Send Amazon Cloudwatch Logs to Loggly
You can push Amazon Cloudwatch Logs (CWL) to Loggly using an Amazon Lambda Blueprint. Please note these instructions are for Cloudwatch Logs, which are different from Cloudwatch metrics. Follow the instructions below.
1. Create a KMS key
Create a symmetric KMS key using the steps provided in the AWS KMS developer guide with the alias name
2. Create a Lamba Role
Sign in to your AWS account and open the IAM console. In your IAM console:
Create a new Role say, ‘CloudWatch-Full-Access-Role‘.
Select Role type as ‘Lambda‘ from the AWS Service Roles.
Attach policy ‘CloudWatchFullAccess‘ and save.
3. Create Lambda Function from the blueprint
Go to AWS Lambda Console. Click the "Create function" button.
Select the "cloudwatch-logs-to-loggly" Loggly blueprint
4. Configure triggers
Configure the triggers to call your Lambda function as below.
- Log Group: Select your log group whose logs you want to send to Loggly.
- Filter Name: Provide your filter name.
- Filter Pattern: This is not a mandatory field. You can keep it empty.
- Enable trigger: Check this option to enable the trigger. You could also come back to this setting later to enable it. Click on Next button.
5. Configure function and Add Environment Variables:
Expand Encryption configuration and click the Enable helpers for encryption in transit check-box. Configure Environment Variables as below:
- kmsEncryptedCustomerToken: Paste your Loggly customer token and click Encrypt. Select
logglyCustomerTokenkey created in step 1 and click Encrypt again.
- logglyTags: Enter
logglyTagsas per your requirement
- logglyHostName: Enter logs-01.loggly.com
Select the correct KMS key to encrypt. Not selecting the right key here will not make this function work.
6. Configure Lambda function handler and role
- Role: Choose an existing role. Select the role created in Step 2.
- Memory (MB): set memory to 512. You can increase it as needed.
- Timeout: Set time out to 1 min.You can increase it as needed.
- KMS key: Select logglyCustomerToken from the dropdown
Click on the next button to review the function and then click on "Create function".
7. Adding CloudWatch-Full-Access-Role to the list of IAM users and roles who can use the key:
You need to add CloudWatch-Full-Access-Role to the list of IAM users and roles who can use this key to encrypt and decrypt data from within the applications and when using AWS services integrated with KMS. Head to the IAM and add the CloudWatch-Full-Access-Role role as shown below.
If you miss this step you will not be able to successfully send Cloudwatch logs to Loggly.
8. Test your function
Configure the test function by clicking under Actions -> Configure test event. A window will open. Select Cloudwatch Logs from the dropdown and click Save and test.
If it tests successfully, you will see the message below. If you get an error, check the troubleshooting section below:
If you haven’t enabled the trigger in step 3, you can go to configure trigger from the left side menu to enable it.
9. Verify Events
Search Loggly events with the tag as cloudwatch2loggly over the past 20 minutes. It may take few minutes to index the events. If it doesn’t work, see the troubleshooting section below.
Advanced Amazon CloudWatch Options
- AWS KMS Key – Set up you KMS key and assign roles to it.
- Cloudwatch Metrics – Send Cloudwatch Metrics to Loggly.
- Loggly Libraries Catalog – See more logging options here.
- Set a log group during setup in Cloudwatch Logs Agent
If you don’t see any data show up in the verification step, check for these common problems.
- Make sure you’ve included your own customer token
- Make sure you are using the latest version of AWS CLI
- Make sure you have configured the same roles as mentioned above
- Create new role from template(s) only
- Go to your Lambda function in AWS Console and click on View logs in Cloudwatch in the Monitoring tab to view logs.
- Search or post your own Amazon Cloudwatch logging questions in the community forum.
When the APM Integrated Experience is enabled, Loggly shares a common navigation and settings with the other integrated experiences' products. How you navigate Loggly and access its features may vary from these instructions. For more information, go to the APM Integrated Experience documentation.
The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.