Send Amazon Cloudwatch Logs to Loggly

Loggly provides the infrastructure to aggregate and normalize log events so they are available to explore interactively, build visualizations, or create threshold-based alerting. In general, any method to send logs from a system or application to an external source can be adapted to send logs to Loggly. The following instructions provide one scenario for sending logs to Loggly.

You can push Amazon Cloudwatch Logs (CWL) to Loggly using an Amazon Lambda Blueprint.

To begin setup, follow the instructions below.

Set up AWS

1. Create a KMS key

Create a symmetric KMS key using the steps provided in the AWS KMS developer guide with the alias name logglyCustomerToken.

2. Create a Lamba Role

Sign in to your AWS account and open the IAM console. In your IAM console:

1. Create a new Role, for example, CloudWatch-Full-Access-Role.

   

2. Select AWS Service as a trusted entity, and then select Lambda as the service that uses the new role.

   

3. Attach the policy named CloudWatchFullAccess, and then click Next: Review.

4. Review settings, make adjustments as needed, and then click Create role.

3. Create Lambda Function from the blueprint

Go to the AWS Lambda Console.

Click Create function.

Select the Loggly blueprint named cloudwatch-logs-to-loggly.

4. Configure triggers

Configure the triggers to call your Lambda function as below.

• Log Group: Select your log group whose logs you want to send to Loggly.
• Filter Name: Provide your filter name.
• Filter Pattern: This is not a mandatory field. You can leave it empty.
• Enable trigger: Check this option to enable the trigger. You could also come back to this setting later to enable it. Click on Next button.

5. Configure function and Add Environment Variables

Expand Encryption configuration and select the Enable helpers for encryption in transit check box. Configure Environment Variables as below:

• kmsEncryptedCustomerToken: Paste your Loggly customer token, and then click Encrypt. Select the logglyCustomerToken key created in Step 1, and then click Encrypt again.
• logglyTags: Enter logglyTags per your requirements.
• logglyHostName: Enter logs-01.loggly.com

• 

  Select the correct KMS key to encrypt.

  Failure to select the correct key here prevent this function from working properly.

  

6. Configure Lambda function handler and role

• Role: Choose an existing role. Select the role created in Step 2.
• Memory (MB): Set memory to 512. You can increase it as needed.
• Timeout: Set time out to 1 min. You can increase it as needed.
• KMS key: Select logglyCustomerToken from the dropdown.

Click on the next button to review the function, and then click Create function.  

7. Add the CloudWatch-Full-Access-Role to the list of IAM users and roles who can use the key

You need to add CloudWatch-Full-Access-Role to the list of IAM users and roles who can use this key to encrypt and decrypt data from within the applications and when using AWS services integrated with KMS. Go to the IAM and add the CloudWatch-Full-Access-Role role as shown below.

8. Test your function

Configure the test function by clicking under Actions > Configure test event. A window opens. SelectCloudwatch Logs from the dropdown, and then click Save and test.

If the test is successful, the message below displays. If you get an error, check Troubleshooting.

9. Verify Events

Search Loggly events with the tag as cloudwatch2loggly over the past 30 minutes. It may take few minutes to index the events. If it doesn’t work, see Troubleshooting.

tag:CloudWatch2Loggly

Advanced Amazon CloudWatch Options

• AWS KMS Key – Set up you KMS key and assign roles to it.
• Cloudwatch Metrics – Send Cloudwatch Metrics to Loggly.
• Set a log group during setup in Cloudwatch Logs Agent

Troubleshooting

If you don’t see any data show up in the verification step, check for these common problems.

• Make sure you’ve included your own customer token.
• Make sure you are using the latest version of AWS CLI.
• Make sure you have configured the same roles as mentioned above.
• Create new role from template(s) only.
• Go to your Lambda function in AWS Console and click on View logs in Cloudwatch in the Monitoring tab to view logs.
• Search or post your own Amazon Cloudwatch logging questions in the community forum.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.