Send Amazon Cloudfront Logs to Loggly

Loggly provides the infrastructure to aggregate and normalize log events so they are available to explore interactively, build visualizations, or create threshold-based alerting. In general, any method to send logs from a system or application to an external source can be adapted to send logs to Loggly. The following instructions provide one scenario for sending logs to Loggly.

You can push Amazon Cloudfront logs to Loggly using an AWS Lambda Script. The script converts Cloudfront gzipped logs written to S3 into JSON format and then sends them to Loggly.

Alternatively, you can use the S3 ingestion service that directly ingests logs into Loggly without requiring a Lambda function.

Set up AWS

1. Get the Lambda Code

   Clone the git repo

   git clone https://github.com/fgheorghe/cloudfront2loggly.git
   cd cloudfront2loggly

   Edit cloudfront2loggly.js with the proper Loggly customer token. In the code below, replace TOKEN with your customer token from the source setup page.

   logglyConfig = {
     tag: "aws-cloudfront",
     token: "TOKEN"
   };

   Install required npm packages.

   npm install

   Zip up your code.

   zip -r cloudfront2loggly.zip cloudfront2loggly.js node_modules

   The resulting zip file (cloudfront2loggly.zip) is what you upload to AWS in Step 2 below.

2. Configure the Lambda Function

   Use the AWS Console to create a cloudfront2loggly Lambda function. See Create a Lambda function with the console in the AWS documentation for more information.

   When prompted, provide the information below:

   Name: cloudfront2loggly
   Upload lambda function: (select the zip file you created in Step 1)
   Handler*: cloudfront2loggly.handler
   Role*: S3 execution role (if a new window opens to create the role, click Allow)
   Memory: Set to 128MB
   Timer: Set to 10 seconds

   Configure the Event Source to call cloudfront2loggly when logs are added to the S3 bucket. When prompted, provide the information below.

   Event source type: S3
   Bucket: Choose the bucket that contains your Cloudfront logs
   Event type: All object create events

   If additional information is needed, see Using AWS Lambda with Amazon S3.

3. Configure Cloudfront Logging

   Go to the Cloudfront app.

   In your distribution settings, enable logging and select the S3 bucket for logs.

4. Verify Events

   Search Loggly events for tag:aws-cloudfront in the last 30 minutes. It may take a few minutes to index the events.

   If indexing doesn’t work, see Troubleshooting.

Advanced AWS Cloudfront Options

AWS Cloudfront Logging – How to setup AWS Cloudfront logging to a S3 bucket

Learn how Loggly can help with all of your AWS Log Management

Troubleshooting

If you don’t see any data show up in the verification step, check for these common problems.

• Wait a few minutes in case indexing needs to catch up.
• Make sure you’ve included your own customer token.
• Make sure you have configured the same roles as mentioned above.
• Search or post your own Amazon Cloudfront logging questions in the community forum.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.