Send Amazon Cloudfront logs to Loggly

You can push your Amazon Cloudfront logs to Loggly using an AWS Lambda Script, originally created by Quidco.com. It converts the Cloudfront gzipped logs written to S3 into JSON format and then sends them to Loggly.

Alternatively, you may use our S3 ingestion service that will directly ingest them into Loggly without requiring a Lambda function.

AWS Setup

1. Get the Lambda Code

Clone the git repo

git clone https://github.com/fgheorghe/cloudfront2loggly.git
cd cloudfront2loggly

Edit cloudfront2loggly.js with the proper Loggly customer token.

logglyConfig = {
  tag: "aws-cloudfront",
  token: "TOKEN"
};

Replace:

• TOKEN: your customer token from the source setup page

Install required npm packages.

npm install

Zip up your code

zip -r cloudfront2loggly.zip cloudfront2loggly.js node_modules

The resulting zip (cloudfront2loggly.zip) is what you will upload to AWS in step 2 below.

2. Configure the Lambda Function

Go to AWS Lambda Console Console. Click the "Create a Lambda function" button. (Choose "Upload a .ZIP file"). Fill the following details.

Name: cloudfront2loggly
Upload lambda function (zip file you made above in Step 1)
Handler*: cloudfront2loggly.handler
Role*: In the drop down click "S3 execution role". (This will open a new window to create the role, click Allow)
Set memory at 128MB
Set Timer to 10 seconds.

Configure the Event Source to call cloudfront2loggly when logs are added to the S3 bucket. Go to the AWS Lambda Console . Make sure the cloudfront2loggly lambda function is selected, then click ‘Actions->Add event source‘. Then fill the following details.

Event source type: S3
Bucket: Choose the bucket that contains your Cloudfront logs.
Event type: ObjectCreated (All)

3. Configure Cloudfront Logging

Goto the Cloudfront app. In your distribution settings, enable logging and select the S3 bucket for logs.

4. Verify Events

Search Loggly events with the tag aws-cloudfront over the past 20 minutes. It may take a few minutes to index the events. If if doesn’t work, see the troubleshooting section below.

tag:aws-cloudfront

Advanced AWS Cloudfront Options

• AWS Cloudfront Logging – How to setup AWS Cloudfront logging to a S3 bucket
• Loggly Libraries Catalog – See more logging options here

Learn how Loggly can help with all your AWS Log Management

Troubleshooting

If you don’t see any data show up in the verification step, then check for these common problems.

• Wait a few minutes in case indexing needs to catch up
• Make sure you’ve included your own customer token
• Make sure you have configured same roles as mentioned above.
• Search or post your own Amazon Cloudfront logging questions in the community forum.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.