Send Amazon Cloudfront logs to Loggly
You can push your Amazon Cloudfront logs to Loggly using an AWS Lambda Script, originally created by Quidco.com. It converts the Cloudfront gzipped logs written to S3 into JSON format and then sends them to Loggly.
Alternatively, you may use our S3 ingestion service that will directly ingest them into Loggly without requiring a Lambda function.
AWS Setup
1. Get the Lambda Code
Clone the git repo
git clone https://github.com/fgheorghe/cloudfront2loggly.git cd cloudfront2loggly
Edit cloudfront2loggly.js with the proper Loggly customer token.
logglyConfig = {
tag: "aws-cloudfront",
token: "TOKEN"
};
Replace:
- TOKEN: your customer token from the source setup page
Install required npm packages.
npm install
Zip up your code
zip -r cloudfront2loggly.zip cloudfront2loggly.js node_modules
The resulting zip (cloudfront2loggly.zip) is what you will upload to AWS in step 2 below.
2. Configure the Lambda Function
Go to AWS Lambda Console Console. Click the "Create a Lambda function" button. (Choose "Upload a .ZIP file"). Fill the following details.
Name: cloudfront2loggly Upload lambda function (zip file you made above in Step 1) Handler*: cloudfront2loggly.handler Role*: In the drop down click "S3 execution role". (This will open a new window to create the role, click Allow) Set memory at 128MB Set Timer to 10 seconds.
Configure the Event Source to call cloudfront2loggly when logs are added to the S3 bucket. Go to the AWS Lambda Console . Make sure the cloudfront2loggly lambda function is selected, then click ‘Actions->Add event source‘. Then fill the following details.
Event source type: S3 Bucket: Choose the bucket that contains your Cloudfront logs. Event type: ObjectCreated (All)
3. Configure Cloudfront Logging
Goto the Cloudfront app. In your distribution settings, enable logging and select the S3 bucket for logs.
4. Verify Events
Search Loggly events with the tag aws-cloudfront over the past 20 minutes. It may take a few minutes to index the events. If if doesn’t work, see the troubleshooting section below.
tag:aws-cloudfront
Advanced AWS Cloudfront Options
- AWS Cloudfront Logging – How to setup AWS Cloudfront logging to a S3 bucket
- Loggly Libraries Catalog – See more logging options here
Learn how Loggly can help with all your AWS Log Management
Troubleshooting
If you don’t see any data show up in the verification step, then check for these common problems.
- Wait a few minutes in case indexing needs to catch up
- Make sure you’ve included your own customer token
- Make sure you have configured same roles as mentioned above.
- Search or post your own Amazon Cloudfront logging questions in the community forum.
When the APM Integrated Experience is enabled, Loggly shares a common navigation and enhanced feature set with the other integrated experiences' products. How you navigate Loggly and access its features may vary from these instructions. For more information, go to the APM Integrated Experience documentation.
The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.