Here's how to sign up for Amazon Web Services, create a bucket for log archives, and share write-only access to Papertrail for nightly uploads.
After setting up S3 archiving it takes approximately 6-7 hours for logs to be available in the archive.
Skip this step if you already have an AWS account, like for Amazon EC2, S3, or another AWS product.
- Visit http://aws.amazon.com/
- Click Create an AWS Account (upper right)
- Enter your email and choose I am a new user
- Complete the signup form. Confirm the activation email.
Skip this step if your AWS account is already activated for S3.
- Visit http://aws.amazon.com/s3/
- Click Sign Up For Amazon S3
- Provide a credit card. You are accepting responsibility for the storage, data transfer, and requests consumed for your logs, and will be charged for it. Typically this is well under $1 (one dollar) per month.
- Visit http://aws.amazon.com/, click Sign In to AWS Management Console and sign in.
If a warning is displayed that your account isn't active yet, try again in 5 minutes.
- Visit http://aws.amazon.com/, click Sign In to AWS Management Console, and sign in.
- Click Services on the top menu, then Amazon S3, and finally Create Bucket.
- Fill in Bucket Name with a unique name, such as
companyname-papertrail. The name should consist of only lowercase letters, numbers, and hyphens. (more info) If you have an existing bucket, you may use it too, though we recommend a bucket just for this purpose.
- Assign the bucket to the appropriate region.
- Skip the Set Properties step unless this is important for other reasons (the defaults are appropriate for Papertrail’s needs).
- On the Set Permissions tab: (To set up permissions with IAM, see Define Sharing Policy with IAM.)
- Next to Access for other AWS Account, click Add Account.
- For the user, enter
- Under the Objects column, check the Write box.
- Save the policy.
- Review and save.
Changes can be made after the fact by selecting the bucket and choosing the Permissions tab.
After submission, Amazon's management console will change the user name to
aws. This is expected.
Amazon also has instructions for editing bucket permissions.
If you followed the instructions above to grant permissions via the AWS Management Console, skip this step.
If you prefer defining a bucket policy to control access, here’s an example policy that permits Papertrail to upload archives:
bucket-name/papertrail/logs/ is the directory for Papertrail. The
s3:DeleteObject action isn’t strictly necessary – it’s only used to clean up the temporary
test_file.txt during initial configuration. (However, since
s3:PutObject also permits overwrites, denying
s3:DeleteObject doesn’t provide any significant protection.)
Under Settings > Archives, enable S3 archive copies and provide the S3 bucket name.
Papertrail will perform a test upload as part of saving the bucket name (and will then delete the test file). Note that a new bucket can sometimes take several hours to become available, due to DNS propagation delays. If it fails, wait two hours, and try again.
When archives are uploaded to the bucket, each file is named under the path (key prefix) provided to Papertrail, typically
<xxx> is an ID. For example, an archive file for 3 PM UTC created on February 25, 2019 would be:
Days are from midnight to midnight UTC.
Papertrail supports S3 rather than Glacier because:
- AWS offers the ability to trickle files from S3 to Glacier using a policy that you define, so by supporting S3, Glacier is automatically a possible destination. Visit S3 Object Lifecycle Management.
- Archived log files compress extremely well, often 15:1 or more, so the total cost of archived logs stored in S3 is extremely small (often pennies per month). Storing a long-term log archive in your S3 bucket will almost always cost less than 1% of the total cost of Papertrail. There's effectively no cost savings.
Yes, Papertrail takes advantage of S3's server-side encryption so that archived data is encrypted at rest using AES-256.
The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.