AWS CloudWatch

Importing CloudWatch Metrics

After you sign in to your AppOptics account for the first time, you will see a welcome page that will guide you through the process of setting up your account. After selecting "Amazon Web Services" you will be asked to provide a name for your new integration, and selecting regions and services, after which you will be prompted to create a new AWS IAM Policy.

You can import metrics from as many instances as you want but as a precautionary measure we have set a limit to 250 instances. If you want that limit removed contact the AppOptics customer support team.

Create a new AWS IAM Policy

1. Login to the AWS console and open the IAM service. Click on the "Policies" tab, then click the Create policy button.
2. Select "Create Your Own Policy". Name it "AppOptics" for scanability reasons and set the description to "Read-only access to CloudWatch for AppOptics".
3. Enter the following policy document:

(If you're not using a service, feel free to omit it):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:Describe*",
        "cloudwatch:Get*",
        "cloudwatch:List*",
        "dynamodb:List*",
        "ec2:Describe*",
        "ecs:Describe*",
        "ecs:List*",
        "elasticache:Describe*",
        "elasticbeanstalk:DescribeEnvironments",
        "elasticloadbalancing:Describe*",
        "elasticmapreduce:List*",
        "rds:Describe*",
        "ses:Get*",
        "sqs:ListQueues",
        "lambda:ListFunctions",
        "tag:GetResources",
        "tag:GetTagKeys",
        "tag:GetTagValues"
     ],
      "Resource": "*"
    }
  ]
}

Create a new AWS IAM Role

1. Open the IAM service. Click on the "Roles" tab, then click on the Create role button.
2. Select the "Another AWS account" role type and check the "Require external ID" option. Leave the "Require MFA" option unchecked and enter the Account ID and External ID that are shown in the AppOptics AWS integration configuration tool.
3. Click on the "Next: Permissions" button and select the Custom Policy you created earlier from the table.
4. Click on the "Next: Review" button and give the new role a Role Name and Role Description.
5. Click on the "Create role" button, then click on the role you created. Copy the Role ARN from the summary and paste it into the Role ARN field in the AppOptics AWS integration configuration tool.

Why do we need so many permissions?

We obviously need the "cloudwatch" permissions to import the actual data, but what about all the others? The Cloudwatch list-metrics API call includes all metrics that have had any data reported for them in the last two weeks, even if the instance, database or other resource has been terminated or deleted. This can result in our integration continuing to make 100s of requests for things that haven't had any new data in several days. To save on costs, both to our integration importer having to do extra work, and also for you having to pay for all those additional AWS API calls, we use the relevant "DescribeX" API call to determine which resources are actively running so we can limit our importing to only those.

Multiple AWS Accounts

You can add multiple AWS accounts by just clicking on "Add Configuration". You will be guided through the steps of setting up the integration. After you are done you will see thje new confgration show up in the list.

Regions and Services

Use the checkboxes for the regions and services to determine which metrics you import. We currently support the following services:

Using AWS Data in AppOptics

After you have completed your setup, you should start seeing metrics flow into your account within 10-15 minutes. If that is not the case, check the Integrations page for any errors that may be present. The error message will give you a hint on what needs to be corrected. If you need help contact the AppOptics customer support team.

Depending on the CloudWatch services you have selected, AppOptics will create metrics in the namespaces following the pattern:

AWS.<service-name>

Default AWS Dashboards

Once your credentials are set up and saved, you can navigate to your list of dashboards we have automatically created for each service. They are easily distinguishable by their "AWS-orange" color.

Default dashboard are intended to give you a starting point to easily view AWS service specific metrics. The tag bar shows the standard, AWS provided tags such as region or az. The dashboards are created as templates using the Dynamic Tags feature. Within the tag bar you can filter or group on tags which will render the corresponding metric streams on your charts.

To see custom tags, or to customize charts, you need to create a custom dashboard.

Custom Tags and Custom Dashboards

If you have added custom tags to your AWS metrics and want to use them in the AppOptics app, you need to create a custom Dashboard. You can either start from scratch by creating a new dashboard or you can copy one of the curated AWS dashboards (see Copy Dashboards). You will notice that the header bar is now blue, indicating that the dashboard is custom and therefore editable.

If you double-click on one of the charts, and click on the tag symbol to the right of a metric stream row, you will see the custom tags you created within AWS. Click on the custom tags and they will appear in the tag bar at the top of the screen.

Filtering Metrics

AppOptics supports the ability to filter by tag for EBS and EC2 resources. In the Cloudwatch integration, the EBS and EC2 service types have an additional input option next to each service type (when checked). This input is a toggle for two states: all or filtered.

The default state is all, which is to collect all resource metrics from CloudWatch for the respective service type. If you click on the link it will toggle to filtered; in this state we white-list metrics coming from resources with the following tag applied:

appoptics:import=true

… where appoptics:import is the key and true is the value. For example, if you are using the EC2 command line tool you would apply the tag to an instance like:

ec2tag <instance id> -t appoptics:import=true

Or in the AWS Console:

If you are using Cloud Formation with Elasticbeanstalk the tag will need to be applied before the environment is setup. see Amazon's ElasticBeanstalk Environment article for more information.

Custom Metrics

AppOptics is a platform designed for custom metrics. You can use the SolarWinds Snap Agent or language bindings to add metrics to AppOptics and then add them to your AWS service graphs or dashboards. For example running the AppOptics Agent on your AWS instances will pull in memory and other metrics that CloudWatch doesn't provide.

You can also pull in CloudWatch custom metrics by checking the box under AWS Service in your AWS Configuration. To learn more about CloudWatch custom metrics, see Amazon's Publish Custom Metrics article. We retrieve up to 10 custom tags created on your custom metrics.

Keep in mind that if you pull in CloudWatch custom metrics, they will effectively be billed by both Amazon and SolarWinds. Amazon currently charges $0.30 per metric per month for the first 10,000. Any AWS custom metrics count against your AppOptics custom metrics quota. As AppOptics was designed for custom metrics, you might consider sending the metrics directly to AppOptics, thereby saving cost and gaining control over the metric resolution.

Reporting Intervals

Depending on the service, CloudWatch metrics are reported at either a 5 minute frequency or a 1 minute frequency. If you want EC2 data at a 1 minute frequency, you'll need to enable "Detailed Monitoring" (see http://aws.amazon.com/cloudwatch/details). Note that ELB, RDS, ElastiCache, and some other services have 1 minute intervals by default. Using the AWS credentials you have provided, AppOptics will poll your CloudWatch metrics every 5 minutes and then write the resulting data into your AppOptics account, back-filling data that is at 1 minute resolution.

Stop Importing CloudWatch Metrics

To stop importing CloudWatch metrics, either remove the IAM role, remove the AWS integration or un-check the AWS Regions and AWS Services boxes and save.

FAQ:

A note about custom metrics…

AppOptics does not charge per Alert or API call and there is no limit to the amount of custom metrics you can send us. If you send custom metrics to AWS and then import them to AppOptics you are paying twice whereas if you only send them to AppOptics you only pay once.

Activating billing metrics

Before you can see data on your estimated charges, you must enable monitoring in AWS, which creates the metric data. To learn how, see the article Enabling the Monitoring of Your Estimated Charges. Note that AWS Billing reports every 4 hours so it may take a while for you to see your data. Also note that you may need to activate the integration for us-east-1 (N. Virginia, US).  From the AWS documentation: "Billing metric data is stored in the US East (N. Virginia) region and represent worldwide charges".

Will I be charged for the AWS API requests?

Yes. Amazon charges a fee (currently $0.01 per 1,000 requests) for CloudWatch API access exceeding 1 million requests/month and these accrue on the account associated with the credentials you provide. Currently we poll the AWS services every 5 minutes and make one API call for each metric instance that we are tracking.

Do you support Auto Scaling Groups?

We do support Auto Scaling Group metrics such as "GroupMinSize", "GroupMaxSize", "GroupTotalInstances", etc as shown in Amazon's Use Amazon CloudWatch to Monitor Your Auto Scaling Instances article.

Can I use Service-Side Aggregation (SSA) on CloudWatch metrics?

Sadly, no. Because of the "polling" nature of the AppOptics / CloudWatch integration, there can be issues with the timing of data coming out of CloudWatch and into AppOptics. Since SSA essentially "waits" for measurements to come in during the aggregation period (say, 60s), and we pull data from CloudWatch based on the "last reported" values, timing issues may occur and the possibility of bad data (double counting) is introduced. For this reason, we do not recommend using SSA for your CloudWatch metrics.

The scripts are not supported under any SolarWinds support program or service. The scripts are provided AS IS without warranty of any kind. SolarWinds further disclaims all warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The risk arising out of the use or performance of the scripts and documentation stays with you. In no event shall SolarWinds or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the scripts or documentation.