Documentation forSolarWinds Incident Response

Datadog

Datadog is a monitoring platform for cloud applications that bring in together data from servers, containers, databases, and third-party services, thus providing observability into the entire stack.

Route detailed monitoring alerts from Datadog to the right users in Incident Response.

How to integrate Datadog with Incident Response

In Incident Response: Using Datadog as an Alert Source

  1. Navigate to Services -> Service Overview -> select or search for your Service. Expand the accordion -> In the Alert Sources section, click Add.

  2. Select Datadog. Copy the displayed Webhook URL to configure it within Datadog. Finish by clicking Add Alert Source -> Done.

When an alert source turns Active, it’ll show up under Configured Alert Sources. You can either generate a test alert from the integration or wait for a real-time alert to be generated by the Alert Source.

An Alert Source is active if there is a recorded incident via that Alert Source for the Service.

In Datadog: Create a Incident Response Webhook

  1. Open the Integrations page from the sidebar

  2. Search for Webhooks. Once the Webhooks tile appears, hover over it and click on "Configure"

  3. Navigate to the Configuration tab

  4. Scroll to the bottom of the page. Click on New Webhook to add a new Webhook URL

  5. (a) Give the Webhook a name in the Name field

(b) Paste the Datadog Webhook URL provided by Incident Response in the URL field

(c) Copy-paste the following JSON in the text box under the Payload section

{
    "alertId": "$ALERT_ID",
    "eventMessage": "$TEXT_ONLY_MSG",
    "title": "$EVENT_TITLE",
    "url": "$LINK",
    "alertTransition": "$ALERT_TRANSITION",
    "hostname": "$HOSTNAME",
    "orgName":"$ORG_NAME",
    "priority":"$PRIORITY",
    "snapshot": "$SNAPSHOT",
    "alertQuery": "$ALERT_QUERY",
    "alertScope": "$ALERT_SCOPE",
    "alertStatus": "$ALERT_STATUS",
    "eventType": "$EVENT_TYPE",
    "event_id": "$ID",
    "alert_metric": "$ALERT_METRIC",
    "alert_priority": "$ALERT_PRIORITY",
    "alert_title": "$ALERT_TITLE",
    "alert_type" : "$ALERT_TYPE",
    "event_msg" : "$EVENT_MSG",
    "incident_pub_id" : "$INCIDENT_PUBLIC_ID",
    "incident_title" : "$INCIDENT_TITLE",
    "incident_url" : "$INCIDENT_URL",
    "incident_msg" : "$INCIDENT_MSG",
    "security_rule_id" : "$SECURITY_RULE_ID",
    "security_rule_name" : "$SECURITY_RULE_NAME",
    "security_signal_severity" : "$SECURITY_SIGNAL_SEVERITY",
    "security_signal_title" : "$SECURITY_SIGNAL_TITLE",
    "security_signal_msg" : "$SECURITY_SIGNAL_MSG",
    "security_rule_query" : "$SECURITY_RULE_QUERY",
    "security_rule_type" : "$SECURITY_RULE_TYPE",
    "tags" : "$TAGS"
}

Click on “Save” to complete the service integration.

Note:

You can find detailed information on the variables supported by Datadog that are available within the alert payload in Incident Response here.

Once the Webhook for Incident Response has been configured, ensure that the same is also selected as a channel within Notify your team about the Monitor's configuration.

That's it, you are good to go! Your Datadog integration is now complete.

  • Now, whenever Datadog fires an alert, an incident will be created in Incident Response for it.

  • When the alert recovers in Datadog, the corresponding incident will automatically get resolved in Incident Response as well.