Documentation forSolarWinds Incident Response

Incident Webhook (Incident Webhook/API)

This document will help you configure Incident Webhook to route alerts from monitoring tools or your internal (bespoke) systems into Incident Response. Incident Webhook can do both, trigger and resolve incidents in Incident Response, through HTTP POST requests.

Route detailed monitoring alerts coming in via Incident Webhook to the right users in Incident Response.

We also support outgoing webhooks. You can find detailed information on how to configure them here.

How to configure Incident Webhook

In Incident Response: Using Incident Webhook as an Alert Source

  1. From the navigation bar on the left, select Services. Pick the applicable Team from the Team-picker on the top. Next, click on Alert Sources for the applicable Service

  2. configuration Incident Webhook as an Alert Source

  3. Search for Incident Webhook from the Alert Source drop-down and copy the Webhook URL. Use this Webhook URL endpoint to send HTTP POST requests

Selection of Incident Webhook from the Alert Source

For an Alert Source to turn active (indicated by a green dot - Receiving alerts against the name of the Alert Source in the drop-down), you can either generate a test alert or wait for a real-time alert to be generated by the Alert Source.

An Alert Source is active if there is a recorded incident via that Alert Source for the Service in the last 30 days.

Add header before making POST request

Ensure that you add a header Content-Type with value application/json while making the HTTP POST request

The body of the POST request should contain the details of your incident in the following format:

{
  "message": "This will be the incident message",
  "description": "This will be the incident description",
  "tags" : {
    "tagname1":"Tag value#1",
     "tagname2":"Tag value#2",
     "tagname3": {
       "color": "Valid HTML HEX Colour Notation goes here",
       "value":"Tag value#3"
     }
  },
  "priority": "P5",
  "status": "trigger",
  "event_id": "6"
}

Information on certain fields within the JSON

Kindly note that the message and description fields in the JSON are no longer mandatory to trigger an incident in Incident Response. You can enrich your incidents by adding other details optionally, in the same format as seen above in the example JSON.

If the message field is missing, the default message template of "Generic Webhook Alert" will be used automatically.

If the status field is missing, Incident Response will automatically default the status field to "Triggered" which would treat the incoming payload as a trigger event.

By default, incidents are triggered with an initial status. However, to resolve an incident, you must explicitly send a status update indicating that it has been resolved.

Important information for Event_ID

  • Each incident requires a unique Event ID to be entered by the user. When resolving a specific incident, you must specify the corresponding unique Event ID. In the event of multiple incidents associated with the same Event ID, resolving one incident will result in the resolution of all incidents linked to that specific Event ID.

  • If you do not provide an Event ID, any previous incidents without an Event ID will be automatically marked as resolved.

  • The Event ID is an optional field and should not be confused with the Incident ID. They serve different purposes.

  • While the Event ID is not mandatory, we strongly recommend including it to prevent the automatic resolution of triggered incidents without an Event ID. Adding the Event ID ensures that all incidents without an assigned Event ID are not auto-resolved.

  1. Allowed values for priority are P1, P2, P3, P4, and P5.

  2. If a value other than the allowed ones or no value is passed, Incident Response will have the priority level as Unset. Users can manually edit the priority via the web or the mobile app.

Payload Size Limitation
The payload size is limited to 30KB. Any payload that crosses this limit will not be processed.
You will receive [HTTP Status Code 413](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/413) to notify you of this.

Event Identification and Resolution

This section will give you an understanding of how one can associate alerts with Incident Response incidents and resolve them with an API call.

Typical Incident JSON

{
  "message": "This will be the incident message",
  "description": "This will be the incident description",
  "status": "trigger",
  "event_id": "6"
}

This triggers an incident and associates the incident with the event_id value as specified. This event_id can be used to resolve the above-created incident with an API call.

To resolve an incident, a JSON with the format shown below should be sent.

{
  "status": "resolve",
  "event_id": "6"
}
  • The status field should be set to the value "resolve"

  • The associated event_id should also be sent along with this

Resolving an Incident with an API call:

To resolve an incident, message and description fields are not required to be sent.

Add a Tag From directly Incident JSON

This section will give you an understanding of how you can add tags to an incident straight from the Incident JSON using the Incident Webhook.

Typical Incident JSON:

{
   "message":"This will be the incident message",
   "description": "This will be the incident description",
   "tags": {
     "tagname1":"Tag value#1",
     "tagname2":"Tag value#2",
     "tagname3": {
       "color": "Valid HTML HEX Colour Notation goes here",
       "value":"Tag value#3"
     }
   }
}

Example 1: Using tags to set the Severity for the incident

{
  	"message": "Error rates higher than usual",
    "description": "HTTP Error rates for srv_90 is above 90 counts/hour",
    "tags": {
    	"severity": "high"
    }
}

The default colour for Tags:

If a color code is not mentioned explicitly, then the system takes the default color "#808080" (gray) for tags

To specify a color explicitly for tags:

{
	"message": "Error rates higher than usual",
  "description": "HTTP Error rates for srv_90 is above 90 counts/hour",
	"severity": {
  	"colour": "#FF0000",
  	"value":"backend"
  }
}

Example 2: Adding different tags to an incident

{
	"message": "Error rates higher than usual",
  "description": "HTTP Error rates for srv_90 is above 90 counts/hour",
	"tags" : {
   	"priority": "P1",
	  "impact_level": 5,
   	"classification": {
    	"color":"#FF0000",
     	"value":"backend"
     }
 	}
 }