Create GitLab issues from PagerDuty Incidents
Problem to solve
Responders need to be automatically notified when an alert is triggered. PagerDuty is one of the most common on-call and paging tools in market and is used by many of our enterprise customers. We want to enable the creation of an issue via API from a PagerDuty incident.
Intended users
Further details
This work contributes to the Incident Management Vision
Proposal
Create webhook that receives PagerDuty incidents and creates GitLab issues in a specific project. The issue will need to be automatically labeled incident
. These issues should contain the payload, any annotations, and a link back to the incident in PagerDuty.
Design
Configuration
The plan is to combine configuration sections for this issue and #119019 into a single space within Settings > Operations > Incidents. The incidents
section already exists, and it currently holds the alert integration. The proposal is to introduce tabs to the incidents
section, and to add an additional tab for the PagerDuty integration. Here's how that would look:
Incidents section collapsed | Alerts tab | PagerDuty tab | Grafana tab |
---|---|---|---|
Note: moving the current Grafana integration into the Incidents section is outside of the scope of this issue. That work can be completed as part of: #219143. Also, what's shown in the PagerDuty section here does not include the fields required for implementing #119019; those fields will be added separately. Only fields required for creating GitLab issues from PagerDuty are included in the mock-up here.
Issue created from PagerDuty incident
In referencing the PagerDuty API, it seems like we could populate the summary section of incident issues created from PagerDuty with the following information:
- Title
- Status (
statuses
) - Opened (which presumably could be a combination of the
since
andtimezone
keys) - Assignee (
user_ids
) - Urgency (
urgencies
) - Incident key (
incident_key
) - Impacted service (
service_ids
) - ...And add the
incident
label as a default for all issues created from PagerDuty?
It's not clear if we'll have anything we can populate with Incident details sections with, however. If we don't have anything to show, perhaps we can consider not adding the details section to the Incident issue automatically. If, however, we are able to pull the PagerDuty incident timeline, that would be a valuable addition to/replacement for the Details section of the Incident issue.
Permissions and Security
Documentation
Documentation Required. Please add a new section here.