Put automatic incident closing behind user setting
Problem to solve
With Incident Management we automatically close an issues created via a Prometheus Alert when that issues transitions from firing to recovered.
When GitLab receives a Recovery Alert, it will automatically close the associated issue. This action will be recorded as a system message on the issue indicating that it was closed automatically by the GitLab Alert bot.
However, this behaviour may not be desirable, based on the specific workflows for different SRE teams. As an example, our internal SRE at Gitlab would like to manually close incidents because there is more work to be done to finish the response workflow following a recovery alert.
You are correct, we would like the incident issue to remain open.
Intended users
User experience goal
User can configure if incident issues close automatically when the corresponding alert closes depending on how their organization handles this scenario.
Proposal
- We add in an additional checkbox in Settings > Operations > Incidents that reads: "Automatically close incident issues when the associated Prometheus alert resolves."
- We have this box checked by default.
Longer-term, we'd also like to:
- We implement this automatic closing functionality for all alerts, not just Prometheus alerts.
- When this functionality is available for all alerts, we'll update the text for this checkbox to read: "Automatically close incident issues when the associated alert resolves."u