Policies and plugins to detect bugs about to breach SLO
Summary
Bugs have a severity label that indicates the SLO for a fix. This automated policy aims to prompt managers about bugs in their group that are approaching the SLO threshold
- Enable this policy to be executed daily for the
gitlab-org/gitlab
project - ~bug issues with severity1 and severity2 labels will be checked daily to see whether a comment should be posted warning the relevant PM and frontend or backend EM about the impending SLO breach
Details
This change adds a new policy and a job for the `gitlab-org/gitlab` project
to find S1/S2 bugs that are approaching the SLO for the applied severity label.
For those issues discovered the relevant EM and PM will be mentioned. Also a
label will be added to each issue affected to ensure that the rule does not repeat.
This applied label also allows bugs approaching SLO to be discovered in the issue
list.
- Detect EM and PM to mention from applied Group label
- Pick Frontend or Backend EM based on whether `frontend` label applied
- Filter issues to comment based on applied severities
- Currently severity::1 and severity::2
- Display breach date in comment to record in the issue when the SLO will be breached
- Apply `approaching-SLO` label to affected issues
Further Iterations
- Expand to more projects in
gitlab-org/gitlab
- Expand to further severity labels
Action items
-
(If applicable) Add documentation to the handbook pages for Triage Operations => gitlab-com/www-gitlab-com!72606 (merged) - (If applicable) Identify the affected groups and how to communicate to them:
-
/cc @ person_or_group
=> -
Relevant Slack channels => #product
#eng-managers
-
Week-in-review
-
Closes #426 (closed)
Edited by Mark Fletcher