Assign root-cause label to deployments report
What does this MR do and why?
Assign root-cause label to deployments report
This commit automatically fetches the 'RootCause::x' label (if it exists) for each production blocker listed on the Deployments report.
Author Check-list
- [-] Has documentation been updated?
Test
Click here for a surprise 🎈
Deployment Blockers - Week: 2023-09-04-2023-09-10
Overview
| Start date | End date | Production deployments blocked for | # of blockers |
|---|---|---|---|
| 2023-09-04 | 2023-09-10 | 33.5 | 11 |
Weekly overview
| Resource | Summary | Blocker type | gstg | gprd |
|---|---|---|---|---|
| gitlab-org/release/tasks#6460 (closed) | QA failure qa/specs/features/ee/browser_ui/2_plan/issues_analytics/issues_analytics_spec.b on gstg-cny | ~"RootCause::Feature-Flag" | 2.0 | 0.0 |
| gitlab-org/release/tasks#6441 (closed) | 2023-09-07 - Deployment blocker - Runners auth error | 3.0 | 2.0 | |
| gitlab-org/release/tasks#6425 (closed) | QA failure: qa/specs/features/browser_ui/3_create/repository/protected_tags_spec.rb | ~"RootCause::Flaky-Test" | 0.5 | 0.5 |
| gitlab-org/release/tasks#6421 (closed) | Tuesday 2023-09-05 11:38 UTC - gitlab-org/gitlab broken 16-3-stable-ee with rspec-ee system pg14 10/10, rspec-ee system pg14 9/10, rspec-ee system pg14 7/10, rspec-ee system pg14 6/10, rspec-ee system pg14 5/10, rspec-ee system pg14 4/10, rspec-ee ... |
0.0 | 0.0 | |
| gitlab-org/release/tasks#6410 (closed) | QA failure: qa/specs/features/api/9_data_stores/user_inherited_access_spec.rb | ~"RootCause::Flaky-Test" | 1.0 | 1.0 |
| gitlab-org/release/tasks#6409 (closed) | QA failure: qa/specs/features/browser_ui/9_data_stores/group/transfer_project_spec.rb | ~"RootCause::Software-Change" | 10.0 | 10.0 |
| gitlab-org/release/tasks#6408 (closed) | QA failure due to artifacts not being available | 1.0 | 1.0 | |
| gitlab-com/gl-infra/production#16323 (closed) | 2023-09-08: QA tests are failing on gstg-cny | ~"RootCause::Software-Change" | 7.0 | 7.0 |
| gitlab-com/gl-infra/production#16322 (closed) | 2023-09-08: gstg-cny-migrations job failing due the wrong Ansible password | ~"RootCause::Needed" | 4.0 | 0.0 |
| gitlab-com/gl-infra/production#16315 (closed) | 2023-09-07: SourceForge.net is DOWN. Deployment blocker | ~"RootCause::External-Dependency" | 10.0 | 10.0 |
| gitlab-com/gl-infra/production#16297 (closed) | 2023-09-04: ops.gitlab.net No active runners available | ~"RootCause::Needed" | 4.0 | 2.0 |
Additional incidents
Below is a list of production incidents created last week.
Click to expand
| Resource | Summary |
|---|---|
| gitlab-com/gl-infra/production#16341 (closed) | 2023-09-11: PrometheusNotificationsBacklog in Ops |
| gitlab-com/gl-infra/production#16340 (closed) | 2023-09-10: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16339 (closed) | 2023-09-10: GCPLoadBalancerImbalancedBackends for gprd-haproxy-main-regional |
| https://gitlab.com/gitlab-com/gl-infra/production/-/issues/16338 | 2023-09-10: Attempted exploit of CVE-2019-11580 |
| gitlab-com/gl-infra/production#16337 (closed) | 2023-09-10: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16336 (closed) | 2023-09-10: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16334 (closed) | 2023-09-10: customers.gitlab.com blackbox probex failing |
| gitlab-com/gl-infra/production#16333 (closed) | 2023-09-09: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16332 (closed) | 2023-09-09: PvsServiceHttpApdexSLOViolation |
| gitlab-com/gl-infra/production#16331 (closed) | 2023-09-09: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16330 (closed) | 2023-09-09: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16329 (closed) | 2023-09-09: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16328 (closed) | 2023-09-09: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16327 (closed) | 2023-09-09: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16326 (closed) | 2023-09-09: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16325 (closed) | 2023-09-09: websockets rails_requests violating slo in us-east1-d |
| gitlab-com/gl-infra/production#16324 (closed) | 2023-09-08: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16319 (closed) | 2023-09-07: Prometheus server unresponsive in ops environment |
| gitlab-com/gl-infra/production#16314 (closed) | 2023-09-07: SSL certificate for kas.staging.gitlab.com:443 expires soon |
| gitlab-com/gl-infra/production#16313 (closed) | 2023-09-06: Prometheus server unresponsive in ops environment |
| gitlab-com/gl-infra/production#16311 (closed) | 2023-09-06: Intermittent failures in health checks for main stage git service when updating feature flags |
| gitlab-com/gl-infra/production#16307 (closed) | 2023-09-05: gstg mailroom pod restarting |
| gitlab-com/gl-infra/production#16305 (closed) | 2023-09-05: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16304 (closed) | 2023-09-04: PostgreSQL queries dominating total query time |
| gitlab-com/gl-infra/production#16303 (closed) | 2023-09-04: Blackbox probes for https://packages.gitlab.com are failing. |
| gitlab-com/gl-infra/production#16301 (closed) | 2023-09-04: Elevated error rate on cny web |
Instructions
-
Review the "Additional incidents" list and add the Deploys-blockedlabel if required. -
Update the "weekly overview" table of this issue to also include: -
any blocking CRs -
the Root Cause type for each blocker (if this one is missing)
-
-
Update the Deployments metric review epic. -
Add a new row to the Overview section: Copy and paste the information from the Overview section in this issue and link to this issue in the Breakdown of blockerscolumn. -
Update the Graph: Update the data on the spreadsheet and then update the graph on the Deployments metric review epic.
-
Edited by Mayra Cabrera