Add Sidekiq queueing application SLIs
What does this MR do and why?
Part of gitlab-com/gl-infra/scalability#2305 (closed)
This is similar to when we added Sidekiq execution application SLIs at !116827 (merged)
Add Sidekiq queueing application SLIs
Similar to Sidekiq execution application SLIs, we're adding Sidekiq queueing application SLIs which eventually replaces the sidekiq_jobs_queue_duration_seconds histogram metric.
Changelog: added
How to set up and validate locally
-
Make sure
production.monitoring.sidekiq_exporter.enabled
istrue
-
Enable feature flag
Feature.enable(:sidekiq_queueing_application_slis)
-
Restart Sidekiq
$ gdk restart rails-background-jobs ok: down: /Users/gregoriusmarco/Documents/workspace/gdk-10-22/services/rails-background-jobs: 0s ok: run: /Users/gregoriusmarco/Documents/workspace/gdk-10-22/services/rails-background-jobs: (pid 8532) 1s, normally down
-
Check that the metrics are initialized
$ curl gdk.test:3807/metrics | rg gitlab_sli_sidekiq_queueing_apdex ...
-
Run a job in Rails console
Chaos::SleepWorker.perform_async(1)
-
Check that the counter is incremented properly
$ curl -s gdk.test:3807/metrics | rg gitlab_sli_sidekiq_queueing_apdex | rg Chaos::SleepWorker gitlab_sli_sidekiq_queueing_apdex_success_total{feature_category="not_owned",urgency="low",worker="Chaos::SleepWorker"} 1 gitlab_sli_sidekiq_queueing_apdex_total{feature_category="not_owned",urgency="low",worker="Chaos::SleepWorker"} 1
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.