Skip to content

Add Sidekiq dead job metrics

Sean McGivern requested to merge add-sidekiq-dead-job-metrics into master

What does this MR do?

This adds a counter, sidekiq_jobs_dead_total, that increments when a job is added to the dead jobs queue. It has the same basic labels (worker, queue, feature category, etc.) as the other Sidekiq metrics. To do this, we use the Sidekiq death handler configuration option, which is global: https://github.com/mperham/sidekiq/wiki/Error-Handling#death-notification

The easiest way to test this is to just make a job that won't work (wrong arguments) locally in a console:

MergeWorker.perform_async('foo')
#=> "e49157fce745b9dca2681822"

After a short time it will show up at http://localhost:3000/admin/sidekiq/morgue:

image

And we can also see it in our new prometheus counter:

image

For gitlab-com/gl-infra/scalability#275 (closed).

Edited by Sean McGivern

Merge request reports

Loading