Skip to content

Create a separate metrics server that can serve Sidekiq metrics

Roy Zwambag requested to merge 345887-separate-metrics-server into master

What does this MR do and why?

This adds a separate metrics server that can serve Sidekiq metrics. This MR only adds the server, but does not actually start/run it yet. We created a PoC where the metrics server is running and serving metrics, see !73049 (closed). This MR extracts the server itself from the whole.

In a future MR (issue here) for this epic, we will enable the metrics server via cli.rb, an example can be seen in the PoC

How to set up and validate locally

We will have to overwrite the default settings for Settings.monitoring.sidekiq_health_checks.port, since they default to Settings.monitoring.sidekiq_exporter.port. If those are the same, we can't run both. With gdk you can change the gitlab.yml and set Settings.monitoring.sidekiq_health_checks.port to any other number. e.g.

sidekiq_exporter:
  enabled: true
  address: 127.0.0.1
  port: 3807
sidekiq_health_checks:
  port: 3907
  1. Open rails console
  2. Start the metrics server:
require_relative 'metrics_server/metrics_server'
MetricsServer.spawn('sidekiq')
  1. See if the sidekiq workers are healthy curl localhost:3907/liveness
  2. See if the metrics server is serving metrics curl localhost:3807/metrics

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #345887 (closed)

Edited by Roy Zwambag

Merge request reports