Adjust buckets for Pull Mirroring metrics
What does this MR do and why?
Contributes to #367846 (closed)
Problem
We already have metrics for Pull Mirroring that calculate update duration and waiting delay. However, their bucket values are not optimal.
Solution
Adjust bucket values to match real scenarios.
For example, update duration is limited to 100 seconds. But it can be possible that it will take longer. I'm increasing that to 10 minutes.
Waiting duration is limited to 10 seconds, but we might have even 1 hour delays. I'm increasing it as well.
How to set up and validate locally
- Enable
sidekiq_exporter
ingitlab/config/gitlab.yml
monitoring: # Time between sampling of Puma metrics, in seconds # puma_sampler_interval: 5 # IP whitelist to access monitoring endpoints ip_whitelist: - 127.0.0.0/8 - 10.0.0.0/8 - 172.16.0.0/12 - 192.168.0.0/16 # Sidekiq exporter is a webserver built in to Sidekiq to expose Prometheus metrics sidekiq_exporter: enabled: true address: 127.0.0.1 port: 3807 # Sidekiq health checks is a webserver built in to Sidekiq to serve health checks for the workers sidekiq_health_checks: enabled: true address: 127.0.0.1 port: 3907
- Test new values with
curl -s 127.0.0.1:3807/metrics | grep gitlab_repository_mirror_update_duration_seconds_bucket
- Test new values with
curl -s 127.0.0.1:3807/metrics | grep gitlab_repository_mirror_waiting_duration_seconds_bucket
Expected results
curl -s 127.0.0.1:3807/metrics | grep gitlab_repository_mirror_update_duration_seconds_bucket
gitlab_repository_mirror_update_duration_seconds_bucket{le="+Inf"} 5
gitlab_repository_mirror_update_duration_seconds_bucket{le="0.1"} 0
gitlab_repository_mirror_update_duration_seconds_bucket{le="1.0"} 3
gitlab_repository_mirror_update_duration_seconds_bucket{le="10.0"} 4
gitlab_repository_mirror_update_duration_seconds_bucket{le="120.0"} 5
gitlab_repository_mirror_update_duration_seconds_bucket{le="30.0"} 5
gitlab_repository_mirror_update_duration_seconds_bucket{le="300.0"} 5
gitlab_repository_mirror_update_duration_seconds_bucket{le="60.0"} 5
gitlab_repository_mirror_update_duration_seconds_bucket{le="600.0"} 5
curl -s 127.0.0.1:3807/metrics | grep gitlab_repository_mirror_waiting_duration_seconds_bucket
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="+Inf"} 8
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="1.0"} 0
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="10.0"} 0
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="1800.0"} 8
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="2400.0"} 8
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="30.0"} 8
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="300.0"} 8
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="3600.0"} 8
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="60.0"} 8
gitlab_repository_mirror_waiting_duration_seconds_bucket{le="600.0"} 8
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Vasilii Iakliushin