Compare live vs actual CI minutes consumption
What does this MR do?
Related to https://gitlab.com/gitlab-org/gitlab/-/issues/20856
Background
This MR builds on top of !59263 (merged) where we added the ability to track the consumption of CI minutes for running builds, called "live consumption". This tracking is not entirely accurate since we would miss some timeframe between the builds being assigned to the runner and the time between last Runner ping about the build and the time the build completes. It's however considered good enough approach for live tracking since we traded accuracy for performance. To balance out with the missed accuracy we are giving 1000 minutes of grace periods to all namespaces before we start dropping builds.
About this MR
When a build completes we run Ci::Minutes::UpdateBuildMinutesService
which is responsible for persisting the actual CI minutes consumption. At this point we can compare the actual consumption with the "live consumption" from !59263 (merged) which is used to determine whether to drop running builds.
The comparison is recorded as a Prometheus histogram in order to understand the accuracy of the live consumption.
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. -
I have not included a changelog entry because _____.
-
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team