Update batch size when calculating secure build metrics
What does this MR do and why?
During the review of my last MR @marc_shaw discovered the reason that we are unable to calculate metrics for jobs
: !77991 (comment 818027397). We need to increase the batch_size
to reduce the number of MAX_LOOPS (10_000
) (verified here).
This MR addresses that by extending lib/gitlab/usage/metrics/instrumentations/database_metric.rb
with ability to specify additional options passed to metric operation and by setting custom batch_size
value for ee/lib/gitlab/usage/metrics/instrumentations/count_ci_builds_metric.rb
.
Based on the ID from the latest Job (https://gitlab.com/gitlab-org/gitlab/-/jobs) - 2097825284
and by changing batch_size
to 1_000_000
, we would have 2097825284 / 1000000 = ~2100
loops which will work as expected.
Queries
- Batch
1_000_000
:
explain SELECT COUNT("ci_builds"."id") FROM "ci_builds" WHERE "ci_builds"."type" = 'Ci::Build' AND "ci_builds"."name" = 'container_scanning' AND "ci_builds"."id" >= 2096667775 AND "ci_builds"."id" < 2097667775;
Time: 32.193 ms
- planning: 31.615 ms
- execution: 0.578 ms
- I/O read: 0.377 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 3 (~24.00 KiB) from the buffer pool
- reads: 4 (~32.00 KiB) from the OS file cache, including disk I/O
- dirtied: 0
- writes: 0
https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/8642/commands/30693
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #344264 (closed)