Use CounterAttribute for ProjectDailyStatistic
-
Please check this box if this contribution uses AI-generated content as outlined in the GitLab DCO & CLA
What does this MR do and why?
During peak hours, an extreme amount of fetches will try to update the count on the same tuple (project_id, date) which can make the LockManager of Postgres to make those queries wait longer which can cause other queries to fail and would create 500s. INSERTING with the Counter Attribute is a perfect improvement of the potential surge of INSERT that pipelines with high concurrency can caused.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
gitlabhq_development=# SELECT "project_daily_statistics".* FROM "project_daily_statistics" WHERE "project_daily_statistics".id=39
;
id | project_id | fetch_count | date
----+------------+-------------+------------
39 | 3 | 2 | 2024-04-14
(1 row)
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested. There might easier ways but to debug that's what I did:
- Run couple CI jobs in a project
- Look into Redis for the key
- Look into Sidekiq admin to see the schedule task to flush the counter.
- Look into the DB to make sure the flush worked:
SELECT "project_daily_statistics".* FROM "project_daily_statistics" WHERE "project_daily_statistics"."project_id" = 3 AND "project_daily_statistics"."date" = '2024-04-11' LIMIT 1;
Closes #426270 (closed)