Use a single query to compute excess storage
What does this MR do?
This improves Namespace#total_repository_size_excess
and Namespace#repository_size_excess_project_count
so that they run only 1 DB query instead of 2.
Related to #325344 (closed)
Before
-
Repos with no project-level limit: https://explain.depesz.com/s/ozOe
SELECT SUM((("project_statistics"."repository_size" + "project_statistics"."lfs_objects_size") - 10737418240)) FROM "projects" INNER JOIN "project_statistics" ON "project_statistics"."project_id" = "projects"."id" WHERE "projects"."namespace_id" IN (...) AND ("project_statistics"."repository_size" + "project_statistics"."lfs_objects_size") > 10737418240 AND "projects"."repository_size_limit" IS NULL
Time: 19.721 ms - planning: 2.297 ms - execution: 17.424 ms - I/O read: N/A - I/O write: N/A Shared buffers: - hits: 9266 (~72.40 MiB) from the buffer pool - reads: 0 from the OS file cache, including disk I/O - dirtied: 0 - writes: 0
-
Repos with project-level limit: https://explain.depesz.com/s/WfOV
SELECT SUM((("project_statistics"."repository_size" + "project_statistics"."lfs_objects_size") - "projects"."repository_size_limit")) FROM "projects" INNER JOIN "project_statistics" ON "project_statistics"."project_id" = "projects"."id" WHERE "projects"."namespace_id" IN (...) AND ("project_statistics"."repository_size" + "project_statistics"."lfs_objects_size") > "projects"."repository_size_limit" AND "projects"."repository_size_limit" != 0
Time: 16.292 ms - planning: 2.409 ms - execution: 13.883 ms - I/O read: N/A - I/O write: N/A Shared buffers: - hits: 4001 (~31.30 MiB) from the buffer pool - reads: 0 from the OS file cache, including disk I/O - dirtied: 0 - writes: 0
After
-
Combined query for both types of repos: https://explain.depesz.com/s/DACC
SELECT SUM((("project_statistics"."repository_size" + "project_statistics"."lfs_objects_size") - COALESCE("projects"."repository_size_limit", 10737418240))) FROM "projects" INNER JOIN "project_statistics" ON "project_statistics"."project_id" = "projects"."id" WHERE "projects"."namespace_id" IN (...) AND ("projects"."repository_size_limit" != 0 OR "projects"."repository_size_limit" IS NULL) AND ("project_statistics"."repository_size" + "project_statistics"."lfs_objects_size") > COALESCE("projects"."repository_size_limit", 10737418240)
Time: 19.858 ms - planning: 2.156 ms - execution: 17.702 ms - I/O read: N/A - I/O write: N/A Shared buffers: - hits: 9270 (~72.40 MiB) from the buffer pool - reads: 0 from the OS file cache, including disk I/O - dirtied: 0 - writes: 0
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. -
I have not included a changelog entry because _____.
-
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Edited by Heinrich Lee Yu