Add storage usage export
What does this MR do and why?
Adds a service for generating a storage statistics usage export for a given plan.
To support exporting namespace storage statistics usage as a CSV, this a service to generate the CSV with a custom iterator so that we can avoid the CsvBuilder's iterator, which causes a slow query due to the order hint. (See this thread for more info)
Initially this will only support the free plan, but will eventually be expanded for others (i.e. premium/ultimate).
To reduce the size of the diff, I've split out the associated worker/controller for exporting the CSV into !133209 (merged)
Refs https://gitlab.com/gitlab-org/gitlab/-/issues/422501
Queries
- root storage statistics batch: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/22793/commands/73507
Time: 18.602 ms - planning: 1.059 ms - execution: 17.543 ms - I/O read: 14.434 ms - I/O write: 0.000 ms Shared buffers: - hits: 877 (~6.90 MiB) from the buffer pool - reads: 7 (~56.00 KiB) from the OS file cache, including disk I/O - dirtied: 0 - writes: 0
- namespace lookups for batch: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/22793/commands/73505
Time: 113.313 ms - planning: 5.382 ms - execution: 107.931 ms - I/O read: 106.603 ms - I/O write: 0.000 ms Shared buffers: - hits: 458 (~3.60 MiB) from the buffer pool - reads: 29 (~232.00 KiB) from the OS file cache, including disk I/O - dirtied: 0 - writes: 0
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Visit http://gdk.test:3000/admin/namespace_limits/export.csv
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.