Backfill project import level on namespace settings
What does this MR do and why?
This batched background migration aligns the project_import_level
in existing Group namespaces to their corresponding values for project_creation_level
.
This is to avoid a breaking change when the ability to configure project import levels go live.
See #358750 (closed)
See this issue comment around why we're making this change.
Sub-Batch Update Query
explain UPDATE "namespace_settings" SET "project_import_level" = 30 WHERE "namespace_settings"."namespace_id" IN (SELECT "namespaces"."id" FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."project_creation_level" IN (2, 3, 4, 5) LIMIT 100)
Plan with Execution
ModifyTable on public.namespace_settings (cost=20160.34..20506.16 rows=100 width=130) (actual time=777.221..777.225 rows=0 loops=1)
Buffers: shared hit=4399 read=1391 dirtied=182
I/O Timings: read=734.220 write=0.000
-> Nested Loop (cost=20160.34..20506.16 rows=100 width=130) (actual time=681.481..730.595 rows=100 loops=1)
Buffers: shared hit=3695 read=1296 dirt
Summary
Time: 777.883 ms
- planning: 0.550 ms
- execution: 777.333 ms
- I/O read: 734.220 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 1049759 (~8.00 GiB) from the buffer pool
- reads: 126086 (~985.00 MiB) from the OS file cache, including disk I/O
- dirtied: 20201 (~157.80 MiB)
- writes: 0
Rows Affected
explain SELECT id FROM "namespaces" WHERE "namespaces"."type"='Group' AND "namespaces"."project_creation_level" IN (0, 2, 3, 4, 5);
Gather (cost=1000.43..1241036.56 rows=6838 width=4)
(actual time=68.188..201608.887 **rows=74718** loops=1)
Workers Planned: 2
Workers Launched: 2
Buffers: shared hit=2235347 read=1074470 dirtied=16366
I/O Timings: read=584775.754 write=0.000
-> Parallel Index Scan using index_groups_on_parent_id_id on public.namespaces (cost=0.43..1239352.76 rows=2849 width=4) (actual
~75K (74718)
Estimated Execution Time
~ 39 minutes
Breakdown:
-
per sub-batch (4 queries): 777.883ms * 4 = 3111.532ms
-
per batch (10 sub-batches per batch): 3111.532ms * 10 = 31115.32ms
-
total (75 batches required to process 75K records): 31115.32ms * 75 = 2333649ms
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #358750 (closed)
Edited by Carla Drago