GraphQL: Add ALL_AVAILABLE membership value
What does this MR do and why?
Describe in detail what your merge request does and why.
This MR does the following:
- adds the
ALL_AVAILABLEvalue to theRunnerMembershipFilterGraphQL enum type; - adds the
:runners_finder_all_availablefeature flag so that we can turn off the new query if it turns out to be too expensive; - adds the
:read_group_all_available_runnersgroup policy that is checked wheneverALL_AVAILABLEis specified;
Part of Group runners should display all runners availa... (#337838 - closed)
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
Namespace structure used for tests
- Top-level group
- Child group
- Child group project
- Child group project runner
- Shared top-level group project runner
- Child group project
- Top-level group project
- Top-level group project runner
- Shared top-level group project runner
- Child group 2
- Child group 2 runner
- Top-level group runner
- Child group
- Other top-level group
- Other top-level group project
- Other top-level group project runner
- Other top-level group runner
- Other top-level group project
-
Enter the following query in http://gdk.test:3000/-/graphql-explorer:
{ group(fullPath: "top-level-group/child-group") { id descendants: runners(membership: DESCENDANTS) { count nodes { id description } } allAvailable: runners(membership: ALL_AVAILABLE) { count nodes { id description } } } } -
The result should list runners from the
top-level-group/child-groupgroup and any from parent groups as well as instance runners:{ "data": { "group": { "id": "gid://gitlab/Group/73", "descendants": { "count": 2, "nodes": [ { "id": "gid://gitlab/Ci::Runner/3", "description": "Shared top-level group project runner" }, { "id": "gid://gitlab/Ci::Runner/2", "description": "Child group project runner" } ] }, "allAvailable": { "count": 4, "nodes": [ { "id": "gid://gitlab/Ci::Runner/9", "description": "Instance test runner" }, { "id": "gid://gitlab/Ci::Runner/6", "description": "Top-level group runner" }, { "id": "gid://gitlab/Ci::Runner/3", "description": "Shared top-level group project runner" }, { "id": "gid://gitlab/Ci::Runner/2", "description": "Child group project runner" } ] } } } }
Database query plans
New Ci::Runner.usable_from_scope(group_id) scope
SQL query
SELECT "ci_runners".*
FROM ((
SELECT "ci_runners".*
FROM "ci_runners"
INNER JOIN "ci_runner_namespaces" ON "ci_runner_namespaces"."runner_id" = "ci_runners"."id"
WHERE 1 = 0)
UNION ALL (
SELECT "ci_runners".*
FROM "ci_runners"
INNER JOIN "ci_runner_namespaces" ON "ci_runner_namespaces"."runner_id" = "ci_runners"."id"
WHERE "ci_runner_namespaces"."namespace_id" IN (
SELECT "ci_namespace_mirrors"."namespace_id"
FROM "ci_namespace_mirrors"
WHERE (traversal_ids @> ARRAY[278964]::int[])))
UNION ALL ( SELECT DISTINCT "ci_runners".*
FROM "ci_runners"
INNER JOIN "ci_runner_projects" ON "ci_runner_projects"."runner_id" = "ci_runners"."id"
WHERE "ci_runner_projects"."project_id" IN (
SELECT "ci_project_mirrors"."project_id"
FROM "ci_project_mirrors"
WHERE "ci_project_mirrors"."namespace_id" IN (
SELECT "ci_namespace_mirrors"."namespace_id"
FROM "ci_namespace_mirrors"
WHERE (traversal_ids @> ARRAY[278964]::int[]))))
UNION ALL (
SELECT "ci_runners".*
FROM "ci_runners"
WHERE "ci_runners"."runner_type" = 1)) ci_runnersQuery plan
https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/12190/commands/43342
Gather (cost=1094.54..6649.54 rows=127 width=270) (actual time=1.359..10.748 rows=73 loops=1)
Workers Planned: 2
Workers Launched: 2
Buffers: shared hit=235
I/O Timings: read=0.000 write=0.000
-> Parallel Append (cost=94.54..5636.84 rows=53 width=270) (actual time=0.019..0.981 rows=24 loops=3)
Buffers: shared hit=235
I/O Timings: read=0.000 write=0.000
-> Unique (cost=5635.95..5636.49 rows=8 width=270) (actual time=1.335..1.338 rows=0 loops=1)
Buffers: shared hit=95
I/O Timings: read=0.000 write=0.000
-> Sort (cost=5635.95..5635.97 rows=8 width=270) (actual time=1.334..1.337 rows=0 loops=1)
Sort Key: ci_runners.id, ci_runners.token, ci_runners.created_at, ci_runners.updated_at, ci_runners.description, ci_runners.contacted_at, ci_runners.active, ci_runners.name, ci_runners.version, ci_runners.revision, ci_runners.platform, ci_runners.architecture, ci_runners.run_untagged, ci_runners.locked, ci_runners.access_level, ci_runners.ip_address, ci_runners.maximum_timeout, ci_runners.runner_type, ci_runners.token_encrypted, ci_runners.public_projects_minutes_cost_factor, ci_runners.private_projects_minutes_cost_factor, ci_runners.config, ci_runners.executor_type, ci_runners.maintainer_note, ci_runners.token_expires_at, ci_runners.allowed_plans
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=95
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=5519.25..5635.83 rows=8 width=270) (actual time=1.200..1.202 rows=0 loops=1)
Buffers: shared hit=62
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=5518.82..5630.13 rows=8 width=4) (actual time=1.200..1.202 rows=0 loops=1)
Buffers: shared hit=62
I/O Timings: read=0.000 write=0.000
-> HashAggregate (cost=5518.40..5520.17 rows=177 width=4) (actual time=1.199..1.201 rows=0 loops=1)
Group Key: ci_project_mirrors.project_id
Buffers: shared hit=62
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=94.13..5517.95 rows=177 width=4) (actual time=1.197..1.198 rows=0 loops=1)
Buffers: shared hit=62
I/O Timings: read=0.000 write=0.000
-> Bitmap Heap Scan on public.ci_namespace_mirrors (cost=93.69..539.28 rows=282 width=4) (actual time=1.186..1.187 rows=1 loops=1)
Buffers: shared hit=59
I/O Timings: read=0.000 write=0.000
-> Bitmap Index Scan using index_gin_ci_namespace_mirrors_on_traversal_ids (cost=0.00..93.62 rows=282 width=0) (actual time=1.184..1.184 rows=1 loops=1)
Index Cond: (ci_namespace_mirrors.traversal_ids @> '{278964}'::integer[])
Buffers: shared hit=58
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_ci_project_mirrors_on_namespace_id on public.ci_project_mirrors (cost=0.44..17.43 rows=22 width=8) (actual time=0.008..0.009 rows=0 loops=1)
Index Cond: (ci_project_mirrors.namespace_id = ci_namespace_mirrors.namespace_id)
Buffers: shared hit=3
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_ci_runner_projects_on_project_id on public.ci_runner_projects (cost=0.43..0.55 rows=7 width=8) (actual time=0.000..0.000 rows=0 loops=0)
Index Cond: (ci_runner_projects.project_id = ci_project_mirrors.project_id)
I/O Timings: read=0.000 write=0.000
-> Index Scan using ci_runners_pkey on public.ci_runners (cost=0.43..0.71 rows=1 width=270) (actual time=0.000..0.000 rows=0 loops=0)
Index Cond: (ci_runners.id = ci_runner_projects.runner_id)
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=94.54..3057.65 rows=2 width=270) (actual time=1.330..1.331 rows=0 loops=1)
Buffers: shared hit=62
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=94.11..3054.60 rows=2 width=4) (actual time=1.329..1.330 rows=0 loops=1)
Buffers: shared hit=62
I/O Timings: read=0.000 write=0.000
-> Bitmap Heap Scan on public.ci_namespace_mirrors ci_namespace_mirrors_1 (cost=93.69..539.28 rows=282 width=4) (actual time=1.302..1.312 rows=1 loops=1)
Buffers: shared hit=59
I/O Timings: read=0.000 write=0.000
-> Bitmap Index Scan using index_gin_ci_namespace_mirrors_on_traversal_ids (cost=0.00..93.62 rows=282 width=0) (actual time=1.294..1.294 rows=1 loops=1)
Index Cond: (ci_namespace_mirrors_1.traversal_ids @> '{278964}'::integer[])
Buffers: shared hit=58
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_ci_runner_namespaces_on_namespace_id on public.ci_runner_namespaces (cost=0.42..8.87 rows=5 width=8) (actual time=0.015..0.015 rows=0 loops=1)
Index Cond: (ci_runner_namespaces.namespace_id = ci_namespace_mirrors_1.namespace_id)
Buffers: shared hit=3
I/O Timings: read=0.000 write=0.000
-> Index Scan using ci_runners_pkey on public.ci_runners ci_runners_1 (cost=0.43..1.52 rows=1 width=270) (actual time=0.000..0.000 rows=0 loops=0)
Index Cond: (ci_runners_1.id = ci_runner_namespaces.runner_id)
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_ci_runners_on_runner_type on public.ci_runners ci_runners_2 (cost=0.43..100.04 rows=117 width=270) (actual time=0.056..0.260 rows=73 loops=1)
Index Cond: (ci_runners_2.runner_type = 1)
Buffers: shared hit=78
I/O Timings: read=0.000 write=0.000
Existing Ci::Runner.belonging_to_group_or_project_descendants(group_id) scope (for comparison)
SQL query
SELECT "ci_runners".*
FROM ((
SELECT "ci_runners".*
FROM "ci_runners"
INNER JOIN "ci_runner_namespaces" ON "ci_runner_namespaces"."runner_id" = "ci_runners"."id"
WHERE "ci_runner_namespaces"."namespace_id" IN (
SELECT "ci_namespace_mirrors"."namespace_id"
FROM "ci_namespace_mirrors"
WHERE (traversal_ids @> ARRAY[278964]::int[])))
UNION (
SELECT "ci_runners".*
FROM "ci_runners"
INNER JOIN "ci_runner_projects" ON "ci_runner_projects"."runner_id" = "ci_runners"."id"
WHERE "ci_runner_projects"."project_id" IN (
SELECT "ci_project_mirrors"."project_id"
FROM "ci_project_mirrors"
WHERE "ci_project_mirrors"."namespace_id" IN (
SELECT "ci_namespace_mirrors"."namespace_id"
FROM "ci_namespace_mirrors"
WHERE (traversal_ids @> ARRAY[278964]::int[]))))) ci_runnersQuery plan
https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/12149/commands/43104
HashAggregate (cost=9299.66..9299.76 rows=10 width=3839) (actual time=12.159..12.167 rows=0 loops=1)
Group Key: ci_runners.id, ci_runners.token, ci_runners.created_at, ci_runners.updated_at, ci_runners.description, ci_runners.contacted_at, ci_runners.active, ci_runners.name, ci_runners.version, ci_runners.revision, ci_runners.platform, ci_runners.architecture, ci_runners.run_untagged, ci_runners.locked, ci_runners.access_level, ci_runners.ip_address, ci_runners.maximum_timeout, ci_runners.runner_type, ci_runners.token_encrypted, ci_runners.public_projects_minutes_cost_factor, ci_runners.private_projects_minutes_cost_factor, ci_runners.config, ci_runners.executor_type, ci_runners.maintainer_note, ci_runners.token_expires_at, ci_runners.allowed_plans
Buffers: shared hit=504
I/O Timings: read=0.000 write=0.000
-> Append (cost=379.53..9299.01 rows=10 width=3839) (actual time=12.157..12.165 rows=0 loops=1)
Buffers: shared hit=504
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=379.53..3341.14 rows=2 width=270) (actual time=5.991..5.994 rows=0 loops=1)
Buffers: shared hit=252
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=379.11..3338.08 rows=2 width=4) (actual time=5.990..5.992 rows=0 loops=1)
Buffers: shared hit=252
I/O Timings: read=0.000 write=0.000
-> Bitmap Heap Scan on public.ci_namespace_mirrors (cost=378.68..824.27 rows=282 width=4) (actual time=5.970..5.971 rows=1 loops=1)
Buffers: shared hit=249
I/O Timings: read=0.000 write=0.000
-> Bitmap Index Scan using index_gin_ci_namespace_mirrors_on_traversal_ids (cost=0.00..378.61 rows=282 width=0) (actual time=5.963..5.964 rows=1 loops=1)
Index Cond: (ci_namespace_mirrors.traversal_ids @> '{278964}'::integer[])
Buffers: shared hit=248
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_ci_runner_namespaces_on_namespace_id on public.ci_runner_namespaces (cost=0.42..8.86 rows=5 width=8) (actual time=0.013..0.013 rows=0 loops=1)
Index Cond: (ci_runner_namespaces.namespace_id = ci_namespace_mirrors.namespace_id)
Buffers: shared hit=3
I/O Timings: read=0.000 write=0.000
-> Index Scan using ci_runners_pkey on public.ci_runners (cost=0.43..1.53 rows=1 width=270) (actual time=0.000..0.000 rows=0 loops=0)
Index Cond: (ci_runners.id = ci_runner_namespaces.runner_id)
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=5841.31..5957.73 rows=8 width=270) (actual time=6.163..6.165 rows=0 loops=1)
Buffers: shared hit=252
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=5840.88..5952.19 rows=8 width=4) (actual time=6.162..6.164 rows=0 loops=1)
Buffers: shared hit=252
I/O Timings: read=0.000 write=0.000
-> HashAggregate (cost=5840.46..5842.23 rows=177 width=4) (actual time=6.160..6.162 rows=0 loops=1)
Group Key: ci_project_mirrors.project_id
Buffers: shared hit=252
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=379.12..5840.01 rows=177 width=4) (actual time=6.156..6.157 rows=0 loops=1)
Buffers: shared hit=252
I/O Timings: read=0.000 write=0.000
-> Bitmap Heap Scan on public.ci_namespace_mirrors ci_namespace_mirrors_1 (cost=378.68..824.27 rows=282 width=4) (actual time=6.134..6.136 rows=1 loops=1)
Buffers: shared hit=249
I/O Timings: read=0.000 write=0.000
-> Bitmap Index Scan using index_gin_ci_namespace_mirrors_on_traversal_ids (cost=0.00..378.61 rows=282 width=0) (actual time=6.129..6.129 rows=1 loops=1)
Index Cond: (ci_namespace_mirrors_1.traversal_ids @> '{278964}'::integer[])
Buffers: shared hit=248
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_ci_project_mirrors_on_namespace_id on public.ci_project_mirrors (cost=0.44..17.57 rows=22 width=8) (actual time=0.014..0.014 rows=0 loops=1)
Index Cond: (ci_project_mirrors.namespace_id = ci_namespace_mirrors_1.namespace_id)
Buffers: shared hit=3
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_ci_runner_projects_on_project_id on public.ci_runner_projects (cost=0.43..0.55 rows=7 width=8) (actual time=0.000..0.000 rows=0 loops=0)
Index Cond: (ci_runner_projects.project_id = ci_project_mirrors.project_id)
I/O Timings: read=0.000 write=0.000
-> Index Scan using ci_runners_pkey on public.ci_runners ci_runners_1 (cost=0.43..0.69 rows=1 width=270) (actual time=0.000..0.000 rows=0 loops=0)
Index Cond: (ci_runners_1.id = ci_runner_projects.runner_id)
I/O Timings: read=0.000 write=0.000MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Pedro Pombeiro