Fix register_job_service.rb generates cross-database query
app/services/ci/register_job_service.rb
, depending if the runner is a shared runner, group runner, or instance runner generates cross-database queries. Queries which join between ci_*
and non ci_*
tables which will not be allowed once ci_*
tables are moved to a new database.
Example
Example failure from https://gitlab.com/gitlab-org/gitlab/-/jobs/1535681506
Failures:
1) Ci::RegisterJobService#execute when a long queue is created picks builds one-by-one
Failure/Error:
raise CrossJoinAcrossUnsupportedTablesError,
"Unsupported cross-join across '#{tables.join(", ")}' modifying '#{schemas.to_a.join(", ")}' discovered " \
"when executing query '#{sql}'"
Database::PreventCrossJoins::CrossJoinAcrossUnsupportedTablesError:
Unsupported cross-join across 'ci_pending_builds, project_features, projects, ci_runner_projects' modifying 'gitlab_ci, gitlab_main' discovered when executing query 'SELECT "ci_pending_builds"."build_id" FROM "ci_pending_builds" WHERE "ci_pending_builds"."project_id" IN (SELECT "projects"."id" FROM "projects" INNER JOIN "ci_runner_projects" ON "projects"."id" = "ci_runner_projects"."project_id" LEFT JOIN project_features ON projects.id = project_features.project_id WHERE "ci_runner_projects"."runner_id" = 17 AND "projects"."pending_delete" = FALSE AND ("project_features"."builds_access_level" > 0 OR "project_features"."builds_access_level" IS NULL)) AND (tag_ids <@ ARRAY[NULL]::int[]) ORDER BY build_id ASC /*application:test,correlation_id:1e4763c14b7e2fa1963dfd7bcbb63eb2*/'
# ./spec/support/database/prevent_cross_joins.rb:45:in `validate_cross_joins!'
# ./spec/support/database/prevent_cross_joins.rb:54:in `block in with_cross_joins_prevented'
# ./app/services/ci/queue/pending_builds_strategy.rb:44:in `build_ids'
# ./app/services/ci/queue/build_queue_service.rb:72:in `execute'
# ./app/services/ci/register_job_service.rb:135:in `block in each_build'
# ./app/services/ci/register_job_service.rb:151:in `block in retrieve_queue'
# ./lib/gitlab/ci/queue/metrics.rb:108:in `observe_queue_time'
# ./app/services/ci/register_job_service.rb:150:in `retrieve_queue'
# ./app/services/ci/register_job_service.rb:135:in `each_build'
# ./app/services/ci/register_job_service.rb:54:in `process_queue'
# ./app/services/ci/register_job_service.rb:30:in `block in execute'
# ./lib/gitlab/ci/queue/metrics.rb:108:in `observe_queue_time'
# ./app/services/ci/register_job_service.rb:29:in `execute'
# ./spec/services/ci/register_job_service_spec.rb:918:in `execute'
# ./spec/services/ci/register_job_service_spec.rb:716:in `block (4 levels) in <module:Ci>'
# ./spec/spec_helper.rb:390:in `block (3 levels) in <top (required)>'
# ./spec/support/sidekiq_middleware.rb:9:in `with_sidekiq_server_middleware'
# ./spec/spec_helper.rb:381:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:377:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/application_context.rb:31:in `with_raw_context'
# ./spec/spec_helper.rb:377:in `block (2 levels) in <top (required)>'
# ./spec/support/database/prevent_cross_joins.rb:85:in `block (3 levels) in <top (required)>'
# ./spec/support/database/prevent_cross_joins.rb:59:in `with_cross_joins_prevented'
# ./spec/support/database/prevent_cross_joins.rb:85:in `block (2 levels) in <top (required)>'
Possible solutions
Edited by Thong Kuah