GraphQL: Fix N+1 queries in RunnerJobsResolver (!106574) · Merge requests · GitLab.org / GitLab

Pedro Pombeiro requested to merge pedropombeiro/385285/fix-n+1-issues-in-RunnerJobsResolver into master Dec 09, 2022

What does this MR do and why?

Describe in detail what your merge request does and why.

This MR implements batch handling for the jobs property of the CiRunner type, which is currently underperforming in the wild (see #385285 (closed) / #384066 (comment 1200589516))

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Notice that the projects/namespace/acts_as_taggable queries are executed for each job, instead of a one-shot. In the screenshot below, the left side represents the current state, and the right side represents the state once the N+1 issues have been fixed:

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Visit http://gdk.test:3000/-/graphql-explorer

Run the following query:

{
  runner(id: "gid://gitlab/Ci::Runner/1220") {
    id
    runnerType
    jobs(first: 8) {
      nodes {
        id
        detailedStatus {
          id
          detailsPath
          group
          icon
          text
        }
        shortSha
        finishedAt
        duration
        queuedDuration
        tags
      }
    }
  }
}

The queries generated by this branch should no longer generate a route query per-runner.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

I have evaluated the MR acceptance checklist for this MR.

Database queries

Old queries

Processing by GraphqlController#execute as */*
  Parameters: {"query"=>"{\n  runner(id: \"gid://gitlab/Ci::Runner/1220\") {\n    id\n    runnerType\n    jobs(first: 8) {\n      nodes {\n        id\n        detailedStatus {\n          id\n          detailsPath\n          group\n          icon\n          text\n        }\n        shortSha\n        finishedAt\n        duration\n        queuedDuration\n        tags\n      }\n    }\n  }\n}\n", "variables"=>"[FILTERED]", "graphql"=>{"query"=>"{\n  runner(id: \"gid://gitlab/Ci::Runner/1220\") {\n    id\n    runnerType\n    jobs(first: 8) {\n      nodes {\n        id\n        detailedStatus {\n          id\n          detailsPath\n          group\n          icon\n          text\n        }\n        shortSha\n        finishedAt\n        duration\n        queuedDuration\n        tags\n      }\n    }\n  }\n}\n", "variables"=>"[FILTERED]"}}
  Ci::Runner Load (0.3ms)  SELECT "ci_runners".* FROM "ci_runners" WHERE "ci_runners"."id" = 1220 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/resolvers/ci/runner_resolver.rb:31:in `block in find_runner'*/
  ↳ app/graphql/resolvers/ci/runner_resolver.rb:31:in `block in find_runner'
  Ci::Build Load (0.4ms)  SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" = 'Ci::Build' AND "ci_builds"."runner_id" = 1220 AND ("ci_builds"."status" NOT IN ('created')) ORDER BY "ci_builds"."id" DESC LIMIT 9 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Ci::Pipeline Load (0.2ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 29 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/ci/processable.rb:100:in `merge_train_pipeline?'*/
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  Ci::BuildMetadata Load (0.3ms)  SELECT "p_ci_builds_metadata".* FROM "p_ci_builds_metadata" WHERE "p_ci_builds_metadata"."build_id" = 322 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/concerns/ci/metadatable.rb:103:in `read_metadata_attribute'*/
  ↳ app/models/concerns/ci/metadatable.rb:103:in `read_metadata_attribute'
  Project Load (0.6ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 22 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/ci/status/build/common.rb:21:in `details_path'*/
  ↳ lib/gitlab/ci/status/build/common.rb:21:in `details_path'
  Namespace Load (0.4ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 70 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:main,line:/config/routes.rb:313:in `block (4 levels) in <main>'*/
  ↳ config/routes.rb:313:in `block (4 levels) in <main>'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 70 AND "routes"."source_type" = 'Namespace' LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:main,line:/app/models/concerns/routable.rb:121:in `full_path'*/
  ↳ app/models/concerns/routable.rb:121:in `full_path'
  ActsAsTaggableOn::Tag Load (0.3ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 322 AND "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/types/ci/job_type.rb:115:in `map'*/
  ↳ lib/gem_extensions/active_record/associations/has_many_through_association.rb:13:in `find_target'
  Ci::Pipeline Load (0.3ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 28 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/ci/processable.rb:100:in `merge_train_pipeline?'*/
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  Ci::BuildMetadata Load (0.2ms)  SELECT "p_ci_builds_metadata".* FROM "p_ci_builds_metadata" WHERE "p_ci_builds_metadata"."build_id" = 321 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/concerns/ci/metadatable.rb:103:in `read_metadata_attribute'*/
  ↳ app/models/concerns/ci/metadatable.rb:103:in `read_metadata_attribute'
  Project Load (0.4ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 21 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/ci/status/build/common.rb:21:in `details_path'*/
  ↳ lib/gitlab/ci/status/build/common.rb:21:in `details_path'
  Namespace Load (0.3ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 22 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:main,line:/config/routes.rb:313:in `block (4 levels) in <main>'*/
  ↳ config/routes.rb:313:in `block (4 levels) in <main>'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 22 AND "routes"."source_type" = 'Namespace' LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:main,line:/app/models/concerns/routable.rb:121:in `full_path'*/
  ↳ app/models/concerns/routable.rb:121:in `full_path'
  ActsAsTaggableOn::Tag Load (0.3ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 321 AND "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/types/ci/job_type.rb:115:in `map'*/
  ↳ lib/gem_extensions/active_record/associations/has_many_through_association.rb:13:in `find_target'
  Ci::Pipeline Load (0.2ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/ci/processable.rb:100:in `merge_train_pipeline?'*/
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  Ci::BuildMetadata Load (0.2ms)  SELECT "p_ci_builds_metadata".* FROM "p_ci_builds_metadata" WHERE "p_ci_builds_metadata"."build_id" = 320 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/concerns/ci/metadatable.rb:103:in `read_metadata_attribute'*/
  ↳ app/models/concerns/ci/metadatable.rb:103:in `read_metadata_attribute'
  Project Load (0.4ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 20 LIMIT 1 /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/ci/status/build/common.rb:21:in `details_path'*/
  ↳ lib/gitlab/ci/status/build/common.rb:21:in `details_path'
  CACHE Namespace Load (0.1ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 22 LIMIT 1
  ↳ config/routes.rb:313:in `block (4 levels) in <main>'
  CACHE Route Load (0.0ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 22 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:121:in `full_path'
  ActsAsTaggableOn::Tag Load (0.4ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 320 AND "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/types/ci/job_type.rb:115:in `map'*/
  ↳ lib/gem_extensions/active_record/associations/has_many_through_association.rb:13:in `find_target'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Project Load (0.0ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 20 LIMIT 1
  ↳ lib/gitlab/ci/status/build/common.rb:21:in `details_path'
  CACHE Namespace Load (0.1ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 22 LIMIT 1
  ↳ config/routes.rb:313:in `block (4 levels) in <main>'
  CACHE Route Load (0.1ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 22 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:121:in `full_path'
  ActsAsTaggableOn::Tag Load (0.3ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 318 AND "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/types/ci/job_type.rb:115:in `map'*/
  ↳ lib/gem_extensions/active_record/associations/has_many_through_association.rb:13:in `find_target'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Project Load (0.0ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 20 LIMIT 1
  ↳ lib/gitlab/ci/status/build/common.rb:21:in `details_path'
  CACHE Namespace Load (0.0ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 22 LIMIT 1
  ↳ config/routes.rb:313:in `block (4 levels) in <main>'
  CACHE Route Load (0.0ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 22 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:121:in `full_path'
  ActsAsTaggableOn::Tag Load (0.3ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 317 AND "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/types/ci/job_type.rb:115:in `map'*/
  ↳ lib/gem_extensions/active_record/associations/has_many_through_association.rb:13:in `find_target'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Project Load (0.0ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 20 LIMIT 1
  ↳ lib/gitlab/ci/status/build/common.rb:21:in `details_path'
  CACHE Namespace Load (0.0ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 22 LIMIT 1
  ↳ config/routes.rb:313:in `block (4 levels) in <main>'
  CACHE Route Load (0.0ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 22 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:121:in `full_path'
  ActsAsTaggableOn::Tag Load (0.3ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 316 AND "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/types/ci/job_type.rb:115:in `map'*/
  ↳ lib/gem_extensions/active_record/associations/has_many_through_association.rb:13:in `find_target'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Project Load (0.0ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 20 LIMIT 1
  ↳ lib/gitlab/ci/status/build/common.rb:21:in `details_path'
  CACHE Namespace Load (0.0ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 22 LIMIT 1
  ↳ config/routes.rb:313:in `block (4 levels) in <main>'
  CACHE Route Load (0.0ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 22 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:121:in `full_path'
  ActsAsTaggableOn::Tag Load (0.3ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 315 AND "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/types/ci/job_type.rb:115:in `map'*/
  ↳ lib/gem_extensions/active_record/associations/has_many_through_association.rb:13:in `find_target'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Project Load (0.0ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" = 20 LIMIT 1
  ↳ lib/gitlab/ci/status/build/common.rb:21:in `details_path'
  CACHE Namespace Load (0.0ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 22 LIMIT 1
  ↳ config/routes.rb:313:in `block (4 levels) in <main>'
  CACHE Route Load (0.0ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 22 AND "routes"."source_type" = 'Namespace' LIMIT 1
  ↳ app/models/concerns/routable.rb:121:in `full_path'
  ActsAsTaggableOn::Tag Load (0.3ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 314 AND "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' /*application:web,correlation_id:01GM46TY37ZSBGA7XGTTT0AG21,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/types/ci/job_type.rb:115:in `map'*/
  ↳ lib/gem_extensions/active_record/associations/has_many_through_association.rb:13:in `find_target'
Completed 200 OK in 218ms (Views: 0.3ms | ActiveRecord: 8.0ms | Elasticsearch: 0.0ms | Allocations: 272042)

New queries

Processing by GraphqlController#execute as */*
  Parameters: {"query"=>"{\n  runner(id: \"gid://gitlab/Ci::Runner/1220\") {\n    id\n    runnerType\n    jobs(first: 8) {\n      nodes {\n        id\n        detailedStatus {\n          id\n          detailsPath\n          group\n          icon\n          text\n        }\n        shortSha\n        finishedAt\n        duration\n        queuedDuration\n        tags\n      }\n    }\n  }\n}\n", "variables"=>"[FILTERED]", "graphql"=>{"query"=>"{\n  runner(id: \"gid://gitlab/Ci::Runner/1220\") {\n    id\n    runnerType\n    jobs(first: 8) {\n      nodes {\n        id\n        detailedStatus {\n          id\n          detailsPath\n          group\n          icon\n          text\n        }\n        shortSha\n        finishedAt\n        duration\n        queuedDuration\n        tags\n      }\n    }\n  }\n}\n", "variables"=>"[FILTERED]"}}
  Ci::Runner Load (0.3ms)  SELECT "ci_runners".* FROM "ci_runners" WHERE "ci_runners"."id" = 1220 /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/resolvers/ci/runner_resolver.rb:31:in `block in find_runner'*/
  ↳ app/graphql/resolvers/ci/runner_resolver.rb:31:in `block in find_runner'
  Ci::Build Load (0.5ms)  SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" = 'Ci::Build' AND "ci_builds"."runner_id" = 1220 AND ("ci_builds"."status" NOT IN ('created')) ORDER BY "ci_builds"."id" DESC LIMIT 9 /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Ci::BuildMetadata Load (0.3ms)  SELECT "p_ci_builds_metadata".* FROM "p_ci_builds_metadata" WHERE "p_ci_builds_metadata"."build_id" IN (322, 321, 320, 318, 317, 316, 315, 314, 313) /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Project Load (0.5ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" IN (22, 21, 20) /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."source_id" IN (20, 21, 22) /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Namespace Load (0.3ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" IN (22, 70) /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."source_id" IN (22, 70) /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Ci::Pipeline Load (0.3ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" IN (29, 28, 27) /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  ActsAsTaggableOn::Tagging Load (0.2ms)  SELECT "taggings".* FROM "taggings" WHERE "taggings"."taggable_type" = 'CommitStatus' AND "taggings"."context" = 'tags' AND "taggings"."taggable_id" IN (322, 321, 320, 318, 317, 316, 315, 314, 313) /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  ActsAsTaggableOn::Tag Load (0.2ms)  SELECT "tags".* FROM "tags" WHERE "tags"."id" IN (1, 2, 4) /*application:web,correlation_id:01GM46WJSEHP4R6H4ZHE5KXKXE,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
Completed 200 OK in 118ms (Views: 0.3ms | ActiveRecord: 3.0ms | Elasticsearch: 0.0ms | Allocations: 115077)

Closes #385285 (closed)

Edited Dec 12, 2022 by Pedro Pombeiro

GraphQL: Fix N+1 queries in RunnerJobsResolver

What does this MR do and why?

Screenshots or screen recordings

How to set up and validate locally

MR acceptance checklist

Database queries

Merge request reports