Skip to content

GraphQL: Preload merge_mequest if needed

Pedro Pombeiro requested to merge pedropombeiro/385285/preload-merge_requests into master

What does this MR do and why?

Describe in detail what your merge request does and why.

This MR adds support for preloading the pipeline.merge_request relationship CiJob.detailedStaus, which are in use for the Runners' admin UI.

Context source: #384066 (comment 1210132291)

The following code in Ci::Pipeline loads the merge_request relationship:

    def merge_request?
      merge_request_id.present? && merge_request.present?
    end

That method gets called from detailed_status, as evidenced below:

[
  "lib/peek/views/active_record.rb:55:in `block in setup_subscribers'",
  "app/models/ci/pipeline.rb:1253:in `merge_request?'",
  "app/models/ci/pipeline.rb:1269:in `merged_result_pipeline?'",
  "ee/app/models/ee/ci/pipeline.rb:180:in `merge_train_pipeline?'",
  "app/models/ci/processable.rb:100:in `merge_train_pipeline?'",
  "ee/app/presenters/ee/ci/build_presenter.rb:11:in `retryable?'",
  "lib/gitlab/ci/status/build/retryable.rb:33:in `matches?'",
  "lib/gitlab/ci/status/factory.rb:38:in `block (2 levels) in extended_statuses'",
  "lib/gitlab/ci/status/factory.rb:38:in `each'",
  "lib/gitlab/ci/status/factory.rb:38:in `find'",
  "lib/gitlab/ci/status/factory.rb:38:in `block in extended_statuses'",
  "lib/gitlab/ci/status/factory.rb:37:in `each'",
  "lib/gitlab/ci/status/factory.rb:37:in `flat_map'",
  "lib/gitlab/ci/status/factory.rb:37:in `extended_statuses'",
  "lib/gitlab/ci/status/factory.rb:14:in `fabricate!'",
  "app/models/ci/build.rb:385:in `detailed_status'",
...

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

image

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Visit http://gdk.test:3000/-/graphql-explorer

  2. Run the following query (update the runner ID to the ID of a runner that has jobs associated with it):

    {
  runner(id: "gid://gitlab/Ci::Runner/1220") {
    id
    runnerType
    jobs {
      nodes {
        detailedStatus {
          detailsPath
        }
      }
    }
  }
}

The queries generated by this branch should no longer generate a ci_pipelines query per job.

The functionality isn't changed in this MR, we only avoid an N+1 issue which is covered by the test.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Database queries

Old queries 
Processing by GraphqlController#execute as */*
  Parameters: {"query"=>"{\n  runner(id: \"gid://gitlab/Ci::Runner/1220\") {\n    id\n    runnerType\n    jobs {\n      nodes {\n        detailedStatus {\n          detailsPath\n        }\n      }\n    }\n  }\n}\n", "variables"=>"[FILTERED]", "graphql"=>{"query"=>"{\n  runner(id: \"gid://gitlab/Ci::Runner/1220\") {\n    id\n    runnerType\n    jobs {\n      nodes {\n        detailedStatus {\n          detailsPath\n        }\n      }\n    }\n  }\n}\n", "variables"=>"[FILTERED]"}}
  Ci::Runner Load (0.3ms)  SELECT "ci_runners".* FROM "ci_runners" WHERE "ci_runners"."id" = 1220 /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/resolvers/ci/runner_resolver.rb:31:in `block in find_runner'*/
  ↳ app/graphql/resolvers/ci/runner_resolver.rb:31:in `block in find_runner'
  Ci::Build Load (0.5ms)  SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" = 'Ci::Build' AND "ci_builds"."runner_id" = 1220 AND ("ci_builds"."status" NOT IN ('created')) ORDER BY "ci_builds"."id" DESC LIMIT 101 /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Ci::BuildMetadata Load (0.3ms)  SELECT "p_ci_builds_metadata".* FROM "p_ci_builds_metadata" WHERE "p_ci_builds_metadata"."build_id" IN (322, 321, 320, 318, 317, 316, 315, 314, 313) /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Project Load (0.5ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" IN (22, 21, 20) /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."source_id" IN (21, 22, 20) /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Namespace Load (0.4ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" IN (22, 70) /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Route Load (0.3ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."source_id" IN (22, 70) /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Ci::Pipeline Load (0.4ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 29 LIMIT 1 /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/ci/processable.rb:100:in `merge_train_pipeline?'*/
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  Ci::Pipeline Load (0.3ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 28 LIMIT 1 /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/ci/processable.rb:100:in `merge_train_pipeline?'*/
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  Ci::Pipeline Load (0.3ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1 /*application:web,correlation_id:01GMARA0VQCP1642187AA4JB4F,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/models/ci/processable.rb:100:in `merge_train_pipeline?'*/
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
  CACHE Ci::Pipeline Load (0.0ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 27 LIMIT 1
  ↳ app/models/ci/processable.rb:100:in `merge_train_pipeline?'
Completed 200 OK in 92ms (Views: 0.2ms | ActiveRecord: 3.8ms | Elasticsearch: 0.0ms | Allocations: 104378)
New queries 
Processing by GraphqlController#execute as */*
  Parameters: {"query"=>"{\n  runner(id: \"gid://gitlab/Ci::Runner/1220\") {\n    id\n    runnerType\n    jobs {\n      nodes {\n        detailedStatus {\n          detailsPath\n        }\n      }\n    }\n  }\n}\n", "variables"=>"[FILTERED]", "graphql"=>{"query"=>"{\n  runner(id: \"gid://gitlab/Ci::Runner/1220\") {\n    id\n    runnerType\n    jobs {\n      nodes {\n        detailedStatus {\n          detailsPath\n        }\n      }\n    }\n  }\n}\n", "variables"=>"[FILTERED]"}}
  Ci::Runner Load (0.8ms)  SELECT "ci_runners".* FROM "ci_runners" WHERE "ci_runners"."id" = 1220 /*application:web,correlation_id:01GMAR8CPR234CT9BRM9GEF1GZ,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/app/graphql/resolvers/ci/runner_resolver.rb:31:in `block in find_runner'*/
  ↳ app/graphql/resolvers/ci/runner_resolver.rb:31:in `block in find_runner'
  Ci::Build Load (0.7ms)  SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" = 'Ci::Build' AND "ci_builds"."runner_id" = 1220 AND ("ci_builds"."status" NOT IN ('created')) ORDER BY "ci_builds"."id" DESC LIMIT 101 /*application:web,correlation_id:01GMAR8CPR234CT9BRM9GEF1GZ,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Ci::BuildMetadata Load (0.8ms)  SELECT "p_ci_builds_metadata".* FROM "p_ci_builds_metadata" WHERE "p_ci_builds_metadata"."build_id" IN (322, 321, 320, 318, 317, 316, 315, 314, 313) /*application:web,correlation_id:01GMAR8CPR234CT9BRM9GEF1GZ,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Ci::Pipeline Load (1.1ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" IN (29, 28, 27) /*application:web,correlation_id:01GMAR8CPR234CT9BRM9GEF1GZ,endpoint_id:GraphqlController#execute,db_config_name:ci,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Project Load (0.6ms)  SELECT "projects".* FROM "projects" WHERE "projects"."id" IN (22, 21, 20) /*application:web,correlation_id:01GMAR8CPR234CT9BRM9GEF1GZ,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."source_id" IN (21, 22, 20) /*application:web,correlation_id:01GMAR8CPR234CT9BRM9GEF1GZ,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Namespace Load (0.3ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" IN (22, 70) /*application:web,correlation_id:01GMAR8CPR234CT9BRM9GEF1GZ,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
  Route Load (0.2ms)  SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."source_id" IN (22, 70) /*application:web,correlation_id:01GMAR8CPR234CT9BRM9GEF1GZ,endpoint_id:GraphqlController#execute,db_config_name:main,line:/lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'*/
  ↳ lib/gitlab/graphql/pagination/keyset/connection.rb:122:in `block in limited_nodes'
Completed 200 OK in 615ms (Views: 0.2ms | ActiveRecord: 28.7ms | Elasticsearch: 0.0ms | Allocations: 185557)
Edited by Pedro Pombeiro

Merge request reports