Add caching to variables calculation of builds (!58286) · Merge requests · GitLab.org / GitLab

Furkan Ayhan requested to merge 326475-reducing-sql-queries into master Apr 01, 2021

What does this MR do?

This MR aims to lower the number of SQL queries used in Ci::CreatePipelineService.

In the step of Gitlab::Ci::Pipeline::Chain::Seed, the method scoped_variables is called for every job. The method scoped_variables depends on Ci::Pipeline#predefined_variables and Project#predefined_variables.

Ci::Pipeline#has_kubernetes_active? is called when calculating Ci::Pipeline#predefined_variables.
Project#requirements_ci_variables is called when calculating Project#predefined_variables.

In this MR, I added strong_memoize to those methods.

In the screenshots section below, you will see that we don't have these repetitive calls anymore:

  RequirementsManagement::Requirement Exists? (0.2ms)
  Clusters::Cluster Load (1.6ms)
  Clusters::Cluster Load (0.3ms)

Screenshots (strongly suggested)

Note: I used the feature flag ci_async_initial_pipeline_processing from !57589 (merged). So, the initial processing is excluded from the logs.

Example CI:

include:
  - local: includes/sub_file_0.yml
  - local: includes/sub_file_1.yml
  - local: includes/sub_file_2.yml

stages:
  - prepare
  - optimize
  - build
  - pre-test
  - test
  - package
  - documentation
  - deploy
  - smoke-test
  - post-deploy

.extend_this_job:
  script: echo I am extended
  stage: prepare

prepare_1:
  extends: ".extend_this_job"

prepare_2:
  script: "echo hello"
  stage: "prepare"

prepare_3:
  script: "echo hello"
  stage: "prepare"

optimize_1:
  script: "echo hello"
  stage: "optimize"

build_1:
  script: "echo hello"
  stage: "build"

build_2:
  script: "echo hello"
  stage: "build"

build_4:
  script: "echo hello"
  stage: "build"

build_5:
  script: "echo hello"
  stage: "build"

build_10:
  script: "echo hello"
  stage: "build"

build_12:
  script: "echo hello"
  stage: "build"

build_13:
  script: "echo hello"
  stage: "build"

pre-test_1:
  script: "echo hello"
  stage: "pre-test"

pre-test_2:
  script: "echo hello"
  stage: "pre-test"

pre-test_3:
  script: "echo hello"
  stage: "pre-test"
  needs: ["build_2","prepare_1","optimize_1"]

test_1:
  script: "echo hello"
  stage: "test"

package_1:
  script: "echo hello"
  stage: "package"
  needs: ["build_2","build_5","optimize_1"]

documentation_1:
  script: "echo hello"
  stage: "documentation"
  needs: ["build_12","optimize_1","build_10","package_1"]

documentation_2:
  script: "echo hello"
  stage: "documentation"
  needs: ["pre-test_2","build_1"]

documentation_3:
  script: "echo hello"
  stage: "documentation"

documentation_5:
  script: "echo hello"
  stage: "documentation"
  needs: ["build_13","prepare_3","pre-test_3","test_1","optimize_1"]

documentation_6:
  script: "echo hello"
  stage: "documentation"

deploy_1:
  script: "echo hello"
  stage: "deploy"

deploy_2:
  script: "echo hello"
  stage: "deploy"

deploy_3:
  script: "echo hello"
  stage: "deploy"

deploy_4:
  script: "echo hello"
  stage: "deploy"

smoke-test_1:
  script: "echo hello"
  stage: "smoke-test"
  dependencies: [deploy_3, deploy_4]

smoke-test_2:
  script: "echo hello"
  stage: "smoke-test"
  needs: [deploy_1, deploy_2]

smoke-test_3:
  script: "echo hello"
  stage: "smoke-test"

smoke-test_4:
  script: "echo hello"
  stage: "smoke-test"

post-deploy_1:
  script: "echo hello"
  stage: "post-deploy"

post-deploy_2:
  script: "echo hello"
  stage: "post-deploy"

post-deploy_3:
  script: "echo hello"
  stage: "post-deploy"

post-deploy_11:
  script: "echo hello"
  stage: "post-deploy"
  dependencies: [smoke-test_3, smoke-test_4]

post-deploy_12:
  script: "echo hello"
  stage: "post-deploy"
  needs: []

Run command:

reload!
project = Project.find(86)
current_user = User.first
create_params = { ref: 'master' }

RequestStore.begin!
pipeline = Ci::CreatePipelineService.new(project, current_user, create_params).execute(:web, ignore_skip_ci: true, save_on_errors: false)
RequestStore.end!
RequestStore.clear!

::Ci::DestroyPipelineService.new(project, current_user).execute(pipeline)

Before optimization:

[9] pry(main)> project = Project.last
  Project Load (0.8ms)
  Route Load (0.5ms)
=> #<Project id:86 root/very-big-ci>>
[10] pry(main)> current_user = User.first
  User Load (1.0ms)
=> #<User id:1 @root>
[11] pry(main)> create_params = { ref: 'master' }
=> {:ref=>"master"}
[12] pry(main)>
[13] pry(main)> RequestStore.begin!
=> true
[14] pry(main)> Ci::CreatePipelineService.new(project, current_user, create_params).execute(:web, ignore_skip_ci: true, save_on_errors: false)
"***** Gitlab::Ci::Pipeline::Chain::Build *****"
  ProjectCiCdSetting Load (0.4ms)
"***** Gitlab::Ci::Pipeline::Chain::Build::Associations *****"
"***** Gitlab::Ci::Pipeline::Chain::Validate::Abilities *****"
  ProjectFeature Load (0.4ms)
  Namespace Load (0.6ms)
  MergeRequest Exists? (0.8ms)
  Group Load (0.4ms)
  ProtectedBranch Load (0.4ms)
  ProtectedBranch::PushAccessLevel Load (0.4ms)
"***** Gitlab::Ci::Pipeline::Chain::Validate::Repository *****"
"***** Gitlab::Ci::Pipeline::Chain::Config::Content *****"
"***** Gitlab::Ci::Pipeline::Chain::Config::Process *****"
  Route Load (0.3ms)
  RepositoryLanguage Load (0.2ms)
  ProjectAutoDevops Load (0.2ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
"***** Gitlab::Ci::Pipeline::Chain::RemoveUnwantedChatJobs *****"
"***** Gitlab::Ci::Pipeline::Chain::Skip *****"
"***** Gitlab::Ci::Pipeline::Chain::SeedBlock *****"
"***** Gitlab::Ci::Pipeline::Chain::EvaluateWorkflowRules *****"
"***** Gitlab::Ci::Pipeline::Chain::Seed *****"
  InternalId Load (0.3ms)
  InternalId Load (0.2ms)
  InternalId Update (0.2ms)
  Ci::Ref Load (0.2ms)
  RequirementsManagement::Requirement Exists? (0.2ms)
  MergeRequest Load (0.4ms)
  Clusters::Cluster Load (0.3ms)
  Ci::FreezePeriod Load (0.3ms)
  Ci::Build Load (0.8ms)
  Ci::Variable Load (0.3ms)
  RequirementsManagement::Requirement Exists? (0.2ms)
  Clusters::Cluster Load (1.6ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.6ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.8ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.6ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.6ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.6ms)
  GitlabSubscription Load (0.5ms)
  Plan Load (0.5ms)
  PlanLimits Load (0.4ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.0ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.8ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.8ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.7ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.7ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.7ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (2.0ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.0ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.7ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.2ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.4ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.5ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.6ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.0ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.7ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.3ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.7ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.1ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.6ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.3ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.5ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (1.9ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (0.8ms)
  RequirementsManagement::Requirement Exists? (0.3ms)
  Clusters::Cluster Load (2.2ms)
  Clusters::Cluster Load (0.4ms)
  Ci::Build Load (1.1ms)
"***** Gitlab::Ci::Pipeline::Chain::Limit::Size *****"
"***** Gitlab::Ci::Pipeline::Chain::Limit::Deployments *****"
"***** Gitlab::Ci::Pipeline::Chain::Validate::External *****"
"***** Gitlab::Ci::Pipeline::Chain::Populate *****"
"***** Gitlab::Ci::Pipeline::Chain::StopDryRun *****"
"***** Gitlab::Ci::Pipeline::Chain::Create *****"
  Ci::Pipeline Create (0.6ms)
  Ci::Stage Create (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.3ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.3ms)
  Ci::Build Create (0.4ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.3ms)
  Ci::Build Create (0.4ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.3ms)
  Ci::Build Create (0.4ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.3ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.7ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.7ms)
  Ci::BuildNeed Bulk Insert (0.5ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Stage Create (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.7ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.7ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildNeed Bulk Insert (0.3ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.7ms)
  Ci::BuildNeed Bulk Insert (0.4ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildNeed Bulk Insert (0.3ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.4ms)
  Ci::BuildNeed Bulk Insert (0.3ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.6ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.7ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.7ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildNeed Bulk Insert (0.2ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.7ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.7ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.4ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (1.2ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.4ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.3ms)
  Ci::Build Load (0.3ms)
  Ci::Build Create (0.4ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::PipelineConfig Create (0.7ms)
"***** Gitlab::Ci::Pipeline::Chain::Limit::Activity *****"
"***** Gitlab::Ci::Pipeline::Chain::Limit::JobActivity *****"
"***** Gitlab::Ci::Pipeline::Chain::CancelPendingPipelines *****"
  Ci::Pipeline Load (2.1ms)
"***** Gitlab::Ci::Pipeline::Chain::Metrics *****"
"***** Gitlab::Ci::Pipeline::Chain::TemplateUsage *****"
"***** Gitlab::Ci::Pipeline::Chain::Pipeline::Process *****"
  MergeRequest Load (1.5ms)

After optimization:

[9] pry(main)> project = Project.last
  Project Load (1.0ms)
  Route Load (0.5ms)
=> #<Project id:86 root/very-big-ci>>
[10] pry(main)> current_user = User.first
  User Load (1.0ms)
=> #<User id:1 @root>
[11] pry(main)> create_params = { ref: 'master' }
=> {:ref=>"master"}
[12] pry(main)>
[13] pry(main)> RequestStore.begin!
=> true
[14] pry(main)> Ci::CreatePipelineService.new(project, current_user, create_params).execute(:web, ignore_skip_ci: true, save_on_errors: false)
"***** Gitlab::Ci::Pipeline::Chain::Build *****"
  ProjectCiCdSetting Load (0.4ms)
"***** Gitlab::Ci::Pipeline::Chain::Build::Associations *****"
"***** Gitlab::Ci::Pipeline::Chain::Validate::Abilities *****"
  ProjectFeature Load (0.4ms)
  Namespace Load (0.8ms)
  MergeRequest Exists? (0.7ms)
  Group Load (0.4ms)
  ProtectedBranch Load (0.3ms)
  ProtectedBranch::PushAccessLevel Load (0.3ms)
"***** Gitlab::Ci::Pipeline::Chain::Validate::Repository *****"
"***** Gitlab::Ci::Pipeline::Chain::Config::Content *****"
"***** Gitlab::Ci::Pipeline::Chain::Config::Process *****"
  Route Load (0.3ms)
  RepositoryLanguage Load (0.3ms)
  ProjectAutoDevops Load (0.2ms)
  RequirementsManagement::Requirement Exists? (0.4ms)
"***** Gitlab::Ci::Pipeline::Chain::RemoveUnwantedChatJobs *****"
"***** Gitlab::Ci::Pipeline::Chain::Skip *****"
"***** Gitlab::Ci::Pipeline::Chain::SeedBlock *****"
"***** Gitlab::Ci::Pipeline::Chain::EvaluateWorkflowRules *****"
"***** Gitlab::Ci::Pipeline::Chain::Seed *****"
  InternalId Load (0.4ms)
  InternalId Load (0.2ms)
  InternalId Update (0.2ms)
  Ci::Ref Load (0.3ms)
  MergeRequest Load (0.9ms)
  Clusters::Cluster Load (0.3ms)
  Ci::FreezePeriod Load (0.4ms)
  Ci::Build Load (1.1ms)
  Ci::Variable Load (0.4ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.8ms)
  Ci::Build Load (0.7ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.7ms)
  GitlabSubscription Load (0.4ms)
  Plan Load (0.3ms)
  PlanLimits Load (0.4ms)
  Ci::Build Load (0.9ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.8ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.8ms)
  Ci::Build Load (0.8ms)
  Ci::Build Load (0.9ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.8ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.7ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.8ms)
  Ci::Build Load (0.8ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (1.0ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.7ms)
  Ci::Build Load (0.8ms)
"***** Gitlab::Ci::Pipeline::Chain::Limit::Size *****"
"***** Gitlab::Ci::Pipeline::Chain::Limit::Deployments *****"
"***** Gitlab::Ci::Pipeline::Chain::Validate::External *****"
"***** Gitlab::Ci::Pipeline::Chain::Populate *****"
"***** Gitlab::Ci::Pipeline::Chain::StopDryRun *****"
"***** Gitlab::Ci::Pipeline::Chain::Create *****"
  Ci::Pipeline Create (0.7ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.7ms)
  Ci::Build Load (0.7ms)
  Ci::Build Create (0.9ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.7ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.8ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::Stage Create (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.7ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Create (0.7ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::Stage Create (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.9ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.5ms)
  Ci::BuildNeed Bulk Insert (0.3ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Stage Create (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.9ms)
  Ci::BuildMetadata Create (0.5ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Create (0.6ms)
  Ci::BuildNeed Bulk Insert (0.3ms)
  Ci::BuildMetadata Create (0.6ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildNeed Bulk Insert (0.4ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildNeed Bulk Insert (0.3ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.2ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.5ms)
  Ci::BuildNeed Bulk Insert (0.3ms)
  Ci::BuildMetadata Create (0.5ms)
  ActsAsTaggableOn::Tagging Load (0.5ms)
  Ci::Build Load (0.8ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::Stage Create (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.4ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.4ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.4ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.6ms)
  Ci::Build Create (0.7ms)
  Ci::BuildNeed Bulk Insert (0.3ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (1.0ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.5ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Stage Create (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.7ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.5ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.4ms)
  ActsAsTaggableOn::Tagging Load (0.3ms)
  Ci::Build Load (0.6ms)
  Ci::Build Load (0.4ms)
  Ci::Build Create (0.5ms)
  Ci::BuildMetadata Create (0.3ms)
  ActsAsTaggableOn::Tagging Load (0.2ms)
  Ci::Build Load (0.7ms)
  Ci::Build Load (0.6ms)
  Ci::Build Create (0.6ms)
  Ci::BuildMetadata Create (0.7ms)
  ActsAsTaggableOn::Tagging Load (0.4ms)
  Ci::PipelineConfig Create (1.2ms)
"***** Gitlab::Ci::Pipeline::Chain::Limit::Activity *****"
"***** Gitlab::Ci::Pipeline::Chain::Limit::JobActivity *****"
"***** Gitlab::Ci::Pipeline::Chain::CancelPendingPipelines *****"
  Ci::Pipeline Load (3.1ms)
"***** Gitlab::Ci::Pipeline::Chain::Metrics *****"
"***** Gitlab::Ci::Pipeline::Chain::TemplateUsage *****"
"***** Gitlab::Ci::Pipeline::Chain::Pipeline::Process *****"
  MergeRequest Load (1.8ms)

Does this MR meet the acceptance criteria?

Conformity

📋 Does this MR need a changelog?
- I have included a changelog entry.
- I have not included a changelog entry because _____.
Documentation (if required)
Code review guidelines
Merge request performance guidelines
Style guides
Database guides
Separation of EE specific content

Availability and Testing

Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
Tested in all supported browsers
Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

Label as security and @ mention @gitlab-com/gl-security/appsec
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
Security reports checked/validated by a reviewer from the AppSec team

Edited Apr 09, 2021 by Furkan Ayhan

Add caching to variables calculation of builds