Fix performance of the pipelines JSON endpoint
What does this MR do and why?
The origin of this performance problem is the pipelines tab of merge requests. When loading the pipelines, we send a request to "merge_requests/:id/pipelines" and return a JSON payload.
Previous improvement: !128097 (merged).
This JSON payload contains all "failed_builds". However, we don't need
this calculated array because we only need the number of failed builds.
In this MR, we are removing the failed_builds
field from the pipelines serializer.
Benchmark shows that this change improves the performance by 59%.
This change is behind
the feature flag ci_fix_performance_pipelines_json_endpoint
. (#427255 (closed))
Screenshots or screen recordings
With a merge request that has many pipelines;
Have this change to benchmark;
expose :failed_builds, if: -> (_, options) { !options[:ci_fix_performance_pipelines_json_endpoint] && can_retry? }, using: Ci::JobEntity do |pipeline|
pipeline.failed_builds.each do |build|
build.project = pipeline.project
end
end
Run the console with RAILS_PROFILE=true rails c
.
Run the benchmark;
require 'benchmark/ips'
user = User.find(1); nil
project = Project.find_by_full_path('root/gitlab-replica'); nil
merge_request = MergeRequest.find(105); nil
pipelines = Ci::PipelinesForMergeRequestFinder.new(merge_request, user).execute; nil
ActiveRecord::Base.logger = nil
::Gitlab::SafeRequestStore.ensure_request_store do
# warmup
PipelineSerializer.new(project: project, current_user: user).represent(pipelines, preload: true)
PipelineSerializer.new(project: project, current_user: user).represent(pipelines, preload: true, ci_fix_performance_pipelines_json_endpoint: true)
Benchmark.ips do |x|
x.warmup = 0
x.time = 20
x.report "without FF" do
PipelineSerializer.new(project: project, current_user: user).represent(pipelines, preload: true)
end
x.report "with FF" do
PipelineSerializer.new(project: project, current_user: user).represent(pipelines, preload: true, ci_fix_performance_pipelines_json_endpoint: true)
end
x.compare!
end; nil
end
result;
Calculating -------------------------------------
without FF 1.723 (± 0.0%) i/s - 34.000 in 20.157212s
with FF 2.731 (± 0.0%) i/s - 54.000 in 20.210030s
Comparison:
with FF: 2.7 i/s
without FF: 1.7 i/s - 1.59x slower
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.