Improve performance of show action for Projects::CompareController under load to meet target
After adding in a new performance test for the Branches Repository Compare page it was found that Projects::CompareController#show
is unperformant:
* Environment: 10k
* Environment Version: 15.10.0-pre `6451e57830a`
* Option: 60s_200rps
* Date: 2023-03-02
* Run Time: 1h 43m 44.43s (Start: 04:40:20 UTC, End: 06:24:04 UTC)
* GPT Version: v2.12.2
NAME | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
---------------------------------------------------------|-------|----------------------|-----------|-----------------------|----------------|---------
web_project_repository_compare | 20/s | 5.68/s (>0.80/s) | 3167.18ms | 3281.06ms (<7500ms) | 100.00% (>99%) | Passed¹ | 100.00% (>99%) | Passed
Looking at the environment metrics it appears that the endpoint is hitting the Rails:
Previous results with Severity::1.
Before Pagination was added in !59162 (merged)
█ Results summary
* Environment: 10k
* Environment Version: 13.11.0-pre `8b407f0a3fb`
* Option: 60s_200rps
* Date: 2021-04-20
* Run Time: 1m 11.77s (Start: 16:46:40 UTC, End: 16:47:51 UTC)
* GPT Version: v2.7.0
NAME | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
----------------------------------------|------|------------------|-----------|-----------------------|----------------|----------------
web_project_repository_compare | 20/s | 1.78/s (>1.60/s) | 9704.03ms | 10522.16ms (<14000ms) | 100.00% (>99%) | Passed
The testing was done on our test 10k Reference Architecture environment as standard at 200 RPS with the project being tested a copy of gitlabhq (tarball can be found here).
For the repository comparison these branches are being used - https://staging.gitlab.com/gpt/large_projects/gitlabhq1/-/compare/12-2-stable...12-1-stable - diff size is 200 commits, 546 changed files with 8342 additions and 2856 deletions
.
Steps to reproduce
- Check out the Performance Tool
- Run the specific test with the
run-k6
command.
What is the current bug behavior?
The results above show that the Branches/Tags Repository Compare has a P90 of 6129.19ms.
What is the expected correct behavior?
As per our performance targets this endpoint is under the second tier target which is severity2. Task is to improve the endpoint's performance into next tier.
@vyaklushin)
Implementation proposal (byThe problem is in the large number of forks. We currently fetch and populate all of them on each page load. That's pretty slow DB request for projects with many forks.
Instead, we can use an alternative approach from the merge request creation page: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/new.
The dropdown for target project doesn't load any elements by default (only the existing project). When user clicks on it, frontend sends a request to load first 20 projects. Search section in the dropdown allows to select the project outside of initial search result.
The backend implementation for search can be reused for compare endpoint as well.
Frontend can re-use the same component from merge request creation page to significantly improve the performance.