Limit the number of results requested when searching for commits
What does this MR do?
Improve performance of non-Elasticsearch commit search by limiting the number of results requested.
In #216908 (closed), it was pointed out that limit
is not passed when we call find_commits_by_message
. If not given, find_commits_by_message
defaults the limit to 1000, but we don't actually need that many results right away.
This MR changes commit search to use the same limiting strategy that is used by blob search. We start by limiting to 100 results (defined by the COUNT_LIMIT
constant), and increase the limit as additional pages are requested.
Performance test results for web search (60s_20rps
against development environment)
Before
✓ { endpoint:commits_count }...: avg=215.78ms min=149.52ms med=168.04ms max=1090.57ms p(90)=271.44ms p(95)=293.89ms
✗ { endpoint:commits }.........: avg=1440.24ms min=1286.12ms med=1413.65ms max=1996.03ms p(90)=1528.53ms p(95)=1675.51ms
After
✓ { endpoint:commits_count }...: avg=198.30ms min=150.31ms med=175.71ms max=738.41ms p(90)=279.20ms p(95)=302.85ms
✗ { endpoint:commits }.........: avg=1007.94ms min=866.59ms med=996.73ms max=1666.52ms p(90)=1103.04ms p(95)=1118.90ms
~28% improvement in p90 response times when searching for commits. The commits_count
endpoint seems relatively unchanged.
A change was also made to the formatting of the number of commit results, to make it consistent with other scopes. This is shown in the screenshots below.
Screenshots
Before | After |
---|---|
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry - [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines - [-] Style guides
- [-] Database guides
- [-] Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec
- [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Closes #216908 (closed)