Get vulnerability_finding from existing queried data
What does this MR do?
Addresses https://gitlab.com/gitlab-org/gitlab/-/issues/338085
- When invoking
Vulnerabilities::CreateService
fromSecurity::StoreReportService
,vulnerability_finding
is already loaded form DB, but still we pass theid
ofvulnerability_finding
. This triggers an unwanted query inVulnerabilities::CreateService
which creates a sub-transaction. This could be avoided by passing thevulnerability_finding
record toVulnerabilities::CreateService
. - The transaction block in
Vulnerabilities::CreateService
rollbacks only whenActiveRecord::RecordNotFound
is raised. This happens when thevulnerability_finding
is not found or already has a different vulnerability associated with it. This is a valid case when the service is invoked from the Create Vulnerability API. But when the service is invoked fromSecurity::StoreReportService
, we are sure that thevulnerability_finding
would not have a different vulnerability associated with it. This means that we don't need to dolock_for_confirmation!
which usesSELECT FOR UPDATE
and it is not performant. - Because the transaction fails only for
ActiveRecord::RecordNotFound
, we don't need to wrapStatistics::UpdateService
andHistoricalStatistics::UpdateService
in the transaction.
Improvements
By conditionally removing the sub-transaction in Vulnerabilities::CreateService
, we can reduce 1 query everytime the service is invoked, which means if the Security::StoreReportService
processes a report with 1000 vulnerabilities, 1000 queries could be reduced from the overall queries count. And since we are moving Statistics::UpdateService
and HistoricalStatistics::UpdateService
out of the sub-transaction, the number of nested sub-transactions from those 2 services are reduced.
I used Measurable
to measure Security::StoreReportsService
by running ee/spec/services/security/store_report_service_spec.rb
following Stan's comment https://gitlab.com/gitlab-org/gitlab/-/issues/338085#note_648108114 with a test gl-sast-report-json
containing 1000 vulnerabilities. These are the results:
Before
I, [2021-08-13T16:58:55.412990 #23242] INFO -- : ---
:class: Security::StoreReportsService
:gc_stats:
:time_to_finish: 44.25615599998855
:number_of_sql_calls: 18713
:memory_usage: 0.0 MiB
:label: process_23242
After
I, [2021-08-13T16:54:18.721069 #22385] INFO -- : ---
:class: Security::StoreReportsService
:gc_stats:
:time_to_finish: 43.77439299999969
:number_of_sql_calls: 17710
:memory_usage: 0.0 MiB
:label: process_22385
Does this MR meet the acceptance criteria?
Conformity
-
I have included changelog trailers, or none are needed. (Does this MR need a changelog?) -
I have added/updated documentation, or it's not needed. (Is documentation required?) -
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) -
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) -
I have self-reviewed this MR per code review guidelines. -
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) -
I have followed the style guides. -
This change is backwards compatible across updates, or this does not apply.
Availability and Testing
-
I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) -
I have tested this MR in all supported browsers, or it's not needed. -
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.
Security
Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team