Add GlobalAdvisoryScanWorker
What does this MR do and why?
This MR makes Continuous Vulnerability Scanning (CVS) on advisory DB change Globally Available (GA) when the flag global_dependency_scanning_on_advisory_ingestion
is enabled.
- Add new development feature flag named
global_dependency_scanning_on_advisory_ingestion
(disabled by default). - Add new
global
param toSbom::PossiblyAffectedOccurrencesFinder
,::GitLab::VulnerabilityScanning::AdvisoryScanner
, andPackageMetadata::AdvisoryScanService
. When it'strue
, the scan is global. However,AdvisoryScanner
skips global scans whenglobal_dependency_scanning_on_advisory_ingestion
is disabled. - Add new
global
param to theTrackCvsService
. The event action isglobal_scan
whenglobal
istrue
. - Add new
GlobalAdvisoryScanWorker
. It subscribes to the::PackageMetadata::IngestedAdvisoryEvent
. It setsglobal
totrue
when calling theAdvisoryScanService
.
The AdvisoryScanWorker
, which handles projects where CVS has been explicitly enabled in security settings, is updated to set global
to false
.
Projects where CVS has been enabled (in project settings) are scanned by the AdvisoryScanWorker
and the GlobalAdvisoryScanWorker
. This is safe b/c the AdvisoryScanner
is idempotent; it won't create duplicate vulnerabilities. There's a bit of a overhead but it's negligible b/c CVS has been enabled in very few projects (~50).
Eventually the AdvisoryScanWorker
needs to be removed. This will be covered by a separate issue, and implemented in a latter milestone.
The documentation is updated in a separate issue.
See CVS on advisory DB changes (GA) (#426089 - closed)
Query plans
When global
is true
, the PossiblyAffectedOccurrencesFinder
behaves like when it was introduced in !131454 (merged).
As a result, we can rely on the query plans that were shared back then; these query plans have already been approved.
PossiblyAffectedOccurrencesFinder
was later updated in !128941 (merged)
to only consider projects where project_security_settings.continuous_vulnerability_scans_enabled
is true
.
Again, this changes is reverted when global
is true
.
Historically the finder has been introduced and modified by the following MRs, in that exact order:
- Only include sbom_occurrences with CVS enabled (!131454 - merged)
- Memoize component_id method (!131211 - merged)
- Add PossiblyAffectedOccurrencesFinder class (!128941 - merged)
Queries should leverage the following DB indexes:
CREATE UNIQUE INDEX index_sbom_components_on_component_type_name_and_purl_type ON sbom_components USING btree (name, purl_type, component_type);
CREATE INDEX index_sbom_occurrences_on_component_id_and_id ON sbom_occurrences USING btree (component_id, id);
These haven't changed since Add PossiblyAffectedOccurrencesFinder class (!128941 - merged).
Logging
As suggested during the review, scan workers now log an error when the advisory cannot be found, instead of an info message.
Performance
Right now global scans are too slow to be enabled in production, so we won't enable the new feature flag until Ingest vulnerabilities from multiple projects a... (#420768 - closed) and Make AdvisoryScanner ingest multiple finding maps (#429834 - closed) are completed. The code introduce in this MR shouldn't change though.
See #423578 (comment 1615208412) for context.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #426089 (closed)