feat: Add MarkDroppedAsResolved service to Security Ingestion flow
What does this MR do and why?
Adds ScheduleMarkDroppedAsResolved
and MarkDroppedAsResolvedWorker
for resolving
vulnerabilities tied to identifiers that are no longer detected.
This worker depends on the presence of report.scan.primary_identifiers
to get
an exhaustive list of all identifiers for which the analyzer scans (as opposed to
all identifiers detected).
See related schema update to support scan.primary_identifiers
field (not strictly
necessary since we use additionalProperties: true
)
gitlab-org/security-products/security-report-schemas!126 (merged)
Relates to #368284 (closed)
Background
Scenario:
-
semgrep-sast
is updated to remove rule that has been identified as overly noisy and prone to producing high false-positive results - Report includes list of included rules executed during scan
- Report parser identifies all existing findings that are not included in list of rules and auto-resolves due to identifier no longer being included
graph LR
repository --> brakeman_sast
subgraph analyzer pipeline
brakeman_sast --> report1a["noisy-rule-123 dropped"]
report1a --> report1b["scan.identifiers populated"]
report1b --> reportout1("gl-sast-report.json")
end
subgraph ingestion
reportout1 --> ingestsvc["IngestReportService"]
ingestsvc--> schedulesvc["ScheduleMarkDroppedAsResolved"]
end
subgraph sidekiq
schedulesvc --> worker["MarkDroppedAsResolvedWorker"]
end
Depends on:
{
"vulnerabilities": [],
"scan": {
"primary_identifiers": [
{
"type": "semgrep_id",
"name": "gosec.G106-1",
"value": "gosec.G106-1"
}
],
"status": "success"
}
}
Screenshots
list | details |
---|---|
How to set up and validate locally
Test project export using modified fixtures: 2022-08-16_16-45-709_root_go_export.tar.gz
- Enable the feature flag:
Feature.enable(:sec_mark_dropped_findings_as_resolved)
- Run default pipeline
- Confirm presence of 3 vulnerabilities on dashboard
- Update
.gitlab-ci.yml
to referencegl-sast-report.tests-go-with-scan-primary-identifiers.json
- Confirm no change in behavior (still 3 vulnerabilities on dashboard)
- Update
.gitlab-ci.yml
to referencegl-sast-report.tests-go-with-scan-primary-identifiers-dropping-G104.json
- Confirm auto-resolution of dropped identifier (2 remaining detected vulnerabilities, 1 resolved and no longer detected)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.