First-class vulnerabilities backend MVC implementation

Overview

This issue is about the ~backstage implementation we need in order to implement vulnerabilities as first-class entities in GitLab.

Development log

Status

(copied from backend MVC implementation plan)

Vulnerabilities and Findings

• Rename existing Vulnerabilities API to Vulnerability Findings API
  • in codebase
  • in documentation | !19029 (merged), it was decided to do as breaking change in %12.5
• Create vulnerabilities table (see schema) and model
• Check (and extend, if necessary) the permissions for the new Vulnerabilities API (basically, clone those from Vulnerability Findings API)
• Create Vulnerabilities API with the following capabilities
  • Get a list of project's Vulnerabilities !18269 (merged)
  • Create Vulnerability from Finding (Findings are backed by vulnerability_occurrences table) | First !19029 (merged), then !18283 (merged)
  • [-] Delete Vulnerability (and disassociate from all of its related Findings as a cascade) removed because is deemed unnecessary
  • Resolve Vulnerability (and resolve all of its related Findings as a cascade) !18439 (merged)
  • Dismiss Vulnerability (and dismiss all of its related Findings as a cascade) !18495 (merged)
  • [-] (Optional, depending on capacity) Associate Finding to (the existing) Vulnerability post-MVC, extracted into #34531 (closed)
• ~"workflow::In review" Add documentation stubs for the new Vulnerabilities API !17364 (merged) and !28423 (merged)

Vulnerabilities and Issues + Vulnerability attributes

• [-] Creating Issues associated with Vulnerabilities: extracted into #34564 (closed)
• [-] Vulnerabilities should be created as confidential by default (mimic the access permissions of confidential Issues): extracted into #34430 (closed)
• [-] Vulnerabilities must have "Report type" attribute: extracted into #34431 (closed)

Decisions

• This issue's work result should not be publicly exposed until #13561 (closed) is implemented, so:
  • the documentation for the API endpoints will be present in a form of stubs: pages but probably with some kind of "Under construction" notices
  • the API endpoints are going to be hidden under a feature flag, disabled
• After the sync+brainstorm on MVC scope of First-class Vulnerabilities (recording (internal), design board), it became clear MVC scope may expand (it's still an open question though):
  • creating Issues from Vulnerabilities
  • Vulnerabilities should be created as confidential by default (mimic the access permissions of confidential Issues)
  • Vulnerabilities must have "Report type" attribute
• Scheduling of the expanded scope for this issue (see the previous item):
  • implementing both of the Vulnerabilities and Findings and Vulnerabilities and Issues + Vulnerability attributes is not realistic to achieve in %12.4
  • there's a suggestion to move the Vulnerabilities and Issues + Vulnerability attributes part to %12.5, put it as the last part to work on for #10242 (closed) to avoid blocking the frontend team while working on #13561 (closed)
  • the precise scheduling (and probably, extraction of Vulnerabilities and Issues + Vulnerability attributes into a separate issue) is TBD in a sync conversation between @vzagorodny, @twoodham, @stkerr, @leipert, and @andyvolpe
• the API call Delete Vulnerability was removed from the API spec; details here
• the API call Dismiss Vulnerability will create Vulnerabilities::Feedback records with feedback type dismissal for all Findings of a Vulnerability in a loop using the current VulnerabilityFeedback::CreateService; it's sufficient for an MVC, a bulk dismiss service could be created later
• the API call Resolve Vulnerability just moves a Vulnerability from open to closed state; but each Finding now has it's own computed state (that is not persisted in the DB), and if a Finding is associated with a closed Vulnerability and it does not have a dismissal Feedback, it's considered resolved

Related issues

• Product discovery
• Engineering discovery
  • Engineering discovery decisions
• Feature MVC implementation

Description

First-class vulnerabilities are needed to deal with advanced vulnerability management flows. At the moment, vulnerabilities are available in individual security reports (like SAST, DAST, etc.), and they are partially persisted into the database.

This implementation, even if an MVC and enough to bring us to the current state, is not scalable anymore. We have requirements, like creating vulnerabilities from issues, or from HackerOne, that cannot fit in the same flow.

The idea is to create vulnerabilities as first-class objects and feed them using many different sources (issues, reports, external, slack commands, etc).

Once the vulnerability is created, it can be managed in the very same way, no matter what was the source.