Skip to content

New SAST and security products jobs organization proposal

Currently, SAST and DAST are 2 single jobs, doing a lot of things. The security reports (CI View and Merge Request Widget) are built on the artifacts generated by these jobs.

While this approach is working well so far, it could be improved by splitting the task inside the jobs. I want to challenge the idea of having multiple SAST jobs, resulting in a single report, for the following reasons:

  • The SAST image needs to install tools and frameworks for each run, once a language is detected. This is long, slow, and heavily using bandwidth.
  • the SAST image is monolithic, which means every small change means releasing a new image
  • the security tests are not run concurrently.
  • Adding a custom test is complex (one has to create a new image from registry.gitlab.com/gitlab-org/security-products/sast and keep up-to-date)
  • Since custom tests are hard to perform, we can't create a community around the security products.
  • SAST being a single image, all variables from all tools are shared, leading to different variables for almost the same purpose.
  • Reports are relying on a single file (and soon, two, refs gitlab-org/gitlab-ee#5105)
  • If one tool is failing, the artifact if not generated

We could instead have multiple jobs in a sast (or dast) stage:

  • Each docker image is a tool, self-contained
  • Each image must generate an artifact following the same convention and specs
  • Users can create (and share) custom images
  • Jobs are run concurrently
  • Images don't need to install anything, everything is ready to run.
  • Variables are specific to each tool
  • If one job failed, only this report will be missing. The single job

The security reports can be created by the aggregation of all the artifacts (which should be trivial). Each job could also generate a raw report (for example, brakeman has a HTML report which can contain more information than what we show in our widgets). This means our report can be seen as a kind of dashboard for the whole picture, with details from each tool.

Also, we could group issues based on the name of the generated file. Tools in sast with dependency-scan-report.json will be categorized automatically as a "Dependency Scanning" tool (again, see gitlab-org/gitlab-ee#5105).

Food for thoughts.

/cc @bikebilly