Migrate Java SAST coverage from SpotBugs to Semgrep
Recap
- In 14.10, we released Semgrep-based scanning for Java. It ran alongside SpotBugs.
- Since then, we have gained confidence in the implementation and worked to ease the transition from SpotBugs scanning to Semgrep-based scanning.
- The CI/CD template change to only trigger Semgrep-based scanning for Java has rolled out in GitLab.com and will be included in 15.4.
See the deprecation issue, particularly at #352554 (comment 1104150078), for details of how to work around any potential issues.
Problem to solve
GitLab SAST uses various analyzers to scan code for vulnerabilities. For technical reasons, some of these analyzers require building the project that's being scanned.
The need to build code in SpotBugs contributes to:
- An increased number of Support tickets. SpotBugs not working out of the box is estimated off-hand to be responsible for around 25% of Static Analysis support tickets, of which about half relate to something about the build (multi-project monorepo, dependency issues, etc.). (We can make this analysis more concrete.)
- Frequent issues being raised in the issue tracker about environmental differences, like Java versions that are not included by default.
- A distinctly "un-magical" out-of-the-box experience, which is problematic because first impressions often come up in situations like POC deployments where we'd like to prove value without lots of effort or help from Support/Customer Success.
- Trouble with certain configurations, like Ant builds, which appear not to be working.
- Larger analyzer image sizes, due to dependencies being included to facilitate out-of-the-box builds.
If we were able to remove the build requirement, we could mitigate or eliminate most of those problems.
User experience goal
The user should be able to run SAST scans against Java code without having to consider build-time dependencies, runtime environment requirements, or other complications. If they bring source code, we should be able to scan it.
Proposal
Port existing rulesets to Semgrep rules.
Change the GitLab-managed SAST CI/CD template to send Java code to Semgrep instead of SpotBugs.
Maintain SpotBugs for languages that Semgrep doesn't support.
Details
Here are the security-related rules currently supported in:
The Programming Languages on GitLab.com Sisense dashboard (internal link; GitLab team members only) shows an order of magnitude more Java than other supported JVM languages (Groovy, Kotlin, Scala), at least 7x higher than the next language down the list.
We have a longer-term strategy of moving more language analyzers toward Semgrep. While this does not fully remove an existing analyzer, it should significantly shift workload to Semgrep.
Documentation
We would need to update Supported languages and frameworks as well as various examples within the main SAST page.
What does success look like, and how can we measure that?
- Reduced number of SpotBugs analyzer failures would be a primary indicator of success (assuming that build failures currently manifest in these metrics)
- Reduced Support interactions for Java projects would be a significant indicator of a better user experience
- Reduced number of SpotBugs-related issues in the GitLab issue tracker would similarly indicate that fewer users are having problems
- Reduced SpotBugs analyzer jobs would indicate that the change is actually meaningfully moving our workload from SpotBugs to Semgrep
Links / references
More issues/Zendesk tickets to come.
See the deprecation issue for details of what has been announced: SAST Deprecation: Analyzer consolidation and CI... (#352554 - closed)
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.