Create SAST analyzer for Python using Semgrep

Release notes

Problem to solve

We need to evaluate semgrep against our existing analyzers. The best way we know to enable this need is to create a semgrep-based analyzer that we can use to compare against one we already have. Based upon usage, an analyzer which scans against python has been identified as our highest priority.

This new analyzer needs to conform to existing SAST features at their identified licensing levels.

Questions to Answer

• Should we have semgrep in a single analyzer project or many analyzer projects for each language/framework?
  • Can semgrep scan multiple languages/frameworks in a single run? This was answered today in a call with r2c, semgrep will scan as many languages for which you have rules defined. As that is true, we should start with a single analyzer project that will work on many projects.

Proposal

• Create https://gitlab.com/gitlab-org/security-products/analyzers/semgrep
• Wrap semgrep with our analyzer pattern
• Bake semgrep rules into Docker image so that the rules are versioned.
• Limit semgrep rules to only python rules so that we don't create duplicates found in other analyzers.
• Ensure feature parity with other analyzers:
  • Enabled by existing SAST vendored template behind the SAST_EXPERIMENTAL_FEATURES env var.
  • Run at same licensure as other SAST analyzers.
  • Match existing support for custom rulesets.
  • Field mappings - severity, location, field descriptions
  • ultimate licensing limit for requisite features
    • custom rulesets (dir/path exlusions)
  • SEARCH_MAX_DEPTH
  • Logging
  • Must work offline
• Investigate SARIF output as a way to generate a report. This could have a great impact on report generation down the line if it isn't too difficult.
• Create downstream qa tests
• Document new semgrep analyzer

Notes

This may be a good starting point:

$ git clone https://github.com/we45/Vulnerable-Flask-App
$ semgrep --config p/bandit Vulnerable-Flask-App