Wrong package manager in Dependency List for dependencies managed using Pipenv, Setuptools, or Gradle

Summary

The Package Manager column of the Dependency List doesn't show the correct information for dependencies managed using Pipenv, Setuptools, or Gradle:

• It contains Pip (Python) for all Python dependencies.
• It contains Maven (Java) for dependencies managed by Gradle.

Further details

The Package Manager column of the Dependency List is inferred from the package_manager JSON field of the Dependency Scanning report. See Dependency List formatter.

Currently the Gemnasium analyzers copy the PackageManager struct field of the dependency file Parser to the package_manager JSON field of the report. Unfortunately, they are cases where the package manager can't be inferred from the parser:

• mvnplugin.go is used to process the output of the Gemnasium Maven plugin and the Gemnasium Gradle plugin. (This will be true even after implementing #337083 (closed) and parsing the output of the dependencies task, because the gemnasiumDumpDependencies task will still be used as a fallback.)
• pipdeptree.go is used to process the JSON output of pipdeptree, and it's also used to process the output of pipenv graph since it uses the same format. As a result, right now this parser is always used when scanning a Python project.

piplock.go also has its PackageManager set to pip, which is incorrect, but right now this parser isn't used.

Steps to reproduce

1. Create a Python project using Pipenv
2. Include the Dependency Scanning CI template
3. Trigger a new pipeline for the default branch
4. Go to the Dependency List page

Example Project

See currently Dependency List for the python-pipenv test project.

What is the current bug behavior?

When scanning a Pipfile.lock we report pip as the package_manager.

What is the expected correct behavior?

When scanning a Pipfile.lock we report pipenv as the package_manager.

Relevant logs and/or screenshots

See currently Dependency List for the python-pipenv test project.

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)
(we will only investigate if the tests are passing)

Possible fixes

See Proposal

Proposal

This can be fixed by using Project.PackageManager (finder package) instead of Parser.PackageManager (parser package) in the Scanner. Project.PackageManager already has the information, and it properly detects Gradle and Pipenv.

Eventually Parser.PackageManager and the corresponding type and constants can be removed.

The Rails backend needs to be updated to support the new values for the package_manager field:

• The DependencyList formatter should render the new values, in the Packager column.
• The Dependencies API should accept these new values in the package_manager param. The documentation needs to be updated as well.

In the expected report for Gradle project, the package_manager field must be set to gradle:

• https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/qa/expect/java-gradle/gl-dependency-scanning-report.json#L2673
• https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/qa/expect/java-gradle-kotlin-dsl/gl-dependency-scanning-report.json#L2673
• https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/qa/expect/java-gradle-multimodules/gl-dependency-scanning-report.json#L3633

In the expected report for Pipenv project, the package_manager field must be set to pipenv:

• https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/qa/expect/python-pipenv/gl-dependency-scanning-report.json

Implementation plan

• Update gemnasium
  • Change Scanner to use Project.PackageManager.
  • Release new version.
• Update gemnasium-maven
  • Upgrade to new version of gemnasium.
  • Change convert, and revert to old behaviour when DS_REPORT_PACKAGE_MANAGER_MAVEN_WHEN_JAVA is true.
  • Change Dockerfile, and set DS_REPORT_PACKAGE_MANAGER_MAVEN_WHEN_JAVA to maintain old behavior.
  • Add integration test for DS_REPORT_PACKAGE_MANAGER_MAVEN_WHEN_JAVA.
  • Release new version.
• Update gemnasium-python
  • Upgrade gemnasium git submodule.
  • Change convert, and revert to old behaviour when DS_REPORT_PACKAGE_MANAGER_PIP_WHEN_PYTHON is true.
  • Change Dockerfile, and set DS_REPORT_PACKAGE_MANAGER_PIP_WHEN_PYTHON to maintain old behavior.
  • Add integration test for DS_REPORT_PACKAGE_MANAGER_PIP_WHEN_PYTHON.
  • Release new version.
• Update Rails backend
  • Make DependencyList formatter render new package managers. See !72021 (diffs) for reference.
  • Change Dependencies API to accept new package managers, and update API docs. See !72022 (diffs) for reference.
  • Change CI template, and enable new behavior.
    • Set DS_REPORT_PACKAGE_MANAGER_MAVEN_WHEN_JAVA to "false".
    • Set DS_REPORT_PACKAGE_MANAGER_PIP_WHEN_PYTHON to "false".