Skip to content

Dependency scanning for Python v3.9 failing on dictalchemy

Summary

Hi,

we recently found out that the dependency scanning (Gemnasium) job started failing for a projects where we use Python v3.9 image and dictalchemy==0.1.2.7 package.

This is because right now the gemnasium:2-python-3.9 Docker image comes with setuptools 58.1.0, and this version of setuptools doesn't provide 2to3 anymore. See https://github.com/pypa/setuptools/issues/2769. However, 2to3 is needed to install dictalchemy==0.1.2.7 and other Python packages.

Workaround: Downgrade to an earlier version of setuptools, like 58.0.1.

Currently this does NOT affect gemnasium:2 (Python 3.6) because it uses setuptools 57.5.0 by default.

Steps to reproduce

Add .gitlab-ci.yml:

stages:
  - test

include:
  - template: Security/Dependency-Scanning.gitlab-ci.yml

gemnasium-python-dependency_scanning:
  image: registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2-python-3.9
  variables:
    SECURE_LOG_LEVEL: debug

Add requirements.txt:

dictalchemy==0.1.2.7

Example Project

https://gitlab.com/vavkamil/depscan-bug-dictalchemy

What is the current bug behavior?

It's failing because of:

[DEBU] [gemnasium-python] [2022-02-21T16:33:40Z] ▶ /usr/local/bin/pip download --disable-pip-version-check --dest ./dist -r requirements.txt
Collecting dictalchemy==0.1.2.7
  Downloading dictalchemy-0.1.2.7.tar.gz (11 kB)
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-download-l9v0jchm/dictalchemy_f1b8f72c2e454be387e6ef956d864114/setup.py'"'"'; __file__='"'"'/tmp/pip-download-l9v0jchm/dictalchemy_f1b8f72c2e454be387e6ef956d864114/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-5qugknlw
         cwd: /tmp/pip-download-l9v0jchm/dictalchemy_f1b8f72c2e454be387e6ef956d864114/
    Complete output (1 lines):
    error in dictalchemy setup command: use_2to3 is invalid.
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/29/31/9451637075e9061c4dcf651199b8666bdee68630160c51e3d82598aac041/dictalchemy-0.1.2.7.tar.gz#sha256=a08b1463b5cd849158c77701da67be4fbb3080de3ad08abb165710aa63987809 (from https://pypi.org/simple/dictalchemy/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement dictalchemy==0.1.2.7 (from versions: 0.1a1, 0.1a2, 0.1a3, 0.1a4, 0.1b1, 0.1b2, 0.1, 0.1.1, 0.1.2b1, 0.1.2b2, 0.1.2b3, 0.1.2, 0.1.2.1, 0.1.2.2, 0.1.2.3, 0.1.2.4, 0.1.2.5, 0.1.2.6, 0.1.2.7)
ERROR: No matching distribution found for dictalchemy==0.1.2.7
exit status 1

What is the expected correct behavior?

The Gemnasium job shouldn't fail, when using registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2-python-3.9 image.

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Implementation plan

Edited by Igor Frenkel