Package Registry: Truncate Pypi metadata description field
requested to merge 431505-twine-push-to-package-registry-fails-with-validation-failed-description-is-too-long-maximum into master
What does this MR do and why?
We recently started uploading more Pypi metadata fields in !131327 (merged).
Unfortunately, because we added a size limit on the description field, packages with a large README file hit this size limit. When a Pypi package with a large README file is uploaded, we return an error response:
Uploading albedo_gscli-0.0.20.dev3+g9502f74-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.6/111.6 kB • 00:00 • 235.3 MB/s
INFO Response from https://gitlab.com/api/v4/projects/<omitted>/packages/pypi:
400 Bad Request
INFO {"message":"Validation failed: Description is too long (maximum is 4000 characters)"}
ERROR HTTPError: 400 Bad Request from https://gitlab.com/api/v4/projects/<omitted>/packages/pypi
Bad Request
This MR will truncate the description so that such package can be uploaded, with the truncated description in the metadata.
Screenshots or screen recordings
No UI changes
How to set up and validate locally
- Install the prerequisites
- Download this barebones python package
- Edit
README.md
so that it is larger than 4KB. Our size limit is 4000 characters. - Build the package. From the package directory, run
python3 -m build
- Setup authentication following our guide
- Upload the package. From the package directory, run
python3 -m twine upload --verbose --repository gitlab dist/*
. NOTE: The upload will fail if you try to upload a version that already exists. Delete the existing version in Package Registry, or build and upload a different version.
Expected response:
Uploading ansible-1.0.14.tar.gz
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.4/114.4 kB • 00:00 • 356.0 MB/s
INFO Response from http://gdk.test:3000/api/v4/projects/34/packages/pypi:
201 Created
INFO {"message":"201 Created"}
- Verify that the package metadata was populated. From a Rails console, run
::Packages::Package.last.pypi_metadatum.description
. The metadatum record should have been created, with thedescription
field set to the truncated contents ofREADME.md
.
In the master
branch, the upload will fail:
Uploading ansible-1.0.15-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.0/114.0 kB • 00:00 • 305.9 MB/s
INFO Response from http://gdk.test:3000/api/v4/projects/34/packages/pypi:
400 Bad Request
INFO {"message":"Validation failed: Description is too long (maximum is 4000 characters)"}
ERROR HTTPError: 400 Bad Request from http://gdk.test:3000/api/v4/projects/34/packages/pypi
Bad Request
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #431505 (closed)
Edited by Radamanthus Batnag