Introduce new DB columns for storing PyPi metadata
This is the first of 2 issues under the epic &11344
- Introduce new DB columns for storing PyPi metadata (this issue)
- Store metadata information when uploading a PyPi package
Context
PyPi packages are uploaded using Twine:
python3 -m twine upload --repository gitlab dist/*
Twine does the hard work of parsing the metadata for us. When twine hits the endpoint, we get this in params
in the upload endpoint (as of Twine 4.0.2):
{
"name"=>"hello-pypi",
"version"=>"0.0.2",
"filetype"=>"bdist_wheel",
"pyversion"=>"py3",
"metadata_version"=>"2.1",
"summary"=>"A small example package",
"home_page"=>"",
"author"=>"",
"author_email"=>"Rad Batnag <rbatnag@gitlab.com>",
"maintainer"=>"",
"maintainer_email"=>"",
"license"=>"",
"description"=>"# README\n",
"keywords"=>"",
"classifiers"=>"Programming Language :: Python :: 3",
"download_url"=>"",
"comment"=>"",
"sha256_digest"=>"0404b3516f7a011953a199889f9a2378c3ab00118e3189a69538d051deca047b",
"project_urls"=>"Bug Tracker, https://github.com/pypa/sampleproject/issues",
"requires_python"=>">=3.7",
"description_content_type"=>"text/markdown",
"md5_digest"=>"e61f4bd4d6ab6d87a5a6bf3793d702eb",
"blake2_256_digest"=>"02171dc2e8c442dfbb3118e492ed6194ebbfba4d16db6d6ef968bda055c9b9cf",
":action"=>"file_upload",
"protocol_version"=>"1",
"content.md5"=>"e61f4bd4d6ab6d87a5a6bf3793d702eb",
"content.name"=>"hello_pypi-0.0.2-py3-none-any.whl",
"content.path"=>"",
"content.remote_url"=>
"https://radamanthus-gdk-packages.s3.amazonaws.com/tmp/uploads/1693301393-40212-0004-3024-92cd798e7e99a5ac7a5f40c46041502d?X-Amz-Expires=15300&X-Amz-Date=20230829T092953Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVL2JV4V7HRAXJ4FN%2F20230829%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=88222156e481752067c33ced7852215616c481d6b85e11b79ee328bfedfd30bf",
"content.size"=>"998",
"content.upload_duration"=>"1.882007875",
"content.sha512"=>"b9bc9c643b6aa63182066465746aae7882b40c83777bddc90b0d19b62bd1df10aca8a8343912dc39d8685b3f24d2e748810534dd686fcd4950a6ea72c9845f38",
"content.remote_id"=>"1693301393-40212-0004-3024-92cd798e7e99a5ac7a5f40c46041502d",
"content.sha256"=>"0404b3516f7a011953a199889f9a2378c3ab00118e3189a69538d051deca047b",
"content.sha1"=>"a1b1fc08782cc39f5b6e7887dc8b6c2560a7dc2f",
"content.gitlab-workhorse-upload"=>
"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1cGxvYWQiOnsibWQ1IjoiZTYxZjRiZDRkNmFiNmQ4N2E1YTZiZjM3OTNkNzAyZWIiLCJuYW1lIjoiaGVsbG9fcHlwaS0wLjAuMi1weTMtbm9uZS1hbnkud2hsIiwicGF0aCI6IiIsInJlbW90ZV9pZCI6IjE2OTMzMDEzOTMtNDAyMTItMDAwNC0zMDI0LTkyY2Q3OThlN2U5OWE1YWM3YTVmNDBjNDYwNDE1MDJkIiwicmVtb3RlX3VybCI6Imh0dHBzOi8vcmFkYW1hbnRodXMtZ2RrLXBhY2thZ2VzLnMzLmFtYXpvbmF3cy5jb20vdG1wL3VwbG9hZHMvMTY5MzMwMTM5My00MDIxMi0wMDA0LTMwMjQtOTJjZDc5OGU3ZTk5YTVhYzdhNWY0MGM0NjA0MTUwMmQ_WC1BbXotRXhwaXJlcz0xNTMwMFx1MDAyNlgtQW16LURhdGU9MjAyMzA4MjlUMDkyOTUzWlx1MDAyNlgtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2XHUwMDI2WC1BbXotQ3JlZGVudGlhbD1BS0lBVkwySlY0VjdIUkFYSjRGTiUyRjIwMjMwODI5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3RcdTAwMjZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3RcdTAwMjZYLUFtei1TaWduYXR1cmU9ODgyMjIxNTZlNDgxNzUyMDY3YzMzY2VkNzg1MjIxNTYxNmM0ODFkNmI4NWUxMWI3OWVlMzI4YmZlZGZkMzBiZiIsInNoYTEiOiJhMWIxZmMwODc4MmNjMzlmNWI2ZTc4ODdkYzhiNmMyNTYwYTdkYzJmIiwic2hhMjU2IjoiMDQwNGIzNTE2ZjdhMDExOTUzYTE5OTg4OWY5YTIzNzhjM2FiMDAxMThlMzE4OWE2OTUzOGQwNTFkZWNhMDQ3YiIsInNoYTUxMiI6ImI5YmM5YzY0M2I2YWE2MzE4MjA2NjQ2NTc0NmFhZTc4ODJiNDBjODM3NzdiZGRjOTBiMGQxOWI2MmJkMWRmMTBhY2E4YTgzNDM5MTJkYzM5ZDg2ODViM2YyNGQyZTc0ODgxMDUzNGRkNjg2ZmNkNDk1MGE2ZWE3MmM5ODQ1ZjM4Iiwic2l6ZSI6Ijk5OCIsInVwbG9hZF9kdXJhdGlvbiI6IjEuODgyMDA3ODc1In0sImlzcyI6ImdpdGxhYi13b3JraG9yc2UifQ.Oep9KdSOk--yv3bp-XOofU_aggnI7w1JNWOMrY7ztkI",
"content"=>
#<UploadedFile:0x00000002cd41cf10
@content_type="application/octet-stream",
@md5="e61f4bd4d6ab6d87a5a6bf3793d702eb",
@original_filename="hello_pypi-0.0.2-py3-none-any.whl",
@remote_id="1693301393-40212-0004-3024-92cd798e7e99a5ac7a5f40c46041502d",
@sha1="a1b1fc08782cc39f5b6e7887dc8b6c2560a7dc2f",
@sha256="0404b3516f7a011953a199889f9a2378c3ab00118e3189a69538d051deca047b",
@size=998,
@upload_duration=1.882007875>,
"id"=>"28"}
Issue
Currently, we're only capturing the requires_python
field. We can capture more fields to make the Additional metadata
section of the package page more useful.
Note: Only Metadata-Version
, Name
, and Version
are required fields. All other fields are optional, see https://packaging.python.org/en/latest/specifications/core-metadata/
Edited by Radamanthus Batnag