Add Nuget metadatum support
What does this MR do?
This MR adds nuget metadata support at the package level.
The .nuspec
file has several fields that we are interested in (see #42680 (closed)). This MR adds support for the following fields:
projectUrl
licenseUrl
iconUrl
Checking the .nuspec
XSD we can see that all these fields are optional:
- https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Packaging/compiler/resources/nuspec.xsd#L67
- https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Packaging/compiler/resources/nuspec.xsd#L66
- https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Packaging/compiler/resources/nuspec.xsd#L68
Once we have those fields saved, they will be exposed in those APIs:
- Nuget Search endpoint
- Nuget metadata endpoint for a package name
- Nuget metadata endpoint for a package name + package version
Approach
- We're going to re-use the same approach used on the metadata for other package types: add a
packages_nuget_metadata
table and link it toPackages::Package
. - We will have 3
text
columns,project_url
,license_url
andicon_url
. All 3 will have a constraint to keep them at a reasonable length. - At the
Packages::NugetMetadatum
, we will validate them using thepublic_url
validator. - In addition, we will validate that we don't have 3 blank values.
- Note that nuget package file can be uploaded several times. Each time, we need to check if a
Packages::Package
already exists and re-use it if that's the case. - From the previous point, on the nuget metadatum, we will need to update all the 3 columns with the latest version and keep a eye that if we have a new version where these 3 values are blank and we have a
Packages::NugetMetadatum
object, we should destroy it. - The above is handled by a dedicated service:
Packages::Nuget::UpdateMetadatumService
. - At the grape entity level, we are using a shared entity to deal with all three fields:
EE::API::Entities::Nuget::Metadatum
Screenshots
Uploading a nuget package:
$ nuget push DummyProject.WithMoreMetadata.1.2.3.nupkg -source local
WARNING: No API Key was provided and no API Key could be found for 'http://gitlab.local:8000/api/v4/projects/1/packages/nuget'. To save an API Key for a source use the 'setApiKey' command.
Pushing DummyProject.WithMoreMetadata.1.2.3.nupkg to 'http://gitlab.local:8000/api/v4/projects/1/packages/nuget'...
PUT http://gitlab.local:8000/api/v4/projects/1/packages/nuget/
Created http://gitlab.local:8000/api/v4/projects/1/packages/nuget/ 9379ms
Your package was pushed.
Visual Studio will use these urls to present more details while searching for packages with the GitLab Nuget Repository:
Database Review
Up migration
== 20200430130048 CreatePackagesNugetMetadata: migrating ======================
-- table_exists?(:packages_nuget_metadata)
-> 0.0004s
-- create_table(:packages_nuget_metadata, {:id=>false})
-> 0.0408s
-- transaction_open?()
-> 0.0000s
-- execute("ALTER TABLE packages_nuget_metadata\nADD CONSTRAINT packages_nuget_metadata_license_url_constraint\nCHECK ( char_length(license_url) <= 255 )\nNOT VALID;\n")
-> 0.0010s
-- execute("SET statement_timeout TO 0")
-> 0.0002s
-- execute("ALTER TABLE packages_nuget_metadata VALIDATE CONSTRAINT packages_nuget_metadata_license_url_constraint;")
-> 0.0004s
-- execute("RESET ALL")
-> 0.0001s
-- transaction_open?()
-> 0.0000s
-- execute("ALTER TABLE packages_nuget_metadata\nADD CONSTRAINT packages_nuget_metadata_project_url_constraint\nCHECK ( char_length(project_url) <= 255 )\nNOT VALID;\n")
-> 0.0002s
-- execute("SET statement_timeout TO 0")
-> 0.0001s
-- execute("ALTER TABLE packages_nuget_metadata VALIDATE CONSTRAINT packages_nuget_metadata_project_url_constraint;")
-> 0.0004s
-- execute("RESET ALL")
-> 0.0001s
-- transaction_open?()
-> 0.0000s
-- execute("ALTER TABLE packages_nuget_metadata\nADD CONSTRAINT packages_nuget_metadata_icon_url_constraint\nCHECK ( char_length(icon_url) <= 255 )\nNOT VALID;\n")
-> 0.0013s
-- execute("SET statement_timeout TO 0")
-> 0.0001s
-- execute("ALTER TABLE packages_nuget_metadata VALIDATE CONSTRAINT packages_nuget_metadata_icon_url_constraint;")
-> 0.0008s
-- execute("RESET ALL")
-> 0.0001s
== 20200430130048 CreatePackagesNugetMetadata: migrated (0.0563s) =============
Down migration
== 20200430130048 CreatePackagesNugetMetadata: reverting ======================
-- drop_table(:packages_nuget_metadata)
-> 0.0161s
== 20200430130048 CreatePackagesNugetMetadata: reverted (0.0161s) =============
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Edited by David Fernandez