Fix nuget metadata endpoint wrong "upper" or "lower" version
What does this MR do and why?
In the NuGet Repository metadata endpoint, the only param we receive is the package_name
. The metadata endpoint should respond with a set of information about the requested package. Among those information, there are the "lower"
& "upper"
versions of the package.
The maximum number of versions the metadata endpoint is allowed to respond with is 300. If a specific package has versions more than 300
, the "lower"
version attribute in the response would be incorrect. Why?
Because in order to determine the "lower"
& "upper"
versions, we iterate on the fetched packages and extract their versions, sort them, and then return the first and last as the "lower"
& "upper"
versions. And since we are limited to 300 versions, the "lower"
version would be the first version of the most recent 300
versions, but not the correct actual lower
version.
The ultimate solution would be caching the metadata for each package and storing it as a JSON file in the Object Storage. However, I tried to figure out the cost of querying all versions of a package if it has more than 300
versions, and it turned out that it's not that expensive query with the right database index in place.
So as an alternative fix, we can rely on fetching all versions from the database when the versions count exceeds the limit of 300. Otherwise, we don't make any extra queries.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
- Create some dummy Nuget packages (say 5) with the same name but different versions and publish them to a project in your local NuGet Repository
- to create a dummy package:
- make sure you have nuget cli installed
- navigate to a new directory and run
nuget spec
in the terminal. A newPackage.nuspec
file will be created. - Open the
.nuspec
file in your text editor and modify the version value (the default is1.0.0
) then save the file and close it. - Run
nuget pack
in the terminal. A newPackage.X.X.X.nupkg
file will be created.X.X.X
is the version you put in the.nuspec
file. -
Publish the
.nupkg
file to the NuGet Repository - Repeat the previous steps by changing the version and creating a new
.nupkg
package file. Publish the new.nupkg
to publish a new version.
- to create a dummy package:
- In the rails console enable the feature flag:
Feature.enable(:nuget_metadata_version_bounds, <project>)
- Change this package finder limit to be less than the number of versions you published. So if you published 5 packages, make this limit 3 for example. That would allow you to validate that the correct lower and upper bounds of the version are returned correctly in the metadata endpoint.
- In the browser open this metadata endpoint URL for the package you are testing against:
http://gdk.test:3000/api/v4/projects/<project_id>/packages/nuget/metadata/<package_name>/index.json
- Check the "lower" & "upper" attributes in the JSON response, they shouldn't be limited to the finder's limit; the correct bounds should be returned.
- On master, the "lower" value will be different and limited to the finder's limit returned packages.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Database analysis
Related to #241376