Add a basic maven virtual registry download endpoint

🔭 Context

With Maven virtual registry MVC (single upstream and... (&14137), we're starting the work on Virtual Registries. Virtual Registries is a feature that could be described as the evolution of the dependency proxy idea: having the GitLab instance play man in the middle between clients and artifacts registries. Artifacts can be any kind but we're going to focus on packages and container images, starting with Maven packages specifically.

In other words, the GitLab instance can be configured to contact a set of upstreams and expose a specific virtual registry url that "talks" the artifact type API, in this case the Maven API. When a request hits this API, we'll check with the set of upstreams and the first one to answer successfully "wins". We will pull the response from that upstream, cache it in the GitLab instance and return it to the client.

At the time of this writing, we're in the first iteration of the implementation. To avoid working on a very large scope, we restricted it to:

• Only maven packages supported.
• Works at a root Group level only.
• A root Group can only have 1 maven virtual registry.
• A virtual maven registry can only have 1 upstream.
• This being a multi MR implementation, everything is gated behind a feature flag.

Obviously, the Maven package API exposed by virtual registries can be pretty complex (upstream and caching handling). The implementation of these has been split in steps.

This MR represents the very first step of the maven virtual registry API implementation. In other words, this MR implements the API (fortunately, only one endpoint to implement) that Maven client can use to pull packages (package publication is not part of the first iteration's scope).

Now, as we said, we're not going to have a super smart download endpoint. Instead, this MR implements the barebones of the download endpoint:

• The Grape API class.
• Check if the related upstream (recall that a maven virtual registry has only one upstream?) has the requested file.
  • If that's a yes, use workhorse send_url to return the file to the client.
  • If that's a no, simply return not found.

The main reason is that we don't want to handle an 🐘 MR and actually, even with the simpler implementation, this MR is already quite chunky.

This is issue Maven Virtual Registry: Maven API endpoint (#467982 - closed).

In a follow up issue, we will update the implementation to support caching (to avoid pinging upstream all the time). That is why this MR implementation is quite "simple".

🤔 What does this MR do and why?

• Add a /api/v4/virtual_registries/packages/maven/:id/*path API endpoint.
  • This endpoint will get the target virtual registry, contact the upstream and respond accordingly.
• Add the related specs.
• Introduce the maven virtual registry feature flag

🏁 MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🆗

🌈 Screenshots or screen recordings

No UI changes.

⚗ How to set up and validate locally

We're not going to use a fully fledged Maven client to verify this MR. Instead, we will simply use $ curl to simulate calls done by Maven clients.

Having said that, we already support public and private upstreams.

1. Enable the feature flag : Feature.enable(:virtual_registry_maven).
2. Have a PAT and a root group (any visiblity) ready.
3. For the virtual registry settings, we don't have an UI or API (yet), we thus need to create them in a rails console:

   r = ::VirtualRegistries::Packages::Maven::Registry.create!(group: <root_group>)
   u = ::VirtualRegistries::Packages::Maven::Upstream.create!(group: <root_group>, url: "http://sandbox.test")
   VirtualRegistries::Packages::Maven::RegistryUpstream.create!(group: <root_group>, registry: r, upstream: u)

1️⃣ Public upstream

Let's update the upstream for a public registry. Being public, we will not have any credentials.

u.update!(url: 'https://repo1.maven.org/maven2')

Let's pull a file:

$ curl --header "Private-Token: <PAT>" "http://gdk.test:8000/api/v4/virtual_registries/packages/maven/< r.id >/org/springframework/boot/spring-boot-starter-web/3.3.2/spring-boot-starter-web-3.3.2.pom"

You get the file 🎉

2️⃣ Private upstream

GitLab team members should have access to this dummy private project that has a maven package. We're going to use that as our upstream.

1. Get a .com PAT ready. Alternative, create a deploy token on the dummy project.
2. Update the upstream:

   u.update!(url: "https://gitlab.com/api/v4/projects/22780791/packages/maven", username: "<gitlab.com username or deploy token username>", password: "gitlab.com PAT or deploy token")

Let's pull a file:

$ curl --header "Private-Token: <PAT>" "http://gdk.test:8000/api/v4/virtual_registries/packages/maven/< r.id >/gl/pru/My.Ananas/13.0.3/My.Ananas-13.0.3.pom"

The file is returned 🎉

3️⃣ Other scenarios to test

You can try:

• Requesting a path that doesn't exist on the upstream.
• Destroy the upstream and request a file.
• Destroy the registry and request a file.