Maven virtual registries: bump restrictions on send dependency call

🐦 Context

The Maven dependency proxy recently received an update around the call of the workhorse send_dependency logic. We're not going to detail here what this logic is doing. See the description of Improve workhorse dependencyproxy logic (#461561 - closed) for a detailed view on that logic.

The update is around setting up or bumping restrictions for the target url passed to send_dependency. For example, we don't want let users target local resources.

The upcoming Maven virtual registry being a very similar feature than the dependency proxy. Thus, in this MR, we're going to apply the exact same restrictions.

Now, note that there is a difference with virtual registries: we actually HEAD the upstream from the rails backend using ::Gitlab::HTTP. By doing this, we already have a first set of protections. For example, ::Gitlab::HTTP will not connect to local IPs. The send_dependency action happens after a successful HEAD. Thus, if the HEAD fails, the entire execution is aborted.

In other words, this MR is simply adding another layer of restriction on top of what we already have.

🤔 What does this MR do and why?

In the Maven virtual registry, update the send_dependencyproxy call to:

• allow localhost urls only under specific conditions.
• allow targeting object storage which could be mounted on a local url.
• enable the ssrf filter.

Update the related spec.

The entire maven virtual registry is behind a feature flag. We, thus, don't need a changelog here.

🏁 MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

✅

🌈 Screenshots or screen recordings

No UI changes.

⚙ How to set up and validate locally

We're going to set up dummy upstream server. This server needs to be running outside the local machine. A tiny AWS lightsail is a good candidate here. The reason is that the Maven Virtual Registry upstream can't target a local url. We need an url reachable across the Internet. A sinatra file will do.

require 'sinatra'

set :bind, '0.0.0.0'

get '*' do
  redirect 'http://192.168.1.1'
end

Notice that it's a redirect to the local IP.

Now, let's have a Group and PAT ready and let's configure the Maven virtual registry to target the (simulated) upstream. In a rails console:

group = Group.find(<id>)

r = ::VirtualRegistries::Packages::Maven::Registry.create!(group: group)
u = ::VirtualRegistries::Packages::Maven::Upstream.create!(group: group, url: 'http://<upstream ip>:4567')
::VirtualRegistries::Packages::Maven::RegistryUpstream.create(group: group, registry: r, upstream: u)

Last thing, without going into technical details, before calling the workhorse send_dependency logic, the backend will head the upstream. This head part has already a few restrictions in place (like not follow redirects to local IPs). We will need to disable this part temporarily to show that the changes of this MR work properly. In reality, this MR is adding a second layer of protection since we have a first one in that head logic.

diff --git a/app/services/virtual_registries/packages/maven/handle_file_request_service.rb b/app/services/virtual_registries/packages/maven/handle_file_request_service.rb
index a1bb1ff06954..865d946b9725 100644
--- a/app/services/virtual_registries/packages/maven/handle_file_request_service.rb
+++ b/app/services/virtual_registries/packages/maven/handle_file_request_service.rb
@@ -71,10 +71,10 @@ def cache_response_still_valid?
 
         def check_upstream(upstream)
           url = upstream.url_for(path)
-          headers = upstream.headers
-          response = head_upstream(url: url, headers: headers)
+          # headers = upstream.headers
+          # response = head_upstream(url: url, headers: headers)
 
-          return ERRORS[:file_not_found_on_upstreams] unless response.success?
+          # return ERRORS[:file_not_found_on_upstreams] unless response.success?
 
           workhorse_upload_url_response(url: url, upstream: upstream)
         end

Ok, now, we can pull a file through the virtual registry:

$ curl --header "Private-Token: <PAT>" "http://gdk.test:8000/api/v4/virtual_registries/packages/maven/<registry_id>/org/springframework/spring-web/6.1.12/spring-web-6.1.12.pom"
Bad Gateway

That Bad Gateway response is returned by the workhorse send_dependency logic detecting a local IP. 🎉