Maven virtual registries: bump restrictions on send dependency call
🐦 Context
The Maven dependency proxy recently received an update around the call of the workhorse send_dependency
logic. We're not going to detail here what this logic is doing. See the description of Improve workhorse dependencyproxy logic (#461561 - closed) for a detailed view on that logic.
The update is around setting up or bumping restrictions for the target url passed to send_dependency
. For example, we don't want let users target local resources.
The upcoming Maven virtual registry being a very similar feature than the dependency proxy. Thus, in this MR, we're going to apply the exact same restrictions.
Now, note that there is a difference with virtual registries: we actually HEAD
the upstream from the rails backend using ::Gitlab::HTTP
. By doing this, we already have a first set of protections. For example, ::Gitlab::HTTP
will not connect to local IPs. The send_dependency
action happens after a successful HEAD
. Thus, if the HEAD
fails, the entire execution is aborted.
In other words, this MR is simply adding another layer of restriction on top of what we already have.
🤔 What does this MR do and why?
In the Maven virtual registry, update the send_dependencyproxy
call to:
- allow localhost urls only under specific conditions.
- allow targeting object storage which could be mounted on a local url.
- enable the ssrf filter.
Update the related spec.
The entire maven virtual registry is behind a feature flag. We, thus, don't need a changelog here.
🏁 MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
🌈 Screenshots or screen recordings
No UI changes.
⚙ How to set up and validate locally
We're going to set up dummy upstream server. This server needs to be running outside the local machine. A tiny AWS lightsail is a good candidate here. The reason is that the Maven Virtual Registry upstream can't target a local url. We need an url reachable across the Internet. A sinatra file will do.
require 'sinatra'
set :bind, '0.0.0.0'
get '*' do
redirect 'http://192.168.1.1'
end
Notice that it's a redirect to the local IP.
Now, let's have a Group
and PAT ready and let's configure the Maven virtual registry to target the (simulated) upstream. In a rails console:
group = Group.find(<id>)
r = ::VirtualRegistries::Packages::Maven::Registry.create!(group: group)
u = ::VirtualRegistries::Packages::Maven::Upstream.create!(group: group, url: 'http://<upstream ip>:4567')
::VirtualRegistries::Packages::Maven::RegistryUpstream.create(group: group, registry: r, upstream: u)
Last thing, without going into technical details, before calling the workhorse send_dependency
logic, the backend will head
the upstream. This head
part has already a few restrictions in place (like not follow redirects to local IPs). We will need to disable this part temporarily to show that the changes of this MR work properly. In reality, this MR is adding a second layer of protection since we have a first one in that head
logic.
diff --git a/app/services/virtual_registries/packages/maven/handle_file_request_service.rb b/app/services/virtual_registries/packages/maven/handle_file_request_service.rb
index a1bb1ff06954..865d946b9725 100644
--- a/app/services/virtual_registries/packages/maven/handle_file_request_service.rb
+++ b/app/services/virtual_registries/packages/maven/handle_file_request_service.rb
@@ -71,10 +71,10 @@ def cache_response_still_valid?
def check_upstream(upstream)
url = upstream.url_for(path)
- headers = upstream.headers
- response = head_upstream(url: url, headers: headers)
+ # headers = upstream.headers
+ # response = head_upstream(url: url, headers: headers)
- return ERRORS[:file_not_found_on_upstreams] unless response.success?
+ # return ERRORS[:file_not_found_on_upstreams] unless response.success?
workhorse_upload_url_response(url: url, upstream: upstream)
end
Ok, now, we can pull a file through the virtual registry:
$ curl --header "Private-Token: <PAT>" "http://gdk.test:8000/api/v4/virtual_registries/packages/maven/<registry_id>/org/springframework/spring-web/6.1.12/spring-web-6.1.12.pom"
Bad Gateway
That Bad Gateway
response is returned by the workhorse send_dependency
logic detecting a local IP.