Skip to content

Add a dependency proxy scope for GitLab tokens

Context

You can use the Dependency Proxy to proxy and cache container images from Docker Hub. This helps you to improve the reliability and performance of your builds. You can authenticate using GitLab predefined environment variables, a deploy token, a personal access token, or a job token.

Problem to solve

The problem is that personal access and deploy tokens require that you give access through the read_registry or write_registry scope. However, certain customers may not want to give permission to the container registry along with the dependency proxy. One way we can mitigate this risk is by adding a new scope to personal access, deploy and project access tokens for the dependency proxy.

Proposal

Add a new read_dependency_proxy and write_dependency_proxy scope to personal access, deploy and project access tokens, so that Admin can set the exact privileges they want to give their team.

UPDATE: Dependency proxy for containers will eventually be renamed to Virtual Registry. Virtual Registry (for Maven) is currently in progress and a read_virtual_registry scope has already been introduced. We'll add the write_virtual_registry scope and have Dependency proxy for containers accept the read_virtual_registry and write_virtual_registry scopes, in the same way it currently accepts the read_registry and write_registry scopes.

Comment from @godfat-gitlab in #332411 (comment 1587469535) that is relevant to this issue.

A workflow can be that there are scheduled pipelines (as an example) which has write_dependency_proxy permission to update the cache, and generally speaking for any other pipelines, it can grant read_dependency_proxy for any tokens.

This means we still limit what can be stored inside dependency proxy, but for anyone they can really speed things up by just reading it. Give it an error if it's not there, which should be fine because it should be handled by the other pipelines.

Further details

This is related to #328765

Technical details

The dependency proxy works as a pull-through cache. In that regard, it will:

  1. Return images from Docker hub.
  2. Cache images that go through so that they can be returned from the cache instead of (1.) (cache hit).

The challenge here is that we have two kinds of access: a read and a write within the same request.

With read_dependency_proxy_container, users should be able to:

  1. Get an image from Docker hub.
  2. Get an image from the cache.

With write_dependency_proxy_container, users should be able to:

  1. same as read_dependency_proxy_container.
  2. Write an image into the cache.

What happens if a read_dependency_proxy_container user requests an image that is not in the cache? Well, we can't write to the cache, so we simply return the image that is on Docker image.

The above should be clearly documented. If the dependency proxy is used by only read_dependency_proxy_container user, then the cache will never be populated and we lose one aspect of the dependency proxy: if Docker hub is down, the dependency proxy will not be of any help here. As such, it is essential that there are users with write_dependency_proxy_container that access it. This way, the cache entries get written.

Implementation

We follow the way cntainer registry does authentication and authorization.

  • Scope checks ("does the user have permissions to read from the container registry?") are done during authentication. Scopes are converted to abilities, and if these abilities are insufficient then we reply with denied or forbidden response. JwtController ContainerRegistryAuthenticationService
  • In the policies - because we do not have access to the token and its scopes - we only check the user level. Example

  1. Add constants for the new abilities. Map the existing scopes to the new abilities: read_registry = read_dependency_proxy_container, write_registry = write_dependency_proxy_container

  2. Add the new scopes to deploy_tokens table. Map the new scopes to the abilities introduced in step 1.

  3. Update the deploy tokens API: accept new scopes as parameters Update the deploy tokens UI: accept new scopes when creating a new deploy token

  4. Implement new caching behavior with the new abilities:

  • read_dependency_proxy_container without write_dependency_proxy_container -> get image from docker hub, get image from cache, but do not write to cache
  • write_dependency_proxy_container -> same as read_dependency_proxy_container, but can write to cache

NOTE: These checks will have to be done in the Auth::DependencyProxyAuthenticationService, similar to how it's done in Auth::ContainerRegistryAuthenticationService (Example). This works because all requests pass through the /v2 endpoint, even after login (see Authentication and authorization)

  1. Cleanup: Remove the scope checks in the policies
Edited by Radamanthus Batnag