Skip to content

Use the GitLab API to purge blobs from the Dependency Proxy cache

Problem to solve

The GitLab Dependency Proxy for the allows users to proxy and cache images hosted on DockerHub, so that they are readily available for use within GitLab CI/CD. However, there is not currently a way to purge the cache, which could result in additional storage costs down the road.

Intended users

Further details

  • The Package group has been discussing utilizing GraphQL to power the front-end. This may be a simple use case, and give us the ability to add a frontend for the API relatively quickly.

Investigation details

Purge all cached blobs for the given group

DELETE /groups/:id/dependency_proxy/cache

To delete all cached blobs for a group, we simply need to run some variation of group.dependency_proxy_blobs.delete_all. Blobs are handled within disk or object storage by DependencyProxy::FileUploader.

  1. Add endpoint to delete cached blobs for a group

Use cases

  1. From the command line, group administrators may utilize the GitLab API to manually purge the blobs from their group's Dependency Proxy.
  2. From the user interface, an admin may click a button and purge the cache for the group's dependency proxy.

Proposal

As an MVC, create a new GitLab API for the Dependency Proxy, which will allow Admin the ability to purge the cache of a group's Dependency Proxy. and define an interval in which to routinely purge the cache.

Examples

  • DELETE /groups/:id/dependency_proxy/cache
    • Purge all cached blobs for the given group
  • POST /groups/:id/dependency_proxy/expiration_cadence
    • Define the cadence for which to regularly purge the group's Dependency Proxy.
    • Set the default for all groups to 60 days.

Beyond the MVC

Permissions and Security

  • The API should be limited to Admin

Documentation

  • Create documentation for the Dependency Proxy API

Testing

What does success look like, and how can we measure that?

Success looks like we give Admin an easy way to manage storage costs.

Metrics

  • Count the number of API events run for self-managed and GitLab.com
  • Track the defined cadence as part of the API event, so we can continue to refine our default settings.

Links / references

Edited by Tim Rizzi