Move a base container repository (and all sub-repositories, if any)
Context
This follows up from the investigation carried out in gitlab#388675 (closed)
Problem
In order to perform a container repository moves (which stem from a project move) as described in gitlab#388675 (closed), we need to be able to update the base repository (and all sub-repositories (if any)) for a given project path in the container registry. There is no way we can do this today using the registry's existing APIs.
Task
-
Extend the container registryAPI-GitLab-V1 rename endpoit PATCH /gitlab/v1/repositories/:project_path/?dry_run=true|false
that performs an atomic bulk update for a base repository (name
andpath
) and all (existing sub-repositories (path
s)) under the requestedpath
to also perform updates to a projects group.-
Access to this route should be restricted to internal (Rails) use only. -
Before attempting the update in the registry database, the operation should start by ensuring that no repository with the new path pre exists (Otherwise, a 409 Conflict
should be returned.) AND the requested path to move to is in the same root namespace as the initial/prior path (Otherwise, a 422 Unprocessable Entity should be returned.) -
For scalability and performance reasons, this feature will start by being limited to projects with no more than 1000 container repositories. For GitLab.com, this covers 99.98% of all projects (source). We can then increase this later based on metrics and pending a decision in https://gitlab.com/gitlab-org/gitlab/-/issues/357014 (internal). Attempting to update more than 1000 repositories (base and sub-repositories) should yield a 422 Unprocessable Entity
response. -
The endpoint should accept a "dry-run" .This option should lead to a check that ensures the target repositories name/path is not yet taken on the registry without performing the move. If the target name/path is already taken a 409 Conflict
response should be returned. -
When processing the dry-run move request, the registry would acquire a repository rename lease for paths that match ^my-group\/(new-name)($|\/.*$)
. This lease would have a short TTL ofN
seconds after which writes/reads against the matching repositories would be no longer blocked. If the dry-run succeeds (the move is technically possible), Rails would be informed in the response that it hadN
seconds to complete the operation. If the dry-run did not succeed, the lease would be released ahead of time. -
When receiving the non-dry-run move request, the registry would ensure a lease is still alive and enough time remains before it expires (e.g., it won't expire before the configured DB transaction timeout of 10 seconds) or (if a lease does not exist / is about to expire) it will grant/refresh a lease first before proceeding with the move operation. Leases would be released at the end of a request, regardless of the result. -
Across the whole registry APIs, a read-write request against a given repository (be it a blob upload, a tag delete, etc.) would only be allowed if a rename lease for the target repository does not exist. -
We should consider breaking down the RenameRepository
handler into smaller functions. It currently is over 200 lines of code!
-