Avoid the copy operation during Generic Package Registry uploads
🔥 Problem
Similar to Avoid copying objects from one bucket to anothe... (#285597 - closed), the package registry can receive uploads for large files. The majority of the package registry uploads (most formats) will use a workhorse direct upload. In this mode, the file is put on object storage in a temporary location and when the upload is confirmed by the backend, the file is moved to its final location (using a copy operation).
The problem is that the GitLab instance can be connected to different object storage providers and that copy operation can take more or less time depending on the file size.
🚒 Solution
- Start with one package format. We can choose the Generic Package Repository as a starting point.
- Add a new column named
file_final_path
to thepackages_package_files
table withtext
type and1024
char length. - Pass two new keyword arguments to
authorize_workhorse!
method.use_final_store_path: true
final_store_path_root_id: <project_or_group_id>
- This change needs a feature flag and close monitoring during the rollout.