Push packages from triggered pipelines to an S3 bucket so they can be downloaded easily
Currently, the packages from triggered pipelines (used for testing MRs - either from package-and-qa
in CE/EE or Trigger:(ce|ee)-package
in omnibus-gitlab) are only available as artifacts. When this was initially designed, we could download artifacts directly from their raw links by passing the PAT, and hence thought it will be redundant to upload them to S3 also. This meant, to download a package, I only had to copy its link. However, we have removed the scope of PAT to do such actions, and one can download artifact only via API. Now, to download a package, I have to create an API link using parts of the URL (project name, job id, etc.).
There are issues open for tackling this like https://gitlab.com/gitlab-org/gitlab-ce/issues/48069 , but none are here yet.
We should probably rethink our design decision, and upload the packages to an S3 bucket also, so it isn't such a pain to get them for testing. We can set a more aggressive cleanup policy for this bucket (3 days, matching to our artifact expiry).