Consolidate object storage config in one place
Currently each object type has its own section in gitlab.yml. For example, for artifacts:
artifacts:
path: tmp/tests/artifacts
enabled: true
object_store:
enabled: false
remote_directory: artifacts # The bucket name
background_upload: false
connection:
provider: AWS
aws_access_key_id: AWS_ACCESS_KEY_ID
aws_secret_access_key: AWS_SECRET_ACCESS_KEY
region: us-east-1
External diffs, LFS, uploads, packages, etc. all have similar independent configuration object storage sections. While this redundancy makes it possible to configure each bucket with different providers or credentials, this causes a configuration explosion that makes GitLab hard to manage.
This change preserves the legacy format but adds a new gitlab.yml
section that enforces a single, common object storage provider for all
object storage types. This will make it possible for the S3 client in
Workhorse to operate with one credential and simplify the configuration
for the end user. An example config:
object_store:
enabled: true
connection:
provider: AWS
aws_access_key_id: AWS_ACCESS_KEY_ID
aws_secret_access_key: AWS_SECRET_ACCESS_KEY
region: us-east-1
proxy_download: true
objects:
artifacts:
bucket: artifacts
proxy_download: false
external_diffs:
bucket: external-diffs
lfs:
bucket: lfs-objects
uploads:
bucket: uploads
packages:
bucket: packages
dependency_proxy:
bucket: dependency_proxy
Note that:
- The consolidated config only gets used if
object_store
is NOT defined within the types themselves. - A bucket needs to be defined for each object type.
- Only
bucket
,enabled
, andproxy_download
can be overridden from the common configuration.
Consolidating support for a single bucket for all types is a larger and more involved change.
First step of #23345 (closed)