Add CI config for GCP Secret Manager
What does this MR do and why?
Add CI config for GCP Secret Manager.
The secret manager is added as gcp_secret_manager
keyword on secrets
. An example of a yaml would be as follows:
job_name:
id_tokens:
GCP_SM_ID_TOKEN:
aud: https://iam.googleapis.com/projects/$GCP_PROJECT_NUMBER/locations/global/workloadIdentityPools/$GCP_WORKLOAD_FEDERATION_POOL_ID/providers/$GCP_WORKLOAD_FEDERATION_PROVIDER_ID # or a custom audience as configured in GCP Workload Identity Pool Provider.
secrets:
DATABASE_PASSWORD:
gcp_secret_manager:
name: my-project-secret # This is the name of the secret defined in GCP Secret Manager
version: '1' # optional: default to `latest`.
token: GCP_SM_ID_TOKEN
In addition to the above config, the user needs to provide the following variables:
- GCP Project Number
GCP_PROJECT_NUMBER
- GCP Workload Identity Federation Pool ID
GCP_WORKLOAD_IDENTITY_FEDERATION_POOL_ID
- GCP Workload Identity Federation Provider ID
GCP_WORKLOAD_IDENTITY_FEDERATION_PROVIDER_ID
Given the above yaml and CI variables, the job payload to Runner should contain the following json:
{
"secrets": {
"DATABASE_PASSWORD": {
"gcp_secret_manager": {
"name": "my-project-secret",
"version": "1",
"server": {
"project_number": "1234",
"workload_identity_federation_pool_id": "pool-id",
"workload_identity_federation_provider_id": "provider-id",
"jwt": "$GCP_SM_ID_TOKEN"
}
}
}
}
}
The runner implementation to handle this payload is done in gitlab-runner!4512 (merged).
Screenshots or screen recordings
The recording can be found in this GitLab Unfiltered video (internal)
Before | After |
---|---|
How to set up and validate locally
It is a rather involved process to test this change. You'd need the following:
- A copy of the Runner implementation gitlab-runner!4512 (merged)
- A GDK that is publicly accessible on HTTPS. This is required by GCP for OIDC verification. The GitLab git repo needs to be checked out to the branch of the MR !139201 (merged). For security reason, it is recommended to run the GDK on a VM and expose the VM, instead of your laptops.
- Configure GCP Workload Identity Federation (see the video above for the detail).
- Create secrets in GCP Secret Manager.
- Configure GCP IAM to allow the Workload Identity Federation Principal to access the GCP Secret Manager secrets (see the vide above for the detail).
- Pull the Runner development branch and build the runner binary.
- Create a test project with the new CI config added in !139201 (merged).
- Add the required GCP related project variables.
- Register the runner and start it.
- Run the pipeline.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #428401 (closed)
Edited by Albert