WIP: Proof-of-Concept: Add native support in Runner for HashiCorp Vault integration with CI_JOB_JWT
What does this MR do?
Implements "native" support for handling HashiCorp Vault secrets in Runner, using the upcoming CI_JOB_JWT
change () - in a way described at gitlab#212252 (comment 317690659).
The only difference is that the current implementation relays on a special variable (VAULT_DEFINITION
), since support for .gitlab-ci.yml
syntax must be added to GitLab in the Rails app. But the syntax of YAML in this variable is exactly what should be added to .gitlab-ci.yml
"natively" in the future, and it uses the "target" API structures. The disadvantage of the variable configuration is that we can't overwrite the configuration with extends:
and the common parts must be repeated.
Example configuration syntax (taken from a test project at https://gitlab.com/tmaczukin-test-projects/test-vault-jwt):
define secrets job:
stage: start
variables:
VAULT_DEFINITION_ALPHA: |
my-vault-server:
server:
url: https://vault.maczukin.dev
auth:
name: jwt
path: ${CI_SERVER_HOST}/jwt
data:
jwt: ${CI_JOB_JWT}
role: pipeline-job
secrets:
SHARED_SECRET:
engine:
name: kv2
path: ${V_SECRETS_PATH}
path: shared
# strategy: read # not defining it, since it's default when none action is defined
WRITE_TEST:
engine:
name: kv2
path: ${V_SECRETS_PATH}
path: ${V_PIPELINE_PATH}/test
strategy: write
script:
- find ${SHARED_SECRET}
- randomKey=$(dd if=/dev/urandom count=10 bs=1 | base64 | sha256sum | cut -f 1 -d " ")
- randomValue=$(dd if=/dev/urandom count=10 bs=1 | base64 | sha256sum | cut -f 1 -d " ")
- echo "${randomValue:0:10}" > ${WRITE_TEST}/${randomKey:0:10}
- find ${WRITE_TEST}
Why was this MR needed?
As an experiment on how hard it would be to add such "native" integration using the JWT authentication method. Thanks to this, the users don't need to use the vault
image or install vault
binary in their job environment nor to execute manually Vault CLI commands.
This is also the first step to make GitLab Runner project to use Vault for storing and sharing secrets between our projects (like credentials for our S3/GCS buckets where we release different binaries).
Are there points in the code the reviewer needs to double check?
Many, starting with missing tests. But this is PoC, and the tests will be added before removing WIP
status
Does this MR meet the acceptance criteria?
-
Documentation created/updated -
Added tests for this feature/bug -
In case of conflicts with master
- branch was rebased