Skip to content

Backend: Disable expansion of file type variables when referenced in gitlab-ci.yml variables section

Status update (2022-10-19)

  • We have a MR ready to ship. When shipped (target 15.7), job variable expansion will be changed and handled as expected for variables of type = file. In summary, going forward, variables that reference/alias file variables are expanded to the file variable's file name/path instead of its value (i.e., the file's content).

This is a breaking change but does resolve a potential security issue as secrets may be leaked with the current bug.

  • The plan is to simply ship the MR with the fix. There is no option as mentioned in some of the threads below, for shipping this fix behind a feature flag, providing a warning to the users.

  • I will keep the option open to pull the ship date into 15.6, as there are solid reasons, as outlined below, to do so. But I do want to give other users and customers that have not been following this issue the opportunity to be made aware of this change as there is a likelihood that the fix does, in fact impact a current workflow. Add comments to the new comment thread below. @DarrenEastman

Link to before and after.

Summary

When using file variables in the "variables" section of a job in .gitlab-ci.yml the contents are being written to the new variable.

Steps to reproduce

The following gitlab-ci.yml

.deployment:
   image: alpine:latest
   script:
      - kubectl config set-cluster ${KUBE_CLUSTER} --server=${KUBE_SERVER} --certificate-authority="${KUBE_CA}"

"Deploy to stage":
   extends: .deployment
   stage: deploy:stage
   when: manual
   variables:
      KUBE_CA: $KUBE_CA_STAGE

"Deploy to prod":
   extends: .deployment
   stage: deploy:prod
   when: manual
   variables:
      KUBE_CA: $KUBE_CA_PROD

The variable KUBE_CA_PROD / KUBE_CA_STAGE are file variables in Gitlab. When using the variable KUBE_CA the contents are no longer a path to a file but the content of the file.

What is the current bug behavior?

Contents of file variables are written to variables

What is the expected correct behavior?

Path of file variable is used as variable

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:         Ubuntu 18.04
Proxy:          no
Current User:   git
Using RVM:      no
Ruby Version:   2.5.3p105
Gem Version:    2.7.9
Bundler Version:1.17.3
Rake Version:   12.3.2
Redis Version:  3.2.12
Git Version:    2.21.0
Sidekiq Version:5.2.7
Go Version:     unknown


GitLab information
Version:        11.11.2-ee
Revision:       cfdecb7c5de
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     9.6.11
URL:            https://gitlab.XXXXX.com
HTTP Clone URL: https://gitlab.XXXXX.com/some-group/some-project.git
SSH Clone URL:  git@gitlab.XXXXX.com:some-group/some-project.git
Elasticsearch:  no
Geo:            no
Using LDAP:     yes
Using Omniauth: yes
Omniauth Providers:


GitLab Shell
Version:        9.1.0
Repository storage paths:
  • default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
not relevant

Proposal

Step Status
1. GitLab: Stop expanding file variables when sending to Runner #365859 (closed)
2. Runner: Refactor the creation of the job temporary file path gitlab-runner#29128 (closed)
3. Fix file variables in Runner 👈 You are here

Technical

#29407 (comment 935276800)

Note

TODO: Update the docs in !75477 (merged) when this issue is closed.

Solution

Once gitlab-runner!3613 (merged) is merged, file variables and variables the reference file variables will work as follows:

Having created a file variable via the GitLab UI named A_FILE_VAR with content this is some super secret content, the following job definition:

work:
   stage: test
   variables:
     REF_FILE_VAR: $A_FILE_VAR
   script:
     - echo $A_FILE_VAR
     - cat $A_FILE_VAR
     - echo $REF_FILE_VAR
     - cat $REF_FILE_VAR

will result in the following before and after outputs:

Before

$ echo $A_FILE_VAR
/builds/avonbertoldi/test-project.tmp/A_FILE_VAR
$ cat $A_FILE_VAR
this is some super secret content
$ echo $REF_FILE_VAR
this is some super secret content
$ cat $REF_FILE_VAR
cat: can't open 'this': No such file or directory
cat: can't open 'is': No such file or directory
cat: can't open 'some': No such file or directory
cat: can't open 'super': No such file or directory
cat: can't open 'secret': No such file or directory
cat: can't open 'content': No such file or directory

After

$ echo $A_FILE_VAR
/builds/avonbertoldi/test-project.tmp/A_FILE_VAR
$ cat $A_FILE_VAR
this is some super secret content
$ echo $REF_FILE_VAR
/builds/avonbertoldi/test-project.tmp/A_FILE_VAR
$ cat $REF_FILE_VAR
this is some super secret content
Edited by Axel von Bertoldi