Draft: Prevent file variable content expansion in downstream pipeline
What does this MR do and why?
Part 2 of #390252 (closed)
Prevent file variable expansion in downstream pipeline.
Before this change, when a trigger job interpolates a file type variable to be used in a downstream pipeline, it would expand the file variable to show the content of the file. This change fixes it such that the content of file variables are not interpolated into a new variable.
This MR depends on the implementation of expand_file_refs:
option in ::ExpandVariables.expand
, implemented in this MR.
Screenshots or screen recordings
Given the following file variables set in the group, project and pipeline: GROUP_FILE_VARIABLE
, PROJECT_FILE_VARIABLE
, PIPELINE_FILE_VARIABLE
.
Without interpolating file variable:
trigger_child:
trigger:
include:
- local: child-pipeline.yml
trigger_cross_project:
trigger:
project: test-group-vars/downstream-project
Without interpolation, there is no change in behaviour. The downstream pipeline will or will not have access to the file variable, depending on the scope (group, project, pipeline). If it has access to the file variable, the file variable will be loaded to the file system and the environment variable set to the file path.
Pipeline | Before | After |
---|---|---|
Child pipeline | ||
Cross project |
With interpolating file variable:
trigger_child_with_interpolation:
variables:
GROUP_FILE_VARIABLE: "$GROUP_FILE_VARIABLE"
PROJECT_FILE_VARIABLE: "$PROJECT_FILE_VARIABLE"
PIPELINE_FILE_VARIABLE: "$PIPELINE_FILE_VARIABLE"
trigger:
include:
- local: child-pipeline.yml
trigger_cross_project_with_interpolation:
variables:
GROUP_FILE_VARIABLE: "$GROUP_FILE_VARIABLE"
PROJECT_FILE_VARIABLE: "$PROJECT_FILE_VARIABLE"
PIPELINE_FILE_VARIABLE: "$PIPELINE_FILE_VARIABLE"
trigger:
project: test-group-vars/downstream-project
With interpolation, there is a change in behaviour. Previously, the downstream pipeline will have the variables interpolated to the content of the files. After the change, the downstream pipeline will get the variables set in the trigger job's variables without interpolation.
Pipeline | Before | After |
---|---|---|
Child pipeline | ||
Cross project |
How to set up and validate locally
- Create a group
test-group
. - Create 2 projects
test-group/main-project
andtest-group/downstream-project
- In
test-group
, create a File variableGROUP_FILE_VARIABLE
. - In
test-group/main-project
, create a File variablePROJECT_FILE_VARIABLE
. - In
test-group/main-project
, create the following CI YMLs for the parent pipeline and child pipeline. - In
test-group/downstream-project
, create the following CI yml - Run pipeline manually on
test-group/main-project
, specifying a File variabelPIPELINE_FILE_VARIABLE
. - Observe the results of the
echo
jobs - Enable feature flag
:ci_prevent_file_var_expansion_downstream_pipeline
- Repeat steps to run pipeline manually
- Observe the results of the
echo
jobs
main-project CI YMLs
# .gitlab-ci.yml
echo:
script:
- echo "GROUP_FILE_VARIABLE $GROUP_FILE_VARIABLE"
- echo "PROJECT_FILE_VARIABLE $PROJECT_FILE_VARIABLE"
- echo "PIPELINE_FILE_VARIABLE $PIPELINE_FILE_VARIABLE"
trigger_child:
trigger:
include:
- local: child-pipeline.yml
trigger_child_with_interpolation:
variables:
GROUP_FILE_VARIABLE: "$GROUP_FILE_VARIABLE"
PROJECT_FILE_VARIABLE: "$PROJECT_FILE_VARIABLE"
PIPELINE_FILE_VARIABLE: "$PIPELINE_FILE_VARIABLE"
trigger:
include:
- local: child-pipeline.yml
trigger_cross_project:
trigger:
project: test-group/downstream-project
trigger_cross_project_with_interpolation:
variables:
GROUP_FILE_VARIABLE: "$GROUP_FILE_VARIABLE"
PROJECT_FILE_VARIABLE: "$PROJECT_FILE_VARIABLE"
PIPELINE_FILE_VARIABLE: "$PIPELINE_FILE_VARIABLE"
trigger:
project: test-group/downstream-project
# child-pipeline.yml
echo:
script:
- echo "GROUP_FILE_VARIABLE $GROUP_FILE_VARIABLE"
- echo "PROJECT_FILE_VARIABLE $PROJECT_FILE_VARIABLE"
- echo "PIPELINE_FILE_VARIABLE $PIPELINE_FILE_VARIABLE"
downstream-project CI yml
echo:
script:
- echo "GROUP_FILE_VARIABLE $GROUP_FILE_VARIABLE"
- echo "PROJECT_FILE_VARIABLE $PROJECT_FILE_VARIABLE"
- echo "PIPELINE_FILE_VARIABLE $PIPELINE_FILE_VARIABLE"
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #390252 (closed)