Stop expanding file variables when sending to Runner

What does this MR do and why?

This MR stops expanding file variables when sending to Runner.

When file variables are referred to in other variables, we should not expand them directly because users expect that file variables are expanded as their file paths, not their values. In order to expand them as their file paths, we should send them to Runner unexpanded.

This change is behind a feature flag ci_stop_expanding_file_vars_for_runners #369907 (closed).

Related to #365859 (closed), however, the main issue is #29407 (closed).

Details: #29407 (comment 947110258) & #29407 (comment 989127953)

This work is actually the first step of this implementation:

Step	Status
1. GitLab: Stop expanding file variables when sending to Runner	👈 You are here
2. Runner: Refactor the creation of the job temporary file path	gitlab-runner#29128 (closed)
3. Fix file variables in Runner	#29407 (closed)

Screenshots or screen recordings

Added some file variables;

Used this config YAML which includes some examples from comments in this and another issue;

variables:
    EXTRA_ARGS: "-f $TEST_FILE"
    DOCKER_REMOTE_ARGS: --tlscacert="$DOCKER_CA_CERT"
    EXTRACTED_CRT_FILE: ${DOCKER_CA_CERT}.crt
    MY_FILE_VAR: $TEST_FILE

test:
    script:
        - echo "run something $EXTRA_ARGS"
        - echo "docker run $DOCKER_REMOTE_ARGS"
        - echo "run --output=$EXTRACTED_CRT_FILE"
        - echo "Will read private key from $MY_FILE_VAR"

Before enabling the feature flag

When Runner asks the job, GitLab send the job variables in this format;

 {:key=>"TEST_FILE", :value=>"hello, this is test", :public=>false, :file=>true, :masked=>false},
 {:key=>"EXTRA_ARGS", :value=>"-f hello, this is test", :public=>true, :masked=>false},
 {:key=>"DOCKER_CA_CERT", :value=>"BEGIN\nthis is secret\nEND", :public=>false, :file=>true, :masked=>false},
 {:key=>"DOCKER_REMOTE_ARGS", :value=>"--tlscacert=\"BEGIN\nthis is secret\nEND\"", :public=>true, :masked=>false},
 {:key=>"EXTRACTED_CRT_FILE", :value=>"BEGIN\nthis is secret\nEND.crt", :public=>true, :masked=>false},
 {:key=>"MY_FILE_VAR", :value=>"hello, this is test", :public=>true, :masked=>false},

As you can see, file variables are expanded into YAML variables.

Job result:

Enable the feature flag

Feature.enable(:ci_stop_expanding_file_vars_for_runners)

After enabling the feature flag

GitLab start sending variables of the job like this;

 {:key=>"TEST_FILE", :value=>"hello, this is test", :public=>false, :file=>true, :masked=>false},
 {:key=>"EXTRA_ARGS", :value=>"-f $TEST_FILE", :public=>true, :masked=>false},
 {:key=>"DOCKER_CA_CERT", :value=>"BEGIN\nthis is secret\nEND", :public=>false, :file=>true, :masked=>false},
 {:key=>"DOCKER_REMOTE_ARGS", :value=>"--tlscacert=\"$DOCKER_CA_CERT\"", :public=>true, :masked=>false},
 {:key=>"EXTRACTED_CRT_FILE", :value=>"${DOCKER_CA_CERT}.crt", :public=>true, :masked=>false},
 {:key=>"MY_FILE_VAR", :value=>"$TEST_FILE", :public=>true, :masked=>false},

As you can see, file variables are not expanded. However, now they are expanded by Runner;

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.