Allow variables in cache policy field

What does this MR do?

Allow variables in cache:policy field

Runner implementation: gitlab-runner!4085 (merged)

Related issues

Fix #371480 (closed)

How to run

1. Checkout this branch and the corresponding cache-policy-variable branch in gitlab-runner!4085 (merged)
2. Run the runner

Testing

The following CI examples show the new behavior:

Overwriting the value based on rules variables

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "web"

variables:
  CACHE_POLICY: pull

stages:
  - test

job_valid_variable_policy:
  stage: test
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
      variables:
        CACHE_POLICY: pull-push
  tags:
    - runner
  script:
    - echo "Hello World" >> test.txt
  cache:
    key: 'foobar'
    policy: $CACHE_POLICY
    unprotect: true
    paths:
      - test.txt

Using a variable with an invalid value:

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "web"

variables:
  CACHE_POLICY: pull

stages:
  - test
 
job_invalid_variable_policy:
  stage: test
  rules:
    - variables:
        CACHE_POLICY: blah
  tags:
    - runner
  script:
    - echo "Hello World" >> test.txt
  cache:
    key: 'foobar'
    policy: $CACHE_POLICY
    unprotect: true
    paths:
      - test.txt

Hard-coded invalid value:

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "web"

variables:
  CACHE_POLICY: pull

stages:
  - test

job_invalid_hardcoded_policy:
  stage: test
  tags:
    - runner
  script:
    - echo "Hello World" >> test.txt
  cache:
    key: 'foobar'
    policy: blah
    unprotect: true
    paths:
      - test.txt

Checklist

Pre-merge

Consider the effect of the changes in this merge request on the following:

• Different pipeline types
• Non-canonical projects:
  • gitlab-foss
  • security
  • dev
  • personal forks
• Pipeline performance

If new jobs are added:

• Change-related rules (e.g. frontend/backend/database file changes): _____
• Frequency they are running (MRs, main branch, nightly, bi-hourly): _____
• Add a duration chart to https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations if there are new jobs added to merge request pipelines

This will help keep track of expected cost increases to the GitLab project average pipeline cost per merge request RPI

Post-merge

• Consider communicating these changes to the broader team following the communication guideline for pipeline changes