Expand parallel:matrix variables in 'needs' and 'dependencies'
What does this MR do and why?
Expand parallel:matrix variables in 'needs' and 'dependencies'
Related to: #396845, #423553 and #255311
This would help to create a chain of parallel matrix stages and have a one-to-one matching between jobs (for fetching artifacts for instance). Only the variables from the parallel:matrix
are expanded (more like an interpolation) during the normalization stage of parallelized jobs. Nested variables are thus not handled.
The current implementation does not break the existing normalization
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
There is no UI change required by the related issues. The CI YAML config is expected to be valid when using the matrix variables in dependencies
, needs:job
or needs:parallel:matrix
keywords
Before | After |
---|---|
Case 1: Using matrix variables with Case 2: Using matrix variables with Case 3: Using matrix variables with |
The CI config YAML should be valid |
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
The following examples should be valid when the matrix variable expansion is fixed. Each parallelized test
job could then fetch artifacts (or depend) on the corresponding parallelized build
job (and that job only).
Case 1: Using matrix variables with dependencies
keyword
default:
image: "ubuntu:22.04"
stages:
- build
- test
.matrix:
parallel:
matrix:
- PROVIDER: [ "centos" ]
STACK: [ "ruby", "python"]
build:
extends: .matrix
stage: build
script:
- echo "Building ... ${PROVIDER} ${STACK}"
test:
extends: .matrix
stage: test
script:
- echo "Building ... ${PROVIDER} ${STACK}"
dependencies:
- "build: [$PROVIDER, $STACK]"
Case 2: Using matrix variables with needs:job
keyword
default:
image: "ubuntu:22.04"
stages:
- build
- test
.matrix:
parallel:
matrix:
- PROVIDER: [ "centos" ]
STACK: [ "ruby", "python"]
build:
extends: .matrix
stage: build
script:
- echo "Building ... ${PROVIDER} ${STACK}"
test:
extends: .matrix
stage: test
script:
- echo "Building ... ${PROVIDER} ${STACK}"
needs:
- job: "build: [$PROVIDER, $STACK]"
Case 3: Using matrix variables with needs:parallel:matrix
keyword
default:
image: "ubuntu:22.04"
stages:
- build
- test
.matrix:
parallel:
matrix:
- PROVIDER: [ "centos" ]
STACK: [ "ruby", "python"]
build:
extends: .matrix
stage: build
script:
- echo "Building ... ${PROVIDER} ${STACK}"
test:
extends: .matrix
stage: test
script:
- echo "Building ... ${PROVIDER} ${STACK}"
needs:
- job: "build"
parallel:
matrix:
- PROVIDER: $PROVIDER
STACK: $STACK