Backend: Allow predefined manipulations in CI config interpolation - part 1
Problem
When using the interpolation feature in a CI config ($[[ inputs.input-id ]]
) we can only replace the literal value of the input as a string.
- As a user I would like to use interpolation where variables expansion is currently not supported (e.g. job name): #387632 (closed)
- As a user I would like to truncate variables to a specific length so that my 3rd party integrations can work safely: &10431
Proposal
By leveraging the new interpolation feature we can have an opportunity to add features here as we interpolate content with its value. We could do that by adding jinja2-style syntax:
$[[ <access> | <func1> | <func2> | ... <funcN> ]]
The <access>
today is given by inputs.input-id
syntax. We could also support other data access like vars.CI_PROJECT_PATH
(exposing variables) or event.name
(exposing CI event data) or simply a string.
The func
will be some predefined functions we support to manipulate values during the interpolation. Multiple functions can be chained together.
Example 1. Force variable expansion from inputs:
##
# component's template.yml
spec:
inputs:
prefix:
---
"job-$[[ inputs.prefix | expand_vars ]]":
script: echo hello world
##
# .gitlab-ci.yml
include:
- component: gitlab.com/path/to/component@1.0
inputs:
prefix: $[[ '$CI_PROJECT_NAME-$MY_VAR' | expand_vars]]
The expand_vars
function could automatically check if a masked variable is being expanded and raise an error.
Example 2. Truncate variables for 3rd party integration:
job:
variables:
TRUNCATED_CI_REF_SLUG: $[[ vars.CI_COMMIT_REF_SLUG | expand_vars | truncate(0,10) ]]
environment: $TRUNCATED_CI_REF_SLUG
script: ...
The truncate(offset,size)
function could take in input a string and truncate it. Note that for truncating a variable's value you should first expand it or it would truncate the literal variable's name.
Next steps
We are going to break this effort into 3 separated issues
- Support for pipeline of functions in the interpolation logic. This implements the mechanism that recognizes the syntax
| <func1> | <func2>
, calls the function and pass the output to the next function in sequence - (this issue) - Implement
expand_vars
function. #387632 (closed) - Implement
truncate
function. #408795
MR Implementation
Description | MR |
---|---|
Allow predefined functions in CI config interpolation; add truncate fxn - This MR solves both Step 1 and Step 3 in the proposal. | !125802 (merged) |