Verify ci config for default and stable branches
What does this MR do and why?
Addresses #34040 (closed) iteration 1 and 5
This MR creates 2 manual jobs in Gitlab Pipeline when changes are made to the Gitlab CI.
pipeline-config-test stable-branch
pipeline-config-test default-branch
These jobs are allowed to fail, and must be manually triggered since they are resource intensive. The following actions will take place:
- a new Gitlab branch will be created (a test branch mimicing a
stable branch
ordefault branch
, depending on the job) - There will be a pipeline created for that branch.
- pipeline jobs will be validated against a reference pipeline(if User provides the reference pipeline ID through a
REFERENCE_PIPELINE_ID
environment variable), or it will compare against the list of jobs stored inexpected_default_branch_jobs
orexpected_stable_branch_jobs
to see if there are any job inconsistencies, either missing or added. - the pipeline is cancelled right after the last step finishes, to avoid pipelines taking up too much resources
- the test branch gets deleted
What needs to be improved:
The jobs have the following 3 scripts:
create_branch "$TEST_BRANCH" "$CURRENT_COMMIT" "$GITLAB_RPOJECT_ID"
create_pipeline_and_validate_jobs "$PIPELINE_CONTEXT" "$TEST_BRANCH" "$GITLAB_RPOJECT_ID"
delete_branch "$TEST_BRANCH" "$GITLAB_RPOJECT_ID"
- If the job failed during the 2nd script, we need to manually delete the branch created by script 1 before we can rerun the jobs.
- The list of
expected jobs
is actually more than what would run in a regular stable or default branch, because the reference job list was generated from a branch that contains ci changes and therefore triggers more jobs than usual. I need to trim down the list of jobs so it doesn't include the ones that only run for CI changes.
Screenshots or screen recordings
Added these 2 manual jobs in the sync
stage below:
Example of a matching test result: https://gitlab.com/gitlab-org/gitlab/-/jobs/3074374225
Example of a test showing that the pipeline jobs are inconsistent with the reference pipeline:
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.