Skip to content

Add suffix configuration option to Pipeline execution policies

What does this MR do and why?

Uniqueness enforcement causes challenges for compliance teams that may use the built-in templates twice, causing the jobs to be duplicate.

With this change we are adding a new policy configuration option suffix: on_conflict/never. The default value is on_conflict.

If policy uses suffix: on_conflict and a job with the same name exists in the pipeline, we will add suffix in a format :policy-<security-policy-project-id>-<policy-index>.

The suffix is added following the policy hierarchy.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before (or with FF disabled) After
CleanShot_2024-08-16_at_17.23.31_2x CleanShot_2024-08-16_at_17.24.50_2x
Dependency on a conflicting job Dependency on a non-conflicting job Dependencies on both jobs
CleanShot_2024-08-20_at_08.56.33_2x CleanShot_2024-08-20_at_08.56.39_2x CleanShot_2024-08-20_at_08.56.40_2x

How to set up and validate locally

  1. Enable feature flag pipeline_execution_policy_suffix.
  2. Create a project
  3. Create .gitlab-ci.yml:
    include:
      - template: Jobs/Secret-Detection.gitlab-ci.yml
    
    duplicate-job:
      stage: test
      script:
        - echo "From project."
  4. Create pipeline execution policy with suffix: on_conflict (or without this option specified) and the following config file:
    include:
      - template: Jobs/Secret-Detection.gitlab-ci.yml
    
    duplicate-job:
      stage: test
      script:
        - echo "From policy."
  5. Run a pipeline and verify that there is no error
  6. Verify that duplicate-job:policy-<spp-id>-<policy-index> is added into the pipeline in addition to the duplicate-job and its output says "From policy."
  7. Update the policy CI file to add more jobs and define dependencies:
    other-policy-job:
      script:
        - "sleep 60 && echo 'policy'"
    
    rspec:
      stage: .pipeline-policy-pre
      script:
        - "sleep 20 && echo 'rspec'"
    
    jest:
      stage: test
      script:
        - "sleep 10 && echo 'jest'"
    
    upload-rspec-coverage:
      stage: deploy
      needs:
        - rspec
      script:
        - echo "requires rspec"
    
    upload-jest-coverage:
      stage: deploy
      needs:
        - jest
      script:
        - echo "requires jest"
    
    deploy:
      stage: deploy
      needs:
        - rspec
        - jest
      script:
        - echo "deploy"
  8. Update project's .gitlab-ci.yml to also define rspec job:
    project-job:
      script:
        - echo "project"
    
    rspec:
      script:
        - echo "project rspec"
  9. Run a pipeline and verify that dependencies were updated correctly for the suffixed job
  10. Change the policy to suffix: never and verify that a job uniqueness error is raised when you try to run a pipeline.

Related to #473189 (closed).

Edited by Martin Čavoj

Merge request reports

Loading