Skip to content

Test scaling security->canonical sync with a fast CI

Alessio Caiazza requested to merge fast-ci-security-canonical-sync into master

What does this MR do and why?

During a security release, the content of the default branch diverges from canonical and security as soon as we start processing the vulnerability fixes.

At the end of each security release, the release managers sync the content of gitlab-org/security/gitlab:master into gitlab-org/gitlab:master, the process does not scale well with the number of changes affecting GitLab, and we recently moved to use the merge-train tool to address some of the problems gitlab-com/gl-infra/delivery#19402 (closed)

In this merge request, I am proposing the first step a long-term solution described in gitlab-com/gl-infra/delivery#19460.

Today we have the merge-train project cloning both repositories, doing a local merge, and pushing the changes into the canonical repo.

This process will fail if another merge request advances the default branch during this operation. To handle that we pull again and perform another merge.

The following diagram explains how the process looks today.

sequenceDiagram
    actor rm as Relese Manager
    participant rt as release-tools
    participant mt as merge-train
    participant gl as gitlab.com
    actor Maintainer
    rm->>rt: Sync security default branch
    rt->>mt: merge sercurity:master->canonical:master
    activate mt
    mt->>+gl: clone gitlab-org/security/gitlab
    gl-->>-mt: security:master
    mt->>+gl: clone gitlab-org/gitlab
    gl-->>-mt: canonical:master
    
    par Release Process
    mt->>mt: merge security:master into canonical:master
    and Development Process
    Maintainer->>+gl: Merge gitlab-org/gitlab!12345
    gl-->>-Maintainer: merged
    end

    mt->>+gl: git push
    gl-->>-mt: ❌ push failure, master advanced
    loop up to 5 times if push fails
    mt->>+gl: pull gitlab-org/gitlab master
    gl-->>-mt: canonical:master
    mt->>mt: merge HEAD into canonical:master
    mt->>+gl: git push
    gl-->>-mt: push result
    end
    deactivate mt

    rm->>rt: Very mirroring status

In this merge request, we are introducing a set of changes to our CI configuration to only have the no-op job when a merge request is from the default branch of the security mirror and targets the default branch of the canonical repo. Every change merged on the security repository has already been reviewed and approved by the relevant maintainers and Appsec.

With this setup, syncing back the security changes could be reduced to an API call to create the above MR.

Screenshots or screen recordings

In !125863 (closed) we can see this MR in action.

To test it before merging this MR, we patched the logic with !125863 (50c8a892)

As we can see in the above MR, its pipeline only took 7 seconds to run

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Alessio Caiazza

Merge request reports