Cyclical multi-project pipelines too strict: "job would create infinitely looping pipelines"
Summary
We use a multi-repo/superproject setup with several services as git submodules.
Our "superrepo" is structured like this:
./services/server
./services/client
These are all git submodules pointing towards their respective repositories.
- On a push to the
app
repo it will check for changes in any of the submodule, export the commit sha and triggers a pipeline on theapp
with all of the refs changed. - This 2nd pipeline within the
app
repo triggers all of the specific services and triggers abuild
stage within the service repo (e.g.server
orclient
) and subsequently adeploy
stage. - Each of the updated services will run the
build
anddeploy
.
This setup has been in use for over a year and has been working well for us.
It has never lead to loops within the process, we have a few except:
and only:
entries in the code to prevent that from happening in the first place.
With !61909 (merged) the behavior is changed where it is currently triggering loop protection for this way of using the CI.
This is a screenshot of the process before !61909 (merged) was implemented/activated:
It appears that the check for these cyclical multi-project pipelines might be too strict.
Steps to reproduce
- Create 2 projects
- In the 2nd project create a sample pipeline
- In the 1st project create a CI config which triggers a pipeline on itself (using the API), with a different set of jobs
- Have this secondary pipeline trigger a downstream pipeline (using the
trigger
keyword andstrategy: depend
) on a the 2nd project repository.
Example Project
- https://gitlab.com/thesio/credits-factory/cbk/app (example of failed pipeline: https://gitlab.com/thesio/credits-factory/cbk/app/-/pipelines/316859125)
-
https://gitlab.com/thesio/ops/monitoring/app (example of failed pipeline: https://gitlab.com/thesio/ops/monitoring/app/-/pipelines/316628954).
Note: attempts were made to try to resolve this issue using this repo (e.g. replacing
only:
andexcept:
byrules:
) but this has not yielded any results yet.
What is the current bug behavior?
The trigger
-job executed by the 2nd pipeline which triggers a job in a different repository is getting killed off because it is detected as causing a loop:
It is unclear as to why it believes this causes infinitely looping pipelines.
What is the expected correct behavior?
The job should not be killed off, as it triggers a task within a different repo which never triggers any jobs "back" to the originating repo.
Relevant logs and/or screenshots
N/A
Output of checks
This bug happens on GitLab.com
Results of GitLab environment info
N/A
Results of GitLab application Check
N/A
Possible fixes
Not a fix but it appears to originate from here: !61909 (diffs)
Technical Proposal
- Add
pipeline.source
to the checksum in order to differentiate pipelines coming from different sources even though they will have the same project + sha. - Add a depth value of 2 where we allow at most 2 pipelines with the same checksum rather than none