Handle Duplication of Resources on Migration
When migrating projects using the new :bulk_import_projects
feature flag, resources can be duplicated due to race conditions.
See the related discussion for examples, but the pipeline will run multiple times for the same resource
The following discussion from !74100 (merged) should be addressed:
-
@acook.gitlab started a discussion: (+14 comments) @georgekoltsov I've investigated this with @Alexand and this change is due to a race condition.
- PipelineSchedulesPipeline starts importing a schedule
- Another worker runs, sees it's started then starts it again
- You end up with multiple copies of the same pipeline schedule imported
When the worker sees a pipeline is already started for this, should it still try to run? This seems counterintuitive to me
There's an explicit test looking for this functionality though added on 9/28 by you so I think we might need to discuss what to do here