[RUN_AS_FOSS] BulkImports: Add PipelineWorker

What does this MR do?

The Pipelines are grouped in Stages, like the CI jobs. Stages run in sequence, one after the other, while the pipelines within a stage run in parallel.

Each pipeline runs on an, individual, BulkImports::PipelineWorker job, which enables:

• smaller/shorter jobs: Background jobs can be interrupted during deploys or other unexpected infrastructure/ops events. To make jobs more resilient it's desirable to have smaller jobs wherever possible.
• faster imports: Some pipelines can run in parallel, which reduces the total time of an import.
• (follow-up) network/rate limits handling: When a pipeline gets rate limitted, it can be schedule to retry after the rate limit timeout. #262024 (closed)

Current Stages

flowchart LR
    subgraph stage0
        StartS0["Start"] --> Group --> FinishS0["Finished"]

        S0Note["All the following <br> Stages depends <br> on Group"]
        S0Note:::note
    end
    subgraph stage1
        StartS1["Start"] --> Subgroup --> FinishSubgroup["Finished"]
        StartS1 --> Members --> FinishMembers["Finished"]
        StartS1 --> Labels --> FinishLabels["Finished"]
        StartS1 --> Milestones --> FinishMilestones["Finished"]
        StartS1 --> EE::Iterations --> FinishIterations["Finished"]
    end
    subgraph stage2
        StartS2["Start"] --> Epic --> FinishEpic["Finished"]

        S2Note["Epics depends on<br>Labels and Members<br>That's why it cannot<br>run in the stage1"]
        S2Note:::note
    end
    subgraph stage3
        StartS3["Start"] --> EpicAwardEmoji --> FinishEpicAwardEmoji["Finish"]
        StartS3 --> EpicEvents --> FinishEpicEvents["Finish"]

        S3Note["Epics subrelations<br>Depends on Epics"]
        S3Note:::note
    end
    subgraph stage4
        StartS4["Start"] --> EntityFinisher --> FinishEntityFinisher["Finish"]

        S4Note["Mark the entity<br>as finished."]
        S4Note:::note
    end

    Start --> stage0
    stage0 --> stage1
    stage1 --> stage2
    stage2 --> stage3
    stage3 --> stage4

    classDef note fill:#ffd,stroke-width:0;

Related to:

• issue: #323384 (closed)
• epic: &5544

Screenshots (strongly suggested)

bulk_import_concurrent2

Does this MR meet the acceptance criteria?

Conformity

• 📋 Does this MR need a changelog?
  • I have included a changelog entry.
  • I have not included a changelog entry because _____.
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team