[RUN AS-IF-FOSS] BulkImports: Track pipeline worker with BulkImports::Tracker#status

What does this MR do?

Context

Create a database representation, with the BulkImports::Tracker, for each BulkImport Pipeline (include BulkImports::Pipeline). This way, the pipeline progress can be tracked using the status column (state machine).

With this change we can run each pipeline on its own background job (#323384 (closed)), which will bring the power of retrying the pipeline in case of failures, like rate limiting.

Besides that, having each pipeline progress in the database will be handy to create a better UI to give an accurate sense of progress to the user in the future.

• Merge request that introduced the status column to BulkImports::tracker: !5568 (merged)

The change

Before this change, BulkImports::Tracker was tracking only the pagination status, when required. So, a record was only created when the pipeline required pagination handling.

Now, before running a pipeline a BulkImports::Tracker record is created, and when the pipeline is finished, failed or skipped, these status are also updated/tracked in the pipeline's BulkImports::Tracker record.

The pipeline status is required to run them in individual background jobs because the pipelines have an order to run. For instance, we cannot import Epics before importing the Group or the Group labels.

• Related to: #324109 (closed)

Next step

• #323384 (closed) - Create the BulkImports::PipelineWorker to run each pipeline on its own job.
• !55928 (merged) - The code duplication on the #after_run method became more evident after this change. This MR aims to DRY this duplication.

References

• Epic: &5544
• Spike where the idea was tested: !54970 (closed)

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

• 📋 Does this MR need a changelog?
  • I have included a changelog entry.
  • I have not included a changelog entry because _____.
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team