Update Ci::PipelineCreationMetadata for merge request pipelines

Change details

This MR updates the Ci::PipelineCreationMetadata class (now renamed to PipelineCreation::Requests), which represents the metadata for the pipeline creation process up until it succeeds and is persisted or fails. It stores the data in Redis and it is retained for 5 minutes.

The data structure looks like this:

 {
    "REDIS_KEY": {
      "CREATION_ID": { "status": "creating" }
    }
 }

What is this for?

Summary: Currently, the FE has no way of knowing when the pipeline creation process has started, and in order to be able to have proper UX when creating pipelines async we need to find a way to let the FE know about the pipeline creation.

Example: in Merge Request pipelines, when a user clicks on Run Pipeline an alert shows up in the bottom left that mentions that the pipeline is being created. This alert is based on the request being completed and can cause confusion, as the pipeline might not be created at all. This also causes a bunch of UI bugs, like the merge request widget not updating from Merge -> Auto merge . This is also why when you click on Run pipeline in the merge request pipeline page, it takes about a minute to show a pipeline, with no feedback about what is happening. The only feedback currently is a recent client side implementation of the skeleton loader, which will be able to be updated with this change to actually be persisted state.

This change will also enable us to implement async pipelines in pipelines/new and the external API endpoints .

But also and perhaps most importantly - this fixes:

An existing bug with the Merge button 😱

This screen grab shows the fixed behavior:

Screen Recording 2024-09-23 at 14.05.23.mov

What's next?

• Follow-up: [Feature flag] Rollout of `ci_redis_pipeline_cr... (#499828) <- Feature flag rollout so no worries 🌞 we merge, we start rolling 🚚
• Follow-up: Extract all the related methods in service class (#494442) <- Assigned to Laura, working on it after the Vuln 🔴
• Follow-up: Use Redis pipeline creations for MR commit push... (#499838) <- Assigned to Avielle, working on it while other MR is in review
• Follow-up: Address race condition in Requests#pipeline_cre... (#499843) <- Assigned to Avielle
• Issue needed (todo -> search before creating): Implement this for pipelines/new which will enable us to create async pipelines in the endpoint.
  • UX/Design ✅ done
  • FE issues

These follow-ups are immediate and will be worked on next alongside product priorities for this and any upcoming milestone :)