Add job to trigger code-suggestion-scenarios

What does this MR do and why?

Adds jobs that trigger a multi-project pipeline in the code-suggestion-scenarios project to perform evaluation of the snippets in that project and then expose the results here.

• The pipeline will be available in all code-patterns MRs with a manual trigger.
• The pipeline can be included and run automatically by applying the new label pipeline:run-CS-evaluation.
• The pipeline (in the code-suggestion-scenarios project):
  • Starts a GDK instance using a Docker image built in the MR that triggered the pipeline.
  • Executes a script to get code suggestions from that GDK instance (via the Staging AI gateway).
  • Saves the results as an artifact named scores.csv.
• Results are exposed in the MR widget as Code Suggestions evaluation results via the e2e:code-suggestions-eval-results job.
  • Artifacts in downstream jobs in multi-project pipelines aren't available to upstream jobs. To get around this limitation, the results are fetched from the downstream job via scripts/download-downstream-artifact.rb, which uses the Jobs API

More details of the pipeline can be found in https://gitlab.com/gitlab-com/create-stage/code-creation/code-suggestion-scenarios/-/blob/main/docs/ci_automation.md

This is part of #429619 (closed).

Example results

• In this MR with e2e:code-suggestions-eval triggered manually:
  • MR pipeline: https://gitlab.com/gitlab-org/gitlab/-/pipelines/1110723272
  • run-scenarios in downstream pipeline: https://gitlab.com/gitlab-com/create-stage/code-creation/code-suggestion-scenarios/-/jobs/5772254160
  • e2e:code-suggestions-eval-results ran automatically in this MR's pipeline: https://gitlab.com/gitlab-org/gitlab/-/jobs/5772100868
• In this MR with e2e:code-suggestions-eval run automatically after pipeline:run-CS-evaluation was applied:
  • MR pipeline: https://gitlab.com/gitlab-org/gitlab/-/pipelines/1110756895
  • run-scenarios in downstream pipeline: https://gitlab.com/gitlab-com/create-stage/code-creation/code-suggestion-scenarios/-/jobs/5772328655
  • e2e:code-suggestions-eval-results ran automatically in this MR's pipeline: https://gitlab.com/gitlab-org/gitlab/-/jobs/5772303790

The results appear in the MR widget as (when expanded):

And for verification that the new jobs aren't included when they shouldn't be:

• Docs-only MR: https://gitlab.com/gitlab-org/gitlab/-/pipelines/1110745739
• QA-only MR: https://gitlab.com/gitlab-org/gitlab/-/pipelines/1110744693

MR breakdown

Description	MR
Add CI config to run the evaluation script in code-suggestion-scenarios	gitlab-com/create-stage/code-creation/code-suggestion-scenarios!19 (merged)
Add a job in gitlab-org/gitlab that runs in MRs and triggers the job added in gitlab-com/create-stage/code-creation/code-suggestion-scenarios!19 (merged)	👈 this MR
Link to the results CSV artifact from the MR widget	👈 this MR
Post the results as a discussion in the MR that triggered the evaluation pipeline	follow-up
Document CI variables, GCP requirements, and maintainance runbook	follow-up

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.