Feedback: collect test selection gaps for tier-1 pipelines

The purpose of this issue

Please let us know if a specific file is changed, what tests we should also run in the pipelinetier-1 pipelines, in the form of 2 links in total:

• A link to the pipelinetier-1 pipeline which was passed (green)
• A link to the pipelinetier-2 or pipelinetier-3 pipeline which then failed (red)

This usually means some tests we run in pipelinetier-2 / pipelinetier-3 pipelines should actually also run in the pipelinetier-1 pipelines, based on specific changes.

Please post this in each its own thread so we can discuss the specific cases in each threads. A merge request to improve it directly is more than welcomed. See the next section for examples how we can improve it.

Examples how we can improve this

Here are some merge requests which improved the mappings, from changes to the tests we need to run in the pipelinetier-1 pipelines:

• !138049 (merged)
• !138525 (merged)

Background

Before a merge request is approved, we run pipelinetier-1 pipelines. It's a kind of pipeline that we only run a very small subset of tests based on changes.

After the merge request is approved, and getting more approvals, we run more and more tests based on the number of approvals. Given that we run more tests, some new failures can be detected.

We should be able to shift them left so that we also run those failed tests in the pipelinetier-1 pipelines, and then we can progress more smoothly.