Investigate any test gap between selective and full Jest jobs in Merge Request pipelines
The following discussions from !103548 (merged) should be addressed:
-
@ddieulivol started a discussion: (+1 comment) thought (non-blocking): Do we have a plan to find out the test gaps? If not, I was thinking about the following:
- Let this MR run for a month or so
- After that time, find all of the MRs for which the post-approval full jest jobs failed, but the pre-approval selected jest jobs succeeded. This would give us some missing data, and we could fill in some of the test gaps.
We would probably have to repeat these steps, or we could do the experiment over a longer period, until we think that we captured most of the test gaps...
Another potential iteration: We could also "fade out" this MR gradually (e.g. on 50% of the pipelines, we still run the full jest specs post-approval, and on 50% not), so that users have a less frustrating experience.
Generally, it would be awesome to have reporting/monitoring for the pipelines that passed the selective tests, and failed the full jest tests (and maybe we already have this, I'm not sure)
Not sure if it's realistic, but I wanted to jot my thoughts down
😆 -
@godfat-gitlab started a discussion: This looks like outdated because we're no longer doing this. It's indeed true back then though: https://gitlab.com/gitlab-org/gitlab/-/blob/3616946936c1adbd9e754c1bd06f86ba670796d8/.gitlab/ci/rules.gitlab-ci.yml#L534-535
-
@godfat-gitlab started a discussion: This doesn't seem to match
frontend-dependency-patterns
, but it looks like matchingfrontend-build-patterns
. To match current behaviour:- when any frontend dependency file is changed (for example, `package.json`, `yarn.lock`, `config/webpack.config.js`, `config/helpers/**/*.js`)
-
@godfat-gitlab started a discussion: This is also outdated, and now only runs for
.gitlab/ci/rules.gitlab-ci.yml
and.gitlab/ci/frontend.gitlab-ci.yml
:- when relevant CI configuration file is changed (`.gitlab/ci/rules.gitlab-ci.yml`, `.gitlab/ci/frontend.gitlab-ci.yml`)
-
@godfat-gitlab started a discussion: 🤔 looks like there are extra period, and the list is outdated. Maybe we can reword it so we don't have to always keep it a full list?| `frontend-dependency-patterns` | Only create job when frontend dependencies are updated (for example, `package.json`, and `yarn.lock`) changes. |