Run minimal rspec jobs before merge request is approved
What does this MR do?
Configures pipeline to run rspec jobs based on a merge request approval status.
- Before a merge request is approved:
- rspec minimal jobs will be created, which runs rspec files that are related to the changes in the Merge Request
- After a merge request is approved:
- full rspec jobs will be created, which runs all rspec files.
Expected result
- Shorter rspec job duration in merge requests that change backend code or database migrations
- No change in merge requests that only change frontend or QA
Part of #222369 (closed)
Verifications
Backend changes
Test Merge Request: !64910 (closed)
- pipeline before approval: https://gitlab.com/gitlab-org/gitlab/-/pipelines/329446625/builds
- pipeline after approval: https://gitlab.com/gitlab-org/gitlab/-/pipelines/329545913/builds
Baseline Merge Request: !64972 (closed)
- baseline pipeline: https://gitlab.com/gitlab-org/gitlab/-/pipelines/328661384/builds
Summary:
- Selected rspec tests are run based on the test mapping: https://gitlab.com/gitlab-org/gitlab/-/jobs/1388117599#L64, resulting in shorter average rspec job duration.
Mean job duration
Note: actual job duration may vary depending on number of tests needed for the MR
jobs | baseline pipeline - full tests | before approval pipeline - minimal tests | after approval pipeline - full tests |
---|---|---|---|
rspec unit pg12 | 21:24 | 11:08 | 21:03 |
rspec integration pg12 | 22:07 | 10:44 | 21:51 |
rspec system pg12 | 16:51 | 13:05 | 16:41 |
rspec-ee unit pg12 | 20:48 | 15:10 | 20:18 |
rspec-ee integration pg12 | 18:26 | 11:34 | 17:58 |
rspec-ee system pg12 | 20:51 | 14:24 | 20:58 |
rspec-ee unit pg12 geo | 15:31 | 9:03 | 15:51 |
rspec-ee integration pg12 geo | 6:41 | 5:01 | 7:06 |
rspec-ee system pg12 geo | 9:01 | 4:20 | 8:49 |
Database changes
Test Merge Request: !64914 (closed)
- pipeline before approval: https://gitlab.com/gitlab-org/gitlab/-/pipelines/335241198/builds
- pipeline after approval: https://gitlab.com/gitlab-org/gitlab/-/pipelines/335349062/builds
Baseline Merge Request: !64972 (closed)
- baseline pipeline: https://gitlab.com/gitlab-org/gitlab/-/pipelines/328661384/builds
Summary:
- Only related database spec needs to be run: https://gitlab.com/gitlab-org/gitlab/-/jobs/1415978872#L60. This spec is run in the job https://gitlab.com/gitlab-org/gitlab/-/jobs/1415978949. Other
rspec
jobs don't have any tests to run, so they end immediately. Thus the average job duration is at ~2 minutes.
Mean job duration
Note: actual job duration may vary depending on number of tests needed for the MR
jobs | baseline pipeline - full tests | before approval pipeline - minimal tests | after approval pipeline - full tests |
---|---|---|---|
rspec migration pg12 | 21:24 | 2:23 | 21:33 |
rspec unit pg12 | 21:24 | 2:39 | 20:42 |
rspec integration pg12 | 22:07 | 2:18 | 21:21 |
rspec system pg12 | 16:51 | 2:21 | 16:56 |
rspec-ee migration pg12 | 20:48 | 2:56 | 18:21 |
rspec-ee unit pg12 | 20:48 | 2:45 | 21:21 |
rspec-ee integration pg12 | 18:26 | 2:49 | 17:00 |
rspec-ee system pg12 | 20:51 | 2:50 | 20:32 |
rspec-ee unit pg12 geo | 15:31 | 2:48 | 15:04 |
rspec-ee integration pg12 geo | 6:41 | 2:16 | 14:52 |
rspec-ee system pg12 geo | 9:01 | 2:48 | 8:47 |
Frontend changes
Test Merge Request: !64913 (closed)
- pipeline before approval: https://gitlab.com/gitlab-org/gitlab/-/pipelines/328168028/builds
- pipeline after approval: https://gitlab.com/gitlab-org/gitlab/-/pipelines/335370218/builds
Baseline Merge Request: !64972 (closed)
- baseline pipeline: https://gitlab.com/gitlab-org/gitlab/-/pipelines/328661384/builds
Summary:
- Frontend change is not mapped to any specific rspec files, so it falls back to running all system specs.
Mean job duration
Note: actual job duration may vary depending on number of tests needed for the MR
jobs | baseline pipeline - full tests | before approval pipeline - minimal tests | after approval pipeline - full tests |
---|---|---|---|
rspec system pg12 | 16:51 | 16:03 | 16:41 |
rspec-ee system pg12 | 20:51 | 21:00 | 21:48 |
QA changes
Test Merge Request: !64912 (closed)
-
!64913 (closed)
- pipeline before approval: https://gitlab.com/gitlab-org/gitlab/-/pipelines/328166629/builds
- pipeline after approval: https://gitlab.com/gitlab-org/gitlab/-/pipelines/335371810/builds
Baseline Merge Request: !64972 (closed)
- baseline pipeline: https://gitlab.com/gitlab-org/gitlab/-/pipelines/328661384/builds
Summary:
- No change to pipeline as there are no
rspec
jobs for QA changes.
Edited by Albert