Reduce the number of backend test predictions
From Draft to Ready
-
gitlab-org/ruby/gems/test_file_finder!52 (merged) was merged and released -
!148212 (merged) is merged
Context
Closes #450374 (closed).
Implements the feature introduced in gitlab-org/ruby/gems/test_file_finder!46 (merged).
What does this MR do and why?
Reduce the number of backend test predictions for the topmost 5% of mappings only.
This is not as important for the DescribedClass
Crystalball strategy as it would have been for the Coverage
Crystalball strategy, but I think it's still useful to have it working for the top 5% cases, and in to have this mechanism available in general if we want to use other crystalball strategies, or the same strategies with different thresholds for making pipelines tiers
Proof of work
Change a file that will trigger 36 backend test files
- Expected: We will reduce the number of backend test files to run by 50%.
-
Actual:
✅
$ cat rspec/changed_files.txt
app/models/concerns/spammable.rb
$ cat crystalball/mapping.json | jq '."app/models/concerns/spammable.rb" | length'
36
# Clean the previous results
rm -rf rspec/matching_tests*
# Run https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/predictive-test-selection.md#generate-test-predictions-like-detect-tests-would locally
$ cat rspec/matching_tests.txt | wc -w
18
Change a file that will trigger 10 backend test files
- Expected: We will keep the number of backend test files to 10.
-
Actual:
✅
$ cat rspec/changed_files.txt
app/finders/issuable_finder.rb
$ cat crystalball/mapping.json | jq '."app/finders/issuable_finder.rb" | length'
10
# Clean the previous results
rm -rf rspec/matching_tests*
# Run https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/predictive-test-selection.md#generate-test-predictions-like-detect-tests-would locally
$ cat rspec/matching_tests.txt | wc -w
10
Edited by David Dieulivol