Exit immediately if running minimal reproduction fails
What does this MR do and why?
This MR ensures that scripts/rspec_bisect_flaky
(added in !75457 (merged)) exits immediately if running a minimal reproduction fails.
Previously, we've run specs for all combinations and continued if a combination failed.
This MR also fixes some shellcheck offenses.
Screenshots or screen recordings
Shellcheck offenses
$ shellckeck scripts/rspec_bisect_flaky
In scripts/rspec_bisect_flaky line 13:
files=( $@ )
^-- SC2206: Quote to prevent word splitting/globbing, or split robustly with mapfile or read -a.
In scripts/rspec_bisect_flaky line 25:
bin/rspec $file $target
^---^ SC2086: Double quote to prevent globbing and word splitting.
^-----^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
bin/rspec "$file" "$target"
In scripts/rspec_bisect_flaky line 29:
bin/rspec --bisect=verbose $@
^-- SC2068: Double quote array expansions to avoid re-splitting elements.
For more information:
https://www.shellcheck.net/wiki/SC2068 -- Double quote array expansions to ...
https://www.shellcheck.net/wiki/SC2206 -- Quote to prevent word splitting/g...
https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
How to set up and validate locally
While trying to reproduce failures from #351431 (closed) I ran
$ scripts/rspec_bisect_flaky ee/spec/features/protected_branches_spec.rb ee/spec/features/groups/analytics/cycle_analytics/filters_and_data_spec.rb ee/spec/features/projects/pipelines/pipeline_spec.rb ee/spec/features/admin/subscriptions/admin_views_subscription_spec.rb ee/spec/features/ci_shared_runner_warnings_spec.rb ee/spec/features/issues/epic_in_issue_sidebar_spec.rb ee/spec/features/boards/boards_spec.rb ee/spec/features/admin/admin_dev_ops_reports_spec.rb ee/spec/features/projects/audit_events_spec.rb ee/spec/features/projects/mirror_spec.rb ee/spec/features/namespace_user_cap_reached_alert_spec.rb ee/spec/features/registrations/user_sees_new_onboarding_flow_spec.rb ee/spec/features/groups/seat_usage/seat_usage_spec.rb ee/spec/features/admin/licenses/show_user_count_threshold_spec.rb ee/spec/features/projects/members/invite_group_and_members_spec.rb ee/spec/features/groups/iterations/user_views_iteration_spec.rb ee/spec/features/epics/filtered_search/visual_tokens_spec.rb ee/spec/features/admin/admin_credentials_inventory_spec.rb ee/spec/features/groups/iterations/user_edits_iteration_spec.rb ee/spec/features/merge_request/user_sets_approval_rules_spec.rb ee/spec/features/ide/user_commits_changes_spec.rb ee/spec/features/groups/billing_spec.rb ee/spec/features/issues/issue_actions_spec.rb ee/spec/features/admin/groups/admin_changes_plan_spec.rb ee/spec/features/projects/insights_spec.rb ee/spec/features/projects/feature_flags/feature_flag_issues_spec.rb ee/spec/features/merge_requests/user_views_all_merge_requests_spec.rb ee/spec/features/groups/analytics/ci_cd_analytics_spec.rb ee/spec/features/projects/integrations/user_activates_jira_spec.rb ee/spec/features/registrations/combined_registration_spec.rb ee/spec/features/projects/quality/test_case_create_spec.rb ee/spec/features/admin/admin_interacts_with_push_rules_spec.rb ee/spec/features/groups/security/compliance_dashboards_spec.rb ee/spec/features/groups/iterations/user_edits_iteration_cadence_spec.rb ee/spec/features/dashboards/activity_spec.rb ee/spec/features/projects_spec.rb ee/spec/features/projects/feature_flags/user_deletes_feature_flag_spec.rb ee/spec/features/groups/new_spec.rb
I was surprised to see that the script kept running if after first failures appeared.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Peter Leitzen