Skip to content

test-selection gap: run system specs when some views changed

Context

We have found two test gaps by looking at the MR pipelines failures:

In the first example, app/views/groups/merge_requests.html.haml was changed, and it broke a system spec: spec/features/groups/merge_requests_spec.rb. Those tests were not run in the selective pipelines, because we don't have a "views -> system specs" mapping.

The second is fairly similar: a view got changed, and running the feature specs would have caught an error.

Goal

When a view gets changed, we should try to run the feature spec that would test that area of the code.

Technical details

We could first try to match the exact feature spec if it exists:

  • app/views/groups/merge_requests.html.haml is changed -> run specs in spec/features/groups/merge_requests_spec.rb.

If it doesn't exist, we could run all specs in the parent folder:

  • app/views/dashboard/_sidebar.html.haml is changed -> run specs in spec/features/dashboard/, since there's no specs called _sidebar_spec.

It would still be faster than having to run ALL feature specs.

There is also a fair overlap of folders between app/views and spec/features folder. Below are the folders that they have in common:

admin
clusters
dashboard
explore
groups
ide
import
issues
jira_connect
profiles
projects
search
snippets
users
Edited by David Dieulivol