Skip to content

Expand test selection for when changing metrics YAML files

David Dieulivol requested to merge ddieulivol-test_selection_for_metrics into master

Context

Follow-up of gitlab-org/quality/engineering-productivity/master-broken-incidents#287 (closed), where some metrics definitions were changed, and a spec that could have detected the problem wasn't triggered because of test selection.

What does this MR do and why?

If we change a ee metric definition in ee/config/metrics/**/*.yml, we'll run the specs in ee/spec/config/metrics/*_spec.rb, as they are testing the metrics data structure.

We are NOT checking the non-ee metrics, as we don't have any specs written for non-ee metrics.

Results

When changing a file in ee/config/metrics/

https://gitlab.com/gitlab-org/gitlab/-/jobs/3530126910:

Changed files: ee/config/metrics/settings/20220218105708_version.yml tests.yml
Related RSpec tests: ee/spec/config/metrics/every_metric_definition_spec.rb
Related FOSS RSpec tests:

When changing a file in ee/lib/ee/gitlab/usage_data_counters/known_events

https://gitlab.com/gitlab-org/gitlab/-/jobs/3530203705:

Changed files: ee/lib/ee/gitlab/usage_data_counters/known_events/analytics.yml tests.yml
Related RSpec tests: ee/spec/config/metrics/every_metric_definition_spec.rb
Related FOSS RSpec tests:

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Rémy Coutable

Merge request reports