Skip to content

Fix flaky test for RefSelector footer

Brian Williams requested to merge bwill/fix-fragile-tags-test into master

What does this MR do and why?

Describe in detail what your merge request does and why.

This fixes the root cause of gitlab-org/quality/engineering-productivity/master-broken-incidents#1024 (closed). The test compares fixture data against a snapshot. The fixture data could change if someone pushes new tags or new branches to https://gitlab.com/gitlab-org/gitlab-test. Branch fixture data is scoped to the search term ma, so it is slightly less fragile, but pushing new branches to gitlab-test which match this search term could have also caused a broken master incident.

Instead of using a snapshot, these tests now generate assertions from the fixture data. This means that the expectations will change as the fixture data changes. Ideally, we should have versioning to prevent the fixture data from changing independently, but there is currently not a system in place for this, and it would be difficult to implement since we need to test all aspects of git repositories.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Brian Williams

Merge request reports