Skip to content

Update insights.yml with flaky-test::state-leak

Jennifer Li requested to merge jennli-add-state-leak-label into main

Proposal

I'd like to add ~"flaky-test::state-leak" as a new flaky test label, to account for flaky tests such as

Previously for these failures, we have been categorizing them as flaky-testdataset-specific , but I feel that these particular errors, which are caused by test data leaking state, have been noticeably rising, and we may benefit from a more descriptive label to highlight the fact that leaked state is the cause here.

Concern

While proposing this idea I tried editing the docs for https://docs.gitlab.com/ee/development/testing_guide/flaky_tests.html#dataset-specific, but I actually don't know what I should update it to if I separate the state leak factor from it. It does feel like flaky-testdataset-specific is quite similar to ~"flaky-test::state-leak", Perhaps these 2 labels share so many traits that we could just rename the label, instead of creating a new one?

Perhaps what we have today is already perfect in which case you can just reject my proposal

Let me know what you think @gl-quality/eng-prod. Thank you 🙇🏻

Edited by Jennifer Li

Merge request reports