Merge Request test report widget shows failing tests when the MR pipeline is green
Encountered this in !90347 (merged).
Some quick investigations:
- The pipeline did not have any failing jobs: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/pipelines/565639209/jobs?scope[]=failed
- The MR pipelines did not have any failure on
rspec unit pg12
, so it seems to eliminate caching issues or report being fetched from an older pipeline.
Other potential problems:
- Could it be that
rspec-retry
retried flaky tests which eventually passed, but the failures are still recorded in the Junit report and it is picked up by the parser?
Other observations:
- https://gitlab.com/gitlab-org/gitlab/-/pipelines/565639209/test_report looking at this test report and the failures, they show the same test failures coming from different test files, which is strange.
Edited by Albert