E2E: Update reliable spec report creation
What does this MR do and why?
Describe in detail what your merge request does and why.
- Fetch data from multiple run types to make sure test is stable on all environments
- Create one report issue for stable specs and reliable specs with failures
- Post only summary tables to slack and link to report issue
Related issue: https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/994
Screenshots or screen recordings
Example report: #347413 (closed)
Running locally
bundle exec rake reliable_spec_report
Bear minimum, command requires following variables:
-
QA_INFLUXDB_URL
- influxdb instance url, https://influxdb.quality.gitlab.net/ -
QA_INFLUXDB_TOKEN
- influxdb read token, can be found in QA vault
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Andrejs Cunskis