Skip to content

Run reliable tests on GDK automatically

Mark Lapierre requested to merge ml-auto-run-reliable-suite-on-gdk into master

What does this MR do and why?

Runs reliable tests automatically (allowed to fail) against GDK in CI in MR and master pipelines.

  • qa/specs/features/api/1_manage/rate_limits_spec.rb doesn't behave as expected so it's excluded from GDK tests until it can be investigated.
  • Smoke test failures with load balancing enabled need investigation. They're now allowed to fail too.

Part of #388533 (closed) and gitlab-org/quality/quality-engineering&4 (closed)

Effect on pipelines

There should be no noticable difference in most cases. No change to the jobs that gdk-qa-reliable is included in (i.e., code/CI/QA changes), but it will now always run instead of being manual. This means MR and master pipelines will have an extra 5 jobs that run in parallel, but they should start at the same time as gdk-qa-smoke and run for about the same duration (give or take a few minutes).

Announcement (to be shared)

To be posted in Slack (#quality, #development, #engineering-fyi) and the engineering WiR doc:

📣 We're now automatically running end-to-end tests in the reliable suite (https://about.gitlab.com/handbook/engineering/quality/quality-engineering/reliable-tests/) against GDK in MR and master pipelines. The jobs are allowed to fail for now. We plan to make them block pipelines, but only after we're confident they do run reliably against GDK in CI.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Mark Lapierre

Merge request reports

Loading