Skip to content

Fix compliance framework E2E test retry failures

Mark Lapierre requested to merge ml-fix-compliance-framework-spec-retry-failures into master

What does this MR do and why?

The test would fail on retry if the first attempt left a default framework behind. For example:

https://gitlab.com/gitlab-org/gitlab/-/jobs/4388928543#L1082

1st Try error in ./qa/specs/features/ee/browser_ui/10_govern/group/compliance_framework_report_spec.rb:62:
Fabrication of QA::EE::Resource::ComplianceFramework using the API failed (200) with `{"errors":[{"message":"Timeout on validation of query","locations":[],"extensions":{"code":"validationTimeout"}}]}`.
Correlation Id: 01H1TB2RAZPZR14939P9WP80NN
      
RSpec::Retry: 2nd try ./qa/specs/features/ee/browser_ui/10_govern/group/compliance_framework_report_spec.rb:62
...
  1) Govern Compliance Framework Report shows the compliance framework for each project
     Failure/Error: expect(project).not_to have_framework
       expected QA::EE::Page::Group::Compliance::Show not to have framework
     # ./qa/specs/features/ee/browser_ui/10_govern/group/compliance_framework_report_spec.rb:86:in `block (6 levels) in <module:QA>'
     # ./qa/ee/page/group/compliance/show.rb:81:in `block in project_row'
     # ./qa/page/base.rb:374:in `within_element'
     # ./qa/support/page/logging.rb:156:in `within_element'
     # ./qa/ee/page/group/compliance/show.rb:80:in `project_row'
     # ./qa/specs/features/ee/browser_ui/10_govern/group/compliance_framework_report_spec.rb:83:in `block (5 levels) in <module:QA>'

This removes any frameworks left behind in the sandbox group after the test finishes.

How to set up and validate locally

From the qa directory:

bundle install
export WEBDRIVER_HEADLESS=false # If you'd like to watch the test in action
export QA_GITLAB_URL="http://gdk.test:3000" # Only needed if GDK is not running on http://127.0.0.1:3000
bundle exec rspec qa/specs/features/ee/browser_ui/10_govern/group/compliance_framework_report_spec.rb

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Mark Lapierre

Merge request reports