Skip to content

Change Feature flag vscode project scope to no scope in specs

Jason Zhang requested to merge jz-old-ide-spec-remove-ff-scope into master

What does this MR do and why?

Seems we are failing on staging all of the sudden with the ff not being able to be disabled causing a lot of ide.wait_until_ide_loads issues:

As advised: https://docs.gitlab.com/ee/development/testing_guide/end_to_end/feature_flags.html we tried enable feature flags only project but it seems that stopped working all of the sudden.

We're changing it to: if no scope is specified. The test will only be skipped on canary, production, and pre-production and thus still running on Staging. From the docs: By default you cannot selectively disable a feature flag by actor In other words, a feature flag that is enabled globally cannot be disabled for an individual project or group.

It is extremely confusing how this was working before. We currently still need both sets of WebIDE specs as VScode is in beta.

Update: It seems it's related to dependency_list_exporter being enabled globally as these tests started to fail immediately after that and consistently throughout the entire day. After turning dependency_list_exporter off temporarily, the tests were passing again. Although we can't explain why - we still need this MR in as dependency_list_exporter has nothing to do with vscode.

How to set up and validate locally

Run one of the specs on staging such as

WEBDRIVER_HEADLESS=false GITLAB_USERNAME="<user_name>" GITLAB_PASSWORD="<password>" GITLAB_QA_USER_AGENT="<user_agent>" GITHUB_ACCESS_TOKEN="58f3080c80a27684005072b4c2d3cf19ecaba72d" bundle exec bin/qa Test::Instance::All https://staging.gitlab.com' qa/specs/features/browser_ui/3_create/web_ide/upload_new_file_in_web_ide_spec.rb

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Jason Zhang

Merge request reports