Update `code_suggestions_in_web_ide_spec` to enter the prompt more like a user
What does this MR do and why?
Updates the e2e test qa/specs/features/ee/browser_ui/3_create/web_ide/code_suggestions_in_web_ide_spec.rb
to enter the code suggestions prompt more like a user would.
-
Waits for the status icon to show that code suggestions is enabled before entering the prompt
-
Enters the prompt to one key at a time to allow the suggestion to be triggered successfully
-
Waits for 60 seconds for the suggestion to appear. Code generation could take longer than 30 seconds sometimes (not likely, but this should avoid that possible source of flakiness)
-
After this is merged the test should be removed from fast quarantine.
Screenshots or screen recordings
Here's the test running against Staging:
Background
- The test failure triggered an incident: gitlab-com/gl-infra/production#17253 (closed)
- The cause was related to the
code_generation_anthropic
feature flag. It needed to be enabled to allow the WebIDE to return suggestions as code generation aftercode-bison
(via Vertex) was no longer used: #434867 (comment 1689333542). The flag is now enabled forgitlab-qa
. - There was still some flakiness after enabling the feature flag, hence this MR. See #434867 (comment 1689631415)
How to set up and validate locally
To run the test against Staging, the following from the qa
directory:
export GITLAB_USERNAME=gitlab-qa
export GITLAB_PASSWORD=<from 1Password>
export GITLAB_QA_ACCESS_TOKEN=<from 1Password>
export GITLAB_QA_USER_AGENT=<from 1Password>
bundle install
bundle exec bin/qa Test::Instance::All https://staging.gitlab.com qa/specs/features/ee/browser_ui/3_create/web_ide/code_suggestions_in_web_ide_spec.rb
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #434867 (closed)