Ensure we advertise screenshots taken when a QA failure happen outside of an RSpec example
What does this MR do?
This happens when the failure happens outside of an RSpec example, in that case we do save the screenshot manually in Session#perform
with screenshot_and_save_page
, we just don't advertise the screenshots. Also, by setting Capybara::Screenshot.autosave_on_failure = false
after we take the screenshot, we can ensure that RSpec won't try to take a screenshot afterward (since we already did manually) and show us the WARN: Screenshot could not be saved.
page.current_path is empty.
message.
Are there points in the code the reviewer needs to double check?
RSpec::Core::ExampleGroup
With an exception while a session is active but outside of an › CHROME_HEADLESS=0 bin/qa Test::Instance http://localhost:3000 -- qa/specs/features/api/users_spec.rb:10
Run options: include {:core=>true, :locations=>{"./qa/specs/features/api/users_spec.rb"=>[10]}}
Randomized with seed 27622
API users
when authenticated
Image screenshot: file:///Users/remy/Code/GitLab/gdk/gitlab/qa/tmp/screenshot_2018-06-13-09-23-30.747.png
HTML screenshot: file:///Users/remy/Code/GitLab/gdk/gitlab/qa/tmp/screenshot_2018-06-13-09-23-30.747.html
get list of users (FAILED - 1)
Failures:
1) API users when authenticated get list of users
Failure/Error: let(:request) { Runtime::API::Request.new(@api_client, '/users') }
RuntimeError:
Exception while a session is active but outside of an RSpec::Core::ExampleGroup
# ./qa/runtime/api/client.rb:33:in `block in create_personal_access_token'
# ./qa/runtime/browser.rb:105:in `perform'
# ./qa/runtime/browser.rb:27:in `visit'
# ./qa/runtime/browser.rb:31:in `visit'
# ./qa/runtime/api/client.rb:31:in `create_personal_access_token'
# ./qa/runtime/api/client.rb:24:in `get_personal_access_token'
# ./qa/runtime/api/client.rb:15:in `personal_access_token'
# ./qa/runtime/api/request.rb:8:in `initialize'
# ./qa/specs/features/api/users_spec.rb:8:in `new'
The manually taken screenshots are advertised.
RSpec::Core::ExampleGroup
With an exception while no session is active inside of an › CHROME_HEADLESS=0 bin/qa Test::Instance http://localhost:3000 -- qa/specs/features/api/users_spec.rb:10
Run options: include {:core=>true, :locations=>{"./qa/specs/features/api/users_spec.rb"=>[10]}}
Randomized with seed 46755
API users
when authenticated
WARN: Screenshot could not be saved. `page.current_path` is empty.
get list of users (FAILED - 1)
Failures:
1) API users when authenticated get list of users
Failure/Error: raise 'Exception while no session is active inside of an RSpec::Core::ExampleGroup'
RuntimeError:
Exception while no session is active inside of an RSpec::Core::ExampleGroup
# ./qa/specs/features/api/users_spec.rb:12:in `block (3 levels) in <module:QA>'
In this case the warning is valid since there's no active session.
RSpec::Core::ExampleGroup
With an exception while a session is active inside of an › CHROME_HEADLESS=0 bin/qa Test::Instance http://localhost:3000 -- qa/specs/features/login/standard_spec.rb
Run options: include {:core=>true}
Randomized with seed 4702
standard user login
user logs in using credentials (FAILED - 1)
HTML screenshot: file:///Users/remy/Code/GitLab/gdk/gitlab/qa/tmp/screenshot_2018-06-13-09-28-55.599.html
Image screenshot: file:///Users/remy/Code/GitLab/gdk/gitlab/qa/tmp/screenshot_2018-06-13-09-28-55.599.png
Failures:
1) standard user login user logs in using credentials
Failure/Error: raise 'Exception while a session is active inside of an RSpec::Core::ExampleGroup'
RuntimeError:
Exception while a session is active inside of an RSpec::Core::ExampleGroup
# ./qa/specs/features/login/standard_spec.rb:6:in `block (2 levels) in <module:QA>'
The automatically (by capybara-screenshot/rspec
) taken screenshots are advertised as before.
Does this MR meet the acceptance criteria?
- Conform by the code review guidelines
-
Has been reviewed by a Backend maintainer
-
-
Conform by the merge request performance guides -
Conform by the style guides -
If you have multiple commits, please combine them into a few logically organized commits by squashing them -
End-to-end tests pass ( package-and-qa
manual pipeline job)
What are the relevant issue numbers?
Fixes gitlab-qa#197 (closed).
Edited by Rémy Coutable