Introduce coverband gem for mapping source code paths
What does this MR do and why?
Relates to: gitlab-org/quality/quality-engineering/team-tasks#2393 (closed)
This is the MR to add source code path mappings to E2E specs. The mapping hash(json file) is stored as a job artifact.
How it works?
- Introduces a gem called
coverband
andrequire
is only called whenCOVERBAND_ENABLED
ENV variable istrue
. By default this feature is disabled. - If
COVERBAND_ENABLED
is true, the api is enabled and the test framework records the coverage for each E2E test. - The
coverage
is cleared at the start of every test and at the end of every test(using coverband_formatter in test framework) so that the source code paths touched are confined only to E2E spec actions. - After each suite of tests, the source code path mapping is written to a JSON file as a job artifact.
- All code mapping job artifact JSON files are combined and uploaded to GCS in
export_code_paths_mapping.rb
How to test locally?
Pre-requisites: gdk is up and running, define COVERBAND_ENABLED=true in env.runit of gdk
in test GITLAB_QA_ADMIN_ACCESS_TOKEN
needs to provided(command below)
- Sometimes env.runit alone doesn't pick up ENV variable inside gdk, so
export COVERBAND_ENABLED=true
- Checkout this branch and start up
gdk
- Run an example E2E spec
COVERBAND_ENABLED=true WEBDRIVER_HEADLESS=false GITLAB_ADMIN_PASSWORD="XXX" GITLAB_QA_ADMIN_ACCESS_TOKEN=XXX GITLAB_PASSWORD="XXX" QA_LOG_LEVEL=DEBUG QA_GITLAB_URL=http://172.16.123.1:3000 bundle exec rspec ./qa/specs/features/ee/browser_ui/10_govern/security_reports_spec.rb:233
- Look for
Saved test code paths mapping to tmp/test-code-paths-mapping-*.json
in the test logs (which should show a hash of code paths)
- Pipeline with
COVERBAND_ENABLED=true
: https://gitlab.com/gitlab-org/gitlab/-/pipelines/1318784722 - Pipeline with
COVERBAND_ENABLED=false
: (https://gitlab.com/gitlab-org/gitlab/-/pipelines/1320432139) https://gitlab.com/gitlab-org/gitlab/-/jobs/7031178722
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
Edited by Harsha Muralidhar