Skip to content

Add experiment for SimpleCov branch coverage

Markus Koller requested to merge simplecov-branch-coverage-experiment into master

What does this MR do and why?

SimpleCov can also detect coverage for code branches in conditionals: https://github.com/simplecov-ruby/simplecov#branch-coverage-ruby--25

We're not sure yet about the impact to pipeline runtimes, so this is only enabled when the $SIMPLECOV_BRANCH_COVERAGE environment variable is set.

Note that this only adds additional branch coverage information to the report, the overall coverage is still based on lines.

Crystalball needs to call Coverage.start with the same arguments as SimpleCov, so we tweak its setup as well.

Issue: #354097

How to set up and validate locally

  1. Run with branch coverage enabled: 
    CRYSTALBALL=true SIMPLECOV=true SIMPLECOV_BRANCH_COVERAGE=true bundle exec rspec spec/models/project_daily_statistic_spec.rb
  2. Run with branch coverage disabled: 
    CRYSTALBALL=true SIMPLECOV=true bundle exec rspec spec/models/project_daily_statistic_spec.rb

Both of these should finish without errors, and generate some data in coverage/ and crystalball/ (which can take a few minutes).

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Markus Koller

Merge request reports