Skip to content

Inline code coverage remarks inside MR diffs

Problem to solve

When a Merge Request is being reviewed comments like "I can't stamp this until you improve code coverage" can be demoralizing. Today it is impossible to tell in the GitLab UI where code is not covered by tests. This leads to a scattershot approach adding tests that may not actually improve the quality of the overall project.

Intended users

  • Sasha (Software Developer) who will be creating and reviewing merge requests and has been asked to hold/improve the code coverage with each change.

Further details

To help developers see what parts of the code can be immediately improved it would be nice to have a visual representation in the diff showing where test coverage is lacking.

This would be really nice but hard to do cross-programming languages.

This could be shown in the Changes / Diff view this way along with Code Quality data.

image

Proposal

Use Cobertura to create a code coverage report on a project. On a pipeline run create utilize this visualize changes to code coverage as shown in the Designs tab.

This can be built on top of #23694

Additional community contributions are welcome to build support for additional coverage formats.

Code Quality data as shown in the original mockup will be incorporated in a follow-up issue gitlab#2526.

Permissions and Security

Documentation

  • Document how to add the new job to the gitlab-ci.yml file
  • Document how the code coverage information will be displayed
  • Document how others can contribute additional coverage support for this feature similar to how the Package team has done this for registry contributions.

Availability & Testing

  • Test for a very large code coverage file to measure performance

What does success look like, and how can we measure that?

This issue will be successful if >= 25% of Projects have included this job in their pipeline within 90 days of launch.

Acceptance Criteria

  • A pipeline that includes the new job will at the same speed as before the job was added
  • A merge request that includes changes to a source file includes the code coverage visualization in the diff view

What is the type of buyer?

This will be available in the Core product as the primary user is an IC.

Links / references

/label feature

Edited by James Heimbuck