Improve incomplete / wrong information for gathering of JaCoCo test coverage percentage
Problem to solve
This concerns the code coverage feature for java projects using the JaCoCo parser. Unless i have missed a third doc page or something else, the way they currently describe the process simply cannot work following the examples they provide (at least for maven).
I followed the relevant documents to set everything up, which are
The issue:
The example given for maven in 1. won't print the test coverage percentage to the log file, so it will never be picked up when using the RegEx example for JaCoCo given in 2.
One can make it work by simply adding a cat target/site/jacoco/index.html
to the end of the test stage, however that will (a) print lots of HTML garbage to the job's log (making it larger than necessary as well as harder to read and debug) and it will (b) only give an integer percentage, which for most projects i too vague and also doesn't match the expected result displayed in the feature screenshot on 2.
Proposal
- Improve the maven example
gitlab-ci.yml
by adding- awk -F, '{ instructions += $4 + $5; covered += $5 } END { print covered, "/", instructions, " instructions covered"; print "Total", 100*covered/instructions "%" }' target/site/jacoco/jacoco.csv
This will make some coverage statistics appear in the job's log such as:36569 / 126018 instructions covered Total 29.0189%
😉 - change the example RegEx for Java/JaCaCo projects on page 2. to
Total.*?([\.0-9]*)%
(using the current one would show 189% code coverage in the example above😆 ) - Add some text, linking from page 1. to page 2. and vice-versa.
Currently users need to be lucky (or perseverant) enough discover them individually, although they seem closely related, and, as far as i currently understand, configuring thegitlab-ci.yml
as described in 1. is a must-have in order to be able to enable the feature outlined in 2. Still, they are distinct features (1. = display coverage line by line, 2. = display total coverage in badges, merge requests and repository graph) - Also link from page 2. to this one: https://docs.gitlab.com/ee/ci/yaml/#coverage
Clarify if setting the regex ingitlab-ci.yml
takes precendence over setting it in the frontend.
With those in place we'd have a much more pleasant experience for users trying to enable this often wanted feature in gitlab (took me roughly three weeks until the penny dropped for me).