Page MenuHomePhabricator

Different coverage results with OpenCppCoverage and Gcov
Closed, WontfixPublic

Assigned To
None
Authored By
j762e
Apr 3 2020, 1:47 PM
Tags
Referenced Files
F1546954: CDash _ MITK.html
Apr 3 2020, 1:47 PM
F1546955: mitkImageStatisticsCalculator.cpp.html
Apr 3 2020, 1:47 PM

Description

I did a code coverage with OpenCppCoverage and compared my output with http://cdash.mitk.org/viewCoverage.php?buildid=723463&status=2.
In my case I got an coverage of 70% but on CDash - which is based on Gcov I think - it's 82%.

Does anybody know what could be the reason for that?

Event Timeline

j762e triaged this task as Normal priority.Apr 3 2020, 1:47 PM
j762e created this task.

Unfortunately I cannot see any colored lines in the CDash html but it's ok since I'm able to see the output in the provided link.
@nolden can you shed any light on this?

If I look at the outputs side by side there is e.g. a difference in the SetSecondaryMask method: it's called 24 times in the CDash run but in the OpenCppCoverage it's red. Which tests did you execute? I think if I remember correctly the numbers in the CTest/CDash result are aggregated over all tests in the default config. Could this make the difference?

kalali added a project: Restricted Project.Apr 8 2020, 10:27 AM

I executed all tests in files.cmake, which are: mitkImageStatisticsCalculatorTest, mitkPointSetStatisticsCalculatorTest, mitkPointSetDifferenceStatisticsCalculatorTest, mitkImageStatisticsTextureAnalysisTest, mitkImageStatisticsContainerManagerTest and mitkImageStatisticsHotspotTest.
But I think the reason for the different results might be that opencppcoverage does not use any testfiles and cannot load pictures because it cannot find these and there is no option to pass them to opencppcoverage.

If I look at the outputs side by side there is e.g. a difference in the SetSecondaryMask method: it's called 24 times in the CDash run but in the OpenCppCoverage it's red. [...]

That's exactly what we werelooking at. Difference in percentage is because of the different number of lines / functions not hit by the tests. The question was, why are the functions not reached by OpenCppCoverage.

But I think the reason for the different results might be that opencppcoverage does not use any testfiles and cannot load pictures because it cannot find these and there is no option to pass them to opencppcoverage.

Ok, I think this is the reason for the difference and I think this has to be fixed to make the output of OpenCppCoverage meaningful. Many tests just don't do anything useful without test images. You can find the proper command line to start the test if you execute "ctest -V -R <test-name>" in the MITK-build directory, or look it up on CDash and adapt the paths to your local setup. E.g. look here and click "Show command line":

http://cdash.mitk.org/testDetails.php?test=13770846&build=724301

kislinsk added a project: Auto-closed.
kislinsk added a subscriber: kislinsk.

Hi there! 🙂

This task was auto-closed according to our Task Lifecycle Management.
Please follow this link for more information and don't forget that you are encouraged to reasonable re-open tasks to revive them. 🚑

Best wishes,
The MITK devs