Subprojects without tests lead to CTest returning 255 instead of 0, resulting in wringly declared failed builds in CI.
Description
Related Objects
Event Timeline
Reverted the removal as CTest still fails, probably because of no tests in superbuild projects. I think the best way to solve this is to contribute to CMake.
In CMake, Source/CTest/cmCTestTestHandler.cxx, cmCTestTestHandler::ProcessHandler(), there is a check if the sum of passed and failed tests equal zero. After another pre-check (see below), "No tests were found!!!" is logged as ERROR_MESSAGE. In Source/cmCTest.cxx, cmCTest::Log(), cmSystemTools::SetErrorOccured() is called if the log type is ERROR_MESSAGE.
The other pre-check I wrote above is "if not cmCTest::GetShowOnly() and not cmCTest::ShouldPrintLabels()". The former condition is set to true if -N/--show-only[=...] is passed as argument to CTest. The latter condition is set to true if --print-labels is passed as argument to CTest.
-N/--show-only[=...] disables the actual execution of tests.
--print-labels prints all available test labels and does not run any tests.
The only option I see is to contribute to CMake. To stay backward-compatible, we probably cannot simply change the log type from ERROR_MESSAGE to WARNING_MESSAGE or similar. I think I will propose a new argument for CTest like --no-tests=<[error|ignore]>.
Here's my CMake merge request: https://gitlab.kitware.com/cmake/cmake/merge_requests/4258
Just for documentation purposes, this is how I ran my tests:
CMake-build/Tests$ RunCMake_TEST_FILTER="^ignore-no-tests" ctest -V -C Release -R '^RunCMake\.CTestCommandLine'
CMake 3.17 is out now. I configured the MITK Nightlies to use CMake 3.17 from tonight on and added the new --no-tests=ignore argument to the call of CTest.
It worked. The nightlies are sunny now and the documentation was uploaded to docs.mitk.org. I changed every MITK and eMITK job to use CMake 3.17 now and passed --no-tests=ignore to CTest.