Page MenuHomePhabricator

Improve error reporting of CTest scripts
Closed, ResolvedPublic

Description

When using a CTest script (calling ctest -S <script-file>) ctest uses something like

make -i <target>

for $(MAKE). This has several implications:

  1. When calling ctest_build() in our ctest script for the complete superbuild, this $(MAKE) command keeps the compiler going and the return code of 'make' is 0. This leads to all external projects being build, even if their dependencies could not be build successfully.
  1. Due to the return code of '0' of make, the stamp files of external projects report a successful build and a subsequent ctest run will not report the same errors again because it will not try to build the external project again.

Further, ctest launchers are not used in external projects because they are usually build without testing enabled and hence do not include(CTest) which would enable launchers based on the CTEST_USE_LAUNCHERS_DEFAULT environment variable. This was discussed here

http://www.na-mic.org/Bug/view.php?id=2468
https://github.com/jcfr/CMake/commit/7efb978694c98012ef74d43a6aba984ecf7c7074

and a solution for CTK was provided here

https://github.com/commontk/CTK/commit/4f9c61eb205c1ef1b26c775c57cd6e978c5c3457

We can fix / work around this by:

  1. Call external project targets separately in our ctest script and check for reported errors.
  1. Remove the <proj>-configure stamp file manually if the external project could not be build.
  1. Include the CTestUseLaunchers.cmake script in external projects as in the CTK solution.

Event Timeline

[7b2d25]: Merge branch 'bug-18843-improve-ctest-error-reporting'

Merged commits:

2015-03-11 23:38:51 Sascha Zelzer [998d24]
Build external projects separately during ctest super-build.


2015-03-11 23:34:25 Sascha Zelzer [fafe8e]
Make sure launchers are used in external projects.

User zelzer has pushed new remote branch:

bug-18843-improve-ctest-error-reporting

[02e16b]: Merge branch 'bug-18843-improve-ctest-error-reporting'

Merged commits:

2015-03-12 10:46:19 Marco Nolden [b7b63a]
COMP: handle missing argument to check_for_errors