diff --git a/Documentation/Doxygen/3-DeveloperManual/Concepts/TestsGeneral.dox b/Documentation/Doxygen/3-DeveloperManual/Concepts/TestsGeneral.dox index 9fa7e1d371..e1481463fb 100644 --- a/Documentation/Doxygen/3-DeveloperManual/Concepts/TestsGeneral.dox +++ b/Documentation/Doxygen/3-DeveloperManual/Concepts/TestsGeneral.dox @@ -1,104 +1,104 @@ /** \page GeneralTests General: Tests in MITK \tableofcontents \section GeneralTestsIntroduction Testing in MITK The basic idea about unit testing is to write a test for every unit (e.g. function) of your software, which automatically checks if the code still returns the expected result for a defined input. The goal is to isolate units as much as possible, in order to be efficient with fixing errors. If this is performed on a low level, this is called unit testing. On higher levels, where various circumstances (such as configuration files, internet connection, hardware, or even input/output) may influence the outcome of the test, this is called integration testing. MITK uses CppUnit, which is a framework for efficient and fast test writing, to support developers in test coding. Furthermore MITK offers currently three types of tests: classic unit tests, rendering tests and interaction tests. Although our unit tests often involve data input (or output), we still consider them to be unit tests. \warning Most of our units tests have been written before adopting CppUnit and have not yet been updated. Therefore we do not recommend to look at a random test for an example of coding a test. This pages gives a general introduction to our test framework. Rendering tests are explained in more detail in the section \ref RenderingTests and interaction tests the section \ref InteractionTestingHowTo. The following sections will explain how to write your own tests with MITK and how to run them. \section GeneralTestsAdding Adding a test to your module Generally, functional code you want to test using unit tests will be located in a module. For an overview of the directory structure see \ref NewModulePageCreateFolder and how to add the files comprising your test compare \ref NewModulePageCreateTestingEnvironment \subsection GeneralTestsAddingFramework Test and test class/suite CppUnit and MITK distinguish between the term test class and test, which is important to understand. A test is simply a call of a single function and a comparison against a reference. The ideal way to write a test is: \code result = myGreatFunction(myParameter); ASSERT_EQUAL(result, reference); \endcode which is just 2 lines of code at the best. Sometimes objects and data may be initialized in a test, but usually this should be done in the setUp() method (see below). If possible, avoid multiple assertions in order to be efficient in error localization and for better readability. For instance, if you instantiate an object and test if its data is not NULL, this is already a complete test and you do not want to repeat this comparison in other tests. All tests check different things and just one at a time. Remember: The goal is to have isolated unit tests and avoid large, confusing tests which are hard to maintain. \note The reference should never be computed and ideally be a distinct and well defined (often hard coded) value. You don't care where you reference comes from, you just want to ensure that your test actually results in the reference value. A test class (also called suite) is a set of tests and often associated to a single functional class in MITK. A test class refers to a TestSuite in CppUnit. These terms are easily confused, so we suggest to use the term test suite for a collection of tests (which is often in a single file), and the term "test" for a single test (usually many per file). In order to create a test class you need to create a new class deriving from mitk::TestFixture. While a purist approach would be to create a new mitk::TestFixture for each method of your class (resulting in as many test source files as your class has methods), we usually follow a more pragmatic approach within MITK. In most cases we suggest having one test source file per class. If your class source file is called mitkGreatClass.cpp we suggest the name mitkGreatClassTest.cpp for your test source file. For very complex and long classes splitting this into several tests may be advisable. Additionally, rendering tests and interaction tests should always get their own test class/suite to mark them as obvious integration tests, because the goal is to isolate unit tests from integration tests. The dashboard for continuous integration will automatically show the results of a test suite and summarize the output. If you run your test manually (e.g. with ctest -VV), you will see the detailed output of all tests of the suite. In order to use CppUnit via MITK you will need to include the corresponding files and register your test: \code #include #include … MITK_TEST_SUITE_REGISTRATION(mitkImageEqual) \endcode \subsection GeneralTestsAddingHow How to structure your test As mentioned before, all test suites derive from mitk::TestFixture. A suggested name for your test suite would be TestSuite. You then create a suite and register your tests. A suggested naming convention for test methods is __. We suggest not to add the word test, because that is most likely the only thing the reader knows anyway about your test, as he is reading a test suite. An example: \code class mitkImageEqualTestSuite : public mitk::TestFixture { CPPUNIT_TEST_SUITE(mitkImageEqualTestSuite); MITK_TEST(Equal_CloneAndOriginal_ReturnsTrue); MITK_TEST(Equal_InputIsNull_ReturnsFalse); MITK_TEST(Equal_DifferentImageGeometry_ReturnsFalse); MITK_TEST(Equal_DifferentPixelTypes_ReturnsFalse); … CPPUNIT_TEST_SUITE_END(); … } \endcode You also may provide a setUp() and a tearDown() function. These will be called before/after each test and should be used to make sure that each test works independently and on freshly initialized members and memory to maximize isolation. That way you avoid only testing whether a function works after another function has already been called. See CppUnit documentation for more information. For an example test suite including tests see \ref GeneralTestsExample \section GeneralTestsRunning Running your test suite The build system of MITK generates a test driver which includes all test suites that have been added to the project. Alternatively you can run MITK test suites by using the program ctest. This is the way all MITK tests run on the continuous dart clients of MITK. The results of these runs can be found at http://cdash.mitk.org. If you use the test driver, you only need to start the executable. If you start it without parameters, it will then give you an overview of all tests which are included in this test driver and you can choose one by typing a number. Alternatively, you can give your test driver the name of your test suite as parameter. If you want to use ctest instead of the test driver you need to start a command line, go to the binary directory of MITK and call ctest. To avoid errors, check if your path variable contains all relevant paths to start MITK. \section GeneralTestsParameterInput Adding parameters to your test -If possible, the setUp() method of the test suite should provide all necessary inputs for the respective tests. MITK provides several helper classes to generate synthetic test data, such as the mitkImageGenerator. If you have to load data from the hard disc for your test, you can use the method GetTestDataFilePath(string fileName). For an example of loading data from the MITK_DATA_DIR check the mitkIOUtilTestSuite. +If possible, the setUp() method of the test suite should provide all necessary inputs for the respective tests. MITK provides several helper classes to generate synthetic test data, such as the mitk::ImageGenerator. If you have to load data from the hard disc for your test, you can use the method GetTestDataFilePath(string fileName). For an example of loading data from the MITK_DATA_DIR check the mitkIOUtilTestSuite. \section GeneralTestsPredefinedAssertions Predefined assertions MITK and CppUnit offer predefined assertions, i.e. helper methods which will help to compare your data against a certain reference. All basic types are covered by CppUnit assertions, such as CPPUNIT_ASSERT. For examples, please check the CppUnit documentation. MITK further offers comparison tools for floating point numbers, vectors, images, surfaces and point sets. A complete list of assertion macros is given in \ref MITKTestingAPI. An example to compare images: \code MITK_ASSERT_EQUAL(image, reference, "Checks if image is equal to a reference"); \endcode By default, the method uses an mitk::eps for floating point comparison, but this can be adapted. It can be necessary to write your own assertion for your own data meeting your special requirements. Recommended examples are all equal test suites for basic data types (mitkImageEqualTest, mitkSurfaceEqualTest and mitkPointSetEqualTest). \section GeneralTestsExample An example \include Examples/FirstSteps/NewModule/test/mitkExampleDataStructureTest.cpp \section GeneralTestsFurtherInfo Further information More examples can be found in the corresponding bugsquashing presentation. */ \section GeneralTestsDeprecatedOldTestingStyle Deprecated macros All tests with MITK_TEST_BEGIN(); and MITK_TEST_END(); use the deprecated old MITK testing style. If possible, they should be replaced with the new CppUnit style as explained before. Most of these test classes contain very unfortunate examples and should not be regarded as reference.