Page MenuHomePhabricator

[Dashboard] mitkNavigationToolStorageSerializerAndDeserializerTest fails frequently
Closed, ResolvedPublic

Description

The test mitkNavigationToolStorageSerializerAndDeserializerTest frequently fails on the MITK dashboard. Example failure: http://cdash.mitk.org/testDetails.php?test=3385341&build=158771

At first, please specify what this test exactly does and double-check if it is acting according to documentation. In other words: Read the documentation and compare if the code really does what its supposed to do.

If there is no documentation, please provide some or find someone who can provide documentation.

Very often one of the following is involved when a test keeps failing randomly:
-Threading
-Rendering tests
-Writing/Reading of data from hard disks
-Special hardware/software of the corresponding dart client

One should double-check, if any of these things apply for this particular test and try to fix it. Do not hesitate to ask for help, if you are not familiar with these mentioned issues.

If there is no easy fix or the reason cannot be found, all findings should be documented here, and the author (or any module responsible person) of the test (or the class which is tested) should contact me.

The priority of this bug is high, because we would like to achieve a green dashboard and high test coverage.

JUST REMOVING THE TEST FROM CMAKE IS NOT WELCOME.
(This is a standard message, copied for all tests with similar behavior)

Event Timeline

All bugs marked with [Dashboard] make the dashboard red are highly important.

Maybe it would help to catch all exceptions in this test and give out the exception message to the console.

At the moment the test only fails randomly on the dart clients which the message "std::exception unknown exception occurred", which is not meaningful.

Surrounded whole test with try catch(std::exception) block. We hope to get a clearer exception message the next time it occurs.

[1ae115]: Merge branch 'bug-12730-CatchallForNavigationToolStorageSerializerAndD

Merged commits:

2012-08-15 14:38:59 Christoph Kolb [707c0a]
catch exception in test

Did the test fail after the last merge? Christoph, do you think your last merge may have fixed the bug? If yes maybe we can close it.

[59cff6]: Merge branch 'bug-12730-CatchallForNavigationToolStorageSerializerAndD

Merged commits:

2012-10-24 16:46:26 Alfred Franz [c09e5f]
tests fails now if an exception is catched in the test

Test only fails on the 32 Bit installer machine, but fails there regularily.
Here's an example:
http://cdash.mitk.org/testDetails.php?test=4125454&build=210098

Since the dart client there produces other bugs as well, I consider this bug closed.

The std::exception: "Access to file denied" is thrown in
NavigationToolStorageSerializerAndDeserializerTestClass::TestSerializerForExceptions();

while trying to serialize a storage to an invalid filename.
the thrown exception is supposed to be an IGTException.

New remote branch pushed: bug-12730-mitkNavigationToolStorageSerializerAndDeserializerTestAddExceptionHandling

added another try/catch block around
filename = mitk::StandardFileLocations::GetInstance()->GetOptionDirectory()+Poco::Path::separator()+".."+Poco::Path::separator()+"TestStorage.storage";

this is assumed to throw a std::exception on 32bit systems, for unknown reasons

[5d759d]: Merge branch 'bug-12730-mitkNavigationToolStorageSerializerAndDeserial

Merged commits:

2012-11-21 13:45:00 Christoph Kolb [4c317a]
more output for debugging


2012-11-21 13:39:42 Christoph Kolb [c07116]
additional exceptionhandling for TestWriteEmptyToolStorage

Maybe the problem is that the name of the temporary directory is not real unique and thus alredy exists. This fail might be an example for this: http://mbits/cdash/testDetails.php?test=3080112&build=155791

New remote branch pushed: bug-12730-DeactivateMitkNavigationToolStorageSerializerAndDeserializerTest

New remote branch pushed: bug-12730-mitkNavigationToolStorageSerializerAndDeserializerTestUseUIDGeneratorForFilenames

Bug is hopefully fixed, only merge the last branch (bug-12730-mitkNavigationToolStorageSerializerAndDeserializerTestUseUIDGeneratorForFilenames) to the master.

Attention, the fix of this branch depends on the fix of T13980!

Updating target milestone to upcoming release

[c71952]: Merge branch 'bug-12730-RemoveTest'

Merged commits:

2012-12-28 12:47:07 Thomas Kilgus [eeea34]
COMP: Removed test until the bug is fixed.

New remote branch pushed: bug-12730-IntegrationBranch2013.Jan.16

New remote branch pushed: bug-12730-IntegrationBranchJan2013

[8bb6ae]: Merge branch 'bug-12730-IntegrationBranchJan2013'

Merged commits:

2013-01-16 16:13:24 Alfred Franz [05a5d3]
activated mitkNavigationToolStorageSerializerAndDeserializerTest again, should work now


2013-01-16 16:11:33 Alfred Franz [eff1eb]
Merge branch 'bug-12730-mitkNavigationToolStorageSerializerAndDeserializerTestUseUIDGeneratorForFilenames' into bug-12730-IntegrationBranch2013.Jan.16


2012-12-12 15:49:13 Christoph Kolb [0300f3]
remove messages and TempDirectoryID


2012-12-12 15:40:40 Christoph Kolb [50bc8a]
replace random number generator with mitk::UIDGenerator

the test failed again with the same error message:

0.61 core.mod.igt.navTlsTorageserializeranddeserializertest ERROR: exception:Access to file denied
Exception occured, test failed!

New remote branch pushed: bug-12730-NavigationToolStorageSerializerAndDeserializerStillRandomFail

[b6990c]: Merge branch 'bug-12730-NavigationToolStorageSerializerAndDeserializer

Merged commits:

2013-02-13 17:17:06 Alfred Franz [dbb8e0]
added additional information on test errors to maybe find the reason for the last random fails

We currently try to find the bug by extending the test of the module SceneSerialization (T14534) because in fact the test always fails in this module, but the tests of this module do not show this bug.

Another idea is to deactivate multi threading of this test. Maybe this would help to prevent multiple access to the same file.

What is the status of this bug? Please change the target milestone to "AfterNextRelease" if this bug is not relevant or cannot be fixed in time for the 2013-06 release.

As far as I am concerned, this test still fails (but not frequently).

@Alfred: Regarding your last suggestion: Is it very important for you that this class works thread safe? Does the crash also occur during regular usage? If it is just a testing issue, I would not spend more time to fix this and rather remove this very advanced test. I assume you test also the important base features without threading, right?

This bug could not be fixed for release 2013-06. Setting target milestone to next release

[a54175]: Merge branch 'bug-12730-TempDisableTest'

Merged commits:

2013-08-11 21:42:27 Stefan Kislinskiy [818fe4]
COMP: Temporarily disabled mitkNavigationToolStorageSerializerAndDeserializerTest since it fails every now and then.

Please fix this test or the cause of test failure before enabling it again. It's annoying to make dummy COMP commits in order to make the dashboard green again.

error message from dashboards: 0.67 core.mod.igt.navTlsTorageserializeranddeserializertest ERROR: exception:Access to file denied

There was another try to fix this bug in branch bug-14534-CorrectTestOfModuleSceneSerialization which is already merged into the master.

The test is activated again, lets wait a couple of days to see if the random fail is actually fixed...

Test failed again on a continuous:
http://cdash.mitk.org/testDetails.php?test=6604780&build=363912

I assume the change (by Adrian) was not related to the failure.

Test failed again on the windows continuous. Please remove the test if possible and redesign it:

http://mbits/cdash/viewTest.php?onlyfailed&buildid=226922

This test is currently blocking our internal dashboard. I would like to remove it until it is in a better shape.

My suggestions is:
Split the test up into two parts:
(1) Everything which is strictly a unit test can run like always
(2) Everything regarding threading, I/O, hard disk connection etc. should be a test which only runs nightly. There is already a CMake flag for this purpose.

The last 2 fails of this test occurred during access to a zip file from the MITK-Data repository.

Line 290 of the mitkNavigationToolStorageSerializerAndDeserializerTest

I don't think the last fails are related to the fails before, so I opened a new bug: T16566

I don't think redesign of this test is necessary unless there are still random fails due to the original problem (I/O access to harddisc).

If there are further random fails please make a comment here, so we can have a look on them.

Added a new bug (T16832) to convert this test to cpp unit. This might help to avoid further random fails.

New remote branch pushed: bug-12730-fileAccessProblems

The last branch bug-12730-fileAccessProblems was moved to T16932 because it also includes some changes to mitk::IOUtil.

Test was converted to cpp unit and splitted up into three different tests in T16832. So closing this bug. If there are any random fails of the new tests please open a new bug for the specific test.