Page MenuHomePhabricator
Create Task
Maniphest T27952

Reduce LabelSetTest and ImageGeneratorTest execution times
Closed, ResolvedPublic
Actions

Description

For example, see nightlies here: https://cdash.mitk.org/index.php?project=MITK&date=2020-11-14

Revisions and Commits

rMITK MITK
Restricted Differential RevisionrMITKd76d2dc174b0 Reduce number of labels in test label set
Restricted Differential RevisionrMITKb72e8c02f3a3 Reduce test image dimensions
Restricted Differential RevisionrMITK28f981c7cd55 Fix comparison between signed and unsigned integers
Restricted Differential RevisionrMITK1198ee0afe28 Reduce execution times of longest tests

Related Objects

Event Timeline

kislinsk triaged this task as High priority.Nov 15 2020, 11:09 AM
kislinsk created this task.
kislinsk updated the task description. (Show Details)Nov 16 2020, 8:09 AM
floca added a subscriber: floca.Nov 16 2020, 4:33 PM

Could it be that the test time out time is 600 sec? Because in the last month all tests runs of LabelSetTest with worked (as long as the duration was below 600 sec). All failed tests took longer then 600 sec.
For what ever reason under win the test is relativaly fast ~150-200 sec. Under linux/mac it is lurking arround 590 sec +- 25...

kislinsk added a comment.Nov 16 2020, 7:39 PM

Yes, eventually the test times out. But even if it would "fix" the test to increase this value, individual unit tests should aim to last not longer than around a second, at maximum a few seconds.

kislinsk added a comment.Nov 17 2020, 2:11 PM

1.5 reasons:

  • mitk::LabelSet::AddLabel() takes a pretty long time to complete while it fiddles around with an std::map
  • Not only does every test unnecessarily initialize a label set with 200 labels, but one test adds another 65535 labels to test the maximum allowed number of labels
kislinsk added a comment.Nov 17 2020, 2:46 PM

Pushed new branch bugfix/T27952-FixLabelSetTest.

kislinsk added a revision: Restricted Differential Revision.Nov 17 2020, 2:47 PM
kislinsk renamed this task from Investigate LabelSetTest to Reduce LabelSetTest execution time.Nov 17 2020, 10:57 PM
kislinsk renamed this task from Reduce LabelSetTest execution time to Reduce LabelSetTest and ImageGeneratorTest execution times.Nov 17 2020, 11:18 PM
kislinsk updated the task description. (Show Details)
kislinsk closed this task as Resolved by committing rMITK1198ee0afe28: Reduce execution times of longest tests.Nov 18 2020, 8:31 PM
kislinsk claimed this task.
kislinsk added a commit: rMITK1198ee0afe28: Reduce execution times of longest tests.
kislinsk added a commit: rMITKb72e8c02f3a3: Reduce test image dimensions.
kislinsk added a commit: rMITK28f981c7cd55: Fix comparison between signed and unsigned integers.
kislinsk added a commit: rMITKd76d2dc174b0: Reduce number of labels in test label set.
kislinsk added a comment.Nov 18 2020, 8:32 PM

Deleted branch bugfix/T27952-FixLabelSetTest.

kislinsk mentioned this in T27977: 2020 Week 47 (Mid November).Nov 19 2020, 2:50 PM
MITK · MIC · IMSY · DKFZ