Page MenuHomePhabricator
Create Task
Maniphest T25697

Optimization of MitkImageStatisticsCalculatorTest
Closed, ResolvedPublic
Actions

Description

~6 Minutes and 1800 lines for a test is too long.

  • use smaller test images
  • test data
    • 2D?
    • 3D
    • 4D
  • API of ImageStatisticsCalculator
    • void SetInputImage(mitk::Image::Pointer image)
      • 2D?
      • 3D
      • 4D
      • LabelSetImage?
  • void SetMask(mitk::MaskGenerator::Pointer mask)
  • void SetSecondaryMask(mitk::MaskGenerator::Pointer mask)
  • unsigned int GetNBinsForHistogramStatistics() const
  • double GetBinSizeForHistogramStatistics() const
  • analyze if all 33 tests are really useful or if they are redundant

Tests:

  • MITK_TEST(TestUninitializedImage)
    • image: nullptr
    • mask/planarfigure: -
    • expected: mitk::Exception
  • MITK_TEST(TestCase1)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: mitk::PlanarPolygon
    • expected: one white pixel x-direction (255), mean, stddev, median tested
  • MITK_TEST(TestCase2)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: mitk::PlanarPolygon
    • expected: half one white pixel x-direction (255), mean, stddev, median tested
  • MITK_TEST(TestCase3)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: mitk::PlanarPolygon
    • expected: half one white pixel xy-direction (255), mean, stddev, median tested
  • MITK_TEST(TestCase4)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: mitk::PlanarPolygon
    • expected: one pixel white pixel + 2 half-pixels white + 1 half pixel black, mean, stddev, median tested
  • MITK_TEST(TestCase5)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: mitk::PlanarPolygon
    • expected: one pixel white pixel + 1 half-pixels gray, mean, stddev, median tested
  • MITK_TEST(TestCase6)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: mitk::PlanarPolygon
    • expected: quarter pixel (black) + whole pixel (white) + half pixel (gray) in x-direction, mean, stddev, median tested
  • MITK_TEST(TestCase7)
  • MITK_TEST(TestCase8)
  • MITK_TEST(TestCase9)
  • MITK_TEST(TestCase10)
  • MITK_TEST(TestCase11)
  • MITK_TEST(TestCase12)
  • MITK_TEST(TestImageMaskingEmpty)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: empty mitk::Image as mask
    • expected: empty statistics container
  • MITK_TEST(TestImageMaskingNonEmpty)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: mitk::Image with 3 activated pixels as mask
    • expected: statistics based on three pixels, mean, stddev, median tested
  • MITK_TEST(TestRecomputeOnModifiedMask)
    • image: ImageStatisticsTestData/testimage.dcm
    • mask/planarfigure: mitk::Image with 3 activated pixels as mask
    • expected: first empty statistics container, then numberOfVoxels==1
  • MITK_TEST(TestPic3DStatistics)
    • image: Pic3D.nrrd
    • mask/planarfigure: -
    • expected: known statistics (from external code/toolkit?)
  • MITK_TEST(TestPic3DAxialPlanarFigureMaskStatistics)
  • MITK_TEST(TestPic3DSagittalPlanarFigureMaskStatistics)
  • MITK_TEST(TestPic3DCoronalPlanarFigureMaskStatistics)
  • MITK_TEST(TestPic3DImageMaskStatistics_label1)
  • MITK_TEST(TestPic3DImageMaskStatistics_label2)
  • MITK_TEST(TestPic3DIgnorePixelValueMaskStatistics)
  • MITK_TEST(TestPic3DSecondaryMaskStatistics)
  • MITK_TEST(TestUS4DCylStatistics_time1
  • MITK_TEST(TestUS4DCylAxialPlanarFigureMaskStatistics_time1)
  • MITK_TEST(TestUS4DCylSagittalPlanarFigureMaskStatistics_time1)
  • MITK_TEST(TestUS4DCylCoronalPlanarFigureMaskStatistics_time1)
  • MITK_TEST(TestUS4DCylImageMaskStatistics_time1_label_1)
  • MITK_TEST(TestUS4DCylImageMaskStatistics_time2_label_1)
  • MITK_TEST(TestUS4DCylImageMaskStatistics_time1_label_2)
  • MITK_TEST(TestUS4DCylIgnorePixelValueMaskStatistics_time1)
  • MITK_TEST(TestUS4DCylSecondaryMaskStatistics_time1)

Summary:

  • 3 test images
  • several planar figures
  • several masks
  • different labels
  • different time
  • unitialized image

Related Objects
Search...

Event Timeline

hentsch triaged this task as Normal priority.Nov 30 2018, 11:18 AM
hentsch created this task.
hentsch added a subscriber: floca.Nov 30 2018, 1:23 PM
hentsch updated the task description. (Show Details)Dec 4 2018, 2:01 PM
hentsch updated the task description. (Show Details)Dec 4 2018, 2:17 PM
hentsch renamed this task from MitkImageStatisticsCalculatorTest takes too long to Optimization of MitkImageStatisticsCalculatorTest.Dec 4 2018, 3:46 PM
hentsch added a project: Restricted Project.Dec 5 2018, 10:43 AM
hentsch updated the task description. (Show Details)Dec 6 2018, 1:55 PM
hentsch updated the task description. (Show Details)Dec 6 2018, 2:25 PM
hentsch added a comment.Dec 17 2018, 9:10 AM

A reason why this test takes so long is that all images are read in setUp() before each test

hentsch added a subscriber: • kraeuter.Feb 5 2019, 2:49 PM
• kraeuter added a comment.Feb 7 2019, 10:44 AM

Breaking down the tests to avoid a timeout I noticed that a lot of these test fail
Right now the only test I can confirm passing are the several TestCase-tests and TestUnitializedImage

• kraeuter added a comment.Feb 7 2019, 1:46 PM

Tested again on release mode, all tests work as intended

hentsch closed subtask T25704: Difference between GUI and expected values MitkImageStatisticsCalculatorTests as Resolved.Feb 7 2019, 3:03 PM
• kraeuter added a comment.Feb 7 2019, 4:00 PM

Is there a specific reason why the test TestPic3DImageMaskStatistics_label2 sets m_Pic3DImageMask instead of m_Pic3DImageMask2?

hentsch added a comment.Feb 7 2019, 4:14 PM

In m_Pic3DImageMask2, only one label is used whereas 2 are used in m_Pic3DImageMask.
And the test is supposed to test the second label: ComputeStatisticsNew(m_Pic3DImage, imgMaskGen.GetPointer(), nullptr, 2);, where 2 is the label.

• kraeuter added a comment.Feb 11 2019, 11:52 AM

The tests show unreliable errors.
Testing a few times in a row, sometimes the tests runs through without any problems, sometimes they fail.
Looking at the logs there are two tests that can fail.
-> TestRecomputeOnModifiedMask
-> TestImageMaskingEmpty
There is nothing written at the tests about failing any conditions, but they display an error called "unknown label"

• kraeuter mentioned this in rMITK4be633af9753: Merge branch 'T26010-RelocateSetUpFunctions' into T25697….Feb 20 2019, 11:28 AM
• kraeuter closed subtask T26009: Find smaller images for MitkImageStatisticsCalculatorTest as Resolved.Feb 20 2019, 2:55 PM
• kraeuter added a comment.Feb 26 2019, 8:44 AM

Pushed new branch T25697-OptimizationOfMitkImageStatisticsCalculatorTest.

• kraeuter mentioned this in rMITK15ac863f87a9: Merge branch 'T23104-deprecateQmitkPlotWidget' into T25697….Feb 26 2019, 8:45 AM
• kraeuter mentioned this in rMITKf29b6e694d1a: Merge branch 'T25697-OptimizationOfMitkImageStatisticsCalculatorTest'.Mar 14 2019, 2:37 PM
• kraeuter mentioned this in rMITK5391bf7ccb47: Merge branch 'T25697-OptimizationOfMitkImageStatisticsCalculatorTest'.Mar 18 2019, 10:29 AM
hentsch closed subtask T26013: Define ground truth as Resolved.Mar 28 2019, 3:22 PM
hentsch added a comment.Mar 28 2019, 3:55 PM

Pushed new branch T25697-fixes.

hentsch mentioned this in rMITK65e1272d7683: Merge branch 'T25697-fixes'.Mar 28 2019, 3:56 PM
hentsch closed subtask T25715: VerifyStatistics should use epsilon instead of boiler plate code to handle small differences as Resolved.Mar 28 2019, 3:59 PM
hentsch closed subtask T26011: Update TestingMacros as Resolved.
hentsch closed subtask T26012: Define test cases as Resolved.
hentsch closed subtask T25716: ImageStatisticsCalculatorTests run through, but are failed as Resolved.
hentsch closed subtask T26010: Avoid loading all data before each test as Resolved.Mar 28 2019, 4:09 PM
kislinsk closed this task as Resolved.Jun 22 2020, 7:01 AM
kislinsk claimed this task.
MITK · MIC · IMSY · DKFZ