@kislinsk I know you have plenty of things to do, but this bug (along with T28516) is a blocker at the moment. I would like to recommend MITK as an annotation tool for our collaborators but cannot do that as along as the slice interpolation does not work (fast enough, T28516) for large images :-)

As it is an inexpensive operation my first guess would be an issue related to the amount of used memory. For example, the image above eats 3,5 GB, Activating the interpolation already triples it. Confirming adds at least another 3,5 GB. That could also explain why it is so slow (swapping). But at this point it is just a first assumption. On a 16 GB Windows machine I was able to complete the interpolation without crash but it took a while.

edit: But maybe it is also not as inexpensive as we though as we rarely have 2048x2048 image stacks.

And we seem to have a huge leak here. Even after deleting all the nodes again, half of my RAM is still in use.

A few things that impact the handling of huge images:

• Loading an image or at least a segmentation currently required twice as much RAM for a short amount of time. I guess there's a block copy happening.

• Interpolation is really heavily computing on a single core and adds another chunk of memory (twice as large) during the operation.

• While one part of that memory is released after the operation is complete, the other half is still not freed until the Segmentation view is closed. Removing the image or closing the project is not enough.

As it is an inexpensive operation my first guess would be an issue related to the amount of used memory. For example, the image above eats 3,5 GB, Activating the interpolation already triples it. Confirming adds at least another 3,5 GB. That could also explain why it is so slow (swapping). But at this point it is just a first assumption. On a 16 GB Windows machine I was able to complete the interpolation without crash but it took a while.

It is not a RAM issue. I checked that :-) No swapping on my 32GB machine. I would have noticed and reported that - I am not new to these things you know :-P

Most computers now have 8 cores or more. This operation takes forever to complete, so it would be nice to have it distributed across the cores

Pushed new branch to rMITK MITK: bugfix/T28491-SpeedUp2DInterpolationsOfLargeSegmentations.

Without looking into parallelizing the interpolation yet, I was already able to algorithmically speed up the interpolation by 7-8 times.

1024³ image block 2d-interpolation
==================================
Without optimizations      392.78s
Distance image cache        53.17s
-- Speedup factor ----------- 7.4x

Interpolation is not done in place to build a Do and Undo operation for the Undo/Redo-Stack. In the test case above this is another 30 seconds (more than half of the time with the first optimizations). While undo/redo is important, we should consider coming up with an heuristic, or option, or something similar to disable it for the interpolation of large images.

Combined with a multithreaded iteration through the interpolation slices, we should be able to reduce the duration to a few seconds.

Sounds like it goes in the write direction. :) thanks.

undo/redo: Why is there a dedicated Filter executed anyways for the undo/redo step? Couldn't we just store/clone the current state?

On my 4 cores / 8 threads workstation, I get a speedup factor of nearly 10x (or 35x, if neglecting the whole do/undo stuff during measurement). This is for a 1024³ block, where only the first and last slices contain segmentations, so the whole image block need to get interpolated.

Since we want to do a snapshot release at the end of the week, I will already request a code review. If accepted in time, 10x is better than nothing I guess. :-)

1024³ image block 2d-interpolation
==================================
Without optimizations      392.78s
Distance image cache        53.17s
-- Speedup factor ----------- 7.4x
Distance image cache +
Extract slice cache         47.22s
-- Speedup factor ----------- 8.3x
Distance image cache +
Extract slice cache +
Multithreading              40.44s
-- Speedup factor ----------- 9.7x



1024³ image block 2d-interpolation
without do/undo operations
(- fixed 30s)
==================================
Without optimizations      362.78s
Distance image cache        23.17s
-- Speedup factor ---------- 15.6x
Distance image cache +
Extract slice cache         17.22s
-- Speedup factor ---------- 21.1x
Distance image cache +
Extract slice cache +
Multithreading              10.44s
-- Speedup factor ---------- 34.8x

Deleted branch from rMITK MITK: bugfix/T28491-SpeedUp2DInterpolationsOfLargeSegmentations.

This is still not fixed, can we please re-open it?

(Ubuntu 18.04, MITK 2021-07-02 shapshot)

@kalali @a178n

I looked into this using the latest Develop-branch on Windows. Using the Debug build I was able to see the stack trace after a crash (using the small, cropped image):

Exception Unhandled:
Unhandled exception at 0x00007FF96A63AFAC (ucrtbased.dll) in MitkWorkbench.exe: An invalid parameter was passed to a function that considers invalid parameters fatal.

_STL_VERIFY(this->_Getcont() == _Right._Getcont(), "vector iterators incompatible");

inside vector -> _Compat, coming from

while ( ci != m_PointsContainer->End() )

inside bool BoundingBox< TPointIdentifier, VPointDimension, TCoordRep, TPointsContainer >::ComputeBoundingBox(void) const (in itkBoundingBox.hxx)
with

ci = itk::VectorContainer<unsigned long,itk::Point<double,3> >::ConstIterator

and

m_PointsContainer = const itk::VectorContainer<unsigned long,itk::Point<double,3> > *

or

m_Iter	{...}	std::_Vector_const_iterator<std::_Vector_val<std::_Simple_types<itk::Point<double,3> > > >

and

r.m_Iter	{...}	const std::_Vector_const_iterator<std::_Vector_val<std::_Simple_types<itk::Point<double,3> > > >

respectively.

Using the large, original image I get a crash inside delete_scalar.cpp in void __CRTDECL operator delete(void* const block) noexcept, coming from void LightObject ::UnRegister(), used by m_BoundingBox->SetPoints(points); inside void mitk::TimeGeometry::UpdateBoundingBox().

I did some testing (with Amir) on ubuntu 20.04 (32GB RAM). The issue could be replicated with a questionable increase in the utilization of RAM.
Fabian provided us a couple of files for testing, viz. tmp.nii.gz and tmp_cropped.nii.gz (same content, just removed some of the zeros around the ROI)

1- Tests using tmp.nii.gz
We tested multiple times the following workflow:

1. open MITK
2. load tmp.nii.gz
3. open segmentation plugin (not multilabel segmentation!)
4. No need to select an image, but make sure the segmentation is selected
5. at the bottom, select Interpolation-> 2-Dimensional
6. at the bottom, click confirm for all slices -> coronal

There are two possible outcomes:

• Aborted (core dumped) crash
• Computation runs without a crash. Preview of interpolated results shown (yellow masks, after Step 5) but Not confirmed (after Step 6). Also, high RAM utilization is shown in the bottom right corner of the workbench.
  • Now Re-run step 6. Outcome: Core dumped OR memory utilization 99%. Same behaviour as above.

2- Tests using tmp_cropped.nii.gz

We test multiple times the workflow as before.

There are two possible outcomes:

• Aborted (core dumped) crash.
• Computation runs without a crash and the result is confirmed/rendered as it should with No questionable rise in RAM usage.

We should check it again on the current code base introduced by Sids rework.

I tested this again with the current version of MITK (develop branch) and can see no problems. Interpolation works without problems on my laptop (Windows 10, 32GB RAM)