Page MenuHomePhabricator

2D interpolation in MultiLabelSegmentation Editor only works for 1. label
Open, NormalPublic

Description

2D interpolation in MultiLabelSegmentation Editor only works for 1. label (that is, the label that was initialized as label 1).

For example, create new segmentation in MultiSegmentation Editor create two labels (L1, L2) in same layer. Select L1, paint two circular patches a few image slices apart. Choose 2D interpolation, click start. --> Works and one can accept all or individual interpolations. Now, select L2 and paint two circular patches a few image slices apart. Choose 2D interpolation, click start. --> Nothing happens and there is no interpolation for L2. In fact, it seems to reference L1 instead of L2 in its attempt to interpolate. Now, delete L1 so that only one label remains, try again with L2, no result. Now, delete L2 so that there are 0 labels left. Make new label L1, it works with new L1.

OS: Ubuntu 18.04.2 LTS
MITK: MITK-v2018.04.2-linux-x86_64

Event Timeline

kislinsk triaged this task as Normal priority.Apr 29 2019, 6:24 AM
kislinsk edited projects, added MITK (2018-04-4); removed MITK.

Is this task still valid? If I try what's written in the description (after showing the interpolation widget), nothing happens, not even on the first label.

Can someone confirm? @floca, @kleina?

Ehm... I cannot do Interpolation in MultiLabelSegmentation I think. Shouldn't there be a drop down?

image.png (431×495 px, 15 KB)

Ehm... I cannot do Interpolation in MultiLabelSegmentation I think. Shouldn't there be a drop down?

image.png (431×495 px, 15 KB)

Oh yes, sorry. It was removed for the relaese with this commit: rMITKc7ee88da8242: Hide interpolation widgets. You need to show it again.

Ah ok, thx. I will try later.

Tested it on Windows. I can confirm what @kalali said in 2018.04.0. In 2018.04.2 it seems to work on single segmentations.
For multi-label segmentations it interpolates always for the first label (label value=1). So the bug described can be confirmed.

In 2021.02, the functionality is already disabled.

Ok I think I found something odd:
In mitk::ShapeBasedInterpolationAlgorithm::ComputeDistanceMap there are two lines (92 / 93), where the invert filter input is defined with

invertFilter->SetInput(binaryImage);
invertFilter->SetMaximum(1);

The comment even states // this assumes the image contains only 1 and 0 (see https://phabricator.mitk.org/source/mitk/browse/develop/Modules/Segmentation/Algorithms/mitkShapeBasedInterpolationAlgorithm.cpp$91-93).

This is not true for a labelsetimage where a label > 1 should be interpolated. I changed this to SetMaximum(2) and created two labels as described above. Now the second label is correctly interpolated, if it is the active label. But the first label cannot be interpolated anymore. So I'm not 100% sure how to fix this but I think this is a good point to continue investigating, maybe @floca?

That is what I assumed... the labels created by multi-label plugin always contain only two values, if I am not mistaken: 0 and x with x e [1,2,3,...]. One could check for the label values and if there is only two different ones, use these two. This does not include the case when a label was created externally,e.g., by U-Net and contains several values in a single image.

that is at least one of the two code locations I assume to make problems. (The other one is when the interpolation results are written back to the segmentation)
For now, I wouldn't try to patch in multi-label support into the interpolation class. Instead I would ensure that only a label image (so an image that just contains the active label) is passed to the interpolation class.
I think that that is the most pragmatic approach to it, procure the current implementation will never work properly when multiple labels are present, due to the assumptions already stated by @kalali.

Things like being aware of the locked labels while doing interpolation should be added later (we have the new data class).

For now, I wouldn't try to patch in multi-label support into the interpolation class. Instead I would ensure that only a label image (so an image that just contains the active label) is passed to the interpolation class.
I think that that is the most pragmatic approach to it, procure the current implementation will never work properly when multiple labels are present, due to the assumptions already stated by @kalali.

I'm not sure if I understood you correctly: What do you mean by ensure that only a label image (so an image that just contains the active label) is passed to the interpolation class?
The mentioned function is called as

auto lowerDistanceImage = this->ComputeDistanceMap(lowerSliceIndex, lowerSlice);
auto upperDistanceImage = this->ComputeDistanceMap(upperSliceIndex, upperSlice);

within mitk::ShapeBasedInterpolationAlgorithm::Interpolate, which is called inside mitk::SliceBasedInterpolationController::Interpolate as algorithm->Interpolate(lowerMITKSlice.GetPointer(), lowerBound, upperMITKSlice.GetPointer(), upperBound, sliceIndex, 0, resultImage);
Here, lowerMITKSlice and upperMITKSlice are created using the ExtractSliceFilter on the whole mitk::LabelSetImage.

What do we need to ensure here?

I think this will not work because this label image (extracted slice) can still have a value other than 1, right?