Page MenuHomePhabricator

[MultiLabel Segmentation] 2D / 3D Fast Marching tool does not work with multiple labels
Closed, ResolvedPublic

Description

Using the Multilabel segmentation plugin view and the 2D / 3D Fast Marching tool, a label set image with multiple labels is not correctly handled.

If using multiple labels on the same layer, the color of the first label is always used for a segmentation mask.

To reproduce:

  1. Load a 3D image, open the multilabel segmentation plugin view, create a new label set image
  2. create a new label on the first layer
  3. create another label on the first layer (distinct color)
  4. create a 2D / 3D Fast Marching segmentation mask on the first label
  5. create a 2D / 3D Fast Marching segmentation mask on the second label
  6. the first label will be removed
  7. the second label will have the same color as the first label

There are many problems with Fast Marching so we might skip solving this task and remove the tool completely.

PS: We tested all tools thoroughly (multi label, multi layer, 4D) but somehow either this slipped through or I couldn't find the corresponding task.

Revisions and Commits

rMITK MITK
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision

Event Timeline

I quickly looked into this since this is also true for the 3D Fast Marching Tool.
I realized that this is connected to the use of the AutoSegmentationWithPreviewTool, which is also used in the Threshold Tool (BinaryThreshold and BinaryThresholdULTool). Here the error also exists (see e.g. T28131: [MultiLabel Segmentation] 3D Threshold tool does not work with multiple labels / layers).

There is also a AutoMLSegmentationWithPreviewTool, so I looked at the differences:

  • AutoMLSegmentationWithPreviewTool is a subclass of AutoSegmentationWithPreviewTool
  • AutoMLSegmentationWithPreviewTool extends the functionality of the auto segmentation tool
  • AutoMLSegmentationWithPreviewTool seems to be used for algorithms that actually CREATE multiple labels, not for algorithms that have to take care of different labels

So it seems fine to use AutoSegmentationWithPreviewTool as a base class for Fast Marching and Thresholding.
However, debugging into this tool shows the following:

  • setting a seed point will eventually lead to a call of AutoSegmentationWithPreviewTool::UpdatePreview
  • here nothing related to labels / labelsetimage is done
  • finally a call to the virtual function DoUpdatePreview happens
  • this leads to a call to FastMarchingBaseTool::DoUpdatePreview
  • this again calls FastMarchingBaseTool::DoITKFastMarching, where nothing is done related to labels / labelsetimage (the word "Label" does not even exist in the class FastMarchingBaseTool)

I debugged also when using the Threshold tool and found a similar process flow.

From this I found the following:

  • in both classes a itk::BinaryThresholdImageFilter is used to finally generate the output image
  • the pixel values of the threshold output image are defined as
thresholdFilter->SetOutsideValue(0);
thresholdFilter->SetInsideValue(1.0);
  • this leads to a binary output image (preview image) with always pixel value 1.0 inside the mask (which is typically the first label, although the second label might be the active label).

Changing this to another value (e.g. 2.0) allows me to use the second label. This also works for the Threshold-Tools, so this is the bug described in T28131: [MultiLabel Segmentation] 3D Threshold tool does not work with multiple labels / layers.

--> @floca Is it an option to make this inside value dynamic by using the value of the active label? I'm not sure if this is the right approach since this needs to be done in all tools, that subclass AutoSegmentationWithPreviewTool and I think it would be better to somehow take care of this in the parent class.
How could this be done? We already had a discussion in T28131: [MultiLabel Segmentation] 3D Threshold tool does not work with multiple labels / layers about the problem but here the things that would be required to do seem complex. Reading the whole discussion now reveals to me that maybe we did not talk about the same thing: It sounds like I was talking about the problem of using the correct label (e.g. using the second label) while you were talking about the problem of overwriting / painting over a label with a new label. These seem to be two different problems and the later was also reported here: T28825: [Segmentation] Thresholding overwrites existing label without warning (and is also valid for 2D / 3D Fast Marching).

kalali renamed this task from [MultiLabel Segmentation] 2D Fast Marching tool does not work with multiple labels to [MultiLabel Segmentation] 2D / 3D Fast Marching tool does not work with multiple labels.Feb 18 2022, 5:47 PM
kalali updated the task description. (Show Details)
kalali added a revision: Restricted Differential Revision.

I added a first draft for a fix as a code review / differential.

let's talk about this in the next MITK meeting. In the end the whole complex boils down to two questions:

  1. how is the active label set and communicated to the tools (and where is it used/ who is responsible to use/ regard it)
  2. how does transfer image in the AutoSegmentationWithPreview work (see my comments in T28131)

With your proposed changes we would have a posibility to regard the active label, but we would still not support multiple label in one image.