Page MenuHomePhabricator

[SEG] Response labels all not displayed on Workbench
Closed, ResolvedPublic

Assigned To
Authored By
a178n
Feb 3 2023, 1:50 PM
Referenced Files
F2652067: totalseg_no_label.png
May 3 2023, 9:56 AM
F2652060: totalseg_no_colour.png
May 3 2023, 9:56 AM
F2622014: mitk_labels_Screenshot.png
Feb 3 2023, 1:50 PM

Description

A MONAILabel response contains segmentation image & a json with label names. These label names can be written into the LabelSetImage object. However, even if assigned correctly, one label name, with label value 1, is remapped back to what is on the QmitkLabelsWidget.
This behaviour is by design, nevertheless, not amicable for MonaiLabel tool.
See screenshot:

mitk_labels_Screenshot.png (1×2 px, 568 KB)

The Label 1 is actually called Spleen.

Revisions and Commits

rMITK MITK
Restricted Differential Revision
Restricted Differential Revision

Event Timeline

a178n triaged this task as Normal priority.Feb 3 2023, 1:50 PM
a178n created this task.

the default implementation is currently to not overwrite label information if already present in the segmentation image only copy labels from the preview that do not exist.

To change that you would override the TransferPerpare function in your tool class.

in that context one could also think about extending the TransferLabelInformation function, which is used by TransferPrepare and thus the actual copying of label information. Here one could add an option that allows to override label information with the label information of the preview even if it already exists.

Hi Ralf,

Sorry if it wasn't clear, the issue happens already while generating Preview. The TransferPrepare function is only called while confirming labels.
Anyway, the solution (at least partly) lies in modifying the SegWithPreviewTool::TransferLabelInformation. The current logic of checking labels already existing in source / target would need to be changed to achieve what's amicable for MonaiLabel.
When I comment out the check if (!target->ExistLabel(targetLabel, target->GetActiveLayer())), it works. "Spleen" is displayed as spleen in addition to "Label 1".
However, this seems to open more bugs. I have to debug more in this direction.

That ist not a good solution (as far as I can see it) i.a. due to other constraints. Lets discuss this topic in the next meeting.

In the meeting it was decided that a decision regarding whether or not to replace or add labels will be still in RFD.
Decision will be later on after further discussion with the group, in conjunction with Ralf's planned changes in the tool API classes.

a178n renamed this task from MONAILabel response labels all not displayed on Workbench to [SEG] Response labels all not displayed on Workbench.Mar 15 2023, 5:38 PM
a178n raised the priority of this task from Normal to High.
a178n edited projects, added MITK (v2023.04); removed MITK.

The issue is also applicable to TotalSegmentator tool where label id 1 (spleen) is predefined yet replaced as "Label 1" by MITK.

Current interim thoughts:

  • There is no reason why tools should not be allowed to alter label information at their will
    • also the simple label widget when selecting the labels that should be transfered should show the content of the preview (thus with all alterations the tool made).
  • We should allow two modes
    • merge (overwrite the content (label information and pixels) with what we have in the preview (if the preview has changed label name i.a. than it also should change (basically the states of the label (all its content should change; but we cannot just clone and replace the instance, because the other label might be wired with some observers and such...)).
    • add (as new layer) so now colission, but the new labels must also change their labelvalue to avoid collision)
  • tool should be able to decide which is its default state (logic is implemented in base class)
  • tool UI should be able to decide if the user is allowed to change the mode (logic is implemented in base ui class).
kislinsk updated the task description. (Show Details)
floca added a revision: Restricted Differential Revision.May 2 2023, 5:32 PM

@a178n the changes in D816 should offer everything to solve the problem. I also adapted TotalSegmentator Tool. But you should verify that it works as expected.
Basically the tool should now change the labels in the preview as they like (see Otsu and TotalSegmentator for example) and use the TransferMode AddLabel. Then the labels (with all there settings) of the preview image will be added instead of mapped.

Hi Ralf,

I tested Totalsegmentator. I am seeing inconsistent behavior.

  1. the issue at hand seems to be fixed. I can see the label Spleen. But, now the label colours can't be seen anymore. Every label is gray.

totalseg_no_colour.png (2×3 px, 466 KB)

  1. Colours exist but no labels in the preview list.

totalseg_no_label.png (2×3 px, 439 KB)

Happens randomly.