Page MenuHomePhabricator

Weak Label Annotation Picking Tool
Closed, ResolvedPublic

Description

Silvia and Ahmed would like a special picking tool for selecting different regions to annotate. How do we go about this?

From Ralfs mail:

It looks sounds like a variation of the picking tool.
 
But to realize that in a clean way it would also mean to advance the AutoSegmentationWithPreviewTool in such a way that tool developers can choose from different transfer strategies for the preview into the segmentation. So far we only have: Replace segmentation image with preview image (that is the current strategy and why most of the auto tools are not available for multi label)
And this ultimately mean that we also finally have to refactor the WithPreview tools to work with multilabel instead of assuming only binary labels as there results….

Revisions and Commits

rMITK MITK
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
AuditedRestricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision
Restricted Differential Revision

Event Timeline

kislinsk triaged this task as Normal priority.Mar 17 2022, 10:54 AM
floca added a revision: Restricted Differential Revision.Apr 5 2022, 5:47 PM
floca added a revision: Restricted Differential Revision.Apr 5 2022, 6:07 PM
floca added a revision: Restricted Differential Revision.Apr 6 2022, 6:51 PM

I tested this with the related review and the differential solves this issue (although there is no clear description of what should be achieved).
From the related D622 I read the following description:

Now you can also pick from multiple other labels and the picked
elements will be added to the active labels.
Be aware if you pick from other labels the must not be locked,
other wise the lock will be ignored.
If you pick from the active label it is irrelevant if it is locked.

I tested the following

  1. created multiple patches for the same label (first label) using the threshold tool
  2. selected multiple patches using the picking tool and confirmed the segmentation
  3. as expected all other patches of the first label are removed and the label only shows the picked patches
  1. created multiple patches for the same label (first label) using the threshold tool
  2. created multiple patches for another label (second label) using the Region Growing 3D tool
  3. this lead to the first label being removed since the Region Growing 3D tool is not modernized so far
  1. created multiple patches for the same label (first label) using the threshold tool
  2. created multiple patches for another label (second label) using the Otsu tool
  3. selected multiple patches using the picking tool:
    • the single patch created by the Otsu tool (second label)
    • multiple patches created by the threshold tool (first label)
  4. confirmed everything for the second label
  5. nothing happens, as the first label is locked
  6. repeated everything with an unlocked first label
  7. as expected selected patches of the first label are added to the second label after confirmation
  8. repeated everything, only selecting a single patch of the first label
  9. as expected all other patches of the second label are removed only the selected single patch of the first label is added to the second label after confirmation
  1. created multiple patches for the first and the second label using the add tool
  2. selected multiple patches using the picking tool, using a third, previously unused label
  3. as expected all picked patches of the first and the second label are added to the third label after confirmation

Note:

  • picking from a different layer is not possible
  • this also works for static segmentations on 4D images
  • this kind of works for dynamic segmentations on 4D images
    • for this to work you have to be on the time step were you pick the patches
    • picking patches on multiple timesteps does not seem to work
    • somehow I had problems with the highlighting of picked regions wen using 4D segmentations

Thanks for extensive testing

  • picking from a different layer is not possible

That's correct. This is a known issue. It will be handled when we have the new data structure.

  • picking patches on multiple timesteps does not seem to work

Could you elaborate. I have no problems (as long as there is "something" below the selected point it will be picked in all time steps)

  • somehow I had problems with the highlighting of picked regions wen using 4D segmentations

More details in a seperate tasks would be helpfull. Cannot directly reproduce it.

I quickly created a short video to show what happens in my version:

As you can see I did the following

  • working only with a single layer, having three labels
  • selecting timestep 0
  • creating two masks for the first label (yellow) and one mask for the second label (green)
  • transferring the yellow labels to the third label (blue) by picking them
    • see how the picked labels are not highlightes
  • selecting timestep 2
  • creating a single mask for each of the three labels
  • picking both the yellow and the green mask on timestep 2 (no highlighting of picked masks)
  • selecting timestep 0 (you can see the green points of the picking)
  • picking both the green mask and the rightmost blue mask (no highlighting of picked masks)
  • only the masks on timestep 0 are transferred to the blue label
    • masks on timestep 2 stay unaffected
    • note: I did not pick the leftmost blue mask on timestep 0 but it was transferred as well, probably because the crosshair of the picking tool in timestep 2 aligned with the mask in timestep 0

Edit: I also get the following output
core.mod.core.eventstatemachine ERROR: Unhandled excaption caught in ExecuteAction(): Dimension 4 is not in (ItkOutputImageType::ImageDimension)

If I haven't overlooked something, in your video it behaves like expected.

  1. Process all time steps is *not* activated, so it will pick only in the current time step.
  2. Pointset is not time step aware. The selected points will be evaluated in all time steps; it does not matter when you have added them.

One can argue about the pros and cons of point 2, but even if we would like to change it, that would be a new feature task and nothing for the release phase (as it is no new behavior).

Or have I missed something?

  1. Ok, then it just wasn't clear to me that you explicitly need to enable "Process all time steps". Now picking over different time steps works.
  2. I'm fine with the pointssets being shown in every time step, but the problem remains that a pointset may accidentally pick a label mask if the pointset is unintentionally placed inside a label mask (see note in my previous comment). But this could be a fix for the future.

The issue that the picked regions are not highlighted is still valid.

  1. I'm fine with the pointssets being shown in every time step, but the problem remains that a pointset may accidentally pick a label mask if the pointset is unintentionally placed inside a label mask (see note in my previous comment). But this could be a fix for the future.

I think that is not decidable with out a usage study / clear usage definition. Depending on the use case, people could find it very tideous to set the points for each frame explicitly.
I deem that as a feature discussion and not a bug/regression. Further it would be a new ticket. As this ticket, focuses on the demands of NCT Annotation, which for know is static.
Might change in the future though...

The issue that the picked regions are not highlighted is still valid.

Partily still exists. hm, Have to live with it as known issue. Don't now why the rendering is sometimes not updated after deleting the points again....

@Siddharth Tourani Please check with Ahmad if the picking tool is now (after D622 is landed on release branch) sufficient. If so, please close the task. Thanks.

  • After the meeting with @floca, Ahmad, Silvia and Jan, they have said that Ralf's suggested union mode would definitely be more appreciated. The task T29146 describes the union mode in a bit more detail. T
  • They also mentioned that by default they would prefer not to have any locks on the labels.
  • Additionally, they would also like the picking tool to work across layers, but @floca suggested we implement this feature subsequently after the new mitk::MultiImage data-structure is finished.
floca edited projects, added MITK (v2022.10); removed MITK.
floca added a revision: Restricted Differential Revision.May 17 2022, 8:12 AM