Page MenuHomePhabricator

[SEG] Add a superpixel tool
Closed, SpitePublic

Description

ITK offers https://itk.org/Doxygen/html/classitk_1_1SLICImageFilter.html.
based on that filter we could realize a segmentation tool that offers superb pixel labelling.

Basic idea of the tool would be as follows:

  • it is an interactive tool
  • an activation it computes the super pixel map where the filter
  • on mouse ckick (or mouse move with pushed mouse button (if it is a new super pixel id)) the super pixel ( each subpixel has its own pixel value) region at the click position will be transferred as active label.

Open questions:

  • Only offer a 2D tool? Or also a 3D tool? (would the later make sense from a UX perspective?)
  • Use of preview or direct manipulation?

Related Objects

Event Timeline

floca triaged this task as Normal priority.Jun 1 2022, 5:19 PM
floca created this task.

only 2D are also 3D?

From the itk::SLICImageFilter page:

The original algorithm was designed to cluster on the joint domain of the images index space and its CIELAB color space. This implementation works with images of arbitrary dimension as well as scalar, single channel, images and most multi-component image types including ITK's arbitrary length VectorImage.

So it should work for 2D and 3D.

So it should work for 2D and 3D.

Thanks for the input! My text was missleading. I meant, if we want only have a tool that works on 2D (e.g. in conjunction with interpolation) or also an tool where you can paint superpixels in 3D.

Ah OK. Thats a decision above my pay grade.

So it should work for 2D and 3D.

Thanks for the input! My text was missleading. I meant, if we want only have a tool that works on 2D (e.g. in conjunction with interpolation) or also an tool where you can paint superpixels in 3D.

Do you see some (dis-)advantages in one option or the other?
In general I think nowadays we could invest time and energy to get more (semi-)automatic 3D tools into MITK. As far as I understood, SLIC can also be used as a first step for other algorithms, so having it available for 2D and 3D makes sense?

One thing to consider might be, is SLIC even useful? If neural networks are going to be taking care of segmentation anyway, what is it's likely sell-by date? This depends on how quickly NN based segmentation algorithms are incorporated into MITK.

Do you see some (dis-)advantages in one option or the other?

Per se, no. Was only 2 thoughts:

  • I think it depends on the performance of the SLIC filter. If 3D has a good performance for the proposed interaction (see task description) I am fine.
  • I was not sure if people will realy use it in 3D, as the would have to constantly scroll through slices to ensure that the 3D superpixels are everywhere correct enough.

One thing to consider might be, is SLIC even useful? If neural networks are going to be taking care of segmentation anyway, what is it's likely sell-by date? This depends on how quickly NN based segmentation algorithms are incorporated into MITK.

It is a point. In the end I think it depends on the cost.

For now I would tend to make it from the structure quite simelar to the way the reworked Paint/Wipe is done (not for contours, but the logic of having an addional slice where you interact with and wich is then merged into the seg after the mous button is released) (thus only for inteactive 2D). I think the costs there are minimal as we have a template and there the benefits are the largest (so this would be the best cost to benefit ratio.

floca claimed this task.

Reopen if the request really appears. Currently I guess things like SAM and medSAM will also do well.