wip

Most likely errors:

• segmentation is always performed on timestep 0 (Threshold/ULThreshold)
• dynamic segmentation is converted into a 3D segmentation (Otsu, Fast Marching, Picking)

Discussion so far:

• perform on all timesteps (threshold / ULthreshold) might cause a huge overhead and should not be to default
• dynamic segmentations should not lead to 3D segmentation result

• not sure whether static execution should happen on the created or executed timestep

Discussion about the default and preferred behavior: T27509: [Segmentation] How to handle 3D and 4D Segmentations on 4D images.

In T27507#206792, @kalali wrote:

Discussion about the default and preferred behavior: T27509: [Segmentation] How to handle 3D and 4D Segmentations on 4D images.

I still back my own default proposal statet in T27509. I think this is the best way today.

My default proposal (see T27509) would lead to this status result:

1: I would make it option to perform on all timesteps (but only a nice to have for this run)
2: should be performed using the current selected timestep of image
3: @thomass I am not sure I understood the problem correctly? What do you mean with initial image when you write the currently selected image will be segmented? The static table cells are empty. Does it mean they where not checked, or that everything is fine?
4: @thomass I am not sure I understood the problem correctly? Does the tool work, but on some images (as US4DCyl) not? First half sounds like a bug, second half sound like a correct behavior.
5: I am not sure if picking makes realy sense on 4D at all. I tend to say you can pick on 4D in the current time step, but it will always produce a static mask with your picked segmentation.
6: Generates 3D segs instead of 4D segs
What do you think?

In addition to the existing results:

Concerning static with t != 0: The problem I encountered is, that the segmentation was done on the structure of timestep 2, but the underlying image itself is the image from timestep 0:

Reinint on US4D image (t = 0), timesteps available, static mask created on timestep 0

Reinint on US4D image (t = 2), timesteps available, static mask created on timestep 2

Reinint on static mask (t = 0), timesteps not available (3D mask), static mask created on timestep 0

Reinint on static mask (t = 2), timesteps not available (3D mask), static mask created on timestep 2

@thomass Just to make sure there is no confussion: In your table, where static t = 2 and the segmentation was created on t = 0: Was it actually computed on the structure of t = 0 or was it computed on t = 2 but the underlying image after reinit showed t = 0, as in this case described?

So I think it is a good thing to create a segmentation based on the structure of the image at timestep 2 and have this as a memory-saving 3D segmentation mask. But either our reinit-algorithm is misleading or the static mask should really only cover this individual timestep, instead of the whole time span. Or am I missing something?

In T27507#206802, @kalali wrote:

But either our reinit-algorithm is misleading or the static mask should really only cover this individual timestep, instead of the whole time span. Or am I missing something?

I think you are missing, that one should never reinit on the segmentation itself, but always on the reference image. Then there is always a time slider apropriate to the reference image where you can select the current time point and the reference image content associated with this time point should be used to generate the (static) segmentation.
Could you please recheck your evaluation? I think we have a fundamental difference in the assumption how the segmentation tools should behave an therefore what the correct outcomes of the tools should be. Thank you.

So I am not quite sure that about the re-init problem but to further clarify my list:

3. What I meant here was that a static segmentation is returned by the tool although a dynamic was created by the plugin -->therefore I would call it a bug. The same holds for Otsu.

4. I couldn't perform the operation on those images / the result was always the same for all steps (I had trouble finding right parameters that change something), but anyways: it created a 3D segmentation in the end which should not be the case if we created a dynamic segmentation before, right?

5. Maybe we should really clarify what the Picking tool should do.

General: I agree that we should have a general Segmentation discussion this week! :)

@kalali / @thomass Can you spawn the identified open tool problems as sub tasks till tomorrow noon or at least indicated here the final results after our todays discussion/clarification how to proceed? That would be very helpfull as they have also to be tackeled. thanks. Ralf

So Region Growing 3D seems to work and could probably be used for reference for other algorithms.

@thomass I checked Fast Marching 3D and re-opened T18436: [Segmentation] Fast Marching 3D Tool only supported for first time step in 4D images for the static mask cases.
Could you re-check the dynamic case? I tested it and with t = 0 I got a dynamic mask (4D) with a segmentation only on the first (0) timestep but with t != 0 I have the same problem as described in the mentioned task (green contour nowhere near the seed point).

I can reproduce the behavior for the US4DCyl but not for the heart 3D+t. But it looks exactly like the error for static segmentation and t=2 (that I showed during the telco).

In T27507#207804, @thomass wrote:

All issues have been adressed in respective subtasks.

@thomass I cannot find a sub task for the watershed tool. Looking at the table/your explination it also is not correct, becuase it always produces 3D masks. Or is this status not correct anymore?

You totally right and the reason why I oversaw this is when I double checked:

If you perform a dynamic segmentation using watershed, it creates a new segmentation no matter if you created that dynamic segmentation before.
The dynamic segmentation stays 4D (which is correct) but the new '_Watershed' segmentation is 3D. The result is stored in that new segmentation not the dynamic one.

To not blow up the datamanager with that, I think it is a bug and the _Watershed should not be created but instead the dynamic segmentation should be used.

2D Segmentation tools:

• Add
• Substract
• Correction
• Paint
• Wipe
• Region Growing
• Fill
• Erase
• Live Wire
• 2D Fast Marching

--> First test Looks suprisingly good, even 2D Fast Marching tool works as expected on US4DCyl.nrrd for dynamic and static Segmentation
--> Region Growing works correctly in terms of timesteps but shows odd behavior (the mouse cursor is released but still more foreground pixels are added --> this often causes an oversegmentation on US4DCyl @kislinsk is this something potentially caused by your recent change?

wip

In T27507#208073, @thomass wrote:

2D Segmentation tools:

• Add
• Substract
• Correction
• Paint
• Wipe
• Region Growing
• Fill
• Erase
• Live Wire
• 2D Fast Marching

--> First test Looks suprisingly good, even 2D Fast Marching tool works as expected on US4DCyl.nrrd for dynamic and static Segmentation
--> Region Growing works correctly in terms of timesteps but shows odd behavior (the mouse cursor is released but still more foreground pixels are added --> this often causes an oversegmentation on US4DCyl @kislinsk is this something potentially caused by your recent change?

wip

@thomass Could you please spawn a sub task for your reagion growing finding?

Region growing task can be found here T27684

It really suprises me that FastMarching2D crashes now since I explictly played with timesteps in T26975. But maybe I truly overlooked something, then I am sorry.

Static Segmentation:
-ADD: A segmentation is drawn on timestep 0, if I change the timestep, the segmentation remains and I can add a new segmentation on timestep !=0, both segmentations are visible since I only have one timestep --> As axpected
-SUBSTRACT: The drawn segmentation can be removed independent from the timestep 0 and timestep !=0 --> As expected
-CORRECTION: The drawn segmentation is removed independent from the timestep 0 and timestep !=0 --> As expected
-PAINT: Painted segmentation occurs on timestep 0. When changing the timestep, the painted segmentation is added to the static segmentation --> As expected
-WIPE: Segmentation is wiped on timestep 0 and wiped on timestep !=0 --> As expected
-RegionGrowing2D: Region Growing is applied on timestep 0 (fits to the image). When changing the timestep, the region growing result fits to the timestep. No crash. Quite slow updates. --> As expected
-FILL: Different timesteps can be filled. When changing the timestep, the segmentation remains --> As expected
-ERASE: Different timesteps can be erased. --> As expected
-LiveWire: T27800
-2D FastMarching: T27721

@thomass Thanks.
Now I might be nasty. Have you also checked everything wenn loading to 4D images with different time geometry? E.G. So that the selected time step in the navigator must not correspond to the same time step of the image you segment on?

In T27507#211481, @thomass wrote:

It really suprises me that FastMarching2D crashes now since I explictly played with timesteps in T26975. But maybe I truly overlooked something, then I am sorry.

I just tested it with both mentioned datasets MITK-Data\UltrasoundImages\4D_TEE_Data_MV.dcm and MITK-Data\3D+t-Heart and it crashed with both. Maybe - because it was not explicitly stated - the newly created segmentation was a static segmentation, not a dynamic segmentation; as it seems that this crash only occurs with a static segmentation (which makes sense looking at the proposed solution in the corresponding differentials).

In T27507#211505, @floca wrote:

@thomass Thanks.
Now I might be nasty. Have you also checked everything wenn loading to 4D images with different time geometry? E.G. So that the selected time step in the navigator must not correspond to the same time step of the image you segment on?

With two 4D images 3D+t-Heart and US4Dcyl it crashes indeed when

• using the Add tool on the static 3D+t Heart on timestep 0
• changing the timestep and then use the Add tool again

I couldn't reproduce the crash when combining 4D_TEE_Data_MV and US4Dcyl (yet)

In T27507#211583, @thomass wrote:

In T27507#211505, @floca wrote:

@thomass Thanks.
Now I might be nasty. Have you also checked everything wenn loading to 4D images with different time geometry? E.G. So that the selected time step in the navigator must not correspond to the same time step of the image you segment on?

With two 4D images 3D+t-Heart and US4Dcyl it crashes indeed when

• using the Add tool on the static 3D+t Heart on timestep 0
• changing the timestep and then use the Add tool again

I couldn't reproduce the crash when combining 4D_TEE_Data_MV and US4Dcyl (yet)

Could you please check (if not done already), if this scenario also crashes if you do it with a static segmentation?
This info is important for the decission, if we have to fix it directly or just partialy deactivate the dynamic option for this cycle. (Curently static segs on 4D images are more relevant then dynamic segs).

In T27507#211659, @floca wrote:

In T27507#211583, @thomass wrote:

In T27507#211505, @floca wrote:

@thomass Thanks.
Now I might be nasty. Have you also checked everything wenn loading to 4D images with different time geometry? E.G. So that the selected time step in the navigator must not correspond to the same time step of the image you segment on?

With two 4D images 3D+t-Heart and US4Dcyl it crashes indeed when

• using the Add tool on the static 3D+t Heart on timestep 0
• changing the timestep and then use the Add tool again

I couldn't reproduce the crash when combining 4D_TEE_Data_MV and US4Dcyl (yet)

Could you please check (if not done already), if this scenario also crashes if you do it with a static segmentation?
This info is important for the decission, if we have to fix it directly or just partialy deactivate the dynamic option for this cycle. (Curently static segs on 4D images are more relevant then dynamic segs).

Since we were having problems with static segmentations, I tested (as shown in the table above) all tools with static segmentations. Therefore, the test with two Images was also carried out using static not dynamic. And (I did also not state that clearly): I checked with the Add tool and it crashed. It didn't have the time to redo all the test for two Images yet. If necessary, I might need a helping hand because static/dynamic*single/double*2DTools/3DTools = many test cases

@thomass OK. Thats enough info, thanks. No further testing for now. We do it in context of a release branch or/and explizit fixes.

We had a discussion about refactoring the segmentation tools (similar to {D380}). The idea is to outsource redundant code into base classes (e.g. time-management, slice extraction, input data checks etc.) to have a single point of truth for the starting point of the different segmentation-tool-algorithms. This would make the tools less error prone and can simplify testing (even allow automatic testing for the input management steps) (see T27800#211467).

The consequences are

• we will not continue testing here for now, as @floca already mentioned
• we will instead focus on the checklists (T27506: Extend checklists for segmentation view) to provide a fundamental test base to test the tools, once they are refactored (hopefully until end of November)
• we (@floca?) will refactor the tools with

In T27507#211660, @thomass wrote:

because static/dynamic*single/double*2DTools/3DTools = many test cases

in mind.

In T27507#207494, @kalali wrote:

So Region Growing 3D seems to work and could probably be used for reference for other algorithms.

Since some tools have been reworked with an additional Create as new segmentation-checkbox this should be consistent. Region Growing 3D was not refactored in that regard --> T27872

@floca I could only find two tasks mentioned here that are not closed, T27872 and T27684, but they are unrelated to the task description. Can this task be closed?