Page MenuHomePhabricator

AutoSegmentationWithPreviewTool tool manager won't support multithreading
Closed, InvalidPublic

Description

MITK segmentation plugins based on the AutoSegmentationWithPreviewTool tool manager API can't be used with QThreads for non-blocking operations.
Specifically, the call
tool->UpdatePreview()
inside a QThread will cause OpenGL error: Cannot make QOpenGLContext current in a different thread.
This occurs arguably due to the encapsulation of segmentation process ComputeMLPreview and consequently, the segmentation data node creation + rendering of the computed LabelSet image inside the single function call.

Event Timeline

a178n renamed this task from AutoSegmentationWithPreviewTool tool manager calls won't support multi threading to AutoSegmentationWithPreviewTool tool manager won't support multi threading.Sep 8 2021, 9:14 AM
a178n created this task.

I did some debugging and found some more insights into the situation and workarounds.

So, the AutoSegmentationWithPreviewTool tool manager not supporting QThreads has threading has 2 implications-

  1. Rendering issues: OpenGL error: Cannot make QOpenGLContext current in a different thread.

    Insight: This rendering issue occurs at 2 places:
    • In mitk::AutoMLSegmentationWithPreviewTool::UpdateCleanUp method; to be exact: at the mitk::AutoSegmentationWithPreviewTool::ResetPreviewNode() call. This cannot be done in a sub-thread.

      Workaround: UpdateCleanUp can be overriden in the plugin child class (ie. eg mitknnUnet.cpp) to make sure the ResetPreviewNode() is not invoked while UpdatePreview() is running. ResetPreviewNode() call can done later in the main thread.
    • In mitk::AutoMLSegmentationWithPreviewTool::DoUpdatePreview method; from where the m_MLPreviewNode is set with the output from ComputeMLPreviewmethod. In that snippet, RenderingModeProperty is created & set. This cannot be done in a sub-thread.

      Workaround - Requires modification to the AutoMLSegmentationWithPreviewTool API. The snippet for data node creation + rendering of the computed LabelSet image can be migrated to another new function eg. SetNodeProperties(mitk::LabelSetImage::Pointer newMLPreview). This new function SetNodeProperties can be overridden in the plugin child class (ie. eg mitknnUnet.cpp) to make sure the node properties are not set immediately after the ComputeMLPreview call, but later from the main thread.

Hence, when the 2 abovementioned situations are handled as per the workarounds, the issue is fixed. Tested in feature/T28553-nnUNet-in-MITK branch.

  1. Exception Handling: leading to Segmentation fault and Workbench crash

Any exceptions occurring in ComputeMLPreview, when thrown back to the calling function (eg. using mitkThrow()<<"I am an exception") will lead to:
QObject::setParent: Cannot set parent, new parent is in a different thread.

Insight (might not be accurate):
This occurs in the first catch block of mitk::AutoSegmentationWithPreviewTool::UpdatePreview, specifically at ErrorMessage.Send(msg).
I suppose, this is not supported by Qt in a sub-thread.

Workaround:
Don't throw back any exception, merely return nullptr and throw a generic exception in the main thread after a null check.
We can discuss more on this.

kislinsk triaged this task as Normal priority.Sep 15 2021, 4:55 AM
a178n renamed this task from AutoSegmentationWithPreviewTool tool manager won't support multi threading to AutoSegmentationWithPreviewTool tool manager won't support multithreading.Aug 11 2022, 9:25 AM
a178n lowered the priority of this task from Normal to Low.
a178n edited projects, added MITK; removed MITK (v2022.10), Request for Discussion.
a178n removed a subscriber: a178n.
a178n removed a subscriber: kalali.

This context and code artifacts referred in this issue seems obsolete or changes. The issue needs to be rephrased if not closed.

@a178n will close this task and create a new dedicated task.