Page MenuHomePhabricator
Create Task
Maniphest T29379

MONAI Label REST Response JSON issues, allegedly
Open, NormalPublic
Actions

Description

This issue is to keep track of issues in the JSON response format. Here is a sample response JSON:

{
  "name": "MONAILabel - Radiology",
  "description": "DeepLearning models for radiology",
  "version": "2.0",
  "labels": [
    "background",
    "spleen",
    "right kidney",
    "left kidney",
    "liver",
    "stomach",
    "aorta",
    "inferior vena cava",
    "gallbladder",
    "esophagus",
    "portal vein and splenic vein",
    "pancreas",
    "right adrenal gland",
    "left adrenal gland"
  ],
  "models": {
    "deepedit": {
      "type": "deepedit",
      "labels": {
        "spleen": 1,
        "right kidney": 2,
        "left kidney": 3,
        "liver": 6,
        "stomach": 7,
        "aorta": 8,
        "inferior vena cava": 9,
        "background": 0
      },
      "dimension": 3,
      "description": "A DeepEdit model for volumetric (3D) segmentation over 3D Images",
      "config": {}
    },
 "deepgrow_2d": {
      "type": "deepgrow",
      "labels": [
        "spleen",
        "right kidney",
        "left kidney",
        "liver",
        "stomach",
        "aorta",
        "inferior vena cava"
      ],
      "dimension": 2,
      "description": "A pre-trained DeepGrow model based on UNET",
      "config": {}
    },
"segmentation_spleen": {
      "type": "segmentation",
      "labels": {
        "spleen": 1
      },
      "dimension": 3,
      "description": "A pre-trained model for volumetric (3D) segmentation of the spleen from CT image",
      "config": {}
    },
"Histogram+GraphCut": {
      "type": "scribbles",
      "labels": {
        "spleen": 1
      },
      "dimension": 3,
      "description": "A post processing step with histogram-based GraphCut for Generic segmentation",
      "config": {
        "num_bins": 64,
        "lamda": 1,
        "sigma": 0.1
      }
    },
    "GMM+GraphCut": {
      "type": "scribbles",
      "labels": {
        "spleen": 1
      },
      "dimension": 3,
      "description": "A post processing step with GMM-based GraphCut for Generic segmentation",
      "config": {
        "num_mixtures": 20,
        "lamda": 5,
        "sigma": 0.5
      }

Related Objects
Search...

Event Timeline

a178n triaged this task as Normal priority.Oct 31 2022, 9:21 AM
a178n created this task.
a178n updated the task description. (Show Details)Oct 31 2022, 9:24 AM
  1. There is no indication in the JSON on which models are interactive click guided, scribble guided or automatic. Eg. Deepedit can be interactive click-guided or automatic, GMM+GraphCut is scribble guided, Segmentation Spleen is purely automatic. Hence, the JSON parameter type can be made more consistent for easy grouping.
a178n added a comment.Oct 31 2022, 9:31 AM
  1. Lack of consistency in models->labels entries. For some models eg. deepedit there are label ids along with names, effectively assuming a Map datatype and for others, eg. deepgrow_2d it's a list. of label names. This creates unnecessary corner cases while parsing using strongly typed languages, like C++.
a178n added a comment.Oct 31 2022, 9:38 AM
  1. The relevance of the labels list outside models? Perhaps, this entry can be used to describe all labels and their label ids used in the host app. And, models can then refer to label ids in their respective labels entries.
a178n edited projects, added MITK (v2023.04); removed MITK.Nov 29 2022, 10:26 AM
floca moved this task from Backlog to Cycle on the MITK (v2023.04) board.Nov 30 2022, 8:52 AM
a178n edited projects, added MITK; removed MITK (v2023.04).Dec 14 2022, 11:37 AM
a178n added a comment.Jan 31 2023, 6:56 PM
  1. Monai label response JSON vs segmentation image mask labels:

After a segmentation is processed, the response JSON, e.g.

"label_names":{"spleen": 1, "right kidney": 2, "left kidney": 3, "liver": 6, "stomach": 7, "aorta": 8, "inferior vena cava": 9, "background": 0}

ie. label id 4 & 5 are not present in the segmentation. However, the real nifty file has the label ids contiguous ie. 0 -7.
Slicer remaps the labels by 'left' shifting to making the labelling continuous, ie.

"liver": 4, "stomach": 5, "aorta": 6, "inferior vena cava": 7

This approach seems intuitive, but certainly needs confirmation and guarantee based on what algorithm makes the remapping of labels.

floca added a subscriber: floca.Feb 1 2023, 9:57 AM

I think you found already pretty many issues. That is far enough to trigger a conversation with the Monai label team, don't you think?

If you want I could also initiate it.

a178n added a comment.Feb 3 2023, 1:52 PM

Hi Ralf,

I would let you initiate it. So how do we proceed- we articulate them better & send mail OR we request for a meeting?

a178n added a comment.EditedMay 8 2023, 9:27 PM

A meeting occured with MonaiLabel devs on 08.05.2023 (6-7pm) to discuss the above-mentioned concerns.

Participants:
Ashis, Ralf, Sachidanand Alle, Andreas Diaz-Pinto, Syed-Ahmed Ahmedi, Y.Tang

Minutes:
Of all, issues 1&4 were discussed in detail. Here is a short summary.

Issue 1 (The taxonomy of model types):
Even though our reasoning was accepted for creating a new "type" name for click-based models &/or overhauling the "type" semantics, the current naming scheme would continue. They have historical reasons to keep the type names "deepedit" & "deepgrow". We have to live with it.
Here is code where types are defined: https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/interfaces/tasks/infer_v2.py#L25
Another discussion along similar lines: https://github.com/Project-MONAI/MONAILabel/issues/1173#issuecomment-1339786638

Issue 4: (Label Id mismatch)
This was accepted as a bug. Andreas will take this further and fix it. A PR will introduced soon and status would be updated.

MITK · MIC · IMSY · DKFZ