Page MenuHomePhabricator

MONAI Label REST Response JSON issues, allegedly
Open, NormalPublic

Description

This issue is to keep track of issues in the JSON response format. Here is a sample response JSON:

{
  "name": "MONAILabel - Radiology",
  "description": "DeepLearning models for radiology",
  "version": "2.0",
  "labels": [
    "background",
    "spleen",
    "right kidney",
    "left kidney",
    "liver",
    "stomach",
    "aorta",
    "inferior vena cava",
    "gallbladder",
    "esophagus",
    "portal vein and splenic vein",
    "pancreas",
    "right adrenal gland",
    "left adrenal gland"
  ],
  "models": {
    "deepedit": {
      "type": "deepedit",
      "labels": {
        "spleen": 1,
        "right kidney": 2,
        "left kidney": 3,
        "liver": 6,
        "stomach": 7,
        "aorta": 8,
        "inferior vena cava": 9,
        "background": 0
      },
      "dimension": 3,
      "description": "A DeepEdit model for volumetric (3D) segmentation over 3D Images",
      "config": {}
    },
 "deepgrow_2d": {
      "type": "deepgrow",
      "labels": [
        "spleen",
        "right kidney",
        "left kidney",
        "liver",
        "stomach",
        "aorta",
        "inferior vena cava"
      ],
      "dimension": 2,
      "description": "A pre-trained DeepGrow model based on UNET",
      "config": {}
    },
"segmentation_spleen": {
      "type": "segmentation",
      "labels": {
        "spleen": 1
      },
      "dimension": 3,
      "description": "A pre-trained model for volumetric (3D) segmentation of the spleen from CT image",
      "config": {}
    },
"Histogram+GraphCut": {
      "type": "scribbles",
      "labels": {
        "spleen": 1
      },
      "dimension": 3,
      "description": "A post processing step with histogram-based GraphCut for Generic segmentation",
      "config": {
        "num_bins": 64,
        "lamda": 1,
        "sigma": 0.1
      }
    },
    "GMM+GraphCut": {
      "type": "scribbles",
      "labels": {
        "spleen": 1
      },
      "dimension": 3,
      "description": "A post processing step with GMM-based GraphCut for Generic segmentation",
      "config": {
        "num_mixtures": 20,
        "lamda": 5,
        "sigma": 0.5
      }

Event Timeline

a178n triaged this task as Normal priority.Oct 31 2022, 9:21 AM
a178n created this task.
  1. There is no indication in the JSON on which models are interactive click guided, scribble guided or automatic. Eg. Deepedit can be interactive click-guided or automatic, GMM+GraphCut is scribble guided, Segmentation Spleen is purely automatic. Hence, the JSON parameter type can be made more consistent for easy grouping.
  1. Lack of consistency in models->labels entries. For some models eg. deepedit there are label ids along with names, effectively assuming a Map datatype and for others, eg. deepgrow_2d it's a list. of label names. This creates unnecessary corner cases while parsing using strongly typed languages, like C++.
  1. The relevance of the labels list outside models? Perhaps, this entry can be used to describe all labels and their label ids used in the host app. And, models can then refer to label ids in their respective labels entries.
  1. Monai label response JSON vs segmentation image mask labels:

After a segmentation is processed, the response JSON, e.g.

"label_names":{"spleen": 1, "right kidney": 2, "left kidney": 3, "liver": 6, "stomach": 7, "aorta": 8, "inferior vena cava": 9, "background": 0}

ie. label id 4 & 5 are not present in the segmentation. However, the real nifty file has the label ids contiguous ie. 0 -7.
Slicer remaps the labels by 'left' shifting to making the labelling continuous, ie.

"liver": 4, "stomach": 5, "aorta": 6, "inferior vena cava": 7

This approach seems intuitive, but certainly needs confirmation and guarantee based on what algorithm makes the remapping of labels.

I think you found already pretty many issues. That is far enough to trigger a conversation with the Monai label team, don't you think?

If you want I could also initiate it.

Hi Ralf,

I would let you initiate it. So how do we proceed- we articulate them better & send mail OR we request for a meeting?