Page MenuHomePhabricator

Investigate MITK <--> python image transfer
Open, NormalPublic

Description

Corresponding email conversation:

Von: Floca, Ralf Omar <r.floca@Dkfz-Heidelberg.de>
Gesendet: Donnerstag, 16. April 2020 22:20
An: Neher, Peter <p.neher@Dkfz-Heidelberg.de>; Samuel Klistorner <samuel.klistorner@sydney.edu.au>; mitk-users@lists.sourceforge.net
Cc: Wasserthal, Jakob <j.wasserthal@dkfz-heidelberg.de>
Betreff: AW: Python and MITK

Hi,

short answer: sadly no. Not in the workbench 

Reason: The mechanism currently used to transfer the image data is very primitive to say it politly. This is already very long on my would be great if we finally would refactor that code part and really make the images transparent accessible. We also already have bindings for mitk::Images that are used by mitk Phenotyping when you want to use it directly out of python. But it is not leveraged in the workbench yet.
@Peter/Samuel maybe you want to have a look.

Best Ralf

Von: Neher, Peter
Gesendet: Donnerstag, 16. April 2020 08:23
An: Samuel Klistorner <samuel.klistorner@sydney.edu.au>; Floca, Ralf Omar <r.floca@Dkfz-Heidelberg.de>; mitk-users@lists.sourceforge.net
Cc: Wasserthal, Jakob <j.wasserthal@dkfz-heidelberg.de>
Betreff: AW: Python and MITK

Hi Samuel,

you mean that the changes you are making in python are directly reflected in the MITK image? Unfortunately I don't know if that is supposed to work. I will try and look into this, but I never really used the python console.

@Ralf, are you a python console user in MITK and maybe know about this or maybe you know someone who might know?

If nobody knows, I will have a look as soon as I can work from the office again. In general it is recommended to ask such questions via the users list so we can leverage the knowledge of the complete MITK developers group.

Best,
Peter


Von: Wasserthal, Jakob
Gesendet: Mittwoch, 15. April 2020 15:14
An: Neher, Peter
Cc: Samuel Klistorner
Betreff: AW: Python and MITK

Hi Peter,

Samuel from Sydney who visited our lab last year is using MITK+Python but is having some more questions. Since I am not familiar with this I refer him to you. So maybe you can help him.

Best
Jakob


Von: Samuel Klistorner <samuel.klistorner@sydney.edu.au>
Gesendet: Mittwoch, 15. April 2020 14:52
An: Wasserthal, Jakob
Betreff: Re: Python and MITK

Hi Jakob,

Great to hear from you. Glad to hear that you are doing well and Germany is coping well.

I did manage to build MITK Diffusion with Python. I worked out that I can drag images from the Data Manager to the Variable Stack and vice versa. My next question is, is there a way to update/refresh the Display (primarily the 3D window) using the Python console?
e.g. I have a SITK image in my python console to which I do some data manipulation on, and then automatically display that in the 3D window – such as erode or dilate a binary mask.

Id be happy to speak to Peter directly if its easier.

How are you going with your work? Any new papers coming out?

Cheers

Kind Regards,
Samuel Klistorner | Software Engineer
The University of Sydney, Save Sight Institute

Event Timeline

neher triaged this task as Normal priority.Apr 17 2020, 11:43 AM
neher created this task.
floca added a subscriber: goetzm.Apr 20 2020, 10:15 PM

@neher let me know when you are heading for this. It certaintly make sense to also have a look at the mitk::Image python bindings @goetzm already has established and to leverage them if possible.

floca added a comment.EditedApr 20 2020, 10:38 PM

Additional infos how to leverage swig wrappings in c++ apps that embed python. If we make it happen this way I think it would be optimal.

General: http://www.swig.org/Doc2.0/Modules.html#Modules_external_run_time

Example how to do it (swig answer is at the very end): https://stackoverflow.com/questions/8225934/exposing-a-c-class-instance-to-a-python-embedded-interpreter

foo.h:

#pragma once
#include <string>

struct Foo{
  Foo();
  Foo(std::string const& s);
  void doSomething();
  std::string m_string;
};

foo.cpp:

#include "foo.h"
#include <iostream>

Foo::Foo() {}

Foo::Foo(std::string const& s) : m_string(s) {}

void Foo::doSomething() {
  std::cout << "Foo:" << m_string << std::endl;
}

foo.i:

%module module
%{
  #include "foo.h"
%}

%include "std_string.i"
%include "foo.h"

Generate the usual SWIG wrapper together with a runtime

swig -python -c++ -Wall foo.i
swig -python -c++ -Wall -external-runtime runtime.h

Generate the SWIG module containing struct Foo:

g++ -fPIC -Wall -Wextra -shared -o _module.so foo_wrap.cxx foo.cpp -I/usr/include/python2.7 -lpython2.7

If you want to share type information across multiple modules, an argument -DSWIG_TYPE_TABLE=SomeName can be added.

Now, here is how a C++ instance of Foo is passed to the interpreter

#include "foo.h"
#include <Python.h>
#include "runtime.h"

int main(int argc, char **argv) {
  Py_Initialize();

  PyObject* syspath = PySys_GetObject((char*)"path");
  PyObject* pName = PyString_FromString((char*) ".");
  int err = PyList_Insert(syspath, 0, pName);
  Py_DECREF(pName);

  err = PySys_SetObject((char*) "path", syspath);

  PyObject *main, *module, *pInstance, *run, *setup;

  try {
    main = PyImport_ImportModule("__main__");
    err = PyRun_SimpleString(
        "a_foo = None\n"
        "\n"
        "def setup(a_foo_from_cxx):\n"
        "    print 'setup called with', a_foo_from_cxx\n"
        "    global a_foo\n"
        "    a_foo = a_foo_from_cxx\n"
        "\n"
        "def run():\n"
        "    a_foo.doSomething()\n"
        "\n"
        "print 'main module loaded'\n");

    // Load Python module
    module = PyImport_ImportModule("module");

    swig_type_info *pTypeInfo = nullptr;
    pTypeInfo = SWIG_TypeQuery("Foo *");

    Foo* pFoo = new Foo("Hello");
    int owned = 1;
    pInstance =
        SWIG_NewPointerObj(reinterpret_cast<void*>(pFoo), pTypeInfo, owned);

    setup = PyObject_GetAttrString(main, "setup");

    PyObject* result = PyObject_CallFunctionObjArgs(setup, pInstance, NULL);
    Py_DECREF(result);

    run = PyObject_GetAttrString(main, "run");

    result = PyObject_CallFunctionObjArgs(run, NULL);
    Py_DECREF(result);
  }
  catch (...) {
    PyErr_Print();
  }

  Py_DECREF(run);
  Py_DECREF(setup);
  Py_DECREF(pInstance);
  Py_DECREF(module);
  Py_DECREF(main);

  Py_Finalize();
  return 0;
}

The above can be compiled by:

g++ -Wall -Wextra -I/usr/include/python2.7 main.cpp foo.cpp -o main -lpython2.7
neher removed neher as the assignee of this task.Fri, Jul 24, 11:35 AM
kahl added a subscriber: kahl.Wed, Jul 29, 3:59 PM

I tried an approach like @floca suggested with the SWIG wrapping that is already available in MITK. Here is my code to transfer a MITK image from C++ to Python:

void CopyImage(mitk::Image *image)
{
  PyGILState_STATE gState = PyGILState_Ensure();
  PyObject *main = PyImport_ImportModule("__main__");

  int err = PyRun_SimpleString("import pyMITK\n"
                               "image = None\n"
                               "\n"
                               "def setup(image_from_cxx):\n"
                               "    print ('setup called with', image_from_cxx)\n"
                               "    global image\n"
                               "    image = image_from_cxx\n"
                               "\n"
                               "def run():\n"
                               "    nr_of_dimensions = image.GetDimension()\n"
                               "    print(str(nr_of_dimensions))\n"
                               "\n"
                               "print ('main module loaded')\n");

  swig_type_info *pTypeInfo = nullptr;
  pTypeInfo = SWIG_TypeQuery("_p_mitk__Image");
  int owned = 0;
  PyObject *pInstance = SWIG_NewPointerObj(reinterpret_cast<void *>(image), pTypeInfo, owned);

  PyObject *setup = PyObject_GetAttrString(main, "setup");
  PyObject *result = PyObject_CallFunctionObjArgs(setup, pInstance, NULL);

  PyObject *run = PyObject_GetAttrString(main, "run");
  result = PyObject_CallFunctionObjArgs(run, NULL);

  m_ThreadState = PyEval_SaveThread();
}

The problem is that currently, image.GetDimension() can't be called. The error I get is

AttributeError: 'SwigPyObject' object has no attribute 'GetDimension'

I guess that's because the object on the Python side has the type

<Swig Object of type 'std::vector< mitk::Image * >::value_type' at 0x0000027C30E3B2A0>

To be able to call functions on the object it should instead be the proxy, so e.g. of the type

<pyMITK.ImagePointer; proxy of <Swig Object of type 'mitk::Image::Pointer *' at 0x00000245250352D0> >

Currently, I don't understand how get a proxy object from the SWIG object to access it's functions. The only thing I found in the documentation (http://www.swig.org/Doc1.3/Python.html) is

In addition, SWIG_NewPointerObj() can automatically generate a proxy class object (if applicable).

If anyone has an idea how to solve this to get the proxy or sees a mistake I made, I would be really thankfull. I will also continue to search for a solution for this and will report if I found something new.

kahl added a comment.Wed, Jul 29, 4:56 PM

Seems like I figured it out. What I had to do was to change SWIG_TypeQuery("_p_mitk__Image") to SWIG_TypeQuery("_p_itk__SmartPointerT_mitk__Image_t") and change the type of image to mitk::Image::Pointer *image. When passing Pic3D now and printing the number of dimensions, 3 is printed as one would expect. I will further investigate how to interact with other libraries like SimpleITK.

hm, strange. Ad hoc I cannot give a hint. Good you found a workarround. For further investigation, may be mitk::Image has special wrapping instruction, that interferes with your approach.