Page MenuHomePhabricator

Static initialiser crash in various DLLs
Closed, InvalidPublic

Description

Following up on http://sourceforge.net/mailarchive/message.php?msg_id=31451146

This happens on Windows 7 and 8 (64-bit), compiled with Visual Studio 2010.

Using the ContourModel crash as an example:

  1. start up our GUI application
  2. eventually ContourModel.dll loads, runs its initialisers.
  3. it registers its factories
  4. other stuff happens
  5. ContourModel.dll is unloaded, the destructor of the initialiser is run.
  6. everything fine so far. application continues loading.
  7. ContourModel.dll is loaded again.
  8. Its initialiser runs, the constructor constructs its m_Factory member
  9. calling mitk::CoreObjectFactory::GetInstance()->RegisterExtraFactory(m_Factory) triggers a segfault

In most cases the segfault is due to a null pointer derefence. Sometimes it tries to call a null function pointer. Sometimes the pointer has a value like 0x0000000900000000.
The call stack in case of a crash is (beware of line-breaks):

ITKCommon-4.3.dll!itk::ObjectFactoryBase::CreateInstance(const char * itkclassname) Line 150 + 0x35 bytes C++

	ContourModel.dll!itk::ObjectFactory<mitk::ContourObjectFactory>::Create()  Line 60 + 0x22 bytes	C++
	ContourModel.dll!mitk::ContourObjectFactory::New()  Line 29 + 0x3c bytes	C++
	ContourModel.dll!RegisterContourObjectFactory::RegisterContourObjectFactory()  Line 142 + 0x31 bytes	C++
	ContourModel.dll!`dynamic initializer for 'registerContourObjectFactory''()  Line 154 + 0x21 bytes	C++
	msvcr100d.dll!_initterm()  + 0x2c bytes	
	ContourModel.dll!_CRT_INIT(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 289	C
	ContourModel.dll!__DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 506 + 0x13 bytes	C
	ContourModel.dll!_DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 477	C
	ntdll.dll!LdrpCallInitRoutine()  + 0x3f bytes	
	ntdll.dll!LdrpInitializeNode()  + 0x192 bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0x7b bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0xf43 bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0xf43 bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0xf43 bytes	
	ntdll.dll!LdrpPrepareModuleForExecution()  - 0x21 bytes	
	ntdll.dll!LdrpLoadDll()  - 0x108a5 bytes	
	ntdll.dll!LdrLoadDll()  + 0xa7 bytes	
	KernelBase.dll!LoadLibraryExW()  + 0xc3 bytes	
	QtCored4.dll!QLibraryPrivate::load_sys()  Line 126 + 0x43 bytes	C++
	QtCored4.dll!QLibraryPrivate::load()  Line 469 + 0xa bytes	C++
	QtCored4.dll!QLibraryPrivate::loadPlugin()  Line 519 + 0xa bytes	C++
	QtCored4.dll!QPluginLoader::load()  Line 228	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertArchive(QSharedPointer<ctkPluginArchiveSQL> * pa, QSqlQuery * query)  Line 432 + 0xe bytes	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertArchive(QSharedPointer<ctkPluginArchiveSQL> * pa)  Line 401 + 0x3e bytes	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertPlugin(const QUrl & location, const QString & localPath)  Line 379	C++
	CTKPluginFramework.dll!ctkPlugins::install(const QUrl & location, QIODevice * in)  Line 119 + 0x51 bytes	C++
	CTKPluginFramework.dll!ctkPluginContext::installPlugin(const QUrl & location, QIODevice * in)  Line 101 + 0x23 bytes	C++
	liborg_blueberry_osgi.dll!berry::InternalPlatform::Initialize(int & argc, char * * argv, Poco::Util::AbstractConfiguration * config)  Line 215 + 0x21 bytes	C++
	liborg_blueberry_osgi.dll!berry::Starter::Run(int & argc, char * * argv, Poco::Util::AbstractConfiguration * config)  Line 54	C++
	NiftyView.exe!ApplicationMain(int argc, char * * argv, QString * appName, QString * orgName, QString * applicationPlugin)  Line 179 + 0x1e bytes	C++
	NiftyView.exe!main(int argc, char * * argv)  Line 23	C++
	NiftyView.exe!__tmainCRTStartup()  Line 555 + 0x19 bytes	C
	NiftyView.exe!mainCRTStartup()  Line 371	C
	kernel32.dll!BaseThreadInitThunk()  + 0x1a bytes	
	ntdll.dll!RtlUserThreadStart()  + 0x21 bytes

Depending on which modules get preloaded the crash happens in different DLLs. Always the same type of segfault.
For example, preloading ContourModell results in:

ITKCommon-4.3.dll!itk::ObjectFactoryBase::CreateInstance(const char * itkclassname) Line 150 + 0x35 bytes C++

	PlanarFigure.dll!itk::ObjectFactory<mitk::PlanarFigureObjectFactory>::Create()  Line 60 + 0x22 bytes	C++
	PlanarFigure.dll!mitk::PlanarFigureObjectFactory::New()  Line 30 + 0x3c bytes	C++
	PlanarFigure.dll!RegisterPlanarFigureObjectFactory::RegisterPlanarFigureObjectFactory()  Line 131 + 0x31 bytes	C++
	PlanarFigure.dll!`dynamic initializer for 'registerPlanarFigureObjectFactory''()  Line 143 + 0x21 bytes	C++
	msvcr100d.dll!_initterm()  + 0x2c bytes	
	PlanarFigure.dll!_CRT_INIT(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 289	C
	PlanarFigure.dll!__DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 506 + 0x13 bytes	C
	PlanarFigure.dll!_DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 477	C
	ntdll.dll!LdrpCallInitRoutine()  + 0x3f bytes	
	ntdll.dll!LdrpInitializeNode()  + 0x192 bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0x7b bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0xf43 bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0xf43 bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0xf43 bytes	
	ntdll.dll!LdrpPrepareModuleForExecution()  - 0x21 bytes	
	ntdll.dll!LdrpLoadDll()  - 0x108a5 bytes	
	ntdll.dll!LdrLoadDll()  + 0xa7 bytes	
	KernelBase.dll!LoadLibraryExW()  + 0xc3 bytes	
	QtCored4.dll!QLibraryPrivate::load_sys()  Line 126 + 0x43 bytes	C++
	QtCored4.dll!QLibraryPrivate::load()  Line 469 + 0xa bytes	C++
	QtCored4.dll!QLibraryPrivate::loadPlugin()  Line 519 + 0xa bytes	C++
	QtCored4.dll!QPluginLoader::load()  Line 228	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertArchive(QSharedPointer<ctkPluginArchiveSQL> * pa, QSqlQuery * query)  Line 432 + 0xe bytes	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertArchive(QSharedPointer<ctkPluginArchiveSQL> * pa)  Line 401 + 0x3e bytes	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertPlugin(const QUrl & location, const QString & localPath)  Line 379	C++
	CTKPluginFramework.dll!ctkPlugins::install(const QUrl & location, QIODevice * in)  Line 119 + 0x51 bytes	C++
	CTKPluginFramework.dll!ctkPluginContext::installPlugin(const QUrl & location, QIODevice * in)  Line 101 + 0x23 bytes	C++
	liborg_blueberry_osgi.dll!berry::InternalPlatform::Initialize(int & argc, char * * argv, Poco::Util::AbstractConfiguration * config)  Line 215 + 0x21 bytes	C++
	liborg_blueberry_osgi.dll!berry::Starter::Run(int & argc, char * * argv, Poco::Util::AbstractConfiguration * config)  Line 54	C++
	NiftyView.exe!ApplicationMain(int argc, char * * argv, QString * appName, QString * orgName, QString * applicationPlugin)  Line 179 + 0x1e bytes	C++
	NiftyView.exe!main(int argc, char * * argv)  Line 23	C++
	NiftyView.exe!__tmainCRTStartup()  Line 555 + 0x19 bytes	C
	NiftyView.exe!mainCRTStartup()  Line 371	C
	kernel32.dll!BaseThreadInitThunk()  + 0x1a bytes	
	ntdll.dll!RtlUserThreadStart()  + 0x21 bytes

Preloading both PlanarFigure and ContourModel results in:

ITKCommon-4.3.dll!itk::ObjectFactoryBase::CreateInstance(const char * itkclassname) Line 150 + 0x35 bytes C++

	Segmentation.dll!itk::ObjectFactory<mitk::ToolManagerProvider>::Create()  Line 60 + 0x22 bytes	C++
	Segmentation.dll!mitk::ToolManagerProvider::New()  Line 67 + 0x3c bytes	C++
	Segmentation.dll!mitk::SegmentationModuleActivator::Load(us::ModuleContext * context)  Line 36 + 0xa bytes	C++
	CppMicroServices.dll!us::Module::Start()  Line 138	C++
	CppMicroServices.dll!us::ModuleRegistry::Register(us::ModuleInfo * info)  Line 132	C++
	Segmentation.dll!us::ModuleInitializer::Register()  Line 24 + 0x23 bytes	C++
	Segmentation.dll!us::ModuleInitializer::ModuleInitializer()  Line 24 + 0x167 bytes	C++
	Segmentation.dll!`dynamic initializer for '_InitializeModule''()  Line 24 + 0x21 bytes	C++
	msvcr100d.dll!_initterm()  + 0x2c bytes	
	Segmentation.dll!_CRT_INIT(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 289	C
	Segmentation.dll!__DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 506 + 0x13 bytes	C
	Segmentation.dll!_DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 477	C
	ntdll.dll!LdrpCallInitRoutine()  + 0x3f bytes	
	ntdll.dll!LdrpInitializeNode()  + 0x192 bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0x7b bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0xf43 bytes	
	ntdll.dll!LdrpInitializeGraph()  + 0xf43 bytes	
	ntdll.dll!LdrpPrepareModuleForExecution()  - 0x21 bytes	
	ntdll.dll!LdrpLoadDll()  - 0x108a5 bytes	
	ntdll.dll!LdrLoadDll()  + 0xa7 bytes	
	KernelBase.dll!LoadLibraryExW()  + 0xc3 bytes	
	QtCored4.dll!QLibraryPrivate::load_sys()  Line 126 + 0x43 bytes	C++
	QtCored4.dll!QLibraryPrivate::load()  Line 469 + 0xa bytes	C++
	QtCored4.dll!QLibraryPrivate::loadPlugin()  Line 519 + 0xa bytes	C++
	QtCored4.dll!QPluginLoader::load()  Line 228	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertArchive(QSharedPointer<ctkPluginArchiveSQL> * pa, QSqlQuery * query)  Line 432 + 0xe bytes	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertArchive(QSharedPointer<ctkPluginArchiveSQL> * pa)  Line 401 + 0x3e bytes	C++
	CTKPluginFramework.dll!ctkPluginStorageSQL::insertPlugin(const QUrl & location, const QString & localPath)  Line 379	C++
	CTKPluginFramework.dll!ctkPlugins::install(const QUrl & location, QIODevice * in)  Line 119 + 0x51 bytes	C++
	CTKPluginFramework.dll!ctkPluginContext::installPlugin(const QUrl & location, QIODevice * in)  Line 101 + 0x23 bytes	C++
	liborg_blueberry_osgi.dll!berry::InternalPlatform::Initialize(int & argc, char * * argv, Poco::Util::AbstractConfiguration * config)  Line 215 + 0x21 bytes	C++
	liborg_blueberry_osgi.dll!berry::Starter::Run(int & argc, char * * argv, Poco::Util::AbstractConfiguration * config)  Line 54	C++
	NiftyView.exe!ApplicationMain(int argc, char * * argv, QString * appName, QString * orgName, QString * applicationPlugin)  Line 179 + 0x1e bytes	C++
	NiftyView.exe!main(int argc, char * * argv)  Line 23	C++
	NiftyView.exe!__tmainCRTStartup()  Line 555 + 0x19 bytes	C
	NiftyView.exe!mainCRTStartup()  Line 371	C
	kernel32.dll!BaseThreadInitThunk()  + 0x1a bytes	
	ntdll.dll!RtlUserThreadStart()  + 0x21 bytes

Cache is deleted each time.

Event Timeline

Contents of the provisioning files:

START file:///C:/n/a/CTK-build/CTK-build/bin/liborg_commontk_configadmin.dll
START file:///C:/n/a/CTK-build/CTK-build/bin/liborg_commontk_eventadmin.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_osgi.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_compat.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_core_runtime.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_core_expressions.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_solstice_common.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_core_commands.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_ui.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_ui_qt.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_ui_qt_help.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_blueberry_ui_qt_log.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_core_services.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_common.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_planarfigure.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_core_ext.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_application.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_ext.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_common.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_stdmultiwidgeteditor.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_common_legacy.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_cmdlinemodules.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_datamanager.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_properties.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_dicom.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_imagenavigator.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_pointsetinteraction.dll
START file:///C:/n/a/MITK-build/MITK-build/bin/plugins/liborg_mitk_gui_qt_volumevisualization.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_gui_qt_commonapps.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_gui_qt_niftyview.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_snapshot.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_thumbnail.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_imagestatistics.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_midaseditor.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_niftyreg.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_gui_qt_common.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_imagelookuptables.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_affinetransform.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_surfaceextractor.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_gui_qt_commonlegacy.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_gui_qt_commonmidas.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_mitksegmentation.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_midasmorphologicalsegmentor.dll
START file:///C:/n/a/NifTK-build/bin/plugins/libuk_ac_ucl_cmic_midasgeneralsegmentor.dll

Thanks for the details.

Unfortunately, I still couldn't reproduce the crash. I used VS 2010 Express 32bit on Windows 7 (we do not have the Professional version with a 64 bit compiler) and played around with additional dependencies of plug-ins on the Segmenetation, ContourModel and PlanarFigure module. So I suspect that one of your plug-ins (not in the MITK distribution) is triggering the crash

So, could you please provide us with the following information:

1.) Try to identify the plug-in which when enabled leads to the crash. You can do so without changing the CMake configuration or re-building stuff by commenting out a specific set of plug-in in the NiftyView.provisioning file (use the '#' character in front of a line to comment it out). Make sure to also comment out plug-ins dependent on the already commented out plug-in. Doing this, you can do a binary search to find the first plug-in triggering a crash. Please send us the link-time dependencies of this plug-in (e.g. by sending the linker command line as shown in the Visuald Studio properties of that plug-in).

2.) After the crash brought up the VS debugger, please send us the list of currently loaded libraries, as shown in the "Modules" window of Visual Studio.

I we can identify the problem today, we could still integrate a fix in the upcoming release. Thanks!

We tracked down one of our MITK_CREATE_MODULE()-modules called niftkMIDAS that depends on mitk's Segmentation. When loading niftkMIDAS, it will lead to the crash described. If we hack our code to not load niftkMIDAS the GUI starts just fine.

We tried to introduce a dummy dependency on Segmentation.dll into one of our other MITK_CREATE_MODULE()-modules, called niftkIGI. Loading niftkIGI and hence loading Segmentation is all fine. No crash.

We'll try to figure out how niftkMIDAS differs from niftkIGI in terms of dependencies, load order, etc...

Fix SegmentationObjectFactory

I attached a patch for the Segmentation module which should fix the unloading of factories similar to the PlanarFigure and ContourModel modules. Additionally, this patch removes the registration of the ContourModelWriterFactory, which is already registerd in the ContourModel module. Maybe this fixes the crashes for your set-up.

If you can confirm that this patch works, I will apply it to the MITK master.

[f4197d]: Merge branch 'bug-16209-fix-segmentation-object-factory'

Merged commits:

2013-10-01 19:44:25 Sascha Zelzer [fcd7ed]
Fix the segmentation object factory registration.

Removed the registration of the ContourModel factories (they are
already registered in the ContourModel module) and fixed unregistering
the SegmentationObjectFactory when unloading the Segmentation module.

Thanks for the patch, sascha!
unfortunately, it didnt help.
Depending on what we preload, it keeps crashing with the mentioned callstacks.

The crash is always in ITKCommon-4.3.dll!itk::ObjectFactoryBase::CreateInstance(const char * itkclassname), different modules calling it.
Code snippet:

LightObject::Pointer
ObjectFactoryBase
::CreateInstance(const char *itkclassname)
{

ObjectFactoryBase::Initialize();

for ( FactoryListType::iterator
      i = ObjectFactoryBasePrivate::m_RegisteredFactories->begin();
      i != ObjectFactoryBasePrivate::m_RegisteredFactories->end(); ++i )
  {
  LightObject::Pointer newobject = ( *i )->CreateObject(itkclassname);
  if ( newobject )
    {
    newobject->Register();
    return newobject;
    }
  }
return 0;

}

itkclassname = "class mitk::PlanarFigureObjectFactory", for example. The line of crash is ( *i )->CreateObject(itkclassname). From what I can tell, the iterator i itself is fine. But digging through the assembly code, derefing it evaluates to a null pointer somewhere, on which CreateObject() is going to be called.

itk::ObjectFactoryBasePrivate::m_RegisteredFactories has 100 element, most of which point to recognisable classes. Maybe there are stale ones in there?

I was checking what stuff is in ObjectFactoryBasePrivate::m_RegisteredFactories (in itk somewhere). There are definitely corrupted entries.
The last valid entry has index 56 and points to mitk::CoordinateAxesDataWriterFactory (is one of ours?). Eventually entry 57 (and following) become corrupted. Before it becomes corrupted, entry 57 points to mitk::ContourModelIOFactory (it always points to this one during my debugging).

The entries after 57 become corrupted too, but where pointing to, for example:
mitk::AnnotationPropertySerializerFactory
mitk::BoolLookupTablePropertySerializerFactory
mitk::BoolPropertySerializerFactory
mitk::ClippingPropertySerializerFactory

I would guess something doesn't deregister properly? Maybe something else is holding on to a smartpointer that then prevents (automagic) cleanup?

Hi Johannes,

as this is urgent for MITK people, please could you try disabling

NifTKCoreObjectFactory

in

Code/Gui/MITK/Apps/NifTKApplication.h

and see if the crashes still occur?

Thanks

Matt

(In reply to Matt Clarkson from comment #9)

Hi Johannes,

as this is urgent for MITK people, please could you try disabling

NifTKCoreObjectFactory

in

Code/Gui/MITK/Apps/NifTKApplication.h

and see if the crashes still occur?

We tried that yesterday, doesn't change anything regarding the crash.
itk::ObjectFactoryBasePrivate::m_RegisteredFactories has fewer entries but there are still plenty corrupted ones.

The entries in itk::ObjectFactoryBasePrivate::m_RegisteredFactories that become corrupt (which then triggers the crash) are all from SceneSerializationBase.dll, i.e. the stuff in MITK-src\Modules\SceneSerializationBase\BasePropertySerializer.
I can't figure out though how that happens...

Thanks for all the digging. There are definitely stale references in the ITK object factory due to incorrect un-registration of factories during library unloading.

There still are a couple of libraries in MITK which do not unregister their stuff properly on un-load. I will fix these in a new branch and you could then test the changes. Unfortunately, this new work will not make it into the 2013.09 release.

When we experienced crashes ourselves, it was very hard to identify the library causing the problem. Probably because there were (are) multiple of these.

Hi Sascha,

is it worth documenting what is the correct pattern..... or else people will copy or invent an incorrect one. Obviously, if all the existing ones are correct, then people can only copy existing correct ones.... so that wont be so bad.

I am even offering to help with the documenting (just to prove I am not being rude :-)

Also, I am not sure I fully understand the Micro Services stuff, and how all this fits in.... so, maybe I should generalise my offer to help with documenting a summary of sorts.

So, what do you think is needed, to try and stop this repeating? And then I can help do some of it.

Matt

(In reply to Johannes Totz from comment #11)

The entries in itk::ObjectFactoryBasePrivate::m_RegisteredFactories that
become corrupt (which then triggers the crash) are all from
SceneSerializationBase.dll, i.e. the stuff in
MITK-src\Modules\SceneSerializationBase\BasePropertySerializer.
I can't figure out though how that happens...

Or maybe not...
There are more candidates:
liborg_blueberry_solstice_common.dll
liborg_blueberry_core_commands.dll
liborg_blueberry_core_expressions.dll
liborg_mitk_core_services.dll

(In reply to Matt Clarkson from comment #13)

Hi Sascha,

is it worth documenting what is the correct pattern..... or else people will
copy or invent an incorrect one. Obviously, if all the existing ones are
correct, then people can only copy existing correct ones.... so that wont be
so bad.

I am even offering to help with the documenting (just to prove I am not
being rude :-)

Also, I am not sure I fully understand the Micro Services stuff, and how all
this fits in.... so, maybe I should generalise my offer to help with
documenting a summary of sorts.

So, what do you think is needed, to try and stop this repeating? And then I
can help do some of it.

Matt

I agree, we should document the correct pattern. The ITK object factories are cumpersome to use (in my opinion) and we are working on replacing the usage within MITK with a micro - services approach. So the currently correct pattern will be different compared to the new approach, which will handle un-registering stuff much more gracefully. One part of that on-going project will be a proper documentation of the system anyway...

(In reply to Johannes Totz from comment #14)

(In reply to Johannes Totz from comment #11)

The entries in itk::ObjectFactoryBasePrivate::m_RegisteredFactories that
become corrupt (which then triggers the crash) are all from
SceneSerializationBase.dll, i.e. the stuff in
MITK-src\Modules\SceneSerializationBase\BasePropertySerializer.
I can't figure out though how that happens...

Or maybe not...
There are more candidates:
liborg_blueberry_solstice_common.dll
liborg_blueberry_core_commands.dll
liborg_blueberry_core_expressions.dll
liborg_mitk_core_services.dll

Finding the root cause for that problem turns out to be a difficult task... liborg_blueberry_solstice_common.dll for example does not even have an ITK dependency, so it cannot touch the ITK object factory stuff.

I found a workaround for us. Preload the following libraries:
libuk_ac_ucl_cmic_midasmorphologicalsegmentor
libuk_ac_ucl_cmic_midasgeneralsegmentor
libuk_ac_ucl_cmic_gui_qt_commonmidas
libuk_ac_ucl_cmic_niftyreg
That makes the GUI start fine.
Preloading any of these individually is not enough. I didn't test all combinations though.

I tried to preload everything at first. Adding libs to the preload string in reverse order they are unloaded (visual studio debugger log), assuming that unloading is what causes dead factory pointers.

Each four is needed. I found previously that switching on any of these individually makes the application crash.

Nice finding!

New remote branch pushed: bug-16209-unregister-object-factories

In the branch mentioned in comment #19 I tried to properly un-register all ITK IO factories during library unloading. It would be great if you could test that branch without using your pre-loading workaround.

Given the four libuk_ac_ucl_cmic_* plug-ins which need to be pre-loaded, it would also be interesting to determine the intersection of their sets of dependencies.

It did not resolve the crash, unfortunately. We will have a look at your fix and check if our four plugins or their dependencies need similar modifications.

That is unfortunate.

Another thing you could check for is the ITK_NO_IO_FACTORY_REGISTER_MANAGER CMake variable and pre-processor define. We set it to "1" within CMake/PackageDepends/MITK_ITK_Config.cmake which affects all MITK modules which declare an ITK dependency via PACKAGE_DEPENDS.

No setting it to "1" leads to registrations of ITK IO factories for each library which depends on ITK. These factories are not unregistered during library un-load. If you for example link to ITK your own way, or somehow this is not set to "1" (except for the MITK "Core" library), this could be the source of your problems.

Setting ITK_NO_IO_FACTORY_REGISTER_MANAGER to 1 under PackageDepends did not solve the problem.

I also modified our code so that it unregisters each IO factory properly. I might have made mistakes, but the app still crashes if those our plugins are not preloaded.

Is there a way to write a unit test that prints the list of the registered factories, loads and unloads a module then prints the list again?

When I enabled that option in PackageDepends, two files called itkImageIOFactoryRegisterManager.h and itkTransformIOFactoryRegisterManager.h have been generated. So far, so good.

However, I do not see our custom IO object factories in this file, only those that are in MITK-build/Code/.../itkImageIOFactoryRegisterManager.h.

Is that all right? Or do we need to do something additionally for this? We have e.g. a custom Analyze reader that is registered by our NifTKCoreObjectFactory.

I searched for the files that contains the "ImageIOFactoryRegisterManager" string in the build folder and found exactly those plugins that we need to preload so that the application does not crash at start:

C:\a-131008\NifTK-build\bin\plugins\Debug\

10/09/2013│     16712956│   A       │libit_unito_cim_intensityprofile.ilk
10/09/2013│     27503616│   A       │libit_unito_cim_intensityprofile.pdb
10/09/2013│     30219740│   A       │libuk_ac_ucl_cmic_midasgeneralsegmentor.ilk
10/09/2013│     50277376│   A       │libuk_ac_ucl_cmic_midasgeneralsegmentor.pdb
10/09/2013│     16265304│   A       │libuk_ac_ucl_cmic_midasmorphologicalsegmentor.ilk
10/09/2013│     26160128│   A       │libuk_ac_ucl_cmic_midasmorphologicalsegmentor.pdb
10/09/2013│     28669708│   A       │libuk_ac_ucl_cmic_niftyreg.ilk
10/09/2013│     37997568│   A       │libuk_ac_ucl_cmic_niftyreg.pdb

That is probably not a coincidence. :)

Is it all right that we have two itkImageIOFactoryRegisterManager.h files now, one in MITK and one in NifTK?

Hi there,

Im not sure if this is relevant, but on my Mac, I do:

Legolas:NifTK-build mattclarkson$ grep -rl ImageIOFactoryRegisterMana * | grep dylib
bin/libniftkCore.dylib
bin/libniftkITK.dylib
bin/libniftkMIDAS.dylib
bin/plugins/libuk_ac_ucl_cmic_midasgeneralsegmentor.dylib
bin/plugins/libuk_ac_ucl_cmic_midasmorphologicalsegmentor.dylib
bin/plugins/libuk_ac_ucl_cmic_niftyreg.dylib

We have libraries niftkCore, niftkITK, niftkMIDAS. Could these be messing things up too?

Also, if we search for any factory:

Legolas:Code mattclarkson$ pwd
/Users/mattclarkson/build/NifTK/Code
Legolas:Code mattclarkson$ find . -name "*Factory*"
./Gui/MITK/Modules/niftkCore/Algorithms/mitkNifTKCoreObjectFactory.cxx
./Gui/MITK/Modules/niftkCore/Algorithms/mitkNifTKCoreObjectFactory.h
./Gui/MITK/Modules/niftkCore/IO/itkPNMImageIOFactory.cxx
./Gui/MITK/Modules/niftkCore/IO/itkPNMImageIOFactory.h
./Gui/MITK/Modules/niftkCore/IO/mitkCoordinateAxesDataReaderFactory.cxx
./Gui/MITK/Modules/niftkCore/IO/mitkCoordinateAxesDataReaderFactory.h
./Gui/MITK/Modules/niftkCore/IO/mitkCoordinateAxesDataWriterFactory.cxx
./Gui/MITK/Modules/niftkCore/IO/mitkCoordinateAxesDataWriterFactory.h
./Gui/MITK/Modules/niftkCore/IO/mitkNifTKItkImageFileIOFactory.cxx
./Gui/MITK/Modules/niftkCore/IO/mitkNifTKItkImageFileIOFactory.h
./Libs/ITK/IO/itkAnalyzeImageIOFactory.cxx
./Libs/ITK/IO/itkAnalyzeImageIOFactory.h
./Libs/ITK/IO/itkDRCAnalyzeImageIOFactory.cxx
./Libs/ITK/IO/itkDRCAnalyzeImageIOFactory.h
./Libs/ITK/IO/itkINRImageIOFactory.cxx
./Libs/ITK/IO/itkINRImageIOFactory.h
./Libs/ITK/IO/itkNIFTKTransformIOFactory.cxx
./Libs/ITK/IO/itkNIFTKTransformIOFactory.h
./Libs/ITK/RegistrationToolbox/Construction/itkImageRegistrationFactory.h
./Libs/ITK/RegistrationToolbox/Construction/itkImageRegistrationFactory.txx

The Libs/ITK creates niftkITK which is direclty linked into niftkCore, niftkMIDAS, but without using anything to do with the PackageDepends stuff. Is this relevant? If that library does anything wrong with factories, it could be "propagated" to almost all other libraries. This might mean we should consider generating the Code/Libs/ITK, Code/Libs/VTK etc. libraries using the MITK mechanisms, so they are valid MITK Modules.

If I remember correctly I checked our factory stuff in itk::ObjectFactoryBasePrivate::m_RegisteredFactories was still present. I.e. they were not unregistered nor corrupted. (But I couldn't tell whether there were all of them; 56 in total.)

Thanks, Sascha, for considering this as a major bug.

We debugged this further with Johannes yesterday, and I believe we found the faulty module. It is SceneSerializationBase. If you look at mitkSerializerMacros.h, you'll see that the registered factories are never unregistered.

We added a debug print statement to itk::ObjectFactoryBase::RegisterFactory, and printed the name of the class being registered, as well the position where it was put into the array of factories.

At some point of the execution when ObjectFactoryBase iterates through the registered factories, the array of factories contains many invalid pointers, as Johannes described above. These items have probably been destructed but not removed from the list of factories. In our case the first corrupted item was at index 57, and from the print statements we could see that the last factory at that position was mitk::AnnotationPropertySerializerFactory:

ObjectFactoryBase::RegisterFactory(): factory type: class mitk::AnnotationProper
tySerializerFactory inserted at 57
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::BoolLookupTableP
ropertySerializerFactory inserted at 58
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::BoolPropertySeri
alizerFactory inserted at 59
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ClippingProperty
SerializerFactory inserted at 60
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ColorPropertySer
ializerFactory inserted at 61
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::DoublePropertySe
rializerFactory inserted at 62
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::EnumerationPrope
rtySerializerFactory inserted at 63

We thought that the problem is that some factory registered by a module is not unregistered when the module is unloaded, but it seems that the problems is a bit more complex. The SceneSerializationBase module is still loaded when the invalid memory access occurs. The end of the debug output window:

'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\MITK-build\bin\plugins\Debug\liborg_mitk_gui_common.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\MITK-build\bin\plugins\Debug\liborg_mitk_core_services.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\MITK-build\bin\Debug\Qmitk.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\VTK-build\bin\Debug\QVTK.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\Qt-4.8.5\lib\QtWebKitd4.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\VTK-build\bin\Debug\vtkViews.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\VTK-build\bin\Debug\vtkInfovis.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\VTK-build\bin\Debug\vtklibxml2.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\VTK-build\bin\Debug\vtkalglib.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\VTK-build\bin\Debug\vtkQtChart.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\MITK-build\bin\Debug\QmitkExt.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\MITK-build\bin\Debug\PlanarFigure.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\MITK-build\bin\Debug\SceneSerializationBase.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\Qwt-build\Debug\qwt.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\Qt-4.8.5\lib\QtSvgd4.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\Qxt-build\Debug\QxtGui.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\Qxt-build\Debug\QxtCore.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\MITK-build\bin\Debug\SceneSerialization.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\MITK-build\Poco-install\lib\PocoZipd.dll', Symbols loaded.
'NiftyIGI.exe': Loaded 'C:\a-131008\NifTK-build\bin\Debug\niftkCoreGui.dll', Symbols loaded.
First-chance exception at 0x002516e4 in NiftyIGI.exe: 0xC0000005: Access violation at location 0x00000000002516e4.

So, I think, those seven factories get destructed somehow, *before* the library is unloaded.

I was not correct, the factories would be unregistered when the module unloads, it is ensured by the destructor of a static object in mitk::SerializerMacros.h:95.

But as I noticed, the memory gets corrupted before the module is unloaded.

I am getting closer.

I added a debug print statement also to the itk::ObjectFactoryBase::UnRegisterFactory() function, and found something interesting.

First 46 internal factories are registered.
Then our 11 factories are registered.
Then there are cycles of registering and unregistering various factories as the modules are loaded and unloaded. Many cycles are go well, then the following happens:

ObjectFactoryBase::RegisterFactory(): factory type: class mitk::AnnotationPropertySerializerFactory insert at back, index: 57
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::BoolLookupTablePropertySerializerFactory insert at back, index: 58
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::BoolPropertySerializerFactory insert at back, index: 59
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ClippingPropertySerializerFactory insert at back, index: 60
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ColorPropertySerializerFactory insert at back, index: 61
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::DoublePropertySerializerFactory insert at back, index: 62
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::EnumerationPropertySerializerFactory insert at back, index: 63
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlaneOrientationPropertySerializerFactory insert at back, index: 64
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ShaderPropertySerializerFactory insert at back, index: 65
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkInterpolationPropertySerializerFactory insert at back, index: 66
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkRepresentationPropertySerializerFactory insert at back, index: 67
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkResliceInterpolationPropertySerializerFactory insert at back, index: 68
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkScalarModePropertySerializerFactory insert at back, index: 69
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkVolumeRenderingPropertySerializerFactory insert at back, index: 70
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ModalityPropertySerializerFactory insert at back, index: 71
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::RenderingModePropertySerializerFactory insert at back, index: 72
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PointSetShapePropertySerializerFactory insert at back, index: 73
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::FloatLookupTablePropertySerializerFactory insert at back, index: 74
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::FloatPropertySerializerFactory insert at back, index: 75
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::GroupTagPropertySerializerFactory insert at back, index: 76
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::IntLookupTablePropertySerializerFactory insert at back, index: 77
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::IntPropertySerializerFactory insert at back, index: 78
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::LevelWindowPropertySerializerFactory insert at back, index: 79
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::LookupTablePropertySerializerFactory insert at back, index: 80
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::Point3dPropertySerializerFactory insert at back, index: 81
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::Point3iPropertySerializerFactory insert at back, index: 82
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::Point4dPropertySerializerFactory insert at back, index: 83
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::StringLookupTablePropertySerializerFactory insert at back, index: 84
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::StringPropertySerializerFactory insert at back, index: 85
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::TransferFunctionPropertySerializerFactory insert at back, index: 86
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::Vector3DPropertySerializerFactory insert at back, index: 87
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarFigureIOFactory insert at back, index: 88
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarFigureSerializerFactory insert at back, index: 89
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarAngleSerializerFactory insert at back, index: 90
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarCircleSerializerFactory insert at back, index: 91
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarCrossSerializerFactory insert at back, index: 92
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarFourPointAngleSerializerFactory insert at back, index: 93
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarLineSerializerFactory insert at back, index: 94
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarPolygonSerializerFactory insert at back, index: 95
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarRectangleSerializerFactory insert at back, index: 96
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlanarEllipseSerializerFactory insert at back, index: 97

ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::JPEGImageIOFactory insert at back, index: 98
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::GDCMImageIOFactory insert at back, index: 99
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::BMPImageIOFactory insert at back, index: 100
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::LSMImageIOFactory insert at back, index: 101
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::PNGImageIOFactory insert at back, index: 102
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::TIFFImageIOFactory insert at back, index: 103
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::VTKImageIOFactory insert at back, index: 104
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::StimulateImageIOFactory insert at back, index: 105
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::BioRadImageIOFactory insert at back, index: 106
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::MetaImageIOFactory insert at back, index: 107
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::NiftiImageIOFactory insert at back, index: 108
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::NrrdImageIOFactory insert at back, index: 109
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::GiplImageIOFactory insert at back, index: 110
ObjectFactoryBase::RegisterFactoryInternal() factory class: class itk::HDF5ImageIOFactory insert at back, index: 111

ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarEllipseSerializerFactory at index 97
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarRectangleSerializerFactory at index 96
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarPolygonSerializerFactory at index 95
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarLineSerializerFactory at index 94
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarFourPointAngleSerializerFactory at index 93
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarCrossSerializerFactory at index 92
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarCircleSerializerFactory at index 91
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarAngleSerializerFactory at index 90
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarFigureSerializerFactory at index 89
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlanarFigureIOFactory at index 88
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::Vector3DPropertySerializerFactory at index 87
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::TransferFunctionPropertySerializerFactory at index 86
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::StringPropertySerializerFactory at index 85
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::StringLookupTablePropertySerializerFactory at index 84
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::Point4dPropertySerializerFactory at index 83
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::Point3iPropertySerializerFactory at index 82
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::Point3dPropertySerializerFactory at index 81
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::LookupTablePropertySerializerFactory at index 80
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::LevelWindowPropertySerializerFactory at index 79
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::IntPropertySerializerFactory at index 78
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::IntLookupTablePropertySerializerFactory at index 77
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::GroupTagPropertySerializerFactory at index 76
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::FloatPropertySerializerFactory at index 75
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::FloatLookupTablePropertySerializerFactory at index 74
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PointSetShapePropertySerializerFactory at index 73
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::RenderingModePropertySerializerFactory at index 72
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::ModalityPropertySerializerFactory at index 71
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::VtkVolumeRenderingPropertySerializerFactory at index 70
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::VtkScalarModePropertySerializerFactory at index 69
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::VtkResliceInterpolationPropertySerializerFactory at index 68
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::VtkRepresentationPropertySerializerFactory at index 67
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::VtkInterpolationPropertySerializerFactory at index 66
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::ShaderPropertySerializerFactory at index 65
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::PlaneOrientationPropertySerializerFactory at index 64
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::EnumerationPropertySerializerFactory at index 63
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::DoublePropertySerializerFactory at index 62
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::ColorPropertySerializerFactory at index 61
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::ClippingPropertySerializerFactory at index 60
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::BoolPropertySerializerFactory at index 59
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::BoolLookupTablePropertySerializerFactory at index 58
ObjectFactoryBase::UnRegisterFactory(): removing class mitk::AnnotationPropertySerializerFactory at index 57

ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PicFileIOFactory insert at back, index: 71
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::AnnotationPropertySerializerFactory insert at back, index: 72
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::BoolLookupTablePropertySerializerFactory insert at back, index: 73
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::BoolPropertySerializerFactory insert at back, index: 74
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ClippingPropertySerializerFactory insert at back, index: 75
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ColorPropertySerializerFactory insert at back, index: 76
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::DoublePropertySerializerFactory insert at back, index: 77
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::EnumerationPropertySerializerFactory insert at back, index: 78
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PlaneOrientationPropertySerializerFactory insert at back, index: 79
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ShaderPropertySerializerFactory insert at back, index: 80
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkInterpolationPropertySerializerFactory insert at back, index: 81
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkRepresentationPropertySerializerFactory insert at back, index: 82
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkResliceInterpolationPropertySerializerFactory insert at back, index: 83
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkScalarModePropertySerializerFactory insert at back, index: 84
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::VtkVolumeRenderingPropertySerializerFactory insert at back, index: 85
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::ModalityPropertySerializerFactory insert at back, index: 86
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::RenderingModePropertySerializerFactory insert at back, index: 87
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::PointSetShapePropertySerializerFactory insert at back, index: 88
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::FloatLookupTablePropertySerializerFactory insert at back, index: 89
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::FloatPropertySerializerFactory insert at back, index: 90
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::GroupTagPropertySerializerFactory insert at back, index: 91
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::IntLookupTablePropertySerializerFactory insert at back, index: 92
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::IntPropertySerializerFactory insert at back, index: 93
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::LevelWindowPropertySerializerFactory insert at back, index: 94
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::LookupTablePropertySerializerFactory insert at back, index: 95
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::Point3dPropertySerializerFactory insert at back, index: 96
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::Point3iPropertySerializerFactory insert at back, index: 97
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::Point4dPropertySerializerFactory insert at back, index: 98
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::StringLookupTablePropertySerializerFactory insert at back, index: 99
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::StringPropertySerializerFactory insert at back, index: 100
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::TransferFunctionPropertySerializerFactory insert at back, index: 101
ObjectFactoryBase::RegisterFactory(): factory type: class mitk::Vector3DPropertySerializerFactory insert at back, index: 102

First the scene serializer factories are registered (57-97).
Then various ITK IO factories are registered (98-111).
Then the scene serializer factories are unregistered.
Then the scene serializer factories are registered again, from 71, since the 14 IO factories have not been removed.

The invalid access happens right after this, because the 14 IO factories in the middle have been destructed meanwhile. We can see the garbage in the watch window.

Those 14 IO factories are registered by the ImageIOFactoryRegisterManager that is compiled into several dll-s, in our case MITKCore.dll, NifTKCore.dll and libit_unito_cim_intensityprofile.dll (our plugin), and probably into a few others, as the grep command showed. (See Matt's comment.)

The following happens:

  • MITKCore is loaded and its ImageIOFactoryRegisterManager registers these 14 IO factories.
  • NifTKCore is loaded and its ImageIOFactoryRegisterManager also registers these 14 IO factories.
  • Various plugins are loaded and unloaded and adds and removes their factories to this stack correctly.
  • The scene serializer module is loaded and registers its factories.
  • The intensity profile plugin is loaded and its ImageIOFactoryRegisterManager registers again the 14 IO factories.
  • The intensity profile is unloaded and it leaves its crap in the factory array.
  • The scene serializer module is unloaded and unregisters its factories. The crap is still at the end of the list.
  • The first time when the crap is accessed, the application crashes.

Does somebody know how to prevent compiling of this file to different dll's?

I guess it should be compiled only into one (niftkCore, I guess) that is loaded when the application starts and is unloaded until the program ends.

The best would be to fix the template in ITK which this file is generated from.

(In reply to Miklos Espak from comment #30)

Does somebody know how to prevent compiling of this file to different dll's?

I guess it should be compiled only into one (niftkCore, I guess) that is
loaded when the application starts and is unloaded until the program ends.

The best would be to fix the template in ITK which this file is generated
from.

Sorry, I didn't have much time for writing an exhaustive answer. Thanks for all the digging... To prevent the automatic registration of the ITK IO factories, you must make sure to set ITK_NO_IO_FACTORY_REGISTER_MANAGER to 1 when including ITK headers into your own library (this is what I was talking about in comment #22). For MITK, we make sure that this variable (which is passes as a preprocessor define to your cpp files) is only set to 0 for the "Mitk" core library. All other MITK modules have set it to "1" so we prevent multiple registrations of these factories (because ITK does not unregister them on library unload).

Since you seem to be linking your own libraries with ITK without setting ITK_NO_IO_FACTORY_REGISTER_MANAGER to 1 and additionally these libraries are linked to your CTK (MITK) plug-ins, the ITK IO factories get registered multiple times but not unregistered when the dependent CTK plug-ins are un-loaded.

Thanks for the explanation. I *did* enable that option after your post, but did not solve the problem.

Are you sure that the factory register manager is compiled only into the MITK core? If I do a file search in the bin folder, I found the "FactoryRegisterManager" string in these files:

DiffusionCore.pdb
ImageStatistics.pdb
IpPicSupport.pdb
Mitk.pdb
MitkDeformableRegistration.pdb
MitkDeformableRegistrationUI.pdb
MitkExt.pdb
mitkOpenCVVideoSupport.pdb
mitkToFProcessing.pdb

It's not a sophisticated way of debugging, I admit. However, it is also interesting that the crash happens after the intensity profile plugin is unloaded, and this plugin depends on ImageStatistics.

(In reply to Miklos Espak from comment #32)

Thanks for the explanation. I *did* enable that option after your post, but
did not solve the problem.

Did you verify in the VS Project settings of your libraries (which link to ITK) that the pre-processor define ITK_IO_FACTORY_REGISTER_MANAGER is *not* set?

Are you sure that the factory register manager is compiled only into the
MITK core? If I do a file search in the bin folder, I found the
"FactoryRegisterManager" string in these files:

DiffusionCore.pdb
ImageStatistics.pdb
IpPicSupport.pdb
Mitk.pdb
MitkDeformableRegistration.pdb
MitkDeformableRegistrationUI.pdb
MitkExt.pdb
mitkOpenCVVideoSupport.pdb
mitkToFProcessing.pdb

Pretty sure. On Windows, the ITK libraries are static (except for ITKCommon). So maybe there are references to same FactoryRegisterManager in the pdb files, but there shouldn*t be any registration code called when the DLLs are loaded.

It's not a sophisticated way of debugging, I admit. However, it is also
interesting that the crash happens after the intensity profile plugin is
unloaded, and this plugin depends on ImageStatistics.

We found that these kind of "relationships" are unfortunately not very reliable in the case of stale IO factory references... of course there could still be bugs in some MITK module but since the MITK Workbench works for all our use cases, I would concentrate on the NifTK libraries which are not MITK modules and have an ITK linker dependency.

(In reply to Sascha Zelzer from comment #33)

Are you sure that the factory register manager is compiled only into the
MITK core? If I do a file search in the bin folder, I found the
"FactoryRegisterManager" string in these files:

DiffusionCore.pdb
ImageStatistics.pdb
IpPicSupport.pdb
Mitk.pdb
MitkDeformableRegistration.pdb
MitkDeformableRegistrationUI.pdb
MitkExt.pdb
mitkOpenCVVideoSupport.pdb
mitkToFProcessing.pdb

Pretty sure. On Windows, the ITK libraries are static (except for
ITKCommon). [...]

Oh oh... this makes me nervous. Mixing DLLs and static libs is not gonna work on Windows. We really need to make sure everything is build as a shared library.

I just checked our ITK build (we are building it as an external project and then passing it down to MITK). There are indeed a whole pile of static libraries. We have BUILD_SHARED_LIBS defined but it doesn't seem to do much?

(In reply to Johannes Totz from comment #34)

Oh oh... this makes me nervous. Mixing DLLs and static libs is not gonna
work on Windows. We really need to make sure everything is build as a shared
library.

I just checked our ITK build (we are building it as an external project and
then passing it down to MITK). There are indeed a whole pile of static
libraries. We have BUILD_SHARED_LIBS defined but it doesn't seem to do much?

ITK has always been build using static libraries on Windows (except ITKCommon). You cannot change that. Linking to shared libraries is not in general "bad". ITK used to work like that (there aren't any "dangerous" static globals). The multiple registration problem for the ITK IO factories is not a static library problem. It happens on Linux too (there, everything is build as a shared library).

(In reply to Miklos Espak from comment #32)

Are you sure that the factory register manager is compiled only into the
MITK core? If I do a file search in the bin folder, I found the
"FactoryRegisterManager" string in these files:

DiffusionCore.pdb
[...]

The pdb files are debug symbols only. There will be all sorts of stuff/text in these files.

(In reply to Sascha Zelzer from comment #33)

Did you verify in the VS Project settings of your libraries (which link to
ITK) that the pre-processor define ITK_IO_FACTORY_REGISTER_MANAGER is *not*
set?

The preprocessor definitions for the niftkCore module, from VS:

WIN32;_WINDOWS;NOMINMAX;BOOST_LIB_PREFIX=;WIN32_LEAN_AND_MEAN;_WIN32_WINNT=0x0601;_DEBUG;QT_DLL;QT_XMLPATTERNS_LIB;QT_GUI_LIB;QT_XML_LIB;QT_NETWORK_LIB;QT_CORE_LIB;Boost_LIB_DIAGNOSTIC_DEFINITIONS;BOOST_ALL_NO_LIB;ITK_IO_FACTORY_REGISTER_MANAGER;BUILD_IGI;USE_NIFTYREG;_CRT_FAR_MAPPINGS_NO_DEPRECATE;_CRT_IS_WCTYPE_NO_DEPRECATE;_CRT_MANAGED_FP_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;_CRT_SECURE_NO_DEPRECATE;_CRT_SECURE_NO_DEPRECATE_GLOBALS;_CRT_SETERRORMODE_BEEP_SLEEP_NO_DEPRECATE;_CRT_TIME_FUNCTIONS_NO_DEPRECATE;_CRT_VCCLRIT_NO_DEPRECATE;_SCL_SECURE_NO_DEPRECATE;QWT_DLL;QT_DEBUG;CMAKE_INTDIR="Debug";niftkCore_EXPORTS;%(PreprocessorDefinitions)

Indeed, the variable is there. I thought the variable has to be set to 1 in each module except the MITK core, that sets it 0.

if(NOT DEFINED ITK_NO_IO_FACTORY_REGISTER_MANAGER)

set(ITK_NO_IO_FACTORY_REGISTER_MANAGER 1)

endif()

I will try to unset it.

Okay, please be careful and read the variable names exactly:

The CMake variable ITK_NO_IO_FACTORY_REGISTER_MANAGER controls wether the ${ITK_USE_FILE} script adds the ITK_IO_FACTORY_REGISTER_MANAGER pre-processor define or not (note the missing "NO"). Setting ITK_NO_IO_FACTORY_REGISTER_MANAGER to "1" before ${ITK_USE_FILE} is included will prevent the script from adding ITK_IO_FACTORY_REGISTER_MANAGER as a define.

[d72aca]: Merge branch 'bug-16209-unregister-object-factories'

Merged commits:

2013-10-07 22:34:06 Sascha Zelzer [e53039]
Properly unregister ITK IO factories.

Our problem has been solved by setting the ITK_NO_IO_FACTORY_REGISTER_MANAGER variable before including the ITK use file, and restructuring our CMake files a bit so that we can have different setting for our MITK based apps and for the command line apps.

Feel free to close this ticket.

Thanks for all your help in sorting this out!

I am happy to hear that the root cause was found and you could fix it.

kislinsk changed the task status from Invalid to Spite.Jun 27 2018, 1:31 PM
kislinsk added a project: Bulk Edit.
kislinsk changed the task status from Spite to Invalid.Jun 27 2018, 1:36 PM
kislinsk removed a project: Bulk Edit.