Page MenuHomePhabricator

Prepare changelog and migration guide for upcoming master merge
Closed, ResolvedPublic

Description

All closed MITK tasks tagged with Noteworthy since v2018.04.2 ordered by date updated (oldest first): /maniphest/query/Jm1.OzqbYTbF/

Related Objects

Mentioned Here
T27510: [MITKDoc] Add/revise developers documentation for new statistics back-/frontend
T22616: Store exact software version in properties when reading DICOM
T23741: Specify default UID generation method
T24302: Should we persiste the ID of Identifiable when loading/saving BaseDatas?
T25466: Enhance TemporoSpatialStringProperty to represent and store information more compact
T26980: MITK_CREATE_MACRO should prohibit dependencies against autoload modules
T27202: Support Planarfigures with fixed size
T27229: [ImageStatistic] Copy to clipboard from statistic table should be directly usabel in Excel
T27272: Double-clicking an entry in the new selection widget should select it and close the selection dialog
T27317: [ModelFit] Functionality to fuse 3D images into a 3D+t image
T27321: [DICOM reader] DICOM 3D+t image reader should condense only images of the same series.
T27322: [DICOM reader] DICOM image reader should only read dicom files from a directory that are in the same image block like the selected file.
T27372: DataNode uses itkCloneMacro(Self) but does not implement a clone method
T24215: Refactor classes to use the QmitkAbstractMultiWidget(Editor)
T24367: SetWidgetPlane-functions in QmitkStdMultiWidget not used
T25483: Apply level window to multiple selected images
T25688: Render window manager with custom multi widget: beta version
T26727: Change dependent files to work with the new concept
T26559: Improve Python 3 integration
T26653: Github pull request 263 (Forward vtkInformation objects to MITK mappers - allow for VTK depth peeling)
T26815: New multi render window widget architecture breaks rendering mode handling
T26936: QmitkStdMultiWidget render window names messed up
T26954: Use Visual Studio project folder feature for MITK and MITK extensions
T27019: Do not disable rendering tests on Linux by default
T27038: Allow CTest subprojects definition in MITK extensions
T27041: Allow to override MITK_REVISION_DESC
T27042: Short-range plan to prevent Differentials from unnecessarily building the superbuild
T27130: Remove Qt variables from Dashboard scripts
T27367: Graphical unit tests fail on some linux setups

Event Timeline

kislinsk triaged this task as High priority.Thu, Jun 18, 11:07 PM
kislinsk created this task.
kislinsk updated the task description. (Show Details)

I went through all Noteworthy tasks of the list above and summarized the ones were I was assigned (we should split it later into API changes and other noteworthy features):

API changes, migration, and other noteworthy features

Python (T26559)

  • MITK_USE_Python CMake variable was renamed to MITK_USE_Python3
  • On macOS, CMAKE_FRAMEWORK_PATH can be used to find a specific version of Python

Rendering mode, depth peeling, and anti-aliasing (T26653, T26815)

  • The rendering mode was removed as it represented different independent concepts
  • Remove any parameters of type mitk::BaseRenderer::RenderingMode::Type from function calls that generate corresponding compiler errors
  • Access to the rendering manager was unified to the static method mitk::RenderingManager::GetInstance()
  • Remove any parameters of type mitk::RenderingManager from function calls that generate corresponding compiler errors
  • Depth peeling was reintroduced for 3d renderers (including support for volumes) for highly improved rendering of overlapping translucent objects
  • Anti-aliasing was reintroduced (using Fast Approximate Anti-Aliasing (FXAA)) for all renderers by default and is managed by mitk::RenderingManager. mitk::RenderingTestHelper disables anti-aliasing for robust image comparisons in rendering tests.

Render window names (T26936)

  • The render window names stdmulti.widget[1-4] were changed to stdmulti.widget[0-3] (n to n-1)
  • For example, the standard 3d render window now is stdmulti.widget3 instead of stdmulti.widget4

Rendering tests on headless Linux clients (T27019, T27367)

  • Rendering tests on headless Linux clients can be easily activated with the MITK_XVFB_TESTING CMake cache variable
  • By default, this will prepend testdriver calls with xvfb-run --auto-servernum, resulting in having a virtual frame buffer available for rendering tests
  • On Ubuntu, install Xvfb with sudo apt install xvfb
  • Having a notoriously buggy history on different Linux distributions, the command can be customized with MITK_XVFB_TESTING_COMMAND, to call scripts with alternative names for example

Custom MITK revision description (T27041)

  • By default, MITK uses the latest Git tag like "v2018.04.2" as revision description in application window titles, about dialogs, and package names.
  • This can be overriden by the value of the MITK_CUSTOM_REVISION_DESC CMake cache variable

Auto-cleaning external projects on updates (T27042)

  • Updates of external projects like CTK, ITK, or VTK were difficult to get right in existing MITK builds as sometimes old build artifacts, header files, or configuration files had to be removed manually to successfully build MITK again
  • The MITK_AUTOCLEAN_EXTERNAL_PROJECTS CMake cache variable was introduced to automate this process, as soon as a corresponding CMakeExternals/*.cmake file was modified since the last build (OFF by default, but highly recommended)

CTest subprojects in MITK extensions (T27038)

  • MITK extensions can add CTest subprojects through a list called MITK_CTEST_SUBPROJECTS in their optional CMake/CTestSubprojectList.cmake file

Visual Studio project folders (T26954)

  • All MITK targets are now categorized in folders like Applications, Modules, or Plugins
  • MITK extension targets have their own root folders

Qt variables removed from dashboard scripts (T27130)

  • QT5_INSTALL_PREFIX and QT_BINARY_DIR were removed
  • Add the Qt path to the PATH environment variable instead, for example, right in the dashboard script with set(ENV{PATH} "$ENV{PATH}...")

Placeholder for external project updates, minimum CMake version, and Qt version

kalali added a comment.EditedMon, Jun 22, 3:07 PM

Level window (T25483)

  • Inside mitkLevelWindowManager the function RecaluclateLevelWindowForSelectedComponent was renamed to RecalculateLevelWindowForSelectedComponent
  • The level window slider now works on multiple selected nodes

Refactoring of QmitkStdMultiWidget and QmitkStdMultiWidgetEditor (T24367, T26727)

  • The level window slider is now controlled by the StdMultiWidgetEditor
  • The following functions inside the QmitkStdMultiWidget have been removed
    • SetWidgetPlaneModeToSlicing
    • SetWidgetPlaneModeToRotation
    • SetWidgetPlaneModeToSwivel
    • SetWidgetPlanesLocked
    • SetWidgetPlanesRotationLocked
    • SetWidgetPlanesRotationLinked
    • EnableStandardLevelWindow
    • DisableStandardLevelWindow
  • QmitkMultiWidgetLayoutManager has been added to change the layout of a multi widget
    • The layout manager supports more general layouts and reacts dynamically, depending on the selected / active render window widget (e.g. enlarge a specific render window widget, move a specific render window to the left, etc.).

Introduction of QmitkAbstractMultiWidget and QmitkAbstractMultiWidgetEditor (T24215)

  • QmitkFunctionality now works with a QmitkAbstractMultiWidget instead of a QmitkStdMultiWidget
  • QmitkIGTTrackingDataEvaluationView and QmitkIGTTrackingSemiAutomaticMeasurementView now work with a QmitkAbstractMultiWidget instead of a QmitkStdMultiWidget

Introduction of QmitkMxNMultiWidget and QmitkRenderWindowManager (T25688)

  • important changes are already mentioned in T25688: Render window manager with custom multi widget: beta version
  • pay attention, especially to:
    • moved implementation of GetRenderWindowPart (with IRenderWindowPartStrategy) from the abstract view to the workbenchutil
      • possible migration conflict: views that rely on the QmitkAbstractView to provide IRenderWindowPartStrategy values now need to access the workbenchutil
      • => e.g. inside the QmitkSegmentationView: GetRenderWindowPart(OPEN) becomes GetRenderWindowPart(mitk::WorkbenchUtil::OPEN)
floca added a comment.Mon, Jun 22, 9:57 PM

MITK_CREATE_MACRO prohibits dependencies to autoload modules (T26980)

  • Modules should never directly depend on autoload modules, but on the public interfaces that are provided by normal modules.
  • To enforce that MITK_CREATE_MACRO now checks the precondition and raises a CMAKE error if one tries to depend on an autoload module.

UX improvement for node selection dialog (T27272)

  • Double clicking a node (in single selection mode) or the selection of several nodes automatically accepts the selection (if valid) and closes the selection dialog.

Functionality to fuse 3D images into a 3D+t image (T27317)

  • New mitk::TemporalJoinImagesFilter added. That takes n 3D images and fuses them to a 3D+t image with n time steps. One may specify the time bounds of each step. Precondition: all images have the same geometry.
  • Added new comandline mini app Fuse3Dto4DImageMiniApp that offers this functionality.

TemporoSpatialStringProperties support compressed json serialization (T25466)

  • JSON format for TemporoSpatialStringProperties supports now compression (one may add the attributes tmax and zmax to indicated that a value is the same for all time points (in a slice) or even for all slices.
  • mitk::PropertyPersistenceSerialization::serializeTemporoSpatialStringPropertyToJSON and mitk::PropertyPersistenceDeserialization::deserializeTemporoSpatialStringPropertyToJSON now support the compression.

DICOM image reader improvements/stability (T27322, T27321, T22616)

  • DICOM image reader only reads DICOM files that are in the same image block as the selected file. Until the change the reader touched all files in the directory and loaded other image blocks in addition to the explicit selection. This was cause of some loading errors and slowed down the loading process in large directories. The fix introduced a change of behavior; the new behavior: If you drag'n'drop a file into the workbench, only the image the file is part of will be loaded.
NOTE: You can still have the old behavior (loading every image in the directory) by drag'n'drop the whole directory into the workbench.
  • DICOM image reader by default only condenses images with the same series instance UID into a 3D+t image. You may change to old behavior (condense everything where the geometry fits) in your reader configuration by setting onlyCondenseSameSeries to false in the configuration XML or by using the SetOnlyCondenseSameSeries() method of mitk::ThreeDnTDICOMSeriesReader.
  • DICOM image reader now stores meta information about the reading process in the loaded data instances to improve provenance tracking of processed data. The following informations are stored:
Property nameDescription
MITK.IO.reader.DICOM.configurationPath to the property containing the name of the dicom reader configuration used to read the data
MITK.IO.reader.DICOM.filesPath to the property containing the files the dicom reader used in a TemporoSpatialProperty
MITK.IO.reader.DICOM.PixelSpacingInterpretationStringPath to the property containing PixelSpacingInterpretationString for the read data
MITK.IO.reader.DICOM.PixelSpacingInterpretationPath to the property containing PixelSpacingInterpretation for the read data
MITK.IO.reader.DICOM.ReaderImplementationLevelStringPath to the property containing ReaderImplementationLevelString for the read data
MITK.IO.reader.DICOM.ReaderImplementationLevelPath to the property containing ReaderImplementationLevel for the read data
MITK.IO.reader.DICOM.GantyTiltCorrectedPath to the property containing the indicator of the gantry tilt was corrected when reading the data
MITK.IO.reader.DICOM.3D+tPath to the property containing the indicator of the data was read as 3D+t
MITK.IO.reader.DICOM.gdcmPath to the property containing the version of GDCM used to read the data
MITK.IO.reader.DICOM.dcmtkPath to the property containing the version of DCMTK used to read the data

Generic reader meta information (T22616)

  • Reader now store meta information about the reading process in the loaded data instances to improve provenance tracking of processed data. The following informations are stored:
Property nameDescription
MITK.IO.reader.descriptionPath to the property containing the name of the reader used
MITK.IO.reader.versionPath to the property containing the version of mitk used to read the data
MITK.IO.reader.mime.namePath to the property containing the mine name detected used to read the data
MITK.IO.reader.mime.categoryPath to the property containing the mime category detected to read the data
MITK.IO.reader.inputlocationPath to the property containing the input location used by the reader if loaded by file
MITK.IO.reader.options.*Path to the properties containing the reader options used to read the data

Statistics view: improvement and bug fixes (T27229)

  • Copy to clipoard now allows to directly paste the statistice as tables into Excel and alike (including correct conversion of numbers (using the locale settings of the current system)).
IMPORTANT: TODO add all other now changes of the statistics view

Planar figures with fixed size (T27202)

  • PlanarCircel and PlanarDoubleEllipse now allow the explizit setting of fixed diameters etc. (e.g. for experimental setups where a defined size is needed and is not allowed to be changed).

Removed Clone() method of DataNode (T27372)

  • Clone () was wrongly implemented and never used -> removed it.

Model fitting / pharmacokinetic modelling (T24244)

  • added simple semi quantitative 2-Step-Linear model.

UID Generation (T23741)

  • Switched implementation of mitk::UIDGenerator to boost::UUIDs.
IMPORTANT: Switching to UUIDs the possibility to define the digits/entropy of the UID makes no sense anymore. Thus the constructor signature of mitk::UIDGenerator is changed, which ,may imply a breaking change for code that uses that UIDGenerator.

BaseData UID serialization (T24302)

The UID provided by mitk::Identifiable is now serialized for all BaseData instances as long as the reader/writer does support the serialization of meta information (e.g. nrrd or mhd format). Also it is serialized in MITK project files.

NOTE: Please be aware that this UID is not a content based UID. Please see the informations in tasks T24302 and class description of mitk::Identifiable for more information.
floca added a comment.Mon, Jun 22, 9:57 PM
IMPORTANT: TODO add all other now changes of the statistics view. Still missing and weren't in the list.

Separate sections of feature highlights like

  • Rewritten Image Statistics
  • New Selection UI
  • MITK extensions with all "hidden" features

New Selection concept / UI (first draft)

The QmitkNodeSelectionDialog is the main widget to be used in order to select data nodes from the data storage. It is a pop-up-dialog that can be included in any other widget, such as the QmitkSingleNodeSelectionWidget or the QmitkMultiNodeSelectionWidget. It uses a tab widget that holds different data storage inspectors. A data storage inspector shows the (possibly filtered) data nodes of the data storage and allows to select one or more nodes. The selection can be accepted inside the node selection dialog and will be send to the including widget (i.e. the single node selection widget).

Generally the selection mechanism was designed to be a local selection mechanism per default. Local in this case means that any selection is only valid for the including widget / view / plugin. However, it is possible to use a QmitkModelViewSelectionConnector and a QmitkSelectionServiceConnector to connect the global selection bus with the selection model of a widget. There are two different modes:

  • QmitkSelectionServiceConnector::SetAsSelectionProvider allows to send a model selection, that has been sent by QmitkModelViewSelectionConnector::CurrentSelectionChanged
  • QmitkSelectionServiceConnector::AddPostSelectionListener allows to receive a bus selection, that will be received by QmitkModelViewSelectionConnector::SetCurrentSelection

For an example, see QmitkDataStorageViewerTest plugin
For more information, see Selection concept page

I think I moved and aligned everything into a draft of the official changelog (click on View Draft Version): 2020 Week 25 (Late June).

@floca The only thing missing is statistics as it seems that the text above is not yet complete.

New Image Statistics UI

The org.mitk.gui.qt.measurementtoolbox plugin provides a view called QmitkImageStatisticsView which has significantly changed.
It is now possible to compute and display the statistics of multiple images at once. Statistics will be generated for each image-mask pair with fitting geometry.
The statistic values are now displayed in a table instead of a list and can be directly copied to e.g. Excel.

Image statistics are now correctly cached to prevent recalculation for the same image (-mask pair).
The UI of the histogram has been improved and is now theme-aware.

NOTE: ImageStatistics with PlanarFigures

I think I moved and aligned everything into a draft of the official changelog (click on View Draft Version): 2020 Week 25 (Late June).

@floca The only thing missing is statistics as it seems that the text above is not yet complete.

Oh damn. Should I add it here or directly in the draft?
@kalali Thanks for starting. I will take over, to give you i.a. time to finish the review of D329 ;)

Just add it here.

floca added a comment.Mon, Jun 29, 5:28 PM

New Image Statistics (preview)

The image statistics (modules ImageStatistics and ImageStatisticsUI as well as the QmitkImageStatisticsView provided by the plugin org.mitk.gui.qt.measurementtoolbox) have been heavily refactored on the backend and the frontend side. The rework is not yet finished as the current state just ensures that we have no regression but does only to a little extend tap into the potential of the new infrastructure.
The following paragraphs briefly summarize what’s new.

Statistics backend

Statistics results are now stored (as helper) in the data storage; opposed to earlier where statistics where always (re)calculated on the fly. The statistics are attributed with meta information regarding the computation (e.g. number of bins or “ignore zeros”), but also the state of computation (e.g. if it is pending, WIP or up to date) and they are associated with the source data (e.g. image and segmentation) by the use of StatisticsToImageRelationRule and StatisticsToMaskRelationRule. The generation/updating is supported by the newly introduced class “family” based on QmitkDataGeneratorBase; e.g. QmitkImageStatisticsDataGenerator that ensures up to date statistics for any specified image-mask-pair. For more details visit the developer’s documentation T27510.
Amongst others the benefits of the new backend are:

  • Use/viewing of statistics becomes faster (especially for large masks or 3D+t images) as statistics can be reused.
  • Image statistics are now correctly cached to prevent recalculation for the same image (-mask pair).
  • The generation of statistics and its usage can be decoupled, so the same statistic can be used by several views.
  • The generation of statistics and there update is efficiently threaded in the background.
  • The infrastructure will be able to unify similar types of calculations (e.g. dose statistics for radiation therapy).
  • The introduction of the generation status allows UIs to be more responsive, as they now can indicated which values are currently (re)computed.

Statistics frontend

The ImageStatisticsView now uses the new QmitkImageStatisticsTreeModel in conjunction with Qt views to show statistic results. QmitkImageStatisticsTreeModel monitors the data storage and reacts to any change of statistics objects in the storage. Currently it allows a representation of the data similar to the old statistics tables. In future the view model concept will allow to represent the statistics data in a more flexible way (e.g. one specific statistic value like mean, images as rows and masks as column), this will be extended in the upcomming release commits.
For more details visit the developer’s documentation T27510.
Amongst others the benefits/additions of the new frontend are:

  • It is now possible to display the statistics of multiple images at once. Statistics will be generated for all selected image-mask pair with fitting geometry.
  • The statistic values are now displayed in a table instead of a list and can be directly copied to e.g. Excel.
  • Outdated values in the statistics (e.g. because the planar figure is moved or the segmentation is modified) table are now indicated and automatically updated in the view.
  • The UI of the histogram has been improved and is now theme-aware.
kislinsk closed this task as Resolved.Mon, Jun 29, 11:01 PM
kislinsk claimed this task.
kislinsk added a comment.EditedFri, Jul 3, 4:13 PM

Deleted branch release/T27486-2020-06-19.