diff --git a/Core/Documentation/Doxygen/Concepts/Interaction.dox b/Core/Documentation/Doxygen/Concepts/Interaction.dox index 99cf22ec85..1eb2bd879f 100644 --- a/Core/Documentation/Doxygen/Concepts/Interaction.dox +++ b/Core/Documentation/Doxygen/Concepts/Interaction.dox @@ -1,122 +1,120 @@ /** \page InteractionPage Interaction and Undo/Redo Concepts \section InteractionPage_Introduction Interaction in MITK \b Interaction is one of the most important tasks in clinically useful image processing software. Due to that, MITK has a special interaction concept, with which the developer can map the desired interaction. For a simple change in interaction he doesn't have to change the code. All information about the sequence of the interaction is stored in an XML-file that is loaded by the application during startup procedure at runtime. That even allows the storage of different interaction patterns, e.g. an interaction behaviour like in MS PowerPoint, in Adobe Photoshop or like the interaction behaviour on a medical image retrieval system. \section InteractionPage_Statemachines_Implementation Statemachines to implement Interaction The interaction in MITK is implemented with the concept of state machines (by Mealy). This concept allows to build the steps of interaction with different states, which each have different conditions, very alike the different interactions that may have to be build to develop medical imaging applications. Furthermore state machines can be implemented using object oriented programming (OOP). Due to that we can abstract from the section of code, that implements the interaction and focus on the sequence of interaction. What steps must the user do first before the program can compute a result? For example he has to declare three points in space first and these points are the input of a filter so only after the definition of the points, the filter can produce a result. The according interaction sequence will inform the filter after the third point is set and not before that. Now the filter after an adaption only needs two points as an input. The sequence of the interaction can be easily changed if it is build up as a sequence of objects and not hard implemented in a e.g. switch/case block. Or the user wants to add a point in the scene with the right mouse button instead of the left. Wouldn't it be nice to only change the definition of an interaction sequence rather than having to search through the code and changing every single if/else condition? \subsection InteractionPage_Statemachine State Machine -So a separation of the definition of a sequence in interaction and its implementation is a useful step in the development of an interactive application. +So a separation of the definition of a sequence in interaction and its implementation is a useful step in the development of an +interactive application. To be able to do that, we implemented the concept of state machines with several classes: States, Transitions and Actions define the interaction pattern. The state machine itself adds the handling of events, that are sent to it. \image html statemachine.jpg \subsubsection InteractionPage_ExampleA Example A: -A deterministic Mealy state machine has always one current state (here state 1). If an event 1 is sent to the state machine, it searches in its current state for a transition that waits for event 1 (here transition 1). The state machine finds transition 1, changes the current state to state2, cause the transition points to it and executes actions 1 and 2. Now state 2 is the current state. The state machine receives an event 2 and searches for an according transition. Transition 2 waits for event 2, and since the transition leads to state 2 the current state is not changed. Action 3 and 4 are executed. Now Event 3 gets send to the state machine but the state machine can't find an according transition in state 2. Only transition 2 , that waits for event 2 and transition 4, that waits for event 4 are defined in that state. So the state machine ignores the event and doesn't change the state or execute an action. Now the state machine receives an event 4 and finds transition 3. So now the current state changes from state 2 to state 1 and actions 5 and 1 are executed. +A deterministic Mealy state machine has always one current state (here state 1). If an event 1 is sent to the state machine, it searches in its current state for a transition that waits for event 1 (here transition 1). The state machine finds transition 1, changes the current state to state2, as the transition points to it and executes actions 1 and 2. Now state 2 is the current state. The state machine receives an event 2 and searches for an according transition. Transition 2 waits for event 2, and since the transition leads to state 2 the current state is not changed. Action 3 and 4 are executed. Now Event 3 gets send to the state machine but the state machine can't find an according transition in state 2. Only transition 2 , that waits for event 2 and transition 4, that waits for event 4 are defined in that state. So the state machine ignores the event and doesn't change the state or execute an action. Now the state machine receives an event 4 and finds transition 3. So now the current state changes from state 2 to state 1 and actions 5 and 1 are executed. Several actions can be defined in one transition. The execution of an action is the active part of the state machine. Here is where the state machine can make changes in data, e.g. add a Point into a list. See mitk::StateMachine, mitk::State, mitk::Event, mitk::Action, mitk::Transition, mitk::Interactor \subsection InteractionPage_GuardState Guard States -Guard States are a special kind of states. The action, that is executed after the state is set as current state, sends a new event to the state machine, which leads out of the guard state. So the state machine will only stay in a guard state for a short time. This kind of state is used to check different conditions, e.g. is an Object is picked of a set of points will be full after the addition of one point. +Guard States are a special kind of states. The action, that is executed after the state is set as current state, sends a new event to the state machine, which leads out of the guard state. So the state machine will only stay in a guard state for a short time. This kind of state is used to check different conditions, e.g. if an Object is picked or whether a set of points will be full after the addition of one point. \image html statemachine_guard.jpg \subsubsection InteractionPage_ExampleB Example B: -Event 1 is sent to the state machine. This leads the current state from state 1 into state check. The action 1 is executed. This action checks a condition and puts the result into a new event, that is sent and handled by the same (this) state machine. E.g. is the the object, the state machine handles the interaction, picked with the received mouse-coordinate? The event, that is generated, will be Yes or No. In case of event No, the state machine sets the current state back to state 1 and executes action 2. In case of event Yes, the state machine changes the state from state check into state 2 and executes action 3, which e.g. can select the object taken care of. +Event 1 is sent to the state machine. This leads the current state from state 1 into state check. The action 1 is executed. This action checks a condition and puts the result into a new event, that is sent and handled by the same (this) state machine. E.g. is the object picked with the received mouse-coordinate? The event, that is generated, will be Yes or No. In case of event No, the state machine sets the current state back to state 1 and executes action 2. In case of event Yes, the state machine changes the state from state check into state 2 and executes action 3, which e.g. can select said object. \subsection InteractionPage_XMLDefinitionStatemachine Definition of a State machine -Due to the separation of the definition of an interaction sequence and its implementation, the definition has to be archived somewhere, where the application can reach it during startup and build up all the objects (states, transitions and actions) that represent the sequence of a special interaction. In MITK, these informations are defined in an XML-file (usually in Interaction/mitkBaseInteraction/StateMachine.xml). +Due to the separation of the definition of an interaction sequence and its implementation, the definition has to be archived somewhere, where the application can reach it during startup and build up all the objects (states, transitions and actions) that represent the sequence of a special interaction. In MITK, these informations are defined in an XML-file (usually in Core/Code/Interactions/StateMachine.xml and Plugins/org.mitk.gui.qt.application/resources/StateMachine.xml, the duplication is currently needed due to design decisions). The structure is the following (from \ref InteractionPage_ExampleA) : \code - - - - - - - - + + + + + + + + \endcode -The identification numbers (ID) inside a state machine has to be unique. Each state machine has to have one state, that is defined as the start-state of that state machine. This means, initially, the current state of the state machine is the start-state. +The identification numbers (ID) inside a state machine have to be unique. Each state machine has to have one state, that is defined as the start-state of that state machine. This means, initially, the current state of the state machine is the start-state. The Event-Ids seen above are also defined in the statemachine.xml file. They specify a unique number for a combination of input-conditions (key, mouse and so on). See \ref InteractionPage_InteractionEvents for further informations. -So a state machine is defined through a few lines of xml-code, which is loaded at runtime. This allows us to change the sequence in interaction in the xml-file and restart the application, where the changes are applied right away. +The statemachine is compiled into an application at compile time. The definition of one single state machine is called the \a statemachine-pattern. Since this pattern is build up during startup with objects (states, transitions and actions) and these objects only hold information about what interaction may be done at the current state, we can also reuse the pattern. -Note, that you as a developer don't necessarily have to implement your own XML-File! We already have defined some interaction-patterns (e.g. for setting Points in 2D or 3D) which you can use and adapt. +\note You as a developer don't necessarily have to implement your own XML-File! We already have defined some interaction-patterns (e.g. for setting Points in 2D or 3D) which you can use and adapt. \subsubsection InteractionPage_ReusePattern Reuse of Interaction Patterns If we for example have a pattern called "pointset", which defines how the user can set different points into the scene and there is an instance of a state machine called "PointSetInteractor". This state machine has a pointer pointing to the current state in its assigned state machine pattern. Several events are send to the state machine, which moves the pointer from one state to the next, according to the transitions, and executes the actions, referenced in the transitions. But now a new instance of the class "PointSetInteractor" has to be build. So we reuse the pattern and let the current state pointer of the new object point to the start state of the pattern "pointset". The implementation of the actions is \b not done inside a class of the pattern (\a state, \a transition, \a action), it is done inside a state machine class (see the reference for mitkStatemachine). -\subsection InteractionPage_StartupInteraction Loading Statemachines at Runtime -During startup all state machine-patterns are checked for not having states, that have no transitions leading out of it (dead states), or states, that have no transitions leading into it (magic states) [Bin99 p. 183]. If an error is found, an error-message will be displayed \b and the state machine will be loaded. - -See mitk::StateMachineFactory - \subsection InteractionPage_InteractionEvents Events -During runtime, events are thrown from e.g. the mouse to the operation system, are then send to your graphical user interface and from there it has to be send to the MITK-object called \a mitkEventMapper. This class maps the events received with an internal list of all events that can be understood in MITK. The definition of all understandable events is also located in the XML-File the state machines are defined in. If the received event can be found in the list, an internal mitk-eventnumber is added to the event and send to the object \a mitkGlobalInteraction. +During runtime, events are thrown from e.g. the mouse to the operating system, are then send to your graphical user interface and from there it has to be send to the MITK-object called \a mitkEventMapper. This class maps the events received with an internal list of all events that can be understood in MITK. The definition of all understandable events is also located in the XML-File the state machines are defined in. If the received event can be found in the list, an internal mitk-eventnumber is added to the event and send to the object \a mitkGlobalInteraction. See mitk::Event, mitk::GlobalInteraction \subsection InteractionPage_GlobalInteraction GlobalInteraction -This object administers the transmission of events to registered state machines. There can be two kinds of state machines, the ones that are only listening and ones that also change data. Listening state machines are here called Listeners and state machines that also change data are called Interactors. \b Note that the discrimination between \a Listener and \a Interactor is only made in mitkGlobalInteraction. As Listener an object derived from class StateMachine can be added and removed from GlobalInteraction and as Interactor an object derived from class Interactor can be added and removed. See the interaction class diagram for further information. +This object administers the transmission of events to registered state machines. There can be two kinds of state machines, the ones that are only listening and ones that also change data. Listening state machines are here called Listeners and state machines that also change data are called Interactors. +\note The discrimination between \a Listener and \a Interactor is only made in mitkGlobalInteraction. +As Listener an object derived from class StateMachine can be added and removed from GlobalInteraction and as Interactor an object derived from class Interactor can be added and removed. See the interaction class diagram for further information. -To add or remove a state machine to the list of registered interactors, call \a AddInteractor or \a RemoveInteractor of \a GlobalInteraction or to add or remove a listener call \a AddListener of \a RemoveListener. Listeners are always provided with the events. Interactors shall only be provided with an event, if they can handle the event. Because of that the method CanHandleEvent is called, which is implemented in each Interactor. This method analyses the event and returns a value between 0 (can't handle event) and 1 (can handle the event). Information, that can help to calculate this jurisdiction can be the bounding box of the interacted data and the picked mouse-position stored in the event. +To add or remove a state machine to the list of registered interactors, call \a AddInteractor or \a RemoveInteractor of \a GlobalInteraction or to add or remove a listener call \a AddListener of \a RemoveListener. Listeners are always provided with the events. Interactors shall only be provided with an event, if they can handle the event. Because of that the method CanHandleEvent is called, which is implemented in each Interactor. This method analyses the event and returns a value between 0 (can't handle event) and 1 (Best choice to handle the event). Information, that can help to calculate this jurisdiction can be the bounding box of the interacted data and the picked mouse-position stored in the event. So after the object \a GlobalInteraction has received an event, it sends this event to all registered Listeners and then asks all registered Interactors through the method \a CanHandleEvent how good each Interactor can handle this event. The Interactor which can handle the event the best receives the event. Also see the documented code in \a mitkGlobalInteraction. To not ask all registered interactors on a new event, the class \a Interactor also has a mode, which can be one of the following: deselected, subselected (deprecated since HierarchicalInteraction has been removed), selected. These modes are also used for the event mechanism. If an interactor is in a state, where the user builds up a graphical object, it is likely that the following events are also for the build of the object. Here the interactor is in mode selected as long as the interactor couldn't handle an event. Then it changes to mode deselected. The mode changes are done in the actions through operations (described further down) and so declared inside the interaction pattern. See mitk::GlobalInteraction \subsection InteractionPage_Interactors Interactors The class \a Interactor is the superclass for all state machines, that solve the interaction for a single data-object. An example is the class \a mitkPointSetInteractor which handles the interaction of the data \a mitkPointSet. Inside the class \a mitkPointSetInteractor all actions, defined in the interaction-pattern "pointsetinteractor", are implemented. Inside the implementation of these actions (\a ExecuteAction(...) ), so called \a mitkOperations are created, filled with information and send to the \a mitkUndoController and to \a mitkOperactionActor (the data, the interaction is handled for). See mitk::Interactor \subsection InteractionPage_ExecOperations Executing Operations The class mitkOperation and its subclasses basically holds all information needed to execute a certain change of data. This change of data is only done inside the data-class itself, which is derived from the interface \a mitkOperationActor. Interactors handle the interaction through state-differentiation and combine all informations about the change in a \a mitkOperation and send this operation-object to the method ExecuteOperation (of data-class). Here the necessary data is extracted and then the change of data is performed. When the operation-object, here called do-operation, is created inside the method \a ExecuteAction (in class \a mitkInteractor), an undo-operation is also created and together with the do-operation stored in an object called \a OperationEvent. After the Interactor has sent the do-operation to the data, the operation-event-object then is sent to the instance of class \a mitkUndoController, which administrates the undo-mechanism. See mitk::Operation, mitk::OperationActor \subsection InteractionPage_UndoController UndoController The instance of class \a mitkUndoController administrates different Undo-Models. Currently implemented is a limited linear Undo. Only one Undo-Model can be activated at a time. The UndoController sends the received operation events further to the current Undo-Model, which then stores it according to the model. If the method \a Undo() of UndoController is called (e.g. Undo-Button pressed from ) the call is send to the current Undo-Model. Here the undo-operation from the last operation event in list is taken and send to the data, referenced in a pointer which is also stored in the operation-event. A call of the method \a Redo() is handled accordingly. See mitk::UndoController, mitk::LimitedLinearUndo \subsection InteractionPage_references References [Bin99] Robert V. Binder. Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley, 1999 */