Extensible Execution Component

From user's Wiki!
Jump to: navigation, search

This component executes processing filters directly by the main thread or in background, using multithreading. This requires a filter execution manager and GUI events communication strategy.

A specific kind of filter called dynamic processor, allows executing filters dynamically at run-time using an XML description. Using the XML, this component analyzes the inputs, outputs and parameters of the filter and calls the specific execution mode.

This XML description is used for the automatic GUI generation component that generates a graphical user interface for setting the parameters (DynWxAGUI). Furthermore, the XML description is used by WebServices Plugin to automatically deploy a web service.

Multithreading

Features

Multi threading capability that allows executing processing tasks in background, see progress, cancel tasks and show list of executed, pending and tasks being processed. A task is a filter being executed. The same filter can be executed several times, and different tasks will be created.

Multi threading

In the status bar (bottom right) you can see new options.

  • Processing icon: will be active when a task is being executed
  • Cancel button: allows to cancel the current task
  • Task list button: opens the list of all tasks. You can double click on a task to see more details
  • Status label: A label that shows the current task being executed
  • Progress bar

When GIMIAS exits, it will check if there are pending tasks and will ask you if you want to cancel them. GIMIAS will wait until all tasks are cancelled.

All data being used by a tak, cannot be unloaded or used by other task at the same time. You will receive an error message.

You can configure two parameters, if you go to the menu Edit->Preferences->Global:

  • Enable multi threading
  • Number of allowed concurrent tasks
Multi threading configuration

Design

To add multi threading feature to GIMIAS, we decided to use the thread pool pattern and use the Boost implementation. Multi threading allows to execute processing tasks in background, while the main GUI thread can process GUI events.

You can read more details in GmKernel#Processor_Manager.

When a processor is added to the ProcessorManager using Execute() method, this is the sequence of operations:

  1. If multi threading is enabled:
    1. Creates a copy of the processor
    2. If copy fails, checks if the processor is already running. If it is, throw an exception
    3. Creates a ProcessorThread and adds it to the list
    4. Add the ProcessorThread to the ProcessorExecutionQueue
  2. If multi threading is not enabled:
    1. Executes the processor

By default all filters have the multi threading option disabled. If you want to enable it, you need to call:

ProcessingWidget::UpdateProcessor( true );

or:

GetProcessor()->SetMultithreading( multithreading );
Core::Runtime::Kernel::GetProcessorManager()->Execute( GetProcessor().GetPointer() );

Note:

  • To execute multiple instances of a processor, it needs to implement the copy function, otherwise, it can only be executed once. The only Processor that has Copy( ) function implemented is DynProcessor.

Events

The most critical point when adding multi threading is that "no more than one thread can call GUI functions". The generic approach is that all worker threads will send events to the main GUI thread to update GUI components. We defined a set of wxWidgets events that will be sent to the main GUI thread:

  • WxUpdatePortEvent: Updates the inputs before processing and the ouputs after processing. This code should be executed by main GUI thread because the observers of rendering data can update the render windows. When the input is updated, it will switch to the needed type for the processor.
  • WxUpdateCallbackEvent: Send progress, status, exception messages to GUI thread and abort processing if user click cancel button

You can see the UML class diagram in GmFiltering.

When a filter is executed using multi threading, this is the sequence of performed steps:

  1. Update input data: BaseProcessor::GetProcessingData( ) is called. This function calls BaseFilterInputPort::UpdateInput( ), that creates a InputUpdater, initializes it, calls Update( ) and finally gets the data using GetData( ) method. The InputUpdater will create a new PortInvocation with multithreading capabilty or not, depending if multi threading has been enabled.
  2. Update processor: Processor is executed and generates new output data from input data
  3. Update output data: Output data is set as output of the processor and observers are notified. The function BaseProcessor::UpdateOutput( ) calls BaseFilterOutputPort::UpdateOutput( ), that creates a new OutputUpdater, initializes it and calls Update( ). OutputUpdater will create a new PortIncovation with multithreading capabilty or not, depending if multi threading has been enabled.

Note:

  • The updating of inputs and outputs will be done by the main GUI thread. You will see that the GUI is still blocked when adding the output to the rendering tree.

Execution progress

ITK Processor

There's an example in the processor ResampleProcessor of the SandboxPlugin. Before executing the filter, you should add an observer to itk::ProgressEvent():

// Call the function 
typedef itk::ResampleImageFilter<ImageType,ImageType> ResampleImageFilterType;
ResampleImageFilterType::Pointer filter = ResampleImageFilterType::New( );
filter->SetInput( itkInputImage );
filter->SetTransform( m_ScaleTransform );
filter->SetOutputParametersFromImage( itkInputImage );
// Add observer
itk::MemberCommand<SandboxPlugin::ResampleProcessor>::Pointer command;
command = itk::MemberCommand<SandboxPlugin::ResampleProcessor>::New();
command->SetCallbackFunction( this, &ResampleProcessor::UpdateProgress );
filter->AddObserver( itk::ProgressEvent(), command );
// Update
filter->Update( );

The observer member function will update the UpdateCallback instance of the processor with the progress percentage and the cancellation of the processor:

void SandboxPlugin::ResampleProcessor::UpdateProgress(
	itk::Object *caller,
	const itk::EventObject& event)
{
	itk::ProcessObject *processObject = (itk::ProcessObject*)caller;
	if (typeid(event) == typeid(itk::ProgressEvent)) 
	{
		GetUpdateCallback()->SetProgress( processObject->GetProgress() );
		GetUpdateCallback()->Modified();

		if ( GetUpdateCallback()->GetAbortProcessing() )
		{
			processObject->AbortGenerateDataOn();
		}
	}
}

VTK processor

There's an example in the processor ShapeScaleProcessor of the SandboxPlugin. Before executing the filter, you should add an observer to ProgressEvent:

vtkSmartPointer<vtkCallbackCommand> progressCallback;
progressCallback = vtkSmartPointer<vtkCallbackCommand>::New();
progressCallback->SetCallback(ProgressFunction);
progressCallback->SetClientData( GetUpdateCallback().GetPointer() );
transformFilter->AddObserver( vtkCommand::ProgressEvent, progressCallback);

The progress function will update the UpdateCallback:

void SandboxPlugin::ShapeScaleProcessor::ProgressFunction(
	vtkObject* caller, long unsigned int eventId, void* clientData, void* callData)
{
	vtkAlgorithm* filter = static_cast<vtkAlgorithm*>(caller);
	Core::UpdateCallback* callback = static_cast<Core::UpdateCallback*> (clientData);
	callback->SetProgress( filter->GetProgress() );
	callback->Modified();

	filter->SetAbortExecute( callback->GetAbortProcessing() );
}


ITK Command Line Plugin

You can find this example in the thirdParty/SLICERAPPS. The Command Line Plugin name is Threshold.

To add progress to the reader, threshold and writer filters you need to add these lines:

itk::PluginFilterWatcher watchReader1(reader1, "Read Volume", CLPProcessInformation);
itk::PluginFilterWatcher watchFilter(filter, "Threshold image", CLPProcessInformation);
itk::PluginFilterWatcher watchWriter(writer, "Write Volume", CLPProcessInformation);

3D+T processing

A filter can be executed multiple times for each 3D+T time step. To configure this option, there's a new button at the right of the input data selection control.

MultipleProcessing.png

Using this control, you can select the data you want to process.

Design

The configuration is stored on the BaseFilterInputPort class. There are two parameters:

  • UPDATE_ACCESS_MODE: Type of access when calling BaseFilter::Update()
    • UPDATE_ACCESS_SINGLE_TIME_STEP: Access a single time step (Default)
    • UPDATE_ACCESS_MULTIPLE_TIME_STEP: Access multiple time steps
  • SELECT_TIME_STEP_TYPE: Type of selection when using UPDATE_SINGLE_TIME_STEP
    • SELECT_ALL_TIME_STEPS: Select all time steps from input DataEntity
    • SELECT_SINGLE_TIME_STEP: Select single time step from input DataEntity (Default)

When BaseFilter::Update() function is called using ProcessingWidget::UpdateProcessor(), it will check the configuration of each input port. If the port is configured as single time step update access mode, depending on the selection and number of time steps of the input data, a loop iteration will be executed. For each loop iteration, a different time step will be automatically selected calling SetSelectedTimeStep().

This feature is available for BaseFilter class and Command Line Plugins.

Example

You can see an example in the file coreSignalTimePropagationProcessor.cxx.

In the constructor, you configure the input as multiple time steps.

Core::SignalTimePropagationProcessor::SignalTimePropagationProcessor( )
{
	SetNumberOfInputs( 2 );
	GetInputPort( 0 )->SetName( "Input data entity" );
	GetInputPort( 0 )->SetDataEntityType( Core::ImageTypeId );
	GetInputPort( 0 )->SetUpdateMode( BaseFilterInputPort::UPDATE_ACCESS_MULTIPLE_TIME_STEP );

	(...)

In the update function, you can retrieve the number of time steps and retrieve each time step using the GetProcessingData( )

void Core::SignalTimePropagationProcessor::Update()
{
	int numOfTimeSteps = GetInputDataEntity(0)->GetNumberOfTimeSteps();

	for ( int i = 0 ; i < numOfTimeSteps ; i++ )
	{
		Core::vtkImageDataPtr inputImage;
		GetProcessingData( 0, inputImage, i );
	}

	(...)

Dynamic Execution

The dynamic execution process is divided in 3 main steps:

  • Pre process: retrieves input data and converts it automatically to the needed format of the filter. This step uses the extensible data component and extensible input/output component.
  • Execution: executes the filter and handles error messages. The execution is implemented in the component DynLib
  • Post process: converts output data to the desired format specified and saves it locally or remotely. This step uses the extensible data component and extensible input/output component

The dynamic execution can be easily extended for local or remote processes. These are the available execution modes:

  • Command Line Plugin (CLP) based on 3D Slicer
    • Shared object plugin: The main advantage of this mode is to transfer data by memory
    • Executable plugin: Executes an application in a separate process
  • Direct DLL: Calls a DLL class directly without linking to it. The benefit of this mode is to extend the framework without developing code because a single XML text file is needed.
  • External Application: Calls an external application previously configured by the user. A custom application wizard guides the user to configure the command line arguments
  • SSH CLP: Executes a CLP remotelly using SSH protocol. All data is transferred automatically to the remote machine. You can read more in SSH Plugin
  • Taverna workflow: Executes a Taverna Workflow using data that is loaded in memory. To compose a workflow, the user can use the embedded window in Taverna plugin.
  • Unicore CLP: Executes a CLP in Unicore. You can read more in Unicore Plugin
  • GIMIAS Processor: This is the base filter class for processing data.
  • GIMIAS clinical workflow: A set of steps with graphical interface where the user can navigate forward and backward. You can read more in Clinical workflow component

You can more details about pre-process and post-process implementation in GmProcessors. To read more about execution, you will find it in DynLib.

Direct DLL execution

GIMIAS can be extended with direct DLL execution. You can create your own dynamic library that only depends on VTK and plug it into GIMIAS Framework. The DLL will be loaded at run-time and all the filters contained in the DLL will be accessible to the end user.

Example of vtkImaging.dll and vtkImageThreshold filter

You can also use the VTK filters directly, only writing the corresponding XML file. GIMIAS creates a dictionary of the exported functions of the DLL and automatically calls the needed functions, using an XML description of the filter. To add a new filter, you need to put the xml file into a plugin folder named “Filters”. You can take a look at the GIMIAS Framework plugins. When you press “Install files to Build Folder” in CSnake, these XML file will be automatically copied to the “Filters” folder of each plugin. At start up of Gimias, the Filters folder will be processed for all the selected plugins of the profile.

Here you can find a more detailed page on HowToAddSimpleDll.

Command Line Plugins

GIMIAS Framework can be extended using Command Line Plugins. This kind of plugins are composed of an XML file, command line application and a DLL. You can use the command line application independently of GIMIAS framework.

CommandLinePluginPic.png

The DLL will be created automatically and will be used by the Framework to pass input and output data. Images are passed to the DLL through memory. Other data types are passed using file disk input/output. All the Command Line Plugins will be placed into the folder “commandlineplugins” of your binary Gimias folder. Gimias will scan this folder at start up.

Here you can find a more detailed page on HowToAddCommandLinePlugin and on HowToDebugWithCommandLinePlugin.

3D Slicer plugin compatibility

Command Line Plugins are compatible with 3D Slicer framework.

3D Slicer

You can plug a 3D Slicer command line plugin into GIMIAS Framework. There are several 3D Slicer plugins available by default:

  • Filtering.Arithmetic: Add Images, Cast Image, Mask Image, Substract Image
  • Registration: Fast Affine Registration, Fast NonRigid BSpline Registration, Fast Rigid Registration, Linear Registration
  • Filtering: CheckerBoard Filter, Histogram Matching, Image Label Combine, Otsu Threshold, Resample Scalar Volume, Resample Scalar/Vector/DWI Volume, Threshold Image, Voting Binary Hole Filling, Zero Crossing Based Edge Detection Filter
  • Converter: Create a DICOM Series, Orient Images
  • Filtering.Denoising: Curvature Anisotropic Denoising, Gaussian Blur, Gradient Anisotropic Diffusion, Median Filter
  • Filtering.Morphology: Grayscale Fill Hole, Grayscale Grind Peak
  • Surface Models: Label Map Smoothing, Merge Models, Polydata to LabelMap
  • Segmentation: Otsu Threshold Segmentation, Simple Region Growing

Data Transfer

By default, when executing a command line plugin (CLP), GIMIAS transfers the images using a memory pointer.

The default execution mode is DLL (unless you check the option "Run as executable"). In DLL execution, GIMIAS will call directly the DLL of the CLP, sharing the same memory space. When the CLP has an input image, GIMIAS will transfer a pointer to the ITK image directly to the CLP, improving the performance of the execution.

This feature is implemented by the class itk::DataEntityIO and only allows to transfer a single 3D image.

Taverna interoperability

Taverna is an open source Workflow Management System written in Java. Interoperability between Taverna and GIMIAS can provide a lot of interesting functionalities for creating medical imaging workflows composed of several filters.

The Taverna plugin for GIMIAS allows discovering all GIMIAS command line plugins when using Taverna. You can read more here Taverna Plugin

External applications

A custom application wizard that will guide the user to configure the execution of external applications.

To execute an external application, XML description used by ModuleDescription is being used, adding some specific tags, like the executable file name. A new dialog allow to modify the general description, the parameters and the inputs/outputs of the executable. All these information will be stored in a single XML file. All XML files will be stored in the Application home path, in the folder "Descriptions".

When the XML is read, a ModuleDescription instance is created. The user can choose it as a processing tool and execute it. During the execution, a specific data transfer (DynDataTransferExternalApp) and Module execution class will be called (dynModuleExecutionExternalApp).

For execution of external applications, these tags have been added to the XML description:

  • executable: specify the executable file path
  • working directory: specify the working directory where to execute the application. In Windows, some executables need specific DLLs
  • format: for each parameter, this tag specify the format of the argument. For example "--grid 14". It's possible to use these variables that will be replaced by the real value during execution:
    • $(name): name of the parameter
    • $(value): value of the parameter

Custom Application Wizard

In order to configure a new external application, you should open the Custom Application Manager that is located in the menu Edit->Custom App Manager.

Custom application wizard

In this window you can manage new external applications: create, edit, delete, or import.

Here you can watch an example video on how to use this feature, here:

Custom application Warp

When editing a custom application, you will see this window:

Custom application wizard

It allows you to configure the general description of the executable, the parameters and the inputs/outputs.

The executable will appear in the Plugin selector, in the menu Edit->Preferences.

Custom application plugin selector

You can load the plugin and use it as a standard command line plugin, with automatic GUI generation. For example, for the external application warp.exe, this is the GUI:

Custom application Warp

To execute the application, you can configure the inputs/outputs and then press Apply. You can also expose the application using WebServices_Plugin

Processor execution

A processor can be executed using the XML description. For using this feature, you need to provide the XML file and implement the two functions for passing parameters:

  • void SetParameter( blTag::Pointer tag );
  • void GetParameter( blTag::Pointer tag );

You can see an example in ChangeOrientationProcessor. The function retrieves the parameter value from blTag and passes it to the filter.

Automated GUI Generation

The library dynWxAGUI allows generating the wxPanel of a filter automatically taking as input an XML description of the filter. You no longer need to use wxGlade to create simple GUI. For each parameter of the filter, a control will be added automatically to the wxPanel. You can also create your own style, with your own controls, overwriting the class wxControlFactory

AutomatedGuiPic1.png


UML class diagram and source code

You can read more details in GmProcessors and see example implementations in SSH Plugin or Taverna Plugin.

The following videos are using this component:

Go back to GIMIAS Architecture