Multi Scale Visualization Plugin allows visualization of data sets in different space/time scales and provides a new interactive approach to navigate through the data.
MSV Plugin provides an interactive visualization framework that allows to navigate through the data using only the rendering window. This approach facilitates the navigation by using markers in the scene, small preview images with a brief description attached to each data object and by creating specific views. It also allows browsing and zooming into the selected object. This approach has similarities to Google Earth and facilitates the navigation in different spatial and temporal scales.
As an example of using this functionality, the datasets collected in the project MSV have been used. These examples can be accessed publicly and can be retrieved from the Sample Data section of the GIMIAS web site. The root dataset is a virtual human, where all the other datasets are hanged. The datasets are organized in different body regions: Head, Heart, Lungs, ... In each region, all related datasets are hanged.
A simple use case for navigation is explained bellow:
- User loads the example dataset collection and the virtual human is visualized
- User moves the mouse pointer over the Virtual Human marker and a tool tip is shown with a brief information
- User clicks on the Virtual Human marker and a brief description window is shown. The user reads a more detailed information or click on the preview image to zoom into the data.
- User clicks on Browse and a new window is shown where he can see all children data with preview images
- User clicks on the Head preview image to zoom into it. He can see all markers that are hanging on the Head: Cerebral Aneurysm, SISCOM and DTI
- User clicks on the Cerebral Aneurysm and later clicks on the Description to get more information about this case. User can come back to the Browse window to see the follow up for this aneurysm case: Aneurysm treatment, Several days after treatment and Six months after treatment.
- User clicks on the Aneurysm treatment session. The title is grey because this session is not loaded yet. User clicks on the preview image to load it. User can visualize all data: 3DRA image, aneurysm geometry or the CFD simulation
- User can get more information about a specific data object clicking on any of the markers
- Once satisfied, the user can navigate to another session and inspect it using the navigation links at the top of the browsing view
Navigation window has four modes:
- Data editing
Description Shows a brief description of the selected data with a preview image. If the window size is not enough to show all text, a read more link will appear. Clicking on the preview image allows navigating to the data.
Data editing User can edit the description and the preview image. The preview image can be captured from the current rendering view or loaded from disk.
Browsing Allows browsing through the data. This mode has a navigation link at the bottom showing all parent sessions. At the left of the title, a small preview image allows navigation to the current selected data. Finally, a table with all children data with previews images allows browsing and navigation to a specific data.
Views Allows managing the views for the current session. A view stores the configuration of the working area with the 3D camera parameters and the visual properties for all data objects of the current session. For example, for an aneurysm dataset, one view can show the surface mesh and the image and another view can show the simulation results.
A view can be added, edited and removed.
- User loads "Virtual Human" session. This is the main navigation object
- User navigates to the Head and adds a new child session called "SISCOM"
- User loads two images to this session: MRI image and a PET image.
- User uses the "Transformation Box Widget" tool to translate the data to the correct position referenced to the Virtual human. First, it selects the first data object and uses the middle mouse button to translate the bounding box. Later it selects the parent session node to apply this transformation to all children.
- User edits the data description and takes a new preview image if the default preview image is not enough. User can also clean the preview image or load it from disk
- Once all description is filled, user navigates to the SISCOM session node and saves it.
MSV Plugin registers a new working area composed of a specialized Rendering Tree (msvRenderingTree) and Render Window (msvMultiRenderWindow). This approach uses the VTK buttons to visualize a marker for each data object. When the user moves the mouse over the marker, a tool tip is shown (wxHtmlTooltipDialog). When the user clicks on the marker, a new window is shown with the description and the allowed actions. These windows are using an HTML page to show the information.
The rendering tree only calls the render window to show/hide the marker. The render window observes the Data Tree to manage the VTK buttons attached to each DataEntity.
The class msvWxVTKButtonsManager manages all VTK buttons and updates the buttons properties from camera or DataEntity.
Each button is represented by the class msvWxVTKButtons. It manages the VTK button and uses the class msvWxVTKAnimate to zoom into the objects.
The class msvWxHtmlWindow is responsible to build the HTML contents depending on the current mode. The class wxHtmlTooltipDialog is the main window that shows the contents to the user. It also manages the current mode and the events triggered by the HTML page.
When the edit data mode is selected, the window msvVTKBtnEdit is shown. When edit view mode is selected, the window msvVTKBtnEditView is shown.
All data is divided in sessions. When navigating, only one session is loaded into memory by default. When saving the data into disk, the sessions are written in sub folders to better organize the dataset collection.
A preview image (Core::DataEntityPreview) is attached to each DataEntity. A specific implementation called Core::MITKPreview using wxWidgets is registed in MITKPlugin.
The preview image is automatically loaded from disk when reading the data and saved into the same folder with the name "prev<name>.png". When taking a snapshot, the working area is captured and saved into an image of 80x80. This image will be stored in a temporary folder until the user saves the data to the final location in disk. This temporary image can be used for displaying it in the HTML window. If data object has no preview image, a default one will be created automatically.
Another information that is gathered when taking a snapshot are the bounds of the 3D view and the camera parameters. This allows to zoom into a specific data.
Each Session uses a blTagMap as processing data to store the children information: Data reference and the Views. The Data tag contains a reference to all children files, while the Views tag contains all views configuration. A view stores the working area name, the rendering properties of all data objects and the 3D camera parameters. The class msvViewManager is responsible to manage all views.
The views are stored on disk in a subfolder called "Views". It stores an xml file with all configuration (.gvw) and the preview image (.png).
- Register itk::Image at reader or another plugin
- Show VPH at the bottom in a small window like ITK-SNAP
- Allow loading a specific timestep window for EEG-fMRI
- Pass current time step to DataEntity
- Allow loading a specific simulation time step
- Allow loading a specific image region
- Animation opacity when camera moves forward and set it to 0 at the end
- Allow only VPH rotation interaction
- Use HTTP/PACS/XNAT servers to retrieve data and save in local cache during navigation
- Show warning when reusing same name for session and child data
- Clean old data when renaming
- Use HTML for context menu
- Add subclass of HTML cell to create browser that highlights current item and allows clicking on all the table cell
- Show space level for each dataset
- Scale control to move through 4 scales: body, organ, tissue, cell