Use Case Human Body Visualisation

Overview

Research areas in online medical education are varied and include among others: content creation, interaction methods, practical application and virtual patients. It is within this context of virtual patient we believe that the use of the remote platform to display the human body in 3D and in real-time can provide a significant advance for the training of medical students and also for the continuous medical training. The problem that arises is the size of these files that is incompatible with the memory of tablets and other devices that don’t have a powerful graphic card.

Platform short description

A very interesting aspect of the remote rendering is the ability to address certain issues on the security of medical data. In fact, the data from a scanner or from an MRI which will then be reconstructed in 3D (or only in map/2D sections) and then streamed to a terminal, doesn’t allow the local copy. The chosen approach for remote rendering will be based on a hybrid solution according to the transmission conditions and the capabilities of the user’s terminal. The objective is to dynamically change the representation method of a 3D medical object being visualised by a user with respect to the variations in the network quality. 

Use Case 1: 3D collection creation

Actors: Power-user; Conversion server; Storage server
Goals: contributes to build the 3D objects database that will be accessed by the end user
Events: Authentication, (2) Populate the 3D database (i.e. upload the 3D medical objects) and (3) Create snapshots of the 3D objects for the web platform
Description: The first requirement for our use case is to have all the 3D objects in a data base on the server. In order to populate the data base we need to follow the following steps: create the 3D objects, upload the 3D objects, convert them in MPEG-4 media files and store them on the server. This process is described in the figure bellow.

Img1.png

Use Case 2: Access and visualize 3D objects

Actors: End-user; virtual machine; rendering engine;
Goals: Access to 3D medical objects database: visualization and interaction
Events: (1) Open an account, (2) Access 3D objects in the browser, (3) Install plugin, (4) Choose a 3D content, (5) Visualize the object: 3D player and Remote rendering and (6) Interact with the object
Description: The following diagram illustrates the typical usage scenario of the system. On the client machine, a player is executed which connects to the remote server and receives the rendered video stream. Commands and controls are streamed back to the server which processes them and updates the state of the demo.

img2.png

On the server side, OGRE is the 3D application serving to execute and render the game. It contains a demonstration 3D content, a rendering engine, the video streaming plug-in and the network input system. On the client side, the video player is Osmo4 (GPAC) and is responsible to receive, decode and display the video stream from the remote server. The player also captures and send to the server the input commands. The following activity diagram illustrates the basic and the alternate flow of the use case:

HBVFLows.png

Requirements

The system has to be capable to analyze the transmission conditions and the capabilities of the user terminal in order to take the decision whether to display the 3D content directly into the end-user's player or to start the remote rendering engine and (2) The system must be able to transport big data objects.

Technical requirements

The chain has two main paths:

  1. Transmission of the 3D graphics object
  2. Remote rendering

Requirements for 3D transmission

Hardware : The end user client should have a 3D graphics hardware acceleration. Compression of the 3D graphics assets are needed to support progressive download. There are no constrains on the resolution of the 3D object, however with respect to the characteristics of the end user terminal, the server will take the decision for the path 4.a or 4.b.
Software : On the client side the only requirements are to have the GPAC player available for download and a web-browser (preferably Firefox).

Requirements for Remote rendering

The high level of details in the texture to be manipulated requires a high video quality and a high spatial resolution. Hence, coupled with the fact that current generation of tablets are natively able to handle applications with an HD/720p (i.e. 1280x720 pixels) resolution, we will consider it as a basis requirement.  More resolution (1080P) will certainly become the expected level at the time of the project. Moreover, the spatial resolution will have to be well-controlled in order to insure a visual comfort to the end-user manipulating conjointly 3D and 2D "object".

End-to-end latency and Temporal resolution are less constrained by the considered use case itself. Thus, a latency of 100-150ms coupled with a frame rate of 10-15 fps seems to be a good basis.
On the Virtual Cluster side the list of required packages includes: the rendering engine Ogre3D, the CUDA architecture and the libraries for 3D reconstruction.


This wiki is licensed under a Creative Commons 2.0 license
XWiki Enterprise 5.4.6 - Documentation - Legal Notice

Site maintained by