Multi-Resolution 3D Human Brain

Visualization through Streaming

Vani V
1
, Pradeep Kumar R
2
and Mohan S
3


1
Department of Information Technology, Dr.N.G.P.IT, Affiliated to Anna University,
Coimbatore, India.
2
Department of Computer Science Engineering, Adithya IT, Affiliated to
Anna University, Coimbatore, India.
3
Department of Computer Science Engineering, Dr.N.G.P. IT, Affiliated to
Anna University, Coimbatore, India.

vasudevan.vani@gmail.com,rpk.ind@gmail.com,
s.mohan77@gmail.com


Abstract. The well established techniques for video and audio streaming have
become the primary inspiration to focus our research towards 3D Streaming. In 3D
streaming, the model is buffered and rendered instead of complete download. The
central idea of this paper is to visualize the typical 3D human brain model at different
resolutions based on the clients requirement at a rapid rate with necessary
interactions. This is achieved by initially streaming the simplified version of 3D
human brain and then the refinements are streamed based on the clients requirement.
The experiment results and analysis affirms the fact that this type of multi resolution
streaming reduces the initial time to present the rendered 3D model to the requested
clients. Experiments are further conducted to compare the quality of the multi
resolution models that are being rendered with respect to the original model.

Keywords: Multi-Resolution model, 3D Streaming, 3D Rendering, 3D Visualization

1 Introduction
In recent years, 3D real time computer graphics [1] over the web has given rise to the
development of 3D Virtual Environments (VE) [2]. The 3D virtual environment
which comprises of set of 3D scenes and the scene in turn consists of set of 3D
complex meshes is available on the web and accessed by multiple users at the same
time. Therefore, an efficient mode of transmitting the 3D complex meshes across the
web has become the need of the hour. In order to cater to the need, the streaming [3]
approach which is promising to deliver the content as per the client request has
evolved and few attempts are already made by the researchers. However, in this
paper, an attempt is made specifically to stream and deliver the 3D human brain with
the required quality requested by the client who could be a medical
student/practitioner who wants to visualize the 3D human brain and study in detail
about its anatomy by interacting with it. The system he uses to stream and render the
3D human brain may have limited buffering capacity from few KBs to MBs.
Mohan S. et al. (Eds.): Proceedings of the International Conference ICSIP 2012, , pp.
springerlink.com Springer India 2012
1 Vani V et al.


Therefore, based on the buffering limits of the client, the corresponding LODs can
be streamed from server to the client. Also, in order to ensure the quality of the
rendered 3D mesh compared with the original mesh a detailed experiment is
conducted and results are analyzed. In the process of developing a 3D visualization
of human brain with required resolution through steaming firstly, Geometric Level
Of Details (GLOD) [8] library which was developed on top of OpenGL is used at
the server end. The GLOD, which focuses on the geometry details of the mesh, has
included polygon reduction technique [4] to reduce the level of details from the
original mesh and gives different levels of meshes by reduction process. This
process is carried out and required levels of multi resolution meshes are identified
and stored in the server for a specific 3D human brain scene which has got only
one complex 3D mesh. Secondly at the server end, the quality of the derived LODs
with respect to the original mesh (LOD14) is also estimated. Finally the real time
streaming is implemented with serialization and de-serialization technique and
results are analyzed. In streaming, if the requested scene is the 3D human brain
then the initial LOD (Level 1) with very less geometrical information is streamed
by computing the total space required and the time spent for streaming. Then, if the
total space estimated is available, the streamed data is buffered and rendered at the
client end with the rendering time estimated. On the client end, the client can
interact (move, rotate, zoom in/out) with the rendered LOD with the help of
keyboard and pointing device. Also, either based on the special key press (+ / -) or
based on the Euclidean distance between updated camera position and a random
point taken from the screen space next/previous LOD would be streamed and
rendered if the space criteria is satisfied. This process is repeated by ensuring that
the levels remain between 1 and 14.
The rest of the paper is organized as follows. In Section 2, we present the overview
of Multi resolution 3D model. Section 3 describes the distance measures used to
check the quality of the original and reduced multi-resolution mesh. In Section 4,
with the 3D human brain model being considered as a specific case, we discussed the
proposed work which includes the streaming and 3D visualization process. In Section
5, we discussed the experiment results and analysis. Section 6 concludes by
reflecting on the insights we gained from our model and its implications.

2 Multi-Resolution 3D Model An Overview


Highly detailed geometric models are necessary to satisfy the demands of user in
3D virtual world for realism in computer graphics [4]. But, due to the increasing
complexity of the models, it makes the virtual world developers feel expensive to
store, transmit, and render requested 3D data by the client. In order to store,
transmit and render the complex models at the rapid rate, the multi-resolution
models [15] are developed whenever, a representation at low resolution is
adequate to meet the needs of the application. Multi-resolution models offer the
possibility to manipulate representations of objects at different levels of detail
(LOD) [5, 6, 7] and accuracy, depending on the needs of each specific
application. In addition to the benefits of speeding up visualization, multi -
resolution techniques also support the interactive modeling of detailed objects.


Multi-resolution 3D visualization 2


2.1 Types of Hierarchies

The level of detail pipeline consists of three basic stages [8]:
geometric simplification, adaptation, and rendering. The simplification process takes
in flat geometry and produces a multi-resolution hierarchy [5, 7] from it. There are
several types of multi-resolution hierarchies, the classification of hierarchies as 1)
Discrete 2) Continuous 3) View-dependent or 4) Hierarchical is based on the multi
resolution granularity as shown in Fig.1.
1) Discrete hierarchies (Discrete Level of Detail (DLOD)): refers to the
creation of several static levels of detail [5, 7] which are swapped out
directly for each other and it is analogous to mip-maps for texturing.

2) Discrete hierarchies are extremely computation efficient as far as rendering
is concerned. The DLOD is the simplest and most common form of
simplification hierarchy.

It encodes multiple LODs at very coarse granularity. In this, each successive LOD
would be half the complexity of the predecessors and the choice of granularity
doubles the storage requirements with respect to higher resolution model. Each LOD
may also contain the error value indicating the fidelity/quality of it. This kind of
simplification is unaware of viewing directions since, the LODs are estimated offline
during preprocessing. Therefore, it uniformly reduces object detail. Thus, Discrete
LODs are also referred to as isotropic or view-independent LODs

The benefits of DLOD are described as below:
LODs are easily compiled into an optimized form for efficient rendering
with the help of processes such as
o Triangular strip reconstruction
o Re-ordering of vertices based on vertex cache size
o Indexed vertex arrays

The management of one or more discrete hierarchies is not too
computationally expensive within the interactive application (i.e.,) with the
given circumstances for each hierarchy appropriate DLOD can be chosen
DLODs are most useful when the data comprises of one or more objects that
are relatively small in spatial extent. In this case, single choice of resolution
is appropriate for the entire model.
DLOD is used by a class of models ranging from 3D scanned objects to that
of virtual environments.
3) Continuous hierarchies (Continuous Level of Detail (CLOD)): creates a
progressive data structure [5, 7] from which it can extract a continuous
spectrum of detail at run time and it is also referred as dynamic levels of
detail. Though it is more computationally complex during rendering, it
addresses many of the limitations of DLOD.
The benefits of CLOD are described as below:
It allows more exact choice of number of primitives to use for an object
It enables more subtle transition between LOD
It provides convenient representation for progressive transmission of
data.


3 Vani V et al.

Since it allows only linear progression and not selective refinement, it is
used for same class models as that of DLODs

4) View dependent LOD extends CLOD by creating a hierarchical data
structure [5, 7] from which it extracts a mesh tailored to the given
viewpoint. . The benefit of the view-dependent hierarchy is overwhelming
though there is a run time overhead in adjusting the detail across the model
on the fly.
5) Hierarchical Level of Detail (HLOD) [5, 9 and 10] works at scene level
with multiple objects and has combined discrete hierarchy with view
dependent hierarchy. This LOD hierarchy is more coarse-grained as
compared to a typical view-dependent hierarchy and roughly corresponds to
the notion of hierarchical scene structure. This coarse-grained representation
is beneficial to visualize complex scenes with large number of objects,
reducing per object overhead for given rendering frame rate both in LOD
selection process as well as the number of primitives to be rendered.



Fig. 1. Types of hierarchies [5], Continuous LOD is large no. of LODs encoded by representing small
changes from one LOD to next. LOD in the view-dependent hierarchy is represented as a cut across the
tree. HLOD is similar to view-dependent, but it is much more coarse-grained, allowing faster
management and more efficient rendering. A cut across an HLOD hierarchy essentially selects a set of
discrete LODs to represent the scene.

3 Distance Measures

One of the efficient geometric error metrics [11, 12, 13, 14] used to calculate the
quality of the rendered 3D LOD is the distance. It gives the distance between the
simplified vertices and the planes of the original surface. Root Mean Square (RMS)
difference is estimated between two LODs and it is treated to be simplest possible
estimation. Also, the Hausdorff distance is considered to be the most popular
distance measure which compares a pair of LODs with different connectivity and
calculates the similarity of two point sets by computing one sided
Discrete Continuous View Dependent Hierarchy




Multi-resolution 3D visualization 4

distance (

), where is the Euclidean distance between two

points ( ) (x,y,z).


(

((

)

(

)

The in general is not symmetrical i.e
(

) (

)

Where (

) is referred as forward distance and (

)
is referred as backward distance.

The two sided (symmetric) distance which provides a more accurate measurement
can be estimated by taking the
((

)(

))

Therefore, the Hausdorff distance has been used to find the geometric error between
a pair of 3-D mesh models which is shortly referred as LODs.

5 Proposed Work

The proposed work attempts to visualize the 3D human brain streamed from the
server and rendered at the client. Fig 2 gives the overview of the 3D Streaming &
Rendering system which has got two major modules.
1) Server Module
2) Client Module










Fig. 2. Overview of 3D Streaming & Rendering
System


Send
Compute LODs
& its Quality
Serialize
Server

De-Serialize
Render
Client

User Input
Receive

5 Vani V et al.
5.1 Server Module

In this module, required levels of LODs are estimated and its quality with respect to
the original well refined mesh is measured. This measure would give the client an
insight on the quality of the rendered mesh based on his request. Based on the clients
request, the initial LOD is streamed by serializing the 3D data. Serializing is the
process of packing the 3D data which comprises of set of vertices and faces for the
level of LOD that is selected for streaming. Further, the server module streams the
LODs based on the interactions made by the client.












































Fig. 3. Work Flow of the Streaming System

13. Display Appropriate
Error & Terminate
6. Start Streaming Timer & Serialize LOD
7. DeSerialize LOD and Stop the Streaming Timer
8. Compute & Display Total Streaming Time and Required Buffer Space
10. Store and Render de-serialized 3D mesh with necessary
interactions
4. Check for 3D Scene Requested with LODs Estimated
2. Is
connection
OK?
3. Set Camera Position & Request for 3D Scene to Visualize
1. Setup Connection between Clients and Server
No
Yes
5. Is the Scene
LOD Available?
9. Is Space
Available?
No
Yes
No
Yes
11. Interact with the Rendered Scene
12. Notify the interaction to Server with the updated move
to stream next level of refined mesh
A
A
A

Multi-resolution 3D visualization 6
5.2 Client Module

In this module, client establishes a connection with the server. Once the connection is
successfully established, it requests for a scene from the server and if the scene is
present then the initial LOD is streamed from the server. In client end, the streamed
data is de-serialized in order to get the vertices and faces of the corresponding LOD
and then it is rendered. Also the time taken for rendering process is also estimated.
Fig. 3 illustrates overall workflow of the system with the client and server
interactions.
6 Experiment Results and its Implications

Experiments are conducted to achieve 3D visualization of human brain through
streaming of multi resolution meshes based on the client request and further
interactions. The experimental setup was run on an Intel Core2 Duo CPU P8600 @ 2.4
GHz with 4 GB RAM and ATI Radeon 1GB graphics card system for both server and
client. For experimental purpose, we have considered 14 levels of LOD from coarse
mesh (LOD1) to refined (LOD14) mesh. Table 1 gives multi-resolution 3D human brain
models.
Table 2 describes the attributes of the LODs considered which includes number of
vertices and faces considered in each LOD with its corresponding % of vertices and %
of faces with respect to LOD14 (original mesh). Fig 4 shows the % of vertices and % of
faces represents the corresponding LODs. Also, in order to understand the quality of the
mesh (LOD) being rendered, we have considered hausdorff distance measure which
would estimate the quality of the LOD being rendered with respect to LOD14 (original
mesh).Fig. 5, highlights the quality of the LODs streamed and rendered for 3D
visualization using one sided and two sided hausdorff distance.

Table 1. Multi-Resolution 3D Human Brain Models (Coarse LOD to Refined LOD)

LOD1 LOD2 LOD3 LOD4
LOD5 LOD6 LOD7 LOD8
LOD9 LOD10 LOD11 LOD12

LOD13 LOD14



7 Vani V et al.

As specified in section 3, hausdorff distance is a generic distance measure which
defines the distance between two empty sets and it is useful in measuring the error
between surfaces. We have considered both one sided and two sided hausdorff distance
measure as some times one sided distance would end up in less accurate measure than
that of the two sided distance as depicted in Fig 6. However, according to the results
highlighted for our LODs, it clearly shows that (

) >> (

)
in all comparisons. Where

is considered to be the original mesh (

) and

is substituted with

to

. Table 3 and Fig. 7 highlight the turnaround

time required to stream each LODs. The results shown in Table1 Table 4 and
Fig.4,5& 7 clearly indicates that by exploiting the human perception characteristics, it is
enough if we stream and render

which could present all the parts of the brain with

approximately 3% of the vertices and 3% of the faces and at the error rate of .09
distance from

and

and .23 distance from

and

. Also, the
buffering capacity required is only 576 KB which is only 2.68% of the total capacity
required to transmit

. From Table 1 and Fig 7, we could infer that if it is sufficient

to stream and render

then the turnaround time required is only 3.78 secs where

we can save 8.5 secs.

Table 2. Attributes of 3D Human Brain LODs

LOD No. of
Vertices
No. of
Faces
%
Vertices
% Faces
LOD1 12 16
0.0041 0.0027
LOD2 39 71
0.0133 0.0121
LOD3 74 142
0.0252 0.0241
LOD4 289 574
0.0983 0.0976
LOD5 578 1148
0.1966 0.1952
LOD6 1152 2296
0.3918 0.3905
LOD7 2299 4594
0.7819 0.7813
LOD8 4593 9188
1.5622 1.5625
LOD9 9186 18376
3.1244 3.125
LOD10 18375 36752
6.2497 6.25
LOD11 36748 73504
12.4988 12.5
LOD12 73500 147008
24.999 25
LOD13 147004 294016
49.9993 50
LOD14 294012 588032
100 100



Fig. 4. % of Vertices and Faces Rendered w.r.t. LODs

Multi-resolution 3D visualization 8


Fig. 5. Quality of the LOD Estimated using (a) One Sided and (b)Two Sided Hausdorff
Distances







Fig. 6. In this case (

) (

) since
here (

) (

)

Table 3. Streaming and RenderingTime of Discrete LODs

LOD
Streaming
Time(sec)
Rendering
Time(sec)
Turnaround
Time(sec)
LOD1 0.024 1.51 1.534
LOD2 0.029 1.526 1.555
LOD3 0.064 1.458 1.522
LOD4 0.092 1.554 1.646
LOD5 0.07 1.742 1.812
LOD6 0.108 1.8 1.908
LOD7 0.146 1.96 2.106
LOD8 0.184 2.39 2.574
LOD9 0.19 3.593 3.783
LOD10 0.208 4.008 4.216
LOD11 0.293 6.196 6.489
LOD12 0.333 8.1 8.433
LOD13 0.666 9.24 9.906
LOD14 1.275 11.034 12.309


Fig. 7. Turnaround Time of Discrete LODs

7 Conclusion
In this paper an attempt has been made to build a 3D human brain visualization system
through streaming by exploiting human perception and multi-resolution capabilities. It
is demonstrated with experimental results that the rendering latency and streaming
A
LOD
2

LOD
1

B


9 Vani V et al.
time are reduced to a greater extent by streaming and rendering appropriate multi
resolution model instead of transmitting the original mesh itself. 3D visualization is
made possible with the predominant data being streamed and rendered with necessary
interactions for further refinement if needed. The results achieved is overwhelming and
would have certainly a greater impact when we have to stream and render much more
complex scenes with time and space constraints enforced on it.

References

[1] Don Brutzman, Mike Macedonia, Theresa-Marie Rhyne and Steve
McCanne,Internetworked 3D Computer Graphics: Beyond the Bottlenecks and
Roadblocks.ACM SIGCOMM '97. (1997).
[2] Blanc, A. L., Bunt, J., Petch, J., and Kwok, Y. The virtual learning space: an interactive
3D environment. In Proceedings of the Tenth international Conference on 3D Web
Technology (Bangor, United Kingdom). Web3D '05. ACM, New York, NY, 93-102.
DOI= http://doi.acm.org/10.1145/1050491.1050505, March 29 - April 01. (2005).
[3] Soumyajit Deb and P. J. Narayanan. Design of a geometry streaming system. In Proc.
ICVGIP, pages 296-301. (2004).
[4] Hugues Hoppe.:Progressive meshes. In Proceedings of the 23rd annual conference on
Computer graphics and interactive techniques (SIGGRAPH '96). ACM, NY, USA, 99-
108. DOI=10.1145/237170.237216 (1996).
[5] Cohen, Jonathan D. and Dinesh Manocha.: Model Simplification for Interactive
Visualization. Visualization Handbook. 13 pages. Eds. Chris Johnson and Chuck
Hansen. Elsevier Butterworth-Heinemann. Chapter 20, pp. 393-410. (2005).
[6] Huimin Ma, Tiantian Huang, Yanzhi Wang, Multi-resolution recognition of 3D objects
based on visual resolution limits, Pattern Recognition Letters, Volume 31, Issue 3,
Pages 259-266, ISSN 0167-8655, 10.1016/j.patrec.2009.08.015.(2010).
[7] Li Xin, Research on LOD Technology in Virtual Reality, Energy Procedia, Volume 13,
Pages 5144-5149, ISSN 1876-6102, 10.1016/j.egypro.2011.12.142.(2011).
[8] Cohen, Jonathan D., David Luebke, Nathaniel Duca, Brenden Schubert, and Chris
Niski.: GLOD: A Minimal Interface for Geometric Level of Detail. 10 pages. Johns
Hopkins University Graphics Lab Technical Report. (2004).
[9] Carl Erikson, Dinesh Manocha, and William V. Baxter, III. 2001. HLODs for faster
display of large static and dynamic environments. In Proceedings of the symposium on
Interactive 3D graphics (I3D '01). ACM, NY, USA, 111-120.
DOI=10.1145/364338.364376. (2001).
[10] Jean Pierre Charalambos. Hoq-Driven HLOD Refinement: Using Hardware Occlusion
Queries Information to Drive Hierarchical Level-Of-Detail Refinement. VDM Verlag,
Saarbrcken, Germany, Germany. (2010).
[11] Nicolas Aspert, Diego Santa-Cruz, Touradj Ebrahimi, MESH : Measuring Errors
Between Surfaces using the Hausdorff Distance In Proc. of the IEEE International
Conference in Multimedia and Expo (ICME) vol. 1, pp. 705-708, Lausanne,
Switzerland, August 26-29. (2002).
[12] Abdullah Bulbul, Tolga Capin, Guillaume Lavou , Marius Preda, Assessing Visual
Quality of 3D Polygonal Models, IEEE Signal Processing Magazine, vol. 28, No. 6,
pp.80-90.(2011).
[13] P. Cignoni, C. Montani, and R. Scopigno, A Comparison of Mesh Simplification
Algorithms,Computers & Graphics, vol. 22, no. 1, pp. 37-54.( 1998).
[14] P. Cignoni, C. Rocchini, R. Scopigno: Metro: Measuring error on simplified surfaces.
Computer Graphics. Forum 17, 2, 167174. (1998).
[15] Eric Shaffer, Michael Garland, A Multi-resolution Representation for Massive Meshes
IEEE Transactions on Visualization and computer graphics, VOL. 11, NO. 2.(2005).