3D Compression and Visual Quality Assessment
Ms. Debarati Kundu and Dr. Brian L.
Motivation
Sources of 3D data
• PAST: Animation studios
• NOW: Kinect, mobile stereo, LIDAR, etc.
Progressive transmission of 3D data adapted to display
device and network conditions in many applications
Applications
• Mobile, massively multiplayer and real-time gaming
• Augmented reality (e.g. 3D maps)
• Virtual avatars in teleconferencing
• Data storage (Google 3D warehouse)
• Telemedicine: Transmission of 3D models of organs
• Scientific visualization
• 6.7% annual growth rate to $86.1B (est.) by 2016
• Biggest growth: massively multiplayer online games
• 50% of users play exclusively on smartphones
• 66% of money on Apple & Android Apps for gaming
Evans
debarati@utexas.edu, bevans@ece.utexas.edu
Aim 1: Static 3D Mesh Compression
Represented by triangular meshes
• Geometry data: Position of n vertices
• Connectivity data: How vertices are joined
• Texture data
Complexity: O(log n) computation and O(n4) storage
m = 40 m = 200
Manifold harmonics extend Fourier bases to surfaces
• Useful for progressive transmission
• Most energy resides in lower frequency components
• 15:1 compression ratios
• Improvements possible due to intrinsic symmetries
Goal: Achieve additional 2x compression
Number of polygons used to represent geometry
reduced by texture images mapped on simplified mesh
3-D Model
p=(x,y,z)
UV Map
P = (u,v)
Texture
• Used for terrain rendering (OpenGL and Direct3D)
• Texture compressed separately by image
compression (JPEG, JPEG 2000)
• Joint geometry texture compression leads to better
compression ratios
Aim 2: Visual Quality Assessment of Textured Meshes
Texture produces masking effects on geometry distortions
Masking of surface geometry artifacts due to Masking of surface geometry artifacts due to
same spatial frequency of texture image same orientation of texture image
• Joint visual evaluation of geometry and texture leads to better
compression and mesh simplification algorithms
m = 500 m=1000
• Ideally work directly on surface of textured 3D objects
• No projection on several viewpoints
• Some algorithms for quantifying geometric distortions correlate well with
human perception
Goal: Develop perceptual quality metrics taking into account texture image
Aim 3: Dynamic 3D Mesh Compression
• Much work on compression of depth maps, which is more noise-prone
• Instead compress dynamic mesh geometry directly
• Motion paths compressed using Principal Components Analysis
• Can be combined with spectral methods for compressing spatial data
Goal: Develop first perceptual measure to quantify distortions of dynamic
meshes of arbitrary connectivity without projecting to 2D
Conclusion
• Compression of 3D static and dynamic meshes result in better rendering of
scenes having same visual quality at lower data rates
• Perceptual quality evaluation leads to better benchmarking of mesh
geometry and texture processing algorithms
• Better understanding of psycho-visual phenomenon of shape perception