OBJECT BASED 2D TO 3D CONVERSION

College Of Engineering,Chengannur

CONTENTS
 TYPICAL 3D TV SYSTEM  3D GENERATION TECHNIQUE  DISADVANTAGES & SOLUTIONS  2D TO 3D CONVERSION:EXISTING    

METHODS OF DEPTH EXTRACTION NOVEL METHODS WHY 2D TO 3D? LIMITATIONS FUTURE OF 3D TV

College Of Engineering,Chengannur

TYPICAL 3D TV SYSTEM

College Of Engineering,Chengannur

3D GENERATION TECHNIQUE
 Stereo camera setting  Different perspectives for left and right eyes  Separate recording

College Of Engineering,Chengannur

DISADVANTAGES & SOLUTIONS

Disadvantages
 Highly skilled camera operators  Costly equipment

Solutions
 Infrared cameras  Depth-image-based rendering (DIBR)techniques  2D-to-3D conversion most effective
College Of Engineering,Chengannur

2D TO 3D CONVERSION: EXISTING METHODS OF DEPTH EXTRACTION

1.Depth From Geometric Constraints 2.Depth from Focus/Defocus(Blur) Analysis 3.Depth From Motion 4.Other Approaches

College Of Engineering,Chengannur

1.Depth From Geometric Constraints

 Depth extraction using geometric constraints  Geometric perspectives vanishing points

and lines  Planar representation of background  Foreground geometrically modelled  Disadvantage Controlled environment required

College Of Engineering,Chengannur

2.Depth from Focus / Defocus (Blur) Analysis

 Depth determination by modeling the effect

of varying focal parameters on the image  Inverse filtering  Drawbacks

Blurs unavailable in general case Blurs due to external factors
o Lens aberration o Atmospheric interference o Blur due to motion

College Of Engineering,Chengannur

3.Depth From Motion

 Based on vision geometry  Modified time difference method(MTD) to

detect object motion & estimate image presentation delay time  Disadvantages
Overall strategy is still heuristic Unable to deal with occlusions No method for recovery of layered motions

College Of Engineering,Chengannur

4.Other Approaches
 Use of edge information to generate sparse

depth maps for DIBR  Depth determination using supervised machine learning  Color based image segmentation

College Of Engineering,Chengannur

10

NOVEL METHODS
1.Determining depth ordinal using optical-flow based occlusion 2.Object segmentation 3.Hybrid depth estimation scheme

College Of Engineering,Chengannur

11

1.DETERMINING DEPTH ORDINAL USING OPTICAL-FLOW BASED OCCLUSIONS

 Optical flow : a pattern representing apparent

motion of objects , surfaces and edges in a visual scene caused by relative motion between observer and scene  Occlusion?

College Of Engineering,Chengannur

12

Principles of Occlusion reasoning

 Relies on optical flow to find the motion of

each object and segment a scene to find the occlusion  Rules of Occlusion Reasoning
Forward Reasoning: Object visible in nth frame and invisible in (n+1)th frame Backward Reasoning : Object visible in (n+1)th frame and invisible in nth frame

College Of Engineering,Chengannur

13

Rules of Occlusion reasoning

 A)forward reasoning B) backward reasoning  (1) and (2) are two consecutive frames In and In+1  (3) denotes relationship between occluded and newly appearing regions from In and In+1
College Of Engineering,Chengannur

14

Using optical based flow occlusion

  

[(a) and (b)] Two consecutive fames from Flower Garden sequence. (c) The optical flow of (a). (d) Layered representation of objects in (a). (e) The occlusion map from (a) to (b). (f) The occlusion map from (b) to (a).
College Of Engineering,Chengannur

15

2.OBJECT SEGMENTATION
 Two steps Depth ordinal considered as initial segmentation Extracting masks of each depth layer
Seeded-Region growing method

College Of Engineering,Chengannur

16

Seeded region growing method

 Let

be binary mask of a depth layer in an image; be remaining mask in the image excluding  Refinement process
Skeletons of and extracted and denoted as So and Sb and used for growing of foreground and background regions , respectively. Ro-foreground region ; Rb -background region Initially Ro=So and Rb=Sb In each loop , all outer neighbouring boundary pixels of Ro and Rb obtained as Co and Cb
College Of Engineering,Chengannur

17

Seeded region growing method

 The minimum distance between Co and Cb to

Ro and Rb determined as do and db.  If do < db , Co is grown to Ro  If do> db , Cb is grown to Rb  With updated values of Ro and Rb repeat above steps till no change in Ro and Rb is made

College Of Engineering,Chengannur

18

3.HYBRID DEPTH ESTIMATION SCHEME

 Recovery of Depth Maps
Matching objects with existing libraries Textural representation using local binary pattern (LBP)

One example to show relationships among (left) disparity maps and (middle) texture maps via stereo pair using the left-image of (right)
College Of Engineering,Chengannur

19

3.HYBRID DEPTH ESTIMATION SCHEME

 Ordinal Based Depth Regularization
Values of depth maps regulated in order to be consistent with depth ordinal. Linear weighting based regularization process is employed

College Of Engineering,Chengannur

20

ADVANTAGES & LIMITATIONS

ADVANTAGES  Reduction in overall cost  Reuse of existing 2-D libraries  Cheapest and most efficient option suitable for academic research labs LIMITATIONS  Conversion process can be time-consuming  Degraded accuracy in sequences of high and medium complexity  Occurrence of depth discontinuities
College Of Engineering,Chengannur

21

FUTURE OF 3D TV
 3D TV without glasses  The potential of sets that contain tiny

cameras which track the viewers position  Holographic technology

College Of Engineering,Chengannur

22

REFERENCE
 Yue Feng; Jinchang Ren; Jianmin Jiang; ,

"Object-Based 2D-to-3D Video Conversion for Effective Stereoscopic Content Generation in 3D-TV Applications," Broadcasting, IEEE Transactions on , vol.57, no.2, pp.500-509, June 2011

College Of Engineering,Chengannur

23

College Of Engineering,Chengannur

24

College Of Engineering,Chengannur

25