Computer Vision

Lecture -07

1
http://www.middlebury.edu/stereo

2
Reconstruction

Only need to match

features across epipolar
lines

3
Correspondence using Discrete Search

4
Sum of Squared Differences (SSD)

5
Problems with window matching

6
 Stereo results
– Data from University of Tsukuba
– Similar results on other images without ground truth

Scene Ground truth

7
 Results with window correlation

Window-based matching Ground truth

(best window size)
8
 Results with better method

State of the art method Ground truth

Boykov et al., Fast Approximate Energy Minimization via Graph Cuts,
International Conference on Computer Vision, September 1999.
NSU-Summer-2011 CV lecture-05 9
Final-Lab assignment- implement window
based stereo algorithm

10
Why estimate motion?
1. Motion Detection
2. Track object behavior
3. Correct for camera jitter (stabilization)
4. Align images (mosaics)
5. 3D shape reconstruction
6. Video Compression

11
 Optical flow

• How to estimate pixel motion from image H to image I?

– Solve pixel correspondence problem

• given a pixel in H, look for nearby pixels of the same color in I

• Key assumptions
– color constancy: a point in H looks the same in I
• For grayscale images, this is brightness constancy
– small motion: points do not move very far
• This is called the optical flow problem

NSU-Summer-2011 CV lecture-05 12
 Optical flow constraints (grayscale images)

– brightness constancy: Q: what’s the equation?

• Let’s look at these constraints more closely

– small motion: (u and v are less than 1 pixel)
• suppose we take the Taylor series expansion of I:

13
Optical flow equation

• Combining these two equations

• In the limit as u and v go to zero, this becomes exact

14
 Optical flow equation

• Q: how many unknowns and equations per pixel?

• Intuitively, what does this constraint mean?

– The component of the flow in the gradient
direction is determined
– The component of the flow parallel to an edge is
unknown 15
Aperture problem

16
Aperture problem

17