Computer Vision and Image Processing

– Fundamentals and Applications

Image Formation in a Stereo Vision Setup

n
ya
hu
.B
.K
M
Course Instructor: Dr. M.K. Bhuyan
Professor

Department of Electronics and Electrical Engineering,

Indian Institute of Technology Guwahati, India.
Goal: disparity map
• Disparity:
• The horizontal displacement between corresponding
points
• Closely related to scene depth

n
ya
hu
.B
.K
M
The matching problem
• Which image entities should be matched?
• Two main approaches
• Pixel/area-based (lower-level)
• Feature-based (higher-level)

n
ya
hu
.B
.K
M
Matching challenges
• Scene elements do not always look the same in the two images
• Camera-related problems
• Image noise, differing gain, contrast, etc..
• Viewpoint-related problems:

n
ya
hu
• Perspective distortions

.B
.K
M
• Occlusions
• Specular reflections
More matching heuristics
• Always valid:
• (Epipolar line)
• Uniqueness
• Minimum/maximum disparity

n
ya
hu
.B
.K
M
• Sometimes valid:
• Ordering
• Local continuity (smoothness)
Area-based matching
• Finding pixel-to-pixel correspondences
• For each pixel in the left image, search for the most similar pixel in the right
image
• Using neighbourhood windows

n
ya
hu
.B
.K
M
 Intensity-based Methods
• Match image sub-windows between the two images
(e.g., using correlation).
left right

n
ya
hu
.B
.K
M
• How to choose the window size W?
– Too small a window may not capture enough image structure, and may
be too noise sensitive (i.e., less distinctive, many false matches).
– Too large a window makes matching less sensitive to noise (desired)
but also harder to match
Area-based matching
• Similarity measures for two windows
• SAD (sum of absolute differences)
• SSD (sum of squared differences)
• CC (cross-correlation)

n
ya
hu
.B
• …

.K
M
M
.K
.B
hu
ya
n
Feature-based matching
• Matching features:
• Edge points
• lines
• corners

n
ya
hu
.B
• …

.K
M
• Sparse reconstruction sets
• Best if scene type is known a priori
M
.K
.B
hu
ya
n
 Intensity-based vs feature-based approaches

• Intensity-based methods
– Provide a dense disparity map.
– Need textured images to work well.
– Sensitive to illumination changes.

n
ya
hu
.B
.K
M
• Feature-based methods:
– Faster than correlation-based methods.
– Provide sparse disparity maps.
– Relatively insensitive to illumination changes.
Three or more viewpoints
• More matching information
• Additional epipolar constraints
• More confident matches

n
ya
hu
.B
.K
M
M
.K
.B
hu
ya
n
Summary
Summary
• Stereo vision:
• A method for 3-D analysis of a scene using images from two or more
viewpoints
• Two subproblems:

n
ya
hu
.B
• Matching

.K
M
• Reconstruction
• Most difficult part: Matching
• Two main approaches:
• Area based: Dense reconstruction
• Feature based: Sparse reconstruction
Image Reconstruction from a Series of

n
ya
hu
Projections

.B
.K
M
M
.K
.B
hu
ya
n
M
.K
.B
hu
ya
n
M
.K
.B
hu
ya
n
 Radon Transform

n
ya
hu
.B
.K
M
⚫ A linear transform
f(x,y) → g(s,)
– Line integral or “ray-sum”
– Along a line inclined at angle 
from y-axis and s away from origin

⚫ Fix  to get a 1-D signal g(s)

M
.K
.B
hu
ya
n
M
.K
.B
hu
ya
n
M
.K
.B
hu
ya
n
Representation in Polar Coordinates

n
ya
hu
.B
.K
M
Example of Radon Transform

n
ya
hu
.B
.K
M
Full circle in RT

n
ya
hu
.B
.K
M
Thin stick in RT

n
ya
hu
.B
.K
M