Professional Documents
Culture Documents
Motion Computing in Image Analysis: - Mani V Thomas CISC 489/689
Motion Computing in Image Analysis: - Mani V Thomas CISC 489/689
Analysis
- Mani V Thomas
CISC 489/689
Roadmap
Optic Flow Constraint
Optic Flow Computation
Gradient Based Approach
Feature Based Approach
Estimation Criterion
Block Matching algorithms
Conclusion
E T
v Et 0
Ex Ex E E u E x Et
x y
E y E x E E y y
v y t
E E
First proposed by Lucas-Kanade in 1981
Summation performed over all the pixels in the window
Lucas-Kanade Optic Flow
Lucas-Kanade Optic flow
Ex Ex E E u E x Et
x y
E y E x E E y y
v E y Et
When is the Lucas-Kanade equations solvable
ATA should be invertible
ATA should not be too small (effects of noise)
Eigenvalues of ATA, 1 and 2 should not be small
ATA should be well conditioned
1/2 should not be large (1 = larger eigenvalue)
Edge
u=1.25 pixels
u=2.5 pixels
u=5 pixels
image H
J image I
For more information: Ramesh Jain, Rangachar Kasturi, Brian Schunck: Machine Vision 1995 (140 -
159)
Roadmap
Optic Flow Constraint
Optic Flow Computation
Gradient Based Approach
Feature Based Approach
Estimation Criterion
Block Matching algorithms
Conclusion
(d )
I k n I k 1 n d
n ,dR
Quadratic function is not good since a single large error can bias the estimate
of the field
Absolute value function is better than the quadratic since cost grows linearly
with error
Does not require multiplications and is better suited for real-time video encoders
Estimation Criterion(contd.)
A more robust criterion is based on the Lorentzian function
2
log 1
2 2
Grows slower than |x| for larger errors
Similarity measure using Correlation
C (d )
I k (n )I k 1 (n d )
n ,dR
Computationally complex because of the multiplications
This criterion requires maximization
I k n I k n I k n d I n d ,1 r 1
r k
I k n I nd
k
6
5
4
3
2
1
0
-6 -4 -2 0 2 4 6
F I k (n ) F I k 1 (n z ) 2u T z
Amplitudes of both the FT are independent of z
Argument difference depends linearly on translation
Global motion is recovered by evaluating the phase difference over a
number of frequencies and solving the resulting system of equations
In practice, this method will work only for a single object moving
across a uniform background
Estimation Criterion(contd.)
Phase Correlation
ˆ ˆ
I (u ) I (u )
k 1,k (n ) F 1 k k1
Iˆk (u ) Iˆk 1 (u )
In the case of a single global translation, the correlation surface
becomes a Kronecker delta function
F e (n z )
j 2uz
k 1,k (n ) I
k ( n ) I k 1 ( n z )
0 x0
( x)
1 x0
In practice, there are numerous peaks which correspond to the
dominant displacements between the two images
The locations are relatively independent to illumination changes
Roadmap
Optic Flow Constraint
Optic Flow Computation
Gradient Based Approach
Feature Based Approach
Estimation Criterion
Block Matching algorithms
Conclusion
d m I k n I k 1 n d m m
nBm
P n n1 , n2 : P n1 P,P n2 P
Bm is an M x N block of pixels with the top-left corner co-ordinate at
m m1 , m2
Block Matching Algorithms(contd.)
-p
N -p N
M M
p p
-p
N
(x,y)
(x+u,y+v)
M
-p p u
p
Block Matching Algorithms(contd.)
Principle of Locality of Reference
Block Matching algorithms
Exhaustive Search
Always finds the “deepest” minimum
Computationally very expensive
If I x J is the picture resolution and rate is F fps the overall
operations in comparing MxN blocks would be
IJF
2 p 12 * MN * 3
MN
This corresponds to 29.89 GOPS for p=15 at 30fps for a 720x480
image (3 operations per pixel of one subtraction, one absolute value
and one addition)
Block Matching Algorithms(contd.)
Logarithmic Search
Sub-optimal and may get trapped in a local minima
Computationally feasible for real-time video encoders
Search Method
Divide the search space at [-p/2, -p/2]
Search at (0,0) and at 8 major points at the perimeter of the rectangle at [-p/2, -
p/2]
Using best match position as starting point, search in the eight perimeter points
at the half distance window
If I x J is the picture resolution and rate is F fps the overall operations in
comparing MxN blocks would be
IJF
8k 1 * MN * 3 k log2 p
MN
This corresponds to 1.03 GOPS for p=15 at 30fps for a 720x480 image
For more information refer the work by Dr. Lai-Man Po and C. K. Cheung
(http://www.ee.cityu.edu.hk/~lmpo/publications/index.html)
Block Matching Algorithms(contd.)
Hierarchical Search
Sub-optimal for regions containing detail and increased storage requirements
Computationally feasible for real-time video encoders
Search method
Form several low resolution images by low pass filtering
At the lowest resolution perform a sub-optimal search like log search
Propagate search vectors to higher resolution images and perform search
If I x J is the picture resolution and rate is F fps the overall operations in
comparing MxN blocks would be
IJF p
2 MN
2 1 180 * *3
MN
4 16
This corresponds to 507.38 MOPS for p=15 at 30fps for a 720x480 image
Conclusion
Motion estimation
Aperture problem
Different algorithms to perform motion analysis
Lucas-Kanade algorithm
Estimation criterion for motion field computation
Block Matching Algorithms
Computational complexity of motion analysis