Optical flow (motion vector)

computation
Course: Computer Graphics and Image Processing

Semester: Fall 2002

Presenter: Nilesh Ghubade (nileshg@temple.edu)

Advisor: Dr Longin Jan Latecki

Dept: Computer and Information Science,
Temple University, Philadelphia, PA-19122
Motion Analysis
Three groups of motion-related problems:
1. Motion detection
Registers any detected motion.
Single static camera.
Used for security purposes.
2. Moving object detection and location
Determination of object trajectory.
Static camera, moving objects OR Moving camera,
static objects OR Both camera and objects moving.
3. Deriving 3D properties
Use of set of 2D projections acquired at different
time instants of object motion.
Object motion assumptions
Maximum velocity.


Small acceleration.


Common motion of object points.
Mutual correspondence.
C
max
* dt
t
0

t
1

t
2

Differential motion analysis
Simple subtraction of images acquired at
different instants in time makes motion
detection possible, assuming stationary
camera position and constant illumination.
Difference image is a binary image
subtract two consecutive images.
Cumulative difference image:
Reveals motion direction.
Time related motion properties.
Slow motion and small object motion.
Constructed from sequence of n images taking first
image as the reference image.
Example
Motion in front of a security camera.

Sobel filter edge detection.
Motion Detection- Sobel filter
10 frames/second 15 frames/second
15 frames/second 25 frames/second
Optical Flow
Optical Flow reflects the image changes due to
motion during a time interval dt.
Optical flow field is the velocity field that
represents the 3D motion of object points across
a 2D image.
It should not be sensitive to illumination changes
and motion of unimportant objects (e.g.
shadows)
Exceptions:
1. Non-zero optical flow fixed sphere illuminated by a moving
source.
2. Zero optical flow smooth sphere under constant
illumination, although there is rotational motion and true non-
zero motion field.
Optical Flow (continued)
Aim is to determine optical flow that
corresponds with true motion field.
Necessary pre-condition of subsequent
higher level motion processing
stationary or moving camera.
Provides tools to determine motion
parameters, relative distances of objects in
the image etc..
Example:
t1
t2
Assumptions
Optical flow computation is based on two
assumptions:

1. The observed brightness of any object
point is constant over time.

2. Nearby points in the image plane move in
a similar manner (the velocity
smoothness constraint).
Optical flow computation
The optical flow field represented in the form
of Velocity vector:
Length of the vector determines the magnitude of
velocity.
Direction of the vector determines the direction of
motion.
Global optical flow estimation
Local constraints are propagated globally.
But errors also propagate across the solution.
Local optical flow estimation
Divide image into smaller regions.
But inefficient in the areas where spatial gradients
change slowly here use global method,
neighboring image parts contribute.

Forms of motion
Translation at constant
distance from the observer.
Set of parallel motion
vectors.
Translation in depth
relative to the observer.
Set of vectors having
common focus of
expansion.
Rotation at constant
distance from view axis.
Set of concentric motion
vectors.
Rotation of planar object
perpendicular to the view
axis.
One or more sets of
vectors starting from
straight line segments.
Representation
Locate the position of a pixel (row,col) in the current image by computing
shortest Euclidean distance with respect to 5-by-5 neighborhood in the next
consecutive frame.
16 15 14 13 12
17 4 3 2 11
18 5 0 1 10
19 6 7 8 9
20 21 22 23 24
Experiments
3-by-3 neighborhood
Experiments (contd)
5-by-5 neighborhood
Experiments (contd)
Experiments (contd)
Applications of optical flow
Object motion detection.

Action recognition.

Active vision or structure of motion
Reconstruction of 3D object by computing depth information.
If you have distance (depth) maps, you can reconstruct surface of the
object.

Facial expression recognition: reference
http://athos.rutgers.edu/~decarlo/pubs/ijcv-face.pdf
Thank you