Motion Detection And Analysis

Michael Knowles
Tuesday 13th January 2004
 Introduction

 Brief Discussion on Motion Analysis and its

applications
 Static Scene Object Tracking
 Motion Compensation for Moving-Camera
Sequences
 Applications of Motion Tracking
 Control Applications
 Object Avoidance
 Automatic Guidance
 Head Tracking for Video Conferencing
 Surveillance/Monitoring Applications
 Security Cameras
 Traffic Monitoring
 People Counting
 Two Approaches

 Optical Flow
 Compute motion within region or the frame as
a whole
 Object-based Tracking
 Detect objects within a scene
 Track object across a number of frames
 My Work

 Started by tracking moving objects in a

static scene
 Develop a statistical model of the
background
 Mark all regions that do not conform to
the model as moving object
 My Work

 Now working on object detection and

classification from a moving camera
 Current focus is motion compensated
background filtering
 Determine motion of background and
apply to the model.
 Static Scene Object Detection and
Tracking
 Model the background and subtract to
obtain object mask
 Filter to remove noise
 Group adjacent pixels to obtain objects
 Track objects between frames to develop
trajectories
Background Modelling
Background Model
After Background Filtering…
 Background Filtering

 My algorithm based on:

“Learning Patterns of Activity using Real-Time Tracking” C.
Stauffer and W.E.L. Grimson. IEEE Trans. On Pattern
Analysis and Machine Intelligence. August 2000

 The history of each pixel is modelled by a

sequence of Gaussian distributions
 Multi-dimensional Gaussian
Distributions
 Described mathematically as:


1
 X t   t T  1  X t   t 
  X t ,  ,  
1 2
1
e
2 
n
2  2

 More easily visualised as:

(2-Dimensional)
 Simplifying….

 Calculating the full Gaussian for every

pixel in frame is very, very slow
 Therefore I use a linear approximation
How do we use this to represent a
pixel?
 Stauffer and Grimson suggest using a
static number of Gaussians for each pixel
 This was found to be inefficient – so the
number of Gaussians used to represent
each pixel is variable
 Weights
 Each Gaussian carries a weight value
 This weight is a measure of how well the
Gaussian represents the history of the pixel
 If a pixel is found to match a Gaussian then the
weight is increased and vice-versa
 If the weight drops below a threshold then that
Gaussian is eliminated
 Matching

 Each incoming pixel value must be

checked against all the Gaussians at that
location
 If a match is found then the value of that
Gaussian is updated
 If there is no match then a new Gaussian
is created with a low weight
 Updating

 If a Gaussian matches a pixel, then the

value of that Gaussian is updated using
the current value
 The rate of learning is greater in the early
stages when the model is being formed
 Colour Spaces

 If RGB is used then the background

filtering is sensitive to shadows
 The use of a colour space that separates
intensity information from chromatic
information overcomes this
 For this reason the YUV colour space is
used
 Colour Spaces
 Background and Frame:

 Channel Differences:
 Isolate Objects

 Groups of object pixels must be grouped

to form objects
 A connected components algorithm is
used
 The result is a list of objects and their
position and size
 Track objects

 Objects are tracked from frame to frame

using:
 Location
 Direction of motion
 Size
 Colour
 The Story So Far…

 Basic principle of background filtering

 Stages necessary in maintaining a
background model
 How it is applied to tracking
 Moving Camera Sequences

 Basic Idea is the same as before

 Detect and track objects moving within a
scene
 BUT – this time the camera is not
stationary, so everything is moving
 Motion Segmentation

 Use a motion estimation algorithm on the

whole frame
 Iteratively apply the same algorithm to
areas that do not conform to this motion
to find all motions present
 Problem – this is very, very slow
 Motion Compensated Background
Filtering
 Basic Principle
 Develop and maintain background model as
previously
 Determine global motion and use this to
update the model between frames
 Advantages

 Only one motion model has to be found

 This is therefore much faster
 Estimating motion for small regions can be
unreliable
 Not as easy as it sounds though…..
 Motion Models

 Trying to determine the exact optical flow

at every point in the frame would be
ridiculously slow
 Therefore we try to fit a parametric model
to the motion
 Affine Motion Model

 u   a0   a1 a2  x 
        
 v   a3   a4 a5  y 

 The affine model describes the vector at each

point in the image
 Need to find values for the parameters that best
fit the motion present
 Minimisation of Error Function
 If we are to find the optimum parameters we
need an error function to minimise:

E ( x, y)  I t ( x, y)  I t 1 ( x  u, y  v)
 But this is not in a form that is easy to
minimise…
 Gradient-based Formulation
 Applying Taylor expansion to the error function:

I I I  u  I
E ( x, y )  u  v   I   
x y t  v  t

 Much easier to work with

 Gradient-descent Minimisation

 If we know how the error changes with

respect to the parameters, we can home
in on the minimum error
 Various methods built on this principle:
 Applying Gradient Descent
E
 We need:
an

 Using the chain rule:

E E u

an u an
 Robust Estimation

 What about points that do not belong to

the motion we are estimating?
 These will pull the solution away from the
true one
 Robust Estimators
 Robust estimators decrease the effect of
outliers on estimation
 Error w.r.t. parameters
 The complete function is:

ER ER E u

an E u an
 Aside – Influence Function
 It can be seen that the first derivative of the
robust estimator is used in the minimisation:
 Pyramid Approach

 Trying to estimate the parameters form

scratch at full scale can be wasteful
 Therefore a ‘pyramid of resolutions’ or
‘Gaussian pyramid’ is used
 The principle is to estimate the
parameters on a smaller scale and refine
until full scale is reached
 Pyramid of Resolutions
 Each level in the pyramid is half the scale of
the one below – i.e. a quarter of the area
 Out pops the solution….
 When combined with a suitable gradient
based minimisation scheme…
 Problems with this approach

 Resampling the background model:

 Model cannot be too complex
 Resampling will bring in errors
 Motion model is only an estimate of what
is really happening
 Can lead to false object detection –
particularly close to boundaries
 Background Model Design

 The background model needs to be robust

to these problems
 We need some way to differentiate
between genuine object detections and
false ones from motion model and
background model errors
 My Approach

 Rather than updating model values with

current, matched values replace them
 In this way resampling errors are not
allowed to accumulate
 Aside – ‘Real Time’

 The ability to process a sequence in real-

time is dependent on THREE key factors:
 The speed of the algorithm
 The frame rate required
 The number of pixels
 Recap

 Introduction to motion analysis

 Principles of background modelling
 Example of a static scene tracker
 Discussion of motion estimation
 Shortcomings when applied to background
filtering