Detection, Tracking and Recognition

in Video Sequences
Project By:

Sonia Kan

Ora Gendler
Supervised By:
Dr. Ofer Hadar
Mr. Uri Perets

Ben-Gurion University of the Negev

Department of Communication Systems Engineering

Outline
Motivation

Our System
Phase 1: Detection

Phase 2: Tracking
Phase 3: Recognition

Result Video

Motivation
National security is a matter of high priority.
Main method of security cameras.

Disadvantage requirement for man power.

A better method is required.

Outline
Motivation

Our System
Phase 1: Detection

Phase 2: Tracking
Phase 3: Recognition

Result Video

Our System
An automatic security system.
Stationary camera.

Goal detect, track and recognize a moving

human object in a video sequence.
Each phase is implemented as a separate algorithm.

Simulation in Matlab.

Hall Monitor

A General View on the System

State machine :
Detect

Recognize

Track

No

Tracking
successful?

Yes

Outline
Motivation

Our System
Phase 1: Detection

Phase 2: Tracking
Phase 3: Recognition

Result Video

Object Detection
Use Canny edge detector to
find all edge maps.
Use four images:
Background edge map Eb.
Difference edge map DEn.
Current edge map En.
Previous algorithm output MEn-1.

Previous gray
level image

Current gray
level image

I n-1

In

Find edge map of difference

| I n-1 - I n |
image

Eb

DEn

En

Find moving object edge

MEn for frame n

MEn-1
Delay

MEn

Object Detection (Cont.)

Background
frame

Extracted background
edge map

Object Detection (Cont.)

Use four images:

Previous gray
level image

Current gray
level image

I n-1

In

Background edge map Eb.

Difference edge map DEn.
Current edge map En.
Previous algorithm output
Men-1.

Difference image contains

the moving parts of the
frame.

Find edge map of difference

| I n-1 - I n |
image

Eb

DEn

En

Find moving object edge

MEn for frame n

MEn-1
Delay

MEn

Object Detection (Cont.)

Edges
extracted from
the difference
image

Edges
extracted from
the original
frame

Current frame

Object Detection (Cont.)

Use four images:
Background edge map Eb.
Difference edge map DEn.
Current edge map En.
Previous algorithm output MEn-1.

Find pixels that belong to the moving parts of the object

MEnchange .
Find pixels that belong to the still parts of the object MEnstill .
Combine the two components MEn MEnchange MEnstill .

Detection Result

Final edge map

Outline
Motivation

Our System
Phase 1: Detection

Phase 2: Tracking
Phase 3: Recognition

Result Video

Object Tracking
Input:
Current edge map

Previous edge map

Goal - estimate objects location in the next

frame.
The estimation is performed for several frames,
until tracking fails. In that case, the object is redetected.

Object Tracking (Cont.)

Divide the previous edge map into square
blocks.
Use thresholding to determine which blocks
contain parts of objects.

Empty block

Block containing object

Object Tracking (Cont.)

Locate the blocks containing objects from the
previous edge map in the current edge map:
To match the blocks, we use correlation.

previous

Matched

Current

Correlation Statistics
Average correlation per frame
0.8

Correlation %

0.75

0.7

0.65

0.6

0.55

30

40

50

frame#

60

70

80

Matching Results
Previous
edge map
with blocks

Previous edge map

Current edge
map with
matched
blocks

Object Tracking (Cont.)

For each matched block, calculate the amount of pixels it
moved in each axis.
Calculate the average E and standard deviation , and
divide into five ranges.
E

Object Tracking (Cont.)

Estimate the location of each block in the
next edge map according to the
corresponding speed.
Verify true location of each block using
correlation.

Tracking Statistics
percentage of detected blocks per frame
1.1
1

detected blocks %

0.9
0.8

Threshold

0.5

0.7
0.6

0.4

30

40

50
frame#

60

70

80

Tracking Results
Current, next and following frames,
double estimation.

Outline
Motivation

Our System
Phase 1: Detection

Phase 2: Tracking
Phase 3: Recognition

Result Video

Scenario Analysis
Static

Environment

Dynamic
Actors

Size
Motion
Interaction

Equipment

Static
Dynamic
Interaction

Recognition
Monitor amount of objects

Recognize behavior of human object

Size based on detected edges
Size indicates motion inwards or outwards

Motion based on tracked blocks

Recognize sudden changes in speed as suspicious

Interaction with environment based on skeleton

Recognize suspicious postures.

Monitoring Amount of Objects

Skeleton Construction
Locate center of mass
Using:

1 Nb
xc
xi ;

Nb i 1

( xc , yc )

1 Nb
yc
yi

Nb i 1

Find end points of limbs

Mark spine and limbs location

Calculate angles between spine
and limbs to recognize the
posture

Outline
Motivation

Our System
Phase 1: Detection

Phase 2: Tracking
Phase 3: Recognition

Result Video

Separated Objects Demo

You!
ClickThank
to edit Master
title style
Click to edit Master subtitle style

32

f*

Freq. Domain Correlation

If there is a match between an image f and an object
h, the correlation will be maximal at the location of
h in f.
Correlation is given by:
1
f ( x, y ) h ( x , y )
M N

M 1 N 1

f * ( m, n ) h ( x m , y n )

m 0 n 0

When: f ( x, y) h( x, y) F * (u, v) H (u, v)

*

f denotes the complex conjugate of f.

Mathematical Background
Difference edge map:
DEn (| I n 1 I n |)

Pixels that belong to the moving parts of the object :

MEnchange {e En | min || e x || Tchange }
xDEn

Pixels that belong to the still parts of the object :

MEnstill {e En | e Eb , min || e x || Tstill }

Result:

xMEn 1

MEn MEnchange MEnstill