Behavior Analysis of Individuals

(Learning Based Approaches)

EE508
Dr Mahesh Kolekar
IIT Patna
Learning-based Behavior Analysis

1. Contour-based feature analysis

2. Motion-based feature analysis
1. Contour-based feature analysis
The Steps
of Learning
Method
Contour-based feature analysis
Target Preprocessing

xc, yc

Centroid
xi,yi
Distance of centroid from contour points
Distance of centroid from contour points

•The distances for human blob are then

transformed into coefficients by means of
DFT.
•20 primary coefficients are selected from
these coefficients.
•Thus 80 coefficients are chosen from
four consecutive blobs.
A running person
A bending down person
Person carrying a bar in a crowd
 Person running in
a shopping mall
Challenges: difficult to
obtain background.
Difficult to segment.

Solution: optical flow

method.
Velocity of each pixel is
calculated and then filtered
using 3x3 template.
Person waving
hand in crowd
Person waving hand in crowd
2. Motion–based Feature
Analysis
 Behaviour Motion Analysis

• Human body contour information

• Search of features  Mean shift method
• Learning method Multiple SVM
• By integrating Bayesian framework with
SVM we improve recognition rate.
• Occlusion handling  Head-contour
detection.
 Human Head Detection and Tracking
• Head is highly
probable of being
detected even in
crowd.
Template Kalman • Template matching +
matching Filter Kalman filter for
tracking (since contour
feature can’t always be
detected in the crowd).
• Details of direction,
speed human height
can be obtained from
head template.
Template
matching

Kalman
Filter
Mean-Shift based Motion Feature Searching
Mean-shift method
to search for the
region that has
concentration of
motion
information.

Motion information
= current frame –
previous frame
 Continuously Adaptive Mean Shift
1. Choose Initial location of search window and search window
size.
2. Compute mean location of search window and store the
zeroth moment.
3. Center the search window in the mean location computed in
the previous step.
4. Repeat 2 & 3 until mean location moves less than the preset
threshold.
5. Set search window size = function of zeroth moment in 2.
6. Repeat 2-5 until the mean location moves less than the preset
threshold.

Note: Initial search widow size= size of human head.

Search window

Body Location
 CAMSHIFT contd…
• Centroid of the moving part within the search window can
be calculated from M00, M10, M01
• Generally Mij= Ʃ Ʃ xi yj I(x,y)
• M00= Ʃ Ʃ I(x,y), M10= Ʃ Ʃ x I(x,y) M01= Ʃ Ʃ y I(x,y)
• I(x,y) value of difference image at (x,y).
• Centroid: xc = M10, yc = M01
M00 M00
 History Motion Image Based Analysis
• HMI image is based on foreground image.
• A threshold is needed to transform foreground image into binary
image
1. Update the HMI template T by a foreground at different time stamp τ:
τ, I(x,y)=1
T(x,y)= 0, T(x,y)< τ-δ I difference image
T(x,y), else δtime window to be considered.

2. To find the gradient on HMI template.

where Gy(x, y), Gx(x, y) are the sobel convolution results with the HMI
template in the x and y directions respectively.
Learning Based Behavioural
recognition
Multi-SVM based Bayesian
Network
 Frame Work Analysis

Two approaches:

1. Model all the normal behaviours .Any behaviour

that doesn’t belong to the model will be recognized
as abnormal.
2. Model the abnormal behaviours directly.
 Support vector machine (SVM)
• Supervised learning method that analyze data and recognize
patterns, used for classification and regression analysis.
• The standard SVM takes a set of input data and predicts for each
given input, which of two possible classes forms the input.
• Given a set of training examples, each marked as belonging to one
of two categories, an SVM training algorithm builds a model that
assigns new examples into one category or the other.
• We are given some training data , a set of n points of the form

• where the yi is either 1 or −1, indicating the class to which the

point belongs.
• Any hyperplane can be written as the set of points satisfying

where .
denotes the dot product and the normal vector to the
hyperplane. The parameter determines the offset of the
hyperplane from the origin along the normal vector .
Maximum-margin hyperplane and margins for an SVM
trained with samples from two classes.
Samples on the margin are called the support vectors.
• For each action type, we will set up a SVM model to
give out the probability P( x|Hi )

• P( x|Hi )  probability that an evidence x, x € X

belongs to hypothetic action type Hi
 Recognition by Bayesian Network
• Hypothesis which has the maximum probability among all
the models will be selected as the recognized action type.
• If all the probabilities are smaller than a certain threshold
value, the action will be recognized as abnormal behavior.

• Recognizing

• Considering the context frame results in a improved result.

• Ideally the probability to be obtained is
P(Hit| xt, xt-1, ……..)
Since it is impossible to take into account all the
history information, we use the previous hypothesis.

P(Hit| xt , xt-1, ……..) = P(Hit| Hit-1 , xt)

= P(Hit | Hit-1 ) P(Hit | xt)
Where P(Hi | x)= P(Hi) P(x| Hi)
Σ P(x| Hi) P(Hi)
Therefore by knowing the initial hypothesis Hi0 and the
transfer matrix P(Hit | Hit-1 ) , the probability can be
obtained from the previous result & SVM model
result
 Recognition by Bayesian Network
• Hypothesis which has the maximum probability among all
the models will be selected as the recognized action type.
• If all the probabilities are smaller than a certain threshold
value, the action will be recognized as abnormal behaviour.
 Reference

Huihuan Qian, Xinyu Wu, Yangsheng Xu,

“Intelligent Surveillance Systems”
Springer Publication, 2011
Thank You !