02-11-11 Hws Slides

Object Detection Using Semi-
Naïve Bayes to Model Sparse

Structure
Henry Schneiderman
Robotics Institute
Carnegie Mellon University
Object Detection
• Find all instances of object X (e.g. X = human faces)

Examples of Detected Objects
Sparse Structure of Statistical Dependency
Chosen variable Chosen variable Chosen variable

Chosen coefficient Chosen coefficient

Chosen coefficient
Chosen coefficient Chosen coefficient Chosen coefficient

Detection using a Classifier
“Object is present”
(at fixed size and alignment)
Classifier
“Object is NOT present”

Proposed Model: Semi-Naïve Bayes
subsets input variables
S1 ,..., S n  {x1 ,..., xr }
e.g. S1 = (x21, x34, x65, x73, x123)

S2 = (x3, x8, x17, x65, x73, x111)
P ( S1 |  1 ) P( S 2 | 1 ) P( S n | 1 )
H ( x1 ,... x r )  log  log  ...  log 
P ( S1 |  2 ) P( S 2 |  2 ) P( S n |  2 )
• Kononenko (1991), Pazzini (1996), Domingos and

Pazzini (1997), Rokach and Maimon (2001)
Goal: Automatic subset grouping
S1 = (x21, x34, x65, x73, x123)

S2 = (x3, x8, x17, x65, x73, x111)
...
Sn = (x14, x16, x17, x23, x85, x101, x103, x107)
P ( S1 |  1 ) P( S 2 | 1 ) P( S n | 1 )
H ( x1 ,... x r )  log  log  ...  log 
P ( S1 |  2 ) P( S 2 |  2 ) P( S n |  2 )
Approach: Selection by Competition
x 1 x2 x3 . . . x m
Generate q candidate subsets

S1 S2 ... Sq
Train q log likelihood function

log [p1(S1|1) / p1(S1|2)] log [p2(S2|1) / p2(S2|2)] . . . log [pq(Sq|1) / pq(Sq|2)]
q functions
Select combination of n candidates
log [pj1(Sj1|1) / pj1(Sj1|2)] log [pj2(Sj2|1) / pj2(Sj2|2)] . . . log [pjn(Sjn|1) / pjn(Sjn|2)]
n functions, n << q
H(x1,…,xr) =
log [pj1(Sj1|1) / pj1(Sj1|2)]+log [pj2(Sj2|1) / pj2(Sj2|2)] +. . .+ log [pjn(Sjn|1) / pjn(Sjn|2)]
x 1 x2 x3 . . . x m

S1 S2 ... Sq


H(x1,…,xr) =
Generation of Subsets
• “modeling error for assuming independence”
P ( x ... x | ) P ( x | )
C1   ... P ( x1...xq )abs[log P ( x11... xqq | 12 )  log PP(( xx11||12 )) P ( x2 |1 )
P ( x2 | 2 ) ... P ( xqq | 12 ) ]dx1...dxq
q is size of the subset

Generation of Subsets
• Selection of variables - “discrimination power”
P ( x ... x | )
C2   ... P ( x1...xq )abs[log P ( x11... xqq | 12 ) ]dx1...dxq
q is size of the subset

Pair-Wise Measurement
• pair-wise measurements
 P( x j , xk )abs[log P ( x jj , xkk |12 )  log P ( x jj|12 )
P ( x , x | ) P ( x | ) P ( xk |1 )
C1 ( x j , xk )  P ( xk | 2 ) ]dx j dxk
x j , xk
 P( x j , xk )abs[log P ( x jj , xkk |12 ) ]dx j dxk

P ( x , x | )
C 2 ( x j , xk ) 
x j , xk
C ( x j , xk )  C1 ( x j , xk )  C2 ( x j , xk ) Pair-affinity
Visualization of C(x,*)
(frontal faces)
x
x x
Measure over a Subset
D( Si )  1
N ( N 1)  C(x , x )
x j , xk S i
j k N  Si Subset-affinity
x j  xk
Generation of Candidate Subsets
x1 x2 x3 . . . . . . . . . . . . . . . . . . . . . . . . x m
C(x1, x2) C(x1, x3) . . . . . . . . C(xm-1, xm)
Heuristic search and selective evaluation of D(Si)
S1 S2 .................. Sp
subset size vs. modeling power
• Model complexity limited by number of

training examples, etc.
• Examples of limited modeling power
– 5 modes in a mixture model
– 7 projection onto principal components
x 1 x2 x3 . . . x m

S1 S2 ... Sq


H(x1,…,xr) =
Log-likelihood function = Table
Si = (xi1, xi2, . . ., xiq)
vector quantization
f i  (1...N i )
table look-up
log( PP(( ffii||21)) )

Sub-Classifier Training by Counting
fi
Pi (fi |1)
fi
Pi (fi |2)
Example of VQ
xi1 xi2 xi3 . . . xiq
projection on to 3 principal components
c1 c2 c 3
quantization to m levels
z1 z2 z3
f = z1m0 + z2m1 + z3m2

x 1 x2 x3 . . . x m

S1 S2 ... Sq


H(x1,…,xr) =
Candidate
log-likelihood h1(S1) h2(S2) ... hP(SP)
functions
Evaluate on
E E1,2 E2,1 E2,2 . . . Ep,1 Ep,2
training data 1,1
Evaluate
ROCs
ROC1 ROC2 ... ROCP
Order top Q
log-likelihood
hj1(Sj1) hj2(Sj2) . . . hjQ(SjQ)
functions
Form pQ pairs hj1(Sj1) + h1(S1) ... hjQ(SjQ) + hp(Sp)
of log-likelihood
functions
Sum
Evaluations Ej1,1+ E1,1 Ej1,2+ E1,2 . . . EjQ,1+ Ep,1 EjQ,2+ Ep,2
Evaluate
ROCs
ROC1 ... ROCQP
Order top Q
pairs of
hk1,1(Sk1,1) + hk1,2(Sk1,2) . . . hkQ,1(SkQ,1) + hkQ,2(SkQ,2)
log-likelihood
functions
. . . Repeat for n iterations
Cross-Validation Selects Classifier
Q Candidates:
H1(x1, x2, . . ., xr) = hk1,1(Sk1,1) + hk1,2(Sk1,2) +. . .+ hk1,n(Sk1,n)
...
HQ(x1, x2, . . ., xr) = hkQ,1(SkQ,1) + hkQ,2(SkQ,2) + . . . + hQ,n(SkQ,n)
H1(x1, x2, . . ., xr) . . . HQ(x1, x2, . . ., xr)
Cross-validation
H*(x1, x2, . . ., xr)

Example subsets learned for telephones
Evaluation of Classifier
Classifier

1) Compute feature values
f1 = #5710
f2 = #3214
fn = #723
2) Look-Up Log-Likelihoods
P1( #5710 | 1)
f1 = #5710 log = 0.53
P1( #5710 | 2)
P2( #3214 | 1)

f2 = #3214 log = 0.03
P2( #3214 | 2)
Pn( #723 | 1)

fn = #723 log = 0.23
Pn( #723 | 2)
3) Make Decision
P1( #5710 | 1)
log = 0.53
P1( #5710 | 2)
P2( #3214 | 1)

log = 0.03
P2( #3214 | 2)
 0.53 + 0.03 + . . . + 0.23 >
<
Pn( #723 | 1)

log = 0.23
Pn( #723 | 2)
Detection using a Classifier
Classifier

View-based Classifiers
Face
Classifier #1
Face
Classifier #2
Face
Classifier #3
Detection: Apply Classifier
Exhaustively
Search in position
Search in scale
Decision can be made by partial evaluation
P1( #5710 | 1)

log = 0.53
P1( #5710 | 2)
P2( #3214 | 1)

log = 0.03
P2( #3214 | 2)
 0.53 + 0.03 + . . . + 0.23 >
<
Pn( #723 | 1)

log = 0.23
Pn( #723 | 2)
Detection Computational Strategy
Apply log [p1(S1|1) / p1(S1|2)] Apply log [p2(S2|1) / p2(S2|2)] Apply log [p3(S3|1) / p3(S3|2)]
exhaustively to scaled input image reduced search space further reduced search space
Computational strategy changes

with size of search space
Compute M2 feature values Look-up M2 log-likelihood values
Repeat for N2 Candidates
Candidate-Based Evaluation
Compute N2 + M2 +2MN feature values Look-up M2 log-likelihood values
Repeat for N2 Candidates
Feature-Based Evaluation
Cascade Implementation
Create candidate subsets
Train candidate log-likelihood functions
Training images of non-

Training images
object
of object Select log-likelihood functions
Retrain selected log-likelihood Adaboost using confidence-rated

functions using Adaboost predictions [Shapire and Singer, 1999]
Cross-validation
images Images that do
Determine detection threshold not contain
object
Automatically select non-object Bootstrapping [Sung and Poggio, 1995]

examples for next stage
Increment stage
Face, eye, ear detection
Frontal Face Detection
• MIT-CMU Frontal Face Test Set [Sung and Poggio, 1995; Rowley, Baluja and Kanade, 1997]
– 180 ms 300x200 image

– 400 ms 300x500 image
• Top Rank Video TREC 2002 Face Detection
• Top Rank 2002 ARDA VACE Face Detection algorithm evaluation
Recognition rate 85.2% 89.7% 92.1% 93.7% 94.2%
False detections 6 13 44 64 79
(this method)
False Detections -- 31 50 167 --
[Viola and Jones,
CVPR, 2001]
AMD Athalon
1.2GHz
Face & Eye Detection for Red-Eye
Removal from Consumer Photos
CMU Face Detector

Eye Detection
• Experiments performed independently at
NIST
• Sequested data set: 29,627 mugshots
• Eyes correctly located (radius of 15 pixels)
98.2% (assumed one face per image)
• Thanks to Jonathon Phillips, Patrick
Grother, and Sam Trahan for their
assistance in running these experiments
Realistic Facial Manipulation:
Earring Example
With Jason Pinto
Telephone Detection
Cart, pose 1
Cart, pose 2
Cart, pose 3
Door Handle Detection
Summary of Classifier Design
• Sparse structure of statistical dependency in many
image classification problem
• Semi-naïve Bayes Model
• Automatic learning structure of semi-naïve Bayes
classifier:
– Generation of many candidate subsets
– Competition among many log-likelihood functions to
find best combination
CMU on-line face detector:

http://www.vasc.ri.cmu.edu/cgi-bin/demos/findface.cgi

02-11-11 Hws Slides

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

02-11-11 Hws Slides

Uploaded by

Copyright:

Available Formats

Object Detection Using Semi-

Naïve Bayes to Model Sparse

• Find all instances of object X (e.g. X = human faces)

Chosen variable Chosen variable Chosen variable

Chosen coefficient Chosen coefficient

Chosen coefficient Chosen coefficient Chosen coefficient

“Object is NOT present”

S1 ,..., S n  {x1 ,..., xr }

e.g. S1 = (x21, x34, x65, x73, x123)

• Kononenko (1991), Pazzini (1996), Domingos and

S1 = (x21, x34, x65, x73, x123)

Generate q candidate subsets

Train q log likelihood function

Generate q candidate subsets

Train q log likelihood function

Select combination of n candidates

q is size of the subset

q is size of the subset

 P( x j , xk )abs[log P ( x jj , xkk |12 ) ]dx j dxk

C(x1, x2) C(x1, x3) . . . . . . . . C(xm-1, xm)

Heuristic search and selective evaluation of D(Si)

• Model complexity limited by number of

Generate q candidate subsets

Train q log likelihood function

Select combination of n candidates

log( PP(( ffii||21)) )

f = z1m0 + z2m1 + z3m2

Generate q candidate subsets

Train q log likelihood function

Select combination of n candidates

H1(x1, x2, . . ., xr) . . . HQ(x1, x2, . . ., xr)

H*(x1, x2, . . ., xr)

“Object is NOT present”

P2( #3214 | 1)

Pn( #723 | 1)

P2( #3214 | 1)

Pn( #723 | 1)

“Object is NOT present”

P1( #5710 | 1)

P2( #3214 | 1)

Pn( #723 | 1)

Computational strategy changes

Repeat for N2 Candidates

Repeat for N2 Candidates

Train candidate log-likelihood functions

Training images of non-

Retrain selected log-likelihood Adaboost using confidence-rated

Automatically select non-object Bootstrapping [Sung and Poggio, 1995]

– 180 ms 300x200 image

Recognition rate 85.2% 89.7% 92.1% 93.7% 94.2%

CMU Face Detector

CMU on-line face detector:

You might also like