FACE DETECTION

OUTLINE
What

is face detection.

The

History. Challenges. 2D-Image Scan. Biometrics : Skin Texture Analysis.

Evaluation of face detection The Viola and Jones method

Applications.
2

INTRODUCTION
Steps:

Face Detection: differentiate a human face

from the background of the image or a real time video.

Feature Detection: record its features. Face Recognition: Compare it to a data

base.

INTRODUCTION

Face interface
Face

detection Face recognition

Face detection

Face database

Output:

Mr.Chan
Face recognition Prof..Cheng

WHAT IS FACE DETECTION

Technique

employed to distinguish a Human face from the rest of the background of the image.
5

FACE DETECTION [1]

To detect faces in an image (Not recognize it yet) Challenges

A picture has 0,1 or many faces Faces are not the same: with spectacles, mustache etc Sizes of faces vary

Available in most digital cameras nowadays The simple method

Slide a window across the window and detect faces

Too

slow, pictures have too many pixels (1280x1024=1.3M pixels)

6

THE HISTORY

During 1964 and 1965, Bledsoe, along with Helen Chan and Charles Bisson , worked on using the computer to recognize human faces. He was proud of this work, but because the funding was provided by an unnamed intelligence agency that did not allow much publicity, little of the work was published.
7

IMPORTANCE OF FACE DETECTION

The first step for any automatic face recognition system system. First step in many Human Computer Interaction systems. Expression Recognition Cognitive State/Emotional State Recognition First step in many surveillance and security systems. Video coding Automatic Target Recognition(ATR)
8

CHALLENGES
In Plane Rotation Out Plane Rotation Lighting Aging Effects Facial Expressions Face Covered by long Hairs or Hand.

CHALLENGES

10

2D IMAGE SCAN

Different Approaches: Knowledge Based Approach Feature Invariant Method Template Matching Method
11

KNOWLEDGE-BASED APPROACH

It uses humancoded rules to model facial features, such as two symmetric eyes, a nose in the middle and a mouth underneath the nose.
12

KNOWLEDGE-BASED APPROACHSUMMARY

pros: Easy to come up with simple rules Based on the coded rules, facial features in an input image are extracted first, and face candidates are identified Work well for face localization in uncluttered background
con: Difficult to translate human knowledge into rules precisely: detailed rules fail to detect faces and general rules may find many false positives Difficult to extend this approach to detect faces in different poses: implausible to enumerate all the possible cases

13

FEATURE INVARIANT METHOD

Feature invariant methods try to find facial features which are invariant to pose, lighting condition or rotation. Skin colors, edges and shapes fall into this category.

14

FEATURE INVARIANT METHODNODAL POINT ANALYSIS

Every face has numerous, distinguishable landmarks, the different peaks and valleys that make up the face o Distance between the eyes o Width of the nose o Depth of the eye sockets o The shape of the cheekbones

15

FEATURE INVARIANT METHODSUMMARY

Pros: Features are invariant to pose and change in orientation. Cons: Difficult to locate facial features due to several corruption (illumination, noise, occlusion) Difficult to detect features in complex
16

TEMPLATE MATCHING METHOD

Template matching methods calculate the correlation between a test image and a preselected facial templates.

17

TEMPLATE MATCHING METHODSUMMARY

Pros: Simple Cons: Templates needs to be initialized near the face images Difficult to enumerate templates for different poses (similar to knowledge-based methods)

18

BIOMETRICS SKIN TEXTURE ANALYSIS

Using skin color to find face segments is a vulnerable technique. Non-animate objects with the same color as skin can be picked up since the technique uses color segmentation. Then the face can be picked up using any of the approaches.
19

SKIN TEXTURE ANALYSIS: ADVANTAGES Lack of restriction to orientation or size of faces. A good algorithm can handle complex backgrounds. It is relatively insensitive to changes in expression, including blinking, frowning or smiling Has the ability to compensate for mustache or beard growth and the appearance of eyeglasses.

20

EVALUATION OF FACE DETECTION

Detection rate Give positive results in locations where faces exist Should be high > 95% False positive rate The detector output is positive but it is false (there is actually no face).Definition of False positive: A result that is erroneously positive when a situation is normal. An
example of a false positive: a particular test designed to detect cancer of the toenail is positive but the person does not have toenail cancer. (http://www.medterms.com/script/main/art.asp?articlekey=3377) Should be low <10-6

A good system has High detection rate Low false positive rate

False positive result

21

EXERCISE

What are the detection rate and false detection rate here?
9 faces in the picture, 8 are correctly detected.

Answer
detection

rate=(8/9)*100% false detection rate=(1/9)*100%

1 window reported to have face is in fact not a face False positive result

22

THE VIOLA AND JONES METHOD [1]

The most famous method Training may need weeks Recognition is very fast, e.g. real-time for digital cameras. Techniques

Integral

image for feature extraction Ada-Boost for feature selection Attentional cascade for fast rejection of non-face sub-windows
23

IMAGE FEATURES REF[3]

Rectangle filters

Rectangle_Feature_value f=
(pixels in white area) (pixels in shaded area)
24

EXERCISE

Find the Rectangle_Feature_val ue (f) of the box enclosed by the dotted line
Rectangle_Feature_value f= (pixels in white area) (pixels in shaded area) f=(8+7)-(0+1) =15-1= 14

1 3 5

2 0 8

3 1 7

3 3 1

25

EXAMPLE: A SIMPLE FACE DETECTION METHOD USING ONE FEATURE

Rectangle_Feature_value f f= (pixels in white area) (pixels in shaded area) If (f) is large it is face ,i.e. if (f)>threshold, then face Else non-face

Result

This is a face:T he eye-part is dark, the nose-part is bright So f is large, hence it is face This is not a face. Because f is small

26

HOW TO FIND FEATURES FASTER INTEGRAL IMAGES FAST CALCULATION METHOD [LAZEBNIK09 ]

The integral image = sum of all pixel values above and to the left of (x,y) Can be found very quickly

(x,y)

27

COMPUTING THE INTEGRAL IMAGE [LAZEBNIK09 ]

28

COMPUTING THE INTEGRAL IMAGE [LAZEBNIK09 ]

ii(x, y-1)

s(x-1, y)
i(x, y)

Cumulative row sum: s(x, y) = s(x1, y) + i(x, y) Integral image: ii(x, y) = ii(x, y1) + s(x, y)

29

CALCULATE SUM WITHIN A RECTANGLE

A,B,C,D are the values of the integral images at the corners of the rectangle R. The sum of image values inside R is: Area_R = A B C + D If A,B,C,D are found , only 3 additions are needed to find Area_R

D Rectangle R Area= Area_R

30

WHY DO WE NEED TO FIND PIXEL SUM OF RECTANGLES? ANSWER: WE WANT TO GET FACE FEATURES

You may consider these features as face features

Two eyes= (Area_A-Area_B) Nose =(Area_C+Area_E-Area_D) Mouth =(Area_F+Area_H-Area_G)

A B C D E F G H

They can be different sizes, polarity and aspect ratios

31

FACE FEATURE AND EXAMPLE

Shaded area

Pixel values inside The areas 10 20 4 7 45 7

-1 216 102 78 Integral White area Image 129 210 111 +2 F=Feat_val = pixel sum in shared area - pixel sum in white area Example Pixel sum in white area= 216+102+78+129+210+111=846
Pixel sum in shared area= 10+20+4+7+45+7=93 Feat_val=F=846-93 If F>threshold, feature=+1 Else feature=-1 End if; We can choose threshold =768 , so feature is +1.

A face

32

THE DETECTION CHALLENGE

(x,y) 24x24Sub-window

Use 24x24 base window For y=1;y<=1024;y++

For x=1;x<=1280;x++{

(1,1)

X-axis 1280

Set (x,y) = the left top corner of the 24x24 sub-window For the 24x24 sub-window, extract 162,336 features and see they combine to form a face or not. }

Yaxis

1024

Conclusion : too slow

33

SOLUTION TO MAKE IT EFFICIENT

The whole 162,336 feature set is too large

Solution:

select good features to make it more

efficient Use: Boosting

Boosting
Combine

many small weak classifiers to become a strong classifier. Training is needed

34

BOOSTING FOR FACE DETECTION

Define weak learners based on rectangle features

value of rectangle feature

1 if pt f t ( x ) ptt ht ( x ) otherwise 0 window

threshold Pt= polarity{+1,-1}

35

FACE DETECTION USING ADABOOST

AdaBoost training
E.g. Collect 5000 faces, and 9400 non-faces. Different scales. Use AdaBoost for training to build a strong classifier. Pick suitable features of different scales and positions, pick the best few. (Take months to do , details is in [Viola 2004] paper)

Testing
Scan through the image, pick a window and rescale it to 24x24, Pass it to the strong classifier for detection. Report face, if the output is positive

36

BOOSTING FOR FACE DETECTION [VIOLA2004]

In the paper it shows that the following two features (obtained after training) in cascaded picked by AdaBoost have 100% detection rate and 50% false positive rate I.e. But 50% false positive rate is not good enough H(face)= Approach [viola2004] :Attentional cascade Sign{1h1(image)
+2h2(image)} Pick a window in the image and rescale it to 24x24 as image
H(face)=+1 face H(face)=-1non-face

h1(image) type2

h2(image) type3

37

BOOSTING FOR FACE DETECTION

A 200-feature classifier can yield 95% detection rate and a false positive rate of 1 in 14084 (Still not god enough) Recall: False positive rate The detector output is positive but it is false (there is actually no face). Definition of False positive: A result that is erroneously positive when a situation is normal. An example of a false positive: a particular test designed to detect cancer of the toenail is positive but the person does not have toenail cancer. (http://www.medterms.com/script/main/art.asp?articlekey=3377)

Correct Detection rate

Still not good enough!

False positive rate X10-3
38

TO IMPROVE FALSE POSITIVE RATE: ATTENTIONAL CASCADE

Cascade of many AdaBoost strong classifiers Begin with with simple classifiers to reject many negative sub-windows Many non-faces are rejected at the first few stages. Hence the system is efficient enough for real time processing.
Adaboost Classifier1

Input image

True

Adaboost Classifier2

True

Adaboost True Classifier3 False Non-face

Face found

False Non-face

False Non-face

39

AN EXAMPLE

More features for later stages in the cascade [viola2004]

type2 type3

2 features Input image Adaboost Classifier1

10 features

25 features

50 features

True

Adaboost Classifier2

True

Adaboost True Classifier3 False Non-face

Face found

False Non-face

False Non-face

40

ATTENTIONAL CASCADE

Chain classifiers that are progressively more complex and have lower false positive rates:
% Detection

Receiver operating characteristic

% False Pos 0 100 50

vs false neg determined by

Input image Adaboost Classifier1

False positive rate

True

Adaboost Classifier2

True

Adaboost True Classifier3 False Non-face

Face found

False Non-face

False Non-face

41

ATTENTIONAL CASCADE [VIOLA2004]

Detection rate for each stage is 0.99 , for 10 stages,

overall detection rate is 0.9910 0.9

False positive rate at each stage is 0.3, for 10 stages

false positive rate =0.310 610-6)

True True

Input image Adaboost Classifier1

Adaboost Classifier2

Adaboost True Classifier3 False Non-face

Face found

False Non-face

False Non-face

42

Face Recognition DETECTION PROCESS IN PRACTICE Face Recognition [SMYTH2007]

Use 24x24 sub-window Scaling

scale

the detection (not the input image) Features evaluated at scales by factors of 1.25 at each level Location : move detector around the image (1 pixel increments)

Final detections
A

real face may result in multiple nearby detections (merge them to become the final result)
43

FACE RECOGNITION

2009 TotallyLooksLike.com

Distinguishing a specific face from other faces

 Biometrics / access control

o o o o

FACE RECOGNITION: APPLICATIONS

no action required scan many people at once places: airports, banks, safes data: laptops, medical info

""Minority Report" 2002 20th Century Fox

Searching mugshot databases Tagging photo albums Detecting fake ID cards

Superbad" 2007 Columbia Pictures

Security measure at ATMs Digital Cameras Public Surveillance (CCTVs) at Airports, Hospitals, etc. Televisions and computers can save energy by reducing the brightness.

APPLICATIONS

46