Face detection and recognition using image processing

Group 11

Group Members
Jam Zia-ul-Haq Hafiz Nisar Ahmad Muhammad Yasir

contents
Introduction Face detection Face recognition Implementation

Introduction
Face interface Face detection Face recognition
Face detection Face recognition Face database

Output: Mr.Chan Prof..Cheng

What is Face Detection?

Given an image, tell whether there is any human face, if there is, where is Click to edit Master text styles it(or where they Second level are).

Click to edit Master text styles Second level Third level Fourth level Fifth level

Third level Fourth level Fifth level

Importance of Face Detection

The first step for any automatic face recognition system system First step in many Human Computer Interaction systems
Expression Recognition Cognitive State/Emotional State Recogntion

First step in many surveillance systems Tracking: Face is a highly non rigid object A step towards Automatic Target Recognition(ATR) or generic object detection/recognition Video coding

Face Detection: current state

State-of-the-art: Front-view face detection can be done at >15 frames per second on 320x240 black-and-white images on a 700MHz PC with ~95% accuracy. Detection of faces is faster than detection of edges! Side view face detection remains to be difficult.

Face Detection: challenges

Out-of-Plane Rotation: frontal, 45 degree, profile, upside down Presence of beard, mustache, glasses etc Facial Expressions Occlusions by long hair, hand In-Plane Rotation Image conditions:
Size Lighting condition Distortion Noise Compression

Different Approaches
Knowledge-based methods:
Encode what constitutes a typical face, e.g., the relationship between facial features

Feature invariant approaches:

Aim to find structure features of a face that exist even when pose, viewpoint or lighting conditions vary

Template matching:
Several standard patterns stored to describe the face as a whole or the facial features separately

Appearance-based methods:
The models are learned from a set of training images that capture the representative variability of faces.

Knowledge-Based Methods
Top Top-down approach: Represent a face using a set of human-coded rules Example:
The center part of face has uniform intensity values The difference between the average intensity values of the center part and the upper part is significant A face often appears with two eyes that are symmetric to each other, a nose and a mouth

Use these rules to guide the search process

Knowledge-Based Method: [Yang and Huang 94]

Level 1 (lowest resolution):
apply the rule the center part of the face has 4 cells with a basically uniform intensity to search for candidates

Level 2: local histogram equalization followed by edge equalization followed by edge detection Level 3: search for eye and mouth features for validation

Knowledge-based Methods: Summary

Pros:
Easy to come up with simple rules Based on the coded rules, facial features in an input image are extracted first, and face candidates are identified Work well for face localization in uncluttered background

Cons:
Difficult to translate human knowledge into rules precisely: detailed rules fail to detect faces and general rules may find many false positives Difficult to extend this approach to detect faces in different poses: implausible to enumerate all the possible cases

Feature-Based Methods
Bottom-up approach: Detect facial features (eyes, nose, mouth, etc) first Facial features: edge, intensity, shape, texture, color, etc Aim to detect invariant features Group features into candidates and verify them

Feature-Based Methods: Summary

Pros: Features are invariant to pose and orientation change Cons: Difficult to locate facial features due to several corruption (illumination, noise, occlusion) Difficult to detect features in complex background

Template Matching Methods

Store a template
Predefined: based on edges or regions

Deformable: based on facial contours (e.g., Snakes) Templates are hand-coded (not learned) Use correlation to locate faces

Click to edit Master text styles Second level Third level Fourth level Fifth level

Template-Based Methods: Summary

Pros: Simple Cons: Templates needs to be initialized near the face images Difficult to enumerate templates for different poses (similar to knowledge-based methods)

Image Features
Rectangle filters

Rectangle_Feature_value f= (pixels in white area) (pixels in shaded area)

Example
1
Find the Rectangle_Feature_valu e (f) of the box enclosed by the dotted line
Rectangle_Feature_value f= (pixels in white area) (pixels in shaded area) f=(8+7)-(0+1) =15-1= 14

2 0 8 2

3 1 7 3

3 3 1 6

3 5 0

Example: A simple face detection method using one Rectangle_Feature_value featurewhite area) f (pixels in shaded f= (pixels in
q q

area)
q

If (f) is large it is face ,i.e. qif (f)>threshold, then q face qElse q non-face

Result

This is a face:T he eye-part is dark, the nose-part is bright So f is large, hence it is face This is not a face. Because f is small

Why do we need to find pixel sum of rectangles? Answer: We want to get face features
You may consider these features as face features
Two eyes=
(Area_A-Area_B)

A B

Nose =(Area_C+Area_E-Area_D) Mouth =(Area_F+Area_HArea_G)

C D E F G H

They can be different sizes, polarity and aspect ratios

Face feature and example

Shaded area 7

Pixel values inside The areas 10 20 4 45 7

-1 216 102 78 Integral White area Image 129 210 111 +2 F=Feat_val = pixel sum in shared area - pixel sum in white area Example Pixel sum in white area= 216+102+78+129+210+111=846
Pixel sum in shared area= 10+20+4+7+45+7=93

A face

Feat_val=F=846-93 If F>threshold, feature=+1 Else feature=-1 End if; We can choose threshold =768 , so feature is +1.

4 basic types of features for white_area-gray_area

Type) Rows x columns Type 1) 1x2 Type 2) 2x1 Type 3) 1x3 Type 4) 3x1
22 22

Each basic type can have difference sizes and aspect ratios.

Feature selection

For a 24x24 detection region, the number of possible rectangle features is ~160,000!

Some examples and their types Fill in the types for the 2nd, 3rd rows
2 3 5 1 4

Type 1) 2) 3) 4) 5)

23 23

The detection challenge

Use 24x24 base window For y=1;y<=1024;y++
For x=1;x<=1280;x++{ Set (x,y) = the left top corner of the 24x24 sub-window For the 24x24 sub-window, extract 162,336 features and see they combine to form a face or not. }
Yaxis (x,y) 24x24Sub-window (1,1) X-axis 1280

1024

Conclusion : too slow

24 24

Solution to make it efficient

The whole 162,336 feature set is too large Solution: select good features to make it more efficient Use: Boosting Boosting Combine many small weak classifiers to become a strong classifier. Training is needed

2525

Boosting for face detection

Define weak learners based on rectangle features

value of rectangle feature

window

threshold Pt= polarity{+1,-1}

26 26

Face detection using Adaboost

AdaBoost

training

E.g. Collect 5000 faces, and 9400 non-faces. Different scales. AdaBoost for training to build a strong classifier. suitable features of different scales and positions, pick the best few. (Take months to do , details is in [Viola 2004] paper) through the image, pick a window and rescale it to 24x24, it to the strong classifier for detection. face, if the output is positive
27 27

Use

Pick

Testing
Scan Pass

Report

Boosting for face detection [viola2004]

In the paper it shows that the following two features (obtained after training) in cascaded picked by AdaBoost have 100% detection rate and 50% false positive rate
I.e. But 50% false positive rate is not good enough H(face)= Sign{1h1(image) Approach [viola2004] :Attentional cascade +2h2(image)}
H(face)=+1 face H(face)=-1non-face

Pick a window in the image and rescale it to 24x24 as image

h1(image) type2

h2(image) type3

28 28

Boosting for face detection

A 200-feature classifier can yield 95% detection rate and a false positive rate of 1 in 14084 (Still not god enough) Recall: False positive rate

The detector output is positive but it is false (there is actually no face). Definition of False positive: A result that is erroneously positive when a situation is normal. An example of a false positive: a particular test designed to detect cancer of the toenail is positive but the person does not have toenail cancer. (http://www.medterms.com/script/main/art.asp?articlekey=3377)

Correct Detection rate

Still not good enough!

False positive rate X10-3 29 29

To improve false positive rate: Attentional cascade

Cascade of many AdaBoost strong classifiers with with simple classifiers to reject many negative sub-windows non-faces are rejected at the first few stages. the system is efficient enough for real time processing.
Face found
30 30

Begin Many

Hence

Input image Adaboost True AdaboostTrueAdaboostTrue Classifier1 Classifier2 Classifier3 False False False Non-face Non-face Non-face

An example

More features for later stages in the cascade

type2 type3

2 features

10 features

25 features

50 features

Input image Adaboost True AdaboostTrueAdaboostTrue Classifier1 Classifier2 Classifier3 False False False Non-face Non-face Non-face

Face found

3131

Attentional cascade

Chain classifiers that are progressively more complex and have lower false positive rates:

Receiver operating characteristic

% False Pos 0 50

% Detection 0

100

vsfalseneg determined by

Input image Adaboost True AdaboostTrueAdaboostTrue Classifier1 Classifier2 Classifier3 False False False Non-face Non-face Non-face

False positive rate

Face found
32 32

Attentional cascade
Detection

rate for each stage is 0.99 , for 10 stages,

overall detection rate is 0.9910 0.9

False

positive rate at each stage is 0.3, for 10 stages

false

positive rate =0.310 610-6)

Face found
33 33

Input image Adaboost True AdaboostTrueAdaboostTrue Classifier1 Classifier2 Classifier3 False False False Non-face Non-face Non-face

Detection process in practice

Use 24x24 sub-window Scaling scale the detection (not the input image) Features evaluated at scales by factors of 1.25 at each level Location : move detector around the image (1 pixel increments) Final detections A real face may result in multiple nearby detections (merge them to become the final result)
3434

Face Recognition

Face Recognition by Humans

Performed routinely and effortlessly by humans Enormous interest in automatic processing of digital images and videos due to wide availability of powerful and low-cost desktop embedded computing

Applications: biometric authentication, surveillance, human-computer interaction multimedia management

Face recognition
Advantages over other biometric technologies: Natural Nonintruisive Easy to use

Among the six biometric attributes considered by Hietmeyer, facial features scored the highest compatibility in a Machine Readable Travel Documents (MRTD) system based on: Enrollment Renewal

Classification
A face recognition system is expected to identify faces present in images and videos automatically. It can operate in either or both of two modes: Face verification (or authentication): involves a one-to-one match that compares a query face image against a template face image whose identity is being claimed. Face identification (or recognition): involves one-to-many matches that compares a query face image against all the template images in the database to determine the identity of the query face.

First automatic face recognition system was developed by Kanade 1973.

Face recognition processing

Face recognition is a visual pattern recognition problem. A face is a three-dimensional object subject to varying illumination, pose, expression is to be identified based on its two-dimensional image ( or three- dimensional images obtained by laser scan). A face recognition system generally consists of 4 modules - detection, alignment, feature extraction, and matching. Localization and normalization (face detection and alignment) are processing steps before face recognition (facial feature extraction and matching) is performed.

Face recognition processing

Face detection segments the face areas from the background. In the case of video, the detected faces may need to be tracked using a face tracking component. Face alignment is aimed at achieving more accurate localization and at normalizing faces, whereas face detection provides coarse estimates of the location and scale of each face.

Face recognition processing

Facial components and facial outline are located; based on the location points, The input face image is normalized in respect to geometrical properties, such as size and pose, using geometrical transforms or morphing, The face is further normalized with respect to photometrical properties such as illumination and gray scale.

Face recognition processing

After a face is normalized, feature extraction is performed to provide effective information that is useful for distinguishing between faces of different persons and stable with respect to the geometrical and photometrical variations. For face matching, the extracted feature vector of the input face is matched against those of enrolled faces in the database; it outputs the identity of the face when a match is found with sufficient confidence or indicates an unknown face otherwise.

Face recognition processing

Click to edit Master text styles Second level Third level Fourth level Fifth level

Face recognition processing flow.