EC1124 - Neural Networks and Fuzzy Logic Assignment

Face Feature Extraction

Adhithya
1041310445

The following project is performed to detect faces in a picture of the users choice and to extract the details (
like outline of the face) of the face using MATLAB. It is achieved using two algorithms, The Viola Jones
Object Detection Framework and Eigenface algorithm. The image processing toolbox for Matlab is
necessary for the project to work. Below is a brief introduction to the algorithms used.

Viola Jones Algorithm :

The ViolaJones object detection framework is the first object detection framework to provide
competitive object detection rates in real-time proposed in 2001 by Paul Viola and Michael Jones. Although
it can be trained to detect a variety of object classes, it was motivated primarily by the problem of face
detection.The problem to be solved is detection of faces in an image. A human can do this easily, but a
computer needs precise instructions and constraints. To make the task more manageable, ViolaJones
requires full view frontal upright faces. Thus in order to be detected, the entire face must point towards the
camera and should not be tilted to either side. While it seems these constraints could diminish the
algorithm's utility somewhat, because the detection step is most often followed by a recognition step, in
practice these limits on pose are quite acceptable.
The characteristics of ViolaJones algorithm which make it a good detection algorithm are:

Robust very high detection rate (true-positive rate) & very low false-positive rate always.

Real time For practical applications at least 2 frames per second must be processed.

Face detection only (not recognition) - The goal is to distinguish faces from non-faces (detection is the
first step in the recognition process).

The algorithm has four stages:

1

Haar Feature Selection

Creating an Integral Image

Adaboost Training

Cascading Classifiers

The features sought by the detection framework universally involve the sums of image pixels within
rectangular areas. As such, they bear some resemblance to Haar basis functions, which have been used
previously in the realm of image-based object detection.[3] However, since the features used by Viola and
Jones all rely on more than one rectangular area, they are generally more complex. The figure on the right
illustrates the four different types of features used in the framework. The value of any given feature is the
sum of the pixels within clear rectangles subtracted from the sum of the pixels within shaded rectangles.
Rectangular features of this sort are primitive when compared to alternatives such as steerable filters.
Although they are sensitive to vertical and horizontal features, their feedback is considerably coarser.

1. Haar Features All human faces share some similar properties. These regularities may be matched
using Haar Features.
A few properties common to human faces:

The eye region is darker than the upper-cheeks.

The nose bridge region is brighter than the eyes.

Composition of properties forming matchable facial features:

Location and size: eyes, mouth, bridge of nose

Value: oriented gradients of pixel intensities

The four features matched by this algorithm are then sought in the image of a face (shown at left).
Rectangle features:

Value = (pixels in black area) - (pixels in white area)

Three types: two-, three-, four-rectangles, Viola & Jones used two-rectangle features

For example: the difference in brightness between the white &black rectangles over a specific area

Each feature is related to a special location in the sub-window

2. An image representation called the integral image evaluates rectangular features in constant time, which
gives them a considerable speed advantage over more sophisticated alternative features. Because each
feature's rectangular area is always adjacent to at least one other rectangle, it follows that any two-rectangle
feature can be computed in six array references, any three-rectangle feature in eight, and any four-rectangle
feature in nine.
The integral image at location (x,y), is the sum of the pixels above and to the left of (x,y), inclusive.

Advantages of ViolaJones algorithm

Extremely fast feature computation

Efficient feature selection

Scale and location invariant detector

Instead of scaling the image itself (e.g. pyramid-filters), we scale the features.

Such a generic detection scheme can be trained for detection of other types of objects (e.g. cars, hands)

Disadvantages of ViolaJones algorithm

Detector is most effective only on frontal images of faces

It can hardly cope with 45 face rotation both around the vertical and horizontal axis.

Sensitive to lighting conditions

We might get multiple detections of the same face, due to overlapping sub-windows.

Example Matlab Code

Eigenface Algorithm :
Eigenfaces is the name given to a set of eigenvectors when they are used in the computer vision problem of
human face recognition. The approach of using eigenfaces for recognition was developed by Sirovich and
Kirby (1987) and used by Matthew Turk and Alex Pentland in face classification. The eigenvectors are
derived from the covariance matrix of the probability distribution over the high-dimensional vector space of
face images. The eigenfaces themselves form a basis set of all images used to construct the covariance
matrix. This produces dimension reduction by allowing the smaller set of basis images to represent the
original training images. Classification can be achieved by comparing how faces are represented by the basis
set.

Examples of Processed Eigenfaces

The Eigenface approach began with a search for a low-dimensional representation of face images. Sirovich
and Kirby (1987) showed that Principal Component Analysis could be used on a collection of face images to
form a set of basis features. These basis images, known as Eigenpictures, could be linearly combined to

reconstruct images in the original training set. If the training set consists of M images, principal component
analysis could form a basis set of N images, where N < M. The reconstruction error is reduced by increasing
the number of eigenpictures, however the number needed is always chosen less than M. For example, if you
need to generate a number of N eigenfaces for a training set of M face images, you can say that each face
image can be made up of "proportions" of all this K "features" or eigenfaces : Face image1 = (23% of E1) +
(2% of E2) + (51% of E3)+...+(1% En).
In 1991 M. Turk and A. Pentland expanded these results and presented the Eigenface method of face
recognition.[3] In addition to designing a system for automated face recognition using eigenfaces, they
showed a way of calculating the eigenvectors of a covariance matrix in such a way as to make it possible for
computers at that time to perform eigen-decomposition on a large number of face images. Face images
usually occupy a high-dimensional space and conventional principal component analysis was intractable on
such data sets. Turk and Pentland's paper demonstrated ways to extract the eigenvectors based on matrices
sized by the number of images rather than the number of pixels.
Once established, the eigenface method was expanded to include methods of preprocessing to improve
accuracy.[4] Multiple manifold approaches were also used to build sets of eigenfaces for different
subjects[5][6] and different features, such as the eyes.
A set of eigenfaces can be generated by performing a mathematical process called principal component
analysis (PCA) on a large set of images depicting different human faces. Informally, eigenfaces can be
considered a set of "standardized face ingredients", derived from statistical analysis of many pictures of
faces. Any human face can be considered to be a combination of these standard faces. For example, one's
face might be composed of the average face plus 10% from eigenface 1, 55% from eigenface 2, and even
-3% from eigenface 3. Remarkably, it does not take many eigenfaces combined together to achieve a fair
approximation of most faces. Also, because a person's face is not recorded by adigital photograph, but
instead as just a list of values (one value for each eigenface in the database used), much less space is taken
for each person's face.
The eigenfaces that are created will appear as light and dark areas that are arranged in a specific pattern.
This pattern is how different features of a face are singled out to be evaluated and scored. There will be a
pattern to evaluate symmetry, if there is any style of facial hair, where the hairline is, or evaluate the size of
the nose or mouth. Other eigenfaces have patterns that are less simple to identify, and the image of the
eigenface may look very little like a face.
The technique used in creating eigenfaces and using them for recognition is also used outside of facial
recognition. This technique is also used for handwriting analysis, lip reading,voice recognition, sign
language/hand gestures interpretation and medical imaging analysis. Therefore, some do not use the term
eigenface, but prefer to use 'eigenimage'.

A set of eigenfaces can be generated by performing a mathematical process called principal component
analysis (PCA) on a large set of images depicting different human faces. Informally, eigenfaces can be
considered a set of "standardized face ingredients", derived from statistical analysis of many pictures of
faces. Any human face can be considered to be a combination of these standard faces. For example, one's
face might be composed of the average face plus 10% from eigenface 1, 55% from eigenface 2, and even

-3% from eigenface 3. Remarkably, it does not take many eigenfaces combined together to achieve a fair
approximation of most faces. Also, because a person's face is not recorded by adigital photograph, but
instead as just a list of values (one value for each eigenface in the database used), much less space is taken
for each person's face.
The eigenfaces that are created will appear as light and dark areas that are arranged in a specific pattern.
This pattern is how different features of a face are singled out to be evaluated and scored. There will be a
pattern to evaluate symmetry, if there is any style of facial hair, where the hairline is, or evaluate the size of
the nose or mouth. Other eigenfaces have patterns that are less simple to identify, and the image of the
eigenface may look very little like a face.
The technique used in creating eigenfaces and using them for recognition is also used outside of facial
recognition. This technique is also used for handwriting analysis, lip reading,voice recognition, sign
language/hand gestures interpretation and medical imaging analysis. Therefore, some do not use the term
eigenface, but prefer to use 'eigenimage'.
These eigenfaces can now be used to represent both existing and new faces: we can project a new (meansubtracted) image on the eigenfaces and thereby record how that new face differs from the mean face. The
eigenvalues associated with each eigenface represent how much the images in the training set vary from the
mean image in that direction. We lose information by projecting the image on a subset of the eigenvectors,
but we minimize this loss by keeping those eigenfaces with the largest eigenvalues. For instance, if we are
working with a 100 x 100 image, then we will obtain 10,000 eigenvectors. In practical applications, most
faces can typically be identified using a projection on between 100 and 150 eigenfaces, so that most of the
10,000 eigenvectors can be discarded.

Example Matlab Code

Image Processing Toolbox :

Image Processing Toolbox supports images and video generated by a wide range of devices, including
webcams, digital cameras, satellite and airborne sensors, medical imaging devices, microscopes, telescopes,
and other scientific instruments. You can use functions and apps to visualize, analyze, and process these
imagesin many data types.
With Image Acquisition Toolbox you can acquire live images and video from frame grabbers, GigE Vision
cameras, DCAM cameras, and other devices.
Key Features

Image analysis, including segmentation, morphology, statistics, and measurement

Image enhancement, filtering, and deblurring

Geometric transformations and intensity-based image registration methods

Image transforms, including FFT, DCT, Radon, and fan-beam projection

Large image workflows, including block processing, tiling, and multiresolution display

Visualization apps, including Image Viewer and Video Viewer

Multicore- and GPU-enabled functions, and C-code generation support

Final Matlab Codes :

FACE DETECTION:
clearall

clc

%Detect objects using Viola-Jones Algorithm

%To detect Face

FDetect = vision.CascadeObjectDetector;

%Read the input image

I = imread('HarryPotter.jpg');

%Returns Bounding Box values based on number of objects

BB = step(FDetect,I);

figure,

imshow(I); holdon

for i = 1:size(BB,1)

rectangle('Position',BB(i,:),'LineWidth',5,'LineStyle','-','EdgeColor','r');

end

title('Face Detection');

holdoff;
BB =52 38 73 73379 84 71 71198 57 72 72

NOSE DETECTION:
%To detect Nose

NoseDetect = vision.CascadeObjectDetector('Nose','MergeThreshold',16);

BB=step(NoseDetect,I);

figure,

imshow(I); holdon

for i = 1:size(BB,1)

rectangle('Position',BB(i,:),'LineWidth',4,'LineStyle','-','EdgeColor','b');

end
Haar Wavelet

In mathematics, the Haar wavelet is a sequence of rescaled "square-shaped" functions which together form
a wavelet family or basis. Wavelet analysis is similar to Fourier analysis in that it allows a target function
over an interval to be represented in terms of an orthonormal basis. The Haar sequence is now recognised as
the first known wavelet basis and extensively used as a teaching example.
The Haar sequence was proposed in 1909 by Alfrd Haar. Haar used these functions to give an example of
an orthonormal system for the space of square-integrable functions on the unit interval [0, 1]. The study of
wavelets, and even the term "wavelet", did not come until much later. As a special case of the Daubechies
wavelet, the Haar wavelet is also known as Db1.
The Haar wavelet is also the simplest possible wavelet. The technical disadvantage of the Haar wavelet is
that it is not continuous, and therefore not differentiable. This property can, however, be an advantage for the
analysis of signals with sudden transitions, such as monitoring of tool failure in machines.

function HaarCascade=GetHaarCasade(FilenameHaarcasade)
% This function reads a Matlab file with a struct containing
% the OpenCV classifier data of an openCV XML file.
% It also changes the structure a little bit, and add missing
% fields
%
% HaarCascade=GetHaarCasade(FilenameHaarcasade);
%
%
% Function is written by D.Kroon University of Twente (November 2010)

load(FilenameHaarcasade);
f=fields(opencv_storage);
HaarCascade=opencv_storage.(f{1});

% Add all missing data-fields

stages=HaarCascade.stages;
for i=1:length(stages)

stages2(i).stage_threshold=stages(i).stage_threshold;
stages2(i).parent=stages(i).parent;
stages2(i).next=stages(i).next;
for j=1:length(stages(i).trees);
for k=1:length(stages(i).trees(j).value)
a=stages(i).trees(j).value(k);
if(~isfield(a,'left_node')) , a.left_node=[]; end
if(~isfield(a,'right_node')), a.right_node=[];end
if(~isfield(a,'left_val')) , a.left_val=[]; end
if(~isfield(a,'right_val')) , a.right_val=[]; end
if(isempty(a.left_val)), a.left_val=-1; end
if(isempty(a.right_val)), a.right_val=-1; end
if(isempty(a.left_node)), a.left_node =-1; end
if(isempty(a.right_node)),a.right_node=-1; end
a.rects1=a.feature.rects(1).value;
a.rects2=a.feature.rects(2).value;
if(length(a.feature.rects)>2),
a.rects3=a.feature.rects(3).value;
else
a.rects3=[0 0 0 0 0];
end
a.tilted=a.feature.tilted;

if(any(a.tilted))
warning('HaarCascade:file','Tilted features are not supported');
end

Eigenface Matlab Code

% Face recognition
clear all
close all
clc
% number of images on your training set.
M=50;
% Chosen std and mean.
% It can be any number that it is close to the std and mean of most of the images.
um=100;
ustd=80;
% read and show image
S=[]; % img matrix
figure(1);
for i=1:M
str=strcat(int2str(i),'.bmp'); % concatenates two strings that form the name of the image
eval('img=imread(str);');
subplot(ceil(sqrt(M)),ceil(sqrt(M)),i)
imshow(img)
if i==3
title('Training set','fontsize',18)
end
drawnow;
[irow icol]=size(img); % get the number of rows (N1) and columns (N2)
temp=reshape(img',irow*icol,1); % creates a (N1*N2)x1 vector

S=[S temp];
end

% S is a N1*N2xM matrix after finishing the sequence

% Here we change the mean and std of all images. We normalize all images.
% This is done to reduce the error due to lighting conditions and background.
for i=1:size(S,2)
temp=double(S(:,i));
m=mean(temp);
st=std(temp);
S(:,i)=(temp-m)*ustd/st+um;
end
% show normalized images
figure(2);
for i=1:M
str=strcat(int2str(i),'.jpg');
img=reshape(S(:,i),icol,irow);
img=img';
eval('imwrite(img,str)');
subplot(ceil(sqrt(M)),ceil(sqrt(M)),i)
imshow(img)
drawnow;
if i==3
title('Normalized Training Set','fontsize',18)
end
end
% mean image
m=mean(S,2); % obtains the mean of each row instead of each column
tmimg=uint8(m); % converts to unsigned 8-bit integer. Values range from 0 to 255
img=reshape(tmimg,icol,irow); % takes the N1*N2x1 vector and creates a N1xN2 matrix
img=img';
figure(3);
imshow(img);
title('Mean Image','fontsize',18)
% Change image for manipulation
dbx=[]; % A matrix
for i=1:M
temp=double(S(:,i));
dbx=[dbx temp];
end
%Covariance matrix C=A'A, L=AA'
A=dbx';
L=A*A';
% vv are the eigenvector for L
% dd are the eigenvalue for both L=dbx'*dbx and C=dbx*dbx';
[vv dd]=eig(L);
% Sort and eliminate those whose eigenvalue is zero
v=[];
d=[];
for i=1:size(vv,2)

if(dd(i,i)>1e-4)
v=[v vv(:,i)];
d=[d dd(i,i)];
end
end
%sort, will return an ascending sequence
[B index]=sort(d);
ind=zeros(size(index));
dtemp=zeros(size(index));
vtemp=zeros(size(v));
len=length(index);
for i=1:len
dtemp(i)=B(len+1-i);
ind(i)=len+1-index(i);
vtemp(:,ind(i))=v(:,i);
end
d=dtemp;
v=vtemp;
%Normalization of eigenvectors
for i=1:size(v,2) %access each column
kk=v(:,i);
temp=sqrt(sum(kk.^2));
v(:,i)=v(:,i)./temp;
end
%Eigenvectors of C matrix
u=[];
for i=1:size(v,2)
temp=sqrt(d(i));
u=[u (dbx*v(:,i))./temp];
end
%Normalization of eigenvectors
for i=1:size(u,2)
kk=u(:,i);
temp=sqrt(sum(kk.^2));
u(:,i)=u(:,i)./temp;
end
% show eigenfaces
figure(4);
for i=1:size(u,2)
img=reshape(u(:,i),icol,irow);
img=img';
img=histeq(img,255);
subplot(ceil(sqrt(M)),ceil(sqrt(M)),i)
imshow(img)
drawnow;
if i==3
title('Eigenfaces','fontsize',18)
end

end
% Find the weight of each face in the training set
omega = [];
for h=1:size(dbx,2)
WW=[];
for i=1:size(u,2)
t = u(:,i)';
WeightOfImage = dot(t,dbx(:,h)');
WW = [WW; WeightOfImage];
end
omega = [omega WW];
end
% Acquire new image
% Note: the input image must have a bmp or jpg extension.
% It should have the same size as the ones in your training set.
% It should be placed on your desktop
InputImage = input('Please enter the name of the image and its extension \n','s');
InputImage = imread(strcat('D:\Documents and Settings\sis26\Desktop\',InputImage));
figure(5)
subplot(1,2,1)
imshow(InputImage); colormap('gray');title('Input image','fontsize',18)
InImage=reshape(double(InputImage)',irow*icol,1);
temp=InImage;
me=mean(temp);
st=std(temp);
temp=(temp-me)*ustd/st+um;
NormImage = temp;
Difference = temp-m;
p = [];
aa=size(u,2);
for i = 1:aa
pare = dot(NormImage,u(:,i));
p = [p; pare];
end
ReshapedImage = m + u(:,1:aa)*p; %m is the mean image, u is the eigenvector
ReshapedImage = reshape(ReshapedImage,icol,irow);
ReshapedImage = ReshapedImage';
%show the reconstructed image.
subplot(1,2,2)
imagesc(ReshapedImage); colormap('gray');
title('Reconstructed image','fontsize',18)
InImWeight = [];
for i=1:size(u,2)
t = u(:,i)';
WeightOfInputImage = dot(t,Difference');
InImWeight = [InImWeight; WeightOfInputImage];
end
ll = 1:M;

figure(68)
subplot(1,2,1)
stem(ll,InImWeight)
title('Weight of Input Face','fontsize',14)
% Find Euclidean distance
e=[];
for i=1:size(omega,2)
q = omega(:,i);
DiffWeight = InImWeight-q;
mag = norm(DiffWeight);
e = [e mag];
end
kk = 1:size(e,2);
subplot(1,2,2)
stem(kk,e)
title('Eucledian distance of input image','fontsize',14)
MaximumValue=max(e) % maximum eucledian distance
MinimumValue=min(e) % minimum eucledian distance

OUTPUT

Contour
Map
Eigenface

Face
Detail
Extractio
n