COMPARISON OF PCAAND ICA IN FACE RECOGNITION
BING LUO1, YU-JIE HAO1, WEI-HUA ZHANG2, ZHI-SHEN LIU1
'University of Electronic Science and Technology of China, Chengdu 610054, China
2Yankuang Cathay Coal Chemicals Co., LTD
E-MAIL: backI127@163.com, haoyujieguestc.edu.cn
Abstract:
Over the last ten years, face recognition has become a
specialized applications area within the larger field of
computer vision. Principal component analysis (PCA) and
independent component analysis (ICA) become common
method for face recognition.
This paper compares Principal component analysis
(PCA) to independent component analysis (ICA) in face
recognition. In this paper, we used PCA derived from
"eigenfaces". ICA derived from a linear representation of
nongaussian data. In the paper, it shows the different
between PCA and ICA.
Keywords:
Face recognition, PCA (principle component analysis),
ICA (independent component analysis).
1. Introduction
A number of current face recognition algorithms use
face representations found by unsupervised statistical
methods. Typically these methods find a set of basis
images and represent faces as a linear combination of
those images. Principal component analysis (PCA) [1, 2, 3]
is a popular example of such methods. The basis images
found by PCA depend only on pairwise relationships
between pixels in the image database. In a task such as
face recognition, in which important information may be
contained in the high-order relationships among pixels, it
seems reasonable to expect that better basis images may be
found by methods sensitive to these high-order statistics.
Independent component analysis (ICA)[4,5,6], a
generalization of PCA, is one such method. We used a
version of ICA derived from the principle of optimal
information transfer through sigmoid neurons.
978-1-4244-3425-1/08/$25.00 ©2008 IEEE
2. Face recognition
In face recognition, the research dates back to
1960s.Most current face recognition techniques, however,
date back only to the appearance based recognition work
of the late 1980s and 1990s. Kirby and Sirovich were the
first to apply principal component analysis (PCA) to face
images, Turky and Pentland used PCA to compute a set of
subspace basis vectors (which they called "eigenfaces")[2]
for a database of face images. Some of the most successful
representations for face recognition, such as eigenfaces,
holons, and local feature analysis are based on PCA.The
most common method for generating spatially localized
features is to apply independent component analysis (ICA)
to produce basis vectors that are statistically independent.
A number of algorithms for performing ICA have been
proposed and has been proved successful for separating
randomly mixed auditory signals (the cocktail party
problem), and for separating electroencephalogram (EEG)
signals and functional magnetic resonance imaging (fMRI)
signals[4]. However, what is different between PCA and
ICA. Some people claim that ICA outperforms PCA for
face recognition, while some people claim that PCA
outperforms ICA and some people claims that there is no
statistical difference in performance between the two. The
relative performance of the two techniques is an open
question.
3. PCA&ICA
3.1 PCA
Principal component analysis, or PCA, is a technique
that is widely used for applications such as dimensionality
reduction, lossy data compression, feature extraction, and
241
data visualization. Sometimes we need the columns of matrix A;
A 2-D facial image can be represented as 1 -D vector denoting them by aj the model can also be written as
by concatenating each row (or column) into a long thin m
vector. Let's suppose we have M vectors of size N (= rows X = Z a1s (6)
of image x columns of image) representing a set of n=l
sampled images. pj's represent the pixel values. The statistical model is called independent component
analysis, or ICA model. The ICA model is a generative
xi =[pi...p,], i=I ...
(1) model, which means that it describes how the observed
data are generated by a process of mixing the components
The images are mean centered by subtracting the
mean image from each image vector. Let m represent the Si It is assumed that the source signals are independent of
mean image. each other and the mixing matrix A is invertible. Based on
these assumptions and the observed mixtures, ICA
x n=X (2) algorithms try to find the mixing matrix A or the separating
matrix W such that:
And let U, be defined as mean centered image. u = Wx = WAs (7)
ui=xi -x (3) is an estimation of the independent source signals.
We can get covariance matrix:
4. Experimental results
5 = ,UU Namely,
lm T
First, use three images employed for comparison.
S = EY (X, X)(Xn X) (4) The image 1 is cos(x); Image 2 is random; Image 3 is
m n=l
Our goal is to find the eigenvectors and eigenvalues
sin(4x). The image of Second row is the mixed. The image
of Third row is processed by ICA, and image of last row is
of the covariance matrix. The size of S is NxN which processed by PCA. We can find the image processed by
could be enormous. This eigenvector is known as the first ICA is better than those processed by PCA.
principal component.
To summarize, principal component analysis involves image 1 image 2 image 3
evaluating the mean x and the covariance matrix S of 5 -1 1rXI 7\7\7X l\1

the data set and then finding the M eigenvectors of S -1

0 500 1000 0 500 1000 0 500 1000
corresponding to the M largest eigenvalues.
Mixed 1 Mixed 2 Mixed 3
3.2 ICA 10 5
54w
0
i5 00f
Independent Component Analysis, or ICA, is a 0 500 1000 0 500 1000 0 500 1000

statistical technique which in many cases characterizes the

data in a natural way. ICA and the related blind source 2
ICA 1
5
ICA 2 ICA 3

separation (BSS) problem have grown important research 0

and application topics both in unsupervised neural learning 0 500 1000 0 500 1000 0 500 1000
and statistical signal processing.
Let us denote by x the random vector whose elements PCA 1 PCA 2 PCA 3
are the mixtures x1, ...,x1, and likewise by s the random 10 50
vector with elements s1, ..., s,. Let us denote by A the 1 |

matrix with elements aij. Generally, bold lower case letters 0 500 1000 5000 1000 0 500 1000
indicate vectors and bold upper-case letters denote Figure 1. ICA vs PCA
matrices. All vectors are understood as column vectors;
thus XT, or the transpose of x, is a row vector. Using this
vector-matrix notation, the above mixing model is written Second, the face images employed for this research
as were a subset of The Olivetti and Oracle Research
X=AS (5) Laboratory (ORL) face database. It is used in order to test

242
our method in the presence of headpose variations. There
are 10 different images of each of 40 distinct subjects. For
some subjects, the images were taken at different times,
varying lighting, facial expressions (open / closed eyes,
smiling / not smiling), facial details (glasses / no glasses)
and head pose (tilting and rotation up to 20 degrees).
The original face images were all sized 92x 112 with a
256-level gray scale. The experiments were performed
with five training images and five test images per person
for a total of 200 training images and 200 test images.
When we use PCA method, the recognition rates is 87.5%.
Figure 1 shows covariance matrix of training images and
figure 2 shows "eigenfaces" of 200 training images. In
ICA method, Figure 3 shows ICA representation.
Figure 2 covariance matrix of 200 training images
Figure 3. "eigenfaces" of 200 training
U1 U2
_bi...1- -h * .1
1+S A--;W 4 _.
4.
Figure 4. ICA representation
5. Conclusions
Comparisons between PCA and ICA are complex,
because differences in tasks, ICA algorithms, and distance
metrics must be taken into account. The paper shows
243
empirically that ICA has its advance. ICA chooses a
different subspace than PCA. PCA is only sensitive to the
power spectrum of images suggests that it might not be
particularly well suited for representing images. However,
ICA is sensitive to high-order statistics in the data, not just
the covariance matrix.
Acknowledgment
This work was supported by the National High
Technology Research and Development Program of China
(No:2007AA01Z423), the National Natural Science
Foundation of China (No: 60703113), China Academy of
Engineering Physical (No:2006Z0604), Sichuan Science
and Technology Project, Sichuan Committee of Economics
Project (No:2008CD00053) and Chongqing Natural
Science Foundation Project.
References
[1] Kwang In Kim, Keechul Jung, and Hang Joon Kim:
Face Recognition Using Kernel Principal Component
Analysis. IEEE SIGNAL PROCESSING LETTERS,
VOL. 9, NO. 2, FEBRUARY 2002.
[2] Kyungnam Kim: Face Recognition using Principle
Component Analysis.
[3] M. Jordan, J. Kleinberg and B. Scholkopf: Pattern
Recognition And Machine Learning.
[4] Marian Stewart Bartlett, Javier R. Movellan and
Terrence J. Sejnowski: Face Recognition by
Independent Component Analysis, IEEE
TRANSACTIONS ON NEURAL NETWORKS,
VOL. 13, NO. 6, NOVEMBER 2002.
[5] M.S. Bartlett, H.M. Lades, T.J. Sejnowski:
Independent component representations for face
recognition, Science and Technology, Conference on
Human Vision and Electronic Imaging III, San Jose,
CA, 1998.
[6] Aapo Hyvarine: survey on Independent Component
Analysis.
[7] K. Baek, B.A. Draper, J.R. Beveridge: PCA vs ICA:
A comparison on the FERET data set, presented at
Joint Conference on Information Sciences, Durham,
NC, 2002.