230 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO.
2, FEBRUARY 2014
A Framework for Making Face Detection
Benchmark Databases
Gee-Sern Hsu, Member, IEEE, and Tsu-Ying Chu
Abstract—The images in face detection benchmark databases
are mostly taken by consumer cameras, and thus are constrained
by popular preferences, including a frontal pose and balanced
lighting conditions. A good face detector should consider be-
yond such constraints and work well for other types of im-
ages, for example, those captured by a surveillance camera.
To overcome such constraints, a framework is proposed to
transform a mother database, originally made for benchmark-
ing face recognition, to daughter datasets that are good for
benchmarking face detection. The daughter datasets can be
customized to meet the requirements of various performance
criteria; therefore, a face detector can be better evaluated on
desired datasets. The framework is composed of two phases:
1) intrinsic parametrization and 2) extrinsic parametrization.
The former parametrizes the intrinsic variables that affect the
appearance of a face, and the latter parametrizes the extrin-
sic variables that determine how faces appear on an image.
Experiments reveal that the proposed framework can generate
not just data that are similar to those available from popular
benchmark databases, but also those that are hardly available
from existing databases. The datasets generated by the proposed
framework offer the following advantages: 1) they can define
the performance specification of a face detector in terms of
the detection rates on variables with different variation scopes;
2) they can benchmark the performance on one single or multiple
variables, which can be difficult to collect; and 3) their ground
truth is available when the datasets are generated, avoiding the
time-consuming manual annotation.
Index Terms—Face database, face detection, performance eval-
uation.
I. Introduction
LMOST all face detection algorithms are evaluated using
A datasets with images collected manually from various
sources. Each collected image has one or a few faces in it
with ground truth manually annotated. The performance is
measured by the difference between the ground truth and those
determined by the algorithm [1]. It is commonly acknowledged
that an ideal face detector should be able to detect faces
of any sizes, orientations, poses, expressions, under various
illumination conditions, and with locations anywhere in the
Manuscript received July 17, 2012; revised March 6, 2013; accepted April
15, 2013. Date of publication May 31, 2013; date of current version February
4, 2014. This paper was recommended by Associate Editor C. Shan.
G.-S. Hsu is with the Artificial Vision Laboratory, National Tai-
wan University of Science and Technology, Taipei 106, Taiwan (e-mail:
jison@mail.ntust.edu.tw).
T.-Y. Chu is with Vols Taipei (e-mail: vacation5588@hotmail.com).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TCSVT.2013.2265571
1051-8215 c 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
image. Some even demand a certain level of robustness against
occlusion. It will be extremely exhausting, if not impossible,
to collect image samples that are good enough to encompass
all or most of the aforementioned variables with a sufficiently
large scope of variation in each variable.
After the works done by Sung and Poggio [2], Rowley et al.
[3], and Schneiderman and Kanade [4], their collections have
been considered as benchmark datasets. The dataset collected
by Sung and Rowley is commonly used for the assessment of
detection of nearly frontal faces, and Schneiderman’s dataset
is used for the detection of nearly profile faces. Both datasets
are often known as the MIT+CMU dataset as a whole. An
extensive review on the works published before 2002 by
Yang et al. [5] reveals that Rowley’s dataset was the most
popular one for performance evaluation by that time. We
have also completed a review that covers 42 face detection
methods, published between 2002 and 2011 (details reported
in the next section), of which 29 (69.1%) use the MIT+CMU
dataset. Although many variables, including pose, orientation,
illumination and size, are sampled in the MIT+CMU dataset,
only a limited scope of variation in each variable is revealed
by the samples. Take illumination as an example. Rowley’s
dataset has 130 images with 507 faces in total. 70% of the
faces have nearly uniform illumination, 9.8% are lit on the
right side, 14.2% on the left, 3.8% on the top, and 2.2% on the
bottom. Because the performance of a face detector strongly
depends on the training set, if the training set contains only
samples with uniform illumination and samples lit on the right
or left side, the resultant face detector will perform poorly on
detecting faces lit on the top or bottom. However, such a face
detector can attain a detection rate 94% on Rowley’s dataset,
which only has 6% of its samples lit on the top or bottom.
The above observations raise the following issues with many
existing benchmark databases.
1) The images captured by consumer cameras are mostly
with general preferences, such as frontal pose, smiling,
and uniform illumination; the variables contained in such
a collection are mostly bandlimited, where the band
refers to the variation scope in each variable. These
databases may be good enough for evaluating the face
detector needed for consumer cameras. However, it can
hardly be justified that they are appropriate for other
applications. The face detector required by an outdoor
surveillance camera spotting a street corner would be
better evaluated on samples with limited poses and sizes
HSU AND CHU: A FRAMEWORK FOR FACE DETECTION BENCHMARK DATABASES
but with a large variation in illumination from dawn to
dusk.
2) Many face detectors report detection rates on, for
example, Rowley’s dataset, including 96.7% in [6],
90% in [7], and 92.3% in [8]. However, only a few
report the characteristics of the samples that make
them fail to be detected. Among the reported cases in
[8]– [10], the causes include unbalanced illumination,
partial occlusion, low contrast, and low image qualities.
Because the samples with these characteristics of vari-
ables are far less than those with generally preferred
characteristics, e.g., balanced lighting and frontal pose,
the reported detection rates can be considered valid only
for the applications with similar characteristics of vari-
ations. If unbalanced illumination, for example, is con-
sidered to be the most common in certain applications,
the performance of the aforementioned face detectors
is better reevaluated using a dataset with unbalanced
illumination samples.
The aforementioned issues motivated the creation of the
proposed framework that can generate application-oriented
and variable-specifiable datasets. The framework takes in
faces from a mother database, which is originally made
for benchmarking face recognition, and generates daughter
datasets with attributes that meet the requirements of a desired
evaluation criterion. The mother database offers facial samples
characterized by intrinsic variables, and each intrinsic variable
covers a large scope of variation. The intrinsic variables refer
to those that are able to change the appearance of a face, such
as illumination and pose. Each daughter dataset is generated
or synthesized with required settings on the intrinsic and
extrinsic variables. The extrinsic variables refer to those that
determine how faces appear on an image, such as the number
and size range of the faces. Extended and improved from our
preliminary work in [11], which lacks a justification on the
validity of the synthesized daughter datasets and does not offer
a guideline on how to use the daughter datasets, this paper
verifies the validity of the daughter datasets by an experimental
comparison with popular benchmark databases, and presents a
case study on how to design daughter datasets to characterize
the performance of a face detector.
The contributions of this paper include the following.
1) Different from existing benchmark datasets with vari-
ables of fixed scopes of variation, the synthesized
daughter datasets can be made with a desired scope
of variables that meets the requirements of various test
criteria.
2) The performance specification of a face detector can be
defined via the daughter datasets with special settings on
the intrinsic and extrinsic variables. The specification is
given in terms of the detection rate on each variable with
a wide range of variation. To the best of our knowledge,
this is the first time that a face detector can be associated
with its performance specification.
Some other advantages deserve to be noted. Unlike manually
collected datasets that require time-consuming manual anno-
tation on the ground truth, the ground truth of the daughter
231
datasets is given when they are generated. Furthermore, the
proposed framework can be applied to other object recognition
databases so that they can be transformed to datasets that are
good for evaluating object detection.
The rest of this paper is organized as follows. Section II
reviews the datasets used by many for performance evaluation
and highlights their limitations. The proposed framework is
presented in Section III, with two phases in it: 1) the intrinsic
parametrization (IP) and 2) extrinsic parametrization (EP). To
demonstrate that the proposed framework is able to generate
daughter datasets for different criteria, Section IV presents
experimental verifications following two protocols. In Protocol
1, the framework synthesizes two daughter datasets with
attributes that are similar to those in Rowley’s dataset [3] and
a new benchmark FDDB dataset [12]. A few face detectors are
tested on both types of datasets to compare the performance,
justifying the validity of the daughter datasets. In Protocol
2, the performance specification of a face detector is defined
via the daughter datasets with specially designed attributes,
offering a case study on the application of the proposed
framework. This paper is concluded in Section V.
II. Issues With Existing Benchmarks
The extensive survey in [5] shows that Rowley’s dataset
was the most popular one for benchmark by 2002. Our review,
which covers 42 face detection methods [6]–[10] and [13]–[49]
published between 2002 and 2011, reveals that many use two
or more datasets for performance evaluation. The following
gives a summary to this review.
1) 69.1% of the reviewed methods select the MIT+CMU
as one of the test sets.
2) 26.2% select other face detection databases. such as
BioID [9], [10], UCD [45], and Champion [14].
3) 40.5% select face recognition databases, including
FERET [9], [17], [21], [41], Yale [19], [44], BANCA
[50], CBCL [19], [51], and PIE [25], [44], [52]. Note
that these databases were selected for offering data with
large variations on illumination and pose.
4) 9.5% also use personal collections of the authors.
The aforementioned databases can be categorized into the
following two types.
1) Type 1: The datasets are composed of images collected
from various sources, e.g., the internet and personal
albums. The variables often include illumination, size,
pose, expression, occlusion, age, race, and background.
Well-known ones include the MIT+CMU [2]–[4] and
the BioID dataset [53]. The former has 130 gray-scaled
images with 507 faces, and the latter has 1521 gray-
scaled images with one face per image. One of the
latest collections, the FDDB dataset [12], has 2845 color
images with 5171 faces.
2) Type 2: These are meant for benchmarking face recog-
nition instead of face detection. They offer faces with
many variables, and each variable covers a wide scope
of variation. For example, the PIE database offers 68
individuals with 13 poses, 43 illumination conditions,
232 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 2, FEBRUARY 2014

TABLE I
Benchmark Databases Segmented Into Different Scopes of Variation in Pose and Illumination

Rowley’s dataset with 507 faces Schneiderman’s dataset with 411 faces FDDB dataset with 5171 faces
(a MIT+CMU subset) (a MIT+CMU subset)
Pose No. (%) Pose No. (%) Pose No. (%)
Frontal 438 (86.4) <30° to either side 38 (9.3) Frontal 2028 (39.22)
<45° leftsided 21 (4.1) 30°–60° leftsided 25 (6.0) <30° to either side 964 (18.64)
<45° rightsided 19 (3.7) 30°–60° rightsided 47 (11.5) 30°–60° leftsided 279 (5.40)
Chin-up 14 (2.8) 60°–100° leftsided 50 (12.0) 30°–60° rightsided 254 (4.91)
Chin-down 11 (2.2) 60°–100° rightsided 219 (53.2) 60°–100° leftsided 481 (9.30)
Other 4 (0.8) Chin-up to either side 16 (4.0) 60°–100° rightsided 475 (9.19)
Chin-down to either side 15 (3.7) Chin-up to either side 329 (6.36)
Other 1 (0.3) Chin-down to either side 152 (2.94)
Other 209 (4.04)
Illumination No. (%) Illumination No. (%) Illumination No. (%)
Uniform 355 (70.0) Uniform 261 (63.5) Uniform 3075 (59.5)
Right lit 50 (9.8) Right lit 63 (15.3) Right lit 455 (8.8)
Left lit 72 (14.2) Left lit 40 (9.7) Left lit 783 (15.2)
Top lit 19 (3.8) Top lit 32 (7.8) Top lit 378 (7.3)
Bottom lit 11 (2.2) Bottom lit 4 (1.0) Bottom lit 68 (1.3)
Other 11 (2.7) Partially shadowed 152 (2.9)
Other 260 (5.0)

and 3 expressions. Just the poses and illumination con-
ditions alone give 559 different combinations for one
individual’s face, more than the overall 507 faces offered
by the Rowley’s dataset.
The images in a Type 1 database are mostly taken by consumer
cameras; the variables are affected by general preferences,
for example, nearly frontal pose and balanced illumination.
To diversify the contents, many Type 1 datasets also include
challenging samples characterized by variables with special
variations, for example, strong shadow, large rotation, partial
occlusion, and cluttered backgrounds. However, the percentage
of the samples with such special variations is often far less than
that with general preferences. We segmented the MIT+CMU
and FDDB databases into subsets with similar variations in
illumination and pose in each subset. The segmentation result
is shown in Table I, where the MIT+CMU database is split into
Rowley’s and Schneiderman’s subsets [3], [4]. Although it is
claimed to be appropriate for evaluating frontal face detection,
Rowley’s dataset has 69 samples, i.e., 13.6% of the whole set,
with nonfrontal poses. Considering illumination conditions,
355 faces (70%) are with nearly uniform illumination versus
52 (30%) with nonuniform illumination. Furthermore, it is
not shown in the table that 474 (93.5%) are with neutral or
smiling versus 33 (6.5%) with other expressions. Schneider-
man’s dataset has 441 faces in 208 images and is generally
considered a benchmark for the detection of profile faces. The
partition in Table I shows that 65.2% of its samples are nearly
profile with a panned angle between 60° and 100°, but 9.3%
are closer to frontal with a panned angle < 30°. Besides, as
many as 264 samples (64.2%) are with uniform illumination,
but merely 15.3% lit on the right, 9.7% lit on the left, and 3.0%
with other conditions. The recently released FDDB database
[12] offers a large collection with a wide range of difficulties,
including occlusion and large poses. However, the challenging
samples are significantly outnumbered by the regular ones.
47.8% are with nearly frontal pose, i.e., < 30°, and 73.6% are
with nearly balanced illumination.
The UCD database [54] is a Type 1, and it is full of chal-
lenging samples, without a good portion of regular samples.
Very few use it for performance evaluation [45]. Its difficulty
seems to deter many from attempting, not to mention report-
ing, performance. A better solution seems to split challenging
samples from regular ones, and report performance separately.
If we can define challenges by variables of different variation
scopes, we can better evaluate an algorithm with data of
different difficulty levels.
Type 2 are primarily used for the evaluation of face recogni-
tion. The faces in these databases are similar in size, appear at
one or a few particular locations in an image, and with one face
per image. These conditions make them inappropriate for the
evaluation of face detection. However, a few select a Type 2 as
one of their test sets, because it offers samples with variables
of specific variation scopes, for example, 30° viewpoint with
illumination from the top.
The proposed framework takes the advantage of the differ-
ences between both types. It selects a Type 2 database as a
mother database for offering a wide range of well-organized
intrinsic variables, and generating daughter datasets, which are
similar to Type 1 but with variables of controllable variation
scopes.
III. From Mother Database to Daughter Datasets
The variables that affect the performance of a face detection
algorithm can be split into two categories: 1) intrinsic and 2)
extrinsic. The intrinsic variables are those that can alter the
appearance of a face, for example, pose, illumination, gender,
age, expression, and accessaries such as glasses, caps, and
masks. The extrinsic variables are those that determine how
faces appear in an image, for example, the size, number, and
spatial distribution of the faces across a background. The pro-
HSU AND CHU: A FRAMEWORK FOR FACE DETECTION BENCHMARK DATABASES
posed framework is designed with two phases in it: 1) Intrinsic
Parameterization (IP) and 2) Extrinsic Parameterization (EP).
The former parametrizes the intrinsic variables and the latter
parametrizes the extrinsic variables.
In the IP phase, we collect faces from a mother database and
parametrize the faces with intrinsic parameters, constituting
a parametrized face dataset (PFD). When a performance
evaluation criterion is given, the EP phase first transforms
the criterion to required intrinsic and extrinsic parameters.
The intrinsic parameters are used to select or synthesize
the required faces from the PFD. The extrinsic parameters
determine the number of faces, their size range, and how they
are distributed over a set of cluttered backgrounds.
By the end of this section a thorough discussion on the
appropriateness and advantages of the proposed framework
is given in Section III-C. It highlights the issues of the
performance gap between human vision and an artificial face
detector, and how to reduce this gap using the framework.
A. IP
Ideal candidates for mother databases must include many
variables and each variable must cover a wide range of vari-
ation. Poses and illumination conditions are often considered
to be two major challenging variables for face detection. If
the numbers of individuals and expressions are also taken
into account, the CMU PIE [52], multi-PIE [55], and CAS-
PEAL [56] can be good candidates. The PIE database is
taken as a sample mother database for presenting the proposed
framework and our experimental study. The procedure can be
repeated for other mother databases.
The IP phase is composed of the following four modules:
1) Face segmentation and pose parametrization.
2) Illumination clustering with DCT decomposition.
3) Scope expansion using illumination cone [57].
4) Parametrization of other intrinsic parameters.
1) Face Segmentation and Pose Parametrization: The faces
in the mother database are first cropped along the facial
contour of each face using an automatic segmentation method
that combines a state-of-the-art cross-pose face detector [58]
and the iterated graph cuts (IGC) [59]. The former is for
locating the face in an image1 and the latter is for cropping
the face. The cross-pose face detector [58] applies a mixture
of trees with a shared pool of features extracted at fiducial
points to capture the cross-pose topological changes of a
face. The tree-structured models are shown to be effective in
detecting faces of various poses using a considerably small
training set, compared to many previous approaches. Because
of the satisfactory performance in locating eyes, mouth, and
facial contours in our tests using the author-supplied source
codes, the face detector is exploited to segment the foreground,
i.e., the face, from the background. The identified foreground
and background regions can automatically initialize the IGC,
which requires manual annotation at initialization.
The IGC [59] is an iterative version of the graph cuts
[60], which formulates the segmentation problem as an energy
minimization via trimaps and probabilistic color models and
1 Each image in the mother database has one face only.
233
solves it by optimization. At initialization, one only has
to specify the background in the trimap using a bounding
box. The region outside of the bounding box is taken as
background, the region inside is taken as the foreground,
and the boundary region is taken as an intermediate region.
Assuming Gaussian mixture models (GMMs) to be in the
foreground and background, one can classify each pixel in the
boundary region by maximizing the likelihood of the GMMs
and update the parameters in the GMMs using the classified
pixels. The segmentation is then obtained by minimizing the
energy computed on the likelihood and a coherence term. This
process can go on until the segmentation converges, giving
a more precise and smooth result than that of graph cuts.
We replace the manual initialization by the cross-pose face
detector [58], and make the face segmentation automatic. The
segmentation accuracy can be further improved considering
that each subject in the mother database is actually aligned
across different illumination conditions when the pose is fixed.
Therefore, one can run the aforementioned algorithm across
multiple images and take the majority of votes to determine
the best segmented facial region.
The faces in the mother database are all associated with
nominal poses. Take PIE as a sample, which offers 13 poses
approximately 22.5° apart [52]. Denote the frontal pose as
(φ0 = 0, θ0 = 0), or simply (0, 0), where φ is the pitch
angle and θ is the yaw angle. Those in horizontal rotations
are parametrized into (0, θj ), where j = ±1, ±2, ±3, ±4
denote for θj = ±22.5°, ±45°, ±67.5°, and ±90°, respectively.
Those with 22.5° chin-up and chin-down from (0, 0) are de-
noted as (φ1 =22.5°,0) and (φ−1 =−22.5°,0), respectively. The
mixed ones, c25 and c31, are parametrized as (φ−1 =−22.5°,
θ−3 =−67.5°) and (φ−1 = −22.5°, θ3 =67.5°), respectively. To
normalize the size, those with (φi , 0) can be normalized to Dh ,
the horizontal baseline between both eyes; those with (0, θj )
can be normalized to Dv , the vertical baseline between the
eyes and mouth. c25 and c31 are normalized with reduced Dh
and Dv . As Dh is zero at profile, Dv along allows the profile
to align with other poses. This step leads to a set of segmented
and normalized faces parametrized in specific poses.
The parametrized poses are limited by the poses available
from the mother database. Taking the aforementioned PIE
mother database as an example, only 13 pairs of (φi , θj )
are available. Although it is not a central concern of this
paper to generate poses that are different from those available
from the mother database, it is desired to have more poses
to offer. This issue by nature is similar to the generation of
additional illumination conditions, because both require some
reconstruction of a 3-D face from 2-D images. The generation
of additional poses and illumination conditions can be solved
by the illumination cone, proposed by Georghiades et al. [57]
and briefly presented in Section III-A3, which can reconstruct
the 3-D face given three aligned facial images of the same
pose but different illumination.
2) Illumination Clustering and Parametrization: Different
mother databases offer different illumination patterns. The
extended Yale database [57], [61] has 64 illumination con-
ditions on nine poses. PIE offers 43 illumination conditions
with 13 poses. While it is possible to come up with 43
234 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 2, FEBRUARY 2014
Fig. 1. Variation of the cost JL in (1) with different numbers of clusters,
with λc = 8 and 4 DCT components.
features to characterize the 43 illumination conditions for each
pose, we reduce this number by clustering those with similar
illumination patterns.
Given a set of faces with a normalized pose (φi∗ , θj∗ ), the
issue here is to cluster the 43 illumination patterns given
by 2924(= 43 × 68) face samples. Assuming that the illu-
mination pattern shown on each face can be approximated
to some extent by its low spatial frequency components, the
discrete cosine transform (DCT) can be applied to extract
the approximated illumination pattern from each aligned face
sample by inverting its DCT coefficients of low frequencies.
A similar application of DCT can be found in [62] for
illumination normalization. To solve this clustering problem,
one must determine the number of clusters to decompose
the 43 illumination patterns cast on 68 different faces. Our
experiments show that a good scheme to determine the cluster
number and simultaneously cluster the 2924 face samples is
to minimize the following cost function JL :
JL ({x̃i }, Nc , λc ) =
Nc 
j=1 ∀x̃i ∈Cj x̃i − mj 
Nc + λc log(Nc ) (1)
j=1 nj mj − m0 
where x̃i is the DCT-based illumination feature vector ex-
tracted from the face xi ; Nc is the number of clusters; nj
is the size of the cluster Cj , i.e., the number of x̃i ’s clustered
to Cj ; mj is the sample mean of Cj , and m0 is the mean
of the overall {x̃i }. λc is a weight that justifies the penalty
on a large Nc . The first part of (1) measures the ratio of
the scattering of the illumination patterns within the same
clusters to the scattering of the illumination patterns across
different clusters. When this ratio is minimized, the within-
cluster scatter of the illumination patterns is at a relative
minimum, and simultaneously the between-cluster scatter is
at a relative maximum. To compromise the condition with a
large Nc , which can reduce the ratio substantially, the second
part of (1) penalizes the model complexity induced by large
Nc with a weight λc .
Assuming that the illumination patterns extracted from a
set of faces with pose (φi∗ , θj∗ ) form a GMM, the expec-
tation maximization can be applied to cluster the extracted
illumination samples. To extract illumination from a facial
image, the DCT coefficients of low frequencies are selected for
clustering and checked whether each cluster collects samples
of similar illumination. We found that the DCT coefficients
chosen between the second to the ninth lowest frequencies
from a face 64×64 in size preserve most of the illumination.
Fig. 2. Nine illumination patterns, I1 –I4 on the top and I5 –I9 on the bottom,
extracted from 68 subjects with frontal pose and 43 illumination conditions.
Given the illumination patterns extracted from the 68 subjects
with frontal poses, (φi∗ = 0, θj∗ = 0), and 43 illumination
conditions, Fig. 1 shows the cost JL varying with Nc , where
λc = 8. The minimum occurs at Nc = 9, and the means of the
nine clustered illumination patterns shown in Fig. 2.
The illumination clustering can be repeated for all poses;
thus, each face can be parametrized in [φi , θj , Ik ]. For the
sample PIE mother database, i ∈ {−1, 0, 1}, j ∈ {0, ±1, ±2,
±3, ±4}, and k ∈ {1, 2, . . . , 9}.
3) Sample Add-On Using an Illumination Cone: Several
approaches, for example, the illumination cone [57] and the
3-D morphable model [63], can use a few sample faces to
generate faces with different poses and illumination condi-
tions. The illumination cone [57] is selected for its relatively
inexpensive storage and computation, and because its require-
ment can be readily satisfied by the samples from the mother
database. The illumination cone exploits the fact that there
exists a set of facial images with the same pose, but taken
under different illumination conditions, is a convex cone in
the image space. Using a few training images of each face
taken under different illumination conditions, the shape and
albedo of the face can be reconstructed using the generalized
bas-relief (GBR) transformation. This reconstruction leads to
a generative model that is able to synthesize the images of
the face under novel poses and illumination conditions. The
requirement on training images of the same pose but different
illumination can be readily met by the samples from the PIE
mother database.
Given k aligned images of a face from the mother database,
denoted as [xi ]ki=1 , xi ∈ Rn , one can form a matrix X =
[x1 , x2 , . . . , xk ] with each column formed by one facial image.
With Lambertian assumption, the illumination cone algorithm
recursively searches for two matrices, B ∈ Rn×3 and S ∈ R3×k ,
to minimize the difference between X and B · S, that is
B∗ , S∗ = arg min X − B · S (2)
B,S
where B = [b1 b2 · · · bn ]T . Each row of B is the product of the
albedo with the unit normal at a point on the face projecting
to a particular pixel in the image, that is
[zx (j) zy (j) − 1]
bTj = αj j = 1, ..., n (3)
z2x (j) + z2y (j) + 1
where αj is the albedo at a point z on the face projecting
to the jth pixel, and zx (j) and zy (j) are the gradients of the
facial surface z(j) along the horizontal and vertical directions,
respectively. Equation (3) can be written into the following
HSU AND CHU: A FRAMEWORK FOR FACE DETECTION BENCHMARK DATABASES
Fig. 3. Three faces on the left are directly available from the PIE mother database; the right five faces are synthesized by an illumination cone.
form for better interpretation with the pixel at (x, y) on the
2-D facial image
[zx (x, y) zy (x, y) − 1]
bT (x, y) = α(x, y) 2 (4)
zx (x, y) + z2y (x, y) + 1
where z(x, y) is the objective facial surface to estimate, and
zx (x, y) and zy (x, y) are the directional derivatives. In (2), S =
[s1 , s2 , . . . , sk ], where each si ∈ R3 shows the ith light source
in the infinity with intensity si  and direction si /si .
To evaluate z(x, y), the algorithm first obtains S0 , an initial
estimate of S, by performing SVD on X and extracting the
three primary basis vectors that best span the row space of X.
The iterative part of the algorithm starts with the evaluation
of B0 , an initial estimate of B, using (2) with X and S0 . An
initial estimate of the albedo α0 (x, y) can be given by taking
the average of [xi ]ki=1 ; the gradient fields zx (x, y) and zy (x, y)
can then be evaluated using (4). To overcome a possible
integrability issue with zx (x, y) and zy (x, y) obtained this way,
z(x, y) is assumed able to be approximated by DCT, that is

z(x, y) = c(w)ψ(x , y; w) (5)
where w defines a local window over which the sum is
performed, and {c(w)} are the DCT coefficients. Because the
basis set {ψ(x , y; w)} is integrable, the gradient fields zx (x, y)
and zy (x, y) approximated by ψx (x , y; w) and ψy (x , y; w), re-
spectively, are also integrable. Given ψx (x , y; w), ψy (x , y; w),
and the estimated S, the albedo α(x, y) can be updated by
least-square minimization, and then B can be recomputed
using (4). The direction and magnitude of each light source si
must be updated independently using the recomputed B. The
aforementioned steps of updating S, then α(x, y), and then B
go on until these estimates converge. Because ψx (x , y; w) and
ψy (x , y; w) are made integrable using DCT approximation, the
GBR surface z̄(x, y) can be obtained by taking an inverse DCT
on the final coefficients {c̄(w)}.
Fig. 3 shows five synthesized cases on the right, different
from the three on the left directly taken from the PIE database.
4) Parametrization of Other Intrinsic Variables: Although
pose and illumination are two major intrinsic variables, the
importance of other intrinsic variables cannot be overempha-
sized. For the diversity of the daughter datasets, we also
consider expression, orientation (in-plane rotation), gender,
and accessaries such as eyeglasses and hats. Some of these
variables are easy to parametrize, but some are constrained by
the mother database, as stated in the following.
1) Limited by the three expressions offered by the sample
PIE mother database, the expression parameters in the
PFD can only be given 1 for smile, 0 for neutral, and
−1 for blinking.
235
2) Gender can take 1 for male and 0 for female.
3) Orientation, i.e., in-plane rotation, can take −180° to
180°, if the upright pose is considered as 0° and turns
clockwise as positive.
4) Partial occlusion is handled according to two different
types: the first is mutual occlusion, measured by the
percentage of a face occluded by another face; and the
second is caused by accessories, such as sun glasses,
masks, and hats. A few accessary templates, collected
from various sources, including the AR database [64],
are made available as an add-on to the faces in the PFD.
Although the generation of different expressions is beyond
the scope of this paper, it highlights the fact that the intrin-
sic parameters available from the proposed framework are
constrained by what the mother database can offer. On the
other hand, the framework would be able to offer intrinsic
parameters with wider scopes of variations if the techniques
for generating novel appearances become available.
In summary, the sample mother database, PIE, contributes
41 368 facial images with different poses, illumination con-
ditions, expressions, and genders. Given a few facial images
aligned to the same pose but taken under different illumination,
the illumination cone [57] is applied to generate faces with
novel poses and illumination conditions, extending the scopes
of the intrinsic parameters directly available from the mother
database.
To this end, the IP phase leads to a PFD with each face
parametrized in p = [φi , θj , ik , ol , g, em , cn ], where φi and θj
specify pose-(i, j), ik for illumination-k, ol for orientation-l, g
for gender, em for expression-m, and cn for occlusion. Given a
specified ∗p , a subset of faces can be segmented or generated
from the PFD.
B. EP
This phase handles how faces of different intrinsic and ex-
trinsic parameters appear across a set of cluttered background
images. The extrinsic parameters include the size density
distribution (SDD), the total number of faces needed, the
overlap between faces, the spatial distribution of the faces,
and the resolution. The settings of each extrinsic parameter
are given in the following.
1) The SDD is given by the number of faces at each
different size over the size range [smin smax ]. It is then
normalized to the total number of faces to make it a
valid distribution. It can be assigned as uniform, single
or multiple Gaussian, or other patterns for different
scenarios. It can also be defined for disjoint segments
with [smin smax ].
236 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 2, FEBRUARY 2014

Fig. 4. Low-resolution samples made by applying scale space with the

Gaussian kernel [65].

2) Nf is the total number of faces needed. Multiplication

with SDD gives the exact number of the faces at each
different size.
3) The aforementioned parameters can render a daughter
dataset if the faces are merged into a set of cluttered
background images. The faces can be distributed arbi-
trarily or according to a preselected distribution. How-
ever, for certain applications, the 2-D spatial distribution
of the faces must reflect the distribution of 3-D faces in
a 3-D scene. With a chosen focal length, we can specify
the number of faces located at different distances from
the camera. Prospective modeling with a normal head
model for each face is used here to obtain its projection
onto a 2-D image.
4) R is the resolution, or more precisely the spatial resolu-
tion, which determines how closely lines can be resolved
in an image. Low-resolution images pose challenges to
face detection. Because the scale-space framework is
successfully applied to capturing the resolution variation
caused by different scales [65], we apply the scale space
with Gaussian kernel to make low-resolution images.
This technique is applied to the whole image, instead
of on faces only to reflect the fact that resolution is a
global extrinsic parameter. However, for demonstration
purposes, Fig. 4 shows a sample face with resolution
reduced by applying the Gaussian kernels for multiple
times.
C. Application Scope of the Framework
A few artificial faces were collected in the MIT+CMU
database, including hand-drawn ones and a few from poker
cards, cartoons, masks, and advertisements, as shown in Fig.
5. Our eyes can easily discern faces and even face-like objects
from others, no matter whether they are real or artificial
ones, and no matter whether they have body parts or even
head contours. An ideal face detector or manmade human-
like vision is expected to do the same. This highlights the
following properties about the application scope of the pro-
posed framework.
1) Best for evaluating scanning-window based face de-
tectors: Almost all face detectors use sliding windows
of various sizes scanning through a give image, and
determine whether the region enclosed by each window
is a face or nonface. Because face detection algorithms
do not consider backgrounds and other body parts, the
daughter datasets generated by the proposed framework
serve well for benchmarking such scanning-window-
based algorithms.
2) Offering suboptimal test set with synthesized data: Given
a criterion, the optimal test sets are presumed to be the
Fig. 5. Artificial faces in the MIT+CMU database, which are easy for
humans to detect, and an ideal face detector is expected to do the same.
Fig. 6. Performance comparison on Rowley’s and FDDB datasets and their
synthesized counterparts.
real data collected in the field that meet the requirements.
However, such optimal real test sets can be extremely
difficult to obtain for some specific test criteria; for
example, pose between 45° to profile 90° with right-
sided illumination. While it is believed that a benchmark
on real data reveals the performance one can expect
in the field, the collection of the right real data must
be ensured. The daughter datasets generated by the
framework offer a suboptimal solution to this difficult-
to-collect issue.
3) Offering a novel perspective to benchmark face detec-
tion: Although the daughter datasets are not made to
contain hand-drawn or cartoon faces like those available
in the MIT+CMU dataset, they can be easily included
to show some extensions of intrinsic and extrinsic pa-
rameters that are rarely seen in real life. However, is
face detection expected to work for real-life samples
only? Because human vision can tell faces from others
regardless of whether the faces are real or artificial,
there is no reason to ask an ideal face detector not to
do the same. As artificial images are made for various
purposes in this era of multimedia, and face detection
in artificial images is important for content-based image
retrieval and categorization, the proposed framework can
be the first one that is able to offer a good scope of
synthesized images for benchmarking the desired face
detectors.
HSU AND CHU: A FRAMEWORK FOR FACE DETECTION BENCHMARK DATABASES
Fig. 7. Original images are labeled in numbers, and with faces replaced labeled in the same numbers but followed by an “r.” (a) Rowley’s (a MIT+CMU
subset). (b) FDDB.
IV. Experimental Validation
Two protocols were designed to validate the proposed
framework. The first one was designed to justify the va-
lidity of the daughter datasets synthesized by the proposed
framework, and the second was designed to show how to
use the framework to define the performance specifications
of a face detector. In the first protocol, we compared the
performance of a few state-of-the-art face detectors tested on
two benchmark datasets, namely the most popular Rowley’s
dataset [3] and the recently released FDDB [12] against that
tested on their synthesized counterparts. The latter were made
by the former with faces replaced by samples with similar
intrinsic parameters from the PFD. In the second protocol, the
three face detectors were tested on several daughter datasets
with different intrinsic and extrinsic parameters so that their
performance specifications and limitations would be clearly
drawn. The datasets used in both protocols can be available at
https://sites.google.com/site/facedetectiondatasets/home.
The selected face detectors include the latest one by Zhu and
Ramanan [58], one by Kalal et al. [66], and one advanced from
Viola and Jones [67], which are, respectively, called Zhu’s,
Kalal’s, and Viola’s face detectors in this paper. Zhu’s and
Kalal’s detectors can be available from the resources provided
by the authors, and Viola’s detector is available from the
OpenCV open source library [68].
A. Performance Comparison on Real and Synthesized
Datasets
The samples from the original databases are called real data,
and the real data with faces replaced by similar ones from the
PFD are called synthesized data in the following. To replace
the faces in the FDDB, the scheme presented in Section III-A1
was applied, which not only segments faces but also estimates
their poses and local features.
From the samples with the same poses in the PFD, those
with similar illumination, justified by low-frequency DCT
components, were selected to replace the segmented faces. Out
of the total 5171 faces, 4808 or 93% were replaced, and 363
or 7% were not replaceable due to extreme poses, expressions,
occlusion, and size less than 20×20.
237
Because the resolution of many samples in Rowley’s dataset
was too low to be processed by Zhu’s detector, we replaced it
by Viola’s face detector and kept the segmentation the same.
Out of the total 507 faces in Rowley’s, 478 or 94.3% were
replaced by similar ones from the PFD and 29 were not
replaceable.
Fig. 6 shows the detection rates of the three detectors on
the real and synthesized with increasing false-positive rates
(FPRs). The detection rates on the synthesized are generally
better than those on the real, but the gaps in between are
all within 5% for FPR >0.03, and decrease to 3% for
FPR >0.08. Such small gaps demonstrate the similarity of
the synthesized to the real. To better understand the causes
of the performance gaps in Fig. 6, one needs to examine
the results on corresponding samples across the real and
synthesized datasets. The following observations are from the
tests using the Viola’s detector, and similar observations can
also be obtained from the other two detectors. Fig. 7 shows a
few samples before and after the replacement, along with the
detection results. Each original image is tagged by a number,
and the one with faces replaced is tagged by the same number
but with a “r” at the end. Except for a few cases where
the real fails to be detected but its synthesized counterpart
can be detected, e.g., samples 1 and 1r in Rowley’s set in
Fig. 7, most faces detected in the real remain detected when
they are replaced in the synthesized, and those missed in the
real mostly remain missed in the synthesized. Some false
positives without overlap with faces also remain after the
replacement, e.g., samples 2 and 2r in Rowley’s set.
Although the overall performance on both the real and
synthesized data seems similar, some differences caught our
attention. Several special cases are revealed in Fig. 8; column
a shows a case in which the real face shadowed by the hat
fails to be detected (a1); it becomes detected when replaced
by a face without shadow (a2), and then fails again when the
synthesized is cast with a shadow. Column b shows that a
false positive overlapped with the real face is removed when
the face is replaced (b1 and b2). A face behind the main
one in b1 fails to be detected because of partial occlusion,
and remains undetected after the replacement by faces with
238 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 2, FEBRUARY 2014

Fig. 8. Cases showing that shadows and accessories, e.g., hats, would result in different performance (columns a, c, and d), and false positives overlapped
with faces may be removed in the synthesized (column b). The top row are real and the rest are synthesized.

Fig. 9. Typical samples from each daughter dataset generated for performance evaluation against a specific intrinsic parameter. (f) Perspective model for the
layout in (b). (a) Pose. (b) Illumination-frontal. (c) Illumination-downward. (d) Occlusion. (e) Orientation. (f) Perspective model.

different fractions of occlusion in b2 and b3. Column c
is similar to column a with hat-cast shadows; however, c2
and c3 show that the synthesized can be detected even with
the eyeglasses, hat, and hat-cast shadow added in, possibly
because the synthesized shadow may not be strong enough or
the shadowed pattern collected in the training set. Column d
shows a case in which a real one with eyeglasses and hat-cast
shadow fails to be detected, but can be detected when replaced
by a face without glasses and weakened shadow. The cases in
Fig. 8 reveal that the proposed framework allows a variety
of conditions and additional parameters to be included in the
daughter datasets to expand the application scopes.
B. Daughter Sets for Performance Specification
To determine the performance specifications of the three
selected face detectors, a few daughter datasets were designed
with the following intrinsic parameters (the labels, such as
c02, c07, c27, follow those used in the PIE database).
1) Pose set: Eight subsets and each with one or a pair
of specific poses, namely (in the PIE pose tags [52]),
c27 (frontal, 0°), c05/c29 (22.5° to the right and left),
c07 (22.5° up), c09 (22.5° down), c11/c37 (45° to both
sides), c02/c14 (67.5° to both sides), c25/c31 (67.5° to
both sides and 22.5° down), and c22/c34 (90° to both
sides). The remaining intrinsic parameters were the same
HSU AND CHU: A FRAMEWORK FOR FACE DETECTION BENCHMARK DATABASES
Fig. 10. Specifications of three face detectors, Zhu’s (green), Viola’s (red), and Kalal’s (blue), in terms of the detection rates on five daughter sets, each
with an intrinsic parameter as the only variable. (a) Pose. (b) Illumination-frontal. (c) Illumination-downward. (d) Occlusion. (e) Orientation.
for each set: illumination was with I3 (shown in Fig. 2)
for its balance across the face, in-plane orientation less
than 10°, and no occlusion.
2) Illumination frontal set: Six subsets with six illumination
patterns, namely, I3 , I5 , I6 , I7 , I8 , and I9 , and all with
frontal pose (c27). The rest of the intrinsic parameters
were the same as the above for the pose set.
3) Illumination downward set: Six subsets with the same
looking-downward pose but viewed from different yaw
angles, including c09 (22.5° down), c25 (22.5° down and
67.5° to the right), and c31 (22.5° down and 67.5° to the
left), and each with a different illumination pattern as for
the illumination frontal set. The rest intrinsic parameters
were the same as the previous cases.
4) Occlusion set: Four subsets with some percentage of a
face covered by another face, the percentage was made
to be 5%, 10%, 20%, and 30%. All were with pose c27,
illumination I3 , and orientation less than 10°.
5) Orientation set: Six subsets with different degrees of in-
plane rotation, namely ±10°, ±20°, and ±30° clockwise
and counterclockwise. All were with pose c27, illumi-
nation I3 , and without occlusion.
Both random pasting and perspective modeling were used to
generate the aforementioned sets. The number of faces in each
set varied from 192 to 400, and the size of each face was within
0.1–0.3 the height of the background image. The size density
SDD was assumed as a bandlimited Gaussian. Fig. 9 shows
typical samples from each set and a perspective model in
Fig. 9(f) considered for the layout in Fig. 9(b).
The test results on these intrinsic-parameter-specific daugh-
ter sets are shown in Fig. 10. Zhu’s detector performs the
best in the pose set with exceptional detection rates on large
239
poses, i.e., 82% on c22 and 92% on c02, far better than
the other two. All three handle the nearly frontal poses well,
but Kalal’s and Viola’s performances drop sharply when the
pose goes beyond c37 (45°). All three perform well on the
illumination frontal set, except that Kalal’s detector seems to
be unable to detect many faces lit on the right side, although it
performs the best for detecting those lit on the left side. Such
a asymmetry performance is also observed in the illumination
downward set, where both Kalal’s and Zhu’s detectors perform
better on one side of illumination than the other. Viola’s
shows the most consistent performance across all lighting
conditions. As for the capability of handling occlusion, Zhu’s
again outperforms the others, as shown in the occlusion set,
with detection rate 94% on 30% occlusion. However, as shown
in the orientation set, Zhu’s detector seems unable to handle
orientation well; it degrades substantially when detecting faces
of ±10° in orientation. Both Kalal’s and Viola’s detectors show
some asymmetry results with better performance handling
orientation 20°–30° than the case with −20°–30°, but Zhu’s
detector shows more symmetry performance.
To the best of our knowledge, the charts in Fig. 10 show
for the first time that the performance of face detectors can
be specified quantitatively with various intrinsic variables and
compared from different aspects of performance needs.
V. Conclusion
A framework was proposed to generate datasets that are
good for benchmarking face detection using a database meant
for benchmarking face recognition. The framework was com-
posed of IP and EP phases. The former categorized the
facial samples from the mother database according to intrinsic
parameters, and the latter generated the daughter datasets
240 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 2, FEBRUARY 2014
according to desired settings on the intrinsic and extrinsic
parameters. Our experiments showed that the daughter dataset
can be made similar to popular benchmarks and the perfor-
mance specification of a face detector can be defined using
daughter datasets with parameters of special settings.
A few questions are yet to be answered and are considered
in the continuing phase of this paper. For example, how should
we define a minimum number of daughter datasets so that the
performance specification of a face detector can be efficiently
defined? Should a generic daughter dataset be applied first to
guide the settings for more specific daughter datasets applied
later when evaluating a face detector? Is a face detector
better trained on limited poses and various illumination or
vice versa? How does its performance vary with training sets
of different variations? These questions and others will be
answered in the near future.
References
[1] R. Kasturi, D. Goldgof, P. Soundararajan, V. Manohar, J. Garofolo,
R. Bowers, M. Boonstra, V. Korzhova, and J. Zhang, “Framework for
performance evaluation of face, text, and vehicle detection and tracking
in video: Data, metrics, and protocol,” IEEE Trans. Pattern Anal. Mach.
Intell., vol. 31, no. 2, pp. 319–336, Feb. 2009.
[2] K.-K. Sung and T. Poggio, “Example-based learning for view-based
human face detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20,
no. 1, pp. 39–51, Jan. 1998.
[3] H. Rowley, S. Baluja, and T. Kanade, “Neural network-based face
detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 1, pp.
23–38, Jan. 1998.
[4] H. Schneiderman and T. Kanade, “A statistical method for 3D object
detection applied to faces and cars,” in Proc. IEEE Conf. Comput. Vis.
Pattern Recognit., vol. 1. Jun. 2000, pp. 746–751.
[5] M.-H. Yang, D. Kriegman, and N. Ahuja, “Detecting faces in images:
A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 1, pp.
34–58, Jan. 2002.
[6] C. A. Waring and X. Liu, “Face detection using spectral histograms and
SVMs,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 3, pp.
467–476, Jun. 2005.
[7] S. Kadoury and M. D. Levine, “Face detection in gray scale images
using locally linear embeddings,” Comput. Vis. Image Understanding,
vol. 105, no. 1, pp. 1–20, 2007.
[8] W.-K. Tsao, A. J. T. Lee, Y.-H. Liu, T.-W. Chang, and H.-H. Lin, “A
data mining approach to face detection,” Pattern Recognit., vol. 43, pp.
1039–1049, Sep. 2009.
[9] J. Wu and Z.-H. Zhou, “Efficient face candidates selector for face
detection,” Pattern Recognit., vol. 36, no. 5, pp. 1175–1186, 2003.
[10] A. Kouzani, “Locating human faces within images,” Comput. Vis. Image
Understanding, vol. 91, pp. 247–279, Sep. 2003.
[11] G.-S. Hsu, T. H. Tran, and S.-L. Chung, “Benchmark face detection
using a face recognition database,” in Proc. 17th IEEE Int. Conf. Image
Process., Sep. 2010, pp. 3821–3824.
[12] V. Jain and E. Learned-Miller, “FDDB: A benchmark for face detection
in unconstrained settings,” Univ. Massachusetts, Amherst, MA, USA,
Tech. Rep. UM-CS-2010-009, 2010.
[13] L.-L. Huang and A. Shimizu, “A multi-expert approach for robust face
detection,” Pattern Recognit., vol. 39, no. 9, pp. 1695–1703, 2003.
[14] R.-L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, “Face detection in color
images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp.
696–706, May 2002.
[15] O. Ayinde and Y.-H. Yang, “Region-based face detection,” Pattern
Recognit., vol. 35, no. 10, pp. 2095–2107, 2002.
[16] L.-L. Huang, A. Shimizu, Y. Hagihara, and H. Kobateke, “Gradient fea-
ture extraction for classification-based face detection,” Pattern Recognit.,
vol. 36, no. 11, pp. 2501–2511, 2003.
[17] C. Liu, “A Bayesian discriminating features method for face detection,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 6, pp. 725–740,
Jun. 2003.
[18] C. Huang, B. Wu, H. Ai, and S. Lao, “Omni-directional face detection
based on real adaboost,” in Proc. IEEE Int. Conf. Image Process., vol. 1.
Oct. 2004, pp. 593–596.
[19] Y.-Y. Lin and T.-L. Liu, “Robust face detection with multi-class boost-
ing,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1. Jun.
2005, pp. 680–687.
[20] H. Zhang, W. Gao, X. Chen, S. Shan, and D. Zhao, “Robust multi-view
face detection using error correcting output codes,” in Proc. 9th Eur.
Conf. Comput. Vis., vol. 3954. 2006, pp. 1–12.
[21] P. Shih and C. Liu, “Face detection using discriminating feature analysis
and support vector machine,” Pattern Recognit., vol. 39, no. 2, pp. 260–
276, 2006.
[22] D. Nguyen, D. Halupka, P. Aarabi, and A. Sheikholeslami, “Real-time
face detection and lip feature extraction using field-programmable gate
arrays,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 4, pp.
902–912, Aug. 2006.
[23] J. Wu, S. C. Brubaker, M. D. Mullin, and J. M. Rehg, “Fast asymmetric
learning for cascade face detection,” IEEE Trans. Pattern Anal. Mach.
Intell., vol. 30, no. 3, pp. 369–382, Mar. 2008.
[24] R. Wang, J. Chen, S. Shan, and W. Gao, “Enhancing training set for
face detection,” in Proc. IEEE 18th Int. Conf. Pattern Recognit., vol. 3.
Aug. 2006, pp. 477–480.
[25] J. Meynet, V. Popovici, and J.-P. Thiran, “Face detection with boosted
Gaussian features,” Pattern Recognit., vol. 40, pp. 2283–2291, Aug.
2007.
[26] C. Huang, H. Ai, Y. Li, and S. Lao, “High performance rotation invariant
multiview face detection,” IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 29, no. 4, pp. 671–686, Apr. 2007.
[27] P. Wang and Q. Ji, “Multi-view face and eye detection using discriminant
features,” Comput. Vis. Image Understanding, vol. 105, no. 2, pp. 99–
111, 2007.
[28] R. Xiao, H. Zhu, H. Sun, and X. Tang, “Dynamic cascades for face
detection,” in Proc. IEEE 11th Int. Conf. Comput. Vis., Oct. 2007, pp.
1–8.
[29] C. Shen, S. Paisitkriangkrai, and J. Zhang, “Face detection from few
training examples,” in Proc. IEEE Int. Conf. Image Process., Oct. 2008,
pp. 2764–2767.
[30] S. Yan, S. Shan, X. Chen, and W. Gao, “Locally assembled binary (LAB)
feature with feature-centric cascade for fast and accurate face detec-
tion,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2008,
pp. 1–7.
[31] M. Toews and T. Arbel, “Detection, localization, and sex classification
of faces from arbitrary viewpoints and under occlusions,” IEEE Pattern
Anal. Mach. Intell., vol. 31, no. 9, pp. 1567–1581, Sep. 2009.
[32] K. Hotta, “View independent face detection based on horizontal rect-
angular features and accuracy improvement using combination kernel
of various sizes,” Pattern Recognit., vol. 42, no. 3, pp. 437–444,
2009.
[33] J. Chen, X. Chen, J. Yang, S. Shan, and W. Gao, “Optimization of a
training set for more robust face detection,” Pattern Recognit., vol. 42,
no. 11, pp. 2828–2840, 2009.
[34] S. Yan, S. Shan, X. Chen, and W. Gao, “FEA-Accu cascade for face
detection,” in Proc. 16th IEEE Int. Conf. Image Process., Nov. 2009,
pp. 1217–1220.
[35] M.-T. Pham, Y. Gao, V.-D. D. Hoang, and T.-J. Chen, “Fast polygonal
integration and its application in extending haar-like features to improve
object detection,” in Proc. IEEE Comput. Vis. Pattern. Recognit., Jun.
2010, pp. 942–949.
[36] V. Subburaman and S. Marcel, “An alternative scanning strategy to detect
faces,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Mar.
2010, pp. 2122–2125.
[37] J.-M. Guo and M.-F. Wu, “Pixel-based hierarchical-feature face detec-
tion,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Mar.
2010, pp. 1638–1641.
[38] J. Shen, W. Yang, and C. Sun, “Learning discriminative features based
on distribution,” in Proc. 20th Int. Conf. Pattern Recognit., Aug. 2010,
pp. 1401–1404.
[39] L. Ding and A. Martinez, “Features versus context: An approach
for precise and detailed detection and delineation of faces and facial
features,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp.
2022–2038, Nov. 2010.
[40] J. Chen, S. Shan, C. He, G. Zhao, M. Pietika, X. Chen, and W. Gao,
“WLD: A robust local image descriptor,” IEEE Trans. Pattern Anal.
Mach. Intell., vol. 32, no. 8, pp. 1705–1720, Sep. 2010.
[41] S. Stein and G. A. Fink, “A new method for combined face detection and
identification using interest point descriptors,” in Proc. IEEE Int. Conf.
Autom. Face Gesture Recognit. Workshops, Mar. 2011, pp. 519–524.
[42] S. Paisitkriangkrai, C. Shen, and J. Zhang, “Incremental training of a
detector using online sparse eigendecomposition,” IEEE Trans. Image
Process., vol. 20, no. 1, pp. 213–226, Jan. 2011.
HSU AND CHU: A FRAMEWORK FOR FACE DETECTION BENCHMARK DATABASES
[43] W. Louis and K. Plataniotis, “Frontal face detection for surveillance
purposes using dual local binary patterns features,” in Proc. 17th IEEE
Int. Conf. Image Process., Sep. 2010, pp. 3809–3812.
[44] P. Phothisane, E. Bigorgne, L. Collot, and L. Prevost, “A robust
composite metric for head pose tracking using an accurate face model,”
in Proc. IEEE Int. Conf. Autom. Face Gesture Recognit. Workshops,
Mar. 2011, pp. 694–699.
[45] U. Yang, M. Kang, K.-A. Toh, and K. Sohn, “An illumination invariant
skin-color model for face detection,” in Proc. 4th IEEE Int. Conf.
Biometrics Theory Appl. Syst., Sep. 2010, pp. 1–6.
[46] Z. Kalal, K. Mikolajczyk, and J. Matas, “Face-TLD: Tracking-learning-
detection applied to faces,” in Proc. 17th IEEE Int. Conf. Image Process.,
Sep. 2010, pp. 3789–3792.
[47] C. Whitelam, Z. Jafri, and T. Bourlai, “Multispectral eye detection: A
preliminary study,” in Proc. 20th IEEE Int. Conf. Pattern Recognit., Aug.
2010, pp. 209–212.
[48] Q. Yuan, A. Thangali, V. Ablavsky, and S. Sclaroff, “Learning a family
of detectors via multiplicative kernels,” IEEE Trans. Pattern Anal. Mach.
Intell., vol. 33, no. 3, pp. 514–530, Mar. 2011.
[49] K. Ali, F. Fleuret, D. Hasler, and P. Fua, “A real-time deformable
detector,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 2, pp.
225–239, Feb. 2012.
[50] E. Bailly-Bailliere, S. Bengio, F. Bimbot, M. Hamouz, J. Kittler,
J. Mariethoz, J. Matas, K. Messer, V. Popovici, F. Poree, B. Ruiz, and
J.-P. Thiran, “The BANCA database and evaluation protocol,” in Proc.
4th Int. Conf. Audio- Video-Based Biometric Person Authenticat., 2003,
pp. 625–638.
[51] B. Weyrauch, J. Huang, B. Heisele, and V. Blanz, “Component-based
face recognition with 3D morphable models,” in Proc. IEEE Conf.
Comput. Vis. Pattern Recognit., Jun. 2004, p. 85.
[52] T. Sim, S. Baker, and M. Bsat, “The CMU pose, illumination, and
expression database,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25,
no. 12, pp. 1615–1618, Dec. 2003.
[53] O. Jesorsky, K. Kirchberg, and R. Frischholz, “Robust face detection
using the Hausdorff distance,” in Proc. 3rd Int. Conf. Audio-Video-Based
Biometric Person Authenticat., vol. 2091. Jun. 2001, pp. 90–95.
[54] P. Sharma and R. Reilly, “A color face image database for benchmark
of automatic facial detection algorithms,” in Proc. 4th Eur. Conf.
Video/Image Multimedia Commun., Jul. 2003, pp. 423–428.
[55] R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, “Multi-pie,”
Image Vis. Comput., vol. 28, pp. 807–813, May 2010.
[56] W. Gao, B. Cao, S. Shan, X. Chen, D. Zhou, X. Zhang, and D. Zhao,
“The CAS-PEAL large-scale Chinese face database and baseline evalu-
ations,” IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no.
1, pp. 149–161, Jan. 2008.
[57] A. Georghiades, P. Belhumeur, and D. Kriegman, “From few to many:
Illumination cone models for face recognition under variable lighting
and pose,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp.
643–660, Jun. 2001.
[58] X. Zhu and D. Ramanan, “Face detection, pose estimation, and landmark
localization in the wild,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., Jun. 2012, pp. 2879–2886.
[59] C. Rother, V. Kolmogorov, and A. Blake, ““GrabCut”: Inter-active
foreground extraction using iterated graph cuts,” ACM Trans. Graph.,
vol. 23, no. 3, pp. 309–314, 2004.
241
[60] Y. Boykov and M.-P. Jolly, “Interactive graph cuts for optimal boundary
and region segmentation of objects in N-D images,” in Proc. 8th IEEE
Int. Conf. Comput. Vis., Jul. 2001, pp. 105–112.
[61] P. N. Belhumeur and D. J. Kriegman, “What is the set of images of an
object under all possible illumination conditions,” Int. J. Comput. Vis.,
vol. 28, no. 3, pp. 1–16, 1998.
[62] W. Chen, M. J. Er, and S. Wu, “Illumination compensation and nor-
malization for robust face recognition using discrete cosine transform in
logarithm domain,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36,
no. 2, pp. 458–466, Apr. 2006.
[63] V. Blanz and T. Vetter, “Face recognition based on fitting a 3D
morphable model,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25,
no. 9, pp. 1063–1074, Sep. 2003.
[64] A. Martinez and R. Benavente, “The AR face database,” Purdue Univ.,
West Lafayette, IN, USA, CVC Tech. Rep. 24, 1998.
[65] D. Lowe, “Distinctive image features from scale-invariant keypoints,”
Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, 2004.
[66] Z. Kalal, J. Matas, and K. Mikolajczyk, “Weighted sampling for large-
scale boosting,” in Proc. Brit. Mach. Vis. Conf., 2008, pp. 42.1–42.10.
[67] P. Viola and M. Jones, “Rapid object detection using a boosted cascade
of simple features,” in Proc. IEEE Comput. Vis. Pattern Recognit., vol. 1.
Dec. 2001, pp. I-511–I-518.
[68] OpenCV. (2012). OpenCV 2.4.4 [Online]. Available:
http://docs.opencv.org/
Gee-Sern Hsu (M’09) received the dual M.S. degree
in electrical and mechanical engineering and the
Ph.D. degree in mechanical engineering from the
University of Michigan, Ann Arbor, MI, USA, in
1993 and 1995, respectively.
From 1995 to 1996, he was a Post-Doctoral Fellow
with the University of Michigan. From 1997 to 2000,
he was a Senior Research Staff with the National
University of Singapore, Singapore. In 2001, he
joined Penpower Technology, where he led research
on face recognition and intelligent video surveil-
lance. In 2007, he joined the Department of Mechanical Engineering, National
Taiwan University of Science and Technology, Taipei, Taiwan, as an Assistant
Professor. His current research interests include object recognition, particularly
face recognition, pedestrian detection, and vehicle license plate recognition.
Dr. Hsu, along with his team at Penpower Technology, was a recipient of
the Best Innovation Award at the SecuTech Expo during 2005–2007.
Tsu-Ying Chu received the B.E. degree in electro-
mechanical engineering from the National Ilan Uni-
versity, I-Lan, Taiwan, in 2010, and the M.S. degree
in mechanical engineering from the National Taiwan
University of Science and Technology, Taipei, Tai-
wan, in 2012.
She is an Image Processing Engineer at Vols
Taipei.