See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/317418143

Facial recognition using histogram of gradients and support vector machines

Conference Paper · January 2017

DOI: 10.1109/ICCCSP.2017.7944082

CITATIONS READS
19 2,064

2 authors:

Josephine Julina J. K. T. Sree Sharmila

Sri Sivasubramaniya Nadar College of Engineering Sri Sivasubramaniya Nadar College of Engineering
5 PUBLICATIONS   32 CITATIONS    71 PUBLICATIONS   301 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

content based image retrieval View project

Computer Vision Based Automated Detection of Object Tracking in Video Surveillance Applications View project

All content following this page was uploaded by Josephine Julina J. K. on 08 November 2017.

The user has requested enhancement of the downloaded file.

IEEE International Conference on Computer, Communication, and Signal Processing (ICCCSP-2017)
Facial Recognition using Histogram of Gradients and
Support Vector Machines
J. Kulandai Josephine Julina
Department of Information Technology
SSN College of Engineering
Chennai, India
josephinejulinajk@ssn.edu.in
Abstract— Face recognition is widely used in computer vision
and in many other biometric applications where security is a
major concern. The most common problem in recognizing a face
arises due to pose variations, different illumination conditions
and so on. The main focus of this paper is to recognize whether a
given face input corresponds to a registered person in the
database. Face recognition is done using Histogram of Oriented
Gradients (HOG) technique in AT & T database with an
inclusion of a real time subject to evaluate the performance of the
algorithm. The feature vectors generated by HOG descriptor are
used to train Support Vector Machines (SVM) and results are
verified against a given test input. The proposed method checks
whether a test image in different pose and lighting conditions is
matched correctly with trained images of the facial database. The
results of the proposed approach show minimal false positives
and improved detection accuracy.
Keywords- AT & T facial database; Face recognition; Feature
extraction; Histogram of Oriented Gradients; Support Vector
Machine
I. INTRODUCTION
The face is an important identity of a person. It is obvious
that humans are experts in recognizing people at a farther
distance, predicting the identity of a person in more prominent
occlusions and in different lighting conditions. However, for
the system, it requires many iterations and tuning of threshold
parameters to recognize a person and to reject appropriately
when a wrong input is given.
The location of facial components namely eyes, nose, and
mouth constitute prime landmarks in marking the presence of
a face in an image. However, this process would be made
difficult when a person tends to exhibit different expressions
and pose variations. Some of the commonly encountered
challenges in facial detection and recognition include
variations in lighting conditions, occlusion, wearing of
spectacles, having more facial hair and so on. Certain
techniques like template matching methods [19], geometry
based approaches [1], appearance-based models [3] are
adopted to overcome such challenges.
Face detection is the first and foremost step in identifying
the presence of faces in an image. Then face recognition or
verification is carried out which checks whether a given test
input containing face is matched with already available faces
978-1-5090-3716-2/17/$31.00 ©2017 IEEE
T. Sree Sharmila
Department of Information Technology
SSN College of Engineering
Chennai, India
sreesharmilat@ssn.edu.in
stored in the database or face gallery. The details about the
face and its different facial components can be obtained based
on features which are chosen in such a way that they are
robust to noise and different orientation. Most of the facial
recognition problems deal with feature extraction and machine
learning techniques. Facial recognition tasks are performed in
many areas of image and vision applications where security is
more focused without any second thought of compromising it.
Many applications deal with mug shot images which are used
to recognize faces of any matched criminals in the database.
The remaining sections of the paper is organized as
follows: Section II provides details about related work carried
in recognition of faces and feature extraction. Section III
provides details about the architecture of the proposed method
for detecting whether an image corresponds to matched face in
the gallery. Section IV provides qualitative results obtained
after testing with images of AT & T facial database [16]. The
system is able to detect 90% of faces over 41 test images
resulting in few false positives. Section V provides conclusion
and future work followed by references.
II. RELATED WORK
The well-known datasets available to detect and recognize
faces and facial expressions [18] includes CVL [21], CMU
pose, illumination & expression (CMU - PIE) [25], FERET
[8], extended M2VTS dataset [17], Cohn-Kanade (CK) [24],
JAFFE dataset [20] and so on. This paper uses AT & T facial
database [16] and real time images to recognize faces based on
feature extraction using HOG [10].
Faces can be detected most generally using Viola-Jones
algorithm [9]. Facial features being extracted should be
invariant to pose variations. Such invariant features can be
obtained based on Active Shape Model (ASM) [4] and Active
Appearance Model (AAM) [12]. Features of an image include
edge details, texture, color information, corners, and interest
points of regions that constitute what is known as the Region
of Interest (ROI). The feature space can be reduced by
identifying dominant and unique feature using Principal
Component Analysis (PCA) [5] technique or by means of
distance metric like Euclidean [2]. Some of the well known
facial recognition algorithms use Gabor filters [30], LBP
(Local Binary Pattern) [30] and Linear Discriminant Analysis
(LDA) [30]. The raw pixel information does not discriminate
IEEE International Conference on Computer, Communication, and Signal Processing (ICCCSP-2017)

unique pattern of images that are prone to pose variations.
Hence, it is essential to extract information in larger blob areas
based on the application of interest. Most of the feature
extraction methods are based on the type of data. There are
many features namely SURF (Speeded-up Robust Features)
[26], FREAK (Fast Retina Keypoint) [27], BRISK (Binary
Robust Invariant Scalable Keypoints) [28], MSER (Maximally
Stable Extremal Regions) [29], SIFT (Scale-invariant Feature
Transformation) [30] and HOG (Histogram of Oriented
Gradients) [23].
The HOG descriptor can overcome the problem of varying
illumination as it is invariant to lighting conditions. It is used
to extract magnitude of edge information and works well even
during variations in poses and illumination conditions. HOG
works well under such challenging situations as it represents
directionality of edge information thereby making it
significant for the study of pattern and structure of the
interested object. The HOG feature descriptor can be used
along with other feature descriptors [10]. SURF features are
mainly used in facial recognition applications. FAST/SURF
and Harris/Freak [6] combination lessens illumination
distortion to a greater extent. The combination of HOG and
SVM [7] yields improved detection accuracy. Fig. 1. shows
general marking of facial landmarks such as eyes, mouth, and
nose using Harris corner points.
based on input data alone. Supervised learning performs
classification if the output is discrete, else regression if output
is continuous. Unsupervised learning deals with clustering as
the pattern is hidden in the data. Some of the algorithms
include Support Vector Machine (SVM) [11], Bayesian
learning [14], neural networks [13], Hidden Markov Model
(HMM) [7], K-Means [14] and so on. Choosing an appropriate
algorithm is somewhat trivial when data is not given
importance. It is also essential to avoid over-fitting and under
fitting problems while bringing out optimal decision boundary
[11].
III. SYSTEM ARCHITECTURE
The system is designed to handle pose and illumination
problems in recognition of faces. This paper focuses on
finding an exact match of the face from 40 subjects in AT & T
facial database in addition to user defined subject, based on
HOG features and matching. Fig. 2 shows the overall system
architecture which covers the entire workflow that includes
setting up of the database, feature extraction using HOG,
facial landmark detection, building up of classifier model and
feature matching.

Fig. 1. Face representation

Machine learning approach is adopted when a system is

trained to take decisions based on the data available to it. It
makes the machine perform tasks as how human perceive the
environment and perform actions. It relies upon the quantum
of data for learning. The results depend on the quality of data
and it is more important than an algorithm itself. The changes
in data convey behavior of the model. It helps in making Fig. 2. System architecture
predictions and it typically falls under two categories such as A. Database creation
supervised and unsupervised learning [11]. The former
learning method knows input and output while the latter works The AT & T database is used to recognize faces. It
contains 40 subjects with 10 images for each. All images in

978-1-5090-3716-2/17/$31.00 ©2017 IEEE

IEEE International Conference on Computer, Communication, and Signal Processing (ICCCSP-2017)
the database are in the same aspect ratio with differences in
poses and expressions with constant illumination. Fig. 3 shows
sample images of AT & T facial database. A new subject is
added with 10 different poses taken in varying lighting
conditions and different periods of time. The database is split
into a training set and testing set. Fig. 4 shows the images
present in the testing set.
Fig. 3. Sample images from AT & T facial database
Fig. 4. Test input
B. Pre-processing techniques
Facial alignment and normalization are essential in
achieving higher recognition rates as all feature vectors work
very well in same feature space. A set of 10 images of the
newly added self-defined images requires pre-processing to
match with images of the database. All faces of the newly
added subject are cropped using Viola-Jones algorithm [9]. It
uses Haar features and Adaboost [31] classifier to detect faces
in an integral image [9]. All images in the new image set
should be resized to match with the common size of
112X 92 pixels. Fig. 5 shows the set of newly added images in
the training set.
Fig. 5. Many appearances of same subject in different illumination and poses
978-1-5090-3716-2/17/$31.00 ©2017 IEEE
C. Feature extraction and classification
Both training and testing phase requires undergoing the
same type of feature extraction technique which in this case is
done by using HOG features.
Algorithm: Feature-based facial recognition
Input: Images of persons S = {1,2,...41} each with
S i = {1,2,...,10} where i = {1,2,....41} forming a total of
410 images
Output: Matched subject in the database based on training
features F and labels generated by means of HOG feature and
Harris feature matching of different facial components
Training set: T = I 1 , I 2 ,..., I 9 from 41 subjects forming a total
of 369 images
Testing set: Te = I 10 each from 41 subjects forming a total of
41 images. These images used to test the system are not
included in the training set.
1. Create database by accessing files from original AT &
T database in which original images in .pgm format
are converted to .jpg format to match with inclusion
of real-time images
2. Include 10 new user images by detecting faces using
Viola-Jones face detector
3. Partition the database into training and testing set
4. Extract facial features for all images in the training
dataset T
5. Train the dataset T using HOG feature descriptor
6. Choose a test image from the test dataset Te
7. Extract feature vector F using HOG feature
descriptor
8. Predict the class of test input using SVM classifier
based on matching data available in training features
and labels
9. Perform feature matching to display the
corresponding matching between a test image and
trained images.
The model built in the training phase is used to predict the
class of the subject by using SVM classifier. SVM can be
used most frequently as a binary classifier. However, it can
be used to classify multi-class data.
The labels and features are used to train a model which
can be used as an input to the SVM classifier for finding an
exact match of the given test data against relevant subjects
available in the database. SVM is trained to correct even
possible misclassification [7] when an unknown test input is
given for recognition.
IEEE International Conference on Computer, Communication, and Signal Processing (ICCCSP-2017)
IV. RESULTS AND DISCUSSION
The training set contains 369 images and testing set
contains 41 images. All images in the training set are trained
using HOG. Hence, a total of 369 feature vectors is obtained
with a dimensionality of 4680 for each image. The larger
dimension of feature vector accommodates magnitude of all
possible edge information in different orientations. Fig. 6
shows the HOG visualization of a new subject.
Fig. 6. HOG visualization
The test set contains 41 images and classification result is
given below:
• Matched correctly (37): s1 – s7, s9, s11 – s18, s20 –
s39, s41
• Not matched or wrong predictions: s8, s10, s19, s40
Fig. 7 shows the classifier result retrieving all matched
subjects from the database against the given test input.
Fig. 7. Classifier output showing matched results from the database
978-1-5090-3716-2/17/$31.00 ©2017 IEEE
The conventional method of evaluating a classification
model belonging to a supervised learning technique is to
formulate confusion matrix [22] over test data. Fig. 8 shows
the outcome of the classification results in the form of
providing them in confusion matrix.
Fig. 8. Confusion Matrix
¦ TruePositive + ¦ TrueNegative
Accuracy = (1)
¦ TotalPopulation
This shows an overall accuracy of 90.2439% computed
based on (1).
V. CONCLUSION AND FUTURE WORK
This paper focused on feature extraction using HOG by
exhibiting greater accuracy primarily in recognizing faces in
AT & T facial database. The increase in cross-validation
testing can evaluate the system in such a way that rejection of
false detections is improved to ensure better classification. As
the number of subjects gets increased, the method of
generating feature vectors using HOG would be time-
consuming. The future work involves the recognition of faces
in 3D images using the concept of deep learning where
convolution neural network [13] is used for training a dataset
as it captures high-level features providing good
representation.
REFERENCES
[1] Ferdinando Silvestro Samaria, "Face Recognition Using Hidden Markov
Models" in Thesis submitted in Trinity College
[2] Henry A. Rowley, Shumeet Baluja, Takeo Kanade, "Neural Network-
Based Face Detection" in IEEE Transactions on Pattern Analysis and
Machine Intelligence, vol. 20, no. 1, pp. 23-38, 1998
[3] Khan RA, Meyer A, Konik H, Bouakaz S., "Framework for reliable,
real-time facial expression recognition for low resolution images" in
Pattern Recognit Lett., pp. 1159–1168, 2013
[4] Leonardo A. Cament, Francisco J. Galdames, Kevin W. Bowyer,
Claudio A. Perez, "Face recognition under pose variation with Gabor
features enhanced by Active Shape and Statistical Models" in Pattern
Recognition, vol. 48, pp. 3371-3384, 2015
[5] M. Turk, A. Pentland, "Eigenfaces for Recognition" in Journal of
Cognitive Neurosicence, vol. 3, no. 1, pp. 71-86, Win. 1991
[6] Madbouly, M. Wafy, Mostafa-Sami M. Mostafa, "Performance
Assessment of Feature Detector-Descriptor Combination in Int. J. of
Computer Science, vol. 12, no. 5, pp. 87-94, 42248
 IEEE International Conference on Computer, Communication, and Signal Processing (ICCCSP-2017)

[7] Martin Schels and Friedhelm Schwenker, "A Multiple Classifier System
Approach for Facial Expressions in Image Sequences Utilizing GMM
Supervectors" in International Conference on Pattern Recognition
[8] P.J. Phillips, H. Moon, S.A. Rizvi, P.J. Rauss, "The FERET Evaluation
Methodology for Face-Recognition Algorithms" in IEEE Transactions
on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090-
1104, October 2000
[9] Paul Viola, Michael J. Jones, "Robust Real-Time Face Detection" in Int.
J. of Computer Vision, vol. 57, no. 2, pp. 137 - 154, 2004
[10] Pierluigi Carcagni, Marco Del Coco, Marco Leo, Cosimo Distante,
"Facial expression recognition and histograms of oriented gradients: a
comprehensive survey" in Springer Open Journal, pp. 1-25, 2015
[11] R. Chellappa, C.L. Wilson, S. Sirohey, "Human and Machine
Recognition of Faces: A Survey" in Proceedings of the IEEE, vol. 83,
no. 5, pp. 705-740, May 1995
[12] Shan Du, Rabab Ward, "Face recognition under pose variations" in
Journal of Franklin Institute, pp. 596-613, 2006
[13] Sun Y, Wang X, Tang X., "Deep learning face representation from
predicting 10,000 classes" in IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), pp. 1898-1898, 2014
[14] Y. Bengio, "Learning deep architectures for AI" in Foundations and
trends in machine learning, vol. 2, no. 1, pp. 1-127, 2009
[15] Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, Face recognition: A
literature survey in ACM Computing Survey, vol. 34, no. 4, pp. 399-
485, 2003
[16] http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html
[17] http://www.ee.surrey.ac.uk/CVSSP/xm2vtsdb/
[18] http://www.face-rec.org/databases/
[19] http://www.i-programmer.info/babbages-bag/1091-face-recognition.html
[20] http://www.kasrl.org/jaffe.html
[21] http://www.lrv.fri.uni-lj.si/database.html
[22] https://en.wikipedia.org/wiki/Sensitivity_and_specificity
[23] https://in.mathworks.com/help/vision/feature-detection-and-
extraction.html
[24] http://www.pitt.edu/~emotion/downloads.html
[25] http://www.ri.cmu.edu/research_project_detail.html?project_id=418&m
enu_id=261
[26] https://www.cse.unr.edu/~bebis/CS491Y/Papers/Bay06.pdf
[27] https://infoscience.epfl.ch/record/175537/files/2069.pdf
[28] https://e-collection.library.ethz.ch/eserv.php?pid=eth:7684&dsID=eth-
7684-01.pdf
[29] http://in.mathworks.com/help/vision/ref/detectmserfeatures.html
[30] http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf
[31] Dwarakesh T P, S Ananda Subramaniam, T Sree Sharmila, Vacant Seat
Detection System using Adaboost and Camshift in International Journal
of Electrical and Computing Engineering, vol. 1, no. 3, pp. 41518, April
2015

978-1-5090-3716-2/17/$31.00 ©2017 IEEE

View publication stats