VISVESVARAYATECHNOLOGICALUNIVERSITY

“JNANASANGAMA”, BELAGAVI-590018.

A Project Synopsis
On
“Vehicle Security System Using Facial Recognition”
Submitted to the Requirement of 8th semester project of
BACHELOR OF ENGINEERING
IN
COMPUTER SCIENCE AND ENGINEERING

Submitted by

SIRISHA.K [1JB17CS152]
VASAVI.M[1JB17CS172]

UNDER THE GUIDANCE OF

PAVITHRA.G.S
Assistant Professor
Dept of CSE

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

SJB INSTITUTE OF TECHNOLOGY

BGS HEALTH AND EDUCATION CITY
Dr. Vishnuvardhan Road,Kengeri, Bangalore-560060.
2020-21
 TABLE OF CONTENTS

Contents

INTRODUCTION ........................................................................................................................... 1

LITERATURE SURVEY ................................................................................................................ 2

MACHINE LEARNING ...............................................................................................................................2
ARTIFICIAL NEURAL NETWORKS(ANN)..............................................................................................2
PERCEPTRONS............................................................................................................................................2
FACIAL RECOGNITION .............................................................................................................................3
ANN FOR FACIAL RECOGNITION ..........................................................................................................4

PROBLEM STATEMENT .............................................................................................................. 7

OBJECTIVES .................................................................................................................................. 8

METHODOLOGY .......................................................................................................................... 9
DESIGN .........................................................................................................................................................9
Data Pre-processing .......................................................................................................................................9
Dataset Gathering ..........................................................................................................................................9
Data cleansing ..............................................................................................................................................10
Data augmentation .......................................................................................................................................10
Face recognition overview ...........................................................................................................................10
Environment ................................................................................................................................................10

REFERENCES .............................................................................................................................. 12
 Facial Recognition in Automobile Security

INTRODUCTION

This project is based on the idea of keeping our automobiles safe and secure. There are many ways to
avoid losing our automobiles to the hands of others. One way is to implement the facial recognition.

When the camera is mounted on the door of the driver’s seat and inside the car on the dashboard, the
camera must be able to recognize the faces it has stored. The car usage will be limited to it’s two
permanent owners and their faces will be captured on the camera and stored in the cloud. When the car
is empty it will be locked by default and when the owner of the car faces the camera on the door of the
driver’s seat, the doors automatically gets unlocked. In order to do a multifactor authentication apart from
facial recognition the user will be challenged with the passphrase on the dashboard of the car for the user
to start the engine. The passphrase consists of a set of numeric (6-12) characters. The location of the car
is also monitored and validated during the start of the engine. The location is validated for a specific
region which is preset by the user in the system.

These features will need systems which are based on the concepts of Artificial Intelligence and the related
applications. One such application is Machine Learning using Convolution Neural Networks (CNN). It
is a part of the neural networks. In deep learning, a convolutional neural network (CNN, or ConvNet) is
a class of deep neural networks, most commonly applied to analyzing visual imagery.

Dept. of CSE, SJBIT 2020-2021 Page 1

 Facial Recognition in Automobile Security

LITERATURE SURVEY

MACHINE LEARNING

This project is about working on the security of the vehicle by facial recognition. The system for facial
recognition is based on the concepts of Machine Learning (ML) which is a branch under Artificial
Intelligence (AI). Artificial Intelligence is the study of how to make computers do things which, at the
moment, people do better. Among the tasks that AI can perform are mundane tasks out of which vision
is one of them. ML is about the systems which are capable of learning based on experience gained without
being explicitly programmed. ML draws on concepts and results from many fields including statistics,
AI, Information Theory and Cognitive Science. Facial Recognition needs ML algorithms like Artificial
Neural Networks (ANN).

ARTIFICIAL NEURAL NETWORKS(ANN)

ANN provide a general, practical method for learning real-valued, discrete-valued and vector-valued
functions from examples. Algorithms such as back propagation use gradient descent to tune network
parameters to best fit a training set of input-output pairs. ANN learning is robust to errors in the training
data and has been successfully applied to problems such as interpreting visual scenes, speech recognition
and learning robot control strategies.

PERCEPTRONS

ANN is analogous to neural networks utilized for perception within brain functions. Biological neural
networks are composed of tiny units called the neurons. In a similar way ANN is composed of artificial
neurons called the perceptron. A perceptron is a neural network unit that does certain computations to
detect features in the input data. The perceptron works by taking in several inputs and producing one
output.

Dept. of CSE, SJBIT 2020-2021 Page 2

 Facial Recognition in Automobile Security

In the example shown the perceptron has three inputs, x1, x2, x3. In general it could have more or fewer
inputs. Rosenblatt proposed a simple rule to compute the output. He introduced weights, w1,w2,… real
numbers expressing the importance of the respective inputs to the output. The neuron's output, 00 or 11,
is determined by whether the weighted sum ∑jwjxj is less than or greater than some threshold value. Just like
the weights, the threshold is a real number which is a parameter of the neuron. In a simple way, perceptron is
the device that makes decisions by weighing up evidences.

FACIAL RECOGNITION

Face recognition is a visual pattern recognition problem. In detail, a face recognition system with the

input of an arbitrary image will search in database to output people’s identification in the input image. A
face recognition system generally consists of four modules as depicted in Figure 1: detection, alignment,
feature extraction, and matching, where localization and normalization (face detection and alignment) are
processing steps before face recognition (facial feature extraction and matching) is performed.
Face detection segments the face areas from the background. In the case of video, the detected faces may
need to be tracked using a face tracking component. Face alignment aims at achieving more accurate
localization and at normalizing faces thereby, whereas face detection provides coarse estimates of the
location and scale of each detected face. Facial components, such as eyes, nose, and mouth and facial
outline, are located; based on the location points, the input face image is normalized with respect to
geometrical properties, such as size and pose, using geometrical transforms or morphing. The face is
usually further normalized with respect to photometrical properties such illumination and gray scale.
After a face is normalized geometrically and photometrically, feature extraction is performed to provide
effective information that is useful for distinguishing between faces of different persons and stable with
respect to the geometrical and photometrical variations. For face matching, the extracted feature vector
of the input face is matched against those of enrolled faces in the database; it outputs the identity of the
face when a match is found with sufficient confidence or indicates an unknown face otherwise.

Dept. of CSE, SJBIT 2020-2021 Page 3

 Facial Recognition in Automobile Security

For face detection module, a three-layer feedforward artificial neural network with Tanh activation
function is proposed that combines AdaBoost to detect human faces so that face detecting rate is rather
high. For face alignment module, a multilayer perceptron (MLP) with linear function (three-layer) is
proposed, and it creates 2D local texture model for the active shape model (ASM) local searching.
For feature extraction module, a method for combination of geometric feature-based method and ICA
method in facial feature extraction is proposed. For face matching, a model which combines many
artificial neural networks applied for geometric features classification is proposed. This case study
demonstrates how to solve face recognition in the neural network paradigm.

ANN FOR FACIAL RECOGNITION

The face detection processing is the first step of the face recognition system. The step will decide the
performance of the system, so it is the most important step of the recognition system. To carry out its
efficiently, many researchers have proposed different approaches. In general, there are four groups of
face detecting methods: (1) Knowledge-based methods; (2) Invariant feature-based methods;
(3) Template matching-based methods; (4) Machine learning-based methods.

One of the popular methods having the same achievement as well is artificial neural networks (ANNs).
ANN is the term on the method to solve problems by simulating neuron’s activities. In detail, ANNs can
be most adequately characterized as “computational models” with particular properties such as the ability
to adapt or learn, to generalize, or to cluster or organize data, and which operation is based on parallel
processing. However, many of the previously mentioned properties can be attributed to nonneural models.

Dept. of CSE, SJBIT 2020-2021 Page 4

 Facial Recognition in Automobile Security

Dept. of CSE, SJBIT 2020-2021 Page 5

 Facial Recognition in Automobile Security

PROBLEM STATEMENT

Vehicle thefts have been increasing and obviously, using reliable safety measure like Face Recognition
would keep car owners at peace even when they’re not around their car. Face recognition could provide
that extra layer of safety and help reduce thefts, besides that, it also brings more convenience personalized
in-cabin experience, like ignition, child protection etc.
 Facial Recognition in Automobile Security

OBJECTIVES

The Main objectives of the project are:

1 Thorough literature study

2 Understanding of the technology stack
3 Face dataset collection
4 Dataset preparation (labeling, cleaning, augmentation, normalization)
5 Model to recognize faces and subsequently identify the ids/owners
6 Validating the model on wide range of images
7 Model deployment
8 Live Demo
 Facial Recognition in Automobile Security

METHODOLOGY

DESIGN

Automobile Web Server

Auth Client Auth Module

Id, passphrase, loc
FR Unit
Location Module

Auth Client: Captures the location and passphrase data as input. The Auth-client also connects to the
internet. It can also detect the location of the device based on the internet connectivity or GPS
Available on the device of the automobile. It sends the data to the web-server deployed on a separate server
FR Unit: The unit captures the image of the face via a built-in camera.
Auth Module: Receives the passphrase from client and authenticates the user or the driver.
Location Module: authenticates based on the location of the user

Data Pre-processing

Data Gathering Data Cleansing Data

Data Pre-processing
Data

training code model

Model
Building Deployme
nt
production
labeled data web
app
codeLearning Workflow
Fig - 1: Typical Machine

Dataset Gathering
Before we can train our model to recognize faces in images, we first need to gather the dataset of faces
itself. If you are already using a pre-curated dataset, such as Labeled Faces in the Wild (LFW), then data
gathering step can be skipped.
But for most of us, we’ll instead want to recognize faces that are not part of any current dataset and
recognize faces of ourselves, family members, etc.
 Facial Recognition in Automobile Security

Data cleansing
It might require to remove the background color from the images to reduce the noise. Other might require
that to brighten or darken the images.
In short, any adjustments that need to apply to the dataset are considered a sort of this step but it can be
avoided if we use open datasets

Data augmentation
Scaling, rotations and other affine transformations are typical in this step. This is done to enlarge the
dataset and expose the neural network to a wide variety of variations of your images. This makes it more
likely that your model recognizes objects when they appear in any form and shape.
The pixel values in images must be scaled prior to providing the images as input to a deep learning neural
network model during the training or evaluation of the model. Traditionally, the images would have to be
scaled prior to the development of the model and stored in memory or on disk in the scaled format.
Face recognition overview
A Convolutional Neural Network (CNN) is a powerful machine learning technique from the field of deep
learning. CNNs are trained using large collections of diverse images which are behind the use cases like
face recognition or object detection. Face recognition or object detection is often learned as the Supervised
Learning problem.

There are 3 distinct phases to solve this problem:

Face Detection
Train the recognizer
Face Recognition

We use OpenCV deep learning-based face detector library to perform face recognition task. The most
common way to detect a face (or any objects), is using the “Haar Cascade classifier” or a complex “Single
Shot Detector” can also be used to detect variety of objects in a given image.

We can train recognizer using OpenCV package LBPH (LOCAL BINARY PATTERNS
HISTOGRAMS) by passing the faces images with ids as the parameters. Once the training is finished its
ready to be served for predictions.

We take any unseen or seen image and can be passed through the model for prediction which returns the
confidence score with the match.
Environment
• Ubuntu 64-bit Linux distribution
• OpenCV
 Facial Recognition in Automobile Security

• Anaconda for Python

• Python Image Library (PIL)
• VSCode or PyCharm
• Flask or Django or RasberryPi web frameworks
 Facial Recognition in Automobile Security

REFERENCES
1. Andrew Ng. Convolutional Neural Networks specialization from Deeplearning.ai
2. Adit Deshpande. A Beginner's Guide To Understanding Convolutional Neural Networks
3. Stanford Edu. CS231n: Convolutional Neural Networks for Visual Recognition
4. Koch, G. et al.: Siamese Neural Networks for One-shot Image Recognition. Presented at the
(2015).
5. Liu et al.: SSD: Single Shot MultiBox Detector ECCV (2016).