Professional Documents
Culture Documents
Major Project PDF
Major Project PDF
A DISSERTATION
Submitted in partial fulfillment of the requirement for the award of the degree of Bachelor of
Engineering
[2019-2020]
Sagar Institute of Research & Technology, Bhopal
Department of Electronics & Communication Engineering
CERTIFICATE
This is to certify that the work embodies in this dissertation entitled“ Extracting
Face from Live Feed ”being Submitted by Shanu Prakash Enrolment no.-
0133EC161141 in partial fulfillment of the requirement for the award of
“Bachelor of Engineering in Electronics & Communication Engineering” to RAJIV
GANDHI PRODYOGIKI VISHWAVIDYALAYA,BHOPAL(M.P.) during the academic
year 2019-2020is a record of Bonafede piece of work, carried out by him under
my supervision and guidance in the Department of Electronics & Communication
Engineering ,Sagar Institute of Research & Technology, Bhopal
Bhopal.
DECLARATION
work presented in this dissertation entitled “Extracting Faces from Live Feed” is
knowledge and this work has been carried out taking care of Engineering Ethics.
The work presented does not infringe any patented work and has not been
submitted to any other University or anywhere else for the award of any degree
I would like to sincerely thanks Dr. Jyoti Jain, HOD. Electronics & Communication
Department for her great support in doing this project.
I am also grateful to Prof. Mukesh Yadav for giving me the necessary facilities to
carry out my project work successfully.
I would like to thanks all my friends for their help and constructive criticism during
my project work. Finally, I have no words to express my sincere gratitude to my
parents who have been shown me this world and for every support they have
given us
Shanu Prakash
0133EC161141
CONTENTS
Face recognition systems are part of facial image processing applications and
it is one of the challenging topics in IT research. Biometric information of
humans are used instead of fingerprint, iris, signature etc., because these
types of biometrics are not much suitable for non-collaborative people. Face
recognition systems are usually used and preferred for security cameras in
metropolitan life. These systems can be used for crime prevention, video
surveillance, person verification, and similar security activities.
Face detection is the first step of face recognition system. Output of the
detection is location of a face region with facial features (i.e. eyes, mouth,
eyebrow, nose etc.) [1]. Detection methods in the literature are classified into
two groups as Knowledge-Based Methods and Image-Based Methods.
Facial recognition system are commonly used for security purposes but are
increasingly being used in a variety of other applications. For example
Physiological Biometrics:-
Based on measurements and data derived from direct the human body
Behavioral Biometrics:-
1.Voice scan
2.Signature scan
TensorFlow is a platform which can be used to train your own model and
the trained mobel can be used to detect any kind of data such as Pen, TV,
Computer, Laptop, Bottle etc. depending upon the model made by you
Many of the face or Object Detection models use Python Environment to run.
FACE DETECTION ALGORITHM
Face detection performs locating and extracting face image operations for
face recognition system. Face detection algorithm of the proposed system.
Our experiments reveal that skin segmentation, as a first step for face
detection, reduces computational time for searching whole image. While
segmentation is applied, only segmented region is searched whether the
segment includes any face or not. After the acquired image is segmented, it
continued with the face candidate search and facial feature extraction to
validate the candidate. To perform the facial feature extraction four different
filtering approaches are performed for useful enhancements on facial
features. Finally, the validated face is extracted based on the facial features
to send to the face recognition algorithm.
Knowledge-Based
Feature Based
Template Match
FACIAL RECOGNITION
The face is an important part of who you are and how people identify you.
Verification is where the system compares the given individual with who that
individual says they are and gives a yes or no decision.
Identification is where the system compares the given individual to all the other
individuals in the database and gives a ranked list of matches.
2. Extracted Faces: Extract faces from all images in a folder and save each
face into a destination folder to create a handy dataset.
The Python file is face_extractor.py
Module Used:
Computer Vision (OpenCV)
Time Module
NumPy
OS Module
PYTHON
Compared to other languages like C/C++, Python is slower. But another important
feature of Python is that it can be easily extended with C/C++. This feature helps
us to write computationally intensive codes in C/C++ and create a Python wrapper
for it so that we can use these wrappers as Python modules.
Python is considered a scripting language, like Ruby or Perl and is often used for
creating Web applications and dynamic Web content. It is also supported by a
number of 2D and 3D imaging programs, enabling users to create custom plug-
ins and extensions with Python. Examples of applications that support a
Python API include GIMP, Inkscape, Blender, and Autodesk Maya.
Scripts written in Python (.PY files) can be parsed and run immediately. They can
also be saved as a compiled programs (.PYC files), which are often used as
programming modules that can be referenced by other Python programs.
The Python 2 language, i.e. Python 2.7.x, was officially discontinued on 1 January
2020 (first planned for 2015) after which security patches and other
improvements will not be released for it. With Python 2's end-of-life, only
Python 3.5.x and later are supported.
There are many different Python Programming IDE available for different
Operating Systems.
Python IDLE
PyCharm
Anaconda
Spyder IDE
Jupyter NoteBook
Python IDLE:
Coded in 100% pure Pyth on, using the tkinter GUI toolkit
Cross-platform: works mostly the same on Windows, Unix, and
macOS
Python shell window (interactive interpreter) with colorizing of
code input, output, and error messages
Multi-window text editor with multiple undo, Python
colorizing, smart indent, call tips, auto completion, and other
features
Search within any window, replace within editor windows, and
search through multiple files (grep)
Debugger with persistent breakpoints, stepping, and viewing
of global and local namespaces
Configuration, browsers, and other dialogs
PyCharm:-
Features
Coding assistance and analysis, with code completion, syntax
and error highlighting, linter integration, and quick fixes
Project and code navigation: specialized project views, file
structure views and quick jumping between files, classes,
methods and usages
Python refactoring: includes rename, extract method,
introduce variable, introduce constant, pull up, push down and
others
Integrated Python debugger
Integrated unit testing, with line-by-line code coverage
Google App Engine Python development [professional edition
only]
Version control integration: unified user interface
for Mercurial, Git, Subversion, Perforce and CVS with change
lists and merge
Support for scientific tools like matplotlib, numpy and scipy
[professional edition only]
Anaconda:-
Anaconda bundles a whole bunch of Python packages that are commonly used by
people using Python for scientific computing and/or data science. It provides a
single download and an install program/script that installs all the packages in one
go. Alternatively, one can install Python first and then individually install all the
required packages using pip. Additionally, it provides its own package manager
(conda) and package repository. But it allows installation of packages from PyPI
using pip if the package is not in Anaconda repositories. It is especially good if you
are installing on Microsoft Windows as it can easily install packages that would
otherwise require you to install C/C++ compilers and libraries if you were using
pip. It is certainly an added advantage that conda, in addition to being a package
manager, is also a virtual environment manager allowing you to install
independent development environments and switch from one to the other
(similar to virtualenv).
Initially created and developed by Pierre Raybaut in 2009, since 2012 Spyder has
been maintained and continuously improved by a team of scientific Python
developers and the community.
Spyder is extensible with first- and third-party plugins, includes support for
interactive tools for data inspection and embeds Python-specific code quality
assurance and introspection instruments, such as Pyflakes, Pylint and Rope. It is
available cross-platform through Anaconda, on Windows, on macOS
through MacPorts, and on major Linux distributions such as Arch Linux, Debian,
Fedora, Gentoo Linux, openSUSE and Ubuntu.
Features
Features include:
Plugins
Jupyter NoteBook:-
IPython
Tornado (web server)
jQuery
Bootstrap (front-end framework)
MathJax
Google Chrome
FireFox
Microsoft Edge
Safari Web Browser (MacOS)
Computer Vision
Computer vision tasks include methods for acquiring, processing, analyzing and
understanding digital images, and extraction of high-dimensional data from the
real world in order to produce numerical or symbolic information, e.g. in the
forms of decisions. Understanding in this context means the transformation of
visual images (the input of the retina) into descriptions of the world that make
sense to thought processes and can elicit appropriate action. This image
understanding can be seen as the disentangling of symbolic information from
image data using models constructed with the aid of geometry, physics, statistics,
and learning theory.
The scientific discipline of computer vision is concerned with the theory behind
artificial systems that extract information from images. The image data can take
many forms, such as video sequences, views from multiple cameras, multi-
dimensional data from a 3D scanner or medical scanning device. The
technological discipline of computer vision seeks to apply its theories and models
to the construction of computer vision systems.
Computer vision is an interdisciplinary field that deals with how computers can be
made to gain high-level understanding from digital images or videos. From the
perspective of engineering, it seeks to automate tasks that the human visual
system can do. "Computer vision is concerned with the automatic extraction,
analysis and understanding of useful information from a single image or a
sequence of images. It involves the development of a theoretical and algorithmic
basis to achieve automatic visual understanding." As a scientific discipline,
computer vision is concerned with the theory behind artificial systems that
extract information from images. The image data can take many forms, such as
video sequences, views from multiple cameras, or multi-dimensional data from
a medical scanner. As a technological discipline, computer vision seeks to apply its
theories and models for the construction of computer vision systems.
Artificial intelligence
Artificial intelligence and computer vision share other topics such as pattern
recognition and learning techniques. Consequently, computer vision is sometimes
seen as a part of the artificial intelligence field or the computer science field in
general.
Information Engineering
Solid-state physics
Solid-state physics is another field that is closely related to computer vision. Most
computer vision systems rely on image sensors, which detect electromagnetic
radiation, which is typically in the form of either visible or infra-red light. The
sensors are designed using quantum physics. The process by which light interacts
with surfaces is explained using physics. Physics explains the behavior
of optics which are a core part of most imaging systems. Sophisticated image
sensors even require quantum mechanics to provide a complete understanding of
the image formation process.[11] Also, various measurement problems in physics
can be addressed using computer vision, for example motion in fluids.
Neurobiology
A third field which plays an important role is neurobiology, specifically the study
of the biological vision system. Over the last century, there has been an extensive
study of eyes, neurons, and the brain structures devoted to processing of visual
stimuli in both humans and various animals. This has led to a coarse, yet
complicated, description of how "real" vision systems operate in order to solve
certain vision-related tasks. These results have led to a subfield within computer
vision where artificial systems are designed to mimic the processing and behavior
of biological systems, at different levels of complexity. Also, some of the learning-
based methods developed within computer vision (e.g. neural net and deep
learning based image and feature analysis and classification) have their
background in biology.
Some strands of computer vision research are closely related to the study
of biological vision – indeed, just as many strands of AI research are closely tied
with research into human consciousness, and the use of stored knowledge to
interpret, integrate and utilize visual information. The field of biological vision
studies and models the physiological processes behind visual perception in
humans and other animals. Computer vision, on the other hand, studies and
describes the processes implemented in software and hardware behind artificial
vision systems. Interdisciplinary exchange between biological and computer vision
has proven fruitful for both fields.[19]
Signal Processing
Yet another field related to computer vision is signal processing. Many methods
for processing of one-variable signals, typically temporal signals, can be extended
in a natural way to processing of two-variable signals or multi-variable signals in
computer vision. However, because of the specific nature of images there are
many methods developed within computer vision which have no counterpart in
processing of one-variable signals. Together with the multi-dimensionality of the
signal, this defines a subfield in signal processing as a part of computer vision.
OpenCV
OpenCV was started at Intel in 1999 by Gary Bradsky and the first release came
out in 2000. Vadim Pisarevsky joined Gary Bradsky to manage Intel’s Russian
software OpenCV team. In 2005, OpenCV was used on Stanley, the vehicle who
won 2005 DARPA Grand Challenge. Later its active development continued under
the support of Willow Garage, with Gary Bradsky and Vadim Pisarevsky leading the
project. Right now, OpenCV supports a lot of algorithms related to Computer
Vision and Machine Learning and it is expanding day-by-day.
OpenCV has a modular structure, which means that the package includes several
shared or static libraries. The following modules are available:
To apply face detection, which detects the presence and location of a face in an
image, but does not identify it
To extract the 128-d feature vectors (called “embeddings”) that quantify each
face in an image.
The model responsible for actually quantifying each face in an image is from
the Open Face project, a Python and Torch implementation of face recognition
with deep learning.
First, we input an image or video frame to our face recognition pipeline. Given the
input image, we apply face detection to detect the location of a face in the image.
After we’ve (optionally) applied face alignment and cropping, we pass the input
face through our deep neural network:
Figure 1: An overview of the OpenCV face recognition pipeline. The key step is a CNN feature
extractor that generates 128-d facial embeddings.
The model responsible for actually quantifying each face in an image is from
the Open Face project, a Python and Torch implementation of face recognition
with deep learning.
First, we input an image or video frame to our face recognition pipeline. Given the
input image, we apply face detection to detect the location of a face in the image.
After we’ve (optionally) applied face alignment and cropping, we pass the input
face through our deep neural network:
The Face Net deep learning model computes a 128-d embedding that quantifies
the face itself.
But how does the network actually compute the face embedding?
To train a face recognition model with deep learning, each input batch of data
includes three images:
The anchor
The positive image
The negative image
Figure 2: How the deep learning face recognition model computes the face embedding.
The anchor is our current face and has identity A.
The second image is our positive image — this image also contains a face of
person A.
The negative image, on the other hand, does not have the same identity, and
could belong to person B, C, or even Y!
The point is that the anchor and positive image both belong to the same
person/face while the negative image does not contain the same face.
The neural network computes the 128-d embeddings for each face and then
tweaks the weights of the network (via the triplet loss function) such that:
The 128-d embeddings of the anchor and positive image lie closer together
While at the same time, pushing the embeddings for the negative image
father away
In this manner, the network is able to learn to quantify faces and return highly
robust and discriminating embeddings suitable for face recognition.
And furthermore, we can actually reuse the Open Face model for our own
applications without having to explicitly train it!
Even though the deep learning model we’re using today has (very
likely) never seen the faces we’re about to pass through it, the model will still be
able to compute embeddings for each face — ideally, these face embeddings will
be sufficiently different such that we can train a “standard” machine learning
classifier (SVM, SGD classifier, Random Forest, etc.) on top of the face
embeddings, and therefore obtain our OpenCV face recognition pipeline.
.
Figure 3: A small example face dataset for face recognition with OpenCV
White Balance Correction
Calculate average value of red channel (Rav), green channel (Gav), and
blue channel (Bav)
Calculate average gray, Grayav=(Rav+Gav+Bav)/3
Then, KR=Grayav/Rav, KG=Grayav/Gav, and KB=Grayav/Bav
Generate new image (New) from original image (Orji) with RGB
channel by New(R)=KR*Orj(R), New(G)=KG*Orj(G), and
New(B)=KB*Orj(B)
r”, “g”, and “b” parameters are red, green and blue channel values of pixel. If
these seven conditions are satisfied, then pixel is said to be skin color and
binary image is created from satisfied pixels.
Face Candidate Search
If the image is not balanced, then some part of the wall will be taken as skin
color as in Figure 3 due to poor lighting.
Based on these conditions, face candidates are extracted from the input
image with modified bounding box. The height of bounding box modified as
1.28 times bigger than width of bounding box because width of face
candidate does not change if candidate includes chest/neck or not. After this
modification, new bounding box covers only face. These face candidates will
be sent to facial feature extraction part to validate the candidates.
Filtering for Facial Feature Extraction
Edge detection methods are nearly insensitive to light change. Sobel edge
detector is used to extract features. Figure 6 shows results of edge detection
on test image 3. Edge detection is not sensitive to light condition as Black-
White conversion. On both images, eyes and mouth can be selected with
human eyes but mouth can be difficult to extract on the images and eye parts
also vary on shapes. Also, edge detection has high responses.
In order to use edge detection, Laplacian of Gaussian (LoG) filter can be used.
LoG filter has low responses than edge detection. It makes useful
enhancements on facial features extraction. Figure 6 shows results of LoG
filter on test image 3.
LoG filter has better performance than the previous three trials. Mouth is
more significant than others and eyes can be selected more accurately.
Facial feature is one of the most significant features of face. Facial features
are eyebrows, eyes, mouth, nose, nose tip, cheek, etc. If some of these
features are found in the candidate image, then the candidate image will be
considered as face. Two eyes and mouth generate isosceles triangle, and
distances between eye to eye and mid-point of eyes distance to mouth are
equal [6]. Some filtering operations are applied to extract feature candidates
and steps are listed below:
Right eye and left eye are determined by criteria which are illustrated by
yellow and green color respectively. Left and right eye are mostly found
correctly but sometimes bottom eyelid is found falsely. If left and right eyes
are detected, then mouth finding application can be applied.
Each label inside down region chooses as mouth candidate and candidate
property vector is calculated. Euclidian distance of right eye to mouth
candidate (right-distance) and Euclidian distance of left eye to mouth
candidate (left-distance) are calculated. Also, Euclidian distance between two
eyes (eye-distance) and Euclidian distance between mid-point of eyes to
mouth candidate (center-distance) are calculated. Then, with respect to the
property vector, the mouth label is found which satisfies isosceles triangle
with the eyes. Required facial features are found which are right eye, left eye
and mouth. Face image can be extracted which covers two eyes and mouth.
Face covering is created with a rectangle in which covers eyes and mouth.
Facial feature extracted image from face candidate and face image.
This proposed algorithm is implemented using MATLAB and tested for more
than hundred images. This algorithm detects not only one face but also more
than one face. Small amount of oriented face are acceptable. Results are
satisfactory for all purpose.
PROJECT STRUCTURE
The Project Aims to develop a system through which whenever a person comes
inside the frame of a CCTV Camera installed across the globe the system perform
a face detection algorithm and extract the face and saves it in a Database which
can be used to perform many different operations in future.
Police Verification – Police Department can use the data for predicting the
criminal by matching the footage from the crime scene with the database.
Location Tracking – Location of an individual can also be tracked using this
as the output data contains the time and location of the face that it extract.
Python 3.7
Os Library
NumPy
OpenCV
Time
DateTime
Hardware Used:
Note
#- This are optional which can be used but not used in this project.
Project
Import libraries
I import os to access various files in the directory. Then, cv2 will be used to work
with images. numpy helps to easily work with multi-dimensional arrays.
Camera Access
This line of code is used to access the camera for detecting the face.
The model’s prototxt and caffemodel is provided in the OpenCV repo itself. I used
the same and placed them in the model_data directory in my project. prototxt file
includes the text description of the network and caffemodel includes the weights.
I read the two files and loaded my model using cv2.
Create directory
If the directory where the resultant images will get stored does not exist, I’ll
create the directory. The output folder is faces.
If the directory where the resultant images will get stored does not exist, I’ll
create the directory. The output folder is updated_images.
Read images
I loop through all images inside the images folder. After extracting the extension, I
check that the files are either of the type .png or .jpg and just operate with those
files only.
One by one, I iterate over all of the faces detected in the image and extract their
start and end points. Then, I extract the confidence of detection. If the algorithm
is more than 50% confident that the detection is a face, I show a rectangle around
it. And the rectangle part which detect the face is cutout and saved in the
directory created as faces
Printing the output In the consol
The output file details which is saved in the directory is shown in the consol
The Entire Code for the Project is as under.
Software Download Links:
Python 3.7
https://www.python.org/downloads/
Anaconda Enviroment
https://www.anaconda.com/distribution/
CONCLUSION
Face recognition systems are part of facial image processing applications
and their significance as a research area are increasing recently.
Implementations of system are crime prevention, video surveillance,
person verification, and similar security activities. The face recognition
system implementation will be part of humanoid robot project at Atılım
University.
Knowledge-Based face detection methods are used to find, locate and
extract faces in acquired images. Implemented methods are skin color and
facial features.
RGB color space is used to specify skin color values, and segmentation
decreases searching time of face images. Facial components on face
candidates are appeared with implementation of LoG filter. LoG filter
shows good performance on extracting facial components under different
illumination conditions. Proposed algorithm is capable of detecting
multiple faces, and performance of system is acceptable.
Proposed system can be affected by pose, presence or absence of
structural components, facial expression, imaging condition, and strong
illumination.
REFERENCES
https://en.wikipedia.org/wiki/Python_(programming_language)
https://en.wikipedia.org/wiki/IDLE
https://en.wikipedia.org/wiki/PyCharm
https://en.wikipedia.org/wiki/Spyder_(software)
https://en.wikipedia.org/wiki/Project_Jupyter
https://en.wikipedia.org/wiki/Computer_vision
https://en.wikipedia.org/wiki/OpenCV
https://docs.opencv.org/master/
https://github.com/kb22/Create-Face-Data-from-Images
https://docs.python.org/3/library/idle.html
https://towardsdatascience.com/extracting-faces-using-opencv-face-detection-neural-network-
475c5cd0c260
https://www.google.com/webhp?hl=en&ictx=2&sa=X&ved=0ahUKEwjouqugm-
XoAhU9H7cAHd_1AcYQPQgH