INTELLIGENT RECOGNITION WITH REAL-TIME VIDEO

DETECTION AND DOOR VERIFICATION

Thesis submitted to the SASTRA Deemed to be University

In partial fulfilment of the requirements
for the award of the degree of

B. Tech Electronics & Communication Engineering

Submitted by

Madesh Balu (Reg. No.: 124004171)

Varun A (Reg. No.: 124004347)
Karunaesh RA (Reg. No.: 124004140)

MAY 2024

SCHOOL OF ELECTRICAL & ELECTRONICS ENGINEERING

THANJAVUR, TAMIL NADU, INDIA – 613 401
 SCHOOL OF ELECTRICAL & ELECTRONICS ENGINEERING
THANJAVUR – 613 401

Bonafide Certificate

This is to certify that the thesis titled “Intelligent Recognition using Real-

Time Video Detection and Door Verification” submitted in partial fulfilment of the

requirements for the award of the degree of B.Tech Electronics & Communication

Engineering to the SASTRA Deemed to be University, is a Bonafide record of the work

done by Mr. Madesh Balu (Reg. No.124004171), Mr. Varun A (Reg. No.124004347)

& Mr. Karunaesh RA (Reg. No.124004140) during the eighth semester of the

academic year 2023-24 academic, in the School of Electrical & Electronics Engineering,

under my supervision. This thesis has not formed the basis for the award of any degree,

diploma, associateship, fellowship or other similar title to any candidate of any University

Signature of Project Supervisor :

Name with Affiliation : Dr. Rakesh Kumar Karn ( Associate Professor )
Date : 06-05-2024

Project Viva voce held on

Examiner 1 Examiner 2
ii
 SCHOOL OF ELECTRICAL & ELECTRONICS ENGINEERING
THANJAVUR – 613 401

Declaration

We declare that the thesis titled “Intelligent Recognition with Real-time Video
Detection and Door Verification” submitted by us is an original work done by us
under the guidance of Dr. Rakesh Kumar Karn, Associate Professor, School of
Electrical and Electronics Engineering, SASTRA Deemed to be University
during the eighth semester of the academic year 2023-24, in the School of Electrical
and Electronics Engineering. The work is original and wherever, we have used
materials from other sources, we have given due credit and cited them in the text of
the thesis. This thesis has not formed the basis for the award of any degree, diploma,
associate-ship, fellowship or other similar title to any candidate of any University.

Signature of the candidates:

Name of the candidates: Madesh Balu

Varun A
Karunaesh RA

Date: 06-05-2024

iii
 ACKNOWLEDGEMENTS

First and foremost, we are thankful to God Almighty for giving us the potential and
strength to work on this project fruitfully. I express our gratitude to Prof. Dr. S.
VAIDHYASUBRAMANIAM, Vice-Chancellor, SASTRA Deemed to be
University for providing us with an encouraging platform for the course of study.
We also thank Dr. R. CHANDRAMOULI, Registrar, SASTRA Deemed to be
University for providing us the opportunity to work in the SEEE, SASTRA Deemed
to be University for the course of the work. We render our solicit thanks to Dr. K.
THENMOZHI, Dean, SEEE, SASTRA Deemed to be University for introducing
this Project to our curriculum.

It is our privilege to express our sincerest regards to our project coordinator, Dr.
BALASUBRAMANIAN.M (SAP/EEE/SEEE), who motivated us during the
project. We owe a debt of earnest gratitude towards our guide Dr. Rakesh Kumar
Karn and Dr. Raju. N for their continued support and guidance throughout the
duration of our project on the topic “Intelligent Recognition with Real-Time Video
Detection and Door Verification” The intriguing questions raised by our panel
members Dr. RAAJAN. N.R and Dr. RAMACHANDRAN. A.V helped us in
shaping this project to its worth. We take this opportunity to thank all our lecturers
who have directly or indirectly helped our project. We also extend our sincere
thanks to our parents for their continuous moral support and friends without whose
thought-provoking suggestions and encouragements, along with our guidance, this
work would not have been what it is today.

iv
 ABSTRACT

In the face of escalating concerns related to intruder attacks and theft, conventional security
is often hindered by their dependency sensors and motion detection devices, which is
susceptible to trigger false alarms due to climate variations or minor movements. This
project presents the design and implementation of a real-time face recognition system
integrated with spoof detection capabilities for door access. The system utilizes a webcam
and Solenoid door lock. Initially, user face data images are captured from the webcam and
processed for face detection using the dib library. Moreover, the system includes a spoof
detection module to differentiate between genuine faces and spoof attempts. Deep Neural
Networks are utilized for training the spoof detection model and recognition model. Upon
detection, the system attempts to recognize the face by comparing it with a database of
known faces. To enhance performance, a threading mechanism is employed for parallel
processing, ensuring real-time operation. If a face is recognized or unrecognized, the user
is promptly notified via email, enhancing system security and user awareness. The
implementation is carried out on the Raspberry Pi 4 Model B, demonstrating the feasibility
of deploying such a system on resource-constrained embedded platforms. The integration
of face recognition and spoof detection capabilities addresses security concerns in various
applications, including access control systems, surveillance, and user authentication.
Experimental results showcase the effectiveness and efficiency of the proposed system in
real-world scenarios.

Specific Contribution:
• Karunaesh RA – User interface, Data alignment and training
• Madesh Balu - Software implementation, Face recognition
• Varun A - Hardware integration, Spoof detection

Specific Learning:
• Karunaesh RA - Threading, Embeddings extraction, OpenCV
• Madesh Balu - Keras Facenet, Dlib, Python
• Varun A - Deep Neural Network, Embedded systems

Signature of the Guide:

Name: Dr. Rakesh Kumar Karn (Associate Professor)

Student Reg.No: 124004171 124004347 124004140

Name: Madesh Balu VarunA Karunaesh RA

v
 Table of Contents
Title Page No.
Bona-fide Certificate ii
Declaration iii
Acknowledgements iv
Abstract v
List of Figures viii
Abbreviations ix

1. Introduction
1.1. What is OpenCV 1
1.2. Dlib 2
1.3. Keras Facenet 2
1.4. Neural Network Classifier Architecture and Training 3
1.5. SMTP 3
1.6. GUI using Tkinter Module 4
1.7. Raspberry Pi 4 Model B 4
1.8. Solenoid door lock 5
1.9. Relay module 5
1.10. BC 547 Transistor 6
1.11. Motivation 7
2. Objectives 8
3. Methodology 9
3.1. Working 9
3.1.1. User Friendly Interface 9
3.1.2. Add and Delete Data 9
3.1.3. Alignment 10
3.1.4. Augmentation 11
3.1.5. Model training 12
3.1.6. Spoof Detection 13
3.1.7. Face Recognition 13
3.1.8. Email System 14
3.1.9 Hardware 15
3.2. Flowchart 16

vi
4. Results and Discussion
4.1. User interface (from user end) 17
4.2. Raspberry Pi Standalone System 17
4.2.1. Face Recognition 17
4.2.2. Uploading model to Raspberry Pi 19
4.2.3. Offline Mode 20
5. Conclusions 21
6. References 22
7. Appendix
7.1Similarity Check Report 23

vii
 List of Figures

Figure No. Title Page No.

1.1 SMTP 3
1.2 Raspberry PI 4B 4
1.3 12V Solenoid Lock 5
1.4 5V Relay 5
1.5 BC 547 Transistor 6
3.1 User friendly GUI 9
3.2 Adding and deleting user data 10
3.3 Before and after alignment 11
3.4 Model Training status 12
3.5 Trained model updated in Google drive 13
3.6 Email Address change GUI and status 14
3.7 Hardware connections 15
3.8 Flowchart 16
4.1 Push button status light 18
4.2 When person is real and identified, lock unlocks 18
4.3 When person is unrecognized or spoofed, lock 19
remains locked
4.4 Model update status in Raspberry Pi 20
4.5 System log stored in Raspberry Pi 20

viii
 ABBREVIATIONS

GUI Graphical User Interface

SMTP Simple Mail Transfer Protocol

TCP Transmission Control Protocol

ARM Advanced RISC Machine

RFID Radio-Frequency Identification

API Application Programming Interface

GPIO General Purpose Input/Output

OpenCV Open Source Computer Vision

DNN Deep Neural Networks

ix
 CHAPTER 1

INTRODUCTION

The security of homes, offices, and other buildings is a major concern for many people.
Traditional methods of security such as locks and keys, security cameras, and alarms
have proven to be effective to some extent, but they are not completely fool proof.
Intruders can still find ways to bypass these measures and gain unauthorized access.
With the increasing advancements in technology, new methods of security are being
developed to address these challenges. One such method is the use of face recognition
technology for intruder detection.

Face recognition is a security technology that utilizes algorithms to recognized faces to

identify unauthorized individuals attempting to gain access to a restricted area or
premises. The technology uses cameras and software to capture and analyse the facial
features of individuals and compare them to a database of authorized personnel. If a
match is not found, the system will alert its user.

The project is implemented using Python programming language and open-source

libraries OpenCV. It is integrated into raspberry pi and is a part of smart home system.
This project report will show how home or office security can be implemented.

1.1 What is Open CV?

OpenCV is an open-source software library that specializes in computer vision and machine
learning. It aims to serve as a universal platform for computer vision applications and to
streamline the incorporation of machine perception into consumer products. OpenCV
provides a broad spectrum of features and can be tailored to suit specific enterprise needs.
The library is equipped with an extensive array of over 2500 fine-tuned algorithms,
covering both conventional and state-of-the-art methods in computer vision and machine
learning. These algorithms facilitate various tasks such as detecting faces, identifying
objects, classifying human actions in videos, tracking camera motion, tracking objects,
extracting 3D models, generating 3D point clouds using stereo cameras, stitching images
for panoramic views, searching for similarities in image databases, removing red-eye in
flash photography, tracking eye movements, recognizing landscapes, and setting up
markers for augmented reality applications.

1
1.2 Dlib

Dlib is a complete toolkit known for its strong skills in facial landmark detection,
recognition, object detection, and machine learning. Its distinguishing characteristic is its
capacity to reliably recognize and locate facial landmarks such as the eyes, nose, mouth,
and jawline in photographs, making it ideal for tasks such as face alignment and biometric
verification. Furthermore, dlib includes facial recognition techniques like as face detection,
alignment, and recognition, which allow persons to be identified based on their facial traits
in applications such as surveillance and security. Beyond faces, dlib excels at object
detection and tracking, with implementations of well-known algorithms such as Histogram
of Oriented Gradients (HOG) and deep learning-based methods. Furthermore, dlib provides
a diverse range of machine learning methods, including support vector machines (SVM),
k-nearest neighbours (k-NN) and clustering algorithms are used to perform classification,
regression, and clustering problems. Its image processing facilities make routine
procedures like picture importing, resizing, and cropping easier while integrating smoothly
with its machine learning capabilities. Furthermore, dlib's cross-platform interoperability
makes it versatile, allowing developers to create apps for major operating systems such as
Windows, macOS, and Linux. Dlib's interaction with Python via bindings makes it
available to a larger audience of developers and academics, allowing for smooth integration
into Python-based projects.

1.3 Keras Facenet

The keras_facenet library is a Python package that implements the FaceNet model with
Keras, a high-level deep learning toolkit. FaceNet, developed by Google researchers, is a
groundbreaking deep convolutional neural network architecture for facial recognition
applications. Its major goal is to learn a mapping from face images to a compact Euclidean
space whose distances are directly proportional to face similarity measures. FaceNet uses
a triplet loss function during training to reduce the distance between photographs of the
same person while increasing the distance between images of different individuals. The
FaceNet model uses a deep CNN followed by global average pooling to build fixed-length
feature vectors (embeddings) from input facial images. Keras_facenet facilitates the use of
FaceNet by offering pre-trained models and methods for extracting facial embeddings.
FaceNet's seamless integration into the Keras environment allows users to simply include
its capabilities into their facial recognition projects, abstracting away the difficulties of
model setup and training. This ease makes keras_facenet an invaluable tool for Python
developers looking to incorporate cutting-edge face recognition techniques into their
projects.

2
1.4 Neural Network Classifier Architecture and Training

The classifier used in the program is a neural network model. Specifically, a Sequential
model is employed, which is a type of feedforward neural network where layers are stacked
sequentially. The model architecture consists of several dense (fully connected) layers
followed by activation functions such as LeakyReLU and BatchNormalization layers for
regularization and improved training stability. The output layer utilizes the softmax
activation function for multi-class classification, as the task involves recognizing
individuals from their facial features. The model undergoes training with the categorical
cross-entropy as the loss function and utilizes the Adam optimizer. Additionally, early
stopping is employed as a callback during training to prevent overfitting and improve
generalization performance. Overall, the neural network classifier is trained to recognize
individuals based on facial embeddings extracted using dlib, facilitating accurate facial
recognition in the program.

1.5 SMTP

SMTP or Simple Mail Transfer Protocol is a system that is responsible for the exchange of
emails between servers. It’s the backbone that allows email clients to dispatch and retrieve
messages from a mail server. The process begins with a connection being established
between the client and the server. Following this, the client transmits the email message to
the server. Which then relays it to the relevant recipient mail server. This recipient server
is then responsible for delivering the email to the recipient’s email client. While SMTP
typically communicates over TCP port 25, it can also utilize other ports like 587 and 465
for secure email transfers. SMTP is crucial for email communication. Thus, enabling users
to effortlessly send and receive emails. Incorporating SMTP into project allows users to
remain informed about the security system’s status and respond appropriately when
necessary. Refer Fig. 1.1.

Fig. 1.1 SMTP

3
1.6 GUI using Tkinter Module

Tkinter is an inbuilt module in Python that offers a toolkit for crafting graphical user
interfaces (GUIs) for desktop applications. It’s the standard GUI package for Python, built
on the foundation of the Tk GUI toolkit. Tkinter provides a collection of widgets, which
are graphical elements like buttons, labels, and text boxes, that can be positioned on an
application’s window. It also offers a range of methods and classes for managing user input,
executing data validation, and reacting to events. Developers can construct GUI
applications ranging from simple to complex, compatible with Windows, macOS and
Linux.

1.7 Raspberry Pi 4 Model B

The Raspberry Pi 4 Model B is a powerful single-board computer known for its adaptability
and compact design. It outperforms its predecessors thanks to the Broadcom BCM2711
quad-core Cortex-A72 (ARM v8) 64-bit processor running at 1.5 GHz. It comes in 2GB,
4GB, and 8GB LPDDR4 SDRAM versions, and its memory capacity determines
multitasking and overall system performance. With a VideoCore VI graphics processor, it
can handle high-definition video playback and 3D graphics processing. It has two USB 2.0
ports, two USB 3.0 ports, Gigabit Ethernet, multi-band 802.11ac wireless LAN, Bluetooth
5.0, dual micro HDMI connections with up to 4K resolution, a 3.5mm audio/video jack,
and a microSD card slot for storage and booting. With forty GPIO pins it makes interacting
with external sensors and hardware components easier. The Raspberry Pi 4 Model B can
be used for a wide range of projects, such as media centers, Internet of Things applications,
educational projects, and more, because it is compatible with a variety of Raspberry Pi
cases and accessories. Its compatibility with a variety of operating systems, such as
Windows 10 IoT Core, Ubuntu, and Raspberry Pi OS, further increases its adaptability and
usefulness in a range of sectors and applications.

Fig. 1.2 Raspberry PI 4B

4
1.8 Solenoid door lock

A solenoid door lock is an electromechanical device designed to secure doors by controlling

the locking mechanism through the application of an electrical current. At its core, the lock
comprises a solenoid, which consists of a coil of wire wrapped around a metal core. When
an electric current passes through the coil, it generates a magnetic field that moves the core,
either engaging or disengaging the locking mechanism, such as a bolt or latch. Typically
controlled by an external circuit or device like a keypad or RFID reader, solenoid door
locks offer a secure and convenient means of access control in various settings, including
residential, commercial, and industrial environments. They provide key advantages such
as reliability, convenience, and durability. When the face recognition system identifies an
authorized user, it sends a signal to activate the solenoid door lock. The solenoid door lock
engages, unlocking the door and allowing access. After a specified period, the door lock
disengages and secures the door again.

Fig.1.3 12V Solenoid Lock

1.9 Relay module

A relay module is an electromechanical switch that allows a low-power circuit (such as

from a microcontroller or a push button) to control a high-power circuit (such as the one
needed to drive the solenoid).The solenoid door lock is connected to the high power side
of the relay module, whereas the control circuit is added to low power side of the relay
module. When the control circuit triggers the relay module, it energizes the relay coil,
which in turn closes or opens the relay switch, depending on its configuration. This action
allows or interrupts the flow of electricity to the solenoid door lock. When the relay is
activated (usually by sending a signal from a controller or pressing a button), it completes
the circuit between the power source and the solenoid. This energizes the solenoid, causing
the door lock to engage or disengage depending on its design. When the relay is
deactivated, the circuit to the solenoid is opened, and the door lock remains in its current
state.

Fig. 1.4 5V Relay

5
1.10 BC 547 Transistor

Fig. 1.5 BC 547 Transistor

The BC547 is an NPN bipolar junction transistor (BJT) commonly used in low-power
switching and amplification applications. It composed of the emitter, base, and collector.
In its default state, the transistor is off, with no current flowing from the collector to the
emitter. To turn it on, a small current is applied to the base, creating a flow of electrons
from the emitter through the base to the collector. This flow allows a much larger current
to pass from the collector to the emitter, making the BC547 suitable for amplifying signals
or switching circuits on and off. The transistor's current gain, known as beta (β), determines
how much the collector current is amplified relative to the base current. The BC547 can
handle collector currents up to 100 mA and collector-emitter voltages up to 45V, making
it versatile for a range of electronic projects and circuits.

6
1.11 Motivation

This project stems from the limitations and vulnerabilities of conventional security systems
in effectively addressing the escalating concerns related to intruder attacks and theft.
Traditional security measures often rely on sensors and motion detection devices, which
are prone to triggering false alarms due to factors such as climate variations or minor
movements. These false alarms not only diminish the reliability of the security system but
also lead to unnecessary disruptions and decreased trust in its effectiveness.
To overcome these challenges, the project focuses on designing and implementing a real-
time face recognition system integrated with spoof detection capabilities for door access.
By leveraging advancements in technology, such as computer vision and deep learning, the
system aims to provide a more robust and accurate means of authentication and access
control.

The integration of face recognition technology addresses the need for a more secure and
convenient method of verifying individuals' identities, while the inclusion of spoof
detection capabilities enhances the system's resilience against fraudulent attempts to bypass
security measures. This dual-layered approach not only improves security but also
mitigates the risks associated with unauthorized access and identity theft. Moreover, the
choice of using a webcam and Solenoid door lock, coupled with the implementation on the
Raspberry Pi 4 Model B, underscores the project's emphasis on scalability, affordability,
and feasibility for deployment on resource-constrained embedded platforms. By
demonstrating the practicality of deploying such a system in real-world scenarios, the
project aims to provide a versatile and accessible solution to address security concerns
across various applications, including access control systems, surveillance, and user
authentication.

7
 CHAPTER 2

OBJECTIVES

• Develop a Raspberry Pi-based system for secure access control using facial
recognition.
• Continuously monitor the environment with the camera.
• Upon user interaction, initiate facial recognition with integrated spoof detection.
• Grant access (unlock solenoid) to recognized and genuine individuals.
• Inform the user of the access attempt status (granted/denied).
• Provide a user-friendly interface for managing the access control system on a
laptop.
• Enable user functionalities:
o Add, delete, or modify authorized user email addresses.
o Upload facial recognition models for training.
o Capture user images and associate them with names for dataset creation.
o Train the facial recognition model using deep learning techniques
(alignment, cropping, embedding extraction, classification).
• Implement cloud storage for the trained model to ensure redundancy and remote
updates.
• To evaluate the effectiveness and accuracy of the system by testing it in various
environments.
• To gain hands-on experience in raspberry pi and software development.

8
 CHAPTER 3

METHODOLOGY

3.1 WORKING

3.1.1 User Friendly Interface:

Fig. 3.1 User Friendly interface

A graphical user interface (GUI) application is implemented using the Tkinter library in
Python for an intelligent recognition system. The GUI window presents several buttons,
each corresponding to a different operation within the system. These operations include
adding data to the dataset, removing data from the dataset, training the model, Uploading
the model to raspberry pi and changing the email address associated with the system. Upon
clicking each button, a separate thread is created to execute the corresponding operation,
ensuring that the GUI remains responsive during the execution of time-consuming tasks.
Additionally, there is an option to exit the program, which prompts a confirmation dialog
before closing the application.

3.1.2 Add and Delete data:

The application allows users to enter their name and then captures a sequence of five images
when the user presses the capture button. These images are saved in a directory named
"Dataset" under separate folders with the user's name. The images are captured with a
countdown displayed on each frame, prompting the user to capture the image by pressing
the 'C' key. Once five images are captured, the application displays a message box
indicating successful capture. The GUI is built using the Tkinter library, and the image
capture functionality is implemented using OpenCV.

9
To Delete data, Users can input a name corresponding to the data they wish to delete, and
upon clicking the "Delete User" button, the application searches for the user's data in three
specified directories: "Dataset", "Aligned Dataset", and "Stored Embeddings". If the user's
data exists in the "Dataset", the application deletes the corresponding folder. It attempts to
delete the user's folder in the "Aligned Dataset" and their embeddings file in the "Stored
Embeddings" directory. Upon successful deletion, appropriate messages are displayed to
the user. Finally, users can exit the application by clicking the "Exit" button.

Fig. 3.2 Adding and Deleting User Data

3.1.3 Alignment:

Facial alignment is a crucial preprocessing step in facial recognition systems, playing a

pivotal role in ensuring accurate and robust face recognition performance. This process
involves detecting faces within images and subsequently identifying key facial landmarks,
such as the eyes, nose, and mouth corners. These landmarks serve as reference points for
aligning facial features to a standardized configuration, compensating for variations in
pose, scale, and orientation.

The code first detects faces in images using the dlib library, aligns them using facial
landmarks, and saves the aligned images to a specified folder. The alignment process
includes handling scenarios by filtering like multiple faces in an image, images with
excessive blur, images too close to the image edge, and black and white images.

10
After alignment, the aligned facial region is cropped and resize to 160 x 160, focusing
exclusively on the facial features while excluding irrelevant background information.
Facial alignment ensures that facial images are uniformly represented for better face
recognition.

Fig. 3.3 Before and After Alignment

3.1.4 Augmentation:

Data augmentation helps to increase the diversity of the dataset. Data augmentation
techniques such as rotation, flipping, and zooming are applied to generate additional
training samples. A class called `AUGMENTATION`, designed to augment images in a
dataset for enhancing training data diversity. It contains methods to load aligned images
from the dataset and augment them using specific transformations like brightness and
contrast adjustment.

11
The augmentation is performed using the Keras `ImageDataGenerator` class, which
generates batches of augmented images based on the specified augmentation parameters.
For each class in the dataset, the script checks the number of images and calculates the
number of new images needed to reach the target number. It then iterates over batches of
augmented images, converting them to RGB format and saving them to disk. Once the
desired number of augmented images is reached for each class, the augmentation process
is completed. This process is necessary to balance each dataset class to prevent any bias on
model training.

3.1.5 Model training:

The training is done using the Facenet pre-trained model and Deep neural network
architecture. It first checks for the existence of an "Aligned Dataset" folder, which should
contain aligned face images for training. Theen the system facilitates the loading of aligned
faces, extraction of embeddings using Facenet, and training of the DNN model. The process
involves iterating through each class (i.e., each individual's face data), extracting
embeddings for each face image, and storing them along with their corresponding labels.

The script then splits the data into training and testing sets, constructs a neural network
model with three hidden layers, compiles it with an Adam optimizer, and trains it using the
extracted embeddings and labels. Early stopping is employed to prevent overfitting, and
the trained model is saved for future use. On successful completion of training, a message
box confirms the successful training of the model.

After the model is trained, “Upload Model” option is clicked. The system provides a
mechanism for compressing, uploading, and managing machine learning model files to
Google Drive. It compresses the contents of a directory into a ZIP file and then uploads it
to a specified Google Drive folder, managing Google authentication and credentials. If
there's an existing file with the same name in the folder, it deletes it before uploading the
new file. It also checks for internet connectivity before initiating the upload process. If the
internet is unavailable, it displays a message box informing the user, ensuring a smooth and
error-free upload operation. The code uses Google Drive APIs for file management and
PyDrive for authentication and operations. After successful uploading, Success message
will be shown. We can check the model which had been uploaded in Google drive as in
Fig.3.5

Fig. 3.4 Model Training Status

12
 Fig. 3.5 Trained Model Updated in Google Drive

3.1.6 Spoof detection:

A python code trains a deep neural network (DNN) model to detect spoof faces, focusing
on distinguishing between real and fake images. It first loads images from specified folders,
either detecting faces in them using dlib's face detector or assuming faces are already
detected, and then resizes them to a fixed size (160x160 pixels). These images are labelled
as real or fake (1 or 0, respectively) based on their source folder. The dataset is split into
training and testing sets, and a DNN model is defined using Keras. The model architecture
consists of dense layers with rectified linear unit (ReLU) activation, batch normalization,
dropout regularization, and L2 regularization to prevent overfitting. The Adam optimizer
is employed with a custom learning rate, and early stopping is utilized to prevent overfitting
during training. Finally, the trained model is saved for use in spoof detection of face
recognition. The trained model predicts the label (real or fake) of each ROI, and the
prediction confidence.

3.1.7 Face recognition:

This code is a face recognition system with spoof detection, implemented using OpenCV,
TensorFlow, and Raspberry Pi GPIO controls. It initializes a webcam for real-time face
detection and recognition, using MediaPipe's BlazeFace for face detection and Keras
FaceNet for facial embeddings. This embedding is used by DNN classifier to compare with
stored embeddings. Which checks for both recognized and unrecognized faces, sending
email alerts for various scenarios: when a recognized face with a high confidence score is
detected, when a spoof (fake) attempt is made, or when an unrecognized face is repeatedly
observed. The GPIO pins control various hardware components, such as LEDs and relays,
to indicate system states and trigger physical responses. The system also supports toggling
face recognition and email notifications through user inputs and has mechanisms for
reloading models from cloud and managing downloads for maintaining up-to-date facial
data.

13
 3.1.8 Email system:

A simple GUI application using Tkinter is created for validating and updating an email
address which is part of user friendly interface. It creates a window with a label prompting
the user to enter their email address and an entry field for input. Upon clicking the "Enter"
button, the entered email address is validated using a regular expression pattern to ensure
it conforms to the standard email format. If the entered email address is valid, a success
message is displayed. Then the saved Email address user gets notified. If the email address
is invalid, an error message is shown, prompting the user to enter it again. Once a valid
email address is entered and successfully saved, the GUI window is closed. The saved
email address will be uploaded to Google drive, from drive Raspberry pi gets the email
address for transferring information.

For sending mail, A Python script defines a function `send_mail` for sending email
notifications. It uses the SMTP protocol to connect to a Gmail SMTP server
(`smtp.gmail.com`) on port 587 for sending emails. The function takes three parameters:
`text_message` (the body of the email), `img_req` (a flag indicating whether an image
attachment is required), and `count` (a delay count before sending the email). It reads the
sender's email address, receiver's email address, and SMTP password from configuration
variables. The function creates an email message object with the specified subject, sender,
receiver, and message body. If an image attachment is required, it reads an image file and
attaches it to the email. Finally, it sends the email using the `smtplib` library.

Fig. 3.6 Email Address change Gui and Status

14
 3.1.9 Hardware:

Fig. 3.7 Hardware Connections

The Raspberry Pi 4 Model B 4GB variant is at the heart of a face recognition-based security
system. It connects to a webcam, which captures the facial data for recognition. A push
button serves as the activation trigger for the face recognition program, allowing the system
to identify individuals and determine access permissions. The relay module used in this
system operates in an active-low state and is responsible for controlling a solenoid that
requires 12V to operate. To interface with the relay module, the system uses a GPIO pin
from the Raspberry Pi. However, because the GPIO pins output only 3.3V, which isn't
enough to activate the 5V relay module, a BC547 transistor is employed as a switch.

In this setup, the base of the BC547 transistor connects to GPIO pin '27' of the Raspberry
Pi, the collector is connected to a 5V source, and the emitter is grounded. The use of a
transistor here allows the low-voltage signal from the GPIO pin to control the higher-
voltage circuit needed for the relay module. When the GPIO pin is inactive, no current
flows to the transistor's base, keeping the transistor in an "open" state. In this state, the 5V
current from the collector flows to the relay module's INPUT pin, keeping it switched off,
which locks the door.

When the push button is pressed, activating the GPIO pin, current flows to the transistor's
base, causing it to "close." This redirects the 5V current to ground, resulting in 0V at the
relay module's INPUT pin. This low signal activated the relay, unlocking the door.
Additionally, LED lights are incorporated to indicate system status: one light signifies
when the recognition code is running, another when the door is unlocked, and another when
access is denied. This setup ensures secure and controlled door access based on facial
recognition, with clear visual indicators of system status.

15
3.2 Flowchart

Fig 3.8 Flowchart

16
 CHAPTER 4

RESULT AND DISCUSSION

4.1 User Interface (from user end)

The main GUI consists of five different operations, which includes Adding data,
removing data from dataset, training and uploading of model and changing email address.
All the operations work effectively when selected. When the user wants to close the
program, user must select “Exit” button which terminates the program.

To register a user's face, select the "Add Dataset" button on the GUI. When you click this
button, a new GUI window opens, where it is prompted to enter personal information in
the input fields.

Clicking "Capture Images" activates the webcam. Press 'C' to capture a total of five
images. These images are then saved and stored in the ‘Dataset’ folder. Then after model
training, the model is successfully uploaded.

4.2 Raspberry Pi Standalone System

The monitoring system was implemented successfully in Raspberry pi 4B. It is capable to

run 24/7 if proper cooling is provided.

4.2.1 Face Recognition

The camera module attached to a Raspberry Pi operates continuously, providing a constant

surveillance feed. The system remains in a standby state until a user activates it by pressing
a push button and its status shown in Fig.4.1. Once activated, the camera begins running a
facial recognition program to identify individuals. If the system detects a person attempting
to gain entry and their facial data does not match any existing records in the database, the
system identifies this person as an unrecognized. As a security measure, an email
notification is immediately sent to the designated user to alert them of the unrecognized
entry attempt as shown in Fig.4.3.

On the other hand, if the recognition program identifies the person and matches their facial
data with an existing record in the database, additional checks are performed to determine
the legitimacy of the image captured by the webcam. If the image is authentic, suggesting
that it's not a photo or a manipulated image but a real person, the system proceeds to unlock
the door, granting the recognized individual entry. Concurrently, a confirmation email is
sent to the registered email address to inform the user that a recognized individual has
entered and the door has been unlocked as shown in Fig.4.2.

17
However, if the system determines that the image is not genuine, which could be due to
various reasons like the use of photos, videos, or other spoofing techniques to gain
unauthorized access, the system recognizes this as a 'spoof' attempt. In this scenario, the
door remains locked, preventing unauthorized entry. Additionally, an email is sent to the
registered email address to alert the user of a potential security threat due to a spoofing
attempt in Fig.4.3. This comprehensive security system ensures that only authorized
individuals can only access while providing real-time alerts to users for any unusual or
unauthorized activity.

Fig. 4.1 Push Button status light

Fig. 4.2 When person is real and identified, lock unlocks

18
 Fig. 4.3 When person is unrecognized or spoofed, lock remain locked

4.2.2 Uploading model to Raspberry Pi

This system checks for updates each minute and downloads a compressed model file from
Google Drive, extracting it locally if there's a newer version. It establishes Google Drive
authentication, retrieves a list of files from a specific folder, and compares the remote file's
last modified timestamp with a locally stored timestamp. If the remote timestamp indicates
a newer version, the code downloads the file, updates the local timestamp, and extracts the
contents of the ZIP file to the current working directory. If there's no update, it indicates
that the local version is up to date. This process ensures that the latest version of the model
is always available locally. Message displayed in raspberry pi screen regarding updated
model and email address is shown in Fig 4.4.

19
 Fig.
4.4 Model update status in Raspberry Pi

4.2.3 Offline Mode

When there is not internet connectivity, raspberry pi can work effectively, it uses recent
model stored in local database, it stores all the status information locally. When internet is
available, it checks for new model and updates it locally. Local stored system log is shown
in Fig 4.5.

Fig. 4.5 System Log stored in Raspberry PI

20
 CHAPTER 5

CONCLUSION

This project successfully developed a secure access control system using facial recognition
on a Raspberry Pi. The system continuously monitors the environment and initiates facial
recognition with spoof detection upon interaction. Real and recognized individuals are
granted access, with clear status notifications provided to the user. A user-friendly interface
allows for managing the system remotely via a laptop. The deep learning-based model
utilizes techniques like image alignment, cropping, embedding extraction, and
classification for accurate recognition.

The system incorporates cloud storage for model redundancy and remote updates, ensuring
continued functionality even in case of network failures. This project demonstrates the
feasibility of a cost-effective and secure access control solution using readily available
hardware and open-source software libraries.

Key Achievements:

• Developed a robust facial recognition system with integrated spoof detection.

• Implemented a user-friendly interface for remote system management.
• Integrated cloud storage for model redundancy and network resilience.
• Showcased the capabilities of Raspberry Pi for real-world security applications.

Future Work:

• Explore incorporating additional security measures like multi-factor authentication.

• Enhance the system's scalability to accommodate a larger number of users.
• Investigate the use of more advanced deep learning architectures for improved
recognition accuracy.
• Develop a mobile application for user interaction and system management.

21
 CHAPTER 6

REFERENCES

1. Li, N., Shen, X., Sun, L., Xiao, Z., Ding, T., Li, T. and Li, X., 2023. “Chinese face
dataset for face recognition in an uncontrolled classroom environment”. IEEE
Access.

2. Kim, H., Choi, N., Kwon, H.J. and Kim, H., 2023. “Surveillance System for Real-
Time High-Precision Recognition of Criminal Faces from Wild Videos”. IEEE
Access.

3. Yashashwini, M., Kumar, K.S., Pitchai, R., Sankeerth, K.S.S., Prasath, G.A. and
Trinath, D., 2023, November. “Face Recognition Based Smart Door Lock System
using Convolution Neural Network”. In 2023 International Conference on
Research Methodologies in Knowledge Management, Artificial Intelligence and
Telecommunication Engineering (RMKMATE) (pp. 1-6). IEEE.

4. Reddy, T.G., Sai, S.C., Kumar, B.P., Venkatesh, R.T., Sathwik, K. and Singh, K.,
2023, May. “Face Recognition Door Lock System Using Raspberry Pi”. In 2023
Third International Conference on Secure Cyber Computing and Communication
(ICSCCC) (pp. 218-221). IEEE.

5. Hussain, T., Hussain, D., Hussain, I., AlSalman, H., Hussain, S., Ullah, S.S. and
Al-Hadhrami, S., 2022. “Internet of things with deep learning-based face
recognition approach for authentication in control medical systems”.
Computational and Mathematical Methods in Medicine, 2022.

22
 CHAPTER 7

APPENDIX

7.1 Similarity Check Report

23
 PLAGIARISM SCAN REPORT

Date 2024-05-05

0% 100%
Words 800
Plagiarised Unique

Characters 5976

Content Checked For Plagiarism

ABSTRACT

In the face of escalating concerns related to intruder attacks and theft, conventional security is often hindered by their
dependency sensors and motion detection devices, which is susceptible to trigger false alarms due to climate variations or
minor movements. This project presents the design and implementation of a real-time face recognition system integrated
with spoof detection capabilities for door access. The system utilizes a webcam and Solenoid door lock. Initially, user face
data images are captured from the webcam and processed for face detection using the dib library. Moreover, the system
includes a spoof detection module to differentiate between genuine faces and spoof attempts. Deep Neural Networks are
utilized for training the spoof detection model and recognition model. Upon detection, the system attempts to recognize
the face by comparing it with a database of known faces. To enhance performance, a threading mechanism is employed for
parallel processing, ensuring real-time operation. If a face is recognized or unrecognized, the user is promptly notified via
email, enhancing system security and user awareness. The implementation is carried out on the Raspberry Pi 4 Model B,
demonstrating the feasibility of deploying such a system on resource-constrained embedded platforms. The integration of
face recognition and spoof detection capabilities addresses security concerns in various applications, including access
control systems, surveillance, and user authentication. Experimental results showcase the effectiveness and efficiency of the
proposed system in real-world scenarios.

Specific Contribution:
* Karunaesh RA – User interface, Data alignment and training
* Madesh Balu - Software implementation, Face recognition
* Varun A - Hardware integration, Spoof detection

Specific Learning:
* Karunaesh RA - Threading, Embeddings extraction, OpenCV
* Madesh Balu - Keras Facenet, Dlib, Python
* Varun A - Deep Neural Network, Embedded systems

Signature of the Guide

Name: Dr. Rakesh Kumar Karn (Associate Professor)

Student Reg.No: 124004171 124004347 124004140

Name: Madesh Balu VarunA Karunaesh RA

List of Figures

Page 1 of 4
Figure No.
Title
Page No.
1.1
SMTP
3
1.2
Raspberry PI 4B
4
1.3
12V Solenoid Lock
5
1.4
5V Relay
5
1.5
BC 547 Transistor
6
3.1
User friendly GUI
9
3.2
Adding and deleting user data
10
3.3
Before and after alignment
11
3.4
Model Training status
12
3.5
Trained model updated in Google drive
13
3.6
Email Address change GUI and status
14
3.7
Hardware connections
15
3.8
Flowchart
16
4.1
Push button status light
18
4.2
When person is real and identified, lock unlocks
18
4.3
When person is unrecognized or spoofed, lock remains locked
19
4.4
Model update status in Raspberry Pi

Page 2 of 4
20
4.5
System log stored in Raspberry Pi
20

ABBREVIATIONS

GUI Graphical User Interface

SMTP Simple Mail Transfer Protocol

TCP Transmission Control Protocol

ARM Advanced RISC Machine

RFID Radio-Frequency Identification

API Application Programming Interface

GPIO General Purpose Input/Output

OpenCV Open Source Computer Vision

DNN Deep Neural Networks

iii

CHAPTER 1

INTRODUCTION

The security of homes, offices, and other buildings is a major concern for many people. Traditional methods of security
such as locks and keys, security cameras, and alarms have proven to be effective to some extent, but they are not
completely fool proof. Intruders can still find ways to bypass these measures and gain unauthorized access. With the
increasing advancements in technology, new methods of security are being developed to address these challenges. One
such method is the use of face recognition technology for intruder detection.

Page 3 of 4
Face recognition is a security technology that utilizes algorithms to recognized faces to identify unauthorized individuals
attempting to gain access to a restricted area or premises. The technology uses cameras and software to capture and
analyse the facial features of individuals and compare them to a database of authorized personnel. If a match is not found,
the system will alert its user.

The project is implemented using Python programming language and open-source libraries OpenCV. It is integrated into
raspberry pi and is a part of smart home system.
This project report will show how home or office security can be implemented.

1.1 What is Open CV?

OpenCV is an open-source software library that specializes in computer vision and machine learning. It aims to serve as a
universal platform for computer vision applications and to streamline the incorporation of machine perception into
consumer products. OpenCV provides a broad spectrum of features and can be tailored to suit specific enterprise needs.
The library is equipped with an extensive array of over 2500 fine-tuned algorithms, covering both conventional and state-
of-the-art methods in computer vision and machine learning. These algorithms facilitate various tasks such as detecting
faces, identifying objects, classifying human actions in videos, tracking camera motion, tracking objects, extracting 3D
models, generating 3D point clouds using stereo cameras, stitching images for panoramic views, searching for similarities
in image databases, removing red-eye in flash photography, tracking eye movements, recognizing landscapes, and setting
up markers for augmented reality applications.

Matched Source

No plagiarism found

Page 4 of 4
 PLAGIARISM SCAN REPORT

Date 2024-05-05

0% 100%
Words 982
Plagiarised Unique

Characters 6906

Content Checked For Plagiarism

1.2 Dlib
Dlib is a complete toolkit known for its strong skills in facial landmark detection, recognition, object detection, and
machine learning. Its distinguishing characteristic is its capacity to reliably recognize and locate facial landmarks such as
the eyes, nose, mouth, and jawline in photographs, making it ideal for tasks such as face alignment and biometric
verification. Furthermore, dlib includes facial recognition techniques like as face detection, alignment, and recognition,
which allow persons to be identified based on their facial traits in applications such as surveillance and security. Beyond
faces, dlib excels at object detection and tracking, with implementations of well-known algorithms such as Histogram of
Oriented Gradients (HOG) and deep learning-based methods. Furthermore, dlib provides a diverse range of machine
learning methods, including support vector machines (SVM), k-nearest neighbours (k-NN) and clustering algorithms are
used to perform classification, regression, and clustering problems. Its image processing facilities make routine procedures
like picture importing, resizing, and cropping easier while integrating smoothly with its machine learning capabilities.
Furthermore, dlib's cross-platform interoperability makes it versatile, allowing developers to create apps for major
operating systems such as Windows, macOS, and Linux. Dlib's interaction with Python via bindings makes it available to a
larger audience of developers and academics, allowing for smooth integration into Python-based projects.

1.3 Keras Facenet

The keras_facenet library is a Python package that implements the FaceNet model with Keras, a high-level deep learning
toolkit. FaceNet, developed by Google researchers, is a groundbreaking deep convolutional neural network architecture for
facial recognition applications. Its major goal is to learn a mapping from face images to a compact Euclidean space whose
distances are directly proportional to face similarity measures. FaceNet uses a triplet loss function during training to reduce
the distance between photographs of the same person while increasing the distance between images of different
individuals. The FaceNet model uses a deep CNN followed by global average pooling to build fixed-length feature vectors
(embeddings) from input facial images. Keras_facenet facilitates the use of FaceNet by offering pre-trained models and
methods for extracting facial embeddings. FaceNet's seamless integration into the Keras environment allows users to
simply include its capabilities into their facial recognition projects, abstracting away the difficulties of model setup and
training. This ease makes keras_facenet an invaluable tool for Python developers looking to incorporate cutting-edge face
recognition techniques into their projects.

1.4 Neural Network Classifier Architecture and Training

Page 1 of 2
The classifier used in the program is a neural network model. Specifically, a Sequential model is employed, which is a type
of feedforward neural network where layers are stacked sequentially. The model architecture consists of several dense (fully
connected) layers followed by activation functions such as LeakyReLU and BatchNormalization layers for regularization and
improved training stability. The output layer utilizes the softmax activation function for multi-class classification, as the task
involves recognizing individuals from their facial features. The model undergoes training with the categorical cross-entropy
as the loss function and utilizes the Adam optimizer. Additionally, early stopping is employed as a callback during training
to prevent overfitting and improve generalization performance. Overall, the neural network classifier is trained to recognize
individuals based on facial embeddings extracted using dlib, facilitating accurate facial recognition in the program.

1.5 SMTP

SMTP or Simple Mail Transfer Protocol is a system that is responsible for the exchange of emails between servers. It’s the
backbone that allows email clients to dispatch and retrieve messages from a mail server. The process begins with a
connection being established between the client and the server. Following this, the client transmits the email message to
the server. Which then relays it to the relevant recipient mail server. This recipient server is then responsible for delivering
the email to the recipient’s email client. While SMTP typically communicates over TCP port 25, it can also utilize other ports
like 587 and 465 for secure email transfers. SMTP is crucial for email communication. Thus, enabling users to effortlessly
send and receive emails. Incorporating SMTP into project allows users to remain informed about the security system’s
status and respond appropriately when necessary. Refer Fig. 1.1.
1.6 GUI using Tkinter Module

Tkinter is an inbuilt module in Python that offers a toolkit for crafting graphical user interfaces (GUIs) for desktop
applications. It’s the standard GUI package for Python, built on the foundation of the Tk GUI toolkit. Tkinter provides a
collection of widgets, which are graphical elements like buttons, labels, and text boxes, that can be positioned on an
application’s window. It also offers a range of methods and classes for managing user input, executing data validation, and
reacting to events. Developers can construct GUI applications ranging from simple to complex, compatible with Windows,
macOS and Linux.

1.7 Raspberry Pi 4 Model B

The Raspberry Pi 4 Model B is a powerful single-board computer known for its adaptability and compact design. It
outperforms its predecessors thanks to the Broadcom BCM2711 quad-core Cortex-A72 (ARM v8) 64-bit processor running
at 1.5 GHz. It comes in 2GB, 4GB, and 8GB LPDDR4 SDRAM versions, and its memory capacity determines multitasking and
overall system performance. With a VideoCore VI graphics processor, it can handle high-definition video playback and 3D
graphics processing. It has two USB 2.0 ports, two USB 3.0 ports, Gigabit Ethernet, multi-band 802.11ac wireless LAN,
Bluetooth 5.0, dual micro HDMI connections with up to 4K resolution, a 3.5mm audio/video jack, and a microSD card slot
for storage and booting. With forty GPIO pins it makes interacting with external sensors and hardware components easier.
The Raspberry Pi 4 Model B can be used for a wide range of projects, such as media centers, Internet of Things
applications, educational projects, and more, because it is compatible with a variety of Raspberry Pi cases and accessories.
Its compatibility with a variety of operating systems, such as Windows 10 IoT Core, Ubuntu, and Raspberry Pi OS, further
increases its adaptability and usefulness in a range of sectors and applications.

Matched Source

No plagiarism found

Page 2 of 2
 PLAGIARISM SCAN REPORT

Date 2024-05-05

0% 100%
Words 917
Plagiarised Unique

Characters 6472

Content Checked For Plagiarism

1.8 Solenoid door lock

A solenoid door lock is an electromechanical device designed to secure doors by controlling the locking mechanism
through the application of an electrical current. At its core, the lock comprises a solenoid, which consists of a coil of wire
wrapped around a metal core. When an electric current passes through the coil, it generates a magnetic field that moves
the core, either engaging or disengaging the locking mechanism, such as a bolt or latch. Typically controlled by an external
circuit or device like a keypad or RFID reader, solenoid door locks offer a secure and convenient means of access control in
various settings, including residential, commercial, and industrial environments. They provide key advantages such as
reliability, convenience, and durability. When the face recognition system identifies an authorized user, it sends a signal to
activate the solenoid door lock. The solenoid door lock engages, unlocking the door and allowing access. After a specified
period, the door lock disengages and secures the door again.

Fig.1.3 12V Solenoid Lock

1.9 Relay module

A relay module is an electromechanical switch that allows a low-power circuit (such as from a microcontroller or a push
button) to control a high-power circuit (such as the one needed to drive the solenoid).The solenoid door lock is connected
to the high power side of the relay module, whereas the control circuit is added to low power side of the relay module.
When the control circuit triggers the relay module, it energizes the relay coil, which in turn closes or opens the relay switch,
depending on its configuration. This action allows or interrupts the flow of electricity to the solenoid door lock. When the
relay is activated (usually by sending a signal from a controller or pressing a button), it completes the circuit between the
power source and the solenoid. This energizes the solenoid, causing the door lock to engage or disengage depending on
its design. When the relay is deactivated, the circuit to the solenoid is opened, and the door lock remains in its current
state.

Fig. 1.4 5V Relay

1.10 BC 547 Transistor

Fig. 1.5 BC 547 Transistor

The BC547 is an NPN bipolar junction transistor (BJT) commonly used in low-power switching and amplification
applications. It composed of the emitter, base, and collector. In its default state, the transistor is off, with no current flowing
from the collector to the emitter. To turn it on, a small current is applied to the base, creating a flow of electrons from the
emitter through the base to the collector. This flow allows a much larger current to pass from the collector to the emitter,

Page 1 of 3
making the BC547 suitable for amplifying signals or switching circuits on and off. The transistor's current gain, known as
beta (β), determines how much the collector current is amplified relative to the base current. The BC547 can handle
collector currents up to 100 mA and collector-emitter voltages up to 45V, making it versatile for a range of electronic
projects and circuits.

1.11 Motivation

This project stems from the limitations and vulnerabilities of conventional security systems in effectively addressing the
escalating concerns related to intruder attacks and theft. Traditional security measures often rely on sensors and motion
detection devices, which are prone to triggering false alarms due to factors such as climate variations or minor movements.
These false alarms not only diminish the reliability of the security system but also lead to unnecessary disruptions and
decreased trust in its effectiveness.
To overcome these challenges, the project focuses on designing and implementing a real-time face recognition system
integrated with spoof detection capabilities for door access. By leveraging advancements in technology, such as computer
vision and deep learning, the system aims to provide a more robust and accurate means of authentication and access
control.

The integration of face recognition technology addresses the need for a more secure and convenient method of verifying
individuals' identities, while the inclusion of spoof detection capabilities enhances the system's resilience against fraudulent
attempts to bypass security measures. This dual-layered approach not only improves security but also mitigates the risks
associated with unauthorized access and identity theft. Moreover, the choice of using a webcam and Solenoid door lock,
coupled with the implementation on the Raspberry Pi 4 Model B, underscores the project's emphasis on scalability,
affordability, and feasibility for deployment on resource-constrained embedded platforms. By demonstrating the
practicality of deploying such a system in real-world scenarios, the project aims to provide a versatile and accessible
solution to address security concerns across various applications, including access control systems, surveillance, and user
authentication.

Page 2 of 3
 CHAPTER 2

OBJECTIVES

• Develop a Raspberry Pi-based system for secure access control using facial recognition.
• Continuously monitor the environment with the camera.
• Upon user interaction, initiate facial recognition with integrated spoof detection.
• Grant access (unlock solenoid) to recognized and genuine individuals.
• Inform the user of the access attempt status (granted/denied).
• Provide a user-friendly interface for managing the access control system on a laptop.
• Enable user functionalities:
o Add, delete, or modify authorized user email addresses.
o Upload facial recognition models for training.
o Capture user images and associate them with names for dataset creation.
o Train the facial recognition model using deep learning techniques (alignment, cropping, embedding extraction,
classification).
• Implement cloud storage for the trained model to ensure redundancy and remote updates.
• To evaluate the effectiveness and accuracy of the system by testing it in various environments.
• To gain hands-on experience in raspberry pi and software development.

Matched Source

No plagiarism found

Page 3 of 3
 PLAGIARISM SCAN REPORT

Date 2024-05-05

2% 98%
Words 948
Plagiarised Unique

Characters 6494

Content Checked For Plagiarism

CHAPTER 3

METHODOLOGY

3.1 WORKING
3.1.1 User Friendly Interface:

Fig. 3.1 User Friendly interface

A graphical user interface (GUI) application is implemented using the Tkinter library in Python for an intelligent recognition
system. The GUI window presents several buttons, each corresponding to a different operation within the system. These
operations include adding data to the dataset, removing data from the dataset, training the model, Uploading the model
to raspberry pi and changing the email address associated with the system. Upon clicking each button, a separate thread is
created to execute the corresponding operation, ensuring that the GUI remains responsive during the execution of time-
consuming tasks. Additionally, there is an option to exit the program, which prompts a confirmation dialog before closing
the application.
3.1.2 Add and Delete data:
The application allows users to enter their name and then captures a sequence of five images when the user presses the
capture button. These images are saved in a directory named "Dataset" under separate folders with the user's name. The
images are captured with a countdown displayed on each frame, prompting the user to capture the image by pressing the
'C' key. Once five images are captured, the application displays a message box indicating successful capture. The GUI is
built using the Tkinter library, and the image capture functionality is implemented using OpenCV.

To Delete data, Users can input a name corresponding to the data they wish to delete, and upon clicking the "Delete User"
button, the application searches for the user's data in three specified directories: "Dataset", "Aligned Dataset", and "Stored
Embeddings". If the user's data exists in the "Dataset", the application deletes the corresponding folder. It attempts to
delete the user's folder in the "Aligned Dataset" and their embeddings file in the "Stored Embeddings" directory. Upon
successful deletion, appropriate messages are displayed to the user. Finally, users can exit the application by clicking the
"Exit" button.

Fig. 3.2 Adding and Deleting User Data

3.1.3 Alignment:
Facial alignment is a crucial preprocessing step in facial recognition systems, playing a pivotal role in ensuring accurate and
robust face recognition performance. This process involves detecting faces within images and subsequently identifying key
facial landmarks, such as the eyes, nose, and mouth corners. These landmarks serve as reference points for aligning facial
features to a standardized configuration, compensating for variations in pose, scale, and orientation.
The code first detects faces in images using the dlib library, aligns them using facial landmarks, and saves the aligned

Page 1 of 3
images to a specified folder. The alignment process includes handling scenarios by filtering like multiple faces in an image,
images with excessive blur, images too close to the image edge, and black and white images.

After alignment, the aligned facial region is cropped and resize to 160 x 160, focusing exclusively on the facial features
while excluding irrelevant background information. Facial alignment ensures that facial images are uniformly represented
for better face recognition.
Fig. 3.3 Before and After Alignment
3.1.4 Augmentation:
Data augmentation helps to increase the diversity of the dataset. Data augmentation techniques such as rotation, flipping,
and zooming are applied to generate additional training samples. A class called `AUGMENTATION`, designed to augment
images in a dataset for enhancing training data diversity. It contains methods to load aligned images from the dataset and
augment them using specific transformations like brightness and contrast adjustment.

The augmentation is performed using the Keras `ImageDataGenerator` class, which generates batches of augmented
images based on the specified augmentation parameters. For each class in the dataset, the script checks the number of
images and calculates the number of new images needed to reach the target number. It then iterates over batches of
augmented images, converting them to RGB format and saving them to disk. Once the desired number of augmented
images is reached for each class, the augmentation process is completed. This process is necessary to balance each dataset
class to prevent any bias on model training.
3.1.5 Model training:
The training is done using the Facenet pre-trained model and Deep neural network architecture. It first checks for the
existence of an "Aligned Dataset" folder, which should contain aligned face images for training. Theen the system facilitates
the loading of aligned faces, extraction of embeddings using Facenet, and training of the DNN model. The process involves
iterating through each class (i.e., each individual's face data), extracting embeddings for each face image, and storing them
along with their corresponding labels.
The script then splits the data into training and testing sets, constructs a neural network model with three hidden layers,
compiles it with an Adam optimizer, and trains it using the extracted embeddings and labels. Early stopping is employed to
prevent overfitting, and the trained model is saved for future use. On successful completion of training, a message box
confirms the successful training of the model.
After the model is trained, “Upload Model” option is clicked. The system provides a mechanism for compressing,
uploading, and managing machine learning model files to Google Drive. It compresses the contents of a directory into a
ZIP file and then uploads it to a specified Google Drive folder, managing Google authentication and credentials. If there's
an existing file with the same name in the folder, it deletes it before uploading the new file. It also checks for internet
connectivity before initiating the upload process. If the internet is unavailable, it displays a message box informing the
user, ensuring a smooth and error-free upload operation. The code uses Google Drive APIs for file management and
PyDrive for authentication and operations. After successful uploading, Success message will be shown. We can check the
model which had been uploaded in Google drive as in Fig.3.5

Fig. 3.4 Model Training Status

Matched Source

Similarity 3%
Title:Prediction in Azure Machine Learning
Jul 16, 2020 — The Score Model. After the model is trained, let us see how the model is predicted using Machine Learning -
> Score -> Score Model control.
https://www.sqlshack.com/prediction-in-azure-machine-learning/

Page 2 of 3
Page 3 of 3
 PLAGIARISM SCAN REPORT

Date 2024-05-05

0% 100%
Words 899
Plagiarised Unique

Characters 5846

Content Checked For Plagiarism

3.1.6 Spoof detection:

A python code trains a deep neural network (DNN) model to detect spoof faces, focusing on distinguishing between real
and fake images. It first loads images from specified folders, either detecting faces in them using dlib's face detector or
assuming faces are already detected, and then resizes them to a fixed size (160x160 pixels). These images are labelled as
real or fake (1 or 0, respectively) based on their source folder. The dataset is split into training and testing sets, and a DNN
model is defined using Keras. The model architecture consists of dense layers with rectified linear unit (ReLU) activation,
batch normalization, dropout regularization, and L2 regularization to prevent overfitting. The Adam optimizer is employed
with a custom learning rate, and early stopping is utilized to prevent overfitting during training. Finally, the trained model
is saved for use in spoof detection of face recognition. The trained model predicts the label (real or fake) of each ROI, and
the prediction confidence.
3.1.7 Face recognition:
This code is a face recognition system with spoof detection, implemented using OpenCV, TensorFlow, and Raspberry Pi
GPIO controls. It initializes a webcam for real-time face detection and recognition, using MediaPipe's BlazeFace for face
detection and Keras FaceNet for facial embeddings. This embedding is used by DNN classifier to compare with stored
embeddings. Which checks for both recognized and unrecognized faces, sending email alerts for various scenarios: when a
recognized face with a high confidence score is detected, when a spoof (fake) attempt is made, or when an unrecognized
face is repeatedly observed. The GPIO pins control various hardware components, such as LEDs and relays, to indicate
system states and trigger physical responses. The system also supports toggling face recognition and email notifications
through user inputs and has mechanisms for reloading models from cloud and managing downloads for maintaining up-
to-date facial data.

3.1.8 Email system:

A simple GUI application using Tkinter is created for validating and updating an email address which is part of user friendly
interface. It creates a window with a label prompting the user to enter their email address and an entry field for input.
Upon clicking the "Enter" button, the entered email address is validated using a regular expression pattern to ensure it
conforms to the standard email format. If the entered email address is valid, a success message is displayed. Then the
saved Email address user gets notified. If the email address is invalid, an error message is shown, prompting the user to
enter it again. Once a valid email address is entered and successfully saved, the GUI window is closed. The saved email
address will be uploaded to Google drive, from drive Raspberry pi gets the email address for transferring information.
For sending mail, A Python script defines a function `send_mail` for sending email notifications. It uses the SMTP protocol

Page 1 of 2
to connect to a Gmail SMTP server (`smtp.gmail.com`) on port 587 for sending emails. The function takes three parameters:
`text_message` (the body of the email), `img_req` (a flag indicating whether an image attachment is required), and `count`
(a delay count before sending the email). It reads the sender's email address, receiver's email address, and SMTP password
from configuration variables. The function creates an email message object with the specified subject, sender, receiver, and
message body. If an image attachment is required, it reads an image file and attaches it to the email. Finally, it sends the
email using the `smtplib` library.

Fig. 3.6 Email Address change Gui and Status

3.1.9 Hardware:

Fig. 3.7 Hardware Connections

The Raspberry Pi 4 Model B 4GB variant is at the heart of a face recognition-based security system. It connects to a
webcam, which captures the facial data for recognition. A push button serves as the activation trigger for the face
recognition program, allowing the system to identify individuals and determine access permissions. The relay module used
in this system operates in an active-low state and is responsible for controlling a solenoid that requires 12V to operate. To
interface with the relay module, the system uses a GPIO pin from the Raspberry Pi. However, because the GPIO pins output
only 3.3V, which isn't enough to activate the 5V relay module, a BC547 transistor is employed as a switch.
In this setup, the base of the BC547 transistor connects to GPIO pin '27' of the Raspberry Pi, the collector is connected to a
5V source, and the emitter is grounded. The use of a transistor here allows the low-voltage signal from the GPIO pin to
control the higher-voltage circuit needed for the relay module. When the GPIO pin is inactive, no current flows to the
transistor's base, keeping the transistor in an "open" state. In this state, the 5V current from the collector flows to the relay
module's INPUT pin, keeping it switched off, which locks the door.
When the push button is pressed, activating the GPIO pin, current flows to the transistor's base, causing it to "close." This
redirects the 5V current to ground, resulting in 0V at the relay module's INPUT pin. This low signal activated the relay,
unlocking the door. Additionally, LED lights are incorporated to indicate system status: one light signifies when the
recognition code is running, another when the door is unlocked, and another when access is denied. This setup ensures
secure and controlled door access based on facial recognition, with clear visual indicators of system status.

Matched Source

No plagiarism found

Page 2 of 2
 PLAGIARISM SCAN REPORT

Date 2024-05-05

0% 100%
Words 879
Plagiarised Unique

Characters 6107

Content Checked For Plagiarism

CHAPTER 4

RESULT AND DISCUSSION

4.1 User Interface (from user end)

The main GUI consists of five different operations, which includes Adding data, removing data from dataset, training and
uploading of model and changing email address. All the operations work effectively when selected. When the user wants
to close the program, user must select “Exit” button which terminates the program.
To register a user's face, select the "Add Dataset" button on the GUI. When you click this button, a new GUI window opens,
where it is prompted to enter personal information in the input fields.
Clicking "Capture Images" activates the webcam. Press 'C' to capture a total of five images. These images are then saved
and stored in the ‘Dataset’ folder. Then after model training, the model is successfully uploaded.
4.2 Raspberry Pi Standalone System
The monitoring system was implemented successfully in Raspberry pi 4B. It is capable to run 24/7 if proper cooling is
provided.
4.2.1 Face Recognition
The camera module attached to a Raspberry Pi operates continuously, providing a constant surveillance feed. The system
remains in a standby state until a user activates it by pressing a push button and its status shown in Fig.4.1. Once activated,
the camera begins running a facial recognition program to identify individuals. If the system detects a person attempting
to gain entry and their facial data does not match any existing records in the database, the system identifies this person as
an unrecognized. As a security measure, an email notification is immediately sent to the designated user to alert them of
the unrecognized entry attempt as shown in Fig.4.3.

On the other hand, if the recognition program identifies the person and matches their facial data with an existing record in
the database, additional checks are performed to determine the legitimacy of the image captured by the webcam. If the
image is authentic, suggesting that it's not a photo or a manipulated image but a real person, the system proceeds to
unlock the door, granting the recognized individual entry. Concurrently, a confirmation email is sent to the registered email
address to inform the user that a recognized individual has entered and the door has been unlocked as shown in Fig.4.2.

However, if the system determines that the image is not genuine, which could be due to various reasons like the use of
photos, videos, or other spoofing techniques to gain unauthorized access, the system recognizes this as a 'spoof' attempt.
In this scenario, the door remains locked, preventing unauthorized entry. Additionally, an email is sent to the registered
email address to alert the user of a potential security threat due to a spoofing attempt in Fig.4.3. This comprehensive
security system ensures that only authorized individuals can only access while providing real-time alerts to users for any
unusual or unauthorized activity.

Page 1 of 3
This system checks for updates each minute and downloads a compressed model file from Google Drive, extracting it
locally if there's a newer version. It establishes Google Drive authentication, retrieves a list of files from a specific folder,
and compares the remote file's last modified timestamp with a locally stored timestamp. If the remote timestamp indicates
a newer version, the code downloads the file, updates the local timestamp, and extracts the contents of the ZIP file to the
current working directory. If there's no update, it indicates that the local version is up to date. This process ensures that the
latest version of the model is always available locally. Message displayed in raspberry pi screen regarding updated model
and email address is shown in Fig 4.4.

When there is not internet connectivity, raspberry pi can work effectively, it uses recent model stored in local database, it
stores all the status information locally. When internet is available, it checks for new model and updates it locally. Local
stored system log is shown in Fig 4.5.

CHAPTER 5

CONCLUSION

This project successfully developed a secure access control system using facial recognition on a Raspberry Pi. The system
continuously monitors the environment and initiates facial recognition with spoof detection upon interaction. Real and
recognized individuals are granted access, with clear status notifications provided to the user. A user-friendly interface
allows for managing the system remotely via a laptop. The deep learning-based model utilizes techniques like image
alignment, cropping, embedding extraction, and classification for accurate recognition.
The system incorporates cloud storage for model redundancy and remote updates, ensuring continued functionality even
in case of network failures. This project demonstrates the feasibility of a cost-effective and secure access control solution
using readily available hardware and open-source software libraries.
Key Achievements:
• Developed a robust facial recognition system with integrated spoof detection.
• Implemented a user-friendly interface for remote system management.
• Integrated cloud storage for model redundancy and network resilience.
• Showcased the capabilities of Raspberry Pi for real-world security applications.
Future Work:
• Explore incorporating additional security measures like multi-factor authentication.
• Enhance the system's scalability to accommodate a larger number of users.
• Investigate the use of more advanced deep learning architectures for improved recognition accuracy.
• Develop a mobile application for user interaction and system management.

Page 2 of 3
Signature of the Guide
Name : Dr. Rakesh Kumar Karn (Associate Professor)

Student Reg.No:124004347 Student Reg.No:124004171 Student Reg.No:124004140

Name : Varun A Name: Madesh Balu Name: Karunaesh RA

Matched Source

No plagiarism found

Page 3 of 3