CAPSTONE PROJECT

ATTENDANCE
SYSTEM
Using Face Detection of Students
 CAPSTONE PROJECT

Team Members

Dommaraju Nikhil Kamparaju Guru Hethish

19BCE0234 19BCI0029
Aim of the project

In this project, we aim to develop a real-time Face recognition system using Deep learning and computer vision
techniques.  

The proposed system uses a web camera to capture images of individuals and processes the images to recognize their
faces using state-of-the-art deep learning models which are trained on Resnet architecture and open datasets to
increase the accuracy of face encodings and key point descriptors.  

The system achieves high accuracy in face recognition, allowing it to accurately and efficiently take attendance in real
time. 

Overall, the proposed real-time attendance management system using deep learning techniques offers a more efficient,
accurate, and reliable alternative to traditional attendance management methods. The system's ability to recognize
faces accurately, even in challenging conditions, makes it a valuable tool for various industries. The following sections
of this document will provide a detailed explanation of the methodologies used in the project and a step-by-step guide
on how to implement the system.
Motivation

Attendance monitoring is an essential part of running any kind of organisation, from businesses to schools. Traditional attendance systems
are prone to mistakes and are simple targets for manipulation, such as manual paper-based or RFID-based systems. A more dependable and
secure alternative is a face detection and recognition-based attendance system. The solution assures precise attendance tracking and stops fraud
by utilising cutting-edge computer vision algorithms to identify and authenticate people based on their distinctive facial traits. This method is
also contactless and doesn't require physical contact, making it a secure and practical choice in the pandemic-driven world of today. Overall,
putting in place a face detection and recognition-based attendance system is a wise investment that may boost any organization's productivity,
accuracy, and security.

Traditional student attendance marking technique is often facing a lot of trouble. The face recognition student  attendance system emphasizes
its simplicity by eliminating classical student attendance marking technique such  as calling student names or checking respective identification
cards. There are not only disturbing the teaching  process but also causes distraction for students during exam sessions.

Thus, face recognition student attendance system is proposed in order to replace the manual signing of the  presence of students which are
burdensome and causes students get distracted in order to sign for their  attendance. Furthermore, the face recognition based automated student
attendance system able to overcome the  problem of fraudulent approach and lecturers does not have to count the number of students several
times to  ensure the presence of the students. Hence, there is a need to develop a real time operating student attendance  system which means the
identification process must be done within defined time constraints to prevent omission.
 CAPSTONE PROJECT

Objectives

The objective of this project is to develop a face recognition based automated

student attendance system. Expected achievements in order to fulfil the objectives
are:
1.To detect the face segment from the video frame.
2.To extract the useful features from the face detected.
3.To classify the features in order to recognize the face detected.
4. To record the attendance of the identified student
PAPER-1
Face recognition based attendance marking system
Authors: senthamilselvi,  chitrakala, antony jenitha

SUMMARY
This system uses camera to capture the images of the employee to do face  detection and recognition. The
captured image is compared one by one with the face database to search  for the worker‟s face where attendance
will be marked when a result is found in the face database. The  main advantage of this system is where
attendance is marked on the server which is highly secure where  no one can mark the attendance of other.
Moreover, in this proposed system, the face detection  algorithm is improved by using the skin classification
technique to increase the accuracy of the  detection process. Although more efforts are invested in the accuracy of
the face detection algorithm,  the system is yet not portable. This system requires a standalone computer which
will need a constant  power supply that makes it not portable. This type of system is only suitable for
marking staff‟s  attendance as they only need to report their presence once a day, unlike students which require
to  report their attendance at every class on a particular day, it will be inconvenient if the attendance  marking
system is not portable. Thus, to solve this issue, the whole attendance management system can be  developed
on an portable module so that it can be work just by executing the python program.
PAPER-2
Smart attendance system using face recognition
Authors : ghalib al-muhaidhri [1], javeed hussain[2]

SUMMARY
The main purpose of this project is to built automated attendance system using  raspberry-pi 3b+
with opencv/python libraries and recognizer algorithm have been  implemented. The proposed system can
be implemented in any field where attendance  system is present and plays a vital role. In addition, as  the
project objectives and the design criteria all met, it’s greatest to say this project is an engineering solution for
all university and colleges to track and manage the attendance.
PAPER-3
Development of an automatic class attendance system using cnn-based face recognition
Authors. : soumitra chowdhury1, sudipta nath2, ashim dey3 and annesha das

SUMMARY
This paper represents the development of a face recognition based automatic  student
attendance system using convolutional neural networks which includes data entry, dataset
training, face recognition and attendance entry. The system  can detect and recognize multiple
person’s face from video stream and automatically record daily attendance.The proposed
system achieved an average  recognition accuracy of about 92%.Using this system, daily
attendance can be  recorded effortlessly avoiding the risk of human error.
PAPER-4
Face recognition-based mobile automatic classroom attendance
management system
Author : refik samet , muhammed tanriverdi

Summary

This paper aims to propose a face recognition-based mobile automatic classroom

attendance management system needing no extra equipment. To this end,a filtering system based
on euclidean distances calculated by Three face recognition techniques, namely
eigenfaces, fisherfaces and local binary pattern, has been developed for Face recognition. 
The proposed system includes three different mobile applications for teachers, students, and parents
to be installed on their smart phones to manage and perform the real-time attendance-taking process.
PAPER-5
Face recognition with local binary patterns
Author: timo ahonen, abdenour hadid, and matti pietik¨ainen

Summary
This paper tells about the novel approach to face recognition which considers both  shape and
texture information to represent face images. The face area is first divided  into small regions from which
local binary pattern (lbp) histograms are extracted  and concatenated into asingle, spatially enhanced
feature histogram efficiently  representing the face image. The recognition is performed using a
nearest neighbour  classifier in the computed feature space with chi square as a dissimilarity measure. 
Extensive experiments clearly show the superiority of the proposed Scheme over all considered methods
(pca, bayesian intra/extrapersonalclassifier and  elastic bunch graph matching) on feret tests which include
testing the robustness  of the method against different facial expressions, lighting and aging of the
subjects.  In addition to its efficiency the simplicity of the proposed method allows for very fast  feature
extraction
PAPER-6
Attendance monitoring system based on face recognition

Authors :N. Nalini ,emerald tuladhar, avinash shah,anusha hegde

Summary:

We have decided to enhance the face recognition for taking attendance by adding a number
of different features.The attendance system will consist of a mechanism that extracts facial features and then
converts them into a numeric representation which can be classified or in this case, recognized. A manual
trigger mechanism to start a timer which records a timestamp in order to maximize efficiency of
the attendee.Moreover, the facial recognition process will take place periodically in 15 minute intervals
to emphasize on attentiveness and discipline even whilst attending a class or job online. Understanding
the scenario, to make the different task of institutions & organizations fruitful, face recognition feature comes
into use that takes out facial features & changes into numeral format. An automatic mail system sends mail to
all the students or staff.
System Architecture :
 The system architecture for the real-time attendance using deep learning techniques consists of
several components: 

• Input(WebCam): The webcam is used to capture real-time images of individuals . 

• Face Detection Module: Histogram of Oriented Gradients(HOG) and pre-trained deep learning model Which was
Developed by Dlib library .

• Feature Extraction Module: Resnet Architecture Along with Dlib library modules .

• Face Recognition Module: The face recognition module compares the extracted features with the features stored
in the database to identify the individual . 

• Database: The database stores the unique facial features of individuals and their attendance records. 

• Attendance Management System: The attendance management system records the attendance of individuals and
generates reports. 
Libraries used:
   Numpy : It provides support for large, multi-dimensional arrays and matrices, along with a large
collection ofmathematical functions to operate on these arrays. Some of the main features of
using numpy are making N dimensional arrays , broadcasting them, performing mathematical functions.

   Cv2 : The cv2 module in Python provides a wide range of functions and tools for image and video processing.
 
   Multiprocessing : The "multiprocessing" module in Python provides support for parallel processing, allowing
you to execute multiple tasks concurrently on different CPUs or cores. This can improve the performance and
efficiency 
  
   Itertools: It is a module that provides a set of functions for working with iterators. When you import itertools,
you gain access to these functions, which you can use in your code.

   Dlib:- It is a general purpose cross-platform software library written in the programming language C++
Models Used:
Face Detection
Histogram of Oriented Gradients (HOG) in Dlib:
HOG (Histogram of Oriented Gradients) is an important algorithm used for feature extraction in object detection tasks, including
face detection. In the context of the Dlib library, HOG refers to the HOG-based face detector implemented in Dlib.

The HOG algorithm in Dlib utilizes the HOG feature descriptors to capture local gradient patterns in an image. These gradient
patterns encode important information about object shapes and structures. The HOG-based face detector is designed to identify
facial features and localize faces in images or video frames.
The 2825 photos in the training set were taken from the LFW dataset and manually annotated by Davis King, the creator
of Dlib.

MMOD Convolutional Neural Network in Dlib:

The MMOD algorithm in Dlib combines the power of HOG (Histogram of Oriented Gradients) features with machine
learning techniques, specifically Structural Support Vector Machines (S-SVM), to perform object detection. This
combination allows the MMOD detector to effectively learn and recognize objects based on their visual characteristics.
Dlib provides both the HOG face detector and the MMOD face detector as separate implementations.  HOG is generally faster but
may be slightly less accurate compared to the MMOD face detector.
we used combination of both detectors to achieve a balance between speed and accuracy.  

HOG detector is used for an initial face region proposal and then apply the MMOD detector to refine the face detection results.
 Facial Landmarks:
shape_predictor_68_face_landmarks :
The dlib is used to estimate the location of 68 coordinates (x, y) that map the facial points on a person's face, as
seen in the image below. It is a landmark's facial detector using pre-trained models

shape_predictor_5_face_landmarks : 
The corners of the eyes and the bridge of the nose are identified using this five point landmarking approach. It was
trained using the 7198-face dlib 5-point facial landmark dataset. Images from the internet are downloaded, then
annotated with dlib's imglab tool to build the dataset. 
Face encoding(Recognition):
dlib_face_recognition_resnet_model_v1:

Dlib provides a top-notch deep learning-based facial recognition system. Modern accuracy is provided by the facial
recognition algorithm used by Dlib. More specifically, the model's face labelling accuracy in the wild database is
99.38%.
This algorithm's implementation was developed by altering the ResNet-34 network.
Rebuilding a neural network with 29 convolutional layers using the standard ResNet structure after removing several
layers. It represents face images as 128 dimensional vectors and anticipates inputs with dimensions of 150x150x3.
​
Explanation of Working Principle:​
​The first step in this process is face detection:-
This is done by using the Histogram of Oriented Gradients (HOG) feature descriptor . 
​
The HOG feature descriptor captures the local gradient information of an image by dividing it into small regions and
computing the histogram of gradient orientations within each region. The resulting feature vector can be used to represent the
appearance of an object in the image.

The image is divided into 8x8 cells to offer a compact representation and make our HOG more robust to noise. Then, we
compute a HOG for each of those cells.histogram is created using gradient direction, gradient magnitude.

Additionally we have used MMOD (Max-Margin Object Detection) based face detector.
 Once the faces are detected, they are cropped and aligned to ensure that they are properly positioned for further
processing.

   We have used "shape_predictor_68_face_landmarks" which is a machine learning model file for facial landmark
detection developed by the Dlib library.it is used to detect the facial landmarks and align the faces in the input images,
which can improve the accuracy and robustness of the face recognition algorithm. 
   The landmark locations can also be used to extract additional features from specific regions of the face, such as the eyes,
nose, or mouth, which can improve the discriminative power of the face recognition system.

   The next step is face encoding. This involves extracting features from the face that can be used for identification and
verification. To accomplish this, Resnet model has been used, It takes as input an image of a face and the facial
landmarks detected by the "shape_predictor_68_face_landmarks" model.
   The output of the neural network is a 128-dimensional feature vector that encodes the unique characteristics of the face in
a compact representation.
The 128-dimensional feature vector generated by the Resnet model can be used for face recognition 
by comparing it to other feature vectors which are already known and are stored in our Database .

Face identification involves matching the input face with one of the faces in the database. 
This is done by computing the Euclidean distance between the feature vector of the input face and 
the feature vectors of the known faces in the database. If the distance is below a certain threshold, 
the input face is considered a match with the corresponding known face.

Now the face is recognized and students attendance can be updated.

Demonstration of Expected Results and Analysis:
Conclusion and Future Development:

The Face-based Attendance System is a computer vision application that leverages face detection and face recognition
techniques to automate the process of tracking and recording student attendance. The system aims to replace manual
attendance management methods, reducing administrative efforts and improving accuracy. It utilizes algorithms such as
HOG or MMOD for face detection, shape_predictor models for face alignment, and the ResNet architecture for face
recognition. The system allows administrators to enroll students by capturing their facial features and associates them
with their unique identifiers. During attendance marking, the system captures images or video frames from cameras,
detects faces, aligns them if needed, and matches them against the enrolled student database using face recognition. The
system records attendance records, including student IDs and entry timestamps, and stores them in a database.
Additionally, it offers functionalities like generating attendance reports and managing student information. The project
involves requirement analysis, system design, algorithm implementation, database management, and user interface
development. By automating the attendance process, the system aims to enhance efficiency, accuracy, and overall
productivity in educational institutions.
References:
[1] Akbar,  Md  Sajid,  et  al.  "Face  Recognition  and  RFID  Verified Attendance System." 2018 International
Conference on Computing, Electronics & Communications Engineering (iCCECE). IEEE, 2018
[2] Rathod, Hemantkumar,  et al.  "Automated attendance  system using machine  learning  approach." 2017 
International  Conference on Nascent Technologies in Engineering (ICNTE). IEEE, 2017. 
[3] Tan and B. Triggs. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE
Transactions on Image Processing, 19(6):1635-1650, 2010.
[4] P. Viola and M. J. Jones, “Robust real-time face detection,” International Journal on Computer Vision, vol. 57, no. 2,
pp. 137–154, May 2004.
[5] C. Lin, 2005,  “Face Detection By Color And Multilayer Feedforward Neural Network”, Proc. 2005 IEEE
International Conference on Information Acquisition, pp.518-523, Hong Kong and Macau, China. 
[6]“Attendance System Using NFC Technology with Embedded Camera on Mobile Device” (Bhise,
Khichi, Korde,Lokare, 2015) 
[7]. K.SenthamilSelvi, P.Chitrakala, A.AntonyJenitha, "Face Recognition Based Attendance Marking System", IJCSMC,
Vol. 3, Issue. 2, February 2014
[8] Hu,   Guosheng,  et   al.   "When   face   recognition   meets with   deep   learning:   an  evaluation   of   convolutional
neural networks  for face recognition."  Proceedings of the   IEEE   International   Conference on ComputerVision
Workshops. 2015.
CAPSTONE PROJECT

THANK
YOU