Drowsiness Drowsiness Detection System

Using DeepLearning
Issac.J
ABSTRACT:

To ensure the Transportation safety, this proposed system focus on driver’s safety and aims
on reducing the number of accidents which happens due to driver’s fatigue. Most of the
accidents happens due to Drivers Drowsiness. Atleast 20% of all the road accidents happens
due to drowsiness of the driver, which in some cases results in loss of life. Road safety
deteriorates due to these unnecessary accidents. Tired eyes indicates the tiredness of the
driver. The Proposed System focuses on this issue. In my research, a system is built to avoid
this category of accidents using python and CNN model. This system analyses the eyes of the
driver to conclude whether he is sleepy or not. OpenCV is used for capturing the face, while
the model is used to recognize whether the driver is in the safe state or sleepy state. A dataset
of 84000 images has been used to train the CNN model.
Keywords – Transportation safety, CNN, OpenCV, dataset
INTRODUCTION:
The National Highway Traffic Safety Administration(NHTSA) estimates that every
year about 100,000  drowsy-driving crashes result in nearly 800 fatalities and about 50,000
injuries. The real number may be much higher. Driving after going more than 20 hours
without sleep is the equivalent of driving with a blood-alcohol concentration of 0.08%. To
provide security to driver, the vehicles are assisted with automated safety system that alerts
driver by sounding alarm. If a person is not concentrating on driving or making normal
movements while driving, it will reflect in his driving quality. Due to a lack of sleep and
irregular sleep cycles, humans tend to feel drowsy at any time of the day. The Objective of
the system is to lessen the accidents which happens in this category. There are numerous
techniques for face detection which is the first step in the process. Realtime web camera /
system camera will acquire the live video frames and give it as input to the CNN classifier.
CNN classifier is trained with a huge dataset of 80000images. The Web Cam should be
mounted in front of the driver in an angle which provides good view of the driver. Else the
Classifier will perform poorly. Then the Trained model will recognize whether he is steady
state or sleepy state.
The Paper presents with 95% accuracy. The System provides an Arithmetic based method to
detect the driver drowsiness. There are totally 2 stages involved in the detection : Face
detection, Eye detection. This system uses a technique called Eye Aspect Ratio(EAR). It
divides the eyes into x and y axis, then computes the coordinates. With Respect to the
coordinate value, it is concluded whether the driver is sleepy or not. This System is capable
of performing in low light and even when the driver wears spectacles.

1
 (a) Performance Analysis of the CNN model with other models

Blue – KNN
Red – SVM
Green - CNN
Accuracy (%) in 1 sec Accuracy (%) in 2 sec Accuracy (%) in 4 sec

Duration

The above graph shows the accuracy percentage of each model. Obviously CNN model
outperforms everyother models.

RELATED WORKS:
S. Kailasam [1] in his paper introduced a system which can detect driver drowsiness and also
prevents the driver from speeding. The Camera is fixed in front of the driver to capture the
eyes and it observes it continuously. A speed control system is used to check the speed of the
vehicle. This system automatically slows down the vehicle when the vehicle starts speeding.
The camera is a night vision camera used to read the face when there is no proper light. The
Alarm is sounded
A. Walavalkar [2] in his paper designed a system in which image processing is done by
extracting the features of face, and identifies whether he is being distracted or not. An
important feature of this system is real-time analysis for alerting drivers and avoiding
accidents caused by distracted driving. PERCLOS algorithm is used to detect the driver’s
distraction. Distraction is computed based on the difference between distance from current
pupil position and ideal pupil position to eye corner. The Accuracy obtained is 75-80%.
B. K. Rajan [3] in his paper designed a system which detects more accurately and precisely
the fatigue condition of the driver. This system is based on image processing. Compared to
physiological signal-based techniques image-based technique is easier to implement. The
face was detected and tracked using both viola jones and KLT algorithm. The KLT algorithm
is used to record the features in the detected face. This System sounds an alarm if the driver
falls to sleep state.
R.Roopalakshmi et al [4] in his paper designed system which can detect driver’s drowsiness
in the basis of blink count. Number of eye blinks are Computed. It continuously analyses the
eye movement. A camera is fixed in front of the driver. A vibrator is also installed in this
system. When the System detects the driver sleeping, the vibrator gets initiated which wakes
up the driver. This implemented on OpenCV and Raspberry Pi.
P. Kanani, [5] in his paper focuses on a System Capable to perform various tasks such as
detection of the car lane changes, alcohol detection , calculating proximity of the objects on
the road. When the driver is not stable/distracted, the alarm goes off. EAR (Eye Aspect Ratio)

2
is used to compute the ratio of openness/closeness of eyes. A pretrained Histogram of
Oriented Gradients + Linear Support Vector Machine Object Detector for Facial Detection is
utilized. The system prevents the engine from starting if the alcohol sensor detects the alcohol
level to be higher than the permitted levels.
Sneha Roy et al [6] in his paper used computer vision to identify eye blinks. And also can
detect a driver's other activities while driving by detecting his body movements. several
technologies are used: 1) driver eye monitoring 2) steering pattern monitoring 3)
psychological measurement. All the monitoring and measurement pieces of information are
stored in the firebase. The System doesn’t detect Yawning. It determines whether the eyes are
in an open position or closed using computer vision. Also in the system, we can check the
body movement of the driver with a set of pressure sensors installed in the drivers seat. We
measure steering pattern deviation with the help of a potentiometer. When drowsiness
detection happens, and if an abnormal situation is found, the system produces an alert sound
and sends a message to the driver which is displayed on the screen.
Aayush Srivastava et al [7] in his paper designed a drowsiness system to detect the
drowsiness of the driver using the Deep Learning model CNN(convolutional neural network).
Dataset is downloaded from MRL_Eye data site which contains 84000 images containing of
37 different subjects. CNN model is built with four convolution layers, three pooling and two
dense layers is done and then training the above model with different configurations to
achieve best results. OpenCV is used to capture the frames. Later the frames are given as
input to the model. So that it will recognize whether he is sleepy or not.
S. Mohanty et al [8] in his paper designed the driver drowsiness system with dlib approach.
Face detector which is based on Histogram of Oriented Gradients (HOG) was implemented.
The Concept of EAR(Ear Aspect Ratio) is used to monitor the blinking pattern and
MAR(Mouth Aspect Ratio) is used to determine whether the driver is yawning. The Camera
is mounted on the dashboard of the car and can able to capture driver’s upperbody, face,
hands. The Dlib model recognizes the tiredness of the driver.
N. Vasudevan et al [9] in his paper proposed a system based on Viola and Jones method. The
camera is mounted such that it points the driver directly. The Viola-Jones method is used to
segment the face and eye of the person from the camera. Then the PERCLOS algorithm is
used to detect the eye closure. Eye Closure has 3 states open, closed, partially closed. If the
Driver’s Eyes are fully closed, then the alarm rings.
K.Sahithya et al [10] in his paper designed a system which detects drowsiness of the driver
using Eye Blink Sensor (IR based) and Uses the Combination of Viola Jones and KLT
algorithm.. This System also has Alcoholic Sensor, to detect whether the person is stable or
not. OpenCV is used for capturing the image through the sensor. This System Shows warning
in a liquid crystal display screen and activates the alarm.
PROPOSED SYSTEM:
To Increase the accuracy of the proposed system, a dataset has been downloaded
from mrl eye website which has 84000 images which contains a mixture of open eyes or
closed eyes. It enhances the performances of the system and increases the accuracy over 95%.

3
 A High Quality webcam makes the classifier to identify the drowsiness of the
driver easily. So overall, it improves the system.

METHODOLOGY:
The Proposed System built to detect the drowsiness of the driver by taking eyes
into account. The Software is entirely built with python. Tiredness can be identified easily by
looking at the eyes. There are totally 2 states for Eyes: Open Eyes, Closed Eyes. This System
recognizes and concludes into one of the states. If eyes of a person are closed for long time,
then the system concludes that the person is sleepy and triggers an alarm to wake up the
driver. If eyes are Opened, then the system remains silent. A Web camera is installed in front
of the driver to record live video frames of the face. Deep Learning model
CNN(Convolutional Neural Network) is implemented in this system. EAR(Eye Aspect Ratio)
technique is used. It computes the eye closure and by that basis it recognizes whether the
driver is sleep state or fresh state.
A) Concepts and Components:
For Capturing the Face of the driver OpenCV, a well know python library used.
Keras which is a deep learning python package is implemented for neural networks.
Pygame package is used to ring the alarm sound. Initially a Script is written to split
the dataset into open eyes and closed eyes images into separate folders. Eye Aspect
Ratio(EAR) is a technique used to compute how much the eyes are closed or opened.
In the backend It depicts the eyes into x and y axis and computes the co-ordinates.
EAR value obtained at the application’s end is compared to a threshold value of 15.
The driver is regarded to be sleepy if the EAR value(Score Value) is more than the
threshold value. In that case an alarm would be triggered alerting the driver. The
Classifier going to be used is CNN(Convolutional neural network). CNN model is
trained with huge dataset to get the essence called accuracy.

B) Facial Marking:

In the Python Script. A forloop is used to draw a rectangle in the face.

Another forloop can be written to draw rectangle on the eyes. But it is avoided as the
Output Screen looks Clumsy.
Shape Predictor models can also be used to draw the rectangles, but it
occupies CPU more ,so it is also avoided in this system.

4
Architecture Diagram:

1. Data Preprocessing and Data Cleaning:

Data Preprocessing:

First Step in data Preprocessing is the splitting of data. Initially the dataset contains a
mixture of open eyes or closed eyes. The open and closed eyes images should be
splitted and stored in different folders for further testing and training of the model.

Second Step data preprocessing is the tuning of the input size. The dataset contains
images with different dimensions. If the input is given without any
tuning(modifications) then the classifier will perform poorly. So the raw images can
converted into a specific dimension which is 80*80.

Third Step in data preprocessing is to convert the raw data to Gray Scale image.
When giving the input as coloured data, the classifier may perform poorly. The input
for the classifier is given as gray scale images.

Data Cleaning:

5
 Data cleaning is a process of removing duplicate images, removing images that are
corrupted, removing irregular images etc…

2. Training and Testing the model:

Training:
CNN model is a neural network model used in this system which will
recognize whether the driver is in sleep state or fresh state. CNN model has many
architectures such as ZF Net, ResNet, VGG16, VGG19, etc…. This System uses
InceptionV3 Architecture. This Architecture has a better performance compared to
other CNN Architectures. It was Developed by Google. And as the name suggests it
is version 3. The Architecture is not built in this system itself, but has been imported
to make use of it.

The DataSet is downloaded from the mrl website. It contains 84000

images as a bundle with mixture of open eyes and closed eyes images. Training with
a huge dataset will give lot of accuracy. When the training starts, the dataset is
broken down to batches, the images are passed onto the algorithm by batches. After
training completed, a model.h5 file should have been created.In the Main module,
the model is loaded and made ready to be used.

Testing:

Then the model evaluation takes place. Most model-performance measures

are based on the comparison of the model's predictions with the (known) values of
the dependent variable in a dataset. The model-performance measures are: F1
measure, Accuracy, Recall.

 F1 measure:
F1 Score is the weighted average of Precision and Recall. F1 is
usually more useful than accuracy.

 Recall:
Recall is the ratio of True Positives to total number of predictions.

Recall = TruePositives / (TruePositives + FalseNegatives)

 Accuracy:
Accuracy is defined as the ratio of Number of Correct Predictions to
the Total number of Predictions.

Accuracy = Number of correct predictions/Total number of

predictions.

3. Computation and Detection:

Computation:

6
 Eye Aspect Ratio(EAR) is the technique used to compute how much the eyes
are closed or opened. In the backend It depicts the eyes into x and y axis and
computes the co-ordinates. EAR value obtained at the application’s end is compared
to a threshold value of 15. The driver is regarded to be sleepy if the EAR
value(Score Value) is more than the threshold value. In that case an alarm would be
triggered alerting the driver. If the EAR value stays less than 15, then the system
stays silent.
Detection:
For Capturing the Face of the driver OpenCV, a well know python library
used. This realtime video is given as input to the well-trained CNN model. The
model recognizes whether he is in sleepy state or fresh state. When it detects that the
person is sleeping, then an alarm would be sounded which wakes up the driver.

PseudoCode:
Program 1: (Data Splitting):
IF i[4] is 0,
Then copy(path1)
ELSE IF i[4] is not 1,
Then copy(path2)

OUTPUT:

(a) Open Eyes:

7
(b) Closed Eyes:

(c) Partially Opened:

8
CONCLUSION:
In this paper, to avoid the intensity of accidents happens everyday due to driver drowsiness,
Driver Drowsiness detection system has been built and the system is designed using Python
with an integrated alert system. A technique called (EAR)Eye Aspect Ratio was
implemented. The dataset which has 84000images was downloaded from the website
successfully and was used to train the CNN model which adopts InceptionV3 Architecture.
The model can able to successfully recognize the sleep state from the fresh state. The
Proposed framework recognizes drowsiness if the eyes are shut for 2 seconds. It also
concludes the driver is in sleep state when the 75% of eyes is closed. From the analysis, the
system can generate an accuracy of over 95%.

REFERENCES:

[1] S. Kailasam, M. Karthiga, R. M. Priyadarshini, K. Kartheeban and K. Anithadevi,

"Accident Alert System for Driver Using Face Recognition," 2019 IEEE International
Conference on Intelligent Techniques in Control, Optimization and Signal Processing
[2] A. Walavalkar, S. Singh, R. Salian and V. Shrivastava, "Driver Distraction Monitoring
and Alerting System," 2019 1st International Conference on Advances in Information
Technology (ICAIT), Chikmagalur, India, 2019
[3] H. Joseph and B. K. Rajan, "Real Time Drowsiness Detection using Viola Jones & KLT,"
2020 International Conference on Smart Electronics and Communication (ICOSEC), Trichy,
India.

9
[4] R.Roopalakshmi, K.Supriya “Driver Drowsiness detection system using based on visual
features” 2018 Second International conference on inventive Communication and
Computational Technologies (ICICCT).
[5] A. Raorane, H. Rami and P. Kanani, "Driver Alertness System using Deep Learning,
MQ3 and Computer Vision," 2020 4th International Conference on Intelligent Computing
and Control Systems (ICICCS), Madurai, India, 2020
[6] Sneha Roy et al , “Driver Drowsiness and Movement Detection based Vehicle Safety
System.” IJERT 2022
[7] Aayush Srivastava et al , “A Deep Learning Approach To Detect Driver Drowsiness”
IJERT  17-05-2021.
[8] S. Mohanty, S. V. Hegde, S. Prasad and J. Manikandan, "Design of Real-time Drowsiness
Detection System using Dlib," 2019 IEEE International WIE Conference on Electrical and
Computer Engineering (WIECON-ECE), Bangalore, India, 2019.
[9] K. S. Sankaran, N. Vasudevan and V. Nagarajan, "Driver Drowsiness Detection using
Percentage Eye Closure Method," 2020 International Conference on Communication and
Signal Processing (ICCSP),Chennai, India, 2020.
[10] A. Subbarao, K.Sahithya “Driver Drowsiness Detection System for Vehicle Safety”
International Journal of Innovative Technology and Exploring Engineering (IJITEE) ISSN:
2278-3075, Volume-8, Issue- 6S4, April 2019.

10