MINI PROJECT

ON
Driver Drowsiness Detection
and Safety Alert System
Using Python
Submitted by: -

K.Yamini (18761A1228)
T.Vivek (18761A1255)
Under the Esteemed Guidance of
V.V.Krishna Reddy
ASST PROFESSOR
Department of IT

LAKIREDDY BALIREDDY COLLEGE OF ENGINEERING

(AUTONOMOUS)
Affiliated to JNTUK, Kakinada & Approved by AICTE, New Delhi
NAAC Accredited with “B+” grade, accredited by NBA, New Delhi &
Certified by ISO 9001:2015
L.B. REDDY NAGAR, MYLAVARAM, KRISHNA Dist.
DEPARTMENT OF INFORMATION TECHNOLOGY 2020-2021
 LAKIREDDY BALIREDDY COLLEGE OF ENGINEERING
(AUTONOMOUS)
Affiliated to JNTUK, Kakinada & Approved by AICTE, New Delhi
NAAC Accredited with “B+” grade, accredited by NBA, New Delhi & Certified by ISO
9001:2015
L.B. REDDY NAGAR, MYLAVARAM, KRISHNA Dist.

DEPARTMENT OF INFORMATION TECHNOLOGY

CERTIFICATE
This is to Certificate that Mini Project work entitled " Drowsy Driver Safety Alert System
Using Python” that is being submitted for the partial fulfilment of I-SEM III B. TECH in
Information technology. It is the bonafide work done by K.Yamini(18761A1228), T.Vivek
(18761A1255).During the academic year 2020-2021 and it has been found worthy of
acceptance according to the requirement of university.

Project Coordinator Head of Department

V.V.Krishna Reddy Dr. B.Srinivasa Rao
Asst. Professor Professor
Department of IT Department of IT
 ACKNOWLEDGEMENT

Behind every achievement lies an unfathomable sea of gratitude to those who activated it,
without whom it would never have been in existence. To them we lay the words of gratitude
imprinted with us.
It is with utmost pleasure that I avail this opportunity to express heart full gratitude to my
beloved guide V.V.Krishna Reddy, ASST PROFESSOR Information Technology for his
mastery supervision through all the phases of our project work with valuable suggestions and
support.His friendly and informal talks helped us to work under excellent working conditions.
We highly oblige to Dr. B.Srinivasa Rao Head of the Department, Information Technology
for his support and encouragement throughout the course of the project work.
We express our special gratitude to Dr.K.AppaRao, Principal of LAKIREDDY
BALIREDDY COLLEGE OF ENGINEERING, who made this endeavour possible. We
would also thank all the teaching and non-teaching staff members of Information Technology
department who have extended their full cooperation during course of our project.
Finally, a word of gratitude to Parents who had to be a constant source of encouragement and
love.

Team Members:
K.Yamini (18761A1228)
T.Vivek (18761A1255)
 DECLARATION

We hereby declare that the Mini Project entitled “Driver Drowsiness Detection and Alert
System using python” is submitted to JNTUK is partial fulfilment of the requirement for the
award of III (B. TECH) is an original work carried out by us.

The matter embodied in this project is a genuine work by us and has not been submitted
earlier to this university or any other university for award of any degree or diploma or prize.

Team Members:

K.Yamini (18761A1228)
T.Vivek (18761A1255)
 INDEX

Contents Page No
1.Abstract 1

2.Introduction 2-3
2.1 Project Area: Computer Vision 2
2.1.1 Basics of CV 2
2.1.2 How does CV work? 2
2.1.3 Features of CV 2
2.1.4 Python Libraries 3
2.1.5 About CNN 3

3.Proposed System 4-5

3.1 Flow of Execution of Application 4
3.2 Requirements of Project 5

4.Model Architecture 6

5.Procedure 7-8
5.1 Flowchart of Application 8

6.UML Diagrams 9-14

6.1 Use case Diagram 9
6.2 State chart Diagram 10
6.3 Class flow Diagram 11
6.4 Sequence Diagram 12
6.5 Collaboration Diagram 13
 6.6 Activity Diagram 14

7.Implentation of Code 15-18

7.1 DriverDrowsinessDetection.py 15-17
7.2 Model.py 17-18

8.Results 19-20
8.1 Closed Eyes Detection 19
8.2 Open Eyes Detection 19
8.3 Sleep Alert 20

9.Conclusion 21
10.References 22

2
 LIST OF FIGURES

Figure Name Page No

1.Process of the Application
1.1 Process of System 4
1.2 Flowchart of Application 8
2.UML Diagrams for Driver Drowsiness Alert System 9-14
2.1 Use case Diagram 9
2.2 State chart Diagram 10
2.3 Class flow Diagram 11
2.4 Sequence Diagram 12
2.5 Collaboration Diagram 13
2.6 Activity Diagram 14
3.Results of Screenshots 19-20
3.1 Closed Eyes Detection 19
3.2 Open Eyes Detection 19
3.3 Sleep Alert 20
 1. ABSTRACT

A countless number of people drive on highway day and night.Taxidrivers,busdrivers,truck

drivers and people traveling long-distance suffer from lack of sleep.Feeling sleepy is
dangerous while driving.The majority of accidents happen due to the drowsiness of the
driver.According to CRRI,40% of accidents due to driver dozing off.To avoid this
problem,we came up with an idea.The project is about alerting driver when he/she feels
drowsy by ringing an alarm.Using Python libraries like OpenCV,Keras and computer vision
technique(CNN) system is built to detect whether the driver is drowsy or not.An alarm will
be rung,if it is detected that the driver feeling drowsy.

2.INTRODUCTION

1
Drowsiness detection is a safety technology that can prevent accidents that are caused by
drivers who fell asleep while driving. The main objective is to build a drowsiness detection
and alert system that will detect that a person’s eyes are closed for a few seconds and also
will alert the driver when drowsiness of driver is detected. The risk, danger, and often tragic
results of drowsy driving are alarming. Drowsy driving is the dangerous combination of
driving and sleepiness or fatigu. Drowsiness detection technologies have attempted to prevent
such incidents by predicting if a driver is falling asleep based on various inputs. The objective
of this project is to measure the current activity of the eyes of the driver which is visualized
by the camera and we can check the drowsiness of the driver. We present a system for
drowsiness detection using CNN,OpenCV and python packages.

2.1 Project Area: Computer Vision

2.1.1 Basics of Computer Vision
Computer Vision is a multidisciplinary field that could broadly be called a subfield of
artificial intelligence and machine learning, which may involve the use of specialized
methods and make use of general learning algorithms. Computer Vision, often abbreviated as
CV, is defined as a field of study that seeks to develop techniques to help computers “see”
and understand the content of digital images such as photographs and videos. The goal of the
field of computer vision and its distinctness from image processing.

2.1.2 How does CV works?

Computer vision algorithms that we use today are based on pattern recognition. We train
computers on a massive amount of visual data—computers process images, label objects on
them, and find patterns in those objects. Deep learning represents a more effective way to do
computer vision—it uses a specific algorithm called a neural network. The neural networks
are used to extract patterns from provided data samples. The algorithms are inspired by the
human understanding of how brains function, in particular, the interconnections between the
neurons in the cerebral cortex.

2.1.3 Features of CV

The most important features of CV on which it works are object classification,object

tracking,object identification and many other features included in below:

1. Image Classification
2. Object Detection
3. Object Tracking
4. Semantic Segmentation
5. Instance Segmentation

2.1.4 Python Libraries

2
OpenCV:OpenCV is the open-source library for computer vision and image
processing tasks in machine learning.In this project,for capturing image from
webcam of pc opencv used.
Keras:A deep learning framework. Here,CNN is used which performs
extremely well for image classification purposes.
Pygame: It includes computer graphics and sound libraries.Here for alerting
driver a sound play is used from library.

2.1.5 About CNN

A Convolutional neural network is a neural network that has one or more

convolutional layers and are used mainly for image processing, classification,
segmentation and also for other auto correlated data.A convolution is essentially
sliding a filter over the input. A Convolutional Neural Network (CNN) is the
foundation of most computer vision technologies. Unlike traditional multilayer
perceptron architectures, it uses two operations called convolution and pooling
to reduce an image into its essential features, and uses those features to
understand and classify the image.

3
 3. Proposed System

In order to overcome the problem of drowsiness of the driver, we introduce a method, which
comprises of the model: the CNN model for feature extraction and OpenCV and keras for
interpreting the features across the frame. The procedure for drowsiness detection is thus as
follows: First, we extract significant CNN features from the frame which is captured from the
webcam.To capture the image from webcam a method provided from OpenCV is used. Then
features representing the sequence of the action (Alert or a Drowsy Driver) for a certain time
interval (fixed number of frames are fed to the as an input. Finally, CNN layers are used to
predict drowsiness of the frame by predicting whether the eyes are opened or closed.

3.1 Flow of Execution of Application

Figure 1.1: Process of the System

4
3.1 Requirements of Project

3.1.1 Software Requirements

i.Python-3.8
ii.OpenCV
iii. Keras
iv.Tensorflow
v.Pygame

3.1.2 Hardware Requirements

i.PC

ii.Webcam

3.1.3 Dataset used

Captures image from a camera and stores in local disk.In our project the dataset used for this
model is created by us. To create the dataset, we wrote a script that captures eyes from a
camera and stores in our local disk. We separated them into their respective labels ‘Open’ or
‘Closed’. The data was manually cleaned by removing the unwanted images which were not
necessary for building the model. The data comprises around 7000 images of people’s eyes
under different lighting conditions. After training the model on our dataset, we have attached
the final weights and model architecture file “models/cnnCat2.h5”.

5
 4. Model Architecture

The model we used is built with Keras using Convolutional Neural Networks (CNN). A
convolutional neural network is a special type of deep neural network which performs
extremely well for image classification purposes. A CNN basically consists of an input layer,
an output layer and a hidden layer which can have multiple numbers of layers. A convolution
operation is performed on these layers using a filter that performs 2D matrix multiplication
on the layer and filter. The CNN model architecture consists of the following layers:

Convolutional layer; 32 nodes, kernel size 3

Convolutional layer; 32 nodes, kernel size 3

Convolutional layer; 64 nodes, kernel size 3

Fully connected layer; 128 nodes

The final layer is also a fully connected layer with 2 nodes. In all the layers, a Relu activation
function is used except the output layer in which we used Softmax.

6
 5. Procedure

Step 1: Take image as input from a camera. Using OpenCV images from webcam is being
gathered and feed them into a deep learning model.

Step 2: Detect the face in the image and create a Region of Interest (ROI).

Step 3: Detect the eyes from ROI and feed it to the classifier.

Step 4: Classifier will categorize whether eyes are opened or closed.

Step 5: Calculate score to check whether the person is drowsy.

Step 1 – Take Image as Input from a Camera

With a webcam, we will take images as input. So to access the webcam, we made an infinite
loop that will capture each frame. We use the method provided by
OpenCV, cv2.VideoCapture(0) to access the camera and set the capture object
(cap). cap.read() will read each frame and we store the image in a frame variable.

Step 2 – Detect Face in the Image and Create a Region of Interest (ROI)

To detect the face in the image, we need to first convert the image into grayscale as the
OpenCV algorithm for object detection takes gray images in the input. We don’t need color
information to detect the objects. We will be using haar cascade classifier to detect faces.
This line is used to set our classifier face = cv2.CascadeClassifier(‘ path to our haar cascade
xml file’). Then we perform the detection using faces = face.detectMultiScale(gray). It
returns an array of detections with x,y coordinates, and height, the width of the boundary box
of the object. Now we can iterate over the faces and draw boundary boxes for each face.

Step 3 – Detect the eyes from ROI and feed it to the classifier

The same procedure to detect faces is used to detect eyes. First, we set the cascade classifier
for eyes in leye and reye respectively then detect the eyes using left_eye =
leye.detectMultiScale(gray). Now we need to extract only the eyes data from the full image.
This can be achieved by extracting the boundary box of the eye and then we can pull out the
eye image from the frame with this code. eye only contains the image data of the eye. This
will be fed into our CNN classifier which will predict if eyes are open or closed. Similarly,
we will be extracting the right eye into r_eye.

Step 4 – Classifier will Categorize whether Eyes are Open or Closed

We are using CNN classifier for predicting the eye status. To feed our image into the model,
we need to perform certain operations because the model needs the correct dimensions to

7
start with. First, we convert the color image into grayscale using r_eye = cv2.cvtColor(r_eye,
cv2.COLOR_BGR2GRAY). Then, we resize the image to 24*24 pixels as our model was
trained on 24*24 pixel images cv2.resize(r_eye, (24,24)). We normalize our data for better
convergence r_eye = r_eye/255 (All values will be between 0-1). Expand the dimensions to
feed into our classifier. We loaded our model using model =
load_model(‘models/cnnCat2.h5’) . Now we predict each eye with our model
lpred = model.predict_classes(l_eye). If the value of lpred[0] = 1, it states that eyes are open,
if value of lpred[0] = 0 then, it states that eyes are closed.

Step 5 – Calculate Score to Check whether Person is Drowsy

The score is basically a value we will use to determine how long the person has closed his
eyes. So if both eyes are closed, we will keep on increasing score and when eyes are open, we
decrease the score. We are drawing the result on the screen using cv2.putText() function
which will display real time status of the person.

5.1 Flowchart of Application

Figure 1.2:Flowchart of Application

8
6.UML Diagrams

6.1 Use Case Diagram:

A Use case diagram describes a system’s functional requirements in terms of use-cases. It is a

model of system’s use-cases and its actors. Here use-cases are similar to the actions
performed.

Figure 2.1: Use case Diagram for Driver Drowsiness Detection and Safety Alert System

9
6.2 State Chart Diagram:
A State chart diagram describes the behavior of system. This diagram helps to visualize the
life cycle of objects and depicts the states and transitions of system during its life cycle.This
diagram provides better understanding for state based system.

Figure 2.2: State chart Diagram for Driver Drowsiness Detection and Safety Alert System

6.3 Class Diagram:

10
A Class diagram describes the types of objects or classes in the system and various kinds of
static relationships which exist between them. A class consists of class name, attributes and
methods.

Figure 2.3: Class Diagram for Driver Drowsiness Detection and Safety Alert System

6.4 Sequence Diagram:

11
A Sequence diagram shows how the objects interact with others in a particular scenario of a
use-case. This diagram describes the flow of events of a particular system.

Figure 2.4: Sequence Diagram for Driver Drowsiness Detection and Safety Alert System

6.5 Collaboration Diagram:

12
Collaboration diagrams (known as Communication Diagram) are used to show how objects
interact to perform the behavior of a particular use case, or a part of a use case. Along with
sequence diagrams, collaboration are used by designers to define and clarify the roles of the
objects that perform a particular flow of events of a use case.  They are the primary source
of information used to determining class responsibilities and interfaces.

Figure 2.5: Collaboration Diagram for Driver Drowsiness Detection and Safety Alert System

6.6 Activity Diagram:

13
Activity diagram is basically a flowchart to represent the flow from one activity to
another activity. The activity can be described as an operation of the system. The control flow
is drawn from one operation to another.

Figure
2.6:
Activity
Diagram
for Driver

Drowsiness Detection and Safety Alert System

7.Implementation of Code

14
7.1 DrowsinessDetection.py

from keras.models import load_model

import os
import cv2
import numpy as np
from pygame import mixer
import time
mixer.init()
sound = mixer.Sound('alarm.wav')
face = cv2.CascadeClassifier('haar cascade files\haarcascade_frontalface_alt.xml')
leye = cv2.CascadeClassifier('haar cascade files\haarcascade_lefteye_2splits.xml')
reye = cv2.CascadeClassifier('haar cascade files\haarcascade_righteye_2splits.xml')
lbl=['Close','Open']
model = load_model('models/cnncat2.h5')
path = os.getcwd()
cap = cv2.VideoCapture(0)
font = cv2.FONT_HERSHEY_COMPLEX_SMALL
count=0
score=0
thicc=2
rpred=[99]
lpred=[99]
while(True):
ret, frame = cap.read()
height,width = frame.shape[:2]
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = face.detectMultiScale(gray,minNeighbors=5,scaleFactor=1.1,minSize=(25,25))
left_eye = leye.detectMultiScale(gray)
right_eye = reye.detectMultiScale(gray)
cv2.rectangle(frame, (0,height-50) , (200,height) , (0,0,0) , thickness=cv2.FILLED )
for (x,y,w,h) in faces:
cv2.rectangle(frame, (x,y) , (x+w,y+h) , (100,100,100) , 1 )
for (x,y,w,h) in right_eye:
r_eye=frame[y:y+h,x:x+w]
count=count+1
r_eye = cv2.cvtColor(r_eye,cv2.COLOR_BGR2GRAY)
r_eye = cv2.resize(r_eye,(24,24))
r_eye= r_eye/255
r_eye= r_eye.reshape(24,24,-1)
r_eye = np.expand_dims(r_eye,axis=0)
rpred = model.predict_classes(r_eye)
if(rpred[0]==1):
lbl='Open'

15
if(rpred[0]==0):
lbl='Closed'
break
for (x,y,w,h) in left_eye:
l_eye=frame[y:y+h,x:x+w]
count=count+1
l_eye = cv2.cvtColor(l_eye,cv2.COLOR_BGR2GRAY)
l_eye = cv2.resize(l_eye,(24,24))
l_eye= l_eye/255
l_eye=l_eye.reshape(24,24,-1)
l_eye = np.expand_dims(l_eye,axis=0)
lpred = model.predict_classes(l_eye)
if(lpred[0]==1):
lbl='Open'
if(lpred[0]==0):
lbl='Closed'
break
if(rpred[0]==0 and lpred[0]==0):
score=score+1
cv2.putText(frame,"Closed",(10,height-20), font, 1,(255,255,255),1,cv2.LINE_AA)
# if(rpred[0]==1 or lpred[0]==1):
else:
score=score-1
cv2.putText(frame,"Open",(10,height-20), font, 1,(255,255,255),1,cv2.LINE_AA)
if(score<0):
score=0
cv2.putText(frame,'Score:'+str(score),(100,height-20), font, 1,
(255,255,255),1,cv2.LINE_AA)
if(score>15):
#person is feeling sleepy so we beep the alarm
cv2.imwrite(os.path.join(path,'image.jpg'),frame)
try:
sound.play()
except: # isplaying = False
pass
if(thicc<16):
thicc= thicc+2
else:
thicc=thicc-2
if(thicc<2):
thicc=2
cv2.rectangle(frame,(0,0),(width,height),(0,0,255),thicc)
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):

16
break
cap.release()
cv2.destroyAllWindows()

7.2 Model.py

import os

from keras.preprocessing import image

import matplotlib.pyplot as plt

import numpy as np

from keras.utils.np_utils import to_categorical

import random,shutil

from keras.models import Sequential

from keras.layers import Dropout,Conv2D,Flatten,Dense, MaxPooling2D,

BatchNormalization

from keras.models import load_model

def generator(dir, gen=image.ImageDataGenerator(rescale=1./255),

shuffle=True,batch_size=1,target_size=(24,24),class_mode='categorical' ):

returngen.flow_from_directory(dir,batch_size=batch_size,shuffle=shuffle,color_mode='grays
cale',class_mode=class_mode,target_size=target_size)

BS= 32

TS=(24,24)

train_batch= generator('data/train',shuffle=True, batch_size=BS,target_size=TS)

valid_batch= generator('data/valid',shuffle=True, batch_size=BS,target_size=TS)

SPE= len(train_batch.classes)//BS

VS = len(valid_batch.classes)//BS

print(SPE,VS)

17
model = Sequential([Conv2D(32, kernel_size=(3, 3), activation='relu',
input_shape=(24,24,1)),

MaxPooling2D(pool_size=(1,1)),

Conv2D(32,(3,3),activation='relu'),

MaxPooling2D(pool_size=(1,1)),

#32 convolution filters used each of size 3x3

Conv2D(64, (3, 3), activation='relu'),MaxPooling2D(pool_size=(1,1)),

#64 convolution filters used each of size 3x3

#choose the best features via pooling

#randomly turn neurons on and off to improve convergence

Dropout(0.25),

#flatten since too many dimensions, we only want a classification output

Flatten(),

#fully connected to get all relevant data

Dense(128, activation='relu'),

#one more dropout for convergence' sake :)

Dropout(0.5),

#output a softmax to squash the matrix into output probabilities

Dense(2, activation='softmax')])

model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])model.
fit_generator(train_batch, validation_data=valid_batch,epochs=15,steps_per_epoch=SPE
,validation_steps=VS)model.save('models/cnnCat2.h5', overwrite=True)

18
 8.Results
8.1: Closed Eye Detection

Figure 8.2:Open Eyes Detection

19
Figure 8.3: Sleep Alert

20
 9.CONCLUSION

Finally,this model helps to prevent accidents.When driver feels drowsy,an alaram is used to
alert the driver from falling asleep. The risk, danger, and often tragic results of drowsy
driving are alarming. Drowsy driving is the dangerous combination of driving and sleepiness
or fatigue .A new perspective towards person doziness identification is presented as features
responsible for decision making are produced by leveraging multi-layer convolutional neural
networks. Described model warns drowsy drivers with an alarm, after successful eye-
detection and tracking with computer vision and deep learning techniques .

21
 10.REFERENCES

 https://data-flair.training/blogs/python-project-driver-drowsiness-
detection-system/
 https://towardsdatascience.com/drowsiness-detection-system-in-real-time-
using-opencv-and-flask-in-python-b57f4f1fcb9e
 https://www.pantechsolutions.net/driver-drowsiness-detection-using-
opencv-and-python
 https://www.skyfilabs.com/project-ideas/driver-drowsiness-detection-
using-python
 https://docs.opencv.org/master/d6/d00/tutorial_py_root.html
 https://learnopencv.com/tag/convolutional-neural-network/
 https://www.pyimagesearch.com/2020/06/22/turning-any-cnn-image-
classifier-into-an-object-detector-with-keras-tensorflow-and-opencv/

22