Driver Alert System Based on Eye Recognition

Chapter 1

INTRODUCTION
Car accident is the major cause of death in which million of people
die. To prevent such accidents we propose a system which alerts the driver if
the driver feels drowsy. Facial landmarks detection is used with help of
image processing of images of the face captured using the camera, for
detection of distraction or drowsiness. Driver fatigue sometimes results in
road accidents every year. It is not easy to estimate the exact amount of sleep
related accidents but research presents that driver fatigue may be a
contributing reason in up to 20% in road accidents. These types of accidents
are about 50% more expected to result in death or serious hurt. They happen
mainly at higher speed impacts. And the driver who has fallen asleep cannot
brake . Drowsiness reduces response time which is a serious element of
secure driving. It also reduces alertness, vigilance, and concentration so that
the capacity to perform attention-based activities i.e. driving is impaired. The
speed at which information is processed is also reduced by drowsiness. The
quality of decision-making may also be affected. It is clear that drivers are
aware when they are feeling sleepy, and so make a conscious decision about
whether to continue driving or to stop for a rest. It may be that those who
persist in driving underestimate the risk of actually falling asleep while
driving. Or it may be that some drivers choose to ignore the risks in the way
drivers drink. Crashes caused by tired drivers are most likely to happen on
long journeys on monotonous roads, such as motorways, between 2pm and
4pm especially after eating or taking an alcoholic drink, between 2am and
6am, after having less sleep than normal, after drinking alcohol, it driver
takes medicines that cause drowsiness and after long working hours or on
journeys home after long shifts, especially night shifts.

Dept. of ISE, NHCE 2018-2019 Page 1

 Driver Alert System Based on Eye Recognition

1.1Motivation of Project

Driver drowsiness is a serious hazard in transportation systems. It has been

identified as a direct or contributing cause of road accident. Driver
drowsiness is one of the major causes of road accident. Drowsiness can
seriously slow reaction time, decrease awareness and impair a driver's
judgment. It is concluded that driving while drowsy is similar to driving
under the influence of alcohol or drugs. In industrialized countries,
drowsiness has been estimated to be involved in 2% to 23% of all crashes.
Systems that detect when drivers are becoming drowsy and sound a warning
promise to be a valuable aid in preventing accidents.

1.2Problem Statement

The countless number of people drives for long distance every day and night
on the highway. Lack of sleep may lead to an accident. To prevent these type
of accident we propose driver alert system based on eye recognition.

Dept. of ISE, NHCE 2018-2019 Page 2

 Driver Alert System Based on Eye Recognition

Chapter 2

System Requirement Specification

Purpose: If the eyes of drivers are closed for a threshold period of time then
it is considered that driver is feeling sleepy and corresponding audio alarm is
used to make the driver aware. If driver don’t keep eyes on the road then it is
observed using facial landmarks and the corresponding alarm is used to
make the driver aware. It prevents the accident related to drowsiness of the
driver.

Scope: Detecting the eye blinks with facial landmarks and OpenCV Facial
landmark detection to localize the eyes in the frame from a video stream.
Compute the eye aspect ratio for each eye, which gives singular value
,relating the distance between vertical eye landmark points to the distance
between horizontal landmark points.The eye aspect ratio will remain
constant when the eyes are open and then will rapidly approach zero during
a blink, then increase increase again as the eyes opens

3.1 Hardware System Configuration:

Processor - Intel Core i5

Speed - 1.8 GHz

RAM - 256 GB

Hard Disk - 10 GB

3.2 Software System Configuration:

Operating System - ubuntu 16.04

Programming language - C++

Compiler - gcc

Dept. of ISE, NHCE 2018-2019 Page 3

 Driver Alert System Based on Eye Recognition

Chapter 3

SYSTEM DESIGN
3.1 Flowchart:

Dept. of ISE, NHCE 2018-2019 Page 4

 Driver Alert System Based on Eye Recognition

3.2 Algorithm:

➢ start video streaming. If face found then apply facial Landmark

detection.

➢ extract eye regions.

➢ compute the eye aspect ratio to determine if the eyes are closed.

➢ If the eye aspect ratio indicates that eyes have been closed for a
sufficiently long enough amount of time.

➢ Sound an alarm to wake up the driver.

Dept. of ISE, NHCE 2018-2019 Page 5

 Driver Alert System Based on Eye Recognition

3.3 Code & Implementation:

Dept. of ISE, NHCE 2018-2019 Page 6

 Driver Alert System Based on Eye Recognition

Dept. of ISE, NHCE 2018-2019 Page 7

 Driver Alert System Based on Eye Recognition

Dept. of ISE, NHCE 2018-2019 Page 8

 Driver Alert System Based on Eye Recognition

Dept. of ISE, NHCE 2018-2019 Page 9

 Driver Alert System Based on Eye Recognition

Chapter 4:

Results and Discussion

4.1 Use of table and explanation:

Various accuracies were taken and a table plotted for them.

Eye blink Eye blink Drowsiness Drowsiness

accuracy for only accuracy for face accuracy for only accuracy for face
eyes and eyes eyes and eyes
90% 95% 90% 90%

It can be seen from the above table that if sample 1 is not taken into
consideration then the system has an accuracy of nearly 94%. That said the
high amount of errors in sample 1 shows that the system is prone to error and
has certain limitations. In sample we did not use the backlight of the
webcam. The resulting poor lighting conditions gave a highly erroneous
output.

4.2Output (Snapshots):

Read the record video using video reader In opencv, the first step is to
read the video. Using the video reader function with read method to read
video data from a file into the opencv workspace. The video reader supports
some file formats that vary platform to platform. These platforms are like-
All platform those are AVI, including uncompressed, indexed, grayscale,
and Motion JPEG-encoded video. All windows, Macintosh, Linux etc. The
video reader constructs the desired objective to read video data from the file.

Dept. of ISE, NHCE 2018-2019 Page 10

 Driver Alert System Based on Eye Recognition

The face region is completely recognized. And now next major step is eye
detection. To detect the eyes, previous two resulted images named as
histogram equalized image and threshold image. Using these two images,
the eyes and low intensity portion of face are located for the further process.
The resulted image with locate eye and low intensity . To darken the eye
region some morphological operations like erosion and dilation are used.
Morphology is a method of image processing based on shapes. The value of
each pixel in the output i.e., binary image is based on a comparison of the
equivalent pixel in the input image with its neighbors. By choosing the size
and shape of the neighborhood, it can be constructed a morphological
operation. This operation is sensitive to specific shapes in the input image.
Using morphological functions, there are many image processing tasks like
contrast enhancement, thinning, filling, noise removal, segmentation is
performed. Some morphological operations are erosion, dilation, opening,
closing and boundary extraction. Erosion and dilation are the fundamental

Dept. of ISE, NHCE 2018-2019 Page 11

 Driver Alert System Based on Eye Recognition

operations of morphological image processing. In this report, erosion and

dilation are applied. The erosion function erodes the grayscale, binary or
packed binary image and returns eroded image. Erosion shrinks or thins
objects in a binary image. When dilation is an operation that grows or
thickens objects in an image. Both functions use a specific element which
does both operations of erosion and dilation.

Dept. of ISE, NHCE 2018-2019 Page 12

 Driver Alert System Based on Eye Recognition

Dept. of ISE, NHCE 2018-2019 Page 13

 Driver Alert System Based on Eye Recognition

After smoothing using Gaussian filter, it is clearly seen that the region of
eye is detected. To find out the stage of eye whether it is close or open, it is
necessary to locate exact location of eye. To trace the accurate spot of eye,
apply the circular Hough transform function. Figure shows the exact region
of eye. The Hough transform is used to detect lines, circles or other
parametric curves. It is computed on binary image. This function returns the
Hough transform matrix with additional parameters i.e., theta in degrees and
rho. These parameters are the arrays of rho and theta values over which the
Hough transform generates the matrix . The binary image can be logical or
numeric. The image must be real and 2-D which is listed. It has been found
the exact region of eyes. Humans must know by watching the image whether
the eyes are open or closed. But opencv toolbox or software does not know
whether the eyes are open or close. To get the absolute result, it is necessary
the conversation of resulted last image into graphical form. To covert the
image into graphical form, histogram function is used. To represent the
allocation of data in a data set, a histogram is used. Each data point is placed
into a bin based on its value. The histogram is a plot of the number of data
points in each bin. A histogram shows the distribution of data values. For
characterizing the spread of data from repeated trial and for determining the
probability of measurement, histograms are widely used. Histogram
processing is also used for computationally adjusting the contrast. Histogram
method can be applied to numerous problems including color balance or tone
transfer. There are mainly three functions of histogram function. First
function introduces basic pixel operations that hoe they do change
histograms. Showing a number of schemes to stretch a histogram to cover
the full range of pixel values. Second function of histogram is histogram
equalization. This function focuses on spreading out the pixel values
consistently over some range. Last but not least the function of histogram
controls modifying the pixel values. It can be used to match any arbitrary

Dept. of ISE, NHCE 2018-2019 Page 14

 Driver Alert System Based on Eye Recognition

distribution. The final step of detecting is analysis the number of histograms

of resulted last image frame. In which, the study of histograms is very
important to define the detection of eye stage. Histogram of image shows
that when the eyes are open, the level of intensity between two random
numbers is high. And at remain numbers, the intensity is almost to zero.
Whenever the eyes are open, the random numbers in which the high intensity
comes will be repeated. And whenever the eyes are close, the intensity
comes low all along the path. Apply fuzzy logic using this pixels for setting
parameters to detect whether eyes are open or closed. Fuzzy Logic is a
problem-solving control system methodology. It lends itself to
implementation in systems ranging from simple, small, embedded micro-
controllers to large, networked, multi-channel PC or workstation-based data
acquisition and control systems. It can be implemented in hardware,
software, or a combination of both. FL provides a simple way to arrive at a
definite conclusion based upon vague, ambiguous, imprecise, noisy, or
missing input information. FL's approach to control problems mimics how a
person would make decisions, only much faster. Fl is different from
conventional control methods. Last figure shows final goal of problem
solution. At last, find the histogram of the circular region and count the bins
for each pixel value. There is distribution of data on the x-axis. There is
easily seen that the intensity level at some points are high and at some points
the intensity level is low or almost zero. This graphical representation is of
one frame among of all frames. Whenever the eyes are open, the intensity
comes high otherwise low. And this intensity comes high in between two
points. These two points will be repeated.

Dept. of ISE, NHCE 2018-2019 Page 15

 Driver Alert System Based on Eye Recognition

Conclusion and Future Enhancements

This is the study that uses a large set of spontaneous facial expressions for
the detection of drowsiness. Previous approaches to drowsiness detection
primarily make pre-assumptions about the relevant behavior, focusing on
blink rate, eye closure, and yawning. Here there is implementation of
learning methods to determine actual human behavior during drowsiness
episodes. Spontaneous expressions have a different brain substrate than
posed expressions. They also typically differ in morphology and dynamics.
This study reveals that facial expressions are very reliable indicators of
driver drowsiness and facial expressions can be used to do fine
discrimination in the different levels of drowsiness and reliably predict the
time to crash. In laboratory conditions computer vision expression
recognition systems can be used to reliably detect drowsiness and predict
crash with high reliability. Field studies are needed to evaluate the
performance of these systems in actual driving environments. Spontaneous
facial expressions under drowsiness are very different from posed
expressions of drowsiness. A non-invasive system is able to localize the eyes
and monitor fatigue was developed. Information about the face and eyes
position is obtained through various self-developed image processing
algorithms. During the monitoring, the system is able to decide if the eyes
are opened or closed. This project can be used in every vehicle currently on
road to ensure the safety and reduce the chances of an accident due to
drowsiness or distraction of driver. This project can be integrated with car,
so that automatic speed control can be imparted if the driver is found
sleeping.

Currently there is not adjustment in zoom or direction of the camera during

operation. Future work may be to automatically zoom in on the eyes once
they are localized. This would avoid the trade-off between having a wide

Dept. of ISE, NHCE 2018-2019 Page 16

 Driver Alert System Based on Eye Recognition

field of view in order to locate the eyes, and a narrow view in order to detect
fatigue. This system only looks at the number of consecutive frames where
the eyes are closed. At that point it may be too late to issue the warning. By
studying eye movement patterns, it is possible to find a method to generate
the warning sooner. Using 3D images is another possibility in finding the
eyes. The eyes are the deepest part of a 3D image, and this maybe a more
robust way of localizing the eyes. Adaptive binarization is an addition that
can help make the system more robust. This may also eliminate the need for
the noise removal function, cutting down the computations needed to find
the eyes. This will also allow adaptability to changes in ambient light. The
system does not work for dark skinned individuals. This can be corrected by
having an adaptive light source. The adaptive light source would measure the
amount of light being reflected back. If little light is being reflected, the
intensity of the light is increased. Darker skinned individual need much more
light, so that when the binary image is constructed, the face is white, and the
background is black. In the real time driver fatigue detection system it is
required to slow down a vehicle automatically when fatigue level crosses a
certain limit. Instead of threshold drowsiness level it is suggested to design a
continuous scale driver fatigue detection system. It monitors the level of
drowsiness continuously and when this level exceeds a certain value a signal
is generated which controls the hydraulic braking system of the vehicle.
Sometime there is a problem with to achieve the optimum range between
face and camera. When the distance between face and webcam is not at
optimum range then certain problems are arising.When face is too close to
webcam (less than 30 cm), then the system is unable to detect the face from
the image. So it only shows the video as output as algorithm is designed so
as to detect eyes from the face region. This can be resolved by detecting eyes
directly using detect objects functions from the complete image instead of
the face region. So eyes can be monitored even if faces are not detected.

Dept. of ISE, NHCE 2018-2019 Page 17

 Driver Alert System Based on Eye Recognition

When face is away from the webcam (more than 70cm) then the backlight is
insufficient to illuminate the face properly. So eyes are not detected with
high accuracy which shows error in detection of drowsiness. This issue is not
seriously taken into account as in real time scenario the distance between
drivers face and webcam doesn’t exceed 50cm. so the problem never arises.
Considering the above difficulties, the optimum distance range for
drowsiness detection is set to 40-70 cm. If more than one face is detected by
the webcam, then system gives an erroneous result. So it is required to
develop such system that will give result even though having multiple faces.
The last but not the least future scope is developing a algorithm which can
work with driver having spectacles. This issue has not yet been solved and is
a challenge for almost all eye detection systems.

In the real time driver fatigue detection system it is required to slow down a
vehicle automatically when fatigue level crosses a certain limit. Instead of
threshold drowsiness level it is suggested to design a continuous scale driver
fatigue detection system. It monitors the level of drowsiness continuously
and when this level exceeds a certain value a signal is generated which
controls the hydraulic braking system of the vehicle. Hardware components
requiredDedicated hardware for image acquisition processing and display 47
Interface support with the hydraulic braking system which includes relay,
timer, stepper motor and a linear actuator. Function When drowsiness level
exceeds a certain limit then a signal is generated which is communicated to
the relay through the parallel port(parallel data transfer required for faster
results).The relay drives the on delay timer and this timer in turn runs the
stepper motor for a definite time period .The stepper motor is connected to a
linear actuator. The linear actuator converts rotational movement of stepper
motor to linear motion. This linear motion is used to drive a shaft which is
directly connected to the hydraulic braking system of the vehicle. When the
shaft moves it applies the brake and the vehicle speed decreases. Since it
Dept. of ISE, NHCE 2018-2019 Page 18
 Driver Alert System Based on Eye Recognition

brings the vehicle speed down to a controllable limit , the chances of

accident occurrence is greatly reduced which is quite helpful for avoiding
crashes caused by drowsiness related cases.

Dept. of ISE, NHCE 2018-2019 Page 19

 Driver Alert System Based on Eye Recognition

References

• https://www.tutorialspoint.com/opencv/
• https://www.opencv-srf.com/p/introduction.html
• https://docs.opencv.org/3.4/d9/df8/tutorial_root.html
• http://dali.feld.cvut.cz/ucebna/matlab/toolbox/images/fspecial.html
• http://wwwrohan.sdsu.edu/doc/matlab/toolbox/images/morph3.htm
l#12508
• http://www.ee.ryerson.ca/~phiscock/thesis/drowsy-
detector/drowsy-detector.pdf
• http://www.cnblogs.com/skyseraph/archive/2011/02/24/196376
5.html
• http://www.scribd.com/doc/15491045/Learning-OpenCV-
Computer-Vision-with-theOpenCV-Library
• http://opencv.willowgarage.com/documentation/reading_and_wr
iting_images_and_vid eo.html
• http://www.scribd.com/doc/46566105/opencv
• Learning OpenCV by Gary Bradski and Adrian Kaehler
• http://note.sonots.com/SciSoftware/haartraining.html

Dept. of ISE, NHCE 2018-2019 Page 20