Traffic Sign Recognition Using Machine Learning Models For

Smart Driving Support System

MSc in Data Analytics (MSCDA_A)

Research in Computing
Submitted: 14th August 2019

Rahul Gopal Jaju

X18125671

School of Computing
National College of Ireland, Dublin

Lecturer: Dr. Catherine Mulwa

1
 Abstract
Modern world is fully simplified with the means of automation. While trying to concentrate on
their driving, the driver unintentionally forgets to pay attention the traffic sign present on the
streets. This can certainly be death-defying for driver and also for the individuals traveling
with him. The issue can be solved if there is an effectual method to inform the driving person
without getting his focus distracted. In this project we propose an outstanding Smart Driving
Support System (SDSS) for successfully detection and recognition of the traffic signs. The first
stage involves identification of characteristics like colour and shape. CIECAM97 model is
employed for extraction of the colour-based information. While FOSTS model is used for
identification of shape. The second stage involves recognition of sign which is achieved by
implementing Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM)
architecture together. This hybrid model is implemented to evaluate the accuracy and recall
for our test dataset. Further, our report also aims to implement other modelling techniques
such as LDA, MLP and Naïve Bayes classifier to find the best model to design our SDSS based
on comparison of their performance metrices.

Keywords: Smart Driving Support System, Long Short-Term Neural Network, Naïve Bayes
classifier.

1. Introduction
These days, we can see an incremental rise in the automobiles on road. This rise is also
responsible for the large number of accidents taking place everywhere. The fatal accident
which occur are primarily due to the driving person’s failure to understand the visual data
which is accessible while driving the vehicle. The (Smart Driving Support System) SDSS is
developed to avoid these accidents and also ensure the drivers safety. The important
applications of SDSS are to detect the blind spots, recognizing the traffic signs, assistance for
emergency brakes, etc.

There are three important categories in traffic signs. They are specific, Mandatory, Cautionary
and Informatory signs. The road traffic sign gives us the info about laws and rules to follow
while driving. Informatory sign provides necessary information to drivers about the directions.
Cautionary sign alerts the driver from any kind of hazardous situation ahead on roads. Many a
times due to heavy traffic or bad weather conditions driver might miss his attention which may
lead to accidents. Another usually confronted issue isn't having the option to comprehend the
significance of the sign. Regardless of whether drivers figure out how to detect the sign, quite
possibly they don't have a clue for its interpretation. The meaning of the sign is conveyed in
the form of text or a voice command. This system supports the driver and gives great amount
of info and guides him to drive safely. This system focuses majorly on the speed limit and road
signs detection. The location and acknowledgment of traffic signs speak to a significant issue
while examining this field. The paper examines the current traffic sign discovery and
acknowledgment strategies. Numerous times, drivers face many difficulties on unknown streets
and in such situations focusing on street and forthcoming signs can turn out to be a challenging
assignment.

2
1.1 Background
Smart Driving Support System (SDSS) is the future of vehicles meant for the driver’s safety.
In spite of the fact that this topic has drawn in research benefits in computer vision network for
a long time (Shustanov & Yakimov 2017), it is for the most part viewed as challenging because
of different complexities, for instance, diversity in the background of images of traffic signs.
The project aims to obtain the data from numerous traffic signs which is critical in numerous
applications that includes road safety, self-driving cars, navigation and mapping.

1.2 Motivation
Although, previous research has been done in this area by following different methodologies.
Unfortunately, there is no SDSS system that could achieve the best accuracy of in recognizing
the road traffic signs which is impossible practically. The present system can detect and
recognize only few sets of sign images. Different weather conditions like cloudy and rainy can
result in inaccurate detection of the signs. Also, the present systems cannot identify the signs
in the night or under the shadows of the trees. That’s where the challenge in this research lies
to accurately recognize these traffic sign under such conditions.

Another challenge lies in unable to interpret the meaning or symbol of traffic sign. With SDSS
whichever sign is spotted by the driver the interpretation of the detected sign will be notified
to the driver in the form of text or a voice command.

1.2 Research Question

“How can the various road traffic signs (warning, regulatory, stop, yield) that are in different
shapes (diamond, rectangle, hexagonal, triangle) can be recognized by different modelling
techniques (C-LSTM, MLP, SVM and Naïve Bayes) to ensure drivers safety on roads?”
The above research question aims to maximize the safety for the drivers by recognizing the
traffic signs which are present in the vicinity. The solution to our research question would be
achieved by the model proposed that consists of traffic sign detection, recognition,
classification, also mapping the traffic infrastructure with the application in the vehicles.

1.3 Objectives
The major objective of this project report is to detect the traffic signs on road and further
accurately recognize it by making use of machine learning techniques which will correctly
classify the detected images into different categories such as warning, regulatory, stop and
yield. Another aim is to interpret the road signs and notify its meaning in the form of text or
audio for drivers’ convenience that also ensures safety. The proposed method can further be
integrated with google maps for inventory management to restore the damage sign boards.

3
2. Related Work
2.1 Introduction
Previously, authors did their research on SDSS by using different machine learning techniques.
This section gives us a brief assessment about the work done in the field of traffic road detection
and recognition.

2.2 CNN – A review of literature

Presently, there exists publications who are dealing with recognition and detection of road
traffic signs for Smart Driving Support System (SDSS). In most of the recent published
approaches the accompanying technique is implemented. Firstly, segmentation of colour and
recognition of shape of sign are done. Then, a final validation of sign is implemented using
classification method based on classifiers. There are few ways for detecting the sign as in
(Shustanov & Yakimov 2017), where the publisher has used the Generalised Hough Transform
modified model in order to determine the coordinates of the traffic sign. And later for the
purpose of recognition CNN is used which resulted in accuracy of 99.94%.
(Girshick et al. 2014) proposed the rich element chain of features for accurately detection of
object and semantic segmentation, the model achieves a better accuracy by utilizing Deep
Convnet for traffic sign classification. But the model proposed by them is slow in performance
as ConvNet is forwarded to each proposed object without any computation being shared.
(Changzhen et al. 2017) have introduced an algorithm completely based on deep Convolutional
Neural Network (CNN) with the use of Regional Proposed Network (RPN). This method was
implemented on the Chinese traffic signs. This CNN model was initially trained on the
collected data and evaluation was based on fine-tune technology. Later, the model was tested
on 33 video sequences of the size 640x480. The videos were from the driving recorder camera
placed in the car. The precision in detecting the signs attained by this model was 99%.

(He et al. 2015) proposed a model based on Spatial Pyramid pooling networks (SPPnets) where
the proposed model speed it up by distribution of feature map based on convolutional network.
This method only calculates the feature map for the full image. The model proposed has some
notable drawbacks as it has a multi stage pipeline for training.

(Qian et al. 2016) have proposed a system which has two main modules. First module generates
a region based simplified and efficient scheme candidate region. The main feature of this
approach is by done by RGB thresholding and later Connected Component Analysis (CCA).
Second module is the model based on CNN which is used to classify and discriminate the each
of the region. For each of the region, CNN is trained to find out whether the detected region is
a traffic sign or not. It rejects the samples which are false and certain true samples are
determined.

(Fistrek & Loncaric 2011) have proposed a two-stage methodology of detection and
acknowledgement of traffic signs based on Neural Network. The detection phase includes
sorting of images into groups. The pixels from the image are extracted that corresponds to the
sign edges. Sorting of images is done by NN. The detected signs are recognised by Hough

4
method. The model proposed by them is applicable for big range of images with different
resolutions and lightning conditions. The only disadvantage of this system was insufficient
speed for execution of this proposed model in real time.

(Changzhen et al. 2017) have introduced an algorithm completely based on deep Convolutional
Neural Network (CNN) with the use of Regional Proposed Network (RPN). This method was
implemented on the Chinese traffic signs. This CNN model was initially trained on the
collected dataset and evaluation was based on fine-tune technology. Later, the model was tested
on 33 video sequences of the size 640x480. The videos were from the driving recorder camera
placed in the car. The precision in detecting the signs attained by this model was 99%.

(Algorry, García & Wofmann 2018) demonstrated two experiments in order to verify that
detection quality of the proposed architecture is fairly improved. The experimented was further
implemented with the use of YOLO v2 that can detect the region of interest (ROI) and further
a light weight CNN is used as a classifier to classify and further enhance the detection quality
with minimum time processing. The only limitation the experiment had was that only few
classes of images were used for training purpose.

(Jurišić, Filković & Kalafatić 2016) model was based on a single CNN. Their model was
designed to analyse the performance over multiple datasets. Their model had only one layer
which was connected fully on top of each convolution layer. In the experiment, training was
performed on two different sets. The two datasets (BTSC and rMASTIF) resulted in
performance of 86.4% and 91.06% respectively.

(Wang 2018) had found that deep convolutional neural network makes its use much efficiently
for training purpose on larger datasets. Larger datasets which were used had many types of
complex signs. For every image as input, the convolution layer extracted the feature from the
image. And the on a different scale the coordinates of the image are analysed. Suppression
algorithm is used to eliminate the redundant text box. In the final experiment, iterations were
performed for 20,000 times iterations at 0.001 learning rate. The accuracy rate reached was
approx. 96%.

2.3 SVM – A Review of Literature

(Biswas, Fleyeh & Mostakim 2014), detected circular traffic signs by using canny edge
detector. The interpretation of the traffic sign is further extracted by using an SVM classifier.
Almost, 270 images that contained 213 sign of speed limit and 288 signs of non-speed limit
clicked in varied light conditions. The accuracy of the recognition was found to be 98%.
(Rahmad et al. 2018) proposed a method which has an effectual feature extraction that is built
on combination of three steps. The first step is HOG method. This method is used to represent
image in a group of histograms. The second step is followed by the application of Gabor filter.
This filter is used to simulate the features of the human visual system to isolate the orientation

5
and frequency of image. Final step is application of LBP (Local Binary Pattern) algorithm to
classify texture of the image. Further, SVM is used as a classification method.

(Yuan, Xiong & Wang 2017) proposed a model for traffic sign detection in an incremental
framework of video. The video frame is fed as an input and further the traffic signs are detected
using offline trained detector. The approach is implemented in five stages. First is application
of Off-line Detection to interpret the traffic signs. The second includes tracking of these signs
using Motion Model. The third stage involves the On-Line sample collection of the detected
traffic sign. Fourth stage involves detection using the incremental detector. The final stage is
Scale-Based recognition which is employed utilizing the multi-class SVM to recognized these
detected signs.

Another research paper by (Kiran et al. 2009) proposed a system that involves three stages.
First is colour segmentation. The main task involves extraction of the colour of the sign. The
second involves shape classification. The shape classification is employed using linear SVM.
The shape classification is further employed as feature extraction of the shape and further
training and testing using the linear SVM. The model successfully accomplished 0.93
Fmeasure for detection and 0.95 for recognition.

(Fifik & Turán 2010) have used a different colour segmentation approach which is grounded
on a two-step detection scheme. The two steps are designed as two subsystems. First subsystem
is called as fast and composed with a classification and CSR block. Second subsystem slower
is enabled with Segmentation, extraction, Feature Memory and Classification block.

After going through a lot of research (Jain & Gianchandani 2019) had developed a hybrid
model which is based on OCR and MSER. The proposed model utilises Lucy-Richardson
calculation. The proposed methodology is divided into two stages. Stage one includes steps
like; removal of noise and de-blurring, colour conversion to binary from RGB image, edge
detection and enhancement followed by the last step as MSER region detection. Second stage
includes steps like geometric filtering, morphological segmentation, connecting the characters,
text line formation and word separation as the last step. The below diagram is the framework
of the model proposed by them.

(Creusen, Hazelhoff & De With 2012) projected a novel paper based on colour transformation
for the interpretation of sign. Colour transformation occurs when extraction of gradient features
takes places from the various channels. The approach is followed by utilizing the different
schemes used for painting the traffic signs. Extraction of colour is largely dependent on the
background of the sign board. The paper shows that the colour transformation enhances the
performance in detection. For the purpose of recognition of identified sign linear SVM was
used. The experiment was performed on dataset of 33,409 images. The experiment resulted in
performance gain of 90%.

6
2.4 MLP (Multi-Layer Perceptron) – A review of Literature
(Swathi & Suresh 2018) have proposed an algorithm where the traffic signs are detected in the
form of video frames from a real time video. In the recognition phase, they have used
Multilayer Perceptron (MLP) which is trained with the HOG (Histogram oriented Gradient)
features. The whole process took place by the real time video fed as an input for the purpose
of pre-processing. Since the signs were designed were some specific colour the ROI of images
was converted to HSV to detect the shape and colour space which was performed using
Douglass-Pecker. The proposed algorithm was implemented on Microsoft Visual Studio which
resulted in recognition rate of 95.71% for circular signs and 95.71% for triangular signs on
real-time videos.

(Höferlin & Zimmermann 2009) developed a design which was organized in three stages of
detection, tracking and classification. The system was an upgraded version of CCD algorithm.
SURF and SIFT were two modules additionally added to the system. The system performed an
experiment on a 30-minute sequence with 94 road traffic signs. For the purpose of recognition
MLP (Multi-Layer Perceptron) with an input layer of 128 neurons was introduced. The system
could identify 96.4% of the total signs.

2.5 Non-Classification Techniques – A Review of Literature

(Houben et al. 2013) discovered the use of Viola-Jones detector for the purpose of sign
detection. The detector is a binary classifier in a cascade which is applied to the input sliding
window. The process of recognition was carried out by training the dataset using AdaBoost.
To achieve robustness, the positives ones were scaled within that range. The same technique
was followed during training that allowed more larger step size. After implementing their
proposed methodology, they found that the accuracy for detecting circular signs was 91.3%
and for triangular shaped signs, it was 90.7%.

(Zavadil, Tuma & Santos 2012) had proposed a paper where MATLAB was used for image
processing. The two techniques are based on pattern recognition and blob analysis have been
combined and utilised. And the results are presented with main algorithms description.
Although, the accuracy was more dependent on surrounding light circumstances. The model
could successfully recognise the traffic sign and symbols at a distance of 2m with a success
rate of 100% almost. Although, the time taken for the purpose of image processing was 0.7 s
approximately and the dataset used had few sets of images.

Later, (Siegler & Kardkov 2011) found a generic system which was both suitable for personal
as well as mobile computer environments. This system was based on a high quality of input
video stream. YUV420 was the output for the encoded MPEG4 video with 25fps and a
resolution of 1600x1200 frames. The experiment was performed in three setups in different
climate conditions. First setup resulted in accuracy of 78.4%. The other two setups resulted in
accuracy of 69.7% and 68.1% respectively.

7
In a research (Torresen, Bakke & Sekanina 2005), where the researcher has used the Norwegian
Signs. The methodology consisted of three steps. First was based on colour filtering. Second
and third were based on locating the sign in the image and detecting the numbers printed on it.
The method was performed on 198 images and the result gave the recognition rate of 91%.

(Wu & Tsai 2013) proposed a method to detect the traffic sign from a video for data
maintenance of road inventory. The algorithm was based on colour segmentation and done on
locally adaptive threshold extraction on ROI which made use of the depth-first-search
algorithm. The result of the algorithm which was performed on 123 images gave 97%
recognition rate.(Liu & Maruya 2009) presented two ways to improve the road sign recognition
system. One made the use of predefined procedures of image processing and the other on
controlling the cameras exposure.(Maldonado Bascón et al. 2010) have proposed a unique
system that gained an improvement in accuracy by 4-5%. The system also made
implementation with support vectors which was reduced to almost 60-70%

2.6 Comparison of previously used methods

Author Method used for Classifier used for Dataset used Result or
Detection Recognition Accuracy

(Qian et al. Thresholding and Multi task CNN GTSRB 95%

2016) CCA (connected
component analysis)

(Rahmad et al. HOG, Gabor, LBP, SVM classifier Indonesian 86.7%

2018) HSV colour space road traffic
sign dataset

(Creusen, Colour SVM 33,409 90%

Hazelhoff & De transformation images
With 2012) based on HOG
detector

(Algorry, García YOLO v2 Lightweight CNN Google street 74.1%

& Wofmann view images
2018)
(Wang 2018) Computer vision VGG-16 CNN GTSRB 96%
method

(Swathi & Douglass Pecker MLP (Multilayer Videoframe Circular-

Suresh 2018) Algorithm Perceptron) clicked on 97.14%
Samsung J5 Triangular-
smartphone 95.71%

8
2.7 Challenges and Limitations
Although, many different models have been proposed by researchers for detecting accurately
acknowledge the road traffic signs. Here are few of the limitations to understand the pitfalls in
the existing project of SDSS (Smart Driving Support System). The challenge faced by (Biswas,
Fleyeh & Mostakim 2014) was that the total number of false negatives were greater than the
total number of false positives. Although, it had a slight better accuracy but it lacked algorithm
for varied digit segmentation which increased the false positive number. The limitation of the
system developed by (Algorry, García & Wofmann 2018) was poor in performance. This was
due to a less images used for training purpose. (Yuan, Xiong & Wang 2017) forces solid spatial
distribution on the location of the signs. While in many situations it can to be sure improve the
discovery execution. Another restriction is that the system also necessitates the disappearing
horizon to be generally in center of the caught picture, which is basic in many situations. Our
proposed system overcomes all these limitations and pitfalls.

2.8 Conclusion
From the above literature we can see the different methodologies used for the purpose of
evaluation and to create more effectiveness for the system. We also saw the different machine
learning algorithms that were used in enhancing the performance of the system. Though many
of the researchers focused on detection and acknowledgement of the traffic signs, still remains
undiscovered parts where we could do more research. Some classification algorithms still
unexplored in this research area and that is where our research and interest lie. We would like
to do research on the same.

3. Methodology and Specification

3.1 Introduction
The three methodologies are Knowledge Discovery Databases (KDD), (SEMMA) – Sample,
Explore, Modify, Model, Access and (CRISP-DM) Cross-Industry Standard Process for Data
Mining. These are widely used for purpose of data mining operations by the experts and also
in industries. In our project we have proposed CRISP-DM methodology for data mining
operations. Let us see in detail how we can use CRISP-DM in our project.

3.2 Justification for the proposed methodology

We have proposed CRISP-DM methodology for our project as it helps in better understanding
of our business goal. It also converts the business goal in our datamining operation. It also
helps in development of our project plan. The approach of CRISP-DM is very flexible and also
helps to improve our hypothesis and the methods for data analysis for further iterations.

3.3 Implemented Methodology

We have modified all six stages of CRISP-DM methodology as per our proposed SDSS (Smart
Driving Support System) which is demonstrated in the figure (1). The first stage includes
detection, recognition and interpretation of the road traffic sign and that’s where our research
is focused on. Traffic signs detection and recognition is at present a significant piece of
undertakings in field of Smart Driving Support System meant for drivers. This stage is like the

9
first stage from CRISP-DM methodology. Unique consideration has been given to the
strategy’s dependent on artificial neural networks. We need to classify about four different kind
of traffic signs (warning, regulatory, stop, yield) and these signs are in different shapes
(diamond, rectangle, hexagonal, triangle). We also aim to interpret this and provide this
information in the form text or voice command for convenience of the driver.
The next stage is to understand the input data which were going to use for our project. Our
input data are images or a video (sequence of images). This is the data understanding stage
from CRISP-DM methodology. The third stage is about data preparation and in this stage, we
are going to prepare our data for the purpose of modeling. In this stage we will separate the
relevant information about the sign from rest of the image. We will also resize the image in
terms of resolution and also process it for noise removal that satisfy our needs. By the end of
this stage we will have the required set of images required for the purpose of modeling.

Figure 1 CRISP-DM Methodology

Now so we have the preprocessed data we are good to go to implement our techniques. We
will implement first CNN-LSTM technique. It is one the adopted machine learning technique
for time series forecasting. Also, this technique is efficient enough to gain a better accuracy

This can also be automated for real-time prediction. Other techniques like LDA (Linear
Discriminant Analysis), MLP (Multilayer Perceptron) and Naïve Bayes will also be
implemented to draw a comparison between their results. Once the proposed models are
implemented the final stage would be the evaluation. This is very important stage as we can
understand the performance metrices and develop an understanding about the best technique
for our data by measuring the results. Our last stage would be to deploy the proposed techniques
for our SDSS (Smart Driving Support System).

10
3.4 Proposed Design
The design which is proposed in the report follows a 3-Tier architecture with three layers.
 Presentation Layer
 Application Layer
 Data persistent Layer
Presentation layer remains our SDSS with which the driver can interpret the signs in the form
of text or a voice command ensuring safety. The second layer is application layer where all our
proposed methodology is executed in our model. Here, we apply the proposed techniques can
compare their performance metrices and evaluate which one is the best classifier for our
system. The last layer is Data Persistent also called as Technical layer where all the techniques
which we have used to build our SDSS exists. We will use R studio to implement our proposed
design.

Figure 2 Proposed Model Design

11
The diagram for process flow from starting stage to the final stage of deployment is given
below in figure 3.

Figure 3 Process flow figure

12
3.5 Datasets

I. Dataset 1: We propose to use two different datasets to implement our model. The first
one is LISA traffic sign dataset. This data is a set of annotated frames and videos that
contain US traffic signs. The dataset has both the pictures and the videos. This dataset
has 47 different traffic signs. This dataset is publicly available to download from
GitHub Repository.
II. Dataset 2: Another dataset is from GTSRB (The German Traffic Sign Benchmark).
This dataset has 40 classes and over 50,000 images. This dataset is publicly available
to download for use from GTSRB official site.

3.6 Conclusion
We have enough database to implement our proposed models using the CRISP-DM
methodology. We will now discuss about the techniques that are proposed to design our SDSS
(Smart Driving Support System).

4. Proposed Implementation

4.1 Introduction
This section we will discuss about the different machine learning techniques that we proposed
to use for implementation. The proposed implementation of the whole project is divided into
objectives. Each of the objective will give a brief idea about it and the technique used behind
it for our SDDS system.

4.2 Proposed Algorithm

Our first objective in designing (Smart Driving Support System) SDSS is to successfully detect
the road traffic signs. In detection, colour and shape are the dominant features used to develop
this system. As the image detected can have different distribution of colour combination
dependent on weather conditions at the time when the image was clicked. We have used a
colour appearance model CIECAM97 for this task. colours commonly used are red, blue, black
and white. Therefore, only Hue and Chroma are considered for segmentation.
The second objective is shape feature extraction is classification of the identified signs on the
basis of their shape. This stage is employed using (Foveal System for Traffic Signs) FOSTS
model. The signs are classified on the basis of their external forms which is done from all the
standard database and also from the real images. The shapes classification is done for
preliminary shapes (circle, triangle and rectangle) by means of histogram orientation.
The third objective is to implement the machine learning models proposed in the report. The
proposed algorithms we would like to implement are CNN LSTM, LDA, MLP and Naïve
Bayes classifier. After implementing them we also compare them and fine the best model for
our SDSS (Smart Driving Support System).

13
Figure 4 Pseudo Code for MLP (Multi-Layer Perceptron) (Mustafa, Allen & Appiah 2019)

Figure 5 Pseudo Code for Naive Bayes classifier (Saputra, Widiyaningtyas & Wibawa 2018)

Figure 6 Pseudo Code for LSTM (Shaikh & Adamuthe 2018)

14
 Figure 7 Pseudo Code for LDA (Huh & Lee 2010)

4.3 Justification for Proposed Algorithms

CNN-LSTM is specially designed for image and videos which is directly used as sequences.
This architecture is generally used to generate textual descriptions where CNN layer is used
for feature extraction and LSTM for sequence prediction. This architecture can also extract the
features from different variety of images (Chen et al. 2017). This is proposed to get the best
performance and further compare it with other techniques.
In addition, we also propose the use of LDA (Linear Discriminant Analysis) (Pang, Ozawa &
Kasabov 2005) that can be effectively used to extract features from large data to with best
discriminability for classification problems when related to additional problems. It also works
as a classifier for classification. Information is in the form of features of the sign which can be
extracted by using this classifier.

MLP (Multi-Layer Perceptron) is a multi-layer neural network. A perceptron is fundamentally

a single neuron that is predecessor to larger neural networks. Mathematically, such network is
proven and are capable for any mapping function. Using this the data structure can learn to
represent features in different resolutions or scale and combine it into higher features.
Finally, we would be using Naïve Bayes classifier as to design our last model for the proposed
dataset. Despite of the design and basic assumptions, naïve Bayes algorithm works very well
in real-life situations. It also needs less training data. It is highly scalable and linear scaling is
done with the predictors. Naïve Bayes can be used for both multi-class classification and binary
problems. It can also handle discrete and continuous data.

15
4.4 Conclusion
The above-mentioned algorithms are proposed for our implementation as there is hardly any
research done in traffic sign detection and acknowledgement by using the above-mentioned
techniques.

5. Proposed Evaluation
5.1 Introduction
This remains the essential part as selecting an incorrect assessment metric might result in
superior performance gain, yet the implemented model may actually fail. This segment
additionally lists all performance metrices for this purpose of evaluation procedure and
likewise give us an insight concerning how our proposed model’s performance would be
assessed for based on its metrices.

5.2 Evaluation Parameters

Since, all our proposed models in this report deals with the problem of classification. The
evaluation parameters that have been selected for our models are Precision Matrix, Confusion
Matrix and Recall.

5.3 Confusion Matrix

It is a measurement for performance for all problem based on classification where the output
can be two or many. It is basically a table with predicted and actual values of different
combinations. It is very useful to understand all measures like recall, Specificity, precision and
accuracy. Even AUC-ROC curve can be understood from confusion matrix.

Figure 8 Confusion Matrix

5.4 Recall and Precision

The other important metrices which we will use for evaluation of these models are Recall and
Precision. Recall is about finding the relevant instances in our dataset and precision gives the
measure of the data points where our models says that actually relevant. Precision defines the
number of true positives by number of false positives plus the true positives. False positives in
the model are incorrectly labelled as positive which were truly negative. The below table very
well explains these terms.

16
 Figure 9 Recall, Precision and Accuracy

5.5 Conclusion
After selecting the performance metrices for each of the model proposed we can evaluate them
and find the best model to design our SDSS (Smart Driving Support System).

6 Ethics
There is no ethical constraint for the data as it is publicly available for users to use and do not
require any permission request from the owner. The quality and integrity of this research is
very much unique and do not intend to harm anyone’s confidentiality. Our stakeholders are the
automobile manufacturing companies for whom this system is designed for. Other stakeholders
would be the road traffic control department as it can also be used as a part of inventory
management for the lost or damaged traffic signs.

7. Project Plan
The plan proposed for thus projects implementation is given below. Progress of the project is
tracked using this smartsheet. Details about the time consumed for each milestone is also stated
below.

Figure 10 Project Plan

17
 Figure 11 Gantt Chart

8. Conclusion
The proposed machine learning models which we will build for our project to detect and
acknowledge the road traffic sign meant for SDSS (Smart Driving Support System) can classify
the detected signs on the basis of their colour and shape (diamond, rectangle, hexagonal,
triangle). Feature selection will be also done to subtract the background and also resize the
image which will result in excellent performance. Finally, all our models will be compared and
evaluated on the basis of their performance metrices. Metrices like precision, recall and
confusion matrix will be the factors for our evaluation. In the end of the project, we will have
our desired SDSS (Smart Driving Support System) with the best performance that would
correctly detect and interpret the road traffic sign on road. This system can also be installed in
the vehicles which will assist drivers and given them information of the detected sign in the
form or text or voice command and also ensuring safe driving.

Acknowledgement
I would like to take this as an opportunity to greet my thanks to my lecturer Dr. Catherine
Mulwa who have guided me throughout this project and my class colleagues for friendly
encouragement and sharing their honest perspectives on different identified issues.

References
Algorry, A.M., García, A.G. & Wofmann, A.G. 2018, ‘Real-Time Object Detection and Classification
of Small and Similar Figures in Image Processing’, Proceedings - 2017 International Conference
on Computational Science and Computational Intelligence, CSCI 2017, pp. 516–9.
Biswas, R., Fleyeh, H. & Mostakim, M. 2014, ‘Detection and classification of speed limit traffic signs’,
2014 World Congress on Computer Applications and Information Systems, WCCAIS 2014, pp. 1–
6.
Changzhen, X., Cong, W., Weixin, M. & Yanmei, S. 2017, ‘A traffic sign detection algorithm based on
deep convolutional neural network’, 2016 IEEE International Conference on Signal and Image

18
 Processing, ICSIP 2016, pp. 676–9.
Chen, J., Wang, Y., Wu, Y. & Cai, C. 2017, ‘An ensemble of convolutional neural networks for image
classification based on LSTM’, Proceedings - 2017 International Conference on Green
Informatics, ICGI 2017, pp. 217–22.
Creusen, I.M., Hazelhoff, L. & De With, P.H.N. 2012, ‘Color transformation for improved traffic sign
detection’, Proceedings - International Conference on Image Processing, ICIP, pp. 461–4.
Fifik, M. & Turán, J. 2010, ‘Real Time Recognition System for Traffic Sign Detection and
Classification’, Pace Pacing And Clinical Electrophysiology, pp. 284–7.
Fistrek, T. & Loncaric, S. 2011, ‘Traffic sign detection and recognition using neural networks and
histogram based selection of segmentation method’, ELMAR, 2011 Proceedings, no. September,
pp. 51–4.
Girshick, R., Donahue, J., Darrell, T. & Malik, J. 2014, ‘Rich feature hierarchies for accurate object
detection and semantic segmentation’, Proceedings of the IEEE Computer Society Conference on
Computer Vision and Pattern Recognition, pp. 580–7.
He, K., Zhang, X., Ren, S. & Sun, J. 2015, ‘Spatial Pyramid Pooling in Deep Convolutional Networks
for Visual Recognition’, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
37, no. 9, pp. 1904–16.
Höferlin, B. & Zimmermann, K. 2009, ‘Towards reliable traffic sign recognition’, IEEE Intelligent
Vehicles Symposium, Proceedings, pp. 324–9.
Houben, S., Stallkamp, J., Salmen, J., Schlipsing, M. & Igel, C. 2013, ‘Detection of traffic signs in real-
world images: The German traffic sign detection benchmark’, Proceedings of the International
Joint Conference on Neural Networks, no. May 2014.
Huh, S. & Lee, D. 2010, ‘Seungil Huh and Donghun Lee ( signature-LDA ) as an extension of LDA
that can be applied to features while selecting more discriminating features using label Linear
discriminant analysis ( LDA ) is one of the most popular feature selection methods for cla’,
Machine Learning, vol. 21, no. 12, pp. 1990–6.
Jain, R. & Gianchandani, P.D. 2019, ‘A hybrid approach for detection and recognition of traffic text
sign using MSER and OCR’, Proceedings of the International Conference on I-SMAC (IoT in
Social, Mobile, Analytics and Cloud), I-SMAC 2018, pp. 775–8.
Jurišić, F., Filković, I. & Kalafatić, Z. 2016, ‘Multiple-dataset traffic sign classification with OneCNN’,
Proceedings - 3rd IAPR Asian Conference on Pattern Recognition, ACPR 2015, pp. 614–8.
Kiran, C.G., Prabhu, L. V., Rahiman V., A. & Rajeev, K. 2009, ‘Traffic sign detection and pattern
recognition using support vector machine’, Proceedings of the 7th International Conference on
Advances in Pattern Recognition, ICAPR 2009, pp. 87–90.
Liu, W. & Maruya, K. 2009, ‘Detection and recognition of traffic signs in adverse conditions’, IEEE
Intelligent Vehicles Symposium, Proceedings, pp. 335–40.
Maldonado Bascón, S., Acevedo Rodríguez, J., Lafuente Arroyo, S., Fernndez Caballero, A. & López-
Ferreras, F. 2010, ‘An optimization on pictogram identification for the road-sign recognition task
using SVMs’, Computer Vision and Image Understanding, vol. 114, no. 3, pp. 373–83.
Mustafa, M.K., Allen, T. & Appiah, K. 2019, ‘A comparative review of dynamic neural networks and
hidden Markov model methods for mobile on-device speech recognition’, Neural Computing and
Applications, vol. 31, pp. 891–9.
Pang, S., Ozawa, S. & Kasabov, N. 2005, ‘Incremental linear discriminant analysis for classification of
data streams’, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol.
35, no. 5, pp. 905–14.

19
Qian, R., Zhang, B., Yue, Y., Wang, Z. & Coenen, F. 2016, ‘Robust Chinese traffic sign detection and
recognition with deep convolutional neural network’, Proceedings - International Conference on
Natural Computation, vol. 2016- Janua, pp. 791–6.
Rahmad, C., Rahmah, I.F., Asmara, R.A. & Adhisuwignjo, S. 2018, ‘Indonesian traffic sign detection
and recognition using color and texture feature extraction and SVM classifier’, 2018 International
Conference on Information and Communications Technology, ICOIACT 2018, vol. 2018-Janua,
no. c, pp. 50–5.
Saputra, M.F.A., Widiyaningtyas, T. & Wibawa, A.P. 2018, ‘Illiteracy Classification Using K Means-
Naïve Bayes Algorithm’, JOIV : International Journal on Informatics Visualization, vol. 2, no. 3,
p. 153.
Shaikh, K.D. & Adamuthe, A.C. 2018, Hierarchical Text Classification using Dictionary based
approach and Long-Short Term Memory, no. April.
Shustanov, A. & Yakimov, P. 2017, ‘CNN Design for Real-Time Traffic Sign Recognition’, Procedia
Engineering, vol. 201, pp. 718–25.
Siegler, A. & Kardkov, Z.T. 2011, Real-time traffic sign recognition system Real-time Traffic Sign
Recognition System, no. July 2014.
Swathi, M. & Suresh, K. V. 2018, ‘Automatic traffic sign detection and recognition in video sequences’,
RTEICT 2017 - 2nd IEEE International Conference on Recent Trends in Electronics, Information
and Communication Technology, Proceedings, vol. 2018- Janua, pp. 476–81.
Torresen, J., Bakke, J.W. & Sekanina, L. 2005, Efficient recognition of speed limit signs, vol. 90, pp.
652–6.
Wang, C. 2018, ‘Research and application of traffic sign detection and recognition based on deep
learning’, Proceedings - 2018 International Conference on Robots and Intelligent System, ICRIS
2018, pp. 150–2.
Wu, J. & Tsai, Y. (James) 2013, ‘Real-time Speed Limit Sign Recognition Based on Locally Adaptive
Thresholding and Depth-First-Search’, Photogrammetric Engineering & Remote Sensing, vol. 71,
no. 4, pp. 405–14.
Yuan, Y., Xiong, Z. & Wang, Q. 2017, ‘An Incremental Framework for Video-Based Traffic Sign
Detection, Tracking, and Recognition’, IEEE Transactions on Intelligent Transportation Systems,
vol. 18, no. 7, pp. 1918–29.
Zavadil, J., Tuma, J. & Santos, V.M.F. 2012, ‘Traffic signs detection using blob analysis and pattern
recognition’, Proceedings of the 2012 13th International Carpathian Control Conference, ICCC
2012, pp. 776–9.

20
21