International Conference On Innovative Computing And Communication (ICICC 2020)

Car auomation simulator using machine learning

Ms. Vinita Rohillaa ,Dr. Sudeshna Chakrabortyb ,Dr. Rajiv Kumarc

a
Department of Computer Science & Engineering,School of Engineering & Technology,Sharda University, Gr. Noida, Maharaja Surajmal Institute of
Technology, JanakPuri, New Delhi
b
Department of Computer Science & Engineering,School of Engineering & Technology,Sharda University, Gr. Noida
c
G. L. Bajaj Institute of Technology & Management,Department of Computer Science & EngineeringSchool of Engineering Gr. Noida

Abstract: Recent advancement in computation power of computer has enabled research in self – driving vehicle. A self-driving vehicle is autonomous in
nature and do not need any driver for making decisions on the movement of vehicle. Self-driving vehicle reads the data by capturing images from the four
sides of the car and use GPS technology to find path between given source and destination. Once path is fixed then there is need for mathematical model
to make decisions on the movement of vehicle. There are two ways to make decisions. One way is to read the images and detect various objects in the
image and then taking movement decisions but this method requires training on huge set of labeled images even to detect few objects. Second way is to
use CNN to extract features from a given image and training a model to take decisions on movement of vehicle. Training of model using neural networks
requires considerably smaller image dataset than required with object detection. Convolutional Neural Network consists of layers of neurons. Successive
layers of neural network are used to extract features from the image. A simulator for self-driving car is required to simulate real world traffic conditions
and then check the trained model in this environment. This simulation is necessary before model can be implemented on actual vehicle to prevent loss of
life and loss of vehicle due to model inaccuracies. Real world traffic can be simulated with the help of mission games having good graphics like Grand
Theft Auto. Proposed project is to develop a simulator for self-driving vehicle using Grand Theft Auto for road and traffic simulation. This project has
four phases. First phase involves reading input to the game and output from the game. Second phase involves collecting data from the game. Third phase
is to train a mathematical model using CNN. Finally, fourth phase is to check trained model and detecting other objects for driving a vehicle in game.

of acceleration, brake and turn on the basis of captured and processed

1. Introduction
Autonomous Vehicles are the vehicles which can travel a particular
distance between given source and destination on their own. Autonomous
Vehicles have several advantages as compared to human driven vehicles
like they can be used on a search and rescue mission without human
driver, they can be used to transport blind people, using the technology of
self-driving vehicle application can be designed to instruct blind people
for moving on the path, also these vehicles can be used to significantly
reduce the accidents due to human mistakes. But all these advantages
requires handling difficult tasks like dealing with noisy sensor data,
handling variable environment conditions and adapting to new condition.
However with advancement in machine learning algorithms and data
processing power of computers these challenges can be solved using
pattern recognition algorithms and end to end learning algorithm [1, 5, 6].
A simulator for autonomous vehicle is used to simulate the real world
traffic conditions and check that whether the predictions of trained
mathematical model are correct or not. This mathematical model is
formulated using a lot of data processing. If the model is tested directly on
autonomous vehicle then there is huge risk of loss of life and loss of
vehicle. The program for autonomous vehicle should be able to capture
image of the view, process that image to find lane and objects like other
vehicles on the road. Also the program should be able to make decisions
view image. Also the simulator must be able to capture the image of
speedometer so as to interpret the current driving speed and this requires
an OCR.
2. Literature Survey
In recent years there has been a lot of research on autonomous vehicle.
The DARPA Autonomous vehicle(DAVE)[5] which was the initial
project of Defense Advanced Research Projects Agency’s (DARPA) was
implemented. It used a radio control (RC) car to move in a junk-filled
way. It was used for driving data. Later based on the DAVE NVIDIA
built an autonomous vehicle that could drive safely on traffic roads and
had the ability to recognize patterns without any explicit need to for
finding railings, lane markings, foot path etc. so as to avoid a collection of
if then else rules. The DAVE-2 was inspired by works of Pomerleau [6]
who designed the Autonomous Land Vehicle in a Neural Network
(ALVINN) system. All the advancement in research is possible due to
advancement in technology especially increase in the computation power
of computers and high storage capacity of computers. All these
advancements enable training of mathematical models in feasible time
period. Also these advancements have enabled the real time computation
and data processing required for handling terra bytes of data. Although

Ms. Vinita Rohillaa ,Dr. Sudeshna Chakraborty ,Dr. Rajiv Kumar 1

Electronic copy available at: https://ssrn.com/abstract=3566915

 International Conference On Innovative Computing And Communication (ICICC 2020)

GPUs are bit expensive to purchase but Cloud Computing has enabled
renting of GPU on clouds. Earlier methods of object detection used
machine learning and with increase in computation power more data was
collected so that more powerful models could be built. Simple recognition
tasks can be solved quite easily with datasets of small size, especially if
they are augmented with some kind of label-preserving the
transformations. Since the object in real world show significant variability
and using object detection to learn about these objects would not be of any
help with small data sets. A model has to be developed which has three
learning capacity. There is a problem of object detection, so the given
model should also compensate for the data not available. For eg.
Convolutional neural networks (CNNs) gives better improvement to
normal object detection. Building of autonomous vehicle requires building
a mathematical model that can take decisions on vehicle movement in a
given scenario. These mathematical models require processing huge
amount of sensor data in the form of images. Pattern recognition
algorithms are used for training of this data. These algorithms try to
recognize patterns by extracting features from the given image data.
Pattern recognition relies heavily on statistics, probability and algebra.
Before the introduction of Convolutional Neural Network (CNN), pattern
recognition was done using feature extraction followed by classification.
The convolution operation in CNN captures the 2D nature of the image.
CNN learning algorithms are usually implemented on parallel graphical
processing unit (GPU) which significantly decreases the time for
inference.
In Hough transform each point crossing corresponds to a
straight line. By Hough transform, the line equation
connecting these points can be calculated.
4. Implementation and Results
The lane line can be displayed by preprocessing of the frames of the
image. It is difficult to detect the lanes due to white residue present in the
frames. So the basic preprocessing for lane finding not work well.
Therefore the HSV color conversion is added in the preprocessing phase
and extract frame’s white features. This will improve detection accuracy
for lanes. For enumerating the lane detection accuracy, the detection rate
is used to estimate the proposed method performance for the detection of
lanes. For the good results of proposed method the size of image is set to
be of same size and randomly taking many images in the test set. The
detection efficiency of proposed work is compared with the basic
processing method for checking the performance. In the proposed work
,the method that selects ROI (Region of Interest) area which is based on
lane color is also compared with the proposed ROI(Region of Interest).
The proposed method results provide highest detection rate.
Following are the observations and experimental results:

3. Lane detection for the designed simulator

Lane detection is performed based on the enhanced pre-

processing. Feature extraction is an important part in the lane
detection.. There are many methods for edge detection like
canny, Sobel and Laplacian transform etc.. In this Canny
transform method is chosen which gives better performance.
After the proposed ROI selection Canny edge detection was
done. Feature-based and model based are the two methods of Fig 1: Detecting passing vehicle
lane detection. The colour and edges of lanes are detected by
method based feature. The efficiency and accuracy of lane In Fig1, passing by truck is detected. The left figure shows the game
detection can be increased by this method. In this Hough line simulation and right figure shows copy of simulation, on which object
detected function that is used for the image processing and detection is shown. The difference between two images is due to
drawing lines in original image. Its gives better performance processing lag which is 0.5-1 seconds.
in terms of speed than the other methods. The Hough
transform efficiently process straight lines and is insensitive to
noise. It is also used for the extraction of the lane line
parameters for the detection of lane. The Hough transform can
detect any shape and can be expressed by mathematical
formula in image processing even if the shape is distorted. In
this work, for straight-line detection, Hough transform is used.

Ms. Vinita Rohillaa ,Dr. Sudeshna Chakraborty ,Dr. Rajiv Kumar 2

Electronic copy available at: https://ssrn.com/abstract=3566915

 International Conference On Innovative Computing And Communication (ICICC 2020)

added to detect straight lanes. In future, we will implement a advanced

lane detection method for improving the performance. Following are some
of the things that can be implemented as future scope.

Fig 2: Movement in traffic

In Fig 2,Two graphs are plotted. First is, Speed Vs. Time. This graph
shows the speed of the vehicle at every instance of time. Second graph is a
histogram, Collision Vs Speed, which shows how many collision occur
for different speed ranges. Both the graphs are real time i.e. both are
plotted when the vehicle is running according to the model. Vehicle is Fig 4: Barrier Collision
shown moving in traffic. The speed is reduced slowly due to vehicle
detection and proper lane is selected by vehicle to stop collision. As there In Fig 4, the vehicle collides with the barrier, instead of moving straight
is a car in front that is why vehicle is trying to reduce speed to prevent on the clear road. The Speed Vs Time graph shows that at the time of
collision. If the speed is greater than threshold value, then breaks are
collision, speed of vehicle is reduced drastically.
applied, so that vehicle does not travel at higher speed. Collision has
higher chance to occur at higher speed.

Fig 3 : Turning left on object detection and moving in lane

Fig 5:Front Collision

In Fig 3, When a car is detected in front, vehicle moves away from it. It
moves in the left side of the lane, but away from the railing, within the
lane, within the lane. In this way it is able to prevent collision.

In Fig 5,the front vehicle could not be detected. So, collision occurred.
The Speed Vs Time graph shows that speed decreased drastically at the
5. Conclusion and Future Scope
time of collision. This is due to some inaccuracies that it is not possible to
In the proposed work, lane detection pre-processing and ROI selection detect all kinds of vehicles.
methods are implemented for lane detection. Before the pre processing
white extraction and edge extraction is added. The Hough transform is

Ms. Vinita Rohillaa ,Dr. Sudeshna Chakraborty ,Dr. Rajiv Kumar 3

Electronic copy available at: https://ssrn.com/abstract=3566915

 International Conference On Innovative Computing And Communication (ICICC 2020)

Fig 6: Moving in wrong side of road.

In fig 6, Vehicle moves in opposite direction, As there is no railing or

barrier, the vehicle moves in wrong side of road.

REFERENCES

M. Daily,S.Medasani ,R.Behringer,M.Trivedi, “Self-Driving Cars”M. Daily, S.

Medasani, R. Behringer, M. Trivedi, "Self-Driving Cars", in Computer,
vol. 50, no. 12, pp. 18-23, December 2017.
N. J. Goodall, "Can you program ethics into a self-driving car?", IEEE
Spectrum, vol. 53, no. 6, pp. 28-58, June 2016.
N. A. Greenblatt, "Self-driving cars and the law", IEEE Spectrum, vol. 53, no.
2, pp. 46- 51, Feb. 2016.

Estonia Allowing a Number of Self-Driving Cars on the Streets Starting Today

Ministry of Economic Affairs and Communication, [online] Available:
https://www.mkm.ee/en/news/estonia-allowing-number-self-drivingcars-
streets-starting- today.
E. Coelingh, J. Nilsson, J. Buffum, "Driving tests for self-driving cars", IEEE
Spectrum, vol. 55, no. 3, pp. 40-45, March 2018.
E. Ackerman, "Lidar that will make self-driving cars affordable [News]", IEEE
Spectrum, vol. 53, no. 10, pp. 14-14, October 2016

Ms. Vinita Rohillaa ,Dr. Sudeshna Chakraborty ,Dr. Rajiv Kumar 4

Electronic copy available at: https://ssrn.com/abstract=3566915