ADA Project Progress Report - 1

Automating the Movement of Car using Images from the front part of
the Car by applying Convolutional Neural Networks
Deep Learning for Behavioral Cloning.

Submitted by
Jai Chaudhry (2K18/SE/069)
Ishaan Jain (2K18/SE/067)

Under the supervision of

DR. RAHUL GUPTA
Professor
Department of Computer Science
and Engineering
Delhi Technological University

DEPARTMENT OF COMPUTER SCIENCE AND

ENGINEERING

DELHI TECHNOLOGICAL UNIVERSITY

(FORMERLY DELHI COLLEGE OF ENGINEERING)

SHAHABAD DAULATPUR, BAWANA ROAD,

DELHI – 110042
 October 11, 2020

Title:
Implementation of Self-Driving Car using Udacity Open Source Simulator.

Team Members:
i. Jai Chaudhry (2K18/SE/069)
ii. Ishaan Jain (2K18/SE/067)

Abstract
Introduction
Problem Definition
Tools and Technology used
Explanation of different modules with snapshots
Implementation code
Bibliography
Abstract
Self-driving cars have become a leading platform due to major advances in computer power
and performance and the effectiveness of Mission and Deep Learning Algorithms training.
The aim of the project is to train a convolutional neural network to drive a car agent in the
direction of Udacity's Simulator environment. Udacity released this simulator as an open
source software for learning purposes. Driving a car independently requires learning to
control steering angle, compression and brakes. The code-making process is used to mimic a
person's driving behavior using information collected in the training mode of the simulator.
After training on this data, the Deep Learning model then drives the car in independent mode.

Introduction:
Problem Definition
The challenge is to imitate the way a person drives in a simulator with the help of a model
trained in deep neural networks. The concept called Behavioral Cloning, to simulate how
people drive a car in a simulator. The simulator consists of two tracks and two modes,
namely, training mode and independent mode. The database created from user to user,
driving the car in training mode. This is followed by a test on the track, to see how the in-
depth learning model works after training by that user data.

1.2 Solution Approach

The high level of startup can be seen in Figure (…)
The problem is solved in the following steps:
• The simulator is used to collect data by driving the car on the training mode keyboard,
provides the input details of "driving well" in the form of driving_log (.csv file) and a set of
images. The simulator works as a server and piping these images and data log to a Python
client.

• Client (Python Program) is a machine learning model developed using Deep Neural
Networks. These types are built into the Kera (advanced API with TensorFlow). The Keras
provide a series of models to build a straight line of network layers. Such models are used in
the project to train more in the database as a second step. Detailed description of CNN
models can be seen in (…)
• Once the model is trained, it provides steering angles and throttle driving in autonomous
server mode (simulator).

• These modules, or inputs, are piped to the back of the server and used to drive the car
independently in the simulator and keep it from falling off the track.

Tools and Technology used

The techniques used to make this project are described in this section.

TensorFlow: This open library of data flow system. It is widely used in machine learning
applications. It is also used as a library of mathematics and arithmetic. For the Camera
project, an advanced API uses TensorFlow as the backend is used. Kera simplifies model
building as it is easy to use. Various libraries are available at Python to assist with machine
learning projects.

Several of those libraries have improved the effectiveness of this project. Few of them are
mentioned in this section.
First, "Numpy" offers a collection of high-quality math work to support multiple metrics and
lists. This is used for the rapid calculation of weights (gradients) in neural networks.
Second, the "matplotlib" is a Python machine learning library with a variety of editing and
visual effects.
The other is OpenCV (Open Source Computer Vision Library) designed for image and
enlargement techniques.

The machine on which the work is built is a personal computer with the following
configuration:
• System: Intel (R) Core i8-8300H @ 2.3GHz
• RAM: 8GB
• System: 64bit OS, x64 processor

Explanation of different modules with snapshots

About the chosen Neural Network CNN:
CNN is a type of feed-forward neural network computing system that can be

used to learn from input data. Learning is accomplished by determining the set of

weights or filter values that allow the network to model behavior in terms of training

data. The desired result and the output produced by CNN launched at random weights

will vary. This difference (error made) is still propagated by using CNN layers to adjust

the weight of the neurons, which reduce the error and allow us to produce the output

closer to what you want.

CNN is good at capturing location data from images. It uses filters that look at

the input regions of the specified window size and then map them to other output. Then

slide the window with a defined line to other regions, covering the entire image. Each

layer of the convolution filter thus captures the properties of this image to be inserted

sequentially in the following layers, capturing details such as image lines, and shapes,

and then objects in the next layers. CNN can be well-suited to feed database images and

classify them into categories.

EXPERIMENTAL CONFIGURATIONS
This section contains configurations used to set up a Python Client training model to provide
Neural Network driving results in a simulator.
Boundary adjustments and rigorous testing have been tried to achieve the best combination.
Although each model has its own unique characteristics and is different in its
performance with each tweak, the following configuration combination can be
considered as the best:
• Consecutive models built on Kera with deep neural network layers
are used to train data.
• Models are only trained using a database from Track_1.
• 90% of the database is used for training, 10% is used for certification.
• Epochs = 40, e.g. Multiplication value or completeness of completion
data. It has been tested with a larger number of epochs as well, but the model has tried to
"overdo it". In other words, the model reads the details in the training well, while
contributing to the performance of the new database.
• Batch size = 32, e.g. Number of image samples distributed by
a network, such as a database as a complete database, is too large to be transmitted
simultaneously.
• Reading level of = 0.0001, 0.00005, 0.000007, eg that coefficients of
weights or gradients change in network.
• The ModelCheckpoint () function provided by Camera to save test locations
and keeping Epoch at the forefront in terms of loss of validation. There is a different layer of
Convolution, Flatten, Dropout, Dense and so on, which can be used to make Neural
Network models. Of the various structures tested, the best is discussed in other parts
of this report.

NETWORK ARCHITECTURE
Many architectural combinations have tried to predict the steering angle, that is how to drive
a car in independent mode. Neural Network layers are arranged in series and a variety of
layers of Time-Distributed Convolution, Flatten, Dropout, Dense etc. are used in the
construction of buildings.

Pg-25

Implementation code
AUGMENTATION AND IMAGE PRE-PROCESSING

• Crop
Images in the database have appropriate features in the lower part where the road is visible.
The external environment above a certain part of the image will not be used to determine
output and can therefore be cropped. About 30% of the upper part of the image is cut and
transferred to a training set. Captions of the code and image conversion after cropping and
resizing the original image can be seen in Figure (…)
• Flip (horizontal)
The image is scanned up (e.g. the original image screenshot has been transferred to the
database). The reason for this is that the model is trained with the same types of turns on
opposite sides. This is important because Track_1 involves a lot of left turns. Captions of the
code and the image conversion after scanning can be seen in Figure (…)

• Translation (horizontal and vertical)

The image is converted to a small amount horizontally and vertically.
Caption of code and image conversion after translation can be seen in Figure (…)

• Brightness
To make standard low light conditions, light enhancement has been a great help. Captions of
the code and image modification after the Brightness transition can be seen in Figure (…)

• Random Shadows
Or after looking at the bright conditions, there is still a chance that there will be shadows on
the road. This will give an example of half and a half illuminated scenes in the picture.
Distributing random shadows and solving this image balancing problem, this expansion is
used in the database. Caption of code and image modification after adding a random shadow
can be seen in Figure (…)

• Random blur
Considering the effect of distortion on the camera while taking pictures, this addition used as
a snapshot is not always clear. Sometimes, the camera doesn't work, but the car still needs to
adjust to the situation and keep the car stable. This random addition of blurring can take into
account such situations. Sample sample code and transformation can be seen in Figure (…)
Bibliography
(REFERENCES)

Nvidia model
References
(To learn more about the latest trends and research in this domain, you can refer to the following papers
referred for this literature survey)

1. Hussain, R., & Zeadally, S. (2018). Autonomous cars: Research results, issues and future challenges. IEEE
Communications Surveys & Tutorials.

2. Badue, C., Guidolini, R., Carneiro, R. V., Azevedo, P., Cardoso, V. B., Forechi, A., … & Oliveira-Santos, T.
(2019). Self-Driving Cars: A Survey. arXiv preprint arXiv:1901.04407.

3. Heylen, J., Iven, S., De Brabandere, B., Oramas, J., Van Gool, L., & Tuytelaars, T. (2018, March). From Pixels to
Actions: Learning to Drive a Car with Deep Neural Networks. In 2018 IEEE Winter Conference on Applications
of Computer Vision (WACV) (pp. 606–615). IEEE.

4. Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., … & Zhang, X. (2016). End to end
learning for self-driving cars. arXiv preprint arXiv:1604.07316.

5. Stafylopatis, A., & Blekas, K. (1998). Autonomous vehicle navigation using evolutionary reinforcement
learning. European Journal of Operational Research, 108(2), 306–318.

6. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017). CARLA: An open urban driving simulator.
arXiv preprint arXiv:1711.03938.

7. Sallab, A. E., Abdou, M., Perot, E., & Yogamani, S. (2017). Deep reinforcement learning framework for
autonomous driving. Electronic Imaging, 2017(19), 70–76.

8. Johnson-Roberson, M., Barto, C., Mehta, R., Sridhar, S. N., Rosaen, K., & Vasudevan, R. (2016). Driving in the
matrix: Can virtual worlds replace human-generated annotations for real world tasks?. arXiv preprint
arXiv:1610.01983.

9. Sharifzadeh, S., Chiotellis, I., Triebel, R., & Cremers, D. (2016). Learning to drive using inverse reinforcement
learning and deep q-networks. arXiv preprint arXiv:1612.03653.

10. Faisal, A., Yigitcanlar, T., Kamruzzaman, M., & Currie, G. (2019). Understanding autonomous vehicles: A
systematic literature review on capability, impact, planning and policy. Journal of Transport and Land Use,
12(1), 45–72.
11. Zhang, X., Gao, H., Guo, M., Li, G., Liu, Y., & Li, D. (2016). A study on key technologies of unmanned driving.
CAAI Transactions on Intelligence Technology, 1(1), 4–13.

12. Du, S., Guo, H., & Simpson, A. (2017). Self-driving car steering angle prediction based on image recognition.
Department of Computer Science, Stanford University, Tech. Rep. CS231–626.

13. Maqueda, A. I., Loquercio, A., Gallego, G., García, N., & Scaramuzza, D. (2018). Event-based vision meets
deep learning on steering prediction for self-driving cars. In Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition (pp. 5419–5427).

14. Santana, E., & Hotz, G. (2016). Learning a driving simulator. arXiv preprint arXiv:1608.01230.

15. Rosenzweig, J., & Bartl, M. (2015). A review and analysis of literature on autonomous driving. E-Journal
Making-of Innovation.