B.M.

S COLLEGE OF ENGINEERING BENGALURU

Autonomous Institute, Affiliated to VTU

An Technical Seminar Report

Controlled Parking for Self-Driving Cars

Submitted in partial fulfillment for the award of degree of

Bachelor of Engineering
in
Computer Science and Engineering

Submitted by:
Chirag Mahaveer Chivate
1BM20CS200

Work carried out at

Internal Guide

Dr. K. Panimozhi, Department of Computer Science and Engineering

B.M.S College of Engineering
Bull Temple Road, Basavanagudi, Bangalore 560 019
2021-2022
 B.M.S COLLEGE OF ENGINEERING
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

DECLARATION

I, Chirag Mahaveer Chivate 1BM20CS200 student of 4th Semester, B.E, Department of

Computer Science and Engineering, B.M.S College of Engineering, Bangalore, hereby declare
that, this technical seminar entitled "Controlled Parkingfor Self-Driving Cars" has been carried
out under the guidance of
DR. K.Panimozhi , Professor, Department of CSE, BMS College of Engineering, Bangalore
during the academic semester April - September 2022. I also declare that to the best of our
knowledge and belief, the technical seminar report is not from part of any other report by any
other students.

Signature of the Candidate,

Chirag Mahaveer Chivate
1BM20CS200
 B.M.S COLLEGE OF ENGINEERING
DEPARTMENT OF COMPUTER SCIENCE AND
ENGINEERING

CERTIFICATE

This is to certify that the Technical Seminar titled “Controlled Parking for Self-
Driving Cars” has been carried out by Chirag Mahaveer Chivate 1BM20CS200
during the academic year 2021-2022.

Signature of the guide

DR K.Panimozhi,
Department of Computer Science and Engineering
BMS College of Engineering, Bangalore
 TABLE OF CONTENTS

S.No Contents PAGE

NO

1. Abstract I

2. Chapter 1: Introduction 1

3. Chapter 2: Literary Survey 2

4. Chapter3: Methodology /Techniques Or Algorithm 4

Used

5. Chapter 4: Description Of Tools Selected 8

6. Chapter 5: Detailed Description Of Modules 12

Implemented

7. Chapter 6: New Learnings From The Topic 13

8. References 14
LIST OF FIGURES AND DIAGRAMS

S.No Contents
1. Figure 3.1: Simple Parking Lot Structure

2. Figure 3.2: Graph Of The Parking Lot

3. Figure 3.3: Entrances And Exits

4. Figure 3.4: Visualization From Simulation

5. Figure 4.1: The Parking Lot

6. Figure 4.2: The Agent

7. Table 4.1: Reward Function

8. Table 4.2: Hyperparameters

 I.Abstract

Abstract—Autonomous parking system is essential in reducing time for waiting to park in the
parking spaces or looking for space, particularly in the smart city. We propose the auto-parking
car simulation framework using proximal policy optimization (PPO) for deep reinforcement
learning in a moderately complex parking scenario which comprises basic and challenging zones
for parking. Different configurations are explored including sparse and dense rewards combining
with checkpoints, orientation reward and stay-in-collision punishment. It shows high success
parking rate in a basic parking zone up to at more than 95%. For the more difficult parking zone,
the model works better when each zone is trained separately.

Keywords—Autonomous Driving Car, Deep Reinforcement Learning, Autonomous Parking Car,

Smart Parking Lot, Proximal Policy Optimization
Chapter 1: Introduction

1.1Overview

An autonomous car (AC) is a vehicle that is adept at sensing its surroundings and navigating
without human involvement. It is sometimes also called a driverless car or a self-driving car or
a robotic car . Google with its self-driving car which is operated by its AI software named
Google Chauffeur and Tesla with its autopilot system on Tesla Model S which designed to be a
computer on wheel are already working on to make these cars publically available. So it can be
anticipated that in the near future, cars on the road will be autonomous as the CEO of Tesla
Motors Elon Musk expects that they would be able to develop mature serial production version
of fully self-driving car by 2018 . These self-driving cars have to blend into the current transport
system without disturbing it, which means that these should be able to perform all the tasks
which a normal car could do when driven by a human being. Since this technology is in its early
stages, there are certain problems that require addressing. The foremost problem would be
situations where perfect solution doesn’t exist; for example, a car braking in front of an AC.

1.2 Motivation
We believe when all the cars in the parking lot are autonomous, we can do better, because the
cars can move concurrently without human intervention. This will lead to improved utilization of
parking space and would minimize the time to move the AC in and out of the parking space.

1.3 Objective
1) We identified the problems for ACs’ parking in some controlled environments.
2) We devised initial algorithms that can be applied to any kinds of parking lots as long as they
can be modeled as graphs.
3) We suggested some parking structures which are suitable for ACs with the tradeoff of space
and energy consumptions.
4) Finally, with experimental results, we proved that the AC parking systems with the help of
some guides are efficient and effective in order to save cost, time and the utilization of parking
spaces.

Page : 1
Chapter 2:LITREATURE SURVEY

Parking an AC in a parking lot is similar in many aspects to the automated parking systems
(APS). APS is a mechanical system designed to minimize the area and /or volume required
for parking cars. A lot of work has been done in the field of APS . Similarly, Sebestyen et al.
present the concept and the implementation of a fully automated and intelligent parking
system, based on prediction through probability algorithms. The difference between APS and
our work is that it is structure independent i.e. we don’t require any special external
mechanical system to park the cars

2.1AUTONOMOUS CAR PARKING SYSTEM:

Every parking lot structure is different so first thing is to represent all different structure into
something more understandable so that we could apply some path finding algorithm on it.
For that we used graphs for the representation of the parking lot structure and used that graph
in our algorithm to find the shortest path.
Graph Generation
To represent the parking lot, we need to build a graph of it in which each parking slot,
pathway and entrance point will be represented as P-node, T-node and E-node, respectively.
Now, in order to link these nodes there are some rules which are needed to be followed to
complete the graph.
• P-node can only be linked to one or more T-node.
• T-node can be linked to any node.
• E-node can only be linked to a T-node.

Page : 2
2.2Nearest Slot Parking :
The basic purpose of this algorithm is to tell the car about the nearest parking slot available
to it from the entrance i.e. to act as a “Parking Control Service” (PCS). To do so, we have
used the graph of a parking lot and a shortest path finding algorithm (Dijkstra's algorithm).
The following are our preprocessing steps:
• Generate the shortest path to every P-node from every E-node and save them in a list such
that every E-node has its own list of closest paths with the values sorted in ascending order of
their cost.
• Generate the shortest path to every E-node from every P-node and save them in a list such
that every P-node has its own list of closest paths with the values sorted in ascending order of
their cost.
The reason for generating these two sets of path separately is that a shortest path from a
particular E-node to some P-node, while parking, may not be the shortest path while exiting,
since it can have more options for E-nodes. So now, whenever a car arrives at a particular E-
node, it will request the PCS to find the closest unoccupied P-node in the graph and direct the
car to that position and change that Pnode’s state to occupied. Similarly, whenever a car
wants to leave a request is made to the PCS for the nearest E-node.

Page : 3
Chapter 3: METHODOLOGY/TECHNIQUES OR ALGORITHM
USED

For our experimentation, we coded our algorithm in Java SE 8 which simulates the nearest
slot parking. Then we used the Processing programming language to visualize our results.
We ran two scenarios on the same kind of parking structures with a capacity of 15 parking
spaces for 7 days where each day 40 cars came in and parked for a duration of 2- 8 hours.
The cost on the links is kept same. We kept the size of the parking lot small to make things
more understandable. As shown in Figure 3.1.

(FIGURE 3.1)

In the first scenario, as shown figure 3.2,the AC is not taking help from the PCS (we refer to
this as the non-guided approach further on). So, in this case, the autonomous car will try to
find the available parking slot just by moving through the parking lot and when it finds an
empty parking slot, it will park itself there. The car will follow this movement pattern; go
straight and look for parking slot on both sides and take turn whenever it comes unless that
area is already explored. If a car enters the parking lot from E-0 (E means Entrance/Exit) and
after moving through the parking lot it finds an empty parking spot at P-16 (P means Parking
Slot) then, it should have followed one of the following paths in the graph to reach that spot 0
to 3 to 2 to 7 to 12 to 17 to 16 or 0 to 3 to 4 to 9 to 14 to 19 to 18 to 17 to 16.

Page : 4
 (FIGURE 3.2)

In the second scenario, the autonomous car will take help from the PCS (we refer to this as
the guided approach further on). The service will calculate the path to the closest empty
parking slot from the car’s current position and will send those directions to the car.
After running both of these scenarios, what we observed at the beginning (when the parking
lot was empty) was that both of the approaches were parking the car at the closest slot. That
was because the path to the closest location was the one near the entrance, but as the path to
the closest node becomes complex, the guided approach becomes much better
For example, one of the parking lot graphs that we used in the experimentation is given in
Fig. 3.3.

Page : 5
 (FIGURE 3.3)
As we discussed earlier, the first step was to convert the parking lot structure into a graph.
The three different kind of nodes i.e. P, T and E, are colored in grey, white and black
respectively. To make things simple, the cost for each link was set to 1. When we ran both of
the scenarios on this graph, we observed these things
In both cases car 1 came from E-0 which was parked at P13 (cost 2), of the closest node from
E-0. Cars 2,3 and 4, which came from E-0, were parked at P-nodes 14(cost 2), 10(cost 3) and
11(cost 3) respectively, in both cases. When Car 5 came from E-0 (in the non-guided
scenario), it was parked at P-7 (cost 4) and in case of the guided scenario, it was parked at P-
15(cost 3) as shown Fig 3. 4.

The non-guided approach will always go for the local optimal solution and the guided
approach will always provide the global optimal solution. So, in the current example by using
the guided approach, the total distance covered was decreased by 21.4% (on average) as
compared to the non-guided approach i.e. if distance covered with the non-guided approach
was 100 meters, it was reduced to 78.6 meters with the guided approach.
We also noticed that by increase the size of the parking lot, the distance covered by the non-
guided approach increases more as compared to the guided approach.
Another advantage of using the controlled approach is that since the system is tracking every
car inside the parking lot, we can handle the load distribution on the exit node as well, which
will decrease the time taken by the car to move out of the parking lot.
Page : 6
(FIGURE 3.4)

Page : 7
Chapter 4: DESCRIPTION OF TOOL SELECTED

4.1 SELF-PARKING SIMULATION FRAMEWORK

We design a moderately complex and perpendicular parking scenario as shown in Fig 1. We
simulate the parking scenarios and a vehicle in Unity Engine and employs the Unity ML-Agents
for passing the information between the simulation and TensorFlow library to construct and
training the neural network and OpenAI Gym library for creating reinforcement learning’s
environment. Unity, which is a popular game engine, is a reliable tool for realistic simulation
using physics and realistic texture in many games nowadays. The example of reinforcement
learning model trained for autonomous driving car using Unity ML-Agents is in . We use
TensorFlow for running Proximal Policy Optimization (PPO) algorithm which generate a model
or an agent which collects the experience for 20 million steps and is trained in an interval every
4096 steps. The agent is trained based on reward functions to provide the next action and state
representation for the vehicle as detailed below

(FIGURE 4.1)
4.1.1 Environment
The environment is a one floor parking scenario displayed in Fig. 4.1 which has total of 72
parking slots. The entrance is on the bottom-right corner and the exit is on the top-right corner.
The parking lot is one-way road as shown in Fig. 4.1. The information of the targeted parking
slot position is submitted to the agent every step.

Page : 8
4.1.2 Agent
The agent, in this case, is a car model in simulation from Eye-maginary which is a 4-wheel low
polygon model. It is suitable for training phase because of fast rendering speed. It is a car
consisted of 8 depth sensors; each has the configuration length of 10 units in the direction as
shown in Fig. 4.2. The maximum speed of this car when training is capped at 80 km/hour for
safety purposes. Note that a unit in Unity is equal to one meter in a physical scenario.

(FIGURE 4.2)
4.1.3 Reward Function

The reward function is how the environment calculate how good of the agent action performs in
each step. Our reward function in this experiment is as in Table I. The negative reward is given to
the action that the agent should not do which can be accumulated until it overcomes the positive
reward it get especially for the action that affect the assets and safety of the people such as
collision. However, the timeout factor has less impact than reward for success parking, so it is set
to a fixed value which is less than the reward from a success parking. We divided the experiment
into 3 configurations based on the reward functions as follows:
•
• 4.1.3 i) Sparse Reward: a reward is given only every 10% decreasing of the distance
between the agent and the target.
•
• 4.1.3 ii) Dense Reward and Checkpoints: a small amount of positive reward is given
at every step to give the agent signal gradually and let it know more frequently
whether it is on the right track. Additionally, the agent receive bonus reward when
reaching the checkpoints .
•
•
•
•
•
Page : 9
4.1.3 iii) Dense Reward and Orientation Reward and Stay-inCollision Punishment: a reward is
given at every step. Punishment (minus reward) is added when crashing occurred and remain
stuck. Also, decrease the reward related to difference between the orientation of parking slot and
the agent when parking successfully. This decreasing reward is capped at a positive value as in
Table 4.1 when parking perpendicular to the slot which help encourage the model to try to get
higher reward from parking rather than capped at zero or negative value which make the model
avoid doing the action.

• (Table 4.1)

4.1.4 State Representation and Action

• The agent receives state for a total of 25 values from the parking lot. Sixteen is from 8
depth sensors, each one sending 2 values. The first one is a Boolean value indicating the
detection of the object and another one is the distance of a detected object to the sensor.
Six values are the vector in 2-Dimension of the agent position, the target parking slot
position and the displacement between agent and target. Three values are the car
orientations in degree angle, the linear velocity, and the angular velocity of the agent. As
the agent receives these above values as input, it returns 2 real numbers which are the
actions for the agent to execute: the torque applied to the front wheel and the steering
angle of the front wheel.

Page : 10
4.1.5 Hyperparameters
• The value of hyperparameters that being used are listed in Table 4.2 which are suggested
values for training the model which has continuous action space from Unity MLAgents
document.

(Table 4.2)

Page : 11
Chapter 5:DETAILED DESCRIPTION OF MODULES
IMPLEMENTED

Handling and navigating an AC in different environment and surroundings is a relatively new so

there is a lot of room to improve and optimize the way things were done before. Since these cars
can communicate with each other make decision on their own so we have lot more information
to process and make more intelligent decisions based on that information. The basic purpose of
this work was to explore what are the possible problems that we can face while parking an AC in
a parking lot. We also suggested some methods for parking ACs in the parking lot with some
tradeoffs.
As a part of the future work, instead of doing experiments with simulated cars' in & out data, we
will collect the real inand-out data from real parking lots to use it on real parking lot structures
so, the cost of the links between the nodes will depend upon the distance as well as the number
of maneuvers required to reach. This will discover practical problems when designing controlled
parking services. We will also work solving the blocked parking lot problem in minimum steps
to minimize the cost of energy being used in such kind of parking lots.

Page : 12
 Chapter 6: NEW LEARNINGS FROM THE TOPIC

Artificial intelligence (AI) has a vital role in human life and society. It is being embed in many
electronic equipment for automate uses and a part of utility software. Reinforcement learning
is one of many techniques in the field of machine learning that is becoming increasingly
popular. It is the method of learning from trial and error to gather the experience and then
adjust the behavior of the model based on the reward of each experience; the reward indicates
the goodness of the selected action in the current state of environment. As of today,
reinforcement learning algorithms have been developed to be more capable in terms of
precision, data efficiency and speed. Deep reinforcement learning, the combination of
reinforcement learning and deep learning, is also extremely flexible and powerful method
which is used to solve many problems. Specifically it is adapted for the simulation programs
to simulate the appropriate environment in the training process which has advantages of
reducing the cost that may occur from changes or degradation of the real objects during the
experimentation.

Page : 13
REFERENCES

➢ IEEE RESEARCH PAPERS

1. Controlled Parking for Self-Driving Cars, 2016 IEEE International Conference on Systems,
Man, and Cybernetics
➢https://sci-hub.se/10.1109/smc.2016.7844509

2. Self-Parking Car Simulation using Reinforcement Learning Approach for Moderate Complexity
Parking Scenario, 2020 17th International Conference on Electrical Engineering/Electronics,
Computer, Telecommunications and Information Technology
➢ https://sci-hub.se/10.1109/ecti-con49241.2020.91582989

Page : 14