NTCC

Term Paper
On
Automation with Reinforcement Learning
Submitted to
Amity School of Engineering and Technology

Guided By:
Dr. Achyut Shankar
Assistant Professor
Department of Computer Science and Engineering

Submitted By:
Aditya Kumar Singh
Enrollment No: A2305220463
B.TECH CSE

AMITY UNIVERSITY NOIDA UTTAR PRADESH

GAUTAMA BUDDHA NAGAR
 Declaration by the Student

I Aditya Kumar Singh, student of B.Tech (CSE) hereby declare

that the project titled “Automation with Reinforcement
Learning” which is submitted by me to the Department Of
Computer Science Engineering, Amity School Of Engineering
And Technology, Amity University, Uttar Pradesh, Noida, in
partial fulfillment of the requirement for the award of the degree
of Bachelor of Technology in “Computer Science Engineering”,
has not been previously formed the basis for the award of any
degree, diploma or other similar title or recognition.

The author attests that permission has been obtained for the use
of any copyrighted appearing in the project report other than
brief excerpts requiring only proper acknowledgment in
scholarly writing and all such use is acknowledged.

Signature:

Place: Noida
Date: 28/7/21
Name: Aditya Kumar Singh
 Certificate by the Faculty Guide

This is to certify that Mr. Aditya Kumar Singh, student of

B.Tech in Computer Science Engineering has carried out the
work presented in the project of the term paper entitle
“Computer Science Engineering” as a part of first year
programme of Bachelor of Technology in Computer Science
Engineering from Amity School of Engineering and
Technology, Amity University, Noida, Uttar Pradesh under my
supervision.

Dr. Achyut Shankar

Assistant Professor
Amity School of Engineering and
Technology
Amity University, Noida, Uttar
Pradesh
 Contents:

S. No Topic Page
No
1. What is Machine Learning and Reinforcement learning? 01 - 02

2. Need for Automation with Reinforcement Learning 03

3. Reinforcement learning v/s Supervised learning 04

4. Algorithms in Reinforcement learning 05 - 06

5. Applications of Automation with Reinforcement Learning 07 - 08

6. CONCLUSION 09

7. REFERENCE 10
 What is Machine Learning:

Machine learning is a method by which a machine analysis data and

automates on it’s own depending on situation. It is basically a branch of
AI (Artificial Intelligence) that helps it to learn on its own. Machine
learning is used to make human task fast and more efficient.
One of the most basic example of machine learning that we see in our
day today life is Google Chrome. Google search consist of many
machine learning system. So, whenever we use google search it learns
about the data we want to see and shows us the result we want to see
first. This enhances the user experience.
Some other examples of machine learning are image recognition,
speech system, fraud detection system and many more.
Machine learning can be divided in 3 categories:
1. Supervised Learning.
2. Unsupervised learning.
3. Reinforcement leaning.

Supervised learning:
Supervised learning uses datasets to train an algorithm so that it can
classify and predict an outcome precisely. The dataset consist of input
and correct output, this allows the model to learn overtime.
Supervised learning helps in two types of problem that are :
 Classification: It is used to sort the data accurately into
specific
categories.
 Regression: It is used to predict outcomes.

Unsupervised Learning:
Unsupervised learning uses an artificial intelligence (AI) algorithm to
identify patterns in a dataset that contain data points that can neither be
classified nor labeled. The algorithms discover hidden patterns in a data
without any human intervention.
It uses clustering to group unlabeled datasets on their similarities and
differences.
Some algorithms and approach used are:
 Exclusive Clustering: In this type of clustering data points
can exist in one group only.
 Hierarchical Clustering: It is an unsupervised clustering
algorithm that can be both agglomerative or divisive.

Reinforcement Learning:
In Reinforcement learning, a machine learning model must be trained to
make series of decisions. The aim is to train a model to learn to achieve
a certain goal in an uncertain, complex situation.
It differs from both supervised learning and unsupervised learning
because the sample dataset used does not train the model.
Alternatively, it uses trail and error to reach the most favorable outcome.
This has made reinforcement learning the most efficient way to indicate
computer imagination.
An example of reinforcement learning is Youtube. Whenever, after
watching a video on Youtube , User see similar titles that he may like.
Suppose user starts a video but end it halfway. Then the program
understand that recommendation was not a good one and will try
another approach.
 Reinforcement learning v/s Supervised learning

Supervised Learning is machine learning that means the process of

learning a practice of developing a function by itself by learning from

several similar examples. This is a process of learning a generalized

concept from few examples provided those of similar ones.

Reinforcement Learning is also an area of machine learning based on

the concept of behavioural psychology that works on interacting directly

with an environment which plays a key component in the area of Artificial

Intelligence.

Machine Learning is a part of Computer Science where the capability of

a software system or application will be improved by itself using only

data instead of being programmed by programmers or coders.

Though both supervised and reinforcement learning use mapping

between input and output, unlike supervised learning where feedback

provided to the agent is correct set of actions for performing a task.

Reinforcement learning uses rewards and punishment as signals for

positive and negative behavior.

 Need For Automation with Reinforcement Learning

As the era of AI (Artificial Intelligence) is approaching,

designing a system which can perform multiple actions is
difficult. The supervised learning and unsupervised learning are
great with dealing with single task at an instant. But this limits
AI to achieve generality.
Reinforcement learning plays major role to make a system that
can perform multiple variety of tasks, instead of specializing at
just one.
A challenge that most AI face is to perceive an environment.
While doing a task an agent does not have the scope to
observe the complete environment. This partial observation
allows the system to perform the best action based on all the
past observations. So, recalling the past states and taking best
action with respect to current observation is the key for RL to
succeed to solve real world problems.
In an industry, learning-based robots are used to perform
various tasks. The robots used are more efficient than humans,
as robots can perform tasks that would be dangerous for
people.
For example: Deepmind used an AI system to cool Google’s
data centers. This action led to a 40% reduction in energy
consumed. Now these centers are fully automated with the help
of Artificial Intelligence system, and no human intervention is
required.
 Algorithms in Reinforcement Learning

The algorithm used in AI applications are:

Q-Learning:
 Q-Learning is a model free reinforcement learning
algorithm.
 Its objective is to analyze a strategy so the system can
take an action to get maximum needed outcomes.
 It is based on Bellman equation.
 Consider a function Q(s,a), here “a” represents an action
while “s” represent a particular state.
 Working of Q-Learning:
o Initialization of Q-table
o Action (a) is selected to perform
o Perform the selected action
o Analyze the outcome
o In end update the Q-table
State Action Reward State action (SARSA):

 System learns the value function according to the current

action derived from the plan currently used.
 In SARSA algorithms maximum reward for the next state
is not required for updating the Q-value in the table.

Deep Q Neural Network (DQN):

 It is neural networks that uses Q learning to provide

models like simulation of intelligent video game play.

 Applications of Automation with Reinforcement
Learning
Some of the best Applications of Automation with
Reinforcement learning are:
1. Industrial Automation
In many industries, robots that can work on their own are
used to do various tasks. This is because they are more
Efficient and accurate to do the same work compared to
humans. This is also time saving.
For example, while browsing on Internet user gets multiple
recommendation and product related ads these are mostly
fraud. They are detected by an Artificial Intelligence (AI).
These applications create a system that assists in analyzing
complex data on a large scale, allow organizations to avoid
unfamiliar risks and recognize beneficial opportunities.
2. Self-Driving Cars

Self-Driving cars uses sensors, actuators, complex algorithms,

machine learning systems, and powerful processors to run a
software.
Self-Driving cars first create a map of the surroundings using
multiple sensors fixed in different parts of the vehicle. Radar
sensors analysis the position of nearby vehicles. Video
cameras detect traffic lights, read road signs, track other
vehicles. Lidar (light detection and ranging) sensors bounce
pulses of light off the car’s surroundings to measure distances,
detect road edges, and identify lane markings. Ultrasonic
sensors in the wheels detect curbs and other vehicles when
parking.

Sophisticated software then processes all this sensory input,

plots a path, and sends instructions to the car’s actuators,
which control acceleration, braking, and steering. Hard-coded
rules, obstacle avoidance algorithms, predictive modeling, and
object recognition help the software follow traffic rules and
navigate obstacles.
Some of the autonomous driving tasks where reinforcement
learning could be applied include trajectory optimization, motion
planning, dynamic pathing, controller optimization, and
scenario-based learning policies for highways.
 CONCLUSION

Reinforcement Learning solves the problem of learning control

strategies for autonomous agents with least or no data.
Reinforcement Learning algorithms are powerful in machine
learning for collecting and labelling a large datasets patterns
that cost more than data itself.
Reinforcement Learning learn itself continuously, so it
continually gets better and better doing the task at hand.
Learning chess game can be a tedious task under supervised
learning but Reinforcement Learning works swiftly for the same
task.
The trial-and-error method as it attempts its task, with the goal
of maximizing long-term reward can show better results here.
By creating a simulation of an entire business or system, it
becomes possible for an intelligent system to test new actions
or approaches, change course when failures happen ,while
building on successes
 REFERENCE

1.javatpoint.com

2.The Reinforcement Learning Repository,

University of Massachusetts, Amherst.

3.Sutton, Richard S. and Barto Andrew G.,

Reinforcement Learning: An Introduction,
MIT,1998.

4.Wikipedia

5. synopsys.com