DEEP LEARNING

Group 8
Abhishek Batni 57
Anirudh Roy 62
Bidush Dey 67
Raviteja Reddy 88
Sachin Sharma 92
Shuvam Jaiswal 100
Vaibhav Bajpai 106
Rishikesh K 146
AI - ML - DL
 Artificial Intelligence
The science and engineering of making intelligent machines, especially intelligent computer programs. ∼ John McCarthy

Applications of AI
Expert Systems
Gaming Applications imparting
Strategic games like reasoning and advising.
chess, poker etc. They provide explanation
 Artificial Intelligence is a way of making a and advice to the users.
computer, a computer-controlled robot, or a Language Processing
software think intelligently, in the similar Intelligent Robots
Interact with the computer that
manner the intelligent humans think. Applications imparting
understands natural language
reasoning and advising.
spoken by humans.
 AI is accomplished by studying how human They provide explanation
brain thinks, and how humans learn, decide, Visual Systems and advice to the users.
and work while trying to solve a problem, and  UAV taking aerial
then using the outcomes of this study as a photographs to study spatial
basis of developing intelligent software and information of the area
Artificial
systems.  Recognizing the face of
criminal with the stored Intelligence
Goals of AI portrait made by forensic
artist Machine
 To Create Expert Systems: The systems Learning
which exhibit intelligent behavior, learn, Handwriting Recognition
demonstrate, explain, and advice its users. Software reads the text written
on paper by a pen or on screen Deep
 To Implement Human Intelligence in by a stylus and converts it into
editable text
Learning
Machines: Creating systems that understand,
think, learn, and behave like humans.
 BASICS OF MACHINE LEARNING
• According to Arthur Samuel, Machine
Learning algorithms enable computers to learn
from data, and even improve themselves,
without being explicitly programmed.

• The basic premise of machine learning is to

build algorithms that can receive input data and
use statistical analysis to predict an output
while updating outputs as new data becomes
available.

Why ML now?

• Flood of available data (especially with the advent of

the Internet)
• Increasing computational power
• Growing progress in available algorithms and theory
developed by researchers
• Increasing support from industries
 TYPES OF MACHINE LEARNING
SUPERVISED LEARNING
• In Supervised learning, an AI system is
presented with data which is labelled, which
means that each data tagged with the correct
label.
• The goal is to approximate the mapping
function so well that when you have new input
data (x) that you can predict the output
variables (Y) for that data.
TYPES OF SUPERVISED LEARNING
• Regression is a technique that aims to
reproduce the output value. We can use it, for
example, to predict the price of some product,
like a price of a house in a specific city or the
value of a stock. 
• Classification is a technique that aims to
reproduce class assignments. It can predict
the response value and the data is separated
into “classes”. Examples? Recognition of a
type of car in a photo, is this mail spam or a
message from a friend, or what the weather
will be today.
 TYPES OF MACHINE LEARNING
UNSUPERVISED LEARNING
• In unsupervised learning, an AI system is
presented with unlabeled, uncategorized data
and the system’s algorithms act on the data
without prior training. The output is dependent
upon the coded algorithms. Subjecting a
system to unsupervised learning is one way of
testing AI.
TYPES OF UNSUPERVISED LEARNING
• Clustering mainly deals with finding a
structure or pattern in a collection of
uncategorized data. Clustering algorithms will
process your data and find natural
clusters(groups) if they exist in the data. 
• Association rules allow you to establish
associations amongst data objects inside large
databases. This unsupervised technique is
about discovering interesting relationships
between variables in large databases. For
example, people that buy a new home most
likely to buy new furniture.
What is Deep Learning?

A Brief Snapshot:

Deep Learning is a subfield of machine learning concerned

with algorithms inspired by the structure and function of the
brain called artificial neural networks

It uses a cascade of layers (tiers) of processing units to

extract features from data and make predictive guesses
about new data

It’s a growing trend within Machine Learning due to some

favorable results in applications where the target function is
very complex and the datasets
are large
 Why Deep Learning?
An overview of some unique features

Self-learning
Part of the machine learning field
of learning representations of
data. Exceptional effective at
learning patterns.

Analogous
Utilizes learning algorithms that
derive meaning out of data by
using a hierarchy of multiple
layers that mimic the neural Intuitive
networks of our brain.
If you provide the system tons of
information, it begins to
understand it and respond in
useful ways.
 Brief History of DL
Modern Innovations
Implementation Of
Backpropagation - 1986

Backpropagation Is Computer
Coded - 1970

The First Backpropagation

Model

Frank Rosenblatt Creates

Perceptron - 1957

McCulloch Pitts Neuron – 1943

 Deep Learning Architectures
RNN LSTM CNN DBN DSN

1990 1995 2000 2005 2010 2015

Architecture Application

Recurrent neural networks (RNN) Speech recognition, handwriting recognition

Natural language text compression, handwriting recognition, speech recognition, gesture

Long short-term memory (LSTM) networks
recognition, image captioning

Convolutional neural networks (CNN) Image recognition, video analysis, natural language processing

Deep belief networks (DBN) Image recognition, information retrieval, natural language understanding, failure prediction

Deep stacking networks (DSN) Information retrieval, continuous speech recognition

 Deep Learning Architecture

Artificial Neural Network

(ANN)

Output Layer

Input Layer
Hidden Layer 1 Hidden Layer 2

Convolution Neural
Network (CNN)

Artificial Neural Network (ANN)
• Artificial Neural Network, or ANN, is a group
of multiple perceptrons/ neurons at each
layer
• ANN is also known as a Feed-Forward
Neural network because inputs are
processed only in the forward direction
Can be used to solve problems related to:
1. Tabular data
2. Image data
3. Text data
Advantages:
• ANN is capable of learning any nonlinear
function. Hence, these networks are
popularly known as Universal Function
Approximators
Challenges:
• One common problem in all these neural
networks is the Vanishing and Exploding
Gradient. This problem is associated with the
backpropagation algorithm
Neural Networks
Recurrent Neural Network (RNN)
• RNN has a recurrent connection on the
hidden state. This looping constraint ensures
that sequential information is captured in the
input data.
Can be used to solve problems related to:
1. Time Series data
2. Audio data
3. Text data
Advantages:
• RNN captures the sequential information
present in the input data i.e. dependency
between the words in the text while making
predictions
• RNNs share the parameters across different
time steps. This is popularly known as
Parameter Sharing. This results in fewer
parameters to train and decreases the
computational cost
Challenges:
• RNNs also suffer from the vanishing and
exploding gradient problem which is a
common problem in all the different types of
neural networks.
Convolution Neural Network (CNN)
• The building blocks of CNNs are filters also
known as kernels. Kernels are used to
extract the relevant features from the input
using the convolution operation.
• CNN models are being used across different
applications and domains, and they’re
especially prevalent in image and video
processing projects.
Advantages:
• CNN learns the filters automatically without
mentioning it explicitly. These filters help in
extracting the right and relevant features
from the input data
• CNN captures the spatial features from an
image. Spatial features refer to the
arrangement of pixels and the relationship
between them in an image.
Challenges:
• CNNs also suffer from the vanishing and
exploding gradient problem which is a
common problem in all the different types of
neural networks.
 Artificial neuron
• An artificial neuron is a connection point in an artificial neural
network.

• Artificial neural networks, like the human body's biological neural

network, have a layered architecture and each network node
(connection point) has the capability to process input and forward
output to other nodes in the network.

• In both artificial and biological architectures, the nodes are called

neurons and the connections are characterized by weights,
which represent the significance of the connection. As new data
is received and processed, the weights change and this is
how learning occurs.
• Artificial neurons are modeled after the hierarchical arrangement
of neurons in biological sensory systems.
Bias: In both artificial and biological networks, when neurons process
the input they receive, they decide whether the output should be
passed on to the next layer as input. The decision of whether or not
to send information on is called bias and it’s determined by an
activation function built into the system
Activation functions are mathematical equations that determine the
output of a neural network. The function is attached to each neuron in
the network, and determines whether it should be activated (“fired”)
or not, based on whether each neuron’s input is relevant for the
model’s prediction
 Training Process
Preset a training
Forward the data to pattern and feed it
a network of hidden through to get
Simple Labeled Data output
layers to get
predictions

Compare with the

target output and
adjust weights
based on errors

Update the Back propagate the Preset another

connection weights errors training pattern and
feed it through to
get output and
compare with target
output
• Repeat this thousands, maybe millions of times – each time taking a
random training instance, and making slight weight adjustments
• Algorithms for weight adjustment are designed to make changes that
will reduce the error
 Autoencoders
• Autoencoders are an unsupervised learning technique in which we
leverage neural networks for the task of representation learning
• In other words, they are used to learn efficient data codings in an
unsupervised manner
Data forced through a bottleneck
We impose a bottleneck in the network which forces a
compressed knowledge representation of the original
input.
However, if some sort of structure exists in the data (ie.
correlations between input features), this structure can be
learned and consequently leveraged when forcing the
input through the network's bottleneck

Applications
Image Processing: They can help compress images for
communication
Anomaly Detection: They can detect errors in images, text
or other media
Noise Reduction: They can help declutter audio clips as
well as images by focusing on background noise and
eliminating it
Translation of languages: They can help identify mistakes
Data sent without a bottleneck while translating from one language to another
 Convolutional Neural Networks (CNN)
• In neural networks, Convolutional neural network (ConvNets or CNNs) is one of the main categories to do images recognition, images
classifications. Objects detections, recognition faces etc., are some of the areas where CNNs are widely used.

The below figure is a complete flow of CNN to process an input image

and classifies the objects based on values.

Technically, deep learning CNN models to train and test, each input image
will pass it through a series of convolution layers with filters (Kernals),
Pooling, fully connected layers (FC) and apply Softmax function to classify
an object with probabilistic values between 0 and 1.
 Recurrent Neural Networks (RNN)
Advantages of Recurrent Neural Network (RNN)
RNN captures the sequential information present in the input data i.e. dependency
between the words in the text while making predictions:

As you can see here, RNN has a recurrent •RNNs share the parameters across different time steps. This is popularly
connection on the hidden state. This looping known as Parameter Sharing. This results in fewer parameters to train and
constraint ensures that sequential information is decreases the computational cost
captured in the input data.

We can use recurrent neural networks to solve

the problems related to:
 Time Series data
 Text data
 Audio data
Applications of Deep Learning

Image Captioning
(CNN+RNN)

NLP - Embeddings

NLP – Word2Vec

NLP - Thought Vectors

Deep Mind – Deep Q

Learning
 Image Captioning : CNN+RNN

A group of people
Vision Deep CNN shopping at an
outdoor market

There are many

vegetables at the
stall
Language Generating RNN

Neural Image Caption Generator generates fitting

natural-language captions only based on the pixels by
combining a vision CNN and a language-generating
RNN

A girl holding a stuffed animal Three pizzas sitting on oven rack

 Natural Language Processing – Embeddings

Embeddings are used to turn textual data (words, sentences, paragraphs) into high dimensional vector
representations and group them together with semantically similar data in a vector space. Thereby,
computer can detect similarities mathematically.
 Natural Language Processing – Word2Vec

Word2Vec is an unsupervised learning algorithm for

It detects similarities mathematically by grouping
obtaining vector representations for words. These
the vectors of similar words together. All it needs
vectors were trained for a specific domain on a very
is words repeated occurrences in the given
large textual data set. GloVe is a better performing
corpus.
alternative.
Natural Language Processing – Word2Vec
 Natural Language Processing – Thought Vectors

Thought vectors is a way of embedding thoughts in vector space.

Their features will represent how each thought relates to other
thoughts. By reading every document on the web, computers might be
able to reason like humans do by mimicking the thoughts expressed in
content.

A neural machine translation is trained on bilingual text using a

encoder and decoder RNN. For translation, the input sentence is
transformed into a thought vector. This vector is used to reconstruct
the given thought in another language.
DeepMind Deep Q-Learning

Deep Q-Learning (DQN) is a model-free

approach to reinforcement learning using
deep networks in environments with
discrete action choices

Policy distillation: Extracts the learned

state (policy) of a reinforcement learning
agent (teacher) and trains a new network
(student) that performs at the expert level
while being dramatically smaller and
more efficient.
 Deep Learning – Usage Requirements

DL

Data Requirements Return on Investment Computational Capacity Scalability

Large Datasets with good Measurable and Large computing power Distribution into group of
quality mappings of input describable goals as per for processing of data. tasks where the basic
and output elements cost constraint. Eg- AWS GPU Instances unit(pixel, word) has very
little meaning in itself.
 Deep Learning: Platform, Frameworks & Libraries
Opensource Tools and their popularity

Platform
• Ersatz Labs -cloud-based deep learning
platform
• H20 –deep learning framework that
comes with R and Python interfaces
Framework
• Caffe-deep learning framework made with expression,
speed, and modularity in mind. Developed by the
Berkeley Vision and Learning Center (BVLC)
• Torch -scientific computing framework with wide
support for machine learning algorithms that puts
GPUs first. Based on Lua programming language
Library
• TensorFlow-open source software
library for numerical computation
using data flow graphs from Google
• Theano-a python library developed
by Yoshua Bengio’s team
.
Tensor Flow
MXNET
MXNET Architecture

Microsoft
The Microsoft Cognitive
Toolkit (CNTK) is an open-
source toolkit for commercial-
grade distributed deep
learning. CNTK allows the
user to easily realize and
combine popular model types
such as feed-forward DNNs
Key players in DL
NVIDIA
NVIDIA AI Platform for
Developers’ GPU-
accelerated deep learning
frameworks offer flexibility
to design and train custom
deep neural networks and
provide interfaces to
commonly-used languages
Amazon
Amazon SageMaker
provides developers and
data scientists with the
ability to build, train, and
deploy machine learning
models quickly. Amazon
SageMaker is a service that
covers the entire machine
learning workflow to label
and prepare data
Google
Google’s AI and machine
learning products make it
easy for developers to
add sight, language,
conversation, and
structured data to their
applications.
Thank You