ARTIFICIAL NEURAL NETWORKS (ANNS)

VS
DEEP NEURAL NETWORKS
 Artificial Neural Network

Introduction to Deep Learning

Deep learning Architectures

Convolutional Neural Network

Deep Belief Net

RNN

Python packages for Deep Learning.

Dr. Robert Hecht-Nielsen, defines a neural network as

A computing system made up of a number of simple, highly interconnected

processing elements, which process information by their dynamic state
response to external inputs.”
 Basic Structure of Human Brain
Working of human brain is imitated by making the right connections, using silicon and wires as living neurons and
dendrites.
Typical diagram of Biological Neural Network
Typical Artificial Neural Network
Relationship between Biological neural network and artificial neural network:

Biological Neural Network Artificial Neural Network

Dendrites Inputs
Cell nucleus Nodes
Synapse Weights
Axon Output
ANNs are composed of

• multiple nodes, which imitate biological neurons of human brain.

• The neurons are connected by links and they interact with each
other.

• The nodes can take input data and perform simple operations on the
data. The result of these operations is passed to other neurons.

• The output at each node is called its activation or node value.

 Types of Artificial Neural Networks

• FeedForward
• Feedback.
FeedForward ANN
• Information flow is unidirectional.
• A unit sends information to other unit from which it does not receive
any information.
• There are no feedback loops.
• They are used in pattern generation/recognition/classification. They
have fixed inputs and outputs.
 FeedBack ANN
• Feedback loops are allowed.
 Shallow neural network
Three layers of neurons:

• An input layer that accepts the independent variables or inputs of the model
• One hidden layer
• An output layer that generates predictions

Deep Neural Network (DNN)

• A Deep Neural Network (DNN) has a similar structure, but it has two or more “hidden layers” of neurons that
process inputs.
On the left side you have neurons (small circles) of x with subscripts 1, 2, … , m carrying data input.

We multiply each of the input by a weight w, also labeled with subscripts 1, 2, …, m, along the arrow
(also called a synapse) to the big circle in the middle. So w1 * x1, w2 * x2, w3 * x3 and so on.

Once all the inputs are multiplied by a weight, we sum all of them up and add another pre-determined
number called bias.

Then, we push the result further to the right. Now, we have this step function in the rectangle. What it
means is that if the result from step 3 is any number equal or larger than 0, then we get 1 as output,
otherwise if the result is smaller than 0, we get0 as output.

The output is either 1 or 0.

 DEEP LEARNING
Makes use of Deep Neural Networks

Deep learning algorithms are constructed with connected

layers.

• The first layer is called the Input Layer

• The last layer is called the Output Layer
• All layers in between are called Hidden Layers.

https://www.guru99.com/deep-learning-tutorial.html
Deep learning Process
Learning occurs in two phases.

The first phase consists of applying a nonlinear transformation of

the input and create a statistical model as output.

The second phase aims at improving the model with a

mathematical method known as derivative.
Convolutional neural networks (CNN)
 Deep Belief Networks
Deep belief Networks are family of deep architecture networks that
uses stacks of Restricted Boltzmann Machines as building blocks.

Furthermore, DBNs can be used in a both unsupervised setting for

tasks such as image generation and in a supervised setting for tasks
such as image classification.

It takes full advantage of great techniques such as unsupervised pre-

training and fine tuning on a down stream task.
 RNN
Derived from feedforward neural networks.

Applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.