What is an Artificial Neural Network?

Artificial Neural Networks are the computational models that are inspired by the human brain.
Many of the recent advancements have been made in the field of Artificial Intelligence,
including Voice Recognition, Image Recognition, Robotics using Artificial Neural Networks.
Artificial Neural Networks are the biologically inspired simulations performed on the computer
to perform certain specific tasks like –

o Clustering
o Classification
o Pattern Recognition

Artificial Neural Networks, in general – is a biologically inspired network of artificial neurons

configured to perform specific tasks.

These biological methods of computing are considered to be the next major advancement in the
Computing Industry.

What is a Neural Network?

The term ‘Neural’ is derived from the human (animal) nervous system’s basic functional unit
‘neuron’ or nerve cells that are present in the brain and other parts of the human (animal) body.
A neural network is a group of algorithms that certify the underlying relationship in a set of data
similar to the human brain. The neural network helps to change the input so that the network
 gives the best result without redesigning the output procedure. You can also learn more
about ONNX in this insight.

Parts of Neuron and their Functions

The typical nerve cell of the human brain comprises of four parts –

Image Source – cs231n.github.io

o Function of Dendrite

It receives signals from other neurons.

o Soma (cell body)

It sums all the incoming signals to generate input.

o Axon Structure

When the sum reaches a threshold value, neuron fires and the signal travels down the axon to the
other neurons.

o Synapses Working

The point of interconnection of one neuron with other neurons. The amount of signal transmitted
depend upon the strength (synaptic weights) of the connections.
The connections can be inhibitory (decreasing strength) or excitatory (increasing strength) in
nature.

So, neural network, in general, is a highly interconnected network of billions of neuron with
trillion of interconnections between them.

What is the Difference between Computer and Human Brain?

Difference between Artificial Neural Networks (ANN) and

Biological Neural Networks (BNN)

Artificial Neural
Characteristics Biological(Real) Neural Network
Network

Faster in processing
Slower in processing information. The
Speed information. Response
response time is in milliseconds.
time is in nanoseconds.

Processing Serial processing. Massively parallel processing.

Highly complex and dense network of

Less size & complexity. It
Size & interconnected neurons containing neurons
does not perform complex
Complexity of the order of 1011 with 1015 of
pattern recognition tasks.
interconnections.<strong
 Information storage is
replaceable means new
Storage
data can be added by
deleting an old one.
Highly complex and dense network of
interconnected neurons containing neurons
of the order of 1011 with 1015 of
interconnections.

Fault intolerant.
Information storage is adaptable means new
Information once
information is added by adjusting the
Fault tolerance corrupted cannot be
interconnection strengths without destroying
retrieved in case of failure
old information
of the system.

There is a control unit for

Control No specific control mechanism external to
controlling computing
Mechanism the computing task.
activities

The similarity of Artificial Neural Networks with Biological

Neural Network

Neural Networks resemble the human brain in the following two ways –

o A neural network acquires knowledge through learning.

o A neural network’s knowledge is stored within inter-neuron connection strengths known as
synaptic weights.

VON NEUMANN
ARCHITECTURE BASED ANN BASED COMPUTING
COMPUTING

Serial processing – processing instruction and Parallel processing – several processors

problem rule one at time (sequential) perform simultaneously (multitasking)

Function logically with a set of if & else rules – Function by learning pattern from given
rule-based approach input (image, text or video, etc.)

Programmable by higher level languages such

ANN is in essence program themselves.
as C, Java, C++, etc.

Requires either big or error-prone parallel

Use of application-specific multi-chips.
processors

Comparison between Artificial Neural Networks (ANN) With

Biological Neural Network (BNN)
The dendrites in the Biological Neural Network are analogous to the weighted inputs based on
their synaptic interconnection in the Artificial Neural Network.

The cell body is comparable to the artificial neuron unit in the Artificial Neural Network which
also comprises of summation and threshold unit.

Axon carries output that is analogous to the output unit in the case of an Artificial Neural
Network. So, ANN is modeled using the working of basic biological neurons.

How Does Artificial Neural Network Works?

Artificial Neural Networks can be viewed as weighted directed graphs in which artificial neurons
are nodes, and directed edges with weights are connections between neuron outputs and neuron
inputs.

The Artificial Neural Network receives information from the external world in the form of
pattern and image in vector form. These inputs are mathematically designated by the notation
x(n) for n number of inputs.

Each input is multiplied by its corresponding weights. Weights are the information used by the
neural network to solve a problem. Typically weight represents the strength of the
interconnection between neurons inside the Neural Network.

The weighted inputs are all summed up inside the computing unit (artificial neuron). In case the
weighted sum is zero, bias is added to make the output not- zero or to scale up the system
response. Bias has the weight and input always equal to ‘1′.

The sum corresponds to any numerical value ranging from 0 to infinity. To limit the response to
arrive at the desired value, the threshold value is set up. For this, the sum is passed through an
activation function.

The activation function is set to the transfer function used to get the desired output. There are
linear as well as the nonlinear activation function.

Some of the commonly used activation function is – binary, sigmoidal (linear) and tan
hyperbolic sigmoidal functions (nonlinear).
o Binary – The output has only two values either 0 or 1. For this, the threshold value is set up. If
the net weighted input is greater than 1, an output is assumed one otherwise zero.
o Sigmoidal Hyperbolic – This function has a ‘S’ shaped curve. Here tan hyperbolic function is
used to approximate output from net input. The function is defined as – f (x) = (1/1+ exp(-????
x)) where???? – Steepness parameter.

You May also Love to Read Overview of Artificial Intelligence & Role of Natural Language
Processing in Big Data

Types of Neural Networks in Artificial

Intelligence

Parameter Types Description

Based on FeedForward, Feedforward – In which graphs have no loops.

connection pattern Recurrent Recurrent – Loops occur because of feedback.

Single Layer – Having one hidden layer. E.g. ,

Based on the Single Perceptron
Single layer,
number of hidden Multilayer – Having multiple hidden layers.
Multi-Layer
layer Multilayer Perceptron

Fixed – Weights are fixed a priori and not changed at

all.
Based on nature of
Fixed, Adaptive Adaptive – Weights are updated and changed during
weights
training.

Static – Memoryless unit. The current output

depends on the current input. E.g. , Feedforward
network
Based on Memory
Static, Dynamic Dynamic – Memory unit – The output depends upon
unit
the current input as well as the current output. E.g. ,
Recurrent Neural Network

Neural Network Architecture Types

o Perceptron Model in Neural Networks

Neural Network is having two input units and one output units with no hidden layers. These are
also known as ‘single layer perceptrons.

o Radial Basis Function Neural Network

These networks are similar to the feed-forward Neural Network except radial basis function is
used as the activation function of these neurons.

o Multilayer Perception Neural Network

These networks use more than one hidden layer of neurons, unlike single layer perception. These
are also known as Deep Feed forward Neural Networks.

o Recurrent Neural Network

 Type of Neural Network in which hidden layer neurons has self-connections. Recurrent Neural
Networks possess memory. At any instance, hidden layer neuron receives activation from the
lower layer as well as its previous activation value.

o Long Short-Term Memory Neural Network (LSTM)

Type of Neural Network in which memory cell is incorporated into hidden layer neurons is
called LSTM network.

o Hopfield Network

A fully interconnected network of neurons in which each neuron is connected to every other
neuron. The network is trained with input pattern by setting a value of neurons to the desired
pattern. Then its weights are computed. The weights are not changed. Once trained for one or
more patterns, the network will converge to the learned patterns. It is different from other Neural
Networks.

o Boltzmann Machine Neural Network

These networks are similar to the Hopfield network except some neurons are input, while others
are hidden in nature. The weights are initialized randomly and learn through backpropagation
algorithm.

o Convolution Neural Network

Get a complete overview of Convolution Neural Networks through our blog Log Analytics with
Machine Learning and Deep Learning.

o Modular Neural Network

It is the combined structure of different types of the neural network like multilayer perception,
Hopfield Network, Recurrent Neural Network, etc. which are incorporated as a single module
into the network to perform independent subtask of whole complete Neural Networks.

o Physical Neural Network

In this type of Artificial Neural Network, electrically adjustable resistance material is used to
emulate the function of synapse instead of software simulations performed in the neural network.

Hardware Architecture for Neural Networks

Two types of methods are used for implementing hardware for Neural Networks.

o Software simulation in conventional computer

o Special hardware solution for decreasing execution time.

When Neural Networks are used with a fewer number of processing units and weights, software
simulation is performed on the computer directly. E.g. , voice recognition, etc.

When Neural Network Algorithms developed to the point where useful things can be done with
1000’s of neurons and 10000’s of synapses, high-performance Neural Networks hardware will
become essential for practical operation.

E.g., GPU (Graphical processing unit) in the case of Deep Learning algorithms in the event of
object recognition, image classification, etc.

The performance of the implementation is measured by connection per the second number (cps),
i.e., the number of the data chunk is transported through the edges of the neural network.

While the performance of the learning algorithm is measured in the connection updates per
second (cups)

Learning Techniques in Artificial Neural

Networks

The neural network learns by adjusting its weights and bias (threshold) iteratively to yield the
desired output. These are also called free parameters. For learning to take place, the Neural
Network is trained first. The training is performed using a defined set of rules also known as the
learning algorithm.
 Training Algorithms For Artificial Neural Networks

o Gradient Descent Algorithm

This is the simplest training algorithm used in case of supervised training model. In case, the
actual output is different from target output, the difference or error is find out. The gradient
descent algorithm changes the weights of the network in such a manner to minimize this mistake.

o Back Propagation Algorithm

It is an extension of the gradient-based delta learning rule. Here, after finding an error (the
difference between desired and target), the error is propagated backward from the output layer to
the input layer via the hidden layer. It is used in case of Multilayer Neural Network.

Learning Data Sets in Artificial Neural Networks

o Training Data Set

A set of examples used for learning that is to fit the parameters [i.e., weights] of the network.
One approach comprises of one full training cycle on the training set.

o Validation Set Approach

A set of examples used to tune the parameters [i.e., architecture] of the network. For example to
choose the number of hidden units in a Neural Network.

o Making Test Set

A set of examples used only to assess the performance [generalization] of a fully specified
network or to apply successfully in predicting output whose input is known.

Five Algorithms to Train a Neural Network

o Hebbian learning Rule

o Self – Organizing Kohonen Rule
o Hopfield Network Law
o LMS algorithm (Least Mean Square)
o Competitive Learning

Artificial Neural Network Architecture

A typical Neural Network contains a large number of artificial neurons called units arranged in a
series of layers.
 In typical Artificial Neural Network, comprises a different layer –

o Input layer – It contains those units (Artificial Neurons) which receive input from the outside
world on which network will learn, recognize about or otherwise process.
o Output layer – It contains units that respond to the information about how it’s learned any task.
o Hidden layer – These units are in between input and output layers. The job of the hidden layer is
to transform the input into something that output unit can use in some way.

Most Neural Networks are fully connected that means to say each hidden neuron is fully linked
to every neuron in its previous layer(input) and to the next layer (output) layer.

Learning Techniques in Neural Networks

o Supervised Learning

In supervised learning, the training data is input to the network, and the desired output is known
weights are adjusted until production yields desired value.
o Unsupervised Learning

The input data is used to train the network whose output is known. The network classifies the
input data and adjusts the weight by feature extraction in input data.

o Reinforcement Learning

Here the value of the output is unknown, but the network provides the feedback on whether the
output is right or wrong. It is Semi-Supervised Learning.

o Offline Learning

The adjustment of the weight vector and threshold is made only after all the training set is
presented to the network. It is also called Batch Learning.

o Online Learning

The adjustment of the weight and threshold is made after presenting each training sample to the
network.

Learning and Development in Neural Networks

Learning occurs when the weights inside the network get updated after much iteration.

For example – Suppose we have inputs in the form of patterns for two different class of patterns
– I & 0 as shown and b -bias and y as the desired output.

Pattern y x1 x2 x3 x4

I 1 1 1 1 -1

O -1 1 1 1 1

We want to classify input patterns into either pattern ‘I’ & ‘O.’

Following are the steps performed:

o Nine inputs from x1 – x9 along with bias b (input having weight value 1) is fed to the network
for the first pattern.
o Initially, weights are initialized to zero.
o Then weights are updated for each neuron using the formulae: Δ wi = xi y for i = 1 to 9 (Hebb’s
Rule)
o Finally, new weights are found using the formulae:
o wi(new) = wi(old) + Δwi
o Wi(new) = [111-11-1 1111]
o The second pattern is input to the network. This time, weights are not initialized to zero. The
initial weights used here are the final weights obtained after presenting the first pattern. By doing
so, the network
o The steps from 1 – 4 are repeated for second inputs.
o The new weights are Wi(new) = [0 0 0 -2 -2 -2 000]

So, these weights correspond to the learning ability of the network to classify the input patterns
successfully.

4 Different Techniques of Neural Networks

o Classification Neural Network

A Neural Network can be trained to classify given pattern or dataset into predefined class. It uses
Feed forward Networks.

o Prediction Neural Network

A Neural Network can be trained to produce outputs that are expected from a given input. E.g., –
Stock market prediction.

o Clustering Neural Network

The Neural network can be used to identify a unique feature of the data and classify them into
different categories without any prior knowledge of the data.

Following networks are used for clustering –

o Competitive networks
o Adaptive Resonance Theory Networks
o Kohonen Self-Organizing Maps.

o Association Neural Network

A Neural Network can be trained to remember the particular pattern so that when the noise
pattern is presented to the network, the network associates it with the closest one in the memory
or discard it. E.g. , Hopfield Networks which performs recognition, classification, and clustering,
etc.
 Neural Networks for Pattern Recognition

Pattern recognition is the study of how machines can observe the environment, learn to
distinguish patterns of interest from their background, and make sound and reasonable decisions
about the categories of the patterns.

Some examples of the pattern are – fingerprint image, a handwritten word, human face or speech
signal.

Given an input pattern, its recognition involves the following task –

o Supervised classification – Given the input pattern is identified as the member of a predefined
class.
o Unsupervised classification – Pattern is assigned to a hitherto unknown class.

So, the recognition problem here is essentially a classification or categorized task.

The design of pattern recognition systems usually involves the following three aspects-

o Data acquisition and preprocessing

o Data representation
o Decision Making

Approaches For Pattern Recognition

o Template Matching
o Statistical
o Syntactic Matching
o Artificial Neural Networks
 Following Neural Network architectures used for Pattern Recognition –

o Multilayer Perceptron
o Kohonen SOM (Self Organizing Map)
o Radial Basis Function Network (RBF)

Neural Network for Deep Learning

Following Neural Network, architectures are used in Deep Learning

o Feed-forward neural networks

o Recurrent neural network
o Multi-layer perceptions (MLP)
o Convolution neural networks
o Recursive neural networks
o Deep belief networks
o Convolution deep belief networks
o Self-Organizing Maps
o Deep Boltzmann machines
o Stacked de-noising auto-encoders

Neural Networks and Fuzzy Logic

Fuzzy logic refers to the logic developed to express the degree of truthiness by assigning values
in between 0 and 1, unlike traditional Boolean logic that represents 0 and 1.

Fuzzy logic and Neural networks have one thing in common. They can be used to solve
problems of pattern recognition and others that do not involve any mathematical model.

Systems combining both fuzzy logic and neural networks are neuron-fuzzy systems.

These systems (Hybrid) can combine advantages of both neural networks and fuzzy logic to
perform in a better way.

Fuzzy logic and Neural Networks have been integrated to use in the following applications –

o Automotive engineering
o Applicant screening of jobs
o Control of crane
o Monitoring of glaucoma

In a hybrid (neuron-fuzzy) model, Neural Networks Learning Algorithms are fused with the
fuzzy reasoning of fuzzy logic.
 The neural network determines the values of parameters, while if-then rules are handled by fuzzy
logic.

Neural Network for Machine Learning

o Multilayer Perception (supervised classification)

o Back Propagation Network (supervised classification)
o Hopfield Network (for pattern association)
o Deep Neural Networks (unsupervised clustering)

Applications of Neural Networks

Neural networks have been successfully applied to the broad spectrum of data-intensive
applications, such as:

Activation
Application Architecture / Algorithm
Function

Process modeling and

Radial Basis Network Radial Basis
control

Tan- Sigmoid
Machine Diagnostics Multilayer Perception
Function

Portfolio Tan- Sigmoid

Classification Supervised Algorithm
Management Function

Tan- Sigmoid
Target Recognition Modular Neural Network
Function

Tan- Sigmoid
Medical Diagnosis Multilayer Perception
Function

Logistic Discriminate Analysis with ANN,

Credit Rating Logistic function
Support Vector Machine

Targeted Marketing Back Propagation Algorithm Logistic function

 Multilayer Perception, Deep Neural
Voice recognition Logistic function
Networks( Convolution Neural Networks)

Financial Forecasting Back propagation Algorithm Logistic function

Intelligent searching Deep Neural Network Logistic function

Gradient – Descent Algorithm and Least Mean

Fraud detection Logistic function
Square (LMS) algorithm.

Advantages of Neural Networks

o A neural network can perform tasks that a linear program cannot.

o When an element of the neural network fails, it can continue without any problem by their
parallel nature.
o A neural network learns and does not need to be reprogrammed.
o It can be implemented in any application.
o It can be performed without any problem.

Limitations of Neural Networks

o The neural network needs the training to operate.

o The architecture of a neural network is different from the architecture of microprocessors,
therefore, needs to be emulated.
o Requires high processing time for large neural networks.

Face Recognition Using Artificial Neural Networks

Face recognition entails comparing an image with a database of saved faces to identify the
person in that input picture. Face detection mechanism involves dividing images into two parts;
one containing targets (faces) and one providing the background.

The associated assignment of face detection has direct relevance to the fact that images need to
be analyzed and faces identified, earlier than they can be recognized.

Click here to read the full case study

Learning Rules in Neural Network

The learning rule is a type of mathematical logic. It encourages a Neural Network to gain from
the present conditions and upgrade its efficiency and performance.
 The learning procedure of the brain modifies its neural structure. The expanding or diminishing
quality of its synaptic associations relies upon their activity.

Learning rules in the neural network:

o Hebbian learning rule ; It determines, how to customize the weights of nodes of a system.
o Perception learning rule; Network starts its learning by assigning a random value to each load.
o Delta learning rule; Modification in sympatric weight of a node is equal to the multiplication of
error and the input.
o Correlation learning rule; It is similar to supervised learning.

How Can Xenon Stack Help You?

Xenon Stack can help you develop and deploy your model solutions based on Neural Networks.
Whatever kind of problem you face – Prediction, Classification or Pattern Recognition – Xenon
Stack has a solution for you.

Machine Learning Consulting Services

Unlock the real value of your data to make smarter decisions. Embrace Machine Learning
Services in your Enterprise to solve complex Business Problems. XenonStack offers Data
Cleansing and Extraction Services, Data Modeling Services, and Algorithm Development.

Fraud Detection & Prevention Services

Xenon Stack Fraud Detection Services offers real-time fraud analysis to increase profitability.
Data Mining is used to quickly detect fraud and search for spot patterns and detect fraudulent
transactions. Data Mining Tools like Machine Learning, Neural Networks, Cluster Analysis are
used to generate Predictive Models to prevent fraud losses.

Data Modeling Services

Xenon Stack offers Data Modeling using Neural Networks, Machine Learning, and Deep
Learning. Data Modeling services help Enterprises to create a conceptual model based on the
analysis of data objects. Deploy your Data Models on leading Cloud Service Providers
like Google Cloud, Microsoft Azure, AWS or on the container environment – Kubernetes &
Dockers.
 PART 2

Artificial Neural Network - Basic Concepts

Neural networks are parallel computing devices, which is basically an attempt to make
a computer model of the brain. The main objective is to develop a system to perform
various computational tasks faster than the traditional systems. These tasks include
pattern recognition and classification, approximation, optimization, and data clustering.

What is Artificial Neural Network?

Artificial Neural Network ANNANN is an efficient computing system whose central
theme is borrowed from the analogy of biological neural networks. ANNs are also
named as “artificial neural systems,” or “parallel distributed processing systems,” or
“connectionist systems.” ANN acquires a large collection of units that are
interconnected in some pattern to allow communication between the units. These units,
also referred to as nodes or neurons, are simple processors which operate in parallel.
Every neuron is connected with other neuron through a connection link. Each
connection link is associated with a weight that has information about the input signal.
This is the most useful information for neurons to solve a particular problem because
the weight usually excites or inhibits the signal that is being communicated. Each
neuron has an internal state, which is called an activation signal. Output signals, which
are produced after combining the input signals and activation rule, may be sent to other
units.

A Brief History of ANN

The history of ANN can be divided into the following three eras −

ANN during 1940s to 1960s

Some key developments of this era are as follows −

 1943 − It has been assumed that the concept of neural network started with the work of physiologist,
Warren McCulloch, and mathematician, Walter Pitts, when in 1943 they modeled a simple neural
network using electrical circuits in order to describe how neurons in the brain might work.

 1949 − Donald Hebb’s book, The Organization of Behavior, put forth the fact that repeated activation
of one neuron by another increases its strength each time they are used.

 1956 − An associative memory network was introduced by Taylor.

  1958 − A learning method for McCulloch and Pitts neuron model named Perceptron was invented by
Rosenblatt.

 1960 − Bernard Widrow and Marcian Hoff developed models called "ADALINE" and “MADALINE.”

ANN during 1960s to 1980s

Some key developments of this era are as follows −

 1961 − Rosenblatt made an unsuccessful attempt but proposed the “backpropagation” scheme for
multilayer networks.

 1964 − Taylor constructed a winner-take-all circuit with inhibitions among output units.

 1969 − Multilayer perceptron MLPMLP was invented by Minsky and Papert.

 1971 − Kohonen developed Associative memories.

 1976 − Stephen Grossberg and Gail Carpenter developed Adaptive resonance theory.

ANN from 1980s till Present

Some key developments of this era are as follows −

 1982 − The major development was Hopfield’s Energy approach.

 1985 − Boltzmann machine was developed by Ackley, Hinton, and Sejnowski.

 1986 − Rumelhart, Hinton, and Williams introduced Generalised Delta Rule.

 1988 − Kosko developed Binary Associative Memory BAMBAM and also gave the concept of Fuzzy
Logic in ANN.

The historical review shows that significant progress has been made in this field.
Neural network based chips are emerging and applications to complex problems are
being developed. Surely, today is a period of transition for neural network technology.

Biological Neuron
A nerve cell neuronneuron is a special biological cell that processes information.
According to an estimation, there are huge number of neurons, approximately 10 with 11

numerous interconnections, approximately 10 . 15

Schematic Diagram
Working of a Biological Neuron

As shown in the above diagram, a typical neuron consists of the following four parts
with the help of which we can explain its working −
 Dendrites − They are tree-like branches, responsible for receiving the information from other neurons
it is connected to. In other sense, we can say that they are like the ears of neuron.

 Soma − It is the cell body of the neuron and is responsible for processing of information, they have
received from dendrites.

 Axon − It is just like a cable through which neurons send the information.

 Synapses − It is the connection between the axon and other neuron dendrites.

ANN versus BNN

Before taking a look at the differences between Artificial Neural Network ANNANN and
Biological Neural Network BNNBNN, let us take a look at the similarities based on the
terminology between these two.
Biological Neural Network BNNBNN Artificial Neural Network ANNANN

Soma Node

Dendrites Input

Synapse Weights or Interconnections

Axon Output
The following table shows the comparison between ANN and BNN based on some
criteria mentioned.

Criteri BNN ANN

a

Proce Massively Massively parallel, fast but inferior than BNN

ssing parallel, slow
but superior
than ANN

Size 1011 neurons 102 to

and 104 nodes mainlydependsonthetypeofapplicationandnetworkdesignermainlydepend
1015 intercon sonthetypeofapplicationandnetworkdesigner
nections

Learni They can Very precise, structured and formatted data is required to tolerate ambiguity
ng tolerate
ambiguity

Fault Performance It is capable of robust performance, hence has the potential to be fault tolerant
tolera degrades
nce with even
partial
damage

Storag Stores the Stores the information in continuous memory locations

e information
capaci in the
ty synapse

Model of Artificial Neural Network

The following diagram represents the general model of ANN followed by its processing.
For the above general model of artificial neural network, the net input can be calculated
as follows −
yin=x1.w1+x2.w2+x3.w3…xm.wmyin=x1.w1+x2.w2+x3.w3…xm.wm

i.e., Net input yin=∑mixi.wiyin=∑imxi.wi

The output can be calculated by applying the activation function over the net input.
Y=F(yin)Y=F(yin)

Output = function netinputcalculatednetinputcalculated

Artificial Neural Network - Building Blocks
Processing of ANN depends upon the following three building blocks −

 Network Topology
 Adjustments of Weights or Learning
 Activation Functions

In this chapter, we will discuss in detail about these three building blocks of ANN

Network Topology
A network topology is the arrangement of a network along with its nodes and
connecting lines. According to the topology, ANN can be classified as the following
kinds −

Feedforward Network
It is a non-recurrent network having processing units/nodes in layers and all the nodes
in a layer are connected with the nodes of the previous layers. The connection has
different weights upon them. There is no feedback loop means the signal can only flow
in one direction, from input to output. It may be divided into the following two types −
 Single layer feedforward network − The concept is of feedforward ANN having only one weighted
layer. In other words, we can say the input layer is fully connected to the output layer.

 Multilayer feedforward network − The concept is of feedforward ANN having more than one
weighted layer. As this network has one or more layers between the input and the output layer, it is
called hidden layers.

Feedback Network

As the name suggests, a feedback network has feedback paths, which means the
signal can flow in both directions using loops. This makes it a non-linear dynamic
system, which changes continuously until it reaches a state of equilibrium. It may be
divided into the following types −
 Recurrent networks − They are feedback networks with closed loops. Following are the two types of
recurrent networks.

 Fully recurrent network − It is the simplest neural network architecture because all nodes are
connected to all other nodes and each node works as both input and output.
  Jordan network − It is a closed loop network in which the output will go to the input again as
feedback as shown in the following diagram.

Adjustments of Weights or Learning

Learning, in artificial neural network, is the method of modifying the weights of
connections between the neurons of a specified network. Learning in ANN can be
classified into three categories namely supervised learning, unsupervised learning, and
reinforcement learning.

Supervised Learning

As the name suggests, this type of learning is done under the supervision of a teacher.
This learning process is dependent.
During the training of ANN under supervised learning, the input vector is presented to
the network, which will give an output vector. This output vector is compared with the
desired output vector. An error signal is generated, if there is a difference between the
actual output and the desired output vector. On the basis of this error signal, the
weights are adjusted until the actual output is matched with the desired output.
Unsupervised Learning

As the name suggests, this type of learning is done without the supervision of a
teacher. This learning process is independent.
During the training of ANN under unsupervised learning, the input vectors of similar
type are combined to form clusters. When a new input pattern is applied, then the
neural network gives an output response indicating the class to which the input pattern
belongs.
There is no feedback from the environment as to what should be the desired output
and if it is correct or incorrect. Hence, in this type of learning, the network itself must
discover the patterns and features from the input data, and the relation for the input
data over the output.

Reinforcement Learning

As the name suggests, this type of learning is used to reinforce or strengthen the
network over some critic information. This learning process is similar to supervised
learning, however we might have very less information.
During the training of network under reinforcement learning, the network receives some
feedback from the environment. This makes it somewhat similar to supervised learning.
However, the feedback obtained here is evaluative not instructive, which means there
is no teacher as in supervised learning. After receiving the feedback, the network
performs adjustments of the weights to get better critic information in future.
Activation Functions

PART 3
Artificial Neural Networks – The basics
Artificial Neural Networks are simulations that are derived from the biological functions of ‘neurons’ which are
present in the brain. Thus, Artificial Neural Networks are essentially artificial neurons configured to carry out a
specific task. ANN has gained a lot of popularity as it is used to model non-linear processes.
Artificial Neural Networking allows solving problems like clustering, classification, pattern recognition,
prediction, and determining outliers. This has made ANN a very useful tool.
How does ANN work?
Artificial Neural Networks acquires knowledge through learning continuously. Like in humans, the knowledge
acquired is stored in the artificial neurons designed within the ANN and used to perform the required task. ANN
has a wide range of syntax, semantics, and speech-tasks which help ANN solve a wide range of problems.
Some of the interesting applications of ANN are discussed below.
1. Text Classification
Applications like web searches, language identification are some of the applications that use text classification.
Neural Networks are widely employed for this type of classification. Experts agree that deep learning can be
applied to enhance the value delivered by text classification. Artificial Neural Networks can be applied from
character-level inputs as well as abstract text content.
CovNets or Convolutional Networks can deliver good outcomes in text classification without prior knowledge of
words or phrases by applying them along with deep learning and Neural Networks.
2. Semantic Parsing
Artificial Neural Networks can be actively helpful in answering questions. A Q&A system will automatically
answer any question asked in natural languages like definition questions, biographical questions and so on.
Using Neural Networks in these systems makes it possible to maintain a high performing question answering
system.
Developers have released semantic parsing framework for answering questions using a specific knowledge
base. ANN uses this framework to quickly identify the type of questions and then answers it using semantic
matching. There are other frameworks available which can further improve neural networks’ performance in this
field.
3. Speech Recognition
Voice technology has advanced and now it is used for automated telephone conversations, speech-only
computing, and much more. Neural Networks are being used extensively in this area. neural networks can
specifically be programmed to handle multiple types of queries over a wide range and with continuous learning,
neural networks help you achieve a great speech recognition software.
4. Character Recognition
Character recognition has become vital in today’s world across different industry verticals. There are many
practical applications in this realm. Some instances include character recognition on receipts, invoices, checks,
or legal billing. The Character Recognition framework for Artificial Neural Networks has been effectively used in
this field and tests have shown the accuracy to be above 85%.
5. Spell Check
Text editors help you find out misspelled words to help you rectify them. Neural networks have been
incorporated in many of these text editors nowadays to provide easier spell checks. It uses the personalized
spell check framework and it outperforms many other text editors that don’t use Artificial Neural Networks
today.
To conclude, we can say that Artificial Neural Networks are very versatile and make a lot of jobs easier in
different functions within an enterprise.