Introduction To ANN

Machine Learning & Pattern Recognition
Basics of Artificial Neural Network
Dr. S. S. Chowhan
School of Computational Sciences

S. R. T. M. University, Nanded
01, November 2014
Dr. S. S. Chowhan (SCS) Machine Learning & Pattern Recognition 01, November 2014 1 / 20
Outline of Course
1 About Brain
2 Biological Neuron
3 Benefits of ANN
4 Definition of ANN
5 Artificial / Mathematical Model of Neuron

Descripition of ANN Model
Architecture of Artificial Neural Network
Outline of Course
1 About Brain
2 Biological Neuron
3 Benefits of ANN
4 Definition of ANN

Outline of Course
1 About Brain
2 Biological Neuron
3 Benefits of ANN
4 Definition of ANN

Outline of Course
1 About Brain
2 Biological Neuron
3 Benefits of ANN
4 Definition of ANN

Outline of Course
1 About Brain
2 Biological Neuron
3 Benefits of ANN
4 Definition of ANN

About Brain
Figure 1: Human Brain
Highly complex, nonlinear and parallel computer. Capability of organizing

neurons so as perform Perceptual Pattern Recognition (recognizing a
familiar face embedded in an unfamiliar scene) in 100 to 200ms. Many
times faster than fastest digital computer in existence today
Biological Neuron
Figure 2: Biological Neuron
Signals are transmitted between neurons by electrical pulses

(action-potentials or ‘spike’ trains) traveling along the axon. These pulses
impinge on the afferent neuron at terminals called synapses. These are
found principally on a set of branching processes emerging from the cell
body (soma) known as dendrites. Each pulse occurring at a synapse
initiates the release of a small amount of chemical substance or
neurotransmitter which travels across the synaptic cleft and which is then
received at post-synaptic receptor sites on the dendrite side of the synapse.
Benefits of Neural Network
(I) Parallel Distributed Massive Structure

(II) VLSI Implementation
(III) Fault Tolerance
(IV) Uniformity of Analysis and Design
(V) Generalization
(VI) Adaptability
(VII) Non-linearity
(VIII) Input-Output Mapping
Definition of Neural Network
ANN is a massively parallel distributed processor which can store

experiential knowledge and making it available for use. It resembles the
brain in two respects:
(I) Knowledge is acquired through learning
(II) Synaptic weights are used to store the knowledge
Artificial / Mathematical Model of Neuron
Figure 3: ANN Model
Artificial / Mathematical Model of Neuron
Figure 4: Artificial Neuron Model
Fundamental Processing Unit a

Three basic elements of the neuron model
Discription of ANN Model
(a) Set of synapses or connecting links

(b) An adder for summing the input signals
(c) An activation function
(d) Threshold input lowers the net input of the neuron (Bias negative of
threshold)
(e) Adder acts as linear combiner and adds weighted inputs
(f) Activation function limits the output of a neuron
(g) Typically normalized output is [0,1] or [1,-1]
Activation Functions
Figure 5: ANN Model
net = X1 ∗ W1 + X2 ∗ W2 + ..... + Xn ∗ Wn >= t (1)
but this is equivalent algebraically equivalent to
net = X1 ∗ W1 + X2 ∗ W2 + ..... + Xn ∗ Wn + (−1) ∗ t >= 0 (2)
So, in an implementation, Consider each TLU ( Threshold Logic Unit)

having an extra input value -1 and the weight is t (threshold) a
As Shown in the below
Linear and Non-linearly separable problems
1 The two classes are therefore separated by the 0 decisionline0 which is

defined by putting the activation equal to the threshold. It turns out
that it is possible to generalize this result to TLUs with n inputs in
3-D the two classes are separated by a decision-plane. In n-D this
becomes a decision − hyperplane.
(The ‘hyper-’ is sometimes dropped even when n > 3 ).
2 The converse of this is that, any TLU is defined by some hyperplane
in its pattern space and any function which cannot be realized in this
way cannot be realized by a TLU. Because the defining equation for
the hyper plane is linear the TLU is a linear classifier.
Linear and Non-Linear Separable Problems
Figure 6: XOR Non-Linear Separable Problem
Linear and Non-Linear Separable Problems
Figure 7: Linear Separable Problem
Neural Network Architecture / Neural Network Topologies
1 FeedforwardNetwork
Singlelayer Feedforward Network
Multilayer Feedforward Network
2 FeedbackNetwork
SingleLayer Neural Network Architecture
A single layer network is a simple structure consisting of m neurons

each having n inputs
The system performs a mapping from the n-dimensional input space
to the m-dimensional output space
To train the network the same learning algorithms as for a single
neuron can be used
This type of network is widely used for linearseparable problems,
Singlelayer network are not capable of classifying
non linear separable data sets.
Multilayer Neural Network Architecture
To Solve Non-Linear Separable Problems Multilayer Neural Network

are used
This network is distinguished by the presence of one or more hidden
layers,
Whose computation nodes are hidden neurons
By adding one or more hidden layers, the network is enabled to
extract higher-order statistics and suitable for nonlilearly separable
pattern classification problems
Figure 8: Toplogy of ANN
References
Marget H. D. and S. Sridhar
Data mining : Introductory and Advanced Topics
Pearson Publishing New Delhi, 2003.
C. Apte and S. Weiss.
Data mining with decision trees and decision rules
Future Generation Computer Systems, 13, 1997.
S. K. Murthy,
Automatic Construction of Decision Trees from Data: A
Multi-Disciplinary Survey, Data Mining and Knowledge Discovery
2(4): 345-389, 1998
J. R. Quinlan.
Induction of decision trees. Machine Learning, 1:81-106, 1986.
I. H. Witten and E. Frank.
Data Mining: Practical Machine Learning Tools and Techniques, 2ed.
Morgan Kaufmann, 2005.
References
Jack.M. Zurada
Introduction to Artificial Neural Systems.
Jaico Publishing Home, 2002.
George J. Klir, Bo Yuan
Fuzzy Sets and Fuzzy Logic Theory and Application
PHI Press 1998.
Satish Kumar
Neural Networks A CLASS ROOM APPROACH
Tata McGraw Hill Publishing 2003.
Robert J.Schalkoff
Artifical Neural Networks
Pearson Education, New Delhi, Tata McGraw Hill Publishing 1997.

Introduction To ANN

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To ANN

Uploaded by

Copyright:

Available Formats

Machine Learning & Pattern Recognition

Basics of Artificial Neural Network

School of Computational Sciences

01, November 2014

5 Artificial / Mathematical Model of Neuron

5 Artificial / Mathematical Model of Neuron

5 Artificial / Mathematical Model of Neuron

5 Artificial / Mathematical Model of Neuron

5 Artificial / Mathematical Model of Neuron

Figure 1: Human Brain

Highly complex, nonlinear and parallel computer. Capability of organizing

Figure 2: Biological Neuron

Signals are transmitted between neurons by electrical pulses

(I) Parallel Distributed Massive Structure

ANN is a massively parallel distributed processor which can store

Figure 3: ANN Model

Figure 4: Artificial Neuron Model

Fundamental Processing Unit a

(a) Set of synapses or connecting links

Figure 5: ANN Model

net = X1 ∗ W1 + X2 ∗ W2 + ..... + Xn ∗ Wn + (−1) ∗ t >= 0 (2)

So, in an implementation, Consider each TLU ( Threshold Logic Unit)

1 The two classes are therefore separated by the 0 decisionline0 which is

Figure 6: XOR Non-Linear Separable Problem

Figure 7: Linear Separable Problem

A single layer network is a simple structure consisting of m neurons

To Solve Non-Linear Separable Problems Multilayer Neural Network

You might also like