Professional Documents
Culture Documents
Artificial Neural Networks
Artificial Neural Networks
NETWORKS
05/04/20 1
DEFINITION OF NEURAL NETWORKS
According to the DARPA Neural Network Study (1988, AFCEA
International Press, p. 60):
05/04/20 3
INTERCONNECTIONS IN BRAIN
05/04/20 4
BIOLOGICAL (MOTOR) NEURON
05/04/20 5
ARTIFICIAL NEURAL NET
Information-processing system.
05/04/20 6
MOTIVATION FOR NEURAL NET
05/04/20 7
The major areas being:
Massive parallelism
Learning ability
Generalization ability
Adaptivity
Fault tolerance
W1
X1 Y
W2
X2
The figure shows a simple artificial neural net with two input neurons
(X1, X2) and one output neuron (Y). The inter connected weights are
given by W1 and W2.
05/04/20 9
ASSOCIATION OF BIOLOGICAL NET
WITH ARTIFICIAL NET
05/04/20 10
PROCESSING OF AN ARTIFICIAL NET
The neuron is the basic information processing unit of a NN. It consists
of:
05/04/20 11
BIAS OF AN ARTIFICIAL NEURON
x1-x2= 1
x1
05/04/20 12
MULTI LAYER ARTIFICIAL NEURAL NET
INPUT: records without class attribute with normalized attributes
values.
HIDDEN LAYER: the number of nodes in the hidden layer and the
number of hidden layers depends on implementation.
05/04/20 13
OPERATION OF A NEURAL NET
- Bias
x0 w0j
x1 w1j
f
Output y
xn wnj
05/04/20 14
WEIGHT AND BIAS UPDATION
Per Sample Updating
05/04/20 15
STOPPING CONDITION
All change in weights (wij) in the previous epoch are below some
threshold, or
05/04/20 16
BUILDING BLOCKS OF ARTIFICIAL NEURAL NET
Network Architecture (Connection between Neurons)
Activation Function
05/04/20 17
05/04/20 18
LAYER PROPERTIES
Input Layer: Each input unit may be designated by an attribute
value possessed by the instance.
05/04/20 19
TRAINING PROCESS
Supervised Training - Providing the network with a series of
sample inputs and comparing the output with the expected
responses.
05/04/20 20
ACTIVATION FUNCTION
ACTIVATION LEVEL – DISCRETE OR CONTINUOUS
05/04/20 21
ACTIVATION FUNCTION
Activation functions:
(A) Identity
(F) Ramp
05/04/20 22
CONSTRUCTING ANN
Determine the network properties:
• Network topology
• Types of connectivity
• Order of connections
• Weight range
05/04/20 23
PROBLEM SOLVING
Select a suitable NN model based on the nature of the problem.
05/04/20 24
NEURAL NETWORKS
Neural Network learns by adjusting the weights so as to be able
to correctly classify the training data and hence, after testing
phase, to classify unknown data.
05/04/20 25
SALIENT FEATURES OF ANN
Adaptive learning
Self-organization
Real-time operation
Fault tolerance via redundant information coding
Massive parallelism
Learning and generalizing ability
Distributed representation
05/04/20 26
McCULLOCH–PITTS NEURON
Neurons are sparsely and randomly connected
All but one neuron are excitatory (tend to increase voltage of other
cells)
05/04/20 27
LINEAR SEPARABILITY
05/04/20 28
HEBB NETWORK
Donald Hebb stated in 1949 that in the brain, the learning is performed
by the change in the synaptic gap. Hebb explained it:
05/04/20 29
HEBB LEARNING
05/04/20 30
FEW APPLICATIONS OF NEURAL NETWORKS
05/04/20 31