Professional Documents
Culture Documents
Chapter2-Neural+Network PartA
Chapter2-Neural+Network PartA
Chapter2-Neural+Network PartA
Learning Outcome
Able to understand the principle of neural network
ASNOR/KEE/UPM/EEE4404
Main events in the history of AI
What is Artificial Neural Network
(ANN)?
• ‘Artificial neural network’ or ANN is the term used to
describe a computer model assumption of the biological
brain.
• It consists of a set of interconnected simple processing
units which combine to output a signal to solve a certain
problem based on the input signals it received.
• The interconnected simple processing units have
adjustable gains that are slowly adjusted through
iterations influenced by the input output patterns given to
the ANN.
What is Artificial Neural Network (ANN)?
(cont)
• An ANN is an information-processing system that has
certain performance characteristics in common with
biological neural networks.
• Basically it is a system that handles many input signals,
processes them, and output them to solve a task that it
has been trained to solve.
What is Artificial Neural Network (ANN)?
(cont)
• ANNs have been developed as generalizations of mathematical
models of human cognition or neural biology, based on the
assumptions that:
An information processing system, developed based on the biological
brain.
Consists of simple processing units called neurons (nodes).
All the neurons are assumed to be interconnected.
Neurons are activated by a certain (activation) function.
For each interconnection, there is an associated weight (adjustable
gains).
Signals are processed from input to output by multiplying the weights
and activated signals.
A learning rule is needed to adapt the weights to solve a particular
problem.
Training patterns are needed in order to train the ANN.
Some Keywords related to ANN
• Neuro-models
• Learning / Teaching Phase
• Activation functions
• Model parameters
• Training Data
• Inputs / Outputs
• Error
• Weights
• Adaptation
• Test / Simulation
• Application
Application of ANN
Output Signals
Input Signals
Middle Layer
(Hidden Layer)
Input Layer Output Layer
The neuron as a simple computing element
Diagram of a neuron
Input Signals Weights Output Signals
x1
Y
w1
x2
w2
Neuron Y Y
wn Y
xn
The State of Activation
• Is the activity of the processing element (neuron).
• Usually the state of activation of neuron can either be ON
(fired or 1) or OFF (unfired or 0).
• Is dependent on the application or task to be solved.
• Activation values can have minimum and maximum levels.
• Activation signals can be continuous or discrete.
• Typical examples of activation signals are:
binary (0,1)
bipolar (-1,1)
• Example of a sigmoid activation function
with continuous binary value
Output of Processing Units
(PU)
• Usually every neuron or PU has an output and the output
is a function of the input.
• The degree of activation of units determines the strength
of signal transmitted out.
• An Output function can have the following expression:
Feedforward
• All signals flow in one direction only, i.e. from lower
layers (input) to upper layers (output)
Feedback
• Signals from neurons in upper layers are fed back to
either its own or to neurons in lower layers.
Learning in ANN
• In all of the neural paradigms, the application of an ANN
involves two phases:
1. Learning phase
2. Recall phase