Professional Documents
Culture Documents
Chapter 3 - Neural Network
Chapter 3 - Neural Network
Synapse Dendrites
Axon
Axon
Soma Soma
Dendrites
Synapse
Output Signals
Input Signals
Middle Layer
Input Layer Output Layer
x1
Y
w1
x2
w2
Neuron Y Y
wn Y
xn
Y Y Y Y
+1 +1 +1 +1
0 X 0 X 0 X 0 X
-1 -1 -1 -1
Inputs
x1 Linear Hard
w1 Combiner Limiter
Output
∑ Y
w2
θ
x2
Threshold
Class A 1
1
2
1
x1
Class A 2 x1
x 1 w 1 + x 2w 2 − θ = 0 x1 w 1 + x2 w 2 + x3 w 3 − θ = 0
x3
(a) Two-input perceptron. (b) Three-input perceptron.
e( p) = Yd ( p) − Y ( p) where p = 1, 2, 3, . . .
where p = 1, 2, 3, . . .
α is the learning rate, a positive constant less than
unity.
Step 2: Activation
Activate the perceptron by applying inputs x1(p),
x2(p),…, xn(p) and desired output Yd (p).
Calculate the actual output at iteration p = 1
n
Y ( p ) = step ∑ x i ( p ) w i ( p ) − θ
i = 1
where n is the number of the perceptron inputs,
and step is a step activation function.
1 1 1
x1 x1 x1
0 1 0 1 0 1
Output Signals
Input Signals
First Second
Input hidden hidden Output
layer layer layer layer
i wij j wjk
xi k yk
m
n l yl
xn
Input Hidden Output
layer layer layer
Error signals
θ3
w13 −1
x1 1 3 w35
w23 θ5
5 y5
w24
x2 2 4 w45
w24
Input θ4 Output
layer layer
−1
Hidden layer
Negnevitsky, Pearson Education, 2005 38
n The effect of the threshold applied to a neuron in the
hidden or output layer is represented by its weight, θ,
connected to a fixed input equal to −1.
n The initial weights and threshold levels are set
randomly as follows:
w13 = 0.5, w14 = 0.9, w23 = 0.4, w24 = 1.0, w35 = −1.2,
w45 = 1.1, θ3 = 0.8, θ4 = −0.1 and θ5 = 0.3.
θ 5 = θ 5 + ∆ θ 5 = 0 . 3 + 0 . 0127 = 0 . 3127
100
Sum-Squared Error
10-1
10-2
10 -3
10 -4
0 50 100 150 200
Epoch
+1.5
−1
+1.0
x1 1 3 −2.0 +0.5
+1.0
5 y5
+1.0
x2 2 +1.0
4
+1.0
+0.5
−1
Negnevitsky, Pearson Education, 2005 46
Decision boundaries
x2 x2 x2
x1 + x 2 – 1.5 = 0 x 1 + x2 – 0.5 = 0
1 1 1
x1 x1 x1
0 1 0 1 0 1