Professional Documents
Culture Documents
Unit II Supervised II
Unit II Supervised II
Supervised Learning II
• Back propagation, also named the Generalized Delta Rule, is an algorithm used in
the training of ANNs for supervised learning. (generalizations exist for other
artificial neural networks) It efficiently computes the gradient of the error
function with respect to the weights of the network for a single input-output
example.
• This makes it feasible to use gradient methods for training multi-layer networks,
updating the weights to minimize loss.
• Since the same training rule is applied recursively for each layer of the neural
network, we can calculate the contribution of each weight to the total error
inversely from the output layer to the input layer.
Architecture
• As shown in the diagram, the architecture of BPN has three interconnected
layers having weights on them.
• The hidden layer as well as the output layer also has bias, whose weight is
always 1, on them.
• As is clear from the diagram, the working of BPN is in two phases.
• One phase sends the signal from the input layer to the output layer, and the
other phase back propagates the error from the output layer to the input layer.
where ek ( p ) yd ,k ( p ) yk ( p )
Calculate the weight corrections:
w jk ( p) y j ( p) k ( p)
Update the weights at the output neurons:
w jk ( p 1) w jk ( p ) w jk ( p )
DEPARTMENT OF INFORMATION TECHNOLOGY 12 11/09/2021
12
Step 3: Weight training (continued)
(b) Calculate the error gradient for the neurons
in the hidden layer:
l
j ( p) y j ( p ) [1 y j ( p )] k ( p ) w jk ( p )
k 1
Calculate the weight corrections:
wij ( p ) xi ( p ) j ( p )
Update the weights at the hidden neurons:
wij ( p 1) wij ( p ) wij ( p )