HOMEWORK 4

NEURAL NETWORKS
SUBMITTED TO

By
Aasim Kamal
Raghul Vishnu
The homework explains the various conditions of training a neural network. There are 4
conditions that can occur in any neural network. The conditions are namely:
Ideal training
Under training
Over training

For all the cases given above the initial weight parameters and alpha are:
U0j=0.5
Uij=-1^(i+j)
V0k=-0.5
Vjk=-1^(j+k)
Alpha = 0.1
Case 1:
Ideal Training: This is the case in which the error curves are flat and the error curves are
below the threshold. In this case we need not take any action as the desired result is acquired.
The output for this case is as follows:

Here the error average of both the validation and the training errors are less than the threshold
and the error curves are flat. Hence this condition is considered as the ideal condition.
Case 2:
Under Training: This is the case in which the validation error is greater than the threshold
error and the error curves become flat. When this happens we can rectify the mistake by
increasing the number of hidden neurons in a neural network.
The output of such a case is as follows:

For the above example the number of hidden neurons is 5. From the output it can inferred
that the values of the validation and training error is higher than the threshold even though the
value of the error curves have become flat.
Now this error can be rectified by increasing the number of hidden neurons, in this case
the number of hidden neurons has been increased from 5 neurons to 10 neurons there by making
the errors converge below the threshold error limit.
 In this above example the validation and the training errors are less than the threshold
error which were decreased by the increase in the number of hidden neurons from 5 to 10 hidden
neurons. Thus in this way the undertraining of a neural network can be identified and rectified.

Case 3:
Over Training: This type of phenomenon occurs when the training error is less than the
threshold error but the validation error is greater than the threshold. When this phenomenon
occurs it can be corrected either by increasing the number of data used for traing the network or
by decreasing the number of hidden neurons.
The output of an overtraining case is as follows:
For the above example the number of hidden neurons is 20. But it can be seen that the value of
the validation error is greater than threshold but the training error is smaller than the threshold.
This phenomenon can either be corrected by increasing the number of data used for training the
system or by decreasing the number of hidden neurons. In this case the number of hidden
neurons has been reduced from 20 hidden neurons to 10 hidden neurons which would yield the
output as follows:
In the above example the error curves are flat and are both below the threshold. This was
possible by the decrease of the number of neurond from 20 to 10 neurons. Thus by using this
method over training can be prevented from happening in a neural network.

MATLAB PROGRAM
% % % % % Program for 10 neurons % % % % %
clc;
clear all;
filename='miteredbendtrain.xlsx';
sheet = 1;
xlRange1 = 'A2:A101';
xlRange3 = 'B2:B101';
xlRange4 = 'C2:C101';
xlRange5 = 'D2:D101';
xlRange6 = 'E2:E101';
xlRange7 = 'f2:f101';
x=zeros(100,2);
d=zeros(100,4);
u=sym('u',[3,10]);
u=0;
% % % % % Declaration of the u and v weights
for i=1
for j=1:10
u(i,j)=0.5;
 end
end
for i=2:3
for j=1:10
u(i,j)=(-1)^((i-1)+j);
end
end
v=sym('v',[11,4]);
v=0;
% v=zeros(6,2);
for i=1
for j=1:4
v(i,j)=-0.5;
end
end
for i=2:11
for j=1:4
v(i,j)=(-1)^((i-1)+j);
end
end
x1 = xlsread(filename,sheet,xlRange1);
x2 = xlsread(filename,sheet,xlRange3);
d1 = xlsread(filename,sheet,xlRange4);
d2 = xlsread(filename,sheet,xlRange5);
d3 = xlsread(filename,sheet,xlRange6);
d4 = xlsread(filename,sheet,xlRange7);
sheet = 1;
xlRange1 = 'A2:b101';
x = xlsread(filename,sheet,xlRange1);
% % % % % Variable declarations
gamma=sym('gamma',[100,10]);
gamma=0;
z=sym('z',[100,10]);
z=0;
y=sym('y',[100,4]);
y=0;
gammaut=sym('gammaut',[100,10]);
gammaut=0;
zut=sym('zut',[100,10]);
zut=0;
yut=sym('yut',[100,4]);
yut=0;
error=sym('error',[100,1]);
error=0;
errorval=sym('error',[200,1]);
errorval=0;
xlRange2 = 'c2:f101';
d = xlsread(filename,sheet,xlRange2);
aer=sym('a',[100,1]);
aer=0;
ber=sym('b',[100,1]);
ber=0;
cer=sym('a',[100,1]);
cer=0;
der=sym('b',[100,1]);
der=0;
aval=sym('aval',[200,1]);
aval=0;
bval=sym('bval',[200,1]);
bval=0;
delu=sym('delu',[2,10]);
delu=0;
delv=sym('delv',[11,4]);
delv=0;
delui=sym('delui',[2,10]);
delui=0;
delvi=sym('delvi',[11,4]);
delvi=0;

% % % % % data for the untrained data

sheet = 2;
xlRange1 = 'A2:A201';
xlRange3 = 'B2:B201';
xlRange4 = 'C2:C201';
xlRange5 = 'D2:D201';
xlRange6 = 'E2:E201';
xlRange7 = 'f2:f201';
xval=zeros(200,2);
dval=zeros(200,4);
x1val = xlsread(filename,sheet,xlRange1);
x2val = xlsread(filename,sheet,xlRange3);
d1val = xlsread(filename,sheet,xlRange4);
d2val = xlsread(filename,sheet,xlRange5);
d3val = xlsread(filename,sheet,xlRange6);
d4val = xlsread(filename,sheet,xlRange7);
xlRange2 = 'A2:b201';
xlRange8 = 'c2:f201';
xval = xlsread(filename,sheet,xlRange2);
dval = xlsread(filename,sheet,xlRange8);
% % % % % % scaling of the training data
for i=1:100
x(i,1)=1-(2*((max(x1)-x(i,1))/(max(x1)-min(x1))));
x(i,2)=1-(2*((max(x2)-x(i,2))/(max(x2)-min(x2))));
d(i,1)=1+((d(i,1)-max(d1))/(max(d1)-min(d1)));
d(i,2)=1+((d(i,2)-max(d2))/(max(d2)-min(d2)));
d(i,3)=1+((d(i,3)-max(d3))/(max(d3)-min(d3)));
d(i,4)=1+((d(i,4)-max(d4))/(max(d4)-min(d4)));
end
% % % % % % scaling of the validation data
for i=1:200
xval(i,1)=1-(2*((max(x1val)-xval(i,1))/(max(x1val)-min(x1val))));
xval(i,2)=1-(2*((max(x2val)-xval(i,2))/(max(x2val)-min(x2val))));
dval(i,1)=1+((dval(i,1)-max(d1val))/(max(d1val)-min(d1val)));
dval(i,2)=1+((dval(i,2)-max(d2val))/(max(d2val)-min(d2val)));
dval(i,3)=1+((dval(i,3)-max(d3val))/(max(d3val)-min(d3val)));
dval(i,4)=1+((dval(i,4)-max(d4val))/(max(d4val)-min(d4val)));
end
stmt='The scaled value of the training inputs are';
disp(stmt)
x
stmt1='The scaled value of the training values of d is';
disp(stmt1)
d
% % % % % % Randomising the values
rand=randperm(100);
x = x(rand,:);
d = d(rand,:);
disp('The randomised values of x and d are as follows');
x % % % % % this prints the randomised rows of x and d
d
% % % % % % feed forward equations for the training data
err=zeros(100,1);
errut=zeros(100,1);
for epoch=1:100
eavg=0;
eavgut=0;
for k=1:100
for j=1:10
gamma(k,j)=0;
end
for i=1:2
for j=1:10
gamma(k,j)=gamma(k,j)+x(k,i)*u(i+1,j);
end
end
for j=1:10
gamma(k,j)=u(1,j)+gamma(k,j);
end
for j=1:10
z(k,j)=1/(1+exp(-gamma(k,j)));
end
for j=1:4
y(k,j)=0;
end
for j=1:4
for i=1:10
y(k,j)=y(k,j)+v(i+1,j)*z(k,i);
end
end
for j=1:4
y(k,j)=y(k,j)+v(1,j);
end
%%%%% feed forward equations end here %%%%%
error(k,1)=0;
% % % error calculations
aer(k,1)=0;
ber(k,1)=0;
cer(k,1)=0;
der(k,1)=0;
aer(k,1)=(y(k,1)-d(k,1));
ber(k,1)=(y(k,2)-d(k,2));
cer(k,1)=(y(k,3)-d(k,3));
der(k,1)=(y(k,4)-d(k,4));
error(k,1)=0.5*((aer(k,1)^2)+(ber(k,1)^2)+(cer(k,1)^2)+(der(k,1)^2));
eavg=eavg+error(k);
% % % % % delta value calculations
alpha=0.1;
for j=1:10
for i=1:3
delu(i,j)=0;
delui(i,j)=0;
 end
end
for j=1:10
for i=1:2
delu(i+1,j)=(aer(k,1)*(v(j+1,1)*z(k,j)*(1-
z(k,j))*x(k,i)))+(ber(k,1)*(v(j+1,2)*z(k,j)*(1-
z(k,j))*x(k,i)))+(cer(k,1)*(v(j+1,3)*z(k,j)*(1-
z(k,j))*x(k,i)))+(der(k,1)*(v(j+1,4)*z(k,j)*(1-z(k,j))*x(k,i)));
delu(1,j)=(aer(k,1)*(v(j+1,1)*z(k,j)*(1-
z(k,j))*x(k,i)))+(ber(k,1)*(v(j+1,2)*z(k,j)*(1-
z(k,j))*x(k,i)))+(cer(k,1)*(v(j+1,3)*z(k,j)*(1-
z(k,j))*x(k,i)))+(der(k,1)*(v(j+1,4)*z(k,j)*(1-z(k,j))*x(k,i)));
delui(i+1,j)=delui(i+1,j)+delu(i+1,j);
delui(1,j)=delui(1,j)+delu(1,j);
end
end
for j=1:4
for i=1:11
delv(i,j)=0;
delvi(i,j)=0;
end
end
for i=1:5
delv(i+1,1)=(aer(k,1)*z(k,i));
delv(i+1,2)=(ber(k,1)*z(k,i));
delv(i+1,3)=(cer(k,1)*z(k,i));
delv(i+1,4)=(der(k,1)*z(k,i));
delvi(i+1,1)=delv(i+1,1)+delvi(i+1,1);
delvi(i+1,2)=delv(i+1,2)+delvi(i+1,2);
delvi(i+1,3)=delv(i+1,3)+delvi(i+1,3);
delvi(i+1,4)=delv(i+1,4)+delvi(i+1,4);
end
delv(1,1)=(aer(1,1));
delv(1,2)=(ber(1,1));
delv(1,3)=(cer(1,1));
delv(1,4)=(der(1,1));
delvi(1,1)=delv(1,1)+delvi(1,1);
delvi(1,2)=delv(1,2)+delvi(1,2);
delvi(1,3)=delv(1,3)+delvi(1,3);
delvi(1,4)=delv(1,4)+delvi(1,4);
end
for i=1:3
for j=1:10
u(i,j)=u(i,j)-(alpha*delui(i,j));
end
end
for i=1:11
for j=1:4
v(i,j)=v(i,j)-(alpha*delvi(i,j));
end
end
eavg=eavg/100;
err(epoch)=eavg;
for k=1:200
for j=1:10
gammaut(k,j)=0;
end
 for i=1:2
for j=1:10
gammaut(k,j)=gammaut(k,j)+xval(k,i)*u(i+1,j);
end
end
for j=1:10
gammaut(k,j)=u(1,j)+gammaut(k,j);
end
for j=1:10
zut(k,j)=1/(1+exp(-gammaut(k,j)));
end
for j=1:4
yut(k,j)=0;
end
for j=1:4
for i=1:10
yut(k,j)=yut(k,j)+v(i+1,j)*zut(k,i);
end
end
for j=1:4
yut(k,j)=yut(k,j)+v(1,j);
end
%%%%% feed forward equations end here %%%%%
% % % % error calculations
aval(k,1)=0;
bval(k,1)=0;
cval(k,1)=0;
dval(k,1)=0;
aval(k,1)=(yut(k,1)-dval(k,1));
bval(k,1)=(yut(k,2)-dval(k,2));
cval(k,1)=(yut(k,3)-dval(k,3));
dval(k,1)=(yut(k,4)-dval(k,4));

errorval(k,1)=0.5*((aval(k,1)^2)+(bval(k,1)^2)+(cval(k,1)^2)+(dval(k,1)^2));
eavgut=eavgut+errorval(k);
end
eavgut=eavgut/200;
errut(epoch)=eavgut;
end
hold on;
title('Error Average vs Epochs plots of Training & Testing Data');
xlabel('Epochs');
ylabel('Average Error');
p1=plot(errut,'-b');
p2=plot(err,'-r');
legend('validation','Training');
hold off;