ARTIFICIAL NEURAL NETWORK

CHAPTER 1

INTRODUCTION

1.1 Background

An artificial neural network is a system based on the operation of biological neural networks, in
other words, is an emulation of biological neural system. Why would be necessary the
implementation of artificial neural networks? Although computing these days is truly advanced,
there are certain tasks that a program made for a common microprocessor is unable to perform;
even so a software implementation of a neural network can be made with their advantages and
disadvantages.

1.2 Basics of ANN

This terminology has been developed from the biological model of the brain. A neural network
consists of a set of connected cells: THE NEURONS. The neurons receive impulses either from
input cells or other neurons and perform some kind of transformation on the input and transmit
the outcome to other neurons or output cells. The neural networks are built from layers of
neurons connected so that one layer receives an input from the receding layer of neurons and
passes the output on to the subsequent.

1.3 Style of Neural Computation

Figure 1.1 The Style Of Neural Computation

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-1-
 ARTIFICIAL NEURAL NETWORK

An input is presented to the neural network and a corresponding desired or target response set at
the output (when this is the case the training is called supervision). An error is composed from
the difference between the desired response and the system output. This error information is fed
back to the system and adjusts the system parameters in a systematic fashion (the learning rule).
The process is repeated until the performance is acceptable. It is clear from this description that
the performance hinges heavily on the data. If one does not have data that cover a significant
portion of the operating conditions or if they are noisy, then neural network technology is
probably not the right solution. On the other hand, if there is plenty of data and the problem is
poorly understood to derive an approximate model, then neural network technology is a good
choice. In artificial neural networks, the designer chooses the network topology, the performance
function, the learning rule, and the criterion to stop the training phase, but the system
automatically adjusts the parameters. So, it is difficult to bring a priori information into the
design, and when the system does not work properly it is also hard to incrementally refine the
solution. But ANN-based solutions are extremely efficient in terms of development time and
resources, and in many difficult problems artificial neural networks provide performance that is
difficult to match with other technologies. Denker 10 years ago said that "artificial neural
networks are the second best way to implement a solution" motivated by the simplicity of their
design and because of their universality, only shadowed by the traditional design obtained by
studying the physics of the problem. At present, artificial neural networks are emerging as the
technology of choice for many applications, such as pattern recognition, prediction, system
identification, and control.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-2-
 ARTIFICIAL NEURAL NETWORK

Figure 1.2: Computation of error

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-3-
 ARTIFICIAL NEURAL NETWORK

CHAPTER 2

DIFFERENT ANN MODELS

2.1 The Biological Model

Artificial neural networks emerged after the introduction of simplified neurons by McCulloch
and Pitts in 1943 (McCulloch & Pitts, 1943). These neurons were presented as models of
biological neurons and as conceptual components for circuits that could perform
computational tasks. The basic model of the neuron is founded upon the functionality of a
biological neuron. "Neurons are the basic signaling units of the nervous system" and "each
neuron is a discrete cell whose several processes arise from its cell body.”

Figure 2.1: Biological structure of a neuron

A neuron has four main regions to its structure. The cell body, or soma, has two offshoots from
it, the dendrites, and the axon, which end in pre-synaptic terminals. The cell body is the
heart of the cell, containing the nucleus. A neuron may have many dendrites, which branch
out in a treelike structure, and receive signals from other neurons. A neuron usually only
has one axon. The axon conducts electric signals down its length. These electric signals are
called action potentials. The other end of the axon may split into several branches, which
end in a pre-synaptic terminal. Action potentials are the electric signals that neurons use to
DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE
-4-
 ARTIFICIAL NEURAL NETWORK

convey information to the brain. All these signals are identical. Therefore, the brain
determines what type of information is being received based on the path that the signal
took. The brain analyzes the patterns of signals being sent and from that information it can
interpret the type of information being received. This ensures that the signal traveling
down the axon travels fast and remains constant (i.e. very short propagation delay and no
weakening of the signal). The synapse is the area of contact between two neurons. The
neurons do not actually physically touch. They are separated by the synaptic cleft. The
neuron sending the signal is called the pre -synaptic cell and the neuron receiving the
signal is post -synaptic cell.

2.2 The Mathematical Model

When creating a functional model of the biological neuron, there are three basic components of
importance. First, the synapses of the neuron are modeled as weights. The strength of the
connection between an input and a neuron is noted by the value of the weight. Negative weight
values reflect inhibitory connections, while positive values designate excitatory connections. The
next two components model the actual activity within the neuron cell. An adder sums up all the
inputs modified by their respective weights. This activity is referred to as linear combination.
Finally, an activation function controls the amplitude of the output of the neuron. An acceptable
range of output is usually between 0 and 1, or -1 and 1. Mathematically, this process is described
in the figure.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-5-
 ARTIFICIAL NEURAL NETWORK

Figure 2.2: Mathematical computation for ANN

From this model the interval activity of the neuron can be shown to be:

The output of the neuron, yk, would therefore be the outcome of some activation function on the
value of vk.

2.3 Activation Functions

As mentioned previously, the activation function acts as a squashing function, such that the
output of a neuron in a neural network is between certain values (usually 0 and 1, or -1 and 1). In
general, there are three types of activation functions, denoted by Φ(.) . First, there is the
Threshold Function which takes on a value of 0 if the summed input is less than a certain

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-6-
 ARTIFICIAL NEURAL NETWORK

threshold value (v), and the value 1 if the summed input is greater than or equal to the threshold
value.

Secondly, there is the Piecewise-Linear function. This function again can take on the values of 0
or 1, but can also take on values between that depending on the amplification factor in a certain
region of linear operation.

Thirdly, there is the sigmoid function. This function can range between 0 and 1, but it is also
sometimes useful to use the -1 to 1 range. An example of the sigmoid function is the hyperbolic
tangent function.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-7-
 ARTIFICIAL NEURAL NETWORK

Figure 2.3: Graphs of activation functions

2.4 Kinematics

Kinematics is the science of motion which treats motion without regard to the forces which cause
it. Within this science one studies the position, velocity, acceleration, and all higher order
derivatives of the position variables. A very basic problem in the study of mechanical
manipulation is that of forward kinematics. This is the static geometrical problem of computing
the position and orientation of the end-effector ('hand') of the manipulator. Specifically, given a
set of joint angles, the forward kinematic problem is to compute the position and orientation of
the tool frame relative to the base frame.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-8-
 ARTIFICIAL NEURAL NETWORK

Figure 2.4: An exemplar robot manipulator

2.5 Inverse Kinematics

This problem is posed as follows: given the position and orientation of the end-effector of the
manipulator, calculate all possible sets of joint angles which could be used to attain this given
position and orientation. This is a fundamental problem in the practical use of manipulators. The
inverse kinematic problem is not as simple as the forward one. Because the kinematic equations
are nonlinear, their solution is not always easy or even possible in a closed form. Also, the
questions of existence of a solution, and of multiple solutions, arise.

2.6 Dynamics

Dynamics is a field of study devoted to studying the forces required to cause

motion. In order to accelerate a manipulator from rest, glide at a constant end-effector velocity,
and finally decelerate to a stop, a complex set of torque functions must be applied by the joint
actuators. In dynamics not only the geometrical properties (kinematics) are used, but also the

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-9-
 ARTIFICIAL NEURAL NETWORK

physical properties of the robot are taken into account. Take for instance the weight (inertia) of
the robot arm, which determines the force required to change the motion of the arm. The
dynamics introduces two extra problems to the kinematic problems.

1) The robot arm has a 'memory'. Its responds to a control signal depends also on its history (e.g.
previous positions, speed, acceleration).

2) If a robot grabs an object then the dynamics change but the kinematics do not change. This is
because the weight of the object has to be added to the weight of the arm (that's why robot arms
are so heavy, making the relative weight change very small).

2.7 End-Effector Positioning

The main aim in robot manipulator control is often the positioning of the hand or end-effector in
order to be able to, e.g., pick up an object. With the accurate robot arms that are manufactured,
this task is often relatively simple, involving the following steps:

1. Determine the target coordinates relative to the base of the robot. Typically, when this
position is not always the same, this is done with a number of fixed cameras or other
sensors which observe the work scene, from the image frame determine the position of
the object in that frame, and perform a pre-determined coordinate transformation;
2. With a precise model of the robot (supplied by the manufacturer), calculate the joint
angles to reach the target (i.e., the inverse kinematics).
3. Move the arm (dynamics control) and close the gripper.

2.8 Need For Involvement Of Neural Networks

So if these parts are relatively simple to solve with a high accuracy, why involve neural
networks? The reason is the applicability of robots. When 'traditional' methods are used to
control a robot arm, accurate models of the sensors and manipulators (in some cases with
unknown parameters which have to be estimated from the system's behavior ; yet still with
accurate models as starting point) are required and the system must be calibrated. Also, systems
which suffer from wear-and-tear (and which mechanical systems don't?) need frequent

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-10-
 ARTIFICIAL NEURAL NETWORK

recalibration or parameter determination. Finally, the development of more complex (adaptive!)

control methods allows the design and use of more flexible (i.e., less rigid) robot systems.

2.9 Camera Robot Co-Ordination

The system we focus on in this section is a work door observed by fixed cameras, and a robot
arm. The visual system must identify the target as well as determine the visual position of the
end-effectors.

The target position Xtarget together with the visual position of the hand Xhand are input to
the neural controller N(.). This controller then generates a joint position θ for the robot:

θ=N(Xtarget ,Xhand)

We can compare the neurally generated θ with the optimal θ0 generated by a fictitious perfect
controller R(.):

θ0 = R(Xtarget,Xhand):

The task of learning is to make the N generate an output 'close enough' to θ0 There are two
problems associated with teaching N(.):

1. Generating learning samples which are in accordance with eq. θ0 = R(Xtarget,Xhand).

This is not trivial, since in useful applications R(.) is an unknown function. Instead, a
form of self-supervised or unsupervised learning is required.
2. Constructing the mapping N(.) from the available learning samples. When the (usually
randomly drawn) learning samples are available, a neural network uses these samples to
represent the whole input space over which the robot is active. This is evidently a form of
interpolation, but has the problem that the input space is of a high dimensionality, and the
samples are randomly distributed.

2.10 Feed Forward Networks

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-11-
 ARTIFICIAL NEURAL NETWORK

When using a feed-forward system for controlling the manipulator, a self-supervised learning
system must be used. One such a system has been reported by (Psaltis, Sideris, & Yamamura,
1988). Here, the network, which is constrained to two-dimensional positioning of the robot arm,
learns by experimentation. Three methods are proposed:

1. Indirect learning: In indirect learning, a Cartesian target point x in world coordinates is

generated, e.g., by a two cameras looking at an object. This target point is fed into the
network, which generates an angle vector θ. The manipulator moves to position θ, and the
cameras determine the new position x' of the end-effector in world coordinates. This x'
again is input to the network, resulting in θ'. The network is then trained on the error
ε1= θ-θ'

Figure 2.5: ANN indirect learning model

2. Indirect learning system for robotics. In each cycle, the network is used in two different
places: first in the forward step, then for feeding back the error.
However, minimization of ε1 does not guarantee minimization of the overall error ε = x-
x'. For example, the network often settles at a 'solution' that maps all x's to a single θ.
3. General learning: Here the plant input θ must be provided by the user. Thus the network
can directly minimize |θ - θ'|. The success of this method depends on the interpolation
capabilities of the network.
4. Specialized learning: Keep in mind that the goal of the training of the network is to
minimize the error at the output of the plant: ε = x - x'. We can also train the neural

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-12-
 ARTIFICIAL NEURAL NETWORK

network by 'back propagating' this error trough the plant (compare this with the back
propagation of the error). This method requires knowledge of the Jacobian matrix of the
plant. A Jacobian matrix of a multidimensional function F is a matrix of partial
derivatives of F, i.e., the multidimensional form of the derivative. For example, if we
have

Y=F(X),

then

or

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-13-
 ARTIFICIAL NEURAL NETWORK

or

where J is the Jacobian matrix of F. So, the Jacobian matrix can be used to calculate the
change in the function when its parameters change. Now, in this case we have

where Pi( θ ) the ith element of the plant output for input θ. The total error ε = x - x' is
propagated back through the plant by calculating the δ as in eq.

where i iterates over the outputs of the plant. When the plant is an unknown function,

where can be approximated by

where ej is used to change the scalar θj into a vector. This approximate derivative can be
measured by slightly changing the input to the plant and measuring the changes in the
output.

Again a two-layer feed-forward network is trained with back-propagation. However, instead of

calculating a desired output vector the input vector which should have invoked the current output
vector is reconstructed, and back-propagation is applied to this new input vector and the existing
output vector. The configuration used consists of a monocular (with 1 eye) manipulator which
has to grasp objects. Due to the fact that the camera is situated in the hand of the robot, the task
is to move the hand such that the object is in the centre of the image and has some predetermined

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-14-
 ARTIFICIAL NEURAL NETWORK

size in which the visual flow-field is used to account for the monocularity of the system, such
that the dimensions of the object need not to be known anymore to the system. One step towards
the target consists of the following operations:

1. Measure the distance from the current position to the target position in camera domain, x.
2. Use this distance, together with the current state θ of the robot, as input for the neural
network. The network then generates a joint displacement vector Δθ.
3. Send Δθ to the manipulator.
4. Again measure the distance from the current position to the target position in camera
domain, x'.

5. Calculate the move made by the manipulator in visual domain, where is

the rotation matrix of the second camera image with respect to the rest camera image;

6. Teach the learning pair to the neural network.

This system has shown to learn correct behavior in only tens of iterations, and to be very
adaptive to changes in the sensor or manipulator.

By using a feed-forward network, the available learning samples are approximated by a single,
smooth function consisting of a summation of sigmoid functions. A feed-forward network with
one layer of sigmoid units is capable of representing practically any function. But how are the
optimal weights determined in finite time to obtain this optimal representation? Experiments
have shown that, although a reasonable representation can be obtained in a short period of time,
an accurate representation of the function that governs the learning samples is often not feasible
or extremely difficult. The reason for this is the global character of the approximation obtained
with a feed-forward network with sigmoid units: every weight in the network has a global effect
on the final approximation that is obtained. Building local representations is the obvious way
out: every part of the network is responsible for a small subspace of the total input space. Thus
accuracy is obtained locally (Keep It Small & Simple). This is typically obtained with a
Kohonean neural network.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-15-
 ARTIFICIAL NEURAL NETWORK

CHAPTER 3

ROBOT MANIPULATOR MECHANISM

3.1 Inverse Kinematics of A Robot Manipulator

The robot manipulator has three degree of freedoms, which consists of two
revolute joints and one prismatic joint. Figure 1 shows picture of the robot
manipulator. The prismatic joint moves the third link up and down when the
drilling process is being done. A drill tool is attached at the third link.
Inverse kinematics is needed for calculating the position of each joint for a
set of given tool coordinate. Because the third link only moves full up and full
down when the drilling process is being done, the inverse kinematics is only
considered for two revolute joints, which determine the drilling position
coordinate. For a given tool coordinate (x, y), the position of the two revolute
joints (θi) can be defined as follows:
θi= f(x, y)
(1)

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-16-
 ARTIFICIAL NEURAL NETWORK

Figure 3.1 Three degree of freedom robot manipulator with two revolute
joints and one prismatic joint.

Assume that there is a coordinate system for the robot manipulator as shown
at figure 2.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-17-
 ARTIFICIAL NEURAL NETWORK

Figure 3.2: Co-ordinate system of the robot manipulator.

The position of point P(x ,y) can be determined by using following equation:
P(x, y) = f (θ1, θ2)
(2)
Assume that vector P is determined by adding vector r1 and vector r2 where
r1 and r2 can be defined as follows:
r1 = [l1 cosθ1 , l1 sinθ1]
(3)
r2 = [l2 cos(θ1 +θ2) , l2 sin(θ1 +θ2)]
(4)

Then

x = l1cosθ1 + l2 cos (θ1 +θ2)

(5)
y = l1sinθ1 + l2 sin (θ1 +θ2)
(6)

Equation (5) and (6) are forward kinematics equation of the robot
manipulator with two revolute joints.
The inverse kinematics equation can be derived from (5) and (6). By using
trigonometry equation, (5) and (6) can be written as follows:

x = l1cosθ1 + l2 cosθ1cosθ 2 − l2 sinθ1sinθ2

(7)
y = l1sinθ1 + l2 sinθ1cosθ 2 + l2 cosθ1sinθ2
(8)

Solving (7) and (8) to find value of θ1 and θ2 results the following equation:

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-18-
 ARTIFICIAL NEURAL NETWORK

θ1=tan-1((y(l1+ l2 cosθ2)-x l2 sinθ2 ) / (x(l1+l2 cosθ2) +y l2 sinθ2 ))

(9)

θ2=cos-1((x*x +y*y-l1*l1-l2*l2) / (2 * l1 * l2))

(10)

Equation (9) and (10) are inverse kinematics equation of the robot
manipulator, which has two revolute joints. For a set of given coordinate P(x ,
y), the position of each link (θ1, θ2) can be calculated by using (9) and (10).

3.2 Modeling Inverse Kinematics Using Artificial Neural Network

Here artificial neural network was applied for modeling the inverse kinematics of the robot
manipulator and the model is only considered for the kinematics of two revolute joints. The
architecture of artificial neural network used in this research is a multi layer perceptron with
steepest descent back propagation training algorithm. Figure 3 shows the architecture of artificial
neural network used for modeling inverse kinematic of the robot manipulator.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-19-
 ARTIFICIAL NEURAL NETWORK

Figure 3.3: Layers of ANN

The network has two input neurons. Both input neurons receive the information of tool
coordinate P(x, y). The output layer of the network has two neurons. Both neurons in the output
layer give information about the position of each link (θ1, θ2). Number of hidden layer of the
network varies from 1 to 10 layers and number of neuron per hidden layer varies from 1 to 50
neurons. If the artificial neural network has m layers and receives input of vector p, then the
output of the network can be calculated by using the following equation:

(a^m) = (f^m)(W^m*f^m-1(W^m-1*f^m-2(…….W^2*f(w^1*p+b^1)+b^2)+b^m-1)+b^m)
(11)
Where f m is log-sigmoid transfer function of the mth layer of the network that can be defined as
following equation:

F( n) = (1+(e^(-n)))^(-1) (12)

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-20-
 ARTIFICIAL NEURAL NETWORK

W m is weight of the mth layer of the network, and b m is bias of the mth layer of the network.
Equation (11) is known as the feed forward calculation. Back propagation algorithm is used as
the training method of the designed artificial neural network. The back propagation algorithm
includes the following steps:
1. Initialize weights and biases to small random numbers.
2. Present a training data to neural network and calculate the output by propagating the input
forward through the network using (11).
3. Propagate the sensitivities backward through the network:

S^m = (-2)*F^m*(n^m)*(t-a)
(13)
S^m = F^m * (n^m)*(W^(m+1))^t * (S^m+1)
(14)

4. Calculate weight and bias updates

Δ(W ^m)(k) = - α * (S^m) *( α ^m-1)^t
(15)
Δ(b^m)^(k) = - α *(S^m)
(16)

Where α is learning rate.

5. Update the weights and biases

(W ^m)(k+1) = (W ^m) + Δ(W ^m)(k)
(17)
(b ^m)(k+1) = (b ^m) + Δ(b ^m)(k)
(18)

6. Repeat step 2 – 5 until error is zero or less than a limit value.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-21-
 ARTIFICIAL NEURAL NETWORK

3.3 Experimental Result

Experiment was done to see performance of artificial neural network in order to model the
inverse kinematic of the robot manipulator. The performance of artificial neural network is
indicated by the RMSE (ROOT MEAN SQUARE ERROR) value. Experiment was done in
various training parameter value of artificial neural network, i.e., various numbers of hidden
layers, various number of neuron per layer, and various value of learning rate. There were 200
training data used in this research for training the artificial neural network. The training data
were created by running the robot manually and measuring the position of each link directly.
Table I shows the experimental result in various learning rate and number of neuron per hidden
layer for the artificial neural network using 1 hidden layer. The best RMSE value is 0.01474.
Overall, the best performance is resulted by artificial neural network with 1 hidden layer and 38
neurons per hidden layer.

Table 3.1: RMSE Value of Network

No of 0.1 0.3 0.5 0.7 0.9

layers
1 0.05005 0.04715 0.04494 0.04358 0.04274

12 0.04382 0.0409 0.03936 0.0390 0.03823

25 0.04128 0.0860 0.03591 0.03535 0.03678

38 0.01474 0.03484 0.03642 0.03946 0.04029

50 0.0324 0.03658 0.04197 0.05223 0.05903

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-22-
 ARTIFICIAL NEURAL NETWORK

CHAPTER 4

ADVANTAGES AND DISADVANTAGES

4.1 Advantages

1. A neural network can perform tasks that a linear program cannot.

2. When an element of the neural network fails, it can continue without any problem
by their parallel nature.
3. A neural network learns and does not need to be reprogrammed.
4. It can be implemented in any application.
5. It can be implemented without any problem.

4.2 Disadvantages

1. The neural network needs training to operate.

2. The architecture of a neural network is different from the architecture of
microprocessors therefore needs to be emulated. Requires high processing time
for large neural networks.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-23-
 ARTIFICIAL NEURAL NETWORK

CHAPTER 5

CONCLUSIONS

From the experimental result, it can be concluded that the artificial neural network can be used to
model the inverse kinematic of robot manipulator. By using artificial neural network, we do not
need to derive mathematical model of inverse kinematics of robot. The inverse kinematic of
robot is modeled into form of number of hidden layer, number of neurons per layer, weight and
bias value of the network. In this case, the best model is achieved by artificial neural network
with 1 hidden layer, 38 neurons per layer. And this model gives RMSE value of 0.01474.

Thus when the traditional method for erecting a robot manipulator becomes extremely difficult,
we need to go for the ANN technology where we don’t need to explicitly write programs for the
robot. Instead, the robot learns through a trial and error method. This saves a lot of programming
time as well as the cost of constructing a robot.

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-24-
 ARTIFICIAL NEURAL NETWORK

REFERENCES

Thiang, Handry Khoswanto, Rendy Pangaldus,” Artificial Neural Network with Steepest Descent
Back propagation Training Algorithm for Modeling Inverse Kinematics of Manipulator”, World
Academy of Science, Engineering and Technology 60 2009

DEPARTMENT OF BIOTECHNOLOGY, SCOE, PUNE

-25-