Unit- I

Introduction to Computational Intelligence;

types of Computational Intelligence,
components of Computational Intelligence.
Concept of Learning/Training model.
Parametric Models, Nonparametric Models.
Multilayer Networks: Feed Forward network,
Feedback network
 Introduction to Computational Intelligence
• A major thrust in algorithmic development is the
design of algorithmic models to solve increasingly
complex problems. Enormous successes have been
achieved through the modeling of biological and
natural intelligence, resulting in so-called “intelligent
systems”. These intelligent algorithms include
artificial neural networks, evolutionary computation,
swarm intelligence, artificial immune systems, and
fuzzy systems. Together with logic, deductive
reasoning, expert systems, case-based reasoning and
symbolic machine learning systems, these intelligent
algorithms form part of the field of Artificial
Intelligence (AI).
• But what is intelligence? Attempts to find definitions
of intelligence still provoke heavy debate. Dictionaries
define intelligence as the ability to comprehend, to
understand and profit from experience, to interpret
intelligence, having the capacity for thought and
reason (especially to a high degree).
• Can computers be intelligent? This is a question that
to this day causes more debate than the definitions of
intelligence. In the mid-1900s, Alan Turing gave
much thought to this question. He believed that
machines could be created that would mimic the
processes of the human brain. Turing strongly
believed that there was nothing the brain could do
that a well-designed computer could not.
This book considers five main paradigms of Computation Intelligence (CI), namely
artificial neural networks (NN), evolutionary computation (EC), swarm intelligence
(SI), artificial immune systems (AIS), and fuzzy systems (FS). Figure 1.1 gives a
summary of the aim of the book. In addition to CI paradigms, probabilistic methods
are frequently used together with CI techniques, which is also shown in the figure.
Soft computing, a term coined by Lotfi Zadeh, is a different grouping of paradigms,
which usually refers to the collective set of CI paradigms and probabilistic methods.
The arrows indicate that techniques from different paradigms can be combined to
form hybrid systems.
 Components of Computational Intelligence
• The five main principles of CI and its applications
• Fuzzy logic.
• Neural networks.
• Evolutionary computation.
• Learning theory.
• Probabilistic methods.
• Working of a Biological Neuron
• As shown in the above diagram, a typical neuron
consists of the following four parts with the help of
which we can explain its working −
• Dendrites − They are tree-like branches, responsible
for receiving the information from other neurons it is
connected to. In other sense, we can say that they are
like the ears of neuron.
• Soma − It is the cell body of the neuron and is
responsible for processing of information, they have
received from dendrites.
• Axon − It is just like a cable through which neurons
send the information.
• Synapses − It is the connection between the axon
and other neuron dendrites.
• In living organisms, the brain is the control unit of the
neural network, and it has different subunits that take care
of vision, senses, movement, and hearing. The brain is
connected with a dense network of nerves to the rest of the
body’s sensors and actors. There are approximately 10¹¹
neurons in the brain, and these are the building blocks of the
complete central nervous system of the living body.
• The neuron is the fundamental building block of neural
networks. In the biological systems, a neuron is a cell just
like any other cell of the body, which has a DNA code and
is generated in the same way as the other cells. Though it
might have different DNA, the function is similar in all the
organisms. A neuron comprises three major parts: the cell
body (also called Soma), the dendrites, and the axon. The
dendrites are like fibers branched in different directions and
are connected to many cells in that cluster.
• Dendrites receive the signals from
surrounding neurons, and the axon transmits
the signal to the other neurons. At the ending
terminal of the axon, the contact with the
dendrite is made through a synapse. Axon is
a long fiber that transports the output signal
as electric impulses along its length. Each
neuron has one axon. Axons pass impulses
from one neuron to another like a domino
effect.
 Why Understand Biological Neural Networks?
• For creating mathematical models for artificial neural networks,
theoretical analysis of biological neural networks is essential as they
have a very close relationship. And this understanding of the brain’s
neural networks has opened horizons for the development of artificial
neural network systems and adaptive systems designed to learn and
adapt to the situations and inputs.
• ANN versus BNN
• Before taking a look at the differences between Artificial Neural
Network ANN and Biological Neural Network BNN, let us take a look
at the similarities based on the terminology between these two.
 • Model of Artificial Neural Network
• The following diagram represents the general model of
ANN followed by its processing.

For the above general model of artificial neural network, the net input can be calculated as follows −

Net input yin=∑mixi.wi

• Processing of ANN depends upon the following three
building blocks −
• Network Topology
• Adjustments of Weights or Learning
• Activation Functions
• Network Topology
• A network topology is the arrangement of a network along
with its nodes and connecting lines. According to the
topology, ANN can be classified as the following kinds −
• Feedforward Network
• It is a non-recurrent network having processing units/nodes
in layers and all the nodes in a layer are connected with the
nodes of the previous layers. The connection has different
weights upon them. There is no feedback loop means the
signal can only flow in one direction, from input to output. It
may be divided into the following two types −
• Single layer feedforward network − The concept is of
feedforward ANN having only one weighted layer. In other words, we can
say the input layer is fully connected to the output layer.

•Multilayer feed forward network − The concept is of feedforward ANN

having more than one weighted layer. As this network has one or more
layers between the input and the output layer, it is called hidden layers.
• Feedback Network
• As the name suggests, a feedback network has feedback paths,
which means the signal can flow in both directions using loops.
This makes it a non-linear dynamic system, which changes
continuously until it reaches a state of equilibrium. It may be
divided into the following types −
• Recurrent networks − They are feedback networks with closed
loops. Following are the two types of recurrent networks.
• Fully recurrent network − It is the simplest neural network
architecture because all nodes are connected to all other nodes
and each node works as both input and output.
• Jordan network − It is a closed loop network in which the
output will go to the input again as feedback as shown in the
following diagram.

Adjustments of Weights or Learning

Learning, in artificial neural network, is the method of
modifying the weights of connections between the neurons of a
specified network. Learning in ANN can be classified into three
categories namely supervised learning, unsupervised learning,
and reinforcement learning.
• Supervised Learning
• As the name suggests, this type of learning is done under the
supervision of a teacher. This learning process is dependent.
• During the training of ANN under supervised learning, the input vector
is presented to the network, which will give an output vector. This
output vector is compared with the desired output vector. An error
signal is generated, if there is a difference between the actual output and
the desired output vector. On the basis of this error signal, the weights
are adjusted until the actual output is matched with the desired output.
• Unsupervised Learning
• As the name suggests, this type of learning is done without the supervision of
a teacher. This learning process is independent.
• During the training of ANN under unsupervised learning, the input vectors of
similar type are combined to form clusters. When a new input pattern is
applied, then the neural network gives an output response indicating the class
to which the input pattern belongs.
• There is no feedback from the environment as to what should be the desired
output and if it is correct or incorrect. Hence, in this type of learning, the
network itself must discover the patterns and features from the input data, and
the relation for the input data over the output.
• Reinforcement Learning
• As the name suggests, this type of learning is used to reinforce or strengthen
the network over some critic information. This learning process is similar to
supervised learning, however we might have very less information.
• During the training of network under reinforcement learning, the network
receives some feedback from the environment. This makes it somewhat
similar to supervised learning. However, the feedback obtained here is
evaluative not instructive, which means there is no teacher as in supervised
learning. After receiving the feedback, the network performs adjustments of
the weights to get better critic information in future.
• Activation Functions
• It may be defined as the extra force or effort applied over the input to obtain
an exact output. In ANN, we can also apply activation functions over the
input to get the exact output. Followings are some activation functions of
interest −
• Linear Activation Function
• It is also called the identity function as it performs no input editing. It can be
defined as −

Sigmoid Activation Function

It is of two type as follows −
•Binary sigmoidal function − This activation function performs input editing
between 0 and 1. It is positive in nature. It is always bounded, which means its
output cannot be less than 0 and more than 1. It is also strictly increasing in
nature, which means more the input higher would be the output. It can be defined
as
 Concept of Learning/Training model
• Training a model simply means learning (determining) good
values for all the weights and the bias from labeled examples.
In supervised learning, a machine learning algorithm builds a
model by examining many examples and attempting to find a
model that minimizes loss; this process is called empirical
risk minimization.
• Loss is the penalty for a bad prediction. That is, loss is a
number indicating how bad the model's prediction was on a
single example. If the model's prediction is perfect, the loss is
zero; otherwise, the loss is greater. The goal of training a
model is to find a set of weights and biases that have low loss,
on average, across all examples. For example, Figure 3 shows
a high loss model on the left and a low loss model on the right.
Note the following about the figure:
• The arrows represent loss.
• The blue lines represent predictions.
 High loss in the left model; low loss in the right model.

Notice that the arrows in the left plot are much longer than their counterparts in the
right plot. Clearly, the line in the right plot is a much better predictive model than the line
in the left plot.
You might be wondering whether you could create a mathematical function—a loss
function—that would aggregate the individual losses in a meaningful fashion.
• Squared loss: a popular loss function
• The linear regression models we'll examine here use a loss function
called squared loss (also known as L2 loss). The squared loss for a single
example is as follows:
= the square of the difference between the label and the prediction
= (observation - prediction(x))2
= (y - y')2
Mean square error (MSE) is the average squared loss per example over
the whole dataset. To calculate MSE, sum up all the squared losses for
individual examples and then divide by the number of examples:
 Parametric Models, Nonparametric Models

• Machine learning models can be parametric or

non-parametric. Parametric models are those
that require the specification of some
parameters before they can be used to make
predictions, while non-parametric models do
not rely on any specific parameter settings and
therefore often produce more accurate results.
This blog post discusses parametric vs non-
parametric machine learning models with
examples along with the key differences.
 What are parametric and non-
parametric models?
• Training machine learning models is about
finding a function approximation built using input
or predictor variables, and whose output
represents the response variable. The reason why
it is called function approximation is because
there is always an error in relation to the value of
function output vs actual or real-world value.
And, an aspect of this error is reducible in the
sense that further features / technique can be used
to improve upon the accuracy. Another aspect of
this error is irreducible which represents the
random error which can’t be dealt with.
• When estimating the function (called function approximation), the
following two steps are followed:
• Identifying the function: The first step is to identify the function such as
linear or non-linear function.
• Identifying the parameters of the function in case this is linear function.
• In case, the function identified is linear function (model), the training or
fitting of the machine learning models boils down to estimating the
parameters. Here is an example of the linear model also called as linear
regression model and related parameters (coefficients).

Such models are called as parametric machine learning models. The parametric
models are linear models which includes determining the parameters such as
that shown above. The most common approach to fitting the above model is
referred to as ordinary least squares (OLS) method. However, least squares is
one of many possible ways to fit the linear model. Example of parametric
models include linear algorithms such as Lasso regression, linear regression
and to an extent, generalized additive models (GAMs).
• Building non-parametric models do not make explicit
assumptions about the functional form such as linear model in
case of parametric models. Instead non-parametric models
can be seen as the function approximation that gets as close
to the data points as possible. The advantage over parametric
approaches is that by avoiding the assumption of a particular
functional form such as linear model, non-parametric models
have the potential to accurately fit a wider range of possible
shapes for the actual or true function. Any parametric approach
brings with it the possibility that the functional form (linear
model) which is very different from the true function, in which
case the resulting model will not fit the data well. Example of
non-parametric models include fully non-linear algorithms
such as bagging, boosting, support vector machines bagging
boosting with non-linear kernels, and neural networks (deep
learning).
• The following is the list of differences between parametric and
non-parametric machine learning models.
• In case of parametric models, the assumption related to the
functional form is made and linear model is considered. In case of
non-parametric models, the assumption about the functional form is
not made.
• Parametric models are much easier to fit than non-parametric
models because parametric machine learning models only require
the estimation of a set of parameters as the model is identified prior
as linear model. In case of non-parametric model, one needs to
estimate some arbitrary function which is a much difficult task.
• Parametric models often do not match the unknown function we are
trying to estimate. The model performance is comparatively lower
than the non-parametric models. The estimates done by the
parametric models will be farther from being true.
• Parametric models are interpretable unlike the non-parametric
models. This essentially means that one can go for parametric
models when the goal is to find inference. Instead, one can choose to
go for non-parametric models when the goal is to make prediction
with higher accuracy and interpretability or inference is not the key
ask.
Multilayer Networks: Feed Forward network, Feedback
network
• Feed-forward network
• Feed-forward neural networks enable signals to travel one method only,
from input to output. There is no feedback (loops) i.e., the output of any
layer does not affect that same layer. Feed-forward networks influence
to be easy networks that relate inputs with outputs. They are
extensively used in pattern recognition. This type of organization is also
defined as bottom-up or top-down.
• Feed-forward neural networks enable signals to travel one method only,
from input to output. There is no feedback (loops) i.e., the output of any
layer does not affect that same layer. Feed-forward networks influence
to be easy networks that relate inputs with outputs. They are
extensively used in pattern recognition. This type of organization is also
defined as bottom-up or top-down.
• The weighted outputs of these units are fed simultaneously to the
second layer of neurons like units known as the hidden layer. The
hidden layer is weighted output which can be input to another hidden
layer and so on. The number of hidden layers is arbitrary and usually,
one is used.
• The weighted outputs of the last hidden layer are inputs to units
making up the output layer, which emits the network's prediction for
given samples. The units in the hidden layers and output layer are
defined as neurodes, because of their symbolic biological basis or as
output units. Multilayer feed-forward networks of linear threshold
functions given through hidden units can closely approximate any
function.
• Feedback Networks
• Feedback networks can have signals traveling in both areas by
learning loops on the web. Feedback networks are very dynamic and
can get extremely complex. Feedback networks are dynamic, their
states are changing continuously until they reach an equilibrium
point.
• The remains at the equilibrium point until the input changes and a
new equilibrium needs to be found. Feedback architectures are also
defined as interactive or recurrent, although the term can indicate
feedback connections in individual-layer organizations.
• When a large database is involved in increasing the accuracy of deep
neural network algorithms, a model of data production and artificial
intelligence learning for behavioral research is essential. In general,
clinical data are used when the user's disease information is
included. At this time, if the clinical data are inaccurate, the results
of the predictions are incorrect.
• When a large database is involved in increasing the accuracy of deep
neural network algorithms, a model of data production and artificial
intelligence learning for behavioral research is essential. In general,
clinical data are used when the user's disease information is
included. At this time, if the clinical data are inaccurate, the results
of the predictions are incorrect.
• Moreover, if the information about the user's behavior and activity,
other than the clinical data is not reflected, time-series data
according to the user’s situation which changes over time, must be
used as an input value to accurately predict the results.
• The feedback model for the deep neural network algorithm includes
an original feedback model and a secondary feedback model that
restate the result.