A

Seminar Report
On

Artificial Neural Network

Submitted in partial fulfilment of requirements for award of the
degree
Bachelor of Technology
In
Computer Science & Engineering

Submitted by:

Under the Supervision of

DEPARTMENT OF COMPUTER SCIENCE &

ENGINEERING
VIGNAN INSTITUTE OF TECHNOLOGY
AND MANAGEMENT BRAHMAPUR

ARTIFICIAL NEURAL NETWORK

1. INTRODUCTION TO NEURAL NETWORK :

If there is one area in data science that has led to the growth of Machine Learning and
Artificial Intelligence in the last few years, it is Deep Learning. From research labs in
universities with low success in the industry to powering every smart device on the planet –
Deep Learning and Neural Networks have started a revolution.

Neural Networks reflect the behaviour of the human brain, allowing computer programs to
recognize patterns and solve common problems in the fields of AI, machine learning and
deep learning.

2. DEEP LEARNING:
Deep Learning is a subfield of machine learning concerned with algorithms inspired by
the structure and function of the brain called artificial neural networks.
It was artificial neural network to perform sophisticated computations on large amounts of
data.

Deep Learning algorithms train machines by learning from examples, Industries such as
health care, e-Commerce, entertainment and advertising commonly use deep learning.

3.
4. WHAT IS ARTIFICIAL NEURAL NETWORK ?
3.1.Introduction:
Artificial neural networks are relatively crude electronic models based on the
neural structure of the brain. The brain basically learns from experience. It is natural proof
that some problems that are beyond the scope of current computers are indeed solvable by
small energy efficient packages. This brain modeling also promises a less technical way to
develop machine solutions. This new approach to computing also provides a more graceful
degradation during system overload than its more traditional counterparts.

These biologically inspired methods of computing are thought to be the next major
advancement in the computing industry. Even simple animal brains are capable of functions
that are currently impossible for computers. Computers do rote things well, like keeping
ledgers or performing complex math. But computers have trouble recognizing even simple
patterns much less generalizing those patterns of the past into actions of the future.

Artificial neutral network is an interconnected group of nodes, inspired by a simplification

of neurons in a brain .

3.2. HISTORICAL BACKGROUND

1|Page
 Neural Network simulations appear to be a recent development. However this field was
established before the advent of computers , and has survived at least one major setback and
several areas .
During this period when funding and professional support was minimal , relatively few
researchers made important advances. The first artificial neuron was produced in 1943 by the
neurophysiologist Warren McCulloch and the logician Walter Pits. But the technology
available at the time did not allow them to do too much.
There were some initial simulations using formal logic. McCulloch and Pitts (1943)
developed models of neural networks based on their understanding of neurology. These
models made several assumptions about how neurons worked. Their networks were based on
simple neurons, which were considered to be binary devices with fixed thresholds. The
results of their model were simple logic functions such as "a or b"and "a and b". Another
attempt was by using computer simulations. Two groups (Farley and Clark, 1954; Rochester,
Holland, Haibit and Duda, 1956). The first group (IBMresearchers) maintained closed contact
with neuroscientists at McGill University. So whenever their models did not work, they
consulted the neuroscientists. This interaction established a multidisciplinary trend, which
continues to the present day .
Significant progress has been made in the field of neural networks-enough to attract a great
deal of attention and fund further research. Advancement beyond current commercial
applications appears to be possible, and research is advancing the field on many fronts.
Neutrally based chips are emerging and applications to complex problems developing.
Clearly , today is a period of transition for neural network technology .

3.3.WHAT IS BIOLOGICAL NEURONS ?

Neural networks grew out research in artificial intelligence; specifically, attempts to mimic
the fault-tolerance and capacity to learn of biological neural systems by modelling the low-
level structure of brain.
In the human brain the axons of one cell connect to the dendrites of another via a synapse.
When a neuron is activated, it fires an electrochemical signal along the axon. This signal
crosses the synapse to other neurons, which may in turn fire.
A neuron fires only if the total signal received at the cell body from the dendrites exceeds
a certain level (threshold).

2|Page
 Fig:
Biological Neuron

4. ARCHITECTURE OF ARTIFICIAL NEURAL NETWORK:-

Artificial Neural Networks are made up of layers and layers of connected input units and
output units called neurons. A single layer neural network is called a Perceptron. Multiple
hidden layers may also be present in an artificial neural network. The input units(receptor),
connection weights, summing function, computation and output units (effectors) are what
makes up an artificial neuron.

Even though neurons are slower than silicon logic gates, their massive interconnection makes
up for the slow rate. The weight value of a connection is the strength of the specified
connection between neurons. Weights are randomly initialized and adjusted via an
optimization algorithm to map aggregations of input stimuli to a desired output function.

The architecture of a neural network are of many types which include: Perceptron, Feed
Forward Neural Network, Multilayer Perceptron, Convulational Neural Network, Radial
Basis Function Neural Network, Recurrent Neural Network, LSTM –Long Short-Term
Memory, Sequence to Sequence models, Modular Neural Network. Learning algorithms
could be supervised, unsupervised or reinforcement methods.

Let us take a look at the different kinds of layers available in an artificial neural network:
3|Page
Input layer:

The Input layers contain those artificial neurons (termed as units) which are to receive input
from the outside world. This is where the actual learning on the network happens, or
recognition happens else it will process.

Output layer:

The output layers contain units that respond to the information that is fed into the system and
also whether it learned any task or not.

Hidden layer:

The hidden layers are mentioned hidden in between input layers and the output layers. The
only job of a hidden layer is to transform the input into something meaningful that the output
layer/unit can use in some way.

Most of the artificial neural networks are all interconnected, which means that each of the
hidden layers is individually connected to the neurons in its input layer and also to its output
layer leaving nothing to hang in the air. This makes it possible for a complete learning
process and also learning occurs to the maximum when the weights inside the artificial neural
network get updated after each iteration.

4.1.WORKING OF ARTIFICIAL NEURAL NETWORK:-

Artificial Neural Network can be best represented as a weighted directed graph, where the
artificial neurons form the nodes. The association between the neurons outputs and neuron
inputs can be viewed as the directed edges with weights. The Artificial Neural Network
receives the input signal from the external source in the form of a pattern and image in the
form of a vector. These inputs are then mathematically assigned by the notations x (n) for
every n number of inputs.

Afterward, each of the input is multiplied by its corresponding weights (these weights are
the details utilized by the artificial neural networks to solve a specific problem). In general
terms, these weights normally represent the strength of the interconnection between neurons
inside the artificial neural network. All the weighted inputs are summarized inside the

computing unit.

4|Page
 Fig:- WORKING OF ARTIFICIAL NEURAL NETWORK

If the weighted sum is equal to zero, then bias is added to make the output non-zero or
something else to scale up to the system's response. Bias has the same input, and weight
equals to 1. Here the total of weighted inputs can be in the range of 0 to positive infinity.
Here, to keep the response in the limits of the desired value, a certain maximum value is
benchmarked, and the total of weighted inputs is passed through the activation function.

The activation function refers to the set of transfer functions used to achieve the desired
output. There is a different kind of the activation function, but primarily either linear or non-
linear sets of functions. Some of the commonly used sets of activation functions are the
Binary, linear, and Tan hyperbolic sigmoid activation functions. Let us take a look at each of
them in details:

Binary:-

In binary activation function, the output is either a one or a 0. Here, to accomplish this, there
is a threshold value set up. If the net weighted input of neurons is more than 1, then the final

output of the activation function is returned as one or else the output is returned as 0.

Sigmoid Hyperbolic:

The Sigmoid Hyperbola function is generally seen as an "S" shaped curve. Here the tan
hyperbolic function is used to approximate output from the actual net input. The function is

defined as:

F(x) = (1/1 + exp (-????x))

5|Page
Where???? Is considered the Steepness parameter

Training Algorithms for Artificial Neural Networks:-

 Gradient Descent Algorithm

This is the simplest training algorithm used in the case of a supervised training model. In case
the actual output is different from the target output, the difference or error is find out. The
gradient descent algorithm changes the weights of the network in such a manner to minimize
this mistake.

 Back Propagation Algorithm

It is an extension of the gradient-based delta learning rule. Here, after finding an error (the
difference between desired and target), the error is propagated backward from the output
layer to the input layer via the hidden layer. It is used in the case of Multilayer Neural
Network.

5. TYPES OF ANN:-

1. Feed forward Neural Network

Feed forward neural network or deep feed forward network is one of the simplest types of
artificial neural networks. In this ANN, data goes through several input nodes (computational
units also known as artificial neurons) until it arrives at an output node.

Simply put, information passes through in a single direction from an entry point or input
node to an exit point or output node. It differs from other more complex ANN types in
that it does not have feedback connections where the output from each layer of neurons is fed
back to it for learning. A fe

edforward neural network does not form a cycle or loop to allow a program to learn.

Applications: Feed forward neural networks are used in computer vision and facial

recognition applications.

2. Radial Basis Function Neural Network (RBFNN)

This type of ANN only has three layers—the input layer, the hidden layer, and the output
layer. It is limited to a single hidden layer compared with other ANN types. The hidden layer

6|Page
is basically hidden in between input and output layers, and it reduces redundancies in data.
Compared with other ANN types that can have several hidden layers, learning is faster in an
RBFNN. 

To further understand what the different ANN layers are for, imagine that you want to inform
your computer that the picture it is shown depicts a car. For the computer to understand, it
needs separate tools (or layers). Your car detector can thus have a wheel detector so it can tell
something has wheels. It should have a vehicle body detector, which could allow it to
differentiate a car from a truck, and a size detector so it can do likewise. These are just some
elements that make up hidden layers in artificial neural networks. They do not present the
entire image but are parts of it.

Applications: RBFNNs can be used in complex power restoration systems. In case of a

blackout, they can be used to restore electrical power to normal conditions with minimal
losses and less societal impact. They can also be extensively applied for time-series
prediction. An example would be in stock trading, where computers predict what stocks are
likely to increase or decrease in value, allowing users to invest wisely.

3. Recurrent Neural Network (RNN)

This type of ANN is similar to a feedforward neural network, but it saves the output of a
specific layer and feeds it back as input. As a result, it can help predict several possible
outcomes from any particular layer.

Drilling down to specifics: If the first layer receives the output and sends it back as input, the
next layers will start the recurrent neural network process. Each node retains a memory from
the previous step. And so, the system remembers wrong predictions and learns from them to
improve its next ones. In short, RNNs can learn from each step to predict the outcome in the
next step.

Applications: RNNs are used in text-to-speech applications that predict what users may want
to say next, depending on the context of their initial input.

4. Multilayer Perceptron (MLP)

This type of ANN has three or more layers that classify data that cannot be
linearly separated (i.e., go through a straight path). Thus, it is fully connected,

7|Page
which means that each node within a layer is connected to the succeeding node
in the next layer.

Applications: MLPs aid in speech recognition and machine translation technologies.

5. Convulational Neural Network (CNN)

This type of ANN applies a different version of MLPs by having several layers that can be
completely interconnected. The primary purpose of CNNs is to decipher specific features of a
given image such as a face. They identify features based on how near or far each pixel (which
makes up every facial feature) is from a reference point.

Applications: CNNs are widely used for accurate face detection even if the input image is of
low resolution. They are also particularly useful for improving a self-driving car’s estimation
of its driving field since they are very good at determining distances. Other applications
include natural language processing (NLP), paraphrase detection, and image classification.

The types of artificial neural networks above use different methods to achieve a desired
outcome. However, all of them work in a way that resembles how neurons in our brains
work. Like the neurons in the human brain, ANNs learn more and improve their functions
every time they receive more data and are used more often. And just like our brain, which it
mimics, its applications can also be limitless.

6. APPLICATION OF ANN:-

Since artificial neural networks allow modeling of nonlinear processes, they have turned into a
very popular and useful tool for solving many problems such as classification, clustering,
regression, pattern recognition, dimension reduction, structured prediction, machine
translation, anomaly detection, decision making, visualization, computer vision, and others.
This wide range of abilities makes it possible to use artificial neural networks in many areas.
In this article, we discuss applications of artificial neural networks in Natural Language
Processing tasks (NLP).

8|Page
1. Text Classification and Categorization

Text classification is an essential part in many applications, such as web searching,

information filtering, language identification, readability assessment, and sentiment analysis.
Neural networks are actively used for these tasks.

2. Named Entity Recognition (NER)

The main task of named entity recognition (NER) is to classify named entities, such as Guido
van Rossum, Microsoft, London, etc., into predefined categories like persons, organizations,
locations, time, dates, and so on. Many NER systems were already created, and the best of
them use neural networks.

3. Part-of-Speech Tagging

Part-of-speech tagging  has many applications including parsing, text-to-speech conversion,

information extraction, and so on. In the work, part of speech tagging Bidirectional Long
Short-Term Memory Recurrent Neural Network a recurrent neural network with word
embedding for part-of-speech (POS) tagging task is presented .The model was tested on
the Wall Street Journal data from Penn Treebank III data set and achieved a performance of
97.40% tagging accuracy.

4. Semantic Parsing and Question Answering

Question Answering systems automatically answer different types of questions asked in

natural languages including definition questions, biographical questions, multilingual
questions, and so on. Neural networks usage makes it possible to develop high performing
question answering systems.

5. Paraphrase Detection

Paraphrase detection determines whether two sentences have the same meaning. This task is
especially important for question answering systems since there are many ways to ask the
same question.

9|Page
6. Language Generation and Multi-document Summarization

Natural language generation has many applications such as automated writing of reports,
generating texts based on analysis of retail sales data, summarizing electronic medical records,
producing textual weather forecasts from weather data, and even producing jokes.

7. Machine Translation

Machine translation software is used around the world despite its limitations. In some
domains, the quality of translation is not good. To improve the results researchers try different
techniques and models, including the neural network approach. The purpose of Neural-based
Machine Translation for Medical Text Domain study is to inspect the effects of different
training methods on a Polish-English machine translation system used for medical data. To
train neural and statistical network-based translation systems The European Medicines
Agency parallel text corpus was used. It was demonstrated that a neural network requires
fewer resources for training and maintenance. In addition, a neural network often substituted
words with other words occurring in a similar context .

8. Speech Recognition
Speech recognition has many applications, such as home automation, mobile telephony,
virtual assistance, hands-free computing, video games, and so on. Neutral networks are widely
used in this area.

9. Character Recognition

Character Recognition systems also have numerous applications like receipt character
recognition, invoice character recognition, check character recognition, legal billing document
character recognition, and so on. The article Character Recognition Using Neural
Network presents a method for the recognition of handwritten characters.

10. Spell Checking

Most text editors let users check if their text contains spelling mistakes. Neural networks are
now incorporated into many spell-checking tools.

10 | P a g e
7.ADVANTAGES AND DISADVANTAGES:-

Advantages of Artificial Neural Networks (ANN)

1. Problems in ANN are represented by attribute-value pairs.

2. ANNs are used for problems having the target function, the output may be
discrete-valued, real-valued, or a vector of several real or discrete-valued
attributes.
3. ANN learning methods are quite robust to noise in the training data. The training
examples may contain errors, which do not affect the final output.
4. It is used where the fast evaluation of the learned target function required.
5. ANNs can bear long training times depending on factors such as the number of
weights in the network, the number of training examples considered, and the
settings of various learning algorithm parameters.

Disadvantages of Artificial Neural Networks (ANN)

1. Hardware Dependence:
1. Artificial Neural Networks require processors with parallel processing
power, by their structure.
2. For this reason, the realization of the equipment is dependent.
2. Unexplained functioning of the network:
1. This most important problem of ANN.
2. When ANN gives a probing solution, it does not give a clue as to why and
how.
3. This reduces trust in the network.
3. Assurance of proper network structure:
1. There is no specific rule for determining the structure of artificial neural
networks.
2. The appropriate network structure is achieved through experience and trial
and error.
4. The difficulty of showing the problem to the network:
1. ANNs can work with numerical information.
2. Problems have to be translated into numerical values before being
introduced to ANN.
3. The display mechanism to be determined will directly influence the
performance of the network.
4. This is dependent on the user's ability.
5. The duration of the network is unknown:
1. The network is reduced to a certain value of the error on the sample
means that the training has been completed.
2. The value does not give us optimum results.

11 | P a g e
8. CONCLUSION:-

Artificial Neural Networks (ANNs) are the main drivers of the contemporary AI revolution. Inspired by the

biological structure of human brain, ANNs are powerful in modelling functions and hypotheses which would

be hard to derive intuitively or logically. Instead of inventing your own function with high-order polynomials,

which may lead to over fitting, one can design an efficient ANN architecture that can automatically fit

complex nonlinear hypotheses to data. This advantage of the ANNs has been leveraged in the algorithmic

feature extraction in computer vision and image recognition. For example, instead of manually specifying a

finite list of image features to choose from, we can design a Convulational Neural Network (CNN) that uses

the same principle as the animal’s visual cortex to extract features. As a human eye, layers of the CNN

respond to stimuli only in a restricted region of the visual field. This allows the network to recognize low-

level features such as points, edges, or corners and gradually merge them into high-level geometric figures

and objects. This example illustrates how good ANNs are in the automatic derivation of hypotheses and

models from complex data that includes numerous associations and relationships.

9. REFRENCES:-

1. An introduction to neural computing. Aleksander, I. and Morton, H. 2nd edition

2. Neural Networks at Pacific Northwest National Laboratory

http://www.emsl.pnl.gov:2080/docs/cie/neural/neural.homepage.html

3. Industrial Applications of Neural Networks (research reports Esprit, I.F.Croall, J.P.Mason)

4. A Novel Approach to Modelling and Diagnosing the Cardiovascular System

http://www.emsl.pnl.gov:2080/docs/cie/neural/papers2/keller.wcnn95.abs.html

5. Artificial Neural Networks in Medicine

http://www.emsl.pnl.gov:2080/docs/cie/techbrief/NN.techbrief.ht

6. Neural Networks by Eric Davalo and Patrick Naim

7. Learning internal representations by error propagation by Rumelhart, Hinton and Williams

(1986).

8. Klimasauskas, CC. (1989). The 1989 Neuro Computing Bibliography. Hammerstrom, D.

(1986). A Connectionist/Neural Network Bibliography.

9. DARPA Neural Network Study (October, 1987-February, 1989). MIT Lincoln Lab. Neural
Networks, Eric Davalo and Patrick Naim

12 | P a g e
10. Assimov, I (1984, 1950), Robot, Ballatine, New York.

11. Electronic Noses for Telemedicine

http://www.emsl.pnl.gov:2080/docs/cie/neural/papers2/keller.ccc95.abs.html

12. Pattern Recognition of Pathology Images http://kopernik-

eth.npac.syr.edu:1200/Task4/pattern.html

13 | P a g e