Introduction to Artificial

Neural Networks
Basic concepts
• Artificial neural network (ANN) is:
popular machine learning technique that simulate the mechanism of learning in
human organisms.
• Biological neural networks V.S Artificial neural networks:
• In Biological neural networks:
- Cell body is called soma
- All cells are connected to one another by axons and dendrites
• In Artificial neural networks:
- The computation units are referred as neurons
- The computation units are connected to one another by weights
- Each input to a neuron is scaled (multiplied) with a weight
Biological Neural Networks
• A biological neuron has three types
of main components;
dendrites, soma (or cell body) and axon.

• Dendrites receives signals from

other neurons.

• The soma, sums the incoming signals. When sufficient input is

received, the cell fires; that it transmits a signal over its axon to other
cells
 Artificial Neurons

• From experience:
examples / training data
• Strength of connection
between the neurons is
stored as a weight-value
for the specific
connection.
• Learning the solution to a
problem = changing the
connection weights (Wij)
 Artificial Neural Network
The work mechanism in ANN can be summarized as follows:
- ANN computes a function of the inputs by propagating the computed
values from the input neurons to the output neurons and using the
weights as intermediate parameters.
- Learning occurs by changing the weights connecting the neurons
- the stimulus is provided by training data containing examples of
input-output pairs of the function to be learned.
- The training data are fed into neural networks by using the input
representation.
Cont.
- The training data provide a feedback to the correctness of the weights
depending on how well the predicted output matches the annotated
output.
- Weights will be changed (adjusted) to modify the computed function
to make the prediction more correct in the future iterations.
- The function computed by neural network is refined over time so that
it provides more accurate prediction
Model generalization
Model generalization:
In the real applications, the performance of Artificial Neural Networks (ANN) is
mostly depends upon its generalization capability.
Generalization is “the ability to accurately handle unseen data by training over a
finite set of input-output pairs”.
This is (the primary usefulness of machine learning models)
• The most basic units in computations in neural networks are inspired by
traditional machine learning algo. (Such as: least-squares and logistic regression).
• The power of NN is gained by connecting these elementary units in particular
ways.
Applications of ANN
ANNs are widely used in many applications such as:
• Signal processing
• Pattern recognition, e.g. handwritten characters or face identification.
• Diagnosis of medical cases.
• Speech recognition
• Human Emotion Detection
• Weather forecasting
• Stock market prediction
• Controlling and others..
Advantages Of ANN
• Non-linearity
It can model non-linear systems
• Input-output mapping
It can derive a relationship between a set of input & output
responses
• Adaptively
The ability to learn allows the network to adapt to changes in
the surrounding environment
• Evidential response
It can provide a confidence level to a given solution
Cont.
• Contextual information
Knowledge is presented by the structure of the network. Every neuron in the
network is potentially affected by the global activity of all other neurons in the
network. Consequently, contextual information is dealt with naturally in the
network.

• Fault tolerance
Distributed nature of the NN gives it fault tolerant capabilities

• Neurobiology analogy
Models the architecture of the brain
Characterization
• Architecture
• a pattern of connections between neurons
• Single Layer Feedforward
• Multilayer Feedforward
• Recurrent
• Strategy / Learning Algorithm
• a method of determining the connection weights
• Supervised
• Unsupervised
• Reinforcement
• Activation Function
• Function to compute output signal from input signal
Types of Architecture

 Feed forward Neural Networks:

– Connections are in one direction only
– Partial biological justification

 Recurrent Neural Network:

- Connections between nodes can create a cycle
- Complex behaviour
Single Layer Feedforward NN
Multilayer Neural Network
Recurrent Neural Network
Types of layers in ANN
• The input layer
– Introduces input values into the network.
– No activation function or other processing.
• The hidden layer(s).
– Perform classification of features
– Two hidden layers are sufficient to solve any problem
– Mostly , more layers perform more better
• The output layer
– Functionally just like the hidden layers
– Outputs are passed on to the world outside the neural network.
Strategy / Learning Algorithm
• Supervised Learning
• Learning is performed by presenting pattern with target
• During learning, produced output is compared with the desired output
• The difference between both output is used to modify learning weights according
to the learning algorithm
• Applications : Recognizing hand-written digits, pattern recognition and etc.
• Network models: perceptron, feed-forward, radial basis function, SVM, etc.

• Unsupervised Learning
• Targets (labels) are not provided
• Appropriate for clustering task
• Applications : Find similar groups of documents in the web, clustering,..
• Network models: Kohonen, self organizing maps, Hopfield networks etc.
Cont.
• Reinforcement Learning
• Target is provided, but the desired output is absent.
• The net is only provided with guidance to determine the produced output is
correct or vise versa.
• Weights are modified in the units that have errors
• Applications: natural language processing, robotics, traffic light control,
autonomous cars, …