ABAF Assignment 1

BY GROUP 5
ANKIT JAIN PGP08008
R A J G U P TA PGP08042
VA I B H AV G A N G WA R PGP08194
V A R U N PA R V AT H A N E N I PGP08195
 Feed Forward Neural Networks
The input travels only in one direction
There is no back propagation
It may or may not contain hidden layers
The sum of the products of inputs and weights is fed to the output

Advantages:
• Easy Design
• Responsive to noisy data
• Easy to maintain
• Ability to work with incomplete knowledge
Business Applications:
1. Validation of signatures using bank cheques 2. Speech recognition

Data Analytics Objective Data Data Analytics Objective Data

To recognize The handwritten Audio recordings of a
handwritten characters character in a bitmap To identify people
particular vowel is used
and validate signatures pattern is used as the suffering from
as the input to train the
on bank cheques to input and the correct Parkinson’s disease using
Feed Forward neural
prevent frauds letter as the output speech recognition
network with affected
patients as the output
 Radial basis function Neural Network
Contains 3 layers
◦ Input layer
◦ Hidden layer
◦ Output layer
Radial basis function Neural Network uses radial functions as activation functions
Hidden units are known as radial centres which transforms the input space to hidden space
Gaussian activation function is commonly used
The Euclidean distance between the input vector and centre of that unit

Advantages :
• Easy Design
• Good generalization
• Strong tolerance to input noise
• Online learning ability
Business Applications:
1. EEG Analysis 2. Real time pressure prediction in Diesel Engines

Data Analytics Objective Data Data Analytics Objective Data

To evaluate epileptic The neural network is Measured in-cylinder
patients and classify trained using To predict in-cylinder
pressure signals and
epilepsy groups by using parameters of EEG pressure in Diesel
time series from the
RBFN signals such as Engines with less
accelerometer are the
Weighted Histogram response time and high
data used to train the
filter, Zero level flexibility
Neural network
saturation
Kohonen Network
Pattern Recognition
◦ It is a breed of neural network designed to group similar inputs together by having them represented by
nearby neurons in a neural network
◦ Each neuron represents a particular input feature vector , and domain of inputs are recognized across
the field of neurons
◦ Each vector is displayed as an RGD triplet cover
Advantages of Kohonen network
◦ Non-Linearity
◦ Input & output mapping
◦ Adaptivity
◦ Evidential response
◦ Contextual information
◦ Fault tolerance
Applications of Kohonen Network
Recognition & Identification
◦ Medical diagnosis
◦ Face recognition
Assessment
◦ Data Mining
Monitoring & Control
◦ E-mail spam filtering
◦ Process control
◦ Vehicle control
Forecasting & Prediction
◦ Financial Applications
Kohonen Learning
A variety of nets use Kohonen Learning
◦ New weight vector is the linear combination of old weight vector and the current input vector.
◦ The weight update for cluster unit (output unit) j can be calculated as:

◦ the learning rate alpha decreases as the learning process proceeds.

 Business application of Kohonen - SOM
Since it is unsupervised environment, so the name is Self Organizing Maps.
Self Organizing NNs are also called Topology Preserving Maps which leads to the idea of neighborhood of the clustering unit.
During the self-organizing process, the weight vectors of winning unit and its neighbors are updated.
Normally, Euclidean distance measure is used to find the cluster unit whose weight vector matches most closely to the input
vector.
For a linear array of cluster units, the neighborhood of radius R around cluster unit J consists of all units j such that:

Data Analytics Objective

◦ To find the shortest route to be travelled to cover all nodes in the most efficient way.

Data
◦ The data is a set of vectors with its x axis and y axis coordinates which are fed into the neural network being used to
compute the shortest path.
Recurrent Neural Network
A recurrent neural network (RNN) is a class of artificial neural network where connections
between nodes form a directed graph along a sequence. This allows it to exhibit temporal
dynamic behaviour for a time sequence.

The core reason that recurrent nets are more exciting is that they allow us to operate over
sequences of vectors: Sequences in the input, the output, or in the most general case both.
Applications of RNN
Applications of Recurrent Neural Networks include:
◦ Machine Translation
◦ Robot control
◦ Time series prediction
◦ Speech recognition
◦ Time series anomaly detection
◦ Rhythm learning
◦ Music composition
◦ Grammar learning
◦ Handwriting recognition
◦ Human action recognition
Business Application of RNN
Character Level Language Models
◦ Data Analytics Objective
◦ We’ll train RNN character-level language models. That is, we’ll give the RNN a huge chunk of text and
ask it to model the probability distribution of the next character in the sequence given a sequence of
previous characters. This will then allow us to generate new text one character at a time.
◦ Data
◦ At test time, we feed a character into the RNN and get a distribution over what characters are likely to
come next. We sample from this distribution, and feed it right back in to get the next letter. Repeat
this process and you’re sampling text!
 Modular Neural Network
Composed of more than one neural network model which are connected by some intermediary
Help allow for sophisticated use of more fundamental neural network systems managed & handled in conjunction
Important principles in modular neural network development includes the "divide and conquer" principal, which
breaks down large problems into more viable smaller parts
Draw on their biological inspiration and emulate the segmentation and modularization found in the brain

Advantages-
Reduces complexity
More efficient
Training algorithm and training data can be implemented more quickly
More robust (by compartmentalizing subtasks, failure and interference are much more readily diagnosed)
Business Applications of Modular Neural
Network
Financial risk and return prediction (Stock market prediction)
 Data- The input consists of several technical and economic indexes. Several modular neural
networks learn the relationships between the past technical and economic indexes and the
timing for when to buy and sell

Option pricing
 Data- The data are daily S&P-500 index European call option prices. For each available
complete year, options across different strike prices and maturities are considered

Exchange rate prediction

 Data- Time series and technical indicators such as moving average so that the network learns
the underlying structure of price fluctuation. For ex- US dollar price against five other major
currencies along with a five year moving average.
Convolutional neural network
 Uses perceptron, a machine learning unit algorithm, for
supervised learning, to analyze data
 Applies to image processing, natural language processing and
various other kinds of cognitive tasks
 A convolutional neural network has an output layer, an input
layer and different hidden layers
 Some layers are convolutional, using a mathematical model
to pass on the results to their successive layers
 Simulates some of the actions of the human visual cortex
 Unlike neural networks, where the input is a vector, here the
input is a multi-channeled image

Advantages-
• Once trained, the predictions are pretty fast.
• CNN can be train with any number of inputs and layers.
• High accuracy
 Business Applications of CNN
Optical character recognition
◦ Govt. agencies are using them to digitize older documents to render them searchable
Data- Bitmap pattern of handwritten characters
◦ Banks- Processing print and handwritten documents (such as reading the amount and account on
cheques)
Data- Training is done using images of cheques to feed the location and format. Bitmap pattern of handwritten
characters to process handwritten documents
◦ Dispatch services like Fed Ex use them to read addresses
Data- Training is done using images of address labels and electronically generated receipts
◦ Financial time-series analysis (forecasting)
Data- Historical time series data along with technical indicators and economic indexes
◦ Algorithmic trading
Data- The input consists of several technical and economic indexes to train the network about the relationships
between the past technical and economic indexes and the timing for when to buy and sell
 Contd..
Image recognition
◦ Auto OEMs are using them for radar imagery to support self-driving cars
Data- Thousands of image road signs, objects found on and around the road, pedestrians, animals etc
◦ Perceptrons are also being used with robots on assembly lines for defect monitoring
Data- Images pertaining to various types of visual defects that are to be detected
◦ Healthcare companies are using them to process medical images such as ultrasounds, X-rays,
MRIs etc
Data- Similar medical images are used for training the network
◦ Aircraft seat ejection system testing
Data- Images of the ejection system from various viewpoints
◦ Reverse Image search, offered by search engines, employ CNN
Data- Library of images to search from
◦ Fraud detection
Data- Images of authorized signatures
 Convolutional neural network – An Example
Real time Image classification
• Dataset
 The data files train.csv and test.csv contain gray-scale images of hand-drawn digits, from zero
through nine
 Each image is 28 pixels in height and 28 pixels in width
 Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that
pixel, with higher numbers meaning darker
 This pixel-value is an integer between 0 and 255, inclusive

• Convolutional Neural Network used is

• Input -> Conv+tanh+Maxpooling -> Conv+tanh+Maxpooling -> Flatten+FC+tanh -> FC -> Softmax
output
• Creating a Neural network model
• Training the model
• Applying the model to the training set and checking the accuracy result
• Finally creating the submission file by applying the model to the test data and saving the results
as a csv
Thank You