TRAINING Deep

Neural networks
with keras

D.DARLING JEMIMA
ASSISTANT PROFESSOR
DEPARTMENT OF CSE
SRI RAMAKRISHNA ENGINEERING COLLEGE
COIMBATORE.
jemima.d@srec.ac.in
AI IN HEALTHCARE
AI IN TRANSPORTATION(OBJECT
IDENTIFICATION)
AI IN FRUIT RIPENING
AI IN AGRICULTURE
AI IN GAMING
AI IN MUSIC
AI IN MILITARY
AI IN FINANCE
A person working in the
field of AI who doesn’t
know math is like a
politician who doesn’t
know how to persuade.
Both have an
“Deep Learning doesn’t do different things,
it does things differently”
 Why it has taken
off now Availability of
Data has
increased due
to explosion in
Smart Mobiles
and devices

Release/develo
pment of new
More algorithms,
Computing APIs and
Power is Platforms for
available due to Deep Learning
coming of Applications
NVIDIA GPUs

25 June
2018
Few Popular Applications: Precision Agriculture, Learner Profiling, Video Captioning,
Exploring Patterns from Satellite images, Image detection in Healthcare, Identifying
specific markers in Genomes, Creating Art and Music, Recommendations, behavior
prediction,
7
 Features
The features are the elements of your input vectors.
The number of features is equal to the
number of nodes in the input layer of the
Category Features
Housing Prices network
No. of Rooms, House Area, Air Pollution, Distance from facilities, Economic
Index city, Security Ranking etc.
Spam Detection presence or absence of certain email headers, the email structure,
the language, the frequency of specific terms, the grammatical
correctness of the text etc.
Speech noise ratios, length of sounds, relative power of sounds, filter matches
Recognition
Cancer Detection Clump thickness, Uniformity of cell size, Uniformity of cell shape, Marginal
adhesion, Single epithelial cell size, Number of bare nuclei, Bland chromatin,
Number of normal nuclei, Mitosis etc.
Cyber Attacks IP address, Timings, Location, Type of communication, traffic details etc.
Video Text matches, Ranking of the video, Interest overlap, history of seen videos,
Recommendatio browsing patterns
ns etc.
Image Pixel values, Curves, Edges etc.
Classification SREC 2
25 June
6
2018
How it Works
How it Works
How it Works
How it Works
 Deep Learning
• Deep learning has an inbuilt automatic multi stage
feature learning process that learns rich
hierarchical representations (i.e. features).
Low Level Mid Level High Output
Trainable (e.g. outdoor,
Features Features Level
Classifier indoor)
Features
Let us see how it all works!
A Deep Neural
Network
 Deep Neural Network
neuron
Input Layer 1 Layer 2 Layer L Output
x1 …… y1
x2 …… y2

……
……

……

……

……
xN …… yM
Input Output
Layer Hidden Layers Layer

Deep means many hidden layers

 Neural Network with
Screen
Size
x
1
one hidden Neuron and
one hidden layer
w
1

Hard Disk x
w2
Size Activation
2 Cost of the
Function
Computer
w
Processor Speed z g(z ŷ
3
1 1)
x3 w
4 z1= x1w1+ x2w2+ x3w3+x4w4+
RAM x x5w5+ b
w Ŷ=g(WTX
Size b
4
5
+b)

Battery x 1
Time
` 5
 Why
W TX
x1 W1

X2 W2
x1
X3 w3 X2
X4 w4 X3
X5 w5 X4
X5
W1 W2 W3 W4 W5

Matrix Matrix
Size: Size:
(1X5) (5X1)
Matrix Matrix
Size: Size:
(5X1) (5X1)
Multiplication of
these two
matrices is not
possible
 Deep Neural Network
x1 …… y1
x 2 W1 W2 ……
WL y2
b1 b2 bL

……
……

……

……

……
xN
x a1 a2…… y yM

 ¿ 𝑓 () Using parallel computing techniques

y x
to speed up matrix operation

  WL …  
¿ 𝜎
𝜎
W 2   𝜎 (
x(
W1 ( +)
) b2 … + bL
b1 + )
Tips for Training DNN
 Recipe for Learning
Modify the Network
• New activation functions, for example, ReLU
or Maxout

Better optimization Strategy

• Adaptive learning rates

Prevent Overfitting
• Dropout
Only use this approach when you already
obtained good results on the training data.
 Optimizers

Optimizers are algorithms or methods used to

change the attributes of your neural network
such as weights and learning rate in order to
reduce the losses.

Different Optimizer Algorithm:

https://towardsdatascience.com/optimizers-for-
training-neural-network-59450d71caf6
 Implementation of DNN using Keras

• https://colab.research.google.com/github/lexf
ridman/mit-deep-
learning/blob/master/tutorial_deep_learning
_basics/deep_learning_basics.ipynb#scrollTo=
b_BU9wUt-Yq6
Thank you