Codeless Deep Learning For Language Modeling and Image Classification

Codeless Deep Learning for Language Modeling
and Image Classification
Corey Weisinger
Corey.Weisinger@knime.com
Rosaria Silipo
Rosaria.Silipo@knime.com
© 2020 KNIME AG. All Right Reserved.

Agenda
• Introduction to the open source tool KNIME Analytics Platform

• Introduction to RNN units and their Applications
• Transfer learning for image classification
© 2020 KNIME AG. All Rights Reserved. 2

What is KNIME Analytics Platform?
• A tool for data analysis, manipulation, visualization, and reporting

• Based on the graphical programming paradigm
© 2020 KNIME AG. All Rights Reserved.

Visual KNIME Workflows
NODES perform tasks on data

Not Configured
Configured
Inputs Outputs Executed
Status Error
Nodes are combined to create

WORKFLOWS

Over 2000 Native and Embedded Nodes Included:
Data Access Transformation Analysis & Mining Visualization Deployment

MySQL, Oracle, ... Row Statistics R via BIRT
SAS, SPSS, ... JFreeChart
Column Data Mining PMML
JavaScript
Excel, Flat, ... Matrix Machine Learning Plotly XML, JSON
Hive, Impala, ... Text, Image Web Analytics Community / 3rd Databases
XML, JSON, PMML Time Series Text Mining Excel, Flat, etc.
Text, Doc, Image, ... Java Network Analysis Text, Doc, Image
Web Crawlers Python Social Media Analysis Industry Specific
Industry Specific Community / 3rd R, Weka, Python Community / 3rd
Community / 3rd Community / 3rd

KNIME Software - Better Decision-making, Faster!
Load Integrate Transform Analyze Visualize
KNIME Community
Integrations Extensions
KNIME
Open Source
Analytics Platform
KNIME Partner
Extensions Extensions
KNIME Server
Collaborate Automate Manage Deploy

Introduction to RNN and their Applications

Neural Network Review

Let’s start with a history lesson
Neural networks are inspired by biological neural networks
Neuron Networks
𝑥
𝑤 𝑏
∑0,8σ 𝑎 = 𝜎(𝑥 𝑤 + 𝑥 𝑤 + 𝑏)
𝑤
𝑥

Fully connected, feed forward networks
Input Hidden Output

Layer Layer Layer Forward pass:
𝑏
∑
𝑤 , 𝑤
𝑤 ,
,
𝑏 𝑏
𝑤 ,
∑ 𝑤 , ∑
𝑤 ,
𝑤 , 𝑤 ,
𝑤 , 𝑏
∑

Frequently used activation functions
Sigmoid Tanh Rectified Linear Unit (ReLU)

Fully connected, feed forward networks- simplified
Input Hidden Output

Layer Layer Layer Forward pass:
𝟐 𝟑
𝒙 ∑ 𝒚
∑ ∑

What are RNNs and LSTMs?

What are RNNs?
• Recurrent Neural Network (RNN) are a family of neural networks used

for processing sequential data
• RNNs are used for all sorts of tasks:
– Language modeling / Text generation
– Text classification
– Neural machine translation
– Image captioning
– Speech to text
– Numerical time series data, e.g. sensor data

Why do we need RNNs for sequential data?
• Goal: Translation network from German to English

∑ σ
“Ich mag Schokolade” ∑ σ
=> “I like chocolate” ∑ σ

𝑥 ∑ σ ∑ σ 𝑦
∑ σ
• One option: Use feed forward network to
translate word by word ∑ σ
∑ σ
• But what about this sentence? Input x Output y

Ich I
“Kathrin mag Schokolade”
mag like
=> “Kathrin likes chocolate”
Schokolade chocolate

Why do we need RNNs for sequential data?
• Problems:
∑ σ
– Each time step is completely independent ∑ σ
– For translations we need context ∑ σ
– More general: we need a network that 𝑥 ∑ σ ∑σ 𝑦
remembers inputs from the past ∑ σ
∑ σ
∑ σ
• Solution: Recurrent neural networks
Input x Output y
Ich I
mag like
Schokolade chocolate

What are RNNs?
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-LSTMs/

From feed forward to recurrent neural networks
𝟐 𝟑
𝒙 𝒚
∑ ∑
𝟐 𝟑
𝒙 𝒚
∑ ∑ ∑
∑ ∑

From feed forward to recurrent neural networks
𝟑 𝟑 𝟑 𝟑
𝒚 𝒚 𝒚 𝒚
∑ ∑ ∑ ∑
∑ ∑ ∑ ∑
∑ ∑ ∑ ∑
𝟐 𝟐 𝟐 𝟐
𝒙 𝒙 𝒙 𝒙

Simple RNN unit
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-LSTMs/

Limitations of simple layer structures
The “memory” of simple RNNs is sometimes too limited to be

useful
– “Cars drive on the ” (road)

– “I love the beach – my favorite sound is the crashing of the “
(cars? glass? waves?)

LSTM = Long Short Term Memory Unit
• Special type of unit with three gates

– Forget gate
– Input gate
– Output gate
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-L

Different network-structures and applications
Many to Many
Ich gehe gerne segeln
I like sailing <eos>

D D D D
A A A A
E E E Ich gehe gerne
<sos> I like sailing

I like sailing
Language model Neural machine translation

Different network-structures and applications
Many to one One to many

English Couple sailing on a lake
A A A A A A A A A A
I like to go sailing
Language classification
Text classification Image captioning
How can we train a recurrent neural network for text generation
• Decide between character, sub word, or word level

– Word level
• How we structure language
• Length of sequences are much shorter
– Character level
• Smaller dictionary => much more manageable
• Ability to generate new words
– Sub-word level
• Happy medium
• What kind of data can we use?

• How can we prepare our data for the training?

The Creative Side of AI: Naming New Products

The Product Naming Phase
What happens when a new product is born?
Naming new products is not as easy as it might sound …

The Case Study: new line of outdoor clothing
Outdoor or hiking clothing line needs

new names that:
• Evoke the feeling of nature
• Sound familiar to customers
• Stand out from the competition
• Are not covered by copyright
• The team also needs many potential candidates to evaluate
• Could a neural network help in this creative process?

Let’s build a model to generate fictional mountain names!

Quick sneak preview – Find the fake mountain names
Set 1 Set 2 Set 3

Barlock Mountain Roblin Hill Terrey Hill
Casterland Mountain Buxley Mountain Walter Hill
Shafford Peak Baldy Rock Little Buck Butte
Correct answer: ALL!

The Dataset
• 33,012 names of mountains in

the US
• Extracted from Wikipedia via a

Wikidata query

Many to many structure

95 = dictionary size
0,0,0,1,0,0,0,0,0, ... , 0
o
0,0,0,0,0,0,1,0,0, ... , 0
u
0,0,0,0,0,0,0,1,0, ... , 0
n
M samples
0,0,0,0,1,0,0,0,0, ... , 0
t
0,0,0,0,0,0,0,0,1, ... , 0
1,0,0,0,0,0,0,0,0, ... , 0
a B
0,1,0,0,0,0,0,0,0, ... , 0
...
k
m
n
b
z
a
softmax
linear
LSTM based Neural Network: Many to Many
33
LSTM states
M = max. Length of mountain name

0,1,0,0,0,0,0,0,0, ... , 0
M
0,0,0,1,0,0,0,0,0, ... , 0
o
0,0,0,0,0,0,1,0,0, ... , 0
u
M samples

0,0,0,0,0,0,0,1,0, ... , 0
N=256 LSTM states

n
0,0,0,0,1,0,0,0,0, ... , 0
t
Mount Baker
0,0,0,0,0,0,0,0,1, ... , 0
1,0,0,0,0,0,0,0,0, ... , 0
B
The KNIME Keras Integration
• Codeless GUI based

• Fully opensource
• Keras functions
https://www.knime.com/deeplearning/keras

Codeless?
LSTM states
Model training workflow
• Read in and transform the input

dataset
• Define the structure of the
neural network
• Train the model using Keras
Learner
• Do a bit of post-training model
editing to introduce temperature
• Convert model to TensorFlow
and save for use in deployment
workflow

Now that we have a model, how do we use it?

Neural network: Code-free
Layers of our neural network:

• Input layer of size [?, 95]
• LSTM for sequence analysis
• Dropout layer to prevent
overfitting
• Dense layer 1 – linear activation
• Dense layer 2 (output) – Softmax
activation
Once we’ve defined the network

structure and pre-processed our
input, we’re ready to train.
Want to see the names?
Advantages:
- They remind you of mountains. Do
not they?
- Sound familiar enough
- Evoke the feeling of nature
- No copyright issues
- Automatic generation in 10 seconds
- You can generate as many as you
want
- No people involved as of now
Can this be considered a creative task?

Yo! AI Generated Rap Songs

Bots & speaking styles
- Gossipy
- Articulate
- Idiomatic
- Chatty
- Incoherent
- Clean
- Informal
- Conversational
- Journalistic
- Crisp
- Literary
- Declamatory
- Lyric
- Diffuse
- Ornate
- Discoursive
- Parenthetical
- Eloquent
- Pejorative
- Emphatic
- Picturesque
- Epigrammatic
- Poetic
- Epistolary
- Prolix
- Euphemistic
- Punchy
- Flowery
- Rambling
- Funny
- Rhetorical
- Fluent
- Rough
- Formal
https://writerswrite.co.za/60-words-used-to-describe-writing-or-speech-style/ - Sesquipedalian

How should my bot speak?
• I want my bot to answer in a polite tone in some situations, even

almost poetic
• I want my bot to answer in an affirmative tone in other situations,

borderline impolite

Let‘s start with impolite ...
• Let‘s generate a rap song!

• This is a free text generation problem
• Similar to the problem of the product name generation, just on longer
complete sentences
• Let’s build a similar network from scratch!
• Maybe not.
• Let’s search Kathrin’s example on the KNIME Hub
• and readapt it!

On the KNIME Hub
https://hub.knime.com

On the KNIME Hub
https://kni.me/w/mGO9nXhmjzIKiqHU

LSTM based Neural Network
M past samples
a
b
s u o h
e
0,0,0,0,0,0,0,1,0, ... , 0
0,0,0,0,0,0,1,0,0, ... , 0
0,0,0,1,0,0,0,0,0, ... , 0
0,1,0,0,0,0,0,0,0, ... , 0
n
LSTM states
N
...
Dictionary size = 86
256 LSTM states z
100 past characters
Training set = 23 popular rap songs

Creative AI: the training workflow

Creative AI: the deployment workflow

Let’s rappify some text
Yo!
This post is about generating free text
When the cat is away
with a deep learning network
The mice shall play This License refers to version of the GNU General Public
When the cat is away particularly it is about
Never mind I'll find someone like you License.
The mice shall play Brick X6, Copyright also means copyright-bi
I wish nothing but the best
For you too, don't forget me lease, yua do not a aze me I around my dramy Phey, cabe, th eaf hard fortars
I beg, I And when I'm finished, bring the yellow tape make you feel soom the way (I smoke good!) I wanna lister for ald be gadg to call up in the
don't need no advice To tape off the scene of the slaughter I probably make (What?) Phantom
You're not here and we both know why, so Still gettin' swoll off bread and water I don't know if they More money in six months, Know *****, they come and kill you on the camera
Move from me when you're extra *** or what Big Rollie, it dancin' bigger than a Pandie
Than what's in your papa's safe (I'm serious)
Move from me with the passa Search a *** down, and grabbin' his *** Go Oscar for Grammy, **** pull up your panty
And on the other hand, without a gun they can't get none Look like I robbed a bank (Okay Okay) Fill up
I'm building up a house where they raised me
But don't let it be a black and a white one I set it off like Queen Latifah I'ma flip it, I got **** pull up and peese vire
You move with me I'll go crazy
Don't switch on me, I got big plans 'Cause they'll slam ya down to the street top 'Cause I'm living single I'm feeling cautious Reepin me sher wing these hoes
We need to forward to the islands and get you gold, Black police showin' out for the white cop I ain't scream when they served a subpoena Don't settle for less one you lise one
no spray tans Ice Cube will swarm And my cas up in the
(Can't go back to jail)
I need you to stop running back to your ex he's a wasteman On any mother**** in a blue uniform Pun I arn’t **** with me
Just cause I'm from, the CPT I heard that he a leader If you wanted to
I wanna know how come we can never slash and stay friends
Punk police are afraid of man en (Who pood, what to be *****' up These expensive, these is red bottoms
I'm blem for real, I might just say how I feel
Together forever I han to we have a keround hord **** it The baugerout Black alro Black X6, These is bloody somes
Good morning, good afternoon, goodnight I'm right Phantom White X6 looks like a panda It the street to you heare it all while I be
I'm here to talk about What more can I say to you? You in the club just to party I'm there,
Goin' out like I'm Montana
More Life On the Phostest, *** tike and push it Get my grown man on I get paid a fee
Let's go! (What more can I say?) Hundred killers, hundred hammers Black X6, I be in and out them banks so much
And push it good for the cremm
Now you know ass is willie Phantom White X6, panda I know they're tired of me
Mand a ray
*** the but bo che turn me When they got you in a mag Pockets swole, Danny Honestly, don't give a ****
But dow of the For like half a billi Sellin' bar, candy 'Bout who ain't fond of me
Purb everydor get my mus, **** And your ass ain't Lilly Dropped two mixtapes in six months
Man I'm the macho like Randy
**** you move my home back hoot on the als to a lot on the White What **** working as hard as me?
That mean that shit you write must be illy The choppa go Oscar for Grammy I don't bother with these hoes
crim just the yourh
Either that or your flow is silly **** **** pull up ya panty Don't let the truth to the young black youth
I'm a sixe and got a proup in the stake of the
Ond I won't stop abusing it It's both I don't mean to boast Hope you killas understand me But shorty's running with my ****, for the man,
To groupie girls stop false accusing it But damn if I don't brag Hey Panda, Panda Panda, I as what
Back to the music Them crackers gonna act like I ain't on they ads I was gang affiliated, got on TV and told on me
Panda, Panda, Panda, Panda
The Maybach roof is translucent The Martha Stewart I guess that's why last winter she got so cold on me
That I got broads in Atlanta She s
**** got a problem Houston
Twistin' dope, lean, and the Fanta
What up be
The Credit cards and the scammers
Hittin' off licks in the bando

To be or not to be … Shakespearian text?

Now for the poetic text ...
“Many a true word hath been spoken in jest.”
• How would Shakespeare say it? ― William Shakespeare, King Lear
“O, beware, my lord, of jealousy;

• I can reuse the same LSTM based network It is the green-ey’d monster, which doth mock
The meat it feeds on.”
• With a different training set ― William Shakespeare, Othello
“There was a star danced, and under that was I born.”

• Training set is the full texts of ― William Shakespeare, Much Ado About Nothing
– “King Lear”
– “Othello”
– “Much ado about nothing”
• The only difference in the network is in the dictionary size

Want to see some AI/Shakespeare text?
– Othello
O my fair warrior! Smokin’ on cookie in the hotbox cookie
F*****’ on your b**** she a thot, thot, thot thot
– Desdemona Cookin fellow,
My dear Othello!
SCENE I. This License refers to version of the GNU General and the best of our that she could else was not a
– Othello
Venice. A street. Public License. Copyright also means copyright- toman.
It gives me wonder great as my content
Enter Roderigo and Iago -bick, Good faith, hold, I beseech your grace,—
To see you here before me. O my soul’s joy! Remade me any thing to his sword -King Lear
I am a man a worm. – Desdemona To his salt and most hidden loose to be so for sings, Then there’s so much to him.
– Gloucester I pray, talk me of Cassio. but not in a libutt of his matter than that shall be sure -King Lear
I know not, sweet: Ay, so I hear the write to prive and there, as will be soldye Thou hast seen a part of this plainness I had thought to
I found your like a thief from the heart That she would seen him that present so lich wored As master compary, do not live in traitor. see the ride on the sea sevel never second to take the
That pirchas will be well. old wat, Bless thy five wits! foul fiend.
The general speaks to be so and the best conscionable than in this revolumance -Kent Still through the way that shoulds: I know not what.
Turn a man, I think, besoou. against him: O pity! -Othello
– Cassio There’s son against father, Sir, where is the patience now, What dost thou see her? O unhappy girl!
I pray you, sir, to lie: in this hand is not a tend and talking and thy father’s son and the best of our time hath no That this is so far from the sea and some bidings to Sir, this desperate, bastard! what news?
of it; exalse your counsel watch dismantle Mistage on my father!
I would not be threaten dispatch. The worst is not a tender to the warlike isle, So many folds of save and honest. -King Lear
Our good old friend, That sunded must needs take the shame -Brabantio What say’st thou st?
Lay comforts the state, seek for him; which the revenges of the self-same malk than the I must not think the Turk of Cassio in the strange -Kent
I will grife you be faster’d! best of our times; metting the cribles of a charmer be the reviling of libe Sir, I do know you;
And the blessed course of dower: keeps our fortunes fend for bearing to a strength, to say And to the Moor and rain, I beserve her take my
Net forth enough to do you; Sight in their natures, letting go safely by the rack: I That I can deceive him to the best advantage, sisters?
And that the Moor is defective in the letter! swear ‘tis, to be so In her prophetic fairs of a little to presently at your -King Lear
Abhorre, heaven, I will go sor; That she will send back my messenger. powers; whereof I thank you, sir. No, no, no, no! where’t the castle.
And the other too. – Gloucester -Albany Enter Gloucester, with King of France and Cordelia
I see the rust in the stocks. Gloucester, I will prove upancy of his sport and first Goneril
– Othello
– King Lear accuriors and guard and talking on the white. Sir, I had thought it not.
I have a seet me such a trial of his speech,
What a trifore be some cartiou, I can tell my way -King Lear -Iago
That he shall live the Moor in the lies
Than should be assurather, despise my brother; Where are the thief? Indeed!
That with daush’er
That I have passed him, tell me I was every think of Thou shalt never have the captains at the letter -Othello
Holds it is a most poor man,
fear, To the Moor and thing we have not the better shall be Indeed! ay, indeed: desceme, sir.
Whose welchers of the state,
That she may be honest yet he hath confess’d in him sure as worth if he be anger— -Cornwall
A man that many call of their life
entertains -Regan Sighing, would!
That have to lost the boy look to’t.
and think the next way to stain it, I pray you, have a countend more than think to do a -Othello
– Regan That the main commet in the least What dost thou see how this paper shall I carry out a
proclaim’d there of my heart, Hot
Sir, to my sister? Would fail her breath, fellow there, that makes his son a poor poor power
– Oswald That she may … That makes his son
I pray you, have your hand: you may receive it all to his When the rain came to the devil wrath!
sorrage, and makes the heavens I have lost the king from the cold wind:
Cassio lies that in the heart Take them what the …
That I may speak: I’ll wast …

Conclusions

Can AI be creative?
Yo!
Caro amico ti scrivo così mi distraggo un po'E
This post is about generating free text siccome sei molto lontano più forte ti scriverò.
with a deep learning network Da quella prima folla strana, che aveva preso il suo
particularly it is about Brick X6, This License refers to version of the GNU General Public License. nome, e di correre alla casa di don Abbondio, con
Phey, cabe, Copyright also means copyright-bick,
make you feel soom the way (I smoke good!)
un viso bene di non poterci andar la casa del padre
Remade me any thing to his sword
I probably make (What?) To his salt and most hidden loose to be so for sings, but not in a Cristoforo, e gli disse che s'avvicinava all'uscio, e si
More money in six months, libutt of his matter than that shall be sure as will be soldye mise a sparse di corsa, e di stare a sé, verso la
Than what's in your papa's safe (I'm serious) As master compary, do not live in traitor. strada di servizio, chiesto le parole che gli andavan
Look like I robbed a bank (Okay Okay) Bless thy five wits!
I set it off like Queen Latifah -Kent dall'altra stanza, e con la sua condizione de'
'Cause I'm living single I'm feeling cautious O pity! cappuccini, e di consigli ricerche di confidenza
I ain't scream when they served a subpoena Sir, where is the patience now, delle gride, nel suo passaggio, se non pensava con
(Can't go back to jail) That this is so far from the sea and some bidings to dismantle
So many folds of save and honest. una certa ripugnanza a casa sua, che andavano a
I heard that he a leader
(Who pood, what to be f*****' up -Brabantio scomparire in un campo di buone ragioni che
I must not think the Turk of Cassio in the strange metting the avevan potuto raccogliere i suoi pensieri, e di
The baugerout Black alro Black X6,
cribles of a charmer be the reviling of libe to say
Phantom White X6 looks like a panda That I can deceive him to the best advantage, sopra non senza interrogare, che la sua avventura
Goin' out like I'm Montana In her prophetic fairs of a little to presently at your powers; aveva fatto predicare, e con la forza d'un fatto
Hundred killers, hundred hammers Black X6, whereof I thank you, sir.
Phantom White X6, panda
come fuggitive che aveva preso il suo nome, e di
-Albany
Pockets swole, Danny Gloucester, I will prove upancy of his sport and first accuriors and correre alla casa di don Abbondio, con un
Sellin' bar, candy guard and talking on the white. cappuccino di quella sorte, con un certo sospiro,
Man I'm the macho like Randy -King Lear alzando le sue finestre, e le diede un'occhiata in
The choppa go Oscar for Grammy Where are the thief? carrozza. Si vendano a metter nelle mani di chi era
B**** n**** pull up ya panty Thou shalt never have the captains at the letter
Hope you killas understand me To the Moor and thing we have not the better shall be sure as stato a sedere sur una strada così fatta con le
Hey Panda, Panda Panda, worth if he be anger— braccia in
Panda, Panda, Panda, Panda -Regan
I got broads in Atlanta I pray you, have a countend more than think to do a proclaim’d
Twistin' dope, lean, and the Fanta there of my heart, Hot
Credit cards and the scammers The words save, honest, thief, master, traitor, and deceive seem to
Hittin' off licks in the bando fit the context. Notice also that the dialogue sprouting from the
start text of the license agreement interestingly involves mainly
minor, less tragic characters from the plays.

References
• https://www.datanami.com/2019/04/04/product-naming-with-deep-learning/
• https://opendatascience.com/how-to-use-deep-learning-to-write-shakespeare/
• https://customerthink.com/ai-generated-rap-songs/

Free Book as a Thank You
Free Copy of e-book:

“Practicing Data Science.
A Collection of Case Studies”
from KNIME Press

https://www.knime.com/knimepress
with this code: AUSTIN-0120

Thank you!
Questions?
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME AG under license from KNIME GmbH,
and are registered in the United States. KNIME® is also registered in Germany.
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by
KNIME AG under license from KNIME GmbH, and are registered in the United States.
KNIME® is also registered in Germany.

Transfer Learning
Corey Weisinger
© 2020 KNIME AG. All Right Reserved.

What is Transfer Learning?
• Transfer learning can be defined

as the attempt to utilize
predictive ability in one input /
output space to aid in the
B
learning of new input spaces,
A C
output spaces, or both.
• In this use case the input space, New
3x64x64, remains the same, but
the output space changes from
1000 image categories to 3
cancer types.

Transfer Learning – Basic Example
Input Data Output
Standard Learning
Sepal Length Prediction

Sepal Width Safe to Eat
Petal Length
Lots of data likely needed to tune
Petal Width weights for logistic regression
Input Data Transferred Output Prediction

Transfer Learning
Sepal Length Learning Safe to Eat

Sepal Width Type of Flower
Easy to train small decision
Petal Length
tree on little data
Petal Width

Cancer Cell Classification Use Case
• Uses Keras Deep Learning Original Task

model.
• Learning Transferred from VGG16 Cat
VGG16 image classifier This Photo by Unknown
• Completely code free with

Author is licensed under
CC BY-SA
KNIME’s Deep Learning New Task

Integration
• Can be adapted to a wide range VGG16 New CLL
of image classification problems
Image From:
https://ome.grc.nia.nih.g
ov/iicbu2008/lymphoma/
index.html

Why is it helpful?
• Deep Learning requires tons of DATA POINTS

data
1,200k
• Image classification requires
tons and tons of data
• We have ~5,000 labeled cancer
images
• VGG16 was trained on more
than 1,000,000 images from
ImageNet dataset. 5k
CANCER CELLS IM A G E N E T

Feature Extraction in Neural Networks
• One of the benefits of

deep learning is its ability
to perform its own feature
engineering
• This can occur through
vector embedding,
convolutional layers, Figure available under Creative Commons Attribution 4.0 International
LSTM layers etc.

Convolutional Layer
• Instead of connecting every

neuron to the new layer a
sliding window is used
• Use when your data has spatial
relationships
– 2D: Image
– 3D: Video
• Some convolutions may detect
edges or corners, while others
may detect cats, dogs, or street
signs inside an image Image from: https://towardsdatascience.com/a-comprehensive-guide-
to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53

Convolutional Layer Cont.
• Stride:
– Distance between adjacent
windows (windows can overlap)
• Window/Kernel/Filter size:
– dimension(s) of windows
• Pooling:
– When the convolution maps
many to one. Commonly taking
the max or average value in a
window.
• Benefits: Image from: https://www.freecodecamp.org/news/an-intuitive-guide-
to-convolutional-neural-networks-260c2de0a050/
– Fewer weights than dense layers
– Less prone to overfitting
– Feature extraction useful for
transfer learning

Final Layer
VGG16 Model We used
VGG16 Architecture: https://neurohive.io/en/popular-networks/vgg16/
• Several blocks of small convolution layers followed by max pooling

layers
• Trained on ImageNet dataset
– Over 1,000,000 images from 1,000 thousand classes
• Won ILSVRC-2013 competition in single network category

Everything in KNIME
1. Download Data Set

2. Pre-Process Data Set
1. Crop, Normalize, Rearrange
3. Configure Network
1. Load VGG16
2. Add and Freeze Layers
4. Train Network
5. Post-Process Predictions
1. Group images, create Classification
Workflow on KNIME Hub:

https://kni.me/w/1_w_dxiltpHmsBQA

Pre-Processing
• Input size
– VGG16: 3x64x64 image
– Us: 1388x1040x3 image
• Image Values
Create Patches Metanode
– VGG16: Normalized 0-1
– Cancer Cells: 0-256
• Pre-Processing
– Multiple patches per
image
– Normalize (per image) Image processing nodes part of KNIME Image Processing Extension:
https://kni.me/e/Uq6QE1IQIqG4q_mp
– Re-arrange dimensions

Configuring the Network
# create the base pre-trained model
model = VGG16(weights='imagenet',
include_top=False, input_shape=(64, 64, 3))
# add Flatten, dense, and dropout layers

model.add(Flatten())
model.add(Dense(32), activation=‘ReLu’)
model.add(Dropout(0.5))
# add a fully-connected layer output layer
model.add(Dense(3, activation=‘Softmax’))
# first: train only the top layers Keras nodes part of KNIME Deep Learning – Keras Integration:
for layer in model.layers[0:-4]: https://kni.me/e/XOee1uZPrzE36EPH
layer.trainable = False
• Load VGG16 Network

• Add custom layers to produce 3 class predictions
• Freeze original VGG16 layers
– Keep feature detection intact
– Significantly reduces number of parameters to be learned

Training the Network
• We train the newly

configured network for 25
epochs
• Only updating the
weights in our unfrozen
end layers
• Using Categorical Cross
Entropy Loss function
Learning Monitor, part of Keras Network Learner node

Post-Processing
• Combine image patches

– Average prediction by class
– Use max probability as
classification
• Scoring
– High Accuracy for small training
set
– High Cohen’s Kappa value shows
significant divergence from
random chance

Finding the Example

Related KNIME Blog Articles
• Original use case, involves

some coding:
– https://www.knime.com/blog/u
sing-the-new-knime-deep-
learning-keras-integration-to-
predict-cancer-type-from-
histopatholog
• Updated use case, totally code
free:
– https://www.knime.com/blog/tr
ansfer-learning-made-easy-with-
deep-learning-keras-integration

Citations
• Convolution Layer: https://towardsdatascience.com/a-

comprehensive-guide-to-convolutional-neural-networks-the-
eli5-way-3bd2b1164a53
• Cancer Cell Slide:
https://ome.grc.nia.nih.gov/iicbu2008/lymphoma/index.html
• VGG16 Architecture: https://neurohive.io/en/popular-
networks/vgg16/

KNIME Course Books
Course books downloadable from

KNIME Press
https://www.knime.com/knimepress
with code: AUSTIN-0120

Thank you!
Questions?
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by
KNIME AG under license from KNIME GmbH, and are registered in the United States.
KNIME® is also registered in Germany.

Next Meetup
• Planning for April, then afterward June

• Next location: IBM campus on Burnet Road
• April Meetup topic ideas?
– Active Learning talk
– KNIME Use Cases talk
– Raw Data to Deployment Learnathon
– Time Series Learnathon
– ...or something else?
• Want to present on KNIME and your project? Let us know!

Codeless Deep Learning For Language Modeling and Image Classification

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Codeless Deep Learning For Language Modeling and Image Classification

Uploaded by

Copyright:

Available Formats

Codeless Deep Learning for Language Modeling

and Image Classification

© 2020 KNIME AG. All Right Reserved.

• Introduction to the open source tool KNIME Analytics Platform

© 2020 KNIME AG. All Rights Reserved. 2

• A tool for data analysis, manipulation, visualization, and reporting

© 2020 KNIME AG. All Rights Reserved.

NODES perform tasks on data

Nodes are combined to create

© 2020 KNIME AG. All Rights Reserved.

Data Access Transformation Analysis & Mining Visualization Deployment

© 2020 KNIME AG. All Rights Reserved.

Load Integrate Transform Analyze Visualize

Collaborate Automate Manage Deploy

© 2020 KNIME AG. All Rights Reserved.

© 2020 KNIME AG. All Rights Reserved. 7

© 2020 KNIME AG. All Rights Reserved. 8

Neural networks are inspired by biological neural networks

© 2020 KNIME AG. All Rights Reserved. 9

Input Hidden Output

© 2020 KNIME AG. All Rights Reserved. 10

Sigmoid Tanh Rectified Linear Unit (ReLU)

© 2020 KNIME AG. All Rights Reserved. 11

Input Hidden Output

© 2020 KNIME AG. All Rights Reserved. 12

© 2020 KNIME AG. All Rights Reserved. 13

• Recurrent Neural Network (RNN) are a family of neural networks used

© 2020 KNIME AG. All Rights Reserved. 14

• Goal: Translation network from German to English

=> “I like chocolate” ∑ σ

• But what about this sentence? Input x Output y

© 2020 KNIME AG. All Rights Reserved. 15

© 2020 KNIME AG. All Rights Reserved. 16

Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-LSTMs/

© 2020 KNIME AG. All Rights Reserved. 17

© 2020 KNIME AG. All Rights Reserved. 18

© 2020 KNIME AG. All Rights Reserved. 19

Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-LSTMs/

© 2020 KNIME AG. All Rights Reserved. 20

The “memory” of simple RNNs is sometimes too limited to be

– “Cars drive on the ” (road)

© 2020 KNIME AG. All Rights Reserved. 21

• Special type of unit with three gates

Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-L

© 2020 KNIME AG. All Rights Reserved. 22

I like sailing <eos>

<sos> I like sailing

Language model Neural machine translation

© 2020 KNIME AG. All Rights Reserved. 23

Many to one One to many

• Decide between character, sub word, or word level

• What kind of data can we use?

© 2020 KNIME AG. All Rights Reserved. 25

© 2020 KNIME AG. All Rights Reserved. 26

What happens when a new product is born?

Naming new products is not as easy as it might sound …

Outdoor or hiking clothing line needs

© 2020 KNIME AG. All Rights Reserved. 28

© 2020 KNIME AG. All Rights Reserved. 29

Set 1 Set 2 Set 3

Correct answer: ALL!

© 2020 KNIME AG. All Rights Reserved. 30

• 33,012 names of mountains in