Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Open in app

Follow 597K Followers

Transfer Learning with VGG16 and Keras

How to use a state-of-the-art trained NN to solve your image classification problem

Gabriel Cassimiro Jun 16 · 4 min read

The main goal of this article is to demonstrate with code and examples how can you use
an already trained CNN (convolutional neural network) to solve your specific problem.

Convolutional Networks are great for image problems however, they are
computationally expensive if you use a big architecture and don’t have a GPU. For that,
we have two solutions:

GPUs
GPUs are much more efficient to train NNs but they are not that common on regular
computers. So that is where google colab come to save us. They offer virtual machines
with GPUs up to 16 GB of RAM and the best part of it all: It is Free.

But even with those upgraded specs, you can still struggle when training a brand new
CNN. That’s where Transfer Learning can help you achieve great results with less
expensive computation.

Transfer Learning
So what is transfer learning?

To better explain that we must first understand the basic architecture of a CNN.

1 of 9 11/15/21, 00:23
Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Open in app

Image by Author

A CNN can be divided into two main parts: Feature learning and classification.

Feature Learning

In this part, the main goal of the NN is to find patterns in the pixels of the images that
can be useful to identify the targets of the classification. That happens in the
convolution layers of the network that specializes in those patterns for the problem at
hand.

I’m not going deep into how this works underneath the hood, but if you want to dig
deeper I highly recommend this article and this amazing video.

Classification

Now we want to use those patterns to classify our images to their correct label. This part
of the network does exactly that job, it uses the inputs from the previous layers to find
the best class to your matched patterns in the new image.

Definition

So now we can define Transfer Learning in our context as utilizing the feature learning
layers of a trained CNN to classify a different problem than the one it was created for.

In other words, we use the patterns that the NN found to be useful to classify images of a
given problem to classify a completely different problem without retraining that part of
the network.

Now I am going to demonstrate how you can do that with Keras, and prove that for a lot
of cases this gives better results than training a new network.

2 of 9 11/15/21, 00:23
Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Open in app

Transfer Learning With Keras

I will use for this demonstration a famous NN called VGG16. This is its architecture:

Image by Author

This network was trained on the ImageNet dataset, containing more than 14 million
high-resolution images belonging to 1000 different labels.

If you want to dig deeper into this specific model you can study this paper.

Dataset
For this demonstration, I will use the tf_flowers dataset. Just as a reminder: The VGG16
network was not trained to classify different kinds of flowers.

This is what the data looks like:

3 of 9 11/15/21, 00:23
Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Open in app

Image by Author

Finally…

The Code
First, we have to load the dataset from TensorFlow:

4 of 9 11/15/21, 00:23
Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Open in app
1 import tensorflow_datasets as tfds
2 from tensorflow.keras.utils import to_categorical
3
4 ## Loading images and labels
5 (train_ds, train_labels), (test_ds, test_labels) = tfds.load(
6 "tf_flowers",
7 split=["train[:70%]", "train[:30%]"], ## Train test split
8 batch_size=-1,
9 as_supervised=True, # Include labels
10 )
11
12 ## Resizing images
13 train_ds = tf.image.resize(train_ds, (150, 150))
14 test_ds = tf.image.resize(test_ds, (150, 150))
15
16 ## Transforming labels to correct format
17 train_labels = to_categorical(train_labels, num_classes=5)
18 test_labels = to_categorical(test_labels, num_classes=5)

transfer_learning_1.py hosted with ❤ by GitHub view raw

Now we can load the VGG16 model.

1 from tensorflow.keras.applications.vgg16 import VGG16

2 from tensorflow.keras.applications.vgg16 import preprocess_input
3
4 ## Loading VGG16 model
5 base_model = VGG16(weights="imagenet", include_top=False, input_shape=train_ds[0].shape
6 base_model.trainable = False ## Not trainable weights
7
8 ## Preprocessing input
9 train_ds = preprocess_input(train_ds)
10 test_ds = preprocess_input(test_ds)

transfer_learning_2.py hosted with ❤ by GitHub view raw

We use Include_top=False to remove the classification layer that was trained on the
ImageNet dataset and set the model as not trainable. Also, we used the preprocess_input
function from VGG16 to normalize the input data.

We can run this code to check the model summary.

5 of 9 11/15/21, 00:23
Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Open in app

base_model.summary()

6 of 9 11/15/21, 00:23
Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Open in app

Extra: comparing to hand-made model

To be sure that this approach can be better in both computational resources and
precision I created a hand-made simple model for this problem.

This is the code:

7 of 9 11/15/21, 00:23
Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Image by Author
Open in app
1 from tensorflow.keras import Sequential, layers
Two
2
main points: the model has over 14 Million trained parameters and ends with a
from tensorflow.keras.callbacks import EarlyStopping
maxpooling
3 layer that belongs to the Feature Learning partimport
from tensorflow.keras.layers.experimental.preprocessing of theRescaling
network.
4
Now
5 we add the last layers for our specific problem.
6 hand_made_model = Sequential()
7 hand_made_model.add(Rescaling(1./255, input_shape=(150,150,3)))
1 from tensorflow.keras import layers, models
8
2
9 hand_made_model.add(layers.Conv2D(16, kernel_size=10, activation='relu'))
3 flatten_layer = layers.Flatten()
10 hand_made_model.add(layers.MaxPooling2D(3))
4 dense_layer_1 = layers.Dense(50, activation='relu')
11
5 dense_layer_2 = layers.Dense(20, activation='relu')
12 hand_made_model.add(layers.Conv2D(32, kernel_size=8, activation="relu"))
6 prediction_layer = layers.Dense(5, activation='softmax')
13 hand_made_model.add(layers.MaxPooling2D(2))
7
14
8
15 hand_made_model.add(layers.Conv2D(32, kernel_size=6, activation="relu"))
9 model = models.Sequential([
16 hand_made_model.add(layers.MaxPooling2D(2))
10 base_model,
17
11 flatten_layer,
18 hand_made_model.add(layers.Flatten())
12 dense_layer_1,
19 hand_made_model.add(layers.Dense(50, activation='relu'))
13 dense_layer_2,
20 hand_made_model.add(layers.Dense(20, activation='relu'))
14 prediction_layer
21 hand_made_model.add(layers.Dense(5, activation='softmax'))
15 ])
22
transfer_learning_3.py
23 hosted with ❤ by GitHub view raw

24 hand_made_model.compile(
25 optimizer='adam',
26 loss='categorical_crossentropy',
And
27 compile and fit the model.
metrics=['accuracy'],
28 )
29
1 from tensorflow.keras.callbacks import EarlyStopping
30
2
31 es = EarlyStopping(monitor='val_accuracy', mode='max', patience=5, restore_best_weights
3 model.compile(
32
4 optimizer='adam',
33 hand_made_model.fit(train_ds, train_labels, epochs=50, validation_split=0.2, batch_size
5 loss='categorical_crossentropy',
transfer_learning_5.py
6 hosted with ❤ by GitHub
metrics=['accuracy'], view raw
7 )
8
9
I used the same final layers and fit parameters to be able to compare the impact of the
10 es = EarlyStopping(monitor='val_accuracy', mode='max', patience=5, restore_best_weights=True
convolutions.
11
12 model.fit(train_ds, train_labels, epochs=50, validation_split=0.2, batch_size=32, callbacks
The accuracy of the hand-made model was 83%. Much worse than the 96% that we got
transfer_learning_4.py hosted with ❤ by GitHub view raw
from the VGG16 model.
Sign up for The Variable
By Towards Data Science

Evaluating this model on the test set we got a 96% Accuracy!

8 of 9 11/15/21, 00:23
Transfer Learning with VGG16 and Keras | by Gabriel ... https://towardsdatascience.com/transfer-learning-with...

Every Thursday, the Variable delivers the very best of Towards Data Science: from hands-on tutorials
Open in app
and cutting-edge
That’s it! research to original features you don't want to miss. Take a look.

Emails will be sent to marco-mp@hotmail.com.

Get thisAnd
It is this simple. newsletter
it is kind ofNot
beautiful
you? right?

How we can find some patterns in the world that can be used to identify completely different
things.
Convolutional Network Keras Vgg16 Neural Networks Deep Learning

If you want to check out the complete code and a jupyter notebook, here’s the GitHubrepo:

gabrielcassimiro17/object-detection
About Writeto gabrielcassimiro17/object-detection
Contribute Help Legal development by creating
an account on GitHub.
github.com
Get the Medium app

9 of 9 11/15/21, 00:23