How to write a classification algorithm

for your project

The gist
The short answer is…pre-trained neural networks. Both PyTorch and Keras integrated
reference architectures during the last years such as: VGG, XCeption, EfficientNet, AlexNet,
ResNet, DenseNet. The advantages of the pre-trained networks are:
1. their weights are already adjusted by using ImageNet for the training.
2. they spare you the time spent on debugging and on writing the network’s architecture.

Are you still in a pickle about using them?

How can I adapt the model to my case?

First, the model is fit to your specific case (i.e.: classes) by using these two steps. The
first action is to eliminate the top layer. At the end of the base model run, your pre-trained CNN
will provide an output for the custom layers. It is marked as x in the provided code. Once the top
layer is disabled, it is high time you created your unique layers for the desired results. The
number of classes is based on the labeling you want as an output. As an example, the template
code uses two classes (see below).

Which architecture is the best choice?

…it depends on your needs and resources. Let’s take ResNet and DenseNet as a
comparison example.
● Do you have a large dataset (e.g.: more than 50.000 labeled cases)? Then,
ResNet will handle the large amount better.
● Is your dataset small (e.g.: around 10.000 labeled cases)? DenseNet maximizes
the use of each case, with better accuracy per epoch.
● How many epochs are you going to run the CNN? There is more bang for your
epoch by using DenseNet rather than ResNet.
● What is your computing power? DenseNet will be more memory intensive in
comparison with ResNet.
● How much time do you have available for training the CNN? With more
compressed timelines comes the adage of a better GPU.
 To summarize the use case of each CNN, the architecture comes in-handy. ResNet
handles large datasets and numerous epochs better due to its identity layers, making use of its
skip connections. By contrast, DenseNet gets more information per epoch and per image due to
its highly interconnected layers. To understand each one better you can watch the video below
and read the original papers (https://arxiv.org/pdf/1608.06993.pdf &
https://arxiv.org/pdf/1512.03385.pdf).

Could these work even with few computing

resources?
Yes and…yes.

For starters, the RAM available on your unit or on a Cloud solution such as Google
Colab is the main limiter. These pre-trained architectures will load the images as they go, the
amount of RAM mounting considerably as each added image leads to a weight update. Finally,
it will crash the moment when the amount of images induces too many weight changes for the
remaining RAM.

The solution is batching. By batching, the CNN deals with one batch of images at once
and then it adapts the weights. The weight of each CNN node updates only once in a 32-image
batch, rather than updating with each image. This reduces the CNN’s training time, without
compromising the learning rate.

For the vanilla example of Keras dataloader (batching) or PyTorch dataloader, you can
access their documentation at these links: https://keras.io/api/data_loading/#timeseries-data-
loading and https://pytorch.org/tutorials/recipes/recipes/loading_data_recipe.html. To label the
datasets, each folder should contain a folder of the class with its corresponding images. For
instance, in the training folder there would be four folders titled with the names of a color
containing the corresponding images.

Template code for your neural network

# Using a pre-trained model
base_model = tf.keras.applications.DenseNet121(
include_top=False, weights='imagenet', input_shape=(224, 224, 3)
)

# Freeze the base model's layers & establish the number of classes
base_model.trainable = False
num_classes = 4

# Create the top layers for the model

x = base_model.output
x = tf.keras.layers.Dense(256, activation='relu')(x)
predictions = tf.keras.layers.Dense(num_classes, activation=’relu’)(x)

model = tf.keras.models.Model(inputs=base_model.input,
outputs=predictions)