Image Classification

Manual
 Table of Contents
Image Annotation - Concept.......................................................................................................................4
How does it work?.....................................................................................................................................4
Types..........................................................................................................................................................4
Bounding Boxes.........................................................................................................................................4
Image Annotation – In Action.....................................................................................................................5
“train_diaMOSNET.py” - Code Explanation...........................................................................................6
Imports.......................................................................................................................................................6
Setup..........................................................................................................................................................6
The __name__ == “__main__”..................................................................................................................7
 Split data........................................................................................................................................7
 Concatenate the Datasets...............................................................................................................7
 Data-saved path..............................................................................................................................8
 Convert Dataset to Dataframe........................................................................................................8
 Save the Dataframes to CSV..........................................................................................................8
 Deep Learning Training Model Loop Proper.................................................................................8
 Creating Data Generators...........................................................................................................8
 Defining Data Generators using MultioutputGenerator Function.............................................9
 Creating and Training the Model.............................................................................................10
“dataset.py” - Custom-made Functions...................................................................................................11
load_data..................................................................................................................................................11
shuffle_split_data.....................................................................................................................................11
split_data..................................................................................................................................................13
“multi_output_generator.py” - Custom-made Functions......................................................................14
MultiOutputGenerator Class....................................................................................................................14
Initializing MultiOutputGenerator Class.................................................................................................15
getGenerator Function.............................................................................................................................16
getNext Function......................................................................................................................................16
labels property of MultiOutputGenerator Class.......................................................................................17
“diaMOSNet.py” - Custom-made Functions...........................................................................................19
MultiOutputGenerator Class....................................................................................................................19
Model Building using Keras......................................................................................................................20
1. Pretrained Model..............................................................................................................................20
2. Transfer learning & fine-tuning.......................................................................................................20
 tf.keras.Input................................................................................................................................21
 tf.keras.layers.BatchNormalization..............................................................................................21
3. tf.keras.Model..................................................................................................................................21
4. compile()..........................................................................................................................................22
Model Training using Scikit-Learn fit()...................................................................................................24
fit() method..............................................................................................................................................24
The Callback API.....................................................................................................................................25
 tf.keras.callbacks.EarlyStopping..................................................................................................25
 tf.keras.callbacks.ModelCheckpoint............................................................................................26
 tf.keras.callbacks.ReduceLROnPlateau.......................................................................................28
Python Methods.........................................................................................................................................30
__init__ method.......................................................................................................................................30
Keras ImageDataGenerator with flow_from_dataframe().......................................................................31
StratifiedShuffleSplit...............................................................................................................................32
Split..........................................................................................................................................................33
Image Annotation - Concept
ASSIGNING OF LABELS TO HELP AI MODEL TO DETECT CERTAIN ASPECTS WITHIN A
VISUAL REPRESENTATION.

1. Practice of assigning labels to an image or set of images. (Manually)

2. Done by: Humans/Team of humans (e.g., shape and label of images)
3. For training dataset for computer vision models.

How does it work?

Prepare dataset  Specify object classes  Assign labels to images  Marking objects within each
image by drawing bounding boxes  Select object class labels for each box  Export annotations in a
format that can be used as a training dataset.

Types
1. Lines – Identify Boundaries
2. Polygons – Placing dots and then manually drawing lines.
3. Markers – Specific significance. s

Bounding Boxes
Image Annotation – In Action
 Click the link: https://github.com/tzutalin/labelImg
 Reference: DiaMOS Plant: A Dataset for Diagnosis and Monitoring Plant Disease
“train_diaMOSNET.py” - Code Explanation
 Reference: https://github.com/mallociFrancesca/leaf-disease-toolbox#dataset
 Open the code named: “train_diaMOSNET.py”

Imports
import os
import datetime
import pandas as pd
from keras.preprocessing.image import ImageDataGenerator
from helpers.dataset import *
from helpers.multi_output_generator import MultiOutputGenerator
from helpers.utils import *
from models.classifiers import *
from models.diaMOSNet import diaMOSNet

These lines import various Python libraries and modules necessary for the project. Notably, it imports
modules from Keras for deep learning, and it imports custom modules from project-specific directories
(helpers and models) for data handling, model training, and other functionalities.

Setup
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

#SETTINGS
BATCH_SIZE = 32
X_COL = "filename"
Y_COL = ['healthy', 'pear_slug', 'leaf_spot','severity_0','severity_1',
'severity_2', 'severity_3','severity_4']
IMG_W = 224
IMG_H = 224
DATA_DIR = "./data/diaMOSPlant/img/"
LABEL_DIR = "./data/diaMOSPlant/csv/"
OUT_PATH = './out/diaMOSPlant/'

networks = ['vgg16','vgg19','resNet50','inceptionV3', 'mobileNetV2',

'efficientNetB0']

 This sets the environment variable ‘CUDA_VISIBLE_DEVICES’ to ‘-1’. In deep learning,

CUDA is a parallel computing platform that enables GPU acceleration for training neural
networks. By setting it to ‘-1’, you’re essentially telling your system not to use any available
GPUs for this project. This can be useful if you want to run your code on a CPU or if you don’t
have access to a GPU.
 BATCH_SIZE: This is the number of data samples (in this case, likely images) that will be
processed in a single iteration during training. A batch size of 32 means that 32 images will be
processed together in each training step.
 X_COL: This seems to be the name of a column in a dataset that holds filenames or paths to
images.
  Y_COL: This is a list of column names. It probably represents the target labels or classes for your
machine learning model. In this case, it appears to be related to plant health classification, with
severity levels (e.g., ‘severity_0’, ‘severity_1’).
 IMG_W and IMG_H: These are the dimensions (width and height) to which the input images will
be resized. In this case, the images will be resized to 224x224 pixels, which is a common size for
many pre-trained deep learning models.
 DATA_DIR: This is the directory path where your image data is stored. The images used for
training and testing will be loaded from this directory.
 LABEL_DIR: This is the directory path where your label data (possibly in CSV format) is stored.
It contains information about the classes and severity levels for each image.
 OUT_PATH: This is the directory path where the output or results of your project will be saved.
This might include trained model checkpoints, logs, or any other output files.
 The NETWORKS list is defined but not explained here. It likely contains the names of deep
learning model architectures that will be used later in the script.

The __name__ == “__main__”

if __name__ == "__main__":

This conditional statement checks if the script is being run as the main program. It ensures that the
code within the block is executed only when the script is run directly and not when it’s imported as a
module into another script.

 Split data
X_train, y_train, X_val, y_val, X_test, y_test =
shuffle_split_data(os.path.join(LABEL_DIR,'diaMOSPlant.csv'),X_COL,Y_CO
L,sep=";",validation=True)

This line calls a function called ‘shuffle_split_data’ to split the data into training, validation, and
testing sets based on a CSV file located in LABEL_DIR. The X_train, y_train, X_val, y_val,
X_test, and y_test variables hold the respective data splits.

 Concatenate the Datasets

train = pd.concat([X_train, y_train], axis=1, sort=False)
val = pd.concat([X_val, y_val], axis=1, sort=False)
test = pd.concat([X_test, y_test], axis=1, sort=False)

In this code, you are using the Pandas library to combine two DataFrames: X_train (features) and
y_train (labels) to create a single DataFrame called train. You do the same for the validation and test
sets.

 pd.concat: This function is used to concatenate pandas objects (a sequence or mapping of

Series or DataFrame objects) along a specified axis (in this case, axis=1, meaning columns).
It effectively combines the columns from X_train and y_train into a single DataFrame.
  Data-saved path
data_save_path = OUT_PATH + '/csv/' +
datetime.datetime.now().strftime('%d_%m%_Y') + '/'
if not(os.path.exists(data_save_path)):
os.makedirs(data_save_path)

This code creates a directory path for saving data based on the current date and sets it to the
variable data_save_path. It then checks if this directory path exists; if not, it creates the directory
using os.makedirs.

 Convert Dataset to Dataframe

df_train = pd.DataFrame(train)
df_val = pd.DataFrame(val)
df_test = pd.DataFrame(test)

These lines convert the training, validation, and test datasets into Pandas DataFrames.

 Save the Dataframes to CSV

df_train.to_csv(os.path.join(data_save_path,'train.csv'), index=False)
df_val.to_csv(os.path.join(data_save_path,'val.csv'), index=False)
df_test.to_csv(os.path.join(data_save_path,'test.csv'), index=False)

These lines save the training, validation, and test dataframe as CSV files in the directory specified by
data_save_path. No index.

 Deep Learning Training Model Loop Proper

for net in networks:

This loop appears to be responsible for training machine learning models with different architectures
for each net in the networks list (e.g., VGG16, ResNet50).

 Creating Data Generators

#base_path = os.path.join(BASE_PATH, net)

train_datagen = ImageDataGenerator(
horizontal_flip=True,
vertical_flip=True,
rescale=1/255
)

val_datagen = ImageDataGenerator(
rescale=1./255,
)

test_datagen = ImageDataGenerator(
rescale=1./255,
)
These lines create data generators for training, validation, and testing data. Data augmentation
techniques are applied to the training data, including horizontal and vertical flips. Additionally, all
data (including validation and test data) is rescaled by dividing by 255 to normalize the pixel
values.

 Defining Data Generators using MultioutputGenerator Function

train_generator = MultiOutputGenerator(
generator = train_datagen,
dataframe = train,
directory=DATA_DIR,
batch_size = BATCH_SIZE,
x_col = X_COL,
y_col = Y_COL,
class_mode="raw",
target_size = (IMG_H, IMG_W),
shuffle=False
)
val_generator = MultiOutputGenerator(
generator = val_datagen,
dataframe = val,
directory=DATA_DIR,
batch_size = BATCH_SIZE,
x_col = X_COL,
y_col = Y_COL,
class_mode="raw",
target_size = (IMG_H, IMG_W),
shuffle=False
)
test_generator = MultiOutputGenerator(
generator = test_datagen,
dataframe = test,
directory=DATA_DIR,
batch_size = 1,
x_col = X_COL,
y_col = Y_COL,
class_mode="raw",
target_size = (IMG_H, IMG_W),
shuffle=False
)

Here, a train, val, and test generators are created using the MultiOutputGenerator class, which
appears to be a custom generator for handling multi-output neural network architectures. It takes
several parameters.
 Creating and Training the Model
model = diaMOSNet(conv_base=net, shape=(IMG_H,IMG_W,3))

model.build()

model.train(train_generator = train_generator,
steps_per_epoch = int(len(train)/32),
epochs = 100,
val_generator = val_generator,
validation_steps = int(len(val)/32),
verbose = 1)

model.save_weights('_weights.h5')

model.save('_arch.h5')

The first line creates an instance of a deep learning model using the diaMOSNet class. The
architecture of the model is determined by the net variable, which corresponds to different model
architectures (e.g., VGG16, ResNet50). The shape parameter defines the input shape of the
model. 3 means the RGB channels.

The third line trains the model using the train_generator for training data and the val_generator
for validation data. It specifies the number of steps per epoch, the number of training epochs, and
the verbosity level for training updates.
“dataset.py” - Custom-made Functions
This defines functions for loading and splitting a dataset into training, validation, and testing sets.

load_data
import pandas as pd
from sklearn.model_selection import train_test_split, StratifiedShuffleSplit

def load_data(path_csv, sep=";"):

dataset = pd.DataFrame(pd.read_csv(path_csv,sep=sep))
print("\nShape dataset {}\n".format(dataset.shape))
print(dataset.head())
return dataset

Reads the CSV file located at path_csv using Pandas and stores it as a DataFrame named dataset. Prints
the shape of the dataset (number of rows and columns) and displays the first few rows of the dataset.
Returns the loaded dataset.

shuffle_split_data
def shuffle_split_data(path_csv, x_col, y_col, sep=";", validation=True, val_size=0.20,
test_size=0.10):

dataset = load_data(path_csv, sep)

X = dataset[x_col]
y = dataset[y_col]
sss1 = StratifiedShuffleSplit(n_splits=1, random_state=0, test_size=test_size)

for train_idx, test_idx in sss1.split(X, y):

X_train, X_test =X.iloc[list(train_idx)], X.iloc[list(test_idx)]
y_train, y_test =y.iloc[list(train_idx)], y.iloc[list(test_idx)]

if validation:
sss2 = StratifiedShuffleSplit(n_splits=1, random_state=0, test_size=val_size)
for train_idx, val_idx in sss2.split(X_train, y_train):
X_train, X_val =X.iloc[list(train_idx)], X.iloc[list(val_idx)]
y_train, y_val =y.iloc[list(train_idx)], y.iloc[list(val_idx)]

print("Training set: {}\n".format(len(y_train)))

print("Validation set: {}\n".format(len(y_val)))
print("Test set: {}\n".format(len(y_test)))

return X_train, y_train, X_val, y_val, X_test, y_test

print("Training set: {}\n".format(len(y_train)))

print("Test set: {}\n".format(len(y_test)))

return X_train, y_train, X_test, y_test

This function loads a dataset, shuffles it, and splits it into training, validation, and testing sets using
stratified sampling. Stratified sampling ensures that each class is proportionally represented in the splits.

It calls the load_data function to load the dataset from the specified CSV file. Separates the features (X)
and labels (y) from the dataset. Performs a stratified shuffle split on the data to create training and test
sets. It ensures that the class distribution is maintained in the splits. If validation is set to True, it further
splits the training set into a training set and a validation set using stratified sampling. Prints the sizes of
the training, validation, and test sets. Returns the features and labels for the training, validation, and test
sets.

X = dataset[X_COL]: This line extracts the "filename" column from the dataset DataFrame and assigns
it to the variable X. X_COL is assumed to be a variable holding the column name, and this line essentially
creates a pandas Series containing the file names.

y = dataset[Y_COL]: This line extracts the "Disease" and "Severity" columns from the dataset
DataFrame and assigns them to the variable y. Y_COL is assumed to be a list of column names, and this
line creates a pandas DataFrame containing these columns.

sss1 = StratifiedShuffleSplit(n_splits=1, random_state=0, test_size=0.1): This line creates an instance

of StratifiedShuffleSplit with the following parameters:

n_splits=1: It specifies that only one split will be created.

random_state=0: It sets the random seed for reproducibility.
test_size=0.1: It indicates that 10% of the data will be used as the test set.

The for loop that follows is used to perform the stratified sampling. It iterates once (due to n_splits=1)
and splits the data into training and test sets while maintaining the same class distribution as specified by
y. train_idx and test_idx are arrays of indices indicating which samples are in the training and test sets,
respectively. The loop prints these indices.

The result is that you have created a stratified split of your data into training and test sets, where each set
maintains a proportional representation of the different classes specified in the "Disease" and "Severity"
columns.
split_data
def split_data(path_csv, x_col, y_col, sep=";", validation=True, val_size=0.20,
test_size=0.10):

dataset = load_data(path_csv, sep)

X = dataset[x_col]
y = dataset[y_col]

sss1 = train_test_split(n_splits=1, random_state=0, test_size=test_size)

for train_idx, test_idx in sss1.split(X, y):
X_train, X_test =X.iloc[list(train_idx)], X.iloc[list(test_idx)]
y_train, y_test =y.iloc[list(train_idx)], y.iloc[list(test_idx)]

if validation:
sss2 = train_test_split(n_splits=1, random_state=0, test_size=val_size)
for train_idx, val_idx in sss2.split(X_train, y_train):
X_train, X_val =X.iloc[list(train_idx)], X.iloc[list(val_idx)]
y_train, y_val =y.iloc[list(train_idx)], y.iloc[list(val_idx)]

return X_train, y_train, X_val, y_val, X_test, y_test

return X_train, y_train, X_test, y_test

This function is similar to shuffle_split_data but uses simple random splitting (without stratified
sampling) instead of stratified shuffling.
 “multi_output_generator.py” - Custom-made Functions
This code defines a custom data generator class called MultiOutputGenerator. This class is designed to
work with the Keras deep learning framework and facilitate the generation of multi-output data for
training deep learning models.

MultiOutputGenerator Class
import numpy as np

class MultiOutputGenerator():

This class is defined and named MultiOutputGenerator.

Initializing MultiOutputGenerator Class
The __init__ method is the constructor method for the MultiOutputGenerator class, and it initializes
various attributes and settings for the generator.

def __init__(self,
generator,
dataframe,
directory=None,
image_data_generator=None,
x_col="filename",
y_col="class",
weight_col=None,
target_size=(256, 256),
color_mode='rgb',
classes=None,
class_mode='categorical',
Properties batch_size=32,
shuffle=False,
seed=None,
data_format='channels_last',
save_to_dir=None,
save_prefix='',
save_format='png',
subset=None,
interpolation='nearest',
dtype='float32',
validate_filenames=True):
'Initialization'

self.keras_generator = generator.flow_from_dataframe(
attribute
dataframe,
directory=directory,
image_data_generator=image_data_generator,
x_col=x_col,
y_col=y_col,
weight_col=weight_col,
target_size=target_size,
color_mode=color_mode,
classes=classes,
class_mode=class_mode,
batch_size=batch_size,
shuffle=shuffle,
seed=seed,
getGenerator Function
def getGenerator(self):

while True:

gnext = self.keras_generator.next()
y = np.float32(gnext[1])
yield (gnext[0],{'disease': y[:,[0,1,2]], 'severity': y[:, [3,4,5,6,7]]})

 This method defines a generator function that generates batches of data suitable for multi-output tasks. It's intended for
use during training of deep learning models.
 The while True loop indicates that this generator will keep generating data indefinitely.
 Inside the loop:
 gnext is used to get the next batch of data from self.keras_generator, which is the Keras generator
initialized in the constructor.
 y is calculated as the second element of gnext. It is converted to a NumPy array of type float32.
 The method yields a tuple containing two elements:
 The first element is the input data (gnext[0]), which typically represents the images.
 The second element is a dictionary with two keys, 'disease' and 'severity'. These keys map to
portions of the label data y:
 'disease' corresponds to the first three elements of y.
 'severity' corresponds to the remaining elements of y.
 When this method is used in a for loop, it will provide batches of data indefinitely.

Example CSV dataset:

getNext Function

def getNext(self):
return self.keras_generator.next()

This method simply retrieves the next batch of data from self.keras_generator and returns it.
labels property of MultiOutputGenerator Class
@property
def labels(self):
if self.keras_generator.class_mode in {"multi_output", "raw"}:
return self.keras_generator._targets
else:
return self.keras_generator.classes

In Python, the @property decorator is used to define a method as a "getter" for a class attribute. This decorator allows you to
access an instance attribute as if it were a class property.

In this code:

@property is a decorator that indicates that the labels method should behave like a property when accessed, rather than a
regular method. It allows you to access the labels attribute of an instance of the MultiOutputGenerator class as if it were a
simple attribute, without using parentheses.

def labels(self): defines the labels method. It takes one argument, self, which is a reference to the instance of the class.

Inside the method:

It checks the class_mode attribute of the self.keras_generator object. self.keras_generator is an instance variable that refers to
a Keras data generator created during the initialization of the MultiOutputGenerator class.

If the class_mode is either "multi_output" or "raw," it returns self.keras_generator._targets. _targets is an attribute of the
Keras data generator that typically represents the target labels for multi-output tasks.

If the class_mode is not "multi_output" or "raw," it returns self.keras_generator.classes. classes is another attribute of the
Keras data generator that stores class labels for classification tasks.

So, in practical terms, when you access the ‘labels’ attribute of an instance of the ‘MultiOutputGenerator class’ like this:

Python will execute the labels method, and the return value will depend on the class_mode of the associated Keras data
generator. The @property decorator makes it possible to access this information in a clean and attribute-like manner.
“diaMOSNet.py” - Custom-made Functions
This code defines a custom data generator class called MultiOutputGenerator. This class is designed to
work with the Keras deep learning framework and facilitate the generation of multi-output data for
training deep learning models.

MultiOutputGenerator Class
import numpy as np

class MultiOutputGenerator():

This class is defined and named MultiOutputGenerator.

Model Building using Keras
1. Pretrained Model
Syntax: model = VGG16(weights='imagenet', include_top=False)

1. Weights (kernel & bias):

 In a neural network, weights are the parameters that the model uses to make predictions.
These weights are learned during the training process, where the model adjusts them to
minimize the difference between its predictions and the actual target values.
 Weights are crucial because they determine how information flows through the
network and how the network transforms input data into output predictions. These
weights are typically represented as matrices and tensors.
2. 'imagenet':
 'imagenet' in this context refers to the ImageNet dataset. ImageNet is a large dataset
containing millions of labeled images across thousands of categories or classes. It's a
widely used benchmark dataset for image classification tasks.
 Pre-trained models on ImageNet have already been trained on this extensive dataset.
During this training, the models have learned to recognize various features, patterns, and
objects within images. As a result, the weights of the neural network layers have been
adjusted to capture this knowledge.
 When you specify weights = 'imagenet' while loading a pre-trained neural network
model, you are essentially telling the framework (e.g., Keras) to initialize the model with
the weights learned from training on ImageNet. This means the model starts with weights
that have already learned general features from a wide range of images, such as edges,
textures, shapes, and even high-level concepts like "cat" or "dog."

Using pre-trained weights from a dataset like ImageNet is a form of transfer learning. Instead of
training a deep neural network from scratch on a specific task, you can start with these pre-trained
weights as a foundation and fine-tune the model on your own dataset or specific task. This can save a
significant amount of training time and data, especially when you have a limited dataset for your task.

2. Transfer learning & fine-tuning

Transfer learning consists of taking features learned on one problem, and leveraging them on a new,
similar problem. For instance, features from a model that has learned to identify racoons may be useful to
kick-start a model meant to identify tanukis.

Transfer learning is usually done for tasks where your dataset has too little data to train a full-scale model
from scratch.

The most common incarnation of transfer learning in the context of deep learning is the following
workflow:
1. Take layers from a previously trained model.
2. Freeze them, so as to avoid destroying any of the information they contain during future training
rounds.
3. Add some new, trainable layers on top of the frozen layers. They will learn to turn the old features
into predictions on a new dataset.
4. Train the new layers on your dataset.
A last, optional step, is fine-tuning, which consists of unfreezing the entire model you obtained above (or
part of it), and re-training it on the new data with a very low learning rate. This can potentially achieve
meaningful improvements, by incrementally adapting the pretrained features to the new data.

 tf.keras.Input
Input() is used to instantiate a Keras tensor.

Syntax: tf.keras.Input(shape=None, batch_size=None, name=None, dtype=None,

sparse=None, tensor=None, ragged=None,
type_spec=None, **kwargs)

A Keras tensor is a symbolic tensor-like object, which we augment with certain attributes that allow us to
build a Keras model just by knowing the inputs and outputs of the model.

Arguments
 shape: A shape tuple (integers), not including the batch size. For
instance, shape=(32,) indicates that the expected input will be batches
of 32-dimensional vectors. Elements of this tuple can be None; 'None'
elements represent dimensions where the shape is not known.
 batch_size: optional static batch size (integer).
 name: An optional name string for the layer. Should be unique in a model
(do not reuse the same name twice). It will be autogenerated if it isn't
provided.
 dtype: The data type expected by the input, as a string
(float32, float64, int32...)
 sparse: A boolean specifying whether the placeholder to be created is
sparse. Only one of 'ragged' and 'sparse' can be True. Note that,
if sparse is False, sparse tensors can still be passed into the input -
they will be densified with a default value of 0.
 tensor: Optional existing tensor to wrap into the Input layer. If set,
the layer will use the tf.TypeSpec of this tensor rather than creating a
new placeholder tensor.
 ragged: A boolean specifying whether the placeholder to be created is
ragged. Only one of 'ragged' and 'sparse' can be True. In this case,
values of 'None' in the 'shape' argument represent ragged dimensions.
For more information about RaggedTensors, see this guide.
 type_spec: A tf.TypeSpec object to create the input placeholder from.
When provided, all other args except name must be None.
 **kwargs: deprecated arguments support.
Supports batch_shape and batch_input_shape.

For instance, if a, b and c are Keras tensors, it becomes possible to do: model = Model(input=[a, b],
output=c)

 tf.keras.layers.BatchNormalization
Batch normalization is a technique used in deep neural networks, including those implemented using
Keras, to stabilize and accelerate the training of neural networks. It normalizes the activations of a layer
by adjusting and scaling them, making the training process more efficient and reducing the likelihood of
overfitting.
3. tf.keras.Model
A model grouping layers into an object with training/inference features.

Syntax: tf.keras.Model(*args, **kwargs)

Args
inputs The input(s) of the model: a keras.Input object or a combination
of keras.Input objects in a dict, list or tuple.
outputs The output(s) of the model: a tensor that originated
from keras.Input objects or a combination of such tensors in a
dict, list or tuple. See Functional API example below.
name String, the name of the model.

There are two ways to instantiate a Model:

1. With the "Functional API", where you start from Input, you chain layer calls to specify the model's
forward pass, and finally you create your model from inputs and outputs:

import tensorflow as tf

inputs = tf.keras.Input(shape=(3,))
x = tf.keras.layers.Dense(4, activation=tf.nn.relu)(inputs)
outputs = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs)

4. compile()
In Keras, the compile method is used to configure the learning process of a neural network model before
training it. When you compile a Keras model, you specify several important settings that define how the
model should be trained.

Syntax: model.compile(optimizer="rmsprop", loss=None, metrics=None,

loss_weights=None, weighted_metrics=None,
run_eagerly=None, steps_per_execution=None,
jit_compile=None, pss_evaluation_shards=0, **kwargs)

Arguments
 optimizer: String (name of optimizer) or optimizer instance.
See tf.keras.optimizers.
 loss: Loss function. May be a string (name of loss function), or
a tf.keras.losses.Loss instance. See tf.keras.losses. A loss function is
any callable with the signature loss = fn(y_true, y_pred),
where y_true are the ground truth values, and y_pred are the model's
predictions. y_true should have shape (batch_size, d0, .. dN) (except in
the case of sparse loss functions such as sparse categorical
crossentropy which expects integer arrays of shape (batch_size, d0, ..
dN-1)). y_pred should have shape (batch_size, d0, .. dN). The loss
function should return a float tensor. If a custom Loss instance is used
and reduction is set to None, return value has shape (batch_size, d0, ..
dN-1) i.e. per-sample or per-timestep loss values; otherwise, it is a
scalar. If the model has multiple outputs, you can use a different loss
on each output by passing a dictionary or a list of losses. The loss
value that will be minimized by the model will then be the sum of all
individual losses, unless loss_weights is specified.
 metrics: List of metrics to be evaluated by the model during training
and testing. Each of this can be a string (name of a built-in function),
function or a tf.keras.metrics.Metric instance. See tf.keras.metrics.
Typically you will use metrics=['accuracy']. A function is any callable
with the signature result = fn(y_true, y_pred). To specify different
metrics for different outputs of a multi-output model, you could also
pass a dictionary, such as metrics={'output_a':'accuracy', 'output_b':
['accuracy', 'mse']}. You can also pass a list to specify a metric or a
list of metrics for each output, such as metrics=[['accuracy'],
['accuracy', 'mse']] or metrics=['accuracy', ['accuracy', 'mse']]. When
you pass the strings 'accuracy' or 'acc', we convert this to one
of tf.keras.metrics.BinaryAccuracy, tf.keras.metrics.CategoricalAccuracy
, tf.keras.metrics.SparseCategoricalAccuracy based on the shapes of the
targets and of the model output. We do a similar conversion for the
strings 'crossentropy' and 'ce' as well. The metrics passed here are
evaluated without sample weighting; if you would like sample weighting
to apply, you can specify your metrics via the weighted_metrics argument
instead.
 loss_weights: Optional list or dictionary specifying scalar coefficients
(Python floats) to weight the loss contributions of different model
outputs. The loss value that will be minimized by the model will then be
the weighted sum of all individual losses, weighted by
the loss_weights coefficients. If a list, it is expected to have a 1:1
mapping to the model's outputs. If a dict, it is expected to map output
names (strings) to scalar coefficients.
 weighted_metrics: List of metrics to be evaluated and weighted
by sample_weight or class_weight during training and testing.
 run_eagerly: Bool. If True, this Model's logic will not be wrapped in
a tf.function. Recommended to leave this as None unless
your Model cannot be run inside a tf.function. run_eagerly=True is not
supported when using tf.distribute.experimental.ParameterServerStrategy.
Defaults to False.
 steps_per_execution: Int. The number of batches to run during
each tf.function call. Running multiple batches inside a
single tf.function call can greatly improve performance on TPUs or small
models with a large Python overhead. At most, one full epoch will be run
each execution. If a number larger than the size of the epoch is passed,
the execution will be truncated to the size of the epoch. Note that
if steps_per_execution is set
to N, Callback.on_batch_begin and Callback.on_batch_end methods will
only be called every N batches (i.e. before/after
each tf.function execution). Defaults to 1.
 jit_compile: If True, compile the model training step with XLA. XLA is
an optimizing compiler for machine learning. jit_compile is not enabled
for by default. Note that jit_compile=True may not necessarily work for
all models. For more information on supported operations please refer to
the XLA documentation. Also refer to known XLA issues for more details.
 pss_evaluation_shards: Integer or 'auto'. Used
for tf.distribute.ParameterServerStrategy training only. This arg sets
the number of shards to split the dataset into, to enable an exact
visitation guarantee for evaluation, meaning the model will be applied
to each dataset element exactly once, even if workers fail. The dataset
must be sharded to ensure separate workers do not process the same data.
The number of shards should be at least the number of workers for good
performance. A value of 'auto' turns on exact evaluation and uses a
heuristic for the number of shards based on the number of workers. 0,
 meaning no visitation guarantee is provided. NOTE: Custom
implementations of Model.test_step will be ignored when doing exact
evaluation. Defaults to 0.
 **kwargs: Arguments supported for backwards compatibility only.

Model Training using Scikit-Learn fit()

fit() method
An estimator is an object that fits a model based on some training data and is capable of inferring some
properties on new data. It can be, for instance, a classifier or a regressor. All estimators implement the fit
method:

Syntax: estimator.fit(X, y)

This is used to estimate some parameters in the model. This is implemented in the fit() method.

The fit() method takes the training data as arguments, which can be one array in the case of unsupervised
learning, or two arrays in the case of supervised learning.

Note that the model is fitted using X and y, but the object holds no reference to X and y. There are,
however, some exceptions to this, as in the case of precomputed kernels where this data must be stored
for use by the predict method.

Parameter/Keyword
Description Example Value
Argument
X Training data features X_train
y Training data labels y_train
sample_weight Optional weights for samples sample_weights
Number of samples in each
batch_size 32
training batch
epochs Number of training epochs 100
Verbosity level for training
verbose 1 (Verbose), 0 (Silent)
output
List of callback functions for [early_stopping,
callbacks
monitoring model_checkpoint]
Validation data for model
validation_data (X_val, y_val)
evaluation
Fraction of training data used 0.2 (20% for
validation_split
for validation validation)
Whether to shuffle the
shuffle True or False
training data
Optional class weights for
class_weight class_weights
imbalanced datasets
Epoch at which to start
initial_epoch 5
training (useful for resuming)
Number of batches to use for
steps_per_epoch 100
each epoch
Number of batches to use for
validation_steps 50
validation
validation_freq Number of training epochs 5
 between validation runs
Maximum size of generator
max_queue_size 10
queue
Number of data loading
workers 4
processes (for generators)
Whether to use multiprocessing
use_multiprocessing True or False
for data loading
Whether to return training
return_dict True or False
results as a dictionary

The Callback API

A callback is an object that can perform actions at various stages of training (e.g. at the start or end of an
epoch, before or after a single batch, etc).

You can use callbacks to:

 Write TensorBoard logs after every batch of training to monitor your metrics
 Periodically save your model to disk
 Do early stopping
 Get a view on internal states and statistics of a model during training
 ...and more

You can pass a list of callbacks (as the keyword argument callbacks) to the .fit() method of a model:

my_callbacks = [
tf.keras.callbacks.EarlyStopping(patience=2),
tf.keras.callbacks.ModelCheckpoint(filepath='model.{epoch:02d}-
{val_loss:.2f}.h5'),
tf.keras.callbacks.TensorBoard(log_dir='./logs'),
]
model.fit(dataset, epochs=10, callbacks=my_callbacks)

 tf.keras.callbacks.EarlyStopping
Syntax:tf.keras.callbacks.EarlyStopping(monitor="val_loss",min_delta=0,
patience=0, verbose=0, mode="auto",
baseline=None,
restore_best_weights=False,
start_from_epoch=0,)

Stop training when a monitored metric has stopped improving.

Assuming the goal of a training is to minimize the loss. With this, the metric to be monitored would
be 'loss', and mode would be 'min'. A model.fit() training loop will check at end of every epoch whether
the loss is no longer decreasing, considering the min_delta and patience if applicable. Once it's found no
longer decreasing, model.stop_training is marked True and the training terminates.

The quantity to be monitored needs to be available in logs dict. To make it so, pass the loss or metrics
at model.compile().

Arguments
 monitor: Quantity to be monitored.
  min_delta: Minimum change in the monitored quantity to qualify as an
improvement, i.e. an absolute change of less than min_delta, will count
as no improvement.
 patience: Number of epochs with no improvement after which training will
be stopped.
 verbose: Verbosity mode, 0 or 1. Mode 0 is silent, and mode 1 displays
messages when the callback takes an action.
 mode: One of {"auto", "min", "max"}. In min mode, training will stop
when the quantity monitored has stopped decreasing; in "max" mode it
will stop when the quantity monitored has stopped increasing;
in "auto" mode, the direction is automatically inferred from the name of
the monitored quantity.
 baseline: Baseline value for the monitored quantity. Training will stop
if the model doesn't show improvement over the baseline.
 restore_best_weights: Whether to restore model weights from the epoch
with the best value of the monitored quantity. If False, the model
weights obtained at the last step of training are used. An epoch will be
restored regardless of the performance relative to the baseline. If no
epoch improves on baseline, training will run for patience epochs and
restore weights from the best epoch in that set.
 start_from_epoch: Number of epochs to wait before starting to monitor
improvement. This allows for a warm-up period in which no improvement is
expected and thus training will not be stopped.

 tf.keras.callbacks.ModelCheckpoint
Syntax:tf.keras.callbacks.ModelCheckpoint(filepath,
monitor: str = "val_loss",
verbose: int = 0,
save_best_only: bool = False,
save_weights_only: bool = False,
mode: str = "auto",
save_freq="epoch",
options=None,
initial_value_threshold=None,
**kwargs)

Callback to save the Keras model or model weights at some frequency.

ModelCheckpoint callback is used in conjunction with training using model.fit() to save a model or
weights (in a checkpoint file) at some interval, so the model or weights can be loaded later to continue the
training from the state saved.

A few options this callback provides include:

 Whether to only keep the model that has achieved the "best performance" so far, or whether to
save the model at the end of every epoch regardless of performance.
 Definition of 'best'; which quantity to monitor and whether it should be maximized or minimized.
 The frequency it should save at. Currently, the callback supports saving at the end of every epoch,
or after a fixed number of training batches.
 Whether only weights are saved, or the whole model is saved.

Note: If you get WARNING:tensorflow:Can save best model only with <name> available,
skipping see the description of the monitor argument for details on how to get this right.
Example

model.compile(loss=..., optimizer=...,
metrics=['accuracy'])

EPOCHS = 10
checkpoint_filepath = '/tmp/checkpoint'
model_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(
filepath=checkpoint_filepath,
save_weights_only=True,
monitor='val_accuracy',
mode='max',
save_best_only=True)

# Model weights are saved at the end of every epoch, if it's the best seen
# so far.
model.fit(epochs=EPOCHS, callbacks=[model_checkpoint_callback])

# The model weights (that are considered the best) are loaded into the
# model.
model.load_weights(checkpoint_filepath)

Arguments
 filepath: string or PathLike, path to save the model file. e.g. filepath
= os.path.join(working_dir, 'ckpt', file_name). filepath can contain
named formatting options, which will be filled the value of epoch and
keys in logs (passed in on_epoch_end). For example:
if filepath is weights.{epoch:02d}-{val_loss:.2f}.hdf5, then the model
checkpoints will be saved with the epoch number and the validation loss
in the filename. The directory of the filepath should not be reused by
any other callbacks to avoid conflicts.
 monitor: The metric name to monitor. Typically the metrics are set by
the Model.compile method. Note:
o Prefix the name with "val_" to monitor validation metrics.
o Use "loss" or "val_loss" to monitor the model's total loss.
o If you specify metrics as strings, like "accuracy", pass the same
string (with or without the "val_" prefix).
o If you pass metrics.Metric objects, monitor should be set
to metric.name
o If you're not sure about the metric names you can check the
contents of the history.history dictionary returned by history =
model.fit()
o Multi-output models set additional prefixes on the metric names.
o __ verbose__: Verbosity mode, 0 or 1. Mode 0 is silent, and mode 1
displays messages when the callback takes an action.
o save_best_only: if save_best_only=True, it only saves when the
model is considered the "best" and the latest best model according
to the quantity monitored will not be overwritten.
If filepath doesn't contain formatting options
like {epoch} then filepath will be overwritten by each new better
model.
o mode: one of {'auto', 'min', 'max'}. If save_best_only=True, the
decision to overwrite the current save file is made based on
either the maximization or the minimization of the monitored
quantity. For val_acc, this should be max, for val_loss this
should be min, etc. In auto mode, the mode is set to max if the
 quantities monitored are 'acc' or start with 'fmeasure' and are
set to min for the rest of the quantities.
o save_weights_only: if True, then only the model's weights will be
saved (model.save_weights(filepath)), else the full model is saved
(model.save(filepath)).
o save_freq: 'epoch' or integer. When using 'epoch', the callback
saves the model after each epoch. When using integer, the callback
saves the model at end of this many batches. If the Model is
compiled with steps_per_execution=N, then the saving criteria will
be checked every Nth batch. Note that if the saving isn't aligned
to epochs, the monitored metric may potentially be less reliable
(it could reflect as little as 1 batch, since the metrics get
reset every epoch). Defaults to 'epoch'.
o options: Optional tf.train.CheckpointOptions object
if save_weights_only is true or
optional tf.saved_model.SaveOptions object if save_weights_only is
false.
o initial_value_threshold: Floating point initial "best" value of
the metric to be monitored. Only applies if save_best_value=True.
Only overwrites the model weights already saved if the performance
of current model is better than this value.
o **kwargs: Additional arguments for backwards compatibility.
Possible key is period.

 tf.keras.callbacks.ReduceLROnPlateau

Syntax:
tf.keras.callbacks.ReduceLROnPlateau(
monitor="val_loss",
factor=0.1,
patience=10,
verbose=0,
mode="auto",
min_delta=0.0001,
cooldown=0,
min_lr=0,
**kwargs
)

Reduce learning rate when a metric has stopped improving.

Models often benefit from reducing the learning rate by a factor of 2-10 once learning stagnates. This
callback monitors a quantity and if no improvement is seen for a 'patience' number of epochs, the learning
rate is reduced.

Example

reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2,

patience=5, min_lr=0.001)
model.fit(X_train, Y_train, callbacks=[reduce_lr])

Arguments
 monitor: quantity to be monitored.
  factor: factor by which the learning rate will be reduced. new_lr = lr *
factor.
 patience: number of epochs with no improvement after which learning rate
will be reduced.
 verbose: int. 0: quiet, 1: update messages.
 mode: one of {'auto', 'min', 'max'}. In 'min' mode, the learning rate
will be reduced when the quantity monitored has stopped decreasing;
in 'max' mode it will be reduced when the quantity monitored has stopped
increasing; in 'auto' mode, the direction is automatically inferred from
the name of the monitored quantity.
 min_delta: threshold for measuring the new optimum, to only focus on
significant changes.
 cooldown: number of epochs to wait before resuming normal operation
after lr has been reduced.
 min_lr: lower bound on the learning rate.

Data Generators
Keras ImageDataGenerator
Keras’ ImageDataGenerator class allows the users to perform image augmentation while training the model.

Keras’ ImageDataGenerator class provide three different functions to loads the image dataset in memory and generates batches of
augmented data. These three functions are:

1. .flow()
2. .flow_from_directory() – images are distributed on folders categorized based on their classes.
3. .flow_from_dataframe() - images are contained in a excel file in CSV or JSON format.

Each of these function is achieving the same task to load the image dataset in memory and generates batches of augmented data,
but the way to accomplish the task is different.
 .flow_from_dataframe()
Python Methods
__init__ method
In Python, __init__ is a special method known as the constructor. It is automatically called when a new instance (object) of a
class is created. The __init__ method allows you to initialize the attributes (variables) of an object.

Initialization: The primary purpose of __init__ is to initialize the attributes (variables) of an object with specific values. These
attributes are defined within the class and represent the characteristics or properties of the object.

Self Parameter: The __init__ method takes at least one parameter, conventionally named self, as its first parameter. This
parameter refers to the instance of the class that is being created. You use self to access and set attributes within the class.

Custom Parameters: Besides self, you can define additional parameters in the __init__ method to pass values when creating an
instance of the class. These parameters are used to set the initial values of the object's attributes.

Attribute Assignment: Inside the __init__ method, you typically assign values to the instance's attributes using
self.attribute_name = initial_value. This allows you to initialize the object's state.

For example:

class Person:
def __init__(self, name, age):
self.name = name
self.age = age

# Creating an instance of the Person class

person1 = Person("Alice", 30)

# Accessing the attributes of the instance

print(person1.name) # Output: Alice
print(person1.age) # Output: 30

In this example, the __init__ method accepts two parameters, name and age, and assigns them to the name and age attributes of
the Person object person1. When you create person1, the __init__ method is automatically called with the provided values,
initializing the object's state.
StratifiedShuffleSplit
Provides train/test indices to split data in train/test sets.

StratifiedShuffleSplit is a method for splitting a dataset into training and testing (or validation) subsets while ensuring that the
class distribution in each subset remains representative of the original dataset. It's a variation of the traditional train-test split
method that is particularly useful when dealing with imbalanced datasets, where some classes have significantly fewer samples
than others.

Syntax:

class sklearn.model_selection.StratifiedShuffleSplit(n_splits=10, *, test_size=None, train_size=None, random_state=None)

Here's how StratifiedShuffleSplit works:

Stratification: It takes into account the class labels of the samples. In a classification problem, each sample is associated with a
class label. Stratified sampling ensures that the proportion of each class in the original dataset is preserved in both the training
and testing subsets.

Shuffling: Before splitting, the dataset is shuffled randomly. Shuffling helps ensure that the samples are not ordered in any
particular way that might affect the split's representativeness.

Splitting: The dataset is divided into two parts: the training subset and the testing (or validation) subset. The division is done in
such a way that it maintains the same class distribution as the original dataset. This means that if, for example, 20% of the
samples in the original dataset belong to Class A, then approximately 20% of the samples in both the training and testing subsets
will also belong to Class A.

By using StratifiedShuffleSplit, you avoid situations where one of the classes is underrepresented or entirely missing in either the
training or testing subsets. This is crucial for building and evaluating machine learning models because an imbalanced
distribution can lead to biased results.

Parameters:
 n_splits : int, default=10
Number of re-shuffling & splitting iterations.
 test_size : float or int, default=None
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents
the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it
will be set to 0.1.
 train_size : float or int, default=None
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int,
represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.
 random_state : int, RandomState instance or None, default=None
Controls the randomness of the training and testing indices produced. Pass an int for reproducible output across multiple
function calls.

Split
Generate indices to split data into training and test set.

Syntax:

split(X, y, groups=None)

Parameters:
 X: array-like of shape (n_samples, n_features)
Training data, where n_samples is the number of samples and n_features is the number of features.
Note that providing y is sufficient to generate the splits and hence np.zeros(n_samples) may be used as a placeholder
for X instead of actual training data.

 Y: array-like of shape (n_samples,) or (n_samples, n_labels)

The target variable for supervised learning problems. Stratification is done based on the y labels.

 groups : object
Always ignored, exists for compatibility.

Yields:
 train : ndarray
The training set indices for that split.
 test : ndarray
The testing set indices for that split.