DEEP LEARNING LAB (MAT6007)

M.Sc. DATA SCIENCE

RECORD

NAME:

REG. NO:

SLOT:

FACULTY NAME:

SEMESTER: FALL SEMESTER 2023-24

SCHOOL OF ADVANCED SCIENCES

CHENNAI – 600127

TAMILNADU

VIT- A Place to learn; A Chance to grow

 SCHOOL OF ADVANCED SCIENCES

DEEP LEARNING LAB (MAT6007)

REGISTRATION NUMBER ___________________________

CERTIFIED THAT THIS IS A BONAFIDE RECORD WORK DONE BY

______________________________________ OF M.Sc. DATA SCIENCE CLASS DURING THE

YEAR 2023-24 AT VIT UNIVERSITY, CHENNAI, 600127.

FACULTY INCHARGE

THIS RECORD IS SUBMITTED FOR THE PRACTICAL EXAMINATION HELD ON

____________________________________

SIGNATURE OF INTERNAL SIGNATURE OF EXTERNAL

EXAMINER EXAMINER
 INDEX

S.No. Date Title of Experiment Page

No.
1. Installation of Python and Exploration of
Python Packages
2. Implementation of McCulloch Pitt Neuron
model
3.
Implementation of Perceptron Model

4. Setting up a neural network in memory

5. Implementation of Convolutional Neural
Network
6. Implementation of RNN
7. Implementation of Simple LSTM
8. Implementation of Bidirectional LSTM
9. Implementation of Simple GRU
10. Implementation of Bidirectional RNN
11. Implementation of Autoencoders
12. Implementation of Generative Adversarial
Networks
13. Implementation of Traffic sign Recognition
using GTSRB Dataset
EXPERIMENT – 1

INSTALLATION OF PYTHON AND EXPLORING THE PACKAGES

AIM

To install python and explore the packages available in python.

INSTALLATION OF PYTHON

Step 1: Download Anaconda

Visit “https://www.anaconda.com/download” and download the anaconda installer for

windows (32 or 64 bit).

Step 2: Run the installer

Double click on the installer file to run the installer.

Step 3: Choose install location

The anaconda installer will open. Click “Next” on the welcome screen, click “I agree” and
choose “Just Me” and select installation location.

Step 4: Complete installation

Click “Install” button. The anaconda will be installed in few minutes. Then click “Finish”.

Step 5: Jupyter-notebook

Open the anaconda navigator and then click on “launch” on “Jupyter-notebook”. The
Jupyter notebook will open in browser. Open a new file and start coding.

EXPLORING THE PACKAGES

 After installing anaconda, open the command prompt.

 Create an environment using “conda create –name Arun python=3.9”. This command will
create an environment named Arun.
 Then activate the environment using the command “Activate Arun”.
 For installing the packages, use “pip install package_name” for example, “pip install
keras” will install “keras” package in python.
 Similarly, we can install other packages like pandas, numpy, scikit-learn etc.

EXPLORING “KERAS” PACKAGE IN PYTHON

Keras is a powerful and easy-to-use deep learning library forTheano and TensorFlow that provides
a high-level neural networks API to develop and evaluate deep learning models.

Few functions from the “keras” package are given below:

1. Loading a dataset.

Keras datasets
from tensorflow.keras.datasets import boston_housing, mnist, cifar10, imdb
(x_train,y_train),(x_test,y_test) = mnist.load_data()
(x_train2,y_train2),(x_test2,y_test2) = boston_housing.load_data()
(x_train3,y_train3),(x_test3,y_test3) = cifar10.load_data()
(x_train4,y_train4),(x_test4,y_test4) = imdb.load_data(num_words=20000)
num_classes = 10

Other datasets
from urllib.request import urlopen
data = np.loadtxt(urlopen(“http://archive.ics.uci.edu/ml/machine-learning-
databases/pima-indians-di abetes/pima-indians-diabetes.data”),delimiter= ”,”)
X = data[:,0:8]
y = data [:,8]

General syntax
from keras.datasets import your_dataset_module
(x_train, y_train), (x_test, y_test) = your_dataset_module.load_data()
In this example, replace your_dataset_module with the specific dataset module you want
to use. For example, common modules include mnist, cifar10, imdb, etc.

2. Preprocessing

Sequence padding
from tensorflow.keras.preprocessing import sequence
x_train4 = sequence.pad_sequences(x_train4,maxlen=80)
x_test4 = sequence.pad_sequences(x_test4,maxlen=80)

General syntax
from keras.preprocessing.sequence import pad_sequences
sequences = [
[1, 2, 3, 4],
[5, 6, 7],
[8, 9]
]
maxlen = 10
padded_sequences = pad_sequences(sequences, maxlen=maxlen, padding='pre',
truncating='pre', value=0.0)

In this example, sequences is a list of lists, where each inner list represents a sequence.
The pad_sequences function is then used to pad or truncate these sequences to a specified
length (maxlen). The padding and truncating parameters determine whether padding or
truncation occurs at the beginning or end of each sequence. Adjust the maxlen parameter
according to the desired length for your sequences. The value parameter allows you to
specify the value used for padding.

One-Hot encoding
from tensorflow.keras.utils import to_categorical
Y_train = to_categorical(y_train, num_classes)
Y_test = to_categorical(y_test, num_classes)
Y_train3 = to_categorical(y_train3, num_classes)
Y_test3 = to_categorical(y_test3, num_classes)

General syntax
from keras.utils import to_categorical
labels = [0, 1, 2, 1, 0, 2]
one_hot_labels = to_categorical(labels, num_classes=num_classes)

In this example, labels is a list of categorical labels. The to_categorical function is then
used to convert these labels into one-hot encoded format. The num_classes parameter
specifies the total number of classes in your dataset. Make sure to replace num_classes
with the actual number of classes in your dataset. The resulting one_hot_labels will be a
NumPy array representing the one-hot encoded labels.

Train and Test sets

from sklearn.model_selection import train_test_split
X_train5,X_test5,y_train5,y_test5 = train_test_split(X, y, test_size=0.33,
random_state=42)

Standardization / Normalization
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler().fit(x_train2)
standardized_X = scaler.transform(x_train2)
standardized_X_test = scaler.transform(x_test2)

General syntax
from sklearn.preprocessing import StandardScaler, MinMaxScaler
# Standardization
scaler_standard = StandardScaler()
standardized_features = scaler_standard.fit_transform(features)
# Normalization (MinMax Scaling)
scaler_minmax = MinMaxScaler()
normalized_features = scaler_minmax.fit_transform(features)

In this example, you replace features with your actual dataset, whether it's a NumPy array
or a Pandas DataFrame. The StandardScaler is used for standardization (z-score
normalization), and the MinMaxScaler is used for normalization to a specific range
(usually [0, 1]).
3. Model architecture

Sequential model
from tensorflow.keras.models import Sequential
model = Sequential()

CNN
from tensorflow.keras.layers import Activation,Conv2D,MaxPooling2D,Flatten
model2.add(Conv2D(32,(3,3),padding= ,input_shape=x_train.shape[1:]))
model2.add(Activation( )) >>> model2.add(Conv2D(32,(3,3)))
model2.add(Activation('relu'))
model2.add(MaxPooling2D(pool_size=(2,2)))
model2.add(Dropout(0.25))
model2.add(Conv2D(64,(3,3), padding= ))
model2.add(Activation( ))
model2.add(Conv2D(64,(3, 3)))
model2.add(Activation( ))
model2.add(MaxPooling2D(pool_size=(2,2)))
model2.add(Dropout(0.25))
model2.add(Flatten())
model2.add(Dense(512))
model2.add(Activation( ))
model2.add(Dropout(0.5))
model2.add(Dense(num_classes))
model2.add(Activation( ))

General syntax
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense
# Create a Sequential model
model = Sequential()
# Add Convolutional layers
model.add(Conv2D(filters=32, kernel_size=(3, 3), activation='relu',
input_shape=(img_height, img_width, channels)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
# Flatten the feature maps
model.add(Flatten())
# Add Fully Connected layers
model.add(Dense(units=128, activation='relu'))
model.add(Dense(units=num_classes, activation='softmax'))
# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# Print the model summary
model.summary()
 In this example,
 Sequential: This is the Keras sequential model, a linear stack of layers.
 Conv2D: Convolutional layer for 2D spatial convolution.
 MaxPooling2D: Max pooling layer for spatial data.
 Flatten: Flattens the input. Does not affect the batch size.
 Dense: Fully connected layer.
 input_shape: Specifies the shape of the input data (height, width, channels) for the
first layer.
 activation: Activation function for the layer.
 pool_size: Size of the pooling window for max pooling.
 units: Number of neurons in the Dense layer.
 optimizer, loss, and metrics: Compilation parameters for the model.
Make sure to adjust the parameters, such as filters, kernel_size, pool_size, units, etc., based
on your specific requirements and the nature of your dataset.

RNN
from tensorflow.keras.klayers import Embedding,LSTM
model3.add(Embedding(20000,128))
model3.add(LSTM(128,dropout=0.2,recurrent_dropout=0.2))
model3.add(Dense(1,activation='sigmoid'))

General syntax
from keras.models import Sequential
from keras.layers import SimpleRNN, Dense
# Create a Sequential model
model = Sequential()
# Add SimpleRNN layer
model.add(SimpleRNN(units=50, activation='relu', input_shape=(timesteps, features)))
# Add Dense layer for output
model.add(Dense(units=num_classes, activation='softmax'))
# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# Print the model summary
model.summary()

In this example,
 Sequential: The Keras sequential model, a linear stack of layers.
 SimpleRNN: Simple recurrent layer.
 Dense: Fully connected layer.
 units: Number of units/neurons in the layer.
 activation: Activation function for the layer.
 input_shape: Shape of the input data (timesteps, features) for the first layer.
 num_classes: Number of output classes in the final Dense layer.
 optimizer, loss, and metrics: Compilation parameters for the model.

Make sure to adjust the parameters such as units, input_shape, num_classes, etc., based on
your specific requirements and the nature of your dataset.
 4. Prediction

model3.predict(x_test4, batch_size=32)
model3.predict_classes(x_test4,batch_size=32)

5. Inspect the model

model.output_shape #Model output shape
model.summary() #Model summary representation
model.get_config() #Model configuration
model.get_weights() #List all weight tensors in the model

6. Compile the model

model.compile(optimizer= ’adam’ , loss= ‘binary_crossentrophy’, metrics=[‘accuracy’])

In this example:

 optimizer: Specifies the optimization algorithm. Common choices include 'adam',

'sgd' (Stochastic Gradient Descent), 'rmsprop', etc.
 loss: Specifies the loss function to be optimized. For a classification task with multiple
classes, 'categorical_crossentropy' is often used. For binary classification,
'binary_crossentropy' is typical.
 metrics: A list of metrics to monitor during training and testing. Common metrics
include 'accuracy' for classification tasks.

You can replace 'adam', 'categorical_crossentropy', and ['accuracy'] with your preferred
optimizer, loss function, and metrics, respectively.

7. Model training

model3.fit(x_train4, y_train4, batch_size=32, epochs=15, verbose=1,

validation_data=(x_test4,y_test4))

8. Evaluate the model’s performance

score = model3.evaluate(x_test, y_test, batch_size=32)

EXPLORING “SCIKIT-LEARN” PACKAGE IN PYTHON

Scikit-learn is an open source python library that implements a range of machine learning,
preprocessing, cross-validation and visualization algorithms using a unified interface.

Few of the functions from the “scikit-learn” package are given below:

1. Importing packages

from sklearn import neighbors, datasets, preprocessing

from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

2. Load the data

iris = datasets.load_iris()

General syntax

from sklearn.datasets import load_XXXX

# Load the dataset
data = load_XXXX()
# Access features and target variables
X = data.data # Features
y = data.target # Target variable

In this example, replace "XXXX" with the specific dataset you want to load, such as
load_iris, load_digits, load_boston, etc.

3. Testing and training data

X, y = iris.data[:, :2], iris.target

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=33)

General syntax

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

In this example,

 X is the feature matrix.

 y is the target variable.
 test_size specifies the proportion of the dataset to include in the test split (e.g., 0.2
for 20%).
 random_state is an optional parameter that sets the random seed for reproducibility.

Adjust the test_size and other parameters based on your specific needs. The train_test_split
function returns the training and testing sets for both features and target variables.

4. Model fitting

lr.fit(X, y)
k_means.fit(X_train)
5. Prediction

y_pred = lr.predict(X_test)
y_pred = k_means.predict(X_test)

6. Preprocessing the data

from sklearn.preprocessing import Normalizer

scaler = Normali zer().fit(X_train)
normali zed_X = scaler.transform(X_train)
normali zed_X_test = scaler.transform(X_test)

In this example,

 X is the original feature matrix.

 You can choose between 'l1' (L1 normalization) and 'l2' (L2 normalization).
 The transform method is then used to apply the normalization to the feature matrix.

Adjust the normalization technique and other parameters based on your specific
requirements. The Normalizer class provides a flexible way to scale features individually.

7. Creating model

Support Vector Machines (SVM)

from sklearn.svm import SVC
svc = SVC(kernel= ‘linear’)

kNN
from sklearn import neighbors
knn = neighbors.KNeighborsClassifier(n_neighbors=5)

Principal Component Analysis (PCA)

from sklearn.decomposition import PCA
pca = PCA(n_components=0. 95)

General syntax
from sklearn.model_selection import train_test_split
from sklearn.YourModelModule import YourModelClass
# Replace YourModelModule and YourModelClass with the actual module and class for
your chosen algorithm
# Assuming X and y are your feature matrix and target variable
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Create an instance of your model
model = YourModelClass()
In this example,
Replace YourModelModule and YourModelClass with the actual module and class for the
algorithm you want to use. For example, if you want to use a Support Vector Machine
 (SVM), you would replace YourModelModule with svm and YourModelClass with SVC
for classification or SVR for regression.

8. Evaluating model’s performance

Classification report
from sklearn.metrics import classification_report
print(classification_report(y_test, y_pred))

Confusion matrix
from sklearn.metrics import confusion _matrix
print (confusion _matrix ( y _test, y _pred))

R2 score
from sklearn.metrics import r2_score
r2_score(y_true, y_pred)

V-measure
from sklearn.metrics import v_measure_score
metrics.v_measure_score(y_true, y_pred)

Cross-Validation
from sklearn.cross_validation import cross_val_score
print(cross_val_score(knn, X_train, y_train, cv= 4))
print(cross_val_score(lr, X, y, cv=2))

RESULT

Thus, in this experiment, the python is installed and few packages, namely, “keras” and “scikit-
learn” are explored successfully.
 EXPERIMENT NO: 2

IMPLEMENTATION OF MCCULLOCH PITT NEURON MODEL

AIM
To implement the various basic logic gates using McCulloch Pitt Neuron model.

REQUIREMENTS
Python, Numpy

PROCEDURE
STEP 1: Import the necessary library of NumPy to handle arrays and randomness.
STEP 2: Set a random seed to ensure reproducibility of random weight vector generation.
STEP 3: Create an input table that represents the input combinations for the logic gate you want
to implement (e.g., AND, OR, etc.).
STEP 4: Display the input table to see the input combinations.
STEP 5: Create a weight vector with random values sampled from {-1, 1}. The length of the
weight vector should match the number of inputs.
STEP6: Display the weight vector, which will determine the behavior of the neuron.
STEP 7: Calculate the dot product of the input table and the weight vector, resulting in a vector
of dot products for each input combination.
STEP 8: Create a function that applies a threshold to determine the binary output. If the dot product
is greater than or equal to a specified threshold, the function returns 1; otherwise, it returns 0.
STEP 9: Define the threshold value based on the logic gate you want to implement (e.g., T=2 for
an AND gate, T=1 for an OR gate, T=-1 for a NOR gate).
STEP 10: Iterate through the dot product vector and apply the threshold function to each element.
Print the activations for each input combination, which represent the gate's behavior.
By adjusting the threshold value (T) in step 9, you can easily implement different logic gates using
the same framework. The code outputs the results for the specified gate based on the chosen
threshold value.

PROGRAM
import numpy as np

# Set a random seed for reproducibility

np.random.seed(seed=0)

# Define the input table (AND function)

input_table = np.array([
[0, 0], # both no
[0, 1], # one no, one yes
[1, 0], # one yes, one no
[1, 1] # both yes
])
print(f'Input table:\n{input_table}')

# Generate a random weight vector sampled from {-1, 1}

W = np.random.choice([-1, 1], size=2)
print(f'Weight vector: {W}')

# Compute the dot product between the input vector and weight vector
dot = input_table @ W
print(f'Dot product: {dot}')

# Define the threshold activation function

def linear_threshold_gate(dot, T):
'''Returns the binary threshold output'''
return int(dot >= T)

# Compute the output based on the threshold value (T)

# FOR AND GATE:
T = 2 # Threshold for AND gate
for i in range(4):
activation = linear_threshold_gate(dot[i], T)
print(f'Activation (AND): {activation}')

#FOR OR GATE:
T = 1 # Threshold for OR gate
for i in range(4):
activation = linear_threshold_gate(dot[i], T)
print(f'Activation (OR): {activation}')

#FOR NOR GATE:

T = -1 # Threshold for NOR gate
for i in range(4):
activation = linear_threshold_gate(dot[i], T)
print(f'Activation (NOR): {activation}')

#FOR NAND GATE:

T = 2 # Threshold for NAND gate
for i in range(4):
activation = linear_threshold_gate(dot[i], T)
print(f'Activation (NAND): {activation}')
OUTPUT

RESULT
The McCulloch Pits Neuron model has been successfully implemented for AND, OR,NOR and
NAND functions.
EXPERIMENT NO:3

IMPLEMENTATION OF PERCEPTRON MODEL

AIM

To implement Perceptron model for AND function using python.

PREREQUISITES

Python 3.9

PROCEDURE

1. Define the predict function that computes the activation of a perceptron and
returns 1 if the activation is greater than or equal to 0; otherwise, it returns 0.
2. Implement the train_perceptron function to train a perceptron on a given dataset.
Specify the initial weights, learning rate, and the number of training epochs.
3. Create training data for the AND gate, where each item is a tuple containing input
values and the corresponding target output (0 or 1).
4. Set the input size, learning rate, and the number of training epochs.

5. Call the train_perceptron function with the AND gate training data, input size,
learning rate, and epochs to obtain the final weights.
6. In the train_perceptron function, use a stopping criterion to check if the weights
have not changed between epochs. If the weights remain the same, stop
training.
7. Print the final weights after training.

8. Verify the AND gate's behavior by using the predict function with the final
weights to obtain the outputs for specific input combinations.
PROGRAM

def predict(inputs,weights):
activation = weights[0]

for i in range(len(inputs)):

activation += inputs[i] * weights[i + 1]

return 1 if activation >= 0 else 0

def train_perceptron(training_data, input_size, learning_rate=0.1, epochs=10):

weights = [0.0] * (input_size + 1) # +1 for bias

previous_weights = None
print("Initial Weights:",
weights) for epoch in
range(epochs):

previous_weights = weights.copy()
print("Epoch:", epoch + 1)

for inputs, target in training_data:

prediction = predict(inputs,
weights) error = target -
prediction

weights[0] += learning_rate * error # Update bias weight

for i in range(len(inputs)):
weights[i + 1] += learning_rate * error * inputs[i]

print("Input:", inputs, "Target:", target, "Prediction:", prediction, "Weights:",

weights)
 # Check if weights have changed

if weights == previous_weights:

print("Stopping training after epoch", epoch

+ 1) break

print("End of Epoch", epoch

+ 1) print("-" * 30)

return weights

# Training data for AND gate (inputs, target)

training_data_and = [([0, 0], 0), ([0, 1], 0), ([1, 0], 0), ([1, 1], 1)]

input_size = 2

learning_rate = 0.1

epochs = 10

final_weights = train_perceptron(training_data_and, input_size, learning_rate, epochs)

print("Final Weights:", final_weights)

# Test the AND gate with the final weights

for inputs in [(0, 0), (0, 1), (1, 0), (1, 1)]:

result=predict_and_gate(inputs,final_weigh
ts)

print(f"AND Gate Output for {inputs}:

{result}")
OUTPUT

RESULT

The result of the experiment demonstrates the successful implementation of a perceptron

model for the AND function.
EXPERIMENT NO: 4

SIMPLE NEURAL NETWORK IN MEMORY

AIM

To set up a neural network using python

PREREQUISITES

Anaconda python with version > 3.6

Packages: numpy, keras, tensorflow.

PROCEDURE

Program1

1. Import Sequential from Keras models, Dense, activation from keras, layers and numpy
libraries.

2. Create numpy arrays for storing inputs (x) and output

3. Define the numbers neural model and its arguments and set the number of neurons for each
layer.

4. Compile the model and calculate its accuracy and print the Summary of the model.

5. Fit the model and print the output.

PROGRAM

Program 1:

# Import python libraries required in this example:

from keras.models import Sequential

from keras.layers import Dense, Activation

import numpy as np

# Use numpy arrays to store inputs (x) and outputs (y):

x = np.array([[0,0], [0,1], [1,0], [1,1]])

y = np.array([[0], [1], [1], [0]])

batch_size = 8

epochs = 50

# Define the network model and its arguments.

# Set the number of neurons/nodes for each layer:

model = Sequential()

model.add(Dense(2, input_shape=(2,)))

model.add(Activation('sigmoid'))

model.add(Dense(1))

model.add(Activation('sigmoid'))

# Compile the model and calculate its accuracy:

model.compile(loss='mean_squared_error', optimizer='sgd', metrics=['accuracy'])

# Print a summary of the Keras model:

model.summary()

#fit the model

model.fit(x, y, batch_size=batch_size, epochs=epochs, validation_split=0.1)

#evaluate the model

score = model.evaluate(x, y, verbose=0)

print("Test loss:", score[0])

print("Test accuracy:", score[1])

OUTPUT

PROCEDURE

Program 2

1.Import Sequential from tensorflow, Keras, module, dense from tensorflow from
tensorflow,keras,layers, train_test_split from sklearn,model Selection and pandas.

2 Import the data and split it into input(x) and output(y) variables.

3. Split the data to train (75%) and test (25%)

4. Define the Keras model and compile the model.

5. Fit the model on the dataset and make predictions with model and print the output.

PROGRAM

PROGRAM 2

# first neural network with keras make predictions

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Dense

from sklearn.model_selection import train_test_split

# load the dataset

import pandas as pd

dataset = pd.read_csv('diabetes.csv')

# split into input (X) and output (y) variables

X = dataset.iloc[:, :-1].values

y = dataset["Outcome"].values

# Split the data to Train, and Test (75%, 25%)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=11111)

# define the keras model

model = Sequential()

model.add(Dense(12, input_shape=(8,), activation='relu'))

model.add(Dense(8, activation='relu'))

model.add(Dense(1, activation='sigmoid'))

# compile the keras model

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# fit the keras model on the dataset

model.fit(X_train, y_train, epochs=150, batch_size=10, verbose=0)

# make class predictions with the model

predictions = (model.predict(X_test) > 0.5).astype(int)

# summarize the first 5 cases

for i in range(5):
print('%s => %d (expected %d)' % (X[i].tolist(), predictions[i], y[i]))

OUTPUT

RESULT

Thus, the program for creating model of neural network and training on diabetes dataset as well
as array input is implemented successfully.
EXPERIMENT NO: 5

IMPLEMENTATION OF CONVOLUTIONAL NEURAL NETWORK

AIM

To create a convolutional neural network and apply on MINIST digit recognition dataset

PREREQUISITES

Python, Keras, Tensorflow, Scikit-learn

DATASET

MINIST Dataset

PROCEDURE

1. Import the necessary libraries:

 Import tensorflow as tf
 Import relevant layers & modules from tensorflow neros.
2. Lead the MINIST dataset
 The MINIST dataset contains hand written digits.
 Lead the dataset into (X_train,y_train for training data & (X_test,y_test) for testing data.
3. Reshape the image dimensions.
 Reshape the data to (28,28,1) to match the CNN requirments.
4. Normalize the pixel values in the range [0,1].
5. Create sequential model.
 Read a conv 2D layer with 6 filters, a kernel size of [3,3] & ‘tanh’ activation.
 Add an average pooling 20 layer.
 Add another conv 2D layer with 16 filters, a kernel size of (3,3) & tanh activation.
 Add another average pooling 2D layer.
 Flatten the output.
 Add an dense layer with 128 units of ‘tanh’ activation.
 Add a final Dense layer with 10 units & softman activation for 10 digit classification.
6. Compile the model using ‘adam’ optimizer sparse categorical crossentropy. Loss
function & track ‘accuracy’ as the evaluation metric.
7. Fit the model to the training data ‘x_train’ ,‘y_train’) with 10 epochs use the validation
data(x_test,y_test) for validation during training function used is model.fit()
8. Model.predict(x_test) returns an array of probabilistics for each class the
np.argmax(predictions [0]) to get the predicted digit for the test sample.
9. Print the predicted digit & display the corresponding test image using ‘plt.inshow()’.

PROGRAM

import tensorflow as tf

from tensorflow import keras

from tensorflow.keras import datasets, layers, models

import matplotlib.pyplot as plt

# Load CIFAR-10 dataset

(X_train, y_train), (X_test, y_test) = datasets.cifar10.load_data()

X_train, X_test = X_train / 255.0, X_test / 255.0

# Define the list of optimizers to compare

optimizers = ['SGD', 'RMSprop', 'Adam', 'Adagrad']

# Initialize lists to store accuracy and loss for each optimizer

accuracy_scores = []

loss_scores = []

# Define a function to create and train a model with a specific optimizer

def train_model(optimizer_name):

# Create a simple CNN model

model = models.Sequential([

layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)),

layers.MaxPooling2D((2, 2)),

layers.Flatten(),

layers.Dense(64, activation='relu'),

layers.Dense(10)

])

# Compile the model with the specified optimizer

model.compile(optimizer=optimizer_name,

loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),

metrics=['accuracy'])

# Train the model

history = model.fit(X_train, y_train, epochs=50, batch_size=32,

validation_data=(X_test, y_test))
 # Evaluate the model and return accuracy and loss

test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=2)

return test_accuracy, history.history['accuracy'], history.history['loss']

# Loop through each optimizer and train models

for optimizer in optimizers:

accuracy, train_acc, train_loss = train_model(optimizer)

accuracy_scores.append(accuracy)

loss_scores.append((train_acc, train_loss))

# Plot accuracy for each optimizer

plt.figure(figsize=(12, 6))

for i, optimizer in enumerate(optimizers):

plt.plot(loss_scores[i][0], label=f'{optimizer} Accuracy')

plt.xlabel('Epochs')

plt.ylabel('Accuracy')

plt.legend()

plt.title('Accuracy Comparison for Different Optimizers')

plt.show()

# Plot loss for each optimizer

plt.figure(figsize=(12, 6))

for i, optimizer in enumerate(optimizers):

plt.plot(loss_scores[i][1], label=f'{optimizer} Loss')

plt.xlabel('Epochs')

plt.ylabel('Loss')

plt.legend()

plt.title('Loss Comparison for Different Optimizers')

plt.show()

# Print the final accuracy scores for all optimizers

for i, optimizer in enumerate(optimizers):

 print(f'{optimizer} Accuracy: {accuracy_scores[i]*100:.2f}%')

OUTPUT

RESULT

Thus, the implementation of convolutional neural network on MINIST dataset is implemented

successfully.
EXPERIMENT NO: 6

IMPLEMENTATION OF RECURRENT NEURAL NETWORKS

AIM

To create and implement a RNN using python.

PREREQUISITES

Python, keras, Jupyter Note Book

PROCEDURE

The provided code is for a Simple Recurrent Neural Network (RNN) to forecast sunspot activity.

1. Data Loading and Preprocessing:

- The code begins by loading monthly sunspot activity data from a CSV file using the `read_csv`
function. The data is scaled using `MinMaxScaler` and then split into training and testing sets.
The split is determined by the `split_percent` parameter (default is 80% training).

2. Data Sequencing:

- The time series data is prepared for training by creating sequences of data for input (`trainX`
and `testX`) and corresponding target values (`trainY` and `testY`). The sequences are created
with a fixed number of time steps (defined by `time_steps`).

3. Model Creation and Training:

- A Simple RNN model is created using the `create_RNN` function. You can customize the
number of hidden units, dense units, and activation functions. The model is compiled with the
mean squared error loss and the Adam optimizer.

- The model is then trained using the training data (`trainX` and `trainY`) for a specified number
of epochs (in this case, 20) and a batch size of 1.

4. Prediction:

- After training, the model makes predictions on both the training and testing data
(`train_predict` and `test_predict`).

5. Error Calculation:

- The code calculates and prints the root mean squared error (RMSE) for both the training and
testing data. RMSE is a common metric for regression tasks.

6. Plotting:

- The code visualizes the actual sunspot activity and the model's predictions. The red vertical
line in the plot separates the training and testing examples.
The code demonstrates a simple time series forecasting task using a Simple RNN model and
provides a visualization of the model's performance. You can adjust the model's architecture and
training parameters to experiment with different settings for your specific forecasting task.

PROGRAM

import numpy as np

import pandas as pd

import math

import matplotlib.pyplot as plt

from sklearn.preprocessing import MinMaxScaler

from sklearn.metrics import mean_squared_error

from keras.models import Sequential

from keras.layers import SimpleRNN, Dense

# Define the read_csv function

def read_csv(url, usecols, engine):

return pd.read_csv(url, usecols=usecols, engine=engine)

# Parameter split_percent defines the ratio of training examples

def get_train_test(url, split_percent=0.8):

df = read_csv(url, usecols=[1], engine='python')

data = np.array(df.values.astype('float32'))

scaler = MinMaxScaler(feature_range=(0, 1))

data = scaler.fit_transform(data).flatten()

n = len(data)

# Point for splitting data into train and test

split = int(n*split_percent)

train_data = data[:split]

test_data = data[split:]

return train_data, test_data, data

# Prepare the input X and target Y

def get_XY(dat, time_steps):

Y_ind = np.arange(time_steps, len(dat), time_steps)

Y = dat[Y_ind]

rows_x = len(Y)

X = dat[:time_steps*rows_x]

X = np.reshape(X, (rows_x, time_steps, 1))

return X, Y

def create_RNN(hidden_units, dense_units, input_shape, activation):

model = Sequential()

model.add(SimpleRNN(hidden_units, input_shape=input_shape, activation=activation[0]))

model.add(Dense(units=dense_units, activation=activation[1]))

model.compile(loss='mean_squared_error', optimizer='adam')

return model

def print_error(trainY, testY, train_predict, test_predict):

# Error of predictions

train_rmse = math.sqrt(mean_squared_error(trainY, train_predict))

test_rmse = math.sqrt(mean_squared_error(testY, test_predict))

# Print RMSE

print('Train RMSE: %.3f RMSE' % (train_rmse))

print('Test RMSE: %.3f RMSE' % (test_rmse))

# Plot the result

def plot_result(trainY, testY, train_predict, test_predict):

actual = np.append(trainY, testY)

predictions = np.append(train_predict, test_predict)

rows = len(actual)
 plt.figure(figsize=(15, 6), dpi=80)

plt.plot(range(rows), actual)

plt.plot(range(rows), predictions)

plt.axvline(x=len(trainY), color='r')

plt.legend(['Actual', 'Predictions'])

plt.xlabel('Observation number after given time steps')

plt.ylabel('Sunspots scaled')

plt.title('Actual and Predicted Values. The Red Line Separates The Training And Test
Examples')

sunspots_url= 'https://raw.githubusercontent.com/jbrownlee/Datasets/master/monthly-
sunspots.csv'

time_steps = 12

train_data, test_data, data = get_train_test(sunspots_url)

trainX, trainY = get_XY(train_data, time_steps)

testX, testY = get_XY(test_data, time_steps)

# Create model and train

model = create_RNN(hidden_units=3, dense_units=1, input_shape=(time_steps,1),

activation=['tanh', 'tanh'])

model.fit(trainX, trainY, epochs=20, batch_size=1, verbose=2)

# make predictions

train_predict = model.predict(trainX)

test_predict = model.predict(testX)

# Print error

print_error(trainY, testY, train_predict, test_predict)

# Plot result

plot_result(trainY, testY, train_predict, test_predict)

OUTPUT

Epoch 9/20
187/187 - 0s - loss: 0.0042 - 405ms/epoch - 2ms/step
Epoch 10/20
187/187 - 0s - loss: 0.0043 - 369ms/epoch - 2ms/step
Epoch 11/20
187/187 - 0s - loss: 0.0043 - 415ms/epoch - 2ms/step
Epoch 12/20
187/187 - 0s - loss: 0.0043 - 322ms/epoch - 2ms/step
Epoch 13/20
187/187 - 0s - loss: 0.0043 - 336ms/epoch - 2ms/step
Epoch 14/20
187/187 - 0s - loss: 0.0041 - 350ms/epoch - 2ms/step
Epoch 15/20
187/187 - 0s - loss: 0.0042 - 402ms/epoch - 2ms/step
Epoch 16/20
187/187 - 0s - loss: 0.0041 - 390ms/epoch - 2ms/step
Epoch 17/20
187/187 - 0s - loss: 0.0042 - 355ms/epoch - 2ms/step
Epoch 18/20
187/187 - 0s - loss: 0.0041 - 437ms/epoch - 2ms/step
Epoch 19/20
187/187 - 0s - loss: 0.0041 - 431ms/epoch - 2ms/step
Epoch 20/20
187/187 - 0s - loss: 0.0041 - 412ms/epoch - 2ms/step
6/6 [==============================] - 0s 3ms/step
2/2 [==============================] - 0s 3ms/step
Train RMSE: 0.063 RMSE
Test RMSE: 0.091 RMSE

RESULT

Overall, the code showcases a basic time series forecasting example using a Simple RNN model.
The RMSE values and the visual plot offer insights into the model's ability to capture patterns and
make predictions on the sunspot activity data. Further model optimization and experimentation
may be required for more accurate and robust forecasting in real-world scenarios.
EXPERIMENT NO: 7

IMPLEMENTATION OF SIMPLE LSTM

AIM

The aim is to demonstrate a simple example of LSTM neural network for univariate time series
forecasting.

PREREQUISITES

1. Python
2. Numpy
3. Keras
4. Matplotlib

PROCEDURE

1. Data Preparation

- Initialize `raw_seq` as a list of numerical values.

- Set `n_steps` to specify the number of previous time steps used for predicting the next value.
2. Data Splitting

- Create the `split_sequence` function to partition the time series into input (X) and output (y)
pairs.
- Iterate through the data, generating input sequences of length `n_steps` and corresponding
output values.
- Result in two NumPy arrays: `X` for input sequences and `y` for output values.
3. Data Reshaping

- Reshape the `X` array to the required 3D format `[samples, timesteps, features]`.
- Set `n_features` to 1, indicating each time step in the univariate time series has one feature.
4. Model Definition

- Create a Keras sequential model for time series forecasting:

a. Begin with an LSTM input layer (50 units, ReLU activation), expecting input of shape (n_steps,
n_features).
b. Add a Dense output layer with a single unit for prediction.
c. Compile the model using the Adam optimizer and Mean Squared Error (MSE) as the loss
function.
5. Model Training

- Train the model with historical data, using input sequences `X` and their corresponding output
values `y`, over 200 training epochs.
6. Prediction

- Define a new input sequence, `x_input`, representing the most recent `n_steps` values in the
time series.
- Reshape `x_input` to match the model's input requirements.
- Utilize the model to predict the next value in the time series, storing the prediction in `yhat`.

PROGRAM

import numpy as np
from keras.models import Sequential
from keras.layers import LSTM, Dense
import matplotlib.pyplot as plt

# Split a univariate sequence into samples

def split_sequence(sequence, n_steps):
X, y = list(), list()
for i in range(len(sequence)):
end_ix = i + n_steps
if end_ix > len(sequence)-1:
break
seq_x, seq_y = sequence[i:end_ix], sequence[end_ix]
X.append(seq_x)
y.append(seq_y)
return np.array(X), np.array(y)

# Define input sequence

raw_seq = [10, 20, 30, 40, 50, 60, 70, 80, 90]

# Choose a number of time steps

n_steps = 3

# Split into samples

X, y = split_sequence(raw_seq, n_steps)

# Reshape from [samples, timesteps] into [samples, timesteps, features]

n_features = 1
X = X.reshape((X.shape[0], X.shape[1], n_features))

# Split the data into training and validation sets

split_ratio = 0.8
split_index = int(len(X) * split_ratio)

X_train, X_val = X[:split_index], X[split_index:]

y_train, y_val = y[:split_index], y[split_index:]

# Define the model

model = Sequential()
model.add(LSTM(50, activation='relu', input_shape=(n_steps, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')

# Fit the model with validation data

history = model.fit(X_train, y_train, epochs=200, verbose=0,
validation_data=(X_val, y_val))
 # Plot training and validation loss
plt.plot(history.history['loss'], label='Training Loss (MSE)')
plt.plot(history.history['val_loss'], label='Validation Loss (MSE)')
plt.title('Model Loss (MSE)')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()

# Demonstrate prediction
x_input = np.array([70, 80, 90])
x_input = x_input.reshape((1, n_steps, n_features))
yhat = model.predict(x_input, verbose=0)
print(yhat)

OUTPUT

RESULT

Thus, Long Short Term memory (LSTM) neural network has been successfully implemented for
univariate time series forecasting.
EXPERIMENT NO: 8

IMPLEMENTATION OF BIDIRECTIONAL LSTM

AIM

To construct a bidirectional LSTM using python.

PREREQUISITES

1. Python
2. Keras
3. Matplotlib

PROCEDURE

1. Data Sequence Generation:

- Create a sequence classification instance using the `get_sequence` function.
- Generate random values and classify them as 0 or 1 based on cumulative sums.
- Reshape the data and return it as `X` and `y`.

2. Define Problem Properties:

- Set the number of timesteps `n_timesteps` to 10.

3. Define LSTM Model:

- Create a Keras sequential model.
- Add an LSTM layer with 20 units, expecting input of shape `(n_timesteps, 1)`, and set it to
return sequences.
- Add a TimeDistributed Dense layer with 1 unit and a sigmoid activation function.
- Compile the model using binary cross-entropy as the loss function and the Adam optimizer,
also tracking accuracy as a metric.

4. Training:
- Initialize lists to store accuracy and loss history during training.
- Train the LSTM model for 1000 epochs.
- For each epoch:
a. Get a new random sequence using `get_sequence`.
b. Fit the model with this sequence for one epoch and batch size 1.
c. Append the accuracy and loss from the training history to the respective lists.

5. Evaluation and Prediction:

- Get a new random sequence for evaluation.
- Use the trained model to predict the sequence.
- Apply a threshold of 0.5 to obtain predicted classes.
- Create plots to visualize model accuracy and loss during training.

6. Display Results:
- Append the expected and predicted labels to respective lists.
- Print the expected and predicted values for each timestep.

7. Classification Report
- Generate a classification report to evaluate the model's performance.
- Print the classification report, including metrics like precision, recall, and F1-score.
PROGRAM

from random import random

from keras.layers import Dense
from keras.layers import TimeDistributed
import matplotlib.pyplot as plt

# create a sequence classification instance

def get_sequence(n_timesteps):
X = array([random() for _ in range(n_timesteps)])
limit = n_timesteps / 4.0
y = array([0 if x < limit else 1 for x in cumsum(X)])
X = X.reshape(1, n_timesteps, 1)
y = y.reshape(1, n_timesteps, 1)
return X, y

# define problem properties

n_timesteps = 10

# define LSTM
model = Sequential()
model.add(LSTM(20, input_shape=(n_timesteps, 1), return_sequences=True))
model.add(TimeDistributed(Dense(1, activation='sigmoid')))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Lists to store accuracy and loss during training

accuracy_history = []
loss_history = []

# train LSTM
for epoch in range(1000):
X, y = get_sequence(n_timesteps)
history = model.fit(X, y, epochs=1, batch_size=1, verbose=2)
accuracy_history.append(history.history['accuracy'][0])
loss_history.append(history.history['loss'][0])

# Evaluate LSTM and make predictions

X, y = get_sequence(n_timesteps)
yhat = model.predict(X, verbose=0)
yhat_classes = (yhat > 0.5).astype(int) # Apply a threshold (0.5) to obtain predicted
classes

# Plot accuracy and loss over training epochs

plt.figure(figsize=(12, 5))
plt.subplot(1, 2, 1)
plt.plot(range(1, len(accuracy_history) + 1), accuracy_history)
plt.title('Model Accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')

plt.subplot(1, 2, 2)
plt.plot(range(1, len(loss_history) + 1), loss_history)
plt.title('Model Loss')
plt.xlabel('Epoch')
 plt.ylabel('Loss')
plt.show()

# Append expected and predicted labels to the respective lists

for i in range(n_timesteps):
expected_labels.append(y[0, i])
predicted_labels.append(yhat_classes[0, i])

# Print expected and predicted values

for i in range(n_timesteps):
print('Expected:', expected_labels[i], 'Predicted', predicted_labels[i])

# Generate and print a classification report

report = classification_report(expected_labels, predicted_labels)
print("Classification Report:\n", report)

OUTPUT

Exp
ecte
d: [
0] P
redi
cted
[0]
Exp
ecte
d: [
0] P
redi
cted
[0]
 Expected: [0] Predicted [0]
Expected: [0] Predicted [0]
Expected: [1] Predicted [0]
Expected: [1] Predicted [1]
Expected: [1] Predicted [1]
Expected: [1] Predicted [1]
Expected: [1] Predicted [1]
Expected: [1] Predicted [1]

Classification Report:
precision recall f1-score support

0 0.89 0.80 0.84 10

1 0.82 0.90 0.86 10

accuracy 0.85 20
macro avg 0.85 0.85 0.85 20
weighted avg 0.85 0.85 0.85 20

RESULT

Thus, bidirectional LSTM has been successfully implemented with model accuracy and loss
graphs.
EXPERIMENT NO: 9

IMPLEMENTATION OF GATED RECURRENT UNIT(GRU)

AIM
The aim of this code is to build and train a recurrent neural network (RNN) model
using the Gated Recurrent Unit (GRU) architecture for sentiment analysis on the
IMDB movie reviews dataset. The model is trained to predict whether a movie
review is positive or negative based on the provided dataset.

PREREQUISITES

Python (3.x recommended), imdb dataset, Keras, matplotlib

PROCEDURE

1. Download the IMDB movie reviews dataset using the imdb.load_data function
from Keras by limiting the dataset to only 5000 words and pad the sequences to
a fixed length of 500 words.
2. Build a sequential neural network model.
3. Add an embedding layer to convert integer indices to dense vectors.
4. Add a GRU layer with 100 units to capture sequential dependencies.
5. Add a dense output layer with a sigmoid activation function for binary
classification.
6. Compile the model using binary crossentropy loss and the Adam optimizer.
7. Fit the model to the training data with a batch size of 64 and over 3 epochs.
8. Use 20% of the training data for validation.
9. Plot the training and validation accuracy over epochs.
10.Plot the training and validation loss over epochs.
11.Use the trained model to predict the sentiment (positive or negative) for each
review in the test set.
12.Evaluate the model on the test set and print the accuracy.

PROGRAM

import matplotlib.pyplot as plt

from keras.datasets import imdb

from keras.models import Sequential

from keras.layers import Dense, GRU, Embedding

from keras.utils import pad_sequences

# Download the dataset and get only 500 words of each review

# Keeping only the top n words

word_count = 5000

(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=word_count)

# Taking the top 500 words

word_max = 500

x_train = pad_sequences(x_train, maxlen=word_max)

x_test = pad_sequences(x_test, maxlen=word_max)

# Building the model

model = Sequential()

model.add(Embedding(word_count, 100, input_length=word_max))

model.add(GRU(100))

model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam',
metrics=['accuracy'])

print(model.summary())

# Model fitting

history = model.fit(x_train, y_train, epochs=3, batch_size=64,

validation_split=0.2)

# Plot training and validation accuracy

plt.figure(figsize=(10, 4))

plt.subplot(1, 2, 1)

plt.plot(history.history['accuracy'], label='Training Accuracy')

plt.plot(history.history['val_accuracy'], label='Validation Accuracy')

plt.xlabel('Epoch')

plt.ylabel('Accuracy')

plt.legend()

# Plot training and validation loss

plt.subplot(1, 2, 2)

plt.plot(history.history['loss'], label='Training Loss')

plt.plot(history.history['val_loss'], label='Validation Loss')

plt.xlabel('Epoch')

plt.ylabel('Loss')

plt.legend()

plt.tight_layout()

plt.show()

# Predicting the next word at each timestep

y_predict = model.predict(x_test)

print(y_predict)

# Finding the accuracy of the model

result = model.evaluate(x_test, y_test, verbose=0)

print("Accuracy = %.2f%%" % (result[1] * 100))

OUTPUT

RESULT

Thus, the code shows the implementation of GRU on IMDB review dataset. The
plots shows the training and validation accuracy as well as the training and
validation loss over epochs. It also prints the predicted sentiments for the test set
and evaluates the overall accuracy of the model on the test data.
EXPERIMENT NO:10

IMPLEMENTATION OF BIDIRECTIONAL RNN

AIM

To create and implement a bidirectional RNN for IMDB dataset using python.

PREREQUISITES

Python, keras, Jupyter Note Book

PROCEDURE

The provided code is a Python script for sentiment analysis on the IMDB movie reviews dataset
using a Bidirectional SimpleRNN-based neural network. The procedure for this code is as follows:

1. Import Libraries and Suppress Warnings:

- Import necessary libraries, including `warnings`, `matplotlib`, `sklearn` for generating

classification reports, and Keras for deep learning.

- Suppress warnings to avoid clutter in the output.

2. Define Parameters:

- Specify parameters, such as the number of features, maximum sequence length (maxlen),
embedding size, and hidden layer size for the model.

3. Load and Preprocess Data:

- Load the IMDB dataset using `imdb.load_data` and preprocess it using `pad_sequences` to
ensure all sequences have the same length.

4. Define the Model:

- Create a Sequential model using Keras.

- Add an Embedding layer for word embeddings.

- Add a Bidirectional SimpleRNN layer to capture bidirectional context.

- Add a Dense layer with a sigmoid activation for binary classification.

- Compile the model with 'adam' optimizer and binary cross-entropy loss.

5. Set Training Parameters:

- Specify the batch size and the number of training epochs.

6. Train the Model:

- Use the training data (`X_train` and `y_train`) to train the model.

- Monitor the training process and save the training history in the `history` variable.
7. Plot Training and Validation Metrics:

- Generate plots for training and validation loss as well as accuracy over epochs.

8. Generate Predictions:

- Use the trained model to generate predictions on the test data (`X_test`).

9. Generate Classification Report:

- Calculate the classification report, which includes metrics like precision, recall, F1-score, and
support, to evaluate the model's performance on sentiment classification.

10. Display the Classification Report:

- Print the generated classification report to assess the model's performance on positive and
negative sentiment classification.

The code provides a comprehensive analysis of the model's performance, including visualizations
of the training process and detailed classification metrics.

PROGRAM

import warnings

import matplotlib.pyplot as plt

from sklearn.metrics import classification_report

from keras.datasets import imdb

from keras.preprocessing.sequence import pad_sequences

from keras.models import Sequential

from keras.layers import Embedding, Bidirectional, SimpleRNN, Dense

# Ignore warnings

warnings.filterwarnings('ignore')

# Define parameters

features = 2000

maxlen = 50

# Load and preprocess the dataset

(X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=features)

X_train = pad_sequences(X_train, maxlen=maxlen)

X_test = pad_sequences(X_test, maxlen=maxlen)

# Define the model

embedding = 128

hidden = 64

model = Sequential()

model.add(Embedding(features, embedding, input_length=maxlen))

model.add(Bidirectional(SimpleRNN(hidden)))

model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Set batch size and number of epochs

batch_size = 32

epochs = 5

# Train the model

history = model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs,

validation_data=(X_test, y_test))

# Plot training and validation loss

plt.figure(figsize=(12, 4))

plt.subplot(1, 2, 1)

plt.plot(history.history['loss'], label='Training Loss')

plt.plot(history.history['val_loss'], label='Validation Loss')

plt.legend()

plt.title('Loss')

# Plot training and validation accuracy

plt.subplot(1, 2, 2)

plt.plot(history.history['accuracy'], label='Training Accuracy')

plt.plot(history.history['val_accuracy'], label='Validation Accuracy')

plt.legend()

plt.title('Accuracy')

plt.show()

# Generate predictions

y_pred_prob = model.predict(X_test)

y_pred = (y_pred_prob > 0.5).astype(int)

# Generate classification report

from sklearn.metrics import classification_report

classification_report_output = classification_report(y_test, y_pred)

print('Classification Report:\n', classification_report_output)

OUTPUT

Epoch 1/5
782/782 [==============================] - 15s 17ms/step - loss: 0.5313 - accuracy: 0.7237 -
val_loss: 0.4514 - val_accuracy: 0.7898
Epoch 2/5
782/782 [==============================] - 12s 16ms/step - loss: 0.4008 - accuracy: 0.8236 -
val_loss: 0.4717 - val_accuracy: 0.7882
Epoch 3/5
782/782 [==============================] - 13s 16ms/step - loss: 0.3102 - accuracy: 0.8678 -
val_loss: 0.5151 - val_accuracy: 0.7655
Epoch 4/5
782/782 [==============================] - 13s 16ms/step - loss: 0.2016 - accuracy: 0.9215 -
val_loss: 0.6243 - val_accuracy: 0.7590
Epoch 5/5
782/782 [==============================] - 12s 16ms/step - loss: 0.1125 - accuracy: 0.9589 -
val_loss: 0.8345 - val_accuracy: 0.7458
782/782 [==============================] - 3s 3ms/step
Classification Report:
precision recall f1-score support

0 0.77 0.71 0.74 12500

1 0.73 0.78 0.75 12500

accuracy 0.75 25000

macro avg 0.75 0.75 0.75 25000
weighted avg 0.75 0.75 0.75 25000

RESULT

The provided code trains a Bidirectional SimpleRNN-based neural network model

on the IMDB movie reviews dataset for sentiment analysis. After training the model,
it generates training and validation loss and accuracy graphs and a classification
report for model evaluation. The conclusion of the code, as indicated by the
generated outputs, is as follows:
1. Training and Validation Loss/Accuracy Graphs:
● These graphs show the training and validation performance of the
model over multiple epochs. They provide insights into the model's
learning progress.
● A decreasing loss indicates that the model is improving its ability to
make predictions, while increasing accuracy suggests the model's
ability to classify sentiment correctly.
2. Classification Report:
● The classification report provides a summary of the model's
performance in terms of precision, recall, F1-score, and support for
both positive and negative sentiment classes.
● It offers detailed insights into the model's classification capabilities,
highlighting its strengths and weaknesses.

The code helps assess how well the Bidirectional SimpleRNN model performs in
sentiment analysis, offering a comprehensive view of its accuracy and the quality of
its predictions. The conclusion will depend on the specific results and metrics
generated, but it allows you to make informed decisions about the model's
effectiveness in the sentiment analysis task.
EXPERIMENT NO:11

IMPLEMENTATION OF AUTOENCODERS

AIM

To employ Autoencoder model in python, train it, plot evaluation metrics such as loss and
predict values.

PREREQUISITES
 Python 3.9
 Keras
 Numpy
 Matplotlib.pyplot
PROCEDURE
1. Import libraries and dependencies, including deep learning framework (keras).
2. Import dataset and preprocess it. If it is image data, normalize pixel values.
3. Define input layer, hidden layers, activation function for autoencoder model and
compile it.
4. While defining encoding and decoding layer, add bottleneck to it. Here, the encoded
and decoded layers give output of dimensionality 128units and bottleneck layer gives
an output of 400units (20x20 pixels).
5. Specify loss function and optimizer. Here, they are mean squared error and adam
respectively.
6. Fit autoencoder model to the training data for both input and target data.
7. Evaluate the performance of autoencoder using loss and accuracy metrics.
8. Plot accuracy and metrics.
9. Display original values and predicted values.
PROGRAM
from keras.layers import Dense, Input from keras.models import Model

from keras.datasets import fashion_mnist import numpy as np

import matplotlib.pyplot as plt

from keras.optimizers import Adam

from keras.layers import Dropout

# Define the dimensions input_dim = 784

encoding_dim = 400 # Adjust as needed

# Load the Fashion MNIST dataset

(x_train, _), (x_test, _) = fashion_mnist.load_data()

# Preprocess the data

x_train = x_train.astype('float32') / 255. x_test =

x_test.astype('float32') / 255.
x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:]))) x_test =
x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))

# Define the input layer

input_img = Input(shape=(input_dim,))

# Encoding layers
encoded = Dense(128, activation='relu')(input_img)
encoded = Dense(encoding_dim, activation='relu')(encoded) # This is the bottleneck layer

# Decoding layers
decoded = Dense(128, activation='relu')(encoded)
decoded = Dense(input_dim, activation='sigmoid')(decoded)

# Autoencoder model
autoencoder = Model(input_img, decoded)

# Encoder model
encoder = Model(input_img, encoded)

# Decoder model
encoded_input = Input(shape=(encoding_dim,)) decoder_layer1 =
autoencoder.layers[-2] decoder_layer2 = autoencoder.layers[-1]
decoder = Model(encoded_input,
decoder_layer2(decoder_layer1(encoded_input)))

# Compile the model op=Adam(lr=0.001)

autoencoder.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])

# Train the model

history = autoencoder.fit(x_train, x_train, epochs=50, batch_size=256, validation_data=(x_test,
x_test))
# Testing/Prediction
encoded_imgs = encoder.predict(x_test) decoded_imgs =
decoder.predict(encoded_imgs)

# Display original and reconstructed images plt.figure(figsize=(20,

4))
for i in range(5):
# Display original
ax = plt.subplot(2, 5, i + 1) plt.imshow(x_test[i].reshape(28, 28), cmap='gray'
 ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False) # Display
reconstruction
ax = plt.subplot(2, 5, i + 1 + 5)
plt.imshow(decoded_imgs[i].reshape(28, 28), cmap='gray')
ax.get_xaxis().set_visible(False) ax.get_yaxis().set_visible(False)
plt.show()

# Plot training & validation loss values plt.figure(figsize=(12,

4))
plt.subplot(1, 2, 1) plt.plot(history.history['loss'])
plt.plot(history.history['val_loss']) plt.title('Model
Loss') plt.ylabel('Loss') plt.xlabel('Epoch')
plt.legend(['Train', 'Test'], loc='upper left') plt.show()
# Plot training & validation accuracy values
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1) plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model accuracy')
plt.ylabel('accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Test'], loc='upper left')
plt.show()

OUTPUT

235/235 [==============================] - 1s 6ms/step - loss: 0.0059 -

accuracy: 0.0666 - val_loss: 0.0061 - val_accuracy: 0.0624
Epoch 47/50
235/235 [==============================] - 1s 6ms/step - loss: 0.0058 -
accuracy: 0.0664 - val_loss: 0.0060 - val_accuracy: 0.0661
Epoch 48/50
235/235 [==============================] - 2s 7ms/step - loss: 0.0058 -
accuracy: 0.0668 - val_loss: 0.0060 - val_accuracy: 0.0696
Epoch 49/50
235/235 [==============================] - 2s 6ms/step - loss: 0.0058 -
accuracy: 0.0687 - val_loss: 0.0059 - val_accuracy: 0.0656
Epoch 50/50
235/235 [==============================] - 1s 6ms/step - loss: 0.0058 -
accuracy: 0.0678 - val_loss: 0.0060 - val_accuracy: 0.0617
313/313 [==============================] - 1s 1ms/step
313/313 [==============================] - 1s 2ms/step
CONCLUSION
It can be observed that model accuracy is very low and it known that accuracy is not a
typical or appropriate metric for evaluating autoencoders. Instead, metrics such as Mean
Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE),
etc. can be used.
There is no indication of overfitting in the loss function as the testing loss doesn’t seem
to be increasing when the training loss decreases. However, training loss is significantly
higher than the testing loss. This implies that the model is not generalizing well to the
validation data which is a case of underfitting. Certain changes in the hyperparameters
can be made to fix this.

RESULT
Thus, an Autoencoder model has been trained and achieved. The autoencoder model was
successfully implemented in this experiment.
EXPERIMENT NO: 12

Implementation of Generative Adversarial Networks

AIM

To implement Generative Adversarial Networks

REQUIREMENTS

 TensorFlow
 Keras
 Python
PROCEDURE

Step 1: Importing the required libraries

Step 2: Loading the Dataset

Step 3: Defining parameters to be used in later processes

Step 4: Defining a Utility Class to Build the Generator

Step 5: Defining a Utility Class to Build the Discriminator

Step 6: Building the Generative Adversarial Network

Step 7: Training the Generative Adversarial Network

PROGRAM

1) Importing the required libraries

import torch

import torch.nn as nn

import torch.optim as optim

import torchvision

from torchvision import datasets, transforms

import matplotlib.pyplot as plt

import numpy as np

# Set device
 device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

2) Loading the Dataset

train_dataset = datasets.CIFAR10(root='./data',\ train=True, download=True,
transform=transform)

dataloader = torch.utils.data.DataLoader(train_dataset, \ batch_size=32, shuffle=True)

3) Defining parameters to be used in later processes

latent_dim = 100

lr = 0.0002

beta1 = 0.5

beta2 = 0.999

num_epochs = 10

4) Defining a Utility Class to Build the Generator

# Define the generator

class Generator(nn.Module):
def __init__(self, latent_dim):
super(Generator, self).__init__()

self.model = nn.Sequential(
nn.Linear(latent_dim, 128 * 8 * 8),
nn.ReLU(),
nn.Unflatten(1, (128, 8, 8)),
nn.Upsample(scale_factor=2),
nn.Conv2d(128, 128, kernel_size=3, padding=1),
nn.BatchNorm2d(128, momentum=0.78),
nn.ReLU(),
nn.Upsample(scale_factor=2),
nn.Conv2d(128, 64, kernel_size=3, padding=1),
nn.BatchNorm2d(64, momentum=0.78),
nn.ReLU(),
nn.Conv2d(64, 3, kernel_size=3, padding=1),
nn.Tanh()
)

def forward(self, z):

img = self.model(z)
return img
5) Defining a Utility Class to Build the Discriminator
# Define the discriminator
class Discriminator(nn.Module):
def __init__(self):
super(Discriminator, self).__init__()

self.model = nn.Sequential(
nn.Conv2d(3, 32, kernel_size=3, stride=2, padding=1),
nn.LeakyReLU(0.2),
nn.Dropout(0.25),
nn.Conv2d(32, 64, kernel_size=3, stride=2, padding=1),
nn.ZeroPad2d((0, 1, 0, 1)),
nn.BatchNorm2d(64, momentum=0.82),
nn.LeakyReLU(0.25),
nn.Dropout(0.25),
nn.Conv2d(64, 128, kernel_size=3, stride=2, padding=1),
nn.BatchNorm2d(128, momentum=0.82),
nn.LeakyReLU(0.2),
nn.Dropout(0.25),
nn.Conv2d(128, 256, kernel_size=3, stride=1, padding=1),
nn.BatchNorm2d(256, momentum=0.8),
nn.LeakyReLU(0.25),
nn.Dropout(0.25),
nn.Flatten(),
nn.Linear(256 * 5 * 5, 1),
nn.Sigmoid()
)

def forward(self, img):

validity = self.model(img)
return validity

6) Building the Generative Adversarial Network

# Define the generator and discriminator
# Initialize generator and discriminator
generator = Generator(latent_dim).to(device)
discriminator = Discriminator().to(device)

# Loss function
adversarial_loss = nn.BCELoss()

# Optimizers
 optimizer_G = optim.Adam(generator.parameters()\
, lr=lr, betas=(beta1, beta2))
optimizer_D = optim.Adam(discriminator.parameters()\
, lr=lr, betas=(beta1, beta2))

7) Training the Generative Adversarial Network

# Training loop
for epoch in range(num_epochs):
for i, batch in enumerate(dataloader):
# Convert list to tensor
real_images = batch[0].to(device)

# Adversarial ground truths

valid = torch.ones(real_images.size(0), 1, device=device)
fake = torch.zeros(real_images.size(0), 1, device=device)

# Configure input
real_images = real_images.to(device)

# ---------------------
# Train Discriminator
# ---------------------

optimizer_D.zero_grad()

# Sample noise as generator input

z = torch.randn(real_images.size(0), latent_dim, device=device)

# Generate a batch of images

fake_images = generator(z)

# Measure discriminator's ability

# to classify real and fake images
real_loss = adversarial_loss(discriminator\
(real_images),
valid)
fake_loss = adversarial_loss(discriminator\

(fake_images.detach()), fake)
d_loss = (real_loss + fake_loss) / 2

# Backward pass and optimize

 d_loss.backward()
optimizer_D.step()

# -----------------
# Train Generator
# -----------------

optimizer_G.zero_grad()

# Generate a batch of images

gen_images = generator(z)

# Adversarial loss
g_loss = adversarial_loss(discriminator(gen_images), valid)

# Backward pass and optimize

g_loss.backward()
optimizer_G.step()

# ---------------------
# Progress Monitoring
# ---------------------

if (i + 1) % 100 == 0:
print(
f"Epoch [{epoch+1}/{num_epochs}]\
Batch {i+1}/{len(dataloader)} "
f"Discriminator Loss: {d_loss.item():.4f} "
f"Generator Loss: {g_loss.item():.4f}"
)

# Save generated images for every epoch

if (epoch + 1) % 10 == 0:
with torch.no_grad():
z = torch.randn(16, latent_dim, device=device)
generated = generator(z).detach().cpu()
grid = torchvision.utils.make_grid(generated,\
nrow=4,
normalize=True)
plt.imshow(np.transpose(grid, (1, 2, 0)))
plt.axis("off")
plt.show()
OUTPUT

Epoch [10/10] Batch 1500/1563 Discriminator Loss: 0.5253 Generator Loss: 1.3269

RESULT
The implementation of Generative Adversarial Networks was performed successfully and
verified.
EXPERIMENT NO: 13

IMPLEMENTATION OF TRAFFIC SIGN RECOGNITION

USING GTSRB DATASET

AIM

The aim is to develop a robust deep learning model for the classification of German traffic

signs (GTSRB) using convolutional neural networks (CNNs).

REQUIREMENTS

 Python >3.9

 German Traffic Sign Recognition Benchmark (GTSRB) dataset

 Packages: pandas, numpy, OpenCV, Keras, Tensorflow, Scikit-learn

PROCEDURE

1. Import necessary libraries, including NumPy, Pandas, OpenCV, matplotlib, tensorflow,

etc.

2. Load the GTSRB dataset and set the paths for the training and test datasets.

3. Resize the images to a consistent size (30x30 pixels) for model compatibility and find

the total number of classes in the dataset.

4. Plot a bar chart showing the number of images in each class for visual inspection.

5. Shuffle the data and split it into training and validation sets.

6. Perform one-hot encoding on the labels.

 7. Define a CNN model with convolutional layers, pooling layers, batch normalization,

and fully connected layers.

8. Compile the model with categorical crossentropy loss, Adam optimizer, and accuracy

as a metric.

9. Implement data augmentation using ImageDataGenerator to enhance model

generalization.

10. Train the model using the augmented training dataset and monitor performance on the

validation set.

11. Visualize training and validation accuracy/loss curves over epochs.

12. Evaluate the model's accuracy on the test dataset.

PROGRAM
#Importing Required Libraries
import numpy as np
import pandas as pd
import os
import cv2
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow import keras
from PIL import Image
from sklearn.model_selection import train_test_split
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.optimizers import Adam
from sklearn.metrics import accuracy_score
np.random.seed(42)
from matplotlib import style
style.use('fivethirtyeight')

#Assigning Path for Dataset

data_dir = 'path to the dataset’

train_path = 'path to training dataset’

test_path = 'path to testing dataset’

# Resizing the images to 30x30x3

IMG_HEIGHT = 30

IMG_WIDTH = 30

channels = 3

#Finding total classes

NUM_CATEGORIES = len(os.listdir(train_path))

NUM_CATEGORIES

# Label Overview

classes = { 0:'Speed limit (20km/h)',

1:'Speed limit (30km/h)',

2:'Speed limit (50km/h)',

3:'Speed limit (60km/h)',

4:'Speed limit (70km/h)',

5:'Speed limit (80km/h)',

6:'End of speed limit (80km/h)',

7:'Speed limit (100km/h)',

8:'Speed limit (120km/h)',

9:'No passing',

10:'No passing veh over 3.5 tons',

11:'Right-of-way at intersection',

12:'Priority road',

13:'Yield',

14:'Stop',

15:'No vehicles',

16:'Veh > 3.5 tons prohibited',

17:'No entry',

18:'General caution',

19:'Dangerous curve left',

20:'Dangerous curve right',

21:'Double curve',

22:'Bumpy road',

23:'Slippery road',

24:'Road narrows on the right',

25:'Road work',

26:'Traffic signals',

27:'Pedestrians',

28:'Children crossing',

29:'Bicycles crossing',

30:'Beware of ice/snow',

31:'Wild animals crossing',

32:'End speed + passing limits',

33:'Turn right ahead',

 34:'Turn left ahead',

35:'Ahead only',

36:'Go straight or right',

37:'Go straight or left',

38:'Keep right',

39:'Keep left',

40:'Roundabout mandatory',

41:'End of no passing',

42:'End no passing veh > 3.5 tons' }

#Visualizing the data set

folders = os.listdir(train_path)

train_number = []

class_num = []

for folder in folders:

train_files = os.listdir(train_path + '/' + folder)

train_number.append(len(train_files))

class_num.append(classes[int(folder)])

# Sorting the dataset on the basis of number of images in each class

zipped_lists = zip(train_number, class_num)

sorted_pairs = sorted(zipped_lists)

tuples = zip(*sorted_pairs)

train_number, class_num = [ list(tuple) for tuple in tuples]

# Plotting the number of images in each class

plt.figure(figsize=(21,10))

plt.bar(class_num, train_number)

plt.xticks(class_num, rotation='vertical')

plt.show()

# Visualizing 25 random images from test data

import random

from matplotlib.image import imread

test = pd.read_csv(data_dir + '/Test.csv')

imgs = test["Path"].values

plt.figure(figsize=(25,25))

for i in range(1,26):

plt.subplot(5,5,i)

random_img_path = data_dir + '/' + random.choice(imgs)

rand_img = imread(random_img_path)

plt.imshow(rand_img)

plt.grid(b=None)

plt.xlabel(rand_img.shape[1], fontsize = 20)#width of image

plt.ylabel(rand_img.shape[0], fontsize =20)#height of image

#Collecting the training data

image_data = []

image_labels = []

for i in range(NUM_CATEGORIES):

path = data_dir + '/Train/' + str(i)

images = os.listdir(path)
 for img in images:

try:

image = cv2.imread(path + '/' + img)

image_fromarray = Image.fromarray(image, 'RGB')

resize_image = image_fromarray.resize((IMG_HEIGHT, IMG_WIDTH))

image_data.append(np.array(resize_image))

image_labels.append(i)

except:

print("Error in " + img)

# Changing the list to numpy array

image_data = np.array(image_data)

image_labels = np.array(image_labels)

print(image_data.shape, image_labels.shape)
#Shuffling the data

shuffle_indexes = np.arange(image_data.shape[0])

np.random.shuffle(shuffle_indexes)

image_data = image_data[shuffle_indexes]

image_labels = image_labels[shuffle_indexes]
#Splitting the dataset

X_train, X_val, y_train, y_val = train_test_split(image_data, image_labels, test_size=0.3,

random_state=42, shuffle=True)

X_train = X_train/255

X_val = X_val/255
print("X_train.shape", X_train.shape)

print("X_valid.shape", X_val.shape)

print("y_train.shape", y_train.shape)

print("y_valid.shape", y_val.shape)
#One hot encoding

y_train = keras.utils.to_categorical(y_train, NUM_CATEGORIES)

y_val = keras.utils.to_categorical(y_val, NUM_CATEGORIES)

print(y_train.shape)

print(y_val.shape)
#model creation

model = keras.models.Sequential([

keras.layers.Conv2D(filters=16,kernel_size=(3,3),activation='relu',input_shape=(IMG_HEI
GHT,IMG_WIDTH,channels)),

keras.layers.Conv2D(filters=32, kernel_size=(3,3), activation='relu'),

keras.layers.MaxPool2D(pool_size=(2, 2)),

keras.layers.BatchNormalization(axis=-1),

keras.layers.Conv2D(filters=64, kernel_size=(3,3), activation='relu'),

keras.layers.Conv2D(filters=128, kernel_size=(3,3), activation='relu'),

keras.layers.MaxPool2D(pool_size=(2, 2)),

keras.layers.BatchNormalization(axis=-1),

keras.layers.Flatten(),

keras.layers.Dense(512, activation='relu'),

keras.layers.BatchNormalization(),

keras.layers.Dropout(rate=0.5),
 keras.layers.Dense(43, activation='softmax')

])

lr = 0.001

epochs = 10

opt = Adam(lr=lr, decay=lr / (epochs * 0.5))

model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])

#Augmenting the data and training the model

aug = ImageDataGenerator(

rotation_range=10,

zoom_range=0.15,

width_shift_range=0.1,

height_shift_range=0.1,

shear_range=0.15,

horizontal_flip=False,

vertical_flip=False,

fill_mode="nearest")

history = model.fit(aug.flow(X_train, y_train, batch_size=32), epochs=epochs,

validation_data=(X_val, y_val))
#Evaluating the model

pd.DataFrame(history.history).plot(figsize=(8, 5))

plt.grid(True)

plt.gca().set_ylim(0, 1)

plt.show()
#loading the dataset and running the predictions
test = pd.read_csv(data_dir + '/Test.csv')

labels = test["ClassId"].values

imgs = test["Path"].values
data =[]

for img in imgs:

try:

image = cv2.imread(data_dir + '/' +img)

image_fromarray = Image.fromarray(image, 'RGB')

resize_image = image_fromarray.resize((IMG_HEIGHT, IMG_WIDTH))

data.append(np.array(resize_image))

except:

print("Error in " + img)

X_test = np.array(data)

X_test = X_test/255

pred = model.predict_classes(X_test)

#Accuracy with the test data

print('Test Data accuracy: ',accuracy_score(labels, pred)*100)

#Confusion matrix

from sklearn.metrics import confusion_matrix

cf = confusion_matrix(labels, pred)

import seaborn as sns

df_cm = pd.DataFrame(cf, index = classes, columns = classes)

plt.figure(figsize = (20,20))

sns.heatmap(df_cm, annot=True)
#classification report

from sklearn.metrics import classification_report

print(classification_report(labels, pred))
#prediction a dataset
plt.figure(figsize = (25, 25))

start_index = 0

for i in range(25):

plt.subplot(5, 5, i + 1)

plt.grid(False)

plt.xticks([])

plt.yticks([])

prediction = pred[start_index + i]

actual = labels[start_index + i]

col = 'g'

if prediction != actual:

col = 'r'

plt.xlabel('Actual={} || Pred={}'.format(actual, prediction), color = col)

plt.imshow(X_test[start_index + i])

plt.show()
 OUTPUT

RESULT

Thus, traffic sign recognition model for the GTSRB dataset was implemented
successfully.