Faculty of Computing and Information Technology (FCIT)

Department of Computing Indus University, Karachi

Machine Learning (SEE-421)

Spring-2024

Department Program Semester Course Instructor IssueDate DueDate FacultySign Total

/ Faculty Title ature Marks

FCIT BS 8th Machine Ms. 30/05/2024 06/06/2024 5

Software Learning Shagufta
Engineering Aftab

Instructions
1. Thisassignmentcontains1Question.Attemptallquestions.
2. This assignment should be completed within assigned time, after the due date, assignment will not
be accepted.
3. Studentsof particularcourse will downloadassignmentexam and submit solution whichwill only be
accepted through CMS portal.
4. Please ensure that no part of your assignment should be copied from any other source without
acknowledgement of the source and proper referencing (IEEE).
5. Please note that copy-paste is a serious nature of academic dishonesty, it is called “Plagiarism” and
the penalties are attached to being found guilty of committing such offences.
6. It is allow using lecture notes, books and other sources, however needing to refer/cite properly,
Reference list must be given at end of the assignment.
7. This assignment should be submitted in PDF file for this purpose first take image of all hand written
pages and then merge using Smartphone app (from PC/Laptop put all images in word file and save as
PDF) including assignment paper in the start of submission.
8. Assignmentcanbecompressedorbreakintwopartsiffilesizeislargerthanuploadinglimit.
9. Thefont sizeshould12and Times New Romanshouldbe used. Allfigures and illustrationsshouldbe

properly titled or numbered on the left side, below.

10. Also ensure that no part of your assignment has been written by any other person, except to the
extent of collaboration and /or group work.
11. Preferablyneatandcleanhand-writtenform,ifamarkercan’treadwhatyou’vewritten,youranswer might
as well be wrong.

THISTABLEISFOROFFICIALUSE.DONOTWRITEANYTHINGON IT
Questions Q -1

CLOs CLO-3

MaximumMarks 5

MarksObtained

Total Score

1
 FacultyofComputingandInformationTechnology(FCIT) Department
of Computing Indus University, Karachi

Machine Learning (SEE-421)

Spring-2024

Question: Apply Forward and Backward propagation algorithms on the given2-layerNeuralnetwork

Introduction

Neuralnetworkisamachinelearningprocessthatteachescomputerstoprocessdatainawaythatmimicsthehuman
brain.It'satypeofartificialintelligencethatusesinterconnectednodesorneuronsinalayeredstructure.Inanyneural
network:

 Forward Propagation is the way tomovefromtheInputlayer(left)totheOutputlayer(right)intheneural

network.
 Similarly,theprocessofmovingfrom therightto lefti.e.,backwardfromtheOutputto theInputlayeris called
the Backward Propagation.

Problem Statement

Objectives

 Implementaneuralnetworkwithtwolayerstoaclassificationproblem
 Implementforwardpropagationusingmatrixmultiplication
 Performbackwardpropagation

Steps to be followed

2
Step-1:
ImportLibraries
→Importsklearn,matplotlib,w3_unittest

3
Step-2: (1.0)
LoadandPreprocessDataset
m=2000
samples,labels=make_blobs(n_samples=m,
centers=([2.5,3],[6.7,7.9],[2.1,7.9],[7.4,2.8]),
cluster_std=1.1,
random_state=0)
labels[(labels==0)|(labels==1)]=1
labels[(labels==2)|(labels==3)]=0 X =
np.transpose(samples)
Y=labels.reshape((1,m))

plt.scatter(X[0,:],X[1,:],c=Y,cmap=colors.ListedColormap(['blue','red']));

Printallrequiredvalues

Step-3:DefiningtheNeuralNetworkStructure (0.5)

 n_x:thesizeoftheinputlayer
 n_h:thesizeofthehiddenlayer(setit equalto2fornow)
 n_y:thesizeoftheoutputlayer

Step-4:PerformForwardPropagation (0.5)

W1 = parameters["W1"]
b1 = parameters["b1"]
W2=parameters["W2"]
b2 = parameters["b2"]
Z1=np.matmul(W1,X)+b1 A1
= sigmoid(Z1)
Z2=np.matmul(W2,A1)+b2 A2
= sigmoid(Z2)

assert(A2.shape==(n_y,X.shape[1]))

cache = {"Z1": Z1,

"A1":A1,
"Z2":Z2,
"A2":A2}
Printallrequiredvalues

Step-5:Performbackwardpropagation (1.0)

m=X.shape[1]

W1=parameters["W1"]
W2=parameters["W2"]

A1=cache["A1"]
A2=cache["A2"]

dZ2=A2-Y
dW2=1/m*np.dot(dZ2,A1.T)
db2=1/m*np.sum(dZ2,axis=1,keepdims=True)
4
 dZ1=np.dot(W2.T,dZ2)*A1*(1-A1) dW1
= 1/m * np.dot(dZ1, X.T)
db1=1/m*np.sum(dZ1,axis=1,keepdims=True)

grads={"dW1":dW1,
"db1": db1,
"dW2":dW2,
"db2":db2}

returngrads

grads=backward_propagation(parameters,cache,X,Y)

Printrequiredvalues

Step-6:Buildyourneuralnetworkmodelin nn_model() (1.0)

defnn_model(X,Y,n_h,num_iterations=10,learning_rate=1.2,print_cost=False): n_x =

layer_sizes(X, Y)[0]
n_y=layer_sizes(X,Y)[2]

parameters=initialize_parameters(n_x,n_h,n_y)

foriinrange(0,num_iterations):

A2,cache=forward_propagation(X,parameters) cost

= compute_cost(A2, Y)

grads=backward_propagation(parameters,cache,X,Y)

ifprint_cost: print("Costafteriteration%i:
%f"%(i,cost))

returnparameters

PrintW1,W2,b1,andb2withsettingnum_iterations=100,learning_rate=1.2

Check results through unit-testing

w3_unittest.test_nn_model(nn_model)

5
Code

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler
from matplotlib import colors

# Number of samples
m = 2000

# Generate blobs for a binary classification

samples, labels = make_blobs(n_samples=m,
centers=([2.5, 3], [6.7, 7.9], [2.1, 7.9], [7.4,
2.8]),
cluster_std=1.1,
random_state=0)

# Combine classes 0 and 1, and classes 2 and 3

labels[(labels == 0) | (labels == 1)] = 1
labels[(labels == 2) | (labels == 3)] = 0

# Transpose X and reshape Y

X = np.transpose(samples)
Y = labels.reshape((1, m))

# Plot the data

plt.scatter(X[0, :], X[1, :], c=Y, cmap=colors.ListedColormap(['blue', 'red']))
plt.show()

# Print required values

print("X shape:", X.shape)
print("Y shape:", Y.shape)
print("Labels distribution:", np.unique(labels, return_counts=True))

def layer_sizes(X, Y):

n_x = X.shape[0] # size of input layer
n_h = 2 # size of hidden layer (set to 2 for now)
n_y = Y.shape[0] # size of output layer
return (n_x, n_h, n_y)

# Print required values

n_x, n_h, n_y = layer_sizes(X, Y)
print(f"Input layer size: {n_x}")
print(f"Hidden layer size: {n_h}")
6
print(f"Output layer size: {n_y}")

def sigmoid(z):
return 1 / (1 + np.exp(-z))

def forward_propagation(X, parameters):

W1 = parameters["W1"]
b1 = parameters["b1"]
W2 = parameters["W2"]
b2 = parameters["b2"]

Z1 = np.dot(W1, X) + b1
A1 = sigmoid(Z1)
Z2 = np.dot(W2, A1) + b2
A2 = sigmoid(Z2)

assert (A2.shape == (n_y, X.shape[1]))

cache = {"Z1": Z1,

"A1": A1,
"Z2": Z2,
"A2": A2}

return A2, cache

# Print required values

parameters = {
"W1": np.random.randn(n_h, n_x),
"b1": np.zeros((n_h, 1)),
"W2": np.random.randn(n_y, n_h),
"b2": np.zeros((n_y, 1))
}
A2, cache = forward_propagation(X, parameters)
print("Z1:", cache["Z1"])
print("A1:", cache["A1"])
print("Z2:", cache["Z2"])
print("A2:", cache["A2"])

def backward_propagation(parameters, cache, X, Y):

m = X.shape[1]
W1 = parameters["W1"]
W2 = parameters["W2"]
A1 = cache["A1"]
A2 = cache["A2"]

dZ2 = A2 - Y
7
 dW2 = 1/m * np.dot(dZ2, A1.T)
db2 = 1/m * np.sum(dZ2, axis=1, keepdims=True)
dZ1 = np.dot(W2.T, dZ2) * A1 * (1 - A1)
dW1 = 1/m * np.dot(dZ1, X.T)
db1 = 1/m * np.sum(dZ1, axis=1, keepdims=True)

grads = {"dW1": dW1,

"db1": db1,
"dW2": dW2,
"db2": db2}

return grads

# Calculate gradients
grads = backward_propagation(parameters, cache, X, Y)
print("dW1:", grads["dW1"])
print("db1:", grads["db1"])
print("dW2:", grads["dW2"])
print("db2:", grads["db2"])

def initialize_parameters(n_x, n_h, n_y):

W1 = np.random.randn(n_h, n_x) * 0.01
b1 = np.zeros((n_h, 1))
W2 = np.random.randn(n_y, n_h) * 0.01
b2 = np.zeros((n_y, 1))

parameters = {"W1": W1,

"b1": b1,
"W2": W2,
"b2": b2}

return parameters

def compute_cost(A2, Y):

m = Y.shape[1]
logprobs = np.multiply(np.log(A2), Y) + np.multiply(np.log(1 - A2), 1 - Y)
cost = - np.sum(logprobs) / m
cost = np.squeeze(cost)
return cost

def nn_model(X, Y, n_h, num_iterations=100, learning_rate=1.2, print_cost=False):

n_x = layer_sizes(X, Y)[0]
n_y = layer_sizes(X, Y)[2]
parameters = initialize_parameters(n_x, n_h, n_y)

for i in range(num_iterations):
8
 A2, cache = forward_propagation(X, parameters)
cost = compute_cost(A2, Y)
grads = backward_propagation(parameters, cache, X, Y)

parameters["W1"] = parameters["W1"] - learning_rate * grads["dW1"]

parameters["b1"] = parameters["b1"] - learning_rate * grads["db1"]
parameters["W2"] = parameters["W2"] - learning_rate * grads["dW2"]
parameters["b2"] = parameters["b2"] - learning_rate * grads["db2"]

if print_cost and i % 10 == 0:
print(f"Cost after iteration {i}: {cost}")

return parameters

# Train the model

parameters = nn_model(X, Y, n_h, num_iterations=100, learning_rate=1.2,
print_cost=True)
print("W1:", parameters["W1"])
print("b1:", parameters["b1"])
print("W2:", parameters["W2"])
print("b2:", parameters["b2"])

# Check results through unit-testing

w3_unittest.test_nn_model(nn_model)

9
1
0
1
1