Huzair Hassan sofi(2021529550)

ASSIGNMENT-3

Ans1:

Estimate the output of the network (y5 and y6) for each of the input patterns, we need to propagate the
input through the network using the given weights and activation function.

For the input pattern (0, 0):

The weighted sum at node 3 is v3 = w13 * x1 + w23 * x2 = 0.3 * 0 + 0.6 * 0 = 0

The activation at node 3 is a3 = 0 (because v3 < d0)

The weighted sum at node 4 is v4 = w34 * a3 = 0.9 * 0 = 0

The activation at node 4 is a4 = 0 (because v4 < d0)

The weighted sum at node 5 is v5 = w45 * a4 = 1.2 * 0 = 0

The activation at node 5 is a5 = 0 (because v5 < d0)

The output at node 6 is y6 = w56 * a5 = 1.1 * 0 = 0

The output at node 5 is y5 = w35 * a3 = 0.8 * 0 = 0

Therefore, for the input pattern (0, 0), the network outputs y5 = 0 and y6 = 0.

For the input pattern (0, 1):

The weighted sum at node 3 is v3 = w13 * x1 + w23 * x2 = 0.3 * 0 + 0.6 * 1 = 0.6

The activation at node 3 is a3 = 1 (because v3 >= d0)

The weighted sum at node 4 is v4 = w34 * a3 = 0.9 * 1 = 0.9

The activation at node 4 is a4 = 1 (because v4 >= d0)

The weighted sum at node 5 is v5 = w45 * a4 = 1.2 * 1 = 1.2

The activation at node 5 is a5 = 1 (because v5 >= d0)

The output at node 6 is y6 = w56 * a5 = 1.1 * 1 = 1.1

The output at node 5 is y5 = w35 * a3 = 0.8 * 1 = 0.8

Therefore, for the input pattern (0, 1), the network outputs y5 = 0.8 and y6 = 1.1.
For the input pattern (1, 0):

The weighted sum at node 3 is v3 = w13 * x1 + w23 * x2 = 0.3 * 1 + 0.6 * 0 = 0.3

The activation at node 3 is a3 = 1 (because v3 >= d0)

The weighted sum at node 4 is v4 = w34 * a3 = 0.9 * 1 = 0.9

The activation at node 4 is a4 = 1 (because v4 >= d0)

The weighted sum at node 5 is v5 = w45 * a4 = 1.2 * 1 = 1.2

The activation at node 5 is a5 = 1 (because v5 >= d0)

The output at node 6 is y6 = w56 * a5 = 1.1

Ans2:

(i) The three inputs can each take one of two possible values (0 or 1), so there are 2^3 = 8
possible input patterns.

The table below shows all possible input patterns for the three input nodes:

X1 | x2 | x3

---|----|---

0| 0| 0

0| 0| 1

0| 1| 0

0| 1| 1

1| 0| 0

1| 0| 1

1| 1| 0

1| 1| 1

(ii) For each input pattern, we first compute the weighted sum of the inputs, v = w1x1 + w2x2 +
w3x3, and then apply the step-function to get the output y.
Using the given weights, we have:

For input pattern (0,0,0): v = 0, so y = 0

For input pattern (0,0,1): v = 1, so y = 1

For input pattern (0,1,0): v = -4, so y = 0

For input pattern (0,1,1): v = -3, so y = 0

For input pattern (1,0,0): v = 2, so y = 1

For input pattern (1,0,1): v = 3, so y = 1

For input pattern (1,1,0): v = -2, so y = 0

For input pattern (1,1,1): v = -1, so y = 0

Ans3:

Backpropagation is a supervised learning algorithm used for training feedforward neural networks. It
involves propagating the error from the output layer back through the network to adjust the weights in
order to minimize the error.

The backpropagation algorithm consists of two phases: forward propagation and backward propagation.
During forward propagation, the input is fed through the network and the output is computed. During
backward propagation, the error between the computed output and the desired output is propagated
backwards through the network to update the weights.

The weight updates are performed using the gradient descent algorithm, which involves computing the
gradient of the error with respect to each weight and adjusting the weight in the direction of the
negative gradient. The learning rate coefficient determines the step size of the weight update and is an
important hyperparameter to tune. A large learning rate can cause the weight update to overshoot the
minimum, while a small learning rate can result in slow convergence.

The role of the learning rate coefficient in the convergence of backpropagation is crucial. A high learning
rate causes the weight update to be larger, which may lead to oscillations or divergence, while a low
learning rate may cause the weight update to be too small, slowing down the convergence of the
algorithm. It is important to choose an appropriate learning rate based on the characteristics of the
dataset and the network architecture. Typically, a learning rate is chosen through experimentation and
validation on a held-out dataset.
Ans4:

A multi-layer feedforward network is effective when it can learn complex patterns and make accurate
predictions on unseen data. The effectiveness of a multi-layer feedforward network depends on various
factors, including the number of hidden layers, the number of neurons in each hidden layer, the
activation functions used, and the training algorithm.

The hidden layers play a crucial role in the effectiveness of the network as they enable the network to
learn complex non-linear relationships between the input and output. The more hidden layers a
network has, the more complex the relationships it can learn. However, adding too many hidden layers
can lead to overfitting, where the network memorizes the training data rather than learning general
patterns.

The output layer is also important in a feedforward network as it produces the final prediction. The
number of neurons in the output layer depends on the problem being solved. For example, a
classification problem with two classes requires one neuron in the output layer, while a problem with
more than two classes requires multiple neurons.

The activation functions used in the hidden layers determine how the neurons in the layer transform
their input. Non-linear activation functions, such as sigmoid or ReLU, enable the network to learn
complex patterns and improve its accuracy.

The training algorithm used to update the weights in the network is also crucial in its effectiveness.
Backpropagation with gradient descent is the most commonly used algorithm for training multi-layer
feedforward networks.