Q1. What is the purpose of forward propagation in a neural network?

A1: Forward propagation is the process of computing the output of a neural network given an input. Its purpose
is to pass the input data through the network's layers, applying weights, biases, and activation functions to
produce a prediction or output.

Q2. How is forward propagation implemented mathematically in a single-layer feedforward

neural network?
A2: In a single-layer feedforward neural network, forward propagation involves multiplying the input features
by the weights, adding biases, and applying an activation function to produce the output.

Q3. How are activation functions used during forward propagation?

A3: Activation functions introduce non-linearity to the network, allowing it to learn complex patterns in the
data. During forward propagation, activation functions are applied to the weighted sum of inputs and biases
at each neuron to introduce non-linearities into the network's output.

Q4. What is the role of weights and biases in forward propagation?

A4: Weights and biases are the parameters that the neural network learns during training. The weights
determine the strength of connections between neurons, while biases allow the network to capture the overall
behavior of the data. During forward propagation, weights are multiplied by input features, and biases are
added to the result to compute the output of each neuron.

Q5. What is the purpose of applying a softmax function in the output layer during forward
propagation?
A5: The softmax function is used in the output layer of a neural network to convert the raw output scores into
probabilities. It ensures that the output values sum up to 1, making it suitable for multi-class classification
problems where the network needs to predict the probability distribution over multiple classes.

Q6. What is the purpose of backward propagation in a neural network?

A6: Backward propagation, also known as backpropagation, is the process of computing gradients of the loss
function with respect to the weights and biases of the network. Its purpose is to update the network's
parameters to minimize the loss function, thus improving the network's performance.

Q7. How is backward propagation mathematically calculated in a single-layer feedforward

neural network?
A7: In a single-layer feedforward neural network, backward propagation involves computing the gradients of
the loss function with respect to the weights and biases using the chain rule. These gradients are then used to
update the parameters of the network using optimization algorithms such as gradient descent.

Q8. Can you explain the concept of the chain rule and its application in backward
propagation?
A8: The chain rule is a fundamental concept in calculus that allows us to compute the derivative of a composite
function. In the context of neural networks, it is used to calculate the gradients of the loss function with respect
to the weights and biases by propagating gradients backward through the network layer by layer. This allows
us to efficiently compute the gradients needed for parameter updates during training.

Q9. What are some common challenges or issues that can occur during backward
propagation, and how can they be addressed?
A9: Common challenges during backward propagation include vanishing or exploding gradients, which can
hinder the training process. Techniques such as gradient clipping, using proper weight initialization schemes,
and using activation functions like ReLU can help mitigate these issues. Additionally, choosing an appropriate
learning rate and employing regularization techniques can also improve the stability of backward propagation.