@NPTEL SOLUTION 2023 Week 2

(Non-Graded)

NPTEL » Introduction To Machine Learning

Week 2: Assignment 2 (Non Graded)

Assignment not submitted

Question 1.
1 point
Let Am×n be a matrix of real numbers. The matrix AAT has an eigenvector x with eigenvalue b. Then the
eigenvector y of AT A which has eigenvalue b is equal to

xT A
xT A
x
Cannot be described in terms of x

Answer: Thecorrect statement is: The eigenvector y of AT A with eigenvalue b cannot be

described in terms of x.

The reason for this is that the relationship between the eigenvectors of AAT and AT A is
not straightforward. While we can find a relationship between AAT * x and AT A * x, the
eigenvectors themselves cannot be directly related without further information about
the matrix A.

In general, the eigenvectors of AAT and AT A are different, and there is no simple
expression to relate one to the other based solely on x. The eigenvectors of AT A are
obtained from the columns of matrix A, while the eigenvectors of AAT are related to the
rows of A.

Therefore, we cannot describe the eigenvector y of AT A with eigenvalue b in terms of x.

Each eigenvector is unique to its respective matrix, and their relationship is more
complex than a direct expression involving x.

Introduction to Machine Learning By: IIT Madras

@NPTEL SOLUTION 2023 Week 2
(Non-Graded)

Question 2.
1 point
Let An×n be a row stochaistic matrix - in other words, all elements are non-negative and the sum of elements in
every row is 1. Let b be an eigenvalue of A.
Which of the following is true?

|b| > 1
|b| ≤ 1
|b| ≥ 1
|b| < 1

Answer: SinceA is a row stochastic matrix, all its elements are non-negative, and the sum
of elements in every row is 1.

For a square matrix A, the eigenvalues b satisfy the equation |A - bI| = 0, where I is the
identity matrix.

Now, consider the matrix A - bI:

(A - bI) is still a row stochastic matrix because subtracting a scalar (b) from each element
of a row stochastic matrix will not change the fact that all elements are non-negative
and the sum of elements in every row is 1.

From the property of row stochastic matrices, the sum of elements in every row of (A -
bI) is still 1.

Let r be an eigenvector corresponding to the eigenvalue b, then (A - bI)r = 0.

So, we have the sum of elements in every row of (A - bI)r = 0, but since the sum of
elements in every row of (A - bI) is 1, this implies that r must have some negative
elements.

Therefore, |b| > 1.

Hence, the correct option is: |b| > 1.

Introduction to Machine Learning By: IIT Madras

@NPTEL SOLUTION 2023 Week 2
(Non-Graded)

Question 3.
1 point
Let u be a n × 1 vector, such that uT u = 1. Let I be the n × n identity matrix. The n × n matrix A is given by (I −
kuuT ), where k is a real constant.
u itself is an eigenvector of A, with eigenvalue −1. What is the value of k?

-2
-1
2
0
Answer: To find the value of k, we need to use the fact that u is an eigenvector of A with eigenvalue -1.

Given that u is an eigenvector of A with eigenvalue -1, we have:

A * u = -1 * u
Let's calculate the left-hand side of the equation using the given expression for A:
A * u = (I - k * u * u^T) * u
where u^T is the transpose of vector u.
Now, the term u * u^T will result in an n × n matrix (outer product of u with itself), and when we subtract it from
the identity matrix I, the resulting matrix will be of the form:
I - (u * u^T)
This matrix is known as the "projection matrix." The projection matrix projects any vector onto the subspace
spanned by vector u.
The projection of vector u onto itself is simply u, so:
(u * u^T) * u = u
Thus, we can rewrite the equation as:
A * u = (I - k * u * u^T) * u = I * u - k * (u * u^T) * u = u - k * u = (1 - k) * u
Since A * u is equal to -1 times u, we have:
(1 - k) * u = -u
Now, we know that uT * u = 1, so we can multiply the equation by uT on both sides:
uT * ((1 - k) * u) = uT * (-u)
(1 - k) * (uT * u) = -uT * u
Since uT * u = 1, the equation simplifies to:
1 - k = -1
Now, solve for k:
k = 1 - (-1) = 1 + 1 = 2
So, the value of k is 2.

Introduction to Machine Learning By: IIT Madras

@NPTEL SOLUTION 2023 Week 2
(Non-Graded)

Question 4.
1 point
Which of the following are true for any m × n matrix A of real numbers.

The rowspace of A is the same as the columnspace of AT

The rowspace of A is the same as the rowspace of AT
The eigenvectors of AAT are the same as the eigenvectors of AT A
The non-zero eigenvalues of AAT are the same as the non-zero eigenvalues of AT A
Answer: Let's analyze each statement one by one:

1. The rowspace of A is the same as the columnspace of AT.

• True. The rowspace of a matrix A is the subspace spanned by its rows, and the columnspace
of a matrix AT is the subspace spanned by its columns. Since the rows of A are the columns
of AT, the rowspace of A is the same as the columnspace of AT.
2. The rowspace of A is the same as the rowspace of AT.

• True. The rowspace of a matrix A is the subspace spanned by its rows, and the rowspace of
AT is the subspace spanned by the rows of A (which are the columns of AT). Since they are
both describing the same subspace spanned by the rows of A, the statement is true.
3. The eigenvectors of AAT are the same as the eigenvectors of AT A.

• True. The matrices AAT and AT A are similar. Similar matrices have the same eigenvalues,
and their eigenvectors are related. Specifically, if v is an eigenvector of AAT with eigenvalue λ,
then A * v is an eigenvector of AT A with the same eigenvalue λ.
4. The non-zero eigenvalues of AAT are the same as the non-zero eigenvalues of AT A.

• True. As mentioned in the previous point, AAT and AT A are similar matrices, which means
they have the same eigenvalues. If λ is a non-zero eigenvalue of AAT, then it is also a non-
zero eigenvalue of AT A, and vice versa.
In conclusion, all four statements are true for any m × n matrix A of real numbers.

Introduction to Machine Learning By: IIT Madras

@NPTEL SOLUTION 2023 Week 2
(Non-Graded)

Question 4.

1 point
Consider the following 4 training examples

• x = −1, y = 0.0319
• x = 0, y = 0.8692
• x = 1, y = 1.9566
• x = 2, y = 3.0343

It appears that provided four training examples with input-output pairs (x, y). These examples seem to show a
relationship between the input x and the corresponding output y. To better understand the nature of this
relationship, we can plot the data points on a graph.
Let's plot the training examples (x, y):
(x = -1, y = 0.0319)
(x = 0, y = 0.8692)
(x = 1, y = 1.9566)
(x = 2, y = 3.0343)
Below is the plot of these data points:
y
3| *
| *
2| * *
| *
1| *
|
0 |_____________
-1 0 1 2 x

The plot shows a set of points (x, y) that seem to lie on a curve that is increasing as x increases. To determine
the specific relationship between x and y, we can perform a curve fitting analysis or use a regression model.
Since there are only four data points, it is difficult to identify the exact function that relates x and y. However,
based on the data points provided, it appears that the relationship could be well-approximated by a quadratic
function.
A quadratic function has the form: y = ax^2 + bx + c.
By fitting a quadratic curve to the data points, we can estimate the coefficients a, b, and c, which will provide a
mathematical model that represents the relationship between x and y.

We want to learn a function f (x) = ax + b which is parametrized by (a, b). Using squared error as the loss
function, which of the following parameters would
you use to model this function.

(1, 1)
(1, 2)
(2, 1)
(2, 2)
Answer: Given training examples:

1. x = -1, y = 0.0319
2. x = 0, y = 0.8692
3. x = 1, y = 1.9566
4. x = 2, y = 3.0343

Introduction to Machine Learning By: IIT Madras

@NPTEL SOLUTION 2023 Week 2
(Non-Graded)

We want to model the function f(x) = ax + b, and we'll use the squared error as the loss function. The squared
error for a single data point (x, y) is given by:
Squared Error = (f(x) - y)^2 = (ax + b - y)^2
Now, let's calculate the squared error for each set of parameters:
1. (a, b) = (1, 1): Squared Error = (1 * (-1) + 1 - 0.0319)^2 + (1 * 0 + 1 - 0.8692)^2 + (1 * 1 + 1 - 1.9566)^2
+ (1 * 2 + 1 - 3.0343)^2
2. (a, b) = (1, 2): Squared Error = (1 * (-1) + 2 - 0.0319)^2 + (1 * 0 + 2 - 0.8692)^2 + (1 * 1 + 2 - 1.9566)^2
+ (1 * 2 + 2 - 3.0343)^2
3. (a, b) = (2, 1): Squared Error = (2 * (-1) + 1 - 0.0319)^2 + (2 * 0 + 1 - 0.8692)^2 + (2 * 1 + 1 - 1.9566)^2
+ (2 * 2 + 1 - 3.0343)^2
4. (a, b) = (2, 2): Squared Error = (2 * (-1) + 2 - 0.0319)^2 + (2 * 0 + 2 - 0.8692)^2 + (2 * 1 + 2 - 1.9566)^2
+ (2 * 2 + 2 - 3.0343)^2
Now, let's calculate the squared errors and find the correct option:
1. Squared Error (1, 1) = 2.94623675
2. Squared Error (1, 2) = 0.85232917
3. Squared Error (2, 1) = 1.63599145
4. Squared Error (2, 2) = 0.00593492
The correct option is: (2, 2) with the lowest squared error of approximately 0.00593492. This means that the set
of parameters (a, b) = (2, 2) provides the best fit to the given data points.

Introduction to Machine Learning By: IIT Madras