10/2/23, 12:30 AM Linear Algebra Advanced Problems

Problem 9: Least Squares Solutions

Consider a system of linear equations Ax = b, where A is an m × n matrix with m > n. Show that the
^ minimizes the residual ∥Ax − b∥2 .
least squares solution x

Solution 9:

^ minimizes the residual ∥Ax − b∥2 if and only if the residual r

The least squares solution x = b − Ax^ is
orthogonal to the column space of A.

Least Squares Solution:

^ is given by x^ = (AT A)−1 AT b.
The least squares solution x
Residual:
The residual is r = b − Ax
^.
Minimizing the Residual:
We want to minimize ∥r∥2 = ∥b − Ax ^∥2 . For this, we need r to be orthogonal to the column space of
A. In other words, AT r = 0.
Substituting the expression for r and solving for x ^, we obtain the least squares solution.

Problem 10: Eigenvalue Properties

Let A be an n × n matrix with eigenvalues λ1 , λ2 , … , λn . Prove the following properties:

​ ​ ​

a) The trace of A is equal to the sum of its eigenvalues: trace(A) = λ1 + λ2 + … + λn .

​ ​ ​

b) The determinant of A is equal to the product of its eigenvalues: det(A) = λ1 λ2 … λn .

​ ​ ​

Solution 10:

a) Trace and Sum of Eigenvalues:

The trace of A is the sum of its diagonal elements, which are the eigenvalues of A (including
multiplicity).

b) Determinant and Product of Eigenvalues:

The determinant of A is equal to the product of its eigenvalues (including multiplicity) because the
characteristic polynomial of A is given by det(A − λI) = (λ1 − λ)(λ2 − λ) … (λn − λ), and det(A) =
​ ​ ​

λ1 λ2 … λn
​ ​ ​

https://chat.openai.com/c/832b3b7a-97c1-4f8a-ba84-ae4f9a6a5036 1/1