Exam – Computational Linear Algebra (2021)

duration: 3 hours; points: 50

Problem 1 (points: 5)
Suppose V is finite-dimensional and let `1 , `2 ∈ L (V). Prove that

range(`1 ) ⊂ range(`2 ) ⇐⇒ ∃`0 ∈ L (V) such that `1 = `2 ◦ `0 .

Problem 2 (points: 5+3+2=10)

Let A ∈ Rn ×n be invertible. Apply Gram-Schmidt decomposition to the columns of A and show that we
can decompose A as A = QR, where Q ∈ Rn ×n is an orthogonal matrix and R is an upper triangular matrix
R whose diagonal elements are positive. By “upper triangular”, we mean that R is of the form
 
× × × × ×
 × × × × 
R= × × × .
 

0 × ×
 
×

Using the decomposition A = QR, explain

(i) How you can solve Ax = b for some b ∈ Rm ?
(ii) How you can verify whether det(A) belongs to the interval [−r, r ] for some given r > 0?

Problem 3 (points: 5)
Prove that P ∈ Rn×n is an orthogonal projector (onto a subspace of Rn ) if and only if P2 = P and P> = P.

Problem 4 (points: 8+7=15)

Let A, B ∈ Rn ×n be symmetric positive semidefinite matrices (all eigenvalues are nonnegative). Prove that

A2 = B2 =⇒ A = B.

(hint: use contradiction; assume that A 6= B and v be an eigenvector of A − B with nonzero eigenvalue).
(ii) If C ∈ Rn ×n be defined as Ci j = Ai j Bi j , 1 ¶ i , j ¶ n , then show that C is symmetric positive semidefinite.

Problem 5 (points: 5)
Let A ∈ Rn ×n be symmetric and b ∈ Rn . Using just the eigendecomposition of A, prove that

Ax = b has a solution ⇐⇒ ∀z ∈ Rn such that Az = 0 : z >b = 0.

Problem 6 (points: 3+4+3=10)

We have seen in a lecture that the PCA problem of finding an optimal k -dimensional subspace in Rn
(k ¶ n) can be reduced to the following: Find orthonormal vectors u 1 , . . . , u k ∈ Rn such that the sum
k
X
u >j Au j (1)
j =1
is maximized. Here, A ∈ Rn ×n is the symmetric positive semidefinite covariance matrix associated with
the data. In particular, let the eigendecomposition of A be
n
X
A= λi q i q >i , (2)
i =1

where λ1 ¾ λ2 ¾ · · · ¾ λn ¾ 0 and assume that that λ1 , . . . , λk are positive.

(i) Substituting (2) in (1), show that we can write (1) as

n
X
λi γi ,
i =1
Pn
where γ1 , . . . , γn ∈ R are such that γi ∈ [0, 1] and i =1 γi = k.

(ii) In particular, explain why the problem of maximizing (1) over all possible orthonormal vectors
u 1 , . . . , u k ∈ Rn can be reduced to the following optimization problem:
n
X n
X
max λi γi subject to γi ∈ [0, 1], γi = k .
γ1 ,...,γn ∈R
i =1 i =1

Prove that the maximum is λ1 +· · ·+λk which is attained when γ1 = · · · = γk = 1 and γk +1 = · · · = γn = 0.

Pn Pn
(hint: show that if γi ∈ [0, 1] and i =1 γi = k , then i =1 λi γi ¶ k ).

(iii) Using (ii), show that (1) is maximized when u j = q j , 1 ¶ j ¶ k . That is, the optimal solution of the
PCA problem is given by top eigenvectors of A.