National University of Computer and Emerging Sciences

Instructor: Dr. M. Ali, Dr. Sara Aziz, Mr. Irfanullah, MT1004 Linear Algebra
M. Usman Rashid BS CS Fall 2022
Practice Questions

Chapter 5 Eigenvalues and Eigenvectors till Chapter 7 Symmetric Matrices and Quadratic Forms

Eigenvalues and Eigenvectors → Diagonalization → Orthogonal Diagonalization → Gram-Schmidt

Algorithm → 𝑸𝑸𝑸𝑸 −Decomposition.

Least-Squares Problem → Pseudoinverse(s) & Singular Value Decomposition.

Symmetric Matrices and Quadratic Forms → Constrained Optimization

Diagonalization

Problem 1: Find a matrix 𝑷𝑷 that diagonalizes

𝟎𝟎 𝟎𝟎 −𝟐𝟐
𝑨𝑨 = �𝟏𝟏 𝟐𝟐 𝟏𝟏 � .
𝟏𝟏 𝟎𝟎 𝟑𝟑

Problem 2: Show that the following matrix is not diagonalizable:

𝟏𝟏 𝟎𝟎 𝟎𝟎
𝑨𝑨 = � 𝟏𝟏 𝟐𝟐 𝟎𝟎� .
−𝟑𝟑 𝟓𝟓 𝟐𝟐

Problem 3: Using 𝑨𝑨𝒌𝒌 = 𝑷𝑷𝑫𝑫𝒌𝒌 𝑷𝑷−𝟏𝟏 , find 𝑨𝑨𝟏𝟏𝟏𝟏 , where

𝟎𝟎 𝟎𝟎 −𝟐𝟐
𝑨𝑨 = �𝟏𝟏 𝟐𝟐 𝟏𝟏 � .
𝟏𝟏 𝟎𝟎 𝟑𝟑

Orthogonal Diagonalization

𝟏𝟏 𝟏𝟏 𝟏𝟏
Problem 1: For the symmetric matrix 𝑨𝑨 = �𝟏𝟏 𝟏𝟏 𝟏𝟏�, find an orthogonal 𝑷𝑷 such that 𝑷𝑷−𝟏𝟏 𝑨𝑨𝑨𝑨 is diagonal.
𝟏𝟏 𝟏𝟏 𝟏𝟏
Then write the spectral decomposition or eigen decomposition of 𝑨𝑨.

Problem 2: Find an orthogonal matrix 𝑷𝑷 that diagonalizes

𝟒𝟒 𝟐𝟐 𝟐𝟐
𝑨𝑨 = �𝟐𝟐 𝟒𝟒 𝟐𝟐� .
𝟐𝟐 𝟐𝟐 𝟒𝟒

Then write the spectral decomposition or eigen decomposition of 𝑨𝑨.

Problem 3 (Optional): What is a geometric interpretation of a spectral decomposition?

 Gram-Schmidt Process/Algorithm

Problem 1: Apply the Gram–Schmidt process to transform the basis vectors

𝟏𝟏 𝟎𝟎 𝟎𝟎
𝒙𝒙𝟏𝟏 = �𝟏𝟏� , 𝒙𝒙𝟐𝟐 = �𝟏𝟏� , 𝒙𝒙𝟑𝟑 = �𝟎𝟎� ,
𝟏𝟏 𝟏𝟏 𝟏𝟏

into an orthogonal basis {𝒗𝒗𝟏𝟏 , 𝒗𝒗𝟐𝟐 , 𝒗𝒗𝟑𝟑 }, and then normalize the orthogonal basis vectors to obtain an
orthonormal basis {𝒒𝒒𝟏𝟏 , 𝒒𝒒𝟐𝟐 , 𝒒𝒒𝟑𝟑 }.

Problem 2: Find a 𝑸𝑸𝑸𝑸 −decomposition of

𝟏𝟏 𝟎𝟎 𝟎𝟎
𝑨𝑨 = �𝟏𝟏 𝟏𝟏 𝟎𝟎� .
𝟏𝟏 𝟏𝟏 𝟏𝟏

Notice that the columns of 𝑨𝑨 are 𝒙𝒙𝟏𝟏 , 𝒙𝒙𝟐𝟐 , and 𝒙𝒙𝟑𝟑 .

Problem 3: Find the 𝑸𝑸𝑸𝑸 factorization of the matrix 𝑴𝑴.

𝟐𝟐 𝟐𝟐
𝑴𝑴 = � 𝟏𝟏 𝟕𝟕 � .
−𝟐𝟐 −𝟖𝟖

Problem 4: Find the 𝑸𝑸𝑸𝑸 factorization of the matrix 𝑺𝑺.

𝟏𝟏 𝟐𝟐
𝑺𝑺 = � �.
𝟐𝟐 𝟒𝟒

Best Approximation; Least Squares

Problem 1: (Unique Least Squares Solution) Find the least squares solution, the least squares error
vector, and the least squares error of the linear system

𝒙𝒙𝟏𝟏 − 𝒙𝒙𝟐𝟐 = 𝟒𝟒, 𝟑𝟑𝒙𝒙𝟏𝟏 + 𝟐𝟐𝒙𝒙𝟐𝟐 = 𝟏𝟏, −𝟐𝟐𝒙𝒙𝟏𝟏 + 𝟒𝟒𝒙𝒙𝟐𝟐 = 𝟑𝟑.

Problem 2: (Infinitely Many Least Squares Solution) Find the least squares solution, the least squares
error vector, and the least squares error of the linear system

𝟑𝟑𝒙𝒙𝟏𝟏 + 𝟐𝟐𝒙𝒙𝟐𝟐 − 𝒙𝒙𝟑𝟑 = 𝟐𝟐, 𝒙𝒙𝟏𝟏 − 𝟒𝟒𝒙𝒙𝟐𝟐 + 𝟑𝟑𝒙𝒙𝟑𝟑 = −𝟐𝟐, 𝒙𝒙𝟏𝟏 + 𝟏𝟏𝟏𝟏𝒙𝒙𝟐𝟐 − 𝟕𝟕𝒙𝒙𝟑𝟑 = 𝟏𝟏.

The Problem of Curve Fitting

Least Squares Straight Line Fit

Find the least squares straight line fit to the four
points (0, 1), (1, 3), (2, 4), and (3, 4).
(See Figure)
Homework: Explore “Fitting a Quadratic Curve to
Data” and so on.

Pseudoinverses 𝑨𝑨+ .
 Singular Value Decomposition SVD

𝟏𝟏 𝟏𝟏
Problem 1: Find the singular values of the matrix 𝑨𝑨 = �𝟎𝟎 𝟏𝟏� .
𝟏𝟏 𝟎𝟎

𝟏𝟏 −𝟏𝟏
Problem 2: Find the singular values of the matrix 𝑨𝑨 = �−𝟐𝟐 𝟐𝟐 � .
𝟐𝟐 −𝟐𝟐

𝟎𝟎 𝟏𝟏 𝟏𝟏
Problem 3: Find the singular values of the matrix 𝑨𝑨 = � �.
𝟏𝟏 𝟏𝟏 𝟎𝟎

𝟔𝟔 𝟐𝟐
Problem 4: Find the singular values of the matrix 𝑨𝑨 = � �.
−𝟕𝟕 𝟔𝟔

𝟏𝟏 𝟑𝟑
Problem 5: Find the singular values of the matrix 𝑨𝑨 = � �.
𝟑𝟑 𝟗𝟗

Symmetric Matrices and Quadratic Forms → Constrained Optimization

Problem 1: Find the maximum and minimum values of the quadratic form

𝒛𝒛 = 𝟓𝟓𝒙𝒙𝟐𝟐 + 𝟓𝟓𝒚𝒚𝟐𝟐 + 𝟒𝟒𝟒𝟒𝟒𝟒,

subject to the constraint 𝒙𝒙𝟐𝟐 + 𝒚𝒚𝟐𝟐 = 𝟏𝟏.

Problem 2: Revisited Problem 1 Using Level Curves.

Problem 3: Diagonalize the quadratic form in Problem 1, that is, remove the cross-product term.

Problem 4: Find the maximum value of 𝟗𝟗𝒙𝒙𝟐𝟐𝟏𝟏 + 𝟒𝟒𝒙𝒙𝟐𝟐𝟐𝟐 + 𝟑𝟑𝒙𝒙𝟐𝟐𝟑𝟑 subject to the constraints 𝒙𝒙𝑻𝑻 𝒙𝒙 = 𝟏𝟏 and 𝒙𝒙𝑻𝑻 𝒖𝒖𝟏𝟏 = 𝟎𝟎,
where 𝒖𝒖𝟏𝟏 = (𝟏𝟏, 𝟎𝟎, 𝟎𝟎). Note that 𝒖𝒖𝟏𝟏 is a unit eigenvector corresponding to the greatest eigenvalue 𝝀𝝀 = 𝟗𝟗 of
the matrix of the quadratic form.
This problem is relevant to a theorem, sometimes known as “A Big Theorem” in literature.

Note that one can have conditions like 𝒙𝒙𝑻𝑻 𝒖𝒖𝟏𝟏 = 𝟎𝟎 on other eigenvectors as well.
Book: David C. Lay
Homework: Please read the relevant terminologies and theorems, especially the principal axes theorem.

Good Luck