Estimation & Adaptive Control

Rajeeb Dey
April 2020

1 Lecture 3 - Post Mid Semester

1.1 Singular Value Decomposition
Following facts about Singular Value Decomposition (SVD) is mentioned below,
• This is a numerical computation technique for decomposing a rectangular
matrix into two orthogonal matrices and one diagonal matrix.
• The diagonal matrix contains the information about the singular values
of the rectangular matrix whose decomposition is carried out.
• SVD finds its wide application in control and communication algorithms
where matrix inversion need to be avoided.
• SVD is widely used for finding Eigenvalues and Eigenvectors too.
• The solution for the decomposition is obtained in the sense of least square
solution.

1.2 Mathematical Preliminaries of SVD

Given a rectangular matrix Ak×m with k > m, its SVD is given as,
A = U DV T
where, U ∈ <k×k , V ∈ <m×m and D ∈ <k×m . Please note that we have
considered an over determined system as k > m. Both U and V matrices are
square and orthogonal, whereas matrix D is diagonal in structure.
Structurally, the decomposition is depicted as,
 
Σr×r 0r×(m−r)
A=U VT
0(k−r)×r 0(k−r)×(m−r)
and the structure of block matrix Σ containing singular values of the matrix A
is given as,  
σ1
Σ=
 .. 
. 
σr

1
where,
σ1 ≥ σ2 ..... ≥ σr > 0

1.3 SVD Properties

Some of the important properties related to the matrices involved in SVD are,
• If Rank(A) = r ≤ min(k, m), then A is full rank.
0
• The σis are the singular values of the matrix A and are unique.
• Singular values are obtained from the eigen values of a symmetric positive
semi-definite matrix AT A.

1.4 Exercise Questions & Further Reading

• What are the properties of orthogonal matrices?

• Given a rectangular matrix Ak×m with k > m, if we find the SVD of

A = U DV T , then what will be the value of AT A?
• What information will be contained in the diagonal matrix of AT A after
applying SVD?

• What is the relation between Singular value and Eigenvalue of the matrix
A?
• Further reading on SVD from any book of Linear Algebra.

1.5 Assignment 3
Prove that SVD of the matrix Ak×m with k > m gives the singular value of the
matrix A along with two orthogonal matrices, using eigenvalue and eigenvector
approach, i.e.,  
Σr×r 0r×(m−r)
A=U VT
0(k−r)×r 0(k−r)×(m−r)

1.6 Pseudo Inverse

Theorem 1. When a matrix HN ×n is full rank with N > n such that,

Rank(H) = r = min(N, n) = n

then,
H † = (H T H)−1 H T
where, H † is called the pseudo inverse of the matrix.

2
1.7 Exercise Question
• If the matrix H is under determined what will be the pseudo inverse in
that case?

1.8 Assignment 4
• Write down all the properties of pseudo inverse of a rectangular matrix
H?
• Identify when right and left pseudo inverse are obtained?

1.9 SVD of Pseudo inverse

Theorem 2. Given a full rank matrix HN ×n with N > n whose SVD is,
 
T Σr×r 0
H = U DV = U VT
0 0

then,
Σ−1
 
0
H† = V UT
0 0
where, Σ−1 = diag(σ1−1 , σ2−1 , ........σr−1 )

1.10 Assignment 5
Prove the fact of Theorem 2.