Linear Algebra by Prof. R.

Vittal Rao

Lecture 2: August 10, 2010

an easy system for y. By analysing this system we get our analysis

2.1 Diagonalization
In the context of reducing a system of n equations in n unknowns
to an easy diagonal system, we have been looking at the question of
diagonalization. Consider an n × n matrix and look at the system
for x.
Hence the question that arises is the following:
Question 2.1 Given an n × n matrix A, can we find an invertible
matrix P such that P−1 AP is a diagonal matrix?

Ax = b (2.1) We remark that the above question has been loosely formulated
and requires more precision, which we shall as see as we analyse
for b ∈ Fn (where F = RorC). further.
We introduce a change of variables,

y = Cx and c = Cb. Remark 2.1 Not all matrices are diagonalizable. Consider the ma-
trix !
where C is an n × n invertible matrix. Then the system (2.1) can be 0 1
A= .
written as 0 0
AC−1 y = C−1 c
We observe that
We denote by ! ! !
P=C −1 0 1 0 1 0 0
A =
2
= = O2 ,
0 0 0 0 0 0
Then the system (2.1) can be written as
where O2 denotes the 2 × 2 zero matrix.
P−1 APy = c (2.2) Suppose A is diagonalizable i.e., suppose ∃ invertible P such that
Suppose we can find P such that P−1 AP = D, a diagonal matrix, d1 0
!
then (2.2) becomes, P AP = D =
−1
; a diagonal matrix
0 d2
Dy = c,

1
2.1 Diagonalization
! 2.1.1 Some examples
d21 0
⇒ (P AP)(P AP) = D =
−1 −1 2
0 d22 !
d21
!
0 0 1
⇒ (P−1 AP)(P−1 AP) = D2 = Example 2.1 Consider the matrix, A =
! 0 d22 1 0
d21 0 We can think of A as a matrix over R.
⇒ P A P=
−1 2
0 ! d22 Now look at !the matrix,
d21 0 1 1
⇒ O2 = since A2 = O2 P=
0 d22 1 −1
!
⇒ d1 = d2 = 0 ⇒ D = O2 1 1
Then P = 1
−1 2 2
⇒ P−1 AP = O2 2
− 12
⇒ A = O2 Then P and P−1 are
! also matrices over R and we have
0 1
A contradiction. P−1 AP = , a real diagonal matrix.
1 0
Thus, in this example the real matrix A can be diagonalized over
R (and hence over C since P can be thought of as a matrix over C).
Thus there are matrices which cannot be diagonalized. Hence
the first basic question in the context of diagonalization is the
following: 0 −1
!
Example 2.2 Consider the matrix, A =
1 0
Suppose ∃P such that
Question 2.2 What is the criterion [C] that A should satisfy in !
d1 0
order that A is diagonalizable? P AP =
−1
0 d2
!
d21 0
⇒P A P=
−1 2
0 d22
Remark 2.2 Once again we remark that we shall fine tune this !
d21 0
question as we go along since the answer to the question also ⇒ −I = since A2 = −I
depends on where are looking for the P. We shall illustrate this by 0 d22
some examples. ⇒ d21 = d22 = −1

2
2.2 The Basic Questions

⇒ d1 , d2 are complex
Remark 2.3 Note that if such a P exists it is not unique.
⇒ D is complex For if ∃ such a P then we have
⇒ P cannot be real (since then D = P−1 AP will also be real)
P−1 AP = D
⇒ A cannot be diagonalized
! over R. !
1 i
1 1 − If α , 0 then if Q = αP we get
Now let P = Then P = 21 i 2
−1
i −i 2 2
We can verify that P−1 = Q−1 and P = α−1 Q
!
i 0 Hence,
P AP =
−1
0 −i D = P−1 AP = αQ−1 Aα−1 Q = Q−1 AQ
Hence, A is diagonalizable over C, whereas it was not diagonaliz- Thus Q is also one such diagonalizing matrix for any α , 0
able over R (even though A is a real matrix). Hence, if we have one P then there are infinite number of such P.
The idea will be to get a P gor which P−1 computation is as easy as
possible.
The above examples suggest that we must make it precise about
the type of matrix P we are seeking for diagonalization.
We shall for the moment look for diagonalization over R or C. Remark 2.4 What we mean by compromise on transformation is
We shall denote by F (either R or C) and make it more precise later. the following:
We had introduced the same change of variable transformation
2.2 The Basic Questions y = Cx
c = Cb
Once we are given an n × n matrix A over F, the first question
we ask is the following: for both the known and the unknown. Suppose we seek different
transforms,say,
y = Cx
,
Question 2.3 Does A satisfy the criterion [C] for diagonalization? c = C1 b
We may get an answer YES or NO.
where C and C1 are invertible n × n matrices, then

These lead us to a string of questions which we list in Figure 2.1. Ax = b

3
2.2 The Basic Questions
Does A satisfy criterion [C]?

YES NO

There exists P s.t. P−1 AP

There is no P s.t. P−1 AP is diagonal
is a diagonal matrix D

What is the struc-

What can we do?
ture of such a P?

How do we find such

Compromise!
a P? (See remark 2.3)

Compromise on what?

Compromise on Compromise on transfor-

Diagonalization
What are the diago-
nal entries of D Are
they uniquely fixed?
Ask for P such that
−1
P AP is nearly diagonal
One such answer is to
triangularize (SCHUR de-
composition and Jordan
decomposition are some
answers in this direction)
mation (See remark 2.4)
Ask for P and Q invert-
ible matrices (n × n)
such that Q−1 AP is
a diagonal matrix
Singular Value Decom-
position (SVD) is an
answer in this direction
Can be generalized to
rectangular matrices as
well. (See remark 2.5)

Figure 2.1: Series of questions from criterion [C] for A

4
2.3 A Third Important Problem

We then define their dot product or inner product as

becomes
Q−1 APy = c n
X
(x, y) := y x =
T
xj yj
(where Q = C−1
1
, P = C−1 )
j=1
Now if we can choose P and Q (i.e. C and C1 ) such that Q−1 AP is
diagonal, then we have an easy system for y. The dot or inner product gives rise to a number.
We now define the outer or tensor product as
 
Remark 2.5 If we had an m × n matrix then b will be m × 1. So we  y1 
 .  
 
see Q an m × m matrix and P and n × n matrix such that 
x ⊗ y := yx =  .  x1 . . . xn
T  
 . 
Q−1 AP  
yn
 
has its leading largest diagonal block as a diagonal matrix.  
= ai j ; where ai j = y j x j
1≤i≤n,1≤j≤n

Thus, the tensor product is an n × n matrix

2.3 A Third Important Problem  y1 x1 y1 x2 . . . y1 xn


 y x y x . . . y x



 2 1 2 2 2 n
x ⊗ y =  ..

We have so far looked at two important problems, (which are .
. . . .
.

 . . . . 
related), namely,

yn x1 yn x2 . . . yn xn
 
1. linear systems, and
Note that x ⊗ y is generated by the single row vector xT since each
2. diagonalization of a matrix. row is a multiple of xT and the components of y are the multiplying
factors (We can also think of the matrix x ⊗ y as generated by the
We shall now look at another important problem.
single column vector y since all the the other columns are multiples
Consider two n × 1 matrices, (say real)
of y, and the components of x are the multiplying factors).
If x and y are nonzero, then x ⊗ y is an n × n matrix generated by
   
x1   y1 
 .   .  a single row vector or a single column vector. We say x ⊗ y is an
   
x =  .  ; y =  .  n × n matrix of rank 1.
   
 . 
 
 . 
  If x(1) , x(2) , . . . , x(r) is a linearly independent set of n × 1 vectors,
xn
 
yn
  and y(1) , y(2) , . . . , y(r) is a linearly independent set of r vectors (n × 1

5
2.4 Exercises

vectors), then

r
2.4 Exercises
X
x(j) ⊗ y(j)  0 1 0 
 
j=1 1. Let A =  0 0 1 
 
0 0 0
 
is an n × n matrix of rank r.
 0 0 0 
 
Similarly if we choose above the x(j) as n × 1 and y( j) as m × 1,
(a) Verify that A3 =  0 0 0  = O3 , the 3 × 3 zero matrix.
 
then
0 0 0
 
Xr
x(j) ⊗ y(j) (b) Examine whether A can be diagonalized over
j=1 i) F2 , ii) Q, iii) R, iv) C
!
is an m × n matrix of rank r. 0 2
2. Let A = . Consider A as a matrix over F7 .
1 0
The question that arises is the following: !
3 1
Let P = (as a matrix over F7 )
1 2

Question 2.4 Given any m × n real matrix A can we express it as (a) Find P−1 over F7
!
s 3 0
X (b) Verify that P−1 AP =
x(j) ⊗ y(j) 0 4
j=1 (and hence A is diagonalizable over F7 ).

for suitable x(1) , x(2) , . . . , x(s) ∈ Rn and y(1) , y(2) , . . . , y(s) ∈ Rm ?

If so, what is the minimum number of terms required,
(i.e., what is the minimal value of s) and how do we find
x(1) , x(2) , . . . , x(s) , y(1) , y(2) , . . . , y(s) ?

This leads us to again to SVD of A but now as a decomposition

of sum of matrices.