7 Special Matrics II

Matrix Theory
––Special Matrices Again

School of Mathematical Sciences
Teaching Group
: Main Reference books
▸ Ilse C. F. Ipsen, Numerical Matrix Analysis: Linear Systems and Least Squares.
SIAM, 2009.
▸ Fuzhen Zhang. Matrix Theory-Basic Results and Techniques, Second Edition.
Springer, 2011.
Reference books:
▸ Roger A. Horn and Charles A. Johnson: Matrix Analysis. Cambridge University
Press, 1985.
▸ Gene H. Golub and Charles F. Van Loan: Matrix Computations, Third Edition.
Johns Hopkins Press, 1996.
▸ Nicholas J. Higham. Accuracy and Stability of Numerical Algorithms, Second
Edition. SIAM, 2002.
▸ Y. Saad. Iterative Methods for Sparse Linear Systems, Second Edition. SIAM,
Philadelphia, 2003.
Matrix Theory Special Matrices Again
Maintained by Yan-Fei Jing
Unitary and Orthogonal Matrices
Outline
Application 1: Givens rotation
Application 2: Permutation matrix
Discovery Journey
References
Matrix Theory Special Matrices - 2/23

Definitions

Definitions
A matrix A ∈ Cn×n is

Definitions
▸ unitary if
AAH = AH A = I.

Definitions
▸ unitary if
AAH = AH A = I.
▸ orthogonal if
AAT = AT A = I.

Definitions
▸ unitary if
AAH = AH A = I.
▸ orthogonal if
AAT = AT A = I.
Remarks:

Definitions
▸ unitary if
AAH = AH A = I.
▸ orthogonal if
AAT = AT A = I.
Remarks:
▸ A unitary matrix: columns (or rows) constitute an
orthonormal basis for Cn .

Definitions
▸ unitary if
AAH = AH A = I.
▸ orthogonal if
AAT = AT A = I.
Remarks:
▸ A orthogonal matrix: columns (or rows) constitute an
orthonormal basis for Rn .

Definitions
▸ unitary if
AAH = AH A = I.
▸ orthogonal if
AAT = AT A = I.
Remarks:
▸ A orthogonal matrix: columns (or rows) constitute an
orthonormal basis for Rn .
▸ The identity matrix is orthogonal as well as unitary.
Outline
Discovery Journey
References

Givens rotation

Givens rotation
Let c and s be scalars with
∣c∣2 + ∣s∣2 = 1.

Givens rotation
Let c and s be scalars with
∣c∣2 + ∣s∣2 = 1.
The matrices
c s c s
( ), ( )
−s c s −c
are unitary.

Definition
If c, s ∈ C so that
∣c∣2 + ∣s∣2 = 1,

Definition
∣c∣2 + ∣s∣2 = 1,
then the unitary 2 × 2 matrix
c s
( )
−s c
is called a Givens rotation.

Definition
∣c∣2 + ∣s∣2 = 1,
then the unitary 2 × 2 matrix
c s
( )
−s c
is called a Givens rotation.
If c and s are also real, then the Givens rotation
c s
( )
−s c
is orthogonal.
Some remarks
▸ When a Givens rotation is real, then both diagonal elements
are the same.

Some remarks
are the same.
▸ When a Givens rotation is complex, then the diagonal
elements are complex conjugates of each other.

Some remarks
are the same.
▸ A unitary matrix of the form
−c s
( )
s c
where the real parts of the diagonal elements have different
signs, is a reflection; it is not a Givens rotation.

Some remarks
are the same.
−c s
( )
s c

Some remarks
are the same.
−c s
( )
s c
In numerical linear algebra, a Givens rotation is a rotation in the

plane spanned by two coordinates axes.
––Givens rotations are named after Wallace Givens, who
introduced them to numerical analysts in the 1950s while he was
working at Argonne National Laboratory.
Outline
Discovery Journey
References

Recall: Permutation
A permutation p on {1, 2, . . . , n} is
▸ even: if p can be restored to natural order by an even
number of interchanges.

Recall: Permutation
A permutation p on {1, 2, . . . , n} is
▸ even: if p can be restored to natural order by an even
▸ odd: if p can be restored to natural order by an odd

Example of permutation
For instance, consider the permutations on {1, 2, 3, 4}.

▸ p = (2, 1, 4, 3) is even
–after interchanging 2 and 1, 4 and 3 (two interchanges)
Ô⇒ (1, 2, 3, 4)

Example of permutation
For instance, consider the permutations on {1, 2, 3, 4}.

▸ p = (2, 1, 4, 3) is even
–after interchanging 2 and 1, 4 and 3 (two interchanges)
Ô⇒ (1, 2, 3, 4)
▸ p = (1, 4, 3, 2) is odd
–after interchanging 4 and 2 Ô⇒ (1, 2, 3, 4)

Definition in Matrix Theory

A square matrix is a permutation matrix if it contains

▸ a single one in each column and in each row,
▸ and zeros everywhere else.


Question: How many permutation matrices of order n?


Question: How many permutation matrices of order n?
n! (n factorial)

Examples
The following are permutation matrices.

Examples
▸ The identity matrix I.

Examples
▸ The exchange matrix (an anti-diagonal matrix)
⎛ 1 ⎞
⎜
J =⎜ ..
. ⎟,
⎟
⎝ 1 ⎠

Examples
⎛ x1 ⎞
⎛ 1 ⎞
⎜ . ⎟, ⎜ x2 ⎟
J =⎜ .. ⎟ J⎜ ⎟
⎜ ⋮ ⎟
⎝ 1 ⎠ ⎝ ⎠
xn

Examples
⎛ x1 ⎞ ⎛ xn ⎞
⎛ 1 ⎞
⎜ . ⎟, ⎜ x2 ⎟ ⎜ ⋮ ⎟
J =⎜ .. ⎟ J⎜ ⎟ =⎜ ⎟
⎜ ⋮ ⎟ ⎜ x2 ⎟
⎝ 1 ⎠ ⎝ ⎠ ⎝ x1 ⎠
xn

Examples
⎛ x1 ⎞ ⎛ xn ⎞
⎛ 1 ⎞
⎜ . ⎟, ⎜ x2 ⎟ ⎜ ⋮ ⎟
J =⎜ .. ⎟ J⎜ ⎟ =⎜ ⎟
⎜ ⋮ ⎟ ⎜ x2 ⎟
⎝ 1 ⎠ ⎝ ⎠ ⎝ x1 ⎠
xn
▸ The upper circular shift matrix
⎛ 0 1 ⎞
⎜ 0 1 ⎟
⎜ ⎟
⎜
Z =⎜ ⋱ ⋱ ⎟,
⎟
⎜ ⋱ 1 ⎟
⎜ ⎟
⎝ 1 0 ⎠

Examples
⎛ x1 ⎞ ⎛ xn ⎞
⎛ 1 ⎞
⎜ . ⎟, ⎜ x2 ⎟ ⎜ ⋮ ⎟
J =⎜ .. ⎟ J⎜ ⎟ =⎜ ⎟
⎜ ⋮ ⎟ ⎜ x2 ⎟
⎝ 1 ⎠ ⎝ ⎠ ⎝ x1 ⎠
xn
⎛ 0 1 ⎞ ⎛ x1 ⎞
⎜ 0 1 ⎟ ⎜ x2 ⎟
⎜ ⎟ ⎜ ⎟
⎜
Z =⎜ ⋱ ⋱ ⎟, Z⎜ ⋮ ⎟
⎟ ⎜ ⎟
⎜ ⋱ 1 ⎟ ⎜ ⎟
⎜ ⎟ ⎜ xn−1 ⎟
⎝ 1 0 ⎠ ⎝ xn ⎠

Examples
⎛ x1 ⎞ ⎛ xn ⎞
⎛ 1 ⎞
⎜ . ⎟, ⎜ x2 ⎟ ⎜ ⋮ ⎟
J =⎜ .. ⎟ J⎜ ⎟ =⎜ ⎟
⎜ ⋮ ⎟ ⎜ x2 ⎟
⎝ 1 ⎠ ⎝ ⎠ ⎝ x1 ⎠
xn
⎛ 0 1 ⎞ ⎛ x1 ⎞ ⎛ x2 ⎞
⎜ 0 1 ⎟ ⎜ x2 ⎟ ⎜ x3 ⎟
⎜ ⎟ ⎜ ⎟ ⎜ ⎟
⎜
Z =⎜ ⋱ ⋱ ⎟, Z⎜ ⋮ ⎟ =⎜ ⋮ ⎟
⎟ ⎜ ⎟ ⎜ ⎟
⎜ ⋱ 1 ⎟ ⎜ ⎟ ⎜ x ⎟
⎜ ⎟ ⎜ xn−1 ⎟ ⎜ n ⎟
⎝ 1 0 ⎠ ⎝ xn ⎠ ⎝ x1 ⎠

Properties of Permutation Matrices
1 Permutation matrices are orthogonal and unitary.

That is, if P is a permutation matrix, then
PP T = P T P = PP H = P H P = I.

That is, if P is a permutation matrix, then
PP T = P T P = PP H = P H P = I.
Hints to understand:
n n
1, for i = j;
(P T P)ij = ∑ pik
T
pkj = ∑ pki pkj = {
k=1 k=1
0, for i ≠ j.

2 The product of permutation matrices is again a

permutation matrix.


permutation matrix.


permutation matrix.
Suppose you have two permutation matrices X and Y .
So by definition both X and Y are n × n matrices having precisely one
non-zero entry in each row and column, and these entries all take the
value 1.


permutation matrix.
value 1.
If the non-zero entry in the first row of X is in the ith column, then what
will the first row of the matrix X ⋅ Y be?


permutation matrix.
value 1.
If the non-zero entry in the first row of X is in the ith column, then what
will the first row of the matrix X ⋅ Y be?
By definition of matrix multiplication it will be
( ∑nk=1 X1,k Yk,1 n

∑k=1 X1,k Yk,2 ... ∑k=1 X1,k Yk,n )
n


( ∑nk=1 X1,k Yk,1 ∑nk=1 X1,k Yk,2 . . . ∑nk=1 X1,k Yk,n )


However, we know that X1,k = δi,k (where δi,k is the Kronecker delta
taking the value 1 when k = i and 0 otherwise).


Hence the first row of X ⋅ Y will be
( X1,i Yi,1 X1,i Yi,2 ... X1,i Yi,n )


( X1,i Yi,1 X1,i Yi,2 ... X1,i Yi,n )
which since X1,i = 1 will be
( Yi,1 Yi,2 ... Yi,n )
So the first row of X ⋅ Y corresponds to the ith row of Y .


( X1,i Yi,1 X1,i Yi,2 ... X1,i Yi,n )
( Yi,1 Yi,2 ... Yi,n )

Continuing in this way, you see that the mth row of X ⋅ Y will be the `th
row of Y , where the ` in the mth row of X appears in the `th position.


( X1,i Yi,1 X1,i Yi,2 ... X1,i Yi,n )
( Yi,1 Yi,2 ... Yi,n )

Continuing in this way, you see that the mth row of X ⋅ Y will be the `th
row of Y , where the ` in the mth row of X appears in the `th position.
Thus when you multiply two permutation matrices together, you are just
rearranging the rows of one of the matrices.
Discovery Journey
Outline
Discovery Journey
References

Discovery Journey
Things to do
1 Prove: If A is unitary, then AH , AT , and A are unitary.

Discovery Journey
Things to do

Tips: unitary if
AAH = AH A = I.

Discovery Journey
Things to do

Tips: unitary if
AAH = AH A = I.
2 What can you say about an involutory matrix that is also

unitary (orthogonal)?

Discovery Journey
Things to do

Tips: unitary if
AAH = AH A = I.

Tips: involutory if
AA = I.

Discovery Journey
Things to do

Tips: unitary if
AAH = AH A = I.

Tips: involutory if
AA = I.
⇒ AH = A(AT = A)

Discovery Journey
Things to do
3 Which idempotent matrix is unitary and orthogonal?

Discovery Journey
Things to do

Tips: idempotent if
AA = A.

Discovery Journey
Things to do

Tips: idempotent if
AA = A.
⇒ A=I

Discovery Journey
Things to do

Tips: idempotent if
AA = A.
⇒ A=I
4 Prove: If A is unitary, so is iA, where i 2 = −1.

Discovery Journey
Things to do

Tips: idempotent if
AA = A.
⇒ A=I
4 Prove: If A is unitary, so is iA, where i 2 = −1.
5 Prove: The product of unitary matrices is unitary.

Discovery Journey
Things to do
6 Partitioned Unitary Matrices.

Discovery Journey
Things to do
Let A ∈ Cn×n be unitary and partition
A = ( A1 A2 ) ,
where, A1 has k columns, and A2 has n − k columns.
Show that
1 A1 = Ik ,
AH 2 A2 = In−k , and A1 A2 = 0.
AH H

Discovery Journey
Things to do
A = ( A1 A2 ) ,
Show that
1 A1 = Ik ,
AH 2 A2 = In−k , and A1 A2 = 0.
AH H
Tips:
I = AH A

Discovery Journey
Things to do
A = ( A1 A2 ) ,
Show that
1 A1 = Ik ,
AH 2 A2 = In−k , and A1 A2 = 0.
AH H
Tips:
AH
1 )( A
I = AH A = ( 1 A2 )
AH
2

Discovery Journey
Things to do
A = ( A1 A2 ) ,
Show that
1 A1 = Ik ,
AH 2 A2 = In−k , and A1 A2 = 0.
AH H
Tips:
AH
1 )( A AH H
1 A1 A1 A2 )
I = AH A = ( H 1 A2 ) = (
A2 A2 A1 AH
H
2 A2

Discovery Journey
Things to do
7 Let x ∈ Cn and x H x = 1.

Discovery Journey
Things to do
Prove: In − 2xx H is Hermitian and unitary.

Discovery Journey
Things to do
Conclude that In − 2xx H is involutory.

Discovery Journey
Things to do
Conclude that In − 2xx H is involutory.
Tips: I = AH A = AA

References
Outline
Discovery Journey
References

References
Extension reading
▸ Julio Benı́tez and Vladimir Rakoc̆ević. Applications of CS decompositon in

linear combinations of two orthogonal projectors, Applied Mathematics and
Computation, 203(2): 761õ769, 2008. (doi:10.1016/j.amc.2008.05.053)

References
Homework
1 Show: If P is a permutation matrix, then P T and P H are also

permutation matrices.
2 Show: If (P1 P2 ) is a permutation matrix, then (P2 P1 ) is

also a permutation matrix.

THANKS FOR YOUR ATTENTION.
QUESTIONS ?

7 Special Matrics II

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

7 Special Matrics II

Uploaded by

Copyright:

Available Formats

Matrix Theory

––Special Matrices Again

Unitary and Orthogonal Matrices

Application 1: Givens rotation

Application 2: Permutation matrix

Matrix Theory Special Matrices - 2/23

Matrix Theory Special Matrices - 3/23

Matrix Theory Special Matrices - 3/23

Matrix Theory Special Matrices - 3/23

Matrix Theory Special Matrices - 3/23

Matrix Theory Special Matrices - 3/23

Matrix Theory Special Matrices - 3/23

Matrix Theory Special Matrices - 3/23

Unitary and Orthogonal Matrices

Application 1: Givens rotation

Application 2: Permutation matrix

Matrix Theory Special Matrices - 4/23

Matrix Theory Special Matrices - 5/23

Let c and s be scalars with

Matrix Theory Special Matrices - 5/23

Let c and s be scalars with

Matrix Theory Special Matrices - 5/23

Matrix Theory Special Matrices - 6/23

then the unitary 2 × 2 matrix

is called a Givens rotation.

Matrix Theory Special Matrices - 6/23

then the unitary 2 × 2 matrix

is called a Givens rotation.

If c and s are also real, then the Givens rotation

Matrix Theory Special Matrices - 7/23

Matrix Theory Special Matrices - 7/23

Matrix Theory Special Matrices - 7/23

Matrix Theory Special Matrices - 7/23

In numerical linear algebra, a Givens rotation is a rotation in the

Unitary and Orthogonal Matrices

Application 1: Givens rotation

Application 2: Permutation matrix

Matrix Theory Special Matrices - 8/23

Matrix Theory Special Matrices - 9/23

Matrix Theory Special Matrices - 9/23

For instance, consider the permutations on {1, 2, 3, 4}.

Matrix Theory Special Matrices - 10/23

For instance, consider the permutations on {1, 2, 3, 4}.

Matrix Theory Special Matrices - 10/23

Definition in Matrix Theory

Matrix Theory Special Matrices - 11/23

Definition in Matrix Theory

A square matrix is a permutation matrix if it contains

Matrix Theory Special Matrices - 11/23

Definition in Matrix Theory

A square matrix is a permutation matrix if it contains

Question: How many permutation matrices of order n?

Matrix Theory Special Matrices - 11/23

Definition in Matrix Theory

A square matrix is a permutation matrix if it contains

Question: How many permutation matrices of order n?

Matrix Theory Special Matrices - 11/23

Matrix Theory Special Matrices - 12/23

Matrix Theory Special Matrices - 12/23

Matrix Theory Special Matrices - 12/23

Matrix Theory Special Matrices - 12/23

Matrix Theory Special Matrices - 12/23

▸ The upper circular shift matrix

Matrix Theory Special Matrices - 12/23