Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

Introduction

Fundamental
Equations

Matrix
Action

Orthonormal
Bases

Perpframes,
Aligners and
Hangers

Matrix Subspaces

Stretchers
Coordinates
Projections
SVD
Matrix
Subspaces
Linear
Systems,
PseudoInverse

Subspaces

Bonus
Facts
Exercises

In the last section you battled to show that every matrix A

could be written as A=(hanger)(stretcher)(aligner).
What's the pay o?

Fundamental equations
2x2 Example
Here is an SVD of a 2 x 2 matrix :
where the two perpframes are shown below.

Condition
Number
Matrix Norm,
Rank One
Data
Compression
Noise
Filtering
Todd Will
UW-La Crosse

1 of 11

Now watch what the matrix A does to an ellipse lined up on the

perpframe
.

03/06/15 21:09

Subspaces

2 of 11

http://websites.uwlax.edu/twill/svd/subspaces/ind...

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

The matrix stretches the ellipse and transfers it from the

perpframe to the
perpframe.
If you look carefully at the "during" plot you'll see that A sends
to

and

to

More generally A,
stretches vectors parallel to
them in the direction of ,

by a factor of 3 and rotates

stretches vectors parallel to

by a factor of

rotates them in the direction of

and

You can verify this by computing what each factor of A does.

Check what each factor of A does to

(The aligner matrix brings

to the y-axis.)

(The stretcher matrix stretches the y-axis by a

factor of

.)

(The hanger rotates the y-axis to the direction of

.)

Theorem:
If you build a 2x2 matrix A=(hanger)(stretcher)(aligner) using

a 2D perpframe

3 of 11

for your

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

a 2D perpframe

numbers
Then

and

for your

for your stretcher

and

These two equations are the fundamental equations of

matrices of the form (hanger)(stretcher)(aligner).
In words they say that A
stretches vectors in
and rotates them to
stretches vectors in
and rotates them to

the
the
the
the

direction
direction
direction
direction

of
of
of
of

by a factor of
,
by a factor of
.

Proof: Outlined above--follow the action of each factor of A on

.

3 x 3 matrices.
The following plot shows the perpframe
perpframe
in red.
See what the matrix
surface lined up on

4 of 11

in blue and the

does to the

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

The surface is stretched and rotated to the perpframe

Just as in the 2x2 case, the matrix A obeys the fundamental

equations:
In fact, matrices of the form (hanger)(stretcher)(aligner)
always obey these fundamental equations, regardless of their
dimension.

Orthonormal bases for fundamental

subspaces.
Suppose the SVD for a matrix

is
.

This decomposition presents orthonormal bases for

5 of 11

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

Proofs:
The Column space of A, Col[A], is the span of the columns of
A.
(The column way to multiply A v.)
(Since the a's are a basis.)
(Linearity)
(Fundamental
Equations)
.
This shows shows that Col[A] = span

Since
are perpendicular unit vectors they are an
orthonormal basis for Col[A].

The nullspace of A, N[A], is the set of vector that A sends to

the zero vector.
Suppose

6 of 11

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

Then

(Linearity and Fundamental

Equations.)

.
Now,

tells you that

and so
.

This tells you that any vector in N[A] is a linear combination of

and , i.e.
.
On the other hand,

, so

The row space of A is the span of the rows of A, which is the

same as the column space of
.
So nding a basis for the row space of A is the same as nding
a basis for the columns space of
.

Here's a look at

7 of 11

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

Now you have

written as (hanger)(stretcher)(aligner),
i.e., you have an SVD of

So from what you did above you know

is an orthonormal basis for the

is an orthonormal basis for

Comments
The proofs above easily generalize to show that
the columns of the hanger matrix corresponding to
non-zero singular values are an orthonormal basis for
Col[A].

the rows of the aligner matrix corresponding to non-zero

singular values are an orthonormal basis for Row[A].

the rows of the aligner matrix corresponding to zero

singular values are an orthonormal basis for N[A].

8 of 11

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

the columns of the hanger matrix corresponding to zero

singular values are an orthonormal basis for
.

Bonus Facts.
Denitions:
The rank of A is usually dened as the dimension of the
column space.
Since the columns of the hanger matrix corresponding to
non-zero singular values form a basis for the column space,
you know that the rank of A is equal to the number of non-zero
singular values.
Similarly, since the nullity of A is the dimension of the null
space, the number of zero singular values equals the nullity of
A.

Theorem: rank[A]+nullity[A]
Proof:
rank[A]+nullity[A]
=(number of non-zero singular values)+(number of zero
singular values)
=(total number of singular values)
=n.

Theorem:
9 of 11

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

Proof:
Rank[A]
=(number on non-zero singular values of A)

(number of non-zero singular values of

=

(1) Since the stretcher matrix for

is the transpose of the
stretcher matrix for A, the singular values for A and
are
identical.

Exercises
1. Go with the 3x4 matrix

Based on the SVD of

A:

give orthonormal bases for

a. the column space of A
b. the row space of A
c. the nullspace of A
d. the nullspace of

2. Let

10 of 11

Here is an SVD for A.

03/06/15 21:09

Subspaces

http://websites.uwlax.edu/twill/svd/subspaces/ind...

Based on the SVD, give an orthonormal basis for the subspace

spanned by

3. Let

. Here is an SVD for A.

Based on the SVD of A and the fundamental equations,

compute
a.

b.
If you check your answer by using

, you may not get

agreement since the SVD for A has been rounded to two

decimals.
1999 Todd Will
Last Modified: 03-Mar-1999

Hit Counter

11 of 11

03/06/15 21:09