Lectures 6,7 & 8: Similarity transformation &

Convolution theorem
AT
Similarity transformation - Change of basis
Let T : (S
1
, ) (S
1
, ) be a linear operator on an n dimensional vector space. Using isomorphism
P
1
: (S
1
, ) R
n
we can nd a representation [T]

: R
n
R
n
of the linear operator T. Let
= {
1
, . . . ,
n
} be a different basis on S
1
. Again using isomorphism P
2
: (S
1
, ) R
n
, we can
compute a representation [T]

: R
n
R
n
of T. The relation between [T]

and [T]

is given by a
similarity transformation.
Let
[M]

=
_

_
.
.
.
.
.
.
.
.
.
[
1
]

. . . [
n
]

.
.
.
.
.
.
.
.
.
_

_
represent the change of representation from basis to basis . Then [M]

=
_
[M]

_
1
. Then the two
representations of T are related as follows
[T]

= [M]

[T]

[M]

.
Similarly,
[T]

= [M]

[T]

[M]

.
In general, two nn matrices Aand B are said to be similar iff M R
nn
such that A = M
1
BM.
It is easy to show that similarity is an equivalence relation.
Eigenvectors as basis
Let us assume a basis of S
1
, = {
1
, . . . ,
n
} such that T
i
=
i

i
, i = 1, . . . , n. Then [T
i
]

=
(0, . . . , 0,
i
, 0, . . . , 0)
t
, where
i
is in the i
th
position. Collected together, we get
[T]

=
_

1
0 . . . . . . 0
0
2
0 . . . 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 . . . . . . . . .
n
_

_
Let us denote this diagonal matrix as . Then [T]

= [M]

[M]

. In order to compute [Tx]

, x
(S
1
, ) we can either use [Tx]

= [T]

[x]

or [Tx]

= [M]

[M]

[x]

. In the former case, we need

O(n
2
) multiplications, whereas for multiplication with we need O(n) multiplications only. One
can argue that we need additional multiplications in the change of basis matrices. We will later see in
1
case of LSI systems there is a specic eigenvector basis for which the change of basis multiplications
do not cost much.
For the given basis , assume
i
=

n
j=1
v
ji

j
, i = 1, . . . , n. Then
[M]

=
_

_
v
11
. . . v
1n
.
.
.
.
.
.
.
.
.
v
n1
.
.
. v
nn
_

_
Linear shift invariant systems
Let us consider a linear shift invariant systemgiven by a circular convolution y[n] =

N1
k=0
x[k]h[(n
k)mod N. Note that this rule applies equally well to the case where x, h and y could be complex
periodic sequences. Therefore we treat such a system as a linear operator between complex vector
spaces T : C
N
C
N
. For the sake of simplicity let us pick N = 4. Whatever we do next applies
to any period N. We work with the standard basis of C
4
denoted by = {
1
, . . . ,
4
}, where

1
= (1, 0, 0, 0)
t
,
2
= (0, 1, 0, 0)
t
,
3
= (0, 0, 1, 0)
t
and
4
= (0, 0, 0, 1)
t
. In terms of our usual
notation we have S
1
= C
4
. We can do away with the isomorphism P
1
. Let the impulse response of
the system T be given as h = (h
0
, h
1
, h
2
, h
3
) C
4
. We have already derived the following in the
previous lecture:
[T]

=
_

_
h
0
h
3
h
2
h
1
h
1
h
0
h
3
h
2
h
2
h
1
h
0
h
3
h
3
h
2
h
1
h
0
_

_
Let us now work with the eigenvectors. The rst eigenvector is obviously v
1
= (1, 1, 1, 1)
t
.
[T]

v
1
= (h
0
+h
1
+h
2
+h
3
)
. .

1
v
1
Let w = exp
_
j

2
_
(for a period N, w = exp
_
j
2
N
_
). With v
2
= (1, w
1
, w
2
, w
3
)
t
, we have
[T]

v
2
= (h
0
jh
1
h
2
+jh
3
)
. .

2
v
2
Simlarly for v
3
= (1, w
2
, w
4
, w
6
)
t
and v
4
= (1, w
3
, w
6
, w
9
)
t
, we get
[T]

v
3
= (h
0
h
1
+h
2
h
3
)
. .

3
v
3
and
[T]

v
4
= (h
0
+jh
1
h
2
jh
3
)
. .

4
v
4
Therefore we have
[T]

= =
_

1
0 0 0
0
2
0 0
0 0
3
0
0 0 0
4
_

_
Note in this case, we have
[M]

=
_

_
1 1 1 1
1 w
1
w
2
w
3
1 w
2
w
4
w
6
1 w
3
w
6
w
9
_

_
2
which gives [T]

[M]

= [M]

which can be written as [T]

= [M]

[M]

. We now need to
compute [M]

.
Let
_
[M]

denote the conjugate transpose of [M]

.
_
[M]

=
_

_
1 1 1 1
1 w
1
w
2
w
3
1 w
2
w
4
w
6
1 w
3
w
6
w
9
_

_
Then we see that
[M]

_
[M]

=
_

_
4

3
i=0
w
i

3
i=0
w
2i

3
i=0
w
3i
w
3

3
i=0
w
i
4

3
i=0
w
i

3
i=0
w
2i
w
6

3
i=0
w
2i
w
3

3
i=0
w
i
4

3
i=0
w
i
w
9

3
i=0
w
3i
w
6

3
i=0
w
2i
w
3

3
i=0
w
i
4
_

_
Let p = w
m
, m = 1, 2 or 3. Let k =

3
i=0
p
i
. Note that w
4
= 1 (in general w
n
= 1, where n is the
period of the sequence). Then,
k =
3

i=0
p
i
p k = p
3

i=0
p
i
=
4

i=1
p
i
Since p
4
= (w
m
)
4
= (w
4
)
m
= 1
m
= 1
p k =
3

i=0
p
i
p k = k
Since p = 1, we get k = 0.
Therefore [M]

_
[M]

= 4 I
4
. Hence [M]

=
_
[M]

_
1
=
1
4
_
[M]

. We then have [T]

=
[M]

[M]

. Let us introduce the following notations: F = 4 [M]

= ([M]

,
F =
_

_
1 1 1 1
1 w
1
w
2
w
3
1 w
2
w
4
w
6
1 w
3
w
6
w
9
_

_
and F
1
=
1
4
[M]

,
F
1
=
1
4
_

_
1 1 1 1
1 w
1
w
2
w
3
1 w
2
w
4
w
6
1 w
3
w
6
w
9
_

_
.
That gives us [T]

= [M]

[M]

= F
1
F. Recollect that the basis correspond to the time-
domain impulses. Observe that [x]

= F[x]

(ignoring the scaling factor of 4 and

1
4
). Rewritten in
more familiar terms, we get the equation dening the Discrete Fourier Transform.
X(k) =
n1

m=0
x[m] exp
_
j
2
n
mk
_
, k = 0, . . . , n 1 (1)
3
We also have [x]

= F
1
[x]

=
1
4
[M]

[x]

, which denes the Inverse Discrete Fourier Transform as

x[m] =
1
n
n1

k=0
X(k) exp
_
j
2
n
mk
_
, m = 0, . . . , n 1
Now let y = Tx. We have [y]

= F
1
F[x]

. This gives F[y]

= F[x]

. Also note that

F[h]

= ([M]

[h]

_
1 1 1 1
1 w
1
w
2
w
3
1 w
2
w
4
w
6
1 w
3
w
6
w
9
_

_
_

_
h
0
h
1
h
2
h
3
_

_
=
_

_
h
0
+h
1
+h
2
+h
3
h
0
jh
1
h
2
+jh
3
h
0
h
1
+h
2
h
3
h
0
+jh
1
h
2
jh
3
_

_
=
_

4
_

_
Therefore we nally have
[y]

= [T]

[x]

[y]

= diag (F[h]

) [x]

(2)
or the more familiar
y[m] =
n1

k=0
x[k]h[(nk)mod n], m = 0, . . . , n1 Y (k) = H(k)X(k), k = 0, . . . , n1 (3)
Some conclusions:
1. The DFT is simply a change of basis matrix and it simply represents the input in a different
basis.
2. The DFT of the impulse response gives the eigenvalues of the LSI system it describes.
3. The DFT diagonalizes any circulant matrix, i.e. the columns of the DFT matrix are eigenvectors
of any LSI system.
Continuous time signals
In this case the convolution integral is given as
y(t) =
_

x(t )h()d
Let us use the input signal x(t) = exp(jt). We get
y(t) =
_

exp(j(t ))h()d
= exp(jt)
_

exp(j)h()d
= exp(jt)H()
= H()x(t)
Thus we can conclude that signals of the form exp(jt), R are eigenvectors of any LSI system
and the eigenvalues are given by the Fourier transform of the impulse response.
4