An Introduction to Signal Expansions and the

Discrete Wavelet Transform

James E. Fowler
Department of Electrical & Computer Engineering
Geosystems Research Institute
Mississippi State University

1 / 38

Hilbert Spaces

Vectors
C: set of complex numbers
R: set of real numbers
Z: set of integers

A vector x is a N-tuple
x = {x1 , x2 , . . . , xN }

(1)

contained in C N (or RN )

2 / 38

Hilbert Spaces

Vector Spaces
A vector space is any subset E C N , coupled with addition and
multiplication operations, which satisfies:
commutation: x + y = y + x
association: (x + y) + z = x + (y + z); () x = (x)
distribution: (x + y) = x + y; ( + ) x = x + x
additive identity: there exists 0 E such that x + 0 = x

additive inverse: there exists x E such that x + (x) = 0

multiplicative identity: 1 x = x

3 / 38

Hilbert Spaces
Subspaces
A subspace is any set M E closed under addition and
multiplication:
x, y M x + y M

x M, C x M

(2)
(3)

For S E, the span of S, span (S), is a subset of E consisting of all

linear combinations of vectors in S:
(
)
X
span (S) =
i xi | xi S, i C
(4)
i

4 / 38

Hilbert Spaces

Bases
The vectors S = {x1 , x2 , . . . } are linearly independent if
X
i xi = 0

(5)

is true only when

i = 0,

(6)

The vectors S = {x1 , x2 , . . . } E form a basis of vector space E if

span (S) = E
the vectors of S are linearly independent

5 / 38

Hilbert Spaces

Basis Expansion
A basis allows the representation, or expansion, of any vector in a
vector space using the basis vectors.
If S = {x1 , x2 , . . . } is a basis of E, and y E, then there exist
constants i C such that
X
y=
i xi .

(7)

The set of expansion coefficients i is unique given y.

6 / 38

Hilbert Spaces
Inner Product
An inner product on vector space E is a function mapping
E E C such that

hx + y, zi = hx, zi + hy, zi
hx, yi = hx, yi
hx, yi = hy, xi
hx, xi 0 with equality only if x = 0

A vector space equipped with an inner product is an inner-product

space
A complete inner-product space is a Hilbert space (completeness:
all Cauchy sequences converge to a vector in the vector space)
p
The norm of x is kxk = hx, xi
Vector x and y are orthogonal iff hx, yi = 0

7 / 38

Hilbert Spaces
Finite-Dimensional Spaces
N is finite: x = {x1 , x2 , . . . , xN } C N
The conventional inner product:

The norm is then

N
X

xi yi

(8)

v
u N
uX 2
kxk = t
|xi |

(9)

hx, yi =

i=1

i=1

Holds for

RN

also

(R3

is 3D Euclidean space)

8 / 38

Hilbert Spaces
Square-Summable Sequences
Let x[n] be a discrete-time signal (sequence):
nZ

x[n] C,

(10)
n

(11)

Hilbert space 2 (Z) is the set of all sequences x[n] C such that
kxk <

(12)

p
hx, xi

X
x [n] y[n]
hx, yi =

(13)

where
kxk =

(14)

n=

9 / 38

Hilbert Spaces
Square-Integrable Functions
Let f (t) be a continuous-time signal (function):
tR

f (t) C,

(15)
t

(16)

Hilbert space L2 (R) is the set of all functions f (t) such that
kfk <

(17)

p
hf, fi
Z
f (t)g(t) dt
hf, gi =

(18)

where
kfk =

(19)

10 / 38

Hilbert Spaces
Orthonormal Bases
A vector x is normalized when
kxk = 1
An orthonormal set S = {xi } satisfies:
(
1, i = j,
hxi , xj i =
0, i 6= j

(20)

(21)

An orthonormal set of vectors that is also a basis is an

orthonormal basis

11 / 38

Hilbert Spaces
Orthonormal Basis Expansion
If S = {xi } is an orthonormal basis of Hilbert space E, then, for all
y E,
X
y=
i xi
(22)
i

where the expansion coefficients are

i = hxi , yi

(23)

If S is merely an arbitrary basis, the expansion coefficients i may

be difficult to calculate for a given vector y. The inner product
provides the coefficients for an orthonormal basis.

12 / 38

Time and Frequency Expansions

Finite-Length Sequences
Consider the Hilbert space C N of length-N sequenceswe will
look at two common bases for C N :

time basis
frequency basis

An important signal: the Kronecker delta

(
1, n = 0,
[n] =
0, n 6= 0

(24)

13 / 38

Time and Frequency Expansions

The Time Basis
For 0 m N 1, define vectors m
(
m [n] = [n m] =

1,
0,

n = m,
n 6= m

(25)

with discrete Fourier transform (DFT) m

h
i N1
X
[n m]ej0 nk = ej0 mk
m [k] = F [n m] =

(26)

n=0

where 0 =

2
N

14 / 38

Time and Frequency Expansions

The Time Basis
The set S = { m , m = 0, . . . , N 1} forms an orthonormal basis of
length-N sequences:
(
N1
X
1, m1 = m2 ,
h m1 , m2 i =
[n m1 ][n m2 ] =
(27)
0, m1 6= m2
n=0
For any length-N sequence x,
h m , xi =
x[n] =

N1
X

n=0
N1
X
m=0

[n m]x[n] = x[m]
x[m][n m] =

N1
X
m=0

h m , xim [n]

(28)

(29)

15 / 38

Time and Frequency Expansions

Orthonormal Time Basis for a Length-N Sequence
0 = [n]

h0 , xi0

x[n]

=
0

h1 , xi
| {z }
x[1]

x[n]

=
0

h2 , xi2

2 = [n 2]

h1 , xi1

1 = [n 1]
expanded as:

h0 , xi
| {z }
x[0]

h2 , xi
| {z }
x[2]

=
0

N
1
X

16 / 38

Time and Frequency Expansions

The Time Basis
In the time domain:
x[n] =

N1
X
m=0

h m , xim [n]

(30)

where m [n] = [n m] support: one value of n

In the frequency domain (take DFT of both sides):

X[k] =

N1
X
m=0

h m , xim [k]

(31)

h
i
where m [k] = F m [n] = ej0 mk support: all values of k
17 / 38

Time and Frequency Expansions

The Frequency Basis
Let l [n] =

1 ej0 nl
N

where 0 =

2
N

The set S = {l , l = 0, . . . , N 1} forms an orthonormal basis of

length-N sequences
The DFT of length-N sequence x[n] is
h
i N1
X
X
N1

x[n]ej0 nl = N
l [n]x[n] = Nhl , xi
X[l] = F x[n] =
n=0

n=0

(32)

The inverse DFT is

N1
N1
h i
X
1 X
hl , xil [n]
X[l]ej0 nl =
x[n] = F 1 X[l] =
N
l=0

(33)

l=0

18 / 38

Time and Frequency Expansions

The Frequency Basis
In the time domain:
x[n] =

N1
X
l=0

where l [n] =

1 ej0 nl
N

hl , xil [n]

(34)

support: all values of n

In the frequency domain (take DFT of both sides):

X[k] =

N1
X
l=0

hl , xil [k]

(35)

h
i
where l [k] = F l [n] = N[k l] support: one value of k
19 / 38

Time and Frequency Expansions

Representation of a Sequence
Example for sequence of length N = 4

Time-basis Representation
m [n] = [n m], m [k] = ej0 mk
(corresponds to the natural time-domain representation)

Frequency-basis Representation

l [n] = 1N ej0 nl , l [k] = N [k l]

(corresponds to the frequency-domain representation given by DFT)

1
0
0

1
2
time
n

k
frequency

k
frequency

0
0

1
2
time
n

20 / 38

Time and Frequency Resolution

Resolution
Resolution: the ability to distinguish between two closely located
signal features
Time resolution: n = time separation between bases
rt =

1
n

(36)

Frequency resolution: k = frequency separation between bases

rf =

1
k

(37)

21 / 38

Time and Frequency Resolution

Time-basis Representation
m [n] = [n m], m [k] = ej0 mk

Frequency-basis Representation

l [n] = 1N ej0 nl , l [k] = N [k l]

1
0
0

1
2
time
n

n = 1, k = N
rt = 1, rf = 1/N

k
frequency

k
frequency

0
0

1
2
time
n

n = N, k = 1
rt = 1/N, rf = 1
22 / 38

Time and Frequency Resolution

Continuous-Time Time/Frequency Bases
Time basis: Dirac delta, (t)
t = 0
=

rt =

1
t

rf =

(38)

=0

Frequency basis: continuous-time Fourier transform

t =
= 0

rt =

rf =

1
t

=0

(39)

23 / 38

General Time-Frequency Tilings

Continuous-Time Time/Frequency Bases
Time and frequency (Fourier) bases are all or nothing (resolution
is 0 or )
We would like bases with 0 < rt , rf <

Define spread (t or ) as second moment:

(t)2 =

R 2
t |(t)|2 dt
R
|(t)|2 dt

time domain

()2 =

2 |()|2 d
R
|()|2 d

(40)

frequency domain

24 / 38

General Time-Frequency Tilings

The Uncertainty Principle

For any (t) with Fourier transform (),
t

1
2

or rt rf 2

(41)

We can have arbitrarily large resolution (time or frequency) only by

giving up resolution of the other quantity
Example:

To get t = 0, must be (which is time basis)

For < , we must use t > 0

25 / 38

The Short-Time Fourier Transform

The Continuous-Time Fourier Transform
F() = F [f (t)] is the set of coefficients of an expansion of
continuous-time function f (t) L2 (R) using basis functions
(t) = ejt

(42)

which provides a basis function for each R

Resolution:

= 0
t =

(43)
(44)

The short-time Fourier transform (STFT) attempts to improve the

time resolution by windowing each (t) to reduce its time extent
26 / 38

The Short-Time Fourier Transform

The STFT
The STFT:

1
f (t) =
2

Z Z
R

S(, ), (t) d d

(45)

Basis functionsFourier basis coupled with window function g(t):

, (t) = g(t )ejt

(46)


S(, ) = , , f
Z

,
(t)f (t) dt
=
ZR
f (t)g(t )ejt dt
=

(47)

Coefficients:

(48)
(49)

27 / 38

The Short-Time Fourier Transform

The STFT Basis Functions
Basis function , (t) is indexed by two indices:

Time:
Frequency:

and determine location in time-frequency plane of the basis

functions corresponding tile:

|2 ,2 (t)|2

|1 ,1 (t)|2

|1 ,1 (t)|2

|2 ,2 (t)|2

t
28 / 38

The Short-Time Fourier Transform

The STFT Window
The window function, g(t), is chosen to give the desired resolution
tradeoff, subject to t 12
Wide window (t large) good frequency resolution ( small)

Narrow window (t small) poor frequency resolution ( large)

Narrow window

Wide window
29 / 38

The Short-Time Fourier Transform

The Balian-Low Theorem

If the STFT basis functions , (t) form an orthogonal basis, then
either t or must be
To have good resolution in time and frequency simultaneously in
STFT, one must use a non-orthogonal basis.

30 / 38

The Short-Time Fourier Transform

Advantages of Orthonormal Bases
If f E is a vector in Hilbert space E, and S = { i } is a basis of E, then
X
f=
i i
(50)
i

If S is an orthonormal basis:
Easy calculation of expansion coefficients:
i = h i , fi

(51)

Parsevals theorem:
kfk2 =

X
i

|i |2

(52)

That is, total energy of f is partitioned among the i coefficients.

31 / 38

Wavelet-Series Expansion
Wavelet-Series Expansion
Commonly called the discrete wavelet transform (DWT)
Expands a continuous-time function into coefficients which are
discrete in time and frequency:
XX
f (t) =
aj,k j,k (t)
(53)
k

time: k Z

frequency: j Z

Similar to STFT in that decomposition is in terms of both time and

frequencybut, orthonormal wavelet bases exist with good time
and frequency resolution simultaneously
32 / 38

Wavelet-Series Expansion

The Mother Wavelet

Wavelet-series expansion systems are generated from one
function, (t), called the mother wavelet
The basis functions are created by scaling and translating the
mother wavelet:
j,k (t) = 2j/2 (2j t k)

for j, k Z

The 2j/2 factor maintains normalization:

j,k 2 = kk2 = 1

(54)

(55)

33 / 38

Wavelet-Series Expansion
Scaling and Translation
(t)

j=0

j=1

j=2

34 / 38

Wavelet-Series Expansion
Scaling and Translation
k controls displacement of j,k (t) in time
j is the scaleincreasing j yields:

compaction of j,k (t) in time

j,k (t) functions spaced closer together in time
taller j,k (t) functions

When j is large:

j,k (t) functions have small t; thus, is large

good time resolution

When j is small:

j,k (t) functions have large t; thus, is small

good frequency resolution

35 / 38

frequency

Wavelet Time-Frequency Tiling

time
36 / 38

Wavelet Time-Frequency Tiling

Why is this a good tiling?

Natural signals are usually lowpasswavelet tiling concentrates
frequency resolution where majority of signal energy resides
Discontinuities (edges) involve high-frequency energy that
accounts for a small portion of total energylocation in time is
more important than frequency compositionwavelet tiling has
good time resolution for high-frequency content

37 / 38

Summary

Wavelets provide an expansion in the form:

XX


j,k , f j,k (t)
f (t) =
k

(56)

The expansion can be orthonormalcoefficients can be

calculated with inner products
The expansion is multiresolutiontime and frequency resolutions
vary across time-frequency plane
The time-frequency tiling matches characteristics of natural
(lowpass) signals with salient discontinuities

38 / 38