STFT (revisited)

• Time/Frequency localization depends on window size.

• Once you choose a particular window size, it will be the same
for all frequencies.
• Many signals require a more flexible approach - vary the
window size to determine more accurately either time or
frequency.
 The Wavelet Transform

• Overcomes the preset resolution problem of the STFT by

using a variable length window:

– Narrower windows are more appropriate at high frequencies

(better time localization)

– Wider windows are more appropriate at low frequencies

(better frequency localization)
 The Wavelet Transform (cont’d)

Wide windows do not provide good localization

at high frequencies.
 The Wavelet Transform (cont’d)

A narrower window is more appropriate at high frequencies.

 The Wavelet Transform (cont’d)

Narrow windows do not provide good localization

at low frequencies.
 The Wavelet Transform (cont’d)

A wider window is more appropriate at low frequencies.

 What is a wavelet?
• It is a function that “waves” above and below the x-axis;
it has (1) varying frequency, (2) limited duration, and (3)
an average value of zero.

• This is in contrast to sinusoids, used by FT, which have

infinite energy.
Sinusoid Wavelet
 Wavelets
• Like sine and cosine functions in FT, wavelets can define
basis functions ψk(t):

f (t )   ak k (t )
k

• Span of ψk(t): vector space S containing all functions f(t)

that can be represented by ψk(t).
 Wavelets (cont’d)

• There are many different wavelets:

Haar Morlet Daubechies

Basis Construction - Mother Wavelet

  jk (t )

(dyadic/octave grid)
 Basis Construction - Mother Wavelet
(cont’d)

 jk (t )  2 j / 2  2 j t  k 

scale =1/2j j
(1/frequency)

k
 Continuous Wavelet Transform (CWT)

translation parameter, scale parameter

measure of time (measure of frequency)

1  t  
Forward C ( , s)   f  t 

 dt
CWT: s t  s 

Mother wavelet
normalization (window)
constant
scale =1/2j=1/frequency
 CWT: Main Steps

1. Take a wavelet and compare it to a section at the start

of the original signal.

2. Calculate a number, C, that represents how closely

correlated the wavelet is with this section of the
signal. The higher C is, the more the similarity.
 CWT: Main Steps (cont’d)

3. Shift the wavelet to the right and repeat steps 1 and 2 until
you've covered the whole signal.
 CWT: Main Steps (cont’d)

4. Scale the wavelet and repeat steps 1 through 3.

5. Repeat steps 1 through 4 for all scales.

 Coefficients of CTW Transform
• Wavelet analysis produces a time-scale view of the input
signal or image.

1  t  
C ( , s )   f  t    dt
s t  s 
Continuous Wavelet Transform (cont’d)

• Inverse CWT:

1 t 
f (t ) 
s
 s C ( , s) ( s )d ds

note the double integral!

FT vs WT
weighted by F(u)

weighted by C(τ,s)
 Properties of Wavelets

• Simultaneous localization in time and scale

- The location of the wavelet allows to explicitly represent
the location of events in time.
- The shape of the wavelet allows to represent different
detail or resolution.
 Properties of Wavelets (cont’d)

• Sparsity: for functions typically found in practice, many of the

coefficients in a wavelet representation are either zero or very
small.

1 t 
f (t ) 
s  s C ( , s) ( s )d ds
• Adaptability: Can represent functions with discontinuities or
corners more efficiently.
• Linear-time complexity: many wavelet transformations can be
accomplished in O(N) time.
Discrete Wavelet Transform (DWT)

a jk   f (t ) *jk (t ) (forward DWT)

t

f (t )   a jk jk (t ) (inverse DWT)

k j

where  jk (t )  2 j/2
  t k
2 j
 DFT vs DWT

• FT expansion: one parameter basis

or f (t )   a l l (t )
l

• WT expansion two parameter basis

f (t )   a jk jk (t )
k j
 Multiresolution Representation using  jk (t )

f (t )

fine
details
wider, large translations

f (t )   a  jk (t )
jk
k j j

coarse
details
 Multiresolution Representation using  jk (t )

f (t )

fine
details

f (t )   a  jk (t )
jk
k j j

coarse
details
 Multiresolution Representation using  jk (t )

f (t )

fine
details
narrower, small translations

f (t )   a  jk (t )
jk
k j j

coarse
details
 Multiresolution Representation using  jk (t )

f (t ) high resolution
(more details)

fˆ1 (t )
j
fˆ2 (t )
…
low resolution
fˆs (t ) (less details)

f (t )   a  jk (t )
jk
k j
 Prediction Residual Pyramid - Revisited
• In the absence of quantization errors, the approximation
pyramid can be reconstructed from the prediction residual
pyramid.

• Prediction residual pyramid can be represented more

efficiently.

(with sub-sampling)
Efficient Representation Using “Details”

details D3

details D2

details D1
L0
(without sub-sampling)
 Efficient Representation Using Details (cont’d)

representation: L 0 D 1 D2 D3
in general: L0 D1 D2 D3…DJ
A wavelet representation of a function consists of
(1)a coarse overall approximation
(2)detail coefficients that influence the function at various scales.
 Reconstruction (synthesis)
H3=H2+D3

H2=H1+D2
details D3

details D2 H1=L0+D1

details D1
L0
(without sub-sampling)
 Example - Haar Wavelets

• Suppose we are given a 1D "image" with a resolution

of 4 pixels:
[9 7 3 5]

• The Haar wavelet transform is the following:

L 0 D 1 D2 D3
(with sub-sampling)
 Example - Haar Wavelets (cont’d)
• Start by averaging the pixels together (pairwise) to get
a new lower resolution image:
[9 7 3 5]

• To recover the original four pixels from the two

averaged pixels, store some detail coefficients.

1
 Example - Haar Wavelets (cont’d)
• Repeating this process on the averages (i.e., low
resolution image) gives the full decomposition:

Harr decomposition:
 Example - Haar Wavelets (cont’d)

• We can reconstruct the original image by adding or

subtracting the detail coefficients from the lower-
resolution versions.
2 1 -1
 Example - Haar Wavelets (cont’d)

Detail coefficients
become smaller and
smaller as j increases.

Dj

Dj-1 How to
compute Di ?

D1
L0
 Multiresolution Conditions
• If a set of functions can be represented by a weighted
sum of ψ(2jt - k), then a larger set,
set including the
original, can be represented by a weighted sum of
ψ(2j+1t - k):
high
resolution

low
resolution
 Multiresolution Conditions (cont’d)
• If a set of functions can be represented by a weighted
sum of ψ(2jt - k), then a larger set, including the
original, can be represented by a weighted sum of
ψ(2j+1t - k):

Vj: span of ψ(2jt - k): f j (t )   ak jk (t )

k

Vj+1: span of ψ(2j+1t - k): f j 1 (t )   bk ( j 1) k (t )

k

V j  V j 1
 Nested Spaces Vj
Basis functions: Vj : space spanned by ψ(2jt - k)
ψ(t - k) V0 f (t )   a  jk (t )
f(t) ϵ Vj k j
jk

ψ(2t - k) V1

ψ(2jt - k) Vj

Multiresolution conditions  nested spanned spaces: V j  V j 1

i.e., if f(t) ϵ V j then f(t) ϵ V j+1
 How to compute Di ? (cont’d)

f(t) ϵ Vj f (t )   a  jk (t )
jk
k j

IDEA: define a set of basis

functions that span the
difference between Vj+1 and
Vj
 Orthogonal Complement Wj
• Let Wj be the orthogonal complement of Vj in Vj+1

Vj+1 = Vj + Wj
 How to compute Di ? (cont’d)
If f(t) ϵ Vj+1, then f(t) can be represented using basis
functions φ(t) from Vj+1:

Vj+1 f (t )   ck (2 j 1 t  k )
k

Alternatively, f(t) can be represented using two sets of basis functions,

φ(t) from Vj and ψ(t) from Wj:
Vj+1 = Vj + Wj

f (t )   ck (2 j t  k )   d jk (2 j t  k )
k k
 How to compute Di ? (cont’d)

Think of Wj as a means to represent the parts of a function

in Vj+1 that cannot be represented in Vj

f (t )   ck (2 j 1 t  k )
k

f (t )   ck (2 j t  k )   d jk (2 j t  k ) differences

k k between
Vj and Vj+1
Vj Wj
 How to compute Di ? (cont’d)
• Vj+1 = Vj + Wj  using recursion on Vj:

Vj+1 = Vj-1+Wj-1+Wj = …= V0 + W0 + W1 + W2 + … + Wj

if f(t) ϵ Vj+1 , then:

f (t )   ck (t  k )   d jk (2 j t  k )
k k j

V0 W0, W1, W2, …

basis functions basis functions
 Summary: wavelet expansion (Section 7.2)
• Wavelet decompositions involve a pair of waveforms
(mother wavelets):
encode low encode details or
resolution info φ(t) ψ(t) high resolution info

f (t )   ck (t  k )   d jk (2 j t  k )
k k j

scaling function wavelet function

 1D Haar Wavelets

• Haar scaling and wavelet functions:

φ(t) ψ(t)

computes average computes details

 1D Haar Wavelets (cont’d)

• V0 represents the space of one pixel images

• Think of a one-pixel image as a function that is constant

over [0,1)

width: 1
Example:
0 1
 1D Haar Wavelets (cont’d)

• V1 represents the space of all two-pixel images

• Think of a two-pixel image as a function having 21
equal-sized constant pieces over the interval [0, 1).

Examples: 0 ½ 1 width: 1/2

• Note that V0  V1

e.g., = +
 1D Haar Wavelets (cont’d)
• V j represents all the 2j-pixel images
• Functions having 2j equal-sized constant pieces over
interval [0,1).

Examples: width: 1/2j

• Note that V j 1  V j

e.g., ϵ Vj ϵ Vj
 1D Haar Wavelets (cont’d)

V0, V1, ..., V j are nested

i.e., V j  V j 1

VJ fine details
…

V1
V0 coarse details
 Define a basis for Vj

• Mother scaling function:

1
0 1

• Let’s define a basis for V j :

note alternative notation:  i ( x )   ji ( x )

j
Define a basis for Vj (cont’d)
 Define a basis for Wj (cont’d)
• Mother wavelet function:

1
-1

0 1/2 1

• Let’s define a basis ψ ji for Wj :

note alternative notation:  i ( x )   ji ( x )

j
Define a basis for Wj

Note
that the dot
product
between basis
functions in Vj
and Wj is zero!
 Define a basis for Wj (cont’d)

Basis functions ψ ji of W j
form a basis in V j+1
Basis functions φ ji of V j
Define a basis for Wj (cont’d)

V3 = V2 + W2
Define a basis for Wj (cont’d)

V2 = V1 + W1
Define a basis for Wj (cont’d)

V1 = V0 + W0
 Example - Revisited

f(x)=

V2
 Example (cont’d)

f(x)=
φ2,0(x)

V2 φ2,1(x)

φ2,2(x)

φ2,3(x)
 Example (cont’d)

(divide by 2 for normalization)

V1and W1 φ1,0(x)
V2=V1+W1
φ1,1(x)

ψ1,0(x)

ψ1,1(x)
Example (cont’d)
 Example (cont’d)
(divide by 2 for normalization)
V0 ,W0 and W1
φ0,0(x)
V2=V1+W1=V0+W0+W1
ψ0,0(x)

ψ1,0 (x)

ψ1,1(x)
Example
Example (cont’d)
 Filter banks
• The lower resolution coefficients can be calculated
from the higher resolution coefficients by a tree-
structured algorithm (i.e., filter bank).

Subband
encoding
(analysis)

h0(-n) is a lowpass filter and h1(-n) is a highpass filter

 Example (revisited)

[9 7 3 5]
low-pass, high-pass,
down-sampling down-sampling

(9+7)/2 (3+5)/2 (9-7)/2 (3-5)/2 V1 basis functions

 Filter banks (cont’d)
Next level:
 Example (revisited)

[9 7 3 5]

low-pass, high-pass,
down-sampling down-sampling

V1 basis functions

(8+4)/2 (8-4)/2
 Convention for illustrating
1D Haar wavelet decomposition
x x x x x x … x x
average
detail
…

re-arrange:

V1 basis functions
re-arrange:
 Examples of lowpass/highpass
(analysis) filters

h0
Haar
h1

h0

Daubechies
h1
 Filter banks (cont’d)
• The higher resolution coefficients can be calculated from
the lower resolution coefficients using a similar structure.

Subband
encoding
(synthesis)
 Filter banks (cont’d)
Next level:
 Examples of lowpass/highpass
(synthesis) filters

g0
Haar (same as
for analysis):
g1

g0
Daubechies
+ g1
 2D Haar Wavelet Transform

• The 2D Haar wavelet decomposition can be computed

using 1D Haar wavelet decompositions.
– i.e., 2D Haar wavelet basis is separable

• Two decompositions (i.e., correspond to different

basis functions):
– Standard decomposition
– Non-standard decomposition
 Standard Haar wavelet decomposition

• Steps

(1) Compute 1D Haar wavelet decomposition of each row of

the original pixel values.

(2) Compute 1D Haar wavelet decomposition of each

column of the row-transformed pixels.
 Standard Haar wavelet decomposition
(cont’d)
average
(1) row-wise Haar decomposition: detail

re-arrange terms
xxx … x … …
xxx … x …
… … . … … .
… … .
xxx ... x …
 Standard Haar wavelet decomposition
(cont’d)
average
(1) row-wise Haar decomposition: detail

row-transformed result

… …
…
… … . …
… … .
…
 Standard Haar wavelet decomposition
(cont’d)
average
detail
(2) column-wise Haar decomposition:

row-transformed result column-transformed result

… …
… …
…
… … . … … .
… …
 Example

row-transformed result
…
re-arrange terms …
…
… … .
…
… … .
Example (cont’d)

column-transformed result
…
…

… … .
…
2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider
all possible outer products of 1D basis functions.

φ0,0(x)

Example:
ψ0,0(x)
V2=V0+W0+W1

ψ1,0(x)

ψ1,1(x)
2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider
all possible outer products of 1D basis functions.

φ00(x), φ00(x) ψ00(x), φ00(x) ψ10(x), φ00(x)

i j ( x)   ji ( x)  i j ( x)   ji ( x)
2D Haar basis of standard decomposition

i j ( x)   ji ( x)
V2

 i j ( x)   ji ( x)
 Non-standard Haar wavelet decomposition

• Alternates between operations on rows and columns.

(1) Perform one level decomposition in each row (i.e., one

step of horizontal pairwise averaging and differencing).

(2) Perform one level decomposition in each column from

step 1 (i.e., one step of vertical pairwise averaging and
differencing).

(3) Repeat the process on the quadrant containing averages

only (i.e., in both directions).
 Non-standard Haar wavelet decomposition
(cont’d)

one level, horizontal one level, vertical

Haar decomposition: Haar decomposition:
xxx … x … …
xxx … x … …
… … . … … .
… … .
xxx ... x … …
 Non-standard Haar wavelet decomposition
(cont’d)
re-arrange terms
…
… one level, horizontal one level, vertical
Haar decomposition Haar decomposition
… … .
on “green” quadrant on “green” quadrant
…
…
…
…
…
… … .
…
 Example

re-arrange terms
…
…
… … .
…
…
 Example (cont’d)

…
…

… … .
…
 2D Haar basis for non-standard decomposition

• Defined through 2D scaling and wavelet functions:

 ( x, y )   ( x) ( y ) 000 ( x, y )   ( x, y )

 ( x, y )   ( x) ( y )  kfj ( x, y )  2 j  (2 j x  k , 2 j y  f )
 ( x, y )   ( x) ( y ) kfj ( x, y )  2 j (2 j x  k , 2 j y  f )
 ( x, y )   ( x) ( y )  kfj ( x, y )  2 j (2 j x  k , 2 j y  f )
2D Haar basis for non-standard decomposition
(cont’d)

i j ( x)   ji ( x)
V2
 i j ( x)   ji ( x)
 2D Haar basis for non-standard decomposition
(cont’d)
• Three sets of detail coefficients (i.e., subband encoding)

000 ( x, y )   ( x, y ) LL: average LL

Detail coefficients

LH: intensity variations along

 ( x, y )  2  (2 x  k , 2 y  f )
j
kf
j j j
columns (horizontal edges)

kfj ( x, y )  2 j (2 j x  k , 2 j y  f ) HL: intensity variations along

rows (vertical edges)
 kfj ( x, y )  2 j (2 j x  k , 2 j y  f ) HH: intensity variations along
diagonals
2D DWT using filter banks (analysis)
LL LH

HL HH
 2D DWT using filter banks (analysis)

LL
H LH
V HL
LL LH LL
D HH
HL HH

The wavelet transform is applied again on the LL sub-image

2D DWT using filter banks (analysis)
 2D Inverse DWT using filter banks
(synthesis)

H LH
V HL
D HH
 Wavelets Applications
• Noise filtering
• Image compression
– Fingerprint compression
• Image fusion
• Recognition
G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, "Face Recognition by
Fusing Thermal Infrared and Visible Imagery", Image and Vision
Computing, vol. 24, no. 7, pp. 727-742, 2006.
• Image matching and retrieval
Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast
Multiresolution Image Querying", SIGRAPH, 1995.
Fast Multiresolution Image Querying
painted low resolution target

queries

Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast

Multiresolution Image Querying", SIGRAPH, 1995.