Lecture 10

Sampling
Theorem

DigDigiitaltal SigSignnaall ProProcessincessingg,, ©© 20062006 RoRobbii Polikar,Polikar, RoRowanwan UnUniversityiversity

 Today in DSP
 The Sampling Theorem
ª The need for sampling and the aliasing
problem
ª Shannon’s
• Nyquistsampling
rate theorem
ª Effect of sampling in frequency domain
ª Sampling explained
graphically!
ª Recovering the original signal.

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Analog Æ Digital Æ Analog
 Most signals in nature are continuous in time
ª Need a way for “digital processing of continuous-time signals.”
 A three-step procedure
ª Conversion of the continuous-time signal into a discrete-time
signal
• Anti-aliasing filter – to prevent potentially detrimental effects of sampling
• Sample & Hold – to allow time to the A/D converter
• Analog to Digital Converter (A/D) – actual conversion in time and amplitude
ª Processing of the discrete-time signal
• Digital Signal Processing – Filter, digital processor
ª Conversion of the processed discrete-time signal back into a cont.-time signal
• Digital to analog converter (D/A) – to obtain the cont. time signal
• Reconstruction / smoothing filter - smooth out the signal from the D/A

Anti- Digital Reconstruction

aliasin S/H A/D processor D/A filter
g
filter
Digital Signal Processing, © 2006 Robi Polikar, Rowan University
 Aliasing

Alias:
Two names for the
same person, or thing.

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Aliasing
 Note that identical discrete-time signals may result from the sampling of more than one
distinct continuous-time function. In fact, there exists an infinite number of continuous-
time signals, which when sampled lead to the same discrete-time signal
x1=cos(2π3t), x2=cos(2π7t), x3=cos(2π13t)
1

0.8

0.6

0.4

0.2
Amplitude

-0.2

-0.4

-0.6

-0.8

-1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
T © Robi Polikar

Digital Signal Processing, © 2006 Robi Polikar, Rowan University ime

,
ms
 Aliasing
 Here is another example:
ª The signals in the following plot shows two sinusoids: x1[n]=cos(0.4πn) and
x2[n]=cos(2.4πn). Note that these two signals are distinct, as the second one
clearly has a higher frequency.
ª However, when sampled at, say integer values of n , they have the same values,
that is x1[n]=x2[n] for all integer n. These two signals are aliases of each other.
More
specifically, in the DSP jargon, we say that the frequencies ω = 0.4π and ω =2.4π
1 2
are aliases of each other.

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Aliasing
 In general, 2π multiples added or subtracted to a sinusoid gives aliases
of the same signal.
ª The one at the lowest frequency is called the principal alias, whereas those at the
negative frequencies are called folded aliases.
ª In summary, the frequencies at ω0 + 2πk and 2πk - ω0 for any integer k, are
aliases of each other.
ª We can further show that for folded aliases, the algebraic sign of the phase angle is
opposite that of the principal alias

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

Alias frequencies
 Houston…
We’ve got a problem!
 The fact that there exists an infinite number of continuous-time
signals, which when sampled lead to the same discrete-time signal,
poses a considerable dilemma in plotting and interpreting signals in
the frequency domain.
 Q: If the same discrete signal can be obtained from several
continuous time signals, how can we uniquely reconstruct the
original continuous time signal that generated the discrete signal at
hand?
 A. Under certain conditions, it is possible to relate a unique
continuous-time signal to a given discrete-time signal.
ª If these conditions hold, then it is possible to recover the original continuous-
time
signal from its sampled values

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Shannon’s
Sampling Theorem
 The solution to this complicated and perplexing phenomenon comes
from the amazingly simple Shannon’s Sampling Theorem, one of the
cornerstones of the modern communications, signal processing and
control.
ª A continuous time signal x(t), with frequencies no higher then Ωmax=2πfmax can be
reconstructed exactly, precisely and uniquely from its samples x[n] =
x(nTs), if
the
that samples are taken
is greater then at aThe
2fmax. sampling rate
frequency (or fs/2 or of ffmax
(frequency)
Ωs/2 s=1/T or (Ωthe
) is scalled s=2π/Ts)

Nyquist frequency (or folding frequency), as it determines the minimum

sampling frequency required. The minimum required sampling frequency is
then called the Nyquist rate.
ª In other words, if a continuous time signal is sampled at a rate that is at least
twice
as high (or higher) as the highest frequency in the signal, then it can be
ª uniquely
reconstructed from its samples.
Aliasing
Digital Signal Processing, can
© 2006 Robi be avoided
Polikar, if a signal is sampled at or above the Nyquist rate.
Rowan University

www.research.att.com
Digital Signal Processing, © 2006 Robi Polikar, Rowan University
Claude Shannon
Shannon, Claude Elwood (1916-2001), American applied mathematician
and electrical engineer, noted for his development of the theory of
communication now known as information theory. Born in Gaylord,
Michigan, Shannon attended the University of Michigan and in 1940
obtained his doctorate from the Massachusetts Institute of Technology,
where he became a faculty member in 1956 after working at Bell
Telephone Laboratories. In 1948 Shannon published “A Mathematical
Theory of Communication,” an article in which he presented his initial
concept for a unifying theory of the transmitting and processing of
information. Information in this context includes all forms of transmitted
messages, including those sent along the nerve networks of living
organisms; information theory is now important in many fields.
© 2003 Microsoft Encarta.
 Effect of Sampling
in the Frequency Domain
 Mathematically, we can show that the spectrum of the discrete
(sampled) signal is simply a 2π replicated and 1/Ts normalized version
of the spectrum of the original continuous time signal

1 ∞
X s (Ω) = ∑ X c (Ω − kΩ s )
Ts k =

−∞
Spectrum of the Sampling
Spectrum of the continuous time signal frequency
sampled signal

ω = ΩTs

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Effect of Sampling
in the Frequency Domain

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Sampling Explained …
 Let ga(t) be a continuous-time signal that is sampled uniformly at
t = nTs, generating the sequence g[n] where

−∞<n< g[n] = g a (nTs ),

∞
 Now, the frequency-domain representation of ga(t) is
given by its
continuous-time Fourier transform
∞ (CTFT):
Ga ( jΩ) = ∫ g a (t )e − jΩt dt
−∞
 The frequency-domain representation of g[n] is given by its discrete-
time Fourier transform (DTFT):

G(e jω ) = ∑∞n=−∞ g[n]e − jωn

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Sampling
 To establish the relation between Ga(Ω) and G(ω), we treat the
sampling operation mathematically as a multiplication of ga(t) by a
periodic impulse train p(t):
∞
p(t ) = ∑ − nTs ) g a (t ) × g p (t )
δ (t p(
n = −∞
t)

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Sampling
 The multiplication operation yields another impulse train:
∞
g p (t ) = g a (t ) p(t ) = ∑ g a (nT )δ (t − nTs )
n=
−∞
 gwith
p(t) is a continuous-time
the impulse at t = nTsignal consisting of
s weighted by the sampled value ga(nTs) of ga(t) at that
a train of uniformly spaced impulses
instant

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Sampling
 Now, note that p(t) is a periodic signal, therefore it can be represented with its FS
1 ∞ 1 ∞
j (2π / Ts )kt
p(t ) = ∑e = ∑ e jΩ s kt Ω s = 2π / Ts
Ts Ts k =
k =−∞
−∞
 The impulse train gp(t) therefore can be expressed as

⎛ 1 ∞ ⎞
g p (t ) =
⎜ Ts ∑ e jΩ s kt
⎟
⋅ g a(t )
⎝ k ⎠
=−
 From the frequency-shifting property∞of the CTFT, the CTFT of e jΩ s kt g a (t ) is
Ga (Ω − kΩ s )

 Hence Gp(Ω) is a periodic function of Ω consisting of a sum of shifted and scaled

replicas of Ga(Ω) , shifted by integer multiples of Ωs and scaled by 1/Ts
1 ∞
G p (Ω) = ∑ Ga (Ω − kΩ s )
Ts k = −∞

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Sampling
 The term on the RHS of this equation for k = 0 is the baseband
portion of Gp(Ω), and each of the remaining terms are the frequency
translated portions of Gp(Ω). 1 ∞
G p (Ω) = ∑ Ga(Ω − kΩ s)
Ts k =
−∞
 The frequency range – Ωs/2 ≤ Ω ≤ Ωs/2 is called the baseband or Nyquist
band

 OK, so the spectrum of the sampled signal is a replicated (by 2π)

version of that of the continuous signal. How does this solve the
aliasing dilemma mentioned earlier, or how does it lead to the
sampling theorem?
ª To see the answer, we look at what is happening in the frequency
domain
graphically
Digital Signal Processing, © 2006 Robi Polikar, Rowan University
 Sampling Explained…
…Graphically
ga(t)

ℑ
⇔
x
p(t)
*

g=
p(t)
=

* x
h(t)

=
^g=
a(t)

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Nyquist Rate
 Note that the key requirement for the Ga(Ω) recovered from Gp(Ω) is
that Gp(Ω) should consist of non-overlapping replicas of Ga(Ω).

 Under what conditions would this be satisfied…?

Ω s − Ω m ≥ Ω m ⇒ Ω s ≥ 2Ω m
?
If Ωs ≥2Ωm, ga(t) can be recovered exactly from gp(t) by passing it through an ideal
lowpass filter Hr(Ω) with a gain Ts and a cutoff frequency Ωc greater than Ωm and
less than Ωs- Ωm. For simplicity, a half-band ideal filter is typically used in exercises.
Digital Signal Processing, © 2006 Robi Polikar, Rowan University
 Aliasing - revisited
 On the other hand, if Ωs < 2Ωm, due to the overlap of the shifted
replicas of Ga(Ω) in the spectrum of Gp(Ω), the signal cannot be
recovered by filtering.
ª This is simply because the filtering of overlapped sections will cause a by
distortion
folding, or aliasing, the areas immediately outside the baseband back into the
baseband.

ª The frequency Ωs /2 is known as the folding frequency.

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 A Summary
 Given the discrete samples ga(nTs), we can recover ga(t) exactly by
generating the impulse train ∞
g p (t ) = ∑ g a (nTs )δ (t − nTs )
n=
and then passing it through an ideal lowpass filter Hr(Ω) with a gain Ts −∞

and a cutoff frequency Ωc satisfying Ωm ≤ Ωc ≤ Ωs- Ωm.

 The highest frequency Ωm contained in ga(t) is usually called the
Nyquist frequency since it determines the minimum sampling
frequency Ωs=2 Ωm that must be used to fully recover ga(t) from its
sampled version.
ª Sampling over or below the Nyquist rate is called oversampling or
undersampling, respectively. Sampling exactly at this rate is critical sampling. A
pure sinusoid may not be recoverable from critical sampling. Some amount of
oversampling is usually used to allow some tolerance.
ª E.g. In phone conversations, 3.4 kHz is assumed to be the highest frequency in
the speech signal, and hence the signal is samples at 8 kHz.
ª In digital audio applications, the full range of audio frequencies of 0 ~ 20kHz
is preserved. Hence, in CD audio, the signal is sampled at 44.1kHz.
Digital Signal Processing, © 2006 Robi Polikar, Rowan University
 A Simulink Demo
Use signal colors!!!

Si ne
Wave Scope
Continuous time signal

Power Spectral While it is sampled using

Densi the pulse generator, all
ty operations are
Scope1
technically in continuous
time
Product
Pul se
Gene
Po
Power Spectral
rator
Densi ty1
Pulse generator represents
the sampling operation.
Vary the period of the Buffer overlap =0
pulse generator

FFT
Zero-Order
Hol d Buffer Spectrum
This lower branch is just Scope
to displayusing
DFT blockset
functions

Scope2

Digital Signal Processing, © 2006 Robi Polikar, Rowan University

 Summary
 Sampling theorem – MUST sample at least twice highest frequency in
the signal!
 Aliasing will occur otherwise.
 If sampled at the appropriate rate, continuous time signal can be
recovered – exactly – from its sampled using low pass filtering.
ª How does this work…? We’ll see it on Wednesday!

Digital Signal Processing, © 2006 Robi Polikar, Rowan University