Notes on Discrete Fourier Transforms

Physics 204B

We’ve discussed Fourier series expansions, which apply to a periodic function or to a function
defined on a finite interval that can be assigned a periodic extension. The original function is
continuous, and to calculate the Fourier series its value must be known throughout the interval.
The result in Fourier space is an infinite set of coefficients corresponding to discrete values of
frequency (or wave vector).

We’ve also discussed continuous Fourier transforms, which are essentially a limiting case
where the interval length becomes infinite. In this case the Fourier transform is itself continuous
and infinite.

Another possibility, hugely important for actual data and for computational work, is that
the original function is sampled at discrete intervals. For example, imagine that a voltage is
measured 100 times per second. As long as the voltage changes slowly compared to the time
interval of 0.01 seconds, this discrete measurement should be capturing the gist of the signal.
We now consider how to decompose the data into Fourier components through a discrete Fourier
transform. The initial data are discrete and span a finite total time; we will see that the Fourier
transform also consists of discrete frequencies confined to a finite range.

T
Let’s assume that N points are sampled over a time T . The sampling times are t = 0, N ,
2T kT (N −1)T
N
,..., N ..., N . We’ll now consider the N lowest-frequency complex exponentials that are
2πn
periodic on [0, T ]. These are ei T t , for 0 ≤ n ≤ N . (These are the first N functions that would
be used in a Fourier series expansion on an interval T , if the function to be expanded were known
at all times.) From each of these complex exponentials, construct an N -dimensional vector from
its values at the N sampling times.

For example, with N = 6, the vectors are:

n = 0 → (1, 1, 1, 1, 1, 1)
2π T π 2π 2T 2π 2π 3T 2π 4T 4π 2π 5T 5π
n = 1 → (1, ei T 6 = ei 3 , ei T 6 = ei 3 , ei T 6 = −1, ei T 6 = ei 3 , ei T 6 = ei 3 )
4π T 2π 4π 2T 4π 4π 3T 4π 4T 2π 4π 5T 4π
n = 2 → (1, ei T 6 = ei 3 , ei T 6 = ei 3 , ei T 6 = 1, ei T 6 = ei 3 , ei T 6 = ei 3 )
n = 3 → (1, −1, 1, −1, 1, −1)
4π 2π 4π 2π
n = 4 → (1, ei 3 , ei 3 , 1, ei 3 , ei 3 )
5π 4π 2π π
n = 5 → (1, ei 3 , ei 3 , −1, ei 3 , ei 3 )

Each entry of each vector is an N th root of 1, and for a given

√ vector the ratio of consecutive
entries is always the same. Each of these vectors has length N . In addition, any two vectors
with different n are orthogonal. (I assume the vectors have the same N , since otherwise even
talking about orthogonality makes no sense.) To demonstrate orthogonality, the key point is that
the N th roots of 1 add to zero, which can be seen pictorially by plotting the roots in the complex
plane, or algebraically by summing a geometric series. Here is the algebraic version, starting
P −1 i 2πnk −i 2πmk
with the inner product of two vectors with 0 ≤ n, m ≤ N and n 6= m: N k=0 e
N e N =
PN −1 i 2π(n−m)k 1−ei2π(n−m)
k=0 e = 2π(n−m) = 0.
N
i
1−e N
 Since for a given N we defined exactly N of these vectors derived from complex exponentials
and then learned that they are mutually √ orthogonal, these N vectors form an orthogonal basis
(orthonormal if we divide each vector by N ). We can express any N -dimensional vector, such
as the data set, in terms of these basis vectors. This is the discrete Fourier transform:
−1 −1
1 NX i 2πnk 1 NX 2πnk
fk = √ cn e N , cn = √ fk e−i N
N n=0 N k=0

As with continuous Fourier transforms, the prefactors can be split in different ways between
the two directions, as long as the product is N1 . I prefer the symmetric version given above, but
many people (and software versions!) use 1 in one direction and N1 in the other.

These formulas use N coefficients fk and N coefficients cn ; in general both the f ’s and c’s can
be complex. However, in many experimental cases, such as the hypothetical voltage measurement
described above, the input values for fk are real. That produces a bit of a puzzle: if we start with
N pieces of information (the real values of fk ), then how can we create 2N pieces of information
(the real and imaginary parts of cn ) from them? The answer is that those 2N values are not
independent. In fact the redundancy shows up in quite a simple way. If all the f ’s are real, then
−1 −1 −1
1 NX 2π(N −n)k 1 NX 2π(−n)k 1 NX 2πnk
c∗N −n = √ (fk e−i N )∗ = √ fk ei2πk ei N = √ fk e−i N = cn .
N k=0 N k=0 N k=0

So only the coefficients up to N2 have independent information. This quick calculation also
illustrates that one could extend the cn ’s periodically beyond the interval 0 ≤ n ≤ N , so that
c−n = cN −n , etc. Also note that taking n = 0 or n = N/2 (the latter for even N only) shows
that c0 and cN/2 are real.

The fact that the cn ’s are not defined for n ≥ N , and in fact have no physical significance for
real data when n ≥ N2 , gives a limit on the highest frequency that can be detected. Returning to
the original exponential basis functions, that frequency (in the case of real data) is 2π(N/2)
T
= ∆tπ
,
T
where δt = N is the time interval between consecutive points. This is known as the Nyquist
limit; you can only hope to resolve frequencies if you sample at least twice per period.

Higher frequencies are “aliased” back to lower-frequency values that fall within the allowed
2πN
range. First imagine a signal ei T t . Sampling at times t = kT
N
always gives a value of 1, so this
2π(N +1) 2π
wave is indistinguishable from a constant function. Similarly, ei T t and ei T t are identical at
times kT
N
, even though they differ at intermediate times. Qualitatively, the point is that you are
blind to any extra wiggles that your signal may undergo in between your sampling times.

We’ve just established that the interval between points in real space sets a maximum frequency
in Fourier space. From the symmetry of the Fourier transform equations, there should be an
analogous relationship between the spacing between allowed frequencies and the total sampling
time in real space. This makes sense from some physical reasoning: even if two waves have very
similar frequencies (small spacing in Fourier space), they can be distinguished by measuring for
a sufficiently long time that they get out of phase with each other (large total time). For shorter
measurement times they may be very close together at all sampled points, although the values
will not be exactly identical as in the case of aliasing of high frequencies.
 On a related note, it’s worth considering what happens when an incoming wave doesn’t
match perfectly with any of the discrete frequencies in the Fourier decomposition. As for Fourier
series expansions, the discrete Fourier transform implicitly continues the signal as a periodic
function. The frequencies used for the Fourier transform are exactly those that truly do give
periodic functions on the relevant interval. Data at other frequencies are expanded to functions
that are not simple waves. The discrete Fourier transform will have a large component at the
closest allowed frequency, plus smaller components at other frequencies. Two waves with nearly
identical frequencies, to small to be properly resolved on the time interval used, will have their
main peak in the same place. The Fourier space distinction between them will rest solely on the
differences in the other, more minor components.

A nice feature of finite data sets is that counting up your information is relatively straightfor-
ward. Warning lights should always go off in your head if you seems to be getting something for
nothing. In particular, it can be tempting to “zero-pad” your data by adding a string of zeroes
after the measured points. You may even be measuring a signal that has an exponential decay,
so it really is very close to zero by the end of the data set. Adding the extra zeroes increases
the total time of the measurement, which decreases the spacing between frequencies. Let’s say
you double the time by adding exactly as many zeroes as you have actual measurements. The
new Fourier transform will look very nice and take on values at twice as many frequencies. How-
ever, these values are not independent of each other. In fact you are essentially “finding” the
intermediate frequency components by averaging the components on either side. That’s why the
transform is typically pretty and smooth, and can tempt unsuspecting students to take it more
seriously than it deserves.

So what do Fourier transforms get you if there’s no new information? It’s just a new way
of viewing the data that can be helpful. For example, a signal at a particular frequency can
easily be obscured by white noise in real space, but still appear clearly in Fourier space. Perhaps
if the human eye worked differently we’d be able to see it just as well in real space. Another
possibility is that you want to fit some functional form to a signal, and the Fourier-space fit
function is easier to deal with. There also could be data processing reasons, such as removing
the entire component at the frequency of a nearby radio station and then examining what’s left
after Fourier transforming back to real space.

You’ll sometimes hear about “fast” Fourier transforms, or FFT’s. This is a particular al-
gorithm for calculating the discrete Fourier transform sums. It has an interesting history; the
mathematics behind it was discovered and then lost more than once, starting in the 1800’s. Be-
fore computers existed the entire calculation was highly impractical, and having an algorithm
that made it slightly less impractical wasn’t very exciting. The basic idea is noting that the same
complex exponential factors show up in all the coefficients, and that while they are permuted in
different ways these permutations do follow a pattern. A brute force calculation of the sums can
be done in N 2 time; that is, N 2 multiplication operations are needed: N terms for each of N
coefficients. (I ignore the time to calculate the N complex exponentials themselves.) The FFT
trick is to break up a length-N Fourier transform into two smaller transforms of length N2 . Here
is how one step works, assuming that N is even:
 N N
N −1 2
−1 2
−1
2π(2k)n 2π(2k+1)n
−i 2πkn −i N
cN f2k+1 e−i
X X X
n = fk e N = f2k e + N

k=0 k=0 k=0

N N
2
−1 2
−1
−i 2πkn 2πkn πn πn
f2k+1 e−i N/2 e−i N/2 = cN/2 + e−i N/2 c0N/2
X X
= f2k e N/2 + n n
k=0 k=0

A discrete Fourier transform using only the N2 even-numbered points from the original data
set gives coefficients cn , while a second discrete Fourier transform using only the odd-numbered
points gives coefficients c0n . Each of these smaller Fourier transforms takes ( N2 )2 multiplications.
The Fourier coefficients for the full data set can be calculated from these two smaller discrete
Fourier transforms, using only one additional multiplication per coefficient to put in the phase
shift between the two transforms. Thus instead of a direct process that takes N 2 multiplications,
we can use 2( N2 )2 + N multiplications instead.

If N = 2Q is in fact a power of 2, then this process can be repeated. Breaking it down one
more step would go from 2 · 22(Q−1) + 2Q multiplications to 2(2 · 22(Q−2) + 2Q−1 ) + 2Q , and one
further step yields 2[2(2 · 22(Q−3) + 2Q−2 ) + 2Q−1 ] + 2Q , etc. Note that each term except the first
one is exactly 2Q . After Q − 1 steps, the first term is reduced to a final 2Q term. With a total of
Q terms, this means 2Q Q = N ln N multiplications. This difference between N 2 time and N ln N
time can be important for large data sets, and twenty years ago it mattered even for moderately
sized data sets. Perhaps the most important idea to take away from this discussion is that there
is a special Fourier transform algorithm when the number of data points is a power of 2. There
are still implementations that demand 2Q data points, in software or in hardware devices that
perform Fourier transforms. The FFT algorithm is the reason.