Fundamental approximation theorems

Kunal Narayan Chaudhury∗

Abstract
We establish two closely related theorems on the approximation of continuous functions, using different
approaches. The first of these concerns the approximation of continuous functions defined on an interval,
while the second is for functions defined on a circle.

Approximation by polynomials
We first show that the polynomials are dense in the class of continuous functions defined on an interval. To
quantify the approximation, we use the supremum (or uniform) norm
n o
k f k∞ = sup | f (x)| : x ∈ [a, b ] .

Theorem 1 (Weierstrass approximation theorem). Let f (x) be a continuous function on [a, b ]. Then for any
arbitrary " > 0, there exists a polynomial P (x) on [a, b ] such that k f − P k∞ < ".
Proof. Without loss of generality, we set the interval [a, b ] as [0, 1]. Fix " > 0. Since a continuous function on a
compact interval is uniformly continuous, there is a δ > 0 (depending on f and ") such that | f (x)− f (y)| < "/2
for all |x − y| < δ. Since f (x) is also bounded, there is a M such that | f (x)| < M for all x in [0, 1].
Corresponding to a sufficiently large n (this will be determined shortly), we set P (x) as
n
‚ Œ 
X k n k
P (x) = f x (1 − x)n−k .
k=0
n k

Clearly, P (x) is a polynomial of degree n. The factor

n k
 
Bk (x) = x (1 − x)n−k
k
can be interpreted as the probability of having k success in n independent trails of an experiment, where x is
the probability of success in each trail (this is where we require the normalization on x). Denoting this random
variable by X , we see that
X n
Bk (x) = P(X ≥ 0) = 1
k=0

for any x ∈ [0, 1]. Moreover, by the Chebyschev inequality,

X n
Bk (x) = P(|X − EX | ≥ λ) ≤ 2 ,
|k−n x|≥λ λ

where we use the fact that the mean and the variance of X are n x and n x(1 − x), respectively. Now, since
n
X
f (x) − P (x) = [ f (x) − f (k/n)]Bk (x),
k=0
∗ kunal.chaudhury@epfl.ch.

1
we can write
X X
| f (x) − P (x)| = | f (x) − f (k/n)|Bk (x) + | f (x) − f (k/n)|Bk (x)
|x−k/n|<δ |k−n x|≥nδ

"X
n X
< Bk (x) + 2M Bk (x)
2 k=1 |k−n x|≥nδ
" 2M
< + .
2 nδ 2
Setting n > 4M /"δ 2 , we arrive at the desired result. ƒ
We can generalize the above result as follows: The system of polynomials Bk (x), k = 0, 1, . . . , n, can be
replaced by a more general system Pk (x), k = 0, 1, . . . , n, defined on [0, 1]. The key criteria for such a system
are

• Partition-of-unity: k Pk (x) = 1 for all x ∈ [0, 1], and

P

• Localization: For every " > 0 and every δ > 0, one can fix a sufficiently large n such that
X
Pk (x) < " (x ∈ [0, 1]).
|k/n−x|≥δ

Remark. Theorem 1 in fact provides us with a countable collection of functions which are dense in the the
C [a, b ], the Banach space of continuous functions on [a, b ] with the supremum norm. In technical jargon,
this means that the space C [a, b ] is separable. To do so, we order the polynomials on [a, b ] by their order,
and within each order consider the (countable) collection of polynomials having rational coefficients, that is,
we consider the set
( )
X N
j
E= c j x : N is a non-negative integer, and c0 , . . . , cN ∈ Q .
j =0

This is clearly countable, and since the reals can be arbitrarily approximated using the rationals (this logic is
kind of circular though), theorem 1 guarantees that E is dense in C [a, b ].

Approximation by trigonometric polynomials

We now show that the trigonometric polynomials, namely, functions of the form
N
X
P (x) = cn en (x)
n=−N

where en (x) = exp( j 2πn x), are dense in the space of periodic continuous functions (P (x) can be seen as a
“bandlimited” function, one having a finite number of frequencies).
Theorem 2 (Approximation of continuous periodic functions). Let f (x) be a complex-valued continuous
function on R that is 1-periodic, and let " > 0. Then there exists a trigonometric polynomial P (x) such that
k f − P k∞ < ".

Note that this result immediately follows from the previous theorem, since one can pass from the interval
[0, 1] to the circle (the periodic domain) using the transformation x 7→ exp( j 2πx). We however provide an
alternative derivation.

2
Proof. To prove this result, we invoke the following ideas in order.
• The desired trigonometric polynomial can be obtained by mollification, namely through the convolution
Z1
( f ∗ P )(x) = f (y)P (x − y)d y
0

between f (x) and an appropriate polynomial P (x). Note that the function ( f ∗ P )(x) is again a trigonometric
polynomial. Indeed, P (x) is a sum of en (x), and for every en (x) we have
Z1
( f ∗ en )(x) = f (y)en (x − y)d y = fˆ(n)en (x).
0

• The particular P (x) that we need is the so-called approximation of the identity. It must have the the
following properties: (i) P (x) must be non-negative, (ii) P (x)d x = 1, and (iii) P (x) < " for δ < |x| < 1 − δ,
R
where 0 < δ < 1/2.
It is not obvious whether such a function exists at all. A particular instance of this function is the so-called
Fejér kernel
1  NX
−1 N −1
‚ Œ
2 X |n|
FN (x) =  ek (x) = 1− ek (x).

N k=0 k=−(N −1)
N

One can verify that by choosing N large enough, the Dirac-like function FN (x) can be made to fulfill the
above-stated properties
• The rest of the proof is as follows. Since f (x) is uniformly continuous, there is a δ > 0 such that
| f (x) − f (y)| < "/2 for all |x − y| < δ. Since f (x) is also bounded, there is a M such that | f (x)| < M for all x
in [0, 1].
We can write f (x) − ( f ∗ P )(x) as
Z1 Z1
f (x) − f (x − y)P (y)d y = ( f (x) − f (x − y))P (y)d y.
0 0

Splitting up the interval [0, 1] into appropriate parts, and using the properties of f (x) and P (x), we have
 Z1  Z1
 f (x) − f (x − y)P (y)d y  ≤ | f (x) − f (x − y)|P (y)d y
 
0 0
Z δ Z 1−δ
= | f (x) − f (x − y)|P (y)d y + | f (x) − f (x − y)|P (y)d y
0 δ
Z1
+ | f (x) − f (x − y)|P (y)d y
1−δ
Zδ Z 1−δ Z 1
< "P (y)d y + 2M "d y + "P (y)d y
0 δ 1−δ
= 2(M + 1)".
Since " was arbitrary, this proves the theorem ƒ

The above theorem can be leveraged to establish the following result concerning the “least-square” approxi-
mation of periodic functions using trigonometric polynomials.
Theorem 3 (Mean-square convergence of Fourier series). Let f (x) be a complex-valued continuous function
that is 1-periodic. Then the trigonometric polynomial SN (x) = Nn=−N fˆ(n)en (x), where
P

Z 1
fˆ(n) = f (x)en (x)d x,
0

3
converges to f (x) in the L2 -norm, as N −→ ∞. In other words,

lim k f − SN k2 = 0.
N →∞

Proof. Fix " > 0, and choose a trigonometric polynomial PN (x) such that k f − PN k∞ < ". In particular, we
have k f − PN k2 ≤ k f − PN k∞ < ".
Let SN (x) = |n|≤N fˆ(n)en (x). We claim that ( f − SN ) is orthogonal to (SN − PN ), so that, using the
P

Pythagoras theorem, we can write

k f − PN k22 = k f − SN k22 + kSN − PN k22 .

This would give us the desired result, since k f − SN k2 ≤ k f − PN k2 < ".

To establish the orthogonality, we note that (SN − PN ) is a trigonometric polynomial, and that for each of
its constituents en (x), |n| ≤ N , we have
N
fˆ(k) 〈ek , en 〉 = fˆ(n) − fˆ(n) = 0,
X
〈 f − SN , e n 〉 = 〈 f , e n 〉 −
k=−N

since 〈en , e m 〉 = 0 for m 6= n, and equals 1 for m = n. ƒ

As a corollary of the above theorem, we obtain the following.

Theorem 4 (Parseval). Let f (x) be a complex-valued continuous function that is 1-periodic. Then the series
| fˆ(n)|2 converges absolutely, and
P∞
n=−∞

∞ Z 1
| fˆ(n)|2 =
X
| f (x)|2 d x.
n=−∞ 0

Proof. By the previous theorem we know that if " > 0, then by choosing N large we have k f − SN k2 < ". By
the triangle inequality,
(k f k2 − ")2 ≤ kSN k22 ≤ (k f k2 + ")2 .
It is easily seen that for the trigonometric polynomial SN ,
N
| fˆ(n)|2 .
X
kSN k22 =
n=−N

In other words,
N
| fˆ(n)|2 ≤ (k f k2 + ")2 .
X
(k f k2 − ")2 ≤
n=−N

Since " is arbitrary, we arrive at the desired result by letting N −→ ∞. ƒ