Professional Documents
Culture Documents
Orthogonal Polys PDF
Orthogonal Polys PDF
Yan-Bin Jia
1 Introduction
We have seen the importance of orthogonal projection and orthogonal decomposition, particularly
in the solution of systems of linear equations and in the least-squares data fitting. In fact, these
ideas can generalize from vectors to functions. For instance, we might let V be the “vector space”
of continuous integrable functions defined on some interval [a, b]. Such a space V has infinite
dimensions.
Many inner products can be defined on V . Which one to use is generally determined by your
application. Here we define the inner product of two functions g and h in one of two ways:
Z b
hg, hi = g(x)h(x)w(x) dx, or (1)
a
n
X
hg, hi = g(xi )h(xi )w(xi ), (2)
i=1
where w(x) is a positive function, called a weighting function. The form (2) is used often when we
do not have information about g and h at all points. Instead, we may have information only at n
discrete points x1 , . . . , xn .
With the inner product defined, we say that the two functions g(x) and h(x) are orthogonal if
hg, hi = 0.
Example 1. It is easy to verify, for example, that the functions g(x) = 1, h(x) = x are orthogonal if the
inner product is Z 1
hg, hi = g(x)h(x) dx,
−1
or if it is
10
X
hg, hi = g(i)h(i),
i=−10
or if it is
1
g(x)h(x)
Z
hg, hi = √ dx.
−1 1 − x2
1
The functions g(x) = sin nx, h(x) = sin mx, integers n and m, are orthogonal if
Z 2π
hg, hi = g(x)h(x) dx,
0
and n 6= m, as are the functions g(x) = sin nx and h(x) = cos mx.
Why are we concerned with orthogonal functions? Because there are two parts to good data
fitting: a) selecting a good basis of functions B and b) approximating a given function or a set of
observed data using the basis functions. We have seen how to carry out part b) but we have yet
to discuss part a). The point here is that if we find an orthogonal basis B, we would be able to
approximate or decompose a function f by the rule
X hf, gi
f∼
= g.
hg, gi
g∈B
The above is an equality if f ∈ span(B), that is, f is a linear combination of some functions in B.
Otherwise, it is an orthogonal projection of f onto span(B).
2 Orthogonal Polynomials
A sequence of orthogonal polynomials consists of p0 (x), p1 (x), p2 (x), . . . (finite or infinite) such that
a) pi (x) is a polynomial of degree i;
b) hpi , pj i = 0 whenever i 6= j.
T0 (x) = 1,
T1 (x) = x,
Tk+1 (x) = 2xTk (x) − Tk−1 (x), k = 1, 2, . . . ,
2
a)
1
g(x)h(x)
Z
hg, hi = √ dx.
−1 1 − x2
In this case,
Z 1
Ti (x)Tj (x) 0, i 6= j
π
hTi , Tj i = √ dx = , i = j 6= 0
1 − x2 2
−1 π, i = j = 0
b)
m−1
X
hg, hi = g(ξk,m )h(ξk,m ),
k=0
The following facts can be proved about a finite sequence of orthogonal polynomials p0 (x), p1 (x),
. . . , pk (x):
Hence we have
hp, pi i
di = .
hpi , pi i
hp, pk i = 0.
By Property i), p(x) = d0 p0 (x)+ d1 p1 (x)+ · · · + dl pl (x), where l ≤ k is the degree of p. Taking
the inner product of pk with both sides of this equation, we obtain that hp, pk i = 0.
iii) If the inner product is given by (1), then pk (x) has k simple real zeros in the interval (a, b).
3
where p−1 (x) = 0 and
In the case where the polynomials are monic (with leading coefficient 1), the following recur-
rence holds:
p−1 (x) = 0,
p0 (x) = 1,
hxp0 , p0 i
p1 (x) = x− p0 (x),
hp0 , p0 i
hxpi , pi i hpi , pi i
pi+1 (x) = x− pi (x) − pi−1 (x), i = 1, 2, . . .
hpi , pi i hpi−1 , pi−1 i
Hence
p1 (x) = (x − 0)p0 (x) = x,
and
1
2
Z
hp1 , p1 i = x2 dx = ,
−1 3
Z 1
hxp1 , p1 i = x3 dx = 0.
−1
So now we have
2
1
p2 (x) = (x − 0)p1 (x) − 3
p0 (x) = x2 − .
2 3
4
Continuing this process we would get
p0 (x) = 1,
p1 (x) = x,
1
p2 (x) = x2 − ,
3
3 3
p3 (x) = x − x,
5
6 3
p4 (x) = x − x2 + ,
4
7 35
..
.
where the inner product is defined by either (1) or (2). And we would like to seek a polynomial
of degree at most k to minimize the above inner product. Such a polynomial is a least-squares
approximation to f (x) by polynomials of degrees not exceeding k.
We proceed by finding an orthogonal sequence of polynomials p0 (x), . . . , pk (x) for the chosen
inner product such that hpi , pj i = 0 whenever i 6= j. Then every polynomial of degree at most k
can be written uniquely as
p(x) = d0 p0 (x) + · · · + dk pk (x)
where
hp, pi i
di =
hpi , pi i
So now we try to minimize
hf (x) − p(x), f (x) − p(x)i = hf (x) − d0 p0 (x) − · · · − dk pk (x), f (x) − d0 p0 (x) − · · · − dk pk (x)i
5
over all possible choices of d0 , . . . , dk , or equivalently, over all polynomials of degree at most k. The
partial derivatives of the above inner product with respect to d0 , . . . , dk must all vanish; in other
words, the “best” coefficients must satisfy the normal equations
di hpi , pi i = hf, pi i, i = 0, . . . , k.
hf, pi i
di = , i = 0, . . . , k. (4)
hpi , pi i
Here is the analogy to the case of the least-squares technique over a vector space. In the
space of all functions, the orthogonal polynomials p0 , . . . pk constitute an “orthogonal basis” for the
subspace of polynomial functions of degree no more than k. The least-squares approximation of a
function f by polynomials in this subspace is then its orthogonal projection onto the subspace. The
coordinates of this projection along the axes p0 , . . . , pk are thus hf, p0 i/hp0 , p0 i, . . ., hf, pk i/hpk , pk i.
Below we illustrate the use of orthogonal polynomials for obtaining least-squares approximations
with respect to both continuous and discrete versions of inner products.
Example 5. Calculate the polynomial at degree at most 3 that best approximates ex over the interval
[−1, 1] in the least-squares sense.
Here we obtain a best approximation by orthogonally projecting ex onto the subspace of functions
spanned by Legendre polynomials p0 , . . . , p3 . In other words,
3
X
p(x) = di pi (x),
i=0
where
hex , pi i
di = .
hpi , pi i
We compute the following inner products:
Z 1
hp0 , p0 i = 1 dx = 2,
−1
Z 1
2
hp1 , p1 i = x2 dx = ,
−1 3
1
2 1 8
Z
hp2 , p2 i = x4 − x2 + dx = ,
−1 3 9 45
Z 1
6 6 4 9 2 8
hp3 , p3 i = x − x + x dx = ,
−1 5 25 175
Z 1
1
hex , p0 i = ex dx = e − ,
−1 e
Z 1
2
hex , p1 i = ex x dx = ,
−1 e
6
1
1 2 14
Z
hex , p2 i = ex x2 − dx = e− ,
−1 3 3 3e
Z 1
3 74
hex , p3 i = ex x3 − x dx = −2e + .
−1 5 5e
Then
1 1
d0 = e− ≈ 1.1752012,
2 e
3 2
d1 = · ≈ 1.1036383,
2 e
45 2 14
d2 = e− ≈ 0.53672153,
8 3 3e
175 74
d3 = −2e + ≈ 0.17613908.
8 5e
So the least-squares approximation to ex on (−1, 1) is
So the inner product (2) is used with w(x) ≡ 1. We start by calculating the following
p0 (x) = 1,
6
X
hp0 , p0 i = 1 · 1 = 6,
i=1
6
X i−1
hxp0 , p0 i = 10 + · 1 = 63.
i=1
5
Therefore
hxp0 , p0 i
p1 (x) = x− · 1 − 0 = x − 10.5,
hp0 , p0 i
6 2
X i−1
hp1 , p1 i = − 0.5 = 0.7,
i=1
5
6 2
X i−1 i−1
hxp1 , p1 i = 10 + − 0.5 = 7.35.
i=1
5 5
7
We can go on to calculate p2 (x), obtaining
hxp1 , p1 i hp1 , p1 i
p2 (x) = x− p1 (x) − p0 (x)
hp1 , p1 i hp0 , p0 i
7.35 0.7
= x− (x − 10.5) −
0.7 6
= (x − 10.5)2 − 0.1166667,
hp2 , p2 i = 0.05973332.
References
[1] S. D. Conte and Carl de Boor. Elementary Numerical Analysis. McGraw-Hill, Inc., 2nd edition,
1972.
[2] M. Erdmann. Lecture notes for 16-811 Mathematical Fundamentals for Robotics. The Robotics
Institute, Carnegie Mellon University, 1998.