Professional Documents
Culture Documents
Notes Ch4
Notes Ch4
Notes Ch4
2
These notes are used by myself in the course. They are provided to students. These
notes are self-contained, and covers almost everything we go through in class. However,
if a student shall be interested in more detailed discussion, he/she should consult the
recommended textbook.
Text: Guide to Scientific Computing, 2nd edition, by Peter Turner. CRC Press, ISBN
0-8493-1242-6.
Chapter 4
Numerical integration
4.1
Introduction
Problem: Given a function f (x), defined on an interval [a, b], we want to find an
approximation to the integral
I(f ) =
f (x) dx .
Main idea:
Cut up [a, b] into smaller sub-intervals;
In each sub-interval, find a polynomial pi (x) f (x);
Integrate pi (x) on each sub-interval, and sum them up.
4.2
Trapezoid rule
The grid:
xi < xi+1 ,
xn = b
On interval [xi , xi+1 ], approximate f (x) by a linear polynomial that interpolates the 2
end points, i.e,
pi (xi ) = f (xi ),
pi (xi+1 ) = f (xi+1 ).
See Figure 4.1. We see that on each sub-interval, the integral of pi equals to the area of
a trapezium:
Z xi+1
1
pi (x) dx = (f (xi ) + f (xi+1 ))(xi+1 xi ).
2
xi
3
pi (x)
f (x)
x0 = a
xi
xi+1
xn = b
f (x) dx
xi+1
pi (x) dx =
xi
1
(f (xi+1 ) + f (xi )) (xi+1 xi ) ,
2
f (x) dx =
n1
X Z xi+1
i=0 xi
n1
X
i=0
f (x) dx
n1
X Z xi+1
i=0
pi (x) dx
xi
1
(f (xi+1 ) + f (xi )) (xi+1 xi )
2
ba
,
n
xi+1 xi = h.
f (x) dx =
a
n1
X
i=0
h
(f (xi ) + f (xi+1 ))
2
h
[(f (x0 ) + f (x1 )) + (f (x1 ) + f (x2 )) + + (f (xn1 ) + f (xn ))]
2"
#
n1
X
h
f (xi ) + f (xn )
=
f (x0 ) + 2
2
i=1
"
#
n1
X
1
1
= h f (x0 ) +
f (xi ) + f (xn )
2
2
i=1
{z
}
|
T (f ; h)
=
b
a
f (x) T (f ; h)
f (x) dx
xi
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
fi
1.4142136
1.2806248
1.1661904
1.077033
1.0198039
1.0
1.0198039
1.077033
1.1661904
1.2806248
1.4142136
Sample codes. Here are some possible ways to program trapezoid rule in Matlab.
Let a, b, n be given. The function f (x) is also defined, such that it takes a vector x and
returns a vector with same size as x. For example, f (x) = x2 + sin(x) could be defined
as:
function v=func(x)
v=x.^2 + sin(x);
end
In the following code, the integral value is stored in the variable T.
h=(b-a)/n;
T = (func(a)+func(b))/2;
for i=1:n-1
x = a+i*h;
Or, one may use directly the Matlab vector function sum, and the code could be very
short:
h=(b-a)/n;
x=[a+h:h:b-h]; % inner points
T = ((f(a)+f(b))/2 + sum(f(x)))*h;
Error estimates. We define the error:
ET (f ; h)=I(f
) T (f ; h) =
where
ET,i (f ; h) =
xi+1
xi
n1
X Z xi+1
i=0
xi
n1
X
ET,i (f ; h),
f (x) pi (x) dx =
f (x) pi (x) dx,
i=0
(i = 0, 1, , n 1)
#
"n1
1 3
1 ba
1 3 X
h f (i ) = h
ET,i (f ; h) =
f (i )
12
12
n | {z
h }
i=0
i=0
}
| i=0 {z
= f ()
=n
n1
X
n1
X
which gives
ET (f ; h) =
b a 2
h f (),
12
Error bound
|ET (f ; h)|
(a, b).
ba 2
h max f (x) .
12
x(a,b)
What is the minimum number of points to be used in the trapezoid rule to ensure en
error 0.5 104 ?
Answer. We have
f (x) = ex ,
so
f (x) = ex ,
a = 0,
b=2
max f (x) = e2 .
x(a,b)
1
|ET (f ; h)| h2 e2 0.5 104
6
2
h 0.5 104 6 e2 4.06 105
p
2
4.3
Simpsons rule
x2n = b,
h=
ba
,
2n
xi+1 xi = h
Consider the interval [x2i , x2i+2 ]. We will find a 2nd order polynomial that interpolates
f (x) at the points
x2i , x2i+1 , x2i+2
See Figure 4.2. Note that in each sub-interval there is a point in the interior, namely,
x2i+1 .
We now use the Lagrange form, and get
(x x2i+1 )(x x2i+2 )
(x x2i )(x x2i+2 )
+ f (x2i+1 )
(x2i x2i+1 )(x2i x2i+2 )
(x2i+1 x2i )(x2i+1 x2i+2 )
(x x2i )(x x2i+1 )
+f (x2i+2 )
(x2i+2 x2i )(x2i+2 x2i+1 )
pi (x) = f (x2i )
pi (x)
f (x)
x2i
x2i+1 x2i+2
Figure 4.2: Simpsons rule: quadratic polynomial approximation (thick line) in each
sub-interval.
With uniform nodes, this becomes
pi (x) =
1
1
f (x2i )(x x2i+1 )(x x2i+2 ) 2 f (x2i+1 )(x x2i )(x x2i+2 )
2
2h
h
1
+ 2 f (x2i+2 )(x x2i )(x x2i+1 )
2h
x2i+2
x2i
we have
Z x2i+2
1
f (x2i )
p (x) dx =
(x x2i+1 )(x x2i+2 ) dx
2h2
x2i
Z x2i+2
1
2 f (x2i+1 )
(x x2i )(x x2i+2 ) dx
h
x2i
Z x2i+2
1
(x x2i )(x x2i+1 ) dx
+ 2 f (x2i+2 )
2h
x2i
i
x2i+2
pi (x) dx =
x2i
h
[f (x2i ) + 4f (x2i+1 ) + f (x2i+2 )] .
3
f (x) dx S(f ; h) =
n1
X Z x2i+2
i=0
x2i
n1
pi (x) dx =
hX
[f (x2i ) + 4f (x2i+1 ) + f (x2i+2 )] .
3
i=0
1
=2
1
1
x2i2
x2i1
x2i
x2i+1
x2i+2
i=1
xi
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
fi
1.4142136
1.2806248
1.1661904
1.077033
1.0198039
1.0
1.0198039
1.077033
1.1661904
1.2806248
1.4142136
h
[f0 + 4(f1 + f3 + f5 + f7 + f9 ) + 2(f2 + f4 + f6 + f8 ) + f10 ] = 2.2955778.
3
(This is somewhat smaller than the number we get with trapezoid rule, and it is actually
more accurate. Could you intuitively explain that for this particular example?)
Sample codes: Let a, b, n be given, and let the function func be defined. To find the
integral with Simpsons rule, one could possibly follow the following algorithm:
10
Error estimate. One can prove that the basic error on each sub-interval is: (see the
proof below)
1
ES,i (f ; h) = h5 f (4) (i ),
i (x2i , x2i+2 ) .
(4.1)
90
Then, the total error is
n1
X
1
1 ba
b a 4 (4)
ES (f ; h) = I(f ) S(f ; h) = h5
f (4) (i )
=
h f (),
90
n 2h
180
(a, b)
i=0
ba 4
h max f (4) (x) .
180
x(a,b)
Proof. for (4.1): (optional) For notation simplicity, lets consider an interval [a, a + 2h],
and approximate the integral by
Z
a+2h
a
f (x) dx
i
hh
f (a) + 4f (a + h) + f (a + 2h) .
3
we get
5
f (a) + 4f (a + h) + f (a + 2h) = 6f (a) + 6hf (a) + 4h2 f (a) + 2h3 f (a) + h4 f (4) (a) +
6
therefore
i
4
2
5
hh
f (a)+4f (a+h)+f (a+2h) = 2hf (a)+2h2 f (a)+ h3 f (a)+ h4 f (a)+ h5 f (4) (a)+
3
3
3
18
Now we go back to the original integral. We observe that
Z
a+2h
f (x) dx =
a
2h
f (a + s) ds.
0
11
s2
s3
s4
f (a) + f (a) + f (4) (a) +
2
6
24
s ds + f (a)
= 2hf (a) + f (a)
ds + f (a)
ds
2
6
0
0
0
Z 2h 4
s
(4)
ds +
+f (a)
24
0
4
2
4
= 2hf (a) + 2h2 f (a) + h3 f (a) + h4 f (a) + h5 f (4) (a) +
3
3
15
Comparing this with the Simpsons rule, we get the error
h4
1
5 i 5 (4)
1
h f (a) + = h5 f (4) (a) + = h5 f (4) (),
ES,i =
15 18
90
90
(a, a + 2h)
Example 4. With f (x) = ex in [0, 2], we now use Simpsons rule. In order to achieve
an error 0.5 104 , how many points must we take?
Answer. We have
2 4 2
h e 0.5 104
180
h4 0.54 180/e2 = 1.218 103
|ES (f ; h)|
h 0.18682
ba
n=
= 5.3 6
2h
We need at least 2n + 1 = 13 points.
Recall: With trapezoid rule, we need at least 314 points. The Simpsons rule uses much
fewer points.
4.4
ba
,
2n
hn+1 =
1
h
2
12
So
#
n 1
2X
1
1
f (a + ihn )
f (a) + f (b) +
T (f ; hn ) = hn
2
2
i=1
2n+1
1
X
1
1
f (a + ihn+1 )
T (f ; hn+1 ) = hn+1 f (a) + f (b) +
2
2
"
i=1
n 1
n 1
2X
2X
hn 1
1
f (a + (2j + 1)hn+1 )
f (a + ihn ) +
T (f ; hn+1 ) =
f (a) + f (b) +
2 2
2
i=1
1
T (f ; hn ) + hn+1
2
n 1
2X
j=0
f (a + (2j + 1)hn+1 )
j=0
Advantages:
1. One can keep the computation for a level n. If this turns out to be not accurate
enough, then add one more level to get better approximation. flexibility.
2. This formula allows us to compute a sequence of approximations to a define integral
using the trapezoid rule without re-evaluating the integrand at points where it has
already been evaluated. efficiency.
4.5
Romberg Algorithm
13
Here an depends on the derivatives f (n) . Note that we only have the even power terms
of h in the error! Due to symmetry of the methods, all terms with hk where k is odd,
would be cancelled out in the error formula!
When we half the grid size h, the error formula becomes
h
h
h
h
h
h
E(f ; ) = I(f ) T (f ; ) = a2 ( )2 + a4 ( )4 + a6 ( )6 + + an ( )n
2
2
2
2
2
2
We have
(1)
(2)
I(f ) = T (f ; h) + a2 h2 + a4 h4 + a6 h6 +
h
h
h
h
h
I(f ) = T (f ; ) + a2 ( )2 + a4 ( )4 + a6 ( )6 + + an ( )n
2
2
2
2
2
The goal is to use the 2 approximations T (f ; h) and T (f ; h2 ) to get one thats more
accurate, i.e., we wish to cancel the leading error term, the one with h2 .
Multiplying (2) by 4 and subtract (1), we get
3 I(f ) = 4 T (f ; h/2) T (f ; h) + a4 h4 + a6 h6 +
1
4
T (f ; h/2) T (f ; h) +
a4 h4 + a
6 h6 +
I(f ) =
3
3
|
{z
}
U (h)
U (h) is of 4-th order accuracy! Much better than T (f ; h). We now write:
U (h) = T (f ; h/2) +
T (f ; h/2) T (f ; h)
22 1
I(f ) = U (h) + a
4 h4 + a
6 h6 +
I(f ) = U (h/2) + a
4 (h/2)4 + a
6 (h/2)6 +
U (h/2) U (h)
24 U (h/2) U (h)
= U (h/2) +
.
24 1
24 1
14
Then
I(f ) = V (h) + a
6 h6 +
So V (h) is even better than U (h).
One can keep doing this several layers, until desired accuracy is reached.
This gives the Romberg Algorithm: Set H = b a, define:
R(0, 0) = T (f ; H) =
H
(f (a) + f (b))
2
R(1, 0) = T (f ; H/2)
R(2, 0) = T (f ; H/(22 ))
..
.
R(n, 0) = T (f ; H/(2n ))
Here R(n, 0)s are computed by the recursive trapezoid formula.
Romberg triangle: See Figure 4.5.
R(0, 0)
R(1, 0)
s
-
R(1, 1)
R(2, 0)
s
-
R(2, 1)
s
-
R(2, 2)
R(3, 0)
s
-
R(3, 1)
s
-
R(3, 2)
R(n, 0)
R(n, 1)
s
-
R(n, 3)
R(n, n)
R(n, m 1) R(n 1, m 1)
22m 1
Accuracy:
I(f ) = R(n, m) + O(h2(m+1) ),
h=
H
.
2m
15
for i = 1 to n 1 do
end
for j = 2 to n do
%2 to n column
for i = j to n do
R(i, j) = R(i, j 1) +
end
1
4j 1
end
4.6
It is intuitive that the error for numerical integration would be larger where the function
changes a lot, i.e., the derivatives f (k) are big. If one uses uniform grid, then the error is
not evenly distributed, and we have little control over the accuracy. Therefore, it would
be desirable to insert more points where error is otherwise large.
We illustrate this idea using Simpsons rile.
For interval [a, b], h =
ba
2 ,
a+b
ba
) + f (b)
f (a) + 4f (
S1 [a, b] =
6
2
Error form:
E1 [a, b] =
1 5 (4)
h f (),
90
(a, b)
Then
I(f ) = S1 [a, b] + E1 [a, b]
Divide [a, b] up in the middle, let c =
a+b
2 .
h
i
1
(h/2)5 f (4) (1 ) + f (4) (2 )
90
16
Assume f (4) does NOT change much, then E1 [a, c] E1 [c, b], and
E2 [a, b] 2E1 [a, c] = 2
1
1
E1 [a, b] = 4 E1 [a, b]
5
2
2
This gives
S2 [a, b] S1 [a, b] = (I E2 [a, b]) (I E1 [a, b]) = E1 E2 = 24 E2 E2 = 15E2
This means, we can compute the error E2 :
E2 =
1
(S2 S1 )
15
24 1
This gives the idea of an adaptive recursive formula:
(A)
I = S1 + E1
(B)
I = S2 + E2 = S2 +
1
E1
24
(24 1)I = 24 S2 S1
I=
S2 S1
24 S2 S1
= S2 +
4
2 1
15
Note that this gives the best approximation when f (4) const.
Pseudocode: f : function, [a, b] interval, : tolerance for error
answer=Simpson(f, a, b, )
compute S1 and S2
If |S2 S1 | < 15
answer= S2 + (S2 S1 )/15;
else
c = (a + b)/2;
Lans=Simpson(f, a, c, /2);
Rans=Simpson(f, c, b, /2);
answer=Lans+Rans;
end
17
In Matlab, one can use quad to compute numerical integration. Try help quad, it will
give you info on it. One can call the program by using:
a=quad(fun,a,b,tol)
It uses adaptive Simpsons formula.
See also quad8, higher order method.
Remark: One could also design an adaptive trapezoid method. Would you like to try it
and work out the algorithm?
Of course, due to the higher accuracy of Simpsons rule, it is the mostly commonly used
one, even in adaptive form.
4.7
We notice that all the numerical integration rule follow the form
Z b
f (x) dx A0 f (x0 ) + A1 f (x1 ) + + An f (xn ) .
(4.2)
Here xi [a, b] are called the nodes, and Ai s are the weights.
For example, the trapezoid rule is:
x0 = a, x1 = b, A0 = A1 = 1
and the Simpsons rule corresponds to
1
2
a+b
, x2 = b, A0 = A2 = (b a) , A1 = (b a) .
2
6
3
In these rules, we fix the points xi , then we adjust the weights Ai .
x0 = a, x1 =
If now we allow the nodes xi also to be adjusted, we gain higher degree of freedom, and
higher accuracy as well. We will explain the details below.
Gaussian quadrature: One chooses the nodes xi and weight Ai (i = 0, 1, 2, , N )
such that (4.2) gives the exact value for polynomial functions f (x) of highest possible
degree m:
Z b
f (x) dx = A0 f (x0 ) + A1 f (x1 ) + + An f (xm ) ,
if f P m .
(4.3)
a
18
This leads to
f (x) = 1 :
A0 + A1 = 2
f (x) = x :
A0 x0 + A1 x1 = 0
2
A0 x20 + A1 x21 =
3
A0 x30 + A1 x31 = 0
f (x) = x2 :
f (x) = x3 :
One could solve this system and find
1
1
x0 = , x1 = ,
3
3
A0 = 1, A1 = 1.
f (x) dx f
+f
.
3
3
1
This will have degree of precision 3.
Now, lets consider N = 2, so we have nodes x0 , x1 , x2 and weights A0 , A1 , A2 , and the
rule should give exact solution for polynomials of degree m = 5. By symmetry, we
immediately have
x0 = x2 , x1 = 0, A0 = A2 ,
which reduces the number of unknowns to 3 and simplifies a lot the computation. Lets
now check it with f (x) = 1, x2 , x4 :
f (x) = 1 :
2A0 + A1 = 2
2
2A0 x20 =
3
2
4
2A0 x0 =
5
f (x) = x2 :
f (x) = x4 :
We get
x0 =
3/5, x1 = 0, x2 =
3/5,
A0 = A2 = 5/9, A1 = 8/9.
(Optional discussion) This is a well-studied problem. It turns out that the nodes xi are
roots of Legendre polynomials qn+1 (x). On the interval [1, 1], they are
q0 (x) = 1,
q1 (x) = x,
3
1
q2 (x) = x2 ,
2
2
q3 (x) =
5 3 3
x x.
2
2
19
q2 : 1/ 3,
q1 : 0,
q3 : 0,
n
Y
li (x) dx,
li (x) =
j=0,j6=i
3/5.
x xj
.
xi xj
See Table 4.1 for a list of Gaussian quadrature nodes and weights for n 5.
Table 4.1: Gaussian Quadrature Nodes and Weights
n
1
2
3
4
Nodes xi
p0
1/3
0.6
0
q
(3 4.8)/7
q
(3 + 4.8)/7
q
p
(5 40/7)/9
0
q
p
(5 + 40/7)/9
Weights Ai
2
1
5/9
8/9
(18 + 30)/36
(18 30)/36
(322 + 13 70)/900
128/225
(322 13 70)/900
2x (a + b)
,
ba
1
1
x = (b a)t + (a + b)
2
2
so for 1 t 1 we have a x b.
Then, for the node and weight (ti , Ai ) on [1, 1], (as in the table), one can change them
into new node and weight (xi , Ai ) for interval [a, b] by
1
1
xi = (b a)ti + (a + b),
2
2
ba
Ai =
Ai .
2
Advantage: (1). Higher order accuracy. (2). Since all nodes are in the interior of the
interval, these formulas can handle integrals of function that tends to infinite value at
one end of the interval (provided that the integral is defined). Examples:
Z
dx,
1
0
(x2 1)1/3
sin(ex 1) dx.