Part 6:

Numerical Integration and

Differentiation
– Newton-Cotes Integration Formulas
– Trapezoidal Rule
– Simpson’s Rules
– Integration with Unequal Segments
– Integration of Equations
– Romberg Integration
– Gauss Quadrature
– Numerical Differentiation
 Newton-Cotes Integration Formula
 Replacing the complicated integrand function or tabulated data by
an approximating function such as a polynomial.
f(x) f(x)

x x
Replacing by a Replacing by a
straight line parabola
 These formulas are applicable to both closed forms (i.e., data points
at the beginning and end of the integration limits are known) and
open forms (otherwise).
 These formulas are applied for equally spaced intervals.
 Trapezoidal Rule
 Complicated integrand function is f(x)
replaced by a first order polynomial
(straight line).
b b

I f ( x ) dx f 1 ( x ) dx
a a

where the first-order polynomial

a b x
f (b ) f (a )
f1 ( x ) f (a ) (x a)
b a
b
f (b ) f (a )
I f (a ) (x a ) dx
a
(b a)

f (a ) f (b )
I (b a) Trapezoidal rule
2
 f(x) f(a)+f(b)
=average height
2

f(a)
I ( width ) ( average height )
f(b)

a b x
width
 All the Newton-Cotes formulas can be formulated in general
form above (only definition of average height changes).
Error for trapezoidal rule: b
''
f ( x ) dx
1 '' 3 1 3
''
f
''
a
Et f ( )( b a) Ea (b a) f (b a)
12 12

Average of the second

Local truncation error
derivative between the interval
EX: Numerically integrate
2 3 4 5
f ( x) 0 .2 25 x 200 x 675 x 900 x 400 x
from a=0 to b=0.8. Find the error. (Note that the exact solution is 1.6405).
f (0) 0 .2 0 .2 0 . 232
I 0 .8 0 . 1728
f ( 0 .8 ) 0 . 232 2

Et 1 . 6405 0 . 1728 1 . 4677 t

89 . 5 %

Normally, we don’t have the knowledge of the true error; we can calculate the
approximate error. We need to calculate the second derivative of the function:
'' 2 3
f ( x) 400 4050 x 10800 x 8000 x
0 .8
2 3
400 4050 x 10800 x 8000 x
''
0
f 60
0 .8 0
1 3 Et and Ea have the
Ea ( 60 )( 0 . 8 ) 2 . 56
12 same order and sign
Multiple-application trapezoidal rule:
We can increase the accuracy of the integration by dividing the
interval into many segments and apply the trapezoidal rule to
each segment.
f(x)
For n+1 equally spaced base
points, there are n segments of
step size:
(b a)
h
n
x0 x1 x2 x
n 1
h
h
I f ( x0 ) 2 f ( xi ) f ( xn )
2 i 1

Initial point final point

 or n 1

f ( x0 ) 2 f ( xi ) f ( xn )
i 1
I (b a)
2n

width average height

Error for the multiple-application trapezoidal rule is found by

summing individual errors.
3 n
b a ''
Et 3
f ( i)
12 n i 1
or
n Convergence:
3 ''
b a '' f ( ) Error is inversely
Ea f where ''
i

12 n
2 f i 1
related to the
n square of n !
EX: Use two-segment trapezoidal rule to integrate
2 3 4 5
f ( x) 0 .2 25 x 200 x 675 x 900 x 400 x
From a=0 to b=0.8.

n=2 h=0.4 f (0) 0 .2 f ( 0 .4 ) 2 . 456 f ( 0 .8 ) 0 . 232

0 .2 2 ( 2 . 456 ) 0 . 232
I 0 .8 1 . 0688
4

Et 1 . 6405 1 . 0688 0 . 57173 34 . 9 % n I t

t
2 1.0688 34.9
1 3
Ea 2
( 60 )( 0 . 8 ) 0 . 64 3 1.3695 16.5
12 ( 2 )
4 1.4848 9.5
5 1.5399 6.1
6 1.5703 4.3
Error is inversely
7 1.5887 3.2
related to the square
of n. 8 1.6008 2.4
 Simpson’s Rules
f(x)
Instead of using a line segment, use
higher-order polynomials to connect
points increase the accuracy.
Simpson’s 1/3 rule
A parabola is substituted for the function. x0 x1 x2 x
Lagrange polynomial forms are used for the replacement.
Replacing Lagrange polynomials for the function at three
points x0 , x1, and x2 :
(x x1 )( x x2 ) (x x 0 )( x x2 )
x2
f ( x0 ) f ( x1 )
( x0 x1 )( x 0 x2 ) ( x1 x 0 )( x1 x2 )
I dx
x0
(x x 0 )( x x1 )
f ( x2 )
( x2 x 0 )( x 2 x1 )
After integration and algebraic manipulations
1h Simpson’s 1/3
I f ( x0 ) 4 f ( x1 ) f x2 rule.
3

In another form:

f ( x0 ) 4 f ( x1 ) f x2 h=(b-a)/2
I (b a)
6

width average height

Error for the single segment Simpon’s 1/3 rule:

> It is more accurate than expected as the
(b a)
5 error is related to the forth order derivative
(4)
Et f (third order term is meaningful but is zero).
2880 > It yields exact results for cubic polynomials
even though it is obtained from a parabola.
EX: Use single application of Simpson’s 1/3 rule to integrate
2 3 4 5
f ( x) 0 .2 25 x 200 x 675 x 900 x 400 x
from a=0 to b=0.8.

n=2 h=0.4 f (0) 0 .2 f ( 0 .4 ) 2 . 456 f ( 0 .8 ) 0 . 232

0 .2 4 ( 2 . 456 ) 0 . 232
I 0 .8 1 . 3675
6

Et 1 . 6405 1 . 3675 0 . 273 t

16 . 6 %
0 .8
(4)
(4) f ( x ) dx
f ( x) 21600 48000 x (4)
0
f 2400
0 .8 0

5 5
(b a) (4) ( 0 .8 ) Et and Ea are equal
Ea f ( 2400 ) 0 . 273
2880 2880 because the integrand is
a fifth order polynomial
Multiple-application Simpson’s 1/3 rule
For n segments:
(b a)
h
n
Total integral can be represented
x2 x4 xn

I f ( x ) dx f ( x ) dx ... f ( x ) dx
x0 x2 xn 2

Substituting Simpson’s 1/3 formula

n 1 n 2

f ( x0 ) 4 f ( xi ) 2 f ( xi ) f ( xn )
i 1, 3 , 5 i 2,4,6
I (b a)
3n

width average height

Error for the multiple (n) segment
Simpon’s 1/3 rule is calculated by
adding error for individual segments.
Hence, we get:
5
(b a) (4)
Et 4
f
180 n

Multiple application Simpson’s rule returns very accurate results

compare to trapezoidal rule. It is the method of choice for most
applications.
 As for all Newton-Cotes formulas, the intervals must be equally
spaced.
Because of the need for three points for applications, the method
is limited to odd number of points (even number of segments).
EX: Use multiple application Simpson’s 1/3 rule (n=4) to integrate
2 3 4 5
f ( x) 0 .2 25 x 200 x 675 x 900 x 400 x
from a=0 to b=0.8.

n=4 h=0.2 f (0) 0 .2 f ( 0 .2 ) 1 . 288 f ( 0 .4 ) 2 . 456

f ( 0 .6 ) 3 . 464 f ( 0 .8 ) 0 . 232

0 .2 4 (1 . 288 3 . 464 ) 2 ( 2 . 456 ) 0 . 232

I 0 .8 1 . 6235
12

Et 1 . 640533 1 . 623467 0 . 017067 t

1 . 04 %

5 5
(b a) (4) ( 0 .8 )
Ea 4
f 4
( 2400 ) 0 . 017067
180 ( n ) 180 ( 4 )

Small error shows very

accurate results are
obtained.
Simpson’s 3/8 rule
Simpson’s 3/8 rule is used when the odd-number of segments
are encountered.
A third-order polynomial is used for replacing the function.
b b

I f ( x ) dx f 3 ( x ) dx
a a

Applying a third order Legendre polynomial to four points gives

3h Simpson’s 3/8
I f ( x0 ) 3 f ( x1 ) 3 f ( x2 ) f x3 formula.
8

f ( x0 ) 3 f ( x1 ) 3 f ( x2 ) f x3
I (b a) h=(b-a)/3
8

width average height

Truncation error for Simpson’s 3/8 rule:

(b a)
5 Simpson’s 3/8 rule is
(4)
Et f somewhat more
6480 accurate than 1/3 rule

In general Simpson’s 1/3 rule is the method of choice because of

obtaining third order accuracy and using three points instead of
four.
3/8 rule has the utility when the number of segments is odd.
In practice, use 1/3 rule for all the even number of segments and
use 3/8 rule for the remaining last three segments.
Higher-order Newton-Cotes formulas:
Higher order (n=4,5) formulas have the same order error as n=1
(Trapezoid) or n=2,3 (Simpson’s) formulas.
Higher-order formulas are rarely used in engineering practices.
To increase the accuracy, just increase the number of segments!
 Integration with Unequal Segments
There may be cases where the spacing between data points may
not be even (e.g., experimentally derived data points)
One way is to use trapezoidal rule for each segment

f ( x0 ) f ( x1 ) f ( x1 ) f ( x2 ) f ( xn 1 ) f ( xn )
I h1 h2 .. hn
2 2 2
width of the segments (h) are not constant so the
formula cannot be written in a compact form.

A computer algorithm can easily be developed to do the

integration for unequal-sized segments.
The algorithm can be constructed such that, use Simpson’s 1/3
rule wherever two consecutive equal-sized segments are
encountered, and use 3/8 rule wherever three consecutive
equal-sized segments are encountered. When adjacent
segments are unequal-sized , just use trapezoidal rule.
 Open Integration Formulas
 There may be cases where
integration limits are beyond the f(x)
range of the data.
General characteristics and order of
error for open forms of the Newton-
Cotes formulas are similar to the
closed forms.
Even segment formulas are usually a x
b
the method of choice as they
require fewer points.
Open forms are not used for definite integration.
They have utility for analyzing improper integrals (discussed later).
They have connection to multi-step method for solving ordinary
differential equations.
 Integration of Equations
Previously discussed methods are appropriate for tabulated data.
If the function to be integrated is available, other modified
methods are available.
> Romberg integration (Richardson extrapolation)
> Gauss quadrature
Newton-Cotes method for functions:
Use of multiple-application of

Percent relative error

trapezoidal and Simpson’s rules are
Trapezoidal
not convenient for analyzing rule
integration of functions.
> Large number of operations
Simpson’s
required to evaluate the functions. rule
> For large values of n, round of
errors starts to dominate.
n
 Romberg Integration
It uses the trapezoidal rule, but much more efficient results are
obtained through iterative refinement techniques .
Richardson’s extrapolation:
A sequence acceleration method to improve the rate of
convergence.
It offers a very practical tool for numerical integration and
differentiation.
Application of iterative refinement techniques to improve the
error at each iteration. For each iteration
I I (h) E (h)

exact approximate truncation

integral integral error

In Richardson extrapolation, two approximate integrals are used

to compute a third more accurate integral.
Assume that two integrals with step sizes of h1 and h2 are
available. Then,

I ( h1 ) E ( h1 ) I ( h2 ) E ( h2 )

(b a)
Error for multiple-equation trapezoidal rule ( n )
h
b a 2
''
E h f
12
Assuming same f ’’ for two step sizes
2
2
E ( h1 ) h 1 h1
2 or E ( h1 ) E ( h2 )
E ( h2 ) h 2 h2
Inserting the error to the previous equation, and solving for E(h2)

I ( h1 ) I ( h2 )
E ( h2 ) 2
1 h1 / h 2

Then,
1
I I ( h2 ) 2
I ( h2 ) I ( h1 )
( h1 / h 2 ) 1

It can be shown that the error for above approximation is O(h4)
although the error from use of trapezoidal rule is O(h2).
For the special case of interval being halved (h2=h1/2), we get

4 1
I I ( h2 ) I ( h1 )
3 3

O(h4) O(h2)
EX: For the numerical integration of
2 3 4 5
f ( x) 0 .2 25 x 200 x 675 x 900 x 400 x

we previously obtained the following, using trapezoidal rule.

n h I t

1 0.8 0.1728 89.5

2 0.4 1.0688 34.9
4 0.2 1.4848 9.5

Use Romberg integration approach to obtain improved approximations.

use n=1 and n=2 for a improved approximation

4 1
I (1 . 0688 ) ( 0 . 1728 ) 1 . 367467 t
16 . 6 %
3 3
If use n=2 and n=4
4 1
I (1 . 4848 ) (1 . 0688 ) 1 . 623467 t
1 .0 %
3 3
The method can be generalized, e.g. combine two O(h4)
approximation to obtain an O(h2) approximation, and so on…
Using two improved O(h4) approximations in the previous
examples, we can use the following formula
16 1 Im: more accurate approx.
I Im Il Il: less accurate approx.
15 15

to obtain an O(h6) approximation.

Similarly, two O(h6) approximation can be combined to obtain an
O(h8) approximation
64 1 Im: more accurate approx.
I Im Il
63 63 Il: less accurate approx.

This sequential improvement convergence technique is named

under the general approach of Richardson's extrapolation.
EX: In the previous example use the two O(h4) approximations to obtain an
O(h6) approximation for the integral.

16 1
I (1 . 623467 ) (1 . 367467 ) 1 . 640533
15 15

True result is 1.640533 (correct answer to 7 S.F.!).

Algorithm:
O(h2) O(h4) O(h6) O(h8)

…
… …
…
… …
… …
…
… …
… … …
… … … …
…
Trapezoidal rule
 Gauss Quadrature
In Newton-Cotes formulation, predetermined and evenly spaced
data are used. Points of integration are covered by the data.
In Gauss quadrature, points of integration are undetermined.

f(x) f(x)

a b x a b x
For the example above, integral approximation can be improved
by using two wisely chosen interior points.
Gauss quadrature offer a method to find these points.
Method of undetermined coefficients:
f(x)
Assume two points (x0, x1) so that the f(x1)
integral can be written in the form
f(x0)

I c0 f ( x0 ) c1 f ( x1 )

c‘s: constants
-1 x0 x1 1 x
This requires calculation of four unknowns.
We choose these four equations according to our preference.
We can choose a function value that is exact to third order.
1

c0 f ( x0 ) c1 f ( x1 ) 1dx 2 In general, integration

1 limits can be made -1..1 for
1
any integral by a simple
c0 f ( x0 ) c1 f ( x1 ) xdx 0 change of variables.
1
 1
2 2
c0 f ( x0 ) c1 f ( x1 ) x dx
1
3
1
3
c0 f ( x0 ) c1 f ( x1 ) x dx 0
1

Or, by evaluating the function values

c0 c1 2

c0 x0 c1 x1 0
2 2 2
c0 x 0
c x 1 1
3
3 3
c0 x0 c1 x 1 0

By these deliberate choices, we force these two points give a an

integral that is exact to cubic order.
Solving for the unknowns
c0 c1 1
1
x0 0 . 5773503 ...
3
1
x1 0 . 5773503 ...
3

Then,
1 1 Two-point Gauss-
I f( ) f( )
3 3 Legendre formula
To change (normalize) the limits of integration, we do the
following change of variable:
x a0 a1 x d

For the lower limit (x=a xd=-1 )

(b a)
a0
a a0 a1 ( 1) 2
For the upper limit (x=b xd=1 )
(b a)
a1
b a0 a1 (1) 2

Substitute into the original formula:

(b a) (b a ) xd These equations are
x subsituted into the
2
original integrand to
Differentiate both sides: transfrom into suitable
(b a) form for application to
dx dx d Gauss-Legendre formula
2
EX: Use two-point Gauss-Legendre quadrature formula to integrate
2 3 4 5
f ( x) 0 .2 25 x 200 x 675 x 900 x 400 x

from x=0 to x=0.8 (remember that exact solution I=1.640533).

First, perform a change of variable to convert the limits -1..1:

x 0 .4 0 .4 x d dx 0 . 4 dx d
Substitute into the original equation to get:
0 .8
2 3 4 5
( 0 .2 25 x 200 x 675 x 900 x 400 x ) dx
0

1 2
0 .2 25 ( 0 . 4 0 .4 x d ) 200 ( 0 . 4 0 .4 x d )
0 . 4 dx d
3 4 5
1 675 ( 0 . 4 0 .4 x d ) 900 ( 0 . 4 0 .4 x d ) 400 ( 0 . 4 0 .4 x d )

RHS is suitable for evaluation using Gauss quadrature. Evaulating for -1/√3
(=0.516741) and for 1/√3 (=1.305837), and gives the following result:
Equivalent to third
I 0 . 516741 1 . 305837 1 . 822578 11 . 1 % order accuracy in
t Simpson’s method.
 Higher-point formulas:
Higher-point versions of Gauss quadrature can be developed in the
form
I c0 f ( x0 ) c1 f ( x1 ) ... cn 1 f ( xn 1 ) n: number of points

For example, for 3 point Gauss-Legendre formulas, c’s and x’s are:
c0 0 . 5555556 x0 0 . 774596669
c1 0 . 8888889 x1 0 .0 Et ~ f(6)( )
c2 0 . 5555556 x2 0 . 774596669

Error for Gauss Quadrature:

2
2n 3
(n 1)!
4
(2n 2)
n= number of
Et 3
f ( ) points minus one
(2n 3) ( 2 n 2 )!
1 1
EX: Use three-point Gauss quadrature formula for the previous problem.

Using the tabulated c and x values we write:

I 0 . 5555556 f ( 0 . 7745967 ) 0 . 8888889 f ( 0 ) 0 . 5555556 f ( 0 . 7745967 )

which is
I 0 . 2813013 0 . 8732444 0 . 4859876 1 . 640533 exact to 7 S.F.!
 Numerical Differentiation
Types of Differentiation:
Forward expansion of Taylor series:
''
' f ( xi ) 2 h=step size
f ( xi 1 ) f ( xi ) f ( xi ) h h ...
2!
' f ( xi 1 ) f ( xi ) first forward
f ( xi ) O (h)
h difference
Backward expansion of Taylor series
''
' f ( xi ) 2
f ( xi 1 ) f ( xi ) f ( xi ) h h ...
2!
f ( xi ) f ( xi 1 ) first backward
'
f ( xi ) O (h) difference
h

Subtract forward expansion from the backward expansion:

' f ( xi 1 ) f ( xi 1 ) 2 centered
f ( xi ) O (h )
2h difference
Higher-accuracy Differentiation Formulas:
High-accuracy diffentiation formulas can be obtained by adding
more terms in Taylor expansion.
Forward Taylor series expansion:
''
' f ( xi ) 2
f ( xi 1 ) f ( xi ) f ( xi ) h h ...
2
or
''
' f ( xi 1 ) f ( xi ) f ( xi ) 2
f ( xi ) h O (h )
h 2

Approximate the second derivative using finite difference formula

'' f ( xi 2
) 2 f ( xi 1 ) f ( xi )
f ( xi ) 2
h
Then,
' f ( xi 1 ) f ( xi ) f ( xi 2
) 2 f ( xi 1 ) f ( xi ) 2
f ( xi ) 2
h O (h )
h 2h
or,
' f ( xi ) 4 f ( xi 1 ) 3 f ( xi ) 2
forward
2
f ( xi ) O (h ) scheme
2h

Backward and centered finite difference formulas can be derived in

a similar way:

' 3 f ( xi ) 4 f ( xi 1 ) f ( xi 2 ) 2 backward
f ( xi ) O (h ) scheme
2h

' f ( xi 2
) 8 f ( xi 1 ) 8 f ( xi 1 ) f ( xi 2 ) 4 centered
f ( xi ) O (h )
12 h scheme
EX: Calculate the approximate derivative of
4 3 2
f ( x) 0 .1 x 0 . 15 x 0 .5 x 0 . 25 x 1 .2

at x=0.5 using a step size of h=0.25 (true value=-0.9125).

First, evaluate the following data points:

xi-2=0 ; f(xi-2)=1.2 xi+1=0.75 ; f(xi+1)=0.6363281
xi-1=0.25 ; f(xi-2)=1.103516 xi+2=1 ; f(xi+2)=0.2
xi=0.5 ; f(xi)=0.925

Forward difference scheme:

' 0 .2 4 ( 0 . 6363281 ) 3 ( 0 . 925 )
f ( xi ) 0 . 859375 t
5 . 82 %
2 ( 0 . 25 )
Backward difference scheme:
' 3 ( 0 . 925 ) 4 (1 . 035156 ) 1 . 2
f ( 0 .5 ) 0 . 878125 t
3 . 77 %
2 ( 0 . 25 )
Centered difference scheme:
' 0 .2 8 ( 0 . 6363281 ) 8 (1 . 035156 ) 1 . 2
f ( xi ) 0 . 9125 t
0%
12 ( 0 . 25 )
Richardson Extrapolation:
As done for the integration, Richardson extrapolation uses two
derivatives of different step sizes to obtain a more accurate
derivative.
In a similar fashion applied for the integration, use two step sizes
such that h2=h1/2. Richardson extrapolation recursive formula:

4 1
D D ( h2 ) D ( h1 )
3 3

O(h4) O(h2) [centered difference scheme]

The approach can be iteratively used by Romberg algorithm to get

higher accuracies.
EX: Use Richardson extrapolation of step sizes h=0.5 and h=0.25 to calculate the
derivative of the function
4 3 2
f ( x) 0 .1 x 0 . 15 x 0 .5 x 0 . 25 x 1 .2

at x=0.5.

Centered scheme finite difference approximation for h=0.5:

0 .2 1 .2
D ( 0 .5 ) 1 .0 t
9 .6 %
1

Centered scheme finite difference approximation for h=0.25:

0 . 6363281 1 . 103516
D ( 0 .5 ) 0 . 934375 t
2 .4 %
0 .5
Apply Richardson extrapolation for improved accuracy:

4 1
D ( 0 .5 ) ( 0 . 934375 ) ( 1) 0 . 9125 t
0%
3 3
Derivatives of Unequally Spaced Data
In the previous discussion, both finite difference approximations
and Richardson extrapolation reqires evenly distributed data. So,
these methods are more suitable to evaulate functions.
Emprically derived data, experimental or from field surveys, are
usually not even.
One way is to fit a second-order Lagrange interpolating polynomial
to each set of three data points. Derivative of the polynomial:

' 2x xi xi 1
2x xi 1
xi 1
2x xi 1
xi
f ( x) f ( xi 1 ) f ( xi ) f ( xi 1 )
( xi 1
x i )( x i 1
xi 1 ) ( xi x i 1 )( x i xi 1 ) ( xi 1
x i 1 )( x i 1
xi )

Using above formula, any point in the range of three data points
can be evaluated.
This equation has the same accuracy of high-accuracy centered
difference approximation even though data is not need to be
equally spaced.
Derivatives and Integrals for Data with Error:
Differentiation process amplifies the error:

y dy/dt

x x
As a remedy, fit a smoother function (low-order polynomial) to the
uncertain data.
On the other hand, integration process reduces the error.
(Succusive negative and positive errors cancel out during
integration). No further action is required.