Lecture 31

Last lecture:
Improper Integral

This lecture:

Adaptive Integration
ODE—Initial value problems
Taylor series solution
Euler explicit method
 CHAPTER VII

ODE'S—INITIAL VALUE PROBLEMS

Objectives: Develop various accurate & efficient methods for solving ODEs’ such as
y' = f(x, y) (1)
Initial-Value & Boundary-Value problems
The Initial-value Conditions ---
ALL conditions are given at the same value of the independent variable (t).

du
For example at t = 0, u = 0.5 & = 1.0
dt

The two conditions are all given at t = 0.

u

1
u = 0.5 &
du 0.5
= 1.0
dt
@t = 0
o t
t=0
 Initial-Value & Boundary-Value Conditions
The B-V Conditions--- Conditions are given at different values
of the independent variable (x).

e.g. at x = 0, y = 1& at x = 2, y = 2.
2-D heat conduction
y
1-D heat conduction T=75
y=2
T=90 T=15 T=100 T=0
y=1
0 2 x T=25
o x=0 x=2 x

The numerical schemes for solving Initial-value and boundary-value problems are different.
 7.1 Taylor series expansion for ODE

◊ Example: Consider the first order ODE:

y' = -2x - y (= f(x, y) ) (2)

with y(0) = -1 (3)

* Exact solution: y(x) = -3e-x - 2x + 2 (4)

* Taylor series (TS) solution:

y(x) = y0 + 𝑦0′ (x-x0) + 2 𝑦0" (x-x0)2 + 3! 𝑦0"′ (x-x0)3 + 4! 𝑦0"" (x-x0)4 +

1 1 1
(5)

At x0 = 0, y0 = -1. Eq. (2)  𝑦0′ = 1 .

Differentiate Eqn. (2)  y" = -2 - y' 

𝑦0" = -2-1 = -3 .
Differentiate y" = -2 –y ' w.r.t. x  y"' = - y"

so that 𝑦0"′ = 3.
Taylor series expansion for ODE y' = -2x - y (= f(x, y) )

Differentiate y"' = - y" w.r.t. x  y"" = - y"' 

𝑦 "" = -3  𝑦 = 3
(5)
0 0
3 1 1
Finally, y(x) = -1 + x - 2 x2 + 2 x3 - 8 x4 + ... (6a)
3 1 1
or y(h) = -1 + h - 2 h2 + 2 h3 - 8 h4 + ... (6b)
* Compare with the exact solution:
h y(TS) yexact yexact -y(TS) h5/40
0.1 -0.9145125 -0.914512254 2.46E-7 2.50E-07

0.2 -0.85620 -0.856192259 7.74E-6 8.00E-06

0.4 -0.81120 -0.81096 2.40E-4 2.56E-04

0.8 -0.95520 -0.947986 7.21E-3 8.19E-03

1
* The TS solution gets worse as h = |x-x0| gets larger, because T.E., 5! h5y(5)(x) in this example, is estimated to be h5/40.
At h=0.8, it gives 0.00819. Such estimate can be unreliable if h is large due to the uncertainty in x.
* Another problem: in general, it is very tedious to get y(n)(x) from the original ODE.

 there is a practical difficulty to get y(n)(x).

* Thus, Taylor series is not used for obtaining numerical solution to the ODEs.
7.2 Euler & Modified Euler Methods for y' = f(x, y), y0 = y(x0)

7.2.1 Euler explicit method

TS: y(x) = y0 + 𝑦0′ (x-x0) + 2 𝑦0" (x-x0)2 + 3! 𝑦0"′ (x-x0)3 +…
1 1
* Rationale: (5)
In TS expansion, if (x-x0) is small, T.E. will be small.
If we just keep upto O(h) term in Eq. (5),
y(x0 + h) = y(x0) + 𝑦0′ h + 2 𝑦0" h2 + 3! 𝑦0"’ h3 + …
1 1
(5)’

 yn+1 = yn + hyn' = yn + hf(xn, yn), n=1, 2, ... (7)

* Euler method is explicit. It is easy to start, but inaccurate.

* Finite difference interpretation:

- 2 𝑦n" h +…
𝑦𝑛+1 −𝑦𝑛 1
Forward difference for y': 𝑦𝑛′ = ℎ

ODE (exact): yn' = f(xn, yn)

- 𝑦n" h +… = f(xn, yn)

𝑦𝑛+1 −𝑦𝑛 1
combine   yn+1 = yn + hf(xn, yn) + 2 𝑦n" h2 +…
1
ℎ 2
(7)
= T.E.
* Geometric interpretation

y
y(xn+1) y(x) Run: h
error {
yn+1 Rise: hyn′

hyn′ Error: yn"h2/2 yn+1 = yn + h * slope at (xn, yn)

h
xn xn+1 x

* Note, since we start from n=1, after n time steps, the global error at xn+1 is: nh2 ~ O(h).
Hence, the Euler method has ONLY a first order accuracy
 Example:

* Example: Solve y' = -2x -y, y(0) = -1 using Euler explicit scheme.
xn yn yn' yexact(xn) y(TS) yexact- yn
0 -1 1
0.1 -0.9 0.7
0.2 -0.83 0.43
0.3 -0.787 0.183
0.4 -0.7683 -0.0317 -0.81096 -0.8112 -0.04266
0.5 -0.77147 -0.22853
0.6 -0.794323 -0.405677
0.7 -0.8348907 -2.2348907
0.8 -1.0583798 -0.947986 -0.9552 0.110394

* Comments:
i. Error is relatively large at x=0.4 with only 4 time steps:
T.E.(x=0.4) = -0.04266.
ii. In order to reduce the T.E., say by a factor of 100, we need to reduce h by 100 which means an
increase in computational effort by a factor of 100.
 Lecture 32
Last lecture:
Taylor series solution
Euler explicit method
This lecture:

Euler implicit
Euler modified
Runge Kutta method
7.2.2 Euler Implicit Method for y = f(x, y)

* Taylor series expansion at x =𝑥𝑛+1 (as opposed to 𝑥𝑛 in the explicit method)

′
ℎ2 ″
𝑦(𝑥𝑛 ) = 𝑦(𝑥𝑛+1 − ℎ) = 𝑦(𝑥𝑛+1 ) − ℎ𝑦 (𝑥𝑛+1 ) + 𝑦 (𝑥𝑛+1 )+. . .
2!
= 𝑛+1 𝑛 + 𝑦 " h + …
𝑦 −𝑦 1
 𝑦′ 𝑥 𝑛+1 n+1
ℎ 2!

* Evaluate the slope f(x, y) also at (xn+1, yn+1):

𝑦𝑛+1 −𝑦𝑛 1 "
+ 2 𝑦n+1 h +… = f(xn+1, yn+1)
ℎ

1 " 2
 yn+1 = yn + hfn+1 - 2 𝑦n+1 h +… = yn + hfn+1 + O(h2)

yn+1 = yn + hfn+1 (8)

It is called Euler implicit scheme because “y” in f(x, y) is an unknown.
* Example: Solve y' = -2x - y, y(0) = -1 using Euler implicit scheme:
yn+1 = yn + hfn+1
yn+1 = yn + h (-2xn+1 - yn+1)
 yn+1 = (yn -2hxn+1)/(1+h)
 xn yn yexact(xn) yexact -yn
0.1 -0.927272 -0.914512254
0.2 -0.8793388 -0.856192259
0.3 -0.8539444 -0.822454662
0.4 -0.84904 -0.810960138 0.03808
* Comment about local T.E.:
1 "
in explicit method: T.E. ~ 𝑦 h2
2 n
 opposite to each other
in implicit method: T.E. ~ - 𝑦 " h2
1
n+1 2

y
y
y(xn+1) y(x) Run: h yn+1 y(x)
Run: h
error { error {
yn+1 Rise: hyn′ y(xn+1) Rise: hy′n+1

hyn′ Error: yn"h2/2

hy′n+1 Error: y"n+1h2/2
h
h
xn xn+1 x
xn xn+1 x
 Two problems with Euler implicit & explicit methods:
* In general f(xn+1, yn+1) depends on yn+1 which is an unknown.
If f(x, y) is a nonlinear function of y, the procedure for getting yn+1
can be complicated in the implicit method.
* In both methods, the accuracy is poor. The T.E. is O(h).!
7.2.3 Euler modified method—predictor-corrector method

 Strategy: combine Euler explicit & implicit methods

to cancel out leading order T.E.
Slope: 𝑦𝑛+1 −𝑦𝑛 1
ℎ
= 2 [f(xn, yn) + f(xn+1, yn+1) ] = average slope in [xn, xn+1].
1
 yn+1 = yn + 2 h [f(xn, yn) + f(xn+1, yn+1) ] (9)
 Why does it improve accuracy?
y
y(x)
Expand yn and yn+1 near x = xn+1/2 using Taylor series: yn+1 Run: h
1 ’ 1 h 2 " 1 h 3 ′′′ error {
y(xn+1)
yn = yn+1/2 - 2 h 𝑦n+1/2 + 2 ( 2 ) 𝑦n+1/2 - 3! ( 2 ) 𝑦n+1/2 + O(h4) Rise: h(y′n+y′n+1)/2
1 ’ 1 h 2 " 1 h 3 ′′′
yn+1 = yn+1/2 + 2 h 𝑦n+1/2 + 2 ( 2 ) 𝑦n+1/2 + 3! ( 2 ) 𝑦n+1/2 + O(h4) Error: -y"'n+1/2h2/12
h(y′n+y′n+1)/2
yn+1− yn ℎ2 h
Subtracting  "
𝑦n+1/2 = - ′′′
𝑦n+1/2
ℎ 24 xn+1 x
xn
h 𝑑𝑓 1 h 2 𝑑 𝑓
2
Similarly, fn = fn+1/2 - 2 𝑑𝑥ቚ + ( ) 𝑑𝑥 2 ቚ
2 2
+ O(h3)
𝑛+1/2 𝑛+1/2
1 ℎ2 𝑑2 𝑓
h 𝑑𝑓 adding  fn+1/2 = (fn + fn+1) - ቚ
1 h 2𝑑 𝑓
2
2 8 𝑑𝑥 2 𝑛+1/2
fn+1 = fn+1/2 + 2 𝑑𝑥ቚ + ( ) 𝑑𝑥 2 ቚ
2 2
+ O(h3)
𝑛+1/2 𝑛+1/2
 Accuracy of Euler modified method

Hence the ODE y' = f at x = xn+1/2: y'n+1/2 = fn+1/2

becomes yn+1− yn 1 ℎ2 𝑑2 𝑓 ℎ2 ′′′
= (fn + fn+1) - ቚ + 𝑦n+1/2
ℎ 2 8 𝑑𝑥 2 𝑛+1/2 24

𝑑2 𝑓 ℎ2 ′′′
Because y"' = , the above gives a T.E.: - 𝑦n+1/2
𝑑𝑥 2 12

 How to get fn+1 with yn+1 yet to be determined?

* Idea: we ESTIMATE yn+1 using Euler explicit method at every step.
Predictor: 𝑦തn+1 = yn + hfn + O(h2) (10)
(as a preliminary, predicted value of y at xn+1)
ℎ
Corrector: yn+1 = yn + 2 [f(xn, yn) + f(xn+1, 𝑦ത n+1)] (11)
* Qualitative error analysis:
In the corrector step for calculating fn+1, yn+1 was replaced by the predictor 𝑦തn+1.
Since 𝑦ത n+1 has a local error of O(h2) from explicit method, fn+1 using 𝑦തn+1 has an error of O(h2), too.
This results in an error of O(h3) in yn+1 in (11) due to O(h2) error in using 𝑦തn+1. No iteration!
 𝑦ത n+1 = yn + hfn
Example of Euler modified method ℎ
yn+1 = yn + 2 [f(xn, yn) + f(xn+1, 𝑦ത n+1)]

Example: Solve y' = -2x - y, y(0) = -1 using Euler modified method & h=0.1
= f(x, y)
xn yn 𝑦തn+1 ҧ
𝑓n+1 yexact -yn
0 -1 -0.9 0.7 𝑦തn+1 = -1 + (-2*0-(-1))* 0.1 = -0.9
0.1 -0.915 -0.8435 0.4435
f(xn+1, 𝑦തn+1) = -2*0.1-(-0.9) =0.7
0.2 -0.857075 -0.811368 0.2113675
0.3 -0.82365288 -0.801288 0.0012876 yn+1 = -1+0.1*(1+0.7)/2 =-1+0.085 = = -0.915
0.4 -0.81240585 -0.811165 -0.188835 1.45E-03
𝑦തn+1 =-0.915 + (-2*0.1-(-0.915))* 0.1 = -0.8435

* The error at x=0.4 in this scheme is 1.45E-3.

It is 29 times smaller than Euler explicit method
* The modified Euler (Predictor-Corrector) method has an O(h2) error.

* To further reduce error, we need to use smaller h.

* We can do better by using higher order schemes—
such as Runge Kutta method.
7.3 Runge-Kutta Method

7.3.1 Second-order Runge-Kutta method

* Let yn+1 = yn + ak1 + bk2 (12a)

With k1 = h f(xn, yn) (12b)

k2 = h f(xn+ ah, yn +bk1) (12c)

where a, b, a, and b are to be determined so that the local error is as small as possible.
* Taylor series expansion for yn+1 based on ODE:
yn+1 = yn + hy′(xn, yn) + 2 𝑦n" h2 +…
1
(7)
h2 𝑑𝑓
= yn + hf(xn, yn) + ቚ +…
2 𝑑𝑥 𝑛

𝑑𝑓 𝑓 𝑓 𝑑𝑦
= 𝑥 + =f +f f
𝑑𝑥 𝑦 𝑑𝑥 x y
 yn+1 = yn + hfn + h2/2 (fx + fyf )n (13)

* Combining Eq. (12a-c)  yn+1 = yn + ah fn + bh f[xn+ah, yn+bhfn] (14)

 2nd order Runge-Kutta Method

yn+1 = yn + hfn + h2/2 (fx + fyf)n (13)

yn+1 = yn + ah fn + bh f[xn+ah, yn+bhfn] (14)

* Taylor series expansion for f[xn+ah, yn+bhfn],

f[xn+ah, yn+bhfn] ~ fn + ahfx(xn, yn) + bhfnfy(xn, yn) + O(h2)

* Eqn. (14), i.e. RK method, for yn+1 becomes

yn+1 = yn + ah fn + bh [fn + ahfx(xn, yn) + bhf fy(xn, yn) + O(h2)]

= yn + (a+b) h fn + bh2 (afx+ bf fy)n + O(h3) (15)

* Comparing (15) (based on the scheme) with (13) (from ODE),
we require: a+b=1
ba = 1/2
bb = 1/2

The above are 3 equations for 4 unknowns → many possible valid choices.
 yn+1 = yn + ak1 + bk2; k2 = h f(xn+ ah, yn +bk1)
2nd order Runge-Kutta Method
a + b = 1; ba = 1/2; bb = 1/2

* Predictor-Corrector method (Heun’s method)

Choose: a = 1/2  b = 1/2, a =1, b = 1 True

value
* Mid-point method y

Choose: a=0  b=1, a =1/2, b = 1/2 yn + k2

 k1 = h f(xn, yn),
yn + k1/2
k2 = h f(xn+ h/2, yn + k1/2)
& yn+1 = yn + k2 xn xn + h/2 xn+1 𝑥
Mid-point method
* Ralston’s method
Choose: a = 1/3  b = 2/3, a = b = 3/4.

* You can make your own reasonable choices for a…

7.3.2 Classical 4th order Runge-Kutta Method

• Algorithm description for RK4 method:

Compute 4 slopes (or derivatives): s1, s2, s3, s4 𝑦𝑛 + 𝑘3
1
𝑦𝑛 + 𝑘1
𝑘1 = ℎ𝑓(𝑥𝑛 , 𝑦𝑛 ) 2 s2
1
s4
1 1 𝑦𝑛 + 𝑘2
𝑘2 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘1 ) 2 s3
2 2
1 1
𝑘3 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘2 )
2 2 
𝑘4 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘3 )
1
𝜑 = (𝑠1 + 2𝑠2 + 2𝑘𝑠3 + 𝑠4 )
6
1
𝑦𝑛+1 = 𝑦𝑛 + (𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4 )
6
• Comments: Local error O(h5); xn xn + h/2 xn + h
global error: O(h4);
highly accurate.
single step; self-starting.
 Lecture 33
Last lecture:
Euler implicit method
Euler modified method
RK2 method
This lecture:

Runge Kutta high order method

Automatic method
Multistep methods
7.3.2 Classical 4th order Runge-Kutta Method

• Algorithm description for RK4 method:

Compute 4 slopes (or derivatives): s1, s2, s3, s4 𝑦𝑛 + 𝑘3
1
𝑦𝑛 + 𝑘1
𝑘1 = ℎ𝑓(𝑥𝑛 , 𝑦𝑛 ) 2 s2
1
s4
1 1 𝑦𝑛 + 𝑘2
𝑘2 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘1 ) 2 s3
2 2
1 1
𝑘3 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘2 )
2 2 
𝑘4 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘3 )
1
𝜑 = (𝑠1 + 2𝑠2 + 2𝑘𝑠3 + 𝑠4 )
6
1
𝑦𝑛+1 = 𝑦𝑛 + (𝑘1 + 2𝑘2 + 2𝑘3 + 𝑘4 )
6
• Comments: Local error O(h5); xn xn + h/2 xn + h
global error: O(h4);
highly accurate.
single step; self-starting.
 Example for RK4 method

* Example: y = f(x, y) = x - y, y(x=0) = 0 1

𝑦𝑛 + 𝑘1 s2
Solution: Choose h = dx = 0.2 2 𝑦𝑛 + 𝑘3
1 s4
@ x0 = 0, y0 = 0,  s1 = f(x0, y0) = 0 𝑦𝑛 + 𝑘2 s3
2

 y0 + s1h/2 = 0+0.0 = 0.0  s2 = f(0+h/2, 0.0) = 0.1- 0 = 0.1

 y0 + s2h/2 = 0+0.01= 0.01  s3 = f(0+h/2, 0.01) = 0.1- 0.01 = 0.09

 y0 + s3 h = 0+0.2*.09 = 0.018  s4 = f(0+h, 0.018) = 0.2- 0.018 = 0.182 xn xn + h/2 xn + h

 y1 = y0 + (s1+ 2s2 +2s3 + s4) h/6 = 0 + (0+2* 0.1+2* 0.09+ 0.182)*0.2/6 = 0.0187333
x y s1 s2 s3 s4 yexact
0 0 0 0.1 0.09 0.182 0 1.2 7.E-06
y Error
0.2 0.0187333 0.18127 0.26314 0.25495 0.33028 0.0187308 1 6.E-06

0.4 0.0703243 0.32968 0.39671 0.39 0.45167 0.07032 5.E-06

0.8
0.6 0.1488168 0.45118 0.50606 0.50058 0.55107 0.1488116 4.E-06
0.8 0.2493346 0.55067 0.5956 0.59111 0.63244 0.249329 0.6
RK4 3.E-06
1 0.3678852 0.63211 0.6689 0.66522 0.69907 0.3678794
0.4 Exact
1.2 0.5011999 0.6988 0.72892 0.72591 0.75362 0.5011942 2.E-06
1.4 0.6466024 0.7534 0.77806 0.77559 0.79828 0.646597 0.2 Error
1.E-06
1.6 0.8019016 0.7981 0.81829 0.81627 0.83484 0.8018965
1.8 0.9653036 0.8347 0.85123 0.84957 0.86478 0.9652989 0 0.E+00
2 1.1353395 1.1353353 0 0.5 1 1.5 2 x
 Performance comparison of four methods

* Comparison of four methods for solving

y = f(x, y) = x - y, y(x=0) = 0
using Euler explicit, Euler implicit, Euler modified, & RK4.
Exact solution : yexact = exp(-x)+ x-1
1.E+00
|yn-exact| y' = x-y; y(0)=0
Euler Euler Euler 1.E-01
x explicit Implicit Modified RK4 yexact
0 0 0 0 0 0 1.E-02
0.2 0 0.033333 0.02 0.0187333 0.01873075
0.4 0.04 0.094444 0.0724 0.0703243 0.07032005 1.E-03
0.6 0.112 0.178704 0.151368 0.1488168 0.14881164 Euler Explicit Euler Implicit

0.8 0.2096 0.282253 0.252122 0.2493346 0.24932896 1.E-04 Euler modified RK4
1 0.32768 0.401878 0.37074 0.3678852 0.36787944
1.2 0.462144 0.534898 0.504007 0.5011999 0.50119421 1.E-05
1.4 0.609715 0.679082 0.649285 0.6466024 0.64659696
1.6 0.767772 0.832568 0.804414 0.8019016 0.80189652 1.E-06
1.8 0.934218 0.993807 0.96762 0.9653036 0.96529889 0 0.5 1 1.5 2 x
2 1.107374 1.161506 1.137448 1.1353395 1.13533528
Butcher’s 6th-order RK Method

1
𝑦n+1 = 𝑦𝑛 + (7𝑘1 + 32𝑘3 + 12𝑘4 + 32𝑘5 + 7𝑘6 )
90
𝑘1 = ℎ𝑓(𝑥𝑛 , 𝑦𝑛 )
1 1
𝑘2 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘1 )
4 4
1 1 1
𝑘3 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘1 + 𝑘2 )
4 8 8
1 1
𝑘4 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 − 𝑘2 + 𝑘3 )
2 2
3 3 9
𝑘5 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 + 𝑘1 + 𝑘 )
4 16 16 4
3 2 12 12 8
𝑘6 = ℎ𝑓(𝑥𝑛 + ℎ, 𝑦𝑛 − 𝑘1 + 𝑘2 + 𝑘 − 𝑘 + 𝑘 )
7 7 7 3 7 4 7 5

--just one version of the 6th-order RK Method

 x y (Butcher6) s1 s2 s3 s4 s5 s6 y_exact Error
0 0 0 0.05 0.04875 0.09525 0.139284375 0.181249286 0
0.2 0.018730767 0.181269233 0.222205772 0.221182358 0.259253339 0.295305636 0.329663600 0.018730753 1.359E-08
0.4 0.070320068 0.329679932 0.363195935 0.362358035 0.393527918 0.423045043 0.451174965 0.070320046 2.225E-08
0.6 0.148811663 0.451188337 0.478628920 0.477942905 0.503462648 0.527629226 0.550660059 0.148811636 2.733E-08
0.8 0.249328994 0.550671006 0.573137456 0.572575795 0.593469593 0.613255514 0.632111565 0.249328964 2.983E-08
1 0.367879472 0.632120528 0.650514502 0.650054653 0.667161048 0.683360391 0.698798420 0.367879441 3.053E-08
1.2 0.501194242 0.698805758 0.713865470 0.713488977 0.727494510 0.740757410 0.753396999 0.501194212 2.999E-08
1.4 0.646596993 0.753403007 0.765732857 0.765424611 0.776891371 0.787750115 0.798098536 0.646596964 2.865E-08 1.E+00
1.6 0.801896545 0.798103455 0.808198282 0.807945912 0.817334101 0.826224489 0.834697060 0.801896518 2.681E-08 |yn-exact| y' = x-y; y(0)=0
1.8 0.965298913 0.834701087 0.842966033 0.842759409 0.850445809 0.857724643 0.864661397 0.965298888 2.469E-08 1.E-01
2 1.135335306 1.135335283 2.246E-08
1.E-02

1.E-03
Euler Explicit Euler Implicit
1.E-04 Euler modified RK4
Butcher6
1.E-05

1.E-06

1.E-07

1.E-08
0 0.5 1 1.5 2
x
7.3.3 Automatic Runge-Kutta methods

Objective: we specify an error tolerance,

let the computer figure out the optimum h in order to maintain accuracy and efficiency.

First approach: Step halving (skip)

▪ Estimate local truncation error using two different step sizes

▪ Solve each step twice, once as a full step and then as two half steps

Second approach: Embedded RK methods

▪ Estimate local truncation error between two predictions using 4th-order & 5th order RK methods
• Compute 6 or 7 slopes (or derivatives)

𝑘1 = ℎ𝑓(𝑥𝑛 , 𝑦𝑛 )
𝑘2 = ℎ𝑓(𝑥𝑛 + 𝟐 𝒉, 𝑦𝑛 + 21 𝑘1 )
𝑘3 = ℎ𝑓(𝑥𝑛 + 𝟑 𝒉, 𝑦𝑛 + 31 𝑘1 + 32 𝑘2 )
𝑘4 = ℎ𝑓(𝑥𝑛 + 𝟒 𝒉, 𝑦𝑛 + 41 𝑘1 + 42 𝑘2 + 43 𝑘3 )
𝑘5 = ℎ𝑓(𝑥𝑛 + 𝟓 𝒉, 𝑦𝑛 + 51 𝑘1 + 52 𝑘2 + 53 𝑘3 + 54 𝑘4 )
𝑘6 = ℎ𝑓(𝑥𝑛 + 𝟔 𝒉, 𝑦𝑛 + 61 𝑘1 + 62 𝑘2 + 63 𝑘3 + 64 𝑘4 + 65 𝑘5 )
𝑘7 = ℎ𝑓(𝑥𝑛 + 𝟕 𝒉, 𝑦𝑛 + 71 𝑘1 + 72 𝑘2 + 73 𝑘3 + 74 𝑘4 + 75 𝑘5 + 76 𝑘6 )
𝑦𝑛+1 = 𝑦𝑛 + 1 𝑘1 + 2 𝑘2 + 3 𝑘3 + 4 𝑘4 + 5 𝑘5 + 6 𝑘6 + 7 𝑘7 4th order accurate
𝑦𝑛+1 = 𝑦𝑛 + 1 𝑘1 + 2 𝑘2 + 3 𝑘3 + 4 𝑘4 + 5 𝑘5 + 6 𝑘6 + 7 𝑘7
5th order accurate
𝐸𝑟𝑟𝑜𝑟 = |𝑦𝑛+1 − 𝑦𝑛+1 | 4th order error estimate

If 𝐸𝑟𝑟𝑜𝑟 < Tolerance, accept yn+1 as the result; go to next interval.

If 𝐸𝑟𝑟𝑜𝑟 >Tolerance, reduce step size by half (replacing h by h/2), repeat the process for y(xn+ h/2).
If 𝐸𝑟𝑟𝑜𝑟 <0.1*Tolerance, double step size (replacing h by 2h).
Butcher tableau for the Dormand–Prince (RKDP) method

i 𝑖 𝑖1 𝑖2 𝑖3 𝑖4 𝑖5 𝑖6

1 0
2 1/5 1/5
3 3/10 3/40 9/40
4 4 44/45 −56/15 32/9

5 8/9 19372/6561 −25360/2187 64448/6561 −212/729

6 1 9017/3168 −355/33 46732/5247 49/176 −5103/18656

7 1 35/384 0 500/1113 125/192 −2187/6784 11/84 0

𝑗 35/384 0 500/1113 125/192 −2187/6784 11/84 0

FSAL
𝑗 5179/57600 0 7571/16695 393/640 −92097/339200 187/2100 1/40

RKDP method is implemented in Matlab ODE45

FSAL = First Same As Last
Butcher tableau for the Cash–Karp method

i 𝑖 𝑖1 𝑖2 𝑖3 𝑖4 𝑖5

1 0
2 1/5 1/5
3 3/10 3/40 9/40
4 3/5 3/10 −9/10 6/5

5 1 −11/54 5/2 −70/27 35/27

6 7/8 1631/55296 175/512 575/13824 44275/110592 253/4096

𝑗 37/378 0 250/621 125/594 0 512/1771

𝑗 2825/27648 0 18575/48384 13525/55296 277/14336 1/4

7.4 Multi-step Method

7.4.1 Basic Idea

* ODE: y = f(x, y)
xn+1 xn+1
* Integrate: න ydx = න f(x, y)dx
xn xn

LHS = y(xn+1) – y(xn); it is exact!

RHS: needs to be as accurate as possible.

* RK2 / RK4: 2 / 4 slopes are computed within ONE step &

then are combined to achieve higher accuracy.
→ More slopes  better interpolation for f(x, y)
 better accuracy for integration.

* If we use fi's from previous intervals => improve the accuracy. x

xn-2 xn-1 xn xn+1

7.4.2 Adams-Bashforth method:

 Suppose (fn-2, fn-1, fn) are already obtained and stored.

* Interpolation for f (--parabolic fit):
x
Let f(x, y(x)) = fn + a (x-xn)/h + b [(x-xn)/h]2 + O(h3)
xn-2 xn-1 xn xn+1
 fn-1 = fn - a + b @ x = xn-1
Quadratic extrapolation of f(x, y)
fn-2 = fn - 2a + 4b @ x = xn-2
 a = (3fn - 4 fn-1 + fn-2)/2
b = (fn - 2fn-1 + fn-2)/2
xn+1
Integrate: න f(x, y)dx = h(fn+ a/2 + b/3) + O(h4)
xn
3
= h(23fn - 16fn-1 + 5fn-2)/12 + 8 h4 y(4)(x)
ℎ 3
 y(xn+1) – y(xn) = (23fn - 16fn-1 + 5fn-2) + 8 h4 y(4)(x)
12

 Adams-Bashforth 3rd order method

Adams-Bashforth 4th order method:

◊ Adams-Bashforth 4th order method:

* Using the same idea in deriving the 3rd order scheme, we get
ℎ 251
yn+1 = yn + (55fn - 59fn-1 + 37fn-2 -9fn-3) + h5 y(5)(x)
24 720

* Locally, AB4 method is 5th order accurate.

* Globally, AB4 method is 4th order accurate (due to the accumulation of truncation error).

* Note: (fn-3, fn-2, fn-1, fn) are all past values already obtained; no estimated values are used;
no additional effort is involved to get (fn-2, fn-1, fn).
* Problem: need (f1, f2, f3) besides f0 to start the process for f4.
Single step method (e.g. RK4) must be used to start.

* However, if AB4 scheme is used, one only needs to employ RK4 (standard program is available in the open literature)
to compute the first 3 points: f1, f2, & f3. One can switch to AB4 to compute f4, …, fn.
* It has the same order of accuracy but takes only 1/4 of effort.
* AB4 is NOT used alone; it is used with AM4 together
7.4.3 Adams-Moulton methods:

* Using information at n+1 x

xn-2 xn-1 xn x n+1
3-point:
ℎ 1 4 (4)
yn+1 = yn + (5fn+1 + 8fn -fn-1) - h y (x)
12 24

4-point:
ℎ 19 5 (5)
yn+1 = yn + (9fn+1 +19fn - 5fn-1+ fn-2) - h y (x)
24 720

Note : T.E. is much (13 times) smaller than AB4 method.

7.4.4 Predictor-Corrector method to improve stability

* Predictor:
Use the 4-point Adams-Bashforth scheme
ℎ
𝑦𝑛+1 = yn + 24 (55fn - 59fn-1 + 37fn-2 -9fn-3)

to evaluate/approximate the slope: fn+1 = f(xn+1, 𝑦𝑛+1 )

* Corrector:
Switch to a more accurate 4-point Adams-Moulton scheme

ℎ
Accept: yn+1 = yn + (9fn+1 +19fn - 5fn-1+ fn-2)
24

* The above scheme is referred as 4th order Adams-Bashforth- Adams-Moulton (ABAM) scheme

* Advantage comparing with RK 4th order scheme

ABAM only uses 2, as opposed to 4, function evaluations in ONE step.

 ABAM is more efficient
 Example for using ABAM 4th order method

Example: Use Adam-Bashforth-Adam-Moulton 4th order method and h=0.2 to solve

y = 1-xy; y(0)=1.
Make pertinent comparisons in terms of performance.
𝑥 1 2
Solution: i) Exact solution: y(x) =exp(-x2/2) [1+ ‫׬‬0 exp 𝑡 𝑑𝑡
2

The integration can be evaluated using Richardson extrapolation

of the results from Simpson’s rule with dt=0.01 and 0.005

ii) For h=0.2, we first compute y(0.2), y(0.4), and y(0.6) using RK4 method.
The results are listed in “y_ABAM4” column from 2nd to 4th row.
iii) At x=0.6, we switch to ABAM4 by computing the predictor

0.2
𝑦(0.8) = y(0.6) + (55*0.1790842 – 59* 0.479021 + 37*0.7644896 -9*1) =1.3754723
24

iv) Compute the slope at x=0.8 using f*(x=0.8, 1.3754723) =1-0.8*1.3754723 = -0.10038
0.2
v) Corrector : y(0.8) = y(0.6) + (9*(-0.10038) +19* 0.1790842 –5*0.479021 +0.7644896) = 1.375431273
24
vi) The rest of the results are given below.
x y_ABAM4 s1 s2 s3 s4 y_pred f *_n+1
0 1 1 0.89 0.8911 0.76436
0.2 1.177551867 0.7644896 0.6238 0.62802 0.4787376 1.E+00 1.8
0.4 1.302447455 0.479021 0.324825 0.332535 0.1786273 Error_ABAM4
0.6 1.368193082 0.1790842 0.029729 0.040184 -0.100984 1.375472 Error y 1.6
0.8 1.375431273 -0.100345 1.331752 -0.10038
1.E-01 Error_RK4
1 1.33119195 -0.3311919 1.248023 -0.33175 1.4
1.2 1.247104567 -0.4965255 1.137995 -0.49763
y_ABAM4
1.4 1.136961971 -0.5917468 1.015278 -0.59319 1.E-02 1.2
1.6 1.014363992 -0.6229824 0.891517 -0.62444
1.8 0.890888776 -0.6035998 0.775323 -0.60473 1
2 0.775046779 -0.5500936 0.671985 -0.55065 1.E-03
2.2 0.67202954 -0.478465 0.583809 -0.47837
0.8
2.4 0.584077441 -0.4017859 0.510829 -0.40114
2.6 0.511201683 -0.3291244 0.451628 -0.32815
2.8 0.452002305 -0.2656065 0.404086 -0.26456 1.E-04 0.6
3 0.40439381 -0.2131814 0.365929 -0.21226
3.2 0.366141256 -0.171652 0.335071 -0.17097 0.4
3.4 0.335189183 -0.1396432 0.309771 -0.13924 1.E-05
3.6 0.309816449 -0.1153392 0.288673 -0.11518 0.2
3.8 0.288670599 -0.0969483 0.270762 -0.09696
4 0.270734064 -0.0829363 0.255298 -0.08305
1.E-06 0
4.2 0.255261919 -0.0721001 0.241751 -0.07225
0 1 2 3 4 5 x
4.4 0.241715986 -0.0635503 0.229737 -0.0637
4.6 0.2297076 -0.056655 0.218976 -0.05679
4.8 0.218953078 -0.0509748 0.209258 -0.05108
5 0.20924143 -0.0462072 0.200423 -0.04629

vii) We can also continue with RK4 to obtain y(x). ABAM4 performs similarly in comparison with RK4.
 Lecture 34
Last lecture:
Runge Kutta high order method
Automatic method
Multistep methods
This lecture:

System of 1st orde ODEs and higher order ODE

Numerical instability
Methods for BVP
7.5 Systems of Equations and High-Order Equations

* Example: consider a mass-spring-dashpot system,

m𝑦ሷ + c𝑦ሶ + ky = f(t, y)
y(t=0) = y0, (t=0) = 𝑦ሶ 0 = v0.
How to solve this 2nd order ODE numerically for t>0?
* In general, a second order ODE of an initial value problem is
y" = f(x, y, y')
y(x0) = y0, y'(x0) = y'0.
* Let y' = u  u(x0)= y'0
& u' = y" = f(x, y, y') = f(x, y, u).
Thus we obtain a system of two first-order nonlinear ODE's:
𝑑𝑦
=u
𝑑𝑥
𝑑𝑢
= f(x, y, u)
𝑑𝑥
with y(x0) = y0; u(x0)= y'0

* Higher orders of ODEs can be similarly handled.

 Methods for solving system of first-order nonlinear ODE's

y1' = f1(x, y1, y2, ..., ym)

y2' = f2(x, y1, y2, ..., ym)
...
ym' = fm(x, y1, y2, ..., ym)
with yi(x0) = yi0, i=1, 2, ..., m
The methods dealing with one equation first order ODE can be easily handled by extended to above system.

 Take the predictor-corrector method for example.

Predictor: Corrector:

𝑛+1 ℎ 𝑛+1 𝑛+1 𝑛+1

𝑦1 = y1𝑛 + hf1(xn, y1𝑛 , y𝑛2 ,…, y𝑛𝑚 )= y1𝑛 +h f1𝑛 y1𝑛+1 = y1𝑛 + 2 [f1(xn+1, 𝑦1 ,𝑦2 ,…, 𝑦𝑚 )+ f1𝑛 ]
𝑛+1
𝑦2 = y𝑛2 + hf2(xn, y1𝑛 , y𝑛2 ,…, y𝑛𝑚 )= y𝑛2 +h f𝑛2 y𝑛+1
ℎ 𝑛+1
= y𝑛2 + 2 [f2(xn+1, 𝑦1 ,𝑦2
𝑛+1 𝑛+1
,…, 𝑦𝑚 )+ f𝑛2 ]
2
… …
𝑛+1 ℎ 𝑛+1 𝑛+1 𝑛+1
𝑦𝑚 = y𝑛𝑚 + hfm(xn, y1𝑛 , y𝑛2 ,…, y𝑛𝑚 )= y𝑛𝑚 +h f𝑛𝑚 y𝑛+1 𝑛 n+1
𝑚 = y𝑚 + 2 [fm(x , 𝑦1 ,𝑦2 ,…, 𝑦𝑚 )+ f𝑛𝑚 ]

 Extend the same idea to RK4 or RKF methods

7.6 Numerical Stability

* Model Equation: Y ' = l Y + g(x), Y(x0)=Y0. (1)

* Exact solution to (1): Y(x)

* Ye(x): solution to (1) with perturbed initial data Ye(0) = Y0 + e:

Ye' = l Ye + g(x), Ye (0)=Y0 + e. (2)

* Take the difference: Ze(x) = Ye(x) - Y(x) (3)

=> Ze' = l Ze, Ze(0)= e (4)
Solution: Ze(x) = e elx (5)

If l< 0, then Ze(x)→ 0 as x gets large.

Is this true for numerical solution as n gets large?
Stability condition for Euler explicit method Y ' = l Y + g(x), Y(x0)=Y0. (1)

* Euler explicit method to equation (1)

yn+1 = yn + h[lyn + g(xn)] (6)
Perturbation: ye,n+1 = ye,n + h[lye,n + g(xn)] (7)
difference (error): Ze,n = ye,n - yn (8)
=> Ze,n+1 = Ze,n + l h Ze,n = Ze,n (1+lh)

i.e. Ze,n+1 = (1+lh)n Ze,0 = e(1+lh)n (10)

For Ze,n+1 to vanish in size, we need

|1+lh|<1 => -2<lh<0 (11)
or h < -2/l (for l<0).
If l>0, the error will inevitably grow.
Unconditional stability of Euler implicit method Y ' = l Y + g(x), Y(x0)=Y0. (1)

* Euler implicit method to equation (1)

yn+1 = yn + h[l yn+1 + g(xn+1)] (12)

Perturbation: ye,n+1 = ye,n + h[l ye,n+1 + g(xn+1 )] (13)

difference (error): Ze,n = ye,n - yn (14)
=> Ze,n+1 = Ze,n + l h Ze,n+1 (15)
i.e. Ze,n+1 = (1-lh)-n ze,0 = e(1-lh)-n (16)
* For l < 0, (1-lh)>1 and (1-lh)-n <1 so that Ze,n+1 always decays so that
the scheme is always stable no matter how large h is.

• If l>0, 1-lh<1, thus (1-lh)-n can increase as long as |1-lh|<1.

However, since y(x) also grows exponentially,
the stability of the numerical scheme is a different issue.
Unconditional instability of mid-point method Y ' = l Y + g(x), Y(x0)=Y0. (1)

* Mid point rule: yn+1 = yn-1 + 2h [lyn +g(xn)] (17)

 Ze,n+1 = Ze,n-1 + 2l h Ze,n (18)
Let d  Ze,n+1 / Ze,n
= growth factor in one step,

then d = d-1 + 2lh.

 d = lh ± 1 + 𝜆2 ℎ2

One of the roots for d is always greater than 1

 Mid point scheme (18) is always unstable for any h
 Stability requirement for RK4 method

* Derivation is complicated; we will study its stability requirement through an experiment/example.

Consider solving the following stiff equation (A»1)
y ' +A y = 𝑒 −𝑡 , y(0)=1.
with A~ - l =100 »1
1 1
Exact solution: y(t) = (1- 𝐴−1) 𝑒 −𝐴𝑡 + 𝐴−1 𝑒 −𝑡

Comments: 10 0
y(t) A=100
• The stability on Dt is controlled by Ah = ADt<2~3
-1
--to catch the rapid transient at small time. 10
exact solution
* The long-time solution is dictated by 𝑒 −𝑡 /(A-1)
* RK4 (explicit method in general) is not suitable 10
-2

for stiff systems of equations (A»1).

-3
10
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
t
RK4 solution for model equation:

1) h= 0.01 (Ah=1.0) 2) h= 0.02785 (Ah=2.785)

t y(RK4) exact t y(RK4) exact t y(RK4)
0.01 0.381213 0.374164 0.02785 0.9992864 0.0709291 80.765 0.274268
0.02 0.149106 0.143869 0.05570 0.9985805 0.0133258 83.55 0.262393
0.03 0.062004 0.059087 0.08355 0.9978821 0.0095242 86.335 0.251033
0.04 0.029281 0.027836 0.11140 0.9971911 0.0090505 89.12 0.240164
0.05 0.016949 0.016278 0.13925 0.9965075 0.0087889 91.905 0.229766
0.06 0.012266 0.011967 0.16710 0.9958302 0.0085467 93.2975 0.224737
0.07 0.010450 0.010321 0.19495 0.9951600 0.0083119 94.69 0.219818
0.08 0.009712 0.009657 0.22280 0.9944964 0.0080836 96.0825 0.215007
0.09 0.009377 0.009354 0.25065 0.9938391 0.0078616 97.475 0.210301
0.1 0.009194 0.009185 0.27850 0.9931883 0.0076456 98.8675 0.205698
0.11 0.009069 0.009065 0.30635 0.9925434 0.0074356 99.95365 0.202178
0.12 0.008967 0.008965 0.33420 0.9919044 0.0072314
0.13 0.008873 0.008872 0.36205 0.9912715 0.0070328 1
0.14 0.008783 0.008782 0.38990 0.9906440 0.0068396 0 20 40 60 80
0.15 0.008694 0.008694 0.41775 0.9900222 0.0066518
0.16 0.008608 0.008608 0.44560 0.9894056 0.0064691
0.17 0.008522 0.008522 0.47345 0.9887943 0.0062914 y(RK4)
0.18 0.008437 0.008437 0.50130 0.9881880 0.0061186
0.19 0.008353 0.008353 0.52915 0.9875864 0.0059506
0.2 0.008270 0.008270 0.55700 0.9869897 0.0057871
0.21 0.008188 0.008188 0.58485 0.9863974 0.0056282
y(RK4) at large x decays to 0
0.22 0.008106 0.008106 0.61270 0.9858100 0.0054736 Computation is stable
0.23 0.008026 0.008026 0.64055 0.9852268 0.0053233 0.1
0.24 0.007946 0.007946 0.66840 0.9846479 0.0051771
Need -l h  2.7852935

3) h= 0.02786 (Ah = 2.786)

x y(RK4) exact 87.759 28.35429 7.78E-41
x y(RK4) exact
0 1 1 1.393 1.0466 0.002508 89.152 29.90514 1.93E-41
0.02786 1.000779 0.070868 2.786 1.101833 0.000623 90.545 31.54081 4.8E-42
0.05572 1.001567 0.013318 4.179 1.161599 0.000155 91.938 33.26594 1.19E-42
0.08358 1.002363 0.009523 5.572 1.225009 3.84E-05 93.331 35.08543 2.96E-43
0.11144 1.003168 0.00905 6.965 1.291981 9.54E-06 94.724 37.00444 7.35E-44
0.1393 1.00398 0.008788 8.358 1.362638 2.37E-06 96.117 39.02841 1.83E-44
0.16716 1.004801 0.008546
9.751 1.437166 5.88E-07 97.51 41.16308 4.53E-45
0.19502 1.005629 0.008311
0.22288 1.006465 0.008083
11.144 1.515772 1.46E-07 98.903 43.41450 1.13E-45
0.25074 1.007308 0.007861 12.537 1.598677 3.63E-08 99.9895 45.25576 3.8E-46
0.2786 1.008159 0.007645 13.930 1.686117 9.01E-09 y(RK4)
0.30646 1.009016 0.007435 15.323 1.77834 2.24E-09 100
0.33432 1.00988 0.007231 16.716 1.875607 5.56E-10 y(RK4) grows exponentially.
0.36218 1.010751 0.007032 18.109 1.978194 1.38E-10 Computation is unstable
0.39004 1.011629 0.006839 19.502 2.086392 3.43E-11
0.4179 1.012513 0.006651
20.895 2.200507 8.51E-12
0.44576 1.013403 0.006468 10
0.47362 1.0143 0.00629 22.288 2.320865 2.11E-12
0.50148 1.015202 0.006118 23.681 2.447805 5.25E-13
0.52934 1.01611 0.005949 25.074 2.581688 1.3E-13
0.5572 1.017024 0.005786 26.467 2.722894 3.24E-14
0.58506 1.017944 0.005627 27.860 2.871824 8.03E-15
0.61292 1.018869 0.005472 1
29.253 3.028899 2E-15 0 20 40 60 80 100
0.64078 1.019799 0.005322
 Supplemental Reading
Chapter 7 Solving ODEs
Example 1 Asymptotic behavior of a nonlinear ODE
Consider
y′ = y2 + x, y(0)=1 (1)
Study the general behavior of the solution (without numerical solution).
Solution:
* First, we note that this is a nonlinear non-homogeneous equation.
* Suppose that we cannot find an exact solution to Eqn. (1), can we deduce
some analytical behavior (to help understanding the behavior of numerical
solution)?
* For simplicity, let’s consider u′ = u2 , u(0)=1 (2)
Let u(x) be the solution to Eqn. (2). It follows that
du/ u2 = dx  -1/u = x + c0.
Since u(0)=1, we see that
u(x)=1/(1-x) (3)
* We note that:
i) u(x) is singular at x=1.
ii) Since y2 + x > y2 , it is easy to see that solution to Eqn. (1), y(x), will
increase more rapidly than u(x) and thus possess a singularity at
x=b<1. That is, near x=b, the solution, y(x) will increase very rapidly
(nearly vertically) while x is almost equal to the constant b.
iii) Thus, we can rewrite (1) as
dy
dx = (4)
y2 + x
 and attempt to solve for x in terms of y. For x near x=b (=a2), y
increases rapidly while x is nearly constant so that x in the
denominator can be approximated by x=b,
dy dy
dx = 
y2 + x y2 + b
The above can be integrated to yield,

tan −1 ( ) + d
1 y
x=
a a
where d is the integration constant. Since y→ as x→b= a2, we have
1 1
a2 = +d d = a2 −
a2 a2
Thus near the singularity, x=b, the solution has the following
asymptotic behavior:

y = a tan[a( x − d )] = a tan[ax − a 3 + ]
2
i.e. y ~ b / tan[ b (b − x)] (asymptotic solution) (5)

Since tan[ b (b − x)] ~ b (b − x) for |x-b|«1, the above may be

further simplified to
y(x) ~ 1/(b - x) (6)
in order to better understand its singular behavior.
* Using automatic 6th order RKF scheme (tolerance =1.0E-9), we can compute
the numerical solution to Eqn. (1) to find that b=0.93056450847…
 1.E+02
RKF6
asymp

1.E+01

1.E+00
0.0 0.2 0.4 0.6 0.8 1.0

1.E+10
y
1.E+09

1.E+08

1.E+07
RKF6 asymp
1.E+06

1.E+05

1.E+04

1.E+03
x=b
1.E+02

1.E+01 x
0.85 0.86 0.87 0.88 0.89 0.90 0.91 0.92 0.93 0.94

* Clearly, the asymptotic solution, y ~ b / tan[ b (b − x)] , gives a very

good analytical description for the solution for x>0.5.
* The relative difference between RKF6 and the asymptotic solution is about
1.49E-5 at x=0.930564 (or b-x=4.6E-7). This is extremely accurate.
* The key in this example is that through a simple analysis we can analytically
determine the structure of the singular solution near the singularity quite
accurately. This simple asymptotic solution can in turn be used to assess the
numerical solutions when less accurate methods are used or when
insufficient resolution is used.
Example 2 Predictor-corrector method
Solve
y′ = f(x, y) = y2 + x; y(0)=1 (1)
using modified Euler (predictor-corrector) method.
Solution:

Predictor: y- n+1 = yn + hfn + O(h2) (2)

(as a preliminary, predicted value of y at xn+1)
h
Corrector: yn+1 = yn + 2 [f(xn, yn) + f(xn+1, y- n+1)]

= 0.5[yn+ y- n+1 + h f(xn+1, y- n+1)] (3)

* Screen shot of excel implementation of predictor-corrector method:

Excel formula:
x corrector predictor
0 1 1
=h+B2 =0.5*(C2+D3+h*(D3^2+B3)) =C2+h*(C2^2+B2)
=h+B3 =0.5*(C3+D4+h*(D4^2+B4)) =C3+h*(C3^2+B3)
=h+B4 =0.5*(C4+D5+h*(D5^2+B5)) =C4+h*(C4^2+B4)
=h+B5 =0.5*(C5+D6+h*(D6^2+B6)) =C5+h*(C5^2+B5)

Result: (based on h=0.001)

x corrector predictor Asymptotic
0 1 1
0.001 1.001001501 1.001
0.002 1.002006009 1.002004505
0.003 1.003013534 1.003012025
0.004 1.004024082 1.00402257
0.005 1.005037663 1.005036147
0.006 1.006054285 1.006052764
0.007 1.007073955 1.00707243
0.008 1.008096682 1.008095153
0.009 1.009122474 1.009120941
0.01 1.010151339 1.010149802
0.011 1.011183286 1.011181745
0.012 1.012218323 1.012216778
0.013 1.013256459 1.013254909
0.014 1.014297702 1.014296148
0.015 1.01534206 1.015340502
0.016 1.016389542 1.016387979
0.017 1.017440156 1.01743859
0.018 1.018493912 1.018492341
0.019 1.019550817 1.019549242
0.02 1.020610881 1.020609301
0.04 1.042489089 1.042487418
0.05 1.053925985 1.053924266
0.06 1.065707589 1.065705819
0.07 1.077844093 1.07784227
0.08 1.090346164 1.090344286
0.09 1.103224978 1.103223041
0.1 1.116492245 1.116490246
0.11 1.130160245 1.130158182
0.12 1.144241863 1.144239733
0.13 1.158750628 1.158748428
0.14 1.173700752 1.173698477
0.15 1.189107175 1.189104822
0.2 1.273563962 1.27356115
0.25 1.372182927 1.372179512
0.3 1.488021425 1.4880172
0.35 1.62519199 1.62518665
0.4 1.789360161 1.789353247
0.45 1.988552851 1.988543639
0.5 2.234529691 2.234516989
0.55 2.545235124 2.545216853
0.6 2.949475513 2.949447792
 0.65 3.496596884 3.49655176
0.7 4.278753976 4.278673021
0.75 5.490175622 5.490007843
0.79 7.075318839 7.074964877 7.070516371
0.8 7.622590064 7.622149047 7.618506783
0.81 8.260283544 8.259724393 8.256883456
0.811 8.32989902 8.329325828 8.326565524
0.812 8.400684946 8.400097238 8.397417759
0.813 8.472671172 8.472068454 8.469470024
0.814 8.545888573 8.545270329 8.542753204
0.815 8.620369092 8.619734784 8.617299252
0.816 8.696145789 8.695494856 8.69314124
0.817 8.773252887 8.772584741 8.770313401
0.818 8.851725823 8.851039853 8.848851188
0.819 8.931601308 8.930896873 8.928791323
0.82 9.012917377 9.012193809 9.010171857
0.821 9.095713457 9.094970057 9.093032231
0.822 9.180030425 9.17926646 9.177413337
0.823 9.265910679 9.265125384 9.263357589
0.824 9.353398206 9.352590779 9.350908995
0.825 9.442538663 9.441708264 9.440113227
0.826 9.533379449 9.532525199 9.531017706
0.827 9.625969797 9.625090773 9.623671684
0.828 9.720360858 9.719456092 9.718126335
0.829 9.816605796 9.815674273 9.814434847
0.83 9.914759891 9.913800545 9.912652525
0.831 10.01488064 10.01389236 10.01283689
0.832 10.11702788 10.11600948 10.11504781
0.833 10.2212639 10.22021414 10.2193476
0.834 10.32765355 10.32657114 10.32580115
0.835 10.43626442 10.43514798 10.43447607
0.836 10.54716695 10.54601504 10.54544284
0.837 10.66043457 10.65924568 10.65877494
0.838 10.77614392 10.77491644 10.77454904
0.839 10.89437495 10.89310719 10.89284513
0.84 11.01521116 11.01390135 11.01374677
0.841 11.13873978 11.13738604 11.13734123
0.842 11.26505198 11.26365231 11.26371972
0.843 11.39424307 11.39279537 11.39297764
0.844 11.52641279 11.52491484 11.52521476
0.845 11.66166552 11.66011498 11.66053554
0.846 11.80011061 11.79850497 11.79904938
0.847 11.94186259 11.94019922 11.94087091
0.848 12.08704158 12.08531767 12.08612033
0.849 12.23577358 12.23398616 12.2349237
0.85 12.38819082 12.38633673 12.38741337
0.851 12.54443222 12.5425081 12.54372833
0.852 12.70464371 12.70264599 12.70401465
0.853 12.8689788 12.86690368 12.86842594
0.854 13.03759899 13.03544242 13.03712382
0.855 13.21067432 13.20843198 13.21027847
0.856 13.38838396 13.38605124 13.38806921
0.857 13.57091682 13.56848879 13.57068511
0.858 13.75847221 13.7559436 13.75832563
0.859 13.95126056 13.94862576 13.95120143
0.86 14.14950427 14.14675724 14.14953507
0.861 14.35343847 14.35057274 14.35356193
0.862 14.56331204 14.56032067 14.5635311
0.863 14.77938856 14.7762641 14.77970645
0.864 15.00194745 14.99868189 15.00236767
0.865 15.23128518 15.22786988 15.23181154
0.866 15.46771655 15.46414223 15.46835322
0.867 15.71157618 15.7078328 15.7123277
0.868 15.96322007 15.95929681 15.96409137
0.869 16.22302733 16.21891247 16.22402382
0.87 16.49140209 16.48708295 16.49252966
0.871 16.76877561 16.76423843 16.7700407
0.872 17.05560857 17.05083844 17.05701825
0.873 17.35239366 17.34737435 17.35395564
0.874 17.65965837 17.65437223 17.66138113
0.875 17.97796815 17.97239591 17.97986097
0.876 18.30792984 18.30205049 18.31000294
0.877 18.6501956 18.64398614 18.65246021
0.878 19.00546714 18.9989024 19.00793565
0.879 19.37450059 19.36755293 19.37718665
0.88 19.7581118 19.75075086 19.76103051
0.881 20.15718245 20.14937479 20.1603505
0.882 20.57266672 20.56437545 20.57610263
0.883 21.00559898 20.99678333 21.00932332
0.884 21.45710236 21.44771717 21.46113802
0.885 21.92839847 21.9183936 21.93277095
0.886 22.42081845 22.41013813 22.42555626
0.887 22.9358155 22.92439755 22.94095056
0.888 23.47497923 23.46275413 23.48054737
0.889 24.04005202 24.02694188 24.04609355
0.89 24.63294783 24.61886512 24.63950821
0.891 25.25577384 25.24061995 25.26290452
0.892 25.91085545 25.89451895 25.91861477
0.893 26.6007653 26.58311988 26.60921967
0.894 27.32835697 27.30925901 27.33758219
0.895 28.09680443 28.07609007 28.10688733
0.896 28.90964828 28.88712985 28.92068869
0.897 29.7708502 29.74631204 29.78296333
0.898 30.6848575 30.65805072 30.69817685
0.899 31.6566798 31.62731598 31.67136075
0.9 32.69198078 32.65972417 32.708205
0.901 33.79718846 33.76164638 33.8151694
0.902 34.9796286 34.94033941 34.99961836
0.903 36.24768692 36.20410501 36.26998495
0.904 37.61100795 37.56248472 37.63597207
0.905 39.08074025 39.02649987 39.10880083
0.906 40.66984127 40.60894951 40.70151951
0.907 42.39345941 42.32478326 42.42939131
0.908 44.26941712 44.19157181 44.31038513
0.909 46.31882763 46.23010641 46.36580291
0.91 48.5668904 48.46517043 48.62109006
0.911 51.04392873 50.92654324 51.1068943
0.912 53.7867601 53.65032239 53.86046691
0.913 56.84053056 56.68068766 56.92754244
0.914 60.261207 60.07228947 60.36489945
0.915 64.11901988 63.89353407 64.24390828
0.916 68.50330694 68.23118359 68.65554046
0.917 73.52946996 73.19692601 73.71759384
0.918 79.34920155 78.93696992 79.58536806
0.919 86.16592217 85.64641534 86.46787871
0.92 94.25880051 93.59140732 94.65328016
0.921 104.0214697 103.144442 104.5502366
0.922 116.0270652 114.8428569 116.7582764
0.923 131.1429922 129.490267 132.1939495
0.924 150.7458917 148.3423996 152.3322938
0.925 177.1550962 173.4711395 179.7086662
0.926 214.592441 208.5399494 219.0802238
0.927 271.5857858 260.6432828 280.5424835
0.928 368.097908 345.3455519 389.937476
0.929 562.650786 503.5949058 639.1779014
0.93 1107.460275 879.227622 1771.45243
0.931
Example 3 Classical Runge Kutta (RK4) method
Solve
y′ = f(x, y) = y2 + x; y(0)=1 (1)
using the classical 4th order Runge-Kutta method.
Solution:
* Screen shot of excel implementation of predictor-corrector method:

* Excel formula:
x y s1 s2 s3 s4
=B2+h/2+ =B2+h/2+ =B2+h+(C2
0 1 =(B2+C2^2) (C2+h*D2/2)^2 (C2+h*E2/2)^2 +h*F2)^2
=C2+h/6*(D2+ =B3+h/2+ =B3+h/2+ =B3+h+(C3
=h+B2 2*E2+2*F2+G2) =(B3+C3^2) (C3+h*D3/2)^2 (C3+h*E3/2)^2 +h*F3)^2
=C3+h/6*(D3+ =B4+h/2+ =B4+h/2+ =B4+h+(C4
=h+B3 2*E3+2*F3+G3) =(B4+C4^2) (C4+h*D4/2)^2 (C4+h*E4/2)^2 +h*F4)^2
=C4+h/6*(D4+ =B5+h/2+ =B5+h/2+ =B5+h+(C5
=h+B4 2*E4+2*F4+G4) =(B5+C5^2) (C5+h*D5/2)^2 (C5+h*E5/2)^2 +h*F5)^2

* Results (based on h=0.01):

x y s1 s2 s3 s4 Asymptotic
0 1.00000000000 1 1.015025 1.01517601 1.03040658
0.01 1.01015134765 1.03040575 1.04584095 1.04599767 1.06164748
0.02 1.02061089840 1.06164661 1.07751006 1.07767282 1.09376044
0.03 1.03138718640 1.09375953 1.11007033 1.11023946 1.12678453
0.04 1.04248912579 1.12678358 1.14356191 1.14373778 1.16076107
0.05 1.05392603253 1.16076008 1.17802732 1.17821031 1.19573383
0.06 1.06570764782 1.19573279 1.21351155 1.21370209 1.23174913
0.07 1.07784416316 1.23174804 1.25006229 1.25026083 1.26885608
0.08 1.09034624711 1.26885494 1.2877301 1.28793711 1.30710676
0.09 1.10322507399 1.30710556 1.32656859 1.3267846 1.34655644
0.1 1.11649235463 1.34655518 1.36663469 1.36686024 1.38726379
0.11 1.13016036936 1.38726246 1.40798886 1.40822455 1.42929116
0.12 1.14424200345 1.42928976 1.45069537 1.45094184 1.47270486
0.13 1.15875078518 1.47270338 1.49482257 1.49508051 1.51757542
0.14 1.17370092678 1.51757387 1.54044322 1.54071339 1.56397798
0.15 1.18910736855 1.56397633 1.58763484 1.58791803 1.61199258
0.16 1.20498582632 1.61199084 1.63648007 1.63677715 1.66170461
0.17 1.22135284278 1.66170277 1.68706705 1.68737896 1.71320519
0.18 1.23822584276 1.71320324 1.73948994 1.7398177 1.76659168
0.19 1.25562319308 1.7665896 1.79384933 1.79419404 1.82196815
0.2 1.27356426724 1.82196594 1.85025284 1.85061569 1.87944598
0.21 1.29206951553 1.87944363 1.90881566 1.90919795 1.93914447
0.22 1.31116054105 1.93914196 1.96966124 1.97006437 2.00119149
0.23 1.33086018218 2.00118882 2.03292197 2.03334749 2.0657243
0.24 1.35119260226 2.06572145 2.09874 2.09918959 2.1328903
0.25 1.37218338713 2.13288725 2.1672681 2.16774357 2.20284799
0.26 1.39385965144 2.20284473 2.2386706 2.23917395 2.275768
0.27 1.41625015449 2.2757645 2.3131245 2.31365789 2.35183417
0.28 1.43938542690 2.35183041 2.39082059 2.39138643 2.43124482
0.29 1.46329790900 2.43124077 2.47196484 2.47256575 2.51421413
0.3 1.48802210246 2.51420978 2.55677981 2.55741865 2.60097373
0.31 1.51359473650 2.60096903 2.64550628 2.64618624 2.69177433
0.32 1.54005495049 2.69176925 2.73840512 2.73912967 2.78688774
0.33 1.56744449474 2.78688224 2.83575924 2.83653223 2.88660897
0.34 1.59580795170 2.88660302 2.93787597 2.93870165 2.99125868
0.35 1.62519297996 2.99125222 3.04508953 3.04597262 3.10118588
0.36 1.65565058397 3.10117886 3.15776397 3.15870968 3.21677099
0.37 1.68723541257 3.21676334 3.2762964 3.27731052 3.3384293
0.38 1.72000609003 3.33842095 3.40112062 3.40220962 3.46661488
0.39 1.75402558389 3.46660575 3.53271133 3.53388241 3.60182498
0.4 1.78936161426 3.60181499 3.67158881 3.67285 3.74460511
0.41 1.82608711045 3.74459413 3.81832424 3.81968455 3.89555467
0.42 1.86428072107 3.89554261 3.97354584 3.97501538 4.05533357
0.43 1.90402738544 4.05532028 4.13794583 4.13953597 4.22466966
0.44 1.94541897470 4.22465499 4.31228842 4.31401196 4.40436727
0.45 1.98855501306 4.40435104 4.49741894 4.49929036 4.59531713
0.46 2.03354349101 4.59529913 4.69427446 4.69631015 4.79850768
0.47 2.08050178437 4.79848767 4.90389593 4.90611452 5.01503827
0.48 2.12955769580 5.01501598 5.1274424 5.12986509 5.2461344
0.49 2.18085063807 5.24610951 5.36620736 5.36885838 5.49316553
0.5 2.23453298226 5.49313765 5.62163799 5.62454507 5.75766583 2.187415
0.51 2.29077159826 5.75763452 5.89535753 5.89855257 6.04135854 2.245848
0.52 2.34974962036 6.04132328 6.18919169 6.19271144 6.34618467 2.306967
0.53 2.41166847737 6.34614484 6.50519966 6.50908663 6.67433685 2.370972
0.54 2.47675023450 6.67429172 6.84571091 6.85001448 7.02829949 2.438086
0.55 2.54524030451 7.02824821 7.21336892 7.2181466 7.41089672 2.508554
0.56 2.61741059779 7.41083824 7.61118332 7.61650241 7.82534963 2.582649
0.57 2.69356319665 7.82528269 8.04259251 8.04853209 8.27534518 2.660673
0.58 2.77403465842 8.27526829 8.5115391 8.51819249 8.76511947 2.742963
0.59 2.85920107664 8.7650308 9.0225613 9.03003913 9.29955891 2.829894
0.6 2.94948406090 9.29945623 9.5809042 9.58933831 9.88432378 2.921889
0.61 3.04535783595 9.88420435 10.1926562 10.2022045 10.5260001 3.019422
0.62 3.14735771217 10.5258606 10.8649169 10.8757695 11.2322875 3.123028
0.63 3.25609024712 11.2321237 11.6060058 11.6183932 12.0122331 3.233315
0.64 3.37224550518 12.0120397 12.4257224 12.4399256 12.8765246 3.350971
0.65 3.49661193914 12.8762951 13.3356741 13.3520378 13.8378606 3.476787
0.66 3.63009457173 13.8375866 14.349691 14.3686418 14.911423 3.611667
0.67 3.77373736381 14.9110937 15.4843577 15.5064268 16.1154823 3.756655
0.68 3.92875093887 16.1150839 16.7596978 16.7855529 17.4721846 3.912962
0.69 4.09654722224 17.4716991 18.2000671 18.2305545 19.008581 4.082
0.7 4.27878309450 19.0079848 19.8353278 19.8715314 20.757992 4.265426
0.71 4.47741591986 20.7572533 21.7024135 21.7457355 22.7618351 4.465197
0.72 4.69477489710 22.7609113 23.8474364 23.8997123 25.0721063 4.683643
0.73 4.93365375543 25.0709394 26.3285665 26.3922295 27.7547968 4.923558
0.74 5.19743263588 27.753306 29.2200215 29.2983422 30.8946685 5.188322
0.75 5.49024047244 30.8927404 32.6176852 32.7151276 34.6020468 5.482064
0.76 5.81717449396 34.5995191 36.6471617 36.7699238 39.0226631 5.809882
0.77 6.18460174910 39.0192988 41.4755497 41.6324019 44.352221 6.178141
0.78 6.60058078731 44.3476667 47.3290382 47.5326591 50.8584652 6.594902
0.79 7.07546333146 50.8521814 54.5198574 54.7890242 58.9155197 7.070516
0.8 7.62277243875 58.9066597 63.4887301 63.8520315 69.058958 7.618507
0.81 8.26051767372 69.0461522 74.8739061 75.376277 82.0772519 8.256883
0.82 9.01322395785 82.0582061 89.6276347 90.3423734 99.1699014 9.010172
0.83 9.91517083084 99.1406126 109.221295 110.273323 122.23421 9.912653
0.84 11.01577759539 122.187356 136.025487 137.639194 154.415866 11.01375
0.85 12.38899857088 154.337286 174.05863 176.663817 201.242052 12.38741
0.86 14.15070562114 201.10247 230.575943 235.064724 273.164646 14.14954
0.87 16.49328637188 272.898495 319.775265 328.201351 391.942485 16.49253
0.88 19.76127672834 391.388058 472.56595 490.361088 609.246681 19.76103
0.89 24.63875808576 607.9584 766.997135 811.649084 1073.80633 24.63951
0.9 32.70385336766 1070.44203 1449.16897 1596.88339 2369.94048 32.70821
0.91 48.59133206694 2362.02755 3649.25256 4468.17909 8700.79546 48.62109
0.92 94.08747589741 8853.37312 19142.8488 36025.6179 206429.087 94.65328
0.93 636.78646470047 405497.932 7098368.19 1305277721 1.70392E+14 1771.452