Numerical solution of ODEs Numerical solution of ODEs

Ordinary Differential Equations 1 Numerical solution of ODEs

Taylors theorem
Taylors theorem for a function of two variables
Barry Croke Example: Taylors series
Example: Initial Value Problem
Eulers method
Example: Eulers method
Semester 1, 2017 Runge-Kutta methods
Second order Runge-Kutta
Fourth order Runge-Kutta
Examples: Runge-Kutta method
Based on notes written by Lilia Ferrario, Linda Stals and Dayal System of equations
Wickramasinge Application: galactic dynamics

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Numerical solution of ODEs Motivation

As you already know, an Initial Value Problem consists of :
 0
x = f (t, x)
,
x(t0 ) = x0

where f is a prescribed function of 2 variables and (t0 , x0 ) is a point

through which the solution should pass.
A solution of this IVP is a function x(t) such that

dx(t)
= f (t, x(t))
dt
for all t in some neighbourhood of t0 and x(t0 ) = x0 .

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

In this section we will see some methods for calculating numerical Taylors theorem
solutions of differential equations. These methods are very useful since
most differential equations cannot be solved analytically, or even if
If the function f (x) has the (n + 1)st derivative f (n+1) (t) exists for all t
analytic solutions exist these may be too complicated to be used.
in an interval containing c and x, then the Taylor series expansion around
The methods that we are going to see are step-by-step methods. We
the point c is:
start with x(t0 ) = x0 and then we proceed by calculating approximate
values of the solution x(t) at: n
X 1 (k)
f (x) = f (c)(x c)k + En (x)
t1 = t0 + h, t2 = t1 + h, t3 = t2 + h, t4 = t3 + h k!
k=0

where h has a certain assigned value and is called the stepsize. These where En (x) is called the Lagrange remainder:
approximate values can be obtained with a Taylor series expansion.
1
En (x) = f (n+1) ()(x c)n+1 .
(n + 1)!

with is some number between c and x.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

However, this formula is not very useful as it is, since we want to Two variable case
advance the solution from x to a neighbouring point x + h. Thus, we
need an expression that gives us the value of the function at x + h in
For a function of two variables f (x, y ), the symbolic expression is:
terms of its value at x. To obtain such an expression, we expand the
function around x. n  (i)
X 1
This can be accomplished by replacing x with x + h and c with x: f (x + h, y + k) = h +k f (x, y ) + En (h, k)
i! x y
i=0
n
X 1 (k)
f (x + h) = f (x)hk + En (h) where
k!
k=0
 (n+1)
1
and: En (h, k) = h +k f (a+h, b+k), 0 1.
1 (n + 1)! x y
En (h) = f (n+1) ()hn+1
(n + 1)!
where is in between x and x + h.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Here:  (0) Example (Taylor series)

h +k f (x, y ) = f (x, y ) Find the Taylors series expansion of
x y


(1) 
f f
 f (x, y ) = cos(xy ).
h +k f (x, y ) = h +k (x, y )
x y x y Solution
(2) 2
The derivatives are:
2f 2
  
2 f 2 f
h +k f (x, y ) = h + 2hk +k (x, y ) f f
x y x 2 xy y 2 = y sin(xy ), = x sin(xy ),
x y

2f
= y 2 cos(xy ),
x 2
2f
= xy cos(xy ) sin(xy ),
xy
2f
= x 2 cos(xy ).
y 2
Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

If n = 1, then
Example (Initial Value Problem)
cos[(x + h)(y + k)] = cos(xy ) hy sin(xy ) kx sin(xy ) + E1 (h, k). Solve the following IVP:
 0
x = f (t, x) = cos t sin x + t 2
.
x(1) = 3

Solution
To solve this IVP numerically, we will use the Taylor expansion:

h2 00 h3 h4
x(t + h) = x(t) + hx 0 (t) + x (t) + x 000 (t) + x iv (t) +
2! 3! 4!

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Obviously, we need to calculate the first few derivatives of x(t). We
already know x 0 (t), since this is given by the IVP itself. lets calculate
x 00 (t), x 000 (t) and x iv (t):
00 0 % taylor
x = sin t x cos x + 2t
000
x = cos t x 00 cos x + (x 0 )2 sin x + 2
x iv = sin t x 000 cos x + 3x 0 x 00 sin x + (x 0 )3 cos x
If we stop here, the terms not included start with h5 and they form the
so-called truncation error in our procedure.
The algorithm is the following:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Find the Taylor expansion of
% c o s ( t ) s i n ( x ) +t 2 a b o u t t h e p o i n t
% ( t , x ) . h i s the step s i z e .
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
f u n c t i o n y = taylor ( t , x , h )
xp = c o s ( t ) s i n ( x )+t 2 ; % f i r s t d e r i v
xpp = s i n ( t )xp c o s ( x ) +2t ; % s e c o n d d e r i v
xppp = c o s ( t )xpp c o s ( x )+xp 2+2; % t h i r d d e r i v
xpppp = s i n ( t ) +(xp3xppp ) c o s ( x ) +3xp xpp s i n ( x ) ; -
% fourth deriv
y = x+h ( xp+h ( xpp+h ( xppp+h xpppp / 4 ) / 3 ) / 2 ) ; % new -
approx

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% When this algorithm was programmed and run, the solution at t = 1 was
% taylor ode x200 = 6.4220.
% Use t h e T a y l o r s e r i e s t o s o l v e t h e ODE
% x ' = f ( t , x ) = c o s ts i n x+t 2 ; x ( 1) = 3
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% set the paramters
m =200; h =0.01 d0 ; t=1.0d0 ; x =3.0 d0 ; k =0;
% p r i n t the in te rm ed ia te values
[k t x]
% apply the algorithm
f o r k =1: m
% f i n d the Taylor expansion of f ( t , x )
x = taylor ( t , x , h )
% take a step forward i n time
t=t+h ;
% print intermediate results
[k t x]
end

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Eulers method The truncation error is proportional to hn , where n is the lowest power of
the terms in the Taylor series that are not included in the calculations.
For example, for the Euler-Cauchy method, the truncation error is
Note that if we use the approximation:
proportional to h2 (since we stop the expansion with the first derivative).
x(t + h) = x(t) + hx 0 (t) For a fixed interval t = t2 t1 in which we want to solve a given DE,
the number of steps is proportional to 1/h. Therefore, the total error is
then the numerical method is called the Euler method or the proportional to h2 h1 = h. For this reason, Eulers method is called a
Euler-Cauchy method. first-order method.
Geometrically, this method approximates the curve f (x, t) with a polygon In the example shown above, since the terms that were not included in
whose first straight line segment is tangent to the exact solution curve at our calculations started with h5 , the resulting numerical method was of
t0 . The good thing about Eulers method is that it doesnt require any order 4.
differentiation of f .

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Eulers method: : step-by-step example The Eulers formula to advance the solution is:

xn+1 = xn + hf (tn , xn )
Example (Eulers method) = xn + 0.05(tn2 xn )
Apply Eulers method to the following IVP: = 0.95xn + 0.05tn2 .

x0 + x = t2 x(0) = 1. We start with:

t0 = 0, x0 = 1
Integrate over the interval [0, 0.4] with steps of stepsize h = 0.05. Solve
the problem analytically and present a table with the numerical and so our first new values are:
analytical results and the error.
t1 = t0 + 0.05 = 0.05,
Solution x1 = 0.95x0 + 0.05t02 = 0.95.
Write the problem as:
The second values are:
x 0 = f (t, x) = t 2 x x(0) = 1.
t2 = t1 + 0.05 = 0.1,
x2 = 0.95x1 + 0.05t12 = 0.95(0.95) + 0.05(0.05)2 = 0.902625.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

The third values are: The sixth values are:

t3 = t2 + 0.05 = 0.15, t6 = t5 + 0.05 = 0.3,

x3 = 0.95x2 + 0.05t22 = 0.95(0.902625) + 0.05(0.1) = 0.857994. 2
x6 = 0.95x5 + 0.05t52 = 0.95(0.777408) + 0.05(0.25)2 = 0.741663.

The fourth values are: The seventh values are:

t4 = t3 + 0.05 = 0.2, t7 = t6 + 0.05 = 0.35,

x4 = 0.95x3 + 0.05t32 = 0.95(0.857994) + 0.05(0.15)2 = 0.816219. x7 = 0.95x6 + 0.05t62 = 0.95(0.741663) + 0.05(0.3)2 = 0.709080.

The fifth values are: And finally, the eighth values are:

t5 = t4 + 0.05 = 0.25, t8 = t7 + 0.05 = 0.4,

x5 = 0.95x4 + 0.05t42 = 0.95(0.816219) + 0.05(0.2) = 0.777408. 2
x8 = 0.95x7 + 0.05t72 = 0.95(0.709080) + 0.05(0.35)2 = 0.679751.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Lets now find the analytical solution of: If we set x(0) = 1, then const = 1 and the analytic solution to the IVP
is
x0 + x = t2 x(0) = 1. x(t) = t 2 2t + 2 e t .
This is a first order non-homogeneous DE, where p(t) = 1 and r (t) = t 2 . So the numerical solution at t = 0.4 is 0.679751, while the analytical
The solution is: solution is 0.689680, giving an error of 0.009929.
R
Z R

p(t) dt p(t) dt
x(t) = e r (t)e dt + const
R
Z R

= e dt t 2 e dt dt + const
Z 
t 2 t
= e t e + const

= e t t 2 e t 2te t + 2e t + const
 

= t 2 2t + 2 + const e t .

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Eulers method - coding example Figure 1 shows the solution when h = 10/25 .

10

Example (Eulers method) 9

Apply Eulers method to the following IVP: 8

x 0 = 2x x(0) = 10. 6

The Matlab code is given in the notes. 5

x
4

0
0 1 2 3 4 5 6 7 8 9 10
t

Figure : Numerical approximation to x 0 + x = t 2 (red crosses) found using

Eulers method compared to the analytical solution (green line).

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Figure 2 shows (the absolute value of) the difference between the According to the theory, if we halve h the error should be divided by 2.
analytical and numerical solutions. Figures 3, 4 and 5 shows how the error changes as h is decreased.
Error (c = 5) Error (c = 6)
1.8 0.7

1.6
0.6

1.4
0.5
1.2

1 0.4
x

x
0.8 0.3

0.6
0.2
0.4

0.1
0.2

0 0
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
t t

Figure : Difference between the numerical and analytical solutions when Figure : Difference between the numerical and analytical solutions when
h = 10/25 . h = 10/26 .
Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Error (c = 7) Error (c = 8)
0.35 0.16

0.3 0.14

0.12
0.25

0.1
0.2
0.08
x

x
0.15
0.06

0.1
0.04

0.05 0.02

0 0
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
t t

Figure : Difference between the numerical and analytical solutions when Figure : Difference between the numerical and analytical solutions when
h = 10/27 . h = 10/28 .

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Stability Lets make a very minor modification to the previous example. Instead of
solving
x 0 = 2x x(0) = 10,
The ODE solvers in Matlab use numerical techniques to find
approximate solutions. In general, you will not need to write your own we are going to solve
numerical solvers like we just did for Eulers method. The ODE solvers
available in packages like Matlab are well written and robust. However, x 0 = 20x x(0) = 10,
it is still important to understand how the solvers work and what their
limitations are. with h = 10/25 .
As shown in Figure 6, the error in the numerical approximation is of order
1023 . This is not an error in the code, it is a consequence of the stability
of Eulers method.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Stability theory is concerned with the consequences of round off errors

23

12
x 10
and it states that h must be smaller than a certain value before Eulers
method will work properly, and that value depends on the ODE you are
10
solving. Sometimes the value of h must be so small that the method is
8 no longer practical. Many other ODE solvers also have this stability
issue, although different solvers place different conditions on h. That is
6
why Matlab offers you a number of different types of solvers.
4
x

4
0 1 2 3 4 5 6 7 8 9 10
t

Figure : Difference between the numerical and analytical solutions when

h = 10/25 if f (x) = 20x.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Runge-Kutta methods Consider again the Taylor expansion:

h2 00 h3 h4
Consider the following IVP: x(t + h) = x(t) + hx 0 (t) + x (t) + x 000 (t) + x iv (t) +
2! 3! 4!
x0

= f (t, x) Let the partial derivatives be denoted with the subscripts x and t, e.g.
. f f
x(t0 ) = x0 x = fx , t = ft . Thus:

The snag with the previous method is that we must calculate all those x0 = f
derivatives (x 0 , x 00 , x 000 , x iv , ), then all these functions need to be x 00
= ft + fx x 0 = ft + fx f
programmed.
x 000 = ftt + ftx f + (ft + fx f )fx + f (fxt + fxx f )
This can all be avoided by means of clever combinations of values of
f (t, x).

(Note: I have used the chain rule for differentiation). So,

1 2
x(t + h) = x(t) + hf + h (ft + ffx ) + O(h3 )
2!
h h
= x + f + [f + hft + hffx ] + O(h3 ).
2 2
Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Recall: Hence, the formula to advance the solution is:

h h h h
x(t + h) = x + f + [f + hft + hffx ] + O(h3 ). x(t + h) = x(t) + f (t, x) + f (t + h, x + hf ).
2 2 2 2
Now we can get rid of the partial derivatives with the help of the first few This can be re-written in compact form as follows.
terms in the Taylor series in two variables. Since
1
x x x(t + h) = x(t) + (F1 + F2 )
f =x 0
= x fh 2
t h
where 
then F1 = hf (t, x)
F2 = hf (t + h, x + F1 )
f (t + t, x + x) f (t + h, x + hf )
= f (t, x) + ft t + fx x + O(h2 ) This is known as a second order Runge-Kutta method. Also known as
Heuns method.
= f + hft + hffx + O(h2 ).
Now we insert this expansion into the expression for x(t + h):
1 1
x(t + h) = x + hf + hf (t + h, x + hf ) + O(h3 ).
2 2
Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Fourth order method Runge-Kutta method: step-by-step example

This formula is very tedious to derive, so I have skipped the proof. The
Example (Runge-Kutta)
classical fourth order Runge-Kutta method is given by:
Apply Runge-Kuttas method to the following IVP:
1
x(t + h) = x(t) + (F1 + 2F2 + 2F3 + F4 )
6 x0 + x = t2 x(0) = 1
where Integrate over the interval [0, 0.4] with steps of stepsize h = 0.05.
F1 = hf (t, x)

Present a table with the numerical and analytical results and the error.
= hf t + h2 , x + 12 F1



F2

.
F = hf t + h2 , x + 12 F2 Solution
3


F4 = hf (t + h, x + F3 ) Write the problem as:

x 0 = f (t, x) = t 2 x, x(0) = 1.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

So, the first values are: Now calculate the advanced solution:

F1 = hf (t, x) = 0.05(0
1) = 0.05 1
x(t + h) = x(t) + (F1 + 2F2 + 2F3 + F4 )
= hf t + h2 , x + 12 F1 6

F2




= 0.05fh 0 + 0.05 1

1
2 , 1 + 2 (0.05)i

= 1+ (0.050000 + 2(0.048719)





= 0.05 0.05

2
1 12 (0.05) = 0.048719 6


2 +2(0.048751) + (0.047437))
= hf t + h2 , x + 12 F2


F3

= 0.951271.
= 0.05 h0 + 0.05 1


2 , 1 + 2 (0.048719) i

2
= 0.05 0.05 1 12 (0.048719) = 0.048751




2
F4 = hf (t + h, x + F3 )




= 0.05fh (0 + 0.05, 1 0.048751)



i
2

= 0.05 (0.05) 1 + 0.048751 = 0.047437

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Now we use this value x(t + h) = 0.951271 and t + h = 0 + 0.05 to Now calculate the advanced solution:
calculate the new functions F1 , F2 , F3 and F4 . Here we go:
1
x(t + h) = x(t) + (F1 + 2F2 + 2F3 + F4 )
6
1
h i
2

F1 = hf (t, x) = 0.05 (0.05) 0.951271 = 0.047439 = 0.951271 + (0.047439 + 2(0.046096)
6
= hf t + h2 , x + 21 F1


F2



+2(0.046130) + (0.044757))
= 0.05fh 0.05 + 0.05 1


2 , 0.951271 + 2 (0.047439)





2 i = 0.905163.
= 0.05 0.05 + 0.05 0.951271 1
(0.047439)




2 2
= 0.046096




= hf t + h2 , x + 21 F2


F3

.
= 0.05 h0.05 + 0.05 1



2 , 0.951271 + 2 (0.046096) i
2
= 0.05 0.05 + 0.05 0.951271 12 (0.046096)





2
= 0.046130




F4 = hf (t + h, x + F3 )




= 0.05fh (0.05 + 0.05, 0.951271 0.046130)



i
2

= 0.05 (0.1) 0.951271 + 0.046130 = 0.044757

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Now we use this value x(t + h) = 0.905163 and x + h = 0.05 + 0.05 to Now calculate the advanced solution:
calculate the new functions F1 , F2 , F3 and F4 . Here we go again:
1
x(t + h) = x(t) + (F1 + 2F2 + 2F3 + F4 )
6
h i
2
F 1 = hf (t, x) = 0.05 (0.05) 0.905163 = 0.044758

1
h 1 = 0.905163 + (0.044758 + 2(0.043358)


F2 = hf t + 2 , x + 2 F1

= 0.05fh 0.1 + 0.05 , 0.905163 + 12 (0.044758)

+2(0.043393) + (0.041963))

2 i
0.05 2

1
= 0.05 0.1 + 0.905163 (0.047439) = 0.043358

= 0.861792.

2 2
h 1


F3 = hf t + 2 , x + 2 F2

. Now we use this value x(t + h) = 0.861792 and t + h = 0.1 + 0.05 to

= 0.05 h0.1 + 0.05 , 0.905163 + 12 (0.043358)


2
calculate the new functions F1 , F2 , F3 and F4 . This goes on and on until

2 i
= 0.05 0.1 + 0.05 0.905163 12 (0.046096) = 0.043393

2 t = 0.4.
F4 = hf (t + h, x + F3 )

= 0.05fh (0.1 + 0.05, 0.905163 0.043393)

i
2

= 0.05 (0.1) 0.905163 + 0.043393 = 0.041963

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

We can now compare the numerical solution with the analytical solution: Runge-Kutta methods - coding example
x(t) = t 2 2t + 2 e t .
Lets go back to the example we looked at previously.
At time t = 0.4, the numerical solution is 0.689680 while the analytical
Example (Runge-Kutta method)
solution is 0.689680. So both answers agree to 6 decimal places. Recall
that the error for Eulers method was 0.009929. Apply the Runge-Kutta method to the following IVP:

x 0 = 2x x(0) = 10.

The Runge-Kutta method is a fourth order method, meaning that as h is

halved we would expect the error to be divided by 24 = 16. Figures 7, 8,
9 and 10 shows that is the case.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

3 Error (c = 5) 4 Error (c = 6)
x 10 x 10
8

6 3

4 2
x

x
3

2 1

0 0
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
t t

Figure : Difference between the numerical and analytical solutions when Figure : Difference between the numerical and analytical solutions when
h = 10/25 . h = 10/26 .

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

5 Error (c = 7) 6 Error (c = 8)
x 10 x 10
2.5 1.4

1.2
2

1.5
0.8
x

x
0.6
1

0.4

0.5
0.2

0 0
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
t t

Figure : Difference between the numerical and analytical solutions when Figure : Difference between the numerical and analytical solutions when
h = 10/27 . h = 10/28 .

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Runge-Kutta method for systems
Consider the IVP
The formula to advance the solution is:
1
x(t + h) = x(t) + (F1 + 2F2 + 2F3 + F4 ) ,
0 6
x = f (t, x, y ) 1
0 y (t + h) = y (t) + (G1 + 2G2 + 2G3 + G4 ) .
y = g (t, x, y ) 6
.
x(t 0 ) = x0
y (t0 ) = y0

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

where Galactic dynamics

F1 = hf (t, x, y ), G1 = hg (t, x, y )
 
h 1 1
F2 = hf t + , x + F1 , y + G1
2 2 2
 
h 1 1
G2 = hg t + , x + F1 , y + G1
2 2 2
 
h 1 1
F3 = hf t + , x + F2 , y + G2
2 2 2
 
h 1 1
G3 = hg t + , x + F2 , y + G2
2 2 2
F4 = hf (t + h, x + F3 , y + G3 )
G4 = hg (t + h, x + F3 , y + G3 ) .

Figure : Cylindrical co-ordinates.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Consider a star that is moving in the potential well created by our Galaxy. The above is a system of two second order differential equations and
This star will obey Newtons second law, which in cylindrical coordinates cannot be solved analytically. Lets now re-write this system in terms of
is (see Figure 11): four first order differential equations:
d 2~r ~ G (R, z)
=F
0
dt 2 R =U
= RLz3 R

0
U
where FG is the gravitational force exerted on the star by the Galaxy.
z0 =W
Note that since disk galaxies are axial symmetric, FG is not a function of

W 0 =

. z
The components of the above equation are: where U and W are the velocities in the R and z direction respectively.
( 2
d R
dt 2 = RLz3 R

d 2z
dt 2 = z

where is the potential of our Galaxy and Lz = R 2 d

dt (d/dt is the
angular velocity), is the angular momentum about the zaxis, which is
conserved.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Now we need the initial conditions. These can be obtained

experimentally; the astronomer measures the components of the velocity
of a star by observing it with various techniques. The Doppler shift in
spectral lines will give the U component of velocity while the W and V
components are found by measuring how far a star has moved over a
certain period of time with respect to background stars on a
photographic or CCD plate. If the stars observed are all in the solar
neighbourhood, then we know where they are in the Galaxy: in the plane
of the Galaxy (z = 0) at R = 8.5 kiloparsecs from its centre, just like the
Sun. So we have a system of four first order differential equations with
initial conditions:

Figure : The spiral Galaxy NGC 4414 (Credit: W. Freedman (Carnigie Obs.) ,
L. Frattare (STScI) et al., & the Hubble Heritage Team (AURA/ STScI/
NASA) ). Our own Galaxy would look a bit like this if seen nearly face-on.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

So, we have now almost everything necessary to start our calculations

R0 = 8.5 kiloparsecs from centre of Galaxy which will allow us to find out where the stars that we have diligently
z0 = 0 kiloparsecs from plane of Galaxy

observed for months and months will be in, say, 300 million years (note:

U(8.5, 0) = observed value in km/sec it takes the Sun about 250 million years to rotate once around the
W (8.5, 0) = observed value in km/sec



Galaxy). All that is still needed is a simple model for the potential
V (8.5, 0) = observed value in km/sec

(R, z) of a disk galaxy. In the simple example presented here, we will
(note: 1 kiloparsec = 1, 000 parsecs = 1, 000 3.26 light years = use a Galaxy of 1.5 1011 solar masses, whose potential is:
1, 000 3.086 1013 km).
z2
 
v0
Furthermore, we also know the value of the angular momentum Lz , since (R, z) = log R 2 + 2
Lz = RV and the initial values of R and V will determine its value which 2 q
will not change during the calculations. where q is the axial ratio and v0 is the circular speed. Generally, a disk
galaxy can be built by adding the contribution of the potentials given by
the galactic bulge, the galactic disk and the halo.
Typical outputs are shown in the figures 13 and 14.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Figure : Orbit of a star in the R z plane with initial velocities U = 130 Figure : Orbit of a star in the R z plane with initial velocities U = 5 km/sec,
km/sec, V = 180 km/sec and W = 95 km/sec. V = 400 km/sec and W = 400 km/sec.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Numerical methods to solve differential equations can also be used to

study the dynamics of galaxies. In 1972, Alar and Juri Toomre
constructed computer simulations of galaxy interactions. In Figure 15
there is a simulation of a close encounter of two galaxies. One is
modelled as a single point mass whilst the other (the large galaxy) is
made up by rings of point masses. The point mass intruder galaxy
moves toward the large galaxy in the same direction as that of the spin of
the large galaxy. In the beginning of the simulation, the interaction
causes the formation of a bridge-like structure of stars joining the two
galaxies. At the end of the simulation the large galaxy doesnt look like
its original self anymore!

Figure : Simulation of a low-mass galaxy passing close to a disk galaxy.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem Taylors theorem

Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Toomres simulations were also very successful in the modelling of

structures similar to that observed in the colliding Antennae (NGC
4038/4039) galaxies (see simulation and picture below). They are
nicknamed Antennae because of their long curvy tails that look like the
antennae of an insect. See figures 16 and ??.

Figure : Another simulation by A. Toomre, closely resembling the interaction of

the Antennae galaxies shown in the picture below.

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations
 Taylors theorem Taylors theorem
Eulers method Eulers method
Numerical solution of ODEs Numerical solution of ODEs
Runge-Kutta methods Runge-Kutta methods
System of equations System of equations

Figure : The interacting Antennae galaxies NGC4038 and NGC4039 (credit: B.
Whitmore (STScI), F. Schweizer (DTM), NASA)
Figure : Two Merging Galaxies NGC 4676. These galaxies are located 300
million light-years away in the constellation Coma Berenices. The galaxies have
been given the nickname The Mice because of the two long tails made up of
stars and gas originating from each galaxy (Credit: NASA and the ACS Science
Team).

Barry Croke Ordinary Differential Equations Barry Croke Ordinary Differential Equations

Taylors theorem
Eulers method
Numerical solution of ODEs
Runge-Kutta methods
System of equations

All these galaxy interaction problems are quite interesting, particularly

since our own Galaxy will collide with the Andromeda galaxy in a few
billion years (the two galaxies are approaching at a speed of about 300
km/sec). Andromeda is about twice as massive as the Galaxy and (at the
moment!) about 2.6 million light years away. See Figure ??.

Barry Croke Ordinary Differential Equations