Finite Differences for Differential Equations 28 Finite Differences for Differential Equations 29

B OUNDARY VALUE P ROBLEMS (I)

Solving a TWO POINT BOUNDARY VALUE PROBLEM with D IRICHLET
B OUNDARY C ONDITIONS :
d2y
=g for x (0, 2)
dx2
y(0) = y(2) = 0
PART II
Finitedifference approximation:
Secondorder center difference formula for the interior nodes:
Finite Difference Methods for Differential y j+1 2y j + y j1
= g j for j = 1, . . . , N
h2
Equations where h = 2
N+1 and x j = jh
Endpoint nodes:
y0 = 0 = y2 2y1 = h2 g1
yN+1 = 0 = 2yN + yN1 = h2 gN

Tridiagonal algebraic system solved very efficiently with the

T HOMAS ALGORITHM (a version of the Gaussian elimination)

Finite Differences for Differential Equations 30 Finite Differences for Differential Equations 31

B OUNDARY VALUE P ROBLEMS (II)

Solving a TWO POINT BOUNDARY VALUE PROBLEM with N EUMANN
B OUNDARY C ONDITIONS :
d2y
dx2
=g for x (0, 2) B OUNDARY VALUE P ROBLEMS (III)
dy dy
(0) = (2) = 0
dx dx
In order to retain the SECOND ORDER ACCURACY in the approximation of
Finitedifference approximation: the Neumann problem need to use higher-order formulae at endpoints, e.g.
Secondorder center difference formula for the interior nodes: y2 + 4y1 3y0 1
y0 = = 0 = y0 = (y2 + 4y1 )
y j+1 2y j + y j1 2h 3
= g j for j = 1, . . . , N
h2
The first row thus becomes
Firstorder Forward/Backward Difference formulae to reexpress
2 2
endpoint values: y1 y0 y2 y1 = h2 g1
= 0 = y0 = y1 3 3
h
yN+1 yN S ECOND ORDER ACCURACY RECOVERED!
= 0 = yN+1 = yN
h
Firstorder only DEGRADED ACCURACY !

Tridiagonal algebraic system Is there any problem? Where?

Finite Differences for Differential Equations 32 Finite Differences for Differential Equations 33

B OUNDARY VALUE P ROBLEMS (IV)

C OMPACT S TENCILS stencils based on three grid points (in every
direction) only: {x j+1 , x j , x j1 } at the j th node
Is is possible to obtain higher (then second) order of accuracy on compact
stencils? YES !
B OUNDARY VALUE P ROBLEMS (V)
d2 y
Consider the central difference approximation to the equation dy2
=g
C OMPACT F INITE D IFFERENCE S CHEMES
y j+1 2y j + y j1 h2 (iv)
y j + O (h4 ) = g j A DVANTAGES :
h2 12
2 (iv)
Increased accuracy on compact grids
h
Re-express the error term 12 y j using the equation in question:
DRAWBACKS:
h2 (iv) h2 h2 g j+1 2g j + g j1 h2 (iv)
 
yj = gj = g + O (h 4
) need to be tailored to the specific equation solved
12 12 12 h2 12 j
can get fairly complicated for more complex equations
Inserting into the original finitedifference equation:
y j+1 2y j + y j1 g j+1 2g j + g j1
= gj + + O (h4 )
h2 12
Slight modification of the RHS = FOURTH ORDER ACCURACY !!!

Finite Differences for Differential Equations 34 Finite Differences for Differential Equations 35

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

G ENERAL R EMARKS
Consider the following C AUCHY PROBLEM :
dy
= f (y,t) with y(t0 ) = y0
dt
The independent variable t is usually referred to as
Equations with higherorder derivatives can be reduced to systems of
firstorder equations
Generalizations to systems of ODEs straightforward
When the RHS function does not depend on y, i.e., f (y,t) = f (t),
solution obtained via a QUADRATURE
Assume uniform timesteps ( h is constant )
C HARACTERIZATION OF I NTEGRATION
M ETHODS
ACCURACY unlike in the Boundary Value Problems, there is no terminal
condition and approximation errors may accumulate in time; consequently, a
TIME . relevant characterization of accuracy is provided by the GLOBAL ERROR
(global error) = (local error) (# of time steps),
rather than the LOCAL ERROR .
STABILITY unlike in the Boundary Value Problems, where boundedness
of the solution at final time is enforced via a suitable terminal condition , in
Initial Value Problems there is a priori no guarantee that the solution will
remain bounded.
Finite Differences for Differential Equations 36 Finite Differences for Differential Equations 37

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

M ODEL P ROBLEM E XPLICIT E ULER S CHEME (I)
S TABILITY of various numerical schemes is usually analyzed by applying
these schemes to the following LINEAR MODEL :
Consider a Taylor series expansion
dy
= y = (r + ii )y with y(t0 ) = y0 ,
dt h2
y(tn+1 ) = y(tn ) + hy (tn ) + y (tn ) + . . .
which is stable when r <= 0 . 2
2 h2 3 h3 Using the ODE we obtain
 
y(t): = y0 et = 1 + h +
E XACT SOLUTION + + . . . y0
2 6 dy
y = =f
M OTIVATION consider the following ADVECTION DIFFUSION PDE : dt
dy df
u u 2 u y = = = ft + f fy
+c a 2 = 0 dt dt
t x x
Neglecting terms proportional to second and higher powers of h yields the
Taking Fourier transform yields (k is the wavenumber):
E XPLICIT E ULER M ETHOD
d uk
+ c i k uk + a k2 uk = 0
dt yn+1 = yn + h f (yn ,tn )
where
the real term a k2 uk represents DIFFUSION Retaining higherorder terms is inconvenient, as it requires differentiation of
the imaginary term c i k uk represents ADVECTION f and does not lead to schemes with desirable stability properties.

Finite Differences for Differential Equations 38 Finite Differences for Differential Equations 39

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

E XPLICIT E ULER S CHEME (II) E XPLICIT E ULER S CHEME (III)

Stability (for the model problem)

L OCAL ERROR analysis:
yn+1 = yn + hyn = (1 + h)yn
yn+1 = (1 + h) yn + [O (h2 )]
Thus, the solution after n time steps
G LOBAL ERROR analysis:
yn = (1 + h)n y0 , n y0 = = 1 + h
T
(global error) = Ch N = Ch = C h
2 2
h For large n, the numerical solution remains stable iff
Thus, the scheme is || 1 = (1 + r h)2 + (i h)2 1
locally secondorder accurate
CONDITIONALLY STABLE for real
globally (over the interval [t0 ,t0 + Nh]) firstorder accurate
UNSTABLE for imaginary
Finite Differences for Differential Equations 40 Finite Differences for Differential Equations 41

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

I MPLICIT E ULER S CHEME (I) I MPLICIT E ULER S CHEME (II)

I MPLICIT S CHEMES based on approximation of the RHS that involve Stability (for the model problem):
f (yn+1 ,t), where yn+1 is the unknown to be determined yn+1 = yn + hyn+1 = yn+1 = (1 h)1 yn
I MPLICIT E ULER S CHEME obtained by neglecting second and 
1
n
1
yn+1 = y0 , n y0 = =
higherorder terms in the expansion: 1 h 1 h
h2 || 1 = (1 r h)2 + (i h)2 1
y(tn ) = y(tn+1 ) hy (tn+1 ) + y (tn+1 ) . . .
2
 Implicit Euler scheme is thus stable for
dy 
Upon substitution dt t = f (yn+1 ,tn+1 ) we obtain
n+1 all stable model problems
yn+1 = yn + h f (yn+1 ,tn+1 ) most unstable model problems

The scheme is R EMARK : When solving systems of ODEs of the form y = A (t)y, each
locally SECOND ORDER accurate implicit step requires solution of an algebraic system: yn+1 = (I hA )1 yn

globally (over the interval [t0 ,t0 + Nh]) FIRST ORDER accurate Implicit schemes are generally hard to implement for nonlinear problems

Finite Differences for Differential Equations 42 Finite Differences for Differential Equations 43

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

C RANK N ICHOLSON S CHEME (I) L EAPFROG S CHEME (I)
Obtained by approximating the formal solution of the ODE
Rt
yn+1 = yn + tnn+1 f (y,t) dt using the TRAPEZOIDAL QUADRATURE :
L EAPFROG as an example of a TWO STEP METHOD :
h
yn+1 = yn + [ f (yn ,tn ) + f (yn+1 ,tn+1 )] yn+1 = yn1 + 2 h yn
2
The scheme is C HARACTERISTIC EQUATION for the AMPLIFICATION FACTOR (yn = n y0 )
locally THIRD ORDER accurate
2 2 h 1 = 0
globally (over the interval [t0 ,t0 + Nh]) SECOND ORDER accurate
where roots give the amplification factors:
Stability (for the model problem):
2 h2
+ . . . = eh + O (h3 )
p
1 + h 1 = h + 1 + 2 h2 1 + h +
!
h 2 2
yn+1 = yn + (yn+1 + yn ) = yn+1 = yn
2 1 h
2
p 2 h2
!n 2 = h 1 + 2 h2 (1 h + . . . ) = eh + O (h3 )
1 + h 1 + h 2
yn+1 = 2
y0 , n y0 = = 2
1 h
2 1 h
2
Thus, the scheme is
|| 1 = (h) 0 locally THIRD ORDER accurate
S TABLE for all model ODEs with stable solutions globally (over the interval [t0 ,t0 + Nh]) SECOND ORDER accurate
Finite Differences for Differential Equations 44 Finite Differences for Differential Equations 45

I NITIAL VALUE P ROBLEMS

I NITIAL VALUE P ROBLEMS M ULTISTEP P ROCEDURES
L EAPFROG S CHEME (II) General form of a MULTISTEP PROCEDURE :
p q
j yn+ j = h j f (yn+ j ,tn+ j )
j=1 j=1
Stability for diffusion problems ( = r ):
q with characteristic polynomials
1 = h + 1 + 2r h2 > 1 for all h > 0 p (z) = p z p + p1 z p1 + + 0
q (z) = q zq + q1 zq1 + + 0
Thus the scheme is UNCONDITIONALLY UNSTABLE for diffusion problems!
Stability for advection problems ( = ii ): if p > q EXPLICIT SCHEME

1
if p q IMPLICIT SCHEME
21/2 = 1 (!!!) for h <
|i | A (, ) procedure converges uniformly in [a, b], i.e.,
Thus, the scheme is CONDITIONALLY UNSTABLE and NON DIFFUSIVE for limh0 maxtn [a,b] |yn y(tn )| = 0 if:
advection problems! the following consistency conditions are verified: (1) = 0 and
(1) = (1) ( CONSISTENCY CONDITION )
Q UESTION analyze dispersive (i.e., related to arg()) errors of the
leapfrog scheme. all roots of the polynomial (z) are such that |zi | 1 and the roots with
|zk | = 1 are simple ( STABILITY CONDITION )

Finite Differences for Differential Equations 46 Finite Differences for Differential Equations 47

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

RUNGE K UTTA M ETHODS (I) RUNGE K UTTA M ETHODS (II)
General form of a FRACTIONAL STEP METHOD :
RK4 an ODE workhorse:
yn+1 = yn + 1 h k1 + 2 h k2 + 3 h k3 + . . .
h h h
where yn+1 = yn + k1 + (k2 + k3 ) + k4
6 3 6
k1 = f (yn ,tn ) h
k1 = f (yn ,tn ) k2 = f (yn + k1 ,tn+1/2 )
2
k2 = f (yn + 1 hk1 ,tn + 1 h)
h
k3 = f (yn + 2 hk1 + 3 hk2 ,tn + 2 h) k3 = f (yn + k2 ,tn+1/2 ) k4 = f (yn + hk3 ,tn+1 )
2
. The amplification factor:
.
.
Choose i , i and i to match as many expansion coefficients as possible in 2 h2 3 h3 4 h4
= 1 + h + + +
h2 h3 2 6 24
y(tn+1 ) = y(tn ) + hy (tn ) + y (tn ) + y (tn ) . . .
2 6 Thus, stability iff || 1
y = f
ACCURACY:
y = ft + f fy
eh = + O (h5 )
y = ftt + ft fy 2 f fyt + f 2 fyt + f 2 fyy
Thus, the scheme is
RungeKutta methods are SELF STARTING with fairly good stability and locally FIFTH ORDER accurate
accuracy properties. globally (over the interval [t0 ,t0 + Nh]) FOURTH ORDER accurate
Finite Differences for Differential Equations 48 Finite Differences for Differential Equations 49

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

RUNGE S P RINCIPLE L AX E QUIVALENCE T HEOREMa
Consider an I NITIAL VALUE P ROBLEM
Let (k + 1) be the order of the local truncation error; denote Y (t, h) an du
= L u with u(t0 ) = u0
approximation of the exact solution y(t) computed with the step size h; then dt
at t = t0 + 2nh: and assume that it is wellposed, i.e., it admits solutions which are unique
y(t) Y (t, h) C 2 n hk+1 = C(t t0 )hk and stable

y(t) Y (t, 2h) C n (2h)k+1 = C(t t0 )2k hk Consider a numerical method defined by a finitedifference operator C(h)
such that the approximate solution is given by
Subtracting:
Y (t, 2h) Y (t, h) C(t t0 )(1 2k )hk uh (nh) = C (h)n u0 , n = 1, 2, . . .
Thus, we can obtain an estimate of the ABSOLUTE ERROR based on solution The above method is CONSISTENT iff
C (h)I
is a convergent approximation
h
with two stepsizes only: of the operator L
Y (t, h) Y (t, 2h) L AX T HEOREM For a CONSISTENT difference method STABILITY is
y(t) Y (t, h)
2k 1 equivalent to CONVERGENCE
a For
Runges principle is very useful for ADAPTIVE STEP SIZE REFINEMENT a more technical discussion, see $ 5.2 in Atkinson & Han (2001)

Finite Differences for Differential Equations 50 Finite Differences for Differential Equations 51

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

C ONSERVATION P ROPERTIES (I) C ONSERVATION P ROPERTIES (II)
Is ACCURACY and S TABILITY all that matters? Implicit Euler
 1  1 1
C ONSERVATION P ROPERTIES conservation by the numerical method || =  = q = 1 2i h2 + < 1 for all h

1 ii h 1 + i h
2 2 2
(i.e., in the discrete sense) of various invariants the original equation may
possess The scheme is thus DISSIPATIVE (i.e., not conservative)
R EMARK conservation properties are particularly relevant for solution FourthOrder RungeKutta
of Hamiltonian / hyperbolic systems
 
2i h2 3i h3 4i h4  1
 q
|| = 1 + ii h i + = 576 86i h6 + 8i h8

Example conservation of the solution norm:  2 6 24  24

In the continuous setting (assume u = |u|ei ) = 1

1 6 6
h + < 1 for small h
144 i
d|u|
= 0 = |u(t)| = |u0 |, The scheme is thus DISSIPATIVE (i.e., not conservative)


du
dt
= ii u
dt
d
= i , Leapfrog |1/2 | 1 for all h < |1 |
i
dt The scheme is thus CONSERVATIVE for all timesteps for which it is
In the discrete setting: |uh (nh)| = |uh ((n 1)h)| = = |uh (0)| stable!!! Leapfrog is an example of a SYMPLECTIC INTEGRATOR which are
Necessary and sufficient condition for discrete conservation: h, |(h)| = 1 designed to have good conservation properties.
Finite Differences for Differential Equations 52 Finite Differences for Differential Equations 53

F INITE D IFFERENCES FOR PDE S

R EVIEW
Classification of linear PDEs in 2D: consider u : 2 R and A, B,C R
such that
2 u 2 u 2 u
A +B +C 2 + f (x, y, u) = 0
x2 x y y
E LLIPTIC P ROBLEMS : B2 4AC < 0
Poisson equation: F INITE D IFFERENCES FOR PDE S
2 u 2 u E LLIPTIC P ROBLEMS
+ = g(x, y)
x2 y2
PARABOLIC P ROBLEMS : B2 4AC = 0 See Homework Assignment # 1 ...
Heat equation:
u 2 u 2 u
 
=a + + g(x, y)
t x2 y2
H YPERBOLIC P ROBLEMS : B2 4AC > 0
Wave equation:
2 u 2 u 2 u
 
=a + + g(x, y)
t 2 x2 y2

Finite Differences for Differential Equations 54 Finite Differences for Differential Equations 55

F INITE D IFFERENCES FOR PDE S

u 2 u F INITE D IFFERENCES FOR PDE S
PARABOLIC P ROBLEMS t = x2 (I) u 2 u
PARABOLIC P ROBLEMS t = x2 (II)
C RANK N ICHOLSON M ETHOD (x j = jx, j = 1, . . . , M, t = nt,
n = 1, . . . , N): M ETHOD
 2 n un 2un +un
allow for a more general approximation in time of the RHS ( [0, 1])
spatial derivative: xu2 = j+1 (x)j 2 j1 + O ((x)2 )
j
un+1
"  n+1  2 n #
u unj 2 u u
 n+1
j 1
time derivative: = + O (t) = + (1 ) + O (t)
t j t 2 x2 j x2 j
un+1
" n+1 n #
u unj 2 u 2 u
 n+1 
j 1
= + O (t) = + + O ((t)2 )
t j t 2 x2 j x2 j
special cases
t  n+1  = 0 = E XPLICIT METHOD : U n+1 = A0U n
un+1 unj = u j+1 2un+1 + un+1 n n n 2 2


j1 + u j+1 2u j + u j1 + O (x) + (t)
j
2(x)2 j
= 21 = C RANK N ICHOLSON METHOD (see previous slide)
t = 1 = I MPLICIT METHOD: A1U n+1 = U n
thus, defining r = (x)2
,we have at every time step n
Stability:
run+1 n+1
j+1 + 2(1 + r)u j run+1 n n n
j1 = ru j+1 + 2(1 r)u j + ru j1 t 1
The E XPLICIT S CHEME is STABLE for r = 2(x)2
< 2
which for U n = [un1 , . . . , unM ]T can be written as an algebraic system
The C RANK N ICHOLSON and I MPLICIT S CHEME are STABLE for all r
(2I A)U n+1 = (2I + A)U n , where A is a tridiagonal matrix
Finite Differences for Differential Equations 56

F INITE D IFFERENCES FOR PDE S

2 u 2 u
H YPERBOLIC P ROBLEMS t 2 = x2 (I)
n unj+1 2unj +unj1
2 u

Spatial derivative: x2
= (x)2
+ O ((x)2 )
j

Time derivative:
un+1 2unj + un1
n n
2 u 2 u
 
j j
= + O ((t)2 ) = + O ((t)3 )
t 2 j (t)2 x2 j

(t)2 n (t)2
 
n1
un+1 n
unj + O (x)2 + (t)4



j = u j+1 + u j1 u j + 2 1
(x)2 (x)2

(t)2
Stability for (x)2
1

R EMARK : need two initial conditions!