Time-stepping techniques

Unsteady flows are parabolic in time ⇒ use ‘time-stepping’ methods to

advance transient solutions step-by-step or to compute stationary solutions

time
Initial-boundary value problem u = u(x, t)
future
zone of influence

∂u
present 

 ∂t + Lu = f in Ω × (0, T ) time-dependent PDE
domain of
past dependence Bu = 0 on Γ × (0, T ) boundary conditions


u = u0 in Ω at t = 0 initial condition

space

Time discretization 0 = t0 < t1 < t2 < . . . < tM = T u0 ≈ u0 in Ω

• Consider a short time interval (tn , tn+1 ) , where tn+1 = tn + ∆t

• Given un ≈ u(tn ) use it as initial condition to compute un+1 ≈ u(tn+1 )
 Space-time discretization

Space discretization: finite differences / finite volumes / finite elements

Unknowns: ui (t) time-dependent nodal values / cell mean values

Time discretization: (i) before or (ii) after the discretization in space

The space and time variables are essentially decoupled and can be discretized
independently to obtain a sequence of (nonlinear) algebraic systems

A(un+1 , un )un+1 = b(un ) n = 0, 1, . . . , M − 1

Method of lines (MOL) L → Lh yields an ODE system for ui (t)

duh
+ Lh uh = fh on (tn , tn+1 ) semi-discretized equations
dt
N
uni ≈ u(xi , tn )
P
FEM approximation uh (x, t) = uj (t)ϕj (x),
j=1
 Galerkin method of lines

∂u
∀t ∈ (tn , tn+1 )
R

Weak formulation Ω ∂t + Lu − f v dx = 0, ∀v ∈ V,

d d
dt (u, v) + a(u, v) = l(v), ∀v ∈ V → dt (uh , vh ) + a(uh , vh ) = l(vh ), ∀vh ∈ Vh

Differential-Algebraic Equations MC du
dt + Au = b t ∈ (tn , tn+1 )

where MC = {mij } is the mass matrix and u(t) = [u1 (t), . . . , uN (t)]T

Matrix coefficients mij = (ϕi , ϕj ), aij = a(ϕi , ϕj ), bi = l(ϕi )

P P R
Mass lumping MC → ML = diag{mi }, mi = mij = (ϕi , ϕj ) = Ω
ϕi dx
j j
P
due to the fact that ϕj ≡ 1. In the 1D case FDM=FVM=FEM + lumping
j
 Two-level time-stepping schemes

Lumped-mass discretization ML du
dt + Au = b, R(u, t) = ML−1 [Au − b]

du

dt + R(u, t) = 0 for t ∈ (tn , tn+1 )
First-order ODE system
 u(tn ) = un , ∀n = 0, 1, . . . , M − 1

Standard θ−scheme (finite difference discretization of the time derivative)

un+1 − un  n+1 n+1 n n


+ θR(u ,t ) + (1 − θ)R(u , t ) = 0 0≤θ≤1
∆t

where ∆t = tn+1 − tn is the time step and θ is the implicitness parameter

θ=0 forward Euler scheme explicit, O(∆t)

θ = 1/2 Crank-Nicolson scheme implicit, O(∆t)2
θ=1 backward Euler scheme implicit, O(∆t)
 Fully discretized problem

Consistent-mass discretization MC du
dt + Au = b, R(u, t) = MC−1 [Au − b]

[MC + θ∆tA]un+1 = [MC − (1 − θ)∆tA]un + ∆t bn+θ

where bn+θ = θbn+1 + (1 − θ)bn , 0 ≤ θ ≤ 1, n = 0, . . . , M − 1

In general, time discretization is performed using numerical methods for ODEs


 du(t) = f (t, u(t))
dt
Initial value problem on (tn , tn+1 )
 u(tn ) = un

R tn+1 du n+1 n
R tn+1
Exact integration tn dt dt = u −u = tn
f (t, u(t)) dt

un+1 = un + f (τ, u(τ ))∆t, τ ∈ (tn , tn+1 ) by the mean value theorem

Idea: evaluate the integral numerically using a suitable quadrature rule

 Example: standard time-stepping schemes

Numerical integration on the interval (tn , tn+1 )

f FE f BE f CN f LF

111111111
000000000 111111111
000000000
000000000
111111111 000000000
111111111 111111111
000000000
000000000
111111111 000000000
111111111 000000000
111111111
111111111
000000000 000000000
111111111 000000000
111111111 000000000
111111111
000000000
111111111 000000000
111111111 000000000
111111111 000000000
111111111
000000000
111111111 000000000
111111111 000000000
111111111 000000000
111111111
000000000
111111111 000000000
111111111 000000000
111111111 000000000
111111111
000000000
111111111 000000000
111111111 000000000
111111111 000000000
111111111
000000000
111111111 000000000
111111111 000000000
111111111 000000000
111111111
000000000
111111111 000000000
111111111 000000000
111111111 000000000
111111111
000000000
111111111 000000000
111111111 000000000
111111111 000000000
111111111
000000000
111111111 000000000
111111111 000000000
111111111 000000000
111111111
t 000000000
111111111 t 000000000
111111111 t t
tn tn+1 tn tn+1 tn tn+1 tn tn+1

left endpoint right endpoint trapezoidal rule midpoint rule

Forward Euler un+1 = un + f (tn , un )∆t + O(∆t)2

Backward Euler un+1 = un + f (tn+1 , un+1 )∆t + O(∆t)2
Crank-Nicolson un+1 = un + 12 [f (tn , un ) + f (tn+1 , un+1 )]∆t + O(∆t)3
Leapfrog method un+1 = un + f (tn+1/2 , un+1/2 )∆t + O(∆t)3
 Properties of time-stepping schemes
T T
Time discretization tn = n∆t, ∆t = M ⇒ M= ∆t

Accumulation of truncation errors n = 0, . . . , M − 1

ǫloc
τ = O(∆t)
p
⇒ ǫglob
τ = M ǫloc
τ = O(∆t)
p−1

Remark. The order of a time-stepping method (i.e., the asymptotic rate at

which the error is reduced as ∆t → 0) is not the sole indicator of accuracy

The optimal choice of the time-stepping scheme depends on its purpose:

• to obtain a time-accurate discretization of a highly dynamic flow problem
(evolution details are essential and must be captured) or
• to march the numerical solution to a steady state starting with some
reasonable initial guess (intermediate results are immaterial)
The computational cost of explicit and implicit schemes differs considerably
 Explicit vs. implicit time discretization

Pros and cons of explicit schemes

⊕ easy to implement and parallelize, low cost per time step
⊕ a good starting point for the development of CFD software
⊖ small time steps are required for stability reasons, especially
if the velocity and/or mesh size are varying strongly
⊖ extremely inefficient for solution of stationary problems unless
local time-stepping i. e. ∆t = ∆t(x) is employed

Pros and cons of implicit schemes

⊕ stable over a wide range of time steps, sometimes unconditionally
⊕ constitute excellent iterative solvers for steady-state problems
⊖ difficult to implement and parallelize, high cost per time step
⊖ insufficiently accurate for truly transient problems at large ∆t
⊖ convergence of linear solvers deteriorates/fails as ∆t increases
 Example: 1D convection-diffusion equation

2
∂u

∂t + v ∂u ∂ u
∂x = d ∂x2 in (0, X) × (0, T ) 2

 u(0) = u(1) = 0, f (t, u(t)) = −v ∂u ∂ u

∂x + d ∂x2
u|t=0 = u0

du(x(t), t) ∂2u
Lagrangian representation =d 2 pure diffusion equation
dt ∂x
d dx(t)
where dt is the substantial derivative along the characteristic lines dt =v

Initial profile is convected at speed v u

t=0 t=T
and smeared by diffusion if d > 0
v d=0
du(x(t), t)
d=0 ⇒ =0
dt
d>0
For the pure convection equation u is
constant along the characteristics 0 X x
 Example: 1D convection-diffusion equation
Uniform space-time mesh t
x
x
X T
xi = i∆x, ∆x = N, i = 0, . . . , N

unknown
T
tn = n∆t, ∆t = M, n = 0, . . . , M
tn+1
un −→ un+1 , u0i = u0 (xi )
t
tn

t
Fully discretized equation 0≤θ≤1 t 1
n

known
un+1
i = uni + [θfhn+1 + (1 − θ)fhn ]∆t

Central difference / lumped-mass FEM 0 x 1 ix x +1 i i X x

" #
n+1 n n+1 n+1 n+1 n+1 n+1
ui − ui u − ui−1 u − 2ui + ui+1
= θ −v i+1 + d i−1
∆t 2∆x (∆x)2
uni+1 − uni−1 uni−1 − 2uni + uni+1
 
+ (1 − θ) −v +d
2∆x (∆x)2
a sequence of tridiagonal linear systems i = 1, . . . , N, n = 0, . . . M − 1
 Example: 1D convection-diffusion equation
Standard θ−scheme (two-level) FE
t
n+1
n+θ n+θ n+θ
un+1
i − uni ui+1 − ui−1 ui−1 − 2uin+θ + ui+1
n+θ
+v =d t
n
∆t 2∆x (∆x)2
uin+θ = θun+1
i + (1 − θ)uni , 0≤θ≤1 n+1
BE
t

Forward Euler (θ = 0) un+1

i = h(uni−1 , uni , uni+1 ) t
n

Backward Euler (θ = 1) un+1

i = h(un+1 n n+1
i−1 , ui , ui+1 ) CN
t
n+1
Crank-Nicolson (θ = 12 ) un+1
i = h(un+1 n n n n+1
i−1 , ui−1 , ui , ui+1 , ui+1 )
t
n

Leapfrog time-stepping un+1

i = un−1
i + 2∆tfhn
LF
t
n+1
un+1
i − un−1
i uni+1 − uni−1 uni−1 − 2uni + uni+1
+v =d
2∆t 2∆x (∆x)2 t
n

(explicit, three-level) un+1

i = h(uni−1 , un−1
i , uni , uni+1 ) t
n 1
 Fractional-step θ−scheme

Given the parameters θ ∈ (0, 1), θ′ = 1 − 2θ, and α ∈ [0, 1] subdivide the time
interval (tn , tn+1 ) into three substeps and update the solution as follows

Step 1. un+θ = un + [αf (tn+θ , un+θ ) + (1 − α)f (tn , un )]θ∆t

Step 2. un+1−θ = un+θ + [(1 − α)f (tn+1−θ , un+1−θ ) + αf (tn+θ , un+θ )]θ′ ∆t

Step 3. un+1 = un+1−θ + [αf (tn+1 , un+1 ) + (1 − α)f (tn+1−θ , un+1−θ )]θ∆t

Properties of this time-stepping method

√
2
• second-order accurate in the special case θ = 1 − 2
1−2θ
• coefficient matrices are the same for all substeps if α = 1−θ

• combines the advantages of Crank-Nicolson and backward Euler

 Predictor-corrector and multipoint methods

Objective: to combine the simplicity of explicit schemes and robustness of

implicit ones in the framework of a fractional-step algorithm, e.g.,

1. Predictor ũn+1 = un + f (tn , un )∆t forward Euler

2. Corrector un+1 = un + 21 [f (tn , un ) + f (tn+1 , ũn+1 )]∆t Crank-Nicolson

or un+1 = un + f (tn+1 , ũn+1 )∆t backward Euler

Remark. Stability still leaves a lot to be desired, additional correction steps

usually do not pay off since iterations may diverge if ∆t is too large

Order barrier: two-level methods are at most second-order accurate, so

extra points are needed to construct higher-order integration schemes

Adams methods tn+1 , . . . , tn−m , m = 0, 1, . . .

Runge-Kutta methods tn+α ∈ [tn , tn+1 ], α ∈ [0, 1]
 Adams methods
Derivation: polynomial fitting f
past present future
Truncation error: ǫglob
τ = O(∆t)p
for polynomials of degree p − 1 which
interpolate function values at p points
t
tn 2 tn 1 tn tn+1
Adams-Bashforth methods (explicit)
p=1 un+1 = un + ∆tf (tn , un ) forward Euler
∆t
p=2 un+1 = un + n n
2 [3f (t , u ) − f (tn−1 , un−1 )]
∆t
p=3 un+1 = un + n n
12 [23f (t , u ) − 16f (tn−1 , un−1 ) + 5f (tn−2 , un−2 )]

Adams-Moulton methods (implicit)

p=1 un+1 = un + ∆tf (tn+1 , un+1 ) backward Euler
∆t
p=2 un+1 = un + 2 [f (t
n+1
, un+1 ) + f (tn , un )] Crank-Nicolson
∆t
p=3 un+1 = un + 12 [5f (t
n+1
, un+1 ) + 8f (tn , un ) − f (tn−1 , un−1 )]
 Adams methods

Predictor-corrector algorithm
1. Compute ũn+1 using an Adams-Bashforth method of order p − 1
2. Compute un+1 using an Adams-Moulton method of order p with
predicted value f (tn+1 , ũn+1 ) instead of f (tn+1 , un+1 )

Pros and cons of Adams methods

⊕ methods of any order are easy to derive and implement

⊕ only one function evaluation per time step is performed
⊕ error estimators for ODEs can be used to adapt the order
⊖ other methods are needed to start/restart the calculation
⊖ time step is difficult to change (coefficients are different)
⊖ tend to be unstable and produce nonphysical oscillations
 Runge-Kutta methods

Multipredictor-multicorrector algorithms of order p

∆t
p=2 ũn+1/2 = un + n n
2 f (t , u ) forward Euler / predictor

un+1 = un + ∆tf (tn+1/2 , ũn+1/2 ) midpoint rule / corrector

∆t
p=4 ũn+1/2 = un + n n
2 f (t , u ) forward Euler / predictor
∆t
ûn+1/2 = un + 2 f (t
n+1/2
, ũn+1/2 ) backward Euler / corrector

ũn+1 = un + ∆tf (tn+1/2 , ûn+1/2 ) midpoint rule / predictor

∆t
un+1 = un + n n
6 [f (t , u ) + 2f (tn+1/2 , ũn+1/2 ) Simpson rule
+ 2f (tn+1/2 , ûn+1/2 ) + f (tn+1 , ũn+1 )] corrector

Remark. There exist ‘embedded’ Runge-Kutta methods which perform extra

steps in order to estimate the error and adjust ∆t in an adaptive fashion
 General comments

Pros and cons of Runge-Kutta methods

⊕ self-starting, easy to operate with variable time steps
⊕ more stable and accurate than Adams methods of the same order
⊖ high order approximations are rather difficult to derive; p function
evaluations per time step are required for a p−th order method
⊖ more expensive than Adams methods of comparable order

Adaptive time-stepping strategy ∆t ∆t ∆t ∆t ∆t ∆t ∆t ∆t

makes it possible to achieve the desired accuracy at a relatively low cost

Explicit methods: use the largest time step satisfying the stability condition

Implicit methods: estimate the error and adjust the time step if necessary
 Automatic time step control

Objective: make sure that ||u − u∆t || ≈ T OL (prescribed tolerance)

Local truncation error Heuristic error analysis

1. u∆t = u + ∆t2 e(u) + O(∆t)4 um∆t − u∆t

e(u) ≈
∆t2 (m2 − 1)
2. um∆t = u + m2 ∆t2 e(u) + O(∆t)4

Remark. It is tacitly assumed that the error at t = tn is equal to zero.

Estimate of the relative error Adaptive time stepping

2
∆t2 (m2 − 1)

∆t∗ ||u∆t − um∆t ||
||u − u∆t∗ || ≈ = T OL ∆t2∗ = T OL
∆t m2 − 1 ||u∆t − um∆t ||

Richardson extrapolation: eliminate the leading term

m2 u∆t − um∆t 4
u= + O(∆t) fourth-order accurate
m2 − 1
 Practical implementation
Automatic time step control can be executed as follows

Given the old solution un do:

1. Make one large time step of size m∆t to compute um∆t
2. Make m small substeps of size ∆t to compute u∆t
3. Evaluate the relative solution changes ||u∆t − um∆t ||
4. Calculate the ‘optimal’ value ∆t∗ for the next time step
5. If ∆t∗ ≪ ∆t, reset the solution and go back to step 1
6. Set un+1 = u∆t or perform Richardson extrapolation

⊖ Note that the cost per time step increases substantially (um∆t may be as
expensive to obtain as u∆t due to slow convergence at large time steps).
⊕ On the other hand, adaptive time-stepping enhances the robustness of the
code, the overall efficiency and the credibility of simulation results.
 Evolutionary PID controller

Another simple mechanism for capturing the dynamics of the flow:

||un+1 −un ||
• Monitor the relative changes en = ||un+1 || of an indicator variable u
ǫ
• If en > ǫ reject the solution and repeat the time step using ∆t∗ = en ∆tn

• Adjust the time step smoothly so as to approach the prescribed tolerance

k P  k I  k D
e2n−1

en−1 T OL
∆tn+1 = ∆tn
en en en en−2

• Limit the growth and reduction of the time step so that

∆tn+1
∆tmin ≤ ∆tn+1 ≤ ∆tmax , l≤ ≤L
∆tn
Empirical PID parameters kP = 0.075, kI = 0.175, kD = 0.01
 Pseudo time-stepping

Solutions to boundary value problems of the form Lu = f represent the

steady-state limit of the associated time-dependent problem
∂u
+Lu = f, u(x, 0) = u0 (x) in Ω
∂t
where u0 is an ‘arbitrary’ initial condition. Therefore, the numerical solution
can be ‘marched’ to the steady state using a pseudo time-stepping technique.
• can be interpreted as an iterative solver for the stationary problem
• the artificial time step represents an adjustable relaxation parameter
• evolution details are immaterial ⇒ ∆t should be as large as possible
• the unconditionally stable backward Euler method is to be recommended
• explicit schemes can be used in conjunction with local time-stepping
• it is worthwhile to perform an adaptive time step control (e.g., PID)