Methods For Solving Ordinary Differential Equations (Lecture Notes) - Peter Philip

Mathematics IV for Geoscience Students:
Methods for Solving Ordinary Differential Equations

Peter Philip∗
Lecture Notes
Created for the Class of Spring Semester 2011 at LMU Munich
July 27, 2018
Contents
1 Ordinary Differential Equations (ODE) of First Order 3
1.1 Definition and Geometric Interpretation . . . . . . . . . . . . . . . . . . 3
1.2 Linear ODE, Variation of Constants . . . . . . . . . . . . . . . . . . . . . 4
1.3 Separation of Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Bernoulli Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Systems of ODE and Higher-Order ODE 16

2.1 Systems of First-Order ODE . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Higher-Order ODE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3 Linear ODE 22
3.1 Existence and Uniqueness . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2 Fundamental System and Variation of Constants . . . . . . . . . . . . . . 24
3.3 Linear ODE of Higher Order and the Wronskian . . . . . . . . . . . . . . 27
3.4 Constant Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.4.1 Linear ODE of Higher Order . . . . . . . . . . . . . . . . . . . . . 30
3.4.2 Systems of Linear ODE . . . . . . . . . . . . . . . . . . . . . . . 35
∗
E-Mail: philip@math.lmu.de
1
CONTENTS 2
4 Numerical Methods 40
4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2 Explicit Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.1 Explicit Euler Method . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.2 General Error Estimates . . . . . . . . . . . . . . . . . . . . . . . 44
4.2.3 Heun’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2.4 Explicit Runge-Kutta Method . . . . . . . . . . . . . . . . . . . . 47
4.3 Implicit Euler Method, Stiff Equations . . . . . . . . . . . . . . . . . . . 49
5 Short Introduction to PDE 55

5.1 Prototypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.2 Laplace’s and Poisson’s Equation . . . . . . . . . . . . . . . . . . . . . . 57
5.2.1 The Fundamental Solution . . . . . . . . . . . . . . . . . . . . . . 57
5.2.2 Boundary Conditions . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.2.3 Green’s Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.2.4 Poisson’s Formula for a Ball . . . . . . . . . . . . . . . . . . . . . 61
5.2.5 Poisson’s Formula for a Half-Space . . . . . . . . . . . . . . . . . 62
A The Vandermonde Determinant 64
B Scalar Products 65
References 67
1 ORDINARY DIFFERENTIAL EQUATIONS (ODE) OF FIRST ORDER 3
1 Ordinary Differential Equations (ODE) of First

Order
1.1 Definition and Geometric Interpretation

Definition 1.1. Let G ⊆ R×R and let f : G −→ R be continuous. An explicit ordinary
differential equation (ODE) of first order is an equation of the form
y ′ = f (x, y), (1.1)
which is an equation for the unknown function y. A solution to this ODE is a differen-
tiable function
φ : I −→ R, (1.2)
defined on a (bounded or unbounded) open interval I ⊆ R satisfying the following two
conditions:
(i) The graph of φ is contained in G, i.e. {(x, φ(x)) ∈ I × R : x ∈ I} ⊆ G.

(ii) φ′ (x) = f x, φ(x) for each x ∈ I.
Note that condition (i) is necessary so that one can even formulate condition (ii).
—
One distinguishes between ordinary differential equations and partial differential equa-
tions (PDE). While ODE contain only derivatives with respect to one variable, PDE
can contain (partial) derivatives with respect to several different variables. In general,
PDE are much harder to solve than ODE. The term first order in Def. 1.1 indicates that
only a first derivative occurs in the equation. Correspondingly, ODE of second order
contain derivatives of second order etc. We will see later that ODE of higher order can
be equivalently formulated and solved as systems of ODE of first order. The explicit in
Def. 1.1 indicates that the ODE is explicitly solved for y ′ . One can also consider implicit
ODE of the form f (x, y, y ′ ) = 0. At first, we will restrict ourselves to studying explicit
ODE of first order.
Remark 1.2. Consider the setting of Def. 1.1. If f does not depend on y, but merely
on x, then (1.1) says φ must be the antiderivative of f according to [Phi18a, Def. 9.20].
As f is assumed to be continuous, according to the fundamental theorem of calculus in
the form [Phi18a, Th. 9.19(a)], the antiderivative of f can be computed by integration,
namely Z x
φ(x) = c + f (t) dt , (1.3)
x0
where x0 ∈ I, c = φ(x0 ). If f does depend also on y, then (1.1) still provides

Z x

φ(x) = c + f t, φ(t) dt , (1.4)
x0
which does not solve the ODE, but only transforms it into an integral equation for φ.
However, in certain cases, this transformation can be useful to find a solution.
—
Geometrically, the ODE (1.1) provides a slope y ′ = f (x, y) for every point (x, y). In
other words, it provides a field of directions. The task is to find a differentiable function
φ such that its graph has the prescribed slope in each point it contains. In certain simple
cases, drawing the field of directions can help to guess the solutions of the ODE.
Example 1.3. (a) Let G := R+ × R and f : G −→ R, f (x, y) := y/x, i.e. we consider
the ODE y ′ = y/x. Drawing the field of directions leads to the idea that the
solutions are functions whose graphs constitute rays, i.e. φc : R+ −→ R, y =
φc (x) = c x with c ∈ R. Indeed, one immediately verifies that each φc constitutes a
solution to the ODE.
(b) Let G := R × R+ and f : G −→ R, f (x, y) := −x/y, i.e. we consider the ODE
y ′ = −x/y. Drawing the field of directions leads to the idea√that √ the solutions
are functions
√ whose graphs constitute semicircles, i.e. φ c : ] − c, c[−→ R, y =
2 +
φc (x) = c − x with c ∈ R . Indeed, we get
−2x −x −x
y ′ = φ′c (x) = √ = = , (1.5)
2 c − x2 φc (x) y
i.e. each φc constitutes a solution to the ODE.
1.2 Linear ODE, Variation of Constants

Definition 1.4. Let I ⊆ R be an open interval and let a, b : I −→ R be continuous
functions. An ODE of the form
y ′ = a(x)y + b(x) (1.6)
is called a linear ODE of first order. It is called homogeneous if, and only if, b ≡ 0; it is
called inhomogeneous if, and only if, it is not homogeneous.
Theorem 1.5 (Variation of Constants). Let I ⊆ R be an open interval and let a, b :
I −→ R be continuous. Moreover, let x0 ∈ I and c ∈ R. Then the linear ODE (1.6) has
a unique solution φ : I −→ R that satisfies the initial condition y(x0 ) = c. This unique
solution is given by
Z x
−1
φ : I −→ R, φ(x) = φ0 (x) c + φ0 (t) b(t) dt , (1.7a)
x0
where Z x Rx
a(t) dt
φ0 : I −→ R, φ0 (x) = exp a(t) dt =e x0
. (1.7b)
x0
Here, and in the following, φ−1

0 denotes 1/φ0 and not the inverse function of φ0 (which
does not even necessarily exist).
Proof. We begin by noting that φ0 according to (1.7b) is well-defined since a is assumed

to be continuous, i.e., in particular, Riemann integrable on [x0 , x]. Moreover, the fun-
damental theorem of calculus [Phi18a, Th. 9.19(a)] applies, showing φ0 is differentiable
with Z x
φ′0 : I −→ R, φ′0 (x) = a(x) exp a(t) dt = a(x)φ0 (x), (1.8)
x0
where the chain rule [Phi18a, (8.14)] was used as well. In particular, φ0 is continuous.
Since φ0 > 0 as well, φ−1
0 is also continuous. Moreover, as b is continuous by hypothesis,
φ−1
0 b is continuous and, thus, Riemann integrable on [x0 , x]. Once again, [Phi18a, Th.
9.19(a)] applies, yielding φ to be differentiable with
φ′ : I −→ R,
Z x
′ ′
φ (x) = φ0 (x) c + φ0 (t) b(t) dt + φ0 (x)φ0 (x)−1 b(x)
−1
x0 Z x
−1
= a(x)φ0 (x) c + φ0 (t) b(t) dt + b(x) = a(x)φ(x) + b(x), (1.9)
x0
where the product rule [Phi18a, Th. 8.5(c)] was used as well. Comparing (1.9) with
(1.6) shows that φ is a solution to (1.6). The computation
φ(x0 ) = φ0 (x0 ) (c + 0) = 1 · c = c (1.10)
verifies that φ satisfies the desired initial condition. It remains to prove uniqueness. To
this end, let ψ : I −→ R be an arbitrary differentiable function that satisfies (1.6) as
well as the initial condition ψ(x0 ) = c. We have to show ψ = φ. Since φ0 > 0, we can
define u := ψ/φ0 and still have to verify
Z x
∀ u(x) = c + φ0 (t)−1 b(t) dt . (1.11)
x∈I x0
We obtain
a φ0 u + b = a ψ + b = ψ ′ = (φ0 u)′ = φ′0 u + φ0 u′ = a φ0 u + φ0 u′ , (1.12)
implying b = φ0 u′ and u′ = φ−1 0 b. Thus, the fundamental theorem of calculus in the

form [Phi18a, (9.50b)], implies
Z x Z x
′
∀ u(x) = u(x0 ) + u (t) dt = c + φ0 (t)−1 b(t) dt , (1.13)
x∈I x0 x0
thereby completing the proof.

Corollary 1.6. Let I ⊆ R be an open interval and let a : I −→ R be continuous.
Moreover, let x0 ∈ I and c ∈ R. Then the homogeneous linear ODE (1.6) (i.e. with
b ≡ 0) has a unique solution φ : I −→ R that satisfies the initial condition y(x0 ) = c.
This unique solution is given by
Z x Rx
a(t) dt
φ(x) = c exp a(t) dt = c e x0 . (1.14)
x0
Proof. One immediately obtains (1.14) by setting b ≡ 0 in in (1.7).

Remark 1.7. The name variation of constants for Th. 1.5 can be understood from
comparing the solution (1.14) of the homogeneous linear ODE with the solution (1.7)
of the general inhomogeneous linear ODE: One obtains (1.7) R x from−1(1.14) by varying the
constant c, i.e. by replacing it with the function x 7→ c + x0 φ0 (t) b(t) dt .
Example 1.8. Consider the ODE
y ′ = 2xy + x3 (1.15)
with initial condition y(0) = c, c ∈ R. Comparing (1.15) with Def. 1.4, we observe we
are facing an inhomogeneous linear ODE with
a : R −→ R, a(x) := 2x, (1.16a)

b : R −→ R, b(x) := x3 . (1.16b)
From Cor. 1.6, we obtain the solution φ0,c to the homogeneous version of (1.15):
Z x
2
φ0,c : R −→ R, φ0,c (x) = c exp a(t) dt = cex . (1.17)
0
The solution to (1.15) is given by (1.7a):
φ : R −→ R,
Z x x
x2 −t2 3 x2 1 2 −t2
φ(x) = e c+ e t dt = e c + − (t + 1)e
0 2 0

2 1 1 2 1 x2 1 2
= ex c + − (x2 + 1)e−x = c + e − (x + 1). (1.18)
2 2 2 2
1.3 Separation of Variables

If the ODE (1.1) has the particular form
y ′ = f (x)g(y), (1.19)
with one-dimensional continuous functions f and g, and g(y) 6= 0, then it can be solved
by a method known as separation of variables:
Theorem 1.9. Let I, J ⊆ R be (bounded or unbounded) open intervals and suppose
that f : I −→ R and g : J −→ R are continuous with g(y) 6= 0 for each y ∈ J. For
each (x0 , y0 ) ∈ I × J, the ODE (1.19) has a unique solution φ that satisfies the initial
condition
φ(x0 ) = y0 . (1.20)
More precisely, given (x0 , y0 ) ∈ I × J, define functions
Z x Z y
dt
F : I −→ R, F (x) := f (t) dt , G : J −→ R, G(y) := . (1.21)
x0 y0 g(t)
Let I ′ ⊆ I be an open interval such that x0 ∈ I ′ and F (I ′ ) ⊆ G(J). Then there exists
a unique solution φ : I ′ −→ R of (1.19) that satisfies (1.20). Moreover, this unique
solution also satisfies
G φ(x) = F (x) for each x ∈ I ′ . (1.22)
Proof. First note that there always exists an open interval I ′ ⊆ I such that x0 ∈ I ′ and
F (I ′ ) ⊆ G(J): According to the fundamental theorem of calculus [Phi18a, Th. 9.19(a)],
G is differentiable with G′ = 1/g. Since g is continuous and nonzero, G is even C 1 . If
G′ (y0 ) = 1/g(y0 ) > 0, then G is strictly increasing on J (due to the intermediate value
theorem [Phi18a, Th. 6.44], g(y0 ) > 0, the continuity of g, and g 6= 0 imply that g > 0
on J). As G(y0 ) = 0, this implies that there is ǫ > 0 such that ] − ǫ, ǫ[⊆ G(J). The
same also holds for G′ (y0 ) = 1/g(y0 ) < 0 by an analogous argument. Again using the
fundamental theorem of calculus [Phi18a, Th. 9.19(a)], F is also differentiable and, in
particular, continuous. Since F (x0 ) = 0, there is δ > 0 such that, for I ′ :=]x0 − δ, x0 + δ[,
one has F (I ′ ) ⊆] − ǫ, ǫ[⊆ G(J) as desired.
We now proceed to show that each solution φ : I ′ −→ R to (1.19) that satisfies (1.20)
must also satisfy (1.22). Since φ is a solution to (1.19),
φ′ (x)
= f (x) for each x ∈ I ′ . (1.23)
g φ(x)
Integrating (1.23) yields

Z x ′ Z x
φ (t)
dt = f (t) dt = F (x) for each x ∈ I ′ . (1.24)
x0 g φ(t) x0
Using the change of variables formula [Phi18a, (9.62)] in the left-hand side of (1.24),
allows us to replace φ(t) by the new integration variable u (note that each solution
φ : I ′ −→ R to (1.19) is in C 1 (I ′ ) since f and g are presumed continuous). Thus, we
obtain from (1.24):
Z φ(x) Z φ(x)
du du
F (x) = = = G φ(x) for each x ∈ I ′ , (1.25)
φ(x0 ) g(u) y0 g(u)
thereby establishing that φ satisfies (1.22).

As already noted above, G must be strictly increasing or strictly decreasing on J. In
consequence, G has a differentiable inverse function H : G(J) −→ J. We obtain that
(1.22) is equivalent to
φ(x) = H F (x) for each x ∈ I ′ . (1.26)
Thus, we have shown that, if φ is a solution to (1.19) and (1.20), then it must equal
H ◦ F , thereby proving uniqueness.
It remains to prove existence, i.e. to verify that φ := H ◦ F does indeed satisfy both
(1.19) and (1.20). Verifying (1.20) is quite simple: φ(x0 ) = H(F (x0 )) = H(0) = y0 . To
see that φ is a solution to (1.19), recall that (1.26) implies (1.22). Differentiating (1.22)
yields, for each x ∈ I ′ ,
(1.22) φ′ (x)
f (x) = F ′ (x) = G′ φ(x) φ′ (x) = , (1.27)
g φ(x)
showing that φ satisfies (1.19).

y
y′ = − on I × J := R+ × R+ (1.28)
x
with the initial condition y(1) = c for some given c ∈ R+ . Introducing functions
1
f : R+ −→ R, f (x) := − , g : R+ −→ R, g(y) := y, (1.29)
x
one sees that Th. 1.9 applies. To compute the solution φ = H ◦ F , we first have to
determine F and G:
Z x Z x
+ dt
F : R −→ R, F (x) = f (t) dt = − = − ln x, (1.30a)
t
Z1 y Z y 1
dt dt y
G : R+ −→ R, G(y) = = = ln . (1.30b)
c g(t) c t c
Here, we can choose I ′ = I = R+ , because F (R+ ) = R = G(R+ ). That means φ is
defined on the entire interval I. The inverse function of G is given by
H : R −→ R+ , H(t) = c et . (1.31)
Finally, we get
c
φ(x) = H F (x) = c e− ln x = . (1.32)
x
—
The advantage of using Th. 1.9 as in the previous example by computing the relevant
functions F , G, and H, is that it is mathematically rigorous. In particular, one can
be sure that one has found the unique solution to the ODE with initial condition.
However, in practice, it is often easier to use the following heuristic (not entirely rigorous)
algorithm. In the end, in most cases, one can easily check by differentiation that the
function found is, indeed, a solution to the ODE with initial condition. The heuristic
algorithm is as follows: Start with the ODE (1.19) written in the form
dy
= f (x)g(y). (1.33a)
dx
Multiply by dx and divide by g(y) (i.e. separate the variables):
dy
= f (x) dx . (1.33b)
g(y)
Integrate: Z Z
dy
= f (x) dx . (1.33c)
g(y)
Change the integration variables and supply the appropriate upper and lower limits for
the integrals (according to the initial condition):
Z y Z x
dt
= f (t) dt . (1.33d)
y0 g(t) x0
Solve this equation for y and set φ(x) := y and check by differentiation that φ is,
indeed, a solution to the ODE. The use of this heuristic algorithm is demonstrated by
the following example:
y ′ = −y 2 on I × J := R × R (1.34)
with the initial condition y(x0 ) = y0 for given values x0 , y0 ∈ R. We manipulate (1.34)
according to the heuristic algorithm described in (1.33) above:
Z Z
dy 2 −2 −2
= −y −y dy = dx − y dy = dx
dx
Z y Z x y
−2 1 1 1
− t dt = dt = [t]xx0 − = x − x0
y0 x0 t y0 y y0
y0
φ(x) = y = . (1.35)
1 + (x − x0 ) y0
Clearly, φ(x0 ) = y0 . Moreover,
′ y02 ′
2
φ (x) = − 2 = − φ (x) , (1.36)
1 + (x − x0 ) y0
i.e. φ does, indeed, provide a solution to (1.34). If y0 = 0, then φ ≡ 0 is defined
on the entire interval I = R. If y0 6= 0, then the denominator of φ(x) has a zero at
x = (x0 y0 − 1)/y0 , and φ is not defined on all of R. In that case, if y0 > 0, then
x0 > (x0 y0 − 1)/y0 = x0 − 1/y0 and the maximal open interval for φ to be defined on
is I ′ =]x0 − 1/y0 , ∞[; if y0 < 0, then x0 < (x0 y0 − 1)/y0 = x0 − 1/y0 and the maximal
open interval for φ to be defined on is I ′ =] − ∞, x0 − 1/y0 [.
1.4 Bernoulli Equations

Definition 1.12. Let I ⊆ R be an open interval, let g, h : I −→ R be continuous
functions, and α ∈ R \ {0, 1}. An ODE of the form
y ′ = g(x)y + h(x)y α (1.37)
is called a Bernoulli differential equation.
—
The cases α = 0 and α = 1 are excluded in Def. 1.12 since, for α = 0, (1.37) has the
form (1.6), i.e. it is a linear ODE; and for α = 1, it can also be written in the form (1.6),
namely with b ≡ 0, i.e. (1.37) is then a homogeneous linear ODE.
If α 6= 0, 1 and h does not vanish identically, then (1.37) is not a linear ODE. However,
the theory of linear ODE of Sec. 1.2 can still be used to determine the solutions to (1.37)
since (1.37) is related to the linear ODE
z ′ = (1 − α)g(x) z + (1 − α)h(x) (1.38)
for the function z.

As it turns out, one encounters different types of situations, depending on the value of
α.
Remark 1.13. (a) If α < 0, then y α is not defined for y = 0, i.e. 0 can not be in the
image of any solution φ = y of (1.37). In particular, each solution to (1.37) must
be either negative on its entire domain or positive on its entire domain.
(b) If α > 0, then y α is also defined for y = 0 and, actually, y ≡ 0 is always a solution
to (1.37) for α > 0.
We now distinguish several cases:

α Is an Odd Integer:
Theorem 1.14. Let α ∈ Z \ {0, 1} be an odd integer and assume the hypotheses of Def.
1.12. Moreover, let x0 ∈ I, c ∈ R \ {0}, and let z : I −→ R be the unique solution
to (1.38) satisfying z(x0 ) = |c|1−α . Then, for each open interval x0 ∈ I ′ ⊆ I such
that z does not have any zeros in I ′ , the Bernoulli ODE (1.37) has a unique solution
φ : I ′ −→ R that satisfies the initial condition y(x0 ) = c; and this unique solution is
given by ( 1
′ z(x) 1−α > 0 if c > 0,
φ : I −→ R, φ(x) = 1 (1.39)
− z(x) 1−α < 0 if c < 0.
If α > 0, then, additionally, φ = y ≡ 0 is the unique solution to (1.37) satisfying
y(x0 ) = 0.
Proof. First, let c 6= 0 and note that Th. 1.5 guarantees the existence of a unique solution
z : I −→ R to the linear ODE (1.38) with initial condition z(x0 ) = z0 := |c|1−α . As
z0 > 0 and z is continuous, there exists an open interval I ′ ⊆ I such that x0 ∈ I ′ and
z > 0 on I ′ (in particular, z(x) 6= 0 for each x ∈ I ′ ).
To prove existence, let φ be defined by (1.39). Then
( 1 1
z(x0 ) 1−α = |c|1−α 1−α = |c| = c if c > 0,
φ(x0 ) = 1 1 (1.40)
− z(x0 ) 1−α = − |c|1−α 1−α = −|c| = c if c < 0,
showing φ satisfies the initial condition. Next, for c > 0, one computes
1 1 (1.38) 1 1
φ′ = z 1−α −1 z ′ = z 1−α −1 (1 − α)g z + (1 − α)h
1−α 1−α
1 α
= z 1−α g + z 1−α h = φ g + φα h, (1.41)
proving φ is a solution to (1.37). For c < 0, the previous computation shows that −φ is
a solution to (1.37). Thus, we obtain
(1.37) α odd
φ′ = −(−φ′ ) = − g (−φ) + h (−φ)α = g φ − h (−φα ) = g φ + h φα , (1.42)
showing φ is, once again, a solution to (1.37).

To verify uniqueness, let ψ : I ′ −→ R be an arbitrary solution to (1.37), satisfying
ψ(x0 ) = c. We claim ψ(x) 6= 0 for each x ∈ I ′ . For α < 0, we already know this from
Rem. 1.13(a). However, here it also holds for α > 0: Otherwise, there exists x̄ ∈ I ′ with
ψ(x̄) = 0. We consider the case x̄ > x0 (the case x̄ < x0 can be treated analogously).
Let
s := inf{x ∈ I ′ : x > x0 , ψ(x) = 0}.
Then the continuity of ψ implies ψ(s) = 0 and, in particular, x0 < s ≤ x̄. Then there
exists ǫ > 0 such that, for I0 :=]x0 − ǫ, s[:
ψ > 0 on I0 for c > 0,

ψ < 0 on I0 for c < 0.
On I0 , define (
ψ 1−α for c > 0,
z̃ :=
(−ψ)1−α for c < 0.
Since
(1.37)
z̃ ′ = (1 − α)ψ −α ψ ′ = (1 − α)ψ −α (g ψ + h ψ α ) = (1 − α)ψ 1−α g + (1 − α) h
= (1 − α) z̃ g + (1 − α) h for c > 0
and
(1.37)
z̃ ′ = −(1 − α)(−ψ)−α ψ ′ = −(1 − α)(−ψ)−α (g ψ + h ψ α )
α odd
= (1 − α)(−ψ)1−α g + (1 − α) h = (1 − α) z̃ g + (1 − α) h for c < 0,
as well as (
ψ 1−α (x0 ) = c1−α for c > 0,
z̃(x0 ) =
(−ψ)1−α (x0 ) = (−c)1−α for c < 0,
z̃ is the unique solution z to (1.38) satisfying z(x0 ) = |c|1−α . In particular, on I0 , z̃ has
to agree with the solution z from the hypothesis of the theorem: z̃ ≡ z on I0 . Since z,
z̃, and ψ all are continuous,
1 1 1
|ψ(s)| = lim |ψ(x)| = lim z̃ 1−α (x) = lim z 1−α (x) = z 1−α (s) 6= 0,
x→s x→s x→s
as z 6= 0 on I ′ . As this is in contradiction to ψ(s) = 0, we have proved ψ 6= 0 and z̃ = z

1 1
on all of I ′ . But then ψ = z 1−α = φ on all of I ′ for c > 0, and ψ = −z 1−α = φ on all of
I ′ for c < 0, proving uniqueness.
For α > 0, y ≡ 0 must be the only solution to (1.37) satisfying y(x0 ) = 0: Indeed,
otherwise, there were s ∈ I and a solution φ to (1.37) satisfying φ(s) = 0 and φ strictly
positive (or negative) on an interval neighboring s, say on I ′ :=]s − ǫ, s[⊆ I. On I ′ , φ
must be given by (1.39) with z as above and limx↑s z(x) = 0. But then, as 1/(1 − α) < 0
for α ≥ 3, limx↑s φ(x) = ±∞, in contradiction to φ(s) = 0.
Example 1.15. Consider the Bernoulli equation

2
y ′ = xy + e−x y 3 , y(0) = −2, (1.43)
2
i.e. with g : R −→ R, g(x) = x, h : R −→ R, h(x) = e−x , and α = 3. We first have to
solve
2
z ′ = (1 − α)g(x) z + (1 − α)h(x) = −2xz − e−x , z(0) = 2−2 = 1/4. (1.44)
According to (1.7), we obtain

Z x
2
z0 (x) = exp −2 t dt = e−x , (1.45a)
0
Z x
−x2 1 t2 −t2 −x2 1
z(x) = e −2 e e dt =e − 2x . (1.45b)
4 0 4
As z has its only zero at x = 1/8, φ is defined on ] − ∞, 1/8[ and (1.39) yields
2
− 12 2 ex /2
φ : ] − ∞, 1/8[−→ R, φ(x) = − z(x) =− √ . (1.46)
1 − 8x
α Is an Even Integer:
This case is basically analogous to the previous case of an odd integer, except that the
formula for the solution is somewhat modified.
Theorem 1.16. Let α ∈ Z \ {0, 1} be an even integer and assume the hypotheses of
Def. 1.12. Moreover, let x0 ∈ I, c ∈ R \ {0}, and let z : I −→ R be the unique solution
to (1.38) satisfying z(x0 ) = |c|1−α sgn c. Then, for each open interval x0 ∈ I ′ ⊆ I such
that z does not have any zeros in I ′ , the Bernoulli ODE (1.37) has a unique solution
φ : I ′ −→ R that satisfies the initial condition y(x0 ) = c; and this unique solution is
given by
1
φ : I ′ −→ R, φ(x) = z(x) 1−α sgn z(x). (1.47)
If α > 0, then, additionally, φ = y ≡ 0 is the unique solution to (1.37) satisfying
y(x0 ) = 0.
Proof. Again start with c 6= 0 and note Th. 1.5 guarantees the existence of a unique
solution z : I −→ R to the linear ODE (1.38), this time with initial condition z(x0 ) =
z0 := |c|1−α sgn c. As z0 6= 0 and z is continuous, there exists an open interval I ′ ⊆ I
such that x0 ∈ I ′ and sgn z = sgn c 6= 0 on I ′ (in particular, z(x) 6= 0 for each x ∈ I ′ ).
To prove existence, let φ be defined by (1.47). Then
1 1
φ(x0 ) = z(x0 ) 1−α sgn z(x0 ) = |c|1−α sgn c 1−α sgn c = |c| sgn c = c, (1.48)
showing φ satisfies the initial condition. Next, one computes

1 1 (1.38) 1 1
φ′ = (sgn z) |z| 1−α −1 (sgn z) z ′ = |z| 1−α −1 (1 − α)g z + (1 − α)h
1−α 1−α
1 α α even
= |z| 1−α g sgn z + |z| 1−α h = φ g + φα h, (1.49)
proving φ is a solution to (1.37).

Apart from slight modifications due to α being even, the proof of uniqueness is conducted
analogous to the uniqueness proof of the previous Th. 1.16: We start by letting ψ :
I ′ −→ R be an arbitrary solution to (1.37), satisfying ψ(x0 ) = c, and claim ψ(x) 6= 0
for each x ∈ I ′ : Otherwise, there exists x̄ ∈ I ′ with ψ(x̄) = 0. Again, we consider the
case x̄ > x0 , and just remark that the case x̄ < x0 can be treated analogously. As in
the proof of Th. 1.16, we let
s := inf{x ∈ I ′ : x > x0 , ψ(x) = 0}
and obtain ψ(s) = 0 and, in particular, x0 < s ≤ x̄. Then there exists ǫ > 0 such that,
for I0 :=]x0 − ǫ, s[:
!
ψ(x) > 0 on I0 for c > 0
∃ ∀ .
ǫ>0 x∈I0 :=]x0 −ǫ,s[ ψ(x) < 0 on I0 for c < 0
On I0 , define
z̃ := |ψ|1−α sgn c.
Since
(1.37)
z̃ ′ = (sgn c) (1 − α)|ψ|−α ψ ′ sgn c = (1 − α)|ψ|−α (g ψ + h ψ α )
α even
= (1 − α)|ψ|1−α g sgn c + (1 − α) h = (1 − α) z̃ g + (1 − α) h
and
z̃(x0 ) = |ψ(x0 )|1−α sgn c = |c|1−α sgn c,
z̃ is the unique solution z to (1.38) satisfying z(x0 ) = |c|1−α sgn c. In particular, on I0 ,
z̃ has to agree with the solution z from the hypothesis of the theorem: z̃ ≡ z on I0 .
Since z, z̃, and ψ all are continuous,
1 1 1
|ψ(s)| = lim |ψ(x)| = lim |z̃(x)| 1−α = lim |z(x)| 1−α = |z(s)| 1−α 6= 0,
x→s x→s x→s
as z 6= 0 on I ′ . As this is in contradiction to ψ(s) = 0, we have proved ψ 6= 0 and z̃ = z

1
on all of I ′ . But then ψ = |z| 1−α sgn z = φ on all of I ′ , proving uniqueness.
The argument from the last paragraph of the proof of Th. 1.16 also works here to
establish that y ≡ 0 must be the only solution to (1.37) satisfying y(x0 ) = 0 for α >
0.
y 1
y′ = + 2, y(0) = 1, (1.50)
1+x y
i.e. with g : ] − 1, ∞[−→ R, g(x) = 1/(1 + x), h : ] − 1, ∞[−→ R, h(x) = 1, and α = −2.
We first have to solve
3z
z ′ = (1 − α)g(x) z + (1 − α)h(x) = + 3, z(0) = 13 = 1. (1.51)
1+x
Z x
3
z0 (x) = exp dt = e3 ln(1+x) = (1 + x)3 , (1.52a)
0 1 + t
Z x x
3 −3 3 1 −2
z(x) = (1 + x) 1 + 3 (1 + t) dt = (1 + x) 1 + 3 − (1 + t) )
0 2 0

5 3 5 3
= (1 + x)3 − (1 + x)−2 = (1 + x)3 − (1 + x). (1.52b)
2 2 2 2
q q q
The set of zeros of z is {−1− 5 , −1, −1+ 5 }, implying φ is defined on ]−1+ 35 , ∞[
3 3
and (1.47) yields

# r " r
3 p 3 5 3
φ : −1 + , ∞ −→ R, φ(x) = 3 z(x) = (1 + x)3 − (1 + x). (1.53)
5 2 2
α < 0 Is Not an Integer:

If α < 0 is not an integer, then y α is not defined for y < 0, i.e. the image of any solution
φ = y of (1.37) must be contained in R+ . This leads to the following result:
Theorem 1.18. Let α ∈ R− \ Z and assume the hypotheses of Def. 1.12. Moreover,
let x0 ∈ I, c ∈ R+ , and let z : I −→ R be the unique solution to (1.38) satisfying
z(x0 ) = c1−α . Then, for each open interval x0 ∈ I ′ ⊆ I such that z does not have any
zeros in I ′ , the Bernoulli ODE (1.37) has a unique solution φ : I ′ −→ R that satisfies
the initial condition y(x0 ) = c; and this unique solution is given by
1
1−α
φ : I ′ −→ R+ , φ(x) = z(x) . (1.54)
Proof. As in the previous theorems, Th. 1.5 guarantees the existence of a unique solution
z : I −→ R to the linear ODE (1.38), this time with initial condition z(x0 ) = z0 := c1−α ;
z0 > 0 implies the existence of an open interval I ′ ⊆ I such that x0 ∈ I ′ and z > 0 on I ′ .
The computations for the existence proof, i.e. for the verification that φ is a solution to
(1.37) that satisfies φ(x0 ) = c are given by the c > 0 case of the proof of Th. 1.14, i.e. by
the first case in (1.40) and (1.41). The uniqueness proof is also the same as in the c > 0
case of the proof of Th. 1.14, except easier since, here, ψ > 0 on I ′ is immediately clear
from Rem. 1.13(a) for every solution ψ : I ′ −→ R to (1.37), satisfying ψ(x0 ) = c.

2 +x 1
y ′ = 4xy + 4e3x y− 2 , y(0) = 4, (1.55)
2 +x
i.e. with g : R −→ R, g(x) = 4x, h : R −→ R, h(x) = 4e3x , and α = −1/2. We first
have to solve
2 +x
z ′ = (1 − α)g(x) z + (1 − α)h(x) = 6xz + 6 e3x , z(0) = 43/2 = 8. (1.56)

Z x
2
z0 (x) = exp 6 t dt = e3x , (1.57a)
0
Z x
3x2 −3t2 3t2 +t 2
z(x) = e 8+6 e e dt = e3x (2 + 6ex ). (1.57b)
0
As z does not have any zeros, φ is defined on R and (1.54) yields

23 2 2
φ : R −→ R, φ(x) = z(x) = e2x (2 + 6ex ) 3 . (1.58)
α > 0 Is Not an Integer:

In this case, we still obtain existence for every initial value problem with c ≥ 0, but, in
general, we lose uniqueness (see Ex. 1.21 below). For c > 0, we still have local uniqueness
in the sense that there exists an open interval I ′ , where the solution is unique, but Ex.
1.21 shows that, for c = 0, there can be uncountably many solutions, even locally.
Theorem 1.20. Let α ∈ R+ \ Z and assume the hypotheses of Def. 1.12. Moreover,
let x0 ∈ I, c ∈ R+ , and let z : I −→ R be the unique solution to (1.38) satisfying
z(x0 ) = c1−α . Then, for each open interval x0 ∈ I ′ ⊆ I such that z does not have any
zeros in I ′ , the Bernoulli ODE (1.37) has a unique solution φ : I ′ −→ R that satisfies
the initial condition y(x0 ) = c; and this unique solution is given by (1.54). For c = 0,
one always has, at least, the solution φ ≡ y ≡ 0.
Proof. Analogous to the proof of the previous theorem, the c > 0 parts of the proof of
Th. 1.14, still prove existence and uniqueness in the present situation for c > 0.

√
y′ = y, y(0) = c ≥ 0, (1.59)
2 SYSTEMS OF ODE AND HIGHER-ORDER ODE 16
i.e. with g : R −→ R, g(x) = 0, h : R −→ R, h(x) = 1, and α = 1/2. There are many

ways to solve this rather simple ODE, but we stick to the Bernoulli equation procedure
and first solve
1 √
z ′ = (1 − α)h(x) = , z(0) = c1−α = c. (1.60)
2
The solution is
x √
z : R −→ R, z(x) = + c. (1.61)
2
√
As z has its only zero at x = −2 c, for c > 0, one obtains the unique solution
√ 2 x √ 2
φ : ] − 2 c, ∞[−→ R, φ(x) = z(x) = + c , (1.62)
2
of (1.59). However, for c = 0, every
(
0 for x ≤ a,
φa : R −→ R, φa (x) := 2
a > 0, (1.63)
(x − a) /4 for a ≤ x,
is a solution to (1.59) (clearly, φa is differentiable at x = a and a solution to (1.59)).
2 Systems of ODE and Higher-Order ODE
2.1 Systems of First-Order ODE

Definition 2.1. In generalization of Def. 1.1 from 1 to n dimensions, n ∈ N, let G ⊆
R × Rn , let f : G −→ Rn be continuous, and call
y ′ = f (x, y), (2.1)
an explicit ODE of first order. A solution to this ODE is a differentiable function
φ : I −→ Rn , (2.2)
defined on a (bounded or unbounded) open interval I ⊆ R satisfying the two conditions
(i) {(x, φ(x)) ∈ I × Rn : x ∈ I} ⊆ G.

(ii) φ′ (x) = f x, φ(x) for each x ∈ I.
Remark 2.2. In the setting of Def. 2.1, using the component functions f1 : G −→ R,
. . . , fn : G −→ R of f , one can interpret (2.1) as the system
y1′ = f1 (x, y1 , . . . , yn ),
y2′ = f2 (x, y1 , . . . , yn ),
.. (2.3)
.
′
yn = fn (x, y1 , . . . , yn )
of 1-dimensional ODE, where a solution φ with component functions φ1 , . . . , φn satisfies

∀ ∀ φ′i (x) = fi x, φ1 (x), . . . , φn (x) . (2.4)
i∈{1,...,n} x∈I
Example 2.3. (a) The most simple types of ODE systems are those, where the 1-
dimensional equations are decoupled, i.e. where the right-hand side of the ith equa-
tion only depends on x and yi . For example, consider

+ 2 2xy1 + x3
f : R × R × R0 −→ R , f (x, y1 , y2 ) = √ , (2.5)
y2
and the initial value problem y ′ = f (x, y), y(0) = (−1/2, 9), which, written in
system form, reads 1
y1′ = 2xy1 + x3 , −2
′ √ y(0) = . (2.6)
y2 = y2 , 9
This is a decoupled combination of Ex. 1.8 and Ex. 1.59 and, thus,
1 2
2 φ1 (x) − 2 (x + 1)
φ : R −→ R , φ(x) = = x
2 , (2.7)
φ2 (x) 2
+ 3
is a solution.
(b) As a first example of a coupled system of ODE consider

3 2 −y2
f : R −→ R , f (x, y1 , y2 ) = . (2.8)
y1
Then, for each r > 0, the initial value problem y ′ = f (x, y), y(0) = (r, 0), which,
written in system form, reads

y1′ = −y2 , r
′ y(0) = , (2.9)
y2 = y1 , 0
has the solution

2 φ1 (x) r cos x
φ : R −→ R , φ(x) = = . (2.10)
φ2 (x) r sin x
It is a remarkable theoretical result that, at least locally, every initial value problem for
the system (2.1) has a solution, provided the function f is at least continuous:
Theorem 2.4 (Peano). If G ⊆ R × Rn is open, n ∈ N, and f : G −→ Rn is continuous,

then, for each (x0 , y0 ) ∈ G, the initial value problem consisting of (2.1) and y(x0 ) = y0
has at least one solution. More precisely, there exists an open interval I ⊆ R and
φ : I −→ Rn such that φ is a solution to (2.1), x0 ∈ I, and φ(x0 ) = y0 .
Proof. See, e.g., [Mar04, Th. 1.9].

Even though Th. 2.4 guarantees the existence of solutions, it is, in general, very difficult
to actually find them. In many cases, this requires numerical approximation methods.
A few such methods will be discussed later.
In Ex. 1.21, we have already seen that, in general, solutions to ODE initial value prob-
lems do not have to be unique. However, under additional hypotheses, they are:
Definition 2.5. Let G ⊆ R×Rn be open, n ∈ N. We call f : G −→ Rn locally Lipschitz
with respect to y if, and only if, for each (x0 , y0 ) ∈ G, there exists an open set U ⊆ G
such that (x0 , y0 ) ∈ U (i.e. U is an open neighborhood of (x0 , y0 )) and f is Lipschitz
continuous with respect to y on U , i.e. if, and only if,

∀ ∃ ∃ ∀ f (x, y) − f (x, ȳ) ≤ Lky − ȳk. (2.11)
(x0 ,y0 )∈G (x0 , y0 ) ∈ U ⊆ G open L≥0 (x,y),(x,ȳ)∈U
Here, the norms on Rn+1 and Rn are arbitrary. If one changes one or both of the norms,
one will, in general, change L, but not the property of f being locally Lipschitz.
Theorem 2.6. If G ⊆ R × Rn is open, n ∈ N, and f : G −→ Rn is locally Lipschitz
with respect to y, then, for each (x0 , y0 ) ∈ G, the solution to the initial value problem
consisting of (2.1) and y(x0 ) = y0 is unique. More precisely, if I ⊆ R is an open
interval and φ, ψ : I −→ Rn are both solutions to (2.1), then φ(x0 ) = ψ(x0 ) for one
x0 ∈ I implies φ(x) = ψ(x) for all x ∈ I:

∃ φ(x0 ) = ψ(x0 ) ⇒ ∀ φ(x) = ψ(x) . (2.12)
x0 ∈I x∈I
Proof. See, e.g., [Mar04, Th. 1.11] or [For08, Th. §12.3].

Remark 2.7. (a) While the local neighborhoods U , where a function locally Lipschitz
(with respect to y) is actually Lipschitz continuous (with respect to y) can be very
small, one can show that a function is locally Lipschitz (with respect to y) if, and
only if, it is Lipschitz continuous (with respect to y) on every compact set K ⊆ G
(see, e.g., [Mar04, Th. 1.14]).
(b) A useful sufficient condition for f : G −→ Rn , G ⊆ R × Rn open, to be locally
Lipschitz with respect to y is f being continuously differentiable with respect to y,
i.e., f is locally Lipschitz with respect to y provided that all partials ∂y1 f, . . . , ∂yn f
exist and are continuous (see, e.g., [Mar04, Th. 1.12] or [For08, Th. §12.2]).
2.2 Higher-Order ODE

We will now also study higher-order ODE, and we will see that they are equivalent to
systems of first-order ODE of the form (2.1) (see Th. 2.9 below).
Definition 2.8. Let G ⊆ R × Rn , n ∈ N, and let f : G −→ R be continuous. An
explicit ODE of nth order is an equation of the form

y (n) = f x, y, y ′ , . . . , y (n−1) . (2.13)
A solution to this ODE is an n times differentiable function
φ : I −→ R, (2.14)
defined on a (bounded or unbounded) open interval I ⊆ R satisfying the following two

conditions:

(i) x, φ(x), φ′ (x), . . . , φ(n−1) (x) ∈ I × Rn : x ∈ I ⊆ G.

(ii) φ(n) (x) = f x, φ(x), φ′ (x), . . . , φ(n−1) (x) for each x ∈ I.
As before, condition (i) is necessary so that one can even formulate condition (ii).
—
As mentioned above, it turns out that the nth-order ODE (1.1) is equivalent to a system
of first-order ODE:
Theorem 2.9. In the situation of Def. 2.8 plus (x0 , y0 ) ∈ G, consider the nth-order
initial value problem

y (n) = f x, y, y ′ , . . . , y (n−1) , (2.15a)
∀ y (i) (x0 ) = y0,i+1 , (2.15b)
i∈{0,...,n−1}
and the first-order initial value problem
y1′ = y2 ,
y2′ = y3 ,
.. (2.16a)
.
′
yn−1 = yn ,
yn′ = f (x, y1 , . . . , yn ),
y(x0 ) = y0 (2.16b)
(note that the unknown function y in (2.15) is R-valued, whereas the unknown function y
in (2.16) is Rn -valued). Then both initial value problems are equivalent in the following
sense:
(a) If φ : I −→ R is a solution to (2.15), then

 
φ
  φ′
Φ : I −→ Rn ,
 
Φ :=  , .. (2.17)
  .
(n−1)
φ
is a solution to (2.16).
(b) If Φ : I −→ Rn is a solution to (2.16), then φ := Φ1 is a solution to (2.15).
Proof. (a): As a solution to (2.15), φ is n times differentiable and Φ is well-defined.

Then (2.15b) implies (2.16b), since
 
φ(x0 )
 φ′ (x0 ) 
  (2.15b)
Φ(x0 ) =  ..  = y0 . (2.18)
 . 
φ(n−1) (x0 )
Next, Def. 2.8(i) implies Def. 2.1(i), since
{(x, Φ(x)) ∈ I × Rn : x ∈ I}
(2.17) Def. 2.8(i)
= x, φ(x), φ′ (x), . . . , φ(n−1) (x) ∈ I × Rn : x ∈ I ⊆ G.
The definition of Φ in (2.17) implies
∀ Φ′i = (φ(i−1) )′ = φ(i) = Φi+1 , (2.19a)

i∈{1,...,n−1}
showing Φ satisfies the first n − 1 equations of (2.16a). As

(2.15a)
Φ′n (x) = (φ(n−1) )′ (x) = φ(n) (x) = f x, φ(x), φ′ (x), . . . , φ(n−1) (x)
∀ (2.19b)
x∈I
= f x, Φ1 (x), . . . , Φn (x) ,
Φ also satisfies the last equation of (2.16a).

(b): As Φ is a solution to (2.16), the first n − 1 equations of (2.16a) imply
φ=Φ1
∀ Φi+1 = Φ′i = φ(i) , (2.20)
i∈{1,...,n−1}
i.e. φ is n times differentiable and Φ has, once again, the form (2.17) (note Φ1 = φ by
the definition of φ). Then, clearly, (2.16b) implies (2.15b) and Def. 2.1(i) implies Def.
2.8(i). Finally, from the last equation of (2.16a), one obtains

∀ φ(n) (x) = Φ′n (x) = f x, Φ1 (x), . . . , Φn (x) = f x, φ(x), φ′ (x), . . . , φ(n−1) (x) ,
x∈I
(2.21)
proving φ satisfies (2.15a).
Example 2.10. The second-order initial value problem
y(0) = 0,
y ′′ = −y, (2.22)
y ′ (0) = r > 0,
is equivalent to the following system of two first-order ODE:

y1′ = y2 , 0
′ y(0) = . (2.23)
y2 = −y1 , r
The solution to (2.23) is

2 Φ1 (x) r sin x
Φ : R −→ R , Φ(x) = = , (2.24)
Φ2 (x) r cos x
and, thus, the solution to (2.22) is
φ : R −→ R, φ(x) = r sin x. (2.25)
One can now easily combine Th. 2.9 with the results of Sec. 2.1 to obtain existence and
uniqueness results for higher-order ODE:
Corollary 2.11. If G ⊆ R × Rn is open, n ∈ N, and f : G −→ Rn is continuous, then,

for each (x0 , y0 ) ∈ G, the initial value problem (2.15) has at least one solution. More
precisely, there exists an open interval I ⊆ R and φ : I −→ R such that φ is a solution
to (2.15).
Proof. If f is continuous, the the right-hand side of (2.16a) is given by the continuous
function  
y2

 y3 

n 
F : G −→ R , F (x, y1 , . . . , yn ) :=  .
. 
(2.26)
 . 

 yn−1 
f (x, y1 , . . . , yn ).
Thus, Th. 2.4 provides a solution Φ : I −→ Rn to (2.16) and, then, Th. 2.9(b) yields
φ := Φ1 to be a solution to (2.15).
Corollary 2.12. If G ⊆ R × Rn is open, n ∈ N, and f : G −→ R is locally Lipschitz

with respect to y, then, for each (x0 , y0 ) ∈ G, the solution to the initial value problem
(2.15) is unique. More precisely, if I ⊆ R is an open interval and φ, ψ : I −→ R are
both solutions to (2.15a) such that (2.15b) holds for some x0 ∈ I with y replaced by φ
and with y replaced by ψ, then φ ≡ ψ on I.
Proof. The right-hand side of (2.16a) is given by the function F in (2.26). Let K be a
compact subset of G and assume f is Lipschitz continuous with respect to y on K with
Lipschitz constant LK ≥ 0. Then, with respect to the max norm on Rn ,
 

y 2 − ȳ 2

 y3 − ȳ3 
 

F (x, y) − F (x, ȳ) ≤
 .. 
max
 . 
∀  
 yn−1 − ȳn−1 
(x,y),(x,ȳ)∈K
LK ky − ȳkmax
max
≤ max{1, LK } ky − ȳkmax .
3 LINEAR ODE 22
Thus, F is locally Lipschitz with respect to y and, if φ, ψ : I −→ R are both solutions

to (2.15a) such that (2.15b) holds for some x0 ∈ I with y replaced by φ and with y
replaced by ψ, then, according to Th. 2.9(a), this gives rise to solutions Φ and Ψ to
(2.16), and Th. 2.6 implies Φ ≡ Ψ on I, implying φ ≡ ψ on I.
Remark 2.13. Of course, one now can proceed to considering systems of nth order
ODE
y (n) = f x, y, y ′ , . . . , y (n−1) , (2.27)
where G ⊆ R × Rn , n ∈ N, and f : G −→ Rm , m ∈ N, is continuous. Then, clearly,
Th. 2.9 applies to each of the m equations making up (2.27), i.e. (2.27) is equivalent to
a system of m · n first-order ODE and Cor. 2.11 and Cor. 2.12 apply, respectively, still
providing existence and uniqueness of solutions under the corresponding hypotheses.
3 Linear ODE
3.1 Existence and Uniqueness

In Sec. 1.2, we saw that the solution of one first-order linear ODE was particularly
simple. One can now combine the ideas from Sec. 1.2 with some linear algebra to obtain
results for systems of linear ODE and, equivalently, for linear ODE of higher order.
Notation 3.1. Let F be a field and m, n ∈ N. The set of all m × n matrices over F is
denoted by M(m, n, F ), and for the set of all n × n matrices, one uses the abbreviation
M(n, F ) := M(n, n, F ). The cases F = R and F = C will be the only ones of interest
to us here.
Definition 3.2. Let I ⊆ R be an open interval, n ∈ N, and let A : I −→ M(n, R) and

b : I −→ Rn be continuous. An ODE of the form
y ′ = A(x)y + b(x) (3.1)
is called a (system of) linear ODE (of first order). It is called homogeneous if, and only
if, b ≡ 0; it is called inhomogeneous if, and only if, it is not homogeneous.
Example 3.3. (a) The ODE in (2.23) is a homogeneous linear ODE with constant A:

y1′ = y2 0 1 y1 0 1
= = Ay, A := . (3.2)
y2′ = −y1 −1 0 y2 −1 0
One easily verifies that the following functions φ, ψ are solutions to (3.2) for each
r > 0:

2 r sin x r cos x
φ, ψ : R −→ R , φ(x) := , ψ(x) := . (3.3)
r cos x −r sin x
3 LINEAR ODE 23
(b) The following is an example of a homogeneous linear ODE with nonconstant A:

y1
y1′ =
!
− y2 1
−1
x x y1
′ y1 2y2 = 1 2 y2
= A(x)y, (3.4a)
y2 = 2 + x2 x
x x !
1
x
−1
A : R+ −→ M(2, R), A(x) := 1 2
. (3.4b)
x2 x
Solutions to (3.4) are given by the following functions φ, ψ:

2
+ 2 x −x2 ln x
φ, ψ : R −→ R , φ(x) := , ψ(x) := . (3.5)
−x x + x ln x
Indeed,

′ 2x x2 /x + x φ1 (x)/x − φ2 (x)
φ (x) = = = ,
−1 x2 /x2 − 2x/x φ1 (x)/x2 + 2φ2 (x)/x

′ −2x ln x − x −x ln x − x − x ln x ψ1 (x)/x − ψ2 (x)
ψ (x) = = = .
1 + ln x + 1 − ln x + 2 + 2 ln x ψ1 (x)/x2 + 2ψ2 (x)/x
Theorem 3.4. Consider the setting of Def. 3.2. Then every initial value problem con-
sisting of the linear ODE (3.1) and y(x0 ) = y0 , x0 ∈ I, y0 ∈ Rn , has a unique solution
φ : I −→ Rn (note that φ is defined on all of I).
Proof. Consider the function

f : I × Rn −→ Rn , f (x, y) := A(x)y + b(x).
The continuity of A and b implies continuity of f : If (ξ, η) ∈ I × Rn and (xk , y k )k∈N is
a sequence in I × Rn with limk∈∞ (xk , y k ) = (ξ, η), then, using an arbitrary norm on Rn
and the corresponding operator norm for A,

A(xk )y k + b(xk ) − (A(ξ)η + b(ξ))

≤ A(xk )y k − A(ξ)y k + A(ξ)y k − A(ξ)η + b(xk ) − b(ξ))

≤ A(xk ) − A(ξ) y k + A(ξ) y k − η + b(xk ) − b(ξ)) → 0 for k → ∞,
showing f is continuous. Then, according to the Peano Th. 2.4, the initial value problem
has a solution on some open interval I ′ with x0 ∈ I ′ ⊆ I. That the solution is defined
on all of I is a consequence of [Mar04, Th. 2.10]. Moreover, if K is a compact subset
of I × Rn , then K is closed and bounded and there exist a, b ∈ I such that (x, y) ∈ K
implies x ∈ [a, b]. Then the continuity of A implies the existence of M ∈ R+ 0 such that
kA(x)k ≤ M for each x ∈ [a, b]. Thus,

∀ A(x)y + b(x) − (A(x)ȳ + b(x)) ≤ A(x) y − ȳ ≤ M y − ȳ ,
(x,y),(x,ȳ)∈K
showing f is locally Lipschitz with respect to y. Thus, Th. 2.6 yields the uniqueness of
the solution to the initial value problem.
3 LINEAR ODE 24
3.2 Fundamental System and Variation of Constants

Remark 3.5. So far, we have only considered ODE over R. However, especially in the
present context of linear ODE, it will be useful to also allow ODE over the complex
numbers C. While functions will now often be allowed to be C-valued, they all remain
defined on a real open interval I ⊆ R. Note that the general results of Sec. 2 also apply
to ODE over C, as C can be identified with R2 in the usual way, and the definition
of derivatives of C-valued functions is compatible with this identification (i.e. f ′ =
(Re f )′ + i (Im f )′ ).
Notation 3.6. We will write K in situations, where we allow K to be R or C.
Remark 3.7. We consider the setting of Def. 3.2 with R replaced by K. Define Li and
Lh to be the respective sets of solutions to (3.1) and its homogeneous version, i.e.
n o
Li := (φ : I −→ Kn ) : φ′ = Aφ + b , (3.6a)
n o
Lh := (φ : I −→ Kn ) : φ′ = Aφ . (3.6b)
If φ ∈ Li is arbitrary, then
Li = φ + Lh = {φ + ψ : ψ ∈ Lh } (3.7)
and Lh is a vector space over K (we will see below that dim Lh = n, in particular, Lh
is finite-dimensional). Thus, one obtains all solutions to the inhomogeneous equation
(3.1) by adding solutions of the homogeneous equation to a particular solution to the
inhomogeneous equation. Note that this is completely analogous to what occurs for
solutions to linear systems of equations in linear algebra.
To verify the above claims, first note that we know Li 6= ∅ and Lh 6= ∅ from Th. 3.4.
Moreover, if λ, µ ∈ K and ψ1 , ψ2 ∈ Lh , then
(λψ1 + µψ2 )′ = λψ1′ + µψ2′ = λ Aψ1 + µ Aψ2 = A(λψ1 + µψ2 ), (3.8)
showing λψ1 + µψ2 ∈ Lh , proving Lh is a subspace of the complex vector space of all
functions from I into C. If φ ∈ Li and ψ ∈ Lh , then
(φ + ψ)′ = φ′ + ψ ′ = Aφ + b + Aψ = A(φ + ψ) + b, (3.9)
showing φ + Lh ⊆ Li . If φ, ϕ ∈ Li , then
(ϕ − φ)′ = Aϕ + b − Aφ − b = A(ϕ − φ), (3.10)
showing ϕ − φ ∈ Lh and ϕ = φ + (ϕ − φ) ∈ φ + Lh , i.e. Li ⊆ φ + Lh .

Theorem 3.8. If I ⊆ R is an open interval, n ∈ N, and A : I −→ M(n, K) is
continuous, then the set Lh defined in (3.6b) constitutes an n-dimensional vector space
over K. Moreover, for each k ∈ N and φ1 , . . . , φk ∈ Lh , the following statements are
equivalent:
3 LINEAR ODE 25
(i) The k functions φ1 , . . . , φk are linearly independent over K.
(ii) There exists x0 ∈ I such that the k vectors φ1 (x0 ), . . . , φk (x0 ) ∈ Kn are linearly
independent over K.
(iii) The k vectors φ1 (x), . . . , φk (x) ∈ Kn are linearly independent over K for every
x ∈ I.
Proof. See, e.g., [For08, Th. §13.2].
Definition 3.9. A basis (φ1 , . . . , φn ) of the n-dimensional vector space Lh over K is

called a fundamental system for the linear ODE (3.1). One then also calls the matrix
 
φ11 . . . φ1n
Φ :=  ... ..  ,

.  (3.11)
φn1 . . . φnn
where the kth column of the matrix consists of the component functions φ1k , . . . , φnk of
φk , k ∈ {1, . . . , n}, a fundamental system or a fundamental matrix solution for (3.1). The
latter term is justified by the observation that Φ : I −→ M(n, K) can be interpreted as
a solution to the matrix-valued ODE
Y ′ = A(x) Y : (3.12)
Indeed,
Φ′ = φ′1 , . . . , φ′n = A(x) φ1 , . . . , A(x) φn = A(x) Φ. (3.13)
Corollary 3.10. Let φ1 , . . . , φn ∈ Lh , n ∈ N, and let Φ be defined as in (3.11). Then

the following statements are equivalent:
(i) Φ is a fundamental system for (3.1).
(ii) There exists x0 ∈ I such that det Φ(x0 ) 6= 0.
(iii) det Φ(x) 6= 0 for every x ∈ I.
Proof. The equivalences are a direct consequence of the equivalences in Th. 3.8.
Example 3.11. (a) For each r > 0, the two solutions in (3.3) provide a fundamental
system for (3.2), since

r sin x r cos x
∀ det = −r2 sin2 x − r2 cos2 x = −r2 6= 0. (3.14)
x∈R r cos x −r sin x
(b) The two solutions in (3.5) provide a fundamental system for (3.4), since
2
x −x2 ln x
∀ + det = x3 + x3 ln x − x3 ln x = x3 6= 0. (3.15)
x∈R −x x + x ln x
3 LINEAR ODE 26
Theorem 3.12 (Variation of Constants). Consider the setting of Def. 3.2 with R re-
placed by K. If Φ : I −→ M(n, K) is a fundamental system for (3.1), then the unique
solution ψ : I −→ Kn of the initial value problem consisting of (3.1) and y(x0 ) = y0 ,
(x0 , y0 ) ∈ I × Kn , is given by
Z x
n −1
ψ : I −→ K , ψ(x) = Φ(x)Φ (x0 ) y0 + Φ(x) Φ−1 (t) b(t) dt . (3.16)
x0
Proof. The initial condition is easily verified:
ψ(x0 ) = Φ(x0 )Φ−1 (x0 ) y0 + 0 = Id y0 = y0 .
To check that ψ satisfies (3.1), one computes, for each x ∈ I,

Z x
′ ′ −1 ′
ψ (x) = Φ (x)Φ (x0 ) y0 + Φ (x) Φ−1 (t) b(t) dt + Φ(x)Φ−1 (x) b(x)
x0
Z x
(3.12) −1
= A(x) Φ(x)Φ (x0 ) y0 + A(x) Φ(x) Φ−1 (t) b(t) dt + b(x)
x0
= A(x) ψ(x) + b(x), (3.17)
thereby establishing the case.
It is an exercise to check that (1.7) is actually a special case of (3.16).

Example 3.13. Consider the initial value problem y(0) = ab ∈ R2 for the following
inhomogeneous version of (3.2):

y1′ = y2 + x, a
y(0) = . (3.18)
y2′ = −y1 + sin x, b
We already know from Ex. 3.11(a) that

sin x cos x
Φ : R −→ M(2, R), Φ(x) := , (3.19)
cos x − sin x
3 LINEAR ODE 27
is a fundamental system for our ODE. Thus, we can apply (3.16) to obtain the solution
ψ : R −→ R2 ,
Z x
−1 a
ψ(x) = Φ(x)Φ (0) + Φ(x) Φ−1 (t) b(t) dt
b 0

sin x cos x − sin 0 − cos 0 a
=−
cos x − sin x − cos 0 sin 0 b
Z x
sin x cos x − sin t − cos t t
− dt
cos x − sin x 0 − cos t sin t sin t
Z x
b sin x + a cos x sin x cos x −t sin t − sin t cos t
= − dt
b cos x − a sin x cos x − sin x 0 −t cos t + (sin t)2
x
b sin x + a cos x sin x cos x − sin t + t cos t + 12 (cos t)2
= −
b cos x − a sin x cos x − sin x − cos t − t sin t + 2t − 12 sin t cos t 0

b sin x + a cos x
=
b cos x − a sin x

sin x cos x − sin x + x cos x − 12 (sin x)2
−
cos x − sin x − cos x − x sin x + x2 − 12 sin x cos x + 1

b sin x + a cos x
=
b cos x − a sin x

(sin x)2 − x cos x sin x + 21 (sin x)3
+
sin x cos x − x(cos x)2 + 21 (sin x)2 cos x

(cos x)2 + x sin x cos x − x2 cos x + 21 sin x(cos x)2 − cos x
+
− sin x cos x − x(sin x)2 + x2 sin x − 12 (sin x)2 cos x + sin x

b sin x + a cos x + 1 + 21 sin x − (1 + x2 ) cos x
= (3.20)
b cos x − a sin x − x + (1 + x2 ) sin x
to (3.18).
3.3 Linear ODE of Higher Order and the Wronskian

In Sec. 2.2, we saw that higher-order ODE are equivalent to systems of first-order ODE.
We can now combine our findings of Sec. 2.2 with the results of Sec. 3.2 to help with
the solution of higher-order linear ODE.
Definition 3.14. Let I ⊆ R be an open interval, n ∈ N. Let b : I −→ K and
a0 , . . . , an−1 : I −→ K be continuous functions. Then a linear ODE of nth order is an
equation of the form
y (n) = an−1 (x)y (n−1) + · · · + a1 (x)y ′ + a0 (x)y + b(x). (3.21)
It is called homogeneous if, and only if, b ≡ 0; it is called inhomogeneous if, and only if,
it is not homogeneous.
3 LINEAR ODE 28
Analogous to (3.6), define the respective sets of solutions
n n−1
X o
Hi := (φ : I −→ K) : φ(n) = b + ak φ(k) , (3.22a)
k=0
n n−1
X o
Hh := (φ : I −→ K) : φ(n) = ak φ(k) . (3.22b)
k=0
Definition 3.15. Let I ⊆ R be an open interval, n ∈ N. For each n-tuple of (n − 1)

times differentiable functions φ1 , . . . , φn : I −→ K, define the Wronskian
 
φ1 (x) ... φn (x)
 φ′1 (x) ... φ′n (x) 
 
W (φ1 , . . . , φn ) : I −→ K, W (φ1 , . . . , φn )(x) := det  .. .. .
 . . 
(n−1) (n−1)
φ1 (x) . . . φn (x)
(3.23)
Theorem 3.16. Consider the setting of Def. 3.14.
(a) If Hi and Hh are the sets defined in (3.22), then Hh is an n-dimensional vector
space over K and, if φ ∈ Hi is arbitrary, then
Hi = φ + Hh . (3.24)
(b) Let φ1 , . . . , φn ∈ Hh . Then the following statements are equivalent:
(i) φ1 , . . . , φn are linearly independent over K (i.e. (φ1 , . . . , φn ) forms a basis of

Hh ).
(ii) There exists x0 ∈ I such that the Wronskian does not vanish:
W (φ1 , . . . , φn )(x0 ) 6= 0.
(iii) The Wronskian never vanishes, i.e. W (φ1 , . . . , φn )(x) 6= 0 for every x ∈ I.
Proof. According to Th. 2.9, (3.21) is equivalent to the first-order linear ODE
    
0 1 0 ... 0 0 y1 0
 0 0 1 ... 0 0    y2   0 
   

 .. ... ... . .
..   ..   .. 
    .
′  . 
y =  + 
 0 0 0 ... 1 0  yn−2   0 
   
 
 0 0 0 ... 0 1  yn−1   0 
a0 (x) a1 (x) a2 (x) . . . an−2 (x) an−1 (x) yn b(x)
=: Ã(x) y + b̃(x). (3.25)

3 LINEAR ODE 29
Define
n o
n ′
Li := (Φ : I −→ K ) : Φ = ÃΦ + b̃ ,
n o
Lh := (Φ : I −→ Kn ) : Φ′ = Ãφ .
(a): Let φ ∈ Hi and define  

φ



 φ′
Φ :=  ...
  .
(n−1)
φ
Then
Th. 2.9(a),(b)
Hh = {Ψ1 : Ψ ∈ Lh }
and
Th. 2.9(a),(b) (3.7)
Hi = {Φ̃1 : Φ̃ ∈ Li } = {Φ̃1 : Φ̃ ∈ Φ + Lh }
= {(Φ + Ψ)1 : Ψ ∈ Lh } = φ + Hh .
As a consequence of Th. 2.9, the map J : Lh −→ Hh , J(Φ) := Φ1 , is a linear isomor-
phism, implying that Hh , like Lh , is an n-dimensional vector space over K.
(l−1)
(b): For φ1 , . . . , φn ∈ Hh , define Φkl := φk for each k, l ∈ {1, . . . , n} and
 
φ1 (x) ... φn (x)
 φ′1 (x) ... φ′n (x) 
 
∀ Φ(x) := (Φ1 (x), . . . , Φn (x)) = (Φkl (x)) =  .. .. 
x∈I  . . 
(n−1) (n−1)
φ1 (x) . . . φn (x)
such that det Φ(x) = W (φ1 , . . . , φn )(x) for each x ∈ I. Since Th. 2.9 yields Φ1 , . . . , Φn ∈
Lh if, and only if, φ1 , . . . , φn ∈ Hh , the equivalences of (b) follow from the equivalences
of Cor. 3.10.
Example 3.17. Consider a0 , a1 : R+ −→ K, a1 (x) := 1/(2x), a0 (x) := −1/(2x2 ), and
y′ y
y ′′ = a1 (x) y ′ + a0 (x) y = − 2. (3.26)
2x 2x
One might be able to guess the solutions
√
φ1 , φ2 : R+ −→ K, φ1 (x) := x, φ2 (x) := x. (3.27)
The Wronskian is
W (φ1 , φ2 ) : R+ −→ K,
√ √ √
x x x √ x (3.28)
W (φ1 , φ2 )(x) = det √ = − x=− < 0,
1 1/(2 x) 2 2
i.e. φ1 and φ2 span Hh according to Th. 3.16(b):
Hh = {c1 φ1 + c2 φ2 : c1 , c2 ∈ K}. (3.29)
3 LINEAR ODE 30
3.4 Constant Coefficients

For 1-dimensional first-order linear ODE, we obtained a solution formula in Th. 1.5 in
terms of integrals (of course, in general, evaluating integrals can still be very difficult,
and one might need effective and efficient numerical methods). In Sec. 3.2, we studied
systems of first-order linear ODE and, in Sec. 3.3, linear ODE of higher order. Unfor-
tunately, there are no general solution formulas for these situations. However, there is
a more satisfying solution theory for the situation of so-called constant coefficients, i.e.
if A in (3.1) and the a0 , . . . , an−1 in (3.21) do not depend on x.
3.4.1 Linear ODE of Higher Order
Definition 3.18. Let I ⊆ R be an open interval, n ∈ N. Let b : I −→ K be continuous

and a0 , . . . , an−1 ∈ K. Then a linear ODE of nth order with constant coefficients is an
equation of the form
y (n) = an−1 y (n−1) + · · · + a1 y ′ + a0 y + b(x). (3.30)
In the present context, it is useful to introduce some notation:
Notation 3.19. Let I ⊆ R be an open interval, n ∈ N. Let Dn (I) := Dn (I, K) denote

the set of all n times differentiable functions f : I −→ K, let P denote the set of all
polynomials over K, Pn := {P ∈ P : deg P ≤ n} (also for n = 0), let
∂x : D1 (I) −→ F(I, K) := {f : I −→ K}, ∂x f := f ′ ,

P
and, for each P ∈ Pn with P (x) = ni=0 ai xi (a0 , . . . , an ∈ K) define the differential
operator
n
X n
X
n
P (∂x ) : D (I) −→ F(I, K), P (∂x )f := ai ∂xi f = ai f (i) . (3.31)
i=0 i=0
Remark 3.20. Using Not. 3.19, the ODE (3.30) can be written concisely as
n−1
X
n
P (∂x )y = b(x), where P (x) := x − ai x i . (3.32)
i=0
The following Prop. 3.21 implies that the differential operator P (∂x ) does not, actually,
depend on the representation of the polynomial P .
Proposition 3.21. Let P, P1 , P2 ∈ P.

3 LINEAR ODE 31
(a) If P = P1 + P2 and n := max{deg P1 , deg P2 }, then

∀ P (∂x )f = P1 (∂x )f + P2 (∂x )f.
f ∈D n (I)
(b) If P = P1 P2 and n := max{deg P, deg P1 , deg P2 }, then

∀n P (∂x )f = P1 (∂x ) P2 (∂x )f .
f ∈D (I)
Proof. See, e.g., [For08, §15, pp. 195–197].

Lemma 3.22. Let λ ∈ K and
f : R −→ K, f (x) := eλx . (3.33)
Then, for each P ∈ P,
P (∂x )f : R −→ K, P (∂x )f (x) = P (λ) eλx . (3.34)
Pn
Proof. There exists n ∈ N0 and a0 , . . . , an ∈ K such that P (x) = i=0 ai xi . One
computes
n
X n
X
P (∂x )f (x) = ai ∂xi eλx = ai λi eλx = eλx P (λ),
i=0 i=0
proving (3.34).
Pn−1
Theorem 3.23. If a0 , . . . , an−1 ∈ K, n ∈ N, and P (x) = xn − i=0 ai xi has the distinct
zeros λ1 , . . . , λn ∈ K (i.e. P (λ1 ) = · · · = P (λn ) = 0), then (φ1 , . . . , φn ), where
∀ φi : I −→ K, φi (x) := eλi x , (3.35)
i∈{1,...,n}
is a basis of the homogeneous solution space

n o
Hh = (φ : I −→ K) : P (∂x )φ = 0 (3.36)
to (3.30) (i.e. to (3.32)).
Proof. It is immediate from (3.34) and P (λi ) = 0 that φi satisfies P (∂x )φ = 0. From
Th. 3.16(a), we already know Hh is an n-dimensional vector space over K. Thus, it
merely remains to compute the Wronskian. One obtains (cf. (3.23)):

1 . . . 1

λ1 . . . λn n−1
(A.2) Y
W (φ1 , . . . , φn )(0) = .. .. = (λi − λj ) 6= 0,
. .
n−1 i,j=0
λ1 . . . λnn−1 i>j
since the λi are all distinct. We have used that the Wronskian, in the present case, turns
out to be a Vandermonde determinant. The formula (A.2) for this type of determinant is
provided and proved in Appendix A. We also used that the determinant of a matrix is the
same as the determinant of its transpose: det A = det At . From W (φ1 , . . . , φn )(0) 6= 0
and Th. 3.16(b), we conclude that (φ1 , . . . , φn ) is a basis of Hh .
3 LINEAR ODE 32
Example 3.24. We consider the third-order linear ODE
y ′′′ = 2y ′′ − y ′ + 2y, (3.37)
which can be written as P (∂x )y = 0 with
P (x) := x3 − 2x2 + x − 2 = (x2 + 1)(x − 2) = (x − i)(x + i)(x − 2), (3.38)
i.e. P has the distinct zeros λ1 = i, λ2 = −i, λ3 = 2. Thus, according to Th. 3.23, the
three functions
φ1 , φ2 , φ3 : R −→ C, φ1 (x) = eix , φ2 (x) = e−ix , φ3 (x) = e2x , (3.39)
form a basis of the C-vector space Hh . If we consider (3.37) as an ODE over R, then
we are interested in a basis of the R-vector space Hh . We can use linear combinations
of φ1 and φ2 to obtain such a basis (cf. Rem. 3.27 below):
eix + e−ix eix − e−ix

ψ1 , ψ2 : R −→ R, ψ1 (x) = = cos x, ψ2 (x) = = sin x. (3.40)
2 2i
As explained in Rem. 3.27 below, since (φ1 , φ2 , φ3 ) are a basis of Hh over C, (ψ1 , ψ2 , φ3 )
are a basis of Hh over R.
—
By working a bit harder, one can generalize Th. 3.23 to the case where P has zeros of
higher multiplicity. We provide this generalization in Th. 3.26 below after recalling the
notion of zeros of higher multiplicity in Rem. and Def. 3.25:
Remark and Definition 3.25. According to the fundamental theorem of algebra, for
every polynomial P ∈ Pn with deg P = n, n ∈ N, there exists r ∈ N with r ≤ n,
k1 , . . . , kr ∈ N with k1 + · · · + kr = n, and distinct numbers λ1 , . . . , λr ∈ C such that
P (x) = (x − λ1 )k1 · · · (x − λr )kr . (3.41)
Clearly, λ1 , . . . , λr are precisely the distinct zeros of P and ki is referred to as the

multiplicity of the zero λi , i = 1, . . . , r.
Pn−1
Theorem 3.26. If a0 , . . . , an−1 ∈ K, n ∈ N, and P (x) = xn − i=0 ai xi has the distinct
zeros λ1 , . . . , λr ∈ K with respective multiplicities k1 , . . . , kr ∈ N, then the set
n o
(φjm : I −→ K) : j ∈ {1, . . . , r}, m ∈ {0, . . . , kj − 1} ,
where
∀ ∀ φjm : I −→ K, φjm (x) := xm eλj x , (3.42)
j∈{1,...,r} m∈{0,...,kj −1}
constitutes a basis of the homogeneous solution space

n o
Hh = (φ : I −→ K) : P (∂x )φ = 0 .
3 LINEAR ODE 33
Proof. See, e.g., [For08, Th. §15.2].

Remark 3.27. If P is a polynomial with only real coefficients and λ ∈ C\R is a nonreal
zero of P , then its complex conjugate λ̄ 6= λ is also a zero of P (see, e.g., [Wal04, Sec.
8.4]). Thus, if, in the situation of Th. 3.26, φjm : I −→ C, φjm (x) = xm eλj x , occurs
in a basis for the C-vector space Hh (with m = 0 in the special case of Th. 3.23), then
φj̃m : I −→ C, φj̃m (x) = xm eλj̃ x , with λj̃ = λ̄j will occur as well. Noting that, for each
x ∈ R and each λ ∈ C,

eλx = ex(Re λ+i Im λ) = ex Re λ cos(x Im λ) + i sin(x Im λ) ,

eλ̄x = ex(Re λ−i Im λ) = ex Re λ cos(x Im λ) − i sin(x Im λ) ,
1 λx
(e + eλ̄x ) = ex Re λ cos(x Im λ),
2
1 λx
(e − eλ̄x ) = ex Re λ sin(x Im λ),
2i
one can define
1
ψjm : I −→ R, ψjm (x) := (φjm (x) + φj̃m (x)) = xm ex Re λj cos(x Im λj ), (3.43a)
2
1
ψj̃m : I −→ R, ψj̃m (x) := (φjm (x) − φj̃m (x)) = xm ex Re λj sin(x Im λj ). (3.43b)
2i
If one replaces each pair φjm , φj̃m in the basis for the C-vector space Hh with the
corresponding pair ψjm , ψj̃m , then one obtains a basis for the R-vector space Hh : This
follows from
1 1
ψjm φjm 2 2 1
=A with A := 1 1 , det A = − 6= 0.
ψj̃m φj̃m 2i
− 2i 2i
Example 3.28. We consider the fourth-order linear ODE
y (4) = −8y ′′ − 16y, (3.44)
which can be written as P (∂x )y = 0 with
P (x) := x4 + 8x2 + 16 = (x2 + 4)2 = (x − 2i)2 (x + 2i)2 , (3.45)
i.e. P has the zeros λ1 = 2i, λ2 = −2i, both with multiplicity 2. Thus, according to Th.
3.26, the four functions
φ10 , φ11 , φ20 , φ21 : R −→ C,
2ix
(3.46)
φ10 (x) = e , φ11 (x) = x e2ix , φ20 (x) = e−2ix , φ21 (x) = x e−2ix ,
form a basis of the C-vector space Hh . If we consider (3.44) as an ODE over R, we can
use (3.43) to obtain the basis (ψ1 , ψ2 , ψ3 , ψ4 ) of the R-vector space Hh , where
ψ10 , ψ11 , ψ20 , ψ21 : R −→ R,
ψ10 (x) = cos(2x), ψ11 (x) = x cos(2x), ψ20 (x) = sin(2x), ψ21 (x) = x sin(2x).
(3.47)
—
3 LINEAR ODE 34
If (3.30) is inhomogeneous, then one can use Th. 3.26 and, if necessary, Rem. 3.27, to
obtain a basis of the homogeneous solution space Hh , then using the equivalence with
systems of first-order linear ODE and variation of constants according to Th. 3.12 to
solve (3.30). However, if the function b in (3.30) is such that the following Th. 3.29
applies, then one can avoid using the above strategy to obtain a particular solution φ
to (3.30) (and, thus, the entire solution space via Hi = φ + Hh ).
Pn−1
Theorem 3.29. Let a0 , . . . , an−1 ∈ K, n ∈ N, and P (x) = xn − i=0 ai xi . Consider
P (∂x )y = Q(x)eµx , Q ∈ P, µ ∈ K. (3.48)
(a) (no resonance): If P (µ) 6= 0 and m := deg(Q) ∈ N0 , then there exists a polynomial
R ∈ P such that deg(R) = m and
φ : R −→ K, φ(x) := R(x) eµx , (3.49)
is a solution to (3.48). Moreover, if Q ≡ 1, then one can choose R ≡ 1/P (µ).
(b) (resonance): If µ is a zero of P with multiplicity k ∈ N and m := deg(Q) ∈ N0 ,

then there exists a solution to (3.48) of the following form:
m+k
X
φ : R −→ K, φ(x) := R(x) eµx , R ∈ P, R(x) = c j xj , ck , . . . , cm+k ∈ K.
j=k
(3.50)
The reason behind the terms no resonance and resonance will be explained in the follow-
ing Example 3.30.
Proof. See, e.g., [For08, Th. §15.3, Th. §15.4].
Example 3.30. Consider the second-order linear ODE
d2 x
+ ω02 x = a cos(ωt), ω0 , ω ∈ R+ , a ∈ R \ {0}, (3.51)
dt2
which can be written as P (∂t )x = a cos(ωt) with
P (t) := t2 + ω02 = (t − iω0 )(t + iω0 ). (3.52)
Note that the unknown function is written as x depending on the variable t (instead of y
depending on x). This is due to the physical interpretation of (3.51), where x represents
the position of a so-called harmonic oscillator at time t, having angular frequency ω0
and being subjected to a periodic external force of angular frequency ω and amplitude
a. We can find a particular solution φ to (3.51) by applying Th. 3.29 to
P (∂t )x = a eiωt . (3.53)
We have to distinguish two cases:

3 LINEAR ODE 35
(a) Case ω 6= ω0 : In this case, one says that the oscillator and the external force are not
in resonance, which explains the term no resonance in Th. 3.29(a). In this case, we
can apply Th. 3.29(a) with µ := iω and Q ≡ a, yielding R ≡ a/P (iω) = a/(ω02 −ω 2 ),
i.e.
a
φ0 : R −→ C, φ0 (t) := R(t) eµt = 2 eiωt , (3.54a)
ω0 − ω 2
is a solution to (3.53) and

a
φ : R −→ R, φ(t) := Re φ0 (t) = cos(ωt), (3.54b)
ω02 − ω2
(b) Case ω = ω0 : In this case, one says that the oscillator and the external force are in
resonance, which explains the term resonance in Th. 3.29(b). In this case, we can
apply Th. 3.29(b) with µ := iω and Q ≡ a, i.e. m = 0, k = 1, yielding R(t) = ct
for some c ∈ C. To determine c, we plug x(t) = R(t) eµt into (3.53):

P (∂t ) ct eiωt = ∂t (c eiωt + ciωt eiωt ) + ω02 ct eiωt
= ciω eiωt + ciω eiωt − cω 2 t eiωt + ω02 ct eiωt
= 2ciω eiωt = a eiωt ⇒ c = a/(2iω). (3.55)
Thus,
a
φ0 : R −→ C, φ0 (t) := t eiωt , (3.56a)
2iω
is a solution to (3.53) and

a
φ : R −→ R, φ(t) := Re φ0 (t) = t sin(ωt), (3.56b)
2ω
3.4.2 Systems of Linear ODE
Definition 3.31. Let I ⊆ R be an open interval, n ∈ N, A ∈ M(n, K) and b : I −→ Kn

be continuous. Then a linear ODE with constant coefficients is an equation of the form
y ′ = A y + b(x), (3.57)
i.e. a linear ODE, where the matrix A does not depend on x.

—
We will see that the solution theory of linear ODE with constant coefficients is related
to the eigenvalues of A. We recall the definition of this notion:
3 LINEAR ODE 36
Definition 3.32. Let n ∈ N and A ∈ M(n, C). Then λ ∈ C is called an eigenvalue of

A if, and only if, there exists 0 6= v ∈ Cn such that
Av = λv. (3.58)
If (3.58) holds, then v 6= 0 is called an eigenvector for the eigenvalue λ.
Theorem 3.33. Let n ∈ N and A ∈ M(n, C).
(a) For each eigenvalue λ ∈ C of A with eigenvector v ∈ Cn \ {0}, the function

φ : I −→ Cn , φ(x) := eλx v, (3.59)
is a solution to the homogeneous version of (3.57).
(b) If {v1 , . . . , vn } is a basis of eigenvectors for Cn , where vi is an eigenvector with
respect to the eigenvalue λi ∈ C of A for each i ∈ {1, . . . , n}, then φ1 , . . . , φn with
∀ φi : I −→ Cn , φi (x) := eλi x vi , (3.60)
i∈{1,...,n}
form a fundamental system for (3.57).
Proof. (a): One computes, for each x ∈ I,

φ′ (x) = λ eλx v = eλx Av = Aφ(x), (3.61)
proving that φ solves the homogeneous version of (3.57).
(b): Without loss of generality, we may consider I = R. We already know from (a)
that each φi is a solution to the homogeneous version of (3.57). Thus, it merely remains
to check that φ1 , . . . , φn are linearly independent. As φ1 (0) = v1 , . . . , φn (0) = vn are
linearly independent by hypothesis, the linear independence of φ1 , . . . , φn is provided by
Th. 3.8.
To proceed, we need a few more notions and results from Linear Algebra:
Theorem 3.34. Let n ∈ N and A ∈ M(n, C). Then the following statements (i) and
(ii) are equivalent:
(i) There exists a basis B of eigenvectors for Cn , i.e. there exist v1 , . . . , vn ∈ Cn and
λ1 , . . . , λn ∈ C such that B = {v1 , . . . , vn } is a basis of Cn and Avi = λi vi for each
i = 1, . . . , n (note that the vi must all be distinct, whereas some (or all) of the λi
may be identical).
(ii) There exists an invertible matrix W ∈ M(n, C) such that
 
λ1 0
W −1 AW = 
 ... 
, (3.62)
0 λn
i.e. A is diagonalizable (if (3.62) holds, then the columns v1 , . . . , vn of W must
actually be the respective eigenvectors to th eigenvalues λ1 , . . . , λn ).
3 LINEAR ODE 37
Proof. See, e.g., [Koe03, Th. 8.3.1].
Unfortunately, not every matrix A ∈ M(n, C) is diagonalizable. However, every A ∈

M(n, C) can at least be transformed into Jordan normal form:
Theorem 3.35 (Jordan Normal Form). Let n ∈ N and A ∈ M(n, C). There exists an
invertible matrix W ∈ M(n, C) such that
B := W −1 AW (3.63)
is in Jordan normal form, i.e. B has block diagonal form

 
B1 0
B=
 ... 
, (3.64)
0 Br
1 ≤ r ≤ n, where each block Bi is a so-called Jordan matrix, i.e.

 
λi 1 0 . . . 0

 λi 1 

Bi = (λi ) or Bi = 
 ... ... 
, (3.65)
 0
 0 λi 1
λi
where λi is an eigenvalue of A.
Proof. See, e.g., [Koe03, Th. 9.5.6] or [Str08, Th. 27.13].
The reason Th. 3.35 regarding the Jordan normal form is useful for solving linear ODE
with constant coefficients is the following result:
Theorem 3.36. Let n ∈ N and A, W ∈ M(n, C), where W is assumed invertible. Then
the following statements (i) and (ii) are equivalent:
(i) φ : I −→ Cn is a solution to y ′ = Ay.
(ii) ψ := W −1 φ : I −→ Cn is a solution to y ′ = W −1 AW y.
Proof. The equivalences
φ′ = Aφ ⇔ W −1 φ′ = W −1 Aφ ⇔ ψ ′ = W −1 AW ψ (3.66)
establish the case.

3 LINEAR ODE 38
Remark 3.37. To obtain a fundamental system for (3.57) with A ∈ M(n, C), it suffices
to obtain a fundamental system for y ′ = By, where B := W −1 AW is in Jordan normal
form and W ∈ M(n, C) is invertible: If φ1 , . . . , φn are linearly independent solutions to
y ′ = By, then A = W BW −1 , Th. 3.36, and W being a linear isomorphism yield that
ψ1 := W φ1 , . . . , ψn := W φn are linearly independent solutions to y ′ = Ay.
Moreover, since B is in block diagonal form with each block being a Jordan matrix
according to (3.64) and (3.65), it actually suffices to solve y ′ = By assuming that
B = λ Id +N, (3.67)
where λ ∈ C and N is a so-called canonical nilpotent matrix, i.e.
 
0 1 0 ... 0

 0 1 

N = 0 (zero matrix) or N = 
 ... ...  , (3.68)
 0
 0 0 1
0
where the case N = 0 is already covered by Th. 3.33. The remaining case is covered by
the following Th. 3.38.
Theorem 3.38. Let λ ∈ C, k ∈ N, k ≥ 2, and assume 0 6= N ∈ M(k, C) is a canonical
nilpotent matrix according to (3.68). Then
 2 xk−2 xk−1

1 x x2 ... (k−2)! (k−1)!
 xk−3 xk−2 
0 1 x ... (k−3)! (k−2)! 
 .. .. 
 ... 
0 0 1 . .

λx  
Φ : I −→ M(k, C), Φ(x) := e  . .. .. ... ... ,
.. (3.69)
 .. . . .
 
 
0 0 0 ... 1 x 
 
0 0 0 ... 0 1
is a fundamental matrix solution to
y ′ = (λ Id +N )y, (3.70)
i.e. the columns of Φ provide k linearly independent solutions to (3.70). Furthermore,
for 0 ∈ I, Φ satisfies the initial condition
Φ(0) = Id . (3.71)
Proof. That (3.71) holds for 0 ∈ I is immediate from (3.69). Since Φ(x) has upper
triangular form with all 1’s on the diagonal, we obtain det Φ(x) = ekλx 6= 0 for each
x ∈ I, showing the columns of Φ are linearly independent. Let φij : I −→ C denote the
ith component function of the jth column of Φ, i.e.
(
xj−i
eλx (j−i)! for i ≤ j,
∀ φij : I −→ C, φij (x) :=
i,j∈{1,...,k} 0 for i > j.
3 LINEAR ODE 39
It remains to show that

(
λφij + φi+1,j for i < k,
∀ φ′ij = (3.72)
i,j∈{1,...,k} λφij for i = k.
One computes,
 xj−i xj−(i+1)
λx
λ e
 (j−i)!
+ eλx (j−(i+1))!
for i < j,
xj−i
∀ φ′ij (x) = λ eλx (j−i)!
+0 for i = j,
i,j∈{1,...,k} 

0 for i > j,
i.e. (3.72) holds, completing the proof.
Example 3.39. For a 2-dimensional real system of linear ODE
y ′ = Ay, A ∈ M(2, R), (3.73)
there exist precisely the following three possibilities (i) – (iii):
(i) The matrix A is diagonalizable with real eigenvalues λ1 , λ2 ∈ R (λ1 = λ2 is

possible), i.e. there is a basis {v1 , v2 } of R2 such that vi is an eigenvector for λi ,
i ∈ {1, 2}. In this case, according to Th. 3.33(b), the two functions
φ1 , φ2 : R −→ K2 , φ1 (x) := eλ1 x v1 , φ2 (x) := eλ2 x v2 , (3.74)
form a fundamental system for (3.73) (over K).
(ii) The matrix A is diagonalizable with two complex conjugate eigenvalues λ1 , λ2 ∈
C \ R, λ2 = λ̄1 . Analogous to (i), one has a basis {v1 , v2 } of C2 such that vi
is an eigenvector for λi , i ∈ {1, 2}, and the two functions in (3.74) still form a
fundamental system for (3.73), however with K replaced by C. However, one can
still obtain a real-valued fundamental system as follows: We have
λ1 = µ + iω, λ2 = µ − iω, where µ ∈ R, ω ∈ R \ {0}. (3.75)
Thus, if Av1 = λ1 v1 with 0 6= v1 = α + iβ, where α, β ∈ R2 , then, letting
v2 := v̄1 = α − iβ, and taking complex conjugates
Av2 = Av̄1 = Av1 = λ1 v1 = λ̄1 v̄1 = λ2 v2 (3.76)
shows v2 is an eigenvector with respect to λ2 . Thus, φ2 = φ̄1 and, similar to the
approach described in Rem. 3.27 above, we can let
1 1
ψ1 2 2 φ1
= 1 1 , (3.77)
ψ2 2i
− 2i φ2
to obtain a fundamental system {ψ1 , ψ2 } for (3.73) over R, where ψ1 , ψ2 : R −→ R,

ψ1 (x) = Re(φ1 (x)) = Re e(µ+iω)x (α + iβ)

= Re eµx cos(ωx) + i sin(ωx) (α + iβ)

= eµx α cos(ωx) − β sin(ωx) , (3.78a)

ψ2 (x) = Im(φ1 (x)) = eµx α sin(ωx) + β cos(ωx) . (3.78b)
4 NUMERICAL METHODS 40
(iii) The matrix A has precisely one eigenvalue λ ∈ R and the corresponding eigenspace
is 1-dimensional. Then there is an invertible matrix W ∈ M(2, R) such that
B := W −1 AW is in (nondiagonal) Jordan normal form, i.e.

−1 λ 1
B = W AW = . (3.79)
0 λ
According to Th. 3.38, the two functions

2 λx 1 λx x
φ1 , φ2 : R −→ K , φ1 (x) := e , φ2 (x) := e , (3.80)
0 1
form a fundamental system for y ′ = By (over K). Thus, according to Th. 3.36,
the two functions
ψ1 , ψ2 : R −→ K2 , ψ1 (x) := W φ1 (x), ψ2 (x) := W φ2 (x), (3.81)
form a fundamental system for (3.73) (over K).
4 Numerical Methods
4.1 Motivation
In the previous sections, we have studied initial value problems for ODE, including
systems of ODE and ODE of higher order. In some simple situations, we were able
to provide solution formulas, e.g. for linear ODE in one dimension (see Th. 1.5) and
for linear ODE of higher order with constant coefficients (see Th. 3.26 and Rem. 3.27).
However, even in these cases, where we have explicit solution formulas, numerical meth-
ods are, in general, necessary for the approximate evaluation of these formulas. For
example, one will, in general, need numerical methods to evaluate the integrals occur-
ring in Th. 1.5 and one will, in general, need numerical methods to approximate the
polynomial roots required for Th. 3.26.
For most ODE, no explicit solution formulas are available, and numerical methods are
required to approximate solutions. There is a vast literature regarding numerical meth-
ods for ODE. Here, we will only be able to briefly touch on the general theory of such
methods and to briefly discuss a few selected concrete methods.
We consider initial value problems
y ′ = f (x, y), (4.1a)

y(ξ) = η, (4.1b)
in the general setting of Def. 2.1, i.e. f : G −→ Rn , G ⊆ R × Rn , n ∈ N, (ξ, η) ∈ G.

This setting allows to include complex-valued ODE via the identification C ∼
= R2 as well
as higher-order ODE via the equivalence of Th. 2.9.
Stable numerical methods virtually always require that small changes of the input data
result in small changes of the output data. The following result Th. 4.1, which generalizes
the uniqueness result of Th. 2.6, shows that initial value problems for ODE are not
hopeless in this regard.
Theorem 4.1 (Continuity in Initial Conditions). If G ⊆ R × Rn is open, n ∈ N, and
f : G −→ Rn is globally L-Lipschitz with respect to y, i.e.

∃ ∀ f (x, y) − f (x, ȳ) ≤ Lky − ȳk, (4.2)
L≥0 (x,y),(x,ȳ)∈G
then the solutions to (4.1a) depend continuously on the initial condition: Let φ, ψ :
I −→ Rn both be solutions to (4.1a) defined on the same open interval I ⊆ R with
ξ ∈ I, then,
∀ φ(x) − ψ(x) ≤ eL(x−ξ) φ(ξ) − ψ(ξ) . (4.3)
x∈I,
x≥ξ
Proof. See, e.g., [Heu09, Sec. 12].
The general idea for the numerical approximation methods is to start at x0 := ξ and
to proceed by discrete steps h0 , h1 , . . . to x1 := x0 + h0 > x0 , x2 := x1 + h1 > x1 , . . . ,
while, simultaneously, starting with u0 := y0 and proceeding to approximations u1 of
y(x1 ), u2 of y(x2 ), . . . This gives rise to the following definition:
Definition 4.2. A (discrete) numerical approximation for the initial value problem
(4.1) is a (finite or infinite) sequence ((xi , ui ))i∈{0,1,... } in the domain G of f such that
(x0 , u0 ) = (ξ, η) and x0 < x1 < . . . Thus, each such numerical approximation comes
with a sequence of stepsizes (hi )i∈{0,1,... } , where
∀ hi := xi+1 − xi . (4.4)
i∈{0,1,... }
(a) We speak of an explicit single-step method for the numerical solution of (4.1) if, and
only if, the sequence (ui )i∈{0,1,... } is given by an explicit recursion of the form
∀ ui+1 = ui + hi ϕ(xi , ui , hi ), (4.5)
i∈{0,1,... }
with a defining function (sometimes also called increment function)

ϕ : Dϕ −→ Rn , where Dϕ ⊆ R × Rn × R+ (4.6)
is a suitable domain.
(b) We speak of an implicit single-step method for the numerical solution of (4.1) if,
and only if, the sequence (ui )i∈{0,1,... } is given by an implicit recursion of the form
∀ ui+1 = ui + hi ϕ(xi , ui , ui+1 , hi ), (4.7)
i∈{0,1,... }
with a defining function or increment function

ϕ : Dϕ −→ Rn , where Dϕ ⊆ R × Rn × Rn × R+ (4.8)
is a suitable domain.
Remark 4.3. Note the crucial difference between explicit methods as defined in Def.
4.2(a) as compared to the implicit methods of Def. 4.2(b): For explicit methods, the
defining function φ does not depend on ui+1 , whereas it does depend on ui+1 for im-
plicit methods. In consequence, explicit methods are computationally much simpler to
execute: For explicit methods, obtaining ui+1 basically means one evaluation of ϕ. For
implicit methods, ui+1 is obtained as a solution to (4.7), which, in general constitutes
a coupled system of n nonlinear equations, and might be very difficult to solve. The
system (4.7) is typically again solved by a suitable numerical method, e.g. Newton’s
method. In particular, obtaining ui+1 usually requires several evaluations of ϕ. How-
ever, depending on the ODE to be solved, the additional computational cost in each
step of an implicit method might be more than compensated by better convergence
properties. This is typically the case for so-called stiff ODE, see Sec. 4.3 below. Thus,
unfortunately, it actually depends on the ODE, which numerical method to prefer; and
even though there are some general guidelines, there is no general selection recipe.
Remark 4.4. In Def. 4.2, we only defined so-called single-step methods, also known
as one-step methods. For a single-step method, the defining function ϕ depends only
on ui (and ui+1 in the implicit case), whereas for so-called multistep methods, ϕ will
also depend on ui−1 , ui−2 , . . . – more precisely, ϕ depends on ui , . . . , ui−m+1 for an m-
step method. Even though multistep methods are important, we will not have time to
consider them in this class. For more on multistep methods, see, e.g., [Pla06, Sec. 8]
and [QSS07, Sec. 11.5, Sec. 11.6].
Remark 4.5. A priori, the methods defined in Def. 4.2 yield discrete approximations
u0 , u1 , . . . to y(x0 ), y(x1 ), . . . , but they do not yield an approximating function u : I −→
Rn of y : I −→ Rn on a suitable open interval x0 ∈ I ⊆ R. To pass from the ui to
a function u requires interpolation. Different interpolation methods are possible, where
interpolation by splines is often used, piecewise linear (i.e. affine) splines being the most
simple possible choice.
4.2 Explicit Methods

4.2.1 Explicit Euler Method
As a first example of an explicit method according to Def. 4.2(a), we consider the explicit
Euler method. For the explicit Euler method, the defining function is
ϕ : G × R+ −→ Rn , ϕ(x, u, h) := f (x, u), (4.9)
i.e., given ξ = x0 < x1 < . . . and hi according to (4.4), the explicit Euler method
consists of the recursion
u0 = η,
∀ ui+1 = ui + hi f (xi , ui ). (4.10)
i∈{0,1,... }
Remark 4.6. In general, there is no guarantee that ui+1 is well-defined by (4.10).

More precisely, ui+1 is well-defined by (4.10) if, and only if, (xi , ui ) ∈ G. Thus, (4.10) is
supposed to mean that the recursion continues as long as (xi , ui ) ∈ G, and it terminates
with ui if (xi , ui ) ∈
/ G.
—
For n = 1, the explicit Euler method can easily be visualized: If y is a solution to

(4.1a), then, at xi , its slope is y ′ (xi ) = f (xi , y(xi )). Thus, in step i + 1, the Euler
method approximates y by the line through (xi , ui ) with slope f (xi , ui ) (using that ui is
supposed to approximate y(xi )). This line is described by
l(x) = ui + (x − xi ) f (xi , ui ), (4.11)
i.e. ui+1 = l(xi+1 ) = ui + hi f (xi , ui ) as in (4.10).
Example 4.7. We consider the initial value problem

y1′ = −y2 , 2
′ y(0) = . (4.12)
y2 = y1 , 0
Here,
3 2 −y2
f : R −→ R , f (x, y1 , y2 ) = , (4.13)
y1
and, from Ex. 2.3(b), we know the exact solution

2 φ1 (x) 2 cos x
φ : R −→ R , φ(x) = = . (4.14)
φ2 (x) 2 sin x
Let us compare with the explicit Euler method for x0 := 0, x1 := π/2, x2 := π,

x3 := 3π/2. The exact values are:
π 0
2 −2 3π 0
φ(0) = , φ = , φ(π) = , φ = . (4.15)
0 2 2 0 2 −2
Noting hi = π/2, the Euler method yields

2
u0 = y(0) = , (4.16a)
0

π 2 π 0 2
u1 = u0 + f (x0 , u01 , u02 ) = + = , (4.16b)
2 0 2 2 π

π 2 π −π 2 − π 2 /2 −2.93
u2 = u1 + f (x1 , u11 , u12 ) = + = ≈ , (4.16c)
2 π 2 2 2π 6.28

π −2.93 π −6.28 −12.8
u3 = u2 + f (x2 , u21 , u22 ) ≈ + = . (4.16d)
2 6.28 2 −2.93 1.67
The following compilation compares the results for hi = h = π/2 and hi = h = π/4:

2 2 2
φ(0) = , u0 (h = π/2) = , u0 (h = π/4) =
0 0 0
π 0
2

0.77

φ = , u1 (h = π/2) = , u2 (h = π/4) =
2 2 π 3.14
(4.17)
−2 −2.93 −4.64
φ(π) = , u2 (h = π/2) = , u4 (h = π/4) =
0 6.28 2.41

3π 0 −12.8 −5.56
φ = , u3 (h = π/2) = , u6 (h = π/4) = .
2 −2 1.67 −6.37
We see that the values produced by the explicit Euler method are (unacceptably) far
from the exact values. One reason is the stepsizes hi being too large to yield accurate
approximations, and decreasing hi does, indeed, yield more accurate approximations as
can already by seen in (4.17). But the bad quality of the ui is also due to the explicit
Euler method not being a very accurate method. This will be made more precise in
the following Sec. 4.2.2 (cf. Th. 4.12), and we will also see examples of more accurate
methods below.
4.2.2 General Error Estimates
We will now introduce notions and results that help to gauge the accuracy (and the
expected error) of explicit single-step methods as defined in Def. 4.2(a) above.
Notation 4.8. Given a real interval [a, b], a < b, (x0 , . . . , xN ) ∈ RN +1 , N ∈ N, is called
a partition of [a, b] if, and only if, a = x0 < x1 < · · · < xN = b. Given such a partition
∆ of [a, b] as above, the number

hmax (∆) := max hi : i ∈ {0, . . . , N − 1} , hi := xi+1 − xi , (4.18)
is called the mesh size of ∆. Moreover, let Π([a, b]) denote the set of all partitions of
[a, b].
Definition 4.9. Consider the initial value problem (4.1) and assume f is such that
(4.1) has a unique solution φ : I −→ Rn on some open interval I ⊆ R. Let b > ξ be
such that [ξ, b] ⊆ I. Moreover, consider an explicit single-step methods as defined in
Def. 4.2(a), given by a defining function
ϕ : Dϕ −→ Rn , Dϕ ⊆ R × Rn × R+ . (4.19)
Then, for each partition ∆ = (x0 , . . . , xN ) of [ξ, b], ϕ defines a recursion
u0 = η,
∀ ui+1 = ui + hi ϕ(xi , ui , hi ), hi := xi+1 − xi . (4.20)
i∈{0,1,... }
(a) We call the method well-defined if, and only if, for each partition ∆ = (x0 , . . . , xN )
of [ξ, b], the recursion (4.20) provides all approximations u0 , . . . , uN (cf. Rem. 4.6),
i.e. if, and only if,
∀ ∀ (xi , ui , hi ) ∈ Dϕ . (4.21)
∆∈Π([ξ,b]) i∈{0,1,...,N −1}
(b) The method is said to have order of convergence p ∈ N if, and only if, it is well-
defined and satisfies
n o
∃ ∀ max kui − φ(xi )k : i ∈ {0, . . . , N } ≤ C hpmax (∆), (4.22)
C≥0 ∆∈Π([ξ,b])
where the quantity on the left-hand side of the inequality in (4.22) is known as the
method’s global truncation error (it depends on both ϕ and ∆).
(c) For each x ∈ [ξ, b] and each h ∈ [0, b − x] such that (x, φ(x), h) ∈ Dϕ , we call
η(x, h) := φ(x) + h ϕ(x, φ(x), h) −φ(x + h) (4.23)
| {z }
cf. (4.20)
the method’s local truncation error at point (x + h, φ(x + h)) with respect to the
stepsize h. The method is said to be consistent of order p ∈ N if, and only if, η(x, h)
is defined for each x ∈ [ξ, b], h ∈ [0, b − x], and
∃ ∀ ∀ kη(x, h)k ≤ C hp+1 . (4.24)
C≥0 x∈[ξ,b] h∈[0,b−x]
Definition 4.10. The function ϕ : Dϕ −→ Rn , n ∈ N, with Dϕ ⊆ R × Rn × R+ is called

(globally) L-Lipschitz with respect to u if, and only if,

∃ ∀ 2
ϕ(x, u, h) − ϕ(x, ū, h) ≤ Lku − ūk. (4.25)
L≥0 (x,u,h),(x,ū,h)∈Dϕ
Theorem 4.11. Consider the setting of Def. 4.9. If the explicit single-step method
under consideration is well-defined, consistent of order p ∈ N, and globally L-Lipschitz
with respect to u, L > 0, then it has order of convergence p. More precisely,
n o
∀ max kui − φ(xi )k : i ∈ {0, . . . , N } ≤ K hpmax (∆), (4.26a)
∆∈Π([ξ,b])
C L(b−ξ)
where K = e −1 , (4.26b)
L
C being the constant given by (4.24).
Proof. See [Pla06, Th. 7.10].

Theorem 4.12. Consider the explicit Euler method, i.e. the method given by the defining
function in (4.9). If the local truncation error η(x, h) is defined for each x ∈ [ξ, b],
h ∈ [0, b − x] (for example, G = Rn+1 is sufficient) and f ∈ C 1 (G, Rn ), then the explicit
Euler method is consistent of order 1. If, in addition, the explicit Euler method is well-
defined (again, G = Rn+1 being sufficient) and f is globally Lipschitz with respect to y,
then the explicit Euler method has order of convergence 1.
Proof. For the proof that the explicit Euler method is consistent of order 1 see, e.g.,
[Pla06, Th. 7.14]. Clearly, if f is globally Lipschitz with respect to y, then ϕ is globally
Lipschitz with respect to u, i.e. the explicit Euler method has order of convergence 1
according to Th. 4.11.
4.2.3 Heun’s Method
For Heun’s method, the defining function is
n 1
ϕ : Dϕ −→ R , ϕ(x, u, h) := f (x, u) + f x + h, u + hf (x, u) , (4.27)
2
i.e., given ξ = x0 < x1 < . . . and hi = xi+1 − xi , Heun’s method consists of the recursion
u0 = η,
hi (4.28)
∀ ui+1 = ui + f (xi , ui ) + f xi + hi , ui + hi f (xi , ui ) .
i∈{0,1,... } 2
As for the explicit Euler method above, we have the issue that, in general, there is no
guarantee that ui+1 is well-defined by (4.28). More precisely, ui+1 is well-defined by
(4.28) if, and only if, (xi , ui ) ∈ G and (xi + hi , ui + hi f (xi , ui )) ∈ G; and (4.28) is
supposed to mean that the recursion continues as long as ui+1 is well-defined, and it
terminates with ui , otherwise.
Example 4.13. We redo Ex. 4.7 with Heun’s method. Thus, once again, we consider

3 2 −y2
f : R −→ R , f (x, y1 , y2 ) = , (4.29)
y1
and x0 := 0, x1 := π/2, x2 := π, x3 := 3π/2; hi = π/2. Heun’s method yields

2
u0 = y(0) = , (4.30a)
0
!
2 π 0 2 π 0 2 π 0 −π
u1 = + + f x1 , + = + +
0 4 2 0 2 2 0 4 2 2

2 − π 2 /4 −0.47
= ≈ , (4.30b)
π π

−0.47 π −π −2.40 −4.82
u2 ≈ + + ≈ , (4.30c)
π 4 −0.47 −5.40 −1.47

−4.82 π 1.47 9.04 3.43
u3 ≈ + + = . (4.30d)
−1.47 4 −4.82 −2.51 −7.23
The following compilation compares the results for hi = h = π/2 and hi = h = π/4:

2 2 2
φ(0) = , u0 (h = π/2) = , u0 (h = π/4) =
0 0 0
π 0
−0.47

−0.28

φ = , u1 (h = π/2) = , u2 (h = π/4) =
2 2 π 2.17
(4.31)
−2 −4.82 −2.32
φ(π) = , u2 (h = π/2) = , u4 (h = π/4) =
0 −1.47 −0.60

3π 0 3.43 0.98
φ = , u3 (h = π/2) = , u6 (h = π/4) = .
2 −2 −7.23 −2.44
While the errors are reduced as compared to the explicit Euler method, the results are
still far too bad to be useful in practise, due to hi being far too large. Here, the example
merely serves illustrative purposes.
Theorem 4.14. If the local truncation error η(x, h) of Heun’s method is defined for each
x ∈ [ξ, b], h ∈ [0, b − x] (for example, G = Rn+1 is sufficient) and f ∈ C 2 (G, Rn ), then
Heun’s method is consistent of order 2. If, in addition, Heun’s method is well-defined
(again, G = Rn+1 being sufficient) and f is globally Lipschitz with respect to y, then
Heun’s method has order of convergence 2.
Proof. For the proof that Heun’s method is consistent of order 2 see, e.g., [Pla06, Ex.
7.18]. Clearly, if f is globally Lipschitz with respect to y, then ϕ is globally Lipschitz
with respect to u, i.e. Heun’s method has order of convergence 2 according to Th.
4.11.
4.2.4 Explicit Runge-Kutta Method
While the Euler method and Heun’s method discussed above, are still relatively simple,
they are also too inaccurate for most practical applications. The Runge-Kutta method
introduced below is slightly more involved, yielding a fourth-order method, often suffi-
ciently good for serious use.
For the explicit Runge-Kutta method, the defining function is
1
ϕ : Dϕ −→ Rn , ϕ(x, u, h) := k1 (x, u) + 2k2 (x, u, h) + 2k3 (x, u, h) + k4 (x, u, h) ,
6
(4.32a)
k1 (x, u) := f (x, u), (4.32b)

h h
k2 (x, u, h) := f x + , u + k1 (x, u) , (4.32c)
2 2

h h
k3 (x, u, h) := f x + , u + k2 (x, u, h) , (4.32d)
2 2

k4 (x, u, h) := f x + h, u + hk3 (x, u, h) , (4.32e)
i.e., given ξ = x0 < x1 < . . . and hi = xi+1 − xi , the explicit Runge-Kutta method
method consists of the recursion
u0 = η,
hi
∀ ui+1 = ui + k1 (xi , ui ) + 2k2 (xi , ui , hi ) + 2k3 (xi , ui , hi ) + k4 (xi , ui , hi ) .
i∈{0,1,... } 6
(4.33)
As for the explicit Euler method and Heun’s method above, we have the issue that, in
general, there is no guarantee that ui+1 is well-defined by (4.33). More precisely, ui+1 is
well-defined by (4.33) if, and only if,

(xi , ui ) ∈ G, xi + hi /2, ui + hi k1 (xi , ui )/2 ∈ G,

xi + hi /2, ui + hi k2 (xi , ui , hi )/2 ∈ G, (4.34)

and xi + hi , ui + hi k3 (xi , ui , hi ) ∈ G,
and (4.33) is supposed to mean that the recursion continues as long as (4.34) holds, and
it terminates with ui , if (4.34) fails.
Example 4.15. As in the previous examples regarding the explicit Euler method and
Heun’s method, respectively, we, once again, consider

3 2 −y2
f : R −→ R , f (x, y1 , y2 ) = , (4.35)
y1

ξ := 0 and η := 20 . The following compilation compares the results for hi = h = π/2
and hi = h = π/4:

2 2 2
φ(0) = , u0 (h = π/2) = , u0 (h = π/4) =
0 0 0
π 0
−1.71

0.01

φ = , u1 (h = π/2) = , u2 (h = π/4) =
2 2 0.07 2.00
(4.36)
−2 −0.10 −1.99
φ(π) = , u2 (h = π/2) = , u4 (h = π/4) =
0 −1.58 0.02

3π 0 1.46 −0.02
φ = , u3 (h = π/2) = , u6 (h = π/4) = .
2 −2 −0.13 −1.98
The improvement over Heun’s method (see (4.31)) is striking, and the values obtained
for hi = h = π/4 can be considered acceptable.
Theorem 4.16. If the local truncation error η(x, h) of the explicit Runge-Kutta method
is defined for each x ∈ [ξ, b], h ∈ [0, b − x] (for example, G = Rn+1 is sufficient)
and f ∈ C 4 (G, Rn ), then the explicit Runge-Kutta method is consistent of order 4. If,
in addition, the explicit Runge-Kutta method is well-defined (again, G = Rn+1 being
sufficient) and f is globally Lipschitz with respect to y, then the explicit Runge-Kutta
method has order of convergence 4.
Proof. For the proof that the explicit Runge-Kutta method is consistent of order 4 see,
e.g., [But03, Sec. 322]. Clearly, if f is globally Lipschitz with respect to y, then ϕ is
globally Lipschitz with respect to u, i.e. the explicit Runge-Kutta method has order of
convergence 4 according to Th. 4.11.
4.3 Implicit Euler Method, Stiff Equations

In Def. 4.2, we defined both explicit and implicit methods, and it was discussed in Rem.
4.3 that explicit methods are computationally easier and should be preferred, unless
the ODE to be solved defies solution by explicit methods. Such ODE, as also already
mentioned in Rem. 4.3 are usually called stiff ODE. A mathematically precise definition
of the notion of stiff ODE seems difficult, and the literature does not seem to have
reached a consensus on this. Here, we will follow [Pla06, Sec. 8.9].
For simplicity, in this section, we will restrict ourselves to open domains G ⊆ R × Rn
such that [ξ, b] × Rn ⊆ G; ξ, b ∈ R, ξ < b. For the convenience of the reader, the initial
value problem, given f : G −→ Rn , is restated:
y ′ = f (x, y), (4.37a)

y(ξ) = η, η ∈ Rn . (4.37b)
Now, typically, (4.37) is stiff if there exists an equilibrium function ψe : I −→ Rn such

that every solution φ : I −→ Rn to (4.37), ξ ∈ I, appoaches ψe rapidly in the sense that
φ(x) ≈ ψe (x) for each x > ξ + ǫ (x ∈ I) with a small ǫ > 0.
The following Def. 4.17 is an attempt at putting the notion of a stiff ODE into a
somewhat more precise form. It is basically reproduced from [Pla06, Sec. 8.9.1]. In
preparation, recall the notion of a scalar product (Def. B.1) and its induced norm (B.6).
Here, we will consider scalar products on Rn , where the Euclidean scalar product is the
most common, but many others also exist (see Ex. B.5).
Definition 4.17. Let h·, ·i : Rn × Rn −→ R denote a fixed scalar product on Rn with

induced norm k · k : Rn −→ R+ 0.
(a) The initial value problem (4.37) is said to satisfy an upper Lipschitz condition with
respect to y and h·, ·i if, and only if, there exists a continuous function M : [ξ, b] −→
R such that

∀ f (x, y) − f (x, ȳ), y − ȳ ≤ M (x) ky − ȳk2 . (4.38)
(x,y),(x,ȳ)∈G
Moreover, the initial value problem is called dissipative if, and only if, (4.38) holds
with M ≤ 0 (in the sense that M (x) ≤ 0 for each x ∈ [ξ, b].
(b) The initial value problem (4.37) is stiff provided that it satisfies the following two
conditions (i) and (ii):
(i) The problem is dissipative or, at least, (4.38) holds with an M that does not
surpass a moderate positive size, say M ≤ 1.
(ii) The expression on the left-hand side of (4.38) divided by ky − ȳk2 can become
strongly negative, i.e.
(
)
f (x, y) − f (x, ȳ), y − ȳ
∀ m(x) := inf 2
: y, ȳ ∈ Rn , y 6= ȳ ≪ 0.
x∈[ξ,b] ky − ȳk
(4.39)
Remark 4.18. Note that the Cauchy-Schwarz inequality (B.2) implies

f (x, y) − f (x, ȳ), y − ȳ f (x, y) − f (x, ȳ)
∀ ≤ . (4.40)
(x,y),(x,ȳ)∈G ky − ȳk2 ky − ȳk
In consequence, (4.39) means that f can be globally L-Lipschitz with respect to y only
with a very large Lipschitz constant L ≥ |m(x)|. In particular, for the explicit Euler
method, Heun’s method, and the explicit Runge-Kutta method, the constant K provided
by (4.26) becomes very large (cf. Theorems 4.12, 4.14, and 4.16), such that reasonable
approximations from these methods can only be expected for exceedingly small stepsizes
h.
Definition 4.19. The implicit Euler method is an implicit single-step method as defined
in Def. 4.2(b), where the defining function is
ϕ : {(x, u, v, h) ∈ R × Rn × Rn × R+ n
0 : (x + h, v) ∈ G} −→ R ,
(4.41)
ϕ(x, u, v, h) := f (x + h, v),
i.e., given ξ = x0 < x1 < . . . and hi = xi+1 − xi , the implicit Euler method consists of
the recursion
u0 = η,
∀ ui+1 = ui + hi f (xi+1 , ui+1 ), (4.42)
i∈{0,1,... }
terminating if the equation for ui+1 does not have a solution.

Example 4.20. Let λ ∈ R. We consider (4.37) with n = 1, ξ = 0, b = 1, and f = fλ ,
where
fλ : R × R −→ R, fλ (x, y) := λy − (1 + λ)e−x , (4.43)
i.e. the initial value problem
y ′ = λy − (1 + λ)e−x , (4.44a)
y(0) = η, η ∈ R. (4.44b)
To compare with (4.38) and (4.40), we compute (using the Euclidean scalar product,
which is just multiplication in one dimension)

2 2
∀ 2
f λ (x, y) − fλ (x, ȳ), y − ȳ = λ(y − ȳ) = λ|y − ȳ| , (4.45)
(x,y),(x,ȳ)∈R
h uh (1) − φ−10 (1) uh (1) − φ−10 (1)

explicit Euler implicit Euler
2−4 = 0.0625 −1.247 · 10−3 1.308 · 10−3

2−6 ≈ 0.0156 −3.174 · 10−4 3.212 · 10−4
2−8 ≈ 0.0039 −7.971 · 10−5 7.994 · 10−5
2−10 ≈ 0.0010 −1.995 · 10−5 1.996 · 10−5
2−12 ≈ 0.0002 −4.989 · 10−6 4.990 · 10−6
Table 1: Numerical rusults to (4.44) with λ = −10, computed for several stepsizes h by
the explicit Euler method and by the implicit Euler method. The problem is not stiff
and both methods perform equally well.
obtaining M (x) = m(x) = λ. Thus Def. 4.17(b) is satisfied for λ ≪ 0. We consider

λ = −10 and λ = −1000, and compute approximations using the explicit Euler method
(4.10) as well as the implicit Euler method (4.42), comparing the results with the exact
solution
φλ : R −→ R, φλ (x) = e−x + (η − 1)eλx (4.46)
(φλ (0) = η is immediate and
φ′λ (x) = −e−x + λ(η − 1)eλx = λe−x + λ(η − 1)eλx − (1 + λ)e−x = λφλ (x) − (1 + λ)e−x ,
shows φλ is a solution to (4.44a)).
h uh (1) − φ−1000 (1) uh (1) − φ−1000 (1)

explicit Euler implicit Euler
2−4 = 0.0625 1.283 · 1024 1.175 · 10−5

2−6 ≈ 0.0156 2.865 · 1069 2.892 · 10−6
2−8 ≈ 0.0039 8.014 · 10112 7.202 · 10−7
2−10 ≈ 0.0010 −1.797 · 10−7 1.799 · 10−7
2−12 ≈ 0.0002 −4.495 · 10−8 4.496 · 10−8
Table 2: Numerical rusults to (4.44) with λ = −1000, computed for several stepsizes h
by the explicit Euler method and by the implicit Euler method. The problem is stiff
and the explicit method is unstable for the largest three values of h.
For the numerical comparison, we fix η := 1. Note
η=1 ⇒ ∀ ∀ φλ (x) = e−x . (4.47)

λ∈R x∈R
Given h > 0, the recursion for the explicit Euler method is
u0 = 1,
∀

ui+1 = ui + h λui − (1 + λ) e−xi . (4.48)
i∈{0,1,... }
For the implicit Euler method, the equation for ui+1 is

ui+1 = ui + h λui+1 − (1 + λ) e−xi+1 , (4.49)
i.e. the recursion is

u0 = 1,
ui − h (1 + λ) e−xi+1 (4.50)
∀ ui+1 = .
i∈{0,1,... } 1 − hλ
We now apply both methods with increasingly small equidistant stepsizes
h = 2−4 , 2−6 , 2−8 , 2−10 , 2−12 ,
recomputing the results of [Pla06, Table 8.3], reproduced in Tables 1 and 2 for the
reader’s convenience, where we write
uh (1) := u1/h ≈ φλ (1) = e−1 . (4.51)
For λ = −10, the problem (4.44) is actually not stiff and the results in Table 1 show
that both the explicit and the implicit Euler method produce reasonable results even
for the largest stepsize h = 2−4 .
For λ = −1000, (4.44) is stiff, and the results in Table 2 show that, while the implicit
Euler method performs reasonably for each value of h, the error of the explicit Euler
method appears to tend to infinity for the first three values of h, but then becomes
reasonably small for h = 2−10 and h = 2−12 (one says the explicit Euler method is
unstable for the larger values of h.
Example 4.21. Let us consider the 2-dimensional example, where n = 2, ξ = 0, and

2 −100 y1 + y2 −100 1
f : R × R −→ R, f (x, y) := Ay = 1 , A := 1 , (4.52)
− 10 y2 0 − 10
i.e. the initial value problem
y1′ = −100 y1 + y2 ,
1
y2′ = − y2 , (4.53a)
10
y(0) = η, η ∈ R2 . (4.53b)
To compare with (4.38) and (4.40), we compute (using the Euclidean scalar product)

f (x, y) − f (x, ȳ), y − ȳ = A(y − ȳ), y − ȳ
1
= −100 (y1 − ȳ1 )2 + (y2 − ȳ2 )(y1 − ȳ1 ) − (y2 − ȳ2 )2
∀ 10 (4.54)
(x,y),(x,ȳ)∈R2 2
1
≤ − 10 |y1 − ȳ1 | − √ |y2 − ȳ2 | ≤ 0 ≤ ky − ȳk22 ,
10
and (
)
f (x, y) − f (x, ȳ), y − ȳ
inf : y, ȳ ∈ Rn , y 6= ȳ = −∞, (4.55)
ky − ȳk2
obtaining M (x) = 0 and m(x) = ∞. Thus, the problem (4.53) is stiff according to Def.
4.17(b).
As in the previous example, the goal is to assess the performance of both the explicit
Euler method (4.10) and the implicit Euler method (4.42), comparing their respective
results with the exact solution. As (4.53a) is a linear ODE with constant coefficients, the
exact solution can be obtained from the results of Sec. 3.4.2. To this end, one observes
A to be diagonalizable,
10
−1 −100 0 1 1 −1 1 − 999
D := W AW = 1 , W = , W = : (4.56)
0 − 10 0 999
10
10
0 999
Indeed,
10
−1 −100 1
−1 1 1 1 − 999 −100 −100 + 999
10
W AW = W 1 999 = 10 999
0 − 10 0 10 0 999 0 − 100

−100 0
= 1 .
0 − 10
As a consequence of Th. 3.33(b),

e−100 x 0
Ψ : R −→ M(2, R), Ψ(x) := , (4.57)
0 e−x/10
constitutes a fundamental matrix solution to y ′ = Dy, and, by Th. 3.36,
Φ : R −→ M(2, R), Φ(x) := W Ψ(x), (4.58)
constitutes a fundamental matrix solution to (4.53a). Thus, by variation of constants

(3.16) with x0 = 0, y0 = η and b(t) ≡ 0, we obtain the following solution to the initial
value problem (4.53):
φ : R −→ R2 , φ(x) = Φ(x)Φ−1 (0) η = W Ψ(x)W −1 Ψ−1 (0) η = W Ψ(x)W −1 η. (4.59)
Introducing the abbreviations

v1 999
v= := W −1 η, w22 := , (4.60)
v2 10
one can rewrite (4.59) as

2 1 1 v1 e−100 x −100 x v1 −x/10 v2
φ : R −→ R , φ(x) = =e +e .
0 w22 v2 e−x/10 0 w22 v2
(4.61)
Given h > 0, the recursion for the explicit Euler method is
u0 = η,
∀ ui+1 = ui + h Aui = (Id +h A)ui , (4.62)
i∈N0
implying
∀ ui = (Id +h A)i u0 = W (Id +h D)i W −1 η. (4.63)
i∈N0
Using
i (1 − 100h)i 0
∀ (Id +h D) = 1
i (4.64)
i∈N0 0 1− 10
h
and (4.60), we obtain
i
v1 1i v2
∀ ui = (1 − 100h) + 1− h . (4.65)
i∈N0 0 10 w22 v2
1
For h > 50
, we have |1 − 100h| > 1 and (4.65) implies
v1 6= 0 ⇒ lim kui k = ∞, (4.66)
i→∞
whereas limx→∞ kφ(x)k = 0, showing the instability of the explicit Euler method for
1
h > 50 . When using (4.62) instead of (4.65), numericallly, due to roundoff errors,
limi→∞ kui k = ∞ will even occur for v1 = 0.
For the implicit Euler method, the equation for ui+1 is
ui+1 = ui + h Aui+1 , (4.67)
i.e. the recursion is
u0 = η,
∀ ui+1 = (Id −hA)−1 ui , (4.68)
i∈N0
implying i i
∀ ui = (Id −hA)−1 u0 = W (Id −h D)−1 W −1 η. (4.69)
i∈N0
Using  
1
i
1+100h
0
−1 i
∀ (Id −h D) =
1
i  (4.70)
i∈N0 0 1
1+ 10 h
and (4.60), we obtain

i i
1 v1 1 v2
∀ ui = + 1 , (4.71)
i∈N0 1 + 100h 0 1 + 10 h w22 v2
5 SHORT INTRODUCTION TO PDE 55
such that
lim kui k = 0, (4.72)
i→∞
independently of the size of h > 0. While (4.72) does not prove the convergence of the
implicit Euler method, a convergence result is provided by the following Th. 4.22.
Theorem 4.22. Assume the initial value problem (4.37) satisfies an upper Lipschitz
condition with respect to y as defined in Def. 4.17(a), however with a constant function
M (x) ≡ M ∈ R. Moreover, assume (4.37) has a unique solution φ defined on [ξ, b],
and consider the implicit Euler method (4.42) for an equidistant partition (x0 , . . . , xN )
of [ξ, b], i.e. h = xi+1 − xi for each i ∈ {0, . . . , N − 1}. Then there exists C ≥ 0 not
depending on the partition and such that the global truncation error can be estimated
n o
max kui − φ(xi )k : i ∈ {0, . . . , N } ≤ K h, where (4.73)
(
C(b − ξ) for M ≤ 0,
K := C M (b−ξ)/(1−HM ) (4.74)
M
e − 1 for M > 0,
where the estimate for M > 0 holds for each h, H ∈ R+ satisfying 0 < h ≤ H < 1/M .
For M ≤ 0, the constant K in (4.73) and (4.74) tends to be of moderate size and it
grows at most linearly with the length of the interval [ξ, b].
Proof. See [Pla06, Th. 8.68].
5 Short Introduction to Partial Differential Equa-

tions (PDE)
5.1 Prototypes
ODE contain only derivatives of one variable, and, correspondingly, the unknown func-
tion y is supposed to be defined on a subset of R. In contrast, PDE contain (partial)
derivatives with respect to different variables, and, correspondingly, the unknown func-
tion u is supposed to be defined on a subset of Rn , n ≥ 2.
A systematic study of PDE, if at all possible, is far beyond the scope of this class, and
we will only be able to briefly consider some prototypes of important classes of PDE.
Notation 5.1. Let Ω ⊆ Rn be open, n ∈ N.
(a) The support of a function f : Ω −→ R, denoted supp(f ), is the closure of the set
of points, where f does not vanish, i.e.
supp(f ) := {x ∈ Ω : f (x) 6= 0} ⊆ Ω. (5.1)

(b) For each k ∈ N0 , let C k (Ω) (resp. C k (Ω)) denote the set of all functions f : Ω −→ R
such that all partials up to order k existTand are continuous on ΩT (resp. extend con-
tinuously to Ω). Moreover, C ∞ (Ω) := k∈N0 C k (Ω), C ∞ (Ω) := k∈N0 C k (Ω). The
corresponding sets C k (Ω, Rm ), C k (Ω, Rm ), C ∞ (Ω, Rm ), C ∞ (Ω, Rm ) of Rm -valued
functions, m ∈ N, are defined analogously – for example, f ∈ C k (Ω, Rm ) if, and
only if, all component functions f1 , . . . , fm of f are in C k (Ω). Let Cck (Ω) denote the
set of all functions f from T C k (Ω) that have a compact support contained in Ω, i.e.
supp(f ) ⊆ Ω; Cc∞ (Ω) := k∈N0 Cck (Ω).
(c) Provided they exist, we denote the gradient, divergence, and Laplacian (also known
as the Laplace operator) of a function f by ∇ f , div f , and ∆f , respectively, where

f ∈ C 1 (Ω) ⇒ ∇ f : Ω −→ Rn , ∇ f (ξ) = ∂1 f (ξ), . . . , ∂n f (ξ) , (5.2a)
X n
f ∈ C 1 (Ω, Rn ) ⇒ div f : Ω −→ R, div f (ξ) = ∂i fi (ξ), (5.2b)
i=1
2 n
f ∈ C (Ω) ⇒ ∆f : Ω −→ R , ∆f (ξ) = div ∇ f (ξ)
Xn
= ∂i ∂i f (ξ). (5.2c)
i=1
Moreover, ∇ f : Ω −→ Rn for f ∈ C 1 (Ω), div f : Ω −→ R for f ∈ C 1 (Ω, Rn ),

∆f : Ω −→ Rn for f ∈ C 2 (Ω).
Definition 5.2. Let Ω ⊆ Rn be an open set, n ∈ N.
(a) Given f : Ω −→ R, the PDE

−∆u = f (5.3)
for the unknown function u : Ω −→ R is called Poisson’s equation. Poisson’s
equation becomes Laplace’s equation if f ≡ 0. Poisson’s equation is the prototype
for so-called elliptic linear PDE. The negative sign in (5.3) might seem arbitrary as
it could be incorporated into f , but this convention is consistent with the convention
for more general elliptic PDE and with the sign conventions of the heat equation
(5.4) and the wave equation (5.5) below.
(b) Given an interval 0 ∈ I ⊆ [0, ∞[ and f : I × Ω −→ R, the PDE
∂t u − ∆u = f (5.4)
for the unknown function u : I × Ω −→ R is called heat equation. Here the

convention is that ∂t refers to the derivative with respect to the first coordinate,
often interpreted as time, whereas ∆ only acts with respect to the remaining n
variables x1 , . . . , xn , often interpreted as the coordinates of n-dimensional space.
The heat equation is the prototype for so-called parabolic linear PDE.
(c) Given an interval 0 ∈ I ⊆ [0, ∞[ and f : I × Ω −→ R, the PDE
∂t ∂t u − ∆u = f (5.5)
for the unknown function u : I × Ω −→ R is called wave equation. As for the

heat equation (5.4), the convention is that ∂t refers to the derivative with respect
to the first coordinate, often interpreted as time, whereas ∆ only acts with respect
to the remaining n variables x1 , . . . , xn , often interpreted as the coordinates of n-
dimensional space. The heat equation is the prototype for so-called hyperbolic linear
PDE.
Each of the above PDE is called homogeneous if, and only if, the right-hand side function
f vanishes, i.e. f ≡ 0. They are called inhomogeneous if, and only if, they are not
homogeneous.
—
Here, we will only have time to look at Laplace’s and Poisson’s equation in more detail,
which we do in the following Sec. 5.2. For similar consideration regarding the heat
equation and the wave equation, we refer to [Eva98, Sec. 2.3] and [Eva98, Sec. 2.4],
respectively.
5.2 Laplace’s and Poisson’s Equation

The subject of the present section are Poisson’s and Laplace’s equation as defined in
Def. 5.2(a).
5.2.1 The Fundamental Solution
Note that for n = 1, (5.3) is actually an ODE that can be solved simply by integrating f
twice. We will therefore restrict ourselves to considering (5.3) for n ≥ 2 in the following.
Notation 5.3. For each n ∈ N, let
Z
αn := voln (B1 (0)) := 1 (5.6)
B1 (0)
denote the n-dimensional volume of the n-dimensional unit ball B1 (0) ⊆ Rn .

Definition 5.4. A function u is called harmonic if, and only if, it is a solution to
Laplace’s equation ∆u = 0.
Definition 5.5. Let n ∈ N, n ≥ 2. Then the function
( v
1 u n
− 2π ln kxk2 for n = 2, uX
Φ : Rn \ {0} −→ R, Φ(x) := 1 1 kxk2 := t x2i , (5.7)
n(n−2)αn kxk2n−2
for n ≥ 3, i=1
is called the fundamental solution of Poisson’s equation (5.3) (or of Laplace’s equation
∆u = 0).
—
A description of how one can find the solution Φ if one does not know it already, can
be found in [Eva98, Sec. 2.2.1.a]. Here, in the following Prop. 5.6, we will merely verify
that Φ is, indeed, harmonic.
Proposition 5.6. For each n ∈ N, n ≥ 2, the fundamental solution Φ of Laplace’s
equation as defined in (5.7) is, indeed, harmonic, i.e.
∆Φ = 0 on Rn \ {0}. (5.8)
Proof. While the constants used in the definition of Φ are useful in the context of Th. 5.7
below, we can ignore them for the purposes of the present proof. To keep notation simple,
we still write Φ below, even though the Φ’s below will defer from the corresponding Φ’s
from (5.7) by a constant.
We start with n = 2: For i ∈ {1, 2} and x = (x1 , x2 ) ∈ R2 \ {0}, we have
p
∂ ln x21 + x22 xi
∂i Φ(x) = = 2 , (5.9a)
∂xi x1 + x22
x2 + x2 − 2x2
∂i ∂i Φ(x) = 1 2 2 2 2 i , (5.9b)
(x1 + x2 )
2x2 + 2x22 − 2x21 − 2x22
∆Φ(x) = ∂1 ∂1 Φ(x) + ∂2 ∂2 Φ(x) = 1 = 0. (5.9c)
(x21 + x22 )2
Now let n = 3: For i ∈ {1, . . . , n} and x ∈ Rn \ {0}, we have
P 1− n n
!− n2
∂ ( ni=1 x2i ) 2 X
∂i Φ(x) = = (2 − n) xi x2i , (5.10a)
∂xi i=1
 ! n !− n2 −1 
n − 2 n
X X
∂i ∂i Φ(x) = (2 − n)  x2i − n x2i x2i , (5.10b)
i=1 i=1
n
!− n2 n
X X n x2
∆Φ(x) = (2 − n) x2i 1 − Pn i 2 = 0, (5.10c)
i=1 i=1 i=1 xi
completing the proof.
The reason the harmonic function Φ of (5.7) is referred to as the fundamental solution
of Poisson’s equation lies in the fact that it can be used to formulate a solution formula
for Poisson’s equation with a general function f , assuming f satisfies certain regularity
requirements. The following Th. 5.7 is quoted from [Eva98, Sec. 2.2], where it is proved
for f ∈ Cc2 (Rn ). The assumption on f is made mainly for simplicity and, in the literature,
one finds variants of Th. 5.7 requiring far less regularity on f .
Theorem 5.7. Let n ∈ N, n ≥ 2, f ∈ Cc2 (Rn ), and Φ as in (5.7).
(a) For each x ∈ Rn , the function y 7→ Φ(x − y)f (y) (with arbitrarily defined value for
y = x) is integrable over Rn .
(b) The function
u : Rn −→ R,
 R
Z − 1 n ln kx − yk2 f (y) dy for n = 2,
2π R
u(x) := Φ(x − y)f (y) dy = 1
R f (y) (5.11)
R n  n(n−2)α
n Rn n−2 dy
kx−yk2
for n ≥ 3,
is a solution to Poisson’s equation −∆u = f on Ω := Rn .
Proof. (a): By hypothesis, f is a continuous function with compact support, imply-

ing f is bounded and zero outside a compact (in particular, bounded) set. Thus, it
merely remains to verify that Φ, as defined in (5.7), is integrable over each ball BR (0),
R > 0. Using the change of variables formula [Phi18b, (4.58)] and n-dimensional polar
coordinates as in [Wal02, Sec. 7.19.4], we obtain, for n = 2,
Z Z Z R
ln kxk2
Φ(x) dx = − dx = − lim r ln r dr
BR (0) BR (0) 2π ρ→0 ρ
R
2 ln r 1 2 ln R 1 1
= − lim r − = −R − − ∈ R, (5.12a)
ρ→0 2 4 ρ 2 4 4
and for n ≥ 3,
Z Z Z R
1 dx rn−1
Φ(x) dx = − n−2 = − lim dr
BR (0) BR (0) n(n − 2)αn kxk2 ρ→0 ρ (n − 2) r n−2
R
r2 R2
= − lim =− ∈ R, (5.12b)
ρ→0 2 (n − 2) 2 (n − 2)
ρ
completing the proof of integrability.

(b): See, e.g., [Eva98, Sec. 2.2, Th. 1].
5.2.2 Boundary Conditions
While (5.7) and (5.11) provide particular solutions to Laplace’s and Poisson’s equation,
respectively, both equations typically allow infinitely many different solutions. This is,
indeed, not really surprising, since we encountered the same phenomenon for ODE. One
calls a problem well-posed if it admits a unique solution (one usually also requires this
solution to depend continuously on the input data, but here we only consider existence
and uniqueness). For a large class of ODE, we were able to obtain well-posedness by
supplementing the ODE with an initial condition (cf. Th. 2.6). For Poisson’s equation,
one can often obtain well-posedness by supplementing the PDE by so-called boundary
conditions. The following are the most commonly considered boundary conditions:
Definition 5.8. A boundary condition for Poisson’s equation (5.3) on an open set Ω ⊆
Rn , n ≥ 2, prescribes conditions for the unknown function f : Ω −→ R to be satisfied
on ∂Ω, i.e. on the boundary of Ω.
(a) A Dirichlet boundary condition prescribes the value of u on ∂Ω, i.e.
u=g on ∂Ω, (5.13)
where g : ∂Ω −→ R is some given function.

(b) A Neumann boundary condition prescribes the value of the normal derivative of u
on ∂Ω, i.e.
n · ∇ u = g on ∂Ω, (5.14)
where n : ∂Ω −→ Rn is the outward unit normal vector with respect to Ω and
g : ∂Ω −→ R is some given function. A Neumann boundary condition only makes
sense if Ω is sufficiently regular such that the outward unit normal vector with
respect to Ω exists everywhere on ∂Ω (or at least almost everywhere with respect
to the n − 1-dimensional surface measure on ∂Ω).
(c) A Robin boundary condition can be seen as a linear combination of Dirichlet and
Neumann boundary conditions. One requires
αu + β n · ∇ u = g on ∂Ω, (5.15)
where 0 6= α, β ∈ R, n : ∂Ω −→ Rn is the outward unit normal vector with respect

to Ω and g : ∂Ω −→ R is some given function. The remark from (b) regarding the
required regularity of Ω applies here as well.
In each case, the combination of the PDE and the boundary condition is called a bound-
ary value problem.
—
We will only study Dirichlet boundary conditions below.
5.2.3 Green’s Functions
Definition 5.9. Let Ω ⊆ Rn be an open set, n ∈ N, n ≥ 2, and let Φ be the fundamental

solution as defined in (5.7).
(a) For each x ∈ Ω, we call φx : Ω −→ R a corrector function for Ω if, and only if, φx
satisfies the following two conditions (5.16a) and (5.16b):
∆φx (y) = 0 for each y ∈ Ω, (5.16a)

φx (y) = Φ(y − x) for each y ∈ ∂Ω. (5.16b)
(b) The function

G : (x, y) ∈ Ω2 : x 6= y −→ R, G(x, y) := Φ(y − x) − φx (y), (5.17)
is called a Green’s function for Ω if, and only if, the φx , x ∈ Ω, are all corrector
functions for Ω as defined in (a).
In many situations, one can now represent solutions to Poisson’s equation as integrals
over Green’s functions as stated in Th. 5.12 below. However, in several respects, this can
only be considered a partial solution to the problem: (a) Green’s functions are typically
very difficult to find, (b) the resulting integrals are typically very difficult to carry out,
(c) it remains to verify that the function given by the representation formula is, indeed,
a solution in the concrete situation at hand.
One of the difficulties regarding PDE is that results often depend on the regularity of the
boundary of the involved sets – Ω in our case. The following Def. 5.10 for C k -boundaries
is taken from [Eva98, Sec. C.1]:
Definition 5.10. Let Ω ⊆ Rn be an open set, n ∈ N, n ≥ 2. The boundary ∂Ω is said
to be C k , k ∈ N, if, and only if, for each x0 ∈ ∂Ω, there exists r > 0 and γ ∈ C k (Rn−1 )
such that – upon relabeling and reorienting the coordinate axes if necessary – we have

U ∩ Br (x0 ) = x ∈ Br (x0 ) : xn > γ(x1 , . . . , xn−1 ) . (5.18)
Remark 5.11. Let Ω ⊆ Rn be an open set, n ∈ N, n ≥ 2. If ∂Ω is C 1 , then the outward

unit normal ν : ∂Ω −→ Rn is continuous. The details are bit tedious. In [Alt06, Sec.
A6.2], ν is constructed for Lipschitz boundaries, where continuity of ν can, in general,
not be expected. However, the construction does show continuity for C 1 -boundaries.
Theorem 5.12. If n ∈ N, n ≥ 2, Ω ⊆ Rn is open, bounded, and ∂Ω is C 1 , f ∈ C(Ω),
g ∈ C(∂Ω), G is a Green’s function for Ω, and u ∈ C 2 (Ω) is a solution to Poisson’s
equation (5.3) with Dirichlet boundary condition (5.13), then
Z Z
∂G
∀ u(x) = − g(y) (x, y) dy + f (y)G(x, y) dy , (5.19)
x∈Ω ∂Ω ∂ν Ω
where
∂G
(x, y) := ν(x, y) · ∇ G(x, y), (5.20)
∂ν
is the directional derivative of G in the direction of the outward unit normal ν(x, y).
Proof. See, e.g., [Eva98, Sec. 2.2, Th. 12].
5.2.4 Poisson’s Formula for a Ball
It is shown in [Eva98, Sec. 2.2.4.c] that a Green’s function for the (open) unit ball B1 (0)
in Rn , n ≥ 2, is given by

G : (x, y) ∈ B1 (0)2 : 0 6= x 6= y −→ R, G(x, y) := Φ(y − x) − Φ y kxk2 − x/kxk2 ,
(5.21)
where Φ is again the fundamental solution as defined in (5.7) (G can be continuously
extended to points (0, y) with y 6= 0, but, for (5.19) the values G(0, y) are irrelevant).
This leads to (5.22), known as Poisson’s formula for a ball:
Theorem 5.13. For each R > 0, let BR (0) denote the (open) unit ball in Rn , n ≥ 2.
Given g ∈ C(∂BR (0)), define
Z
R2 − kxk22 g(y)
u : BR (0) −→ R, u(x) := n
dy . (5.22)
nR αn ∂BR (0) kx − yk2
Then u ∈ C ∞ (BR (0)) and u is a solution to Laplace’s equation with Dirichlet boundary
condition (5.13) in the sense that
∆u = 0 on BR (0) and, (5.23a)

∀ lim u(y) = g(x). (5.23b)
x∈∂BR (0) y→x
Proof. This is stated as [Eva98, Sec. 2.2, Th. 15], but the proof is left as an exercise.
For a proof, see, e.g., [RR04, Th. 4.14].
5.2.5 Poisson’s Formula for a Half-Space
Even though, in Th. 5.12, Ω is assumed to be bounded, (5.19) can sometimes also be
used to obtain solutions for unbounded sets Ω. For example, it is shown in [Eva98, Sec.
2.2.4.b] that a Green’s function for the half-space
Rn+ := {(x1 , . . . , xn : xn > 0} (n ∈ N) (5.24)
is given by

G : (x, y) ∈ (Rn+ )2 : x 6= y −→ R,
(5.25)
G(x, y) := Φ(y − x) − Φ(y − x̃), where x̃ := (x1 , . . . , xn−1 , xn ),
and then (5.19) is used to obtain Poisson’s formula (5.26) for a half-space:
Theorem 5.14. For each n ∈ N, n ≥ 2, and each bounded g ∈ C(Rn−1 ), define

Z
n 2xn g(y)
u : R+ −→ R, u(x) := dy . (5.26)
n αn ∂Rn+ kx − ykn2
Then u ∈ C ∞ (Rn+ ) is bounded and u is a solution to Laplace’s equation with Dirichlet

boundary condition (5.13) in the sense that
∆u = 0 on Rn+ and, (5.27a)

∀ n lim u(y) = g(x). (5.27b)
x∈∂R+ y→x
Proof. See, e.g., [Eva98, Sec. 2.2, Th. 14].

Example 5.15. For each R > 0, let


0
 for t ≤ −R,
g : R −→ R, g(t) := t2 − R2 for −R ≤ t ≤ R, (5.28)


0 for R ≤ t,
and consider Laplace’s equation with Dirichlet boundary conditions on the half-space
∆u = 0 on R2+ , (5.29a)
u = g on ∂R2+ ∼= R. (5.29b)
Specializing (5.26) to n = 2 yields
Z ∞
y g(t)
u: R2+ −→ R, u(x, y) := dt , (5.30)
π −∞ (x − t)2 + y 2
where (x, y) in (5.30) plays the role of x = (x1 , x2 ) of (5.26), and t in (5.30) plays the
role of y = (y1 , 0) of (5.26). Thus, for each (x, y) ∈ R2+ ,
Z
y R t2 − R 2
u(x, y) = dt
π −R (x − t)2 + y 2

y −R2 + x2 − y 2 t−x 2 2
t=R
= t+ arctan + x ln (t − x) + y
π y y t=−R
2 2 2

y −R + x − y R−x R+x
= 2R + arctan + arctan
π y y y
!
(R − x)2 + y 2
+ x ln . (5.31)
(R + x)2 + y 2
Let us check directly that u, indeed, satisfies (5.27). We start with (5.27b):
 2 2
−R +ξ π

 π 2
− π2 = 0 = g(ξ) for ξ < −R,


0 = g(ξ)

 for ξ = −R,
2 2
lim u(x, y) = −R π+ξ π2 + π2 = ξ 2 − R2 = g(ξ) for −R < ξ < R, (5.32)
(x,y)→(ξ,0) 



 0 = g(ξ) for ξ = R,
 −R2 +ξ2 − π + π = 0 = g(ξ)

for R < ξ.
π 2 2
It remains to verify (5.27a). To this end, we calculate

y 2x R−x R+x
∂x u(x, y) = arctan + arctan
π y y y

2 2 2 1 1
+ (R − x + y ) −
y 2 + (R − x)2 y 2 + (R + x)2
!
(R − x)2 + y 2 2x(R − x) 2x(R + x)
+ ln − −
(R + x)2 + y 2 y 2 + (R − x)2 y 2 + (R + x)2
!
y 2x R−x R+x (R − x)2 + y 2
= arctan + arctan + ln , (5.33a)
π y y y (R + x)2 + y 2
A THE VANDERMONDE DETERMINANT 64
!
1 R−x R+x (R − x)2 + y 2
∂y u(x, y) = 4R − 2y arctan + arctan + x ln ,
π y y (R + x)2 + y 2
(5.33b)
2
∂x ∂x u(x, y) =
π (R2 + 2Rx + + x2 y 2 ) (R2
− 2Rx + x2 + y 2 )
R−x R+x

4 2 2 2 2 2 2
· R − 2R (x − y ) + (x + y ) arctan + arctan
y y
!

− 2Ry R2 + x2 + y 2 , (5.33c)
∂y ∂y u(x, y) = −∂x ∂x u(x, y), (5.33d)
where (5.33d) is proved by differentiating (5.33b) with respect to y and rearranging

terms. Thus,
∀ 2 ∆u(x, y) = ∂x ∂x u(x, y) + ∂y ∂y u(x, y) = 0, (5.34)
(x,y)∈R+
concluding the proof of (5.27a).
A The Vandermonde Determinant

Theorem A.1. Let n ∈ N and λ0 , λ1 , . . . , λn ∈ C. Moreover, let
 
1 λ0 . . . λn0
 1 λ1 . . .
 λn1 

V :=  .. ..  (A.1)
. .
1 λn . . . λnn
be the corresponding Vandermonde matrix. Then its determinant, the so-called Vander-
monde determinant is given by
n
Y
det(V ) = (λi − λj ). (A.2)
i,j=0
i>j
Proof. The proof can be conducted by induction with respect to n: For n = 1, we have
1
1 λ0 Y
det(V ) = = λ1 − λ0 = (λi − λj ), (A.3)
1 λ1
i,j=0
i>j
showing (A.2) holds for n = 1. Now let n > l. We know from Linear Algebra that the
value of a determinant does not change if we add a multiple of a column to a different
B SCALAR PRODUCTS 65
column. Adding the (−λ0 )-fold of the nth column to the (n + 1)st column, we obtain
in the (n + 1)st column  
0
λn − λn−1 λ0 
 1 1 
 .. . (A.4)
 . 
λnn − λnn−1 λ0
Next, one adds the (−λ0 )-fold of the (n − 1)st column to the nth column, and, succes-
sively, the (−λ0 )-fold of the mth column to the (m + 1)st column. One finishes, in the
nth step, by adding the (−λ0 )-fold of the first column to the second column, obtaining

1 λ0 . . . λn 1 0 0 ... 0
0
1 λ1 . . . λn 1 λ1 − λ0 λ2 − λ1 λ0 . . . λn − λn−1 λ0
1 1 1 1
det(V ) = .. .. = .. .. .. .. .. . (A.5)
. . . . . . .

1 λn . . . λnn 1 λn − λ0 λ2n − λn λ0 . . . λnn − λnn−1 λ0
Applying the rule for determinants of block matrices to (A.5) yields

λ1 − λ0 λ2 − λ1 λ0 . . . λn − λn−1 λ0
1 1 1
det(V ) = 1 · ... .. .. ..

. . . . (A.6)

λn − λ0 λ2n − λn λ0 . . . λnn − λnn−1 λ0
As we also know from Linear Algebra that determinants are linear in each row, for each
i, we can factor out (λi − λ0 ) from the ith row of (A.6), arriving at

n−1
Yn

1 λ 1 . . . λ 1
(λi − λ0 ) ... ... .. .. .

det(V ) = . . (A.7)
n−1
i=1 1 λn . . . λn
However, the determinant in (A.7) is precisely the Vandermonde determinant of the n−1
numbers λ1 , . . . , λn , which is given according to the induction hypothesis, implying
n
Y n
Y n
Y
det(V ) = (λi − λ0 ) (λi − λj ) = (λi − λj ), (A.8)
i=1 i,j=1 i,j=0
i>j i>j
completing the induction proof of (A.2).
B Scalar Products
Definition B.1. Let X be a real vector space. A function h·, ·i : X × X −→ R is called
an inner product or a scalar product on X if, and only if, the following three conditions
are satisfied:
(i) hx, xi > 0 for each 0 6= x ∈ X.

(ii) hλx + µy, zi = λhx, zi + µhy, zi for each x, y, z ∈ X and each λ, µ ∈ R.
(iii) hx, yi = hy, xi for each x, y ∈ X.

Lemma B.2. For each inner product h·, ·i on a real vector space X, the following
formulas are valid:
(a) hx, λy + µzi = λhx, yi + µhx, zi for each x, y, z ∈ X and each λ, µ ∈ R. Together
with Def. B.1(ii), this means that h·, ·i is a bilinear form.
(b) h0, xi = hx, 0i = 0 for each x ∈ X.
Proof. (a): One computes, for each x, y, z ∈ X and each λ, µ ∈ R,

Def. B.1(iii) Def. B.1(ii)
hx, λy + µzi = hλy + µz, xi = λhy, xi + µhz, xi
Def. B.1(iii)
= λhx, yi + µhx, zi. (B.1a)
(b): One computes, for each x ∈ X,

Def. B.1(iii) Def. B.1(ii)
hx, 0i = h0, xi = h0x, xi = 0hx, xi = 0, (B.1b)
thereby completing the proof of the lemma.

Theorem B.3. The following Cauchy-Schwarz inequality (B.2) holds for each inner
product h·, ·i on a real vector space X:
|hx, yi| ≤ kxk kyk for each x, y ∈ X, (B.2)
where p p
kxk := hx, xi, kyk := hy, yi. (B.3)
Moreover, equality in (B.2) holds if, and only if, x and y are linearly dependent, i.e. if,
and only if, y = 0 or there exists λ ∈ R such that x = λy.
Proof. If y = 0, then it is immediate that p both sides of (B.2) vanish. If x = λy with

2
λ ∈ R, then |hx, yi| = |λhy, yi| = |λ|kyk = λ2 hy, yikyk = kxk kyk, showing that (B.2)
holds with equality. If x and y are not linearly independent, then y 6= 0 and x − λy 6= 0
for each λ ∈ R, i.e.
0 < hx − λy, x − λyi = hx, xi − 2λhx, yi + λ2 hy, yi = kxk2 − 2λhx, yi + λ2 kyk2 . (B.4)
Since (B.4) is valid for each λ ∈ R, one can set λ := hx, yi/kyk2 (using y 6= 0) to get
2hx, yi2 hx, yi2 kxk2 kyk2 − hx, yi2

0 < kxk2 − + = , (B.5)
kyk2 kyk2 kyk2
or hx, yi2 < kxk2 kyk2 . Finally, taking the square root on both sides shows that (B.2)
holds with strict inequality.
Proposition B.4. If X is a real vector space with an inner product h·, ·i, then the map
p
k · k : X −→ R+ 0 , kxk := hx, xi, (B.6)
defines a norm on X. One calls this the norm induced by the inner product.
Proof. If x = 0, then hx, xi = 0 and kxk = 0 as well. Conversely, if x 6= 0, then

hx, xi > 0 and kxk > 0p as well, showing that k · k is positive definite. For λ ∈ R and
x ∈ X, one has kλxk = λ2 hx, xi = |λ|kxk, showing that k · k is homogeneous of degree
1. Finally, if x, y ∈ X, then
kx + yk2 = hx + y, x + yi = kxk2 + 2hx, yi + kyk2

(B.2) 2
≤ kxk2 + 2kxk kyk + kyk2 = kxk + kyk , (B.7)
establishing that k · k satisfies the triangle inequality. In conclusion, we have shown that
k · k constitutes a norm on X.
Example B.5. We consider the real vector space Rn , n ∈ N.
(a) The most common scalar product on Rn is the Euclidean scalar product defined by
n
X
h·, ·i : Rn × Rn , hx, yi := x · y := xi yi . (B.8)
i=1
One readily verifies that (B.8), indeed, defines a scalar product on Rn .
(b) Let the matrix B ∈ M(n, R) by symmetric (i.e. B t = B) and positive definite (i.e.
xt Bx > 0 for each 0 6= x ∈ Rn , where, for the purposes of matrix multiplication
x ∈ Rn is considered a column vector). Then
h·, ·iB : Rn × Rn , hx, yiB := xt By, (B.9)
defines a scalar product on Rn : Indeed, for h·, ·iB , Def. B.1(i) is precisely the state-
ment that B is positive definite. Next, for each x, y, z ∈ Rn and each λ, µ ∈ R,
hλx + µy, ziB = (λx + µy)t Bz = (λxt + µy t )Bz = λxt Bz + µy t Bz

= λhx, ziB + µhy, ziB , (B.10)
proving h·, ·iB satisfies Def. B.1(ii). Finally, using that B is symmetric, for each
x, y ∈ Rn ,
hy, xiB = hy, xitB = (y t Bx)t = xt B t y = xt By = hx, yiB , (B.11)
showing h·, ·iB also satisfies Def. B.1(iii).

Note that the Euclidean scalar product from (a) is recovered as the special case
B = Id.
REFERENCES 68
References
[Alt06] Hans Wilhelm Alt. Lineare Funktionalanalysis, 5th ed. Springer-Verlag,
Berlin, 2006 (German).
[But03] John Charles Butcher. Numerical Methods for Ordinary Differential

Equations. John Wiley, Chichester, UK, 2003.
[Eva98] Lawrence C. Evans. Partial Differential Equations. Graduate Studies in

Mathematics, Vol. 19, American Mathematical Society, Providence, Rhode
Island, 1998, reprinted with corrections in 2008.
[For08] Otto Forster. Analysis 2, 8th ed. Vieweg+Teubner, Wiesbaden, Germany,

2008 (German).
[Heu09] Harro Heuser. Gewöhnliche Differentialgleichungen, 6th ed.

Vieweg+Teubner, Wiesbaden, Germany, 2009 (German).
[Koe03] Max Koecher. Lineare Algebra und analytische Geometrie, 4th ed. Sprin-
ger-Verlag, Berlin, 2003 (German), 1st corrected reprint.
[Mar04] Nelson G. Markley. Principles of Differential Equations. Pure and Applied

Mathematics, Wiley-Interscience, Hoboken, NJ, USA, 2004.
[Phi18a] P. Philip. Calculus I for Computer Science and Statistics Students. Lec-
ture Notes, Ludwig-Maximilians-Universität, Germany, 2017/2018, available
in PDF format at http://www.math.lmu.de/~philip/publications/lectureNot
es/philipPeter_Calc1_forInfAndStatStudents.pdf.
[Phi18b] P. Philip. Calculus II for Statistics Students. Lecture Notes, Ludwig-

Maximilians-Universität, Germany, 2018, available in PDF format at
http://www.math.lmu.de/~philip/publications/lectureNot
es/philipPeter_Calc2_forStatStudents.pdf.
[Pla06] Robert Plato. Numerische Mathematik kompakt, 3rd ed. Vieweg Verlag,
Wiesbaden, Germany, 2006 (German).
[QSS07] Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri. Numerical

Mathematics, 2nd ed. Texts in Applied Mathematics, Vol. 37, Springer-Verlag,
Berlin, 2007.
[RR04] M. Renardy and R.C. Rogers. An Introduction to Partial Differential

Equations, 2nd ed. Texts in Applied Mathematics, Vol. 13, Springer-Verlag,
New York, 2004.
[Str08] Gernot Stroth. Lineare Algebra, 2nd ed. Berliner Studienreihe zur Math-
ematik, Vol. 7, Heldermann Verlag, Lemgo, Germany, 2008 (German).
REFERENCES 69
[Wal02] Wolfgang Walter. Analysis 2, 5th ed. Springer-Verlag, Berlin, 2002 (Ger-
man).
[Wal04] Wolfgang Walter. Analysis 1, 7th ed. Springer-Verlag, Berlin, 2004 (Ger-
man).

Methods For Solving Ordinary Differential Equations (Lecture Notes) - Peter Philip

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Methods For Solving Ordinary Differential Equations (Lecture Notes) - Peter Philip

Uploaded by

Copyright:

Available Formats

Mathematics IV for Geoscience Students:

Methods for Solving Ordinary Differential Equations

July 27, 2018

2 Systems of ODE and Higher-Order ODE 16

5 Short Introduction to PDE 55

A The Vandermonde Determinant 64

1 Ordinary Differential Equations (ODE) of First

1.1 Definition and Geometric Interpretation

y ′ = f (x, y), (1.1)

(i) The graph of φ is contained in G, i.e. {(x, φ(x)) ∈ I × R : x ∈ I} ⊆ G.

where x0 ∈ I, c = φ(x0 ). If f does depend also on y, then (1.1) still provides

1.2 Linear ODE, Variation of Constants

Here, and in the following, φ−1

Proof. We begin by noting that φ0 according to (1.7b) is well-defined since a is assumed

φ(x0 ) = φ0 (x0 ) (c + 0) = 1 · c = c (1.10)

a φ0 u + b = a ψ + b = ψ ′ = (φ0 u)′ = φ′0 u + φ0 u′ = a φ0 u + φ0 u′ , (1.12)

implying b = φ0 u′ and u′ = φ−1 0 b. Thus, the fundamental theorem of calculus in the

thereby completing the proof.

Proof. One immediately obtains (1.14) by setting b ≡ 0 in in (1.7).

a : R −→ R, a(x) := 2x, (1.16a)

The solution to (1.15) is given by (1.7a):

1.3 Separation of Variables

Integrating (1.23) yields

thereby establishing that φ satisfies (1.22).

showing that φ satisfies (1.19).

1.4 Bernoulli Equations

z ′ = (1 − α)g(x) z + (1 − α)h(x) (1.38)

for the function z.

We now distinguish several cases:

showing φ is, once again, a solution to (1.37).

ψ > 0 on I0 for c > 0,

as z 6= 0 on I ′ . As this is in contradiction to ψ(s) = 0, we have proved ψ 6= 0 and z̃ = z

Example 1.15. Consider the Bernoulli equation

According to (1.7), we obtain

showing φ satisfies the initial condition. Next, one computes

proving φ is a solution to (1.37).

s := inf{x ∈ I ′ : x > x0 , ψ(x) = 0}

as z 6= 0 on I ′ . As this is in contradiction to ψ(s) = 0, we have proved ψ 6= 0 and z̃ = z

and (1.47) yields

α < 0 Is Not an Integer:

Example 1.19. Consider the Bernoulli equation

According to (1.7), we obtain

As z does not have any zeros, φ is defined on R and (1.54) yields

α > 0 Is Not an Integer:

Example 1.21. Consider the Bernoulli equation

i.e. with g : R −→ R, g(x) = 0, h : R −→ R, h(x) = 1, and α = 1/2. There are many

2 Systems of ODE and Higher-Order ODE

2.1 Systems of First-Order ODE

(i) {(x, φ(x)) ∈ I × Rn : x ∈ I} ⊆ G.

(b) As a first example of a coupled system of ODE consider

has the solution

Theorem 2.4 (Peano). If G ⊆ R × Rn is open, n ∈ N, and f : G −→ Rn is continuous,

Proof. See, e.g., [Mar04, Th. 1.9].

Proof. See, e.g., [Mar04, Th. 1.11] or [For08, Th. §12.3].

2.2 Higher-Order ODE

A solution to this ODE is an n times differentiable function

defined on a (bounded or unbounded) open interval I ⊆ R satisfying the following two

and the first-order initial value problem

(a) If φ : I −→ R is a solution to (2.15), then

(b) If Φ : I −→ Rn is a solution to (2.16), then φ := Φ1 is a solution to (2.15).

Proof. (a): As a solution to (2.15), φ is n times differentiable and Φ is well-defined.

Next, Def. 2.8(i) implies Def. 2.1(i), since