Stoch Proc Sim

Stochastic processes
(Updated
16/5/2014)
Generalities
Brownian Motion
Stochastic differential
equations
Simulation of SDEs
(Updated 16/5/2014)
References
S.M. Iacus 2014 Simulation of SDEs 1 / 105

Generalities
A stochastic process is a family of random variables {Xt , t } defined
(Updated
on taking values in R.
16/5/2014)
Generalities
Brownian Motion X(t, ) 7 R
equations For example = [0, T ], or = N or Z (integers with sign), etc.
Simulation of SDEs
References For a given random event , we call X(t, ) a trajectory or path of

the random process, seen as a function of time.
A trajectory represents the dynamic evolution of the stochastic process in

time.
We use the notation Xt = X(t) = X(t, ) whenever needed.

Paths of stochastic processes
X(t, 3)
X(t, 1)
X(t, 2)
0 t

Classification of stochastic processes
State space
(Updated
16/5/2014) Discrete Continuous
Generalities Time
Brownian Motion
equations
Discrete Random walks ARIMA, GARCH
Simulation of SDEs
Markov Chains
References
Telegraph, Lvy
Continuous Poisson Wiener, Diffusion
SDE

Brownian Motion
Brownian motion (or Wiener process) is a stochastic process (Bt , t 0)
(Updated
starting from zero at time 0, i.e. B0 = 0, with the following properties:
16/5/2014)
Generalities (i) it has independent increments: i.e. Bt Bs and Bu Bv are
Brownian Motion
independent for (s, t) (v, u) = ;
equations (ii) it has stationary increments: i.e. the distribution of Bt Bs for
Simulation of SDEs t > s 0 depends only on t s and not by t and/or s separately;
References (iii) it has Gaussian increments: i.e. Bt Bs N (0, t s)
in particular Bt N (0, t). We also use the notation Wt = Bt

Brownian Motion
Given a fixed time increment t > 0, one can easily simulate a trajectory
(Updated of the Wiener process in the time interval [0, T ]. Indeed, for Wt it holds
16/5/2014)
Generalities
true that
Brownian Motion
Stochastic differential W (t) = W (t) W (0) N (0, t) t N (0, 1),
equations
Simulation of SDEs and the same is also true for any other increment W (t + t) W (t); i.e.,
References

W (t + t) W (t) N (0, t) t N (0, 1) .
Thus, we can simulate one such path as follows.

Brownian Motion
Divide the interval [0, T ] into a grid such as
(Updated
0 = t1 < t2 < < tN 1 < tN = T with ti+1 ti = t. Set i = 1 and
16/5/2014)
Generalities W (0) = W (t1 ) = 0 and iterate the following algorithm.
Brownian Motion
1. Generate a (new) random number z from the standard Gaussian
equations distribution.
Simulation of SDEs 2. i = i + 1.
References 3. Set W (ti ) = W (ti1 ) + z t.
4. If i N , iterate from step 1.

Naive simulator of the Brownian motion
> set.seed(123)
(Updated > N <- 100 # number of end-points of the grid including T
16/5/2014) > T <- 1 # length of the interval [0,T] in time units
Generalities > Delta <- T/N # time increment
Brownian Motion > W <- numeric(N+1) # initialization of the vector W
> t <- seq(0,T, length=N+1)
equations > for(i in 2:(N+1))
+ W[i] <- W[i-1] + rnorm(1) * sqrt(Delta)
Simulation of SDEs
> plot(t,W, type="l", main="Wiener process" , ylim=c(-1,1))
References

Naive simulator of the Brownian motion
> set.seed(123)
(Updated > N <- 100 # number of end-points of the grid including T
16/5/2014) > T <- 1 # length of the interval [0,T] in time units
Generalities > Delta <- T/N # time increment
Brownian Motion > W <- numeric(N+1) # initialization of the vector W
equations > for(i in 2:(N+1))
+ W[i] <- W[i-1] + rnorm(1) * sqrt(Delta)
Simulation of SDEs
> plot(t,W, type="l", main="Wiener process" , ylim=c(-1,1))
References
But the code above is highly inefficient as it implements an iteration using

a for statement when this is not needed at all. In this simple case, the
whole trajectory can be simulated in one line of R code as follows.
W <- c(0,cumsum( sqrt(Delta) * rnorm(N)))
due to the fact that the algorithm

is just simulating a random walk with
Gaussian increments of size t N (0, 1).

R efficiency
> BM.1 <- function(N=10000){ # brutal code
(Updated + W <- NULL
16/5/2014) + for(i in 2:(N+1))
Generalities + W <- c(W, rnorm(1) / sqrt(N))
Brownian Motion + }
> BM.2 <- function(N=10000){ # smarter
equations + W <- numeric(N+1)
+ Z <- rnorm(N)
Simulation of SDEs
+ for(i in 2:(N+1))
References + W[i] <- W[i-1] + Z[i-1] / sqrt(N)
+ }
> BM.vec <- function(N=10000) # awesome !
+ W <- c(0,cumsum(rnorm(N)/sqrt(N)))
> set.seed(123)
> system.time(BM.1())
[1] 1.354 1.708 3.856 0.000 0.000
> set.seed(123)
> system.time(BM.2())
[1] 0.281 0.011 0.347 0.000 0.000
> set.seed(123)
> system.time(BM.vec())
[1] 0.008 0.001 0.010 0.000 0.000
Example of Brownian motions trajectory
Wiener process
1.0
0.5
0.0
W
0.5
1.0
0.0 0.2 0.4 0.6 0.8 1.0
t
Brownian motion
Let and random walksX1 , X2 , . . . , Xn be a sequence of i.i.d. random
(Updated variables taking only values -1 and +1 with probability 12 . Let
16/5/2014)
Generalities
Brownian Motion Sn = X1 + X2 + + Xn
equations be a random walk. Then,
Simulation of SDEs
S[nt]

References
, t 0 (B(t), t 0)
n
where convergence is in distribution.

So Brownian motion is the limit of a random walk (R).

Notice that, by the Central Limit Theorem we already know that Sn / n N (0, 1). In the above,
the convergence takes place for each t > 0 and it is a Gaussian law with variance t.
Exercise: write R code to show this empirically.

Brownian motion paths are nowhere differentiable
Brownian motion can be approximated using the following series
(Updated expansion
16/5/2014) X
Generalities
Wt = W (t) = Zi i (t)
Brownian Motion
i=0
equations
with
Simulation of SDEs 2 2T (2i + 1)t
i (t) = sin
References
(2i + 1) 2T
and Zi a sequence of i.i.d. Gaussian random variables.
From this formula we can show that Brownian motion has continuous but
nowhere differentiable trajectories.

To graphically view the weirdness of the trajectory of the Brownian
(Updated motion and understand why it is nowhere differentiable, we simulate the
16/5/2014)
Generalities
increments of the Brownian motion in two subsequent time points lagged
Brownian Motion by a time t, say W (0.5) and W (0.5 + t), and we let t 0.
equations
Next plot shows the explosive behaviour of
Simulation of SDEs
References |W (0.5 + t) W (0.5)|

lim +
t0 t

Non differentiability of Brownian motion
1e+05
W(0.5 + t) W(0.5) t
1e+03
1e+01
0.000 0.002 0.004 0.006 0.008 0.010
t
R code
> set.seed(123)
(Updated > phi <- function(i,t,T){
16/5/2014) + (2*sqrt(2*T))/((2*i+1)*pi) * sin(((2*i+1)*pi*t)/(2*T))
Generalities + }
Brownian Motion > n <- 100
> Z <- rnorm(n)
equations > Delta <- seq(1e-7, 1e-2,length=30)
> W <- sum(Z*sapply(1:n, function(x) phi(x,0.5,T)))
Simulation of SDEs
> for(i in Delta)
References + Wh <- sum(Z*sapply(1:n, function(x) phi(x,0.5+i,T)))
> inc.ratio <- abs(Wh-W)/Delta
> plot(Delta,inc.ratio,type="l",log="y",xlab=expression(Delta*t),
+ ylab=expression(abs(W(0.5+Delta*t)-W(0.5))/Delta*t))

This happens because of the independence of the increments of the
(Updated Brownian motion and also, moreimportantly, because the increments
16/5/2014)
Generalities
W (t + t) W (t) behave like t instead of t. Thus, in the limit one
Brownian Motion can expect
equations

|W (t + t) W (t)| | t|
Simulation of SDEs lim lim = + .
t0 t t0 t
References

Brownian bridge
Another useful and interesting manipulation of the Wiener process is the
(Updated so-called Brownian bridge, which is a Brownian motion starting at x at
16/5/2014)
Generalities
time t0 and passing through some point y at time T , T > t0 . It is defined
Brownian Motion as
t t0
WtT,y
equations
0 ,x
(t) = x + W (t t0 ) (W (T t0 ) y + x) .
Simulation of SDEs T t0
References
More precisely, this is the process
{W (t), t0 t T |W (t0 ) = x, W (T ) = y}. This process is easily
simulated using the simulated trajectory of the Wiener process.

Brownian bridge
t t0
WtT,y
0 ,x
(t) = x + W (t t0 ) (W (T t0 ) y + x) .
T t0
> set.seed(123)
> N <- 100
> T <- 1
> Delta <- T/N
> W <- numeric(N+1)
> for(i in 2:(N+1)) W[i] <- W[i-1] + rnorm(1) * sqrt(Delta)
> x <- 0
> y <- -1
> BB <- x + W - t/T * (W[N+1] - y +x)
> plot(t,BB,type="l")
> abline(h=-1, lty=3)

(Updated
16/5/2014)
equations
What is an sde?
Examples
It Integral Stochastic differential equations
Properties
Simulation of SDEs
References

What is an sde?
(Very) Roughly speaking, given a smooth, non stochastic dynamical
(Updated system Xt = X(t), its evolution with respect to time can be represented as
16/5/2014)
Stochastic differential Xt+dt Xt dXt
equations = = b(Xt ) or dXt = b(Xt )dt
What is an sde? dt dt
Examples
It Integral A stochastic differential equation models the noise (or the stochastic part)
Properties of this system by adding the variation of some stochastic process to the
Simulation of SDEs above dynamics, e.g. the Wiener process
References
dXt = b(Xt )dt + (Xt )dWt
i.e.
deterministic trend + stochastic noise

Meaning of and SDE
Unfortunately
(Updated dXt dWt
16/5/2014) = b(Xt ) + (Xt )
dt dt
equations does not exists in the usual sense because we have seen that Wiener
What is an sde? process is nowhere differentiable.
Examples
It Integral
Properties SDEs exist in the It sense, and the correct definition is given in integral
Simulation of SDEs form Z t Z t
References Xt = X0 + b(Xs )ds + (Xs )dWs
0 0
where the last term is the It, or stochastic, integral.

Model of interest
We will consider only a particular simple class of SDEs.
(Updated
16/5/2014) Let {Xt }, t [0, T ] be a diffusion process solution to the 1-dim SDE
equations
What is an sde? dXt = b(Xt ; )dt + (Xt ; )dWt (1)
Examples
It Integral with some initial condition X0 (eventually stochastic). Where
Properties
Simulation of SDEs Wt the Wiener process

References
b(x; ) and (x; ) are two smooth functions such that the SDE is well
defined
Rp , are the parameters of interest

Fields of application
physics (see e.g. Papanicolaou, 1995)

astronomy (Schuecker et al., 2001)
mechanics (Kushner, 1967)
economics (Bergstrom, 1990)
mathematical finance (Hull, 2000)
geology (Ditlevsen et al.. 2002)
genetic analysis (see e.g. Holland, 1976, Lange, 2002)
ecology (Holmes, 2004)
cognitive psychology (see e.g. Hanes & Schall, 1988)
neurology (Holden, 1976)
biology (Ricciardi, 1977)
biomedical sciences (Banks, 1975)
epidemiology (Bailey, 1957)
political analysis and social processes (Cobb, 1981)
...and many other fields of science and engineering

Examples
geometric Brownian motion (gBm)
(Updated
16/5/2014) dXt = Xt dt + Xt dWt
equations
What is an sde?
Examples Cox-Ingersoll-Ross (CIR)
It Integral
Properties
p
dXt = (1 + 2 Xt )dt + 3 Xt dWt
Simulation of SDEs
References
Chan-Karolyi-Longstaff-Sanders (CKLS)
dXt = (1 + 2 Xt )dt + 3 Xt4 dWt

Examples
nonlinear mean reversion (At-Sahalia)
(Updated
16/5/2014) dXt = (1 Xt1 + 0 + 1 Xt + 2 Xt2 )dt + 1 Xt dWt
equations
What is an sde?
Examples double Well potential (bimodal behaviour, highly nonlinear)
It Integral
Properties
dXt = (Xt Xt3 )dt + dWt
Simulation of SDEs
References
Jacobi diffusion (political polarization)

1 p
dXt = Xt dt + Xt (1 Xt )dWt
2

Examples
Ornstein-Uhlenbeck (OU)
(Updated
16/5/2014) dXt = Xt dt + dWt
equations
What is an sde?
Examples radial Ornstein-Uhlenbeck
It Integral
Properties
dXt = (Xt1 Xt )dt + dWt
Simulation of SDEs
References
hyperbolic diffusion (dynamics of sand)

" #
2 Xt
dXt = p dt + dWt
2 2
+ (Xt ) 2
...and so on.

Weak and Strong solutions
Weak solution means that the Xt which solves (1), is one of the solutions
(Updated
with the property of being equivalent in the distributional sense.
16/5/2014)
equations Existence of a strong solutions is a pathwise equivalence definition.
What is an sde?
Examples
In simulation problems sometimes the existence of a strong solution do
It Integral
Properties
matter. In other cases, like e.g. Monte Carlo option pricing, weak
Simulation of SDEs
solutions are enough.
References
In statistics, we are mainly interested in distributional properties of the
observations, hence existence of weak solutions is enough.
A strong solution is also a weak solution.

Assumptions
Assumption 1 (Global Lipschitz). For all x, y R and t [0, T ], there
(Updated
exists a constant K < + such that
16/5/2014)
equations |b(t, x) b(t, y)| + |(t, x) (t, y)| < K|x y| . (L)
What is an sde?
Examples Assumption 2 (Linear growth, NO EXPLOSION). For all x, y R and
It Integral
Properties
t [0, T ], there exists a constant C < + such that
Simulation of SDEs
|b(t, x)| + |(t, x)| < C(1 + |x|) . (G)
References
Under the above assumptions the SDE has a unique strong solution.

Local conditions
The above conditions might be too restrictive
(Updated
16/5/2014) dXt = Xt3 dt + dWt . (2)
equations
What is an sde? Assumption 3 (Local Lipschitz). For any N < , |x|, |y| N , there
Examples exists a constant LN > 0 such that
It Integral
Properties
|b(x) b(y)| + |(x) (y)| LN |x y|
Simulation of SDEs
References and
2xb(x) + 2 (x) B(1 + x2 ) . (3)
Indeed, these conditions are satisfied in (2). For the class of ergodic
diffusion processes that will be discussed later, it is usually true that
xb(x) < 0. Hence (3) is just a condition on the growth of the diffusion
coefficient.

Ergodicity
Diffusion processes possess the Markov property and may or may not be
(Updated ergodic.
16/5/2014)
equations The ergodic property implies that, for any measurable function h(), the
What is an sde? following result holds with probability 1:
Examples
It Integral T +
1
Z Z
Properties h(Xt )dt h(x)(x)dx = Eh(),
Simulation of SDEs
T 0
References
where () is called the invariant or stationary density of the diffusion
process and is some random variable with () as density.

Invariant density
The stationary distribution, when it exists, can be expressed in terms of
(Updated the scale measure and the speed measure defined respectively as
16/5/2014)
Z x
Stochastic differential b(y)
equations s(x) = exp 2 2
dy (4)
What is an sde? x0 (y)
Examples
It Integral and
Properties 1
m(x) = 2 . (5)
Simulation of SDEs (x)s(x)
References
In particular, the density of the invariant distribution () is proportional,
up to a normalizing constant, to the speed measure m(); i.e.,
m(x)
(x) = , (6)
M
R
where M = m(x)dx.

Transition and invariant density
From the Markovian property of the diffusion, it is also possible to define
(Updated the transition density from value x at time s to value y at time t by
16/5/2014)
p(t, y|s, x) or as p(t s, y|x).
equations
What is an sde? For parametric models, we will later use the notation p(t, y|s, x; ) or
Examples
p (t, y|s, x) and p(t s, y|x; ) or p (t s, y|x), respectively.
It Integral
Properties
Simulation of SDEs
References

Transition and invariant density
The transition density satisfies the Kolmogorov forward equation
(Updated
16/5/2014) 1 2 2
p(t, y|s, x) = (b(y)p(t, y|s, x)) + 2
( (y)p(t, y|s, x)) (7)
equations t y 2 y
What is an sde?
Examples and Kolmogorov backward equation
It Integral
Properties 1 2 2
p(t, y|s, x) = b(x) p(t, y|s, x) + (x) 2 p(t, y|s, x); (8)
Simulation of SDEs s x 2 x
References
these equations relate the transition density to the invariant density and the
coefficients of the SDE

Invariant density and coefficients of the SDE
Taking the limit as t diverges in the previous equations
(Updated
16/5/2014) d2 2 d
Stochastic differential 2
( (x)(x)) = 2 (b(x)(x)) ,
equations dx dx
What is an sde? and by subsequent integration we obtain
Examples
It Integral
1 d 2
Properties b(x) = ( (x)(x)),
Simulation of SDEs
2(x) dx
References and finally
x
2
Z
2
(x) = b(u)(u)du
(x) 0
from which nonparametric inference is usually possible as is usually done.

Infinitesimal generator of a diffusion
Given X solution to dXt = b(Xt )dt + (Xt )dWt , X0 = x0 , the
(Updated
differential operator L defined as
16/5/2014)
2 (x)
equations
(Lf )(x) = f (x) + b(x)f (x) (9)
What is an sde? 2
Examples
It Integral is called the infinitesimal generator of X and Let
Properties
Z t
Simulation of SDEs
Z(t) = f (Xt ) (Lf )(Xs )ds ,
References
0
is a martingale. Moreover,
Z t
Ef (Xt ) = f (x0 ) + E (Lf )(Xs )ds
0
(L is the basis of many statistical procedures)

Trick: The Lamperti transform
Take dXt = b(t, Xt )dt + (Xt )dWt and apply the Lamperti transform to
(Updated
it
16/5/2014) Z Xt
1
Stochastic differential Yt = F (Xt ) = du (10)
equations z (u)
What is an sde?
Examples
where z is any arbitrary value in the state space of X. Then Yt solves the
It Integral SDE
Properties dYt = bY (t, Yt )dt + dWt ,
Simulation of SDEs
where
b(t, F 1 (y)) 1
References
1
bY (t, y) = 1
x (F (y)),
(F (y)) 2
which can also be written as

b(t, Xt ) 1
dYt = x (Xt ) dt + dWt . (11)
(Xt ) 2

Stochastic integral
Let
(Updated Rt
EX 2 du <
16/5/2014) 0 u
Stochastic differential Xu independent of B(s) B(u) (with s > u), i.e. X is a process
equations
adapted to the natural filtration of the Brownian motion
What is an sde?
Examples
It Integral
The stochastic integral of the Rprocess X with respect to the Brownian
Properties t
motion, denoted by It (X) = 0 Xs dBs , is the limit of
Simulation of SDEs
References 2
Z t n1
X
It (X) = Xs dBs = lim E X(tj ) (B (tj+1 ) B (tj ))
0 n
j=0

Properties of the stochastic integral
EIt (X) = 0
(Updated Rt
2
VarIt (X) = E{It (X)} = 0 EXs2 ds : (It isometry)
16/5/2014) Rt Rt
adB(s) = a 0 dB(s) = aB(t)
Stochastic differential R0t
0 (aX(s) + Y R
(s))dB(s) = aIt (X) + bIt (Y ) : linearity
equations

What is an sde?
t
Examples It (X) N (0, 0 EXs2 ds)
It Integral
Properties
Simulation of SDEs Notice that, for each t, It (X) is a random variable, hence the family
References {It (X), t 0} is a stochastic process.

(Updated
16/5/2014)
equations
Simulation of SDEs
Simulation strategies
gBm
Simulation of SDEs
Simulation
Numerical stability
Trick
Euler-Maruyama
pitfalls
Likelihood
Exact sampling
sde package
Yuima
References

There are at least four (although not unrelated) types of approaches in
(Updated simulation for SDEs
16/5/2014)
Stochastic differential discretization of the SDE
equations
via conditional distribution
Simulation of SDEs
approximation of the of SDE before discretization
gBm via exact sampling (acceptance/rejection method)
Simulation
Numerical stability
Before looking at the schemes for general SDEs, we consider the case of
Trick geometric Brownian motion.
Euler-Maruyama
pitfalls
Likelihood
Exact sampling
sde package
Yuima
References

Geometric Brownian motion
In the Black & Scholes theory of option pricing, we model an underlying
(Updated asset using a stochastic process St called geometric Brownian motion
16/5/2014)
which satisfies the stochastic differential equation
equations
Simulation of SDEs
dSt = St dt + St dWt
gBm with volatility , interest rate r and drift .
Simulation
Numerical stability
Trick
Euler-Maruyama
pitfalls
Likelihood
Exact sampling
sde package
Yuima
References

Geometric Brownian motion
In the Black & Scholes theory of option pricing, we model an underlying
(Updated asset using a stochastic process St called geometric Brownian motion
16/5/2014)
which satisfies the stochastic differential equation
equations
Simulation of SDEs
dSt = St dt + St dWt
gBm with volatility , interest rate r and drift .
Simulation
Numerical stability
Trick
the solution of the SDE is explicit:
Euler-Maruyama
2

pitfalls

Likelihood St = x exp r t + Wt , t > 0,
Exact sampling 2
sde package
Yuima with S0 = x, x R is the initial value.
References

Simulation of the Geometric Brownian motion
Assuming that W contains a trajectory of W and t is the vector containing
(Updated all the time points, a path can be generated as follows
16/5/2014)
equations > set.seed(123)
> r <- 1
Simulation of SDEs > sigma <- 0.5
Simulation strategies > x <- 10
gBm > N <- 100 # number of end points of the grid including T
Simulation > T <- 1 # length of the interval [0,T] in time units
Numerical stability > Delta <- T/N # time increment
Trick > W <- numeric(N+1) # initialization of the vector W
Euler-Maruyama > t <- seq(0,T, length=N+1)
pitfalls > for(i in 2:(N+1))
Likelihood + W[i] <- W[i-1] + rnorm(1) * sqrt(Delta)
Exact sampling > S <- x * exp((r-sigma^2/2)*t + sigma*W)
sde package > plot(t,S,type="l",main="geometric Brownian motion")
Yuima
References

geometric Brownian motion

40
35
30
25
S
20
15
10
0.0 0.2 0.4 0.6 0.8 1.0

An equivalent way of simulating a trajectory of the geometric Brownian
(Updated motion is by simulating the increments of S. Indeed,
16/5/2014)
2

r 2 (t+tt)+(W (t+t)W (t))
equations S(t + t) = S(t)e ,
Simulation of SDEs
which simplifies to
gBm
Simulation
2

Numerical stability
S(t + t) = S(t) exp r t + tZ
Trick 2
Euler-Maruyama
pitfalls
Likelihood with Z N (0, 1).
Exact sampling
sde package
Yuima
References

Consider the SDE of gBm
(Updated
16/5/2014) dSt = St dt + St dBt
equations
Let us choose a grid of times ti = i, i = 0, 1, . . . , n, such that
Simulation of SDEs
Simulation strategies n = T.
gBm
Simulation We can discretize the SDE as follows (Sti = S(ti ) = S(i) )
Numerical stability
Trick
Euler-Maruyama S((i + 1)) S(i) = S(i) + S(i)(B((i + 1)) B(i))
pitfalls
Likelihood

Exact sampling
remind that B((i + 1)) B(i) N (0, ) = Zi with
sde package Zi N (0, 1), thus
Yuima

References S((i + 1)) = S(i) + S(i) + S(i) Zi

Let us denote by S[i] the value of S(i), Z[i] the pseudo-RNG of the Zi ,
(Updated S0 the initial value of S0 .
16/5/2014)
equations > set.seed(123)
> n <- 101 # total length of the sequence
Simulation of SDEs
> Delta <- 0.1 # time mesh
>
gBm > S0 <- 1 # initial value
Simulation > mu <- 0.1 # the value of the drift
Numerical stability > sigma <- 0.2 # the volatility
Trick >
Euler-Maruyama > S <- numeric(n) # we prepare a vector
pitfalls
> S[1] <- S0 # initialize the starting point
Likelihood
>
Exact sampling > for(i in 2:n) # generation of the the trajectory
sde package + S[i] <- S[i-1] + mu*S[i-1]*Delta +
Yuima + sigma*S[i-1]*sqrt(Delta)*rnorm(1)
References
>
> t <- (0:(n-1))*Delta
> plot(t, S, type="l", main="gBm")

An alternative way is to load the package sde
(Updated
16/5/2014)
> library(sde)
equations
> set.seed(123)
> n <- 100
Simulation of SDEs > Delta <- 0.1
Simulation strategies >
gBm > S0 <- 1 # fix initial value
Simulation > mu <- 0.1 # fix the drift value
Numerical stability > sigma <- 0.2 # and one for the volatility
Trick >
Euler-Maruyama > S <- sde.sim(X0=S0, N=n, delta=Delta, model="BS",
pitfalls + theta=c(mu, sigma))
Likelihood
Exact sampling >
sde package > plot(S, main="gBm")
Yuima
References We will discuss the sde package later on.

The sde package contains these functions as well
(Updated
16/5/2014)
equations
BBridge(x=0, y=0, t0=0, T=1, N=100)
Simulation of SDEs
BM(x=0, t0=0, T=1, N=100)
Simulation strategies GBM(x=1, r=0, sigma=1, T=1, N=100)
gBm
Simulation you can try
Numerical stability
Trick
Euler-Maruyama
pitfalls plot(BM())
Likelihood
plot(BBridge())
Exact sampling
sde package plot(GBM())
Yuima
References

Properties of simulation schemes
To assess the quality of a simulation scheme some order of convergence
(Updated has to be defined. Let X be the simulated version of X, simulated on a
16/5/2014)
grid of times ti with maximum mesh (ti ti1 ) equal to .
equations
A simulation method is of strong order > 0 if at time T we have
Simulation of SDEs
gBm E|XT XT | C , < 0
Simulation
Numerical stability X is said to converge weakly of order to X if for any fixed T and
Trick
Euler-Maruyama
any 2( + 1) continuous differentiable function g of polynomial
pitfalls
growth
Likelihood
Exact sampling
|Eg(XT ) Eg(XT )| C , < 0
sde package
Yuima
References
In both cases 0 > 0 and C not depending on

Discretization
The stochastic differential equation
(Updated
16/5/2014) dXt = b(Xt )dt + (Xt )dWt
equations
is thought as a finite difference system
Simulation of SDEs
gBm Xti+1 Xti = something
Simulation
Numerical stability and something is simulated accordingly and conditionally on Xti .
Trick
Euler-Maruyama
pitfalls None of the methods exposed here require t = ti+1 ti to be fixed, but
Likelihood
Exact sampling
to ease the notation we make this assumption. We also write Xi instead of
sde package Xti = X(ti ).
Yuima
References

Euler-Maruyama scheme
The idea behind Euler-Maruyama scheme is very old (as per the name)
(Updated
16/5/2014) Xi+1 = Xi + b(Xi )t + (Xi )W
equations = current value + innovation
Simulation of SDEs
Simulation strategies where
gBm W = Wi+1 Wi tZ
Simulation
Numerical stability with Z N (0, 1)
Trick
Euler-Maruyama
1
pitfalls Euler-Maruyama scheme is strongly convergent of order = 2 and
Likelihood
Exact sampling
weakly convergent of order = 1.
sde package
Yuima
References

Milstein scheme
By It formula, it is possible to obtain the following discretization
(Updated
16/5/2014) Xi+1 =Xi + b(Xi )t + (Xi )W
equations 1 2

+ (Xi )x (Xi ) (W ) t
Simulation of SDEs 2
gBm here x is the partial derivative of w.r.t. argument x.
Simulation
Numerical stability
Milstein scheme is of strong order = 1.
Trick
Euler-Maruyama
pitfalls
For all SDEs with constant diffusion coefficients, Euler-Maruyama and
Likelihood
Exact sampling
Milstein method coincide, so are of the same order of convergence.
sde package
Yuima Both methods work for nonhomogeneus SDE with b = b(t, x) and
References = (t, x).

Speed of Convergence
2.4
(Updated
2.2
16/5/2014)
equations
Euler
S(T)
Simulation of SDEs 2.0 Milstein

Simulation strategies True
gBm
Simulation
Numerical stability
1.8
Trick
Euler-Maruyama
pitfalls
Likelihood 2 4 6 8 10 12 14
Exact sampling
sde package l o 2g(N)
Yuima
References
Simulation of gBm S(t). The interest is in S(T ) (e.g. for evaluating a
european call price). Convergence as a function of N with = 1/N .

Milstein scheme 2

1 1
(Updated Xi+1 = Xi + b x t + Z t + x tZ 2
16/5/2014) 2 2

Stochastic differential 3 1 1 1 2
equations + t 2 bx + bx + xx Z
2 2 4
Simulation of SDEs
2 1 1
+ t bbx + bxx 2 .
gBm 2 4
Simulation
Numerical stability where we omitted the argument Xi in b and .
Trick
Euler-Maruyama
pitfalls More regularity on the coefficients is needed (e.g. existence of partial
Likelihood derivatives up to order 2 of the coefficients b and )
Exact sampling
sde package
Yuima For this scheme = 2
References

KPS method
Kloeden-Platen-Schurz-Srensen method of strong order = 1.5

1 2
1 1 2
bbx + bxx t2

Xi+1 = Xi + bt + Wt + x (Wt ) t +
2 2 2

1 2
+ bx Ut + bx + xx {Wt t Ut }
2

1 1
+ (x )x (Wt )2 t Wt ,
2 3
where Z ti+1 Z s
Ut = dWu ds
t0 ti
is a Gaussian random variable with zero mean and variance 13 t3 and such that
E(Ut Wt ) = 21 t2 . All the pairs (Wt , Ut ) are mutually independent.
Method requires samples from the multivariate normal distribution

Markov property
A discrete time stochastic process {Xn , n 1} is said to be Markovian if
(Updated
the conditional distribution of Xk given the past (Xk1 , Xk2 , . . .) equals
16/5/2014)
the conditional distribution of Xk given Xk1 solely, i.e.
equations
Simulation of SDEs L(Xk |Xk1 , Xk2 , . . .) = L(Xk |Xk1 ).

gBm
Simulation
Numerical stability
Trick
This means that the future of the process Xk depends only on the
Euler-Maruyama
pitfalls
present Xk1 and the knowledge of the past Xk2 , . . . , X1 does not
Likelihood contribute any additional information.
Exact sampling
sde package
Yuima
References

Sampling from the conditional distribution
(Updated Diffusion processes are also Markovian, hence the conditional distribution
16/5/2014) of Xt |Xs = x always exists.
equations
Simulation of SDEs
So, by using the conditional distribution
gBm p(, y|x) = P r(Xt+ dy|Xt = x)
Simulation
Numerical stability
it is possible to generate a sequence X0 , X1 , etc sampling from p(, y|x).
Trick
Euler-Maruyama
pitfalls
Unfortunately this possibility is almost theoretical because the conditional
Likelihood
Exact sampling distribution is known in explicit form for essentially only 3 non trivial
sde package models: OU, gBm, CIR !
Yuima
References For OU p(, y|x) is Gaussian, for gBm is log-normal, for CIR is 2 .

Conditional distribution: Ornstein-Uhlenbeck
Recall that the Ornstein-Uhlenbeck or Vasicek process solution to
(Updated
16/5/2014) dXt = (1 2 Xt )dt + 3 dWt , X 0 = x0 ,
equations
It has a Gaussian transition density p (, y|Xt = x) with mean and
Simulation of SDEs
Simulation strategies variance as in (12) and (13), respectively
gBm
Simulation 1 1
Numerical stability m(t, x) = E (Xt |X0 = x0 ) = + x0 e2 t (12)
2 2
Trick
Euler-Maruyama
pitfalls and
32 e22 t

Likelihood 1
Exact sampling v(t, x) = Var (Xt |X0 = x0 ) = . (13)
sde package
22
Yuima
References

Therefore,
(Updated Xt+t |Xt = x N (m(t, x), v(t, x))
16/5/2014)
Stochastic differential Hence we can just make use of the rnorm function to build our random
equations
number generator.
Simulation of SDEs
gBm rcOU <- function(n=1, t, x0, theta){
Simulation Ex <- theta[1]/theta[2]+(x0-theta[1]/theta[2])*exp(-theta[2]*t)
Numerical stability Vx <- theta[3]^2*sqrt((1-exp(-2*theta[2]*t))/(2*theta[2]))
Trick rnorm(n, mean=Ex, sd = sqrt(Vx))
Euler-Maruyama }
pitfalls
Likelihood
Exact sampling and, in a simulator, we will generate X[i] using a pseudo random number
sde package
generated from the law of Xt+t |Xt = X[i 1].
Yuima
References

Consider again the Ornstein-Uhlenbeck model
(Updated
16/5/2014) dXt = (1 2 Xt )dt + 3 dWt
equations
This is a mean reverting process. It means that, in the long run, the process
Simulation of SDEs
Simulation strategies tends to oscillate around some given level. We can rewrite this process as
gBm
Simulation dXt = ( Xt )dt + dWt
Numerical stability
Trick
Euler-Maruyama (where 1 = and 2 = ). The value of is this global level and is
pitfalls
the speed of mean reversion at which the process tends to go around the
Likelihood
Exact sampling value of . The parameter is the usual volatility parameter.
sde package Let us consider some examples.
Yuima
References

Mean reverting property
Suppose we want to simulate the process
(Updated
16/5/2014) dXt = ( Xt )dt + dWt
equations
we define the corresponding drift and diffusion functions in R
Simulation of SDEs
gBm > d <- expression(1*(5-x))
Simulation > s <- expression(0.5)
Numerical stability
Trick
and then we simulate the trajectory using sde.sim from the sde package.
Euler-Maruyama We set some value for , and .
pitfalls
Likelihood
Exact sampling > set.seed(123)
sde package > sde.sim(X0=10,drift=d, sigma=s,T=1000,N=1000) -> X
Yuima > plot(X,main="Ornstein-Uhlenbeck")
> abline(h=5, lty=3, col="red")
References
play with different values of and

Conditional distribution: gBm
The Black and Scholes or geometric Brownian motion process solution of
(Updated
16/5/2014) dXt = 1 Xt dt + 2 Xt dWt
equations
has a log-normal transition density p (, y|x), where the log-mean and
Simulation of SDEs
Simulation strategies log-variance are given in (14)
gBm
Simulation 1
Numerical stability = log(x0 ) + 1 22 t, 2 = 22 t, (14)
2
Trick
Euler-Maruyama
pitfalls The following code implements the random number generator from the
Likelihood conditional law.
Exact sampling
sde package
Yuima rcBS <- function(n=1, Dt, x0, theta){
lmean <- log(x0) + (theta[1]-0.5*theta[2]^2)*Dt
References
lsd <- sqrt(Dt)*theta[2]
rlnorm(n, meanlog = lmean, sdlog = lsd)
}

Simulation
Suppose we want to simulate the gBm using sde.sim.
(Updated
16/5/2014) dXt = 1 Xt dt + 2 Xt dWt
equations
Simulation of SDEs
gBm > d <- expression(1*x)
Simulation
> s <- expression(0.5*x)
Numerical stability
Trick
Euler-Maruyama
pitfalls
Likelihood > set.seed(123)
Exact sampling
> sde.sim(X0=10,drift=d, sigma=s,T=10,N=1000) -> X
> plot(X,main="gBm")
sde package
Yuima
References

Conditional distribution: Cox-Ingersoll-Ross
The conditional density of Xt+ |Xt = x for the Cox-Ingersoll-Ross
(Updated
process solution of
16/5/2014)
p
equations dXt = (1 2 Xt )dt + 3 Xt dWt
Simulation of SDEs
Simulation strategies is a noncentral chi-squared distribution. It is known that p (, y|x) is
gBm
Simulation
(u+v)
u q/2
Numerical stability p (t, y|x0 ) = ce Iq (2 uv)
Trick
v
Euler-Maruyama
pitfalls with u = cx0 e2 t , v = cy, q = 21 /32 1, where Iq () is the modified
Likelihood
Bessel function of the first kind of order q.
Exact sampling
sde package
Yuima For all other processes used in finance, the conditional distribution is not
References known and so other methods should be applied.

Simulation
Suppose we want to simulate the CIR process using sde.sim.
(Updated
16/5/2014)
p
dXt = (1 2 Xt )dt + 3 Xt dWt
equations
Simulation of SDEs
gBm > d <- expression(1-2*x)
Simulation > s <- expression(0.5*sqrt(x))
Numerical stability
Trick
Euler-Maruyama
pitfalls
Likelihood > set.seed(123)
Exact sampling > sde.sim(X0=10,drift=d, sigma=s,T=10,N=1000) -> X
sde package > plot(X,main="CIR")
Yuima
References

Approximation of the SDE before discretization
(Updated Euler-Maruyama scheme assumes a constant drift between two simulated
16/5/2014) points
equations Xi+1 = Xi + b(Xi )t + W
Simulation of SDEs
Ozaki propose a linear approximation of b instead a simple constant
gBm function. The method is called Local Linear Localization because
Simulation linearization is allowed to change on the local interval. The resulting
Numerical stability
approximate solution of the SDE is
Trick
Euler-Maruyama
pitfalls
Z t+t
Kt t
Likelihood Xt+t = Xt e + eKt (t+tu) dWu
Exact sampling t
sde package
Yuima where
1 b(Xt )
References Kt = log 1 + ebx (Xt )t 1
t Xt bx (Xt )

Local Linearization Method
The integral
Z t+t
eKt (t+tu) dWu
t
is a Gaussian r.v., hence the conditional distribution of the approximated solution is such
that Xt+t |Xt = x N (Ex , Vx ), where
b(x) bx (x)t
Ex = x+ e 1 ,
bx (x)
e 2Kx t 1
Vx = 2 ,
2Kx
with
1 b(x)
Kx = log 1 + ebx (x)t 1
t xbx (x)
For non homogeneous SDEs the method is called Shoji-Ozaki.

Comparison: numerical stability
Consider
(Updated
16/5/2014) dXt = (5 11Xt + 6Xt2 Xt3 )dt + dWt , X0 = 5
equations
Simulation of SDEs
We simulate a path from Xt using the Euler-Maruyama/Milstein, Ozaki
gBm and Shoji-Ozaki method for = 0.1 (up in the next plot) and = 0.25
Simulation (down)
Numerical stability
Trick
Euler-Maruyama For small values of , all three methods gave similar results, but this is
pitfalls
Likelihood
not the case for = 0.25
Exact sampling
sde package
Yuima
References

Numerical stability. Up|Down = 0.1|0.25

Lamperti transform
Given any SDE with (t, x) = (x) (and provided regularity conditions)
(Updated
by applying the Lamperti transform
16/5/2014)
Stochastic differential Xt
1
Z
equations
F (Xt ) = du
Simulation of SDEs (u)
gBm we have that Yt = F (Xt ) has unitary diffusion coefficient, i.e.
Simulation
Numerical stability
Trick
dYt = bY (t, Yt )dt + dWt ,
Euler-Maruyama
pitfalls
where
Likelihood
b(t, F 1 (y)) 1 1
Exact sampling bY (t, y) = 1
x (F (y))
sde package (F (y)) 2
Yuima
References

Trick
If possible, it is better to apply the Lamperti transform F to X and
(Updated simulate Y = F (X) by Euler-Maruyama scheme.
16/5/2014)
equations This makes the simulation more stable and usually of higher strong order
Simulation of SDEs .
gBm Then apply F 1 (Y ) to obtain X on the original scale. Example...
Simulation
Numerical stability
Trick
Euler-Maruyama
pitfalls
Likelihood
Exact sampling
sde package
Yuima
References

Lamperti transform
dXt = (0 + 1 Xt + 2 Xt2 + 3 Xt3 )dt + Xt dWt ,

where (0 = 6, 1 = 11, 2 = 6, 3 = 1, = 0.5, = 1). The Lamperti transform is
Z x
1 1 1 x1 1
1
F (x) = du = and its inverse F (y) = (y(1 )) 1
0 u 1
or Shoji-Ozaki method can be used in this case. The drift function of process Yt is then
b(t, x) 1
by (t, x) =
x ,
x 2
which has to be calculated in F 1 (y). We have that F 1 (y) = (y/2)2 . Hence the process
Yt satisfies
23 11Yt2 + 32 Yt4 1
2 4 Y t
6 p
dYt = dt + dWt , Y0 = 2 X0 .
2Yt
Given a trajectory of Y , X = (Y /2) 2.

S.M. Iacus 2014 t t t Simulation of SDEs 72 / 105
Simulation
This is the corresponding code camparing Shoji-Ozaki and Lamperti
(Updated transform
16/5/2014)
equations > DT <- 0.25
> set.seed(123)
Simulation of SDEs > X <- sde.sim(drift=bX, delta=DT,X0=x0)
Simulation strategies > plot(X, main="Euler-Maruyama")
gBm > set.seed(123)
Simulation > Y <- sde.sim(drift=bX, method="ozaki",delta=DT,X0=x0)
Numerical stability > plot(Y,main="Ozaki")
Trick > set.seed(123)
Euler-Maruyama > Z <- sde.sim(drift=bX, method="shoji",delta=DT,X0=x0)
pitfalls > lines((Y/2)^2,col="red")
Likelihood #plot(Z,main="Shoji-Ozaki")
Exact sampling
sde package
Yuima
References

Predictor-corrector method
Euler-Maruyama and Milstein discretization schemes consider the
(Updated coefficients b and as not varying during the time interval t.
16/5/2014)
equations This is clearly untrue for a generic stochastic differential equation, as b
Simulation of SDEs and can depend on both the time t and the state of the process Xt .
gBm One way to recover the varying nature of these coefficients is to average
Simulation
Numerical stability
their values in some way.
Trick
Euler-Maruyama
pitfalls
This method highly increases the stability of the simulation.
Likelihood
Exact sampling The predictor-corrector algorithm is as follows.
sde package
Yuima
References

Predictor-corrector method
First consider the simple approximation (the predictor)
(Updated

16/5/2014) Yi+1 = Yi + b(ti , Yi )t + (ti , Yi ) tZ.
equations
Then choose two weighting coefficients and in [0, 1], and calculate the
Simulation of SDEs
Simulation strategies corrector as
gBm
Simulation Yi+1 = Yi + b(ti+1 , Yi+1 ) + (1 )b(ti , Yi ) t
Numerical stability

Trick
Euler-Maruyama
+ (ti+1 , Yi+1 ) + (1 )(ti , Yi ) tZ
pitfalls
Likelihood
Exact sampling
with
sde package b(ti , Yi ) = b(ti , Yi ) (ti , Yi )x (t, Yi ) .
Yuima
References
If = = 0, then the scheme is just the standard Euler-Maruyama

scheme. Usually = = 12 .
More consideration about Euler-Maruyama scheme
Consider again the geometric Brownian motion Xt , which is a
(Updated non-negative process or, better, if X0 > 0, this process is always positive,
16/5/2014)
being an exponential functional of the Brownian motion. This is always
equations the case for financial data.
Simulation of SDEs
Simulation strategies The application of the Euler-Maruyama approximation can lead to
gBm
unexpected results. Indeed, the Euler-Maruyama scheme for Xt reads as
Simulation
Numerical stability
Trick Yi+1 = Yi + 1 Yi t + 2 Yi tZ
Euler-Maruyama
pitfalls = Yi (1 + 1 t + 2 tZ)
Likelihood
Exact sampling
sde package
and if t is too small, it can happen in one or more simulations that a
Yuima pseudo random number Z is drawn from the Gaussian distribution such
References that
1 + 1 t
Z<
2 t
and therefore Yi+1 takes negative values.
Euler-Maruyama as Local Gaussian Approximation
Under high-frequency sampling, the Euler-Maruyama scheme performs relatively well. In

theory!
Xt+t Xt = b(Xt , )t + (Xt , )(Wt+t Wt ),
and the increments Xt+t Xt are then independent Gaussian random variables with
mean b(Xt , )t and variance 2 (Xt , )t. Therefore the transition density of the
process can be written as
2

1 1 (y x b(x, )t )
p (t , y|x) = exp
t 2 (x, )
p
2
2t (x, ) 2
In theory, because even for Gaussian models, like OU, the bias is strictly related to .
Moreover, the Gaussian assumption is quite strong! A few examples.

Euler-Maruyama, and bias
Consider OU model
(Updated
16/5/2014) dXt = (1 2 Xt )dt + 3 dWt , X 0 = x0
equations
Both true and Euler-Maruyama approximation are Gaussian respectively
Simulation of SDEs
Simulation strategies with mean and variance
gBm
2 2

Simulation 2 1

2
3 1 e 2
Numerical stability
m(, x) = xe + 1e , v(, x) = ,
2 22
Trick
Euler-Maruyama
pitfalls and (Euler-Maruyama)
Likelihood
Exact sampling
mEuler (, x) = x(1 2 ) + 1 , v Euler (, x) = 32 ,
sde package
Yuima
Only under high-frequency setting, i.e. 0, the approximation is
References
acceptable.

Euler-Maruyama, and gaussianity
Consider the gBM model
dXt = 1 Xt dt + 2 Xt dWt , X 0 = x0
The true transition density is log-normal but Euler-Maruyama approximation is Gaussian !

Exact sampling (i.i.d. case)
In the i.i.d. case, let f and g be two densities such that f (x) g(x) for
(Updated
some > 0. If we know how to simulate from g but not from f , we can
16/5/2014)
use exact sampling (or acceptance/rejection) algorithm to simulate from f
equations
Simulation of SDEs
1. Sample y from g, y g.
gBm 2. Sample u from the uniform law, u U (0, 1).
Simulation
3. If u < f (y)/g(y), retain y; otherwise iterate from 1.
Numerical stability
Trick
Euler-Maruyama
pitfalls Then y will be f -distributed.
Likelihood
Exact sampling
sde package
If the number of iterations to accept n points is small, then the method is
Yuima worth using.
References
What are f and g in the SDE case?

Exact sampling (i.i.d. case)
In the i.i.d. case, let f and g be two densities such that f (x) g(x) for
(Updated
some > 0. If we know how to simulate from g but not from f , we can
16/5/2014)
use exact sampling (or acceptance/rejection) algorithm to simulate from f
equations
Simulation of SDEs
1. Sample y from g, y g.
gBm 2. Sample u from the uniform law, u U (0, 1).
Simulation
3. If u < f (y)/g(y), retain y; otherwise iterate from 1.
Numerical stability
Trick
Euler-Maruyama
pitfalls Then y will be f -distributed.
Likelihood
Exact sampling
sde package
If the number of iterations to accept n points is small, then the method is
Yuima worth using.
References
What are f and g in the SDE case? We need Girsanov theorem.

Girsanov theorem
Consider
(Updated
16/5/2014) (1)
dXt = b1 (Xt )dt + (Xt )dWt , X0 , 0 t T,
equations dXt = (Xt )dWt , X0 , 0 t T,
Simulation of SDEs
Simulation strategies and denote by P1 , and P the probability measures induced by the
gBm solutions of the SDEs. Assume further that the initial values are either
Simulation
Numerical stability
random variables with densities f1 (), and f () with the same common
Trick support or nonrandom and equal to the same constant.
Euler-Maruyama
pitfalls
Likelihood Then P1 and P are equivalent and the corresponding Radon-Nikodm
Exact sampling derivative is
sde package
Z T Z T 2
Yuima
dP1 f1 (X0 ) b1 (Xs ) 1 b1 (Xs )
References (X) = exp 2
dX s 2
ds (15)
dP f (X0 ) 0 (Xs ) 2 0 (Xs )

Likelihood
Remark that (15) is also the likelihood function for continuous time observations from an
SDE
A nice example inspired by M.Keller-Ressel and T.Steiner (WikiPedia). Suppose we have

a standard Brownian motion Wt on [0, 1] rescaled by a constant ,
Xt = Wt ,
and the same rescaled Brownian motion with drift
Yt = t + Wt , 0 t 1,
with < 0 and > 0. We apply Girsanovs formula (15) to weight the trajectories of the
process Yt .

An example
We have that
(Updated
1 1
0 02
2
Z
dP2 1
Z
16/5/2014)
(Y ) = exp 2
dY s 2
dt
equations
dP1 0 2 0
Z 1 2

Simulation of SDEs 1
= exp 2
(ds + dWs ) +
Simulation strategies 0 2 2
gBm
2

Simulation 2 1
= exp + W1 +
Numerical stability 2 2
Trick
Euler-Maruyama W1 1 2
pitfalls = exp .
Likelihood 2
Exact sampling
sde package
Yuima In the next plot, the darker trajectories are more likely to come from Yt
References

Likelihood
1.0
0.5
0.5
0.0 0.2 0.4 0.6 0.8 1.0

1.0
t
0.5
0.5
0.0 0.2 0.4 0.6 0.8 1.0

Exact sampling for SDEs
(Beskos & Roberts, 2005) Consider
(Updated
16/5/2014) dXt = b(Xt )dt + dWt , 0 t T, X0 = x
equations
We want to simulate the random variable X for some > 0 and initial
Simulation of SDEs
Simulation strategies value X0 = x.
gBm
Simulation The main assumption of the EA algorithm (besides regularity) are the
Numerical stability
Trick
following:
Euler-Maruyama
pitfalls 1 1
Likelihood k1 , k2 : k1 b2 (x) + bx (x) k2 , x R (16)
Exact sampling
2 2
sde package
Yuima
Denote (x) = 12 b2 (x) + 21 bx (x) k1 , so 0 (x) M , with
References
M = (k2 k1 ).
For identifiability it is required that 1/M

The algorithm
Rz
Let A(z) = 0 b(u)du and
x)2

(z
h(z) = exp A(z) /K
2
where K is a normalizing constant. EA is as follows:

1. Simulate Y = y according to distribution h.
2. Simulate k from the Poisson distribution with intensity = M .
3. Draw (Ti , Vi ) = (ti , vi ) according to U [(0, ) (0, M )], i = 1, . . . , k.
4. Generate a Brownian bridge starting at x at time 0 and ending in y at time at time
instants ti ; i.e., generate Yti = yi , i = 1, . . . , k.
Yk
5. Compute the indicator function I = {(yi )vi } .
i=1
6. If I = 1, the trajectory (x, Yt1 = y1 , . . . , Ytk = yk , Y = y) is accepted and Y is an
exact draw of X . Otherwise, restart from step 1.

Facts & Limitations
It is important to mention that the probability of the event I = 1 in the
(Updated algorithm above, which is the probability of accepting a simulated path,
16/5/2014)
exponentially decreases to zero as 0 and is at least e1 , which
equations justifies using this rejection algorithm from the point of view of efficiency.
Simulation of SDEs
gBm
Assumption (16)
Simulation
1 2 1
Numerical stability k1 b (x) + bx (x) k2
Trick 2 2
Euler-Maruyama
pitfalls is strictly related to the ability of bounding the likelihood ratio which is
Likelihood needed to perform exact sampling.
Exact sampling
sde package
Yuima
Unfortunately it may be too restrictive. Many quite important processes
References
dont met this assumption.
Versions 2 and 3 of the algorithm have been proposed to remove one

bound or both bounds. The resulting algorithm is much more involved.
Periodic drift example (SINE process)
Consider the following example from [?]. We have a process satisfying the stochastic
differential equation
dt = sin(t )dt + dWt , 0 = 0 . (17)
The drift b(x) = sin(x) satisfies the usual hypotheses, and is differentiable and such that
1 1 2 1 1 2 1 5
= k1 b(u) + bx (u) = sin(u) + cos(u) k2 = ;
2 2 2 2 2 8
hence M = k2 k1 = 9/8 and A(u) = 1 cos(u). The following code simulates an
exact path of the SINE process.
> set.seed(123)
> d <- expression(sin(x))
> d.x <- expression(cos(x))
> A <- function(x) 1-cos(x)
> sde.sim(method="EA", delta=1/20, X0=0, N=500, drift=d,
drift.x = d.x, A=A) -> X
> plot(X, main="Periodic drift")
Hyperbolic process
Consider the process q

dXt = 1 Xt dt + 2 1 + Xt2 dWt
with 1 + 22 /2 > 0. Using the Lamperti transform Yt = F (Xt )
dYt = (1 /2 + 2 /2) tanh(2 Yt )dt + dWt , Y0 = F (X0 )
1 2 8+16(sinh(2x))2
Choose 1 = 6 and 2 = 2. In this case 2 (b (x) + bx (x)) = 2(cosh(2x))2
, and
1
4 = k1 (b(x)2 + bx (x)) k2 = 8
2
and Z x
A(x) = 4 tanh(2u)du = 2 log(cosh(2x)).
0
x2
A(x) 2T
R
Hence M = k2 k1 = 12, 0 (x) 1/M , and the constant K = Re can be
found numerically.
Hyperbolic process
Original scale X vs transformed Y

8
0
6
50
X
4
(b(x)2 + bx)
2 150
2
1
2
0
0
2
Y
4
2
6
4
0 5 10 15 20 25
4 2 0 2 4 Time
Figura 1: Graph of the curve 12 (b(x)2 + bx (x)).
Once we have simulated a path of Yt using the exact algorithm, we can obtain a trajectory
of Xt using the inverse of F ; i.e., Xt = F 1 (Yt ) = sinh(2 Yt ). See R code.

Hyperbolic process
> set.seed(123)
> d <- expression(-4*tanh(2*x))
> d.x <- expression(-(4 * (2/cosh(2 * x)^2)))
> A <- function(x) -(0.5+6/4)*log(cosh(2*x))
> X0 <- rt(1, df=4)/2
> F <- function(x) log(x + sqrt(1+x^2))/2
> Y0 <- F(X0)
> sde.sim(method="EA", delta=1/20, X0=Y0, N=500, drift=d,
+ drift.x=d.x, A=A, k1=-4,k2=8) -> Y
> X <- sinh(Y)
> ts(cbind(X,Y),start=0,delta=1/20) -> XY
> plot(XY,main="Original scale X vs transformed Y")

CIR process
Consider the Cox-Ingersoll-Ross process

p
dXt = (1 2 Xt )dt + 3 Xt dWt , X0 = 10 ,
for which the Lamperti transform

x
1 2 x
Z
F (x) = du =
0 3 u 3
gives Yt = F (Xt ), satisfying
41 32 2
dYt = 2 dt Yt dt + dWt .
23 Yt 2
41 32
The transformed drift b(x) = 22 x is not bounded in zero from above or below
232 x
and hence the EA algorithm is not applicable to this process. The same situation occurs for
the Ornstein-Uhlenbeck process.
Simulation of diffusion bridges
Consider
(Updated dXt = b(Xt )dt + (Xt )dWt
16/5/2014)
Stochastic differential and take a and b as two points in the state space of X. A solution of the
equations
previous equation in the interval [t1 , t2 ] such that Xt1 = a and Xt2 = b is
Simulation of SDEs
called a (t1 , x1 , t2 , x2 )-diffusion bridge.
gBm
Simulation In SLE (Simulated Likelihood Estimation) the interest is in the simulation
Numerical stability
of a (0, a, 1, b)-diffusion bridges.
Trick
Euler-Maruyama
pitfalls
Likelihood
Exact sampling
sde package
Yuima
References

Simulated diffusion bridges
Let W 1 and W 2 be two independent Wiener processes and define X 1 and X 2 as solutions
to
dXti = b(Xti )dt + (Xti )dWti
with X01 = a and X02 = b.
2 }, where the inf over the empty set is taken to be .
Let = inf{0 t 1|Xt1 = X1t
Define (
Xt1 if 0 t ,
Zt = 2
X1t if < t 1.
Then, the distribution of {Zt }0t1 conditional on the event { 1} is equal to the
conditional distribution of {Xt }0t1 given X0 = a and X1 = b; i.e., Z is a
(0, a, 1, b)-diffusion bridge. (Bladt, Srensen, 2007).

4
3 Diffusion bridges
Y1
2
1
0.0 0.2 0.4 0.6 0.8 1.0

4
Time
3
Y1
2
1
0.0 0.2 0.4 0.6 0.8 1.0

Diffusion Bridges
> d <- expression((3-x))

> s <- expression(1.2*sqrt(x))
> par(mar=c(3,3,1,1))
> par(mfrow=c(2,1))
> set.seed(123)
> X <- DBridge(x=1.7,y=0.5, delta=0.01, drift=d, sigma=s)
> plot(X)
> X <- DBridge(x=1,y=5, delta=0.01, drift=d, sigma=s)
> plot(X)

The sde package
The package R package sde on CRAN provides functionalities to simulate
(Updated
multiple independent trajectories of processes solution to 1-dimensional
16/5/2014)
SDEs via the function
equations
Simulation of SDEs sde.sim

gBm
by any of these methods.
Simulation
Numerical stability
Trick
Euler-Maruyama
pitfalls
Likelihood
Exact sampling
sde package
Yuima
References

The sde package
The sde package implements many simulation schemes, including: Euler-Maruyama,

Milstein, Milstein2, Elerian, Ozaki, Ozaki-Shoji, Exact Simulation Scheme, Simulation
from conditional distribution, Predictor-Correction scheme, etc via the unique sde.sim
function
sde.sim(t0 = 0, T = 1, X0 = 1, N = 100, delta, drift, sigma,

drift.x, sigma.x, drift.xx, sigma.xx, drift.t,
method = c("euler", "milstein", "KPS", "milstein2",
"cdist","ozaki","shoji","EA"),
alpha = 0.5, eta = 0.5, pred.corr = T, rcdist = NULL,
theta = NULL, model = c("CIR", "VAS", "OU", "BS"),
k1, k2, phi, max.psi = 1000, rh, A, M=1)

The sde.sim function
For the OU process, dXt = 5Xt dt + 3.5dWt , it is as easy as
> d <- expression(-5 * x)

> s <- expression(3.5)
> sde.sim(X0=10,drift=d, sigma=s) -> X
> str(X)
Time-Series [1:101] from 0 to 1: 10 9.32 8.79 8.89 8.48 ...

The sde.sim function

For the CIR model dXt = (6 3Xt )dt + 2 Xt dWt
> d <- expression( 6-3*x )

> s <- expression( 2*sqrt(x) )
> sde.sim(X0=10,drift=d, sigma=s) -> X
or, via model name
> sde.sim(X0=10, theta=c(6, 3, 2), model="CIR") -> X
or, via exact conditional distribution rcCIR (also implemented in sde)
> sde.sim(X0=10, theta=c(6,3,2), rcdist=rcCIR, method="cdist") -> X

One step beyond
The sde package is very powerful but it is 1-dim only, next step is
(Updated
16/5/2014) Yuima
equations
Simulation of SDEs
gBm
Simulation
Numerical stability
Trick
Euler-Maruyama
pitfalls
Likelihood
Exact sampling
sde package
Yuima
References

(Updated
16/5/2014)
equations
Simulation of SDEs
References References

Some references
Stochastic processes At-Sahalia, Y. (1996) Testing continuous-time models of the spot interest rate, Rev. Financial Stud., 9(2), 385426.
(Updated Banks, H.T. (1975) Modeling and Control in the Biological Sciences, Letc. Notes Biotmath, 6, Springer-Verlag, Berlin.
Bailey, N.T.J. (1957) The Mathematical Theory of Epidemics, Griffin, London.
16/5/2014)
Bandi, F., Phillips, P. (2003) Fully nonparametric estimation of scalar diffusion models, Econometrica, 71, 241283.
Stochastic differential Bladt, M., Srensen, M. (2007) Simple simulation of diffusion bridges with application to likelihood inference for diffusions, Working
equations Paper, University of Copenhagen.
Simulation of SDEs Bergstrom, A.R. (1990) Continuous Time Econometric Modeling, Oxford University Press, Oxford.
Beskos, A. and Roberts, G.O. (2005) Exact simulation of diffusions, Ann. Appl. Probab., 4, 24222444.
References
Bibby, B., Jacobsen, M., Srensen, M. (2007) Estimating functions for discretely sampled diffusion-type models, in Handbook of
Financial Econometrics, At-Sahalia, Y., Hansen, L.P., eds, North-Holland, Amsterdam.
Bibby, B., Srensen, M. (1995) Martingale estimating functions for discretely observed diffusion processes, Bernoulli, 1, 1739.
Black, F., Scholes, M.S. (1973) The pricing of options and corporate liabilities, Journal of Political Economy, 81, 637-654.
Chan, K.C., Karolyi, G.A., Longstaff, F.A., Sanders, A.B. (1992) An empirical investigation of alternative models of the short-term
interest rate, J. Finance, 47, 12091227.
Cobb, L. (1981) Stochastic Differential Equations for the Social Sciences, in Mathematical Frontiers of the Social and Policy Sciences,
Cobb & Thrall eds, Westview Press, 1-26.
Cox, J.C., Ingersoll, J.E., Ross, S.A. (1985) A theory of the term structure of interest rates, Econometrica, 53, 385-408.
Dacunha-Castelle, D., Florens-Zmirou, D. (1986) Estimation of the coefficients of a diffusion from discrete observations, Stochastics,
19, 263284.
Ditlevsen, P.D., Ditlevsen, S., Andersen, K.K. (2002) The fast climate fluctuations during the stadial and interstadial climate states,
Ann. Glaciol., 35, 457-462.
Elerian, O. (1998) A note on the existence of a closed form conditional density for the Milstein scheme, Working Paper, Nuffield
College, Oxford University. Available at http://www.nuff.ox.ac.uk/economics/papers/

Some references
Stochastic processes Hanes, D.P., Schall, J.D. (1988) Neural control of voluntary movement initiation, Science, 274, 427-430.
(Updated Holden, A.V. (1976) Models for Stochastic Activity of Neurones, Springer-Verlag, New York.
Holland , C.J. (1976) On a Formula in Diffusion Processes in Population Genetics, Proceedings of the American Mathematical
16/5/2014) Society, 54(1), 316-318.
Stochastic differential Holmes, E.E. (2004) Beyond theory to application and evaluation: diffusion approximations for population viability analysis,
equations Ecological Applications, 14(4), 1272-1293.
Honore, P. (1997) Maximum Likelihood Estimation of Non-Linear Continuous-Time Term-Structure Models, Working Paper 1997-7,
Simulation of SDEs Department of Finances, Aahrus School of Business. Available at http://ssrn.com/abstract=7669.
References Hsu, D.A. (1977) Tests for variance shift at an unknown time point, Appl. Stat., 26(3), 279284.
Hull, J. (2000) Options, Futures and Other Derivatives, Prentice-Hall, Englewood Cliffs, NJ.
Iacus, S.M., De Gregorio, A. (2008) Least suqare change-point estimation for discretely observed diffusion processes, Comm. Stat.
Th. & Meth., 15, ...
Kessler, M. (1997) Estimation of an ergodic diffusion from discrete observations, Scand. J. Stat., 24, 211229.
Kushner, H.J. (1967) Stochastic Stability and Control, Academic Press, New York.
Kutoyants, Y. (1994) Identification of dynamical systems with small noise, Kluwer, Dordrecht.
Kutoyants, Y. (2004) Statistical inference for ergodic diffusion processes, Springer-Verlag, London.
Lange, K. (2002) Mathematical and Statistical Methods for Genetic Analysis, Springer-Verlag, New York.
Milstein, G.N. (1978) A method of second-order accuracy integration of stochastic differential equations, Theory Probab. Appl., 23,
396401.
ksendal, B. (1998) Stochastic Differential Equations. An Introduction with Applications, 5th ed., Springer-Verlag, Berlin.
Ozaki, T. (1992) A bridge between nonlinear time series models and nonlinear stochastic dynamical systems: A local linearization
approach, Statistica Sinica, 2, 25-83.

Some references
Stochastic processes Papanicolaou, G. (1995) Diffusions in random media, in Surveys in Applied Mathematics, J.B. Keller, D. McLaughin and G.
Papanicolaou (Eds), Plenum Press, 205-255.
(Updated
Pedersen, A. R. (1995) A new approach to maximum likelihood estimation for stochastic differential equations based on discrete
16/5/2014) observations. Scand. J. Statist., 22, 55-71.
Prakasa Rao, B.L.S. (1999) Statistical Inference for Diffusion Type Processes, Oxford University Press, New York.
equations Ricciardi, L.M. (1977) Diffusion Processes and Related Topics in Biology, Lecture Notes in Biomathematics, Springer, New York.
Santa-Clara, P. (1995) Simulated likelihood estimation of diffusion with application to the short term interest rate, Anderson Graduate
Simulation of SDEs School of Management, University of California at Los Angeles.
References Shoji, L., Ozaki, T. (1998) Estimation for nonlinear stochastic differential equations by a local linearization method, Stochastic
Analysis and Applications, 16, 733-752.
Schuecker, P., Bhringer, H., Arzner, K.,Reiprich, T. H. (2001) Cosmic mass functions from Gaussian stochastic diffusion
processes, Astronomy & Astrophysics, 370, 715-728.
Srensen, M. (1997) Estimating functions for discretely observed diffusions: a review, in Selected Proceedings of the Symposium on
Estimating Functions, Basawa, I.V., Godabme, V.P., Taylor, R.L., eds, Volume 32, IMS Lecture Notes, Institute of Mathematical
Statistics, Hayward, CA, 305325.
Srensen, M. (1999) On asymptotics of estimating functions, Braz. J. Probab. Stat., 13, 111136.
Uchida, M., Yoshida, N. (2005) AIC for ergodic diffusion processes from discrete observations, preprint MHF 2005-12, March 2005,
Faculty of Mathematics, Kyushu University, Fukuoka, Japan.
Uhlenbeck, G. E., Ornstein, L. S. (1930) On the theory of Brownian motion, Phys. Rev., 36, 823-841.
Vasicek, O. (1977) An Equilibrium Characterization of the Term Structure, Journal of Financial Economics, 5, 177-188.
Yoshida, N. (1992) Estimation for diffusion processes from discrete observation, Journal of Multivariate Analysis, vol 41, n. 2,
220-242.
Zhou, H. (2001) Finite sample properties of EMM, GMM, QMLE, and MLE for a square-root interest rate diffusion model, J. Comput.
Finance, 5, 89122.

Stoch Proc Sim

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Stoch Proc Sim

Uploaded by

Copyright:

Available Formats

Stochastic processes

S.M. Iacus 2014 Simulation of SDEs 1 / 105

References For a given random event , we call X(t, ) a trajectory or path of

A trajectory represents the dynamic evolution of the stochastic process in

We use the notation Xt = X(t) = X(t, ) whenever needed.

S.M. Iacus 2014 Simulation of SDEs 2 / 105

S.M. Iacus 2014 Simulation of SDEs 3 / 105

S.M. Iacus 2014 Simulation of SDEs 4 / 105

in particular Bt N (0, t). We also use the notation Wt = Bt

S.M. Iacus 2014 Simulation of SDEs 5 / 105

Thus, we can simulate one such path as follows.

S.M. Iacus 2014 Simulation of SDEs 6 / 105

S.M. Iacus 2014 Simulation of SDEs 7 / 105

S.M. Iacus 2014 Simulation of SDEs 8 / 105

But the code above is highly inefficient as it implements an iteration using

W <- c(0,cumsum( sqrt(Delta) * rnorm(N)))

due to the fact that the algorithm

S.M. Iacus 2014 Simulation of SDEs 8 / 105

0.0 0.2 0.4 0.6 0.8 1.0

where convergence is in distribution.

S.M. Iacus 2014 Simulation of SDEs 11 / 105

S.M. Iacus 2014 Simulation of SDEs 12 / 105

References |W (0.5 + t) W (0.5)|

S.M. Iacus 2014 Simulation of SDEs 13 / 105

0.000 0.002 0.004 0.006 0.008 0.010

S.M. Iacus 2014 Simulation of SDEs 15 / 105

S.M. Iacus 2014 Simulation of SDEs 16 / 105

S.M. Iacus 2014 Simulation of SDEs 17 / 105

S.M. Iacus 2014 Simulation of SDEs 18 / 105

S.M. Iacus 2014 Simulation of SDEs 19 / 105

dXt = b(Xt )dt + (Xt )dWt

deterministic trend + stochastic noise

S.M. Iacus 2014 Simulation of SDEs 20 / 105

S.M. Iacus 2014 Simulation of SDEs 21 / 105

Simulation of SDEs Wt the Wiener process

Rp , are the parameters of interest

S.M. Iacus 2014 Simulation of SDEs 22 / 105

physics (see e.g. Papanicolaou, 1995)

S.M. Iacus 2014 Simulation of SDEs 23 / 105

dXt = (1 + 2 Xt )dt + 3 Xt4 dWt

S.M. Iacus 2014 Simulation of SDEs 24 / 105

Jacobi diffusion (political polarization)

S.M. Iacus 2014 Simulation of SDEs 25 / 105

hyperbolic diffusion (dynamics of sand)

S.M. Iacus 2014 Simulation of SDEs 26 / 105

A strong solution is also a weak solution.

S.M. Iacus 2014 Simulation of SDEs 27 / 105

S.M. Iacus 2014 Simulation of SDEs 28 / 105

S.M. Iacus 2014 Simulation of SDEs 29 / 105

S.M. Iacus 2014 Simulation of SDEs 30 / 105

S.M. Iacus 2014 Simulation of SDEs 31 / 105

S.M. Iacus 2014 Simulation of SDEs 32 / 105

S.M. Iacus 2014 Simulation of SDEs 33 / 105

from which nonparametric inference is usually possible as is usually done.

S.M. Iacus 2014 Simulation of SDEs 34 / 105

(L is the basis of many statistical procedures)

S.M. Iacus 2014 Simulation of SDEs 35 / 105

S.M. Iacus 2014 Simulation of SDEs 36 / 105

S.M. Iacus 2014 Simulation of SDEs 37 / 105

S.M. Iacus 2014 Simulation of SDEs 38 / 105

S.M. Iacus 2014 Simulation of SDEs 39 / 105

S.M. Iacus 2014 Simulation of SDEs 40 / 105

S.M. Iacus 2014 Simulation of SDEs 41 / 105