Standard errors Computer simulation §§15.

1–3

wi3425TU—Monte Carlo methods

L.E. Meester

Week 2

1/ 35
 Standard errors Computer simulation §§15.1–3

Week 2—Program for this week

1 Canonical Monte Carlo and standard errors

Canonical Monte Carlo
Standard errors: the two most important cases
2 Higham Chapter 4: Computer simulation
Stochastic simulation: is it for real?
Pseudo random number generators (background to §4.2)
How to see if the RNG is OK (Higham §4.3??)
3 Higham Sections 15.1–3
Computational example §15.2: effect of estimating σX (= b)
Pricing options by Monte Carlo simulation (§15.3)
Vectorising code

Don’t forget: Do the weekly practice material and upload your

solutions and we’ll provide you with feedback!
2/ 35
 Standard errors Computer simulation §§15.1–3 Canonical Monte Carlo Standard errors

1 Canonical Monte Carlo and standard errors

Canonical Monte Carlo
Standard errors: the two most important cases

2 Higham Chapter 4: Computer simulation

Stochastic simulation: is it for real?
Pseudo random number generators (background to §4.2)
How to see if the RNG is OK (Higham §4.3??)

3 Higham Sections 15.1–3

Computational example §15.2: effect of estimating σX (= b)
Pricing options by Monte Carlo simulation (§15.3)
Vectorising code

3/ 35
 Standard errors Computer simulation §§15.1–3 Canonical Monte Carlo Standard errors

The canonical Monte Carlo situation

Recall:
I.i.d. sequence (of random variables)
A sequence X1 , X2 , . . . of random variables is called independent
and identically distributed (or iid) if
all Xi have the same distribution;
the collection X1 , . . . , Xn is independent, for any n.

In a typical Monte Carlo simulation

1 you produce a large number of iid replications of your model,
2 you compute an average to estimate some quantity, and
3 you compute a standard error (or confidence interval) to know
its accuracy.

4/ 35
 Standard errors Computer simulation §§15.1–3 Canonical Monte Carlo Standard errors

Known rules:
Variance of iid sums and averages
If X1 , . . . , Xn are iid with E [X1 ] = µ and Var(X1 ) = σ 2 . Then

σ2
 
2 X1 + · · · + Xn
Var(X1 + · · · + Xn ) = nσ and Var = .
n n

√
The average we denote by X̄n , so SD(X̄n ) = σ/ n.

When X̄n is used to estimate µ, we call SD(X̄n ) the standard error.

5/ 35
 Standard errors Computer simulation §§15.1–3 Canonical Monte Carlo Standard errors

Standard errors: the two most important cases

First: suppose we know σ and p.

If X1 , . . . , Xn are iid with E [X1 ] = µ and Var(X1 ) = σ 2 , the
standard error is
σ
s.e.(X̄n ) = √ .
n
If we are estimating a probability, say p, our estimate is the
average of a sequence of 0s and 1s, where the 1s occur with
probability p.
This is as in the previous case where now X1 , . . . , Xn are iid
Bernoulli with parameter p, whence σ 2 = p (1 − p).
So, if we use p̂ to denote the estimate, then
r
p (1 − p)
s.e.(p̂) = .
n

6/ 35
 Standard errors Computer simulation §§15.1–3 Canonical Monte Carlo Standard errors

. . . and their estimates

In practice, we usually don’t know σ and p:

If we don’t know σ, we use the sample standard deviation of
our data (Matlab: function std; R: sd) to estimate it.

σ̂
s.e.(X̄n ) ≈ √ .
n

In case we need s.e.(p̂), we insert our estimate p̂ into the

formula, which (if n is not too small) usually gives a good
enough approximation:
r
p̂ (1 − p̂)
s.e.(p̂) ≈ .
n

7/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

1 Canonical Monte Carlo and standard errors

Canonical Monte Carlo
Standard errors: the two most important cases

2 Higham Chapter 4: Computer simulation

Stochastic simulation: is it for real?
Pseudo random number generators (background to §4.2)
How to see if the RNG is OK (Higham §4.3??)

3 Higham Sections 15.1–3

Computational example §15.2: effect of estimating σX (= b)
Pricing options by Monte Carlo simulation (§15.3)
Vectorising code

8/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Stochastic simulation, how is that possible?

Generating random numbers:

Suppose I only had a roulette wheel or some dice,

could I do last week’s simulation for π?
How do computers produce random numbers?
Do they, actually?
9/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Background to Higham §4.2

Pseudo-random number generators (PRNGs)

PRNGs aim to produce sequences U1 , U2 , . . . , such that

Each Ui is uniformly distributed on the interval [0, 1].
U1 , U2 , . . . , is a collection of independent random variables.
So, for short: an iid standard uniform sequence.
This raises some questions, philosophical as well as practical,
for example: How could one check if someone claims to have
accomplished this?
Designing PRNGs is really a specialists’ topic. Big names:
Pierre L’Ecuyer (links): TestU01, RngStreams, his website.
George Marsaglia: lattice structure of LCGs, diehard tests.
Donald Knuth: The art of computer programming.

10/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

General framework for PRNGs (L’Ecuyer)

Each PRNG can be seen as a quadruple (E , µ, f , g ), where

E is some finite set of states, e.g., {0, 1, . . . , M};
µ: a “random” mechanism for the seed, the starting value x0 ;
f : E → E is the recursion function: xn+1 = f (xn );
g : E → [0, 1] is the output map: un = g (xn ).
Usually g (E ) ⊂ (0, 1), and always it is a strict subset of [0, 1].

The period of an RNG: smallest integer d such that an integer `

exists such that x`+d = x` , i.e., the length of the smallest cycle.

History: the midsquare method (John von Neumann . . . ).

11/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

General considerations for PRNGs

Period length:
Longer is better.
It can be no longer than the number of elements of E .
Reproducibility: same seed always produces the same
numbers.
Speed:
Functions like log, exp, sin take 5–10 as much time as a single
RNG-call (so it says in Asmussen and Glynn somewhere).
Presently, no reason to sacrifice quality for greater speed.
Portability: identical behavior of PRNG on different platforms.
How to judge “randomness”:
theoretical properties are used to narrow down the search for
good candidates;
then statistical tests are used to make final selection(s).

12/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

RANDU (IBM 360/370 series, 1970s)

In the 1970s still some bad generators were used:

RANDU’s iteration (IBM 360/370 series, 1970s):
xi
xi = 65539 · xi−1 modulo 231 output: ui = .
231
N.B.: 65539 = 216 + 3.
Easy to implement on binary computer.
Does not satisfy conditions to guarantee maximal period.
But: if seed x0 odd then period: 229 .
However: triples (Ui , Ui−1 , Ui−2 ) exhibit serious dependence!
Easy to prove and illustrate randu.m, RANDUgen.m (Matlab).

13/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Linear congruential generators (LCGs)

RANDU is an example from this class, which also contains good

generators:
Provide x0 (seed) and iterate for i ≥ 1:

xi = a xi−1 + c modulo m, output: ui = xi /m.

Parameters (nonnegative integers):

a (the multiplier), c, and m (the modulus).
Intuition: for well-chosen parameters the remainder of the
a x +c
division i−1m is a “random” number xi ∈ {0, 1, . . . , m − 1}.
LCGs are necessarily periodic (as are all PRNGs), the period
depends on a, c, m (and the seed). Knuth’s theorem gives
conditions for maximal period m.

14/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Modifications, combinations, other methods

LCG but “multiply with carry:”

axn + cn
xn+1 = (axn + cn ) mod m, cn+1 = b c.
m
Multiple recursive generators:

xi = (a1 xi−1 + a2 xi−2 + · · · + ak xi−k ) mod m,

which needs k seed values x0 , x1 , . . . xk−1 .

Generalized feedback shift registers (GFSRs, exploit binary
hardware architecture).
Overcoming defects and/or extend period by combining
generators.

15/ 35
Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Matlab: rand(’state’,42) uses a combined generator:

Marsaglia’s “subtract with borrow” and an LCG with carry;
there are some known dependency problems.
Matlab: rand(’twister’,42) uses Matsumoto and
Nishamura’s Mersenne twister generator, which uses a
generalized feedback shift register;
properties: good uniformity, period approx 106000 ;
even here some funny things have been found.
R: uses the Mersenne twister by default; other choices
possible, via command RNGkind.
Parallel processors: sometimes provisions for splitting random
streams available (see e.g., L’Ecuyer’s RngStreams).

16/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Reproducibility

Matlab program HigT41.m:

rand(’state’,100);
randn(’state’,100);
u=rand(10,1);
z=randn(10,1);
Table41=[u,z]

produces Higham Table 4.1:

17/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

1 Canonical Monte Carlo and standard errors

Canonical Monte Carlo
Standard errors: the two most important cases

2 Higham Chapter 4: Computer simulation

Stochastic simulation: is it for real?
Pseudo random number generators (background to §4.2)
How to see if the RNG is OK (Higham §4.3??)

3 Higham Sections 15.1–3

Computational example §15.2: effect of estimating σX (= b)
Pricing options by Monte Carlo simulation (§15.3)
Vectorising code

18/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Checking the RNG & Higham §4.3 Statistical “tests”

Higham’s section title suggests: this is about checking the RNG.

It is not, really: it merely shows how some distribution parameters
are reproduced. The results just illustrate the LLN and the CLT.
Of course: (distribution) characteristics should be reproduced by
the pseudo-random sequence, in a way similar to a truly random
sequence. For example expectation and variance: HigT42.m

19/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Does this prove that the Matlab generator is of good quality??

It merely illustrates the law of large numbers. . .

Would you know more rigorous checks?
20/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

What about this: Histograms of rand-calls?

What do we see here?

21/ 35
 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Or this: Histograms of randn-calls?

Do these look correct? 22/ 35

 Standard errors Computer simulation §§15.1–3 Simulation: for real? PRNGs Checking randomness

Has “everything” been tested?

Have we checked:
The distribution? A bit: one-dimensional marginal
Independence? Not at all
Both figures illustrate the LLN: the histogram converges to the
true density. We could try to answer the question: Does the
wiggliness in these plots agree with what follows from the CLT?
How: by computing a standard error for the plotted numbers
and comparing that to what we see.

Proper checks are performed using proper statistical tests, more

formal and stringent than what is done in § 4.3.
The Mersenne Twister passes most of them . . .

23/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

1 Canonical Monte Carlo and standard errors

Canonical Monte Carlo
Standard errors: the two most important cases

2 Higham Chapter 4: Computer simulation

Stochastic simulation: is it for real?
Pseudo random number generators (background to §4.2)
How to see if the RNG is OK (Higham §4.3??)

3 Higham Sections 15.1–3

Computational example §15.2: effect of estimating σX (= b)
Pricing options by Monte Carlo simulation (§15.3)
Vectorising code

24/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

Higham Chapter 15: Monte Carlo method

We covered most of Section 15.1–2 last week:

Rv X with E [X ] = a and Var(X ) = b 2 can be simulated.

Unbiased simulation estimators for a and b:
M M
1 X 2 1 X
aM = Xi and bM = (Xi − aM )2 .
M M −1
i=1 i=1
√
Then Var(aM ) = b 2 /M; the standard error for aM is b/ M.
√
M large, then aM ± 1.96 · b/ M is a 95% confidence interval
for a. Also when b is replaced with the estimate bM .
More accuracy needed? Then: M larger, or look for smaller b.

25/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

1 Canonical Monte Carlo and standard errors

Canonical Monte Carlo
Standard errors: the two most important cases

2 Higham Chapter 4: Computer simulation

Stochastic simulation: is it for real?
Pseudo random number generators (background to §4.2)
How to see if the RNG is OK (Higham §4.3??)

3 Higham Sections 15.1–3

Computational example §15.2: effect of estimating σX (= b)
Pricing options by Monte Carlo simulation (§15.3)
Vectorising code

26/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

Computational Example §15.2—effect of estimating bM

Purpose of this example: to study the role of the estimate bM .

Setup:
X = eZ , with Z ∼ N (0, 1).
√
Then E [X ] = e ≈ 1.65,
Var(X ) = e2 − e = e (e − 1) = E bM
 2
≈ 4.67.
We simulate with M = 25 , 26 , . . . , 217 ,
results are depicted in Higham Figure 15.1:

27/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

N.B.: Figures 15.1 and 15.2 are to show the effect of the sample
size. In practice you naturally do only one simulation.
28/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

What should happen (on average) to

the standard error;
the size of the confidence interval;
if M is doubled?
In the figure: interval for M = 26 larger than for M = 25 . . .

Empirical study to understand the variation in bM :

Run program HigS152.m
For each M-value one thousand samples are drawn.
Conclusion: Given
√ M the length of the confidence interval
aM ± 1.96 bM / M varies as a consequence of variation in bM .
For large samples, the effect on the determination of a is
minimal.

29/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

1 Canonical Monte Carlo and standard errors

Canonical Monte Carlo
Standard errors: the two most important cases

2 Higham Chapter 4: Computer simulation

Stochastic simulation: is it for real?
Pseudo random number generators (background to §4.2)
How to see if the RNG is OK (Higham §4.3??)

3 Higham Sections 15.1–3

Computational example §15.2: effect of estimating σX (= b)
Pricing options by Monte Carlo simulation (§15.3)
Vectorising code

30/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

Asset price model (Ch 6 and 7)

Model for prices S(t0 ), . . . , S(tn ) at instances

t0 = 0 < t1 < · · · < tn = T based on the empirical “fact” that the
returns S(ti+1 )−S(ti )
S(ti ) appear to have a normal distribution:
2 /2)(t √
i+1 −ti )+σ ti+1 −ti ·Zi
S(ti+1 ) = S(ti ) · e(µ−σ ,

with µ: the drift,

σ: the volatility, and
Z0 , . . . , Zn : iid N (0, 1) random variables.
Later we will need a price path, for now: n = 1 and t0 = 0, t1 = T :
 √ 
ST = S0 exp (r − σ 2 /2)T + σ T · Z .

31/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

§15.3 Pricing options

Many option prices can be determined as discounted expected

pay-offs under this model, with µ = r (interest rate); this is
the so-called risk-neutral distribution.
Call option: Pay-off Λ(ST ) = max(ST − E , 0) = [ST − E ]+ .
Λ: so-called pay-off function;
T : expiry, time of exercise;
E : strike price.
Option price:
 √ 
e−rT E [Λ(ST )] with ST = S0 exp (r − σ 2 /2)T + σ T Z .

This is all simple, straightforward, and known to you; see

programs EurCall3.m and EurCall4.m (adapted ch15.m).
Next week: §15.4 Estimating derivatives/Greeks.
32/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

1 Canonical Monte Carlo and standard errors

Canonical Monte Carlo
Standard errors: the two most important cases

2 Higham Chapter 4: Computer simulation

Stochastic simulation: is it for real?
Pseudo random number generators (background to §4.2)
How to see if the RNG is OK (Higham §4.3??)

3 Higham Sections 15.1–3

Computational example §15.2: effect of estimating σX (= b)
Pricing options by Monte Carlo simulation (§15.3)
Vectorising code

33/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

Matlab, vectorising code, efficiency

Matlab stands for “Matrix Laboratory”—the paradigm used to be:

All objects are matrices.

Vectorising and “matrixifying” of algorithms:

Generating a sample of Monte Carlo replications by writing
everything in terms of a few vector/matrix operations.
Often faster, because of the absence of for–loops.
Compact algorithms through use of Matlab/R’s strengths.
Often harder: deeper insight in algorithm is needed.
In really big “runs” objects may become too big and cause
memory problems.

34/ 35
 Standard errors Computer simulation §§15.1–3 Accuracy of bM Pricing options Vectorization

Examples
EurCall4.m improves over EurCall3.m because now one
statement generates a vector of M replications of S(T ):
Sfinal =
Snul*exp((r-0.5*sigma^2)*T+sigma*sqrt(T)*randn(M,1));
In HigF71.m one statement generates a whole price path; in
HigF73.m even several of them using Matlab’s cumprod:
tvals = [0:dt:T];
Svals = S*cumprod(exp((mu-0.5*sigma^2)*dt
+ sigma*sqrt(dt)*randn(M,L)),2);
Svals = [S*ones(M,1) Svals]; % insert initial price
plot(tvals,Svals(1:20,:))
If you find this hard to understand try this example:
A=[1 2 3; 4 5 6 ; 7 8 9]
cumprod(A,1)
cumprod(A,2)
35/ 35