Basics of Monte Carlo Simulation

He Shuangchi

IE5004 Basics of Monte Carlo Simulation 1 / 36

Example: Monty Hall problem1

Example
You are on a game show. You are given the choice of three doors: Behind
one door is a car; behind the others, goats. You pick a door, say door 1.
Then, the host, who knows what are behind the doors, opens another
door, say door 3. It has a goat. He then says to you, “Do you want to
pick door 2?” Is it to your advantage to switch your choice?

1 2

? ?

1
http://en.wikipedia.org/wiki/Monty_Hall_problem
IE5004 Basics of Monte Carlo Simulation 2 / 36
Two opposite intuitions

“After the host reveals a goat, you now have a one-in-two chance of
being correct. Whether you change your selection or not, the odds are
the same.”—No need to switch
“The car has a 1/3 chance of being behind your pick and a 2/3
chance of being behind one of the other two doors. The host opens a
door, the odds for the two sets don’t change but the odds move to 0
for the open door and 2/3 closed door.”—You should switch

Question
Which argument is correct?

IE5004 Basics of Monte Carlo Simulation 3 / 36

Assumptions

Before we proceed, let us make the following assumptions about the host.
The host must open a door that was not picked by you.
The host must open a door to reveal a goat and never the car.
The host must offer the chance to switch between the originally
chosen door and the remaining closed door.

IE5004 Basics of Monte Carlo Simulation 4 / 36

Do a simulation!

We would like to write a program to check these two intuitions. Let us

introduce several commands in Matlab.
Command rand returns a random number that is uniformly
distributed in the open interval (0, 1).
Command randi(n) returns a random integer from the discrete
uniform distribution from 1 to n.
Command randn returns a standard normal random number.

IE5004 Basics of Monte Carlo Simulation 5 / 36

Tree diagram

Suppose that the player initially picks door 1.

Car Host Total
opens: probability: Stay: Switch:
location:

1/2 Door 2 1/6 Car Goat

Door 1
1/3 1/2 Door 3 1/6 Car Goat

1/3 1
Door 2 Door 3 1/3 Goat Car

1/3 1
Door 3 Door 2 1/3 Goat Car

IE5004 Basics of Monte Carlo Simulation 6 / 36

Why simulation?

In formulating a probability model to describe a real phenomenon, we have

to compromise between
choosing a model that is a realistic replica of the actual situation
choosing a model that is mathematically tractable
The advent of fast and inexpensive computational power has opened up
another approach:
try to model the phenomenon as faithfully as possible
rely on a simulation study to analyze it

IE5004 Basics of Monte Carlo Simulation 7 / 36

Monte Carlo methods2

From the almighty Wikipedia:

Monte Carlo methods are a broad class of computational algorithms
that rely on repeated random sampling to obtain numerical results;
typically one runs simulations many times over in order to obtain the
distribution of an unknown probabilistic entity.
The Monte Carlo method was developed by Stanislaw Ulam, while he
was working on nuclear weapons projects at the Los Alamos National
Laboratory.
It was named by Nicholas Metropolis, after the Monte Carlo Casino,
where Ulam’s uncle often gambled.

2
http://en.wikipedia.org/wiki/Monte_Carlo_method
IE5004 Basics of Monte Carlo Simulation 8 / 36
 (−1, 1) (1, 1)
Example: Estimating π

Let X and Y be independent uniform random variables over (−1, 1). The
joint probability density function is given by
(
1/4 if − 1 < x < 1 and − 1 < y < 1,
f (x, y ) = fx (x)fY (y(−1,) =
−1) (1, −1)
0 otherwise.
= (0, 0)

Figure 3.1. Square.

(−1, 1) (1, 1)

(−1, −1) (1, −1)

= (0, 0)

Figure 3.2. Circle within Square.

If X2 + Y2 < 1, then (X , Y ) must be within the circle.

Since the density function of (X, Y ) is constant in the square, it thus follows (by
definition) that (X, Y ) is uniformly distributed in the square. Now if U is uniform
IE5004 on (0, 1) then 2UBasics
is uniform on (0, 2),Carlo
of Monte 2U − 1 is uniform on (−1, 1).
and so Simulation 9 / 36
Example: Estimating π
With radius r = 1, the circle has area πr 2 = π. Then,
area of the circle π
P[X 2 + Y 2 < 1] = P[(X , Y ) is in the circle] = =
area of the square 4
1

0.5

−0.5

−1
−1 −0.5 0 0.5 1

Let β = the relative frequency that (X , Y ) is in the circle. Then, by the

strong law of large numbers,
π = 4 × P[(X , Y ) is in the circle] ≈ 4 × β
IE5004 Basics of Monte Carlo Simulation 10 / 36
Using random numbers to evaluate an integral

Example
Let g (x) be a function. Compute
Z 1
θ= g (x) dx
0

Let U be a random variable uniformly distributed over (0, 1). Then

Z 1
E[g (U)] = g (x) dx = θ
0

If U1 , . . . , Un are iid uniformly distributed over (0, 1), g (U1 ), . . . , g (Un ) are
iid all having expected value θ. By the strong law of large numbers,
n
1X
g (Uk ) → E[g (U)] = θ as n → ∞
n
k=1

IE5004 Basics of Monte Carlo Simulation 11 / 36

Example

Exercise
Use the Monte Carlo method to evaluate
Z 1
θ= x 4 dx
0

In the simulation, we use

n
1X 4
θ≈ Uk
n
k=1

where U1 , . . . , Un are iid random variables uniformly distributed over (0, 1).

IE5004 Basics of Monte Carlo Simulation 12 / 36

Computing an arbitrary definite integral

Example
Let g (x) be a function. Compute
Z b
θ= g (x) dx for a < b
a

Let z = x − a. Then, Z b−a

θ= g (a + z) dz
0
Put y = z/(b − a). Then, dy = dz/(b − a). It follows that
Z 1
θ= (b − a)g (a + (b − a)y ) dy
0
Let h(y ) = (b − a)g (a + (b − a)y ). Then,
Z 1
θ= h(y ) dy = E[h(U)]
0
IE5004 Basics of Monte Carlo Simulation 13 / 36
Computing an arbitrary definite integral

Example
Let g (x) be a function. Compute
Z ∞
θ= g (x) dx
0

We may apply the substitution y = 1/(x + 1). Then,

1 dy
x= −1 and dx = −
y y2
It follows that
1
g (y −1 − 1)
Z
θ= dy
0 y2
Let h(y ) = g (y −1 − 1)/y 2 . Then,
Z 1
θ= h(y ) dy = E[h(U)]
0

IE5004 Basics of Monte Carlo Simulation 14 / 36

Example

Exercise
Use the Monte Carlo method to evaluate
Z ∞
θ= e −x dx
0

Let
e 1−1/y
h(y ) =
y2
In the simulation, we use
n
1X
θ≈ h(Uk )
n
k=1

where U1 , . . . , Un are iid random variables uniformly distributed over (0, 1).

IE5004 Basics of Monte Carlo Simulation 15 / 36

Random number generation

The building block of simulation is the ability to produce random numbers.

A random number represents the value of a random variable uniformly

distributed on (0, 1).
Random numbers were originally generated manually or mechanically,
by spinning wheels, dice rolling, and card shuffling, etc.
The modern approach is to use a compute to successively generate
pseudorandom numbers.
Using uniform (0, 1) random numbers, we may generate the values of
an arbitrary random variable by a transform.

IE5004 Basics of Monte Carlo Simulation 16 / 36

Pseudorandom numbers

Because a computer can follow only very specific steps, the generated
numbers are not truly random.
They are deterministically generated.
They have all the appearances of being independent uniform (0,1)
random variables.
These pseudorandom numbers must satisfy the following conditions.
All generated numbers are within (0, 1).
For 0 ≤ a < b ≤ 1, the relative frequency that generated numbers lie
in (a, b) is b − a.
The sequence of numbers can pass a statistical test of independence.

IE5004 Basics of Monte Carlo Simulation 17 / 36

Linear congruential generators

A linear congruential generator produces pseudorandom numbers by a

recursive procedure. It first produces integers by

xn = (axn−1 + c) mod m for n = 1, 2, . . .

where
x0 , a, c, and m are given nonnegative integers
the initial value x0 is called the seed
a is called the multiplier
c is called the increment
m is called the modulus
Then, pseudorandom numbers are generated by
xn
un = for n = 0, 1, . . .
m

IE5004 Basics of Monte Carlo Simulation 18 / 36

Example

Example
Consider a linear congruential generator given by

xn = (2xn−1 + 1) mod 7

with seed x0 = 3. Find x1 , . . . , x7 .

x1 = (2 × 3 + 1) mod 7 = 0, u1 = 0/7 = 0,
x2 = (2 × 0 + 1) mod 7 = 1, u2 = 1/7 = 0.1429,
x3 = (2 × 1 + 1) mod 7 = 3, u3 = 3/7 = 0.4286,
x4 = (2 × 3 + 1) mod 7 = 0, u4 = 0/7 = 0,
x5 = (2 × 0 + 1) mod 7 = 1, u5 = 1/7 = 0.1429,
x6 = (2 × 1 + 1) mod 7 = 3, u6 = 3/7 = 0.4286,
x7 = (2 × 3 + 1) mod 7 = 0, u7 = 0/7 = 0,
IE5004 Basics of Monte Carlo Simulation 19 / 36
Period of a random number generator

Since each xn assumes one of 0, 1, . . . , m − 1, after some finite number (at

most m) of generated values, the sequence will repeat itself.

Definition
The period of a random number generator is the number of values that
can be produced before the exact sequence of values repeats.

Question
What is the period of the linear congruential generator in the previous
example?

IE5004 Basics of Monte Carlo Simulation 20 / 36

Criteria of a good random number generator

The constants a, c, and m should be chosen according to three criteria.

For any initial seed, the resultant sequence has the “appearance” of
being a sequence of independent uniform (0, 1) random variables
For any initial seed, the number of variables that can be generated
before repetition begins is large.
The values can be computed efficiently on a digital computer. (The
modulus is often chosen to be fitted to the computer word size, e.g.,
m = 232 or m = 264 ).

Example
A linear congruential generator with

a = 214013, m = 232 , c = 2531011

is used by Microsoft Visual C++ (32-bit version).

IE5004 Basics of Monte Carlo Simulation 21 / 36

Generating a Bernoulli random variable

Let U be a uniform (0, 1) random variable.

Question
Let X be a Bernoulli random variable with P[X = 1] = p. How can we use
the values of U to generate the values of X ?

Let (
1 if 0 < U ≤ p,
X =
0 if p < U < 1.
Then,

P[X = 1] = P[0 < U ≤ p] = p,

P[X = 0] = P[p < U ≤ 1] = 1 − p.

So that X is a desired Bernoulli random variable.

IE5004 Basics of Monte Carlo Simulation 22 / 36

Generating a discrete random variable

Let X be a discrete random variable with probability mass function

P[X = xj ] = pj for j = 0, 1, . . .
P
where x0 < x1 < · · · and j pj = 1.

Question
How can we generate the values of X using the values of U?

IE5004 Basics of Monte Carlo Simulation 23 / 36

Discrete inverse transform method

Let F be the cumulative distribution function of X . Then,

k
X
F (xk ) = P[X ≤ xk ] = pj
j=0

Let
X = xk if F (xk−1 ) ≤ U < F (xk )
Then,

P[X = xk ] = P[F (xk−1 ) ≤ U < F (xk )] = F (xk ) − F (xk−1 ) = pk

So that X has the desired distribuiton.

IE5004 Basics of Monte Carlo Simulation 24 / 36

Generating a discrete uniform random variable

Example
Let X be a discrete uniform random variable that is equally likely to take
each value of 1, . . . , n, i.e., P[X = k] = 1/n. Generate the values of X .

Let F be the cumulative distribution function of X . Then,

k
F (k) = P[X ≤ k] =
n
By the discrete inverse transform method,
k −1 k
X =k if ≤U<
n n
Therefore,
X =k if k − 1 ≤ nU < k
or equivalently,
X = bnUc + 1
where bxc is the largest integer that is less than or equal to x.
IE5004 Basics of Monte Carlo Simulation 25 / 36
Generating a geometric random variable

Example
Let X be a geometric random variable with P[X = k] = q k−1 p for
q = 1 − p and k = 1, 2, . . .. Generate the values of X .

Let F be the cumulative distribution function of X . Then,

F (k) = P[X ≤ k]
= 1 − P[X > k]
= 1 − P[the first k trials are all failures]
= 1 − qk

By the discrete inverse transform method,

X =k if 1 − q k−1 ≤ U < 1 − q k

IE5004 Basics of Monte Carlo Simulation 26 / 36

Generating a geometric random variable (continued)

If 1 − q k−1 ≤ U < 1 − q k holds, we must have

q k < 1 − U ≤ q k−1

Then,
k log q < log(1 − U) ≤ (k − 1) log q
Because log q < 0,
log(1 − U)
k −1≤ <k
log q
Because U has the same distribution as 1 − U, we can use
 
log U
X = +1
log q

IE5004 Basics of Monte Carlo Simulation 27 / 36

Generating a continuous random variable

The inverse transform method can also be used for generating the values
of a continuous random variable.
Proposition
Let U be a uniform (0, 1) random variable. Let F be the cumulative
distribution function of a continuous random variable. Then,

X = F −1 (U)

has the cumulative distribution function F . In the above expression,

F −1 (u) is defined to be the value of z such that F (z) = u.

IE5004 Basics of Monte Carlo Simulation 28 / 36

Proof

The cumulative distribution function of X = F −1 (U) is given by

P[X ≤ x] = P[F −1 (U) ≤ x]

Because F is monotone increasing and F (F −1 (U)) = U, it follows that

P[X ≤ x] = P[F (X ) ≤ F (x)]

= P[F (F −1 (U)) ≤ F (x)]
= P[U ≤ F (x)]
= F (x)

So that F is the cumulative distribution function of X .

IE5004 Basics of Monte Carlo Simulation 29 / 36

Generating an exponential random variable

Example
Let X be an exponential random variable with rate 1. Generate the values
of X using the values of U.

The cumulative distribution function is given by

F (x) = 1 − e −x for x ≥ 0

Let u = 1 − e −x . Then,
x = − log(1 − u)
Therefore, F −1 (u) = − log(1 − u). We may generate the desired random
variable by
X = F −1 (U) = − log(1 − U)
Because 1 − U has the same distribution as U, we can use

X = − log U
IE5004 Basics of Monte Carlo Simulation 30 / 36
The rejection method

It is not always convenient to use the inverse transform method because

many cumulative distribution functions do not have closed-form
expressions, e.g., a normal cumulative distribution function
the inverse function could be complex
To generate a random variable X with density f (x), suppose that
we can efficiently generate a random variable Y with density g (x)
there exists a constant c such that
f (y )
≤c for all y
g (y )

Then, we may use the rejection method to generate the values of X .

IE5004 Basics of Monte Carlo Simulation 31 / 36

The rejection method
5.2 The Rejection Method 73

Start
Generate a Is Yes
Generate Set X = Y
random number f (Y )
Y~g U ≤ cg (Y ) ?
U
No

Figure 5.1. The rejection method for simulating a random variable X having density
function1 f .Generate Y that has density g
Step
Step 2 Generate a uniform (0, 1) number U
5.2 3The
Step Rejection
Set X = Y Method
if
f (Y )
Suppose we have a method for generatingU ≤ a random variable having density
c · g (Y ) from the continuous
function g(x). We can use this as the basis for generating
distribution having density
Otherwise, function
return of f (x)
to Step 1. by generating Y from g and then
accepting this generated value with a probability proportional to f (Y )/g(Y ).
Specifically, let c be a constant such that
Proposition
f (y)
The random variable X generated  c the
by for rejection
all y method has density f .
g(y)
IE5004 Basics of Monte Carlo Simulation 32 / 36
Generating a standard normal random variable

Example
Let Z be a standard normal random variable. Generate the values of Z .
The procedure is as follows.
Generate X = |Z | using the rejection method.
Set Z = X with probability 0.5 and Z = −X with probability 0.5.
Note that X = |Z | has density

2 2
f (x) = √ e −x /2 for x > 0
2π
Let Y be an exponential random variable with rate 1, whose density is

g (x) = e −x for x > 0

IE5004 Basics of Monte Carlo Simulation 33 / 36

Generating a normal random variable (continued)

We have r
f (x) 2 x−x 2 /2
= e
g (x) π
Because
x2 1 1
x− = − (x − 1)2 +
2 2 2
It follows that x − x 2 /2 ≤ 1/2 for all x. Then,
r
f (x) 2e
≤ for all x > 0
g (x) π
p
We take c = 2e/π and then

f (x) 2
= e −(x−1) /2
c · g (x)

IE5004 Basics of Monte Carlo Simulation 34 / 36

Generating a normal random variable (continued)

We may generate a standard normal random variable as follows

Step 1 Generate Y , an exponential random variable with rate 1.
Step 2 Generate a uniform (0, 1) random number U1 .
2
Step 3 If U1 ≤ e −(Y −1) /2 , set X = Y and proceed to Step 4.
Otherwise, return to Step 1.
Step 4 Generate another uniform (0, 1) random number U2 and set
(
X if U2 ≤ 0.5
Z=
−X if U2 > 0.5

IE5004 Basics of Monte Carlo Simulation 35 / 36

Homework

Reading assignment
Read Chapters 3, 4.1, and 5.1–5.2 in Simulation by Ross
Exercise problems
3.2, 3.3, 3.5, 3.7, 4.1, 5.1, and 5.5 in Simulation by Ross

IE5004 Basics of Monte Carlo Simulation 36 / 36