Chap5

Chapter 5: Monte Carlo È © ~ E â
Contents
1 Monte Carlo Integration and Variance Reduction 1

1.1 Monte Carlo Integration . . . . . . . . . . . . . . . . . . . . 3
1.1.1 Simple Monte Carlo estimator . . . . . . . . . . . . 3
1.1.2 Variance and Efficiency . . . . . . . . . . . . . . . . 12
1.2 Variance Reduction . . . . . . . . . . . . . . . . . . . . . . . 14
1.3 Antithetic Variables . . . . . . . . . . . . . . . . . . . . . . 15
1.4 Control Variates . . . . . . . . . . . . . . . . . . . . . . . . 24
1.4.1 Antithetic variate as control variate . . . . . . . . . 30
1.4.2 Several control variates . . . . . . . . . . . . . . . . 31
1.5 Importance sampling . . . . . . . . . . . . . . . . . . . . . . 35
1.6 Stratified Sampling . . . . . . . . . . . . . . . . . . . . . . . 43
1.7 Stratified Importance Sampling . . . . . . . . . . . . . . . . 47
Previous Next First Last Back Forward 1

Chapter 1
Monte Carlo Integration and Variance
Reduction
AkÛ(Monte Carlo)È©´«ÄuÅÄÚO{. AkÛ{

Ù¢ ´é«g¡, 3)û¯K, |^)þÅ§
, é ù (J?1VÇ©Û, l 5ýÿ(J{, Ñ±¡A
kÛ{.
'X¦ ±Çπ, Í¶Ò´¥ÆÆL“ {”(4 (A,
3.1416), yÀ(H, 3.1415926< π <3.1415927)). y3±¦Â
kÛÈ©{: dVÇØ
e r.v. X, Y i.i.d U (0, 1), K P (X + Y ≤ 1) = π4
2 2

Ïd, π = 4P (X 2 + Y 2 ≤ 1) ≈ 4#{x2 + y 2 ≤ 1}/n.
• n = 1000, π̂ = 3.168.
• n = 100, 000, π̂ = 3.14312.
• n = 107 , π̂ = 3.141356.

1.1 Monte Carlo Integration
bg´È¼ê, ·OR g(x)dx. £Á3VÇØ¥, eÅCb

a
þX Ýf (x), KÅCþY = g(X)Ï"

Z ∞
Eg(X) = g(x)f (x)dx
−∞
XJlX ©Ù¥) Åê, KEg(X)Ã OÒÁ²

þ.
1.1.1 Simple Monte Carlo estimator
ÄOθ = R 1
0 g(x)dx. eX , · · · , X þ!©ÙU (0, 1)oÄ, K
1 m
drêÆ m
1 X
θ̂ = gm (X) = g(Xi )
m i=1
±VÇ1ÂñÏ"Eg(X). ÏdR 1
0 g(x)dx {üMonte Carlo Oþ
g (X).
m

~1: ({üMonte Carlo È©) O
Z 1
θ= e−x dx
0
{üMonte CarloO±9È©'.
↑Example
m <- 10000
x <- runif(m)
theta.hat <- mean(exp(-x))
print(theta.hat)
print(1 - exp(-1))
#[1] 0.6355289
#[1] 0.6321206
↓Example
O .
È©θ = 1 − e
θ̂ = 0.6355, −1 .
= 0.6321.
eO Rb
d?a < bkê. KÈ©Cþ¦È©
a g(x)dx,
l01. =Cy = (x − a)/(b − a), Ïd

Z b Z 1
g(x)dx = g(y(b − a) + a)(b − a)dy
a 0
, «{Ò´|^þ!©ÙU (a, b), =
Z b Z b 1
g(x)dx = (b − a) g(x) dx
a a b−a
~2: {üMonte Carlo È©(Y) O

Z 4
θ= e−x dx
2
Monte CarloO¿ÚÈ©'.
↑Example
m <- 10000
x <- runif(m, min=2, max=4)
theta.hat <- mean(exp(-x)) * 2
print(theta.hat)
print(exp(-2) - exp(-4))
#[1] 0.1172158
#[1] 0.1170196

↓Example
=, O .
θ̂ = 0.1172, ýθ = − e−2 e−4
.
= 0.1170.
o(e, È© Rb
a g(x)dx {üMonte CarloO{
1. lþ!©ÙU (a, b)¥)i.i.dX , · · · , X 1 m;
2. Og (X) = g(X )

m
1
m i
3. θ̂ = (b − a)gm (X).
~3: Monte CarloÈ©, Ã¡È© 'X¦^Monte CarloÈ©{OI

O©Ù¼ê x t2
Z
1
Φ(x) = √ e− 2 dt.
−∞ 2π
Äk·ØU¦^±c{(ÏÈ©«mÃ.), ´·±ò
d¯K©ü«/: x > 0Úx ≤ 0. e x > 0, 5¿éIO©Ù, È©
«m±©(−∞, 0)Ú(0, x),ÏdIOÈ©θ = R e dt =. x −t
0
2
2
±¦^c{. Ílþ!©ÙU (0, x)¥Ä, exu)Cz,

Kþ!©ÙÒCz . e¦lU (0, 1)¥Ä, K±Cy = t/x, K
Z 1 2
θ= xe−(xy) /2
dy
0
Ïd, θ = E [xe
Y ], Ù¥Y
−(xY )2 /2 ∼ U (0, 1). l )U (0, 1)i.i.dÅ
êu , · · · , u , K θO
1 m
m
1 X −(xu)2 /2
θ̂ = gm (u) = xe .
m i=1
déx > 0,Φ(x)O0.5 + θ̂/√2π; éx ≤ 0, OΦ(x) = 1 − Φ(−x).

↑Example
x <- seq(.1, 2.5, length = 10)

m <- 10000
u <- runif(m)

cdf <- numeric(length(x))
for (i in 1:length(x)) {
g <- x[i] * exp(-(u * x[i])^2 / 2)
cdf[i] <- mean(g) / sqrt(2 * pi) + 0.5
}
Phi <- pnorm(x)
print(round(rbind(x, cdf, Phi), 3))
↓Example
~4: Monte CarloÈ©, Ã¡È©(Y) éþ~, ·±¦^, «

ª(hit-or-miss). PZ ∼ N (0, 1), Ké?Û¢êxk
Φ(x) = P (Z ≤ x) = EI(Z ≤ x).
l lIO¥)Åz , · · · , z 1 m , =Φ(x)O
m
1 X
Φ̂(x) = I(zi ≤ x).
m i=1

Ù±VÇ1ÂñΦ(x).
↑Example
x <- seq(.1, 2.5, length = 10)

m <- 10000
z <- rnorm(m)
dim(x) <- length(x)
p <- apply(x, MARGIN = 1,
FUN = function(x, z) {mean(z <= x)}, z = z)
Phi <- pnorm(x)
print(round(rbind(x, p, Phi), 3))
↓Example
o(Xe: OÈ©
Z
θ= g(x)f (x)dx
A
Ù¥f ±A| VÇ¼ê, =R A f (x)dx = 1. K)f i.i.dÅ

êx , · · · , x
1 m , dêÆθO
m
1 X
θ̂ = g(xi ).
m i=1
duθ̂ σ /m, Ù¥σ = V ar (g(X)). ÅCþX ©Ù,

2 2
f
·±¦^x , · · · , x ²©Ù¼ê, l σ /mO

1 m
2
m
1 X
σ̂ 2 /m = 2
[g(xi ) − g(x)]2 .
m i=1
2d¥%4½nm → ∞
θ̂ − E θ̂
q
V ar(θ̂)
©ÙÂñIO©Ù. Ïdé, ìC5±ÑÈ©

Monte CarloOØ ., ±d± 5uÂñ5.

~5: Monte CarloÈ©Ø . ïáΦ(2)ÚΦ(2.5)Monte CarloÈ©
O%95&«m.
↑Example
x <- 2
m <- 10000
z <- rnorm(m)
g <- (z <= x) #the indicator function
v <- mean((g - mean(g))^2) / m
cdf <- mean(g)
c(cdf, v)
c(cdf - 1.96 * sqrt(v), cdf + 1.96 * sqrt(v))
#[1] 9.7800e-01 2.1516e-06
#[1] 0.975125 0.980875
↓Example
ÅCþI(Z ≤ 2)1VÇΦ(2) ≈ 0.977. d?g(X) ∼ B(10000, Φ(2)),

Ïd g(X) 0.977(1 − 0.977)/10000 = 2.223e − 06. Monte CarloÈ
©O 2.1516e − 06, ®²~C .
1.1.2 Variance and Efficiency
Monte Carlo {3OÈ©R b

a òÙL«þ!ÅC
g(x)dx ,
þÏ", l
Z b Z b 1
θ= g(x)dx = (b−a) g(x) dx = (b−a)E[g(X)], X ∼ U (a, b).
a a b−a
l {Xe
1. lU (a, b)¥)i.i.dX , · · · , X 1 m.
2. Og(X) = 1 m
P
m
g(X ).
i=1 i
3. θ̂ = (b − a)g(X).
´,
(b − a)2
E θ̂ = θ, V ar(θ̂) = (b − a)2 V ar(g(X)) = V ar(g(X)).
m

k¥%4½n, g(X)©ÙìC©Ù, Ïdθ̂ìC©Ù.
Hit-or-miss Monte Carlo {K¦^ , «OÈ©ª, Ù
Úþ¡`{ØÓ. LãXe: bf ÅCþX VÇ¼ê, ¦
^“hit-or-miss”{OÈ©F (x) = R f (t)dt: x
−∞
1. lX ©Ù¥)i.i.dX , · · · , X 1 m.
2. Og(X) = 1 m
P
I(X ≤ x).
m i=1 i
3. F[
(x) = g(X).
w,ézkx, Y = g(X) = I(X ≤ x) ∼ B(1, p), p = F (x). Ï

d
E[F[
(x)] = F (x), V ar[F[
(x)] = F (x)(1 − F (x))/m.
F[
(x) ±ÏLF[ (x))/m5O.
(x)(1 − F[
ùü«{ ØÓ, g,¬¯=«` , =kÇ.
θ̂ Úθ̂ ´θüÃ Oþ, Kθ̂ 'θ̂ k, XJ
1 2 1 2
Definition
V ar(θ̂1 ) ≤ V ar(θ̂2 ).
XJOþ , K·ÏLòÙOÑ5. , , Oþ

o´±ÏLO\þ 5~.
1.2 Variance Reduction
·®²Monte CarloÈ©{±^5OE[g(X)]ù«a.È©.
e¡·?ØXÛ~θ = E[g(X)] þOþ.

eθ̂ Úθ̂ ë êθ ü O þ, V ar(θ̂2 ) < V ar(θ̂1 ), K¦
^θ̂ '¦^θ̂ ~z©'
1 2
2 1
! Definition
V ar(θ̂1 ) − V ar(θ̂2 )
100 .
V ar(θ̂1 )
¦+±O\þm5~ , Ó¤¬ép. 'XIO

õe, V ar(g(X)) = σ , KIþm ≥ . XrIO l0.01~
2 σ2
e2
0.0001, Kþ10000.
1.3 Antithetic Variables
5¿ÑéüÅCþU , U , 1 2
1
V ar((U1 + U2 )/2) = (V ar(U1 ) + V ar(U2 ) + 2Cov(U1 , U2 ).
4

Ïd, U , U K'(U + U )/2 'ÙpÕá. ù¯
1 2 1 2
¢r¦·Ä¦^K'ÅCþ 5~ .
'X, bX , . . . , X ÏL_C{), édmÅþ(X =
1 n
(X , . . . , X ))z, ·®²)U ∼ U (0, 1)±9OX

1 n = F (U ), j =
j
(j) −1
X j
1, . . . , n. 5¿eU ∼ U (0, 1),K 1 − U ∼ U (0, 1), U 1 − U K'. K
−1 −1 (j) (j)
Yj = g(X (j) ) = g(FX (U1 ), . . . , FX (Un ))

−1 −1 (j) (j)
Yj0 = g(X (j) ) = g(FX (1 − U1 ), . . . , FX (1 − Un ))
Ó©Ù.
@oo/eY Y ´K'? e¡½n`²XJg´üN, j j
0
KY ÚY ´K'.
j j
0
½Â(x , . . . , x ) ≤ (y , . . . , y ) XJx ≤ y , j = 1, . . . , n. n

1 n 1 n j j
¼ê g = g(x , . . . , x )¡Ó, XJÙézCþÑÓ, =X

1 n

J(x , . . . , x ) ≤ (y , . . . , y ), Kg(x , . . . , x ) ≤ g(y , . . . , y ). aq
1 n 1 n 1 n 1 n
¡g4~,XJÙézCþÑ´~. , ¡g´üN,XJÙÓ
½ö~.
Theorem 1. X = (X , · · · , X )©þpÕá, f Úg ÓüN¼
1 n
ê(ÓO½öÓ~), K
E[f (X)g(X)] ≥ E[f (X)]E[g(X)].
Proof. Øf, gÓO. y²{´én8B. bn = 1, Ké¤kx, y ∈

R, (f (x)−f (y))(g(x)−g(y)) ≥ 0. Ïd E[(f (X)−f (Y ))(g(X)−g(Y ))] ≥ 0,
=k
E[f (X)g(X)] + E[f (Y )g(Y )] ≥ E[f (X)g(Y )] + E[f (Y )g(X)].
d?X, Y i.i.d, Ïd
2E[f (X)g(X)] = E[f (X)g(X)] + E[f (Y )g(Y )]

≥ E[f (X)g(Y )] + E[f (Y )g(X)] = 2E[f (X)]E[g(X)],
=(Øén = 1¤á. y3bén − 1¤á, Kdu

E[f (X)g(X)|Xn = xn ] ≥ E[f (X1 , . . . , Xn−1 , xn )]E[g(X1 , . . . , Xn−1 , xn )]
= E[f (X|Xn = xn )]E[g(X|Xn = xn )],
ªzX O¼ê, Ïdân = 1(Øk

n
E[f (X)g(X)] = E{E[f (X)g(X)|Xn ]}
≥ E{E[f (X|Xn = xn )]}E{E[g(X|Xn = xn )]}
= E[f (X)]E[g(X)].
Corollary 1. g = g(X , . . . , X ) ´üN, K

1 n
−1 −1
Y = g(FX (U1 ), . . . , FX (Un ))

−1 −1
Y 0 = g(FX (1 − U1 ), . . . , FX (1 − Un ))
´K'.
Proof. Ø5, bg ´4O. K
−1 −1
Y = g(FX (U1 ), . . . , FX (Un ))

−1 −1
−Y 0 = f = −g(FX (1 − U1 ), . . . , FX (1 − Un ))
´ü4O¼ê. ÏdE[g(U )f (U )] ≥ E[g(U )]E[f (U )], = E[Y Y ] ≤ 0
E[Y ]E[Y ], d=¿X

0
Cov(Y, Y 0 ) = E[Y Y 0 ] − E[Y ]E[Y 0 ] ≤ 0.
l Y ÚY K'.0

éóCþ(Antithetic Variables){éNÁ^. XJImMonte
E, K)m/2E
Carlo
−1 (j) −1 (j)
Yj = g(FX (U1 ), . . . , FX (Un ))
±9em/2E
−1 (j) −1 (j)
Yj0 = g(FX (1 − U1 ), . . . , FX (1 − Un )),
Ù¥U (j)
i ,i = 1, . . . , n; j = 1, . . . , m/2. i.i.d U (0, 1). KéóOþ
1
θ̂ = {Y1 + Y10 + Y2 + Y20 + · · · + Ym/2 + Ym/2
0
}
m
m/2 0
2 X Yj + Yj
= .
m j=1 2
ÏL¦^éóCþ¦Monte Carlo O ~.

~6: éóCþ{ ¦^c~f, 'XOÈ©
Z x 1 −t2 /2
Φ(x) = e dt.
−∞ 2π
¦^éóCþ{OdÈ©, ¿éÑIO ~þ.

Cþ , 8IÈ©´θ = E [xe ], Ù¥U ∼ U (0, 1). w,
U
−(xU )2 /2
d?g¼ê´üN, Ïdc¡íØ^÷v. )Åêu , · · · , u ∼ 1 m/2
U (0, 1), O
2
Yj = g (j) (u) = xe−(xuj ) /2
, j = 1, · · · , m/2.
±9
2
Yj0 = xe−(x(1−uj )) /2
, j = 1, · · · , m/2.
Kþ
m/2 2 2
1 X xe−(uj x) + xe−(x(1−uj )) /2
θ̂ = gm (u) =
m/2 j=1 2

ÂñEθ̂.
l ex > 0,KΦ(x)O0.5+θ̂/√2π; ex < 0, KΦ(x) = 1−Φ(−x).
↑Example
MC.Phi <- function(x, R = 10000, antithetic = TRUE) {

u <- runif(R/2)
if (!antithetic) v <- runif(R/2) else
v <- 1 - u
u <- c(u, v)
cdf <- numeric(length(x))
for (i in 1:length(x)) {
g <- x[i] * exp(-(u * x[i])^2 / 2)
cdf[i] <- mean(g) / sqrt(2 * pi) + 0.5
}
cdf
}
↓Example
Ú{üMonte CarloÈ©{':
↑Example
x <- seq(.1, 2.5, length=5)
Phi <- pnorm(x)
set.seed(123)
MC1 <- MC.Phi(x, anti = FALSE)
set.seed(123)
MC2 <- MC.Phi(x)
print(round(rbind(x, MC1, MC2, Phi), 5))
↓Example
~þ±[5':
↑Example
m <- 1000
MC1 <- MC2 <- numeric(m)
x <- 1.95
for (i in 1:m) {
MC1[i] <- MC.Phi(x, R = 1000, anti = FALSE)
MC2[i] <- MC.Phi(x, R = 1000)
}
print(sd(MC1))
print(sd(MC2))
print((var(MC1) - var(MC2))/var(MC1))

↓Example
éx = 1.95, éóCþ{'u{üMonte Carlo{O ~

99.5%.
1.4 Control Variates
O¥, «~ {Ćþ(Control Variates)

Monte Carlo
¦^. Oþθ = E[g(X)], f ¼ê, ÙÏ"E[f (X)] = µ®
, f Úg'.
é?Û~êc, Oþθ̂ = g(X) + c(f (X) − µ)Ã ,
c
V ar(θ̂c ) = V ar(g(X)) + c2 V ar(f (X)) + 2cCov(g(X), f (X))
écz, K3
Cov(g(X), f (X))
c∗ = −
V ar(f (X))

?,
[Cov(g(X), f (X))]2
V ar(θ̂c∗ ) = V ar(g(X)) − .
V ar(f (X))
ÅCþf (X)¡g(X)Cþ(Control Variate). w, ~

Ç
[Cov(g(X), f (X))]2
100 = 100[Cor(g(X), f (X))]2 .
V ar(g(X))V ar(f (X))
±wÑ, ù«{3f Úgr'´k`³, ef ÚgØ', KØ¬
~.
~7: Cþ{ ¦^Cþ{OÈ©
Z 1
θ = E[eU ] = eu du,
0
Ù¥U ∼ U (0, 1).

d~¥,,È©θ = e − 1 =. 1.718282, ·E,¦^CþMonte
Carlo{5OÈ©, ^±`²ù«{¦^. XJ¦^{üMonte

CarloÈ©{, K
e2 − 1 .
V ar(g(U )) = V ar(eU ) = − (e − 1)2 = 0.2420351.
2
EmgO V ar(g(U ))/m.

Cþg,ÀJU ∼ U (0, 1), KCov(e U , U) .
= 1 − (e − 1)/2 =
0.1408591. Ïd
−Cov(eU , U ) .
c∗ = = −12 + 6(e − 1) = −1.690309.
V ar(U )
¦^Cþ{Oθ̂ = e c∗
U − 1.690309(U − 0.5), m gE
V ar(θ̂ )/m, Ù¥V ar(θ̂ )
c∗ c∗
[Cov(eU , U )]2 e2 − 1 e−1 2 .

V ar(eU )− = −(e−1)2 −12[1− ] = 0.003940175.
V ar(U ) 2 2
Ïd¦^Cþ{{üMonte CarloOþ ~Ç100(1 −

0.003940175/0.2429355) = 98.3781%.

e¡·¦^Cþ{5OÙ² ~Ç.
↑Example
x <- seq(.1, 2.5, length=5)

Phi <- pnorm(x)
set.seed(123)
MC1 <- MC.Phi(x, anti = FALSE)
set.seed(123)
MC2 <- MC.Phi(x)
print(round(rbind(x, MC1, MC2, Phi), 5))
↓Example
~þ±[5':
↑Example
m <- 10000
a <- - 12 + 6 * (exp(1) - 1)
U <- runif(m)
T1 <- exp(U) #simple MC
T2 <- exp(U) + a * (U - 1/2) #controlled
mean(T1)

mean(T2)
(var(T1) - var(T2)) / var(T1)
↓Example
~8: ¦^Cþ{Monte CarloÈ© ¦^Cþ{OÈ©

1 e−x
Z
dx.
0 1 + x2
d~¥a,þθ = Eg(X), g(x) = , Ù¥X ∼ U (0, 1). · e−x

1+x2
Ï¦v Cg¼êf ÙÏ"®, Úg'. 'Xf (x) =
´±, eU ∼ U (0, 1), K
e−0.5
(1+x2 )
Z 1 1 π
Ef (U ) = e−0.5 du = e−0.5 .
0 1 + u2 4
·±OÑCor(g(U ), f (U )) ≈ 0.974. Ïd
↑Example
f <- function(u) exp(-.5)/(1+u^2)
g <- function(u) exp(-u)/(1+u^2)
set.seed(510) #needed later
u <- runif(10000)
B <- f(u)
A <- g(u)
a <- -cov(A,B) / var(B) #est of c*
m <- 100000
u <- runif(m)
T1 <- g(u)
T2 <- T1 + a * (f(u) - exp(-.5)*pi/4)
c(mean(T1), mean(T2))
c(var(T1), var(T2))
(var(T1) - var(T2)) / var(T1)
↓Example

1.4.1 Antithetic variate as control variate
éóCþ{¢SþĆþ{A~. 5¿Cþ{ ´Ã
O5|Ü. /, eθ̂ Úθ̂ θÃ Oþ, Ké?Û~êc,k
1 2
θ̂c = cθ̂1 + (1 − c)θ̂2
EθÃ O. Ù
V ar(θ̂2 ) + c2 V ar(θ̂1 − θ̂2 ) + 2cCov(θ̂2 , θ̂1 − θ̂2 ).
AO/, θ̂ Úθ̂ Ó©Ù,

1 2 Cor(θ̂1 , θ̂2 ) = −1. KCov(θ̂ , θ̂ ) = −V ar(θ̂ ),
1 2 1

V ar(θ̂c ) = 4c2 V ar(θ̂1 ) − 4cV ar(θ̂1 ) + V ar(θ̂1 ) = (4c2 − 4c + 1)V ar(θ̂1 ),
Ïd`c ∗ = 1/2. dCþOþ

θ̂1 + θ̂2
θ̂c∗ =
2
ù´(ù«A½ÀJe)éóCþ{eOþ.
1.4.2 Several control variates
òÃ Oþ|Üå5ëêθO,±~ {±í2õ
Cþ|Ü: k
X
θ̂c = g(X) + ci (fi (X) − µi ))
i=1
Ù¥µ i = Efi (X), i = 1, · · · , k.,

Pk
i=1 ci =1 ±9
k
X
E θ̂c = E[g(X)] + ci E[fi (X) − µi ] = θ
i=1
Cþ{eOþθ̂ ±9`c ±ÏL£8.5O.

ĉ i
3 k = 1 |Ü, Ä((g(X ), f (X )), · · · , (g(X ), f (X

1 1 n n ))), b
g(X)f (X)m35'X: g(X) = β + β f (X) + e, 0 1
E[g(X)] = β0 + β1 E[f (X)].

β1 ¦O
Pn
i=1 (g(Xi ) − g(X))(f (Xi ) − f (X)) Cov(g(X), f (X))
d
β̂1 = Pn ¯ 2 = = −c∗ .
i=1 (f (Xi ) − f (X)) V ar(f (X))
d
ù`²·±¦^g(X)éf (X)£85Oc:
↑Code
L<-lm(gx~fx)
c.star<--L$coef[2]
↓Code
å¦Oβ̂ 0 = g(X) − (−c∗ )f (X), Ïd3µ?ýÿ

β̂0 + β̂1 µ = g(X) + ĉ∗ (f (X) − µ) = θ̂c∗
=Cþ{eOþ´ýÿ.
Ø O
be2 = Vd
σ ar(g(X)−ĝ(X)) = Vd ar(g(X)+ĉ∗ f (X))
ar(g(X)−(β̂0 +β̂1 f (X))) = Vd

Cþ{eOþ O
ar(g(X) + ĉ∗ f (X))
Vd σ̂ 2
ar(g(X) + ĉ∗ (f (X) − µ)) =
Vd = e.
n n
Ïd3R¥, Cþ{eOþIO O

↑Code
se.hat<-summary(L)$sigma/sqrt(n)
↓Code
Úc¡Cþ!¥(JÓ.
ék, K±¦^£8
k
X
g(X) = β0 + βi f (X) + e
i=1
5O`~êc = (c , · · · , c ). K−c = (β̂ , · · · , β̂ ), ±9d

∗ ∗
1
∗
k
∗
1 k
Cþ{eO3µ = (µ , · · · , µ )?ýÿĝ(X). O

1 k
σ /n.
2
e

~9: CþÚ£8 ¦^£8{OÈ©
1 e−x
Z
g(x) = dx.
0 1 + x2
ùpCþf (x) = e .5 (1 + x2 )−1 , 0 < x < 1. , µ = E[f (X)] =

e.5 π/4. O`~êc , ∗
↑Example
set.seed(510)
u <- runif(10000)
f <- exp(-.5)/(1+u^2)
g <- exp(-u)/(1+u^2)
L <- lm(g~f)
c.star <- - L$coeff[2] # beta[1]
mu <- exp(-.5)*pi/4
c.star
theta.hat <- sum(L$coeff * c(1, mu)) #pred. value at mu
theta.hat

summary(L)$sigma^2
summary(L)$r.squared
↓Example
ùp·¦^ Úc~¥Ó«f, ÏdÓc O. y3θ̂ ´ ∗

ĉ∗
3µ = 0.4763681?ýÿ. Oþθ̂9ÙIO , ~ÇÑÚc~

Ó.
1.5 Importance sampling
3c¡SN¥,·òk«mþÈ©À´d«mþþ!©ÙÅCþ
,¼êÏ". ù«{":´ØUûÃ¡È©O, ±9
È¼ê3È©«mþØ´éþ!ÿ Çé$. @oég,/±
Äþ!©Ù± Ù¦\¼ê. ùÒ/5Ä{0.
bÅCþX Ý¼êf (x), ÷v38Ü{x : g(X) > 0}þf (x) >
0. KPY = g(X)/f (X),
Z Z
g(x)
g(x)dx = f (x)dx = EY.
f (x)
ÏL{üMonte Carlo{OEY :
m m
1 X 1 X g(Xi )
Yi = .
m i=1 m i=1 f (Xi )
d?ÅCþX , · · · , X lf ¥Ä. f ¡¼ê. O

1 m
V ar(Y )/m, Y C~ê, Y ¬', dÝf AT“C” ¼

êg(x). ,, Ýf ATÀJN´Ä©Ù.
5Ä{`:´±ÏLÀJ¼ê5ü$ Monte Carlo
O . bf (x)´| AÝ¼ê, eφ(x) > 0, ∀x ∈ A, KÈ©
Z
θ= g(x)f (x)dx,
A
± Z
f (x)
θ= g(x) φ(x)dx.
φ(x)
eφ(x)AþÝ¼ê, KθOþ
A
n
1X f (Xi )
θ̂ = g(Xi ) ,
n i=1 φ(Xi )

Ù¥X , · · · , X lφ(x)¥)Å. φ(x)¡5Ä¼ê½
1 n
ö ä¼ê(envelope). φÀJk«, ;.ÀJ¦3Aþk

φ(x) ≈ |g(x)|f (x), ,φkk . ù´du
g 2 (x)
Z
V ar(θ̂) = dx − θ2 .
A φ(x)
dCauchy-SchwarzØª, V ar(θ̂)
Z 2
|g(x)|dx − θ2 .
A
±ÏL
|g(x)|
φ(x) = R .
A |g(x)|dx
ØLduOR |g(x)|dx, Ïd©1. ¦+ùU
A
φ(x)éJ, ´XJÀJÚ|g(x)|/GCφ(x), K ±C`

.
~10: 5¼êÀJ ¦^ØÓ5¼ê5OÈ©
1 e−x
Z
dx.
0 1 + x2
'X, ±ÄXe5¼ê:
f0 (x) = 1, 0 < x < 1,
f1 (x) = e−x , 0 < x < ∞,

1
f2 (x) = , − ∞ < x < ∞,
π(1 + x2 )
f3 (x) = e−x (1 − e−1 )−1 , 0 < x < 1,
4
f4 (x) = , 0<x<1
π(1 + x2 )
ù5«¼êÚgm'X:

Importance sampling
function f0,...f4 with g
2.0
g
f0
f1
f2
f3
f4
1.5
1.0
0.5
0.0
0.0 0.2 0.4 0.6 0.8 1.0

Ratios of g/f
g/f0
3.0
g/f1
g/f2
g/f3
g/f4
2.5
2.0
1.5
1.0
0.5
0.0
0.0 0.2 0.4 0.6 0.8 1.0

3ù5«5¼ê¥, f , f | '(0, 1), ÏdéõÑéOÈ
1 2
©Ã^, ÏdÇ$e. f Cg. 'ù5«5¼êeÈ©OÚ

3
O:
↑Example
m <- 10000
theta.hat <- se <- numeric(5)
g <- function(x) {
exp(-x - log(1+x^2)) * (x > 0) * (x < 1)
}
x <- runif(m) #using f0

fg <- g(x)
theta.hat[1] <- mean(fg)
se[1] <- sd(fg)
x <- rexp(m, 1) #using f1

fg <- g(x) / exp(-x)
se[2] <- sd(fg)

x <- rcauchy(m) #using f2
i <- c(which(x > 1), which(x < 0))
x[i] <- 2 #to catch overflow errors in g(x)
fg <- g(x) / dcauchy(x)
se[3] <- sd(fg)
u <- runif(m) #f3, inverse transform method

x <- - log(1 - u * (1 - exp(-1)))
fg <- g(x) / (exp(-x) / (1 - exp(-1)))
se[4] <- sd(fg)
u <- runif(m) #f4, inverse transform method

x <- tan(pi * u / 4)
fg <- g(x) / (4 / ((1 + x^2) * pi))
se[5] <- sd(fg)
rbind(theta.hat, se)

[,1] [,2] [,3] [,4] [,5]
theta.hat 0.5243630 0.5188012 0.5367005 0.52389845 0.5259339
se 0.2436977 0.4199031 0.9594029 0.09657142 0.1414429
>
↓Example
1.6 Stratified Sampling
|^È©55ÚÚrêÆ, ±¦È©R g(x)dx±©A

È©Ú, ézÈ©±üÕ¦^Monte CarloÈ©{. ØòdÈ©
©kÈ©Ú, ézÈ© Äm g, m = m + · · · + m , ÷v8I
i 1 k
V ar(θ̂k (m1 , · · · , mk )) < V ar(θ̂).
~11: Äc~f òÈ©«m(0, 1)'X©of«m, 3zf

«mþÄm/4g, ùpmoÄgê. , òù O\å5È
©R e (1 + x ) dxO.
1 −x
0
2 −1

↑Example
M <- 20 #number of replicates
T2 <- numeric(4)
estimates <- matrix(0, 10, 2)
g <- function(x) {
exp(-x - log(1+x^2)) * (x > 0) * (x < 1) }
for (i in 1:10) {
estimates[i, 1] <- mean(g(runif(M)))
T2[1] <- mean(g(runif(M/4, 0, .25)))
T2[2] <- mean(g(runif(M/4, .25, .5)))
T2[3] <- mean(g(runif(M/4, .5, .75)))
T2[4] <- mean(g(runif(M/4, .75, 1)))
estimates[i, 2] <- mean(T2)
}
estimates
apply(estimates, 2, mean)
apply(estimates, 2, var)
↓Example

Theorem 2. PM gÄe{üMonte CarloÈ©Oþ(=lþ!©Ù
¥Ä)θ̂ , ±9 M
k
1X
θ̂S = θ̂i
k i=1
L«© Oþ, z Ägêm/k. 31j þ, g(U )þÚ
©O Pθ Úσ , j = 1, · · · , k. KV ar(θ̂ ) ≥ V ar(θ̂ ).
j
2
j
M S
Proof. ^J L«ÅÀJ , P (J = j) = 1/k, j = 1, · · · , k. K

V ar(g(U )) 1
V ar(θ̂M ) = = [V ar(E(g(U |J))) + E(V ar(g(U |J)))]
M M
1
= (V ar(θJ ) + EσJ2 )
M
k
1 1X 2
= (V ar(θJ ) + σ )
M k i=1 j
1
= V ar(θJ ) + V ar(θ̂S ) ≥ V ar(θ̂S ).
M

~12: ¦^© Ä{Oc¡~f¥È© éÈ©R e (1 + 1 −x
0
x ) dxA^ k Ä{O, ¿Ú{üMonte CarloÈ©{'.

2 −1
↑Example
M <- 10000 #number of replicates
k <- 10 #number of strata
r <- M / k #replicates per stratum
N <- 50 #number of times to repeat the estimation
T2 <- numeric(k)
estimates <- matrix(0, N, 2)
g <- function(x) {
exp(-x - log(1+x^2)) * (x > 0) * (x < 1)
}
for (i in 1:N) {
estimates[i, 1] <- mean(g(runif(M)))
for (j in 1:k)
T2[j] <- mean(g(runif(M/k, (j-1)/k, j/k)))
estimates[i, 2] <- mean(T2)
}

apply(estimates, 2, mean)
apply(estimates, 2, var)
↓Example
1.7 Stratified Importance Sampling
© Äg±^35Ä{¥. 35Ä{¥, θ =
g(x)dxOþ σ /M , Ù¥σ = V ar(g(X)/f (X)), X ∼ f. A
2 2
R
^© Ä{, ò©kf«m I = {x : a ≤ x < a }, j j−1 j
Ù¥ a = −∞, a = F (j/k), j = 1, · · · , k − 1, a = ∞. Pg (x) =

0 j
−1
k j
g(x)I(a ≤ x < a ), ±9
j−1 j
Z aj
θj = gj (x)dx, j = 1, · · · , k.
aj−1

Kθ = θ 1 + · · · + θk . ézf«mI , 5¼ê±^Ý:
j
f (x)I(aj−1 ≤ x < aj )
fj (x) = f (x|Ij ) = = kf (x)I(aj−1 ≤ x < aj )
P (aj−1 ≤ X < aj )
2Pσ = V ar(g (X)/f (X)), j = 1, · · · , k. , θ© 5Ä

2
j j j
eO k
X
θ̂SI = θ̂j
i=1
Ù k k
X 1 X 2
V ar(θ̂SI ) = V ar(θ̂j ) = σ .
i=1
m i=1 j
Ù¥m = M/k. KF"

k
k X 2
σ 2 /M > σ .
M i=1 j
·±y²Xe(Ø:

bM = km5ÄeOþθ̂ Äê, θ̂ =
Theorem 3. I SI
Pk
© 5ÄeOþ, ùpθ̂ 1j θ 5Äe
i=1 θ̂j j j
Oþ, Äêm. eV ar(θ̂ ) = σ /M , ±9V ar(θ̂ ) = σ /m,K

I 2
j
2
j
k
X
σ2 − k σj2 ≥ 0,
j=1
Ò¤á =θ = · · · = θ .
1 k
d(Ø`²© ÄýØ¬* , 3g~ê|Üo´3±

~ © .
~13: 3c¡~f¥, Á¦^f (x)5¼ê, òÈ©«m©
(j/5, (j + 1)/5), j = 0, 1, · · · , 4ù 5f«m, ézf«mA^5Ä
3
{, OdÈ©Oþ9Ù² .

Chap5

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chap5

Uploaded by

Copyright:

Available Formats

Chapter 5: Monte Carlo È ©   ~ E â

1 Monte Carlo Integration and Variance Reduction 1

Previous Next First Last Back Forward 1

AkÛ(Monte Carlo)È©´«ÄuÅÄÚO{. AkÛ{

Previous Next First Last Back Forward 1

• n = 100, 000, π̂ = 3.14312.

Previous Next First Last Back Forward 2

bg´È¼ê, ·OR g(x)dx. £Á3VÇØ¥, eÅCb

þX Ýf (x), KÅCþY = g(X)Ï"

XJlX ©Ù¥)  Åê, KEg(X)Ã OÒ´A²

Previous Next First Last Back Forward 3

l01. =Cy = (x − a)/(b − a), Ïd

~2: {üMonte Carlo È©(Y) O

Previous Next First Last Back Forward 5

2. Og (X) = g(X )

~3: Monte CarloÈ©, Ã¡È© 'X¦^Monte CarloÈ©{OI

±¦^c{. ´Ilþ!©ÙU (0, x)¥Ä, exu)Cz,

déx > 0,Φ(x)O0.5 + θ̂/√2π; éx ≤ 0, OΦ(x) = 1 − Φ(−x).

x <- seq(.1, 2.5, length = 10)

Previous Next First Last Back Forward 7

~4: Monte CarloÈ©, Ã¡È©(Y) éþ~, ·±¦^, «

Previous Next First Last Back Forward 8

x <- seq(.1, 2.5, length = 10)

Ù¥f ±A| VÇ¼ê, =R A f (x)dx = 1. K)f i.i.dÅ

duθ̂ σ /m, Ù¥σ = V ar (g(X)). ÅCþX ©Ù,

·±¦^x , · · · , x ²©Ù¼ê, l σ /mO

©ÙÂñIO©Ù. Ïdé, ìC5±ÑÈ©

Previous Next First Last Back Forward 10

ÅCþI(Z ≤ 2)1VÇΦ(2) ≈ 0.977. d?g(X) ∼ B(10000, Φ(2)),

Monte Carlo {3OÈ©R b

Previous Next First Last Back Forward 12

w,ézkx, Y = g(X) = I(X ≤ x) ∼ B(1, p), p = F (x). Ï

XJOþ , K·ÏLòÙOÑ5. , , Oþ

Previous Next First Last Back Forward 14

¦+±O\þm5~ , ´O¤¬ép. 'XIO 

Previous Next First Last Back Forward 15

(X , . . . , X ))z, ·®²)U ∼ U (0, 1)±9OX

1, . . . , n. 5¿eU ∼ U (0, 1),K 1 − U ∼ U (0, 1), U 1 − U K'. K

½Â(x , . . . , x ) ≤ (y , . . . , y ) XJx ≤ y , j = 1, . . . , n. n

¼ê g = g(x , . . . , x )¡´O, XJÙézCþÑ´O, =X

Previous Next First Last Back Forward 16

Proof. Øf, gÓO. y²{´én8B. bn = 1, Ké¤kx, y ∈

Previous Next First Last Back Forward 17

=(Øén = 1¤á. y3bén − 1¤á, Kdu

= E[f (X|Xn = xn )]E[g(X|Xn = xn )],

 ªzX O¼ê, Ïdân = 1(Øk

E[f (X)g(X)] = E{E[f (X)g(X)|Xn ]}

≥ E{E[f (X|Xn = xn )]}E{E[g(X|Xn = xn )]}

Corollary 1. g = g(X , . . . , X ) ´üN, K

Previous Next First Last Back Forward 18

´ü4O¼ê. ÏdE[g(U )f (U )] ≥ E[g(U )]E[f (U )], = E[Y Y ] ≤ 0

E[Y ]E[Y ], d=¿X

Cov(Y, Y 0 ) = E[Y Y 0 ] − E[Y ]E[Y 0 ] ≤ 0.

Previous Next First Last Back Forward 19

ÏL¦^éóCþ¦Monte Carlo O ~.

Previous Next First Last Back Forward 20

¦^éóCþ{OdÈ©, ¿éÑIO ~þ.

d?g¼ê´üN, Ïdc¡íØ^÷v. )Åêu , · · · , u ∼ 1 m/2

Previous Next First Last Back Forward 21

MC.Phi <- function(x, R = 10000, antithetic = TRUE) {

Previous Next First Last Back Forward 23

éx = 1.95, éóCþ{'u{üMonte Carlo{O ~

O¥, «~ {´Cþ(Control Variates)

Chapter 5: Monte Carlo È © ~ E â

AkÛ(Monte Carlo)È©´«ÄuÅÄÚO{. AkÛ{

bg´È¼ê, ·OR g(x)dx. £Á3VÇØ¥, eÅCb

þX Ýf (x), KÅCþY = g(X)Ï"

XJlX ©Ù¥) Åê, KEg(X)Ã OÒ´A²

l01. =Cy = (x − a)/(b − a), Ïd

~2: {üMonte Carlo È©(Y) O

2. Og (X) = g(X )

~3: Monte CarloÈ©, Ã¡È© 'X¦^Monte CarloÈ©{OI

±¦^c{. ´Ilþ!©ÙU (0, x)¥Ä, exu)Cz,

déx > 0,Φ(x)O0.5 + θ̂/√2π; éx ≤ 0, OΦ(x) = 1 − Φ(−x).

~4: Monte CarloÈ©, Ã¡È©(Y) éþ~, ·±¦^, «

Ù¥f ±A| VÇ¼ê, =R A f (x)dx = 1. K)f i.i.dÅ

duθ̂ σ /m, Ù¥σ = V ar (g(X)). ÅCþX ©Ù,

·±¦^x , · · · , x ²©Ù¼ê, l σ /mO

©ÙÂñIO©Ù. Ïdé, ìC5±ÑÈ©

ÅCþI(Z ≤ 2)1VÇΦ(2) ≈ 0.977. d?g(X) ∼ B(10000, Φ(2)),

Monte Carlo {3OÈ©R b

w,ézkx, Y = g(X) = I(X ≤ x) ∼ B(1, p), p = F (x). Ï

XJOþ , K·ÏLòÙOÑ5. , , Oþ

¦+±O\þm5~ , ´O¤¬ép. 'XIO

(X , . . . , X ))z, ·®²)U ∼ U (0, 1)±9OX

1, . . . , n. 5¿eU ∼ U (0, 1),K 1 − U ∼ U (0, 1), U 1 − U K'. K

½Â(x , . . . , x ) ≤ (y , . . . , y ) XJx ≤ y , j = 1, . . . , n. n

¼ê g = g(x , . . . , x )¡´O, XJÙézCþÑ´O, =X

Proof. Øf, gÓO. y²{´én8B. bn = 1, Ké¤kx, y ∈

=(Øén = 1¤á. y3bén − 1¤á, Kdu

ªzX O¼ê, Ïdân = 1(Øk

Corollary 1. g = g(X , . . . , X ) ´üN, K

´ü4O¼ê. ÏdE[g(U )f (U )] ≥ E[g(U )]E[f (U )], = E[Y Y ] ≤ 0

E[Y ]E[Y ], d=¿X

ÏL¦^éóCþ¦Monte Carlo O ~.

¦^éóCþ{OdÈ©, ¿éÑIO ~þ.

d?g¼ê´üN, Ïdc¡íØ^÷v. )Åêu , · · · , u ∼ 1 m/2

éx = 1.95, éóCþ{'u{üMonte Carlo{O ~

O¥, «~ {´Cþ(Control Variates)

ÅCþf (X)¡g(X)Cþ(Control Variate). w, ~

EmgO V ar(g(U ))/m.

Ïd¦^Cþ{{üMonte CarloOþ ~Ç100(1 −

~8: ¦^Cþ{Monte CarloÈ© ¦^Cþ{OÈ©

d~¥a,þθ = Eg(X), g(x) = , Ù¥X ∼ U (0, 1). · e−x

AO/, θ̂ Úθ̂ Ó©Ù,

Ïd`c ∗ = 1/2. dCþOþ

Cþ{eOþθ̂ ±9`c ±ÏL£8.5O.

3 k = 1 |Ü, Ä((g(X ), f (X )), · · · , (g(X ), f (X

å¦Oβ̂ 0 = g(X) − (−c∗ )f (X), Ïd3µ?ýÿ

Ïd3R¥, Cþ{eOþIO O

5O`~êc = (c , · · · , c ). K−c = (β̂ , · · · , β̂ ), ±9d

Cþ{eO3µ = (µ , · · · , µ )?ýÿĝ(X). O

ùpCþf (x) = e .5 (1 + x2 )−1 , 0 < x < 1. , µ = E[f (X)] =

ùp·¦^ Úc~¥Ó«f, ÏdÓc O. y3θ̂ ´ ∗

3µ = 0.4763681?ýÿ. Oþθ̂9ÙIO , ~ÇÑÚc~

d?ÅCþX , · · · , X lf ¥Ä. f ¡¼ê. O

V ar(Y )/m, Y C~ê, Y ¬', dÝf AT“C” ¼

ö ä¼ê(envelope). φÀJk«, ;.ÀJ¦3Aþk

φ(x)éJ, ´XJÀJÚ|g(x)|/GCφ(x), K ±C`

©Ã^, ÏdÇ$e. f Cg. 'ù5«5¼êeÈ©OÚ

|^È©55ÚÚrêÆ, ±¦È©R g(x)dx±©A

~11: Äc~f òÈ©«m(0, 1)'X©of«m, 3zf

Proof. ^J L«ÅÀJ , P (J = j) = 1/k, j = 1, · · · , k. K

x ) dxA^ k Ä{O, ¿Ú{üMonte CarloÈ©{'.

^© Ä{, ò©kf«m I = {x : a ≤ x < a }, j j−1 j

2Pσ = V ar(g (X)/f (X)), j = 1, · · · , k. , θ© 5Ä

Oþ, Äêm. eV ar(θ̂ ) = σ /M , ±9V ar(θ̂ ) = σ /m,K

d(Ø`²© ÄýØ¬* , 3g~ê|Üo´3±