1.4. Chap3

Chapter 3: Åê){
Contents
1 Methods for Generating Random Variables 1

1.1 Generating Uniform(0,1) random number . . . . . . . . . . 2
1.2 Random Generators of Common Probability Distribution
in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 The Inverse Transform Method . . . . . . . . . . . . 8
1.2.2 The Acceptance-Rejection Method . . . . . . . . . . 18
1.2.3 Transformation Methods . . . . . . . . . . . . . . . . 22
1.2.4 Sums and Mixtures . . . . . . . . . . . . . . . . . . . 25
1.3 Multivariate Distribution . . . . . . . . . . . . . . . . . . . 33
1.3.1 Multivariate Normal Distribution . . . . . . . . . . . 33
1.3.2 Mixtures of Multivariate Normals . . . . . . . . . . . 40
1.3.3 Wishart Distribution . . . . . . . . . . . . . . . . . . 42
1.3.4 Uniform Distribution on the d−Sphere . . . . . . . . 43
1.4 Stochastic Process . . . . . . . . . . . . . . . . . . . . . . . 46
Previous Next First Last Back Forward 1

Chapter 1
Methods for Generating Random
Variables
ÚOO¥Ä:¯KÒ´lA½VÇ©Ù¥)ÅCþ(Åê).
3OÅþ^êÆ{)(þ!) ÅêÚì½O{ )
ê, §äkaqu(þ!) ÅCþÕáÄS5, ù êQ,
´ì(½{), BØU´ýÅê, Ïd~r^êÆ{)
Åê¡Åê.
3{ü/, lkoN¥)Å*ÿ, ÒI«llÑ
þ! oN¥)Å*ÿ{. l , Ü·þ!()Åê)ì´
lþ¤I, lÙ¦VÇ©Ù¥)ÅêÑ6uþ!Åê)ì.
• ÅêØUýÅ; ´, ÅêU ÏLÅêXÚ

Ou, ·Ò±r§ýÅê % ¦^.
• é)Åê?1«ÚObuyÙÎÜÕá5^ ©Ù
¦©Ù;
• Ðþ!©ÙU(0,1) Åêu)ì({) AäIe¡n
5:
– )SAäkþ!oNÅÚO5: X©Ùþ!
5, ÄÅ5, SmÕá5;
– )SÝ¯, Ó^S3, äkE5;
– )S±Ïv .
1.1 Generating Uniform(0,1) random number
5Ó{u)ì:
Xn+1 = (aXn + c) (mod) m

ùp:
(1) m: = 2 − 1(32 OÅ±L«ê).
31
(2) a: ¦ê, %ÀJ

(3) c: Oþ(±)=0.
(4) X : Ð©(«f).
0
• Xn ∈ {0, 1, . . . , m − 1}, Un = Xn /m.
•¦^Ûê«f.
• êÚ+nØkÏuaÀJ.
• F")¤ì±Ïé.
• Ø)õm/1000ê.
3R¥, ¦^ÏÌK5 ).Random.seed½öRNGkind'u"þ

!Åê )ì[&E. «ØÓa.Åê)ì9Ù5±ë
êO¡].
1.2 Random Generators of Common Probability
Distribution in R
3 R ¥þ!Åê)ì´runif. )þ!Åêª
) þÝ þ
↑Example
) þÝ þ
runif{n} # 0 1 n
)01þnxmÝ
runif{n,a,b} # a b n
matrix(runif(n*m),nrow=n,ncol=m) #
↓Example
~^VÇ©ÙVÇþ¼ê(pmf)½öVÇÝ¼ê(pdf), \
È©Ù¼ê(cdf), © ê¼ê(quantile function)±9Åê )ì¼ê
3R¥®²8¤, 'Xé©Ù, ±ëwÏ©
↑Code
dbinom(x, size, prob, log = FALSE)
pbinom(q, size, prob, lower.tail = TRUE, log.p = FALSE)
qbinom(p, size, prob, lower.tail = TRUE, log.p = FALSE)
rbinom(n, size, prob)
↓Code
R ¥~©Ù¼ê

©Ù cdf Åê)ì ëê
©Ù
beta pbeta rbeta shape1, shape2
χ ©Ù
pbinom rbinom size, prob
2
ê©Ù
pchisq rchisq df
F©Ù
pexp rexp rate
pf rf df1,df2
AÛ©Ù
gamma pgamma rgamma shape,rate or scale
éê©Ù
pgeom rgeom prob
K©Ù
plnorm rlnorm meanlog,sdlog
©Ù
pnbinom rnbinom size, prob
Poisson©Ù
pnorm rnorm mean,sd
t©Ù
ppois rpois lambda
þ!©Ù
pt rt df
punif runif min,max

¦^sample¼êlklÑoN¥Ä:
±¢ylkoN¥k£½öØ£ü«Äª?1Ä.
sample
↑Example
M1
#
i1
sample(0:1,size=10,replace=TRUE)
# a-z
õ©ÙÄ
sample(letters)
#
x<-sample(1:3,size=100,replace=TRUE,prob=c(0.2,0.3,0.5))
#> table(x)
#x
# 1 2 3
#12 30 58
↓Example

1.2.1 The Inverse Transform Method
o ëY.|Ü
)¤Åê_C{´Äu±eÙ½n
Theorem 1 (Probability Integral Transformation). eX ëY.ÅC
þ, Ùcdf F , K U = F
X X (X) ∼ U (0, 1).
Proof. ½Â
−1
FX (u) = inf{x : FX (x) = u}, 0 < u < 1.
eÅCþU ∼ U (0, 1), Ké¤kx ∈ R, k

−1
P (FX (U ) ≤ x) = P (inf{t : FX (t) = U } ≤ x)
= P (U ≤ FX (x)) = FU (FX (x)) = FX (x).
ÏdF −1
X (U )ÚÅCþX Ó©Ù.
l , e)X Å*ÿx, ±
1. Ñ_C¼ê F (u). −1
X
2. lþ!©ÙU (0, 1)¥)Åêu, -x = F −1

X (u).
ù«{¦F _¼êN´¦Ñ.
~: ¦^_C{)ëY.Ýf (x) = 3x I(0 < x < 1)Å*ÿ 2
.
d?F X (x) = x3 , (0 < x < 1), Ïd F −1
X (u) = u1/3 , Ïd
↑Code
n<-1000
u<-runif(n)
x<-u^(1/3)
hist(x,prob=TRUE,main=expression(f(x)==3*x^2))
y<-seq(0,1,0.01)

lines(y,3*y^2)
↓Code
±ã¥êÆÎÒL«õSNëwÏ© ?plotmath.
~: ¦^_C{)ê©ÙÅê.
X ∼ Exp(λ), Ïdéx > 0, F (x) = 1 − e
X , l F (u) = −λx −1
X
− log(1 − u). 5¿U Ú1 − U Ó©Ù, Ïd)ëê´λê©ÙÝ

1
λ
n Åê·-
↑Code
-log(runif(n))/lmbda ↓Code
3R¥±¦^rexp5).

o lÑ.|Ü
X lÑ.ÅCþ, ÙUP
· · · < xi−1 < xi < xi+1 < · · ·
½Â
−1
FX (u) = inf{x : FX (x) ≥ u}
K_CF −1
X (u) = xi , Ù¥F X (xi−1 ) < u ≤ FX (xi ). l )Å
êxª
1. lþ!©ÙU (0, 1)¥)Åêu

2. x = x , eF (x
i ) < u ≤ F (x ).
X i−1 X i

é, ©Ù5`, OF (x ) < u ≤ F (x )U'(J. 3lÑ.
X i−1 X i
|ÜA^_C{)ÅêØÓª±ëDevroye1nÙ.
~: ¦^_C{)0-1©ÙÅê.
dF (0) = f (0) = 1 − p±9F (1) = 1. Ïdeu > 1 − p,
X X X
KF (u) = 1; eu ≤ 1 − p, K F (u) = 0. Ïd
−1
X
−1
X
↑Code
n<-1000
p<-0.4
u<-runif(n)
nØ
x<-as.integer(u>1-p)
nØ
mean(x) # p
var(x) # p(1-p)
↓Code
3R¥,·¦^rbinom(n,1,p)5)0-1©ÙnÅ*ÿþ.
~: ¦^_C{)AÛ©ÙÅê.
duAÛ©Ù©Ù¼êF (x) = 1 − q , x = 1, 2, · · · . ÏdzÅ
X
x

ê ÑO
1 − q x−1 < u ≤ 1 − q x .
ùØªdux − 1 < log(1 − u)/log(q) ≤ x, Ù)x = dlog(1 −

u)/log(q)e, ùpdteL«Øutê(ceiling). Ïd
↑Code
n<-1000
p<-0.4
u<-runif(n)
x<-ceiling(log(1-u)/log(1-p))
↓Code
5¿U Ú1 − U Ó©Ù, ±9log(1 − U )/log(q)êVÇ0, l þã

è Ú±d
↑Code
k<-floor(log(u)/log(1-p))+1 ↓Code

3R¥,·¦^rgeom(n,p)5)ëêpAÛ©Ùn
Å*ÿþ.
~: ¦^_C{)Poisson©ÙÅê.
éPoisson©Ù5`, ¦)F (x − 1) < u ≤ F (x)'åAÛ©Ù/E
,õ. )Poisson©Ù Ä{´ÏLXe4í'X)Ú;©Ù¼
ê:
λf (x)
f (x + 1) = ; F (x + 1) = F (x) + f (x + 1).
x+1
l ±ÏL;e©Ù¼êS¥Ï¦F (x − 1) < u ≤ F (x)). R¥
±ÏLrpois(n,lambda)5)Poisson©Ùn|*ÿþ.
~: ¦^_C{)Xeéê©ÙÅê.
aθx
f (x) = P (X = x) = , x = 1, 2, · · ·
x
Ù¥0 < θ < 1±9a = (−log(1 − θ)) −1 .

w,, 4í'X
θx
f (x + 1) = f (x), x = 1, 2, · · ·
x+1
nØþù, f (x)±ÏLXþ4í'X¦, ´ééxù«O

{°ÝØ , ª ¬f (x) = 0, F (x) < 1. Ïd·ÏLf (x)L
ªexp(log(a) + xlog(θ) − log(x))5 Of (x). ¦)Øª, òF (x)
;e5, Ð©±ÀJêN , þF (x), x = 1, · · · , N , , 3
k7ÿ2ON . , , éA½u¦)F (x − 1) < u ≤ F (x), O
÷vF (x − 1) < uxê. F þ, F < u Ü6þ, Ù i
TRUE½öFALSE, u > F Ú=x − 1. ¤±

i
↑Code
rlogarithmic<-function(n,theta){
#return a random logarithmic(theta) sample size n
u<-runif(n)
#set the initial length of cdf vector
N<-ceiling(-16/log10(theta))

k<-1:N
a<--1/log(1-theta)
fk<-exp(log(a)+k*log(theta)-log(k))
Fk<-cumsum(fk)
x<-integer(n)
for(i in 1:n){
x[i]<-as.integer(sum(u[i]>Fk))
while(x[i]==N){
logf<-log(a)+(N+1)*log(theta)-log(N+1)
fk<-c(fk,exp(logf))
Fk<-c(Fk,Fk[N]+fk[N+1])
N<-N+1
x[i]<-as.integer(sum(u[i]>Fk))
}
}
x+1
}
↓Code
l ) logarithmic(0.5) ©ÙÅê
↑Example
n<-1000
theta<-0.5
Oéê©Ù©ÙÆ5é'
x<-rlogarithmic(n,theta)
#
k<-sort(unique(x))
IO
p<--1/log(1-theta)*theta^k/k
se<-sqrt(p*(1-p)/n) #
↓Example
e¡(JL²ªÇ©ÙÎÜnØ©Ù:
↑Result
> round(rbind(table(x)/n,p,se),3)
1 2 3 4 5 6 7 8 9
0.720 0.185 0.058 0.018 0.012 0.003 0.002 0.001 0.001
p 0.721 0.180 0.060 0.023 0.009 0.004 0.002 0.001 0.000
se 0.014 0.012 0.008 0.005 0.003 0.002 0.001 0.001 0.001
↓Result

1.2.2 The Acceptance-Rejection Method
bX ÚY ´ÅCþ, ÙVÇ¼ê©Of Úg. ÷v

f (t)
≤ c, ∀ t s.t. f (t) > 0
g(t)
KÀ{(Acceptance-Rejection Method)±^5)¤X Åê.
é±B)¤ÅêÅCþY , ÙVÇ¼
êg÷vf (t)/g(t) ≤ c, ∀ t s.t. f (t) > 0.
1.
2. lg ¥)Åêy.
3. lþ!©ÙU (0, 1)¥)Åêu.
4. eu < f (y)/(cg(y)), KÉx = y, ÄKáýy. E2-

4, )½êx.

3lÑ.|Ü,
h f (Y ) i X f (y) 1
P (A) = E[P (A|Y )] = E = g(y) = .
cg(Y ) y
cg(y) c
éz¦f (k) > 0kk

P (A|k)g(k) [f (k)/(cg(k))]g(k)
P (k|A) = = = f (k).
P (A) 1/c
éëY.ÅCþ|Ü, =Iy²P (Y ≤ y|U ≤ f (Y )

cg(Y )
) = FX (y). ¯¢þ
f (Y )
f (Y ) P (U ≤ cg(Y )
,Y ≤ y)
P (Y ≤ y|U ≤ ) =
cg(Y ) 1/c
f (Y )
Z y P (U ≤ cg(Y )
|Y = ω ≤ y)
= g(ω)dω
−∞ 1/c
Z y
f (ω)
= c g(ω)dω
−∞ cg(ω)
= FX (y)
À{A::
• `:ÓmØXxO\;
• ":´Ï¦N´)Åê %Cf éÐ©Ùg '(J, Ïd
ØÐg¬ÉVÇé$,E¤ $1mL.
~: ¦^À{lXe©Ù)Åê.
f (x) = 6x(1 − x), 0<x<1
ld©Ù¥)nÅêIÌoê6uÉVÇ1/c.
g(x)U (0, 1)Ý, Kc = 6, l lg¥)ÅêxÉ, Ø
f (x)
= x(1 − x) > u
cg(x)
Ïd,²þ5wIcngÌ, =2cnÅêI).
↑Code
n<-1000
j<-k<-0

y<-numeric(n)
while(k<n){
u<-runif(1)
l ¥)Åê
j<-j+1
x<-runif(1) # g
if(x*(1-x)>u) {
k<-k+1
y[k]<-x
}
}
#>j
#[1] 6153
↓Code
3ùg[¥, I6153gÌ5)n = 1000IÅê. 'Ù²

© êÚnØ© ê
↑Example
p<-seq(.1,.9,.1)
Qhat<-quantile(y,p)

Q<-qbeta(p,2,2)
se<-sqrt(p*(1-p)/(n*dbeta(Q,2,2)))
round(rbind(Qhat,Q,se),3)
↓Example
ÝC0IþE5O© ê.
1.2.3 Transformation Methods
Øc¡0 { , NõCa.±^5)¤Åê. 'X

1. eZ ∼ N (0, 1), KZ ∼ χ (1).
2 2
2. eU ∼ χ (m) ±9 V ∼ χ (n), KF =
2 2 U/m
∼ F (m, n).
V /n
3. eZ ∼ N (0, 1) ±9 V ∼ χ (n) pÕá, KT = √

2 ∼ t(n).Z
V /n
4. eU, V ∼ U (0, 1) pÕá, KZ = −2logU cos(2πV )Z

√
1 2 =
−2logV sin(2πU ) pÕáIOÅCþ.
√

'X
• eU ∼ Gamma(r, λ), V ∼ Gamma(s, λ), pÕá, K U
U +V
∼
Beta(r, s).
~: ¦^C{l)éê©ÙÅê.
5¿eU, V ∼ U (0, 1) pÕá, K

logV
X = 1+
log(1 − (1 − θ)U )
Ñlëêθéê©Ù. l
1. lU (0, 1)¥)u
2. lU (0, 1)¥)v
3. x = b1 + log(v)/log(1 − (1 − θ) u )c

¢yè
↑Code
n<-1000
theta<-0.5
u<-runif(n)
v<-runif(n)
Oéê©ÙVÇ
x<-floor(1+log(v)/log(1-(1-theta)û))
k<-1:max(x)#
p<--1/log(1-theta)*theta^k/k
se<-sqrt(p*(1-p)/n)
p.hat<-tabulate(x)/n
round(rbind(p.hat,p,se),3)
↓Code
'u_C{, C{kÇ(o?)
↑Code
rlogarithmic<-function(n,theta){
stopifnot(all(theta>0 & theta <1))
th<-rep(theta,length=n)
u<-runif(n)

v<-runif(n)
x<-floor(1+log(v)/log(1-(1-theta)û))
return(x)
}
↓Code
1.2.4 Sums and Mixtures
ÅCþÚ½ö·Ü´«AÏa.C.
ÚòÈ'©Ùk
• χ (ν): ν iid IO©Ù²òÈ.
2
• K©Ù NegBin(r, p): r iid Geom(p) òÈ.
• Gamma(r, λ): r Õá Exp(λ) òÈ.
~: ¦^ÅCþÚ)χ ©ÙÅê. 2

ν IOÅCþ²Úχ (ν)©ÙÅCþ, Ïd2
↑Code
n<-1000
nu<-2
{
X<-matrix(rnorm(n*nu),n,nu)^2
# 1
{
y<-rowSum(X)
# 2
y<-apply(X,MARGIN=1,FUN=sum)
↓Code
éÅCþ·Ü, ·kw½Â:

¡ Å C þ l Ñ · Ü, X J Ù © Ù , Å C
þX , X , · · · ©Ù\:
1 2
Definition
X
FX (x) = θi FXi (x).
θi > 0
P
.
θi = 1
éëY.ÅCþ, aqk
¡ÅCþëY·Ü, XJÙ©Ù,©ÙxX|Y = y\

: Z
FX (x) = FX|Y =y (x)fY (y)dy. Definition
Ù¥R f Y (y)dy = 1.

~: )Xe·Ü©ÙÅê.
FX (x) = pFX1 (x) + (1 − p)FX2 (x)
= p Pr(X1 ≤ x) + (1 − p) Pr(X2 ≤ x)
= Pr(B = 1) ∗ Pr{B ∗ X1 + (1 − B) ∗ X2 ≤ x|B = 1}
+ Pr(B = 0) ∗ Pr{B ∗ X1 + (1 − B) ∗ X2 ≤ x|B = 0}
= Pr{B ∗ X1 + (1 − B) ∗ X2 ≤ x},
ùp B ∼ Binomial(p).
)dÅê{w,
1. )êk ∈ {1, 2}, ùpP (1) = p, P (2) = 1 − p.
2. ek = 1, KlF ¥)x , ¿-x = x .

X1 1 1
3. ek = 2, KlF ¥)x , ¿-x = x .

X2 2 2

~: Γ©Ù·Ü.
5
X
FX = θj FXj
i=1
Ù¥X j ∼ Gamma(r = 3, λj = 1/j) pÕá, θ j = j/15, j = 1, · · · , 5.
↑Code
n<-1000
k<-sample(1:5,size=n,replace=TRUE,prob=(1:5)/15)
rate<-1/k
x·ÜÝ±9©þÝ
x<-rgamma(n,shape=3,rate=rate)
#
plot(density(x),xlim=c(0,40),ylim=c(0,.3),lwd=3,xlab="x",main="")
for(i in 1:5)
lines(density(rgamma(n,3,1/i)))
↓Code

~: ΓÝ·Ü.
5
X
f (x) = θj fj (x), x > 0
j=1
Ù¥f Gamma(3, λ )Ý. xd·ÜÝã/, IOÙ

j j
↑Code
f<-function(x,lambda,theta){
sum(dgamma(x,3,lambda)*theta)
}
↓Code
ùpdgamma(x, 3, lambda) ∗ theta´þ(θ f (x), · · · , θ f (x)).

1 1 5 5
↑Code
¦^ ¼êI
x<-seq(0,8,length=200)
dim(x)<-length(x) # apply
p<-c(.1,.2,.2,.3,.2)
O·ÜÝ3 ?
lambda<-c(1,1.5,2,2.5,3)
# x
y<-apply(x,1,f,lambda=lambda,theta=p)

plot(x,y,type="l",ylim=c(0,.85),lwd=3,ylab="Density")
for(j in 1:5){
y<-apply(x,1,dgamma,shape=3,rate=lambda[j])
lines(x,y)
}
↓Code

~: P oisson − Gamma·Ü.
eX|Λ = λ ∼ P ois(λ), Λ ∼ Gamma(r, β), K X ∼ N B(r, p = β/(1 +
β)). ~`²Poisson-Gamma·Ü,¿ÚK ©Ù'.
↑Code
#Poisson-Gamma Mixture
n<-1000
r<-4
´Å
beta<-3
lambda<-rgamma(n,r,beta) #lambda
x<-rpois(n,lambda)
#compare with negative binomial
mix<-tabulate(x+1)/n
negbin<-round(dnbinom(0:max(x),r,beta/(1+beta)),3)
se<-sqrt(negbin*(1-negbin)/n)
round(rbind(mix,negbin,se),3)
↓Code

1.3 Multivariate Distribution
1.3.1 Multivariate Normal Distribution
ÅþX = (X , · · · , X )Ñld©ÙN (µ, Σ), XJÙéÜÝ

1 d d
k/ª
1
f (x) = (2π)−d/2 |Σ|−1/2 exp{− (x − µ)0 Σ−1 (x − µ)}, x ∈ Rd .
2
)N (µ, Σ)Åê, ÏLüÚ¢y:

d
1. )i.i.d.IO©ÙÅCþZ , · · · , Z . 1 d
2. òZ = (Z , · · · , Z )CX = (X , · · · , X )¦ÙÏ"µ,
1 d 1 d
Σ:
X = CZ + µ,
ùpCC = Σ. ©)Σ±´Ì©){(eigen), Choleski©)(chol)½

0
öÛÉ©)(svd).
Ø¬éÅþ?1g5C, ²~´é¤
k|¤Ý ?1C. bZ = (Z )´ n × dÝ , Ù ij
¥Z iid N (0, 1). KZ 1ńd IOÅCþ*ÿ. KdI

ij
?1C´
X = ZQ + JµT ,
Ù¥Q Q = Σ, J = J = (1, · · · , 1) . KX 1Ń (µ, Σ)Åê. l

T
n×1
T
d
o(Xe
1. )kIO©ÙÅê¤n × dÝ Z.
2. O©)Σ = Q Q. T
3. A^C X = ZQ + Jµ . T
Ù¥X = ZQ + Jµ 3R¥±Xe¢y
T
↑Code

Z<-matrix(rnorm(n*d),nrow=n,ncol=d)
X<-Z%*%Q+matrix(mu,n,d,byrow=TRUE)
↓Code
)¤N (µ, Σ)ÅêÌ©){ du

d
Σ1/2 = P Λ1/2 P T
¤±
↑Code
mu <- c(0, 0)
Sigma <- matrix(c(1, .9, .9, 1), nrow = 2, ncol = 2)
rmvn.eigen <-
function(n, mu, Sigma) {
# generate n random vectors from MVN(mu, Sigma)
# dimension is inferred from mu and Sigma
d <- length(mu)
ev <- eigen(Sigma, symmetric = TRUE)
lambda <- ev$values

V <- ev$vectors
R <- V %*% diag(sqrt(lambda)) %*% t(V)
Z <- matrix(rnorm(n*d), nrow = n, ncol = d)
X <- Z %*% R + matrix(mu, n, d, byrow = TRUE)
X
}
# generate the sample
X <- rmvn.eigen(1000, mu, Sigma)
plot(X, xlab = "x", ylab = "y", pch = 20)
print(colMeans(X))
print(cor(X))
↓Code
) ¤N (µ, Σ) Å ê Û É © ) { âÛÉ©)´Σ

d
1/2 =
U D V . ¤±
1/2 T
↑Code
rmvn.svd <-

d <- length(mu)
S <- svd(Sigma)
R <- S$u %*% diag(sqrt(S$d)) %*% t(S$v) #sq. root Sigma
Z <- matrix(rnorm(n*d), nrow=n, ncol=d)
X <- Z %*% R + matrix(mu, n, d, byrow=TRUE)
X
}
↓Code
)¤N (µ, Σ)ÅêCholeski©){

d
↑Code
rmvn.Choleski <-
d <- length(mu)
Q <- chol(Sigma) # Choleski factorization of Sigma
Z <- matrix(rnorm(n*d), nrow=n, ncol=d)
X <- Z %*% Q + matrix(mu, n, d, byrow=TRUE)
X
}

↓Code
'«)¤ì5U
·®²?Ø )ÅêØÓ{, @o=«{ÐQ? «Ä
±mE,5, O¡Ä±´â[8Ó½õë
ê, ½öÓþ , ?1 µ(3± §¥ò9).
ùp·¦^system.time5µ)¤ìmE,5.
↑Code
library(MASS)
library(mvtnorm)
n <- 100 #sample size
d <- 30 #dimension
N <- 2000 #iterations
mu <- numeric(d)
set.seed(100)
system.time(for (i in 1:N)

rmvn.eigen(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
rmvn.svd(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
rmvn.Choleski(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
mvrnorm(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
rmvnorm(n, mu, cov(matrix(rnorm(n*d), n, d))))
set.seed(100)
cov(matrix(rnorm(n*d), n, d)))
detach(package:MASS)
detach(package:mvtnorm)
↓Code

3õ©ÙÅê)¤L§¥, Ü©´ó´©) Ý . ùp¦
^ Ý ´ü Ý , Ïd, Å) Ý Σ
3zgÌĆz, ´Σ~Cü . 3Ó Ý e
'«ØÓ{, zg$1 ÑòÅê«f¡E. g$1==´
) Ý , ±Úom'.
1.3.2 Mixtures of Multivariate Normals
õ·Ü
pNd (µ1 , Σ1 ) + (1 − p)Nd (µ2 , Σ2 )
)d·Ü©ÙnÅþ:
↑Code
library(MASS) #for mvrnorm
loc.mix <- function(n, p, mu1, mu2, Sigma) {
#generate sample from BVN location mixture
n1 <- rbinom(1, size = n, prob = p)

n2 <- n - n1
x1 <- mvrnorm(n1, mu = mu1, Sigma)
x2 <- mvrnorm(n2, mu = mu2, Sigma)
X <- rbind(x1, x2) #combine the samples
return(X[sample(1:n), ]) #mix them
}
↓Code
^d§S)10004©ÙÅþ:
↑Example
x <- loc.mix(1000, .5, rep(0, 4), 2:5, Sigma = diag(4))

r <- range(x) * 1.2
par(mfrow = c(2, 2))
for (i in 1:4)
hist(x[ , i], xlim = r, ylim = c(0, .3), freq = FALSE,
main = "", breaks = seq(-5, 10, .5))
↓Example

1.3.3 Wishart Distribution
eM = X X, X lN (0, Σ)¥Än × dÅÝ , KM ÑlWishart©

T
d
ÙW (Σ, n). d = 1, W (σ , n) = σ χ (n).

d 1
2 2 2
w,, lWishart©Ù¥)Åê±ÏLXeª:
1. lN (0, Σ)¥)¤X
d
2. -W = X X. T
ù«{´«$Ç{. ù´du7L)ndÅê5û½d(d +
1)/2. «Çp{´ÄuBartlett©): -T = (T ) ij
d × denÝ , Ù÷v
1. Tij i.i.d ∼ N (0, 1), i > j.
p
2. Tii ∼ χ2 (n − i + 1), i = 1, · · · , d.

KÝ A = T T ÑlW (I , n). Ïd)W (Σ, n)±ÏLΣCholeski©
T
d d d
)Σ = LL , KLAL ∼ W (Σ, n).

T T
d
1.3.4 Uniform Distribution on the d−Sphere
¥¡=8Ü{x ∈ R : kxk = 1}. 3d−¥¡þþ!©ÙÅþk

d− d 2
ÓU. )dÅê±ÄuXe5:
eX , · · · , X i.i.d N (0, 1), K(U , · · · , U )ÑlR ¥ü ¥¡þþ!
1 d 1 d
d
©Ù, Ù¥
Xj
Uj = , j = 1, · · · , d. (1.1)
kxk
{Xe: ézu , E i
1. lN (0, 1)¥)Åêx , · · · , x i1 id
2. Oîªkxk = (x + · · · + x ) 2
i1
2 1/2 .
id
3. -u = x /kxk, u = (u , · · · , u
ij ij i i1 id ).

3R¥±ÏLXeªJpÇ:
1. )ndÅê¤n × dÝ M , Ù1i1éAu1iÅê
þ.
2. éz1O(1.1),rdn3þL¥.
3. ézêM [i, j]Ø±L[i], U .
¢yèXe
↑Code
runif.sphere <- function(n, d) {
# return a random sample uniformly distributed
# on the unit sphere in R ^d
M <- matrix(rnorm(n*d), nrow = n, ncol = d)
L <- apply(M, MARGIN = 1,
FUN = function(x){sqrt(sum(x*x))})
D <- diag(1 / L)
U <- D %*% M

U
}
↓Code
xÑ2002 ±þþ!©ÙÅêÑ:ã:
↑Example
X <- runif.sphere(200, 2)
par(pty = "s")
plot(X, xlab = bquote(x[1]), ylab = bquote(x[2]))
par(pty = "m")
↓Example

1.4 Stochastic Process
àgPoissonL§
ÏL[5ïÄOêL§, ·±) L§km´».
OêL§{N (t), t ≥ 0}Ç´λàgPoissonL§, X
J
1. N (t)kÕáOþ5
(λt)n −λt
2. P (N (s + t) − N (s) = n) = n!
e .
éàgPoissonL§, ¯5m T , T , · · · í.i.dëêλ

1 2
ê©ÙÅCþ. Ïd, )àgPoissonL§{)5m m

T , · · · , T , · · · , K1nm5S = T + · · · + T , S
1 n n 1 n

{Tn }∞
n=1½ö {S } =±L«L§´».∞
n n=1
, «ª´|^S , · · · , S |N (t) = nÓ©ÙulU (0, t)¥Än

1 n
{ügSÚOþ. L§3½t?Gu3[0, t]Sê

8,min{k : S > t} − 1. = eS ´mLt, KN (t) = n − 1.
k n
[àgPoissonL§3[0, t ]«mþG{Xe 0
1. S = 0
1
2. é÷vS ≤ t j = 1, 2, . . .,
j 0
(a) )T ∼ exp(λ). j
(b) -S = T + · · · + T .
j 1 j
3. N (t0 ) = minj (Sj > t0) − 1.
'X, )Ç2àgPoissonL§3[0, 3]SG,

↑Code
lambda <- 2
t0 <- 3

Tn <- rexp(100, lambda) #interarrival times
Sn <- cumsum(Tn) #arrival times
n <- min(which(Sn > t0))-1 #arrivals in [0, t0]
↓Code
¦^, «ªÒ´
↑Code
lambda <- 2
t0 <- 3
upper <- 100
pp <- numeric(10000)
for (i in 1:10000) {
N <- rpois(1, lambda * upper)
Un <- runif(N, 0, upper) #unordered arrival times
Sn <- sort(Un) #arrival times
n <- min(which(Sn > t0)) #arrivals+1 in [0, t0]
pp[i] <- n - 1 #arrivals in [0, t0]
} ↓Code

dPoisson©Ù5, N (3)Ï"Ú ÑAT´6:
c(mean(pp), var(pp))
àg Poisson L§
OêL§{N (t), t ≥ 0}rÝ´λ(t)PoissonL§, XJ
1. N (t)kÕáOþ5
, Ù
(m̃(s,t))n −m̃(s,t)
2. P (N (s + t) − N (s) = n) = e
¥m̃(s, t) = R λ(y)dy.
n!
s+t
s
XJàgPoissonL§rÝ¼êk., =λ(t) ≤ λ < ∞,, K± 0
ÏL làgPoissonL§Ä). ù´ÏlrÝλàgPoissonL
§¥ Ä, ¦3tu)¯±λ(t)/λÉ(Oê), Oê
L§ÒÑlrÝλ(t)àg PoissonL§. )[0, t ]àgPoissonL 0 [ o?]
§±ÏLd{¢y. kéλ < ∞, ¦λ(t) ≤ λ , ∀0 ≤ t ≤ t .
0 0 0

, l rÝλ àgPoissonL§¥)¯5{S }, , ±V
0 j
Çλ(S )/λ ÉzS . {Xe

j 0 j
1. S = 0 1
2. é÷vS ≤ t j = 1, 2, . . .,
j 0
(a) )T ∼ exp(λ ), -S = T + · · · + T .

j 0 j 1 j
(b) )þ!ÅêU ∼ U (0, 1). j
(c) eU ≤ λ(S )/λ , KÉS , -I = 1, ÄK-I = 0.

j j 0 j j j
3. £m {S : I = 1}. j j
'Xλ(t) = 3cos (t), Kλ(t) ≤ 3 = λ . Ïd1j mU ≤

2
0 j
3cos (S )/3 = cos (S ) É.

2
j
2
j
↑Example
lambda <- 3
upper <- 100

N <- rpois(1, lambda * upper)
Tn <- rexp(N, lambda)
Sn <- cumsum(Tn)
Un <- runif(N)
keep <- (Un <= cos(Sn)^2) #indicator, as logical vector
Sn[keep]
round(Sn[keep], 4)
↓Example
L§3t?G=
sum(Sn[keep]<=t)
#L§
OêL§{N (t), t ≥ 0}#L§, XJ mm
T , T , . . .í.i.d.
1 2

~: bmm ÑlëêpAÛ©Ù, =mm K
ê, Ïd S = T + · · · + T ÑlK©Ù,ëêjÚp, Ïd
j 1 j
↑Example
t0 <- 5
Tn <- rgeom(100, prob = .2) #interarrival times
Sn <- cumsum(Tn) #arrival times
n <- min(which(Sn > t0)) #arrivals+1 in [0, t0]
↓Example
N (t0 ) ©Ù±ÏLEXþL§5O
↑Example
Nt0 <- replicate(1000, expr = {

Sn <- cumsum(rgeom(100, prob = .2))
min(which(Sn > t0)) - 1

})
table(Nt0)/1000
Nt0
↓Example
þ¼êEN (t)±ÏLXeO
↑Example
t0 <- seq(0.1, 30, .1)

mt <- numeric(length(t0))
for (i in 1:length(t0)) {
mt[i] <- mean(replicate(1000,
{
Sn <- cumsum(rgeom(100, prob = .2))
min(which(Sn > t0[i])) - 1
}))

}
plot(t0, mt, type = "l", xlab = "t", ylab = "mean")
abline(0, .25)
↓Example
é¡ÅiÄ
X , X , · · · í.i.dÅCþ
P , ÷vPr(X
Pr(X = −1) = 1/2, -S = X , ±9S K
1 2 i = 1) =
n
L§{S , n ≥ 0}¡é¡ÅiÄ.
i n i=1 i 0 = 0,
n
)é¡ÅiÄL§é{ü
n <- 400
incr <- sample(c(-1, 1), size = n, replace = TRUE)
S <- as.integer(c(0, cumsum(incr)))

plot(0:n, S, type = "l", main = "", xlab = "i")
eÐ©S = 0, L§3m[1, 400]S£0gêwhich(S==0).

0
XJI´L§S 3mnG, Ø´{¤, é'n,

n
±ÏLXeª5 pÇ):
bS = 0L§Ð©G, eL§3mnc£0, K3
0
)S ÿ, ·± ÑL§3Cg£0cm.

n
-T L§Äg£0, K)S , ±{zd¯Kk) n
mT , om(T Ú)ÄgLn, , lncCg£0

m©, )OþX , , ò§\.
i
T ©Ù
n−1 1 Γ(2n − 1)
p2n = P (T = 2n) = C2n−2 = , n ≥ 1.
n22n−1 n22n−1 Γ2 (n)
P (T = 2n + 1) = 0, n ≥ 0.
K{Xe:
-W 1jg£Ð© Im,
j
1. W = 0 1
2. é÷vW ≤ n j = 1, 2, . . .,
j
(a) lm©Ù¥)T , Äg£0. j
(b) -W = T + · · · + T .
j 1 j
3. -t = W − T
0 j j
4. s = 0. 1
5. )lt + 1nOþ. éi = 1, 2, . . . , n − t
0 0
(a) )ÅOþx ∼ P (X = ±1) = 1/2. i
(b) s = x + · · · + x .i 1 i
(c) es = 0, Oêìi = 1.
i
6. £s . i

lT ©Ù¥pÇ)Åê{ë Devroye, p.754. ùp·
¦^{ü´1ÇØp{. 5¿
1
p2n = P (X = n − 1), X ∼ B(2n − 2, 1/2)
2n
ÏdO
↑Example
#compute the probabilities directly

n <- 1:10000
p2n <- exp(lgamma(2*n-1)
- log(n) - (2*n-1)*log(2) - 2*lgamma(n))
↓Example
½ö¦^©Ù±9_C{O
↑Example

P2n <- (.5/n) * dbinom(n-1, size = 2*n-2, prob = 0.5)
pP2n <- cumsum(P2n)
#|^_C{)T
# u<-runif(1)
# Tj<-2*(1+sum(u>pP2n))
↓Example
y3b½n,·O3(0, n]S g£0m, K

↑Example
#given n compute the time of the last return to 0 in (0,n]

n <- 200
sumT <- 0
while (sumT <= n) {
u <- runif(1)

s <- sum(u > pP2n)
if (s == length(pP2n))
warning("T is truncated")
Tj <- 2 * (1 + s)
#print(c(Tj, sumT))
sumT <- sumT + Tj
}
sumT - Tj
↓Example

1.4. Chap3

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1.4. Chap3

Uploaded by

Copyright:

Available Formats

Chapter 3: Åê){

1 Methods for Generating Random Variables 1

Previous Next First Last Back Forward 1

Previous Next First Last Back Forward 1

1.1 Generating Uniform(0,1) random number

Previous Next First Last Back Forward 2

(2) a: ¦ê, %ÀJ

• Xn ∈ {0, 1, . . . , m − 1}, Un = Xn /m.

3R¥, ¦^ÏÌK5 ).Random.seed½öRNGkind'u"þ

Previous Next First Last Back Forward 5

Previous Next First Last Back Forward 6

Previous Next First Last Back Forward 7

eÅCþU ∼ U (0, 1), Ké¤kx ∈ R, k

= P (U ≤ FX (x)) = FU (FX (x)) = FX (x).

2. lþ!©ÙU (0, 1)¥)Åêu, -x = F −1

Previous Next First Last Back Forward 9

− log(1 − u). 5¿U Ú1 − U Ó©Ù, Ïd)ëê´λê©ÙÝ

Previous Next First Last Back Forward 10

1. lþ!©ÙU (0, 1)¥)Åêu

Previous Next First Last Back Forward 11

Previous Next First Last Back Forward 12

ùØªdux − 1 < log(1 − u)/log(q) ≤ x, Ù)x = dlog(1 −

5¿U Ú1 − U Ó©Ù, ±9log(1 − U )/log(q)êVÇ0, l þã

Previous Next First Last Back Forward 13

Ù¥0 < θ < 1±9a = (−log(1 − θ)) −1 .

Previous Next First Last Back Forward 14

nØþù, f (x)±ÏLXþ4í'X¦, ´ééxù«O

TRUE½öFALSE, u > F Ú=x − 1. ¤±

Previous Next First Last Back Forward 15

Previous Next First Last Back Forward 17

bX ÚY ´ÅCþ, ÙVÇ¼ê©Of Úg. ÷v

KÀ{(Acceptance-Rejection Method)±^5)¤X Åê.

3. lþ!©ÙU (0, 1)¥)Åêu.

4. eu < f (y)/(cg(y)), KÉx = y, ÄKáýy. ­E2-

Previous Next First Last Back Forward 18

éz¦f (k) > 0kk

éëY.ÅCþ|Ü, =Iy²P (Y ≤ y|U ≤ f (Y )

Previous Next First Last Back Forward 20

3ùg[¥, I6153gÌ5)n = 1000IÅê. 'Ù²

Previous Next First Last Back Forward 21

Øc¡0 { , NõCa.±^5)¤Åê. 'X

3. eZ ∼ N (0, 1) ±9 V ∼ χ (n) pÕá, KT = √

4. eU, V ∼ U (0, 1) pÕá, KZ = −2logU cos(2πV )Z

Previous Next First Last Back Forward 22

Previous Next First Last Back Forward 23

Previous Next First Last Back Forward 24

1.2.4 Sums and Mixtures

• K©Ù NegBin(r, p): r  iid Geom(p) òÈ.

• Gamma(r, λ): r Õá Exp(λ) òÈ.

Previous Next First Last Back Forward 25

Previous Next First Last Back Forward 26

¡ÅCþëY·Ü, XJÙ©Ù,©ÙxX|Y = y\

Previous Next First Last Back Forward 27

= Pr(B = 1) ∗ Pr{B ∗ X1 + (1 − B) ∗ X2 ≤ x|B = 1}

+ Pr(B = 0) ∗ Pr{B ∗ X1 + (1 − B) ∗ X2 ≤ x|B = 0}

2. ek = 1, KlF ¥)x , ¿-x = x .

3. ek = 2, KlF ¥)x , ¿-x = x .

Previous Next First Last Back Forward 28

Ù¥X j ∼ Gamma(r = 3, λj = 1/j) pÕá, θ j = j/15, j = 1, · · · , 5.

Previous Next First Last Back Forward 29

Ù¥f Gamma(3, λ )Ý. xd·ÜÝã/, IOÙ

ùpdgamma(x, 3, lambda) ∗ theta´þ(θ f (x), · · · , θ f (x)).

Previous Next First Last Back Forward 30

Chapter 3: Åê){

(2) a: ¦ê, %ÀJ

3R¥, ¦^ÏÌK5 ).Random.seed½öRNGkind'u"þ

eÅCþU ∼ U (0, 1), Ké¤kx ∈ R, k

2. lþ!©ÙU (0, 1)¥)Åêu, -x = F −1

− log(1 − u). 5¿U Ú1 − U Ó©Ù, Ïd)ëê´λê©ÙÝ

1. lþ!©ÙU (0, 1)¥)Åêu

ùØªdux − 1 < log(1 − u)/log(q) ≤ x, Ù)x = dlog(1 −

5¿U Ú1 − U Ó©Ù, ±9log(1 − U )/log(q)êVÇ0, l þã

nØþù, f (x)±ÏLXþ4í'X¦, ´ééxù«O

TRUE½öFALSE, u > F Ú=x − 1. ¤±

bX ÚY ´ÅCþ, ÙVÇ¼ê©Of Úg. ÷v

KÀ{(Acceptance-Rejection Method)±^5)¤X Åê.

3. lþ!©ÙU (0, 1)¥)Åêu.

4. eu < f (y)/(cg(y)), KÉx = y, ÄKáýy. E2-

éz¦f (k) > 0kk

éëY.ÅCþ|Ü, =Iy²P (Y ≤ y|U ≤ f (Y )

3ùg[¥, I6153gÌ5)n = 1000IÅê. 'Ù²

Øc¡0 { , NõCa.±^5)¤Åê. 'X

3. eZ ∼ N (0, 1) ±9 V ∼ χ (n) pÕá, KT = √

4. eU, V ∼ U (0, 1) pÕá, KZ = −2logU cos(2πV )Z

• K©Ù NegBin(r, p): r iid Geom(p) òÈ.

• Gamma(r, λ): r Õá Exp(λ) òÈ.

¡ÅCþëY·Ü, XJÙ©Ù,©ÙxX|Y = y\

2. ek = 1, KlF ¥)x , ¿-x = x .

3. ek = 2, KlF ¥)x , ¿-x = x .

Ù¥X j ∼ Gamma(r = 3, λj = 1/j) pÕá, θ j = j/15, j = 1, · · · , 5.

Ù¥f Gamma(3, λ )Ý. xd·ÜÝã/, IOÙ

ùpdgamma(x, 3, lambda) ∗ theta´þ(θ f (x), · · · , θ f (x)).

ÅþX = (X , · · · , X )Ñld©ÙN (µ, Σ), XJÙéÜÝ

)N (µ, Σ)Åê, ÏLüÚ¢y:

ùpCC = Σ. ©)Σ±´Ì©){(eigen), Choleski©)(chol)½

¥Z iid N (0, 1). KZ 1´nd IOÅCþ*ÿ. KdI

Ù¥Q Q = Σ, J = J = (1, · · · , 1) . KX 1´N (µ, Σ)Åê. l

)¤N (µ, Σ)ÅêÌ©){ du

) ¤N (µ, Σ) Å ê Û É © ) { âÛÉ©)´Σ

)¤N (µ, Σ)ÅêCholeski©){

eM = X X, X lN (0, Σ)¥Än × dÅÝ , KM ÑlWishart©

ÙW (Σ, n). d = 1, W (σ , n) = σ χ (n).

¥¡=8Ü{x ∈ R : kxk = 1}. 3d−¥¡þþ!©ÙÅþk

3. ézêM [i, j]Ø±L[i], U .

éàgPoissonL§, ¯5m T , T , · · · ´i.i.dëêλ

ê©ÙÅCþ. Ïd, )àgPoissonL§{)5m m

, «ª´|^S , · · · , S |N (t) = nÓ©ÙulU (0, t)¥Än

{ügSÚOþ. L§3½t?Gu3[0, t]Sê

(a) )T ∼ exp(λ). j

'X, )Ç2àgPoissonL§3[0, 3]SG,

XJàgPoissonL§rÝ¼êk., =λ(t) ≤ λ < ∞,, K± 0

Çλ(S )/λ ÉzS . {Xe

(a) )T ∼ exp(λ ), -S = T + · · · + T .

(b) )þ!ÅêU ∼ U (0, 1). j

(c) eU ≤ λ(S )/λ , KÉS , -I = 1, ÄK-I = 0.

'Xλ(t) = 3cos (t), Kλ(t) ≤ 3 = λ . Ïd1j mU ≤

3cos (S )/3 = cos (S ) É.

eÐ©S = 0, L§3m[1, 400]S£0gêwhich(S==0).

XJI´L§S 3mnG, Ø´{¤, é'n,

)S ÿ, ·± ÑL§3Cg£0cm.

-T L§Äg£0, K)S , ±{zd¯Kk) n

mT , om(T Ú)ÄgLn, , lncCg£0

(a) lm©Ù¥)T , Äg£0. j

(a) )ÅOþx ∼ P (X = ±1) = 1/2. i

y3b½n,·O3(0, n]S g£0m, K