Alternative

Alternative RPM’s to DP
Bayesian nonparametric methods and random

distributions
Ramsés H. Mena
IIMAS
1 Alternative RPM’s to DP
RPMs via normalized log-Gaussian process
RPMs via normalization of subordinators N-Prior(α, ν)
Generalizations / alternatives for Q

All these constructions / representations of Dα served as starting
point for generalizations or alternative models to DP.
Normalized log-Gaussian processes (Lenk, 1988)
Species sampling models (Pitman, 1996)
→ Stick breaking priors (Hjort, 2000; Ishwaran and James, 2001)
Poisson-Kingman models (Pitman, 2003)
Normalized random measures with ind. inc. (Regazzini, Lijoi and
Prünster, 2003)

Prünster, 2003)
Other constructions:
Neutral to the right (NTR) processes (Doksum, 1974)
NTR species sampling models (James, 2007)
Berstein polynomials (Petrone, 1999)

Prünster, 2003)
Other constructions:
Neutral to the right (NTR) processes (Doksum, 1974)
NTR species sampling models (James, 2007)
Berstein polynomials (Petrone, 1999)
→ An excellent recent review is given by Lijoi and Prünster (2010),

where many generalizations can be encompassed through the
concept of completely random measures by Kingman (1967,1993).
Normalized log-Gaussian process
• Normalized log-Gaussian process
exp{Zx }
P(dx) := R ,
exp{Zs }dλ(s)
{Zt } ∼ PG(µ, σ) ⇒ paths can be regarded as continuous densities

Zt P (dx)
Define W := {Wt }t∈X con Wt = exp{Zt }, the log-Gaussian process

(W ∼ LG(µ, σ))
Normalized log-Gaussian process
• Normalized log-Gaussian process
exp{Zx }
P(dx) := R ,
exp{Zs }dλ(s)
{Zt } ∼ PG(µ, σ) ⇒ paths can be regarded as continuous densities

Zt P (dx)
Define W := {Wt }t∈X con Wt = exp{Zt }, the log-Gaussian process

(W ∼ LG(µ, σ))
Generalization by Lenk 1988

The Logistic normal process, denoted by LNSX (µ, σ, ξ), is defined as the random
density given by the transformation
W ∗ (x)
f (x) = R ,
E
W ∗ (s) dλ(s)
with the distribution of W ∗ given by

R R ξ
A X
W (s) dλ(s) dΛ(W )
Λξ (A) = ,
C(ξ, µ)
where ξ = 1, 2, . . ., Λ ∼ W ∼ LNX (µ, σ) and

"Z  
K # Z X K YK
C(K , µ)=E Wdλ = exp σ(si , sj ) W0 (sk ) dλ(s),
X XK 
i<j

k=1
for K > 0 and where W0 (s) = E[W (s)] = exp{µ(s) + σ(s, s)/2}
Let f ∼ LNSX (µ, σ, ξ) and x ∈ XK , hence

"K #
Y C(ξ − K , µ∗ )
E f (xi ) = M(x) ,
C(ξ, µ)
i=1
PK
where µ∗ (s) = µ(s) + i=1 σ(s, xi ) y
K
" #
Y
M(x) = E W (xk )
k=1
 
K K
!
 X σ(xk , xk ) X 
= exp µ(xk ) + + σ(xi , xj ) .
 2 
k=1 i<j

iid
Let X1 , X2 , . . . X-valued exchangeable r.v.’s and Xi ∼ f and
f ∼ LNSX (µ, σ, ξ) hence the posterior RPM is given by
{f | X1 , . . . , Xn } ∼ LNSX (µ∗ , σ, ξ ∗ )
namely it is a conjugate RPM

iid
{f | X1 , . . . , Xn } ∼ LNSX (µ∗ , σ, ξ ∗ )
µ∗ (s) = µ(s) + ni=1 σ(s, xi ) ξ∗ = ξ − n
P

iid
{f | X1 , . . . , Xn } ∼ LNSX (µ∗ , σ, ξ ∗ )
µ∗ (s) = µ(s) + ni=1 σ(s, xi ) ξ∗ = ξ − n
P
The Bayes estimator (that coincides with the predictive dist.) is given by
( n )
X
fn (y ) = E[f (y ) | x1 , . . . , xn ] ∝ W0 (y ) exp σ(y , xi ) C(ξ − n − 1, µ∗y )
i=1
where µ∗y = µ∗ (s) + σ(s, y )


iid
{f | X1 , . . . , Xn } ∼ LNSX (µ∗ , σ, ξ ∗ )
µ∗ (s) = µ(s) + ni=1 σ(s, xi ) ξ∗ = ξ − n
P
The Bayes estimator (that coincides with the predictive dist.) is given by
( n )
X
fn (y ) = E[f (y ) | x1 , . . . , xn ] ∝ W0 (y ) exp σ(y , xi ) C(ξ − n − 1, µ∗y )
i=1
where µ∗y = µ∗ (s) + σ(s, y )

→ The dependence enters through the covariance σ
→ Differentiability of σ ⇒ controls the smoothness of f
A choice of covariance function can be chosen to be

√ b √
∗ a 2 ντ 2 ντ
σ(x, y ) = σ (τ ) = ν−1
Kν , a, κ, ν > 0
Γ(ν)2 κ κ
where τ := |x − y | and Kν denotes the modified Bessel function of order

ν.

√ b √
∗ a 2 ντ 2 ντ
σ(x, y ) = σ (τ ) = ν−1
Kν , a, κ, ν > 0
Γ(ν)2 κ κ

ν.
⇒ Gaussian process with (ν − 1) differentiable paths


√ b √
∗ a 2 ντ 2 ντ
σ(x, y ) = σ (τ ) = ν−1
Kν , a, κ, ν > 0
Γ(ν)2 κ κ

ν.
⇒ Gaussian process with (ν − 1) differentiable paths

Continuous realizations: Q(P : P << Leb) = 1
Issue: Negative moments, C(K , µ), K < 0 in general not easy to
deal with.
Models via normalization of subordinators
Fix X = R. Let {ξt } a Lévy subordinator characterized by its Laplace

Transform
Lξt (λ) = E(e −λξt ) = e −tψ(λ)
with characteristic exponent given by
Z
ψ(λ) = (1 − e −y λ )ν(dy )
R+
with Lévy measure, ν, satisfying ν(R+ ) = ∞.

We term a RPM P, N-prior(α, ν) if
ξα(A)
P(A) := , A∈B
ξθ
with α > 0 a finite measure on (R, B) y θ := α(R)

If {Xi }∞
i=1 are exchangeable with with directing RPM, N-Prior(α, ν),
hence
α(B)
P(X2 ∈ B | X1 ) = [1 − Iθ ] + δX1 (B) Iθ
θ
where Z Z
Iθ := θ u e −θ ψ(u) v 2 e −uv ν(dv )du
R+ R+
y Z
ψ(u) = (1 − e −xu )ν(dx)
R+
→ Iθ can be interpreted as the probability of a tie X2 = X1


The corresponding EPPF is given by
k
θk
Z
(n)
Y
Πk (n1 , . . . , nk ) = u n−1 e −θψ(u) µnj (u)du
Γ(n) R+ j=1
where Z
µn (u) := v n e −uv ν(dv )
R+
Hence
k
(n) (n) α(B) 1 X n
P(Xn+1 ∈ B | X ) =w0 + wj δXj∗ (B)
θ n
j=1
where {Xj∗ }kj=1 denote the k distinct values in X(n) := (X1 , . . . , Xn ) and
(n+1)
(n) θ Πk+1 (n1 , . . . , nk , 1)
w0 = (n)
n Πk (n1 , . . . , nk )
and for j = 1, . . . , k
(n+1)
(n) Πk (n1 , . . . , nj + 1, nk )
wj = (n)
Πk (n1 , . . . , nk )
(n+1)
(n) θ Πk+1 (n1 , . . . , nk , 1)
w0 = (n)
n Πk (n1 , . . . , nk )
and for j = 1, . . . , k
(n+1)
(n) Πk (n1 , . . . , nj + 1, nk )
wj = (n)
Πk (n1 , . . . , nk )
• A generalized Pólya urn scheme
The Dirichlet process
e −x
ν(dx) = x dx
Z ∞ Z ∞
Iθ = θ ue −θψ(u) v 2 e −uv ν(dv )du
0 0
Z ∞ Z ∞
−θ log(1+u)
= θ ue ve −(1+u)v dv du
0 0
Z ∞
1
= θ u(1 + u)−(θ+2) du =
0 θ+1
Z Z ∞
n −uv Γ(n)
µn (u) = v e ν(dv ) = v n−1 e −(u+1)v dv =
R+ 0 (1 + u)n
Hence the expression for the EPPF is given by

k ∞
θk Y
Z
(n)
Πk (n1 , . . . , nk ) = (ni − 1)! u n−1 (u + 1)−(θ+n) du
Γ(n) 0
i=1
k
θk Y Γ(θ)Γ(n)
= (ni − 1)!
Γ(n) Γ(n + θ)
i=1
k
θk Y
= (ni − 1)!
(θ)n
i=1
The corresponding weights in the generalized Pólya urn scheme are
(n) θ Γ(n + 1)Γ(n + θ) θ

w0 = =
n Γ(n + θ + 1)Γ(n) θ+n
and
(n) Γ(n + 1)Γ(n + θ) n
wj = nj = nj
Γ(n + θ + 1)Γ(n) n+θ
for j = 1. . . . , k. Finally the predictive distribution can be simplified as
k
θ α(B) X nj
P(Xn+1 ∈ B | X(n) ) = + δX ∗ (B)
θ+n θ θ+n j
j=1
Normalized stable process
ν(dx) = c x −(1+γ) dx, γ ∈ (0, 1)
Z ∞ Z ∞
−θψ(u)
Iθ = θ ue v 2 e −uv ν (dv ) du
0 0
Z ∞ cΓ(1−γ)u γ
Z ∞
−θ
= θ ue γ v 1−γ e −uv dv du
0 0
Z ∞ cΓ(1−γ)u γ
−θ
= θ c Γ (2 − γ) u γ−1 e γ du
0
= 1−γ
Normalized stable process
Z ∞
µn (u) = v n e −uv ν (dv )
0
Z ∞
= c v n−1−γ e −uv dv
0
= cu −n+γ Γ (n − γ) .
Following a similar procedure we see that
k
(n) γ k−1 Γ(k) Y
Πk (n1 , . . . , nk ) = (1 − γ)ni −1
Γ(n)
i=1
Resulting in
k
k 1X
P Xn+1 ∈ B | X(n) = γ P0 + (nj − γ) δXj∗ (B) .
n n
j=1
Gibbs-type priors Gσ
Let (Xi )i≥1 a set of exchangeable observations, we say that it is governed
by a Gibbs-type prior (Gσ ) if its predictive distribution can be expressed as
k
(n) Vn+1,k+1 Vn+1,k X
P Xn+1 ∈ A X = P0 (A) + (nj − σ) δXj∗ (A),
Vn,k Vn,k
j=1
for σ ∈ [0, 1) and where {Vn,k }n≥1,1≤k≤n is a set of weights s.t.

Vn,k = (n − kσ)Vn+1,k + Vn+1,k+1 , V1,1 = 1.
Gibbs-type priors Gσ
Let (Xi )i≥1 a set of exchangeable observations, we say that it is governed
by a Gibbs-type prior (Gσ ) if its predictive distribution can be expressed as
k
(n) Vn+1,k+1 Vn+1,k X
P Xn+1 ∈ A X = P0 (A) + (nj − σ) δXj∗ (A),
Vn,k Vn,k
j=1
for σ ∈ [0, 1) and where {Vn,k }n≥1,1≤k≤n is a set of weights s.t.

Vn,k = (n − kσ)Vn+1,k + Vn+1,k+1 , V1,1 = 1.
Its EPPF’s are of the form (Gnedin and Pitman (2005))

k
(n)
Y
Πk (n1 , . . . , nk ) = Vn,k (1 − σ)nj −1
j=1
→ A product partition model with cohesion c(nj ) = (1 − σ)nj −1

(See Quintana and Iglesias (2003) )
Gibbs-type priors Gσ : Poisson-Dirichlet process case
The two-parameter Poisson-Dirichlet process (PD(σ, θ)) [Pitman 1996]

Qk−1
i=1 (θ+iσ)
can be seen as Gσ with Vn,k = (θ+1)n−1 , θ > −σ

Qk−1
i=1 (θ+iσ)
Predictive distribution
k
(n) θ + kσ 1 X
P Xn+1 ∈ A X = P0 (A) + (nj − σ)δXj (A).
θ+n θ+n
j=1

Qk−1
i=1 (θ+iσ)
k
(n) θ + kσ 1 X
P Xn+1 ∈ A X = P0 (A) + (nj − σ)δXj (A).
θ+n θ+n
j=1
P ∼ PD(σ, θ) also has a stick-breaking construction but with sticks

iid
Vi ∼ Be(1 − σ, θ + iσ). See Ishwaran and James (2001).

Qk−1
i=1 (θ+iσ)
k
(n) θ + kσ 1 X
P Xn+1 ∈ A X = P0 (A) + (nj − σ)δXj (A).
θ+n θ+n
j=1

iid
→ If σ ↓ 0, P ∼ DθP0

Qk−1
i=1 (θ+iσ)
k
(n) θ + kσ 1 X
P Xn+1 ∈ A X = P0 (A) + (nj − σ)δXj (A).
θ+n θ+n
j=1

iid
→ If σ ↓ 0, P ∼ DθP0
See book by Feng (2010) for more on PD processes
Gibbs-type priors Gσ : Generalized gamma prior case
The generalized gamma process prior, P ∼ GG(β, σ), can be seen as Gσ

with
n−1
σ k−1 e β X n − 1

i i/σ i
Vn,k = (−1) β Γ k − ; β .
Γ(n) i σ
i=0
See Lijoi, M. and Prünster (2007).

The generalized gamma process prior, P ∼ GG(β, σ), can be seen as Gσ

with
n−1
σ k−1 e β X n − 1

i i/σ i
Vn,k = (−1) β Γ k − ; β .
Γ(n) i σ
i=0
See Lijoi, M. and Prünster (2007).

• Predictive distributions
k
(n) (n)
X
(n)
P[Xn+1 ∈ A | X ]= w0 P0 (A) + w1 (nj − σ) δXj∗ (A),
j=1
Pn n i i/σ Pn n i i/σ
i=0 ( i )(−1) β Γ(k+1− σi ; β ) i=0 ( i )(−1) β Γ(k− σi ; β )
with w0(n) = σ
n Pn−1 (n−1)(−1)i β i/σ Γ(k− i ; β )
(n)
and w1 = Pn−1 n−1
(−1)i β i/σ Γ(k− σi ; β )
i=0 ( i )
i=0 i σ
n
P ∼ GG(β, σ) can be also constructed as a normalized subordinator

with Lévy measure
exp(−τ x) x −(1+σ)
ν(dx) = dx [with β := τ σ /σ]
Γ(1 − σ)
with σ ∈ (0, 1) y τ ≥ 0, e.g. a generalized gamma process (Brix 99’)


with Lévy measure
exp(−τ x) x −(1+σ)
Γ(1 − σ)

Lijoi, Prünster and Walker (2008) prove that GG(β, σ) is the
intersection between the normalized random measures with indep.
inc. (NRMI) and Gibbs-type priors. (NRMI) ∩ Gσ = GG(β, σ)

with Lévy measure
exp(−τ x) x −(1+σ)
Γ(1 − σ)

If β = 0 ⇒ normalized σ-stable process (Kingman 75’)

with Lévy measure
exp(−τ x) x −(1+σ)
Γ(1 − σ)

If σ = 12 ⇒ normalized inverse Gaussian (NIG) prior
(Lijoi, M. and Prünster (2005)) ← also available through fdds!

with Lévy measure
exp(−τ x) x −(1+σ)
Γ(1 − σ)

If σ = 12 ⇒ normalized inverse Gaussian (NIG) prior
(Lijoi, M. and Prünster (2005)) ← also available through fdds!
If σ ↓ 0 ⇒ Dβ
P[Kn = k] derived from generalized gamma process
n−1
β C (n, k, σ) X n − 1

e i
P[Kn = k] = (−1)i β i/σ Γ k − ; β ,
σ Γ(n) i σ
i=0
n−1
β C (n, k, σ) X n − 1

e i
P[Kn = k] = (−1)i β i/σ Γ k − ; β ,
σ Γ(n) i σ
i=0
“ Sampling formulas ”
. β = 0 ⇒ Kingman , 1975
n−1
β C (n, k, σ) X n − 1

e i
P[Kn = k] = (−1)i β i/σ Γ k − ; β ,
σ Γ(n) i σ
i=0
. β = 0 ⇒ Kingman , 1975
. σ = 1/2 ⇒ LMP, 2005
n−1
β C (n, k, σ) X n − 1

e i
P[Kn = k] = (−1)i β i/σ Γ k − ; β ,
σ Γ(n) i σ
i=0
. β = 0 ⇒ Kingman , 1975
. σ = 1/2 ⇒ LMP, 2005
. σ → 0 ⇒ Ewens, 1972
n−1
β C (n, k, σ) X n − 1

e i
P[Kn = k] = (−1)i β i/σ Γ k − ; β ,
σ Γ(n) i σ
i=0
β= 1
GG( β,0.25 )
0.175 β= 500
β= 2 n = 50
0.150 β= 5
β= 300
“ Sampling formulas ” β= 10
0.125
β= 30 β= 50 β= 70 β= 100
. β = 0 ⇒ Kingman , 1975
0.100
. σ = 1/2 ⇒ LMP, 2005
0.075
. σ → 0 ⇒ Ewens, 1972
0.050
0.025
0 5 10 15 20 25 30 35 40 45 50
β controls the location (β ↑ ⇒ E(Kn ) ↑)

n−1
β C (n, k, σ) X n − 1

e i
P[Kn = k] = (−1)i β i/σ Γ k − ; β ,
σ Γ(n) i σ
i=0
σ= 0.1
0.35
n = 50
0.30
GG ( 1 , σ )
“ Sampling formulas ” 0.25
σ= 0.2
. β = 0 ⇒ Kingman , 1975
0.20
. σ = 1/2 ⇒ LMP, 2005

0.15
σ= 0.3
. σ → 0 ⇒ Ewens, 1972
σ= 0.4
0.10
σ= 0.5
σ= 0.6
“ Flexible clustering structure ” 0.05
σ= 0.7 σ= 0.8
0 5 10 15 20 25 30 35 40 45 50
σ controls the flatness (σ ↑ ⇒ flatness ↑)

References
Antoniak, C.E. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Statist.,
2, 1152–1174.
Banfield, J.D. and Raftery, A.E. (1993) Model-based Gaussian and non-Gaussian clustering. Biometrics, 7, 1-34.
Barrientos, A.F., Jara, A. and Quintana (2012) On the Support of MacEachern?s Dependent Dirichlet Processes and
Extensions. Bayesian Analysis, 49, 803-821.
Blackwell, D. and MacQueen (1973). Discreteness of Ferguson selections. Ann. Statist. 1, 356–358.
Brix, A. (1999). Generalized gamma measures and shot-noise Cox processes. Adv. in Appl. Probab., 31, 929–953.
Caron, F., Davy, M., Doucet, A., Duflos, E. and Vanheeghe, P (2006). Bayesian inference for dynamic models with
Dirichlet process mixtures. In 9th IEEE International Conference on Information Fusion, Florence, Italy.
Dahl, D.B. (2009). Modal Clustering in a Class of Product Partition Models. Bayesian Analysis., 4, 243–264.
Doksum, K. (1974). Tailfree and neutral random probabilities and their posterior distributions. Ann. Probab., 2, 183–201.
Dunson, D.B. and Park, J.H. (2008). Kernel stick-breaking processes. Biometrika, 95, 307–323.
Escobar, M.D. (1988). Estimating the means of several normal populations by nonparametric estimation of the distribution
of the means. Unpublished Ph.D. dissertation, Department of Statistics, Yale University
Escobar, M.D. (1994). Estimating normal means with a Dirichlet process prior. J. Am. Stat. Assoc., 89, 268?-277
Escobar, M.D. and West, M. (1995). Bayesian density estimation and inference using mixtures. J. Amer. Stat. Assoc., 90,
577–588.
Ewens, W. J. (1972). The sampling theory of selectively neutral alleles. Theor. Popul. Biol., 3, 87-112.
Favaro, S., Lijoi, A., Mena, R.H. and Prünster, I. (2009). Hierarchical mixture modeling with normalized inverse
Gaussian Priors J. Roy. Stat. Soc., Series B, 71, 993-1008.
Feigin, P.D. and Tweedie, R.L. (1989). Linear functionals and Markov chains associated with Dirichlet processes. Math.
Proc. Cambridge Philos. Soc., 105, 579–585.
Feng, S. (2010). The Poisson-Dirichlet Distribution and Related Topics: Models and Asymptotic Behaviors. Springer.
References
Ferguson, T.S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1, 209–230.
Ferguson, T.S. (1974). Prior distributions on spaces of probability measures. Ann. Statist. 2, 615–629.
Ferguson, T.S. (1974). Prior distributions on spaces of probability measures. Ann. Statist. 2, 615–629.
Fuentes-Garcı́a, R., Mena, R. H. and Walker, S. G. (2009). A nonparametric dependent process for Bayesian
regression. Statistics and Probability Letters. 79, 1112–1119.
Fuentes-Garcı́a, R., Mena, R. H. and Walker, S. G. (2010). A new Bayesian nonparametric mixture model.
Communications in Statistics-Simulation and Computation. 39, 669–682.
Fuentes-Garcı́a, R., Mena, R. H. and Walker, S. G. (2010). A probability for classification based on the mixture of
Dirichlet process model. Journal of Classification. 27, 389–403.
Gnedin, A. and Pitman, J. (2005). Exchangeable Gibbs partitions and Stirling triangles. Zap. Nauchn. Sem. S.-Peterburg.
Otdel. Mat. Inst. Steklov. (POMI) 325, 83–102.
Griffin, J.E. and Steel, M.F.J. (2006). Order-based dependent Dirichlet processes. J. Amer. Statist. Asoc., 101, 179–194.
Hjort, N.L. (2000). Bayesian analysis for a generalized Dirichlet process prior. Statist. Research Report, 7, University of Oslo.
Ishwaran, H. and James, L.F. (2001). Gibbs sampling methods for stick-breaking priors. J. Amer. Stat. Assoc., 96,
161–173.
James, L.F. (2007). Neutral-to-the-right species sampling mixture models. In Advances in statistical modeling and inference,
425–439, Ser. Biostat. 3 World Sci. Publ., Hackensack, NJ.
Kalli, M., Griffin, J. and Walker, S.G. (2010) Slice Sampling Mixture Models. Statistics and Computing. In press.
Kingman, J.F.C. (1967). Completely random measures. Pacific J. Math. 21, 59–78.
Kingman, J.F.C. (1975). Random discrete distributions. J. Roy. Statist. Soc. Ser. B, 37, 1–22.
Kingman, J.F.C. (1978). The representation of partition structures. Journal of the London Mathematical Society, 18,
374–380.
Kingman, J.F.C. (1993). Poisson processes Oxford University Press.
References
Lenk, P. J. (1988). The logistic normal distribution for Bayesian nonparametric, predictive densities. J. Amer. Statist. Asoc.,
83, 509–516.
Lijoi, A., Mena, R.H. and Prünster, I. (2007). Controlling the reinforcement in Bayesian nonparametric mixture models.
J. R. Statist. Soc. B, 69, 715–740.
Lijoi, A., Mena, R.H. and Prünster, I. (2007). Bayesian nonparametric estimation of the probability of discovering new
species. Biometrika, 94, 769–786.
Lijoi, A., Mena, R.H. and Prünster, I. (2005). Bayesian nonparametric estimation of the probability of discovering new
species. JASA, 100, 1278–1291.
Lijoi, A. and Prünster, I. (2010). Models Beyond the Dirichlet process. In Bayesian nonparametrics. (Eds. Hjort, N.,
Holmes, C., Müller, P. and Walker, S.G.). Cambridge Univ. Press.
Lijoi, A., Prünster, I. and Walker, S.G. (2008). Bayesian nonparametric estimators derived from conditional Gibbs
structures. Ann. Appl. Probab., 18, 1519-1547.
Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates: I Density estimates. Annals of Statistics, 12, 351–357.
MacEachern, S.N. (1994). Estimating normal means with a conjugate style Dirichlet process prior. Commun. Statist.
Simulation Comp., 23, 727–741.
MacEachern, S.N. (1998). Computational methods for mixture of Dirichlet process models. In Practical nonparametric and
semiparametric Bayesian statistics (eds D. Dey, P. Müller and D. Sinha). New York: Springer, 23–43.
MacEachern, S.N. (1999). Dependent nonparametric processes. In ASA Proceedings of the Section on Bayesian Statistical
Science. Alexandria: American Statistical Association, 50-55.
Petrone, S. (1999). Random Bernstein polynomials. Scandinavian Journal of Statistics, 26, 373–393.
Pitman, J. (1995). Exchangeable and partially exchangeable random partitions. Probab. Theory Related Fields 102, 145–158.
Pitman, J. (1996). Some developments of the Blackwell-MacQueen urn scheme. In Statistics, Probability and Game Theory.
Papers in honor of David Blackwell (Eds. Ferguson, T.S., et al.). Lecture Notes, Monograph Series, 30, 245–267. Institute of
Mathematical Statistics, Hayward.
References
Pitman, J. (2003). Poisson–Kingman partitions. In Science and Statistics: A Festschrift for Terry Speed (ed D.R. Goldstein),
IMS Lecture Notes Monograph Series, vol. 40, 1–35.
Pitman, J. (2005). Combinatorial Stochastic Processes. LNM1875 Springer.
Quintana, F. A. and Iglesias, P. L. (2003). Bayesian clustering and product partition models. J. R. Statist. Soc. B, 65,
557?-574.
Regazzini, E., Lijoi, A. and Prünster, I. (2003). Distributional results for means of random measures with independent
increments. Ann. Statist., 31, 560–585.
Rodriguez, A. and Ter Host, E. (2008). Bayesian dynamic density estimation. Bayesian Analysis., 3, 339–366.
Ruggiero, M. and Walker, S.G. (2009a). Bayesian nonparametric construction of the Fleming-Viot process with fertility
selection. Statistica Sinica, 19, 707–720.
Ruggiero, M. and Walker, S.G. (2009b). Countable representation for infinite-dimensional difusions derived from the two
parameter Poisson Dirichlet process. Electronic Communications in Probability, 14, 501–517.
Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statist. Sinica 4, 639–650.
Walker, S.G. (2007). Sampling the Dirichlet mixture model with slices. Communications in Statistics, 36, 45–54.
Walker, S.G., Hatjispyros, S.J. and Nicoleris, T. (2007). A Fleming-Viot process and Bayesian nonparametrics.
Annals of Applied Probability, 17, 67–80.

Alternative

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Alternative

Uploaded by

Copyright:

Available Formats

Alternative RPM’s to DP

Bayesian nonparametric methods and random

Generalizations / alternatives for Q

Generalizations / alternatives for Q

Generalizations / alternatives for Q

→ An excellent recent review is given by Lijoi and Prünster (2010),

Normalized log-Gaussian process

• Normalized log-Gaussian process

{Zt } ∼ PG(µ, σ) ⇒ paths can be regarded as continuous densities

Define W := {Wt }t∈X con Wt = exp{Zt }, the log-Gaussian process

Normalized log-Gaussian process

• Normalized log-Gaussian process

{Zt } ∼ PG(µ, σ) ⇒ paths can be regarded as continuous densities

Define W := {Wt }t∈X con Wt = exp{Zt }, the log-Gaussian process

Generalization by Lenk 1988

with the distribution of W ∗ given by

where ξ = 1, 2, . . ., Λ ∼ W ∼ LNX (µ, σ) and

Generalization by Lenk 1988

Let f ∼ LNSX (µ, σ, ξ) and x ∈ XK , hence

Generalization by Lenk 1988

Generalization by Lenk 1988

Generalization by Lenk 1988

where µ∗y = µ∗ (s) + σ(s, y )

Generalization by Lenk 1988

where µ∗y = µ∗ (s) + σ(s, y )

Generalization by Lenk 1988

A choice of covariance function can be chosen to be

where τ := |x − y | and Kν denotes the modified Bessel function of order

Generalization by Lenk 1988

A choice of covariance function can be chosen to be

where τ := |x − y | and Kν denotes the modified Bessel function of order

⇒ Gaussian process with (ν − 1) differentiable paths

Generalization by Lenk 1988

A choice of covariance function can be chosen to be

where τ := |x − y | and Kν denotes the modified Bessel function of order

⇒ Gaussian process with (ν − 1) differentiable paths

Models via normalization of subordinators

Fix X = R. Let {ξt } a Lévy subordinator characterized by its Laplace

with Lévy measure, ν, satisfying ν(R+ ) = ∞.

with α > 0 a finite measure on (R, B) y θ := α(R)

Models via normalization of subordinators

→ Iθ can be interpreted as the probability of a tie X2 = X1

Models via normalization of subordinators

Models via normalization of subordinators

Models via normalization of subordinators

The Dirichlet process

The Dirichlet process

Hence the expression for the EPPF is given by

The Dirichlet process

The corresponding weights in the generalized Pólya urn scheme are

(n) θ Γ(n + 1)Γ(n + θ) θ

Normalized stable process

ν(dx) = c x −(1+γ) dx, γ ∈ (0, 1)

Normalized stable process

for σ ∈ [0, 1) and where {Vn,k }n≥1,1≤k≤n is a set of weights s.t.

for σ ∈ [0, 1) and where {Vn,k }n≥1,1≤k≤n is a set of weights s.t.

Its EPPF’s are of the form (Gnedin and Pitman (2005))

→ A product partition model with cohesion c(nj ) = (1 − σ)nj −1

Gibbs-type priors Gσ : Poisson-Dirichlet process case

The two-parameter Poisson-Dirichlet process (PD(σ, θ)) [Pitman 1996]

Gibbs-type priors Gσ : Poisson-Dirichlet process case

The two-parameter Poisson-Dirichlet process (PD(σ, θ)) [Pitman 1996]

Gibbs-type priors Gσ : Poisson-Dirichlet process case