STAT6201 Lecture Note 3 Fall 2021

3 Common Families of Distributions

3.1 Introduction X
3.2 Discrete Distributions X
3.3 Continuous Distributions
3.3.1 Uniform Distribution
• pdf: X ∼ Unif[a, b] if
1
f (x | a, b) = , a ≤ x ≤ b.
b−a
1
Pk
• Moments: E(X) = (b + a)/2, E(X k ) = k+1 i=0 ai bk−i , Var(X) = (b − a)2 /12.
etb −eta
• MGF: M (t) = E(etX ) = t(b−a)

3.3.2 Exponential Distribution

• pdf: X ∼ Exp(β) if
1 −x/β
f (x | β) = e , x > 0.
β

• cdf: F (x) = 1 − e−x/β .

• Moments: E(X) =?, Var(X) =?, E(X k ) =?
• MGF: M (t) = (1 − tβ)−1 .
• Memoryless Property:

P (X > a + b | X > a) = P (X > b), a > 0, b > 0.

3.3.3 Gamma Distribution

• Gamma function: For any α > 0,
Z ∞
Γ(α) = e−x xα−1 dx.
0
R∞
– Γ(1) = e−x dx = 1.
0
√
– Γ(1/2) = π.
– If α > 0, then by integration by parts,
∞
Z ∞
Γ(α + 1) = xα (−e−x ) 0 − (−e−x )αxα−1 dx = αΓ(α).

0

– In particular, for any positive integer n ≥ 1,

Γ(n) = (n − 1)Γ(n − 1) = · · · = (n − 1)(n − 2) · · · 1Γ(1) = (n − 1)!.

• Gamma distribution: X ∼ Gamma(α, β) if the pdf of X is

xα−1 e−x/β
f (x) = , x > 0,
β α Γ(α)
where
STAT6201 Lecture Note 3 Page 2 of 3

– α > 0: shape parameter, which most influences the peakedness of the distribution.
– β > 0: scale parameter, which influences the spread of the distribution.

alpha=1, beta=2 alpha=2, scale=2

0.4

0.15
0.3

0.10
Density

Density
0.2

0.05
0.1

0.00
0.0

0 5 10 15 0 5 10 15 20

x x

alpha=5, scale=2 alpha=10, scale=2

0.06
0.08

0.04
Density

Density
0.04

0.02
0.00

0.00

0 10 20 30 40 10 20 30 40 50

x x

alpha=5, scale=0.1 alpha=5, scale=1

2.0

0.00 0.05 0.10 0.15

1.5
Density

Density
1.0
0.5
0.0

0.0 0.5 1.0 1.5 0 5 10 15 20

x x

alpha=5, scale=3 alpha=5, scale=10

0.020
0.06
0.04
Density

Density

0.010
0.02

0.000
0.00

0 10 20 30 40 50 0 50 100 150

x x

– When α increases, distribution becomes more symmetric.

– When β increases, distribution is stretched along the x-axis.
Γ(k+α) k
• Moments: E(X) = αβ, E(X k ) = Γ(α) β , Var(X) = αβ 2 .

• MGF: M (t) = E(etX ) = (1 − βt)−α .

• Properties:
– aX ∼ Gamma(α, aβ).
– X/β ∼ Gamma (α, 1).
– Gamma(1, β)=Exp(β).
– Gamma(p/2, 2)=χ2p , the chi-squared distribution with p degree of freedom.

Cont.
STAT6201 Lecture Note 3 Page 3 of 3

– If X ∼ Gamma(1, β), i.e., Exp(β), then Y = X 1/γ ∼ Weibull(β, γ) (γ > 0) with pdf
γ γ−1 −yγ /β
f (y | β, γ) = y e , y > 0.
β
The Weibull distribution plays a very important role in the analysis of failure time data. See
Textbook Exercises 3.25 and 3.26.
– Gamma-Poisson relationship: Textbook Example 3.3.1, pp.100.

3.3.4 Normal Distribution/Gaussian Distribution

• pdf: X ∼ N(µ, σ 2 ) if
1 2 2
f (x | µ, σ) = √ e−(x−µ) /2σ , −∞ < x < ∞.
σ 2π
• Moments: E(X) = µ, Var(X) = σ 2 .
2 2
• MGF: M (t) = eµt+σ t /2
.
• Properties:
– Linear transformation: If X ∼ N (µ, σ 2 ), then Y = aX + b ∼ N (aµ + b, a2 σ 2 ).
– Standardization (i.e., z-score transformation):
X −µ
Z= ∼ N (0, 1) (standard normal distribution)
σ
∗ The pdf of Z is symmetric over 0.
∗ E(Z 2k+1 ) = 0, k integer.
∗ E(Z 2k ) = (2k − 1)(2k − 3) · · · 3 · 1, k integer.
∗ Skewness = 0; Kurtosis = 0.
Pp
∗ If Z1 , . . . , Zp are i.i.d. N (0, 1), then k=1 Zk2 ∼ χ2p .
– Lognormal distribution: X follows a lognormal distribution if log X is normally distributed.
• cdf: not expressible in terms of elementary functions. If we denote the cdf of Z by
Z z
1 2
Φ(z) = √ e−t /2 dt,
−∞ 2π
and create a table for Φ, then since for X ∼ N (µ, σ 2 ),
FX (x) = P (X ≤ x) = P {Z ≤ (x − µ)/σ} = Φ{(x − µ)/σ},
the cdf of X can be related to the table for Φ.
Φ(0) = 0.5, Φ(−∞) = 0, Φ(∞) = 1, Φ(−z) = 1 − Φ(z). Φ(1.96) ≈ 0.975, Φ(−4) ≈ 0, Φ(4) ≈ 1.
• Normal approximation to binomial distribution: Suppose X ∼ binomial(n, p). If n is large
(at least 30 or 40), and p is not too close to 0 or too close to 1, then X approximately follows
N (np, np(1 − p)).

Acknowledgement
The lecture notes of this course are based on the textbook and Prof. Huixia Judy Wang’s lecture slides. The
instructor thanks Prof. Wang for kindly sharing them.

The End.