Moment Generating Functions

Continuous Distributions
The Uniform distribution from a to b

f x

1
ba
0

a xb
otherwise

0.4

f x

0.3
0.2

ba

0.1

0
0

10

15

The Normal distribution

(mean , standard deviation )

1
f x
e
2

x 2
2 2

The Exponential distribution

e x

f x

x0
x0

0.2

0.1

0
-2

10

Weibull distribution with parameters and.

Thus

and

F x 1 e

f x F x x 1e

x0

The Weibull density, f(x)

0.7

( = 0.9, = 2)

0.6
0.5

( = 0.7, = 2)

0.4

( = 0.5, = 2)

0.3
0.2
0.1
0
0

The Gamma distribution

Let the continuous random variable X have
density function:

1 x
x e

f x

x0
x0

Then X is said to have a Gamma distribution

with parameters and .

Expectation of functions of
Random Variables

X is discrete

E g X g x p x g xi p xi
x

X is continuous

E g X

g x f x dx

Moments of Random Variables

The kth moment of X.

k E X k
k
x
p x

if X is discrete

x k f x dx if X is continuous

the kth central moment of X

k

E X

0
k

x p x

if X is discrete

x f x dx
k

if X is continuous

where = 1 = E(X) = the first moment of X .

Rules for expectation

Rules:
1. E c c where c is a constant
2. E aX b aE X b where a, b are constants
2

3. var X E X

E X 2 E X
0
2

4. var aX b a 2 var X

2 12

Moment generating functions

Moment Generating function of a R.V. X

tx
e
p x

mX t E e

tX

if X is discrete

etx f x dx if X is continuous

Examples
1. The Binomial distribution (parameters p, n)
mX t E etX etx p x
x

e
x 0
n

x 0

n
n x
x
p 1 p
x
n
x
n
n x n x
n

x
t
e p 1 p a b
x
x
0 x

tx

a b e p 1 p
n

2. The Poisson distribution (parameter )

x
p x
e
x!

x 0,1, 2,K

The moment generating function of X , mX(t) is:

mX t E etX etx p x
x

e
x 0

et 1

e
t

x!

etx
e
x!
x 0
n

e ee

x
u
using eu
x 0 x !

3. The Exponential distribution (parameter )

e x x 0
f x
x0
0
The moment generating function of X , mX(t) is:
mX t E etX

e
0

t x

tx
tx
x
e
f
x
dx

e
dx

t x

e
dx

t
t
undefined t

4. The Standard Normal distribution ( = 0, = 1)

1 x22
f x
e
2

The moment generating function of X , mX(t) is:

mX t E etX

tx
e
f x dx

tx

1 x22
e dx
2
1 x2 22 tx
e
dx
2

We will now use the fact that

1
2 a e dx 1 for all a 0, b
x b 2
2 a2

We have
completed
the square

t2
x 2 2 tx t 2
1 x2 22 tx
1

e
dx e 2
e 2 dx
2
2

This is 1

t2
1 x2t 2
e
dx e 2
2

t2
2

mX t

4. The Gamma distribution (parameters , )

1 x
x e

f x

x0
x0

The moment generating function of X , mX(t) is:

mX t E etX

tx
e
f x dx

etx
x 1e x dx

0

1 t x

x e
dx

0

We use the fact

b a a 1 bx
0 a x e dx 1 for all a 0, b 0

1 t x
mX t
x e
dx

0

dx

Equal to 1

1 t x

t 0

Properties of
Moment Generating Functions

1.

mX(0) = 1

mX t E etX , hence mX 0 E e0X E 1 1

Note: the moment generating functions of the following

distributions satisfy the property mX(0) = 1
i)

Binomial Dist'n

mX t e p 1 p

ii) Poisson Dist'n mX t e

et 1

iii) Exponential Dist'n mX t

t2 t
iv) Std Normal Dist'n mX t e 2

v) Gamma Dist'n mX t

2.

k k
2 2 3 3
mX t 1 1t t t K t K
2!
3!
k!

We use the expansion of the exponential function:

2
3
k
u
u
u
eu 1 u K K
2! 3!
k!

mX t E etX

t2 2 t3 3
tk k
E 1 tX X X K X K
2!
3!
k!

k
2
3
t
t
t
2
3
1 tE X E X E X K E X k K
2!
3!
k!
t2
t3
tk
1 t 1 2 3 K k K
2!
3!
k!

3.

k
d
k
mX 0 k mX t k
dt
t 0

Now
k k
2 2 3 3
mX t 1 1t t t K t K
2!
3!
k!
3 2
k k 1
2
mX t 1
2t 3t K
kt K
2!
3!
k!
3 2
k
1 2t t K
t k 1 K
2!
k 1 !
and mX 0 1
k
4
mX t 2 3t t K
t k 2 K
2!
k 2 !
and mX 0 2
continuing we find mX 0 k
k

Property 3 is very useful in determining the moments of a

random variable X.
Examples
n
t
i) Binomial Dist'n mX t e p 1 p

m 0 n e

p 1 p pe np
m t np n 1 e p 1 p e p e e p 1 p

npe e p 1 p n 1 e p e p 1 p
npe e p 1 p ne p 1 p
mX t n e p 1 p
t

n 1

pet

n 1

n2

n2

n2

np np 1 p np np q n 2 p 2 npq 2

n 1

e t

ii) Poisson Dist'n mX t e

et 1

mX t e

e t e e 1 t

e 1 t
e 1 t
t
2 e 1 2 t

mX t e
e
e 1 e
et 1

t
2 e 1 2 t

mX t e

e 2 e

e 3 e

t
2 e 1 2 t

t
3 e 1 3t

et 1 t

et 1

et 1 t

t
2 e 1 2 t

3 e

et 1 t

To find the moments we set t = 0.

1 mX 0 e

e0 1 0

0
2 e 1 0

2 mX 0 e

e0 1 0

3 mX 0 3e0 3 2 e0t e0 3 3 2


iii) Exponential Dist'n mX t

d t
d
mX t

dt t
dt
1 t
mX t 2 t

mX t 2 3 t

1 t

mX t k ! t

1 2 3 t

mX t 2 3 4 t
k

1 2 t

k 1

1 4! t

Thus
1
1 mX 0

2
3
2 mX 0 2 2

M
k mX 0 k !
k

k 1

k!
k

The moments for the exponential distribution can be calculated in

an alternative way. This is note by expanding mX(t) in powers of t
and equating the coefficients of tk to the coefficients in:

k k
2 2 3 3
mX t 1 1t t t K t K
2!
3!
k!

1
1
mX t

1 u u2 u3 L
t 1 t
1 u

t t2 t3
1 2 3 L

Equating the coefficients of tk we get:

k
1
k!
k or k k
k!

The moments for the standard normal distribution

mX t e

t2
2

We use the expansion of eu.

k
2
3
k

u
u
u
u
eu 1 u L L
2! 3!
k!
k 0 k !
t2
2

mX t e 1

t2
2

t2
2

t2
2

t2
2

2!
3!
k!
1 4
1 6
1 2k
1 2
1 2 t 2 t 3 t L k t L
2 2!
2 3!
2 k!

We now equate the coefficients tk in:

k k
2 k 2 k
2 2
mX t 1 1t t K t K
t K
2!
k!
2k !

If k is odd: k = 0.
For even 2k:

or
Thus

2 k
1
k
2k ! 2 k !

2 k

2k !

2 k!

2!
4!
1 0, 2 1, 3 0, 4 2
3
2
2 2!

Summary
Moments
Moment generating functions

Moments of Random Variables

k E X k

k
x
p x

if X is discrete

x k f x dx if X is continuous

The moment generating function

tx
e
p x

mX t E e

tX

if X is discrete

etx f x dx if X is continuous

Examples
1. The Binomial distribution (parameters p, n)
n
n x
x
p x p 1 p
x

mX t e p 1 p
t

x 0,1, 2,K , n

e p q
n

2. The Poisson distribution (parameter )

x
p x
e
x 0,1, 2,K
x!
et 1
mX t e

3. The Exponential distribution (parameter )

e x x 0
f x
x0
0

mX t
t
undefined t
4. The Standard Normal distribution ( = 0, = 1)
1 x22
f x
e
2

mX t e

t2
2

5. The Gamma distribution (parameters , )

1 x
x e
x0

f x

0
x0

mX t

6. The Chi-square distribution (degrees of freedom )

(

1
1 x
2 x e
x0

f x 2

0
x0

12

mX t 1 1 2t
2 t
2

1
2

Properties of Moment Generating Functions

1.
2.
3.

mX(0) = 1
k k
2 2 3 3
mX t 1 1t t t K t K
2!
3!
k!
k
d
k
mX 0 k mX t k
dt
t 0
i.e. mX 0 1
mX 0 2
mX 0 3 , etc

The log of Moment Generating Functions

Let lX (t) = ln mX(t) = the log of the moment generating
function

Then
l X t

l X 0 ln mX 0 ln1 0
1

mX t

mX t

mX t

l X 0

mX t

mX t mX t mX t
l X t
2
mX t

mX 0
mX 0

mX 0 mX 0 mX 0
l X 0
2
mX 0

2 1 2
2

Thus lX (t) = ln mX(t) is very useful for calculating the

mean and variance of a random variable
1.

l X 0

2.

l X 0 2

Examples
1. The Binomial distribution (parameters p, n)

e p q
t n ln e p q

mX t e p 1 p
t

l X t ln mX

l X t n

l X 0 n
2

et p et p q e t p e t p

e p q
t

1
l X 0 n
p np
pq

1
l X t n t
et p
e pq

p p q p p

p q

npq

2. The Poisson distribution (parameter )

mX t e

et 1

l X t ln mX t et 1
l X t et

l X 0

l X t e

2 l X 0

3. The Exponential distribution (parameter )

mX t

t
t
undefined t

l X t ln mX t ln ln t if t
1
1
l X t
t
t
l X t 1 t

1
1
2
Thus l X 0
and l X 0 2

4. The Standard Normal distribution ( = 0, = 1)

mX t e

t2
2

l X t ln mX t

t2
2

l X t t , l X t 1
Thus l X 0 0 and 2 l X 0 1

5. The Gamma distribution (parameters , )

mX t

l X t ln mX t ln ln t

l X t

t
t

l X t 1 t

2
Hence l X 0
and l X 0 2

6. The Chi-square distribution (degrees of freedom )

mX t 1 2t

l X t ln mX t ln 1 2t
2
1

lX t
2
2 1 2t
1 2t
l X t 1 1 2t

1 2t

Hence l X 0 and 2 l X 0 2

Name
Discrete
Uniform
Bernoulli
Binomial
Geometric
Negative
Binomial
Poisson
Hypergeometric

SummaryofDiscreteDistributions

probabilityfunctionp(x)
1
p(x)=Nx=1,2,...,N
p x=1
p(x)= q x=0

N
p(x)= x pxqNx
p(x)=pqx1x=1,2,...
x1
p(x)= k1 pkqxk
x=k,k+1,...
x
p(x)= x! e x=1,2,...
A NA

x nx
p(x)=

Mean
N+1
2
p

Variance
N21
12
pq

Moment
generating
functionMX(t)
et etN1
N et1
q+pet

Np

Npq

(q+pet)N

1
p
k
p

p2
kq

p2

pet

1qet
pet k

1qet

e(e 1)

A
n N

A A Nn
n N 1N N1

notuseful

Name
Continuous
Uniform
Exponentia
l

SummaryofContinuousDistributions

probability
densityfunctionf(x)

1
ba
0

f ( x)

le lx

f ( x)

Gamma

f(x)= f(x)

d.f.
Normal
Weibull

a xb

Mean
a+b
2

(ba)2
12

Momentgenerating
functionMX(t)
ebteat
[ba]t

t fort<

t fort<

Variance

otherwise
x0

x0
0
la
x a 1e lx x 0

G (a )
0
x0

(1/2) (1/2)x
x
e
x?0
f(x)= (/2)

0
x<0
1
2
2

2
f(x)=
e(x) /2
2
+1

x
+2

(
)
x e
x? 0

(
)

f(x)=

0
x<0

/2

1
12t fort<1/2

et+(1/2)t

[( )]
+1

22

not
avail.