Moment Generating Functions

Recall that for a discrete random variable X {x1 , x2 , . . .} if a function h : R R is such that |h (xi )| pX (xi ) <
i1

(1)

then E (h (X)) is dened and equals h (xi ) pX (xi ) .

i1

An especially important choice of h () is h(x) = etx for complex numbers t C. We dene the moment generating function (MGF for short) of X by mX (t) = E etX =
i1

etxi pX (xi )

(2)

when this exists (i.e. the sum at (1) converges). It can be shown that if X is such that for all t in a circular disk of complex plane centred at the origin mX (t) exists, then X has nite moments of all orders (i.e. E (X n ) exist for all n 1) and the following series expansion holds for MGF: mX (t) = 1 + E(X)t + E (X 2 ) 2 E (X 3 ) 3 t + t + 2! 3! (3)

In other words from a Taylor series of mX (t) the moments of X can be read o. Recall form calculus that the Taylor series expansion mX (t) is: mX (t) = mX (0) + mX (0)t + Comparing (3) and (4) we have that: 1 = mX (0) E(X) = mX (0) E X 2 = m (0) X dn mX (t) E (X n ) = dtn (5) (6) (7) (8)
t=0

m (0) 2 m (0) 3 X t + X t + 2! 3!

(4)

Example Find the mean and the variance of the discrete random variable X whose pmf is pX (i) = p (1 p)i1 for i 1 1

where 0 < p < 1. Solution: Note that E etX =

j1

p (1 p)j1 etj p 1p p 1p (1 p) et
j1 j

= =

, convergent for et <

1 1p

(9)

(1 p) e 1 (1 p) et
1

= p et (1 p)

To use equations (6) and (7) we nd the rst two derivative of mX (t) as: mX (t) = p et (1 p)
2

et
3

(10)
2

m (t) = (2p) et (1 p) X So that E(X) =

e2t pet et (1 p)

(11)

1 2 1 and E X 2 = 2 . p p p

(12)

Using the fact that V (X) = E (X 2 ) (EX)2 , from (12) we obtain V (X) = 1p p2

Even when MGFs exist they are not always easy to caculate in closed form as we did in the above example. However, when a closed form is available then the job of calculating moments is simplied. We mention in passing that MGFs are useful for things much more sophisticated than just calculating moments. In digital communication you will see how they are used to obtain very sharp bounds on probabilities of events which are very dicult to calculate exactly.