Lecture 10: Characteristic

Functions

1. Definition of characteristic functions

1.1 Complex random variables

1.2 Definition and basic properties of characteristic functions
1.3 Examples
1.4 Inversion formulas

2. Applications

2.1 Characteristic function for sum of independent random vari-

ables
2.2 Characteristic functions and moments of random variables
2.3 Continuity theorem
2.4 Weak law of large numbers
2.5 Central Limit theorem
2.6 Law of small numbers

1. Definition of characteristic function

1.1 Complex random variables

(1) Complex numbers has a form u = a + ib where a, b are real

numbers and i is so-called imaginary unit which has a property
i2 = 1.

1
(2) The summation of complex numbers u = a+ib and v = c+id
is performed according the rule u + v = (a + c) + i(b + d) and
the multiplication as uv = (ac bd) + i(ad + bc).

(3) For a complex number u = a + ib, the conjugate complex

number is defined as u = a ib and the norm is defined by the
formula |u|2 = uu = a2 + b2 ; |uv| = |u||v|; |u + v| |u| + |v|.

(4) The basic role in what follows plays the formula

eit = cos t + i sin t, t R1 .

Note that |eit | = cos2 t + sin2 t = 1.

A complex random variable is defined as Z = X + iY where

X = X() and Y = Y () are usual real-valued random vari-
ables defined on some probability space < , F, P >.

The distribution of a complex random variable Z is determined

by the distribution of the random vector (X, Y ).

The expectation of the complex random variable Z = X + iY is

defined as the complex number EZ = EX + iEY .

Two complex random variables Z1 = X1 +iY1 and Z2 = X2 +iY2

are independent if the random vectors (X1 , Y1 ) and (X2 , Y2 ) are
independent.

(5) The product of two complex random variables Z1 Z2 = (X1 X2

Y1 Y2 )+i(X1 Y2 +Y1 X2 ) then EZ1 Z2 = E(X1 X2 Y1 Y2 )+iE(X1 Y2 +
Y1 X2 ). If the random variables Z1 and Z2 are independent, then

2
EZ1 Z2 = EZ1 EZ2 .

-
EZ1 Z2 = E(X1 X2 Y1 Y2 ) + iE(X1 Y2 + Y1 X2 ) = (EX1 EX2
EY1 EY2 ) + i(EX1 EY2 + EY1 EX2 ) = EZ1 EZ2 .
-

(6) If |X| M then |EX| M .

-
Let X = Y +iZ, then |EX|2 = |EY +iEZ|2 = (EY )2 +(EZ)2
EY 2 + EZ 2 = E|X|2 M 2 .
-

(7) |EX| E|X|.

1.2 Definition and basic properties of characteristic

functions

Definition 10.1. Let X be a random variable with the dis-

tribution FX (A) and the distribution function FX (x). The char-
acteristic function of the random variable X (distribution FX (A)
and the distribution function FX (x)) is called the complex-valued
function X (t) defined on a real line as function of t by the for-
mula, Z
X (t) = E exp{itX} = eitx FX (dx)

Z Z
= E cos tX +iE sin tX = cos txFX (dx)+i sin txFX (dx).

(1) If X is a discrete random variable with the discrete distri-
bution pX (xn ) = P (X = xn ), n = 0, 1, . . . ( n pX (xn ) = 1), then
P

X itx
X (t) = E exp{itX} = en
pX (xn ).
n

3
(2) If X is a continuous random variable with the probability
density function fX (x), then
Z
X (t) = E exp{itX} = eitx fX (x)dx,

where the integration the Riemann integration is performed if
the density fX (x) is Riemann integrable (otherwise the Lebesgue
integration should be used).

(3) In the latter case the characteristic function is also known

as the Fourier transform for function fX (x).

Theorem 10.1. The characteristic function X (t) of a ran-

dom variable X possesses the following basic properties:
(1) X (t) is a continuous function in t R1 ;
(2) X (0) = 1
(3) X (t) is positively defined function, i.e., a quadratic form
1j,ln cj cl X (tj tl ) 0 for any complex numbers c1 , . . . , cn
P

and real t1 , . . . , tn and n 1.

-
(a) |X (t+h)X (t)| |Eei(t+h)X EeitX | = |EeitX (eihX 1)|
E|eitX (eihX 1)| = E|eihX 1|.

(b) |eihx 1| 2 and also |eihx 1| |hx|. Choose A > 0 and

get the following estimates
Z Z A
E|eihX 1| |eihx 1|FX (dx) + |eihx 1|FX (dx)
|x|>A A
Z Z A
2 FX (dx) + |hx|FX (dx)
|x|>A A
2P (|X| A) + hAP (|X| A) 2P (|X| A) + hA.

4
(c) Choose an arbitrary > 0. Then choose A so large that
2P (|X| A) 2 and then h so small that hA 2 . Then the
expression on the right hand side in the estimate given in (b)
will be less or equal .

(d) X (0) = Eei0X = E1 = 1.

(e) To prove (3) we get

cj cl Eei(tj tl )X
X X
cj cl X (tj tl ) =
1j,ln 1j,ln

cj eitj X cj eitj X = E| cj eitj X |2 0.

X X X
=E
1jn 1jn 1jn
-

Theorem 10.2 (Bochner)**. A complex-valued function

(t) defined on a real line is a characteristic function of some
random variable X, i.e., can be represented in the form (t) =
R
EeitX = eitx FX (dx), t R1 if and only if it possesses the
properties (1) - (3) formulated in Theorem 1.

An additional properties of characteristic functions are:

(4) |X (t)| 1 for any t R1 ;

(5) a+bX (t) = eiat X (bt);

(6) |eitX | 1 and, thus, |X (t)| = |EeitX | 1;

(7) a+bX (t) = Eeit(a+bX) = eiat EeitbX = eiat X (bt);

1.3 Examples

5
(a) If X a where a = const, then X (t) = eiat .

(1) Binomial random variable.

X = B(m, p), m = 1, 2, . . . , 0 p 1;
n n
pX (n) = Cm p (1 p)mn , n = 0, 1, . . . , m;
X (t) = (peit + (1 p))m .

(2) Poisson random variable.

X = P(), > 0;
n
pX (n) = n! e , n = 0, 1, . . . ;
it
X (t) = e(e 1) .

(3) Normal random variable.

X = N (m, 2 ), m R1 , 2 > 0;
x2
fX (x) = 12 e 2 , x R1 ;
t2
X (t) = e 2

Y = m + X;
(xm)2
fY (x) = 1 e 2 2 , x R1
2
2 2
iat 2t
Y (t) = e

(4) Gamma random variable.

X = Gamma(, ), , > 0;
x
1
fX (x) = () x1 e , x 0 where () = 0 x1 ex ;
R

1
X (t) = ( 1it ) .

6
 1.4 Inversion formulas

F (A) and G(A) are probability measures on a real line;

R R
(t) = eitx F (dx) and (t) = eitx G(dx) are their charac-
teristic functions.

Lemma 10.1 (Parseval identity). The following so-called

Parseval identity takes place,
Z Z
eity (t)G(dt) = (x y)F (dx).

-
(a) The following equality follows from the definition of a char-
acteristic function,
Z
ity
e (t) = eit(xy) F (dx)

(b) Integrating it with respect to the measure G(A) and using

Fubinni theorem we get,
Z Z Z
ity
e (t)G(dt) = [ eit(xy) F (dx)]G(dt)

Z Z Z
it(xy)
= [ e G(dt)]F (dx) = (x y)F (dx).

-

Theorem 10.3 (inversion). Different characteristic func-

tion correspond to different distributions and vise versa.

This theorem is a corollary of the inversion formula given be-

low.

7
 Theorem 10.4. Let F (x) be a distribution function and CF
is the set of all point of continuity of this distribution function.
R
Let also (t) = eitx F (dx) be the characteristic function that
corresponds to this distribution function. The following inver-
sion formula takes place
1 Z z Z ity 2 2
t 2
F (z) = lim [ e (t)e dt]dy, z CF .
2 0 2
-
(a) Let G(A) is a normal distribution with parameters m = 0
and 12 . Then Parseval identity takes the following form
Z
ity t2 2 Z
(xy)
2

e (t) e 2 dt = e 2 2
F (dx),
2

or equivalently
1 Z ity t2 2
Z 1 (yx)2
e (t)e 2 dt = e 22 F (dx).
2 2

(b) Integrating with respect to the Lebesgue measure (Riemann

integration can be used!) over the interval (, z] we get
1 Z z Z ity 2 2
t 2
Z z Z 1 (yx)
2

[ e (t)e dt]dy = [ e 2 2
F (dx)]dy
2 2
Z Z z 1 (xy)2
= [ e 22 dy]F (dx) = P (N (0, 2 ) + X z),
2

where N (0, 2 ) is a normal random variable with parameters 0

and 2 independent of the random variable X.

(c) If 2 0 then the random variables N (0, 2 ) converge in

probability to 0 (P (|N (0, 2 )| ) 0 for any > 0) and, in

8
sequel, the random variables N (0, 2 ) + X weakly converge to
the random variable X that is P (N (0, 2 )+X z) P (X z)
as 2 0 for any point z that is a point of continuity for the
distribution function of the random variable X.
-

Theorem 10.5. Let F (x) be a distribution function and

R
(t) = eitx F (dx) be the characteristic function that cor-
responds to this distribution function. Let (t) is absolutely
R
integrable at real line, i.e., |(t)|dt < . Then the distribu-
tion function F (x) has a bounded continuous probability density
function f (y) and the following inversion formula takes place
1 Z ity
f (y) = e (t)dt, y R1 .
2
-
(a) The proof can be accomplished by passing 2 0 in the
Parseval identity
1 Z ity 2 2
t 2
Z 1 (yx)
2

e (t)e dt = e 2 2
F (dx).
2 2

(b) Both expressions on the left and right hand sides of the above
identity represent the probability density function f (y) for the
distribution function F (y) of random variable N (0, 2 ) + X.

(c) F () F () as 0.

(d) From the left hand side expression in the above identity, we
see that.
f (y) f (y) as 0.

9
(e) Note that f (y) and f (y) are bounded continuous functions;

(f) It follows from (c), (d) and (e) that for any x0 < x00
Z x00 Z x00
00 0
F (x ) F (x ) = f (y)dy f (y)dy
x0 x0

(g) But, if x0 , x00 are continuity point of F (x) that

F (x00 ) F (x0 ) F (x00 ) F (x0 ) as 0.

(h) Thus,
Z x00
F (x00 ) F (x0 ) = f (y)dy,
x0
i.e., indeed, f (y) is a probability density of the distribution func-
tion F (x).

2. Applications

2.1 Characteristic function for sum of independent ran-

dom variables

Lemma 10.2. If a random variable X = X1 + + Xn is a

sum of independent random variables X1 , . . . , Xn then

X (t) = X1 (t) Xn (t).

-
X (t) = Eeit(X1 +Xn ) = E nk=1 eitXk = nk=1 EeitXk .
Q Q

10
 Examples

(1) Let random variables Xk = N (mk , k2 ), k = 1, . . . , n be in-

dependent normal random variables. In this case X = X1 +
+ Xn is also a noramal random variable with parameters
m = m1 + + mn and 2 = 12 + + n2 .

- 2 t2
k 2 t2
X (t) = nk=1 eitmk 2 = eitm 2 .
Q

(2) Let random variables Xk = P(k ), k = 1, . . . , n be inde-

pendent Poissonian random variables. In this case X = X1 +
+ Xn is also a Poissonian random variable with parameters
= 1 + + n .

-
it it
X (t) = nk=1 ek (e 1) = e(e 1) .
Q

2.2 Characteristic functions and moments of random

variables

Theorem 10.6. Let X be a random variable such that

R
E|X|n = |x|n FX (dx) < for some n 1. Then, there
exist n-th continuous derivatives for the characteristic function
R
X (t) = eitx FX (dx) for all t R1 given by the formula,
Z
(k)
X (t) = ik xk eitx FX (dx), k = 1, . . . , n,

11
and, therefore, the following formula takes place
(k)
EX k = ik X (0), k = 1, . . . , n.

-
itx eihx 1
(1) X (t+h) X (t)
R
h = e h FX (dx);
ihx
(2) | e h1 | |x| that let to perform the limit transition in the
ihx
above relation as h 0 using also that e h1 ix as h 0;
(k)
(3) In general case, the formula for X (t) can be proved by in-
duction.
-

Examples

(1) Let X = N (m, 2 ) be a normal random variable. Then

2 t2
X (t) = eitm 2 and, therefore,
2 t2
(1) 0X (t) = (im 2 t)eitm 2 and EX = i1 0X (0) = m.
2 t2 2 t2
(2) 00X (t) = 2 eitm 2 + (im 2 t)2 eitm 2 and EX 2 =
i2 00X (0) = 2 + m2 .

(2) Let random variables X = P() be a Poissonian random

it
variable. Then X (t) = e(e 1) and, therefore,
it
(1) 0X (t) = ieit e(e 1) and, thus, EX = i1 0X (0) = .
it it
(2) 00X (t) = i2 eit e(e 1) +i2 2 e2it e(e 1) and EX 2 = i2 00X (0) =
+ 2 .

2.3 Continuity theorem

Xn , n = 0, 1, . . . is a sequence of random variables with distri-

bution functions, respectively, Fn (x), n = 0, 1, . . .;

12

n (t) = EeitXn = eitx Fn (dx), n = 0, 1, . . . are the corre-
R

sponding characteristic functions;

CF denotes the set of continuity points for a distribution func-
tion F (x).

Theorem 10.7 (continuity)**. Distribution functions Fn

F0 as n if and only if the corresponding characteristic func-
tions n (t) 0 (t) as n for every t R1 .

-
(a) Let Fn F0 as n .

(b) Then by Helly theorem, for every t R1 ,

Z Z
itx
n (t) = e Fn (dx) eitx F0 (dx) = 0 (t) as n .

(c) Let n (t) 0 (t) as n , t R1 .

(d) As was proved in Lecture 9 using Cantor diagonal method,

for any subsequence nk as k , one can select a sub-
sequence n0r = nkr as r such that

Fn0r (x) F (x) as n , x CF ,

where F (x) is proper or improper distribution function (non-

decreasing, continuous from the right and such that 0 F ()
F () 1).

(e) Let us write down the Parseval identity

1 Z ity t2 2
Z 1 (yx)2
e n0r (t)e 2 dt = e 22 Fn0r (dx).
2 2

13
(f) By passing to the limit as r in the above identity and
takin into account (c), and Lebesque theorem, and the fact that
(yx)2
1 e 2 2 0 as x , we get
2

1 Z ity 2 2
t 2
Z 1 (yx)
2

e 0 (t)e dt = e 2 2
F (dx).
2 2

(g) By multiplying both part in (f) by 2 and taking y = 0,
we get
Z t2 2 Z x2
0 (t) e 2 dt = e 22 F (dx) F (+) F ().
2

(g) By passing to the limit as 0 in the above equal-

R t2 2
ity and taking into account that (a) e 2 dt = 1, (b)
2
t 2 2
e 0 as , for t 6= 0, and (c) (t) is continuous
2
2
in zero and (0) = 1, we get,

Z t2 2
1 =
lim 0 (t) e 2 dt F (+) F ().
2
(h) This, F (+) F () = 1, i.e., the distribution F (x) is a
proper distribution.

(i) In this case, the weak convergence Fn0r F as n

implies by the first part of the theorem that
Z
n0r (t) (t) = eitx F (dx) as n , t R1 .

(j) This relation together with (c) implies that

Z
(t) = eitx F (dx) = 0 (t), t R1 .

14
(k) Therefore due to one-to-one correspondence between char-
acteristic functions and distribution functions F (x) F0 (x).

(l) Therefore, by the subsequence definition of the weak conver-

gence Fn F0 as n .
-

Theorem 10.8* (continuity). Distribution functions Fn (x)

weakly converge to some distribution function F (x) if and only if
the corresponding characteristic functions n (t) converge point-
wise (for every t R1 ) to some continuous function (t). In
this case, (t) is the characteristic function for the distribution
function F (x).

-
(a) Let n (t) (t) as n for every t R1 . Then (0) = 1;

(b) (t) is positively defined since functions n (t) possess these

properties.

(c) Since (t) is also continuous, the properties (a)-(c) pointed in

Theorem 10.1 hold and, therefore, (t) is a characteristic func-
tion for some distribution function F (x). Then Fn (x) weakly
converge to F (x) by Theorem 10.7.
-

2.2 Weak law of large numbers (LLN)

< , F, P > is a probability space;

Xn , n = 0, 1, 2, . . . are independent identically distributed (i.i.d.)

15
random variables defined on a probability space < , F, P >.

Theorem 10.9 (weak LLN). If E|X1 | < , EX1 = a, then

random variables
X1 + + Xn d
Xn = a as n .
n
-
(a) Let (t) = EeitX1 , n (t) = EeitX n be the corresponding char-
acteristic functions. Then n (t) = (( nt ))n .
(b) Since E|X1 | < , there exists the continuous first derivative
0 (t) and, therefore, Taylor expansion for (t) in point t = 0 can
be written down that is (t) = 1 + 0 (0)t + o(t)
(c) Since 0 (0) = iEX1 = ia, the above Taylor expansion can be
re-written. in the following form (t) = 1 + iat + o(t).
(d) Therefore, n (t) = (( nt ))n = (1 + ia nt + o( nt ))n .

(e) If un 0 and vn then lim(1 + un )vn exists if and only

if there exists lim un vn = w and, in this case, lim(1 + un )vn = ew .

(f) Using this fact, we get limn n (t) = limn (1 + ia nt +

o( nt ))n = eiat , for every t R1 .
(g) Thus by the continuity theorem for characteristic functions
d
X n a as n .
-

10.10 Central Limit Theorem (CLT)

< , F, P > is a probability space;

Xn , n = 0, 1, 2, . . . are independent identically distributed (i.i.d.)

16
random variables defined on a probability space < , F, P >;
Z = N (0, 1) is standard normal random variable with the dis-
2
1
Rx y2
tribution function F (x) = n
e dy, < x < .

Theorem 10.10 (CLT). If E|X1 |2 < , EX1 = a, V arX1 =

2 > 0, then random variables
X1 + + Xn an d
Zn = Z as n .
n
(1) Since, the limit distribution F (x) is continuous, the above
relation means that P (Zn x) F (x) as n for every
< x < .

-
Xn a
(a) Let introduce random variables Yn = ,n = 1, 2, . . ., then
n an = Y1 ++Y
Zn = X1 ++X n
.
n n

(b) Let (t) = EeitY1 , n (t) = EeitZ n . Then n (t) = ((t/ n))n .

(c) If E|X1 |2 < then E|Y1 |2 < and, moreover, EY1 =

1 (EX1 a) = 0, E(Y1 )2 = 2 E(X1 a)2 = 1.
(d) Since E|Y1 |2 < , there exist the continuous second deriva-
tive 00 (t) and, therefore, Taylor expansion for (t) in point t = 0
can be written down that is (t) = 1 + 0 (0)t + 21 00 (0)t2 + o(t2 ).
(e) Since 0 (0) = iEY1 = 0 and 00 (0) = i2 E(Y1 )2 = 1, the
above Taylor expansion can be re-written in the following form
(t) = 1 12 t2 + o(t2 ).
t2 2
(f) Therefore, n (t) = (( tn ))n = (1 2n + o( tn ))n .
(g) Using the above formula, we get limn n (t) = limn (1
2
t2 t2 n t2
2n + o( n )) = e , for every t R1 .

17
(h) Thus by the continuity theorem for characteristic functions
d
Zn Z as n .
-

2.6 Law of small numbers

Let Xn,k , k = 1, 2, . . . are i.i.d. Bernoulli random variables

defined on a probability space < n , Fn , Pn > and taking values
1 and 0, respectively, with probabilities pn and 1 pn , where
0 pn 1.
N () = P() is a Poissonian random variable with parameter
r
, i.e., P (N () = r) = r! e , r = 0, 1, . . ..

Theorem 10.11 (LSN). If 0 < pn n < , then

random variables
d
Nn = Xn,1 + + Xn,n N () as n .

(1) The random variable Nn is the number of successes in n

Bernoulli trials.

(2) The random variables Nn = Xn,1 + + Xn,n , n = 1, 2, . . .

are determined by series of random variables
X1,1
X2,1 , X2,2
X3,1 , X3,2 , X3,3
......

This representation gave the name to this type of models. as

so-called triangle array mode.

18
-
(a) Let n (t) = EeitNn The characteristic function EeitXn,1 =
pn eit + 1 pn and, thus, the characteristic function n (t) =
EeitNn = (pn eit + 1 pn )n .
(b) Using the above formula, we get n (t) = (pn eit + 1 pn )n
it it
epn (e 1)n e(e 1) as n , for every t R1 .
(c) Thus by the continuity theorem for characteristic functions
d
Nn N () as n .
-

LN Problems

1. What is the characteristic function of random variable X

which takes value 1 and 1 with probability 12 .

2. Prove that the function cosn t is a characteristic function

for every n = 1, 2, . . ..

3. Let X is a uniformly distributed random variable on the

interval [a, b]. Please find the characteristic function of the ran-
dom variable X.

4. Let f (x) is a probability density for a random variable X.

Prove that the characteristic function is a real-valued function
if f (x) ia a symmetric function, i.e. f (x) = f (x), x 0.

5. Let random variables X1 and X2 are independent and have

gamma distributions s parameters 1 , and 2 , , respectively.
Prove that the random variable Y = X1 + X2 also has gamma
distribution with parameters 1 , +2 , .

19
 6. Let X has a geometrical distribution with parameter p.
Find EX and V arX using characteristic functions.

7.Let Xn are geometrical random variables with discrete dis-

tribution pXn (k) = pn (1 pn )k1 , k = 1, 2, . . ., where 0 < pn 0
d
as n . Prove that pn Xn X as n , where X =
Exp(1) is an exponential random variable with parameter 1.

20