Module 22

DISCRETE RANDOM VECTORS

() Module 22 DISCRETE RANDOM VECTORS 1 / 33

 X = (X1 , . . . , Xp ) : a p-dimensional random vector (r.v.), defined on
a probability space (Ω, P(Ω), P).

FX (·) : d.f. of X .
Definition 1:
(a) The r.v. X is said to be a discrete r.v. if there exists a countable set
SX ⊆ Rp such that P({X = x}) > 0, ∀ x ∈ SX and
X
P({X ∈ SX }) = P({X = x}) = 1.
x∈SX

(b) Under the notation of (a), the set SX is called the support of X and
the function
fX (x) = P({X = x}), x ∈ Rp ,
which
P is such that fX (x) ≥ 0, ∀ x ∈ Rp , fX (x) > 0, ∀ x ∈ SX and
x∈SX fX (x) = 1, is called the joint probability mass function (p.m.f.)
of X .
() Module 22 DISCRETE RANDOM VECTORS 1 / 33
Remark 1 :

Let X = (X1 , . . . , Xp ) be a p-dimensional discrete r.v. with support SX ,

p.m.f. fX (·) and d.f. FX (·).
(a) For given r > 0 and x ∈ Rp , let
v
 u p 
p t
uX
Nr (x) = t∈R : 2
(ti − xi ) < r
i=1

denote the p-dimensional ball of radius r centered at x.

It can be shown that

SX = {x ∈ Rp : P({X ∈ Nr (x)}) > 0, ∀ r > 0}.

(b) Since P({X ∈ SX }) = 1, we have P({X ∈ SXc }) = 0 and therefore

P({X = x}) = 0, ∀ x ∈ SXc .

() Module 22 DISCRETE RANDOM VECTORS 2 / 33

(c) As in the one-dimensional case (p = 1) it can be shown that if there
is a countable set S ⊆ Rp and a function g : PRp → R such that,
p
g (x) ≥ 0, ∀ x ∈ R , g (x) > 0, ∀ x ∈ S and x∈S g (x) = 1, then
there exists a probability space (Ω, P(Ω), P) and a p-dimensional
discrete r.v. X defined on (Ω, P(Ω), P) such that g (·) is p.m.f. of X .

() Module 22 DISCRETE RANDOM VECTORS 3 / 33

(d) Let Y = (Y1 , . . . , Yp ) be a p-dimensional r.v. (not necessarily
discrete or continuous) with d.f. FY (·). For a = (a1 , . . . , ap ) ∈ Rp ,
define an = (a1 − n1 , . . . , ap − n1 ), n = 1, 2, . . . . Then
{X = a} = X −1 ({a})
\ ∞ 
−1
= X (an , a]
n=1
∞
\  
= X −1 (an , a]
n=1
∞
\  
⇒ P({X = a}) = P X −1 (an , a]
n=1
  
−1
= lim P X (an , a]
n→∞
p
X X
= lim (−1)k FX (zn ).
n→∞
k=0


z n ∈∆k,p (an ,a]

() Module 22 DISCRETE RANDOM VECTORS 4 / 33

(e) Using (d) it follows that the joint p.m.f. of a discrete r.v. X is
determined by its joint d.f. Conversely the joint d.f. of X


FX (x) = P {X ∈ (−∞, x]}


= P {X ∈ (−∞, x) ∩ SX }
X
= fX (t)
t∈(−∞,x]∩SX

is determined by its p.m.f. Thus to study the probability function

PX (·), induced by a discrete r.v. X , it is enough to study its p.m.f.

() Module 22 DISCRETE RANDOM VECTORS 5 / 33

(f) Let A ⊆ Rp . Then


P({X ∈ A}) = P {X ∈ A ∩ SX }
 [ 
= P {X = x}
x∈A∩SX
X

= P {X = x}
x∈A∩SX
X
= fX (x)
x∈A∩SX
X
= fX (x)IA (x).
x∈SX

() Module 22 DISCRETE RANDOM VECTORS 6 / 33

Result 1:
Let X = (X1 , . . . , Xp ) be a p-dimensional (p ≥ 2) discrete r.v. with
support SX and p.m.f. fX (·). For fixed k ∈ {1, . . . , p − 1}, let
Y = (X1 , . . . , Xk ) and Z = (Xk+1 , . . . , Xp ) so that X = (Y , Z ). For
y ∈ Rk , define Ay = {z ∈ Rp−k : (y , z) ∈ SX }. Then the r.v. Y is
discrete with support SY = {y ∈ Rk : (y , z) ∈ SX , for some z ∈ Rp−k }
and joint marginal p.m.f.
(P
z∈Ay fX (y , z), if y ∈ SY
fY (y ) = .
0, otherwise
Proof: Note that
{X ∈ SX } = {(Y , Z ) ∈ SX } ⊆ {Y ∈ SY }.
Thus
P({Y ∈ SY }) ≥ P({X ∈ SX }) = 1
⇒ P({Y ∈ SY }) = 1.
() Module 22 DISCRETE RANDOM VECTORS 7 / 33
Also SY is countable (since SX is countable), and for y ∈ SY ,
 \ 
P({Y = y }) = P {Y = y } {X ∈ SX }
 \ 
= P {Y = y } {(y , Z ) ∈ SX }
 \ 
= P {Y = y } {Z ∈ Ay }
 [ 
= P {(Y , Z ) = (y , z)}
z∈Ay
X

= P {(Y , Z ) = (y , z)}
z∈Ay
X
= fX (y , z).
z∈Ay

Note that, for y ∈ SY , Ay 6= φ and, for z ∈ Ay , (y , z) ∈ SX . Therefore

P({Y = y }) > 0, ∀ y ∈ SY . Hence the assertion follows.

() Module 22 DISCRETE RANDOM VECTORS 8 / 33

Remark 2:

(a) The marginal distributions of discrete distributions are discrete.

(b) The above results suggests that to get a marginal p.m.f. from joint
p.m.f. one needs to sum out the arguments of unwanted variables in
the joint p.m.f.
Example 1: Let Z = (X , Y ) have the joint p.m.f.
(
cy , if 1 ≤ x ≤ y ≤ n, x, y ∈ N
fX ,Y (x, y ) = ,
0, otherwise

where n (≥ 2) is a fixed positive integer and c is a fixed real constant.

(a) Find the value of constant c;
(b) Find the marginal p.m.f.s of X and Y ;
(c) Find P({X > Y }), P({X = Y }) and P({X < Y }).

() Module 22 DISCRETE RANDOM VECTORS 9 / 33

Solution:
(a) Clearly
SZ = support of Z
= {(s, t) ∈ R2 : s, t ∈ {1, . . . , n}, s ≤ t}.
Also,
X
fX ,Y (z) = 1
z∈SZ
y
n X
X
⇒ cy = 1
y =1 x=1
Xn
⇒ c y2 = 1
y =1
6
⇒ c = .
n(n + 1)(2n + 1)
() Module 22 DISCRETE RANDOM VECTORS 10 / 33
(b) For x ∈ SX = {1, 2, . . . , n}

fX (x) = P({X = x})

Xn
= P({X = x, Y = y })
y =x
X n
= c y
y =x
h n(n + 1) (x − 1)x i
= c −
2 2
Thus   
 3 n(n+1)−(x−1)x
, if x = 1, 2, . . . , n
fX (x) = n(n+1)(2n+1) .
0, otherwise

() Module 22 DISCRETE RANDOM VECTORS 11 / 33

For y ∈ SY = {1, 2, . . . , n}
y
X
P({Y = y }) = cy
x=1
2
= cy .

Therefore the marginal p.m.f. of Y is

( 2 6y
n(n+1)(2n+1) , if y = 1, 2, . . . , n
fY (y ) = .
0, otherwise

(c)
X
P({X > Y }) = fX ,Y (x, y )
(x,y )∈SZ
x>y
= 0.

() Module 22 DISCRETE RANDOM VECTORS 12 / 33

 X
P({X = Y }) = fX ,Y (x, y )
(x,y )∈SZ
x=y
X n
= c y
y =1
3
= .
2n + 1

P({X < Y }) = 1 − P({X = Y }) − P({X > Y })

3
= 1−
2n + 1
2n − 2
= .
2n + 1

() Module 22 DISCRETE RANDOM VECTORS 13 / 33

Conditional Distribution of Discrete Random Vectors

(Ω, P(Ω), P): a given probability space.

X = (X1 , . . . , Xp ) : Ω → Rp : a p-dimensional r.v. (not necessarily

discrete or A.C.) with d.f. FX (·) (p ≥ 2).
Definition 2:
(a) Let D ⊆ Rp be such that P({X ∈ D}) > 0. Then the conditional d.f.
of X given that X ∈ D is defined by


FX |D (x) = P {X ≤ x}|{X ∈ D}
P({X ≤ x, X ∈ D})
= , x ∈ Rp .
P({X ∈ D})

() Module 22 DISCRETE RANDOM VECTORS 14 / 33

(b) Let X = (X1 , . . . , Xp ) and Y = (Y1 , . . . , Yq ) be p and q dimensional
r.v.s, respectively, and let Z = (X , Y ) (a (p + q)-dimensional r.v.).
Let SZ and fZ (·), respectively, denote the support and joint p.m.f. of
Z . Let SY and fY (·), respectively, denote the support and joint p.m.f.
of Y . For a fixed y ∈ SY , define SX |y = {x ∈ Rp : (x, y ) ∈ SZ }. Then
the conditional d.f. and conditional p.m.f. of X given Y = y (y ∈ SY
is fixed) are defined by

FX |Y (x|y ) = P({X ≤ x|Y = y })

P({X ≤ x, Y = y })
=
P({Y = y })
P
t∈SX |y ,t≤x fZ (t, y )
= P , x ∈ Rp
f
t:(t,y )∈SZ Z (t, y )

() Module 22 DISCRETE RANDOM VECTORS 15 / 33

and
fX |Y (x|y ) = P({X = x|Y = y })
P({X = x, Y = y })
=
P({Y = y })
fZ (x, y )
= , x ∈ Rp ;
fY (y )
respectively.

Remark 3:
(a) It is easy to verify that the function FX |D (·), defined in above
definition, is a proper d.f. (i.e., it satisfies the four properties of a
d.f.).

(b) It is straightforward to establish that, for every fixed y ∈ SY , the

function FX |Y (·|y ) is a proper d.f. and fX |Y (·|y ) is the p.m.f.
corresponding to FX |Y (·|y ) with support SX |y .
() Module 22 DISCRETE RANDOM VECTORS 16 / 33
Result 3:

Let X = (X1 , . . . , Xp )0 be a p-dimensional discrete r.v. with support SX

and joint p.m.f. fX (·). Let SXi and fXi (·), respectively, denote the support
and marginal p.m.f of Xi , i = 1, . . . , p. Then
(a) X1 , . . . , Xp are independent iff
p
Y
fX (x1 , . . . , xp ) = fXi (xi ), ∀ x = (x1 , . . . , xp ) ∈ Rp . (1)
i=1

(b) X1 , . . . , Xp are independent ⇒ SX = SX1 × · · · × SXp .

() Module 22 DISCRETE RANDOM VECTORS 17 / 33

Proof:
Take p = 2, for simplicity of notation.
(a) Suppose that (1) holds. Then
SX = {x ∈ R2 : fX (x) > 0}
= {x ∈ R2 : fX1 (x1 )fX2 (x2 ) > 0}
= {x ∈ R : fX1 (x) > 0} × {y ∈ R : fX2 (y ) > 0}
= SX1 × SX2 .
Moreover, for x = (x1 , x2 ) ∈ R2 ,
FX1 ,X2 (x1 , x2 ) = P({X1 ≤ x1 , X2 ≤ x2 })
X
= fX (t)
t∈SX
t≤x
X X
= fX1 (t1 )fX2 (t2 )
t1 ∈SX1 t2 ∈SX2
t1 ≤x1 t2 ≤x2
! !
() Module 22 DISCRETE RANDOM VECTORS
X X 18 / 33
 = FX1 (t1 )FX2 (t2 ),
implying that X1 and X2 are independent.
Conversely, suppose that X1 and X2 are independent, i.e.,
FX1 ,X2 (x1 , x2 ) = FX1 (x1 )FX2 (x2 ), ∀ x = (x1 , x2 ) ∈ R2 .
Then, for x = (x1 , x2 ) ∈ R2 ,
fX (x1 , x2 ) = P({X1 = x1 , X2 = x2 })
= lim P({x1 − h < X1 ≤ x1 , x2 − h < X2 ≤ x2 })
h↓0

= lim FX (x1 , x2 ) − FX (x1 − h, x2 ) − FX (x1 , x2 − h)
h↓0

+FX (x1 − h, x2 − h)

= lim FX1 (x1 )FX2 (x2 ) − FX1 (x1 − h)FX2 (x2 )
h↓0

−FX1 (x1 )FX2 (x2 − h) + FX1 (x1 − h)FX2 (x2 − h)

() Module 22 DISCRETE RANDOM VECTORS 19 / 33

 = FX1 (x1 )FX2 (x2 ) − FX1 (x1 −)FX2 (x2 ) − FX1 (x1 )FX2 (x2 −)
+FX1 (x1 −)FX2 (x2 −)
 
= FX1 (x1 ) − FX1 (x1 −) FX2 (x2 )
 
− FX1 (x1 ) − FX1 (x1 −) FX2 (x2 −)
  
= FX1 (x1 ) − FX1 (x1 −) FX2 (x2 ) − FX2 (x2 −)
= fX1 (x1 )fX2 (x2 ).

(b) The proof of this assertion is contained in the proof of (a).

() Module 22 DISCRETE RANDOM VECTORS 20 / 33

Result 4:

Let X 1 , . . . , X p be discrete random vectors of (possibly) different

dimensions. Then
(a) X 1 , . . . , X p are independent ⇔
p
Y
P({X i ∈ Ai , i = 1, . . . , p} = P({Xi ∈ Ai }), ∀ A1 , . . . , Ap ⊆ Rp ;
i=1

(b) X 1 , . . . , X p are independent ⇔ ψ1 (X 1 ), . . . , ψp (X p ) are independent,

for any functions (not necessarily real-valued) ψ1 , . . . , ψp .

() Module 22 DISCRETE RANDOM VECTORS 21 / 33

Remark 4:

(a) Result 3 above remains valid (with obvious modifications) for

independence of discrete random vectors (of possibly different
dimensions).

(b) Random vectors X and Y are independent

⇔ FX |Y (x|y ) = FX (x), ∀ x ∈ Rp , y ∈ SY

⇔ fX |Y (x|y ) = fX (x), ∀ x ∈ Rp , y ∈ SY .

() Module 22 DISCRETE RANDOM VECTORS 22 / 33

Result 5:
Let X = (X1 , . . . , Xp )0 be a discrete r.v. with joint p.m.f. fX (·). Then
X1 , . . . , Xp are independent iff
p
Y
fX (x) = gi (xi ), x = (x1 , . . . , xp ) ∈ Rp , (2)
i=1
for some non-negative functions gi (·), i = 1, . . . , p, defined on R. In that
case the marginal p.m.f. of Xi is
fXi (x) = ci gi (x), x ∈ R,
for some positive constant ci , and support of Xi is
SXi = {x ∈ R : gi (x) > 0}, i = 1, . . . , p.
Proof: Take p = 2, for simplicity of notation.
First suppose that X1 and X2 are independent. Then
fX (x) = fX1 (x1 )fX2 (x2 ), ∀ x = (x1 , x2 ) ∈ R2 ,
so that (2) holds.
() Module 22 DISCRETE RANDOM VECTORS 23 / 33
Conversely suppose that (2) holds. Then

SX = {x ∈ R2 : fX (x) > 0}
= {x1 ∈ R : g1 (x1 ) > 0} × {x2 ∈ R : g2 (x2 ) > 0}
= S1 × S2 , say.

() Module 22 DISCRETE RANDOM VECTORS 24 / 33

We have

fX1 (x1 ) = P({X1 = x1 })

X
= P({X1 = x1 , X2 = t})
t:(x1 ,t)∈SX
X
= g1 (x1 )g2 (t)
t:(x1 ,t)∈S1 ×S2
( P
g1 (x1 ) t∈S2 g2 (t), if x1 ∈ S1
= .
0, otherwise

Similarly
( P
g2 (x2 ) s∈S1 g1 (s), if x2 ∈ S2
fX2 (x2 ) = .
0, otherwise

() Module 22 DISCRETE RANDOM VECTORS 25 / 33

Also
X
fX (s, t) = 1
(s,t)∈SX
X
⇒ g1 (s)g2 (t) = 1
(s,t)∈S1 ×S2
X  X 
⇒ g1 (s) g2 (t) = 1
s∈S1 t∈S2
(
g1 (x1 )g2 (x2 ), if x1 ∈ S1 , x2 ∈ S2
⇒ fX1 (x1 )fX2 (x2 ) = .
0, otherwise
= fX (x1 , x2 ), ∀ x = (x1 , x2 ) ∈ R2 ,

implying that X1 and X2 are independent.

() Module 22 DISCRETE RANDOM VECTORS 26 / 33

Example 2 :
Let Z = (X , Y ) have the joint p.m.f.
(
y
, if 1 ≤ x ≤ y ≤ 5, x, y ∈ N
fX ,Y (x, y ) = 55 .
0, otherwise

(a) Find the conditional p.m.f. of X given Y = y (y ∈ {1, . . . , 5}) and of

Y given X = x (x ∈ {1, . . . , 5}).
(b) Are X and Y independent?
(c) Find P({Y ≥ 3}|{X = 2}).
Solution : We have SX ,Y = {(s, t) ∈ N × N : 1 ≤ s ≤ t ≤ 5}.
(a) Fix y ∈ {1, . . . , 5}. Then

P({X = x, Y = y })
fX |Y (x|y ) = , x ∈ R.
P({Y = y })

() Module 22 DISCRETE RANDOM VECTORS 27 / 33

 X
P({Y = y }) = P({X = s, Y = y })
s:(s,y )∈SX ,Y
y
X y
=
55
s=1
y2
= .
55
Thus, for fixed y ∈ {1, . . . , 5},
(
1
y, x ∈ {1, . . . , y }
fX |Y (x|y ) = .
0, otherwise

For fixed x ∈ {1, . . . , 5},

X
fX (x) = P({X = x, Y = t})
t:(x,t)∈SX ,Y

() Module 22 DISCRETE RANDOM VECTORS 28 / 33

 5
X t
=
t=x
55
(6 − x)(5 + x)
=
110
P({X = x, Y = y })
fY |X (y |x) =
P({X = x})
(
2y
(6−x)(5+x) , if y ∈ x, x + 1, . . . , 5
= .
0, otherwise

(b) Since fX |Y (x|y ) is not independent of y , we infer that X and Y are

not independent. Also note that

SX = {1, . . . , 5} = SY

() Module 22 DISCRETE RANDOM VECTORS 29 / 33

and

SX ,Y = {(s, t) ∈ N × N : 1 ≤ s ≤ t ≤ 5}
6= SX × SY .

(c) We have, from (a),

(
y
fY |X (y |2) = 14 , if y = 2, 3, 4, 5
.
0, otherwise

P({Y ≥ 3}|{X = 2}) = 1 − P({Y = 2}|{X = 2})

1 6
= 1− = .
7 7

() Module 22 DISCRETE RANDOM VECTORS 30 / 33

Take Home Problem
Let X = (X1 , X2 , X3 )0 be a discrete r.v. with joint p.m.f.
(
c x1 x2 x3 , x1 = 1, 2, x2 = 1, 2, 3, x3 = 1, 3
fX (x1 , x2 , x3 ) = ,
0, otherwise

where c is a fixed real constant.

(a) Find the value of c, support of X and support of Xi , i = 1, 2, 3;
(b) Find the marginal p.m.f. of Xi , i = 1, 2, 3;
(c) Find the marginal p.m.f. of Y = (X1 , X3 );
(d) Find P({X1 = X2 = X3 });
(e) Find the conditional p.mf. of X1 given (X2 , X3 ) = (2, 1);
(f) Find the conditional p.mf. of (X1 , X3 ) given that X2 = 3;
(g) Given X2 = 3, are X1 and X3 independent?
(h) Are X1 , X2 and X3 independent r.v.s?
(i) Are X1 and X3 independent r.v.s?
() Module 22 DISCRETE RANDOM VECTORS 31 / 33
Abstract of Next Module

We will introduce continuous and absolutely continuous random

vectors and study properties of their probability distributions.

() Module 22 DISCRETE RANDOM VECTORS 32 / 33

 Thank you for your patience

() Module 22 DISCRETE RANDOM VECTORS 33 / 33