Joint Probability Mass Function (PMF)

Remember that for a discrete random variable XX, we define the

PMF as PX(x)=P(X=x)PX(x)=P(X=x). Now, if we have two random
variables XX and YY, and we would like to study them jointly, we
define the joint probability mass function as follows:

The joint probability mass function of two discrete random

variables XX and YY is defined as

PXY(x,y)=P(X=x,Y=y).PXY(x,y)=P(X=x,Y=y).

Note that as usual, the comma means "and," so we can write

PXY(x,y)=P(X=x,Y=y)=P((X=x) and (Y=y)).PXY(x,y)=P(X=x,Y=y)=P((X=x) an

d (Y=y)).

We can define the joint range for XX and YY as

RXY={(x,y)|PXY(x,y)>0}.RXY={(x,y)|PXY(x,y)>0}.

In particular, if RX={x1,x2,...}RX={x1,x2,...} and RY={y1,y2,...}RY={y1,y2,...},

then we can always write

RXY⊂RX×RY={(xi,yj)|xi∈RX,yj∈RY}.RXY⊂RX×RY={(xi,yj)|xi∈RX,yj∈RY}.

In fact, sometimes we define RXY=RX×RYRXY=RX×RY to simplify the

analysis. In this case, for some
pairs (xi,yj)(xi,yj) in RX×RYRX×RY, PXY(xi,yj)PXY(xi,yj) might be zero. For
two discrete random variables XX and YY, we have

∑(xi,yj)∈RXYPXY(xi,yj)=1∑(xi,yj)∈RXYPXY(xi,yj)=1

We can use the joint PMF to find P((X,Y)∈A)P((X,Y)∈A) for any

set A⊂R2A⊂R2. Specifically, we have
P((X,Y)∈A)=∑(xi,yj)∈(A∩RXY)PXY(xi,yj)P((X,Y)∈A)=∑(xi,yj)∈(A∩RXY)PXY(xi,yj)

Note that the event X=xX=x can be written

as {(xi,yj):xi=x,yj∈RY}{(xi,yj):xi=x,yj∈RY}. Also, the event Y=yY=y can be
written as {(xi,yj):xi∈RX,yj=y}{(xi,yj):xi∈RX,yj=y}. Thus, we can write

PXY(x,y)=P(X=x,Y=y)=P((X=x)∩(Y=y)).PXY(x,y)=P(X=x,Y=y)=P((X=x)∩(Y=y
)).

Marginal PMFs
The joint PMF contains all the information regarding the
distributions of XX and YY. This means that, for example, we can
obtain PMF of XX from its joint PMF with YY. Indeed, we can write
PX(x)=P(X=x)=∑yj∈RYP(X=x,Y=yj)=∑yj∈RYPXY(x,yj).law of total
probablityPX(x)=P(X=x)=∑yj∈RYP(X=x,Y=yj)law of total
probablity=∑yj∈RYPXY(x,yj).

Here, we call PX(x)PX(x) the marginal PMF of XX. Similarly, we can

find the marginal PMF of YY as

PY(Y)=∑xi∈RXPXY(xi,y).PY(Y)=∑xi∈RXPXY(xi,y).

Marginal PMFs of XX and YY:

PX(x)PY(y)=∑yj∈RYPXY(x,yj), for any x∈RX=∑xi∈RXPXY(xi,y), for
any y∈RY(5.1)PX(x)=∑yj∈RYPXY(x,yj), for
any x∈RXPY(y)=∑xi∈RXPXY(xi,y), for any y∈RY(5.1)

Let's practice these concepts by looking at an example.

Example
Consider two random variables XX and YY with joint PMF given in
Table 5.1.
Table 5.1 Joint PMF of XX and YY in Example 5.1

Y=0Y=0 Y=1Y=1 Y=2Y=2

X=0X=0 1616 1414 1818

X=1X=1 1818 1616 1616

Figure 5.1 shows PXY(x,y)PXY(x,y).

Figure 5.1: Joint PMF of XX and YY (Example 5.1).

a. Find P(X=0,Y≤1)P(X=0,Y≤1).
b. Find the marginal PMFs of XX and YY.
c. Find P(Y=1|X=0)P(Y=1|X=0).
d. Are XX and YY independent?
 Solution
o
a. To find P(X=0,Y≤1)P(X=0,Y≤1), we can write

P(X=0,Y≤1)=PXY(0,0)+PXY(0,1)=16+14=512.P(X=0,Y≤1)=PX
Y(0,0)+PXY(0,1)=16+14=512.

b. Note that from the table,

RX={0,1} and RY={0,1,2}.RX={0,1} and RY={0,1,2}.

Now we can use Equation 5.1 to find the marginal

PMFs. For example, to find PX(0)PX(0), we can write

PX(0)=PXY(0,0)+PXY(0,1)+PXY(0,2)=16+14+18=1324.PX(0)=P
XY(0,0)+PXY(0,1)+PXY(0,2)=16+14+18=1324.

We obtain

PX(x)=⎧⎩⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪132411240x=0x=1otherwisePX
(x)={1324x=01124x=10otherwise

PY(y)=⎧⎩⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪7245127240y
=0y=1y=2otherwisePY(y)={724y=0512y=1724y=20oth
erwise

c. Find P(Y=1|X=0)P(Y=1|X=0): Using the formula for

conditional probability, we have

P(Y=1|X=0)=P(X=0,Y=1)P(X=0)=PXY(0,1)PX(0)=141324=613.P(
Y=1|X=0)=P(X=0,Y=1)P(X=0)=PXY(0,1)PX(0)=1413
24=613.

d. Are XX and YY independent? XX and YY are not

independent, because as we just found out
 P(Y=1|X=0)=613≠P(Y=1)=512.P(Y=1|X=0)=613≠P(Y=1)=
512.

Caution: If we want to show that XX and YY are

independent, we need to check
that P(X=xi,Y=yj)=P(X=xi)P(Y=yj)P(X=xi,Y=yj)=P(X=xi)P(
Y=yj), for all xi∈RXxi∈RX and all yj∈RYyj∈RY. Thus,
even if in the above calculation we had
found P(Y=1|X=0)=P(Y=1)P(Y=1|X=0)=P(Y=1), we
would not yet have been able to conclude
that XX and YY are independent. For that, we would
need to check the independence condition for
all xi∈RXxi∈RX and all yj∈RYyj∈RY.