Module IIIc - Annotated

III C :
J OINT D ISTRIBUTION AND C OVARIANCE
Jiheng Zhang
III C :
J OINT D ISTRIBUTION AND C OVARIANCE
Jiheng Zhang
Joint Distribution Independence LOTUS – 2D Conditional Distributions Variance & Covariance
Joint Distribution
Sometimes, we need to deal with random variables having more than 1 dimension.
(X, Y)
The joint cumulative distribution function is
F(x, y) = P{X  x, Y  y}
The cdf of X:
^
FX (x) = P{X  x}
marginal
= P{X  x, Y < 1}
= F(x, 1)
The cdf of Y:
FY (y) = F(1, y)
III c: Joint Distribution and Covariance Jiheng Zhang
Joint PMF – Discrete Random Variables

When both X and Y are discrete, define the joint probability mass function
Note that
p(xi , yi ) = P{X = xi , Y = yj }
E-
I
[Y.CI Fj:)
=
Fjn Fj ,
=
¢
{X = xi } = {X = xi , Y = yj } U Fj =
S
' IF j I IF i
X
pX (xi ) = P{X = xi } = P{X = xi , Y = yj }
j
X
= p(xi , yj )
j
X
pY (yj ) = P{Y = yj } = p(xi , yj )
i
The pmf of X (or Y) are also call marginal pmf
Joint PMF – Discrete Random Variables
Example: Suppose that☐ 3 batteries are randomly chosen from a group of 3 new, 4
used but still working, and 5 defective batteries.
X: number of new batteries

Y: number of used but still working
The probability mass function p(i, j) = P{X = i, Y = j}

✓ ◆ ✓ ◆
5 12 10
p(0, 0) = / =
3 3 220
···
✓ ◆✓ ◆ ✓ ◆
3 4 12 18
p(1, 2) = / =
1 2 3 220

Joint PMF
98 – Discrete Random Variables Chapter 4: Random Variables and Expectation
TABLE 4.1 P{X = i, Y = j }

j Row Sum
i 0 1 2 3 = P {X = i }
0 10 40 30 4 84
220 220 220 220 220
1 30 60 18 0 108
220 220 220 220
2 15 12 0 0 27
220 220 220
3 1 0 0 0 1
220 220
Column
Sums =
P{Y = j } 56 112 48 4
220 220 220 220

Joint PDF – Continuous Random Variables
When both X and Y are continuous, define the joint probability density function to
be a function f (x, y) such that
0
Z x Z y
P{X  x, Y  y} = f (x, y)dxdy
1 1
.
Using a more general notation,

ZZ
P{(X, Y) 2 C} = f (x, y)dxdy
C
for every set C of pairs of real numbers.

An Illustration of Joint PDF
Fixed

An Illustration of Joint PDF
÷t:¥F
"

fix / =
3×2 fix)=6✗
Upon differentiation, we have
.
@2 FIX,y ) =3
xy -155
☐
f (a, b) = F(a, b)
←f
@a@b
@
@a F(a, b) denotes the partial differentiation :#2+07=3 .si
¥¥f*H=Éx -110yd ?
Consider (a, a + da] ⇥ (b, b + db],
Z b+db Z a+da
P{a < X  a + da, b < Y  b + db} = f (x, y)dxdy
b a
⇡ f (a, b)dadb
f (a, b) indicates how likely (X, Y) will be near (a, b)

P{X 2 A} = P{X 2 A, Y 2 ( 1, 1)}
I☒☐☐
Z Z 1
= f (x, y)dydx
A 1
R1
Let fX (x) =
-
☐
f (x, y)dy, then
1
Z
P{X 2 A} = fX (x)dx
A
So
R1
fX (x) = f (x, y)dy is the probability density function of X
R 11
fY (y) = 1 f (x, y)dx is the probability density function of Y
Example
Example: The joint density function of X and Y is given by
(
2e x e 2y 0 < x < 1, 0 < y < 1
f (x, y) =
0 otherwise
Compute: (a) P{X < a}; (b) P{X > 1, Y < 1}; (c) P{X < Y}.

Example
Example: The joint density function of X and Y is given by
(
2e x e 2y 0 < x < 1, 0 < y < 1
f (x, y) =
0 otherwise
Compute: (a) P{X < a}; (b) P{X > 1, Y < 1}; (c) P{X < Y}.
Z aZ 1 ← f¥✗)
P{X < a} = 2y x
2e e dydx
②
15=1
0 0
Z a Z 1 "
= e x
2e 2y dydx
D- e-
-
0 0
Z a
= .
= e x dx
☐ 0
a
=1 e

Example
Z 1Z 1
P{X> 1, Y< 1} = 2e x e 2y dxdy
0 1
Z 1 Z 1
= 2e 2y e x dxdy
0 1
Z 1
1
= 2e 2y [ e x 1 ]dy ☐
0
Z 1
1
=e 2e 2y dy
0
1 2
=e (1 e )

Example D il
as
" e- " e-
Ya < Y ZZ
a.
" 2e x e 2y
^
-
'
.
P{X < Y} =
.
-
i
☐yy=
=
Z
Z0 1 0
(x,y):x<y
1Z y
2e x e 2y dxdy
2e 2y [
Z y
e x dx]dy
=D e-3
dxdy
÷
.
=
0
Z 1 0 ☐
=
Z0 1
e-
2e 2y (1 e y )dy
Z 1
= 2e 2y dy 2e 3y dy
0 0
. 2 1
=1 =
3 3
s :S :
Independence
Consider two events {X 2 A} and {Y 2 B}, the following is NOT always true:
P{X 2 A, Y 2 B} = P{X 2 A}P{Y 2 B} How

←
many
If true for ALL A and B, then X and Y are independent.
In terms of joint distribution function,
F(a, b) = FX (a)FY (b) for all a. b.
,

Independence
Consider two events {X 2 A} and {Y 2 B}, the following is NOT always true:
P{X 2 A, Y 2 B} = P{X 2 A}P{Y 2 B}
If true for ALL A and B, then X and Y are independent.

In terms of joint distribution function,
F(a, b) = FX (a)FY (b)
In discrete case
p(x, y) = pX (x)pY (y)
In continuous case
f (x, y) = fX (x)fY (y)

Uniform Distribution on a Square
Suppose (X, Y) are uniformly distributed on a square, then = ply > as) .
(
1
1  x  1, 1  y  1
±g÷÷µ÷,
f (x, y) = 4
0 otherwise
y f±ix =/ >
-
-
fans ) dy =
¥ dy =
-12
i
- -
-
of
-
-
- -
€10.2 fy- LY ) = - - - - i - -
- - -
-
-
=L
0.6
x
f*Wfy-
"
f- ix. g)
"
all
- l I =
¥ =
Ext = is )
,
for
0 .
-1 < ✗ C /
,
p ,, , , , , , , , ,
P (F)
HE
III c: Joint Distribution and Covariance
TE a
=
÷
Jiheng Zhang
Uniform Distribution on a Disc
Suppose (X, Y) are uniformly distributed on a disc, then

(
- 1 2
⇡ 0x +y 1
2
y
-10.62=12--79--0.
'
5T¥ 8 f (x, y) =
0 otherwise
yyñ=i zf-xifx-II-I-fx.us
y)dy=S+F" ¥ dy -
E.
=
I
✗ = -11
to .
%
CJ .
-5*(0-6)=1?j%=¥ .
✗ =ab
t.ge
.
0.6 ,
x
% fix .gs#f-*-cx)-fy-cy) .
0.8
-
- - -
PCY > of / ×> at ) 0
fq.co
.
-_
PCY > as ) -40
Jiheng Zhang
,
Chicken-egg Problem e-i-e-NH-e-NP-H-e-dP.eu't

Suppose the total number of eggs N produced at a farm is a Poisson random
variable with parameter . Suppose each egg will hatch with probability p. Let X
denote the number of hatches, and Y denote the number of eggs do not hatch.
Are X and Y independent? PCE) TÉ PIE/ Fj ) PIF;) = -
conditioning 5=0
.
g 1
X
P(X = i, Y = j) = P(X = i, Y = j|N = n)P(N = n) a- 3,5=5 ,
n=lo .
n=1 0
if itj=n
= P(X = i, Y = j|N = i + j)P(N = i + j) = P(X = i|N = i + j)P(N = i + j)
✓ ◆ - I
②②
i+j i 1 P(Binomial city;p)=z ,
= p (1 p)j e i+j
pmf
i (i + j)!
.
pref
(i + j)! i j 1
- pi qj
= pq e pe q i j = e p i e q j
i!j! -(i + j)! -
i! j! -
-
>
Poisson lip) .
Poisson CXG
Jiheng Zhang
Independence
Example: Suppose X and Y are independent and have common density function
f (a) = e a , a > 0. Find the density of X/Y. 1¥ - Y ] =
.
By independence, we have that f (x, y) = e x e y , x, y > 0. =3 ✗< at ]

8- Ha - ZZ
Ya FX/Y (a) = P{X/Y  a} = f (x, y)dxdy
☐
¥
x/ya
Z 1 Z ay
= e x e y dxdy
0 0
Z 1
= (1 e ay )e y dy
0
h e (a+1)y i 1 1
y
= e + =1
→'#
The density function is f X/Y (a)
0
= FX/Y (a) =
a+1
1
(1+a)2
0
, a > 0.
a+1

Joint Distributions for Multiple Variables

Let X1 , X2 , . . . , Xn be n random variables.
The joint distribution is defined as
F(a1 , a2 , . . . , an ) = P{X1  a1 , X2  a2 , . . . , Xn  an }
If discrete, define the joint pmf by
p(x1 , x2 , . . . , xn ) = P{X1 = x1 , X2 = x2 , . . . , Xn = xn }
If continuous, define the joint pdf by

Z Z
P{(X1 , X2 , . . . , Xn ) 2 C} = ··· f (x1 , x2 , . . . , xn )dx1 dx2 · · · dxn
(x1 ,x2 ,...,xn )2C

Law of the Unconscious Statistician (LOTUS) – 2D
LOTUS – 2D
If (X, Y) is discrete X
E[g(X, Y)] = g(xi , yj )p(xi , yj )
i,j
If X is continuous
Z 1 Z 1
E[g(X, Y)] = g(x, y)f (x, y)dxdy
1 1

Ei×-Y]=E _ t"¥'=±
Example: Suppose X and Y are independent Uniform⇠ [0, 1] random variables.
,
Find E[|X Y|].

Solution: IX. y ) /
f IX y
=
Lotus
-
Z 1Z Z 1Z 1
.
1
I
Q
E[|X Y|] = |x y|f (x, y)dxdy = |xy|dxdy
0 0 0 0
ZZ .
Z 1Z 1 Z 1
1 1
(x2 /2
=2 (x y)dxdy = 2 (x
☐ y)dxdy = 2 yx) dy =
3
¥171
0 y 0 .
y
x>y -
÷IÉx III c: Joint Distribution and Covariance Jiheng Zhang


-
Find E[|X Y|]. ✗
3
Solution:
Z 1Z 1 Z 1Z 1
E[|X Y|] = |x y|f (x, y)dxdy = |x y|dxdy 1×-41--2=5-3
0 0 0 0
ZZ Z 1Z 1 Z 1
1 1
=2 (x y)dxdy = 2 (x y)dxdy = 2 (x2 /2 yx) dy =
0 y 0 y 3
x>y
Let M = max(X, Y) and L = min(X, Y). Can you find E[M] and E[L]?
Solution: |X Y| = M L, E[|X Y|] = 00
E[M] E[L] = 1/3.
On the other hand, 0 E[M] +☐ E[L] = 1. So E[M] = 2/3 and E[L] = 1/3.
III c: Joint Distribution and Covariance E- ET Jiheng Zhang
Conditional Distributions
If (X, Y) is discrete, the conditional probability mass function of X given that
0 Y = yj is
pX|Y (xi |yj ) =
p(xi , yj )
pY (yj )
.
for all xi
If (X, Y) is continuous, the conditional probability density function of X given

that Y = y is
f (x, y)
fX|Y (x|y) = for all x
fY (y)

• Formula for0
Conditional Expectation
If (X, Y) is discrete, then the expectation of X given that Y = yj is
X
. = yj ] =
E[X|Y xi pX|Y (xi |yj )
xi
The expectation of X can be calculated as

X
E[X|Y = yj ]pY (yj )
yj
If (X, Y) is continuous, then the expectation of X given that Y = y is

Z 1
E[X|Y = y] = xfX|Y (x|y)dx
1
The expectation of X can be calculated as
Z 1
E[X|Y = y]fY (y)dy
1
Conditional Formula for Expectation

Example: A miner is trapped in a mine containing three doors. The first door leads to a tunnel that
takes him to safety after two hours of travel time. The second door leads to a tunnel that takes him to
safety after three hours of travel time. The third door leads to a tunnel that returns him to his mine
after five hours. Assuming that the miner is at all times equally likely to choose any one of these
doors, what is the expected length of time until the miner reaches safety?
Solution: Let X be the time until the miner reaches safety and Y denote the door he initially choose
E[X|Y = 1] = 2
E[X|Y = 2] = 3 n
E[X|Y = 3] = 5 + E[X]
-
So
1 1 1
E[X] = E[X|Y = 1] + E[X|Y = 2] + E[X|Y = 3]
3 3 3
Thus, E[X] = 5. =
2×1-3 +
30¥ +
(5+-6*7) §
Conditional Formula for Expectation

Find E[|X Y|]. 1 I 1- Ñ= .
< ✗ <
.
Solution: Let Z = |X Y|, we first compute E[Z|Y = y] for any 0  y  1.

Z Z Z
A
1 y 1
1
→
E[Z|Y = y] = |x y|fx (x)dx
☐= (y x)dx + (x y)dx = y2 y+
0 0 y 2
Then using the conditioning formula,

Z 1 Z 1
1
E[Z] = E[Z|Y = y]fY (y)dy = (y2 y + )dy = -
2
.
. .
-
0 0
-

Review of Variance
For any random variable X, the variance Var(X) is defined by
Var(X) = E[(X E[X])2 ]

=
foulX ,
X) =
Varix) = E [(X -
ETD ) ( X -
Et XD ]
Alternatively,
Var(X) = E[X 2 ] (E[X])2
For any constants a and b,
Var(aX + b) = a2 Var(X)
If X and Y are independent, then
Var(X + Y) = Var(X) + Var(Y).

Covariance
What if X and Y are not independent?

Let’s consider a special case where X = Y,
Var(X + X) = Var(2X) = 4Var(X)

Var(X) + Var(X) = 2Var(X)

Covariance
What if X and Y are not independent?

Let’s consider a special case where X = Y,
Var(X + X) = Var(2X) = 4Var(X)

Var(X) + Var(X) = 2Var(X)
Let µx and µy be the means of X and Y, respectively.
D EFINITION (C OVARIANCE )
The covariance of X and Y is defined by
Cov(X, Y) = E[(X µx )(Y µy )]

Covariance
Cov(X, Y) = E[XY µx Y µy X + µx µy ]
= E[XY] µx E[Y] µy E[X] + µx µy
linearity →
= E[XY] µx µy µy µx + µx µy
= E[XY] µx µy
= E[XY] E[X]E[Y]

Covariance
z-¥
Cov(X, Y) = E[XY µx Y µy X + µx µy ]
= E[XY] µx E[Y] µy E[X] + µx µy
✗+
= E[XY] µx µy µy µx + µx µy
= E[XY] µx µy
¥
= E[XY] E[X]E[Y]
Properties: ✗+ 2-
→
¥Ix
• Cov(X, Y) = Cov(Y, X)
• Cov(X, X) = Var(X)
linear
-41
↳ • Cov(aX, Y) = aCov(X, Y)
• Cov(X + Z, Y) = Cov(X, Y) + Cov(Z, Y)
n
X n
X
• Cov( Xi , Y) = Cov(Xi , Y)
III c: Joint Distribution and Covariance i=1 i=1 Jiheng Zhang
Covariance
G ENERAL C OVARIANCE F ORMULA
0 1
Xn m
X n X
X m
Cov @ Xi , A
Yj = Cov(Xi , Yj )
i=1 j=1 i=1 j=1
4) I 1 2 34 5
4
÷÷¥¥#¥
5
.
¥¥ tij ,

Covariance
G ENERAL C OVARIANCE F ORMULA
0 1
Xn m
X n X
X m
Cov @ Xi , A
Yj = Cov(Xi , Yj )
i=1 j=1 i=1 j=1
0 1 0 1
Xn m
X n
X m
X
Cov @ Xi , Yj A = Cov @Xi , Yj A
i=1 j=1 i=1 j=1
0 1
n
X m
X
= Cov @ Yj , Xi A
i=1 j=1
n X
X m
= Cov(Xi , Yj )
i=1 j=1
Covariance
Example: Find Cov(X1 + X2 + X3 , Y1 + Y2 ).

!
Cov(X1 , Y1 ), Cov(X2 , Y1 ), Cov(X3 , Y1 )
Cov(X1 , Y2 ), Cov(X2 , Y2 ), Cov(X3 , Y2 )
Sum up all six numbers in the above matrix.

The above matrix is also called covariance matrix.

Covariance
VARIANCE OF S UM
n
! n n X
n
X X X
Var Xi = Var(Xi ) + Cov (Xi , Xj )
i=1 i=1 i=1 j=1
j6=i
÷
" "
.
! 0 1
n
X n
X n
X
Var Xi = Cov @ Xi , Xj A
i=1 i=1 j=1
'0 1
Cov(X1 , X1 ), Cov(X2 , X1 ), · · · Cov(Xn , X1 )
B C
BCov(X1 , X2 ), Cov(X2 , X2 ), · · · Cov(Xn , X2 )C
B .. .. .. C
B .. C
@ . . . . A
Cov(X1 , Xn ), Cov(X2 , Xn ), · · · Cov(Xn , Xn )
Hypergeometric Random Variable – revisit

Suppose
X ⇠ Hypergeometric(N, M, n)
N M
i n i
P{X = i} = N+M
n
Label the n selected batteries by i = 1, 2, 3, . . . n

(
1 if battery i is acceptable
Xi =
0 otherwise
Then, the number of acceptable batteries

X n
X= Xi
i=1

Since each battery is picked from a pool of N + M,
N
P{Xi = 1} =
N+M
Are X1 , X2 , X3 , . . . independent?


Since each battery is picked from a pool of N + M,
N
P{Xi = 1} =
N+M
Are X1 , X2 , X3 , . . . independent?
Consider for i 6= j,
P{Xi = 1, Xj = 1} = P{Xi = 1}P{Xj = 1|Xi = 1}
N N 1
pc ,
=
N+MN+M 1 ☐
6= P{Xi = 1}P{Xj = 1}
It does not affect finding the expectation

Xn Xn
nN
E[X] = E[Xi ] = P{Xi = 1} =
III c: Joint Distribution and Covariancei=1
N+M Jiheng Zhang
i=1

For variance, we have to use
n
X X
Var(X) = Var(Xi ) + Cov(Xi , Xj )
i=1 i6=j


n
X X
i=1 i6=j
Var(Xi ) = P{Xi = 1} (P{Xi = 1})2

N M
= P{Xi = 1}(1 P{Xi = 1}) =
N+MN+M


n
X X
i=1 i6=j
Var(Xi ) = P{Xi = 1} (P{Xi = 1})2

N M
= P{Xi = 1}(1 P{Xi = 1}) =
N+MN+M
NM
Cov(Xi , Xj ) =
(N + M)2 (N +M 1)
✓ ◆
nNM n 1
Var(X) = 1
(N + M)2 N+M 1
Let-Y-xi.is/CEF7--PCElF)-pCF
Since we have computed
g
1 it Xi=×j=1
} ) P{X = 1|X = 1} =
j i
N 1
N+M 1
y= 0 OW
PlY=D=PlXIXj=I
Cov(Xi , Xj ) = E[Xi Xj ] E[Xi ]E[Xj ] F F-
@☐
-
N(N 1) N N
-
= ,, , .my,
(M + N)(M + N 1) M+NM+N
NM
= 2
(N + M) (N + M 1) =P(Xj=1IXi=D.p=i
for ¥÷
.
i=j
.
any
?⃝
Covariance
For the simple case with two random variables
Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y)
Council Covlx 4)
( Cook
,
,
,
✗1
,
loud

Covariance
For the simple case with two random variables
Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y)
C OVARIANCE BETWEEN I NDEPENDENT R ANDOM VARIABLES

Cov(X, Y) = 0
Since Cov(X, Y) = E[XY] E[X]E[Y] and we have shown that
E[XY] = E[X]E[Y]
when X and Y are independent.

Covariance
VARIANCE OF S UM WITH I NDEPENDENCE

Var(X + Y) = Var(X) + Var(Y)
If X1 , X2 , . . . , Xn are independent, then

n
! n
X X
Var Xi = Var(Xi )
i=1 i=1

Covariance
Example: Compute the variance of the number of heads resulting from 10 #a
independent tosses of a fair coin. Module
Method 1: Compute the distribution of X and use definition.
9
}
P
Method 2: Let X = 10 j=1 Ij , where
( d
"" ""
"
1 if the jth toss lands head
'
Ij =
0 if the jth toss lands tail
P10
We just need to find Var( j=1 Ij ).
1 1 1
Var(Ij ) = E[Ij2 ] (E[Ij ])2 = =
2 4 4
Thus
10
X 10
X 10
Var( Ij ) = Var(Ij ) =
4
III c: Joint Distribution and Covariance j=1 j=1 Jiheng Zhang
Covariance
f- ÷É④
Example: Compute the variance of the sum of 10 independent rolls of a fair die.
Let Xi be the outcome of the ith roll, we have that
10
! 10
X X
Var Xi = Var(Xi )
i=1 i=1
35 35 35
= + + ··· +
12 12 12
35 175
= 10 ⇥ =
12 6

Correlation
Covariance measures the strength of the relationship between X and Y.
If X and Y are independent, then Go ( too X

,
too 4) =
100 ✗ too Cool
Cov(X, Y) = 0
If Y = X, then
Cov(X, Y) = Cov(X, X) = Var(X) > 0
If Y = X, then
Cov(X, Y) = Cov(X, X) = Var(X) < 0

Correlation
The correlation, scaled covariance, is defined as
-
Cov(X, Y)
Corr(X, Y) = p
Var(X)Var(Y)
Come ax.BY ) =
Corr IX X) ¥1 =*÷E;¥¥
¥va*T-=I
,
=

Correlation
The correlation, scaled covariance, is defined as
Cov(X, Y)
Corr(X, Y) = p
Var(X)Var(Y)

Module IIIc - Annotated

Uploaded by

Copyright:

Available Formats

You might also like

Module IIIc - Annotated

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Module IIIc - Annotated

Uploaded by

Copyright:

Available Formats

III C :

J OINT D ISTRIBUTION AND C OVARIANCE

Joint PMF – Discrete Random Variables

Joint PMF – Discrete Random Variables

X: number of new batteries

The probability mass function p(i, j) = P{X = i, Y = j}

III c: Joint Distribution and Covariance Jiheng Zhang

TABLE 4.1 P{X = i, Y = j }

III c: Joint Distribution and Covariance Jiheng Zhang

Joint PDF – Continuous Random Variables

Using a more general notation,

for every set C of pairs of real numbers.

III c: Joint Distribution and Covariance Jiheng Zhang

An Illustration of Joint PDF

III c: Joint Distribution and Covariance Jiheng Zhang

An Illustration of Joint PDF

III c: Joint Distribution and Covariance Jiheng Zhang

Joint PDF – Continuous Random Variables

f (a, b) indicates how likely (X, Y) will be near (a, b)

III c: Joint Distribution and Covariance Jiheng Zhang

Joint PDF – Continuous Random Variables

P{X 2 A} = P{X 2 A, Y 2 ( 1, 1)}

III c: Joint Distribution and Covariance Jiheng Zhang

III c: Joint Distribution and Covariance Jiheng Zhang

III c: Joint Distribution and Covariance Jiheng Zhang

P{X 2 A, Y 2 B} = P{X 2 A}P{Y 2 B} How

III c: Joint Distribution and Covariance Jiheng Zhang

P{X 2 A, Y 2 B} = P{X 2 A}P{Y 2 B}

If true for ALL A and B, then X and Y are independent.

III c: Joint Distribution and Covariance Jiheng Zhang

Uniform Distribution on a Square

Uniform Distribution on a Disc

Suppose (X, Y) are uniformly distributed on a disc, then

PCY > of / ×> at ) 0

Chicken-egg Problem e-i-e-NH-e-NP-H-e-dP.eu't

By independence, we have that f (x, y) = e x e y , x, y > 0. =3 ✗< at ]

III c: Joint Distribution and Covariance Jiheng Zhang

Joint Distributions for Multiple Variables

If discrete, define the joint pmf by

If continuous, define the joint pdf by

III c: Joint Distribution and Covariance Jiheng Zhang

Law of the Unconscious Statistician (LOTUS) – 2D

III c: Joint Distribution and Covariance Jiheng Zhang

Law of the Unconscious Statistician (LOTUS) – 2D

Find E[|X Y|].

÷IÉx III c: Joint Distribution and Covariance Jiheng Zhang

Law of the Unconscious Statistician (LOTUS) – 2D

Example: Suppose X and Y are independent Uniform⇠ [0, 1] random variables.

If (X, Y) is discrete, the conditional probability mass function of X given that

If (X, Y) is continuous, the conditional probability density function of X given

III c: Joint Distribution and Covariance Jiheng Zhang

The expectation of X can be calculated as

If (X, Y) is continuous, then the expectation of X given that Y = y is

Conditional Formula for Expectation

Conditional Formula for Expectation

Example: Suppose X and Y are independent Uniform⇠ [0, 1] random variables.

Solution: Let Z = |X Y|, we first compute E[Z|Y = y] for any 0  y  1.

Then using the conditioning formula,