SIEO 3600 (IEOR Majors) Assignment #7

Introduction to Probability and Statistics March 9, 2010

Assignment #7– due March 9th, 2010

(Expectation, variance and covariance of random variables)

1. Ten balls are randomly chosen from an urn containing 17 white and 23 black balls. Let X
denote the number of white balls chosen.

(a) If everytime we put the chosen ball back to the urn (with replacement), compute E[X]
by defining appropriate indicator variables Xi , i = 1, . . . , 10 so that
10
X
X= Xi .
i=1

(b) If everytime we don’t put the ball back (without replacement), compute E[X] by

m
er as
defining appropriate indicator variables Yi , i = 1, . . . , 17 so that

co
eH w
17
X
X= Yi .

o.
i=1

rs e
ou urc
Solution:

(a) For i = 1, . . . , 10, let

o


1 if the ith pick is a white ball,
aC s

Xi =
0 otherwise.
vi y re

With replacement, everything we have the same number of white and black balls.
Hence,
P10 E(Xi ) = P(Xi = 1) = 17/40. Linearity of expectation implies that E(X) =
ed d

i=1 E(X i ) = 10 · E(Xi ) = 17/4.

ar stu

(b) With all 17 white balls labeled with j = 1, . . . , 10, let


1 if white ball number j is picked,
Yj =
0 otherwise.
sh is

Without replacement, this is equivalent to picking 10 balls out of 40 at one time. Note
Th

that

E(Yj ) = P(Yj = 1) = P(white ball number j is picked)

total number of ways to pick 10 balls out of 40 with white ball number j included
=
total number of ways to pick 10 balls out of 40
39


9 1
= 40
= .
10
4

Hence, linearity of expectation implies that E(X) = 10

P
j=1 E(Yj ) = 17 · E(Yj ) = 17/4.
Although the polices in (a) and (b) are different (with or with replacement), the answers
are the same. (Unbelievable!)

https://www.coursehero.com/file/6444234/Hw7Sol/
 2 SIEO 3600, Assignment #7

2. Let X be a continuous random variable with CDF F and PDF f , then define its median as
the value m such that F (m) = 1/2. (Think about the sample median of a data set). The
median, like the mean, is important in predicting the value of a random variable. It can
be shown that the mean E(x) is the best predictor from the point of view of minimizing the
expected value of the square of the error. Prove that the median is the best predictor if one
wants to minimize the expected value of the absolution error. That is, E[|X − c|] is minimized
when c is the median of X.
Hint: Write
Z ∞
E[|X − c|] = |x − c|f (x)dx
Z−∞
c Z ∞
= |x − c|f (x)dx + |x − c|f (x)dx
−∞ c
Z c Z ∞
= (c − x)f (x)dx + (x − c)f (x)dx

m
−∞ c

er as
Z c Z ∞
= c F (c) − x f (x)dx + x f (x)dx − c[1 − F (c)],

co
−∞ c

eH w
and then choose the best c to minimize E[|X − c|].

o.
rs e
ou urc
Solutions Following the hint, we take the derivative of the right hand side with respect to
c set the derivative to be 0:

0 = F (c) + cf (c) − cf (c) − cf (c) − [1 − F (c)] + cf (c),

o
aC s

which yields F (c) = 12 . Therefore, we have c = m is the unique solution. Finally, since the
vi y re

second derivative is f (m) > 0 we have that this is indeed the minimizing point.

3. A community consists of 100 married couples (200 people). If during a given year 50 of the
members of the community die, what is the expected number of marriages that remain intact?
ed d

Hint: Define random variables X1 , . . . , X100 such that

ar stu


1, if neither member of couple i dies
Xi =
0, otherwise
sh is

Solution: For i = 1, ·100, let

Th


1 if neither member of couple i dies,
Xi =
0 otherwise.

Since

E(Xi ) = P(neither member of couple i dies)

total number of ways to choose 50 out of 198
=
total number of ways to choose 50 out of 200
198


50 149 · 150
= 200
= .
50
199 · 200
P100
Linearity of expectation implies that E(X) = i=1 E(Xi ) = 100 · E(X1 ) = 56.2.

https://www.coursehero.com/file/6444234/Hw7Sol/
 SIEO 3600, Assignment #7 3

4. (continued with Problem 1 in HW6) X and Y are two continuous random variables with joint
probability density function

10xy 2 , 0 < x < y < 1;


fX,Y (x, y) =
0, otherwise.

Compute the covariance cov(X, Y ) and the correlation coefficient ρ(X, Y ).

5
Solutions: Using the density we found in the last homework we get that E[X] = 9 and
5
E[X 2 ] = 14 while E[Y ] = 56 and E[Y 2 ] = 57 . Next, we have that

Z 1Z 1
10
E[XY ] = xy10xy 2 dydx = ,

m
21

er as
0 x
5 25

co
var(X) = − ,
14 81

eH w
5 25
var(X) = − .

o.
7 36
rs e
ou urc
Hence, we have that

10 25
cov(X, Y ) = − ,
o

21 54
10/21 − 25/54
aC s

cov(X, Y )
ρ(X, Y ) = =p = 0.426.
vi y re

p p p
var(X) var(Y ) 5/14 − 25/81 5/7 − 25/36

5. A product is classified according to the number of defects it contains and the factory that
ed d

produces it. Let X and Y be the random variables that represent the number of defects per
ar stu

unit (taking on possible values 0, 1, 2, or 3) and the factory number (taking on possible values
1 or 2), respectively. The entries in the table represent the joint probability mass function of
a randomly chosen product.
sh is

X ↓, Y → 1 2
Th

0 1/8 1/16
1 1/16 1/16
2 3/16 1/8
3 1/8 1/4

(a) Find the marginal probability mass functions of X and Y .

(b) Find E[X], E[Y ], var(X), var(Y ), and cov(X, Y ).

Solution:

https://www.coursehero.com/file/6444234/Hw7Sol/
 4 SIEO 3600, Assignment #7

(a) The PMF of X is

2
X
pX (0) = PX,Y (0, j) = 1/8 + 1/16,
j=1
2
X
pX (1) = PX,Y (1, j) = 1/16 + 1/16,
j=1
2
X
pX (2) = PX,Y (2, j) = 3/16 + 1/8,
j=1
2
X
pX (3) = PX,Y (3, j) = 1/8 + 1/4.
j=1

m
The PMF of Y is

er as
3

co
X
pY (1) = PX,Y (i, 1) = 1/8 + 1/16 + 3/16 + 1/8,

eH w
i=0

o.
3

rs e pY (2) =
X
PX,Y (i, 2) = 1/16 + 1/16 + 1/8 + 1/4.
ou urc
i=0

(b) The expectation of X is

o

3
aC s

X
E(X) = i PX (i) = 1.875,
vi y re

i=0
X2
E(Y ) = j PY (j) = 1.5,
j=1
ed d

3
X
var(X) = E(X 2 ) − E(X)2 = i2 PX (i) − E(X)2 = 1.234,
ar stu

i=0
X2
var(Y ) = E(Y 2 ) − E(Y )2 = j 2 PY (j) − E(Y )2 = 0.25,
sh is

j=1
3 X
2
Th

X
cov(X, Y ) = E(XY ) − E(X)E(Y ) = ij PX,Y (i, j) − E(X)E(Y ) = 0.125.
i=0 j=1

p p
6. Let X and Y have standard deviations σX ≡ var(X) and σY ≡ var(Y ). Show that the
correlation coefficient ρ(X, Y ) satisfies

(a) −1 ≤ ρ(X, Y )
Hint: Start with 0 ≤ var(X/σX + Y /σY ).
(b) ρ(X, Y ) ≤ 1
Hint: Start with 0 ≤ var(X/σX − Y /σY ).

from which conclude that −1 ≤ ρ(X, Y ) ≤ 1.

https://www.coursehero.com/file/6444234/Hw7Sol/
 SIEO 3600, Assignment #7 5

Solutions:

(a) Following the hint, we have

X Y X Y X Y
0 ≤ var( + ) = var( ) + var( ) + 2 cov( , )
σX σY σX σY σX σY
var(X) var(X) cov(X, Y )
= 2 + 2 +2
σX σX σX σY
= 1 + 1 + 2 ρ(X, Y ),

which implies that ρ(X, Y ) ≥ −1.

(b) Following the hint, we have

X Y X Y X Y
0 ≤ var( − ) = var( ) + var(− ) + 2 cov( ,− )
σX σY σX σY σX σY

m
var(X) var(X) cov(X, Y )

er as
= 2 + 2 −2
σX σX σX σY

co
eH w
= 1 + 1 − 2 ρ(X, Y ),

o.
which implies that ρ(X, Y ) ≤ 1.
rs e
ou urc
7. Consider n indepenent trials, each of which results in any of the outcomes i, i = 1, 2, 3, e.g.,
the defendent is guilty for i = 1, not guilty for i = 2, jury cannot reach
P3a verdict for i = 3
(Law and order is my favorate drama), with probabilities p1 , p2 , p3 , i=1 pi = 1. Let Ni
o

denote the number of trials that result in outcome i.

aC s

(a) Show that cov(N1 , N2 ) = −np1 p2 .

vi y re

(b) Explain why it is intuitive that this covariance is negative.

Hint: For i = 1, . . . , n, let

ed d


1 if trial i results in outcome 1 (guilty)
Xi =
ar stu

0 if trial i does not results in outcome 1 (not guilty or trial dismissed)


1 if trial i results in outcome 2 (not guilty)
Yi =
0 if trial i does not results in outcome 2 (guilty or trial dismissed)
sh is

Note that
Th

n
X n
X
N1 = Xi , N2 = Yi .
i=1 i=1

Then try to compute the coviance of N1 and N2 using those properties discussed in class.

Solution: Define Xi and Yj as above. Note that independence implies that cov(Xi , Yj ) = 0
for any i 6= j. Moreover,

cov((Xi , Yi ) = E(Xi Yi ) − E(Xi )E(Yi )

= [(1 · 0)p1 + (0 · 1)p2 + (0 · 0)p3 ] − [1 · p1 + 0 · (p2 + p3 )][1 · p2 + 0 · (p1 + p3 )]
= −p1 p2 .

https://www.coursehero.com/file/6444234/Hw7Sol/
 6 SIEO 3600, Assignment #7

Hence, we have by linearity of covariance,

n X
X n
cov(N1 , N2 ) = cov(Xi , Yj )
i=1 j=1
Xn XX
= cov(Xi , Yi ) + cov(Xi , Yj )
i=1 i6=j
n
X
= cov(Xi , Yi ) + 0
i=1
= −n p1 p2 .

8. Let X1 and X2 have the same cumulative distribution function, show that

m
cov(X1 − X2 , X1 + X2 ) = 0

er as
co
Note that independence is not being assumed.

eH w
o.
Solution:
rs e
We have by linearity of covariance,
ou urc
cov(X1 − X2 , X1 + X2 ) = cov(X1 − X2 , X1 ) + cov(X1 − X2 , X2 )
= [cov(X1 , X1 ) + cov(X2 , −X1 )] + [cov(X1 , X2 ) + cov(X2 , −X2 )]
o

= [cov(X1 , X1 ) − cov(X2 , X1 )] + [cov(X1 , X2 ) − cov(X2 , X2 )]

aC s

= var(X1 ) − var(X2 ) = 0,
vi y re

where the last step holds since having the same distributions implies having the same vari-
ances.
ed d

9. (continued with Problem 9 of HW6) Consider the letter-envelop matching problem where n
ar stu

letters are put into n envelopes at random. Let Y be the number of letters that are placed
in the correct envelopes. Recall that we defined

1 if letter i is put in its correct envelop
Xi =
sh is

0 otherwise
Th

for i = 1, 2, . . . , n. We have shown that E[Xi ] = 1/n and E[Y ] = E[ ni=1 Xi ] = 1 in HW6.
P
Now

(a) Compute cov(Xi , Xj ) for i 6= j.

Hint: it suffices to find E(Xi Xj ) since cov(Xi , Xj ) = E(Xi Xj ) − E(Xi )E(Xj ). Note that
Xi Xj = 1 if both letter i and j are in their correct envelops; Xi Xj = 0 otherwise.
(b) Use (a) to compute var(Y ).
Hint: Recall that we have discussed in class that
n
X n
X XX
var( Xi ) = var(Xi ) + 2 cov(Xi , Xj ).
i=1 i=1 i<j

https://www.coursehero.com/file/6444234/Hw7Sol/
 SIEO 3600, Assignment #7 7

Solution:

(a) Note that for i 6= j,

E(Xi Xj ) = P(Xi = 1, Xj = 1) = P(both letter i and j are in their correct envelops)

total number of ways to put n − 2 letters into n − 2 envelops
=
total number of ways to put n letters into n envelops
n−2
Pn−2 (n − 2)! 1
= = = .
Pnn n! n(n − 1)
1 1 1
Hence, cov(Xi , Xj ) = E(Xi Xj ) − E(Xi )E(Xj ) = n(n−1) − n2
= n2 (n−1)
.
(b) Since P(Xi = 1) = P(letter i is in its correct envelop) = 1/n from HW 6, we have

var(Xi ) = E(Xi2 ) − E(Xi )2

m
er as
1
= 12 · P(Xi = 1) −
n2

co
eH w
1 1
= − 2.
n n

o.
rs e
Hence, we have
ou urc
n
X n
X XX
var( Xi ) = var(Xi ) + 2 cov(Xi , Xj )
i=1 i=1 i<j
o

 
1 1 n 1
= n · ( − 2) − 2
aC s

2
= 1.
n n 2 n (n − 1)
vi y re

10. From past experience, a professor knows that the test score of a student taking her final
examination is a random variable with mean 75.
ed d

(a) Give an upper bound to the probability that a student’s test score will exceed 85.
ar stu

Suppose in addition the professor knows that the variance of a student’s test score is equal
to 25.
sh is

(b) What can be said about the probability that a student will score between 65 and 85?
Th

(c) How many students would have to take the examination so as to ensure, with probability
at least .9, that the class average would be within 5 of 75?

Solutions:
E(X) 75
(a) By Markov inequality, P(X > 85) ≤ 85 = 85 .
(b) By Chebyshev’s inequality, we have

P(65 ≤ X ≤ 85) = P(−10 ≤ X − 10 ≤ 10) = P(|X − 75| ≤ 10) = P(|X − E(X)| ≤ 10)
var(X) 25 3
= = 1 − P(|X − E(X)| > 10) ≥ 1 − 2
=1− = .
10 100 4

https://www.coursehero.com/file/6444234/Hw7Sol/
 8 SIEO 3600, Assignment #7

(c) Suppose we need n students to take the exam. Let X̄ ≡ n1 ni=1 Xi be the class average,
P

which is a random variable with mean E(X̄) = 75 and variance var(X) = var(X n
1)
= 25
n.
Applying chebyshev’s inequality on random variable X̄, we have

var(X̄) 1
P(|X̄ − E(X)| > 5) = P(|X̄ − 75| > 5) ≤ = ,
25 n
which implies that P(|X̄ − 75| ≤ 5) > 1 − 1/n. In order for this probability to be at least
0.9, we require 1 − 1/n > 0.9, that is n > 10.

m
er as
co
eH w
o.
rs e
ou urc
o
aC s
vi y re
ed d
ar stu
sh is
Th

https://www.coursehero.com/file/6444234/Hw7Sol/

Powered by TCPDF (www.tcpdf.org)