EE278 Statistical Signal Processing Stanford, Autumn 2022

Midterm Solutions

• Please sign the honor code.

• The exam duration is 80 mins.

• Start a new page for every problem.

• The only allowable aids are a double-sided sheet of notes and a calculator.

• The total number of points is 100, but questions have different number of points.

Good luck!

1
1. Small questions
 
X1
a) Let X = X2  where X1 , X2 , X3 are independent continuous random variables.
X3
Express each of the following quantities in terms of the cumulative distribution
functions FX1 (x1 ), FX2 (x2 ) and FX3 (x3 ).
i. (5 points) The cumulative distribution function of the vector X, i.e. Fx (x) =
FX1 ,X2 ,X3 (x1 , x2 , x3 ).
ii. (5 points) Pr |X1 | ≤ 5, X2 < 4, X33 > 8 .


iii. (5 points) Pr {min(X1 , X2 , X3 ) ≤ 2}.

b) (5 points) Let X and Y be two independent random variables with moment gen-
erating functions MX (s) and MY (s) respectively. Let Z = aX + bY + c for some
real numbers a, b and c. Express the moment generating function of Z in terms of
MX (s) and MY (s).
c) (10 points) A drunkard executes a “random walk” in the following way: Each
minute he takes a step north or south, with probability 21 each, and his successive
step directions are independent. His step length is 0.5 m. Use the central limit
theorem to approximate the probability distribution of his location after 1h.

2
2. Let X1 , X2 , X3 be independent random variables with unit variance and zero mean, and

Y1 = X1 , Y2 = X1 − X2 Y3 = X1 − X2 + X3 .

Let X and Y denote the corresponding vectors containing these random variables as
their entries.
a) (5 points) Find a matrix A, such that Y = AX.
b) (5 points) Find the covariance matrix of Y.
c) (10 points) Find a matrix B such that W = BY and W has identity covariance
matrix.
d) (10 points) Assume we want to approximate Y by Yapp = Z1 q1 where Z1 = qT1 Y
 
0√
and q1 = 1/√2. What is the approximation error E[||Y − Yapp ||2 ]?
1/ 2

3
3. You have a six-sided die and you suspect that it is biased towards six. You decide to
estimate the probability of six, P (X = 6), by rolling the die n times and noting the
outcome of each roll.
a) (10 points) Suggest a way to estimate P (X = 6) from the outcome of the n rolls,
and argue that your estimate converges to P (X = 6) when n → ∞.
b) (5 points) Find n to guarantee that your estimate is within 0.2 of the true probability
P (X = 6) with probability at least %99.93 by using Chebyshev’s inequality. Your
answer should be an actual number.
c) (10 points) Repeat the previous part by using Hoeffding’s inequality. See below for
the statement of the Hoeffding’s inequality.
d) (15 points) Assume now that you are given a box of 10 dice. You suspect that each
of them is biased possibly with a different bias and you would like to choose the die
with the largest bias towards six, i.e. largest P (X = 6). You decide to roll each die
100 times, and you observe that one of the dice results in six in all 100 out of 100
trials, while all the remaining ones result in at most 60 sixes out of 100 trials. You
choose the die that gave you six in all 100 out of 100 rolls. Bound the probability
that you chose the wrong die.

Hoeffding’s inequality: Let X1 , X2 , . . . , Xn be a sequence of i.i.d. random vari-

1 Pn
ables with mean µ and a ≤ Xi ≤ b. Then for any t ≥ 0, X̄n = n i=1 Xi , we
have
2 2
P (|X̄n − µ| ≥ t) ≤ 2e−2nt /(b−a) .