EE: Modern Discrete Probability (Due: 15/02/22)

Homework Assignment #3

Instructor: Ohad Elishco

• Please submit the assignment via Moodle or by email.

Problem 1: Total variation norm (25 points)

Let P be a transition probability matrix for an irreducible, aperiodic Markov chain on a finite state space
S with stationary distribution π. Recall the definitions we saw in class

d(t) = sup kP t (x, ·) − πkT V , ¯ = sup kP t (x, ·) − P t (y, ·)kT V .

d(t)
x∈S x,y∈S

Recall also that if (X, Y ) is a coupling of µ, ν then kµ − νkT V 6 Pr(X 6= Y ). Use coupling (or any other
¯ + s) 6 d(t)
¯ d(s).
¯ 1/t
method) to show that d(t Deduce that the limit ρ = limt→∞ (d(t)) exists.

¯ 6 2d(t). You may also use Fekete’s lemma: a

Hint: you may use the inequalities showed in class d(t) 6 d(t)
subadditive function f is a function such that f (t + s) 6 f (t) + f (s) for all s, t > 0. Fekete’s lemma states that
if f is subadditive, then the limit limt→∞ 1t f (t) exists.

Remark 0.1 It is possible to show that if P is reversible, then ρ = γ ∗ where γ ∗ = max {|λ| | λ is an eigenvalue 6= 1}.
This means that the rate at which Pt converges to stationarity is exponential γ t where γ is the second largest
eigenvalue of P .

Problem 2: A gas experiment (30 points)

Let us consider the following physics experiment. Assume we have two chambers A and B, each one filled
with a different gas such that every chamber contains a different number of gas particles. Assume that initially
there is a partition between the chambers and that the partition is removed at time 0. At each subsequence
discrete stage in time, a particle from the population of the two chambers is chosen at random. The chosen
particle switches sides. We would like to study the behaviour of the number of particles in each chamber.

1. (5 points) Consider the urn model given by

 
−1 1
M= .
1 −1

Notice that the population of this urn does not change in size over time. Explain how this urn model can
be used to model the gas experiment above.
2. (10 points) Consider the urn model M and assume we begin with one black ball and one white ball. The
state of the urn at time n (its composition) is denoted by Sn , n > 0 and can be described as a Markov
chain with 3 states. Analyze the Markov chain Sn - write the graph representation (with 3 states), write
the transition probabilities matrix, and find the stationary (binomial) distribution.
3. (5 points) Does the system above converge to the stationary distribution? Explain your answer.
4. (10 points) Suppose we start with W0 ∼ Bin(τ, 21 ) white balls and B0 = τ − W0 black balls. Let Wn
denote the number of white balls after n draws. Show that
 
1
Wn → Bin τ, .
2

Hint: There are two approaches here:

1
– Homework Assignment #3 2

(a) Show straightforward that the binomial distribution is the stationary distribution of the general
Markov chain and explain why this is enough.
(b) Try to write Pr(Wn = k) as a function of Pr(Wn−1 = k+1) and Pr(Wn−1 = k−1) and use the fact that
a stationary distribution exists (Why?). Assume the Markov chain is in its stationary distribution
and explain why we can write Pr(Wn = j) as pj (disregarding the n). Obtain a recurrence formula
and write pk as a function of k and p0 = Pr(W0 = 0). Obtain the wanted result.

Problem 3: Martingale transform (20 points)

Let {Xn }n>0 be a stochastic process where Xi obtain values on a countable set X . The sequence of real-
valued random variables {Hn }n>1 is called an X0n predictable process if for all n > 1,

Hn = gn (X0n−1 )

for some function gn : X n → R. Let {Yn } be another sequence of real-valued random variables. The sequence
{(H ◦ Y )n }n>0 defined by
n
X
(H ◦ Y )0 = 0, (H ◦ Y )n = (Yk − Yk−1 )Hk (n > 1)
k=1

is called the transform of {Yn } by {Hn }.

Prove the following (You may assume that E[|Yn |] < ∞).
1. (5 points) Let {Yn } be sub-martingale with respect to X0n . And let {Hn } be a bounded, non-negative,
X0n -predictable process. Then {(H ◦ Y )n }n>1 is a sub-martingale with respect to X0n .
2. (5 points) Let {Yn } be martingale with respect to X0n . And let {Hn } be a bounded X0n -predictable
process. Then {(H ◦ Y )n }n>1 is a martingale with respect to X0n .
3. (10 points) Let {Yn } be martingale with respect to X0n and let τ be a stopping time for the process {Xn }.
Let n ∧ τ = min {n, τ } and show that Yn∧τ is martingale with respect to X0n .
Hint: try to define Hk = 1k6τ and apply the previous sections.

Problem 4: (15 points)

Consider the examples that were given in class for the sequence of i.i.d. random {−1, 1} variables, the m
balls and N bins, and the gambling strategy (examples VII.3 in the lecture notes). Show the following.
1. (5 points) For the sequence of i.i.d. random variables, denote by Xn the number of +1s after n variables
were selected. Show that for λ > 0,
λ2
Pr(|Xn − E[Xn ]| > λ) 6 2e− n .

Hint: It may be easier to think of it as an experiment with n tosses of coins where X is the total number
2
of HEADS and we need to show that Pr(|X − E[X]| > λ) 6 2e−λ /n . Notice that as written above, the
process is not martingale. Recall the analysis made in class for the Lipschitz condition - we created there
a ”new process” which is martingale.
2. (5 points) For the balls and bins scheme, let Z denote the number of empty bins. Show that
λ2
Pr(|Z − E[Z]| > λ) 6 2e− m .

Hint: Again, we need to use a different process to what we have done in class. Use a similar method to
the one used in the previous part.
3. (5 points) For the gambling strategy, assume the gambler now can borrow whatever amount he needs so
the game never stops and bn+1 (Y0n ) can be any value. In return for this borrowing privilege, the functions
bn are bounded by M for all n > 1. Recall that Y0 = a. Prove that
λ2
Pr(|Yn − a| > λ) 6 2e− 2nM 2 .
– Homework Assignment #3 3

Problem 5: (10 points)

Let {Xn }n>0 be a stochastic process taking values in X . Let f : X N → R be a function such that E[|f |] < ∞.
1. Define {Mn }n>0 as Mn = E[f (X) | X0n ] and show {Mn } is martingale with respect to X0n . Do not forget
to show that Mn is a function of X0n and that for every n, E[|Mn |] < ∞.
Hint: you may use Jensen inequality for conditional expectation: if φ is a convex function then φ(E[Y | C]) 6
E[φ(Y ) | C].
2. Use martingale convergence theorem to deduce Kolmogorov’s zero-one law: Let A be an event depending
on infinite number of variables Xi , i.e., does not depend on any finite number of Xi s. Define 1A as the
indicator function of the event A and define the corresponding Mn . use martingale convergence theorem
to deduce that Mn → 1A in probability which implies that Pr(A) ∈ {0, 1}.
Hint: To use the convergence theorem, do not forget to show that supn E[|Mn |] < ∞ (this is not the same
as ∀n, E[|Mn |] < ∞. You can use Jensen inequality again).