Alexander Stolyar, ELEN E6711.001.2011.

3
MID-TERM EXAM
Due: November 1, 11:59 pm;
email PDF to eeliuchang@gmail.com,
cc: as4007@columbia.edu
Please put E6711 assignment at the beginning of the email subject line.
1
Lawler, ex. 1.7.
2
Xn is a discrete-time Markov chain, with

q0 q1

qm q0

P =

q2 q3
q1 q2

the transition probability matrix P equal to

q2 . . . qm

q1 . . . qm1

q4 . . . q1
q3 . . . q0

with q0 > 0 and qm > 0.

(a) Does the limn P n exist, and what is it equal to?
(b) What are the exact additional conditions under which this chain is reversible (with respect
to its stationary distribution)?
3
Consider the discrete-time Markov chain with state space {0, 1, 2, ...}, with transition
probabilities p(k, k 1) = 0.7 0.6/k, p(k, k + 2) = 0.3 + 0.6/k for k 1, and p(0, 10) = 1.
Is this chain positive recurrent?
4
Consider the graph consisting of 3 legs, M segments each, connected at the central node
O. (See Figure 1.) Let Xn be the simple random walk on with absorbtion at end-vertices
A, B, C: once the process gets there, it stays there.
(a) Find the probability that the process starting at the node k steps from O in the
direction of A, will be eventually absorbed at A.
(b) Find the probability that the process starting at the node k steps from O in the direction
of B, will be eventually absorbed at A.
1

Figure 1: Example with M = 5.

5
Lawler, ex. 3.3.