MATH-F302 - Probabilités II

Exercise sheet 8 - April 30th, 2018

Exercise 1. Let p, q, r > 0 be such that p + q + r = 1. Let P be the stochastic matrix

 
r 1−r 0 ··· 0
... .. 
 p r q . 


P = 
 . . . . . . .
.

 0 . . . . 

 .
 ..

p r q 
0 ··· 0 1−r r

of size N + 1.
(a) Determine the Markov chain {Xn , n ≥ 1} associated to P as a simple random
walk on {0, . . . , N } with holding/reflecting boundary conditions at 0 and N. What is
the difference with the Markov chain of Exercise 3 in Sheet 7?
(b) Compute the unique probability vector π ∈ RN +1 such that

πP = π.

and that its value does not depend on r when p = q.

(c) Show that P satisfies the conditions of the particular case of Gershgorin’s theorem
which was handled in the lecture, and deduce that
d
Xn −→ π as n → ∞

for any entrance law X0 ∼ µ.

Exercise 2. Let {pn , n ≥ 0} be a sequence in ]0, 1[. Consider the homogeneous

Markov chain {Xn , n ≥ 0} valued in N whose transition matrix is given by

··· ···
 
q0 p0 0 0 0

 q1 0 p1 0 ··· 0 ··· 

 q2 0 0 p2 ··· 0 ··· 
P = 
 .. .. .. .. ...

. . . . 0 ··· 


qn 0 0 0 ··· pn ··· 
 

.. .. .. .. .. .. ..
. . . . . . .

with qn = 1 − pn pour tout n. In other words, we have

Pij = Pi [X1 = j] = P[X1 = j | X0 = i] = P[Xn+1 = j | Xn = i]

1
 Q∞ Qk−1
for every i, j, n ≥ 0. Set β0 = 1, β∞ = i=0 pi et βk = i=0 pi for every k ≥ 1.

(a) Show that for every i, j ≥ 0 there exists n0 ≥ 1 such that Pnij > 0 for every n ≥ n0 .
Deduce that {Xn , n ≥ 0} is irreducible.
(b) Set τj = inf{n ≥ 1, Xn = j} for the hitting time of the state j ≥ 0. Show that
for every i, k ≥ 0 one has
βi+k
Pi [τ0 > k] = ·
βi
Deduce that {Xn , n ≥ 0} is recurrent if and only if β∞ = 0.

(c) Suppose {Xn , n ≥ 0} is recurrent. Show that it has an invariant probability if

and only if X
βk < ∞
k≥1

and compute the invariant probability, which is unique, under this assumption.

Exercise 3. Let P be a transition matrix on N×N such that there exists a probability
p on N and a constante c > 0 such that

P (x, y) ≥ cp(y) for every x, y ∈ N.

Let l1 (N) be the set of absolutely summable sequences endowed with the norm
X
||u|| = |un |
n≥0

We will identify every probability on N as an element of l1 (N) with norm 1 and non
negative components.

(a) Show that c ≤ 1. What happens if c = 1 ? In the following, we will assume c < 1.
P
(b) Let u ∈ l1 (N) be such that n≥0 un = 0. Show that ||uP || ≤ (1 − c)||u||.
(c) Show that if µ, ν are two probabilities on N, then

||µP − νP || ≤ (1 − c)||µ − ν||.

(d) Deduce that if P admits an invariant probability, then it is unique.

(e) Show that if µ is any probability on N, the sequence {µP n , n ≥ 0} is a Cauchy
sequence on l1 (N). Deduce that P has a unique invariant probability π and that

||µP n − π|| ≤ Cρn

for some explicit constants C > 0 and ρ ∈ (0, 1).