Exam: Markov theory 30/5 2008 (translated from Swedish)

Examiner: Torgny Lindvall

1. A particle makes a random walk between the corners of a regular (five-sided) pentagon: it
always walks to one of the closest two, with probability 1/2 for each. We number the corners 1,
2, 3, 4 and 5, clockwise. Assume that the particular is at corner 1 at time 0. Determine the
probability that it returns to the starting point without having visited corner 4.
2. Cars arrive to each entrance of a 1 km long car tunnel as two independent Poisson processes
with equal rates: 1 car/minute. The cars all run at velocity 60 km/hour. Determine the
probability that at a given time point there is at most 3 cars in the tunnel.
3. We consider a random walk on the set , where a step to the right has probability ,
, and a step to the left has probability , except for state 0: here the walk
stops with probability . For which values of is there a stationary distribution, and what does it
look like? Is this Markov chain ergodic for these values of ?
4. We study an -queueing system, however with the deviation from the standard model
that we have queue aversion: a customer that arrives to a system with customers present
joins the system with probability , where as . Show that there always
exists a stationary distribution for the the birth-death process , where is the number
of customers in the system at time .
5. Describe how to, with the coupling method, prove ergodicity for a Markov chain which is
irreducible, aperiodic, and with a finite state space.
6. We analyze an on-off process, that is a Markov process with state space . Let this process
have transition rates , . Use Kolmogorov’s forward equation to exactly
determine , , in terms of and .

TURN FOR BRIEF SOLUTIONS!

1. For let where is the position of the particle after
steps. Let | ; we want to determine . The usual method of conditioning
on gives the equations , , ,
. It is easily seen that .
2. The number of cars in the tunnel at a given time point is the sum of two independent Poisson-
distributed variables with parameter 1, since the cars drive with velocity 1 km/min. This sum is
Poisson-distributed med parameter 2, and thus the desired probability is ∑ .
3. –
4. Let and be the rates for arrival and service time, respectively. In our modified -
system we get birth rates (arrival rates) for . The death rates (leaving rates) are
all . We know that if ∑ , we have a stationary distribution. To
show this convergence, take big enough such that for . Now it is
easy to use dominated convergence and dominate ∑ with
∑ ( ) , and we are done.
5. –
6. Kolmogorov’s forward equation gives:

Let and for simplicity. Thus we have the differential equation

, which gives , which in turn gives (note: )
. We can conclude: