UNIVERSITY OF ZIMBABWE

HSTS416
BSc Honours Level 4 in Statistics

STOCHASTIC PROCESSES I

November/December 2018
Time : 2 hours

Candidates should attempt all questions in Section A and TWO Questions from Section B.
Marks will be allocated as indicated.

SECTION A (40 marks)

Candidates may attempt ALL questions being careful to number them A1 to A4.

A1. Define the following terms:

(a) a stochastic process, [1]
(b) a Markov chain, [1]
(c) transient state, [1]
(d) a recurrent state, [1]
(e) counting process, [2]
(f) memorylessness, [2]
(g) a pure birth process, and [2]
(h) a time reversible Markov chain. [2]

A2. (a) If for all s ≥ 0, t ≥ 0,

∞
X
Pij (t + s) = Pik (t)Pkj (s),
k=0
prove these Chapman-Kolmogorov equations remembering that for the process to
go from state i to state j in time t + s it must be somewhere at time t . [7]
(b) Specify the classes for the following Markov chains and determine whether they
are transient or recurrent.
(i)

1 1

2
0 2
0 0
1 1 1
0 0

4 2 4

1 1

P=
2
0 2
0 0
1 1

0 0 0 2 2


1 1
0 0 0


2 2

[5]

page 1 of 4
 HSTS416/

(ii)

0 0 0 1

0 0 0 1
P=

1 1
0 0


2 2
0 0 1 0

[4]

A3. There are two types of claims that are made to an insurance company. Let Ni (t) denote
the number of type i claims made by t, and suppose that {N1 (t), t ≥ 0} and
{N2 (t), t ≥ 0} are independent Poisson processes with rates λ1 = 10, λ2 = 1. The
amounts of successive type 1 claims are independent exponential random variables
with mean $1000 whereas the amounts from type 2 claims are independent exponen-
tial random variables with mean $5000. A claim has just been received; what is the
probability it is a type 1 claim? [7]

A4. Describe the birth and death process represented by the Immigration-Emigration
Model. [5]

SECTION B (60 marks)

Candidates may attempt TWO questions being careful to number them B5 to B7.

B5. (a) Show that if state i is recurrent and state i communicates with state j, then state
j is recurrent. [4]
(b) Suppose that sociologists want to determine the proportion of society that has
an upper, middle or lower class occupation. Assume that transitions between
social classes of successive generations in a family can be regarded as transitions
of a Markov chain and that the occupation of a child depends only on his or her
parent’s occupation. Let us suppose that such a model is appropriate and that
the transition probability matrix is given by,


.45 .48 .07
P=
.05 .70 .25

.01 .50 .49

Determine the proportion in the lower, middle and upper class using the limiting
probabilities [12]
(c) Let X1 , X2 , · · · , Xn be independent identically distributed exponential random
variables with mean 1/λ. Using mathematical induction show that
X1 + X2 + X3 + · · · + Xn has a gamma distribution with parameters n and λ. [6]

page 2 of 4
 HSTS416/

(d) Consider a post office that is run by two clerks. Suppose that when Mr Smith
enters the system he discovers that Mr Jones is being served by one of the clerks
and Mr Brown by the other. Suppose also that Mr Smith is told that his ser-
vice will begin as soon as either Jones or Brown leaves. If server i serves at an
exponential rate λi , i = 1, 2, show that
!2 !2
λ1 λ2
P {Smith is not last} = +
λ1 + λ2 λ1 + λ2

[8]

B6. (a) Define the non-homogeneous Poisson process. [3]

(b) Given u ≥ 0, s ≥ 0, Ns (t) = N (s + t) − N (s), g(t) = E[exp{−uNs (t)}] derive
a differential equation for g(t) starting with g(t + h) = E[exp{−uNs (t + h)}]
and show that the counting process {N (t), t ≥ 0} is a non-homogeneous Poisson
process with intensity function λ(t), t ≥ 0, if
(i) N (0) = 0.
(ii) {N (t), t ≥ 0} has independent increments.
(iii) P {N (t + h) − N (t) ≥ 2} = o(h).
(iv) P {N (t + h) − N (t) = 1} = λ(t)h + o(h).
Hint
If we let m(t) = 0t λ(y)dy then it can be shown that
R
n
P {N (s + t) − N (s)} = e−[m(s+t)−m(s)] [m(s+t)−m(s)]
n!
, n≥0
where N (s + t) − N (s) is a Poisson random variable with mean
m(s + t) − m(s). [15]
(c) A store opens at 8 A.M. From 8 to 10 customers arrive at a Poisson rate of four
per an hour. Between 10 and 12 they arrive at a Poisson rate of eight an hour.
From 12 to 2 the arrival rate increases steadily from eight per hour at 12 to ten
per hour at 2; and from 2 to 5 the arrival rate drops steadily from ten per hour
to four per hour at 5. Determine the probability distribution of the number of
customers that enter the store on a given day. [12]

B7. (a) Define a continuous time Markov chain. [3]

(b) Describe the birth and death process. [2]
(c) Consider a taxi station where taxis and customers arrive in accordance with
Poisson processes with respective rates of one and two per minute. A taxi will
wait no matter how many other taxis are present. However, an arriving customer
that does not find a taxi waiting leaves. Find
(i) the average number of taxis waiting, and [6]
(ii) the proportion of arriving customers that get taxis. [4]

page 3 of 4
 HSTS416/

(d) Consider a shoeshine shop with two chairs where a customer upon arrival goes
initially to chair 1 where his shoes are cleaned and polish is applied. After this
is done the customer moves to chair 2 where his shoes are buffed and shined.
Potential customers arrive in accordance with a Poisson process with rate λ, and
a potential customer will enter the system only if it is empty. The service times
at the two chairs are assumed to be independent exponential random variables
with respective rates µ1 and µ2 .
(i) Describe the state space. [2]
(ii) Explain why this continous time Markov chain is not a birth and death pro-
cess. [3]
(iii) Obtain the balance equations for this process and solve them to obtain the
probabilities. [10]

END OF EXAMINATION PAPER.

page 4 of 4