XIAMEN UNIVERSITY MALAYSIA

ONLINE MIDTERM EXAMINATION

Course Code: MAT313

Course Name: Stochastic Processes
Question Paper Setter: Liu Jie
Academic Session: 2021/09 Question Paper: A o B o
Total No. of Pages: 5 Time Allocated: 2 hours
Additional Materials: Draft Paper
Apparatus Allowed: Calculator

INSTRUCTIONS TO CANDIDATES

1. This paper consists of 5 questions. Answer all questions.

2. Write or paste all the corresponding answers on the answer book provided.

3. Communication between candidates in any means is forbidden. Answers must be entirely

individual candidate’s independent effort. If you are found sharing your solutions with
other candidates, or suspected of doing so, you would be penalized accordingly.

(Student ID: Full Name: )

CONFIDENTIAL 202109/MAT313

Question 1 (20 marks)

(a) The time required to repair a machine is Exp(3). What is the probability that a repair time
exceeds 2 given the fact that it is greater than 1? [10]

(b) Let X1 ∼ Exp(µ1 ), X2 ∼ Exp(µ2 ) be independent. Define X(1) = min(X1 , X2 ) and

X(2) = max(X1 , X2 ). Determine

(i) E[X(1) ]. [2]

(ii) Var(X(1) ). [3]

(iii) E[X(2) ]. [5]

Page 1 of 5
CONFIDENTIAL 202109/MAT313

Question 2 (20 marks)

Consider the Markov chain with states 1, 2, 3, 4 and transition matrix
 
1 1 1
0
 4 4 2 

 0 0 3 13 2
 
P=  .
1 1
 2 2 0 0 

 
0 13 13 13

(a) Specify the class of the Markov chain and determine whether they are transient, positive
recurrent, or null recurrent. [5]

(b) If X0 = 1, determine the probability that the chain enters state 4 before state 3. [8]

(c) If P(X0 = 1) = 0, P(X0 = 2) = 21 , P(X0 = 3) = 0, P(X0 = 4) = 12 , determine E[X2 ]. [7]

Page 2 of 5
CONFIDENTIAL 202109/MAT313

Question 3 (20 marks)

Two white and four black balls are distributed in two urns in such a way that each urn contains
three balls. The system is in state i, i = 0, 1, 2, if the first urn contains i white balls. At each
step, we swap a ball from each urn with balls being randomly selected. Let Xn be the state of
the system after the nth step.

(a) Write down the transition matrix P. [10]

(b) Use the result on Page 59 of the notes to determine if this Markov chain is time reversible.
Justify your answer. [10]

Page 3 of 5
CONFIDENTIAL 202109/MAT313

Question 4 (20 marks)

(a) Let us consider a fixed (no births and no deaths) population of N individuals. Suppose that
an individual can be in one of three states — Susceptible (S), Infected (I), and Recovered
(R). Let S(t) be the number of susceptible individuals, let I(t) be the number of infected
individuals, and let R(t) be the number of recovered individuals. Each individual in state
S has a “S → I” clock. When it rings, that person goes to state I. Each individual in
state I has a “I → R” clock. When it rings, that person goes to state R. At time t,
every “S → I” clock ∼ Exp(βI(t)) while every “I → R” clock ∼ Exp(γ). Note that
I(t)
the individual infection rate, which is the parameter in Exp(βI(t)), is proportional to N

which is the density of infected people. Since N is a fixed constant, we have lumped it
into the constant β. Introduce an appropriate continuous-time Markov chain for the vector
(S(t), I(t), R(t)) and determine all the appropriate parameters, namely vi and Pij , for the
model. [Hint: Recall Question 2 of Exercises of Continuous Time Markov Chains.] [10]

(b) A single repairperson looks after three machines 1, 2, and 3. Each time a machine is
repaired, it stays up for an exponential time with rate λ. When a machine fails, it requires
an exponentially distributed amount of time with rate µ to complete its repair. Define the
states and introduce an appropriate continuous-time Markov chain to describe this process.
Determine all the appropriate parameters, namely vi and Pij , for the model. [Hint: It is
simpler than Question 3 but slightly harder than Question 8 of Exercises of Continuous
Time Markov Chains.] [10]

Page 4 of 5
CONFIDENTIAL 202109/MAT313

Question 5 (20 marks)

Let S(t) denote the price of a security at time t. A popular model for the process {S(t), t ≥ 0}
supposes that the price remains unchanged until a “shock” occurs, at which time the price is
multiplied by a random factor. If we let N (t) denote the number of shocks by time t, and let
Xi denote the ith multiplicative factor, then this model supposes that
N (t)
Y
S(t) = S(0) Xi
i=1

QN (t)
where i=1 Xi is equal to 1 when N (t) = 0. Suppose that the Xi are independent
exponential random variables with rate µ; that {N (t), t ≥ 0} is a Poisson process with rate λ;
that {N (t), t ≥ 0} is independent of the Xi ; and that S(0) = S0 .

(a) Find E[S(t)]. [10]

(b) Find Cov (N (t), S(t)). [10]

[Hint: Recall Questions 46 and 50 of Exercises of the Exponential Distribution and the Poisson
Process.]

-END OF PAPER-

Page 5 of 5