IE 325: Stochastic Models

Homework Assignment 5
Solutions
Spring 2020

Question 1. Patients arrive at a doctor’s office according to a Poisson process with a rate
of λ = 2 patients per hour.

(a) What is the probability that no patients arrive to the doctor’s office from 10:00 to
12:30?

(b) Assume each patient arriving needs examination which takes an exponentially dis-
tributed amount of time with a mean of 15 minutes. Let patient A arrive at time
10:30. Given that patient A is still present in the office by 11:00, what is the probabil-
ity that patient A will still be in the office by 12:00?

(c) It is known that the office was visited by 10 patients in the afternoon, i.e. between
12:30 and 17:30. What is the probability that there were no patients during the first
two hours?

Solution of Question 1. (a) Let {Nt , t ≥ 0} be the Poisson process mentioned above
where t = 0 is 10:00. We will have hours as time units. Then,

e−5 (5)0
P{N2.5 = 0} =
0!
−5
= e

(b) The time the patient A spends in the office from 11:00 on is exponentially distributed
with a mean of 15 minutes (a rate of 11 = 4) regardless how much time he spent before
4
11:00 because of the memoryless property of exponential distribution. Accordingly, let
T be the time patient A spends in the office. Then,

P{T > 1} = 1 − FT (1)

= 1 − (1 − e−4 )
= e−4

(c) We know that there were 10 patients visiting the office between 12:30 and 17:30 (5
hours). Since we have a stationary Poisson process, each of these patients can arrive

1
 during the first two hours with probability 25 . Let X be the number of patients visiting
in the first two hours. Then X ∼ Binomial(n = 10, p = 52 ). Then we have,
 
10 2
0 3
10
P{X = 0} =
0 5 5
3
10
=
5
Question 2. Buses arrive at a certain stop according to a Poisson process with rate λ.
Suppose that John knows the first bus will arrive between t0 and t1 , and he arrives at the
bus stop at t2 (t0 < t2 < t1 ). What is the probability that John catches the first bus?

Solution of Question 2. S1 : Arrival time of the first bus

{Nt , t ≥ 0}: Arrival process (Poisson process with rate λ)
It is given that Nt = 1 (t = t1 − t0 ).
Then, (S1 |Nt = 1) is uniformly distributed over (t0 , t1 ).
−t0
Then, P{S1 ≤ t2 |Nt = 1} = P{Nt2 = 1|Nt = 1} = tt12 −t 0
.

Question 3. An IT center serves two offices. Offices report their complaints to the IT
center according to independent Poisson processes with respective rates λ1 = 1/120 per
minute and λ2 = 1/150 per minute.

(a) After the offices start working, what is the probability that the first report is from
office 1?

(b) Given that the first report came from office 1 at time t, what is the probability that
the next report is from office 2?

(c) Assume the IT center sends someone to office 1 for investigation if office 1 works
without complaint reports for 60 minutes. After they all start working, what is the
expected time until the first inspection?

Solution of Question 3. (a) Let,

– X1 be the time until the first report in office 1

– Y1 be the time until the first report in office 2

Since reports occur according to Poisson processes, X1 ∼ Expon(1/120) and Y1 ∼

Expon(1/150). Moreover, X1 and Y1 are independent as the respective Poisson pro-

2
 cesses are independent.
Z ∞
P{X1 < Y1 } = P{X1 < y|Y1 = y}fY1 (y)dy
Zy=0
∞
= (1 − e−λ1 y )λ2 e−λ2 y dy
Zy=0
∞ Z ∞
−λ2 y
= λ2 e dy − e−λ1 y λ2 e−λ2 y dy
| y=0 {z } y=0

=1
Z ∞

− λ1 +λ2 y
= 1 − λ2 e dy
y=0
h 1 Z ∞ i
= 1 − λ2 (λ1 + λ2 )e−(λ1 +λ2 )y dy
λ1 + λ2 y=0
| {z }
=1
λ2
= 1−
λ1 + λ2
λ1
=
λ1 + λ2
5
=
9
Remark: Note that for two independent exponential random variables X ∼ Expon(λ)
λ
and Y ∼ Expon(µ), we have P{X < Y } = λ+µ , in general.

(b) Reports occur according to Poisson processes. Therefore times between reports are
exponentially distributed. We are given that X1 = t and the first report is sent from
office 1. Let X2 be the time from t until the second report of office 1. We want to
compute
P{Y1 < t + X2 |Y1 > t} = P{Y1 < X2 }
.
Note that we have seen the memoryless property of exponential distribution for discrete
times, that is, P{Y1 > t + u|Y1 > t} = P{Y1 > u}. It is also true that we have
P{Y1 > t + X|Y1 > t} = P{Y1 > X},
where X is random. One can see this by conditioning on X. Here, we use the property
directly:
P{Y1 < t + X2 |Y1 > t} = 1 − P{Y1 ≥ t + X2 |Y1 ≥ t}
= 1 − P{Y1 ≥ X2 }
= P{Y1 < X2 }.
The, using the previous remark we have
1
λ2 150 4
P{Y1 < X2 } = = 1 1 =
λ1 + λ2 120
+ 150
9

3
 (c) Let T be the time until the inspection. Let’s condition T on the arrival of the first
report from office 1.
 
E[T ] = E E[T |X1 ]
Z ∞
= E[T |X1 = x]fX1 (x)dx
0

Now consider two cases. The first report from office 1 can arrive either before 60th
minute or after 60th minute. In the first case, let x be the arrival of the first report.
The expected time until the inspection in this condition is then x + E[T ]. In the
second case, we have the inspection done at 60th minute since no report arrives by
60th minute. Accordingly,
Z ∞


= 1x<60 x + E[T ] + 1x>60 60 fX1 (x)dx
Z0 ∞ Z ∞


= 1x<60 x + E[T ] fX1 (x)dx + 60 1x>60 fX1 (x)dx
0 0
Z 60 Z ∞


= x + E[T ] fX1 (x)dx + 60 fX1 (x)dx
0 60
Z 60 Z 60


= xfX1 (x)dx + E[T ] fX1 (x)dx + 60 1 − FX1 (60)
Z0 60 0
Z 60
1 − 1 x 1 − 1 x



1
− 120 60
= x e 120 dx + E[T ] e 120 dx + 60 e
0 120 0 120
− 21 − 21 1
+ 60e− 2



= 60 2 − 3e + E[T ] 1 − e

Arranging the terms gives us:

1
1 − e− 2
E[T ] = 120 1 ≈ 77.85 minutes.
e− 2
Question 4. Customer arrivals into a textbook store form a Poisson process with λ = 5
per hour. There are hard-cover and paperback books available in the store. It is known that
a customer buys a hard-cover book with probability 0.1, a paperback book with probability
0.2 or leaves the store without buying a book with probability 0.7. On average, the price
of a hard-cover book is 300 TL and the price of a paperback book is 100 TL. Let H be the
total number of customers who buy a hard-cover book, P be the total number of customers
who buy a paperback book and L be the total number of customers who do not buy a book,
during an 8 hour day.

(a) What are the expected values and the variances of H, P, L?

(b) Let R be the total amount of money that the customers pay during an 8 hour day.
Compute E[R], Var(R).

4
Solution of Question 4. Let {Nt , t ≥ 0} be the Poisson process with rate 5.
By decomposition theorem, we have three independent Poisson processes:
{Ht , t ≥ 0} counting the arrivals of customers who buy hard-cover books
{Pt , t ≥ 0} counting the arrivals of customers who buy paperback books
{Lt , t ≥ 0} counting the arrivals of customers who leave without buying a book
Their respective rates are λH = 5 × 0.1 = 0.5, λP = 5 × 0.2 = 1, λL = 5 × 0.7 = 3.5.

(a) E[H] = E[H8 ] = Var(H8 ) = 8λH = 4

E[P ] = E[P8 ] = Var(P8 ) = 8λP = 8
E[L] = E[L8 ] = Var(L8 ) = 8λN = 28

(b) Let R be the total amount of money that the customers pay during an 8 hour day,
then we have R = 300H + 100P + 0L.
E[R] = E[300H + 100P + 0L]
= 300E[H] + 100E[P ]
= 2000 TL
Var(R) = Var(300H + 100P + 0L)
= 3002 Var(H) + 1002 Var(P )
= 440000 TL,
by independence of H, P and L.
Question 5. A coffee shop has three branches X,Y and Z in distant regions of a city and
the number of complaints that the coffee shop receives for those branches follow independent
Poisson processes with rates 15, 17, and 12 complaints per month, respectively.
(a) What is the probability that the coffee shop receives less than 50 complaints in total
in a typical month?
(b) What is the probability that branch X receives exactly 3 complaints before Z receives
any complaints?
Solution of Question 5.
Let Xt : Number of complaints that branch X receives in t months ∼ Poisson(15t),
Yt : Number of complaints that branch Y receives in t months ∼ Poisson(17t),
Zt : Number of complaints that branch Z receives in t months ∼ Poisson(12t).
Note that {Xt , t ≥ 0}, {Yt , t ≥ 0} and {Zt , t ≥ 0} are independent Poisson processes.
(a) If we denote the total number of complaints that the coffee shop receives in t months
by Nt , then by superposition theorem, {Nt , t ≥ 0} is again a Poisson process, with rate
15+17+12=44. Then probability that the coffee shop receives less than 50 complaints
in total in t = 1 month is
49
X e−44 44k
P{N1 < 50} =
k=0
k!

5
 (b) Let us define
SiX : Arrival time ith complaint to branch X
SiZ : Arrival time ith complaint to branch Z.

Then, the question is to compute P{S3X < S1Z }. Let us consider the superposed
process Mt = Xt + Zt . Clearly, {Mt , t ≥ 0} is a Poisson process with rate 15 + 12 = 27.
Moreover, by the superposition theorem, each arrival of the superposed process is from
15
Xt with probability 27 and from Zt with probability 12 27
independent from all the other
arrivals. Note that
{S3X < S1Z } = {The first three arrivals are from Xt }{The forth arrival is from Zt }.
Hence, we have  3
15 12
P{S3X < S1Z } =
27 27

Question 6. Four individuals need kidney transplants. New kidneys arrive according to a
Poisson process with rate λ > 0. Independent from the lifetimes of the other patients and
of the the kidney arrivals, if any of these patients does not receive a new kidney, he/she will
die after an exponential time with parameter µ > 0 (same µ for all four).

(a) What is the probability that all four patients receive kidneys?

(b) Suppose four patients are numbered according to their priorities of receiving kidneys.
Find the probability that the second patient couldn’t survive to get a kidney.

Solution of Question 6. Kidney arrivals is a Poisson process with rate λ > 0. Lifetime
of each patient is exponentially distributed with parameter µ > 0 and independent of every-
thing.
(a)

P (all four patientsreceive kidneys) = P (first kidney arrives before anyone dies)×
P (second kidney arrives before any of the remaining three patients’ death)×
P (third kidney arrives before any of the remaining two patients’ death)×
P (fourth kidney arrives before the last patient’s death)
λ λ λ λ
= · · ·
λ + 4µ λ + 3µ λ + 2µ λ + µ
Consider superposition of kidney arrivals and deaths of all patients. All four patients
receive kidneys if one of them receives kidney and leaves the system, then the second one re-
ceives kidney and leaves the system, then the third one receives kidney and leaves the system,
and so on. The first one receives kidney if the first event in superposition is a kidney arrival,
λ
then P (first kidney arrives before anyone dies ) = λ+4µ by superposition theorem. In subse-
quent system, there are three remaining patients. By memoryless property, the system acts
as if there are three patients in the system at the beginning independently from the past,
λ
hence P (second kidney arrives before any of the remaining three patients death) = λ+3µ by

6
superposition theorem again. Other probabilities are calculated similarly.

(b) Note that the second patient’s status does not depend on the third and fourth patients’
conditions. Thus, we can model the problem as if there are only two patients.
We superpose the three independent Poisson processes, where we consider the death of
patient 1 and patient 2 as the first arrival of some Poisson processes with rates µ. Then we
condition on the first event of this superposed process, say M .
P (the second patient couldn’t survive to get a kidney)
= P (First arrival of M is the death of second patient)
+ P (First arr. of M is the death of patient 1)P (Patient 2 dies before the kidney after patient 1 dies)
+ P (first arr. of M is a kidney)P (Patient 2 dies before the first kidney after patient 1 leaves the system)
   
µ µ µ λ µ
= + × + ×
2µ + λ 2µ + λ µ + λ 2µ + λ µ + λ
2µ
= .
2µ + λ
Question 7. A barbershop, operated by two barbers, has one extra chair, so that at most
one customer can wait in the shop (customers would leave if the shop is full when they arrive).
Potential customers arrive at a Poisson rate of λ per hour, and barbers work independently
where each cutting time is independent, exponentially distributed. An arriving customer
chooses one of the barbers if both are free, chooses the barber which is free if the other one
is busy and starts to wait in the line if both are busy. Describe this system as a Continuous
Time Markov Chains (CTMC) in the cases below. In your description, write the transition
rates from one state to another and also draw a transition diagram.
(a) Suppose the cutting times of barbers are identical with rate µ and the customers are
equally likely to choose any barber if both are free.
(b) Suppose the cutting times are not identical and has rates of µ1 and µ2 , and again the
customers are equally likely to choose any barber if both are free.
(c) Suppose the cutting times are not identical as in part (b), however now, the customers
do prefer the first barber if both are free.
Solution of Question 7. (a) Barbers are identical, customers equally likely to choose
any of them with equal probabilities. Let X(t) be number of customers in the shop
and state space is E = {0, 1, 2, 3}.

(b) Cutting times of the two barbers have non-identical exponential distributions, where
for barber 1 the rate is µ1 and for barber 2 the rate is µ2 . Let X(t) = (i, j) where i is
the number of customers in the shop and j indicates which barber is busy.

7
 – j = N indicates that none of the barbers are busy,
– j = I indicates that barber 1 is busy,
– j = II indicates that barber 2 is busy,
– j = B indicates that both barbers are busy.

E = {(0, N ), (1, I), (1, II), (2, B), (3, B)}.

(c) Customers prefer the first barber. X(t) and E are as defined in part (b).

Question 8. Travelers arrive at a taxi station according to a Poison process with rate λ.
Whenever there are 4 travelers in the station, a taxi will be sent from the center to pick them
up and take them to their destinations. We assume that a taxi have the maximum capacity
of 4 people and also we assume that the travelers leave and go to the station of different taxi
company if they come to the station and see that there are already 5 people waiting. The
time it takes for a taxi to come to the station follows an exponential distribution with mean
1/µ.

(a) Describe this system as a CTMC. In your description, write the transition rates from
one state to another and draw a transition diagram.

(b) Suppose λ = 1 and µ = 3. Find the proportion of time that the process spends at each
state. Express the mean time the process spends at each state, i.e. sojourn times.

Solution of Question 8. (a) Let X(t) be the number of travelers waiting at the station
at time t. The state space is E = {0, 1, 2, 3, 4, 5}.

8
(b)
0 1 2 3 4 5
 
0 −1 1 0 0 0 0
 0 −1 1
1 0 0 0 

2 0
 0 −1 1 0 0 
Λ=  
3 0 0 0 −1 1 0 

4 3 0 0 0 −4 1 
5 0 3 0 0 0 −3

Solving πΛ = 0 and π1 = 1, where 0 and 1 vector of zeros and ones, respectively.

Letting λ = 1 and µ = 3:

π0 = 3π4
π1 = π0 + 3π5
π2 = π1
π3 = π2
4π4 = π3
3π5 = π4
5
X
πi = 1
i=0

Solving above sets of equations π0 = 9/49, π1 = π2 = π3 = 12/49, π4 = 3/49, and

π5 = 1/49. Let Ti be the expected time that process spends in state i ∈ {0, 1, 2, 3, 4, 5}.
Then,

1
E[Ti ] = , i ∈ {0, 1, 2, 3}
λ
1
E[T4 ] =
µ+λ
1
E[T5 ] =
µ