IE 325 STOCHASTIC MODELS

HOMEWORK 2

FALL 2021

1. The number of students graduated from a university in a typical year has Poisson distribution with
mean 600 students. Each graduate finds independently a job in two months after graduation with
probability 0.9 if the economy is booming and with probability 0.4 if the economy is in recession. In
a typical graduation year, the economy is booming with probability 0.6 and undergoes a recession with
probability 0.4.
(a) What is the expected total number of graduates who find a job in two months after graduation?
(b) What is the probability mass function of the total number of graduates who find a job in two
months after graduation?
2. An individual whose level of exposure to a certain pathogen is x will contract the disease caused by
this pathogen with probability P (x) = (1 − e−x ). Exposure level of a randomly chosen member of the
population is exponentially distributed with parameter λ, independent of the other members.
(a) Find the probability that a randomly chosen member of the population has the disease that is
caused by this pathogen.
(b) Among 100 people that have the check-up, find the expected number of people who have the
disease that is caused by this pathogen.
3. Consider the Markov Chain defined recursively by:


X + ξ
n n+1 , if Xn + ξn+1 ≤ 2
Xn+1 =
X + ξ
n − 3,
n+1 if Xn + ξn+1 ≥ 3
where ξ1 , ξ2 , . . . are independent identically distributed random variables and,
P{ξi = 0} = p0 , P{ξi = 1} = p1 , P{ξi = 2} = p2 ,
p0 > 0, p1 > 0, p2 > 0, p0 + p1 + p2 = 1 and X0 = 0
(a) Is X = (Xn )n≥0 a Markov Chain? Explain briefly.
(b) What are the states of the Markov Chain?
(c) Determine the one step transition probability matrix P .
4. Consider the Markov chain with one-step transition probability matrix:
0 1
" #
0 1/2 1/2
P =
1 1/4 3/4
and initial probabilities p0 = p1 = 21 .
(a) Compute the two-step transition probability matrix.
(b) Find P{X0 = 1, X2 = 0}
(c) Find P{X2 = 0}
(d) Find P{X0 = 1, X1 = 0|X2 = 0}
(e) Suppose X0 = 1. Find E[X2 |X0 = 1] and Var[X2 |X0 = 1]

Date: October 5, 2021.

1
2 FALL 2021

5. Assume that the preferred universities in Ankara are METU, Ankara University and Bilkent University.
In addition, the following information is provided:
• 60% of the daughters of METU female graduates study at METU; 10% of them study at Bilkent
University and the rest study at Ankara University;
• 40% of the daughters of Ankara University female graduates study at Ankara University and the
rest are split evenly between METU and Bilkent University.
• 70% of the daughters of Bilkent University female graduates study at Bilkent University, 20% of
them study at METU and the rest study at Ankara University.
(a) Model the process as a Markov chain and find the one step transition matrix.
(b) What is the probability that a granddaughter of a METU female graduate studies at METU?
(c) Given that the initial probabilities of a female graduate being from any of the three universities
are distributed equally, find the probability that a granddaughter studies at a different school
from her grandmother.
6. Mustafa lives near university commutes to university everyday. His friend drives him to his workplace
in the morning, but in the evening he takes the bus. He misses the bus with probability 0.1. When he
misses the bus, he walks to home. Since he lives in a very cold city, if he walks to home for two days in
a row, he gets sick the following day.
(a) Model the process as a Markov chain and find the one step transition probability matrix.
(b) Mustafa’s friends, on Tuesday, have invited him to go to museum on Friday. Mustafa told them
that he would go if he feels well that day. Given that Mustafa feels sick on Tuesday morning,
what is the probability that he will make it to the museum?
7. In an institute, a lab rat is used for experimentation of a certain medication. Each evening, there are
two treatment options: applying 1 ml of the medication or not. If the medication is not applied, the
response is positive with probability 0.6, negative with probability 0.38 and the rat dies with probability
0.02. If 1 ml of the medication is applied, the response is positive with probability 0.65, negative with
probability 0.3 and the rat dies with probability 0.05. If the response is negative, the treatment is
switched.
(a) Model the process as a Markov chain and find the one step transition probability matrix.
(b) If the medication was not applied on Monday evening, what is the probability that the treatment
will be applied on Thursday evening?
(c) The lab rat was given the medication on Sunday evening. The rat will be transferred to another
testing facility on next Sunday morning (if it still lives). An employee is tasked with bringing
enough supplies of the medication until transfer. What is the expected amount of medication
that should be brought?
Hint: You can use MATLAB for matrix calculations.
8. An urn contains four balls, of which two are red and two are green. A ball is randomly selected from
the urn and replaced with a ball of the opposite color. Explain why this system may be analyzed by
using a Markov chain. Define the state space. Calculate the one-step transition probability matrix. Find
the probability that after 2 selections there are still two red and two green balls in the urn.
 IE 325 STOCHASTIC MODELS HOMEWORK 2 3

9. Consider the Markov chain with one-step transition probability matrix:

0 1 2 3
 
0 0.15 0.1 0.5 0.25
 
1 0.4 0.3 0.2 0.1 
P =  
2 0
 0.5 0.3 0.2 

3 0.2 0 0.1 0.7
Suppose that the chain is equally likely to start in either 0 or 3. Calculate
(a) P{X0 = 0, X1 = 2, X2 = 1, X3 = 3}
(b) P{X2 = 1|X1 = 1}
(c) P{X3 = 2|X1 = 3}
(d) P{X2 = 2}
(e) E[X2 |X1 = 1]