Professional Documents
Culture Documents
ISYE 3232C Fall 2015 Quiz 9
ISYE 3232C Fall 2015 Quiz 9
ISYE 3232C Fall 2015 Quiz 9
I,
, do swear that I abide by the Georgia Tech Honor Code. I understand that
any honor code violation will result in a zero for this test and one letter grade reduction.
Signature:
1. (3) True or False.
(T/F) In a MC with finite state space, all states can be transient. (F)
(T/F) A transient state is not accessible from any recurrent state. (T)
(T/F) At least one, possibly more, recurrent states are accessible from a given transient state. (T)
(T/F) A recurrent state is accessible from all states in its class, and it is also accessible from
recurrent states in other classes. (F)
(T/F) If DTMC is periodic with period d 2, then the limiting probability exist. (F)
(T/F) A DTMC can have infinite number of stationary distributions. (T)
2. Consider a DTMC on state space S = {1, 2, 3, 4} with transition probability
1 0.2
2 0
P= 3 0
4 0
0.2
1
0
0
0.4
0
0.7
0.5
0.2
0
0.3
0.5
0.2
0.2
0.2
0.5
0.4
0.3
3
0.7
(b) (1.5) Identify the classes and classify them as recurrent or transient.
{1}: Transient
{2}: Recurrent
{3, 4}: Recurrent
(c) (0.5) What is the period of state 3?
d(3) = 1
?
P =
?
?
?
?
?
?
?
?
?
?
,
?
?
? 0.375
Fill the P .
Note that
j=1
3
4
Moreover
f1,2 + f1,{3,4} = 1
Thus f1,2 =
1
4
Therefore
0 41
0 1
P =
0 0
0 0
3
4
0.625
0
0.625
0.625
3
4
0.375
0
,
0.375
0.375