Assignment 1.

Date of submission : 3pm of 20th February 2020,

1. Consider a communication system which transmits the digits 0 and 1. Each digit
transmitted must pass through several stages, at each of which there is a probability ‘p’
that the digit entered will be unchanged when it leaves. Transform this situation into a
Markov chain and write its transition probability matrix.

2. Suppose that there are five types of breakfast cereal, which we call A, B, C, D and E.
Customers tend to stick to the same brand. Those who choose type A, choose it the next
time around with probability 0.8; those who choose type B, choose it the next time with
probability 0.9. The probabilities for types C, D and E are given by 0.7, 0.8 and 0.6,
respectively. When customers do change brand, they choose one of the other four equally
probably. Explain how this may be modelled by a Markov chain and give the transition
probability matrix.

3. Examine the nature of the states of the Markov chain whose transition probability matrix
is given by P=[ 0 1 00 0 0 0 0.5 0.5 0 0 0 0 0.7 0.3 0 0 0 0 0 0 1 0 0 0 0 0 00.4 0.6 0 0 0 0 0.20.8 ].
The state space is S= { 1,2,3,4,5,6 }. If there exist a recurrent class, determine the stationary
distribution of the states in that class.

4. The transition probability matrix of a discrete time Markov chain whose state space is
S={1, 2, 3, 4, 5} is given by A=[ 0 0 0 01 0 0 1 0 0 0 0 01 0 0 0.8 0.2 0 0 0.4 0 0.6 0 0 ] . Draw
all the sample paths of length 4 that begin in state 1. What is the probability of being in
each of the states 1 to 5 after four steps, beginning at state 1?

5. Consider the transition probability matrix corresponding to the mood 0, 1, 2 of an

individual, i.e., P=[ 0.5 0.4 0.1 0.3 0.4 0.3 0.2 0.3 0.5 ]. In the long run, what proportion of
time is the process in each of the three states?

6. Draw the state transition diagram for the Markov chain whose states are 0, 1, 2, 3, 4
having transition probability matrix as
A=[ 0.4 0.6 0 0 0 0.5 0.5 0 0 0 0 0 0.3 0.7 0 0 0 0.5 0.4 0.1 0 0 0 0.80.2 ]
Which of the states are accessible from state ‘0’?
Which of the states are accessible from state ‘3’?

7. The transition probability matrix of a Markov chain

 X n , n  1, 2,... having three states
 0.1 0.5 0.4 
P   0.6 0.2 0.2 

with the initial distribution 0 

{ 1,2,3 } is  0.3 0.4 0.3   0.7 0.2 0.1
. Find
 i  P  X 2  3
 ii  P  X 3  2, X 2  3, X 1  3, X 0  2
 8. Examine the nature of the states { a, b, c, d, e} of the Markov chain having transition
probability matrix as
 0.5 0 0.5 0 0 
 0 0.25 0 0.75 0 

P 0 0 13 0 2 3
 
0.25 0.5 0 0.25 0 
 1 3 0 13 0 1 3 

9. A radioactive source emits particles at a rate of 5 per minute in accordance with a Poisson
process. Each particle emitted has a probability 0.6 of being recorded. Find the
probability that the number of particles in a 4-minute interval is equal to 10.

10. The number of customers arriving at a grocery store can be modelled by a Poisson
process with intensity λ=10 customers per hour.
(i) Find the probability that there are 2 customers between 10:00 a.m. and 10:20 a.m.
(ii) Find the probability that there are 3 customers between 10:00 a.m. and 10:20 a.m.
and 7 customers between 10:20 a.m. and 11a.m.

************