Markov

Processes
Characteristics of a Markov System
1. It will operate or exist for a number of periods.
2. In each period, the system can assume one of a number of
states or conditions.
3. The states are both mutually exclusive and collectively
exhaustive.
4. System changes between states from period to period can
be described by transition probabilities, which remain
constant.
5. The probability of the system being in a given state in a
particular period depends only on its state in the preceding
period and the transition probabilities. It is independent of
all earlier periods.
Markov Analysis:
Assumptions
• The probability that an item in the system
either will change from one state to another or
remain in its current state is a function of the
transition probabilities only.
• The transition probabilities remain constant.
• The system is a closed one; there will be no
arrivals to the system or exits from the system.
 • A small town has three grocery stores.
• A total of 100,000 people shop at the three
grocery stores during any given month.
Markov • Forty thousand people may be shopping at
Process- American Food Store (state 1)
Example • 30,000 people may be shopping at Food
Mart (state 2);
• 30,000 people may be shopping at Atlas
Foods (state 3).
 • 80% of the customers who shop at American Food
Store for one month will return to that store next
month.
• However, of the other 20% of American’s customers,
Markov 10% will switch to Food Mart, and the other 10%
will switch to Atlas Foods for their next purchase.
Process- • For customers who shop this month at Food Mart,
Example 70% will return, 10% will switch to American Food
Store, and 20% will switch to Atlas Foods.
• Of the customers who shop this month at Atlas
Foods, 60% will return, but 20% will go to American
Food Store and 20% will switch to Food Mart.
Tree
Diagram
However..
• The tree would soon get very large.
• It is easier to use a matrix of transition probabilities.
• A transition probability is the probability of moving from one particular state to another
particular state.
• A matrix of transition probabilities is used along with the current state probabilities to
predict future conditions.
 • Matrix of conditional probabilities of being in
Matrix of Transition a future state given a current state

Probabilities • Pij = conditional probability of being in state j

in the future given the current state of i
Predicting Future Market Shares
•𝜋 1 =𝜋 0 𝑃
• 𝜋 𝑛+ 1 =𝜋 𝑛 𝑃

•𝜋 1 =

• = (0.41,0.31,0.28)

• The market shares for American Food Store and Food Mart have increased, while the market share
for Atlas Foods has decreased. Will this trend continue in the next period and the one after that?
Question
• Paul Tolsky, owner of Tolsky Works, has recorded the operation of his milling machine
for several years. Over the past 2 years, 80% of the time the milling machine functioned
correctly during the current month if it had functioned correctly in the preceding month.
This also means that only 20% of the time did the machine not function correctly for a
given month when it was functioning correctly during the preceding month. In addition, it
has been observed that 90% of the time the machine remained incorrectly adjusted for any
given month if it was incorrectly adjusted the preceding month. Only 10% of the time did
the machine operate correctly in a given month when it did not operate correctly during
the preceding month. In other words, this machine can correct itself when it has not been
functioning correctly in the past, and this happens 10% of the time.

• What is the probability that the machine will be functioning correctly 1, 2 month from
now, given that it is now functioning correctly?
• What are the steady state probabilities?
Question
• George Walls, president of Bradley School, is concerned about declining enrollments. Bradley School is a
technical college that specializes in training computer programmers and computer operators. Over the years,
there has been a lot of competition among Bradley School, International Technology, and Career Academy. The
three schools compete in providing education in the areas of programming, computer operations, and basic
secretarial skills. To gain a better understanding of which of these schools is emerging as a leader, George
decided to conduct a survey. His survey looked at the number of students who transferred from one school to the
other during their academic careers. On average, Bradley School was able to retain 65% of those students it
originally enrolled. Twenty percent of the students originally enrolled transferred to Career Academy, and 15%
transferred to International Technology. Career Academy had the highest retention rate: 90% of its students
remained at Career Academy for their full academic program. George estimated that about half the students who
left Career Academy went to Bradley School, and the other half went to International Technology. International
Technology was able to retain 80% of its students after they enrolled. Ten percent of the originally enrolled
students transferred to Career Academy, and the other 10% enrolled in Bradley School.
• Currently, Bradley School has 40% of the market. Career Academy, a much newer school, has 35% of the
market. The remaining market share—25%—consists of students attending International Technology. George
would like to determine the market share for Bradley for the next year.
• What are the equilibrium market shares for Bradley School, International Technology, and Career Academy?