CHAPTER 11

MARKOV ANALYSIS
Lesson 11.1

Markov Analysis Assumptions

1. The process has a finite number of discrete states, meaning once a
state has been entered, it cannot be removed.
2. The process' condition or state in any given period depends only on
its on condition in the previous period and on the transition
probabilities.
3. The transition probabilities become constant over time.
4. Changes in the process happen only once in every period.
5. All time duration is equal in length.
6. The market size remains constant in all periods.
Lesson 11.2

Information Flow in the Markov Analysis

Figure 11.1. Information Flow

Lesson 11.3

Transition Movements and Transition Probabilities

Go Cleaners is one of the three janitorial cleaning companies that have served in Muntinlupa
City for many years. For the past 10 years, Go Cleaners has been the largest janitorial company
in the area, with 55% of the market. Anytime 24-hour Maids serves 27% of the Muntinlupa
area. Sweep Cleaning is the smallest among the three companies and has served 18% of the
community.

Sweep Cleaning was recently bought by a new Chinese family. This company has launched new
marketing and service packages that have attracted customers away from Go Cleaners and
Anytime 24-hour Maids. Mr. Go, the owner of Go Cleaners, is very concerned about the decrease
in his company's market share. He hired a market research firm to find out the extent of
customer switching and to determine the future of stability of his company.
Lesson 11.3
The janitorial services offered by the three companies are used on a monthly basis by 12,000
Muntinlupa residents. The market research study revealed that in January 2020 (period 1),
6,600 households purchased from Go Cleaners, 3,240 from Anytime 24-hour Maids, and 2,160
from Sweep Cleaning. In February 2020 (Period 2), Go Cleaners retained 4,000 of their
customers. Their remaining customers switched to:

Anytime 24-hour Maids 900

Sweep Cleaning 1,700
Lesson 11.3

Anytime 24-Hour Maids 'retained 2,500 of its 3,240 customers. The others who left
went to:

Go Cleaners 400
Sweep Cleaning 340
Sweep Cleaning retained 1,900 of its customers and lost 130 each to Go Cleaners
and Anytime 24-hour Maids.
Lesson 11.3

Transition Movements and Transition Probabilities

The market research company submitted a Markov analysis to Mr. Go to predict the future of
his company. There are four factors that will interact to predict the future market shares of Go
Cleaners using Markov Analysis. These are:
1. Present Market Share. The percentage of the market accounted for by each of the
companies.
2. Retention Rate. The likelihood of a company's present customers to buy from the
same company.
3. Switching-out Rate. The likelihood of a company's present customers to buy from
another company in the next period.
4. Switching-in Rate. The likelihood of a competitor's customers to buy from the
company in the next period.
 Flow of Customers from Period to Period Lesson 11.3
Table 11.1
Flow of Customers of Janitorial Services
Supplying Firm in Period 2
Period 1
FIRM
Customers
A B C

A 6,600 4,000 900 1,700

B 3,240 400 2,500 340

C 2,160 130 130 1,900

Period 2
4,530 3,530 3,940
Customers

A = Go Cleaners B = Anytime 24-hour Maids C = Sweep Cleaning

Lesson 11.3
Matrix of Transition Probabilities
The matrix of transition probabilities provides more useful information. Table 11.2
shows the transition probabilities by dividing the number of customers of a particular
company in one period by the number of customers in the preceding period. A row
indicates the percentage of customers that a firm will retain from period to period
and the percentage of customers that will be patronizing its competitors. Each column
indicates the percentage of customers a firm will retain and the percentage of other
company's customers that the firm will win in the subsequent period. To summarize,
the rows show retention and loss, while the columns reveal retention and gain.
Lesson 11.3

How to Calculate Market Shares for the Next Period

We will use the matrix transition probabilities
(Table 11.2) to forecast the market share in the
second period.
 Matrix of Transition Probabilities

Market Share in Period 1 A B C Market Share in Period 2

0.606 0.136 0.258
0.550 0.270 0.180 0.377 0.294 0.328
0.123 0.772 0.105
0.061 0.061 0.878

We calculate the new market share by multiplying the matrix of market shares of the three
companies for the initial period 55% for A, 27% for B and 18% for C.
1. The market share of A in period 2 is computed by multiplying the
market share for period 1 by column 1 in the matrix of transition
probabilities.
Ability of A to retain its customers
0.606 x 0.550 = 0.333
multiplied by its share of the market
Ability of A to obtain customers from B
0.123 x 0.270 = 0.033
multiplied by the market share of B
Ability of A to obtain customers from C
0.061 x 0.180 = 0.011
multiplied by the market share of C
Market Share of A in Period 2 0.377
2. The market share of B in period 2 is computed by multiplying the
market share for period 1 by column 2 in the matrix of transition
probabilities.
Ability of B to obtain customers from A
0.136 x 0.550 = 0.075
multiplied by the share of the market of A
Ability of B to retain its customers multiplied
0.772 x 0.270 = 0.208
by its share of the market
Ability of B to obtain customers from C
0.061 x 0.180 = 0.011
multiplied by the market share of C
Market Share of B in Period 2 0.294
3. The market share of C in period 2 is computed by multiplying the
market share for period 1 by column 3 in the matrix of transition
probabilities.
Ability of C to obtain customers from A
0.258 x 0.550 = 0.142
multiplied by the share of the market of A
Ability of C to obtain customers from B 0.105 x 0.270 = 0.028
multiplied by the market share of B
Ability of C to retain its customers multiplied
0.878 x 0.180 = 0.158
by its share of the market
Market Share of B in Period 2 0.328
The calculations for period 2 are already known. For period 2, Go Cleaners has a
37.7% market share, Anytime 24-hour Maids and Sweep Cleaning have 29.4% and
32.8%, respectively. Now, for period 3, the calculations will be as follows:
Matrix of Transition Probabilities

Market Share in Period 2 Market Share in Period 3

0.606 0.136 0.258
0.377 0.294 0.328 0.285 0.299 0.416
0.123 0.772 0.105
0.061 0.061 0.878
 Market Share of A. (0.606 x 0.377) + (0.123 x 0.294) + (0.061 x 0.328) = 0.285
Market Share of B. (0.136 x 0.377) + (0.772 x 0.294) + (0.061 x 0.328) = 0.299
Market Share of C. (0.258 x 0.377) + (0.105 x 0.294) + (0.878 x 0.328) = 0.419

For period 3, the data shows that Go Cleaners will have 28.5%, Anytime 24. hour
Maids at 29.9%, and Sweep Cleaning at 41.6%. Again, the succeeding data can be
used to estimate the relative market shares for period 4. The process can be
repeated to find shares in all of the subsequent periods. Table 11.3 presents the
market share data for periods 1-10.
 Table 11.3
Market Share for Periods 1-10 for Go Cleaners
Lesson 11.4

Steady State Probabilities

An equilibrium condition or steady state happens when in the long
run, the process has stabilized. A stabilized system occurs when the
shares do not change. In other words, the state probabilities in one
period are the same as in the previous period.
Lesson 11.4
CHAPTER 12
DYNAMIC
PROGRAMMING
Lesson 12.1

Philosophy of Dynamic Programming

What distinguished dynamic programming from the "divide and conquer" method is
that the problem to be solved should have overlapping sub-problems.

The method works when the problem has two features: optimal substructure and
overlapping sub-problems. Optimal substructure means that an optimal solution
must contain an optimal sub-solutions. Overlapping sub-problems manifest
when a recursive algorithm visits the same sub-problems repeatedly.
Lesson 12.2 Dynamic Programming Problem
Lesson 12.2
Solution by Dynamic Programming
Starts with the segmentation of the given into smaller problems
termed as stages.
State is referred to as a condition in which a process or a problem is
in a particular stage.
Rollback analysis means that the end of the problem is analyzed
ahead of other problems. Thus, the computation will start at the last
stage (stage 1). The traveler would have to decide in each stage the
best way in which the travel time would be minimized.
 Table 12.2
The First Stage
ALTERNATIVE HOURS OF
STATE BEST ROUTE (HOURS)
ROUTE TRAVEL

7 7-11 7 7

8 8-11 4 4

9 9-11 5 5

10 10-11 6 6
 Table 12.3
The Second Stage
BEST DISTANCE
ALTERNATIVE DISTANCE TO FROM STAGE 1 TO TOTAL
STATE BEST ROUTE
ROUTE STAGE 1 ILOCOS NORTE DISTANCE
( TABLE 12.2 )

4-7 4 7 11
4
4-8 6 4 10

5-8 5 4 9
5
5-9 3 5 8

6-9 4 5 9
6
6-10 2 6 8
 Table 12.4
The Third Stage
BEST DISTANCE
ALTERNATIVE DISTANCE TO FROM STAGE 2 TO TOTAL
STATE BEST ROUTE
ROUTE STAGE 1 ILOCOS NORTE DISTANCE
( TABLE 12.3 )

2-4 4 10 14
2
2-5 3 8 11

3-5 2 8 10
3
3-6 3 8 11
 Table 12.4
The Fourth Stage
BEST DISTANCE
ALTERNATIVE DISTANCE TO FROM STAGE 3 TO TOTAL
STATE BEST ROUTE
ROUTE STAGE 1 ILOCOS NORTE DISTANCE
( TABLE 12.4 )

1-2 2 11 13
1
1-3 4 10 14
Lesson 12.3

Six Major Terms of Dynamic Programming

STAGES
Subdividing the problem is the initial step in
solving a dynamic programming problem. A
stage is defined as a particular decision point
on the course of the solution.
STATES
The traveler moved along bus
stations from Manila to Ilocos
Norte. Each bus station is
equivalent to a state.

DECISION PROCESS
REWARD
The decision process in a dynamic The reward in the example is the
programming solution is done by moving from ‘hours of travel.’
one stage to another. Directions can either
go forward or backward.
Lesson 12.3

Three Types of Reward

IMMEDIATE REWARD
The result or payoff from moving between two
adjacent stages is referred to as the immediate
reward. It is shown under the column ‘Distance
to Stage No.’
TOTAL REWARD
The total reward is equivalent to the sum of the
immediate reward and the optimal reward
obtained from the previous stage.

OPTIMAL TOTAL REWARD

The optimal reward is the best of all rewards in each state, for
each given stage. In the example, it is the shortest route from
one bus station going to the destination.
Lesson 12.3

Six Major Terms of Dynamic Programming

RECURSIVE RELATION
Recursive relation is the function that binds the immediate reward,
the total reward and the optimal total reward. It is a relationship
that is based on the principle of optimality.

POLICY
Dynamic policy is about a predetermined plan for making a decision.