INGB472: DECISION-SUPPORT SYSTEMS

Study Unit 3: Markov Chains Part 2

ABSORBING MARKOV CHAIN
An absorbing Markov chain is where every state can reach an absorbing state.
An absorbing state is a state that, once entered, the probability of staying in that
state is 1 (100%).
ABSORBING MARKOV CHAIN
A Markov chain is an absorbing chain if:
• There is at least one absorbing state.
• It is possible to go from any state to at least one absorbing state in a finite
number of steps.
A state that is not absorbing is known as a transient state or just transient.
ABSORBING MARKOV CHAIN
Let an absorbing Markov chain with transition matrix P comprises ”t” transient
states and ”r” absorbing states. Let:
𝑰𝑰 𝟎𝟎
𝑷𝑷 = 𝒓𝒓
𝑨𝑨 𝑩𝑩
Where: 𝑰𝑰𝒓𝒓 is the 𝑟𝑟 × 𝑟𝑟 identity matrix; 0 an 𝑟𝑟 × 𝑡𝑡 zero matrix; 𝑨𝑨 is an t × 𝑟𝑟 matrix and
B an t × 𝑡𝑡. Thus, B describes the probability of transitioning from some transient
state to another while A describes the probability of transitioning from some
transient state to some absorbing state.
ABSORBING MARKOV CHAIN
B describes the probability of transitioning from some transient state to another,
while A describes the probability of transitioning from some transient state to
some absorbing state.
FUNDAMENTAL MATRIX
A basic property about an absorbing Markov chain is the expected number of
visits to a transient state ”j” starting from a transient state ”i” (before being
absorbed). The probability of transitioning from ”i" to ”j” in exactly ”k” steps is the
(i, j)-entry of Bk. Summing this for all k (from 0 to ∞) yields the fundamental matrix,
denoted by F. It can be proven that:
FUNDAMENTAL MATRIX
𝑭𝑭 = � 𝐵𝐵𝑘𝑘 = 𝑰𝑰𝒕𝒕 − 𝑩𝑩 −1

𝑘𝑘

Where 𝑰𝑰𝒕𝒕 is the 𝑡𝑡 × 𝑡𝑡 identity matrix.

The theory will be explained by means of an example:

ABSORBING STATES
In the examples discussed thus far, we assume that it is possible for the process or
system to go from one state to any other state between any two periods. In
some cases, however, if you are in a state, you cannot go to another state in the
future. In other words, when you are in a given state, you are “absorbed” by it,
and you will remain in that state. Any state that has this property is called an
absorbing state. An example of this is the accounts receivable application. An
accounts receivable system normally places debts or receivables from its
customers into one of several categories or states depending on how overdue
the oldest unpaid bill is (the exact categories or states depend on the policy set
by each company). Four typical states or categories for an accounts receivable
application follow:
ABSORBING STATES
State 1 𝜋𝜋1 : paid, all bills.
State 2 𝜋𝜋2 : bad debt, overdue more than three months.
State 3 𝜋𝜋3 : overdue less than one month.
State 4 𝜋𝜋4 : overdue between one and three months.
ABSORBING STATES
• At any given period, in this case one month, a customer can be in one of these
four states.
• For this example it will be assumed that if the oldest unpaid bill is over three
months due, it is automatically placed in the bad debt category.
• Therefore, a customer can be paid in full (state 1), have the oldest unpaid bill
overdue less than one month (state 3), have the oldest unpaid bill overdue
between one and three months inclusive (state 4), or have the oldest unpaid
bill overdue more than three months, which is a bad debt (state 2).
ABSORBING STATES
• As in any other Markov process, we can set up a matrix of transition
probabilities for these four states.
• This matrix will reflect the propensity of customers to move among the four
accounts receivable categories from one month to the next.
• The probability of being in the paid category for any item or bill in a future
month, given that a customer is in the paid category for a purchased item this
month, is 100% or 1.
• It is impossible for a customer to completely pay for a product one month and
to owe money on it in a future month.
ABSORBING STATES
• Another absorbing state is the bad debts state.
• If a bill is not paid in three months, we are assuming that the company will
completely write it off and not try to collect it in the future.
• Thus, once a person is in the bad debt category, that person will remain in that
category forever.
• For any absorbing state, the probability that a customer will be in this state in
the future is 1, and the probability that a customer will be in any other state is 0.
ABSORBING STATES
• These values will be placed in the matrix of transition probabilities. But before
we construct this matrix, we need to know the future probabilities for the other
two states—a debt of less than one month and a debt that is between one
and three months old.
• For a person in the less than one month category, there is a 0.60 probability of
being in the paid category, a 0 probability of being in the bad debt category,
a 0.20 probability of remaining in the less than one month category, and a
probability of 0.20 of being in the one to three month category in the next
month.
ABSORBING STATES
• Note that there is a 0 probability of being in the bad debt category the next
month because it is impossible to get from state 3, less than one month, to
state 2, more than three months overdue, in just one month.
• For a person in the one to three month category, there is a 0.40 probability of
being in the paid category, a 0.10 probability of being in the bad debt
category, a 0.30 probability of being in the less than one month category, and
a 0.20 probability of remaining in the one to three month category in the next
month.
ABSORBING STATES

Following month
This month Paid Bad debt <1 month 1 to 3
months
Paid 1 0 0 0
Bad debt 0 1 0 0
<1 month 0.6 0 0.2 0.2
1 to 3 0.4 0.1 0.3 0.2
months
ABSORBING STATES
• How can we get a probability of 0.30 of being in the one to three month
category for one month, and in the one month or less category in the next
month?
• Because these categories are determined by the oldest unpaid bill, it is
possible to pay one bill that is one to three months old and still have another
bill that is one month or less old.
• In other words, any customer may have more than one outstanding bill at any
point in time.
• With this information, it is possible to construct the matrix of transition
probabilities of the problem.
ABSORBING STATES
Therefore:
1 0 0 0
𝑷𝑷 = 0 1 0 0
0.6 0 0.2 0.2
0.4 0.1 0.3 0.2

1 0 0 0 0.6 0 0.2 0.2

and 𝑰𝑰 = , 𝟎𝟎 = , 𝐀𝐀 = , 𝑩𝑩 =
0 1 0 0 0.4 0.1 0.3 0.2
ABSORBING STATES
𝑭𝑭 = 𝑰𝑰 − 𝑩𝑩 −1

−1
0.8 −0.2
𝐅𝐅 =
−0.3 0.8

1.38 0.34
∴ 𝐅𝐅 =
0.52 1.38
ABSORBING STATES
• Now we are in a position to use the fundamental matrix in computing the
amount of bad debt money that we could expect in the long run.
• First we need to multiply the fundamental matrix, F, times the matrix A. This is
accomplished as follows:
ABSORBING STATES
1.38 0.34 0.6 0
𝐅𝐅𝐀𝐀 = ×
0.52 1.38 0.4 0.1

0.97 0.03
∴ 𝑭𝑭𝑭𝑭 =
0.86 0.14
ABSORBING STATES
• The new FA matrix has an important meaning.
• It indicates the probability that an amount in one of the non-absorbing states
will end up in one of the absorbing states.
• The top row of this matrix indicates the probabilities that an amount in the less
than one month category will end up in the paid and the bad debt category.
The probability that an amount that is less than one month overdue will be
paid is 0.97, and the probability that an amount that is less than one month
overdue will end up as a bad debt is 0.03.
ABSORBING STATES
• The second row has a similar interpretation for the other non-absorbing state,
which is the one to three month category.
• Therefore, 0.86 is the probability that an amount that is one to three months
overdue will eventually be paid, and 0.14 is the probability that an amount
that is one to three months overdue will never be paid but will become a bad
debt.
ABSORBING STATES
• This matrix can be used in a number of ways. If we know the amount of the less
than one month category and the one to three month category, we can
determine the amount of money that will be paid and the amount of money
that will become bad debts.
• We let the matrix M represent the amount of money that is in each of the non-
absorbing states as follows:
ABSORBING STATES
𝑀𝑀 = (𝑀𝑀1 , 𝑀𝑀2 , … , 𝑀𝑀𝑛𝑛 )
Where:
𝑛𝑛 = the number of non-absorbing states.
𝑀𝑀𝑖𝑖 = the amount in the 𝑖𝑖𝑡𝑡𝑡 state where 𝑖𝑖 ≤ 𝑛𝑛.
ABSORBING STATES
Say there is R 4000 in the less than one month state and R 10000 in the one to
three month state.
∴ 𝑀𝑀 = (4000, 10000)
The quantity that will likely be paid as oppose to bad debt write-offs can be
predicted through MFA.
ABSORBING STATES
0.97 0.03
𝑴𝑴𝑴𝑴𝑴𝑴 = 4000, 10000 = (12480, 1520)
0.86 0.14

Therefore, from R 14000 of debt, R 1520 will probably be written off as bad debt.