Markov Analysis

Introduction

Markov analysis is a technique that deals with the probabilities of future occurrences by ana-
lyzing presently known probabilities.1 The technique has numerous applications in business, in-
cluding market share analysis, bad debt prediction, university enrollment predictions, and
determining whether a machine will break down in the future.
Markov analysis makes the assumption that the system starts in an initial state or condi-
tion. For example, two competing manufacturers might have 40% and 60% of the market
sales, respectively, as initial states. Perhaps in two months the market shares for the two
companies will change to 45% and 55% of the market, respectively. Predicting these future
states involves knowing the system’s likelihood or probability of changing from one state to
The matrix of transition another. For a particular problem, these probabilities can be collected and placed in a matrix
probabilities shows the likelihood or table. This matrix of transition probabilities shows the likelihood that the system will
of change. change from one time period to the next. This is the Markov process, and it enables us to pre-
dict future states or conditions.
Like many other quantitative techniques, Markov analysis can be studied at any level of
depth and sophistication. Fortunately, the major mathematical requirements are just that you
know how to perform basic matrix manipulations and solve several equations with several un-
knowns. If you are not familiar with these techniques, you may wish to review Module 5 (on the
Companion Website for this book) which covers matrices and other useful mathematical tools,
before you begin this chapter.
There are four assumptions of Because the level of this course prohibits a detailed study of Markov mathematics, we limit
Markov analysis. our discussion to Markov processes that follow four assumptions:
1. There are a limited or finite number of possible states.
2. The probability of changing states remains the same over time.
3. We can predict any future state from the previous state and the matrix of transition
probabilities.
4. The size and makeup of the system (e.g., the total number of manufacturers and customers)
do not change during the analysis.

15.2 States and State Probabilities

States are used to identify all possible conditions of a process or a system. For example, a ma-
chine can be in one of two states at any point in time. It can be either functioning correctly or
not functioning correctly. We can call the proper operation of the machine the first state, and we
can call the incorrect functioning the second state. Indeed, it is possible to identify specific states
for many processes or systems. If there are only three grocery stores in a small town, a resident
can be a customer of any one of the three at any point in time. Therefore, there are three states
corresponding to the three grocery stores. If students can take one of three specialties in the
management area (let’s say management science, management information systems, or general
management), each of these areas can be considered a state.
Collectively exhaustive and In Markov analysis we also assume that the states are both collectively exhaustive and
mutually exclusive states are two mutually exclusive. Collectively exhaustive means that we can list all of the possible states of a
additional assumptions of system or process. Our discussion of Markov analysis assumes that there is a finite number of
Markov analysis. states for any system. Mutually exclusive means that a system can be in only one state at any
point in time. A student can be in only one of the three management specialty areas and not in
two or more areas at the same time. It also means that a person can only be a customer of one of
the three grocery stores at any point in time.
 Markov Processes: Examples
Here we give some more Markov processes examples.

1. In analysing switching by Business Class customers between airlines the

following data has been obtained by British Airways (BA):

Next flight by
BA Competition
Last flight by BA 0.85 0.15
Competition 0.10 0.90

For example if the last flight by a Business Class customer was by BA the
probability that their next flight is by BA is 0.85. Business Class customers make 2
flights a year on average.

Currently BA have 30% of the Business Class market. What would you forecast
BA's share of the Business Class market to be after two years?

Solution

We have the initial system state s1 given by s1 = [0.30, 0.70] and the transition
matrix P is given by
2
P = 0.85 0.15 = 0.7375 0.2625
0.10 0.90 0.1750 0.8250

where the square term arises as Business Class customers make 2 flights a year on
average.

Hence after one year has elapsed the state of the system s 2 = s1P = [0.34375,
0.65625]

After two years have elapsed the state of the system = s3 = s2P = [0.368, 0.632]

and note here that the elements of s2 and s3 add to one (as required).

So after two years have elapsed BA's share of the Business Class market is 36.8%
 2. An admissions tutor is analysing applications from potential students for a
particular undergraduate course at Imperial College (IC). She regards each
potential student as being in one of four possible states:

 State 1: has not applied to IC

 State 2: has applied to IC but an accept/reject decision has not yet been made
 State 3: has applied to IC and has been rejected
 State 4: has applied to IC and has been accepted (been made an offer of a place)

At the start of the year (month 1 in the admissions year) all potential students are in
state 1.

Her review of admissions statistics for recent years has identified the following
transition matrix for the probability of moving between states each month:

To1 2 3 4
From 1 0.97 0.03 0 0
2 0 0.10 0.15 0.75
3 0 0 1 0
4 0 0 0 1
 What percentage of potential students will have been accepted after 3
months have elapsed?
 Is it possible to work out a meaningful long-run system state or not (and
why)?

The admissions tutor has control over the elements in one row of the above
transition matrix, namely row 2.

The elements in this row reflect:

 transition from 2 to 2: the speed with which applications are processed each
month
 transition from 2 to 3: the proportion of applicants who are rejected each
month
 transition from 2 to 4: the proportion of applicants who are accepted each
month.
To be more specific, at the start of each month the admissions tutor has to decide
the proportion of applicants who should be accepted that month. However she is
constrained by a policy decision that, at the end of each month, the total number of
rejections should never be more than one-third of the total number of offers, nor
should it ever be less than 20% of the total number of offers.

Further analysis reveals that applicants who wait longer than 2 months between
applying to IC and receiving a decision (reject or accept) almost never choose to
come to IC, even if they get an offer of a place.

Formulate the problem that the admissions tutor faces each month as a linear
program. Comment on any assumptions you have made in so doing.

Solution

We have the initial system state s1 given by s1 = [1, 0, 0, 0] and the transition
matrix P given by

P = | 0.97 0.03 0 0 |
| 0 0.10 0.15 0.75 |
| 0 0 1 0 |
| 0 0 0 1 |

Hence after one month has elapsed the state of the system s 2 = s1P = [0.97, 0.03, 0,
0]

After two months have elapsed the state of the system = s3 = s2P = [0.9409, 0.0321,
0.0045, 0.0225]

After three months have elapsed the state of the system = s4 = s3P = [0.912673,
0.031437, 0.009315, 0.046575]

and note here that the elements of s2, s3 and s4 add to one (as required).

Hence 4.6575% of potential students will have been accepted after 3 months have
elapsed.

It is not possible to work out a meaningful long-run system state as the admissions
year is only (at most) 12 months long. In reality the admissions year is probably
shorter than 12 months.
 3. A petrol station owner is considering the effect on his business (Superpet) of
a new petrol station (Global) which has opened just down the road.
Currently (of the total market shared between Superpet and Global) Superpet
has 80% of the market and Global has 20%.

Analysis over the last week has indicated the following probabilities for customers
switching the station they stop at each week:

To
Superpet Global
From Superpet 0.75 0.25
Global 0.55 0.45

 What will be the expected market share for Superpet and Global after
another two weeks have past?
 What would be the long-run prediction for the expected market share for
Superpet and Global?

Solution

Letting

 state 1 = Superpet
 state 2 = Global

we have the initial system state s1 given by s1 = [0.80, 0.20] and the transition
matrix P given by

P = |0.75 0.25 |
|0.55 0.45 |

Hence after one week has elapsed the state of the system s2 = s1P = [0.71, 0.29] and
so after two weeks have elapsed the state of the system = s 3 = s2P = [0.692, 0.308]
and note here that the elements of s2 and s3 add to one (as required).

Hence the market shares after two weeks have elapsed are 69.2% and 30.8% for
Superpet and Global respectively.

Assuming that in the long-run the system reaches an equilibrium [x1, x2] where

[x1, x2] = [x1, x2]P and x1 + x2 = 1

we have that

x1 = 0.75x1 + 0.55x2 (1)

x2 = 0.25x1 + 0.45x2 (2)
and x1 + x2 = 1 (3)

From (3) we have that x2 = 1-x1

so substituting into (1) we get

x1 = 0.75x1 + 0.55(1-x1)

i.e. (1-0.75+0.55)x1 = 0.55

i.e. x1 = 0.55/0.80 = 0.6875

Hence x2 = 1-x1 = 1-0.6875 = 0.3125

Note that as a check we have that these values for x 1 and x2 satisfy equations (1) -
(3) (to within rounding errors).

Hence the long-run market shares are 68.75% and 31.25% for Superpet and Global
respectively.

4. A company is considering using Markov theory to analyse brand switching

between three different brands of floppy disks. Survey data has been
gathered and has been used to estimate the following transition matrix for
the probability of moving between brands each month:

To Brand
1 2 3
From Brand 1 | 0.80 0.10 0.10
2 | 0.03 0.95 0.02
3 | 0.20 0.05 0.75

The current (month 1) market shares are 45%, 25% and 30% for brands 1, 2 and 3
respectively.

 What will be the expected market shares after two months have elapsed (i.e.
in month 3)?
  What is the long-run prediction for the expected market share for each of the
three brands?
 Would you expect the actual market share to approach the long-run
prediction for the market or not (and why)?

Solution

We have the initial system state s1 given by s1 = [0.45, 0.25, 0.30] and the
transition matrix P given by

P = |0.80 0.10 0.10 |

|0.03 0.95 0.02 |
|0.20 0.05 0.75 |

Hence after one month has elapsed the state of the system s2 = s1P = [0.4275,
0.2975, 0.2750] and so after two months have elapsed the state of the system = s 3 =
s2P = [0.4059, 0.3391, 0.2550] and note here that the elements of s 2 and s3 add to
one (as required).

Hence the market shares after two months have elapsed are 40.59%, 33.91% and
25.50% for brands 1, 2 and 3 respectively.

Assuming that in the long-run the system reaches an equilibrium [x1, x2, x3] where

[x1, x2, x3] = [x1, x2, x3]P and x1 + x2 + x3 = 1

we have that

x1 = 0.80x1 + 0.03x2 + 0.20x3

x2 = 0.10x1 + 0.95x2 + 0.05x3

x3 = 0.10x1 + 0.02x2 + 0.75x3

x1 + x2 + x3 = 1

Rearranging we get

0.20x1 = 0.03x2 + 0.20x3 (1)

0.05x2 = 0.10x1 + 0.05x3 (2)
0.25x3 = 0.10x1 + 0.02x2 (3)
x1 + x2 + x3 = 1 (4)
Now subtracting equation (3) from equation (2) we get

0.05x2 - 0.25x3 = 0.05x3 - 0.02x2

i.e. 0.07x2 = 0.30x3

i.e. x2 = (0.30/0.07)x3 (5)

Now from equation (4) we have

x1 = 1 - x2 - x3

so substituting in equation (1) we get

0.20(1 - x2 - x3) = 0.03x2 + 0.20x3

i.e. 0.20 = 0.23x2 + 0.40x3

Substituting from equation (5) for x2 we get

0.20 = 0.23(0.30/0.07)x3 + 0.40x3

i.e. x3 = 0.1443

Hence from equation (5)

x2 = (0.30/0.07)x3 = 0.6184

and x1 = 1 - x2 - x3 = 1 - 0.6184 - 0.1443 = 0.2373

Note that as a check we have that these values for x1, x2 and x3 satisfy equations (1)
- (4) (to within rounding errors).

Hence the long-run market shares are 23.73%, 61.84% and 14.43% for brands 1, 2
and 3 respectively.

Have that after two months s3 (calculated above) not close to the long-run market
shares and would expect changing circumstances to render the transition matrix
invalid.
 5. In analysing switching between different brands of copper pipe survey data
has been used to estimate the following transition matrix for the probability
of moving between brands each month:

To Brand
1 2 3 4
From Brand 1 | 0.95 0.02 0.02 0.01
2 | 0.05 0.90 0.02 0.03
3 | 0.10 0.05 0.83 0.02
4 | 0.13 0.13 0.02 0.72

The current (month 1) market shares are 45%, 23%, 20% and 12% for brands 1, 2,
3 and 4 respectively.

 What will be the expected market shares after two months have elapsed (i.e.
in month 3)?
 What is the long-run prediction for the expected market share for each of the
four brands?

Solution

We have the initial system state s1 given by s1 = [0.45, 0.23, 0.20, 0.12] and the
transition matrix P given by

P = | 0.95 0.02 0.02 0.01 |

| 0.05 0.90 0.02 0.03 |
| 0.10 0.05 0.83 0.02 |
| 0.13 0.13 0.02 0.72 |

Hence after one month has elapsed the state of the system s 2 = s1P = [0.4746,
0.2416, 0.1820, 0.1018] and so after two months have elapsed the state of the
system = s3 = s2P = [0.494384, 0.249266, 0.16742, 0.08893] and note here that the
elements of s2 and s3 add to one (as required).

Hence the market shares after two months have elapsed are 49.44%, 24.93%,
16.74% and 8.89% for brands 1, 2, 3 and 4 respectively.

Assuming that in the long-run the system reaches an equilibrium [x1, x2, x3, x4]
where

[x1, x2, x3, x4] = [x1, x2, x3, x4]P and x1 + x2 + x3 + x4 = 1

we have that

x1 = 0.95x1 + 0.05x2 + 0.10x3 + 0.13x4

x2 = 0.02x1 + 0.90x2 + 0.05x3 + 0.13x4

x3 = 0.02x1 + 0.02x2 + 0.83x3 + 0.02x4

x4 = 0.01x1 + 0.03x2 + 0.02x3 + 0.72x4

x1 + x2 + x3 + x4 = 1

Rearranging we get

0.05x1 = 0.05x2 + 0.10x3 + 0.13x4 (1)

0.10x2 = 0.02x1 + 0.05x3 + 0.13x4 (2)
0.17x3 = 0.02x1 + 0.02x2 + 0.02x4 (3)
0.28x4 = 0.01x1 + 0.03x2 + 0.02x3 (4)
x1 + x2 + x3 + x4 = 1 (5)

Now from equation (3) we have

0.17x3 = 0.02(x1 + x2 + x4)

and from equation (5) we have

x1 + x2 + x4 = 1 - x3

Hence

0.17x3 = 0.02(1-x3)

i.e. 0.19x3 = 0.02

i.e. x3 = (0.2/0.19) = 0.10526

Now subtracting equation (2) from equation (1) we get

0.05x1 - 0.10x2 = 0.05x2 + 0.10x3 - 0.02x1 - 0.05x3

i.e. 0.07x1 - 0.15x2 = 0.05x3 (6)

Also substituting for x4 from equation (5) in equation (4) we have

0.28(1 - x1 - x2 - x3) = 0.01x1 + 0.03x2 + 0.02x3

i.e. 0.28 = 0.29x1 + 0.31x2 + 0.30x3

i.e. 0.29x1 + 0.31x2 = 0.28 - 0.30x3 (7)

Multiplying equation (6) by 0.31 and equation (7) by 0.15 and adding we get

(0.31)(0.07)x1 + (0.15)(0.29)x1 = (0.31)(0.05)x3 + (0.15)(0.28) - (0.15)(0.30)x3

and since we know x3 = 0.10526 we have x1 = 0.59655

Hence from equation (6) we find that x2 = 0.24330 and from equation (5) that x4 =
0.05489

Note that as a check we have that these values for x1, x2, x3 and x4 satisfy equations
(1) - (5) (to within rounding errors).

Hence the long-run market shares are 59.66%, 24.33%, 10.53% and 5.49% for
brands 1, 2, 3 and 4 respectively.