CH 16 Markov Analysis

Chapter 16 Markov Analysis
Prepared by Lee Revere and John Large

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna
16-1
2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458
Learning Objectives
Students will be able to:
Determine future states or conditions using Markov analysis. Compute long-term or steady-state conditions using only the matrix of transition. Understand the use of absorbing state analysis in predicting future conditions.
16-2
Chapter Outline
16.1 Introduction 16.2 States and State Probabilities 16.3 Matrix of Transition Probabilities 16.4 Predicting Future Market Share 16.5 Markov Analysis of Machine Operations 16.6 Equilibrium Conditions 16.7 Absorbing States and the Fundamental Matrix: Accounts Receivable Applications
16-3
Introduction
Markov Analysis
A technique dealing with probabilities of future occurrences with currently known probabilities Numerous applications in
Business (e.g., market share analysis), Bad debt prediction University enrollment predictions Machine breakdown prediction
16-4
Markov Analysis
Matrix of Transition Probabilities
Shows the likelihood that the system will change from one time period to the next This is the Markov Process.
It enables the prediction of future states or conditions.
16-5
States and State Probabilities

States are
used to identify all possible conditions of a process or system
A system can exist in only one state at a time. Examples include:

Working and broken states of a machine
Three shops in town, with a customer able to patronize one at a time

Courses in a student schedule with the student able to occupy only one class at a time
16-6
Assumptions of Markov Analysis

1. A finite number of possible states 2. Probability of changing states remains the same over time 3. Future state predictable from previous state and the transition matrix 4. Size and states of the system remain the same during analysis 5. States are collectively exhaustive
All possible states have been identified
6. States are mutually exclusive

Only one state at a time is possible
16-7
States and State Probabilities continued

1. Identify all states. 2. Determine the probability that the system is in this state.
This information is placed into a vector of state probabilities.
p(i) = vector of state probabilities for period i
= (p1, p2, p3,,pn)

where n = number of states p1, p2,,pn = P (being in state 1, 2, , state n)
16-8

For example:
If dealing with only 1 machine, it may be known that it is currently functioning correctly. The vector of states can then be shown.
p(1) = (1,0)
where
p(1) = vector of states for the machine in period 1 p1 = 1 = P (being in state 1) = P (machine working) p2 = 0 = P (being in state 2) = P (machine broken)
Most of the time, problems deal with more than one item!
16-9

Three Grocery Store example:
100,000 customers monthly for the 3 grocery stores

State 1 = store 1 = 40,000/100,000 = 40% State 2 = store 2 = 30,000/100,000 = 30% State 3 = store 3 = 30,000/100,000 = 30% p(1) = (0.4,0.3,0.3)
vector of state probabilities: where p(1) = vector of state probabilities in period 1 p1 = 0.4 = P (of a person being in store 1) p2 = 0.3 = P (of a person being in store 2) p3 = 0.3 = P (of a person being in store 3)
16-10

Three Grocery Store example, continued:
Probabilities in the vector of states for the stores represent market share for the first period. In period 1, the market shares are Store 1: 40% Store 2: 30%
Store 3: 30%
But, every month, customers who frequent one store have a likelihood of visiting another store. Customers from each store have different probabilities for visiting other stores.
16-11

Store-specific customer probabilities for visiting a
store in the next month:

Store 1: Return to Store 1 = 80% Visit Store 2 = 10% Store 2: Visit Store 1 = 10% Return to Store 2 = 70%
Visit Store 3 = 10%

Store 3: Visit Store 1 = 20% Visit Store 2 = 20% Return to Store 3 = 60%
Visit Store 3 = 20%
16-12

Combining the starting market share with the
customer probabilities for visiting a store next period

yields the market shares in the next period: Initial Share P(1) P(2) P(3)
Store 1:
Next Period: Store 2: Next Period: Store 3: Next Period: New Shares:
40%
30% 30%
80%
32% 10% 3% 20% 6% 41%
16-13
10%
4% 70% 21% 20% 6% 31%
10%
4% 20% 6% 60% 18% 28% = 100%
Matrix of Transition Probabilities

To calculate periodic changes, it is much more convenient to use
a matrix of transition probabilities. a matrix of conditional probabilities of being in a future state given a current state.
Let Pij = conditional probability of being in state j in the future given the current state of i, P (state j at time = 1 | state i at time = 0) For example, P12 is the probability of being in state 2 in the future given the event was in state 1 in the prior period
16-14
Matrix of Transition Probabilities continued

Let P = matrix of transition probabilities
Row Sum
P11 P12 P13 ***** P1n P21 P22 P23 ***** P2n
1
1
P=
* * * *
* * * *
Pm1
Important:
******
Pmn
Each row must sum to 1. But, the columns do NOT necessarily sum to 1.
16-15
Matrix of Transition Probabilities continued

Three Grocery Stores, revisited The previously identified transitional probabilities for each of the stores can now be put into a matrix: 0.8 P= 0.1 0.2 0.1 0.7 0.2 0.1 0.2 0.6
Row 1 interpretation:
0.8 = P11 = P (in state 1 after being in state 1)

16-16
Predicting Future Market Shares

Grocery Store example
A purpose of Markov analysis is to predict the future
Given the
1. 2. vector of state probabilities and matrix of transitional probabilities.
It is easy to find the state probabilities in the future.

This type of analysis allows the computation of the probability that a person will be at one of the grocery stores in the future.
Since this probability is equal to market share, it is possible to determine the future market shares of the grocery store.
16-17
Predicting Future Market Shares continued

Grocery Store example
When the current period is 0, finding the state
probabilities for the next period (1) can be found

using: p(1) = p(0)P Generally, in any period n, the state probabilities for
period n+1 can be computed as:

p(n+1) = p(n)P
16-18
Predicting Future States continued

p (0) = [.4 .3 .3
= state probabilities
.8 .1 .1 P = . 1 .7 .2 .2 .2 .6
p (1) = p (0)P p (1) = [.4 .3 .3
.8 .1 .1 .1 .7 .2 .2 .2 .6
p ( 1) = [(.4*.8 + .3*.1 + .3*.2) +

(.4*.1 + .3*.7 + .3*.2) +
(.4*.1 + .3*.2 + .3*.6)]
p (1) = [ 0.41 0.31 0.28]

16-19
Predicting Future Market Shares continued

In general, p(n) = p(0)Pn
Therefore, the state probabilities n periods in the future can be obtained from the
current state probabilities and the matrix of transition probabilities.
16-20
Markov Analysis of Machine Operations

P=
where 0.8 0.2 0.1 0.9
P11 = 0.8 = probability of machine working this period if working last period
P12 = 0.2 = probability of machine not working this period if working last P21 = 0.1 = probability of machine working this period if not working last P22 = 0.9 = probability machine not working this period if not working last
16-21

continued
What is the probability the machine will be working next month? p(1) = p(0)P 0.8 0.2 = (1,0) 0.1 0.9 = [(1)(0.8)+(0)(0.1), (1)(0.2)+(0)(0.9)] = (0.8, 0.2) If the machine works this month, then there is an 80% chance that it will be working next month and a 20% chance it will be broken.
16-22

continued
What is the probability the machine will be working in two months? p(2) = p(1)P
0.8 0.2 = (0.8, 0.2) 0.1 0.9 = [(0.8)(0.8)+(0.2)(0.1), (0.8)(0.2)+(0.2)(0.9)] = (0.66, 0.34) If the machine works next month, then in two months there is a 66% chance that it will be working and a 34% chance it will be broken.
16-23
Equilibrium Conditions
Equilibrium state probabilities are the long-run average probabilities for being in each state. Equilibrium conditions exist if state probabilities do not change after a large number of periods. At equilibrium, state probabilities for the next period equal the state probabilities for current period.
16-24
Equilibrium Conditions continued

One way to compute the equilibrium share of the market is to use Markov analysis for a large number of periods and see if the future amounts approach stable values. On the next slide, the Markov analysis is repeated for 15 periods for the machine example. By the 15th period, the share of time the machine spends working and broken is around 66% and 34%, respectively.
16-25
Machine Example: Periods to Reach Equilibrium

Period
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
State 1
1.0 .8 .66 .562 .4934 .44538 .411766 .388236 .371765 .360235 .352165 .346515 .342560 .339792 .337854
16-26
State 2
0.0 .2 .34 .438 .5066 .55462 .588234 .611763 .628234 .639754 .647834 .653484 .657439 .660207 .662145
The Markov Process

Equilibrium Conditions
p(n)
Current State
P =
Matrix of Transition
p(n+1)
New State
16-27
Equilibrium Equations
p (i + 1) = p (i)P
p Assume: p (i) = [ 1
Then:
p11 p 2 ], P = p21
p12 p22
[p1
p 2 ]= [ 1 P + p 2 P21 p 1 P + p 2 P22 ] p 11 12
p 2 = p 1 P + p 2 P22 12 p 1 p12
or : p 1 = p 1 P + p 2 P21, 11
Therefore: p 2 p21 p1 = 1 - p11

and p 2 =
1 - p22
16-28
Equilibrium Equations continued

It is always true that p (next period) = p (this period) P or p (n+1) = p (n) P At Equilibrium:
* p (n+1) = p (n) = p (n) P

* p (n) = p (n) P Dropping the n term: p=pP
16-29

Machine Breakdown example At Equilibrium: p=pP
(p1, p2) = (p1, p2)
0.8 0.2 0.1 0.9
Applying matrix multiplication:

(p1, p2) = [(p1)(0.8) + (p2)(0.1), (p1)(0.2) + (p2)(0.9)
Multiplying through yields:

p1 = 0.8 p1 + 0.1 p2 p2 = 0.2 p1 + 0.9 p2
16-30

Machine Breakdown example The state probabilities must sum to 1, therefore: S ps = 1 In this example, then: p1 + p2 = 1 In a Markov analysis, there are always n state equilibrium equations and 1 equation of state probabilities summing to 1.
16-31

Machine Breakdown Example Summarizing the equilibrium equations: p1 = 0.8 p1 + 0.1 p2 p2 = 0.2 p1 + 0.9 p2
p1 + p2 = 1 Solving by simultaneous equations:

p1 = 0.333333 p2 = 0.666667 Therefore, in the long-run, the machine will be functioning 33.33% of the time and broken down 66.67% of the time.
16-32
Absorbing States
Any state that does not have a probability of moving to another state is called an absorbing state. If an entity is in an absorbing state now, the probability of being in an absorbing state in the future is 100%. An example of such a process is accounts receivable.
Bills are either paid, delinquent, or written off as bad debt. Once paid or written off, the debt stays paid or written off.
16-33
Absorbing States
Accounts Receivable example The possible states are
Paid Bad debt Less than 1 month old 1 to 3 months old
A transition matrix for this would look similar to:

Paid Bad <1 1-3
Paid 1 0 0.6 0.4 Bad 0 1 0 0.1 <1 0 0 0.2 0.3 1-3 0 0 0.2 0.2
16-34
Markov Process Fundamental Matrix

I 0
P=
1 0 0 1 0.6 0 0.4 0
A
0 0 0.2 0.3
B
0 0 0.2 0.2
Partition the probability matrix into 4 quadrants to make 4 new sub-matrices: I, 0, A, and B
16-35
Markov Process Fundamental Matrix continued

I 0 Let P = A B
Where I = Identity matrix, and 0 = Null matrix
Then
F = [I - B]
-1
Once F is found, multiply by the A matrix: FA The FA indicates the probability that an amount in one of the non-absorbing states will end up in one of the absorbing states.
16-36
Markov Process Fundamental Matrix continued

Once the FA matrix is found, multiply by the M vector, which is the starting values for the non-absorbing states, MFA, where M = (M1, M2, M3, Mn) The resulting vector will indicate how many observations end up in the first nonabsorbing state and the second nonabsorbing state, respectively.
16-37

CH 16 Markov Analysis

Uploaded by

Copyright:

Available Formats

CH 16 Markov Analysis

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CH 16 Markov Analysis

Uploaded by

Copyright:

Available Formats

Chapter 16 Markov Analysis

Prepared by Lee Revere and John Large

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

States and State Probabilities

A system can exist in only one state at a time. Examples include:

Three shops in town, with a customer able to patronize one at a time

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

Assumptions of Markov Analysis

6. States are mutually exclusive

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

States and State Probabilities continued

p(i) = vector of state probabilities for period i

= (p1, p2, p3,,pn)

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

States and State Probabilities continued

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

States and State Probabilities continued

100,000 customers monthly for the 3 grocery stores

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

States and State Probabilities continued

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

States and State Probabilities continued

store in the next month:

Visit Store 3 = 10%

Visit Store 3 = 20%

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

States and State Probabilities continued

customer probabilities for visiting a store next period

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

Matrix of Transition Probabilities

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

Matrix of Transition Probabilities continued

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

Matrix of Transition Probabilities continued

0.1 = P12 = P (in state 2 after being in state 1)

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

Predicting Future Market Shares

It is easy to find the state probabilities in the future.

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

Predicting Future Market Shares continued

probabilities for the next period (1) can be found

period n+1 can be computed as:

To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna

2006 by Prentice Hall, Inc. Upper Saddle River, NJ 07458

Predicting Future States continued

p (1) = p (0)P p (1) = [.4 .3 .3

p ( 1) = [(.4.8 + .3.1 + .3*.2) +

(.4.1 + .3.2 + .3*.6)]