Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 237 views

    04-Markov Chain

    Uploaded by

    aziz
    The document describes several examples of Markov chains modeling different scenarios. For the first scenario, the document represents the computer purchase preferences of an engineering professor as a Markov chain and calculates the probability of purchasing the current model in four years. For the second scenario, it models the book borrowing habits from a library as a Markov chain and calculates the average number of weeks a book is kept before being returned. The third scenario models population dynamics between rural, suburban and inner city populations as a Markov chain and calculates the population distribution in future time periods and the long-run population levels. It then provides answers to questions about additional Markov chain examples modeling ex-con recidivism rates and costs, a token game

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    04-Markov Chain

    Uploaded by

    aziz
    237 views29 pages
    The document describes several examples of Markov chains modeling different scenarios. For the first scenario, the document represents the computer purchase preferences of an engineering professor as a Markov chain and calculates the probability of purchasing the current model in four years. For the second scenario, it models the book borrowing habits from a library as a Markov chain and calculates the average number of weeks a book is kept before being returned. The third scenario models population dynamics between rural, suburban and inner city populations as a Markov chain and calculates the population distribution in future time periods and the long-run population levels. It then provides answers to questions about additional Markov chain examples modeling ex-con recidivism rates and costs, a token game

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document describes several examples of Markov chains modeling different scenarios. For the first scenario, the document represents the computer purchase preferences of an engineering professor as a Markov chain and calculates the probability of purchasing the current model in four years. For the second scenario, it models the book borrowing habits from a library as a Markov chain and calculates the average number of weeks a book is kept before being returned. The third scenario models population dynamics between rural, suburban and inner city populations as a Markov chain and calculates the population distribution in future time periods and the long-run population levels. It then provides answers to questions about additional Markov chain examples modeling ex-con recidivism rates and costs, a token game

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    237 views29 pages

    04-Markov Chain

    Uploaded by

    aziz
    The document describes several examples of Markov chains modeling different scenarios. For the first scenario, the document represents the computer purchase preferences of an engineering professor as a Markov chain and calculates the probability of purchasing the current model in four years. For the second scenario, it models the book borrowing habits from a library as a Markov chain and calculates the average number of weeks a book is kept before being returned. The third scenario models population dynamics between rural, suburban and inner city populations as a Markov chain and calculates the population distribution in future time periods and the long-run population levels. It then provides answers to questions about additional Markov chain examples modeling ex-con recidivism rates and costs, a token game

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 29

    Markov Chain

    Question 1
    An engineering professor purchases a new computer every two years
    with preferences for the three model: M1, M2, and M3.
    If the present model is M1, the next computer may be M2 with
    probability 0.2, or M3 with probability 0.15.
    If present model is M2, the probabilities of switching to M1 and M3
    are 0.6 and 0.25, respectively.
    And, if the present model is M3, then the probabilities of switching to
    M1 and M2 are 0.5 and 0.1, respectively.
    Represent the situation as a Markov chain and determine the
    probability that the professor will purchase the current model in four
    years
    States:
    1. M1
    2. M2
    3. M3

    The Markov Chain is:


    The probability that the professor will purchase the current model in 4
    years. P (2-step)

    𝑃 𝑀1 𝑀1 = 0.6175

    𝑃 𝑀2 𝑀2 = 0.1675

    𝑃 𝑀3 𝑀3 = 0.26
    Question 2
    When I borrow a book from the city library, I usually try to return it
    after one week. Depending on the length of the book and my free time,
    there is a 30% chance that I may keep it for another week. If I have had
    the book for two weeks, there is a 10% chance that I will keep it for an
    additional week. Under no condition do I keep it for more than three
    weeks.
    • Express the situation as a Markov chain.
    • Determine the average number of weeks I keep a book before
    returning it to the library.
    States:
    1. 1: 1 Week
    2. 2: 2 Weeks
    3. 3: 3 Weeks
    4. L: Library
    The Markov Chain is:
    b) Determine the average number of weeks before returning a book to
    the library is 𝐼 − 𝑁 −1 ∗ 1
    I will keep the book 1.33 week on an average
    Question 3
    Population dynamics is impacted by the continual movement of people
    who are seeking better quality of life or better employment. The city of
    Mobile has an inner city population, a suburban population, and a
    surrounding rural population. The census taken in 10-year intervals
    shows that 10% of the rural population move to the suburbs and 5% to
    the inner city. For the suburban population, 30% move to rural areas
    and 15% to the inner city. Inner-city population would not move into
    suburbs, but 20% of them move to the quiet rural life.
    • Express the population dynamics as a Markov chain.
    • If the greater Mobile area currently includes 20,000 rural residents,
    100,000 suburbanites, and 30,000 inner city inhabitants, what will the
    population distribution be in 10 years? In 20 years?
    • Determine the long-run population picture of Mobile.
    States:
    1. Inner city population
    2. Suburban population
    3. Surrounding rural population

    a). The Markov Chain is:


    b). Initial Probabilities
    0 30000 100000 20000
    𝑎 = 150000 150000 150000
    𝑎 0 = 0.2 0.67 0.13

    Population distribution in 10 years


    Population = 150000 * P[1-step] = 150000 * a0 * P
    State Absolute [1-step] Population
    Inner City 0.267 40050
    Suburban 0.387 57225
    Surrounding rural 0.352 52725
    Population distribution in 20 years
    Population = 150000 * P[2-step] = 150000 * a0 * P2

    State Absolute [2-step] Population


    Inner City 0.288 43200
    Suburban 0.245 36750
    Surrounding rural 0.467 70150
    c). Long run population = 150000 * 𝜋𝑖
    𝜋1 𝜋2 𝜋3 = 𝜋1 𝜋2 𝜋3 ∗ 𝑃

    𝜋1 + 𝜋2 + 𝜋3 = 1

    𝜋1 = 0.254238, 𝜋2 = 0.135593, 𝜋3 = 0.61017


    c). Long run population

    𝜋1 = 0.254238, 𝜋2 = 0.135593, 𝜋3 = 0.61017

    State Steady State Population


    Inner City 0.254238 0.254238*150,000
    = 38135
    Suburban 0.135593 0.135593*150,000
    = 20339
    Surrounding rural 0.61017 0.61017*150,000
    = 91526
    Question 4
    Some ex-cons spend the rest of their lives in one four of states: free, on
    trial, in jail, or on probation. At the start of each year, statistics show
    that there is 50% chance that a free ex-con will commit a new crime
    and go on trial. The judge may send the ex-con to jail with probability
    0.6 or grant probation with probability 0.4. Once in jail, 10% of ex-cons
    will be set free for good behavior. Of those who are on probation, 10%
    commit new crimes and are arraigned for new trials, 50% will go back
    to finish their sentence for violating probation orders, and 10% will be
    set free for lack of evidence. Taxpayers underwrite the costs associated
    with the punishment of the ex-felons. It is estimated that a trial will
    cost about $5,000, an average jail sentence will cost $20,000, and an
    average probation period will cost $2,000.
    • Determine the expected cost per ex-con.
    • How often does an ex-con return to jail? Go on trial? Get set free?
    States:
    1. F: free
    2. T: on trial
    3. J: in jail
    4. P: on probation

    The Markov Chain is:


    𝜋1 𝜋2 𝜋3 𝜋4 = 𝜋1 𝜋2 𝜋3 𝜋4 ∗ 𝑃

    The solution is
    𝜋1 = 0.153123 𝜋2 = 0.081206 𝜋3 = 0.719257 𝜋4 = 0.046404

    Expected cost per ex-con


    E[cost/ex-con] = 0 ∗ 0.153123 + 5000 ∗ 0.081206
    +20000 ∗ 0.719257 + 2000 ∗ 0.046404
    = $ 14884
    b). How often does an ex-con return to jail? Go on trial? Be set free?

    1
    𝜇𝐽𝐽 = = 1.39 𝑦𝑒𝑎𝑟𝑠
    𝜋3

    1
    𝜇𝑇𝑇 = = 12.31 𝑦𝑒𝑎𝑟𝑠
    𝜋2

    1
    𝜇𝐹𝐹 = = 6.53 𝑦𝑒𝑎𝑟𝑠
    𝜋1
    Question 5
    Jim and Joe start a game with five tokens, three for Jim and two for Joe.
    A coin is tossed and if the outcome is heads, Jim gives Joe a token; else,
    Jim gets a token from Joe. The game ends when Jim or Joe has all the
    tokens. At this point, there is 30% chance that Jim and Joe will continue
    to play the game, again starting with three tokens for Jim and two for
    Joe.
    a. Represent the game as a Markov chain.
    b. Determine the probability that Joe will win in three-coin tosses.
    That Jim will win in three coin tosses.
    c. Determine the probability that a game will end in Jim's favor. Joe's
    favor.
    d. Determine the average number of coin tosses needed before Jim
    wins. Joe wins.
    States:
    (Jim-Joe)
    3-2, 2-3, 1-4, 4-1, 0-5, 5-0
    Initial Probabilities:
    𝑎0 = 1 0 0 0 0 0
    The Markov Chain is:
    b. probability that Joe will win in three-coin tosses, and that Jim will win in
    three coin tosses.

    Output (3-step) transition matrix

    P{Jim wins in 3 tosses} = P 3 − 2 5 − 0 = 0.175


    P{Joe wins in 3 tosses} = P 3 − 2 0 − 5 = 0.125
    c. probability that a game will end in Jim's favor and Joe's favor.

    Steady State Probabilities:


    𝜋1 𝜋2 𝜋3 𝜋4 𝜋5 𝜋6

    𝜋1 = 0.257143 𝜋2 = 0.171429
    𝜋3 = 0.085714 𝜋4 = 0.128571
    𝜋5 = 0.142857 𝜋6 = 0.214286

    P{game ends in Jim’s favor} = 𝜋6 = 0.214286


    P{game ends in Joe’s favor} = 𝜋5 = 0.142857
    d. average number of coin tosses needed before Jim wins, and Joe wins

    average 12.2 number of coin tosses till Jim wins


    average 20 number of coin tosses till Joe wins
    Question 6
    The registration committee of the IT department at AAA university has modeled the path
    of the student through the department a Markov chain. The student in this department
    can be in the first year, second year, or in he third year of studies. If he finished successfully
    the third then he graduate. Any student can quit the department at any year of studies. We
    give the following transition probabilities matrix.

    a. What is the probability that the student will be in the third year after two years if he is
    currently in the second year?
    b. What is the steady state probability of the Q state?
    c. What is the probability that the student will graduate from this school if he is currently
    in the first year?
    d. What is the probability that the student will graduate from this school in exactly three
    years if he is currently in the first year?
    e. What is the average number of years that the student may spend in this school?
    a. What is the probability that the student will be in the third year after two
    years if he is currently in the second year?

    (2)
    𝑃23 = 𝑃21 ∗ 𝑃13 + 𝑃22 ∗ 𝑃23 + 𝑃23 ∗ 𝑃33 + 𝑃24 ∗ 𝑃43 + 𝑃25 ∗ 𝑃53

    (2)
    𝑃23 = 0 ∗ 0 + 0.3 ∗ 0.5 + 0.5 ∗ 0.2 + 0.2 ∗ 0 + 0 ∗ 0

    (2)
    𝑃23 = 0.25

    b. What is the steady state probability of Q State?

    This is not an ergodic Markov chain, So we cannot compute the steady State
    probabilities.
    c. What is the probability that the student will graduate from this
    school if he is currently in the first year?
    𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑡𝑖𝑒𝑠 𝑜𝑓 𝑎𝑏𝑠𝑜𝑟𝑝𝑡𝑖𝑜𝑛 = 𝐼 − 𝑁 −1 ∗ 𝐴

    probability that the student will graduate from this school if he is


    currently in the first year is 0.55
    d. The probability that the student will graduate from this school in exactly three years.
    P(3-step)

    𝑎0 ∗ 𝑃 3 = 0.01 0.13 0.25 0.37 0.25

    So, The probability that the student will graduate from this school in exactly three years is
    0.25
    e. What is the average number of years that the student may spend in
    this school?

    the average number of years that the student may spend in this school
    is 3.28 years

    You might also like