Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Ergodic and adaptive control of hidden Markov models

  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

A partially observed stochastic system is described by a discrete time pair of Markov processes. The observed state process has a transition probability that is controlled and depends on a hidden Markov process that also can be controlled. The hidden Markov process is completely observed in a closed set, which in particular can be the empty set and only observed through the other process in the complement of this closed set. An ergodic control problem is solved by a vanishing discount approach. In the case when the transition operators for the observed state process and the hidden Markov process depend on a parameter and the closed set, where the hidden Markov process is completely observed, is nonempty and recurrent an adaptive control is constructed based on this family of estimates that is almost optimal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bielecki TR, Pliska SR. (1999). Risk sensitive dynamic asset management. JAMO 39:337–360

    MATH  MathSciNet  Google Scholar 

  • Bielecki TR, Rutkowski M. (2002). Credit risk: modelling, valuation and hedging. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  • Borkar VS. (2003). Dynamic programming for ergodic control with partial observation. Stoch Proc Appl 103:293–310

    Article  MATH  MathSciNet  Google Scholar 

  • Borkar VS, Budhiraja A. (2004). Further remark on dynamic programming for partially observed Markov processes. Stoch Proc Appl 112(1):79–93

    Article  MATH  MathSciNet  Google Scholar 

  • Borkar VS, Mundra SM. (1998). Bayesian parameter estimation and adaptive control of Markov processes with time - averaged cost. Appl Math 25(3):339–358

    MATH  MathSciNet  Google Scholar 

  • Brennan MJ, Schwarz ES, Lagnado R. (1997). Stategic asset allocation. J Econ Dyn Control 21:1377–1403

    Article  MATH  Google Scholar 

  • Di Masi GB, Stettner L. (1995). Bayesian ergodic control of discrete time Markov processes. Stoch Stoch Rep 54:301–316

    MATH  MathSciNet  Google Scholar 

  • Doob JL. (1953). Stochastic processes. Wiley, New York

    MATH  Google Scholar 

  • Duncan TE, Pasik-Duncan B, Stettner L. (1998). Adaptive control of a partially observed discrete time Markov process. Appl Math Optim 37:269–293

    Article  MATH  MathSciNet  Google Scholar 

  • Elliott RJ, Aggoun L, Moore JB. (1996). Hidden Markov models: estimation and control. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Hernandez-Lerma D, Lasserre JB. (1996). Discrete time Markov control processes: basic optimality criteria. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Kazangey T, Sworder DD (1971) Effective federal policies for regulating residential housing. In: Proceedings of summer computer simulation conference, Los Angeles, pp 1120–1128

  • Korn R. (1997). Optimal portfolios. stochastic models for optimal investment and risk management in continuous time. World Scientific, Singapore

    MATH  Google Scholar 

  • Mariton M. (1990). Jump linear systems in automatic control. Marcel Dekker, New York and Bassel

    Google Scholar 

  • Montgomery RC. (1983). Reliability considerations in placement of control systems components. In: Proceedings of AIAA guidance and control conference, Gatlinburg

  • Royden HL. (1968). Real analysis. MacMillan, New York

    Google Scholar 

  • Runggaldier WJ, Stettner L. (1994). Approximations of discrete time partially observed control problems. Giardini, Pisa

    Google Scholar 

  • Sethi SP, Zhang Q. (1994). Hierarchical decision making in stochastic manufacturing systems. Birkhäuser, Boston Basel Berlin

    MATH  Google Scholar 

  • Stettner L. (1993). Ergodic control of partially observed Markov processes with equivalent transition probabilities. Appl Math 22:25–38

    MATH  MathSciNet  Google Scholar 

  • Stettner L. (2002). Bayesian adaptive control of discrete time partially observed Markov processes. In: Proceedings of workshop stochastic theory and control, Lect Notes Control Info Sci 280:435–446

  • Willsky AS, Levy BC. (1979). Stochastic stability research for complex power systems. Lab. Inf. Decision Systems, Mass. Inst. Technology, report no. ET-76-C-01-2295.

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Kansas, Lawrence, KS, 66045, USA

    T. E. Duncan & B. Pasik-Duncan

  2. Institute of Mathematics, Polish Academy of Sciences, Warsaw, Poland, USA

    L. Stettner

Authors
  1. T. E. Duncan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. B. Pasik-Duncan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. L. Stettner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. E. Duncan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Duncan, T.E., Pasik-Duncan, B. & Stettner, L. Ergodic and adaptive control of hidden Markov models. Math Meth Oper Res 62, 297–318 (2005). https://doi.org/10.1007/s00186-005-0010-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-005-0010-z

Keywords

Search

Navigation