Article

Blind construction of optimal nonlinear recursive predictors for discrete sequences

Authors:

Cosma Rohilla Shalizi,

Kristina Lisa ShaliziAuthors Info & Claims

UAI '04: Proceedings of the 20th conference on Uncertainty in artificial intelligence

Pages 504 - 511

Published: 07 July 2004 Publication History

Abstract

We present a new method for nonlinear prediction of discrete random sequences under minimal structural assumptions. We give a mathematical construction for optimal predictors of such processes, in the form of hidden Markov models. We then describe an algorithm, CSSR (Causal-State Splitting Reconstruction), which approximates the ideal predictor from data. We discuss the reliability of CSSR, its data requirements, and its performance in simulations. Finally, we compare our approach to existing methods using variable-length Markov models and cross-validated hidden Markov models, and show theoretically and experimentally that our method delivers results superior to the former and at least comparable to the latter.

References

[1]

C. R. Shalizi. Causal Architecture, Complexity and Self-Organization in Time Series and Cellular Automata. PhD thesis, University of Wisconsin-Madison, 2001. http://bactra.org/thesis/.

Digital Library

[2]

C. R. Shalizi, K. L. Shalizi, and J. P. Crutchfield. An algorithm for pattern discovery in time series. Technical Report 02--10-060, Santa Fe Institute, 2002. arxiv.org/abs/cs.LG/0210025.

[3]

S. Kullback. Information Theory and Statistics. Dover Books, New York, 2nd edition, 1968.

[4]

D. Blackwell and M. A. Girshick. Theory of Games and Statistical Decisions. Wiley, New York, 1954.

[5]

J. P. Crutchfield and K. Young. Inferring statistical complexity. Physical Review Letters, 63:105--108, 1989.

[6]

C. R. Shalizi and J. P. Crutchfield. Computational mechanics: Pattern and prediction, structure and simplicity. Journal of Statistical Physics, 104:817--879, 2001. arxiv.org/abs/cond-mat/9907176.

[7]

F. B. Knight. A predictive view of continuous time processes. The Annals of Probability, 3:573--596, 1975.

[8]

W. C. Salmon. Scientific Explanation and the Causal Structure of the World. Princeton University Press, Princeton, 1984.

[9]

M. L. Littman, R. S. Sutton, and S. Singh. Predictive representations of state. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 1555--1561, Cambridge, Massachusetts, 2002. MIT Press.

[10]

S. Singh, M. L. Littman, N. K. Jong, D. Pardoe, and P. Stone. Learning predictive state representations. In T. Fawcett and N. Mishra, editors, Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), pages 712--719. AAAI Press, 2003.

Digital Library

[11]

J. Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge, England, 2000.

Digital Library

[12]

H. R. Lewis and C. H. Papadimitriou. Elements of the Theory of Computation. Prentice-Hall, Upper Saddle River, New Jersey, second edition, 1998.

Digital Library

[13]

D. R. Upper. Theory and Algorithms for Hidden Markov Models and Generalized Hidden Markov Models. PhD thesis, University of California, Berkeley, 1997. http://www.santafe.edu/projects/CompMech/papers/TAHMMGHMM.html.

Digital Library

[14]

T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, New York, 1991.

Digital Library

[15]

L. Devroye and G. Lugosi. Combinatorial Methods in Density Estimation. Springer-Verlag, Berlin, 2001.

[16]

J. Feldman and J. F. Hanna. The structure of responses to a sequence of binary events. Journal of Mathematical Psychology, 3:371--387, 1966.

[17]

K. Marton and P. C. Shields. Entropy and the consistent estimation of joint distributions. The Annals of Probability, 23:960--977, 1994. See also Correction, Annals of Probability,24 (1996):541--545.

[18]

J. Rissanen. A universal data compression system. IEEE Transactions on Information Theory, 29:656--664, 1983.

Digital Library

[19]

P. Bühlmann and A. J. Wyner. Variable length Markov chains. The Annals of Statistics, 27:480--513, 1999.

[20]

F. Willems, Y. Shtarkov, and T. Tjalkens. The context-tree weighting method: Basic properties. IEEE Transactions on Information Theory, 41:653--664, 1995.

Digital Library

[21]

P. Tino and G. Dorffner. Predicting the future of discrete sequences from fractal representations of the past. Machine Learning, 45:187--217, 2001.

Digital Library

[22]

M. B. Kennel and A. I. Mees. Context-tree modeling of observed symbolic dynamics. Physical Review E, 66:056209, 2002.

[23]

D. Ron, Y. Singer, and N. Tishby. The power of amnesia: Learning probabilistic automata with variable memory length. Machine Learning, 25:117--149, 1996.

[24]

B. Weiss. Subshifts of finite type and sofic systems. Monatshefte für Mathematik, 77:462--474, 1973.

[25]

R. Badii and A. Politi. Complexity: Hierarchical Structures and Scaling in Physics. Cambridge University Press, Cambridge, 1997.

[26]

T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag, New York, 2001.

[27]

R. A. McCallum. Instance-based utile distinctions for reinforcement learning with hidden state. In A. Prieditis and S. J. Russell, editors, Proceedings of the Twelth International Machine Learning Conference (ICML 1995), pages 387--395, San Francisco, 1995. Morgan Kauffman.

[28]

H. Jaeger. Observable operator models for discrete stochastic time series. Neural Computation, 12:1371--1398, 2000.

Digital Library

[29]

H. J. Bussemaker, H. Li, and E. D. Siggia. Building a dictionary for genomes: Identification of presumptive regulatory sites by statistical analysis. Proceedings of the National Academy of Sciences, 97:10096--10100, 2000.

Cited By

Liu YZeng YZhu HTang YLarson KWinikoff MDas SDurfee E(2017)Making and Improving Predictions of Interest Using an MDP ModelProceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems10.5555/3091125.3091379(1610-1612)Online publication date: 8-May-2017
https://dl.acm.org/doi/10.5555/3091125.3091379
Parikh NMarathe MSwarup SJonker CMarsella SThangarajah JTuyls K(2016)Simulation SummarizationProceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems10.5555/2936924.2937205(1451-1452)Online publication date: 9-May-2016
https://dl.acm.org/doi/10.5555/2936924.2937205
Harada JDarmon DGirvan MRand W(2015)Forecasting High TideProceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 201510.1145/2808797.2809392(504-507)Online publication date: 25-Aug-2015
https://dl.acm.org/doi/10.1145/2808797.2809392
Show More Cited By

Index Terms

Blind construction of optimal nonlinear recursive predictors for discrete sequences
1. Mathematics of computing
  1. Probability and statistics
    1. Probabilistic representations
      1. Markov networks
    2. Stochastic processes
      1. Markov processes
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Markov decision processes

Index terms have been assigned to the content through auto-classification.

Recommendations

Product models for sequences
Optimal control of affine nonlinear discrete-time systems
MED '09: Proceedings of the 2009 17th Mediterranean Conference on Control and Automation

In this paper, direct neural dynamic programming techniques are utilized to solve the Hamilton Jacobi-Bellman equation in real time for the optimal control of general affine nonlinear discrete-time systems. In the presence of partially unknown dynamics, ...
Discrete-Time Inverse Optimal Control for Nonlinear Systems

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

UAI '04: Proceedings of the 20th conference on Uncertainty in artificial intelligence

July 2004

657 pages

ISBN:0974903906

Conference Chair:
Christopher Meek
Microsoft Research
,
Program Chairs:
Max Chickering
Microsoft Reasearch
,
Joseph Halpern
Cornell University

Sponsors

Alberta Ingenuity Centre for Machine Learning
Sun Microsystems of Canada
Hewlett-Packard Laboratories
Information Extraction and Transportation
Informatics Circle of Research Excellence
Yahoo! Research Labs
IBMR: IBM Research
Intel: Intel
Microsoft Research: Microsoft Research
Pacific Institute of Mathematical Sciences
Boeing
University of Alberta: University of Alberta
Northrop Grumman Corporation

Publisher

AUAI Press

Arlington, Virginia, United States

Publication History

Published: 07 July 2004

Check for updates

Qualifiers

Article

Conference

UAI '04

Sponsor:

IBMR
Intel
Microsoft Research
University of Alberta

UAI '04: Uncertainty in artificial intelligence

July 7 - 11, 2004

Banff, Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
396
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)4

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu YZeng YZhu HTang YLarson KWinikoff MDas SDurfee E(2017)Making and Improving Predictions of Interest Using an MDP ModelProceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems10.5555/3091125.3091379(1610-1612)Online publication date: 8-May-2017
https://dl.acm.org/doi/10.5555/3091125.3091379
Parikh NMarathe MSwarup SJonker CMarsella SThangarajah JTuyls K(2016)Simulation SummarizationProceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems10.5555/2936924.2937205(1451-1452)Online publication date: 9-May-2016
https://dl.acm.org/doi/10.5555/2936924.2937205
Harada JDarmon DGirvan MRand W(2015)Forecasting High TideProceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 201510.1145/2808797.2809392(504-507)Online publication date: 25-Aug-2015
https://dl.acm.org/doi/10.1145/2808797.2809392
Wen YRay APhoha S(2013)Hilbert space formulation of symbolic systems for signal representation and analysisSignal Processing10.1016/j.sigpro.2013.02.00293:9(2594-2611)Online publication date: 1-Sep-2013
https://dl.acm.org/doi/10.1016/j.sigpro.2013.02.002
Wen YMukherjee KRay A(2013)Adaptive pattern classification for symbolic dynamic systemsSignal Processing10.1016/j.sigpro.2012.08.00293:1(252-260)Online publication date: 1-Jan-2013
https://dl.acm.org/doi/10.1016/j.sigpro.2012.08.002
Henter GKleijn W(2013)Picking up the piecesPattern Recognition Letters10.1016/j.patrec.2012.11.01334:5(587-594)Online publication date: 1-Apr-2013
https://dl.acm.org/doi/10.1016/j.patrec.2012.11.013
Mahmud M(2010)Constructing states for reinforcement learningProceedings of the 27th International Conference on International Conference on Machine Learning10.5555/3104322.3104415(727-734)Online publication date: 21-Jun-2010
https://dl.acm.org/doi/10.5555/3104322.3104415
Pfau DBartlett NWood F(2010)Probabilistic deterministic infinite automataProceedings of the 24th International Conference on Neural Information Processing Systems - Volume 210.5555/2997046.2997111(1930-1938)Online publication date: 6-Dec-2010
https://dl.acm.org/doi/10.5555/2997046.2997111
Chen CNagl SClack C(2008)A method for validating and discovering associations between multi-level emergent behaviours in agent-based simulationsProceedings of the 2nd KES International conference on Agent and multi-agent systems: technologies and applications10.5555/1787839.1787841(1-10)Online publication date: 26-Mar-2008
https://dl.acm.org/doi/10.5555/1787839.1787841
Auer CWüchner PMeer H(2008)A Method to Derive Local Interaction Strategies for Improving Cooperation in Self-Organizing SystemsProceedings of the 3rd International Workshop on Self-Organizing Systems10.5555/1485753.1485773(170-181)Online publication date: 10-Dec-2008
https://dl.acm.org/doi/10.5555/1485753.1485773
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten