research-article

Derivatives of Entropy Rate in Special Families of Hidden Markov Chains

Authors:

Guangyue Han,

B. MarcusAuthors Info & Claims

IEEE Transactions on Information Theory, Volume 53, Issue 7

Pages 2642 - 2652

https://doi.org/10.1109/TIT.2007.899467

Published: 01 July 2007 Publication History

Abstract

Consider a hidden Markov chain obtained as the observation process of an ordinary Markov chain corrupted by noise. Recently Zuk et al showed how, in principle, one can explicitly compute the derivatives of the entropy rate of at extreme values of the noise. Namely, they showed that the derivatives of standard upper approximations to the entropy rate actually stabilize at an explicit finite time. We generalize this result to a natural class of hidden Markov chains called "black holes." We also discuss in depth special cases of binary Markov chains observed in binary-symmetric noise, and give an abstract formula for the first derivative in terms of a measure on the simplex due to Blackwell.

Cited By

View all

Tadić VDoucet A(2019)Analyticity of Entropy Rates of Continuous-State Hidden Markov ModelsIEEE Transactions on Information Theory10.1109/TIT.2019.293577765:12(7950-7975)Online publication date: 19-Nov-2019
https://dl.acm.org/doi/10.1109/TIT.2019.2935777
Guangyue Han (2015)A Randomized Algorithm for the Capacity of Finite-State ChannelsIEEE Transactions on Information Theory10.1109/TIT.2015.243209461:7(3651-3669)Online publication date: 1-Jul-2015
https://dl.acm.org/doi/10.1109/TIT.2015.2432094
Guangyue Han Marcus B(2015)Analyticity of Entropy Rate of Hidden Markov Chains With Continuous AlphabetIEEE Transactions on Information Theory10.1109/TIT.2015.242355861:6(3013-3028)Online publication date: 15-May-2015
https://dl.acm.org/doi/10.1109/TIT.2015.2423558
Show More Cited By

Derivatives of Entropy Rate in Special Families of Hidden Markov Chains
1. Mathematics of computing
  1. Probability and statistics
    1. Probabilistic representations
    2. Stochastic processes

Recommendations

Asymptotics of entropy rate in special families of hidden Markov chains

We derive an asymptotic formula for entropy rate of a hidden Markov chain under certain parameterizations. We also discuss applications of the asymptotic formula to the asymptotic behaviors of entropy rate of hidden Markov chains as outputs of certain ...
Analyticity of Entropy Rate of Hidden Markov Chains

We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is ...
Analyticity of Entropy Rate of Hidden Markov Chains With Continuous Alphabet
We first prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is analytic with respect to the input Markov ...

Comments

Information & Contributors

Information

Published In

IEEE Transactions on Information Theory Volume 53, Issue 7

July 2007

357 pages

ISSN:0018-9448

Issue’s Table of Contents

Publisher

IEEE Press

Publication History

Published: 01 July 2007

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Tadić VDoucet A(2019)Analyticity of Entropy Rates of Continuous-State Hidden Markov ModelsIEEE Transactions on Information Theory10.1109/TIT.2019.293577765:12(7950-7975)Online publication date: 19-Nov-2019
https://dl.acm.org/doi/10.1109/TIT.2019.2935777
Guangyue Han (2015)A Randomized Algorithm for the Capacity of Finite-State ChannelsIEEE Transactions on Information Theory10.1109/TIT.2015.243209461:7(3651-3669)Online publication date: 1-Jul-2015
https://dl.acm.org/doi/10.1109/TIT.2015.2432094
Guangyue Han Marcus B(2015)Analyticity of Entropy Rate of Hidden Markov Chains With Continuous AlphabetIEEE Transactions on Information Theory10.1109/TIT.2015.242355861:6(3013-3028)Online publication date: 15-May-2015
https://dl.acm.org/doi/10.1109/TIT.2015.2423558
Tadić V(2010)Analyticity, convergence, and convergence rate of recursive maximum-likelihood estimation in hidden Markov modelsIEEE Transactions on Information Theory10.1109/TIT.2010.208111056:12(6406-6432)Online publication date: 1-Dec-2010
https://dl.acm.org/doi/10.1109/TIT.2010.2081110
Han GMarcus B(2010)Asymptotics of entropy rate in special families of hidden Markov chainsIEEE Transactions on Information Theory10.1109/TIT.2009.203909456:3(1287-1295)Online publication date: 1-Mar-2010
https://dl.acm.org/doi/10.1109/TIT.2009.2039094
Schönhuth A(2009)On analytic properties of entropy rateIEEE Transactions on Information Theory10.1109/TIT.2009.201601555:5(2119-2127)Online publication date: 1-May-2009
https://dl.acm.org/doi/10.1109/TIT.2009.2016015

Abstract

Cited By

Recommendations

Asymptotics of entropy rate in special families of hidden Markov chains

Analyticity of Entropy Rate of Hidden Markov Chains

Analyticity of Entropy Rate of Hidden Markov Chains With Continuous Alphabet

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Share

Share this Publication link

Share on social media

Affiliations