Abstract
We summarize some results of geometric measure theory concerning rectifiable sets and measures. Combined with the entropic chain rule for disintegrations (Vigneaux, 2021), they account for some properties of the entropy of rectifiable measures with respect to the Hausdorff measure first studied by (Koliander et al., 2016). Then we present some recent work on stratified measures, which are convex combinations of rectifiable measures. These generalize discrete-continuous mixtures and may have a singular continuous part. Their entropy obeys a chain rule, whose “conditional term” is an average of the entropies of the rectifiable measures involved. We state an asymptotic equipartition property (AEP) for stratified measures that shows concentration on strata of a few “typical dimensions” and that links the conditional term of the chain rule to the volume growth of typical sequences in each stratum.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A function \(f:X\rightarrow Y\) between metric spaces \((X, d_X)\) and \((Y,d_Y)\) is called Lipschitz if there exists \(C>0\) such that
$$\forall x,x'\in X,\quad d_Y(f(x),f(x')) \le C d_X(x,x').$$The Lipschitz constant of f, denoted \({\text {Lip}}(f)\), is the smallest C that satisfies this condition.
- 2.
Given a measure \(\mu \) on a \(\sigma \)-algebra \(\mathfrak {B}\) and \(B\in \mathfrak {B}\), \(\mu |_B\) denotes the restricted measure \(A\mapsto \mu |_B(A) := \mu (A\cap B)\).
References
Alberti, G., Bölcskei, H., De Lellis, C., Koliander, G., Riegler, E.: Lossless analog compression. IEEE Trans. Inf. Theory 65(11), 7480–7513 (2019)
Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Oxford Science Publications, Clarendon Press (2000)
Cover, T.M., Thomas, J.A.: Elements of Information Theory. A Wiley-Interscience Publication. Wiley (2006)
Csiszár, I., Körner, J.: Information Theory: Coding Theorems for Discrete Memoryless Systems. Probability and Mathematical Statistics. Academic Press (1981)
De Lellis, C.: Rectifiable Sets, Densities and Tangent Measures. Zurich Lectures in Advanced Mathematics. European Mathematical Society (2008)
Federer, H.: Geometric Measure Theory. Classics in Mathematics. Springer, Heidelberg (1969). https://doi.org/10.1007/978-3-642-62010-2
Koliander, G., Pichler, G., Riegler, E., Hlawatsch, F.: Entropy and source coding for integer-dimensional singular random variables. IEEE Trans. Inf. Theory 62(11), 6124–6154 (2016)
Mattila, P.: Geometry of Sets and Measures in Euclidean Spaces: Fractals and Rectifiability. Cambridge Studies in Advanced Mathematics. Cambridge University Press (1995)
Morgan, F.: Geometric Measure Theory: A Beginner’s Guide. Elsevier Science (2008)
Rényi, A.: On the dimension and entropy of probability distributions. Acta Mathematica Academiae Scientiarum Hungarica 10(1), 193–215 (1959)
Vigneaux, J.P.: Topology of statistical systems: a cohomological approach to information theory. Ph.D. thesis, Université de Paris (2019)
Vigneaux, J.P.: Entropy under disintegrations. In: Nielsen, F., Barbaresco, F. (eds.) GSI 2021. LNCS, vol. 12829, pp. 340–349. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-80209-7_38
Vigneaux, J.P.: Typicality for stratified measures. IEEE Trans. Inf. Theor. arXiv preprint arXiv:2212.10809 (2022, to appear)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vigneaux, J.P. (2023). On the Entropy of Rectifiable and Stratified Measures. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2023. Lecture Notes in Computer Science, vol 14071. Springer, Cham. https://doi.org/10.1007/978-3-031-38271-0_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-38271-0_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-38270-3
Online ISBN: 978-3-031-38271-0
eBook Packages: Computer ScienceComputer Science (R0)