Optimal Quantization for Distribution Synthesis
Abstract
Finite precision approximations of discrete probability distributions are considered, applicable for distribution synthesis, e.g., probabilistic shaping. Two algorithms are presented that find the optimal $M$-type approximation $Q$ of a distribution $P$ in terms of the variational distance $| Q-P|_1$ and the informational divergence $\mathbb{D}(Q| P)$. Bounds on the approximation errors are derived and shown to be asymptotically tight. Several examples illustrate that the variational distance optimal approximation can be quite different from the informational divergence optimal approximation.
- Publication:
-
arXiv e-prints
- Pub Date:
- July 2013
- DOI:
- 10.48550/arXiv.1307.6843
- arXiv:
- arXiv:1307.6843
- Bibcode:
- 2013arXiv1307.6843B
- Keywords:
-
- Computer Science - Information Theory
- E-Print:
- Submitted to the IEEE Transactions on Information Theory