Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
60 views

Entropy Coding - Wikipedia

ergerg

Uploaded by

rommel baldago
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
60 views

Entropy Coding - Wikipedia

ergerg

Uploaded by

rommel baldago
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

9/24/21, 4:18 PM Entropy coding - Wikipedia

Entropy coding
In information theory an entropy coding (or entropy encoding) is a lossless data compression
scheme that is independent of the specific characteristics of the medium.

One of the main types of entropy coding creates and assigns a unique prefix-free code to each unique
symbol that occurs in the input. [1] These entropy encoders then compress data by replacing each
fixed-length input symbol with the corresponding variable-length prefix-free output codeword. The
length of each codeword is approximately proportional to the negative logarithm of the probability of
occurrence of that codeword. Therefore, the most common symbols use the shortest codes.[2]

According to Shannon's source coding theorem, the optimal code length for a symbol is ,
where is the number of symbols used to make output codes and is the probability of the input
symbol.

Two of the most common entropy coding techniques are Huffman coding and arithmetic coding.[3]
If
the approximate entropy characteristics of a data stream are known in advance (especially for signal
compression), a simpler static code may be useful.
These static codes include universal codes (such as
Elias gamma coding or Fibonacci coding) and Golomb codes (such as unary coding or Rice coding).

Since 2014, data compressors have started using the Asymmetric Numeral Systems family of entropy
coding techniques, which allows combination of the compression ratio of arithmetic coding with a
processing cost similar to Huffman coding.

Contents
Entropy as a measure of similarity
See also
References
External links

Entropy as a measure of similarity


Besides using entropy coding as a way to compress digital data, an entropy encoder can also be used
to measure the amount of similarity between streams of data and already existing classes of data. This
is done by generating an entropy coder/compressor for each class of data; unknown data is then
classified by feeding the uncompressed data to each compressor and seeing which compressor yields
the highest compression. The coder with the best compression is probably the coder trained on the
data that was most similar to the unknown data.

See also
Arithmetic coding

https://en.wikipedia.org/wiki/Entropy_coding 1/2
9/24/21, 4:18 PM Entropy coding - Wikipedia

Asymmetric numeral systems (ANS)


Context-adaptive binary arithmetic coding (CABAC)
Huffman coding
Range coding

References
1. "Education - Entropy Encoding" (http://www.pcs-ip.eu/index.php/main/edu/8). www.pcs-ip.eu.
Retrieved 2020-10-13.
2. "What is Entropy Coding | IGI Global" (https://www.igi-global.com/dictionary/entropy-coding/1004
0). www.igi-global.com. Retrieved 2020-10-13.
3. Huffman, David (1952). "A Method for the Construction of Minimum-Redundancy Codes".
Proceedings of the IRE. Institute of Electrical and Electronics Engineers (IEEE). 40 (9): 1098–
1101. doi:10.1109/jrproc.1952.273898 (https://doi.org/10.1109%2Fjrproc.1952.273898).
ISSN 0096-8390 (https://www.worldcat.org/issn/0096-8390).

External links
Information Theory, Inference, and Learning Algorithms (http://www.inference.phy.cam.ac.uk/mack
ay/itila/book.html), by David MacKay (2003), gives an introduction to Shannon theory and data
compression, including the Huffman coding and arithmetic coding.
Source Coding (http://iphome.hhi.de/wiegand/assets/pdfs/VBpart1.pdf), by T. Wiegand and H.
Schwarz (2011).

Retrieved from "https://en.wikipedia.org/w/index.php?title=Entropy_coding&oldid=1045258762"

This page was last edited on 19 September 2021, at 17:25 (UTC).

Text is available under the Creative Commons Attribution-ShareAlike License;


additional terms may apply. By using this
site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia
Foundation, Inc., a non-profit organization.

https://en.wikipedia.org/wiki/Entropy_coding 2/2

You might also like