Fundamentals of Image Compression

ƒ Image Compression – reduce the size of image data file

while retaining necessary information

01101…
Compression Decompression
(encoding) (decoding)
Compressed
File
Original Decompressed
uncompressed Image
Image

1 Fundamentals of Image Compression

Fundamentals of Image Compression

ƒ Compression ratio – the ration of the original,

uncompressed image file and the compressed file

Uncompressed File Size SIZEU

Compression Ratio = =
Compressed File Size SIZEC

2 Fundamentals of Image Compression

Image Compression (cont.)

ƒ Bits per pixel (bpp) – denote the number of bits required

to represent an image pixel

Number of Bits 8(Number of Bytes)

bpp = =
Number of Pixels N×N

ƒ Data – the pixel gray-level values that correspond to the

brightness of a pixel at a point in space
ƒ Information – an interpretation of the data in a meaningful
way

3 Fundamentals of Image Compression

Image Compression Techniques

ƒ Information preserving (lossless compression)
• No data are lost – original image can be recreated exactly
from compressed data
• Compression ratio – 2:1 to 3:1
• Useful in image archiving (such as legal or medical images)
ƒ Information lossy (lossy compression)
• Allow a loss in the actual image data – original image
cannot be recovered exactly from compressed data
• Compression ratio – 10:1 to 20:1or more
• Useful for broadcast television, video-conferencing, and
facsimile transmission, etc.
4 Fundamentals of Image Compression
 Image/Video Compression Techniques

Image/Video
Image/Video
Compression
Compression

Predictive
Predictive Transform
Transform Importance
Importance Hybrid
Hybrid
Coding
Coding Coding
Coding Oriented
Oriented Coding
Coding Coding
Coding

Delta
Delta modulation
modulation DCT
DCT Bit
Bit allocation
allocation JPEG
JPEG
DPCM
DPCM KL
KL Sub-sampling
Sub-sampling H.26X
H.26X
ME/MC
ME/MC Subband
Subband Quantization
Quantization MPEG-X
MPEG-X
Wavelet
Wavelet
Scalar
Scalar Vector
Vector
…

5 Fundamentals of Image Compression

How To Do Image Compression

ƒ Basis - redundancy exists in image data and can be removed
ƒ Types of image data redundancy:
• Coding redundancy (statistical redundancy) - variable-
length code (RLC, Huffman Coding, Arithmetic Coding,
LZW)
• Interpixel redundancy (spatial redundancy) – mapper
(DPCM, DCT, Subband, Wavelet)
• Psychovisual redundancy (perceptual redundancy) –
quantization (SQ, VQ, fractal)
• Temporal redundancy (interframe redundancy), for
video) – DPCM, MC/ME
6 Fundamentals of Image Compression
Interpixel Redundancy
ƒ The gray levels of adjacent pixels in normal images are
highly correlated, resulting in interpixel redundancy
ƒ The gray level of a given pixel can be predicted by its
neighbors and the difference is used to represent the image;
this type of transformation is called mapping
ƒ Run-length coding can also be employed to utilize interpixel
redundancy in image compression
ƒ Removing interpixel redundancy is lossless

7 Fundamentals of Image Compression

Interpixel Redundancy (cont.)

ƒ In run-length coding, each
run Ri is represented by a
pair (gi , ri) with
gi = gray level of Ri
ri = length of Ri

8 Fundamentals of Image Compression

Psychovisual Redundancy
ƒ Certain information are more important than other information
to the human visual system
ƒ Humans perceives spatial frequencies below about 50 cycles
per degree so that higher-frequency information is less
important
ƒ Requantization using less bits can eliminate psychovisual
redundancy, but false contour will appear
ƒ For color images, color subsampling can be used to reduce the
size of color components since human eyes are less sensitive
to variations in color than variations in light intensity
ƒ Removing psychovisual redundancy is information lossy

9 Fundamentals of Image Compression

Requantization
ƒ Requantization – the process of limiting the value of a function
at any sample to one of a predetermined number of permissible
values
ƒ A general usage or gray-scale reduction is to reduce the number
of bits per pixel (bpp)
• False contouring – as the number of gray levels decreases,
false edges or lines will appear in the image
• The false contouring effect can be visually improved upon
by using an IGS (Improved Gray-Scale) quantization –
adding a small random number to each pixel before
quantization

10 Fundamentals of Image Compression

Requantization (cont.)

8 bits 7 bits 6 bits 5 bits

4 bits 3 bits 2 bits 1 bits

11 Fundamentals of Image Compression

Requantization (cont.)

ƒ Types of Quantizer
• Scalar Quantizers
» Uniform Scalar Quantizer
» Non-uniform Scalar Quantizer
» The Lloyd-Max Quantizer
» Entropy-constrained Quantizer
• Vector Quantizer

12 Fundamentals of Image Compression

Uniform Scalar Quantizer
ƒ Uniform scalar quantizer – the simplest quantizer
• Masking the lower bits via an AND operation
• Example – reduce 8-bit pixel (256 levels) to be 3-bit pixel
(8 levels) by ANDing with the bit string 11100000

0 32 64 96 128 160 192 224 256

13 Fundamentals of Image Compression

Uniform Scalar Quantizer

Input Output Reconstruction MSE Reconstruction MSE

range code value value
0-31 000 00000000 (0) 00010000 (16)
32-63 001 00100000 (32) 00110000 (48)
64-95 010 01000000 (64) 01010000 (80)
96-127 011 01100000 (96) 325.5 01110000 (112) 85.5
128-159 100 10000000 (128) 10010000 (144)
160-191 101 10100000 (160) 10110000 (176)
192-223 110 11000000 (192) 11010000 (208)
224-255 111 11100000 (224) 11110000 (240)

14 Fundamentals of Image Compression

IGS quantization
ƒ IGS quantization procedure:
1. Set initial SUM = 0000 0000
2. if most significant 4 bits of current pixel A = 1111
new_SUM = A + 0000
else
new_SUM = A + least significant 4 bits of old SUM
Pixel Gray Level Sum IGS Code
i-1 N/A 0000 0000 N/A
i 0110 1100 0110 1100 0110
i+1 1000 1011 1001 0111 1001
i+2 1000 0111 1000 1110 1000
i+3 1111 0100 1111 0100 1111
15 Fundamentals of Image Compression

IGS quantization (cont.)

16 Fundamentals of Image Compression

Non-uniform Scalar Quantizer

ƒ Idea - the quantization levels are finely spaced in the range that
gray levels occur frequently, while coarsely spaced outside of it

17 Fundamentals of Image Compression

The Lloyd-Max Quantizer

™ Optimal quantization
• A quantization function can be regarded as a staircase, in
which si is called decision levels and ti reconstruction levels
• s is mapped to ti if it lies in the half-open interval (si -1, si]

18 Fundamentals of Image Compression

The Lloyd-Max Quantizer (cont.)

19 Fundamentals of Image Compression

The Lloyd-Max Quantizer (cont.)

ƒ How to select si and ti?
ƒ The Llooyd-Max quantizer
• Assuming the criterion is MSE, E{(s - ti)2}, and p(s) is even
The condition for minimal error are
si
∫s i −1
( s − ti ) p ( s ) ds = 0 i = 1,2,..., L/2

⎧0 i=0
⎪
si = ⎨(ti + ti +1 )/2 i = 1, 2, . . ., L/2 − 1
⎪∞ i = L/2
and ⎩
s-i = -si, t-i = -ti
» Each ti is the centroid of the area under p(s) over two si
» si is halfway between two ti
20 Fundamentals of Image Compression
Entropy-constrained Quantizers

ƒ Aim at transmitting the image with as few bits as possible

rather than looking for a quantizer that provides the lowest
distortion for a given number of output levels
¾ Entropy-constrained quantizer (ECQ) – a quantizer that
minimizes distortion for a given entropy measured at the
quantizer output
• The quantizer output is variable-length encoded
• In image compression, the uniform quantizer followed by
variable-length coding will produce the results close to the
optimum ECQ

21 Fundamentals of Image Compression

Color Quantization

16,777,216 colors 256 colors

22 Fundamentals of Image Compression
Color subsampling 128 × 128

Interpolation

256 × 256
23 256 × 256 Fundamentals of Image Compression

Fidelity Criteria

ƒ Fidelity criteria is used to measure information loss

ƒ Two classes:
• objective fidelity criteria - measured mathematically
about the amount of error in the reconstructed data
• subjective fidelity criteria - measured by human
observation

24 Fundamentals of Image Compression

 Fidelity Criteria (cont.)
ƒ Subjective fidelity criteria:
• Impairment test – “how bad the images are”
• Quality test – “how good the images are”
• Comparison test –
Impairment Quality Comparison
5 - Imperceptible A - Excellent +2 much better
4 – Perceptible, not annoying B - Good +1 better
3 – Somewhat annoying C - Fair 0 the same
2 – Severely annoying D - Poor -1 worse
1 - Unusable E - Bad -2 much worse

25 Fundamentals of Image Compression

Fidelity Criteria (cont.)

ƒ Objective criteria
• root-mean-square (rms) error: eRMS

M −1 N −1
1
e RMS =
MN
∑ ∑ [ Iˆ ( r, c ) − I ( r, c )]
r =0 c =0
2

26 Fundamentals of Image Compression

Fidelity Criteria (cont.)

ƒ Objective criteria (cont.)

• root-mean-square signal-to-noise ratio: SNRRMS

M −1 N −1

∑∑ Iˆ(r , c) 2

SNR RMS = r =0 c =0
M −1 N −1

∑∑ [ Iˆ(r , c) − I (r , c)]
r =0 c =0
2

27 Fundamentals of Image Compression

Fidelity Criteria (cont.)

ƒ Objective criteria (cont.)

• peak signal-to-noise ratio: SNRPEAK

( L − 1) 2
SNR PEAK = 10 log10
1 M −1 N −1 ˆ
∑∑
MN r =0 c =0
[ I (r , c) − I (r , c)]2

where L is the maximum sample value

28 Fundamentals of Image Compression

Image Compression Model

I(r, c) Preprocessing Encoder

010010…
Compression
Compressed
(encoding) File

010010…
Î(r, c) Postprocessing Decoder

Decompression
(decoding)
ƒ The encoder removes input redundancies (interpixel,
psychovisual, and coding redundancy)
29 Fundamentals of Image Compression

Encoder and Decoder

I(r, c) Quantizer Symbol

Mapper encoder

encoder Compressed
File
Inverse Symbol
Î(r, c) mapper decoder
decoder
ƒ mapper - reduce interpixel redundancy, reversible (predictive
coding, run-length coding, Transform coding)
ƒ quantizer - reduce psychovisual redundancy, irreversible
(uniform quantization, nonuniform quantization)
ƒ symbol encoder - reduce coding redundancy, reversible
(Huffman coding, arithmetic coding, LZW)
30 Fundamentals of Image Compression
 Predictive Coding

31 Fundamentals of Image Compression

Predictive Coding

+ Bit
+ Quantization Allocation
–

+
Predictor +
+

32 Fundamentals of Image Compression

Predictive Coding (cont.)
Encoder
I
Input n +
en ên Compressed
∑ Symbol
data Quantizer encoder data
-
∑
+
Ĩn Predictor
În
+

Decoder
Compressed Symbol ên + În Decompressed
data ∑ data
decoder
+
Ĩn Predictor
33 Fundamentals of Image Compression

Predictive Coding (cont.)

ƒ In general, the prediction is formed by a linear combination of

m previous samples:
m
Iˆn = round [ ∑ α i I n − i ]
i =1
where m is the order of the linear predictor, round is a
function to denote the rounding or nearest integer operation,
and the αi are prediction coefficients

34 Fundamentals of Image Compression

Predictive Coding (cont.)

ƒ For 2-D images, the prediction is a function of the previous

pixels in a left-to-right, top-to-bottom scan of an image

I(r-1 , c-1) I(r-1 , c) I(r-1 , c+1)

I(r , c-1) I(r , c)

35 Fundamentals of Image Compression

Predictor (cont.)
I(r-1 , c-1) I(r-1 , c) I(r-1 , c+1)
ƒ Examples:
• First-order predictor:
I(r , c-1) I(r , c)
Iˆ ( r , c ) = I ( r, c - 1)
• Second-order predictor:
Iˆ ( r , c ) = 1 / 2[ I ( r, c - 1) + I ( r - 1, c )]
• Third-order predictor:
» Iˆ ( r , c ) = I ( r, c - 1) - I ( r - 1, c - 1) + I ( r - 1, c )

» Iˆ ( r , c ) = 0 .75 I ( r, c - 1) - 0.5 I ( r - 1, c - 1) + 0 .75 I ( r - 1, c )

36 Fundamentals of Image Compression

Predictive Coding (cont.)

ƒ In 3-D case, the prediction is based on the above pixels and the
previous pixels of preceding frames

37 Fundamentals of Image Compression

Differential Pulse Code Modulation (DPCM)

ƒ Differential pulse code modulation (DPCM) – a form of

predictive coding suitable for use with Markov sources
• Use the fact that the intensity value of a pixel is likely to be
similar to that of the surrounding pixels
• Eliminating interpixel redundancies
• Instead of coding an intensity value, we predict the value
from the values of nearby pixels
• The difference between the actual and predicted value of
that pixel is coded
• There is a high probability that the difference will be small

38 Fundamentals of Image Compression

DPCM (cont.)

ƒ The predictive coding system consists of an encoder and a

decoder, each containing an identical predictor
• The predictor transforms a set of high-entropy but
correlated values into a set of low-entropy but less
correlated values
¾ The amount of information is reduced
ƒ Optimal predictor
We shall minimize the variance of e m = x m − xˆ m ,
⎧⎪ ⎛ ⎞ ⎫⎪
2

{ }
m −1
(
i.e., σ e = E x m − x m
2
ˆ )2
= E ⎨⎜ x m − ∑ α i x i ⎟ ⎬
⎜ ⎟
⎪⎩ ⎝ i =0 ⎠ ⎪⎭
39 Fundamentals of Image Compression

DPCM (cont.)

ƒ To find the minimal value, we partial differentiate the

following and let it be zero :

[
∂E ( xm − xˆm )
2
]
∂αi

=
{
∂E [xm − (α0 x0 + α1 x1 + L + αm −1 xm −1 )]
2
}
∂αi
= -2 E [( xm − xˆm )xi ] = 0 i = 0 ,1,L , m-1 (1)

40 Fundamentals of Image Compression

DPCM (cont.)
Define the autocorrelation as Rij = E ( xi x j )

From (1) :
⎧ m −1 ⎫
E{xm xi } = E{xˆm xi } = E ⎨(∑ αi xi ) xi ⎬
⎩ i =0 ⎭
Rmi = E{ α0 x0 xi + α1 x1 xi + L + αm −1 xm −1 xi }
= α0 R0i + α1R1i + L + αm-1R(m −1 )i
i = 0 ,1,L, m-1 (2)

α i , i = 0, 1, L, m − 1 can be obtained by solving

the m equations in (2)
41 Fundamentals of Image Compression

DPCM (cont.)
If xˆm is obtained by the predictor that uses the optimal
αi thus obtained, then
{
σ e2 = E ( xm − xˆ m ) 2 }
= E {( xm − xˆ m ) xm } -E {( xm − xˆ m ) xˆ m }
But E {( xm − xˆ m ) xˆ m } = 0 from Eq. (1) Thus,
σ e2 = E {( xm − xˆ m ) xm }
[ ]
= E xm2 - E [xˆ m xm ]
= Rmm − (α0 R0 m + α1 R1m + L + α m −1 Rm −1m ) (3)

σ e2 : can be considered as the variance of the error signal

R mm : can be considered as the variance of original signal
42 Fundamentals of Image Compression
DPCM (cont.)

Eq. (3) verifies that variance of the error signal is

smaller than that of the original signal !

Thus, it worths DPCM !

43 Fundamentals of Image Compression

DPCM (cont.)

44 Fundamentals of Image Compression

Adaptive Predictor

ƒ Adaptive predictor – several predictor are predefined, select

the best predictor according to the characteristics of the data
• Try out a number of predictors and determine which is the
best, then tell the decoder which predictor has been used
• Consider blocks of m pixels and choose among n predictors,
the overhead is log2n bits per block, or (log2n)/m bpp
ƒ Determine the choice of predictor from the data values
⎧min[I (r,c - 1), I (r - 1,c)] if I (r - 1, c - 1) ≥ max[I (r,c - 1), I (r - 1, c)]
ˆI (r, c) = ⎪⎨max[I (r,c - 1), I (r - 1,c)] if I (r - 1, c - 1) ≤ min[I (r,c - 1), I (r - 1, c)]
⎪I (r - 1,c) + I (r,c - 1) - I (r - 1,c - 1) otherwise
⎩

45 Fundamentals of Image Compression

Results (cont.)

ƒ DPCM at 1.0 bpp

46 Fundamentals of Image Compression

Results (cont.)
ƒ DPCM at 2.0 bpp

47 Fundamentals of Image Compression

Results (cont.)

ƒ DPCM at 3.0 bpp

48 Fundamentals of Image Compression

Results (cont.)

ƒ DPCM at 2.0 bpp

49 Fundamentals of Image Compression

Conclusions
Prediction
ƒ Lossless JPEG
0 No

1 A

2 B
前一行 .......... B C D ..........
3 C
.......... A X ... ..........
4 A+B-C

5 A-(B-C)/2

6 B-(A-C)/2

7 (A+B)/2
50 Fundamentals of Image Compression
Conclusions (cont.)

ƒ Post processor
• JPEG: transformed dc components
• MPEG-4: transformed dc, ac

51 Fundamentals of Image Compression

Entropy Coding

52 Fundamentals of Image Compression

Entropy Coding

ƒ Information Theory
ƒ Huffman Coding
ƒ Arithmetic Coding

53 Fundamentals of Image Compression

Information Theory
ƒ Information theory provides the mathematical framework to
determine the minimum amount of data that is sufficient to
describe completely an image without loss of information
ƒ A event E that occurs with probability P(E) will contain
1
I ( E ) = log = − log P ( E )
P( E )
unit of information, I(E) is called the self-information of E

54 Fundamentals of Image Compression

Information Theory (cont.)
ƒ I(E) is a measure of uncertainty/unpredictability and is
inversely related to P(E)
• If P(E) = 1, I(E) = 0: no information (surprise) is contained
• On the other hand, if the probabilities are all very different,
then when a symbol with a low probability arrives, you feel
more surprised, get more information, than when a symbol
with a higher probability arrives
• If the base r logarithm is used, the measurement is said to
be in r-ary unit; if r = 2, the resulting is called a bit

55 Fundamentals of Image Compression

Information Theory (cont.)

ƒ If we get I(si) units of information when we receive the symbol
si, how much do we get on the average?
ƒ A source alphabet S for a set of n symbols {s1, s2, ..., sn} with
the symbol probabilities {P(s1), P(s2 ), ..., P(sn )}, define
H ( S ) = p( s1 ) I ( s1 ) + p ( s2 ) I ( s2 ) + L + p ( sn ) I ( sn )
n n
= ∑ p( si )I ( si ) = − ∑ p ( si ) log p( si )
i =1 i =1

• H(S) is called the uncertainty or entropy of the source

• The entropy defines the average amount of information
obtained by observing a single source output
56 Fundamentals of Image Compression
Information Theory (cont.)

ƒ If the source symbols are equally probable, the entropy is

maximized and the source provides the greatest possible
average information per source symbol
1.00

0.80

0.60

H(P)

0.40

0.20

0.00

0.00 0.20 0.40 0.60 0.80 1.00

P
57 Fundamentals of Image Compression

Information Theory (cont.)

ƒ Example:

S p(si) I(si) p(si) I(si) p(si) I(si)

s1 0.7 0.515 0.25 2 0.5 1
s2 0.1 3.322 0.25 2 0.25 2
s3 0.1 3.322 0.25 2 0.125 3
s4 0.1 3.322 0.25 2 0.125 3
H(S) 1.375 2.0 1.75

58 Fundamentals of Image Compression

Information Theory (cont.)

ƒ Instantaneous Codes : a code is said to be instantaneous

if when a complete symbol is received, the receiver
immediately knows this, and do not have to look further before
deciding what message symbol is received.
ƒ No encoded symbol of this code is a prefix of any other
symbol

59 Fundamentals of Image Compression

Information Theory (cont.)

ƒ Example :
s1 = 0 s1 = 0
s2 = 10 s2 = 01
s3 = 110 s3 = 011
s4 = 111 s4 = 111
instantaneous non-instantaneous
101100111- s2s3s1s4 011111111- s3s4s4
ƒ No code word si which is a prefix of another code word, sj

60 Fundamentals of Image Compression

Information Theory (cont.)

ƒ Shannon’s noiseless source coding theorem – for a uniquely

decodable code, the average number of bits per symbol used by
the code must be at least equal to the entropy of the source
q
H ( S ) ≤ Lavg = ∑ pi li
i =1

• The entropy H(S) provides the minimum amount of data

that is sufficient to describe data without loss of information

61 Fundamentals of Image Compression

Information Theory (cont.)

ƒ Example:

S p(si) I(si) Code (length) p(si) I(si) Code (length)

s1 0.7 0.515 0 (1) 0. 5 1.0 0 (1)
s2 0.1 3.322 10 (2) 0.25 2.0 10 (2)
s3 0.1 3.322 110 (3) 0.125 3.0 110 (3)
s4 0.1 3.322 111 (3) 0.125 3.0 111 (3)
Average bits 1.5 1.75

62 Fundamentals of Image Compression

Variable-Length Coding
ƒ Idea - assigning the shortest possible code words to the most
probable symbols
ƒ The source symbols may be the gray levels of an image or a
gray-level mapping operation (pixel differences, run-lengths,
etc.)
ƒ Two widely used variable-length coding method
• Huffman coding – generate an optimal code in terms of
the number of code symbols subject to the constraint that
the source symbols are coded one at a time
» Adopted in JPEG, MPEG
• Arithmetic coding – an entire sequence of symbols is
assigned a single arithmetic codeword

63 Fundamentals of Image Compression

Huffman Coding
ƒ Encoding steps:
Initialization: Given the probability distribution of the source
symbols
Step 1: source reduction
» Order the probabilities of the source symbols
» Combine two lowest probability symbols into a single
symbol that replaces them in the next source reduction
Step 2: codeword construction
» Code each reduced source starting with the smallest
source and work back to the original source

64 Fundamentals of Image Compression

Huffman Coding (cont.)
Source reduction process:

n
H ( z ) = −∑ P(ai ) logP (ai )
i =1

= −[0.4 log(0.4) + 0.3 log(0.3) + 2 × 0.1log(0.1) + 0.06 log(0.06) + 0.04 log(0.04)]

= 2.14
65 Fundamentals of Image Compression

Huffman Coding (cont.)

Codeword construction process:

L−1
Lavg = ∑l (ak ) p(ak )
k =0

= 1× (0.4) + 2 × (0.3) + 3× (0.1) + 4 × (0.1) + 5× (0.06) + 5× (0.04)

= 2.2
66 Fundamentals of Image Compression
Huffman Decoding

ƒ Concept – reconstruct the binary coding tree to the decoder

from the symbol to codeword table
ƒ Decoding steps:
Step 1: Read the input compressed stream bit by bit and
traverse the tree until a leaf is reached
Step 2: Discard each input bits
Step 3: When a leaf node is reached, output the symbol at the
leaf node, this will complete the decoding for this
symbol; go to Step 1 for the decoding of next symbol

67 Fundamentals of Image Compression

Modified Huffman Codes

ƒ An Example: assume the source has a range of –510 to +511

S p(si) I(si) Code Bits
0 0.20 2.232 10 2
+1 0.10 3.322 110 3
-1 0.10 3.322 1110 4
+2 0.05 4.322 11110 5
-2 0.05 4.322 11111 5
All other values 0.5 1.000 0 plus value 11
combined (10 bits)

68 Fundamentals of Image Compression

Arithmetic Coding

ƒ Huffman coding
• encodes individual symbols
• relies heavily on accurate knowledge of symbol statistics
• disadvantage of Huffman coding – the length of Huffman
codes have to be an integer, while the entropy value of a
symbol may be a faction number
¾ Theoretical possible compressed message cannot be
achieved
ƒ For example, if p(s) = 0.9, then
I(s) = –log(0.9) = 0.15 bits

69 Fundamentals of Image Compression

Arithmetic Coding (cont.)

ƒ Arithmetic coding
• An entire sequence of symbols is assigned a single
arithmetic codeword
• The codeword defines an interval of real numbers between
0 and 1
• Theoretically achieve the maximum compression

70 Fundamentals of Image Compression

Arithmetic Coding (cont.)

ƒ Example: 1
?
Si Pi subinterval 0.90
k 0.05 [0.00,0.05) r
l 0.20 [0.05,0.25) 0.70
u 0.10 [0.25,0.35) e
w 0.05 [0.35,0.40) 0.40
e 0.30 [0.40,0.70) w
0.35
r 0.20 [0.70,0.90) u
0.25
? 0.10 [0.90,1.00)
l
Alphabet (k, l, u, w, e, r, ?) 0.05
? : end-of-message 0 k

71 Fundamentals of Image Compression

Arithmetic Coding (cont.)

l l u u r e ?
1 .25 .10 .074 .0714 .07136 .071336 0.0713360
? ? ? ? ? ? ? ?

r r r r r r r r

e e e e e e e e

w w w w w w w w
u u u u u u u u

l l l l l l l l

k k k k k k k k
0 .05 .06 .070 .0710 .07128 .071312 0.0713336
72 Fundamentals of Image Compression
Arithmetic Coding (cont.)

ƒ Encoding process
Low = 0.0 ; high =1.0 ;
while not EOF do
range = high - low ; read(c) ;
high = low + range×high_range(c) ;
low = low + range×low_range(c) ;
endofwhile
output(low);

73 Fundamentals of Image Compression

Arithmetic Coding (cont.)

New character Low value high value
l 0.05 0.25
l 0.06 0.10
u 0.070 0.074
u 0.0710 0.0714
r 0.07128 0.07136
e 0.071312 0.071336
? 0.0713336 0.0713360

74 Fundamentals of Image Compression

Arithmetic Coding (cont.)

ƒ Decoding is the inverse process

• Since 0.0713336 falls between 0.05 and 0.25, the first character must be
‘L’.
• Removing the effect of ‘L’ from 0.0713336 by first subtracting the low
value of ‘L’, 0.05:
0.0713336 – 0.05 = 0.0213336
• Then divided by the range of ‘L’, 0.2:
0.0213336/0.2 = 0.106668
• Then calculate where that lands, which is in the range of the character
‘L’
• The process repeats until 0 or the known length of the message is
reached

75 Fundamentals of Image Compression

Arithmetic Coding (cont.)

ƒ Decoding algorithm
r = input_code
repeat
search c such that r falls in its range ;
output(c) ;
r = r - low_range(c) ;
r = r/(high_range(c) - low_range(c));
until EOF or the length of the message is reached

76 Fundamentals of Image Compression

Arithmetic Coding (cont.)

r c Low High range

0.0713336 L 0.05 0.25 0.2
0.106668 L 0.05 0.25 0.2
0.28334 U 0.25 0.35 0.1
0.3334 U 0.25 0.35 0.1
0.834 R 0.7 0.9 0.2
0.67 E 0.4 0.7 0.3
0.9 ? 0.9 1.0 0.1

77 Fundamentals of Image Compression

Arithmetic Coding (cont.)

ƒ In summary, the encoding process is simply one of narrowing

the range of possible numbers with every new symbol
ƒ The new range is proportional to the predefined probability
attached to that symbol
ƒ Decoding is the inverse procedure, in which the range is
expanded in proportion to the probability of each symbol as it
is extracted

78 Fundamentals of Image Compression

Arithmetic Coding (cont.)

ƒ Coding rate approaches high-order entropy theoretically

ƒ Not so popular as Huffman coding because ×, ÷ are needed.
ƒ Average bits/byte on 14 files (program, object, text, and etc.)
Huff. Arithmetic
4.99 2.48

79 Fundamentals of Image Compression

Conclusions

ƒ JPEG, MPEG-1/2 uses Huffman and arithmetic coding –

preprocessed by DPCM
ƒ JPEG2000, MPEG-4 uses arithmetic coding only

80 Fundamentals of Image Compression