Huffman Coding

Topic 20: Huffman Coding
The author should gaze at Noah, and ...

learn, as they did in the Ark, to crowd a
great deal of matter into a very small
compass.
Sydney Smith, Edinburgh Review
Agenda
Encoding
Compression
Huffman Coding
Encoding
UT CS
85 84 32 67 83
01010101 01010100 00100000 01000011
01010011
what is a file? how do some OS use file extensions?
open a bitmap in a text editor
open a pdf in word
ASCII - UNICODE
Text File
Text File???
Bitmap File
Bitmap File????
JPEG File
JPEG VS BITMAP
JPEG File
10
Encoding Schemes
"It's all 1s and 0s"

What do the 1s and 0s mean?
50 12 109
ASCII -> 2ym
Red Blue Green ->
dark teal?
11
Agenda
Encoding
Compression
Huffman Coding
12
Compression
Compression: Storing the same information
but in a form that takes less memory
lossless and lossy compression
Recall:
13
Lossy Artifacts
14
Why Bother?
Is compression really necessary?
2 Terabytes
500 HD, 2 hour movies or 500,000 songs
15
Little Pipes and Big Pumps

Home Internet Access
CPU Capability
40 Mbps roughly $40 per

month.
12 months * 3 years * $40 =
$1,440
10,000,000 bits /second
= 1.25 * 106 bytes / sec
$1,500 for a laptop or

desktop
Intel i7 processor
Assume it lasts 3 years.
Memory bandwidth
25.6 GB / sec
= 2.6 * 1010 bytes / sec
on the order of
5.0 * 1010 instructions /
second
16
Mobile Devices?
Cellular Network
iPhone CPU
Your mileage may vary

Mega bits per second
AT&T
Apple A6 System on a Chip

Coy about IPS
2 cores
Rough estimates:
1 x 1010 instructions per
second
17 download, 7 upload
T-Mobile & Verizon

12 download, 7 upload
17,000,000 bits per second

= 2.125 x 106 bytes per
second
http://tinyurl.com/q6o7wan
17
Little Pipes and Big Pumps

CPU
Data In
From Network
18
Compression - Why Bother?
Apostolos "Toli" Lerios

Facebook Engineer
Heads image storage group
jpeg images already
compressed
look for ways to compress even
more
1% less space = millions of
dollars in savings
19
Agenda
Encoding
Compression
Huffman Coding
20
Purpose of Huffman Coding

Proposed by Dr. David A. Huffman
A Method for the Construction of Minimum
Redundancy Codes
Written in 1952
Applicable to many forms of data transmission

Our example: text files
still used in fax machines, mp3 encoding, others
21
The Basic Algorithm

Huffman coding is a form of statistical coding
Not all characters occur with the same
frequency!
Yet in ASCII all characters are allocated the
same amount of space
1 char = 1 byte, be it e or x
22
The Basic Algorithm

Any savings in tailoring codes to
frequency of character?
Code word lengths are no longer fixed like
ASCII or Unicode
Code word lengths vary and will be
shorter for the more frequently used
characters
23
The Basic Algorithm

1. Scan text to be compressed and tally
occurrence of all characters.
2. Sort or prioritize characters based on
number of occurrences in text.
3. Build Huffman code tree based on
prioritized list.
4. Perform a traversal of tree to determine
all code words.
5. Scan text again and create new file
using the Huffman codes
24
Building a Tree
Scan the original text
Consider the following short text
Eerie eyes seen near lake.
Count up the occurrences of all characters in
the text
25
Building a Tree

What characters are present?
E e r i space
y s n a r l k .
26
Building a Tree

What is the frequency of each character in the
text?
Char Freq. Char Freq.

E
1
y
1
k 1
e
8
s
2
. 1
r
2
n
2
i
1
a
2
space 4
l
1
Char Freq.
27
Building a Tree
Prioritize characters
Create binary tree nodes with character
and frequency of each character
Place nodes in a priority queue
The lower the occurrence, the higher the
priority in the queue
28
Building a Tree
The queue after inserting all nodes
sp
Null Pointers are not shown

29
Building a Tree
While priority queue contains two or more
nodes
Create new node
Dequeue node and make it left subtree
Dequeue next node and make it right subtree
Frequency of new node equals sum of frequency of
left and right children
Enqueue new node back into queue
30
Building a Tree
sp
31
Building a Tree
sp
2
E
1
32
Building a Tree
2
E
sp
33
Building a Tree
2
E
sp
2
k
1
34
Building a Tree
sp
35
Building a Tree
sp
2
y
1
36
Building a Tree
sp
37
Building a Tree
sp
4
a
2
38
Building a Tree
sp
39
Building a Tree
E
1
2
i
1
sp
8
a
4
r
2
40
Building a Tree
E
1
2
i
1
sp
8
a
41
Building a Tree
sp
8
a
4
2
1
42
Building a Tree
sp
43
Building a Tree
6
sp
2
y
44
Building a Tree
e
sp
What is happening to the characters

with a low number of occurrences?
45
Building a Tree
e
sp
4
8
4
4
a
2
46
Building a Tree
sp
e
8
4
a
47
Building a Tree
e
8
10
a
sp
1
48
Building a Tree
e
8
10
4
4
a
sp
49
Building a Tree
10
16
sp
8
4
4
a
2
50
Building a Tree
10
16
e
sp
8
4
4
a
51
Building a Tree
26
16
10
4
sp
4
a
52
Building a Tree
After
enqueueing
this node
there is
only one
node left in
priority
queue.
26
16
10
4
sp
4
a
2
53
Building a Tree
Dequeue the single node
left in the queue.
This tree contains the
new code words for each
character.
Frequency of root node
should equal number of
characters in text.
26
16
10
4
2
6
2
E i k l y .
sp
4
1 1 1 1 1 1
Eerie eyes seen near lake. 4 spaces,

26 characters total
54
8
4
a n
r s
2 2
2 2
Encoding the File

Traverse Tree for Codes
Perform a traversal of the tree
to obtain new code words
left, append a 0 to code word
right append a 1 to code word
code word is only completed
when a leaf node is reached
26
16
10
4
2
6
2
E i
k l y .
1 1
1 1 1 1
sp
55
8
4
a n
r s
2 2
2 2
Encoding the File

Traverse Tree for Codes
Char
E
i
k
l
y
.
space
e
a
n
r
s
Code
0000
0001
0010
0011
0100
0101
011
10
1100
1101
1110
1111
26
16
10
4
2
6
2
E i
k l y .
1 1
1 1 1 1
sp
56
8
4
a n
r s
2 2
2 2
Encoding the File

Rescan text and encode file
using new code words
000010111000011001110
010010111101111111010
110101111011011001110
011001111000010100101
Char
E
i
k
l
y
.
space
e
a
n
r
s
Code
0000
0001
0010
0011
0100
0101
011
10
1100
1101
1110
1111
57
Encoding the File

Results
Have we made things any
better?
82 bits to encode the text
ASCII would take 8 * 26 =
208 bits
000010111000011001110
010010111101111111010
110101111011011001110
011001111000010100101
hIf modified code used 4 bits per

character are needed. Total bits
4 * 26 = 104. Savings not as great.
58
Decoding the File

How does receiver know what the codes are?
Tree constructed for each text file.
Considers frequency for each file
Big hit on compression, especially for smaller files
Tree predetermined
based on statistical analysis of text files or file types
59
Decoding the File

Once receiver has tree it
scans incoming bit stream
0 go left
1 go right
1010001001111000111111
11011100001010
26
10
4
2
A.
B.
C.
D.
E.
elk nay sir

eek a snake
eek kin sly
eek snarl nil
eel a snarl
16
e
6
2
E i k l y .
sp
4
1 1 1 1 1 1
60
8
4
a n
r s
2 2
2 2
Assignment Hints
reading chunks not chars

header format
the pseudo eof character
the GUI
61
Assignment Example
"Eerie eyes seen near lake." will result in different
codes than those shown in slides due to:
adding elements in order to PriorityQueue
required pseudo eof character (PEOF)
62
Assignment Example
Char Freq. Char Freq. Char Freq.
E
1
y
1
k 1
e
8
s
2
. 1
r
2
n
2
PEOF 1
i
1
a
2
space 4
l
1
63
Assignment Example
.
1
E
1
i
1
k
1
l
1
y
1
PEOF
1
a
2
n
2
r
2
s
2
64
SP
4
e
8
Assignment Example
i
1
k
1
l
1
y
1
PEOF
1
a
2
n
2
r
2
s
2
2
.
1
E
1
65
SP
4
e
8
Assignment Example
i
1
k
1
l
1
y
1
PEOF a
1
2
n
2
r
2
s
2
SP
4
2
.
1
E
1
66
e
8
Assignment Example
l
1
y
1
PEOF
1
a
2
n
2
r
2
s
2
2
.
1
E
1
i
1
k
1
67
SP
4
e
8
Assignment Example
PEOF
1
a
2
n
2
r
2
s
2
2
.
1
E
1
i
1
SP
4
2
k
1
l
1
y
1
68
e
8
Assignment Example
n
2
r
2
s
2
2
.
1
E
1
i
1
2
k
1
l
1
SP
4
3
y
1
PEOF
1
a
2
69
e
8
Assignment Example
s
2
2
.
1
E
1
i
1
2
k
1
l
1
SP
4
3
y
1
PEOF
1
e
8
a
2
n
2
70
r
2
Assignment Example
2
i
1
2
k
1
l
1
SP
4
3
y
1
PEOF
1
a
2
n
2
e
8
4
s
2
r
2
2
.
1
71
E
1
4
n
2
r
2
s
2
2
.
1
2
E
1
i
1
e
8
SP
4
3
k
1
l
1
y
1
PEOF
1
a
2
72
4
2
2
i
1
k
1
l
1
PEOF
1
SP
4
3
y
1
e
8
a
2
n
2
4
r
2
s
2
2
.
1
73
E
1
11
e
8
4
n
2
r
2
4
s
2
.
1
SP
4
3
E
1
i
1
k
1
l
1
y
1
PEOF
1
74
a
2
11
16
e
8
4
2
SP
4
3
i
1
k
1
l
1
y
1
PEOF
1
a
2
8
4
4
n
2
s
2
r
2
2
.
1
75
E
1
27
11
16
e
8
4
2
SP
4
3
i
1
k
1
l
1
y
1
PEOF
1
a
2
8
4
4
n
2
s
2
r
2
2
.
1
76
E
1
Codes
value:
value:
value:
value:
value:
value:
value:
value:
value:
value:
value:
value:
value:
32, equivalent char: , frequency: 4, new code 011

46, equivalent char: ., frequency: 1, new code 11110
69, equivalent char: E, frequency: 1, new code 11111
97, equivalent char: a, frequency: 2, new code 0101
101, equivalent char: e, frequency: 8, new code 10
105, equivalent char: i, frequency: 1, new code 0000
107, equivalent char: k, frequency: 1, new code 0001
108, equivalent char: l, frequency: 1, new code 0010
110, equivalent char: n, frequency: 2, new code 1100
114, equivalent char: r, frequency: 2, new code 1101
115, equivalent char: s, frequency: 2, new code 1110
121, equivalent char: y, frequency: 1, new code 0011
256, equivalent char: ?, frequency: 1, new code 0100
77

Huffman Coding

Uploaded by

Copyright:

Available Formats

Huffman Coding

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Huffman Coding

Uploaded by

Copyright:

Available Formats

Topic 20: Huffman Coding

The author should gaze at Noah, and ...

"It's all 1s and 0s"

Little Pipes and Big Pumps

40 Mbps roughly $40 per

$1,500 for a laptop or

Your mileage may vary

Apple A6 System on a Chip

T-Mobile & Verizon

17,000,000 bits per second

Little Pipes and Big Pumps

Compression - Why Bother?

Apostolos "Toli" Lerios

Purpose of Huffman Coding

Applicable to many forms of data transmission

The Basic Algorithm

The Basic Algorithm

The Basic Algorithm

Eerie eyes seen near lake.

Eerie eyes seen near lake.

Char Freq. Char Freq.

Null Pointers are not shown

What is happening to the characters

Eerie eyes seen near lake. 4 spaces,

Encoding the File

Encoding the File

Encoding the File

Encoding the File

hIf modified code used 4 bits per

Decoding the File

Decoding the File

elk nay sir

reading chunks not chars

32, equivalent char: , frequency: 4, new code 011

You might also like