Professional Documents
Culture Documents
Greedy Algorithms: Part - 4: Huffman Coding
Greedy Algorithms: Part - 4: Huffman Coding
Greedy Algorithms: Part - 4: Huffman Coding
2
Two Ways to Compress data
• Fixed-Length Codewords
• Variable-Length Codewords
3
Example
E.g.:
a b c d e f
Frequency (thousands) 45 13 12 16 9 5
4
Fixed-Length Codes
E.g.: Data file containing 100,000 characters
a b c d e f
Frequency (thousands) 45 13 12 16 9 5
• 3 bits needed
5
Huffman Codes
• Idea:
– Variable Length Codes
6
Constructing a Huffman Code
• Idea:
– Arranging the characters in Ascending order of their
frequencies
f: 5 e: 9 c: 12 b: 13 d: 16 a: 45
7
Example
f: 5 e: 9 c: 12 b: 13 d: 16 a: 45 c: 12 b: 13 14 d: 16 a: 45
0 1
f: 5 e: 9
14 d: 16 25 a: 45 25 30 a: 45
0 1 0 1 0 1 0 1
f: 5 e: 9 c: 12 b: 13 c: 12 b: 13 14 d: 16
0 1
f: 5 e: 9
a: 45 55 0 100 1
0 1
a: 45 55
25 30 0 1
0 1 0 1
25 30
c: 12 b: 13 14 d: 16 0 1 0 1
0 1 c: 12 b: 13 d: 16
14
f: 5 e: 9 0 1
f: 5 e: 9 8
Variable-Length Codes
E.g.: Data file containing 100,000 characters
a b c d e f
Frequency (thousands) 45 13 12 16 9 5
Variable length
0 101 100 111 1101 1100
Codewords
a b c d e f
Frequency (thousands) 45 13 12 16 9 5
Fixed Length
000 001 010 011 100 101
Codewords
Variable length
0 101 100 111 1101 1100
Codewords
300000 – 224000
X 100
300000
= 25.33%
Encode : abcde
Decode: 101100011111011100
12
Dynamic Programming
Introduction
Dynamic Programming
• An algorithm design technique for optimization problems
(similar to divide and conquer)
14
Dynamic Programming
• For example,
{10, 22, 9, 33, 21, 50, 41, 60, 80}
9 2 5 3 7 11 8 10 13 6
17
Increasing subsequences
• e.g. 9 2 5 3 7 11 8 10 13 6
2 3 7
5 7 10 13 are increasing subsequences.
2 5 7 8 10 13
2 3 7 8 10 13
We want to find a longest one.
So LIS = 2 3 7 8 10 13 OR 2 5 7 8 10 13
And Length of LIS = 6
19
A naive approach for LIS
• Let LIS[i] be the length of a longest increasing
subsequence ending at position i.
LIS[i] = 1 + max j = 0..i-1{L[j] | aj < ai}
(use a dummy a0 = minimum, and L[0]=0)
i = 1; j = 0 to i-1
Index 0 1 2 3 4 5 6 7 8 9
Input 9 2 5 3 7 11 8 10 13 6
Length
Prev
20
A naive approach for LIS
Index 0 1 2 3 4 5 6 7 8 9
Input 9 2 5 3 7 11 8 10 13 6
Length 1 1 2 2 3 4 4 5 6 3
Prev -1 -1 1 1 2 4 4 6 7 2
21
Psudocode or LIS
LIS( int arr[], int n )
• for (i = 0; i < n; i++ )
• lis[i] = 1;
•
• /* Compute optimized LIS values in bottom up manner */
• for (i = 1; i < n; i++ )
• for (j = 0; j < i; j++ )
• if ( arr[i] > arr[j] && lis[i] < lis[j] + 1)
• lis[i] = lis[j] + 1;
•
• /* Pick maximum of all LIS values */
• for (i = 0; i < n; i++ )
• if (max < lis[i])
• max = lis[i];
•
• return max;
22