Analysis of Algorithm

SE-305
Lecture 14 and 15

Dr. Huma Qayyum (Assistant Professor)

Department of Software Engineering
huma.ayub@uettaxila.edu.pk
 Greedy Algorithms

MINIMUM SPANNING TREE

Definition
• A tree is a connected graph without cycles.
• A graph is a tree if and only if there is one and only one path joining
any two of its vertices.
• A connected graph is a tree if and only if it has N vertices and N-1
edges.
• A subgraph that is a tree and that spans (reaches out to) all vertices of
the original graph is called a spanning tree.
• Among all the spanning trees of a weighted and connected graph, the
one (possibly more) with the least total weight is called a minimum
spanning tree (MST).
Example of a Problem that Translate into a MST

The Problem
• Several pins of an electronic circuit must be connected
using the least amount of wire.
• Network Design

Modeling the Problem

• The graph is a complete, undirected graph
G = ( V, E ,W ), where V is the set of pins, E is the set of all
possible interconnections between the pairs of pins and
w(e) is the length of the wire needed to connect the pair
of vertices.
• Find a minimum spanning tree.
Greedy/Prims 4
Greedy Choice
We will show two ways to build a minimum
spanning tree.
• KRUSKAL’S ALGORITHM

• PRIM’S ALGORITHM

Greedy/Prims 5
Kruskal's algorithm
• In graph theory this algo finds a minimum spanning tree for a
connected weighted graph.
• This means it finds a subset of the edges that forms a tree that
includes every vertex, where the total weight of all the edges in the
tree is minimized.
 Kruskal’s algorithm
• In Kruskal’s algorithm, the set A is a forest. The safe edge added
to A is always a least-weight edge in the graph that connects two
distinct components.

.
• It finds a safe edge to add to the growing forest by finding, of all
the edges that connect any two trees in the forest, an edge (u, v)
of least weight.
• Kruskal’s algorithm is a greedy algorithm, because at each step it
adds to the forest an edge of least possible weight.
• We can determine whether two vertices u and v belong to the
same tree by testing whether FIND-SET (u) equals FIND-SET
(v).
• The combining of trees is accomplished by the UNION procedure
Explanation

• Kruskal’s logic is a greedy logic.

• We first arrange the edges in ascending order of weights.
• We also make ‘n’ sets of one node each.
• We then start the iterations. We pick up the edge with the edge with
the lowest weight and we check whether the nodes of the edge belong
to the same set. If the do we make the union of the two sets. Else we
move on to the next edge in the queue.
• In this way, when all the nodes belong to the same set, our minimum
spanning tree is ready.
Prim’s Algorithms
In this case we start with a source node, say ‘s’. We maintain an array of the
distance of each node from the MST being built.
We also maintain an array of the predecessor of each node. When a node has
not yet been selected into the MST it’s predecessor is NULL.
Initially we set all distances as infinity except that of the source which is set to
zero.
We now start the iterations.
We select the node that has the least distance, say ‘u’. In the first loop, the
source is selected because its distance is zero. In subsequent iterations this
process ensures that the minimum weight edge in the cut of the current tree is
selected. We now include the node in the tree.
Then we relax all the nodes connected to the node. This means that we see for
each node v connected to u whether the distance of v is less than the weight of
edge (u,v). If it is, we assign w(u,v) to the distance of v.
This step ensures that after each new node is added to the tree, the distance of
the remaining nodes from the tree is updated.
Thus when we have all the edges in the tree, our minimum spanning tree is
ready.
PRIM’S Algorithm
PRIM(G, w, r)
for each u belongs to V [G]
do key[u] ← ∞
π[u] ← NIL
key[r] ← 0
Q ← V [G]
while Q ≠ Ø
do u ← EXTRACT-MIN(Q)
for each v belongs to Adj[u]
do if v belongs to Q and w(u, v) < key[v]
then π[v] ← u
key[v] ← w(u, v)
while Q ≠ φ v Adj {u} For each V ε Adj u
if ∏{V} Key {V}
1 T a b,h T a key[b] = 4, key[h] = 8
2 T b c,h T b key[c] = 8
3 T c d,f,i T c key[d] = 7, key[f] = 4, key[i] = 2
4 T i g,h T i key[g] = 6, key [h] = 7
5 T f d,e,g T f key[e] = 10, key [g] = 2
6 T g f,h,i T g key [h] = 1
7 T h a,b,g,i F - -
8 T d c,e,f T d key [e] = 9
9 T e d,f F - -

while loop terminated

A = |V-Q| = vertices in the MST = {a,b,c,i,f,g,h,d,e}
Huffman Coding
 The Basic Algorithm
1. Scan text to be compressed and tally
occurrence of all characters.
2. Sort or prioritize characters based on
number of occurrences in text.
3. Build Huffman code tree based on
prioritized list.
4. Perform a traversal of tree to determine
all code words.
5. Scan text again and create new file
using the Huffman codes
6
 Building a Tree
Scan the original text

• Consider the following short text

Eerie eyes seen near lake.

• Count up the occurrences of all characters in the

text

7
 Building a Tree
Scan the original text

Eerie eyes seen near lake.

• What characters are present?

E e r i space
y s n a r l k .

8
 Building a Tree
Scan the original text
Eerie eyes seen near lake.
• What is the frequency of each character in the
text?
Char Freq. Char Freq. Char Freq.
E 1 y 1 k 1
e 8 s 2 1
r 2 n 2
i 1 a 2
space 4 l 1

9
 Building a Tree
Prioritize characters

• Create binary tree nodes with character and

frequency of each character
• Place nodes in a priority queue
- The lower the occurrence, the higher the priority
in the queue

10
 Building a Tree
• The queue after inserting all nodes

E i y l k r s n a sp e
1 1 1 1 1 1 2 2 2 2 4 8

11
 Building a Tree

y l k r s n a sp e
1 1 1 1 2 2 2 2 4 8

E i
1 1

12
 Building a Tree

y l k r s n a sp e
2
1 1 1 1 2 2 2 2 4 8
E i
1 1

13
 Building a Tree

k r s n a sp e
2
1 1 2 2 2 2 4 8
E i
1 1

y l
1 1

14
 Building a Tree

2
k r s n a 2 sp e
1 1 2 2 2 2 y l 4 8
E i 1 1
1 1

15
 Building a Tree

r s n a 2 2 sp e
2 2 2 2 4 8
y l
E i 1 1
1 1

k
1 1

16
 Building a Tree

r s n a 2 s
2 p
2
2 2 2 e 4 8
k 2
E i y l
1 1 1 1 1 1

17
 Building a Tree

n a 2 s
2 p
2 2 4 8
e
E i y 2
l k
1 1 1 1 1 1

r s
2 2

18
 Building a Tree

n a 2 e
2 2 sp
2 2 4 8
4
E i y l k r s
1 1 1 1 1 1 2 2

19
 Building a Tree

2 2 2 4 e
sp
8
4 r s
E i y l k
1 1 1 1 2 2
1 1

n a
2 2

20
 Building a Tree

2 2 2 4 4 e
sp
8
4 r s n a
y l k
E i 1 1 1 1 2 2 2 2
1 1

21
 Building a Tree

4 4 e
2 sp
8
4 a
k r s n
1 1 2 2 2 2

2 2

E i y l
1 1 1 1

22
 Building a Tree

4 4 4
2 sp e

4 2 2 8
k r s n a
1 1 2 2 2 2
E i y l
1 1 1 1

23
 Building a Tree

4 4 4
e
2 2 8
r s n a
2 2 2 2
E i y l
1 1 1 1

2 sp
4
k
1 1

24
 Building a Tree

4 4 4 6 e
2 sp 8
r s n a 2 2
4
2 2 2 2
k
E i y l 1 1
1 1 1 1

25
 Building a Tree

4 6 e
2 2 2 sp 8
4
E i y l k
1 1 1 1 1 1
8

4 4

r s n a
2 2 2 2

26
 Building a Tree

4 6 e 8
2 2 2 sp 8
4 4 4
E i y l k
1 1 1 1 1 1
r s n a
2 2 2 2

27
 Building a Tree

8
e
8
4 4
10
r s n a
2 2 2 2 4
6
2 2 2 sp
4
E i y l k
1 1 1 1 1 1

28
 Building a Tree

8 10
e
8 4
4 4
6
2 2 2
r s n a sp
2 2 2 2 4
E i y l k
1 1 1 1 1 1

29
 Building a Tree

10
16
4
6
2 2 e 8
2 sp 8
4
E i y l k 4 4
1 1 1 1 1 1

r s n a
2 2 2 2

30
 Building a Tree

10 16
4
6 e 8
2 2 8
2 sp
4 4 4
E i y l k
1 1 1 1 1 1
r s n a
2 2 2 2

31
 Building a Tree

26

16
10

4 e 8
6 8
2 2 2 sp 4 4
4
E i y l k
1 1 1 1 1 1 r s n a
2 2 2 2

32
 Building a Tree
•After
enqueueing
26 this node
there is only
16
10 one node left
4 e 8
in priority
6 8 queue.
2 2
2 sp 4 4
4
E i y l k
1 1 1 1 1 1 r s n a
2 2 2 2

33
 Encoding the File
Traverse Tree for Codes
• Perform a traversal of the tree
to obtain new code words LEFT
• Going left is a 0 going right is a RIGHT
1 26

• code word is only completed 10

16
when a leaf node is reached
4 e 8
6 8
2 2 2 sp 4 4
4
E i y l k
1 1 1 1 1 1 r s n a
2 2 2 2

35
 Encoding the File
Traverse Tree for Codes
Char Code
E 0000 LEFT
i 0001
y 0010 RIGHT 26
l 0011 16
k 0100 10
0101 4 e 8
space011 6 8
e 10 2 2 2 sp 4 4
r 1100 4
E i y l k
s 1101 1 1 1 1 1 1 r s n a
n 1110 2 2 2 2

a 1111
36
 Encoding the File
• Rescan text and encode file
Char Code
using new code words E 0000
Eerie eyes seen near lake. i 0001
y 0010
0000101100000110011 l 0011
k 0100
1000101011011010011 0101
1110101111110001100 space 011
1111110100100101 e 10
r 1100
s 1101
n 1110
a 1111
37
 Encoding the File
Results
• 73 bits to encode the text 0000101100000110011
• ASCII would take 8 * 14 1000101011011010011
= 1110101111110001100
1111110100100101
112 bits
Compression ratio = bits for ASCII representation / number of bits
transmitted
= 112 / 73 = 1.534

Thus, the size of the transmitted file is 100 / 1.534 = 65 % of the

original ASCII file

Or 38

73/112*100= 65 % of the original ASCII file

 Decoding the File
• How to decode this file ?
26
• Once receiver has tree it scans
incoming bit stream 16
•0 ⇒ go left 10
•1 ⇒ go right 4 e 8
6 8
2 2 2 sp 4 4
0000101100000110011 4
E iy l k
1000101011011010011 1 1 1 1 1 1 r s n a
2 2 2 2
1110101111110001100
1111110100100101

39
 Huffman Coding Example-2
• Character (or symbol) frequencies
- A: 20% (.20)
• e.g., ‘A’ occurs 20 times in a 100 character document, 1000
times in a 5000 character document, etc.
- B: 9% (.09)
- C: 15% (.15)
- D: 11% (.11)
- E: 40% (.40)
- F: 5% (.05)

• Must know frequency of every characters in the

document
51
 Huffman Coding Example-2
• Here are the symbols and their associated frequencies.
• Now we combine the two least common symbols (those
with the smallest frequencies) to make a new symbol
string and corresponding frequency.

E B F D A C
.4 .09 .05 .11 .20 .15

52
 Huffman Coding Example-2
• Here’s the result of combining symbols once.
• Now we repeat until we have combined all the symbols
into a single string.

E BF D A C
.4 .14 .11 .20 .15

B F
.09 .05

53
Huffman Coding Example-2
ABCDEF
1.0

ABCDF E
.6 .4

BFD AC
.25 .35

BF D A C
.14 .11 .20 .15

B F
.09 .05

54
 Huffman Coding Example ABCDEF
1.0
• Now we assign 0s/1s to each branch 0 1
• Codes (reading from top to bottom) ABCDF E
.6 .4
- A: 010 0 1
- B: 0000
- C: 011 BFD AC
.25 1 0 .35 1
- D: 001 0
- E: 1 BF D A C
- F: 0001 .14 .11 .20 .15
0 1
B F
.09 .05
• Note
- None are prefixes of another
• Decode this. Try decoding right
- 0100111100010000 to left

– What’s the first character? The second? 55

Prefix property
• A code has the prefix property if no character code is the prefix (start of the code) for
another character
• Example:
Symbol Code
P 000
01001101100010
Q 11
R 01 RSTQPT
S 001
T 10

• 000 is not a prefix of 11, 01, 001, or 10

• 11 is not a prefix of 000, 01, 001, or 10 …
Code without prefix property
• The following code does not have prefix property
Symbol Code
P 0
Q 1
R 01
S 10
T 11

• The pattern 1110 can be decoded as QQQP, QTP, QQS, or TS

O(nlog n)