Note 18 Jan 2024 (2) 5

Algorithms
Lesson #3:
Greedy Algorithms (cont.)
(Based on slides by Prof. Dana Shapira)
1
717N
9174422M
Binary codes
7108212418By 2177147318
737 9187021217 11371
Example
a b c d e f
Frequency 45 13 12 16 9 5
2127 April 3 to
Fixed-length 000 001 010 011 100 101 L(C1) = 300
Codeword (C1)
74 97 1.45 3 13 12161 419 5

Variable-length 0 101 100 111 1101 1100 L(C2) = 224
codeword (C2)
By99
28,81071 Which one is cheaper?
2
231
p
7510 010 2977 be a
a
Prefix-free Codes
 Codes in which no codeword is also a prefix of some
other codeword.
 Example:
a b c d e f
Variable-length 0 101 100 111 1101 1100
codeword
DNA is if f
frame
110001001101
 Easy to encode and decode using prefix-free codes.
 No Ambiguity !! Uniquely Decipherable (UD) NOW
IN 77130
2217 I
It try ftp.s non
3
Prefix-free Codes
a b c d e f
Variable-length 0 101 100 111 1101 1100
codeword
0 101 100 111 110 1100
1100 = 110 0 = “f"

rath

Has
 or JUD
 1100 = 110 + 0 = “ea"
4
Prefix-free Codes
 Represented as a binary tree
 each edge represents either 0 (left) or 1 (right)

h em I in Nobel
 each leaf corresponds to a codeword
ad
which is the day
sequence of 0s and 1s traversed from the root
to reach it.
 Since no prefix is shared, all codewords are at the leaves,

and decoding a string means following edges, according to
the sequence of 0s and 1s in the string, until a leaf is
5
reached.
0 1
1107701
20
M a
6 10
cla
o 1
b c
Tree Representation
C Ca
a b c d e f a b c d e f
Frequency 45 13 12 16 9 5 Frequency 45 13 12 16 9 5
Fixed-length 000 001 010 011 100 101 Variable-length 0 101 100 111 1101 1100
codeword codeword
C 8yydyyeayx.ws
100 C 0
100
1
771 MR
0
1
5770140 I'M
86 14 a:45 0 55
0 1 0 1
14 25 30
58 28 0 1 1
1 0 1 0 1 0
0
c:12 b:13 14 d:16
a:45 b:13 c:12 d:16 e:9 f:5 0 1
f:5 e:9
mspnkfhany.tk H.pk
eppdsi P
1 6
1 I 5
do Mr pox I for
Building an optimal binary
code
Problem: We are given a set of symbols with frequencies.
How can we build an optimal code for them?
V W X Y Z
7 15 6 5 6
Walk If MAR
Shannon-Fano algorithm
 Sort the symbols by frequency
 Always split as close to the middle as possible (w.r.t.
frequency) 03Ndlast 6 741
V W X Y Z 514 W
7 15 6 5 6 b 2871817
Y X Z V W
exe
JOHNNY O
5 6 6 7 15 Total: 39 YXZ VW
17 22
YX Z V W
Y X Z
5 6 6 Z V W
Y X
11 6
Y X
V W X Y Z
7 15 6 5 6
The Shannon-Fano code:

Is this optimal?
0 1 A better code:
IN 214
0 1 0 1 0 1
Z:6 V:7 W:15 W:15

0 1 0 1
Y:5 X:6
0 1 0 1
Y:5 X:6 V:7 Z:6
L = 89
L = 87
9
Walk If SF e mar 14 A
Huffman Algorithm
 Greedy
 The two smallest nodes are chosen at each step,
and this local decision results in a globally optimal
encoding tree.
 bottom-up manner
 Starts with a set of || leaves and performs a
sequence of || - 1 "merging" operations to create
the final tree.
Professor David A. Huffman 10
(August 9, 1925 - October 7, 1999)

Huffman Algorithm
HUFFMAN( )
1 n | |
2 Q 
What is the running time?
3 for i 1 to n - 1
4 do ALLOCATE-NODE(z)
5 left[z] x EXTRACT-MIN(Q)
6 right[z] y EXTRACT-MIN(Q)
7 w[z] w[x] + w[y]
8 INSERT(Q, z)
9 return EXTRACT-MIN(Q)
11
NOT3172171 an HIM A He 710005
Huffman Algorithm
f:5 e:9 c:12 b:13 d:16 a:45 c:12 b:13 14 d:16 a:45
0 1
MAX f:5 e:9
30
1 d:16 25 a:45
14 25 a:45 0 1 0 1
0 1 0 1
c:12 b:13 14 d:16
f:5 e:9 c:12 b:13
1
0
DX
I A f:5 e:9
OC's
a:45 55 0
100 1
0 1 a:45 55
25 30 0 1 Mr HANNA
0 0 1
1 25 30
c:12 b:13 14 d:16 0 1 0 1 IN
0 1 d:16
c:12 b:13 14 DID it
f:5 e:9 0 1
12
f:5 e:9
Optimality of Huffman Codes
Theorem:
Given weights w1,…,wn. Huffman Algorithm assigns code
lengths l1,…,ln such that is minimal.
Lemma 1:
An optimal code for a file is always represented by a full
binary tree, in which every non-leaf node has two
children.
Lemma 2:
In an optimal tree the two lowest weights wn-1 and wn are
in the lowest level.
13
wa lemma
I 28ND May bf.PE in
I do exp pills win

µ
I
p apr 6
a 8 8 60 or
walk lemma2
DID a 20 113s em a effa 2
2 weak 80 lemma
15 2 Y
775 7H PM RH 2PM ftp.GJYH IN 7 117 0 a
lemma
7341216 Box away 3014 ND XD
for
µ
an
If P I am 800 DX
121 2 WON
Ʃ at
yo Me an
NXX 8 PM A Nd Py n No 718
T 78201 2
NAGH Odor pox Ip
T
Noth yh ye If Not ok se Wbk The ex
Tl PH walk fao Mark

lengtht
Whiffy
G F fWattown a 7 R'M
LITTCLT 8W TN DC to be the ya
G CL T LIT Worth a alien B

119CLT Whether
4 4 WhatWas L Hwathin
or a No L T LT
LIT't a3 19 A

Lemma 3:
In an optimal tree the two lowest weights wn-1 and wn can
be assumed to be brothers.
14
Proof that Huffman is optimal:
By induction on n.
Given Σ = (s1,…,sn) with multiplicities (w1,…,wn), Huffman
replaces symbols sn-1, sn by a new symbol z with
multiplicity wn-1+wn, then constructs a tree T' for
Σ'=(s1,…,sn-2,z), then obtains T by replacing
z
by
sn-1 sn
By induction assumption, T' is optimal

15
What is the relation between L(T) and L(T')?
L(T) = L(T') + wn-1 + wn (1)
Suppose for a contradiction that there exists a tree

T* for Σ with L(T*) < L(T)
W.l.o.g. sn-1, sn are neighbors in T*
Can replace in T* by z
sn-1 sn
By (1), we got a tree for Σ' better than T'. Contradiction

16
Algorithms
Lesson #4:
Dynamic Programming
1
Fibonacci Sequence
13141005720
 F(0)=0
 F(1)=1
 F(n)=F(n-1)+F(n-2)
 Write a recursive algorithm!

 What is its running time?
TH Th 1 Tn2 Fat fat Fol
o on Fo FAI 7146
071.6 7171 461
ABDON gas pot add 275 2

77828 on ipad p I pse post MH DP
pH AND 2x 0 0380
Binomial Coefficients
n! n 
 
k ! n  k  !  k 
 Recursive equation:
 n  n  1   n  1 
k    k   k  1
     
 Write a recursive algorithm!

408 3
Dynamic Programming Approach
to Optimization Problems
1. Characterize structure of an optimal solution.
2. Recursively define value of an optimal solution.
3. Compute value of an optimal solution in bottom-up

fashion.
4. Construct an optimal solution from computed

information.
4
World Series Odds1710
 Two teams A and B play to see who is the first to win
n games. In world series games n=4.
 Team A wins any particular game with probability q

(same for all the games).
 What is the probability of A winning the tournament?
5
groupA groupB 131
us we jan 3107
A n B
P 19 3
A on
my pli it sxdgfdmwr.BY
Bd Ad yy IN i
8 Br an
P 8,87 73107ft and 8
18 april pipe
World Series Odds
P(i,j) - The probability that A will win the

tournament, given that it is missing i games to win and B
is missing j games to win.
1 I 1 am of M of steam
P(0,j)=1 j>0 in
P(i,0)=0 i>0 7H ok a non µ
0 n dapr8
P(i,j)=q*P(i-1,j)+(1-q)*P(i,j-1) i>0 and j>0
P a 3
P 21 31 P2,3 1
6
Recursive Solution
P(i,j){
if i==0 return 1
elseif j==0 return 0
else return q*P(i-1,j)+(1-q)*P(i,j-1)
} 37
1141
If psfNan a 7am ped
rol
7
52Nd 13 6070318 DP 04 Apr
Dynamic Programming
0 1 2 3 4
0 1 1 1 1
P(n,m){
1 0 1/2 3/4 7/8 15/16
for(i=0; in; i++)
T[0,i] = 1 2 0 1/4 1/2
for(j=1; jm; j++)
T[j,0] = 0 3 0 1/8
for(i=1; in; i++)
4 0 1/16
for(j=1; jm; j++)
BSB.EC q=1/2
T[i,j]= q*T[i-1,j]+ (1-q)* T[i,j-1];
return T[n,m]
} What is its running time? 042
1210,17
PAX P1,0 16,0 1.1 1.0 12 Arpad 8
J
1378
Matrix Chain Multiplication
-Problem:
Bid 600 pm R
Given n matrices M1,M2,…,Mn, compute the product
M1M2M3…Mn, where Mi has dimension di-1 x di , for i = 1,…,n.
-objective
compute M1,M2,…,Mn with the minimum number of scalar
multiplications.
-Given matrices A with dimension p x q and B with

dimension q x r, multiplication AB takes pqr scalar
multiplications
9
-Problem: Parenthesize the product M1M2…Mn in a way
to minimize the number of scalar multiplications.
-Example: M1 --- 2 x 5
M2 --- 5 x 3
M3 --- 3 x 4
M1(M2M3) --- 60 + 40 = 100 multiplications

(M1M2)M3 --- 30 + 24 = 54 multiplications
10
of B
IE
dogfoo P
P got p.VN 7001807700N
AB A
5807 7 1 an AxBXc Axβxc
AN jbox
Me 25
My 5 3
M 3 4
M MÉM 5 3 4 1 2 5 4 60 40 10
MoMa Ms 2 5 3 2 3 4 30 24 54
I'M 24 730 7
 Let m(i,j) be the number of multiplications
performed using optimal parenthesis of
MiMi+1…Mj-1Mj.
i
Me m(i , j )   0mama i j
min {m(i,k) + m(k+1,j)+di-1dk dj } i  j
 i k<j
p or 6
ya  Let S(i,j) be the optimal split point for Mi…Mj
(meaning S(i,j)=k means we split (Mi…Mk)(Mk+1…Mj))
 S(i,j) is needed to reconstruct the solution

11
grap be 0
MUL(i,j){
if (i==j) return 0; Mamp if Kyle Ip 73
else{ 7 04
x MM
for k=i to j-1 MA
yMUL(i,k)+MUL(k+1,j)+di-1dkdj
if y<x {
xy
S(i,j)k
}
}
}
What is its running time?
12
Recursive Running Time
n 1
T  n    T  k   T  n  k   1
k 1
n 1
T  n   2 T  k    n  1
k 1
n2
T  n  1  2 T  k    n  2 
k 1
T  n   T  n  1  2T  n  1  1
T  n   3T  n  1  1  3T  n  1  32 T  n  2     3k T  n  k 
1
001
13
112 8308
pon3yylepptfmb.br
Dynamic Programming
-Example: M1 --- 2 x 5
ij is
m
of a Ese
1 2 3
M2 --- 5 x 3
1 0 30 54
8
1
M3 --- 3 x 4
2 0 60
No
3 I 0
 0 i j S 1 2 3
m (i , j )  min {m(i,k) + m(k+1,j)+d d d } i  j
 i k<j i-1 k j
1 1 2
MHA mind
m
m's
12 Kl
I
I
s 3
as
dat
2
3
2
If
Ip
14
m2,31 minds b 0 0 5 34 60
mom
mail.mn as 1 0 60 2 5 4 10
3 4
MARI m3,3 dodads 30 0 2 54
4211131
MUL(n){
for (i=1;in;i++) m[i,i]=0;
for (diff=1;diffn-1;diff++){
for (i=1;in-diff;i++){
ji+diff
x
for (k=i;kj-1;k++)
ym[i,k]+m[k+1,j]+di-1dkdj
if y<x {
xy
S(i,j)k
} ex
m[i,j]=x;
PP alex
}
} 01m 737 16
15
The Integer Knapsack Problem
- again 000018 PN 07 d 160
• Problem –
Given G1,G2,…,Gn, each Gi with integer weight wi
and value vi.
Maximize the profit out of the goods you can put
in the knapsack with capacity C.
 (can’t cut goods in the middle)
n
 Maximize fv
i 1
i i
 Subject to f w
i 1
i i C
 How can we solve this using DP?

AND are yy 2781 Here a
16
 Let T[i,B] be the maximum profit achievable with
goods G1,…,Gi, under weight limit B
 We are interested in T[n,C] 17

HIGH
Kien id
 Recursive formula for T[i,B]:
BE
p
BIT
in 3 1
4 4
T[i,B] = max(T[i-1,B], vi + T[i-1,B-wi])
TOB3 0 PIN µ
 Boundary constraints… 70
T i o 0 HIM
5 02 03 64 55 Go
weight 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
limit
up to
item
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Bike
1 0 0 0 0 0 0 0 0 0 4 4 4 4 4 4 4
2 0 0 0 0 0 0 8 8 8 8 8 8 8 8 8 12
3 0 0 0 7 7 7 8 8 8 15 15 15 15 15 15 15
4 0 0 0 7 7 7 8 8 8 15 15 15 15 16 16 16
5 0 0 0 7 7 7 8 8 9 15 15 15 15 16 17 17
6 0 0 10 10 10 17 17 17 18 18 19 25 25 25 25 26
Basalese T6,153786 man

T 3,9 Max T 2,9 7 T 2,6 Max 8,78 Max 815 15
kxikdkxbfxyyfaplaxb
A
pfklfyy
73 is .my
paid
p3M b 8839
T 6,95 max T 5,15 10 1 513 Max 17,1016 moxa267 260
61 Bayyad
5 13 Max T 4,133 2 4,83 max16,28 16
SAM
T 4,133 Max T 3,133,1 43,9 Max15,16 16
T 3,93
T 2,63 Max 6,63 8 6,051 Max 0,8 80

0,0 Base case
10 1 7 8 26
W 2 4 3 6 15 034
runtime no c is J dam
input c
1118123
Vein vn
WaWaoooWn input s
read
B
C logo if Gcp
lo9 Wo
Verano.vn 1094,109k co
Wa o.e Wnlo9We lo9Wayoe lo9Wn
J
O
What is the running time?
Running time = Θ(Size of the table)
Size of the table = n*C
Is this polynomial time in the size of the input?
NO! Because the size of C in the input is O(log C)!
“Pseudopolynomial”
Had GH the c
Bps 7037
The table just gives us the profit. How do we reconstruct
the actual packing…?
Algorithms
Lesson #5:
BFS
1
Graph
 A graph is a pair (V, E), where
 V is a set of nodes, called vertices
 E is a collection of pairs of vertices, called edges  (VV)
 If edge pairs are ordered, the graph is directed, otherwise
undirected.
V
p V
µ
U X Z U X Z
W W
2
Y Y
Nth
IEFM 8
Paths
 Path
 sequence of vertices v0 , v1 ,..., v p  where  vi , vi 1   E
 Simple path
100 717 211217 6254 18
 If no vertex in the path appears more than
once V
a b
P1
U d X Z
P2 h
c e
W g
f 3
Y
Cycles
 Circuit grandnoted a
 A path v , v ,..., v  where

0 1 p
v0  v p .
 Simple circuit
 If no vertex, other than the start-end vertex,
appears more than once, and the start-end vertex
does not appear elsewhere V
in the circuit
U X Z
C2
C1
W
4
Y
ADD NO
Graph Representations
 Adjacency Lists.
3011 I did
a b a b d c
b a c
c d c d a b
d a c
 Adjacency Matrix.
1 2 1 2 3 4
a b
1 0 1 1 1 ply 100N
2 1 0 1 0 5125281 D List 824
3 1 1 0 1
3
c d
4 4 1 0 1 0
654 7017 field
Bw 5
Breadth-First Search &
Depth-First Search
Objective: Visit all the vertices of the graph, by
following the edges
app 7170730 787 8217
For each node u:
v1 v2 v3 BFS: Visit all the nodes reachable from

u before continuing with nodes
reachable from v1 017178
DFS: Visit all the nodes reachable from

u v1 before continuing with v2
pod
Keep some info about the search process – will be useful
for applications
6
Breadth-first Search
 Input: Graph G = (V, E), directed or undirected,
and source vertex s  V.
 Output: AN AID
for all v  V
 d[v] = distance from s to v.
urspymxmalxo.de no
 [v] = u such that (u, v) is the last edge on shortest path from s to v. is
 Builds breadth-first tree with root s that contains all reachable vertices.
 Colors the vertices to keep track of progress.

to 77667
– Undiscovered. eyed
– Discovered but not finished.
– Finished.
T
6 P's
MP a 2912177
7
700 ID'D
BFS for Shortest Paths
A In 2437717241
7 may
7Ham
2 3 3
2
2 3
1 S
S 2 S
1 1
2
2 3 3
8
plan 8581770
BFS(G,s)
1. for each vertex u in V[G] – {s}
2 do color[u]  white
3 d[u]  
4 [u]  NULL
5 color[s]  gray pictograph
6 d[s]  0
7 [s]  NULL f 3174881410
null
not
8 Q
www.efif
9 enqueue(Q,s)
10 while Q  
Running time is O(|V|+|E|)
11 do u  dequeue(Q)
12 for each v in Adj[u] pray
13 do if color[v] = white
14 then color[v]  gray 7100
15 d[v]  d[u] + 1
16 [v]  u
17 enqueue(Q,v)
18 color[u]  black 1083
1 5 0
9
Example
Let G be the following graph:
A B
F J
I C
H
E D
What is BFS(G,A)?
BFS(G,D)? ShortestPath
NON BFS
10
BFS correctness
Let G = (V,E) be a graph and sV.
 δ(s,v) - the shortest path distance from s to v (the minimum
number of edges).
 δ(s,v) = ∞ if there is no path from s to v.
Theorem:
1. During its execution, BFS discovers every vertex vV that
is reachable from s.
2. Upon termination, d[v] = δ(s,v).
3. For all reachable vertices v except for s, one of the shortest
paths from s to v is a shortest path from s to π[v] followed
by the edge (π[v],v).
11
CondED CSM
BFS
V53 873
as
8274
92112177 60 tensor 3 and

daged Hbd
87202 xD are da 6
a
may MM
Had A
a
pm
do
IN 1317 distance
Q 53 87319
1 DON Na
8 0
Q 445,553
tpsop.im 7379
2 445555,5 71
µs 7108119
Mayor
1 8 fy a da pth and
WHICH 0943171
MY
By dist of 214 M y es are a
554 0120 JUDS SV
S
non SP 4H My 137
84fromBFS
737 a drone all
a v
make
3111737 A MINK
JEWISH
Dc 02
BFS correctness
Lemma 1:  edge (u,v)E, δ(s,v) ≤ δ(s,u) + 1
Lemma 2: Upon termination of BFS:vV, d[v] ≥ δ(s,v).
12
BFS correctness
 Lemma 3: If the queue Q contains vertices <v1, .. vr>

where v1 and vr are Q’s head and tail, then:
1. d[vr] ≤ d[v1] + 1
2. d[vi] ≤ d[vi+1] for i=1,2,..,r–1
 Corollary : If vertex vi was enqueued before vertex vj

during BFS, then d[vi] ≤ d[vj] when vj is enqueued.
13
Proof of the Theorem
 By contradiction, assume that there is a vertex v such that
d[v]≠ δ(s,v), and choose v with minimum δ(s,v). Using Lemma
2 d[v]>δ(s,v).
 v ≠ s, since d[s]=0=δ(s,s).
 v is reachable from s for otherwise δ(s,v)=∞ ≥d[v].
Let u be the vertex immediately preceding v on a shortest path

from s to v.
 δ(s,v) = δ(s,u)+1.  δ(s,u)<δ(s,v)  from minimality δ(s,v)
, d[u] = δ(s,u).
* d[v]>δ(s,v)=δ(s,u)+1=d[u] + 1
14
Proof (cont.)
Consider now the time when u is dequeued. Vertex v is either
white, gray, or black.
 v is white: then in line 15 d[v]=d[u]+1 which contradicts
*.
 v is gray: There exists wV that was dequeued before u

d[v]=d[w]+1. but d[w]d[u] (corollary)  d[v]d[u]+1
which contradicts *.
 v is black: v was already dequeued  d[v]  d[u]
(corollary) thus d[v]<d[u]+1 which contradicts *.
d[v] = δ(s,v)
 If π[v]=u, then d[v]=d[u]+1. Thus, we obtain the shortest path
from s to v by taking a shortest path from s to π[v] and then
traversing the edge (π[v],v).
15
Breadth-first Tree
 For a graph G = (V, E) with source s, the predecessor subgraph
of G is G = (V , E) where
 V ={vV : [v]  NULL}{s}
 E ={([v],v)E : v  V - {s}}
 The predecessor subgraph G is a breadth-first tree if:

 V consists of the vertices reachable from s and
 for all vV , there is a unique simple path from s to v in G

that is also a shortest path from s to v in G.
 The edges in E are called tree edges.
|E | = |V | - 1.
16
Breadth-first Tree
After performing BFS we can print the vertices on the shortest path
from s to v in linear time:
print_path(G,s,v)
1. if v=s print s
2. else if [v] =NULL
3. print (“no path”)
4. else
5. print_path(G,s, [v] )
6. print(v)
7.
17
Algorithms
Lesson #5b:
DFS
Topological Sort
Strongly Connected Components
1
Graph Representations
 Adjacency Lists.
a b a b d c
b a c
c d c d a b
d a c
 Adjacency Matrix.
1 2 1 2 3 4
a b
1 0 1 1 1
2 1 0 1 0
c d 3 1 1 0 1
3 4 4 1 0 1 0
2
Depth-first Search
Input: G = (V, E), directed or undirected.

 Output:
 for all v  V.
 d[v] = discovery time (v turns from white to gray) key
pokers
 f [v] = finishing time (v turns from gray to black) GOWN
1108M [v] : predecessor of v = u, such that v was discovered during
Bd the scan of u’s adjacency list.
 Forest of depth-first trees:
Gπ = (V,Eπ) Eπ = {(π[v],v), vV and π[v] ≠ null}
 Colors the vertices to keep track of progress.

– Undiscovered.
– Discovered but not finished.
– Finished.
3
DFS(G) 00 xxx 217217 IN 000GPD
DFS-Visit(u)
Main Loop 2119110 do
1. color[u]  GRAY
1. for each vertex u  V[G] 2. time  time + 1 1131420
2. do color[u]  white 3. d[u]  time
3. [u]  NULL 4. for each v  Adj[u]
4. time  0 5. do if color[v] = WHITE
5. for each vertex u  V[G] 6. then [v]  u
6. do if color[u] = white 7. DFS-Visit(v)
7. then DFS-Visit(u) 8. color[u]  BLACK
III 9. f[u]  time  time + 1
Running time is Θ(|V|+|E|)

4
a
Example (DFS) A 790
u v w
DF
1/8
1/8 2/7 9/12
4/5 3/6 10/11

x y z
call stack
(x)
DFS-visit(y)
DFS-visit(v) (z)
DFS-visit(u) DFS-visit(w)
time 5
P 76107 6N
Parenthesis Theorem
Theorem
For all u, v, exactly one of the following holds:
1. d[u] < f [u] < d[v] < f [v] or d[v] < f [v] < d[u] < f [u] and neither
u nor v is a descendant of the other.
2. d[u] < d[v] < f [v] < f [u] and v is a descendant of u.
3. d[v] < d[u] < f [u] < f [v] and u is a descendant of v.
 So d[u] < d[v] < f [u] < f [v] cannot happen.

 Corollary
v is a proper descendant of u if and only if
d[u] < d[v] < f [v] < f [u]. 6
8
June
t.am
ññÑ net ar y 149.1313 JFK
Iran
naw Vava Pds 13130
7104 2PM m8o3 IFS Visitly Sky
Wav 71 08231 JFSVisit a S 1 004 7 5ND JUN
van Hereal
p.gl if WB Vd by 514 7042 5H
won 63 pie
atte adept set her set told pola AsVisitor
Another example
y z s t
3/6 2/9 1/10 11/16
B F C
Yo B
4/5 7/8 12/13

C 14/15
C C
x w v u
(s (z (y (x x) y) (w w) z) s) (t (v v) (u u) t)
7
ph 4ND JUN
White-path Theorem
Theorem
v is a descendant of u if and only if at time d[u],
there is a path u v consisting of only white
vertices.
Proof
 direction: Algorithm only proceeds on white vertices, so path must
have been white.
 direction: By induction on the length of the white path. Let w be
the vertex preceding v in the path. By induction assumption, w is a
descendant of u. What color was v when we examined the edge (w, v)?
White?... Gray/black?...
8
1H For W
28 461 3 3 V I 10
5 2150 U 6 40 DAD
Froney apr yxb.pk
depot for a U lid 50 111 s
73T
PPI 83 000
pm don't B
1313 in Uk 7 Pathlength 4 AND
4 6
K Path
6 101
length
Vkt I AM got VK To afford Van 8 PM ID w 4 I

4 6 1337
JN's
pH Kala poi pra't too
tendon Cola µ
or
Apr 6 Has a prom
pylon Had if
Hinder add Ged of a add her
a
µ 10 83k
1
214 7088
I 43137
In 77351 Rdaperson bar don
ph
roud Mm U ask.NU 1313
2130
737.0104 ftp 37yyt
Nov 9317 theeetge
I
Classification of Edges
 Tree edge: Edges in Gπ. v was found by exploring (u, v).
 Back edge: (u, v), where u is a descendant of v in Gπ .

841 18 1513 2X b 00 722 jan1788 34 2179M bx UN NOB e
 Forward edge: (u, v), where v is a descendant of u, but
not a tree edge. A me dbc be 3 1h
µ
PM µ payed
 Cross edge: any other edge. Can go between vertices
in same depth-first tree or in different depth-first
trees.
IT 3 I 2H pnp cross
9
Identification of Edges
 Edge type for edge (u, v) can be identified when
it is first explored by DFS. v8 sell Us
rajon
 Identification is based on the color of v.
 White – tree edge.
 Gray – back edge.
 Black – forward or cross edge.
d[u] < d[v] d[u] > d[v]
10
Identification of Edges
Theorem:
In DFS of an undirected graph, we get only tree and
back edges. No forward or cross edges.
Proof:
 Let (u,v)E. w.l.o.g let d[u] < d[v].
Then v must be discovered and finished before u is finished.
 If the edge (u,v) is explored first in the direction uv, then
v is white until that time then it is a tree edge .
 If the edge is explored in the direction, vu, u is still gray
at the time the edge is first explored, then it is a back edge.
11
Afca forward
µ µ
Directed
Characterizing a DAG Graph
Lemma
A directed graph G is acyclic iff a DFS of G yields
no back edges.
Proof: a safe da e SE BE pink

 : 030
a 66 1130731 Gky He Had 750 75dB
Back
 Suppose there is a back edge (u, v). Then v is an Edge
ancestor of u in depth-first forest.

 Therefore, there is a path v u, so v u v
is a cycle. dad O back e.pk
v u back 144
U 4313 a
B rad JUN 8rd 12
Jan 01 e
Back
Edge Back I c dad e

JAN 1TH
room lean v
60N no jor Ma a
down 8D at aged h 20 UN papaya 6
star pA U
ph
VG Is
pox 4,0
backor
Characterizing a DAG
Lemma
A directed graph G is acyclic iff a DFS of G yields no back edges.
Proof (Cont.):
  :
 c : cycle in G, v : first vertex discovered in c, (u, v) :
preceding edge in c.
 At time d[v], vertices of c form a white path v u.
Why?
 By white-path theorem, u is a descendent of v in
depth-first forest. v u
 Therefore, (u, v) is a back edge.
B 13
Topological Sort
Want to “sort” a directed acyclic graph (DAG).
A B D
Order the vertices such
that all edges go forward
C E
A B C D E
D P
8 or A B D C E
Max
14
NDC
µ
68N a DAF XD IN 3017 a 301 I m in
DFS a 21H tape yipe DFS a 6 MAN

MAR µ
timeF I D 917214 or AN
Hd F A MAR 9191 F 08 7211471 Y
MF IN 14177 E
F ON D SH
F Joe D D8 77 c 1 NO
B SH
did A Rod
A B c D E
IN Rda ad 0317 Ped is idiola
ATTIC DE
XD Xp MEGA Nso
IV El DFS IN IN Ns
3
dad is
ex 710k too a 861 UH XD DX
pot
7207 of Baylorp53 7124 V 001 FCA 1811114
µ
SON 13H w DAG ago as sea
628
W fu du m UH 710k at V8
TE n ist Rf is so good to
U V f SFU
austere V6 ft 710k a 7H Upk
pl UNM ffassful
Topological Sort
 Performed on a DAG.
 Linear ordering of the vertices of G such that if
(u, v)  E, then u appears somewhere before v.
Topological-Sort (G)
1. call DFS(G) to compute finishing times f [v] for all v  V
2. as each vertex is finished, insert it onto the front of a
linked list
3. return the linked list of vertices
Running time is Θ(|V|+|E|) 15

Example
A B D
9/10 5/8 1/4
6/7 2/3
C E
Linked List:
9/10 5/8 6/7 1/4 2/3
A B C D E
16
Correctness Proof
 Just need to show if (u, v)  E, then f [v] < f [u].
 When we explore (u, v), what are the colors of u and v?
 u is gray.
 Is v gray, too?
 No, because then v would be an ancestor of u.
  (u, v) is a back edge.
  contradiction of Lemma (DAG has no back edges).
 Is v white?
 v is a descendant of u.
 By parenthesis theorem, d[u] < d[v] < f [v] < f [u].
 Is v black?
 Then v is already finished.
 Since we’re exploring (u, v), we have not yet finished u.
  f [v] < f [u].
17
Strongly Connected
Components
 G is strongly connected if every pair (u, v) of
vertices in G is reachable from each other.
 A strongly connected component (SCC) of G is a
maximal set of vertices C  V such that for all u,
v  C, both u v and v u exist.
a b c d e
f g h i j
no He to 18
5070177 Scc
µ I pynd 0017 May 8 217217Goady
e 7k
JAY app 2 far
Not be
130 8179771 730 Bad MA a377 1 s Mar 1th
97 IS BY 03 06 8
Component Graph
 GSCC = (VSCC, ESCC).
 VSCC has one vertex for each SCC in G.
 ESCC has an edge if there is an edge between the
corresponding SCC’s in G.
GSCC for the example considered:
abfg cdh ej 213031

7Nd
i
apr and 201
5 1 DFS 1707 to xp11
fyek.sn
anykphS eldNddOd 1 19
for AD se can n d
S
Claim: GSCC is a DAG

Proof:
 Suppose for a contradiction that GSCC contains a
cycle
 All the vertices in these components are reachable

from one another
 They should have been in the same component
20
231 Pood I 1777170
Asa jul 4St july 07 alla ist da 7138µg k

o b
yo done't Id pyar
Karin 8D pie
751 MA and exam pl

WP W 2128 98 me xD By
m am nice be a Dfs a bear
141 ROD of 12001 dfs 7 Me 0305 17 Y

µ
570 7377N 100737
USUN
µ I C c
psych 077 l to 879 H
fan de 77 a
C f D ON
C job cite pig y yo fake
It ow ply da n
III PW be ajar PD dmin TX Apr 4 co 217911 I
6H of Urup It e µ vs an dm e pot 441 Po
pi
flat for a fix a melon ftp.odl.ph how
thin stamina 2 DX
M Id 14 PBI And today eyes 8D
pool 9N
cp Hb apr 8 into pit as far
p
C a fax fax C
538 jeans 6yd As 8nd apr
dyl Is an oar 61 and ya yo If a
A G fol Mas 6 8301 family

2706368 dat xsy.pt on
p mm 8 001as ke
716 3770 to
6271720 DFS
007 0 707 870 4 BLT bi GT 8301
In DFS 7 P.IN DESI
ON 37 5
Algorithm to determine SCCs
Idea: Do DFS, going “from the bottom up” in the Main
Loop, so that each SCC becomes a separate tree
4 2
5 3
21
Idea (cont.):
 We will do 2 rounds of DFS. The first round will give us
the vertex order for the Main Loop of the second round
 In the first DFS, the Main Loop vertex order is

arbitrary
Notation:
 d[u] and f [u] always refer to first DFS.
 Extend notation for d and f to sets of vertices U V:
 dmin(U) = minuU{d[u]} (earliest discovery time)
 fmax(U) = maxuU{ f [u]} (latest finishing time)
2171911
Gp plasma d enjoy fee IN Paxman P'G on 22
in e sa
111
SCCs and DFS finishing times

Lemma
Let C and C be distinct SCC’s in G = (V, E). Suppose there is a path
u v such that u  C and v C. Then fmax(C) > fmax(C).
x
Proof: C C
 Case 1: dmin(C) < dmin(C)
u v
 Let x be the first vertex discovered in C.
 At time d[x], all vertices in C and C are white. Thus, there exist paths
of white vertices from x to all vertices in C and C.
 By the white-path theorem, all vertices in C and C are descendants of x
in depth-first tree.
 By the parenthesis theorem, f [x] = fmax(C) > fmax(C).
23
C C
u v
 Case 2: dmin(C) > dmin(C) y
 Let y be the first vertex discovered in C.

 At time d[y], all vertices in C are white and there is a white path
from y to each vertex in C  all vertices in C become
descendants of y. Again, f [y] = fmax(C).
 At time d[y], all vertices in C are also white.
 By earlier lemma, we cannot have a path from C' to C.
 So no vertex in C is reachable from y.
 Therefore, at time f [y], all vertices in C are still white.
 Therefore, for all w  C, f [w] > f [y], which implies that
fmax(C) > fmax(C).
24
Example
First DFS
a b c d
13/14 11/16 1/10 8/9
12/15 3/4 2/7 5/6

e f g h
25
finishing times
This almost works: fmax gives opposite order
Solution: Reverse all the graph’s edges for 2nd DFS
fmax = 10
14 16 10 9
fmax = 16
15 4 7 6
fmax = 7 fmax = 6
26
Transpose of a Directed
Graph
 GT = transpose of directed G.
 GT = (V, ET), ET = {(u, v) : (v, u)  E}.
 GT is G with all edges reversed.
 Can create GT in Θ(V + E) time if using
adjacency lists.
 G and GT have the same SCC’s. (u and v are
reachable from each other in G if and only
if reachable from each other in GT.)
27
SCC(G)
1. call DFS(G) to compute finishing times f [u] for all u
2. compute GT
3. call DFS(GT), but in the main loop, consider vertices in
order of decreasing f [u] (as computed in first DFS)
4. output the vertices in each tree of the depth-first forest
formed in second DFS as a separate SCC
Running time is (V+E)

28
Example (continued)
Second DFS
1 2
16 10
7 6
3 4
29
Example
cd
abe
fg h
30
Convex hull computation –
Graham’s scan
Gabriel Nivasch
Convex hull
Let 𝑆 be a set of 𝑛 points in the plane
The convex hull of 𝑆 is a convex

polygon 𝑃, with vertices in 𝑆,
that contains all the points of 𝑆
Convex hull

Convex hull

How do we compute the convex

hull efficiently?
Basics: Orientation of 3 points
Given points 𝑝1 , 𝑝2 , 𝑝3 , not on the same line, 𝑝3
𝑝2 𝑝1
𝑝1
right turn left turn
𝑝3 𝑝2
The path 𝑝1 → 𝑝2 → 𝑝3 can make either a left turn or a right turn at 𝑝2
Basics: Orientation of 3 points
Given points 𝑝1 , 𝑝2 , 𝑝3 , not on the same line, 𝑝3
𝑝2 𝑝1
𝑝1
right turn left turn
𝑝3 𝑝2
The path 𝑝1 → 𝑝2 → 𝑝3 can make either a left turn or a right turn at 𝑝2
left 𝑝2
𝑝1
right
How do we calculate whether 𝑝1 , 𝑝2 , 𝑝3 make a left or a right turn?
1 1 1 Positive: Left turn

Claim: Look at the sign of det 𝑥1 𝑥2 𝑥3 Negative: Right turn
𝑦1 𝑦2 𝑦3 Zero: Points are collinear
1 1 1 Positive: Left turn

Claim: Look at the sign of det 𝑥1 𝑥2 𝑥3 Negative: Right turn
𝑦1 𝑦2 𝑦3 Zero: Points are collinear
Example: 𝑝3 = (4,7)
1 1 1
det 1 7 4 = 33 Left turn
2 1 7
𝑝1 = (1, 2)
𝑝2 = (7,1)
Proof: We first prove the 0 case
Recall: A line is a set of the form 𝐿 = { 𝑥, 𝑦 : 𝑎𝑥 + 𝑏𝑦 = 𝑐} where 𝑎, 𝑏 are not both 0

Example: 3𝑥 + 4𝑦 = 5

Given 𝑝1 , 𝑝2 , 𝑝3 let 𝐿 = { 𝑥, 𝑦 : 𝑎𝑥 + 𝑏𝑦 = 𝑐} be the line through 𝑝1 , 𝑝2

Is 𝑝3 also in 𝐿?


Say 𝑎 ≠ 0. Suppose w.l.o.g. 𝑎 = 1


1 1 1
Take the matrix 𝑥1 𝑥2 𝑥3 and do the row operation 𝑟2 ← 𝑟2 + 𝑏𝑟3 − 𝑐𝑟1
𝑦1 𝑦2 𝑦3


1 1 1
𝑦1 𝑦2 𝑦3
Determinant is unchanged


1 1 1
𝑦1 𝑦2 𝑦3
Determinant is unchanged Row 𝑟2 becomes 0 if and only if 𝑝3 ∈ 𝐿



1 1 1
𝑦1 𝑦2 𝑦3
Determinant is unchanged Row 𝑟2 becomes 0 if and only if 𝑝3 ∈ 𝐿

(Also 𝑦1 ≠ 𝑦2 )
Proof (cont.):
Suppose 𝑃 = (𝑝1 , 𝑝2 , 𝑝3 ) make a left turn, and 𝑄 = (𝑞1 , 𝑞2 , 𝑞3 ) also
We can transform 𝑃 into 𝑄 continuously, without the points ever becoming collinear
𝑃 𝑃′ 𝑃′′ 𝑄
rotate change change

angle lengths
Proof (cont.):

angle lengths
Determinant varies continuously, and never becomes 0

Proof (cont.):

angle lengths
Determinant varies continuously, and never becomes 0
➔ 𝑃 and 𝑄 have the same determinant sign

Proof (cont.):
Verify an easy “left turn” case: 𝑝1 = 0, 0 , 𝑝2 = 1, 0 , 𝑝3 = (0, 1)
𝑝3
𝑝1 𝑝2
Proof (cont.):
𝑝3 Determinant is positive
𝑝1 𝑝2
Proof (cont.):
𝑝1 𝑝2
➔ Every left turn has positive determinant

Proof (cont.):
𝑝1 𝑝2
➔ Every left turn has positive determinant
“Right turn”: Similarly QED

Graham’s scan
Input: Sequence of 𝑛 points 𝑝1 = 𝑥1 , 𝑦1 , … , 𝑝𝑛 = (𝑥𝑛 , 𝑦𝑛 )
Output: Vertices of the CH, in clockwise order
Graham’s scan
(Assume general position: No 3 points collinear)

Graham’s scan
(Assume general position: No 3 points collinear)
Step 1: Find a point 𝑞1 that is certainly a CH vertex, say point with

lowest y-coordinate
Time: 𝑂(𝑛)
Step 2: Sort the remaining points clockwise w.r.t. 𝑞1
𝑞7
𝑞3 Whenever the sorting algorithm
𝑞6 asks “is 𝑟 < 𝑠?” we check
𝑞5 𝑞8 whether 𝑟, 𝑞1 , 𝑠 make a left turn
𝑞2 𝑞4 𝑞9
𝑞10 Time: 𝑂(𝑛 log 𝑛)

𝑞11
𝑞1
Step 3:
𝑞7
• Initialize a stack to 𝑆 = (𝑞1 , 𝑞2 ) 𝑞3
• For 𝑖 = 3 to 𝑛: 𝑞5
𝑞6
𝑞8
• While 𝑆 −2 , 𝑆 −1 , 𝑞𝑖 make a left turn
• 𝑆. pop()
𝑞4
• 𝑆. push(𝑞𝑖 ) 𝑞2 𝑞9
• Return the points in 𝑆 𝑞10

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Step 3:
𝑞7
𝑞6
𝑞8
• 𝑆. pop()
𝑞4

𝑞11
𝑞1
Running time of step 3?
• For 𝑖 = 3 to 𝑛:
• 𝑆. pop()
• 𝑆. push(𝑞𝑖 )
• While 𝑆 −2 , 𝑆 −1 , 𝑞𝑖 make a left turn 𝑂(𝑛) worst case
• 𝑆. pop()
• 𝑆. pop()
Worst-case time 𝑂(𝑛2 )??

• 𝑆. pop()
No! Each point is pushed once, and popped at most once

• 𝑆. pop()
No! Each point is pushed once, and popped at most once
Step 3 running time: 𝑂(𝑛)

This is called amortized analysis
Time to handle each point 𝒒𝒊 in step 3:

• Worst case: 𝑂 𝑛
• Amortized: 𝑂(1)
Meaning, handling all 𝑛 points together
takes time 𝑂(𝑛), so we divide that by 𝑛
This is called amortized analysis
Time to handle each point 𝒒𝒊 in step 3:

• Worst case: 𝑂 𝑛
• Amortized: 𝑂(1)
Meaning, handling all 𝑛 points together
takes time 𝑂(𝑛), so we divide that by 𝑛
Overall running time of Graham’s scan: 𝑂(𝑛 log 𝑛)

Note 18 Jan 2024 (2) 5

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Note 18 Jan 2024 (2) 5

Uploaded by

Copyright:

Available Formats

Algorithms

(Based on slides by Prof. Dana Shapira)

74 97 1.45 3 13 12161 419 5

1100 = 110 0 = “f"

 each edge represents either 0 (left) or 1 (right)

 Since no prefix is shared, all codewords are at the leaves,

Problem: We are given a set of symbols with frequencies.

How can we build an optimal code for them?

The Shannon-Fano code:

Z:6 V:7 W:15 W:15

Professor David A. Huffman 10

(August 9, 1925 - October 7, 1999)

MAX f:5 e:9

I do exp pills win

Tl PH walk fao Mark

G CL T LIT Worth a alien B

Optimality of Huffman Codes

By induction assumption, T' is optimal

L(T) = L(T') + wn-1 + wn (1)

Suppose for a contradiction that there exists a tree

W.l.o.g. sn-1, sn are neighbors in T*

By (1), we got a tree for Σ' better than T'. Contradiction

(Based on slides by Prof. Dana Shapira)

 Write a recursive algorithm!

ABDON gas pot add 275 2

 Write a recursive algorithm!

2. Recursively define value of an optimal solution.

3. Compute value of an optimal solution in bottom-up

4. Construct an optimal solution from computed

 Team A wins any particular game with probability q

 What is the probability of A winning the tournament?

P(i,j) - The probability that A will win the

-Given matrices A with dimension p x q and B with

M1(M2M3) --- 60 + 40 = 100 multiplications

 S(i,j) is needed to reconstruct the solution

 How can we solve this using DP?

 We are interested in T[n,C] 17

Basalese T6,153786 man

T 2,63 Max 6,63 8 6,051 Max 0,8 80

(Based on slides by Prof. Dana Shapira)

 A path v , v ,..., v  where

v1 v2 v3 BFS: Visit all the nodes reachable from

DFS: Visit all the nodes reachable from

 Colors the vertices to keep track of progress.

92112177 60 tensor 3 and

Lemma 1:  edge (u,v)E, δ(s,v) ≤ δ(s,u) + 1

Lemma 2: Upon termination of BFS:vV, d[v] ≥ δ(s,v).

 Lemma 3: If the queue Q contains vertices <v1, .. vr>

 Corollary : If vertex vi was enqueued before vertex vj

Let u be the vertex immediately preceding v on a shortest path

 The predecessor subgraph G is a breadth-first tree if:

 for all vV , there is a unique simple path from s to v in G

 Colors the vertices to keep track of progress.

Running time is Θ(|V|+|E|)

4/5 3/6 10/11

 So d[u] < d[v] < f [u] < f [v] cannot happen.

4/5 7/8 12/13

Vkt I AM got VK To afford Van 8 PM ID w 4 I

 Back edge: (u, v), where u is a descendant of v in Gπ .

d[u] < d[v] d[u] > d[v]

Characterizing a DAG Graph

Proof: a safe da e SE BE pink

ancestor of u in depth-first forest.

Edge Back I c dad e