BITS, PILANI – K. K.

BIRLA GOA CAMPUS

Design & Analysis of Algorithms

(CS F364)

Lecture No. 11

1
 Observations
Question 1
How many subproblems are used in an optimal solution
for the original problem?
• Coin Change Problem
Two Which?
• Matrix Chain Multiplication
Two Which?
Question 2
How many choices we have in determining which
subproblems to use in an optimal solution?
• Coin Change Problem
Two Why?
• Matrix Chain Multiplication
j - i choices for k (splitting the product)
 DP – Time Complexity
Intuitively, the running time of a dynamic
programming algorithm depends on two factors:
1. Number of subproblems overall
2. How many choices we have for each subproblem
Matrix multiplication:
– O(n2) subproblems
– At most n-1 choices
Therefore, Time complexity is O(n3) overall
 Longest Common Subsequence (LCS)
Given two sequences
X = x1, x2, …, xm
Y = y1, y2, …, yn
find a maximum length common subsequence
(LCS) of X and Y
Example
X = A, B, C, B, D, A, B
Subsequences of X:
– A subset of elements in the sequence taken in order
For example, A, B, D, B, C, D, B, etc.
 Example
Example
X = A, B, C, B, D, A, B
Y = B, D, C, A, B, A
B, C, B, A is a longest common subsequence of
X and Y (length = 4)
B, D, A, B is also a longest common
subsequence of X and Y (length = 4)
B, D, A, however is not a LCS of X and Y
 Brute Force Solution
Let length of X be m & length of Y be n
Brute Force
For every subsequence of X,
check whether it’s a subsequence of Y
Question? How many subsequences are there of X?
There are 2m subsequences of X
Question?
What is time required to check for each subsequence?
Each subsequence takes O(n) time to check
Scan Y for first letter, then scan for second, & so on
Therefore, Running time: O(n2m) Exponential
 Making the choice
X = A, B, Z, D
Y = Z, B, D
Choice: include one element into the common
sequence (D) and solve the resulting subproblem
X = A, B, E, Z
Y = Z, B, E
Choice: exclude an element from a string and solve
the resulting subproblem
 Notations
• Given a sequence X = x1, x2, …, xm
The i-th prefix of X, for i = 0, 1, 2, …, m is
Xi = x1, x2, …, xi

• c[i, j] = the length of a LCS of the sequences

Xi = x1, x2, …, xi and Yj = y1, y2, …, yj
 Recursive Solution
Case 1: xi = yj (Last characters match)
Example
Xi = <D, B, Z, E>
Yj = <Z, B, E>

c[i, j] = c[i - 1, j - 1] + 1

• Append xi = yj to the LCS of Xi-1 and Yj-1

• Must find a LCS of Xi-1 and Yj-1
 Recursive Solution
Case 2: xi  yj (Last characters do not match)
In this case xi and yj cannot both be in the LCS
Thus either xi is not part of the LCS, or yj is not part of the LCS
(and possibly both are not part of the LCS).
• In the 1st case LCS of Xi and Yj is the LCS of Xi-1 and Yj
• In the 2nd case LCS of Xi and Yj is the LCS of Xi and Yj -1
Example
Xi = A, B, Z, G & Yj = A, D, Z
1. LCS of Xi-1 = A, B, Z and Yj = A, D, Z
2. LCS of Xi = A, B, Z, G and Yj-1 = A, D

Therefore, c[i, j] =max { c[i - 1, j], c[i, j-1] }

 Recursive Solution
0 if i = 0 or j = 0
c[i, j] = c[i-1, j-1] + 1 if xi = yj
max(c[i, j-1], c[i-1, j]) if xi  yj
 Optimal Substructure
Optimal Substructure Property
Easy to prove that in both the cases
Optimal solution to a problem includes optimal
solutions to subproblems
Cut-Paste Argument
 Overlapping Subproblems
To find a LCS of X and Y
We may need to find the LCS between
X and Yn-1 and that of Xm-1 and Y
Both the above subproblems has the subproblem
of finding the following:
LCS of Xm-1 and Yn-1
Subproblems share subsubproblems
 Computing the Length of the LCS
0 if i = 0 or j = 0
c[i, j] = c[i-1, j-1] + 1 if xi = yj
max(c[i, j-1], c[i-1, j]) if xi  yj
0 1 2 n
yj: y1 y2 yn
0 xi: 0 0 0 0 0 0
1 x1 0 first

2 x2 0 second
0
0
m xm 0 c[m, n]
 Computing the table
0 if i = 0 or j = 0
c[i, j] = c[i-1, j-1] + 1 if xi = yj
max(c[i, j-1], c[i-1, j]) if xi  yj

Along with c[i, j] we also compute and record b[i, j] which

tells us what choice was made to obtain the optimal value
If xi = yj
b[i, j] = “ ”

Else, if c[i - 1, j] ≥ c[i, j-1]

b[i, j] = “  ”

Else
b[i, j] = “  ”
 Pseudo Code for LCS
1. for i ← 1 to m
2. do c[i, 0] ← 0
3. for j ← 0 to n
4. do c[0, j] ← 0
5. for i ← 1 to m
6. do for j ← 1 to n
7. do if xi = yj
8. then c[i, j] ← c[i - 1, j - 1] + 1
9. b[i, j ] ← “ ”
10. else if c[i - 1, j] ≥ c[i, j - 1]
11. then c[i, j] ← c[i - 1, j]
12. b[i, j] ← “↑”
13. else c[i, j] ← c[i, j - 1]
14. b[i, j] ← “←”
15.return c and b Running time: O(mn)
 Example
X = A, B, C, B, D, A, B 0 if i = 0 or j = 0
Y = B, D, C, A, B, A c[i, j] = c[i-1, j-1] + 1 if xi = yj
max(c[i, j-1], c[i-1, j]) if xi  yj
0 1 2 3 4 5 6
yj B D C A B A
If xi = yj
0 xi
b[i, j] = “ ” 0 0 0 0 0 0 0

Else if c[i - 1, j] ≥ c[i, j-1] 1 A 0 0 0 0  1 1 1
b[i, j] = “  ”   
2 B 0 1 1 1 1 2 2
else    
3 C 0 1 1 2 2 2 2
b[i, j] = “  ”
4 B 0   
1 1 2 2 3 3
5 D 0     
1 2 2 2 3 3
6 A 0    
1 2 2 3 3 4
7 B 0    
1 2 2 3 4 4
 Constructing a LCS

Start at b[m, n] and

follow the arrows yj B D C A B A

0 xi
0 0 0 0 0 0 0

When we
1 A 0 0 0 0  1 1 1
encounter a “ “ in   
b[i, j] 2 B 0 1 1 1 1 2 2
 xi = yj is an 3 C 0    
1 1 2 2 2 2
element of the LCS
4 B 0   
1 1 2 2 3 3
LCS is BCBA 5 D 0     
1 2 2 2 3 3
6 A 0    
1 2 2 3 3 4
7 B 0    
1 2 2 3 4 4
 The 0-1 Knapsack Problem
Given: A set S of n items, with each item i having
wi - a positive “weight”
vi - a positive “benefit”
Goal:
Choose items with maximum total benefit but with weight
at most W.
And we are not allowed to take fractional amounts
In this case, we let T denote the set of items we take

Objective : maximize ෍ 𝒗𝒊
𝒊∈𝑻

Constraint : ෍ 𝒘𝒊 ≤ 𝑾
𝒊∈𝑻
 Greedy approach
Possible greedy approach:
Approach1: Pick item with largest value first
Approach2: Pick item with least weight first
Approach3: Pick item with largest value per
weight first
We know that none of the above approaches work
Exercise:
Prove by giving counterexamples.
 Brute Force Approach
• Brute Force
The naive way to solve this problem is to go
through all 2n subsets of the n items and pick
the subset with a legal weight that maximizes
the value of the knapsack.