Dynamic Programming

Dynamic Programming 1
Outline and Reading
Matrix Chain-Product (5.3.1)
The General Technique (5.3.2)
0-1 Knapsack Problem (5.3.3)

Dynamic Programmin 2
 Matrix Chain-Products
Dynamic Programming is a general
algorithm design paradigm.
f
Rather than give the general structure,
let us first give a motivating example:
Matrix Chain-Products
B
Review: Matrix Multiplication. j
e
C = A*B
A is d e and B is e f
e
O(def ) time
e 1 A C
C[i, j ] A[i, k ] * B[k , j ] d i i,j d
k 0

f
Dynamic Programmin 3
Matrix Chain-Products
Matrix Chain-Product:
Compute A=A0*A1**An-1
Ai is di di+1
Problem: How to parenthesize?
Example
B is 3 100
C is 100 5
D is 5 5
(B*C)*D takes 1500 + 75 = 1575 ops
B*(C*D) takes 1500 + 2500 = 4000
ops
Dynamic Programmin 4
Enumeration Approach
Matrix Chain-Product Alg.:
Try all possible ways to parenthesize
A=A0*A1**An-1
Calculate number of ops for each one
Pick the one that is best
Running time:
The number of parenthesizations is
equal to the number of binary trees
with n nodes
This is exponential!
It is called the Catalan number, and it is
almost 4n.
This is a terrible algorithm!
Dynamic Programmin 5
Greedy Approach
Idea #1: repeatedly select the product that
uses the fewest operations.
Counter-example:
A is 101 11
B is 11 9
C is 9 100
D is 100 99
Greedy idea #1 gives A*((B*C)*D)), which takes
109989+9900+108900=228789 ops
(A*B)*(C*D) takes 9999+89991+89100=189090
ops
The greedy approach is not giving us the
optimal value.
Dynamic Programmin 6
Recursive Approach
Define subproblems:
Find the best parenthesization of Ai*Ai+1**Aj.
Let Ni,j denote the number of operations done by this
subproblem.
The optimal solution for the whole problem is N0,n-1.
Subproblem optimality: The optimal solution can
be defined in terms of optimal subproblems
There has to be a final multiplication (root of the expression
tree) for the optimal solution.
Say, the final multiplication is at index i: (A0**Ai)*(Ai+1*
*An-1).
Then the optimal solution N0,n-1 is the sum of two optimal
subproblems, N0,i and Ni+1,n-1 plus the time for the last
multiplication.
Dynamic Programmin 7
Characterizing
Equation
The global optimal has to be defined in terms of
optimal subproblems, depending on where the
final multiplication is at.
Let us consider all possible places for that final
multiplication:
Recall that Ai is a di di+1 dimensional matrix.
So, a characterizing equation for Ni,j is the following:

N i , j min{N i ,k N k 1, j d i d k 1d j 1}
i k j

Note that subproblems are not independentthe

subproblems overlap.
Dynamic Programmin 8
Subproblem Overlap
Algorithm RecursiveMatrixChain(S, i, j):
Input: sequence S of n matrices to be multiplied
Output: number of operations in an optimal parenthesization of S
if i=j
then return 0
for k i to j do
Ni, j min{Ni,j, RecursiveMatrixChain(S, i ,k)+ RecursiveMatrixChain(S, k+1,j)+ di dk+1
dj+1}
return Ni,j

Dynamic Programmin 9
Subproblem Overlap
1..4

1..1 2..4 1..2 3..4 1..3 4..4

2..2 3..4 2..3 4..4 1..1 2..2 3..3 4..4 ...

3..3 4..4 2..2 3..3

Dynamic Programmin 10
Dynamic
Programming
Since
Algorithm
subproblems
overlap, we
Algorithm matrixChain(S):
Input: sequence S of n matrices to be multiplied
dont use Output: number of operations in an optimal
recursion.
parenthesization of S
Instead, we
construct for i 1 to n 1 do
optimal Ni,i 0
subproblems
bottom-up. for b 1 to n 1 do
Ni,is are easy, so { b j i is the length of the problem }
start with them
for i 0 to n b - 1 do
Then do
problems of jib
length 2,3, Ni,j
subproblems,
and so on. for k i to j 1 do
Running time: Ni,j min{Ni,j, Ni,k + Nk+1,j + di dk+1 dj+1}
O(n3)
return N0,n-1
Dynamic Programmin 11
Dynamic
Programming
Algorithm
The bottom-up
construction fills in the
N min{N
i, j
i k j
i ,k N k 1, j d i d k 1d j 1}
N array by diagonals
Visualization
N gets values from
i,j
N 0 1 2 i j n-1

previous entries in i-th 0

row and j-th column 1

Filling in each entry in answer
i
the N table takes O(n)
time.
Total run time: O(n3)
j
Getting actual
parenthesization can
be done by n-1
remembering k for
each N entry
Dynamic Programmin 12
Dynamic Programming
Algorithm Visualization
A0: 30 X 35; A1: 35 X15; A2: 15X5;
A3: 5X10; A4: 10X20; A5: 20 X 25

N i , j min{N i ,k N k 1, j d i d k 1d j 1}
i k j

N1, 4 min{
N1,1 N 2, 4 d1d 2 d 5 0 2500 35 *15 * 20 13000,
N1, 2 N 3, 4 d1d 3 d 5 2625 1000 35 * 5 * 20 7125,
N1,3 N 4, 4 d1d 4 d 5 4375 0 35 *10 * 20 11375
}
7125

Dynamic Programmin 13
Dynamic
Programming Algorithm matrixChain(S):
Since
Algorithm
subproblems
overlap, we
Input: sequence S of n matrices to be multiplied
Output: number of operations in an optimal
dont use parenthesization of S
recursion.
for i 1 to n 1 do
Instead, we
construct Ni,i 0 ; Tii i;
optimal for b 1 to n 1 do
subproblems
bottom-up. { b j i is the length of the problem }
Ni,is are easy, so for i 0 to n b - 1 do
start with them jib
Then do Ni,j ; Tij i;
problems of for k i to j 1 do
length 2,3,
subproblems, If (Ni,j> Ni,k + Nk+1,j + di dk+1 dj+1)
and so on. Ti,jk
Running time: Ni,j min{Ni,j, Ni,k + Nk+1,j + di dk+1 dj+1}
O(n3)
return N0,n-1
Dynamic Programmin 14
Dynamic Programming
Algorithm Visualization

(A0*(A1*A2))*((A3*A4)*A5)
Dynamic Programmin 15
The General Dynamic
Programming Technique
Applies to a problem that at first seems to
require a lot of time (possibly
exponential), provided we have:
Subproblem optimality: the global
optimum value can be defined in terms of
optimal subproblems
Subproblem overlap: the subproblems are
not independent, but instead they overlap
(hence, should be constructed bottom-up).

Dynamic Programmin 16
The 0/1 Knapsack Problem
Given: A set S of n items, with each item i having
wi - a positive weight
bi - a positive benefit
Goal: Choose items with maximum total benefit
but with weight at most W.
If we are not allowed to take fractional amounts,
then this is the 0/1 knapsack problem.
In this case, we let T denote the set of items we take

Objective: maximize b
iT
i

Constraint: w
iT
i W

Dynamic Programmin 17
 Example
Given: A set S of n items, with each item i having
bi - a positive benefit
wi - a positive weight
Goal: Choose items with maximum total benefit but
with weight at most W.
knapsack

Items:
1 2 3 4 5 box of width 9 in

Weight: 4 in 2 in 2 in 6 in 2 in Solution:
item 5 ($80, 2 in)
Benefit: $20 $3 $6 $25 $80
item 3 ($6, 2in)
item 1 ($20, 4in)
Dynamic Programmin 18
A 0/1 Knapsack
Algorithm, First Attempt
Sk: Set of items numbered 1 to k.
Define B[k] = best selection from S k.
Problem: does not have subproblem optimality:
Consider set S={(3,2),(5,4),(8,5),(4,3),(10,9)} of
(benefit, weight) pairs and total weight W = 20

Best for S4:

Best for S5:

Dynamic Programmin 19
A 0/1 Knapsack
Algorithm, Second
Attempt
S : Set of items numbered 1 to k.
k

Define B[k,w] to be the best selection from S k with

weight at most w
Good news: this does have subproblem optimality.

B[k 1, w] if wk w
B[k , w]
max{B[k 1, w], B[k 1, w wk ] bk } else
I.e., the best subset of Sk with weight at most w is
either
the best subset of Sk-1 with weight at most w or
the best subset of Sk-1 with weight at most wwk plus
item k
Dynamic Programmin 20
0/1 Knapsack Algorithm
Consider set S={(1,1),(2,2),(4,3),(2,2),(5,5)} of
(benefit, weight) pairs and total weight W = 10

Dynamic Programmin 21
0/1 Knapsack Algorithm
Trace back to find the items picked

Dynamic Programmin 22
0/1 Knapsack Algorithm
Each diagonal arrow corresponds to adding one item into
the bag
Pick items 2,3,5
{(2,2),(4,3),(5,5)} are what you will take away

Dynamic Programmin 23
0/1 Knapsack Algorithm
B[k 1, w] if wk w
B[k , w]
max{B[k 1, w], B[k 1, w wk ] bk } else
Algorithm 01Knapsack(S, W):
Recall the definition of Input: set S of n items with benefit bi
B[k,w] and weight wi; maximum weight W
Since B[k,w] is defined in Output: benefit of best subset of S with
terms of B[k1,*], we can weight at most W
use two arrays of instead let A and B be arrays of length W + 1
of a matrix
for w 0 to W do
Running time: O(nW).
B[w] 0
Not a polynomial-time for k 1 to n do
algorithm since W may be
large copy array B into array A
for w wk to W do
This is a pseudo-
polynomial time algorithm if A[w wk] bk > A[w] then
B[w] A[w wk] bk
Dynamic Programmin return B[W] 24
Longest Common
Subsequence
Given two strings, find a longest subsequence that they share
substring vs. subsequence of a string
Substring: the characters in a substring of S must occur contiguously in S
Subsequence: the characters can be interspersed with gaps.
Consider ababc and abdcb

alignment 1
ababc.
abd.cb
the longest common subsequence is ab..c with length 3
alignment 2
aba.bc
abdcb.
the longest common subsequence is ab..b with length 3

Dynamic Programmin 25
Longest Common
Subsequence
Lets give a score M an alignment in this way,
M=sum s(xi,yi), where xi is the i character in the first aligned sequence
yi is the i character in the second aligned sequence
s(xi,yi)= 1 if xi= yi
s(xi,yi)= 0 if xiyi or any of them is a gap

The score for alignment:

ababc.
abd.cb
M=s(a,a)+s(b,b)+s(a,d)+s(b,.)+s(c,c)+s(.,b)=3

To find the longest common subsequence between sequences S 1 and S2

is to find the alignment that maximizes score M.

Dynamic Programmin 26
Longest Common
Subsequence
Subproblem optimality
Consider two sequences S1: a1a2a3ai
S2: b1b2b3bj

Let the optimal alignment be

x1x2x3xn-1xn
y1y2y3yn-1yn

There are three possible cases

for the last pair (xn,yn):

Dynamic Programmin 27
 Longest Common
Subsequence
There are three cases for (xn,yn) pair:

S1: a1a2a3
ai
S2: b 1b 2b3
bxj 1x2x3xn-
xn1
y1y2y3yn-
i,j
1y{M
M = MAX n , + S (a b ) (match/mismatch)
i-1 j-1 i, j
Mi,j-1 + 0 (gap in sequence #1)
Mi-1,j + 0 (gap in sequence #2) }

Mi,j is the score for optimal alignment between strings a[1i] (substring of a from index 1 to i) and b[1j]

Dynamic Programmin 28
Longest Common
Subsequence
Mi,j = MAX {
Mi-1, j-1 + S(ai,bj)
Mi,j-1 + 0
Mi-1,j + 0
}

s(ai,bj)= 1 if ai=bj
s(ai,bj)= 0 if aibj or any of them is a gap

Examples:
G A A T T C A G T T A (sequence #1)
G G A T C G A (sequence #2)

Dynamic Programmin 29
 Longest Common
Subsequence
Fill the score matrix M and trace back table B

M1,1 = MAX[M0,0 + 1, M1, 0 + 0, M0,1 + 0] = MAX [1, 0, 0] = 1

Score matrix M Trace back table B

Dynamic Programmin 30
Longest Common
Subsequence
Score matrix M Trace back table B

M7,11=6 (lower right corner of Score matrix)

This tells us that the best alignment has a score of 6
What is the best alignment?
Dynamic Programmin 31
 Longest Common
Subsequence
We need to use trace back table to find out the best alignment,
which has a score of 6

(1) Find the path from lower right

corner to upper left corner

Dynamic Programmin 32
 Longest Common
Subsequence
(2) At the same time, write down the alignment backward
S1
:Take one
character from
each sequence
:Take one character
S2 from sequence S1
(columns)
:Take one character
from sequence S2
(rows)

Dynamic Programmin 33
Longest Common
Subsequence
:Take one
character from
each sequence
:Take one character
from sequence S1
(columns)
:Take one character
from sequence S2
(rows)

Dynamic Programmin 34
 Longest Common
Subsequence
Thus, the optimal alignment is

The longest common subsequence is

G.A.T.C.G..A

There might be multiple longest common subsequences (LCSs)

between two given sequences.

These LCSs have the same number of characters (not include gaps

Dynamic Programmin 35
Longest Common
Subsequence
Algorithm LCS (string A, string B) {
Input strings A and B
Output the longest common subsequence of A and B

M: Score Matrix
B: trace back table (use letter a, b, c for )
n=A.length()
m=B.length()
// fill in M and B
for (i=0;i<m+1;i++)
for (j=0;j<n+1;j++)
if (i==0) || (j==0)
then M(i,j)=0;
else if (A[i]==B[j])
M(i,j)=max {M[i-1,j-1]+1, M[i-1,j], M[i,j-1]}
{update the entry in trace table B}
else
M(i,j)=max {M[i-1,j-1], M[i-1,j], M[i,j-1]}
{update the entry in trace table B}

then use trace back table B to print out the optimal alignment

Dynamic Programmin 36