Talgo Dy P

Contents
3 Dynamic Programming
1 Repeated occurrences of
sub-problems
4 Coin changing problem
2 Multiple decomposition
5 Chain matrix multiplication
options along with repeated
sub-problems
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
CM (IIT Kharagpur) Algorithms March 14, 2024 1 / 25

Repeated occurrences of sub-problems
Section outline
Fibonacci number
computation
1 Repeated occurrences of
sub-problems
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Repeated occurrences of sub-problems Fibonacci number computation
Fibonacci number computation


 0 n=0
Fn = 1 n=1
Fn−2 + Fn−1 n > 1

Direct coding of this recurrence requires exponential execution
time
Observe that, Fn−1 = Fn−3 + Fn−2 , so Fn−2 is a repeated
sub-problem
If sub-problems are solved repeatedly, a lot of time is wasted
Naive coding of Fibonacci recursion takes exponential time
But, if solutions of sub-problems computed earlier are reused,
then the computation takes O(n) additions
This is an important feature of the dynamic programming
paradigm
Further optimisations yield O(lg n) algorithm, as discussed earlier, TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
but not considered now

yog, kms kOflm^

Repeated occurrences of sub-problems Fibonacci number computation
Repeating sub-problems for Fibonacci number

computation
Fn Note the occurrence of

repeated sub-problems
Fn−2 Fn−1 Repeated sub-problems
need not be solved
Fn−3 Fn−2 So, Fn may be computed in
linear time, as depicted
Fn−4 Fn−3 Only two previous values
need to be stored
Fn−5 F2 F2 F1 F0
F3 F2 F1
F0 F1
··· TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
Fn Fn−1 Fn−2
IND

19 5 1
yog, kms kOflm^

Multiple decomposition options, repeated sub-problems
Section outline
Coin changing 11 using

denominations {1, 2, 5}
2 Multiple decomposition Chain matrix multiplication
options along with repeated
sub-problems
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Multiple decomposition options, repeated sub-problems Coin changing 11 using denominations {1, 2, 5}
Coin changing 11 using denominations {1, 2, 5}
11 Each edge is a
5 min 1
coin change
2
option
6 9 10 Denominations
5 min 1 5 min 1 5 min 1
shown in blue
2 2 2
Denominations
1 4 5 4 7 8 5 8 9 have cost of 1
2 min
1 1
5
min 2 5
2
min 1 Otherwise, cost
is 1 + minimum
2 3 6 2 5 3 6 7 child cost
2 min
1 Repeated
Denomination values shown in blue sub-problem
1 2
Evaluation of red nodes is redundant (why?) instances solved TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
only once
IND

Repeated problem instances are shown in purple 19 5 1
yog, kms kOflm^

Multiple decomposition options, repeated sub-problems Chain matrix multiplication
Chain matrix multiplication

A1 × A2 × A3 × A4
| {z }
min
×1 ×3
×2
A1 × A2 × A3 × A4 A1 × A2 × A3 × A4 A1 × A2 × A3 × A4
|{z} | {z } | {z } | {z } | {z } |{z}
min min
×1 ×2 ×1 ×2
A2 × A3 × A4 A2 × A3 × A4 A1 × A2 × A3 A1 × A2 × A3
|{z} | {z } | {z } |{z} |{z} | {z } | {z } |{z}
There are n − 1 ways to split the product A1 × A2 × . . . × An

Each splitting may result in a different cost, the minimum is to be chosen
The cost of computing a m × l matrix A and a l × n matrix B is m · n · l
Cost of computing a longer matrix multiplication chain may be
determined by splitting recursively
Note the emergence of repeated sub-problem instances while splitting TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
Repeated sub-problem instances solved only once
IND

19 5 1
yog, kms kOflm^

Dynamic Programming
Section outline
Algorithm design paradigms

DP examples
3 Dynamic Programming
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Dynamic Programming Algorithm design paradigms
Divide-and-conquer (DC) Break up a problem into two or more

sub-problems, solve each sub-problem independently,
and combine solution to sub-problems to form a solution
to original problem
Dynamic programming (DP) Break up a problem into a series of
sub-problems and use the solutions to build solutions to
larger and larger sub-problems
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Dynamic Programming Algorithm design paradigms
Divide-and-conquer (DC) Break up a problem into two or more

sub-problems, solve each sub-problem independently,
and combine solution to sub-problems to form a solution
to original problem
Dynamic programming (DP) Break up a problem into a series of
sub-problems and use the solutions to build solutions to
larger and larger sub-problems
Difference between DC and DP can be confusing
DP makes repeated use of the solutions of sub-problems
Sub-problems identified for DP are usually overlapping, leading to
identification of common smaller sub-problems later
DP often exhibits optimal substructure where an optimal solution
is constructed from the optimal solution of its sub-problems
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Dynamic Programming DP examples
DP examples
Fibonacci number computation

Chain Matrix Multiplication
Coin changing
Minimum edit distance
Longest common subsequence
All shortest paths (Floyd-Warshall)
Box stacking
Bridge building
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Coin changing problem
Section outline
Formulation for coin

changing
4 Coin changing problem DP solution for coin change
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Coin changing problem Coin changing problem
Definition
Coin change problem
Coin denominations can be modeled by a set of n distinct positive
integer values, arranged in increasing order as w1 = 1 through wn
Given a positive integral amount W , find non-negative integers
{x1 , x2 , . . . , xn } such that
n
P
xi wi = W
i=1
Pn
xi is minimised
i=1
Each xj represents the number of coins of denomination wj used
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Coin change problem
Sometimes the greedy method of picking coins of largest

denomiation works
Coin systems for which the greedy method works are called
canonical coin systems
The greedy algorithm does not work arbitrary coin systems
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Coin change problem
Sometimes the greedy method of picking coins of largest

denomiation works
Coin systems for which the greedy method works are called
canonical coin systems
The greedy algorithm does not work arbitrary coin systems
Consider forming coin change of 6 units in the coin system:
{1, 3, 4}
The greedy method would yield {4, 1, 1}, however, the optimal
change for this system is {3, 3}
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Coin changing problem Formulation for coin changing
Formulation for coin changing

Consider a denomination wj ≤ W
If we know how make the change optimally for amount W − wj , then we
can make change for W by including a coin of denomination wj
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


→
−
Let C(W , w ) be the function that returns the change count, given the
→
−
vector of denominations w
C(0, ) = 0
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


→
−
→
−
C(0, ) = 0
To get optimal change for amount W , we consider all possibilities
→
− →
−
C(W , w ) = min C(W − wj , w ) + 1
wj ≤W
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


→
−
→
−
C(0, ) = 0
→
− →
−
C(W , w ) = min C(W − wj , w ) + 1
wj ≤W
This function may not be defined for certain amounts

→
− →
−
If w = {2, 3}, C(1, w ) is not defined
→
−
We will not have this problem if 1 ∈ w
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


→
−
→
−
C(0, ) = 0
→
− →
−
C(W , w ) = min C(W − wj , w ) + 1
wj ≤W
This function may not be defined for certain amounts

→
− →
−
If w = {2, 3}, C(1, w ) is not defined
→
−
We will not have this problem if 1 ∈ w
A direct coding of this recursive formulation will lead to an exponential
time solution TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
But, we can efficiently form the solution bottom-up in O(|w|W ) time

Coin changing problem DP solution for coin change
DP solution for coin change

change(w[], k, W) // coins of k denominations in w
1 C[0] ← 0 // initialisation for p=0
2 for p ← 1 to W // do coin changes bottom-up
3 min ← ∞
4 for i ← 0 to k-1
5 if w[i] ≤ p then // don’t overpay
// decide whether w[i] is a good choice
6 if 1 + C[p − w[i]] < min then
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
9 D[p] ← coin // best coin to pick for W
10 C[p] ← min
11 return C and S
Working of change({1, 2, 5}, 3, 14)
p: 0
C[p]: 0 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1
C[p]: 0 1 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2
C[p]: 0 1 1 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3
C[p]: 0 1 1 2 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4
C[p]: 0 1 1 2 2 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5
C[p]: 0 1 1 2 2 1 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5 6
C[p]: 0 1 1 2 2 1 2 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5 6 7
C[p]: 0 1 1 2 2 1 2 2 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5 2
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5 6 7 8
C[p]: 0 1 1 2 2 1 2 2 3 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5 2 1
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5 6 7 8 9
C[p]: 0 1 1 2 2 1 2 2 3 3 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5 2 1 2
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5 6 7 8 9 10
C[p]: 0 1 1 2 2 1 2 2 3 3 2 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5 2 1 2 5
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5 6 7 8 9 10 11
C[p]: 0 1 1 2 2 1 2 2 3 3 2 3 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5 2 1 2 5 1
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5 6 7 8 9 10 11 12
C[p]: 0 1 1 2 2 1 2 2 3 3 2 3 3 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5 2 1 2 5 1 2
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min
11 return C and S
p: 0 1 2 3 4 5 6 7 8 9 10 11 12 13
C[p]: 0 1 1 2 2 1 2 2 3 3 2 3 3 4 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5 2 1 2 5 1 2 1
19 5 1
yog, kms kOflm^


3 min ← ∞
4 for i ← 0 to k-1
7 min ← 1 + C[p − w[i]]
8 coin ← w[i]
10 C[p] ← min Time complexity of DP solution: Linear
11 return C and S in W , exponential in number of bits of W
p: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
C[p]: 0 1 1 2 2 1 2 2 3 3 2 3 3 4 4 TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

D[p]: 0 1 2 1 2 5 5 2 1 2 5 1 2 1 2
19 5 1
yog, kms kOflm^

Section outline
matrix multiplication
DP formulation for chain
5 Chain matrix multiplication matrix multiplication
Chain matrix multiplication Bottom-up chain matrix
Formulation for chain matrix multiplication
multiplication DP solution for chain matrix
Naive solution for chain multiplication
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Chain matrix multiplication Chain matrix multiplication

Example (Chain matrix multiplication)
Let A1 be a 10 × 100 matrix
Need to compute A1 × A2 × A3
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


Cost of (A1 × A2 ) × A3 : (10 × 100 × 5) + (10 × 5 × 50) = 7, 500
Cost of A1 × (A2 × A3 ): (100 × 5 × 50) + (10 × 100 × 50) = 75, 000
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


Cost of (A1 × A2 ) × A3 : (10 × 100 × 5) + (10 × 5 × 50) = 7, 500
Cost of A1 × (A2 × A3 ): (100 × 5 × 50) + (10 × 100 × 50) = 75, 000
Definition
Chain matrix multiplication Given n matrices, A1 , . . . , Aı , . . . , An , where
for 1 ≤ ı ≤ n, Aı is a pı−1 × pı , matrix, parenthesise the product
A1 × . . . × Aı × . . . × An so as to minimize the total cost of
multiplication, assuming that the cost of multiplying a pı−1 × pı matrix TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

by a pı × pı+1 matrix using the naive algorithm is pı−1 × pı × pı+1

19 5 1
yog, kms kOflm^

Chain matrix multiplication Formulation for chain matrix multiplication

If there is just a single matrix, there is nothing to decide
For n (n ≥ 2), we need to divide the problem into two parts
suitably, in one of the n − 1 possible ways
The sub-problems are solved optimally to get the requisite solution
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


However, the quality of the solution is dependent on the point of
division
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


division
So, we need to consider all possible ways to divide the problem
into two parts and retain the best choice
The total
 number of possible solutions to be handled is huge:
 1 n=1
Nn = n−1
P
 Nk Nn−k n > 1
k=1
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


division
So, we need to consider all possible ways to divide the problem
into two parts and retain the best choice
The total
 number of possible solutions to be handled is huge:
 1 n=1  1 n=0
Nn = n−1
P ≡ Cn−1 = n−2
P
 Nk Nn−k n > 1  Ck Cn−k n ≥ 1
k=1 k=0

n
(n − 1) Catalan number, Cn−1 ∈ Ω 43
th TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
n2 yog, kms kOflm^

Chain matrix multiplication Naive solution for chain matrix multiplication
Naive solution for chain matrix multiplication

chnMatMulSim(int p[x-1 .. y]) { // dimensions
1 int n=y-x+1;
2 if (n==1) return 0; // single matrix
3 for L = 2 to n { // lengths of subchains
4 for i = x to n-L+1 { // starts of subchains
5 j = i+L-1; // ends of subchains
6 c = ∞; // init to find min cost
7 for k = i to j - 1 { // check all splits
8 q = chnMatMulSim(p[i-1,k] +
chnMatMulSim[k,j] + p[i-1]×p[k]×p[j];
9 if (q < c) c = q; // check for lower cost
10 }
11 }
12 }
13 return c; TE
OF
TECHNO
LO
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
}
Chain matrix multiplication Naive solution for chain matrix multiplication
Analysis of naive method


 0 n=1
T (n) = n−1
P
 (T (k) + T (n − k) + c) n > 1
k=1
n−1
P
= 2 T (k) + (n − 1)c n>1
k=1
Pn
T (n + 1) = 2 T (k) + nc n>1
k=1
n−1
X
= 2 T (k) + (n − 1)c +2T (n) + c n>1
| k=1 {z }
T (n)
= 3T (n) + c n>1
T (n) = 3T (n − 1) + c n>1
= 12 3n−1 − 1 c OF
TECHNO
LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
Thus, the naive method works in exponential time.
IND

19 5 1
yog, kms kOflm^

Chain matrix multiplication DP formulation for chain matrix multiplication
DP formulation for chain matrix multiplication
Key observation:
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Key observation: Occurrence of common sub-problems

5
Q 4
Q
To solve for Aı , need to consider A1 × A2 while solving for Aı
ı=1 ı=1
3
Q
and also Aı
ı=1
May other sub-problems are also repeated
Considerable savings possible by reusing solutions to earlier
identified sub-problems (memoization), thereby avoiding solving
those again and again
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Key observation: Occurrence of common sub-problems

5
Q 4
Q
To solve for Aı , need to consider A1 × A2 while solving for Aı
ı=1 ı=1
3
Q
and also Aı
ı=1
May other sub-problems are also repeated
Considerable savings possible by reusing solutions to earlier
identified sub-problems (memoization), thereby avoiding solving
those again and again
Problems can be solved bottom-up to obtain required solution
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Chain matrix multiplication Bottom-up chain matrix multiplication
Bottom-up chain matrix multiplication

Example (Efficient computation of A1 × A2 × . . . × A7 )
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


TECHNO
OF
A1 A2 A3 A4 A5 A6 A7
LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


(A1 × A2 ) (A2 × A3 ) (A3 × A4 ) (A4 × A5 ) (A5 × A6 ) (A6 × A7 )

TECHNO
OF
A1 A2 A3 A4 A5 A6 A7
LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


3 4 5 6 7

Q Q Q Q Q
Aı Aı Aı Aı Aı
i=1 i=2 i=3 i=4 i=5
(A1 × A2 ) (A2 × A3 ) (A3 × A4 ) (A4 × A5 ) (A5 × A6 ) (A6 × A7 )

TECHNO
OF
A1 A2 A3 A4 A5 A6 A7
LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


4 5 6 7

Q Q Q Q
Aı Aı Aı Aı
i=1 i=2 i=3 i=4
3 4 5 6 7

Q Q Q Q Q
Aı Aı Aı Aı Aı
i=1 i=2 i=3 i=4 i=5
(A1 × A2 ) (A2 × A3 ) (A3 × A4 ) (A4 × A5 ) (A5 × A6 ) (A6 × A7 )

TECHNO
OF
A1 A2 A3 A4 A5 A6 A7
LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


5 6 7

Q Q Q
Aı Aı Aı
i=1 i=2 i=3
4 5 6 7

Q Q Q Q
Aı Aı Aı Aı
i=1 i=2 i=3 i=4
3 4 5 6 7

Q Q Q Q Q
Aı Aı Aı Aı Aı
i=1 i=2 i=3 i=4 i=5
(A1 × A2 ) (A2 × A3 ) (A3 × A4 ) (A4 × A5 ) (A5 × A6 ) (A6 × A7 )

TECHNO
OF
A1 A2 A3 A4 A5 A6 A7
LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


6 7

Q Q
Aı Aı
i=1 i=2
5 6 7

Q Q Q
Aı Aı Aı
i=1 i=2 i=3
4 5 6 7

Q Q Q Q
Aı Aı Aı Aı
i=1 i=2 i=3 i=4
3 4 5 6 7

Q Q Q Q Q
Aı Aı Aı Aı Aı
i=1 i=2 i=3 i=4 i=5
(A1 × A2 ) (A2 × A3 ) (A3 × A4 ) (A4 × A5 ) (A5 × A6 ) (A6 × A7 )

TECHNO
OF
A1 A2 A3 A4 A5 A6 A7
LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


7
Q
Aı
6 i=1
7
Q Q
Aı Aı
5 i=1
6 i=2
7
Q Q Q
Aı Aı Aı
4 i=1
5 i=2
6 i=3
7
Q Q Q Q
Aı Aı Aı Aı
3 i=1
4 i=2
5 i=3
6 i=4
7
Q Q Q Q Q
Aı Aı Aı Aı Aı
i=1 i=2 i=3 i=4 i=5
(A1 × A2 ) (A2 × A3 ) (A3 × A4 ) (A4 × A5 ) (A5 × A6 ) (A6 × A7 )

TECHNO
OF
A1 A2 A3 A4 A5 A6 A7
LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Bottom-up chain matrix multiplication (contd.)

Example (Upper triangular DP matrix for A1 × A2 × . . . × A7 )
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^


A1
A2
A3
A4
A5
A6
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
A7

A1 (A1 × A2 )
A2 (A2 × A3 )
A3 (A3 × A4 )
A4 (A4 × A5 )
A5 (A5 × A6 )
A6 (A6 × A7 )
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
A7

3
Q
A1 (A1 × A2 ) Aı
i=1
4
Q
A2 (A 2 × A 3 ) Aı
i=2
5
Q
A3 (A3 × A4 ) Aı
i=3
6
Q
A4 (A4 × A5 ) Aı
i=4
7
Q
A5 (A5 × A6 ) Aı
i=5
A6 (A6 × A7 )
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
A7

3 4
Q Q
A1 (A1 × A2 ) Aı Aı
i=1 i=1
4 5
Q Q
A2 (A 2 × A 3 ) Aı Aı
i=2
i=2
5
6
Q Q
A3 (A3 × A4 ) Aı Aı
i=3 i=3
6 7
Q Q
A4 (A4 × A5 ) Aı Aı
i=4 i=4
7
Q
A5 (A5 × A6 ) Aı
i=5
A6 (A6 × A7 )
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
A7

3 4 5
Q Q Q
A1 (A1 × A2 ) Aı Aı Aı
i=1 i=1 i=1
4 5 6
Q Q Q
A2 (A 2 × A 3 ) Aı Aı Aı
i=2
i=2
5
i=2
6
7
Q Q Q
i=3 i=3 i=3
6 7
Q Q
A4 (A4 × A5 ) Aı Aı
i=4 i=4
7
Q
A5 (A5 × A6 ) Aı
i=5
A6 (A6 × A7 )
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
A7

3 4 5 6
Q Q Q Q
A1 (A1 × A2 ) Aı Aı Aı Aı
i=1 i=1 i=1 i=1
4 5 6 7
Q Q Q Q
A2 (A 2 × A 3 ) Aı Aı Aı Aı
i=2
i=2
5
i=2
6
i=2
7

Q Q Q
i=3 i=3 i=3
6 7
Q Q
A4 (A4 × A5 ) Aı Aı
i=4 i=4
7
Q
A5 (A5 × A6 ) Aı
i=5
A6 (A6 × A7 )
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
A7

3 4 5 6 7
Q Q Q Q Q
A1 (A1 × A2 ) Aı Aı Aı Aı Aı
i=1 i=1 i=1 i=1 i=1
4 5 6 7
Q Q Q Q
A2 (A 2 × A 3 ) Aı Aı Aı Aı
i=2
i=2
5
i=2
6
i=2
7

Q Q Q
i=3 i=3 i=3
6 7
Q Q
A4 (A4 × A5 ) Aı Aı
i=4 i=4
7
Q
A5 (A5 × A6 ) Aı
i=5
A6 (A6 × A7 )
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
A7
Chain matrix multiplication DP solution for chain matrix multiplication
DP solution for chain matrix multiplication

chainMatMul(int p[0 .. n], int n) { // dimensions
1 int s[1 .. n - 1, 2 .. n]; // split positions
2 for i = 1 to n m[i, i] = 0; // single matrix
3 for L = 2 to n { // lengths of subchains
4 for i = 1 to n-L+1 { // starts of subchains
5 j = i+L-1; // ends of subchains
6 m[i, j] = ∞; // init to find min cost
7 for k = i to j - 1 { // check all splits
8 q = m[i,k] + m[k+1,j] + p[i-1]×p[k]×p[j];
9 if (q < m[i, j]) { // check for lower cost
10 m[i, j] = q;// found, so update
11 s[i, j] = k; // record split for min cost
12 }
13 }
14 }
15 }
16 return m[1, n] and s;
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^
}
Optimal chain matrix multiplication

matMuOptl(i, j) {
1 if (i == j) return A[i]; // base case
2 else {
3 k = s[i, j]; // split position for Ai × . . . × Aj
4 X = matMulOpt(i, k); // X = Ai × . . . × Ak
5 Y = matMulOpt(k + 1, j); // Y = Ak +1 × . . . × Aj
6 return X×Y;
7 }
}
TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

19 5 1
yog, kms kOflm^

Optimal chain matrix multiplication

matMuOptl(i, j) {
1 if (i == j) return A[i]; // base case
2 else {
3 k = s[i, j]; // split position for Ai × . . . × Aj
4 X = matMulOpt(i, k); // X = Ai × . . . × Ak
5 Y = matMulOpt(k + 1, j); // Y = Ak +1 × . . . × Aj
6 return X×Y;
7 }
}
Space requirement with DP: O(n2 )
Time requirement with DP: O(n3 ) – DP solution scheme
encounters n2 sub-problems each of size n2 and requiring
evaluation of n2 − 1 parenthesisations
Time requirement with naive (non DP) approach: Θ 3n−1

TECHNO
OF LO
TE
GY
ITU
IAN INST
KH
ARAGPUR
IND

Better algorithms are also available

19 5 1
yog, kms kOflm^

Talgo Dy P

Uploaded by

Copyright:

Available Formats

You might also like

Talgo Dy P

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Talgo Dy P

Uploaded by

Copyright:

Available Formats

Contents

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 1 / 25

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 2 / 25

Fibonacci number computation

but not considered now

CM (IIT Kharagpur) Algorithms March 14, 2024 3 / 25

Repeating sub-problems for Fibonacci number

Fn Note the occurrence of

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 4 / 25

Coin changing 11 using

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 5 / 25

Coin changing 11 using denominations {1, 2, 5}

Repeated problem instances are shown in purple 19 5 1

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 6 / 25

Chain matrix multiplication

There are n − 1 ways to split the product A1 × A2 × . . . × An

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 7 / 25

Algorithm design paradigms

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 8 / 25

Algorithm design paradigms

Divide-and-conquer (DC) Break up a problem into two or more

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 9 / 25

Algorithm design paradigms

Divide-and-conquer (DC) Break up a problem into two or more

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 9 / 25

Fibonacci number computation

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 10 / 25

Formulation for coin

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 11 / 25

Coin changing problem

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 12 / 25

Coin change problem

Sometimes the greedy method of picking coins of largest

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 13 / 25

Coin change problem

Sometimes the greedy method of picking coins of largest

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 13 / 25

Formulation for coin changing

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 14 / 25

Formulation for coin changing

yog, kms kOflm^

CM (IIT Kharagpur) Algorithms March 14, 2024 14 / 25

Formulation for coin changing