Unit 3

Dynamic Programming
 Dynamic Programming
• Dynamic Programming is a general algorithm
design technique for solving problems defined
by or formulated as recurrences with
overlapping subinstances.
• Dynamic programming, like the divide-and-
conquer method, solves problems by
combining the solutions to subproblems.
(“Programming” in this context refers to a
tabular method, not to writing computer
code.)
 Steps
• When developing a dynamic-programming algorithm,
we follow a sequence of
• four steps:
• 1. Characterize the structure of an optimal solution.
• 2. Recursively define the value of an optimal solution.
• 3. Compute the value of an optimal solution, typically
in a bottom-up fashion.
• 4. Construct an optimal solution from computed
information.
 Principle of optimality
• The principle of optimality states that an
optimal sequence of decisions has the
property that whatever the initial state and
decision are, the remaining decisions must
constitute an optimal decision sequence with
regard to state resulting from the first
decision.
• Many decision sequences may be generted,
Subsequnces can not be optimal
Exactly the same as divide-and-conquer …
but store the solutions to subproblems for
possible reuse.
A good idea if many of the subproblems are the
same as one another.
There might be O(2n)
nodes in this tree,
but only e.g. O(n3)
different nodes.
 f(7)
Fibonacci series
f(6) f(5)
• 0, 1, 1, 2, 3, 5, 8, 13, 21, …
• f(0) = 0. f(5) f(4) f(4) f(3)
• f(1) = 1.
f(4) f(3) f(3) f(2) f(3) f(2) f(2) f(1)
• f(N) = f(N-1) + f(N-2)
if N 2.
… … … … … … … 1
int f(int n) { f(n) takes exponential time
if n < 2 to compute.
return n Proof: f(n) takes more than
else twice as long as f(n-2), which
return f(n-1) + f(n-2) therefore takes more than
twice as long as f(n-4) …
} Don’t you do it faster?

600.325/425 Declarative Methods - J.

6
Eisner
 Reuse earlier results! f(7)
(“memoization” or “tabling”) f(6)
• 0, 1, 1, 2, 3, 5, 8, 13, 21, …
f(5)
• f(0) = 0.
• f(1) = 1. f(4)
• f(N) = f(N-1) + f(N-2) …
if N 2.

int f(int n) { int fmemo(int n) {

does it matter
if n < 2 if f[n] is undefined
which of these
return n we call first? f[n] = f(n)
else return f[n]
return fmemo(n-1) + fmemo(n-2) }
}
600.325/425 Declarative Methods - J.
7
Eisner
 Backward chaining vs. forward chaining
• Recursion is sometimes called “backward chaining”: start with the
goal you want, f(7), choosing your subgoals f(6), f(5), … on an as-
needed basis.
– Reason backwards from goal to facts
(start with goal and look for support for it)

• Another option is “forward chaining”: compute each value as soon

as you can, f(0), f(1), f(2), f(3) … in hopes that you’ll reach the goal.
– Reason forward from facts to goal
(start with what you know and look for things you can prove)

• Either way, you should table results that you’ll need later
• Mixing forward and backward is possible (future topic)

600.325/425 Declarative Methods - J.

8
Eisner
 Reuse earlier results! f(7)
(forward-chained version)
f(6)
• 0, 1, 1, 2, 3, 5, 8, 13, 21, …
f(5)
• f(0) = 0.
• f(1) = 1. f(4)
• f(N) = f(N-1) + f(N-2) …
if N 2.
Which is more efficient, the
int f(int n) { forward-chained or the backward-
f[0] = 0; f[1]=1 chained version?
for i=2 to n
f[i] = f[i-1] + f[i-2] Can we make the forward-chained
return f[n] version even more efficient?
(hint: save memory)
}
600.325/425 Declarative Methods - J.
9
Eisner
 How to analyze runtime of backward chaining
f(7)

Each node does some “local” computation

to obtain its value from its children’s values
f(6)
(often O(# of children))
(here, f(n) needs O(1) time itself)

Total runtime = total computation at all

f(5) nodes that are visited

(often O(# of edges))

(here, f(n) needs O(n) total time)
…
Memoization is why you only have to
f(4) count each node once! Let’s see …
…
600.325/425 Declarative Methods - J.
Eisner
 How to analyze runtime of backward chaining
f(7)

fmemo(6) fmemo(5)
? ?
f(6)

fmemo(5) fmemo(4)
? ?
f(5)

fmemo(4) fmemo(3)
? …
f(4)
…
600.325/425 Declarative Methods - J.
Eisner
 How to analyze runtime of backward chaining
fmemo(…) is fast. Why? f(7)
Just looks in the memo So only O(1) work within
each box (for Fibonacci)
table & decides whether fmemo(6) fmemo(5)
to call another box
? ?
f(6)

fmemo(5) fmemo(4) Although fmemo(5) gets called twice,

at most one of those calls will
? ? pass the “?” and call the f(5) box
f(5)
So each box gets called only once!
fmemo(4) fmemo(3) So total runtime
? … = # boxes * average runtime per box
f(4)
…
600.325/425 Declarative Methods - J.
Eisner
 How to analyze runtime of backward chaining
f(7)

fmemo(6) fmemo(5)
? ?
f(6)

fmemo(5) fmemo(4)
? ?
Caveat: Tempting to try to divide
f(5) up work this way:

fmemo(4) fmemo(3) How many calls to fmemo(n)?

And how long does each one take?
? …
f(4)
…
600.325/425 Declarative Methods - J.
Eisner
 How to analyze runtime of backward chaining
f(7)

fmemo(6) fmemo(5)
? ?
f(6)
Caveat: Tempting to try to divide
up work this way:
fmemo(5) fmemo(4)
? ? How many calls to fmemo(n)?
And how long does each one take?
f(5)
But hard to figure out how many.
fmemo(4) fmemo(3) And the first one is slower than rest!

? … So instead, our previous trick associates

f(4) runtime of fmemo(n) with its caller
… (green boxes, not brown blobs)

600.325/425 Declarative Methods - J.

Eisner
 Example
• Let us consider that the capacity of the
knapsack is W = 8 and the items are as shown
in the following table.
Item A B C D
Profit 1 2 5 6
Weight 2 3 4 5
 0/1 Knapsack
0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0
2 3 2 0
5 4 3 0
6 5 4 0
 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 1
2 3 2 0
5 4 3 0
6 5 4 0
 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1
2 3 2 0
5 4 3 0
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0
5 4 3 0
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 2
5 4 3 0
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2
5 4 3 0
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 3
5 4 3 0
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3
5 4 3 0
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 5
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 0 1 2 5
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 0 1 2 5 6
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 0 1 2 5 5 6
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 0 1 2 5 5 6 7
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 0 1 2 5 5 6 7 7
6 5 4 0
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 0 1 2 5 5 6 7 7
6 5 4 0 6
c 0 1 2 3 4 5 6 7 8
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 0 1 2 5 5 6 7 7
6 5 4 0 0 1 2 5 6
 c 0 1 2 3 4 5 6 7 8
V [i, w]  max{V [i  1, w],V [i  1, w  w[i ]]  p[i ]}
pi wi 0 0 0 0 0 0 0 0 0 0
1 2 1 0 0 1 1 1 1 1 1 1
2 3 2 0 0 1 2 2 3 3 3 3
5 4 3 0 0 1 2 5 5 6 7 7
6 5 4 0 0 1 4 5 6 6 7 8

This is just the data collection, for that we can use the
following formula:
V [i, w]  max{V [i  1, w],V [i  1, w  w[i ]]  p[i ]}
• Sequence of decisions are as follows:
x1 x2 x3 x4 W=8
1 8-6=2
x1 x2 x3 x4 W=8
0 1 8-6=2

2 is also available row before that

Can say the changes are not because of this
x1 x2 x3 x4 W=8
1 0 1 8-6=2
x1 x2 x3 x4 W=8
0 1 0 1 8-6=2

0 is also available row before that

2 is also available row before that
Can say the changes are not because
Can sayofthe
thischanges
item are not because of this
 Algorithm
• Dynamic-0-1-knapsack (v, w, n, W)
for w = 0 to W do
c[0, w] = 0
for i = 1 to n do
c[i, 0] = 0
for w = 1 to W do
if wi ≤ w then
if vi + c[i-1, w-wi] then
c[i, w] = vi + c[i-1, w-wi]
else c[i, w] = c[i-1, w]
else c[i, w] = c[i-1, w]

This algorithm takes θ(n, w) times as table c has (n + 1).(w + 1)

entries, where each entry requires θ(1) time to compute.
 0/1 Knapsack Example
• Let us consider that the capacity of the
knapsack is W = 25 and the items are as
shown in the following table.
Item A B C D
Profit 24 18 18 10
Weight 24 10 10 7