0/1 Knapsack problem using Dynamic

programming.

Prepared by: Bushra Tufail

Presented to: Miss Sadaf Yasmeen

 What is dynamic programming?

• Dynamic programming is a method for solving

optimization problems.

The idea: Compute the solutions to the sub-

problems once and store the solutions in a table,
so that they can be reused (repeatedly) later.

• It is a Bottom-Up approach.
Applications of Dynamic Programming
Approach
• Matrix Chain Multiplication
• Longest Common Subsequence
• Travelling Salesman Problem
• Knapsack Problem
• Why dynamic problem?

Its about knowing the sub-

problem and optimizing the
solution of problem

• Why 0/1?
we can either take an item or
reject.
We are going to look for all
subsets that has time
complexity in exponential O(2
exp n) while dynamic
programming reduces it to
O(n*m)
 The 0-1 Knapsack Problem
(Rucksack Problem)
• Given: A set S of n items, with each item i having
– wi - a positive weight
– bi - a positive benefit (value)

• Goal: Choose items with maximum total benefit but

with weight at most W (weight of Knapsack).
EXAMPLE:
• Capacity 200
• Items :
X1 : v1/w1 = 12, weight : 50
X2 : v2/w2 = 10, weight :
55 X3 : v3/w3 = 8, weight :
10 X4 : v4/w4 = 6, weight :
100

To solve 0-1 Knapsack, Dynamic Programming approach is

required.
 Recursive formula for sub-problems:

3 cases:
1. There are no items in the knapsack, or the weight of the knapsack is 0 -
the benefit is 0
2. The weight of itemi exceeds the weight w of the knapsack - itemi cannot be
included in the knapsack and the maximum benefit is V[i-1, w]
3. Otherwise, the benefit is the maximum achieved by either not including
itemi ( i.e., V[i-1, w]),
or by including itemi (i.e., V[i-1, w-wi]+bi)

if wi <= w // item i can be part of the solution

if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w] // maximum value
 0-1 Knapsack Algorithm
for w = 0 to W // Initialize 1st row to
0’s
i = 1 to=n0 // Initialize 1st column to 0’s
for V[0,w]
V[i,0] = 0
for i = 1 ton
for w = 0
to W
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Example

Let’s run our algorithm on the following

data:

n = 4 (# of elements) W = 5
(max weight)
Elements (weight, benefit): (2,3),
(3,4), (4,5), (5,6)
 Example (1)

i\W 0 1 2 3 4 5
0
1
2
3
4
 Example (2)

i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1
2
3
4

for w = 0 to W
V[0,w] = 0
 Example (3)

i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0

for i = 1 to n
V[i,0] = 0
 Items:
Example (4) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0
wi=2
2 0
3 0
w-w
w=1i =-1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (5) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3
wi=2
2 0
3 0
w-w
w=2i =0
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (6) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3
wi=2
2 0
3 0
w-w
w=3i =1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (7) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3 3
wi=2
2 0
3 0
w-w
w=4i =2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (8) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3 3 3
wi=2
2 0
3 0
w-w
w=5i =3
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (9) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0
3 0
w-w
w=1i =-2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (10) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3
3 0
w-w
w=2i =-1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (11) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4
3 0
w-w
w=3i =0
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (12) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4
3 0
w-w
w=4i =1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (13) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4 7
3 0
w-w
w=5i =2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (14) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4
w= 1..3
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (15) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
0 0 3 4 5
w=
3
0 4
4
w-
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
wi=0
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (16) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
0 0 3 4 5 7
w=
3
0 5
4
w-
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
wi=1
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (17) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7
w= 1..4
4 0 0 3 4 5
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (18) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
0 0 3 4 5 7
w=
3
0 0 3 4 5 7 5
4
w-
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
wi=0
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
We’re DONE!!
The max possible value that can be carried
in this knapsack is $7
• This algorithm only finds the max possible
value that can be carried in the knapsack
– i.e., the value in V[n,W]
• To know the items that make this maximum
value, we need to trace back through the
table.
 Items:
Finding the Items 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7
3 0 0 3 4 5 7 wi=5
4 0 0 3 4 5 7 V[i,k
]=7
V[i
1,k]
=7
 Items:
Finding the Items (2) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7
3 0 0 3 4 5 7 wi=5
4 0 0 3 4 5 7 V[i,k
]=7
V[i
1,k]
=7
 Items:
Finding the Items (3) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 wi=4
4 0 0 3 4 5 7 V[i,k
]=7
V[i
1,k]
=7
 Items:
Finding the Items (4) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 7 wi=3
4 0 0 3 4 5 7 V[i,k
]=7
V[i1,k] =3
k  wi=2
 Items:
Finding the Items (5) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 k= 2
1 0 0 3 3 3 3 bi=3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 wi=2
4 0 0 3 4 5 7 V[i,k
]=3
V[i1,k] =0
k  wi=0
 Items:
Finding the Items (6) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=0 4: (5,6)
0 0 0 0 0 0 0 k= 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
0 0 3 4 5 7 knapsack
4
should contain
{1, 2}
 Items:
Finding the Items (7) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 4: (5,6)
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
4 0 0 3 4 5 7 knapsack
should contain
i=n, k=W {1, 2}
while i,k > 0
if V[i,k]

V[i1,k
] then
mar
 Running time
for w = 0 to W
O(W)
V[0,w] = 0
for i = 1 to n
V[i,0] =
0 Repeat n times
for i = 1 to n O(W)
for w = <0 the
to rest of the code >
W
What is the running time of this
algorithm?
O(n*W)
 Applications of Knapsack Problem
• Resourse allocation with financial constraints
• Construction and Scoring of Heterogenous
test
• Selection of capital investments
ANY QUESTIONS ???
THANK YOU