Knapsack Problem

Knapsack Problem
• Input: a Knapsack (or a Container) of a
given capacity (in wt.)
• Input: number of items with given Profit
and Weight
• Outcome: Select items to be put in the
Container (within its capacity), ensuring
maximum profit

2
 Knapsack Problem
• Two variations possible:
• Part of an item can be included

• Only whole items can be included

3
 Knapsack Problem
• Two variations possible:
• Part of an item can be included
Fractional Knapsack

• Only whole items can be included

0-1 Knapsack

4
 Fractional Knapsack
• If a whole item cannot be put in the
Knapsack (not enough capacity), a part of
it can be included, to fill in the remaining
capacity of Knapsack
• Other items included in Knapsack are
whole items
• In proportion to actual weight included of
the last item, partial profit is calculated

5
 0-1 Knapsack
• Only whole items to be included, part of an
item not allowed
• Not possible to use the Fractional
Knapsack technique
• Need to look at all combinations of items
to ensure that total profit is maximised

6
 Fibonacci Series
• Very familiar series in Mathematics
• Defined as : F(n) = F(n-1) + F(n-2)
• Recursive Definition, easy to implement as
recursive function

7
 Fibonacci Series
• Very familiar series in Mathematics
• Defined as : F(n) = F(n-1) + F(n-2)
• Recursive Definition, easy to implement as
recursive function
Proc Fib (n)

Begin

If n=0 Return 0 End If

If n=1 Return 1 End If

Return (Fib(n-1)+ Fib(n-2))

End Proc

8
 Fibonacci Series
Fib(7)

Fib(6) Fib(5)

Fib(5) Fib(4) Fib(4) Fib(3)

Fib(3) Fib(3) Fib(3)

• Large number recursive calls are made

• Same value (Fib(3)) computed many times
• Execution time very high for any n (exponential)
9
 Fibonacci Series
Fib(7) F Table

0
Fib(6) Fib(5)
1

2
Fib(5) Fib(4) Fib(4) Fib(3)

Fib(3) Fib(3) Fib(3)

• Introducing a Table F
• Stores intermediate values of series
• Every value need not be recomputed multiple times
10
 Fibonacci Series
• New Algorithm for Fib
Proc N_Fib (n)

Begin

If n=0

Set F[0] := 0

Return F[0]

End If

If n=1

Set F[1] := 1

Return F[1]

End If

If F[n]=0

Set F[n] := N_Fib(n-1)+N_Fib(n-2)

End If

Return F[n]

End Proc

11
Fibonacci Series

• New table
reduces
computation
time
• Time required
for F(n) = ?

12
 Fibonacci Series
• Simpler Algorithm for Fib, using Table F[n]
• Iterative instead of Recursive

Proc Iter_Fib (n)

Begin

Set F[0] := 0

Set F[1] := 1

End If

Repeat for I:=2 to N Step 1

Set F[I] := F[I-1]+F[I-2]

End IF

End Proc

13
 0-1 Knapsack
• Given an input of n items, calculations of
all combinations will require 2n steps
• To reduce computational complexity, we
will use a temporary table for storing
intermediate values
• Reduces exponential complexity to
Polynomial

14
 0-1 Knapsack
Item List
Items Weight Profit
I1 2 3
I2 3 4
I3 4 5
I4 5 7

Knapsack Capacity: 7

15
 0-1 Knapsack
Table for storing item combinations and profits

0 1 2 3 4 5 6 7
0
I1
I2
I3
I4

16
 0-1 Knapsack
0 1 2 3 4 5 6 7
0
I1
I2
I3
I4

• A Table of 5 rows and 8 columns (for given example)

• Rows are Items and Columns are capacity of Knapsack (0 to
max)
• Each cell represents Profit of all elements entered into
Knapsack
17
 0-1 Knapsack
0 1 2 3 4 5 6 7
0 0 0 0 0 0 0 0 0
I1 0
I2 0
I3 0
I4 0

• Initialization of Table
• First Row and First Column initialized to 0
• First Row indicates zero profit when no items are added
• First Column indicates zero profit when Knapsack capacity
is 0
18
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0
I3 0
I4 0

• Adding I1 – Weight: 2, Profit: 3

• When Knapsack Capacity is 1, I1 cannot be added, Profit=0
• At Capacity=2, I1 is added to Knapsack and Profit = 3
• For other entries in the same row, only having I1, Profit
remains 3 even if Knapsack Capacity increases to 7
19
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3
I3 0
I4 0

• Adding I2 – Weight: 3, Profit: 4

• When Capacity < 3, I2 cannot be added, Profit remains same
• At Capacity=3, I2 can be added to Knapsack
• At Capacity=3, either I1 or I2 can be included in Knapsack,
but not both
20
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3 4 4
I3 0
I4 0

• Adding I2 – Weight: 3, Profit: 4

• As Profit of I2 is more than I1, I2 is included and I1 is removed
from Knapsack
• Hence, Profit = 4
• Profit remains same at Capacity = 4
21
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3 4 4 7 7 7
I3 0
I4 0

• Adding I2 – Weight: 3, Profit: 4

• When Capacity = 5, both I1 and I2 can be included in
Knapsack
• Hence, Profit = 3 + 4 = 7
• Profit remains same when Capacity > 5
22
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3 4 4 7 7 7
I3 0 0 3 4 5
I4 0

• Adding I3 – Weight: 4, Profit: 5

• Till Capacity <4, Profit remains same as above
• At Capacity = 4, I3 replaces I2 (more profit), profit = 5
• At Capacity = 5, profit = ?

23
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3 4 4 7 7 7
I3 0 0 3 4 5 7
I4 0

• Adding I3 – Weight: 4, Profit: 5

• At Capacity = 5, either (I1 + I2) can be included in Knapsack,
or, only I3
• As Profit of (I1 + I2) exceeds profit of I3, hence (I1 + I2)
included and I3 taken out, Profit = 7
24
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3 4 4 7 7 7
I3 0 0 3 4 5 7 8
I4 0

• Adding I3 – Weight: 4, Profit: 5

• At Capacity = 6, (I1 + I3) can be included in Knapsack, which
would have more profit (8) than profit of (I1+I2)
• Hence, I2 is taken out and I3 is included along with I1
• Profit = 8
25
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3 4 4 7 7 7
I3 0 0 3 4 5 7 8 9
I4 0

• Adding I3 – Weight: 4, Profit: 5

• At Capacity = 7, (I2 + I3) can be included in Knapsack, which
would have more profit (9) than profit of (I1+I3)
• Hence, I1 is taken out and I2 included in Knapsack
• Thus, at Capacity = 7, profit is (4+5) or 9
26
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3 4 4 7 7 7
I3 0 0 3 4 5 7 8 9
I4 0

• Adding I4 – Weight: 5, Profit: 7

• Calculate Profit …

27
 0-1 Knapsack
0 1 2 3 4 5 6 7 I1 2 3
I2 3 4
0 0 0 0 0 0 0 0 0
I3 4 5
I1 0 0 3 3 3 3 3 3 I4 5 7
I2 0 0 3 4 4 7 7 7
I3 0 0 3 4 5 7 8 9
I4 0

• Adding I4 – Weight: 5, Profit: 7

• Calculate Profit …

• Final Profit will be available at last cell

28
 0-1 Knapsack Algorithm (part)
Input: N elements with Weights[1..N] and Profit[1..N], Knapsack Capacity: C

Associated Structure: B[0..N, 0..C], for calculation of Profit

Initialization: B[0, 0..C], B [0..N,0] both set to Zero

Repeat for I := 1 to N step 1

Repeat for J := 1 to C step 1
If Weight[I] is <= J
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
Else
Set B[I,J]:= B[I-1,J]
End If
Else
Set B[I,J]:= B[I-1,J]
End If
End Repeat
End Repeat 29
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0

Repeat for I := 1 to N step 1

Repeat for J := 1 to C step 1
If Weight[I] is <= J
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
Else
Set B[I,J]:= B[I-1,J]
End If
Else
Set B[I,J]:= B[I-1,J]
End If
End Repeat
End Repeat
30
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0 0

Repeat for I := 1 to N step 1 I=4,N=4

Repeat for J := 1 to C step 1 J=1

If Weight[I] is <= J (Weight[4])=5<=1?
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
Else
Set B[I,J]:= B[I-1,J]
End If
Else
Set B[I,J]:= B[I-1,J] B[4,1]=B[3,1]=0
End If
End Repeat
End Repeat
31
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0 0 3

Repeat for I := 1 to N step 1 I=4,N=4

Repeat for J := 1 to C step 1 J=2

If Weight[I] is <= J 5<=2?
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
Else
Set B[I,J]:= B[I-1,J]
End If
Else
Set B[I,J]:= B[I-1,J] B[4,2]=B[3,2]=3
End If
End Repeat
End Repeat
32
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0 0 3 4

Repeat for I := 1 to N step 1 I=4,N=4

Repeat for J := 1 to C step 1 J=3

If Weight[I] is <= J 5<=3?
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
Else
Set B[I,J]:= B[I-1,J]
End If
Else
Set B[I,J]:= B[I-1,J] B[4,3]=B[3,3]=4
End If
End Repeat
End Repeat
33
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0 0 3 4 5

Repeat for I := 1 to N step 1 I=4,N=4

Repeat for J := 1 to C step 1 J=4

If Weight[I] is <= J 5<=4?
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
Else
Set B[I,J]:= B[I-1,J]
End If
Else
Set B[I,J]:= B[I-1,J] B[4,4]=B[3,4]=5
End If
End Repeat
End Repeat
34
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0 0 3 4 5 7

Repeat for I := 1 to N step 1 I=4,N=4

Repeat for J := 1 to C step 1 J=5

If Weight[I] is <= J
5<=5?
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
(Profit[4]=7 + B[3,0(5-5)]=0)=7 > B[3,5]=7 ?
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
Else
Set B[I,J]:= B[I-1,J] B[4,5]=B[3,5]=7
End If
Else
Set B[I,J]:= B[I-1,J]
End If
End Repeat 35

End Repeat
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0 0 3 4 5 7 8

Repeat for I := 1 to N step 1 I=4,N=4

Repeat for J := 1 to C step 1 J=6

If Weight[I] is <= J
5<=6? If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
(Profit[4]=7 + B[3,1(6-5)]=0)=7 > B[3,6]=8 ?
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
Else
Set B[I,J]:= B[I-1,J] B[4,5]=B[3,5]=8
End If
Else
Set B[I,J]:= B[I-1,J]
End If
End Repeat
End Repeat 36
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0 0 3 4 5 7 8 10

Repeat for I := 1 to N step 1 I=4,N=4

Repeat for J := 1 to C step 1 J=7

If Weight[I] is <= J 5<7?
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
(Profit[4]=7 + B[3,2(7-5)]=3)=10 > B[3,7]=9 ?
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
B[4,7] = 7 + 3 = 10
Else
Set B[I,J]:= B[I-1,J]
End If
Else
Set B[I,J]:= B[I-1,J]
End If
37
End Repeat
End Repeat
 0-1 Knapsack
I\J 0 1 2 3 4 5 6 7 I1 2 3
0 0 0 0 0 0 0 0 0 I2 3 4
I1 0 0 3 3 3 3 3 3 I3 4 5
I2 0 0 3 4 4 7 7 7 I4 5 7

I3 0 0 3 4 5 7 8 9
I4 0 0 3 4 5 7 8 10

Repeat for I := 1 to N step 1 I=3,N=4

Repeat for J := 1 to C step 1 J=7

If Weight[I] is <= J (Weight[3])=4<7?
If(Profit[I] + B[I–1,J–Weight[I]]) > B[I–1,J])
(Profit[3]=5 + B[2,3(7-4)]=4)=9 > B[2,7]=7 ?
Set B[I,J]:= Profit[I]+B[I-1,J-Weight[I]]
B[3,7] = 5 + 4 = 9
Else
Set B[I,J]:= B[I-1,J]
End If
Else
Set B[I,J]:= B[I-1,J] Calculations
End If for B[3,7]
38
End Repeat
End Repeat
The End