Knapsack Algorithm

At Qargo we are currently experimenting with different Fintech algorithms and

applicability of quantum computing in resolving them. We are starting this serios of articles
in order to go over some typical problems that can be resolved with quantum computing.
This is the first in a series of articles exploring a number of algorithm design paradigms
used in solving the knapsack problem. Our journey starts with the standard backtracking
algorithm, ends with the quantum computing, and includes everything in between.

First algorithm that we are going to explore is a knapsack algorithm. The knapsack problem
is a problem in combinational optimization and as such is usually used in decision making
problems; investment selections and portfolio selections. So let’s dig into classical way of
implementation of this problem. For our examples we are going to use Python and all the
source code can be found on our Github repository.

Knapsack Problem
In this problem, the input is specified by 2n + 1 positive integers: n item values v 1, v2 , . . . , vn
, n item weights w1, w2, . . . , wn, and a knapsack capacity C. The aim is to find a subset S ⊆ {1,
2, . . . , n} of items with the maximum-possible sum ∑ ❑ vi of values, subject to having total
i∈ S

weight ∑ ❑ wi at most C. In other words, the objective is to make use of a scarce resource
i∈ S
in the most valuable way possible.
The most common variant, and the subject of this series of articles, is the 0-1 knapsack
problem. In this case, each item is either placed in the knapsack in its entirety or discarded.
This 0/1 property is what makes the knapsack problem quite challenging .

Backtracking
Backtracking is an “intelligent” exhaustive search strategy. An “intelligent” brute-force
strategy assumes avoiding repetitions and search misses. Backtracking algorithms acquire
their “intelligence” by combining recursion and iteration.
Backtracking algorithms search for solutions in a systematic way. They build a partial
solution, which could eventually become a valid complete solution. Partial solutions are
built incrementally by adding candidate items. At each step, the algorithm attempts to
extend existing partial solution by adding another item. Thinking recursively, it creates a
tree of partial solutions where each node corresponds to a recursive call. If node c was
created by extending node p, then there is an edge from p to c. The root node corresponds
to the first call to the algorithm, while valid complete solutions are found when reaching
leaf nodes.
Effective use of backtracking assumes efficient analysis of partial solutions. If a partial
solution satisfies the problem constraints, then it could potentially be extended into a valid
complete solution. Otherwise, the algorithm “backtracks” to a previous valid partial
solution and continues exploring a search space. This technique of restricting a depth-first
tree traversals the instant the problem constraints are violated is called pruning. Pruning a
search tree as soon as possible can have huge impact on the algorithm running time.

Backtracking Algorithm
The valid complete solution to the knapsack 0-1 problem is a subset of n distinct items.
Therefore, the starting point is to design an algorithm that generates all subsets of n
distinct elements. Its recursion tree is a binary tree since an item can either be excluded or
included in the valid complete solution.
The algorithm uses list data structure to represent solutions (subsets). Subsets are
represented as a binary list of n zeros and ones. For instance, given the list of items [x, y,
z], the list [1, 0, 1] represents the subset {x, z}. Zeros and ones are the candidate
elements specifying the partial solution. As the algorithm progresses by calling itself
recursively, it evolves the partial solution by including new candidates at every level.
Reaching a base case indicates that the algorithm has generated one of the 2 n possible
subsets.
def knapsack(index, soluton, items):
if index == len(items):
# Base case (Process valid solution)
else:
# Recursive case (Generate candidate)
for k in range(0, 2):
# Include candidate
solution[index] = k
# Extend partial solution
knapsack(index + 1, solution, items)

For the knapsack 0-1 problem, we are only interested in partial solutions which do not
violate the problem constraints. Given a partial solution index, the algorithm either
excludes or includes the corresponding item subject to the constraints of the problem - the
sum of included weights must not exceed the knapsack capacity. If the i-th item is excluded,
then both the remaining capacity and the total value are unchanged. If the i-th item is
included in the knapsack, then the remaining capacity is decreased by the item weight w i,
while the total value is increased by the item value vi. Instead of a list of items, the
algorithm now receives the lists of values v and weights w, as well as the capacity C. With
these parameters it is possible to compute the remaining capacity in the knapsack in n + 1
steps. However, it is more efficient to define an extra parameter holding the remaining
capacity, w_left, since the remaining capacity can be updated in a single step.
def knapsack(i, sol, w_left, v, w, C):
if i == len(sol):
# Process valid solution
else:
# Generate candidate
for k in range(0, 2):
# Check constraints (pruning recursion tree)
if k * w[i] <= w_left:
# Expand partial solution
sol[i] = k
# Update remaining capacity
w_left = w_left - k * w[i]
# Expand partial solution
knapsack(i + 1, sol, w_left, v, w, C)

def knapsack_0_1(v, w, C):

sol = [None] * len(v)
knapsack(0, sol, C, v, w, C)

Since the algorithm solves an optimisation problem, it has to keep track of the best valid
solution (i.e. optimal value). An additional parameter, opt_sol, serves this purpose.
Similarly, opt_val parameter contains the optimum sum of values corresponding to
opt_sol. This parameter is also redundant since it is possible to recalculate its value given
opt_sol. Lastly, for the sake of efficiency another redundant parameter, val, is introduced
so that the partial sum of values can be updated in a single step.
def knapsack(i, sol, val, opt_sol, opt_val, w_left, v, w, C):
if i == len(sol):
# Check if better than current best
if val > opt_val:
# Update optimal value and solution
opt_val = val
for k in range(0, len(sol)):
opt_sol[k] = sol[k]
else:
# Generate candidate
for k in range(0, 2):
# Check constraints (pruning recursion tree)
if k * w[i] <= w_left:
# Expand partial solution
sol[i] = k
# Update remaining capacity
w_left = w_left - k * w[i]
# Update partial value
 val = val + k * v[i]
# Expand partial solution
opt_val = knapsack(i + 1, sol, val, opt_sol, opt_val, w_left,
v, w, C)
return opt_val

def knapsack_0_1(v, w, C):

sol = [None] * len(v)
opt_sol = [None] * len(v)
return knapsack(0, sol, 0, opt_sol, 0, C, v, w, C)

# List of item values

v = [7, 2, 10, 4]
# List of item weights
w = [3, 6, 9, 5]
# Knapsack capacity
C = 15
print(knapsack_0_1(v, w, C))

Regarding the wrapper function knapsack_0_1, index i is initially set to 0, w_left stores the
capacity C, while val and opt_val are initialised to 0 since the knapsack is empty.

Knapsack Problem

About Qargo