A Comparision Between Solve 0/1 Knapsack

Problem Using Dynamic ,Greedy and Genetic

Algorithm
Marwa Alabdali m abdali84@yahoo.com
Aysha Alshareef lamita2004h@yahoo.com
ALBalqa Applied University
May 7, 2010

Abstract
During the last decades solving problem to accomplish the best opti-
mization using different algorithms has attracted the attention of many
researchers. In this paper we describe a research on solving 0/1 knapsack
problem using different algorithm .The Knapsack Problem is an exam-
ple of optimization problem,which seek the maximum object in knapsack
without exceeding its capacity,is defined to be is recognized to be NP-hard.
This paper describe aresearch project on using differant algorithm to
solve the 0-1 knapsack problem,Subsequently comparisons are made with
a these algorithm to decide which one is best in solving 0/1 knapsack
problem .
In this paper we use the same computational example to solve this prob-
lem using differant algorithem the output shows the behaviour of each
algorithm in the same problem space.

1 Introduction
This paper proposes different algorithms to solve 0/1 knapsack problem,we use
different Algorithms to solve the problem since we want to focus on how each
affects search behavior in the same problem space.The goals of the paper are to
provide additional insights into how each algorithm works,and to decide which
algorithm gives best performance in problem like 0/1knapsack problem.
The paper contains brief description of the basic idea and elements of the 0/1
Knapsack Problem ,implementation of the 0-1 Knapsack Problem using genetic
algorithm greedy and dynamic algorithm, finally computational results shows
the behaviour of each algorithm in the same problem space.

1
 Genetic algorithm focus on probabilistic and stochastic basis to search through
a space of potential solutions using randomness as a major factor to make de-
cisions,since they can provide solutions to problems where often standard al-
gorithms have failed. In this paper we implemented Roulette-wheel selection
as a selection functions. The results from both of them differed depending on
whether we used elitism or not. Elitism significantly improved the performance
of the roulette-wheel function. Moreover, we tested the program with different
crossover ratios and single and double crossover points but the results given
were not that different.

Dynamic programming is a bottom-up technique focus on avoid calculating

the same thing twice, usually by keeping a table of known results, which we fill
up as sub instances are solved.
Greedy algorithm is an Algorithms that solves optimization problems basi-
cally by making locally optimal choices in the hope that these choices will create
a globally optimal solutions,it always makes the choice that looks best at the
moment,it does not always yield optimal solutions, but for many problems it
do.Greedy strategy is a top-down fashion strategy.
The list of this articleis programize as folow;The Knapsack Problem Defini-
tion ,Implementation of the 0-1 KP Using genetic algorithm ,Implementation
of the 0-1 KP Using dynamic algorithm,Implementation of the 0-1 KP Using
Greedy algorithm,comparision between differant strategy and result and con-
clusion.

2 The Knapsack Problem Definition

The KP problem is an example of optimization problem, which seeks for a best
solution from among many other solutions.It is concerned with a knapsack that
has positive integer volume (or capacity)V. There are n distinct items that may
potentially be placed in the knapsack. Item i has a positive integer volume Vi
and positive integer benefit Bi[*],in addition there are Qi copies of item i avail-
able, where quantity Qi is a positive integer satisfying 1 ≤ Qi ≤ ∞.
LetXidetermineshowmanycopiesof itemiaretobeplacedintotheknapsack.
T hegoalistomaximize
Xn
BiXi
i=1
Subjecttotheconstraints
Xn
Vi Xi .And0 ≤ Xi ≤ Qi
i=1

If one or more of the Qi is infinite,the KP is unbounded; otherwise, the KP

is bounded.The bounded KP can be either 0-1 KP or Multiconstraint KP. If Qi
= 1 for i = 1, 2,, N, the problem is a 0-1 knapsack problem In the current paper,
we have worked on the bounded 0-1 KP, where we cannot have more than one
copy of an item in the knapsack[*].

2
2.1 Example of a 0-1 Knapsack
In this paper we solved a given knapsack problem in different algorithem ,given
some items, pack the knapsack to get the maximum total value,each item has
some weight and some value,and the total knapsack capacity is 10.In order to
find the best solution we have to identify a subset that meets the constraint and
has the maximum total benefit.

Item Weight Value

A 5 10
B 4 40
C 6 30
D 3 50

3 Implementation
3.1 Implementation of the 0-1 KP Using Genetic Algo-
rithm
Genetic algorithm are among search procedures based on natural selection and
natural genetics,they randomly create an initial population of individuals. Then
they use genetic operators to yield new offspring[*].
For this problem there are 24 possiblesubsetsof items :
A B C D weight of set benefit of set subset meets constraint
0 0 0 0 0 0 reject
0 0 0 1 3 50 reject
0 0 1 0 6 30 reject
0 0 1 1 9 80 accept
0 1 0 0 4 40 reject
0 1 0 1 7 90 accept
0 1 1 0 10 70 accept
0 1 1 1 15 120 reject
1 0 0 0 5 10 accept
1 0 0 1 8 60 accept
1 0 1 0 11 40 reject
1 0 1 1 14 90 reject
1 1 0 0 9 50 accept
1 1 0 1 12 100 reject
1 1 1 1 18 130 reject
subsetmeetsconstrain : meanmeetknapsackcapacity.

3
3.2 Representation Of Items And Encoding Of The Chro-
mosomes
We use a data structure, called cell, with two fields (Value and weight) to rep-
resent every item[*]. Then we use an array of type cell to store all items in it,
which looks as follows:

weight value weight value weight value weight value

5 10 4 40 6 30 3 50
0 1 2 3
The chromosomes in GAs represent the space of candidate solutions can be
represented in an array having size equal to the number of the items (in our ex-
ample of size 4),using binary encoding and represent each chromosome as string
of bits Each element from this array denotes whether an item is included in the
knapsack (1) or not (0).In order to find the best solution we have to identify a
subset that meets the constraint and has the maximum total benefit.

3.3 Main Operation

Genetic algorithem perform the follwing basic operations:
1. Initialize array items whitch contained (value and weight) for each item.
2. Initialize the first population by randomly generating a population of Size
chromosomes.
p1:{ 0,0,1,1 }
p2:{ 0,1,0,1 }
p3:{ 0,1,1,0 }
p4:{ 1,0,0,1 }
p5:{ 1,1,0,0 }

3. Calculate the fitness and weight of all chromosomes.

A B C D weight of set benefit of set
0 0 1 1 9 80
0 1 0 1 7 90
0 1 1 0 10 70
1 0 0 1 8 60
1 1 0 0 9 50
4. Check what percentage of the chromosomes in the population has the same
fitness value,here the persantage is 40%
5. Does chromosome have the same fitness value?
If not randomly select 2 chromosomes from the population,using roulette-
wheel selection to select two chromosome.

4
 A B C D weight of set benefit of set
0 0 1 1 9 80
0 1 0 1 7 90
0 1 1 0 10 70
1 0 0 1 8 60
1 1 0 0 9 50
If yes select the chromosome with the most benefit in this example we
selecte chromosome with greater benefit.

A B C D weight of set benefit of set

0 0 1 1 9 80
1 1 0 0 9 50
We select {0,0,1,1} with benefit of 80$
(a) Perform crossover on the 2 chromosomes selected,suppose the crossover
probability is 0.80 all 4 chromosomes are selected for crossover ,the
selected chromosomes are paired up randomly,a crossover point is
randomly chosen for each crossover.Suppose the 1st digit in the first
pair is chosen as the crossover point.
v1’ = (0 1,0,1)
v2’ = (0 0,1,1)
After crossover, we will have:
v1” = (0 0,1,1) this chromosome still meet constrain
v2” = (0 1,0,1) this chromosome still meet constrain
In this process no changing in chromosome happened, so we do the
crossover process to other chromosome
v3’ = (1 0,0,1)
v4’ = (0 1,1,0)
After crossover we will have:
v3” = (1 1,1,0) this chromosome does not meet constrain since totat weight=15
v4” = (0 0,0,1) this chromosome still meet constrain , but the overall benefit is too small

(b) Perform mutation on the chromosomes obtained.

6. Does 40% have the same fit value? and Is the number of generations greater
than the limit?
If not go to step 3 If yes exit.

3.4 Termination conditions

In this problem termination happened when reach to the best solution by repeat-
ing the genetic algorithem operation many time to achieved the best reslt,here in
our example the best capacity of the knapsack is represent by this chromosome
(0,1,0,1) is equal to 7 which accomplish the best benefit equal to 90%

5
3.5 Using Roulette-Wheel Selection As Selection Func-
tion
In this paper we use Roulette-wheel combined with elitism,roulette-wheel is a
simple method of implementing fitness-proportionate selection it is conceptually
equal to giving each individual a slice of a circular roulette wheel equal in area
to the individuals fitness.The wheel is spun N times, where N is the number
of the individuals in the population (in our case N = Size).On each spin, the
individual under wheels marker is selected to be in the pool of parents for the
next generation [*].

This method can be implemented in the following way

1. Sum the total expected fitness (benefit) of all individuals in the population.
in our example,the total sum of fitness values is = 80+90+70+60 = 300 so
the probability of each chromosome to be selected into the next generation
is as follows:
prob(v1) = 80/300= 27%
prob(v2) = 90/300=30%
prob(v3) = 70/300=23%
prob(v4) = 60/300=20%

2. Next arrange these probabilities into different ranges from 0 to 1 :

v1 : 0.00 to 0.27
v2 : 0.27 to 0.57
v3 : 0.57 to 0.80
v4 : 0.80 to 1.00

3. Choose a random integer (limit) between 0 and Size,suppose we have the

following random numbers: 0.4007 ,0.2210 ,0.8823 ,0.5954

The following individuals will be chosen:

0.4007 v2 (0,1,0,1) v1’
0.2210 v1 (0,0,1,1) v2’
0.8823 v4 (1,0,0,1) v3’
0.5954 v3 (0,1,1,0) v4’

4 Implementation of the 0-1 KP Using Dynamic

Algorithm
Dynamic programming is a method for solving optimization problems,where
we want to find the best way of doing something is a recursive approach that
involves breaking a global problem down into more local subproblems,the idea
is to compute the solutions to the subsub-problems once and store the solutions
in a table, so that they can be reused (repeatedly) later[*].

6
4.1 Main Operation
Step1: Structure: Characterize the structure of an optimal solution by decom-
pose the problem into smaller problems,and find a relation between the structure
of the optimal solution of the original problem and the solutions of the smaller
problems.
Step2:Principle of Optimality: Recursively define the value of an optimal so-
lution;express the solution of the original problem in terms of optimal solutions
for smaller problems.
Step 3:Bottom-up computation: Compute the value of an optimal solution
in a bottom-up fashion by using a table structure,using iteration not recursion.
Bottom: V[0,w]=0 0≤ w ≤ W
Bottomupcomputationmeancomputingthetableusing
V [i, w] = max(V [i − 1, w], vi + V [i − 1, W − wi])
rowbyrow.
Step4 : Construction of optimal solution:Constructoptimalsolutionf romcomputedinf ormation.
Steps 3 and 4 may often be combined.

In dynamic programming we define the initial Settings to be:

set V[0,w] = 0 for 0≤ w ≤ W

V [i, w] = ∞f orw ≤ 0
Step5 : Recursive Step:U se

V [i, w] = max(V [i − 1, w], vi + V [i − 1, W − wi])f orall1 ≤ i ≤ n, 0 ≤ w ≤ W.

v[i,w] 0 1 2 3 4 5 6 7 8 9 10
i=0 0 0 0 0 0 10 10 10 10 10 10
1 0 0 0 0 40 40 40 40 40 50 50
2 0 0 0 0 40 40 40 40 40 50 70
2 0 0 0 50 50 50 50 90 90 90 90
T hef inaloutputisv[4, 10] = 90

4.2 Dynamic Programming Algorithm

KnapSack ( v , w , n , W )
{
for ( w = 0 to W ) V [ 0 , w ] = 0;
for ( i = 1 to n )
for( w = 0 to W )
if ( wi≤ w)
V [i, w] = max(V [i − 1, w], vi + V [i − 1, W − wi])
else
V [i, w] = V [i − 1, w];
returnV [n, W ]

7
}
T imecomplexity : Clearly, O(nW )