Introduction to Genetic Algorithm

Solutions to GA are represented by data structures, often a fixed length vector,

called chromosomes. Chromosomes are composed of genes which have value called
alleles, and the position of the gene in the chromosome is called locus (Goldberg,
1989). Chromosomes have been represented by vectors, strings, arrays, and tables.
Alleles have been represented by a simple binary system of zeroes and ones, integers,
real number, and letter of alphabet, or other symbols (Gen and Cheng, 1997). Each
chromosome has a fitness value which is indicative of the suitability of the
chromosome as a solution to the problem (Goldberg, 1989).

A simple genetic algorithm can be summarized as follows.

1. Representation: Encode the characteristics of each individual in the initial
population as a chromosome (typically, a chromosome is a bit string). Set the
current population to this initial population.
2. Evaluation: Evaluate fitness of each chromosome.
3. Selection: Select chromosomes of parent P from the current population via
proportional selection, a chromosome with high fitness is more likely to be
selected.
4. Recombination: Choose at random a pair of parents for mating. Generate two
offspring from the two parents by exchanging sub strings between parent
chromosomes (crossover).
5. Mutation: process each offspring by the mutation operator, and insert the
resulting offspring in the new population.
6. Repeat steps (4) and (5), until P offspring are created.
7. Replace the old population of chromosomes by the new one.
8. Evaluate the fitness of each chromosome in the new population.
9. Go back to Step 3 if the number of generations is less than a fixed number of
generation or until convergence to a population of similar individuals is obtained.
Then, the best chromosome generated during the search is decoded into the
corresponding individual (Potvin, 1996). The procedure of GA can be described in
Figure 1 (Gen and Cheng, 1997).

Irhamah Introduction to Genetic Algorithm – page 1

 Procedure: Genetic Algorithms
begin
tÅ 0
initialize P(t);
evaluate P(t);
while (not termination condition) do
recombine P(t) to yield C(t);
evaluate C(t);
select P(t + 1) from P(t) and C(t);
t Å t + 1;
end
end

Figure 1 : Procedure of genetic algorithm

1 Selection Methods

In selection process, chromosomes are selected from the population to be

parents for crossover. According to Darwin's theory of evolution the best ones
survive to create new offspring. There are many methods in selecting the best
chromosomes. Some of them will be described in following (Obitko, 1998).

a. Roulette Wheel Selection

Parents are selected according to their fitness. The better the chromosomes are, the
more chances for it to be selected. Imagine a roulette wheel where all the
chromosomes in the population are placed. The size of the section in the roulette
wheel is proportional to the value of the fitness function of every chromosome: the
bigger the value is, the larger the section is. For an example, see Figure 2.

Figure 2 : Illustration of roulette wheel selection

Irhamah Introduction to Genetic Algorithm – page 2

A marble is thrown in the roulette wheel and the chromosome where it stops is
selected. Clearly, the chromosomes with bigger fitness value will be selected more
times. This process can be described by the following algorithm.
1. [Sum] Calculate the sum of the fitness values of all chromosomes in the
population - sum S.
2. [Select] Generate a random number between 0 and the sum of the fitness
value.
3. [Loop] Select the chromosome whose fitness value added to the sum of the
fitness values of the previous chromosomes is greater or equal to the random.
(Obitko, 1998)

b. Rank Selection
The previous type of selection will have problems when they are big differences
between the fitness values. For example, if the best chromosome fitness is 90% of
the sum of all fitnesses then the other chromosomes will have very few chances to be
selected. Rank selection ranks the population first and then every chromosome
receives fitness value determined by this ranking. The worst will have the fitness 1,
the second worst will have the fitness 2 etc. and the best will have the fitness PopSize
(number of chromosomes in population). The following picture describes how the
situation changes after changing fitness to the numbers determined by the ranking.

Figure 3 : Situation before ranking (graph of fitnesses)

Figure 4 : Situation after ranking (graph of order numbers)

Irhamah Introduction to Genetic Algorithm – page 3

Now all the chromosomes have a chance to be selected. However this method can
lead to slower convergence, because the best chromosomes do not differ so much
from other ones (Obitko, 1998).

c. Steady-State Selection
This is not a particular method of selecting parents. The main idea of this type of
selection is that a big part of chromosomes can survive to next generation. The
steady-state selection GA works in the following way. In every generation a few
good (with higher fitness) chromosomes are selected for creating new offspring.
Then some bad (with lower fitness) chromosomes are removed and the new offspring
is placed in their place. The rest of population survives to new generation (Obitko,
1998).

d. Elitism
When creating a new population by crossover and mutation, we have a big chance,
that we will loose the best chromosome. Elitism is the name of the method that first
copies the best chromosome (or few best chromosomes) to the new population. The
rest of the population is constructed in ways described above. Elitism can rapidly
increase the performance of GA, because it prevents a loss of the best found solution
(Obitko, 1998).

2 Crossover

The simplest type of crossover is one point crossover. As illustrated in Loft

and Snow (2006) where chromosomes are strings of symbols (here, 0’s and 1’s),
there are two chromosomes; one from Parent A and one from Parent B, both the of
same length. A crossover point is selected randomly. The two chromosomes are cut
at this point, and new chromosomes are formed by using the chromosome from
Parent A before the crossover point and from Parent B after the crossover point and
vice versa. This is depicted in Figure 5.

Irhamah Introduction to Genetic Algorithm – page 4

 Parent A: 0 1 1 0 1 0 1 1 segment of A

Parent B: 1 0 1 1 1 0 0 0 segment of B

Offspring 1 0 1 1 1 1 0 0 0

Offspring 2 1 0 1 0 1 0 1 1
Figure 5 : One point crossover

A first generalization of one point crossover is two point crossover. In two

point crossover, two crossover points are randomly chosen. Genetic material is
copied from A before the first crossover point. Between the crossover points,
material is taken from Parent B. After the second crossover point, material is again
taken from A. This is depicted in Figure 6.

initial and final

Parent A: 0 1 1 0 1 0 1 1 segments of A

Parent A: 1 0 1 1 1 0 0 0 middle segment of B

Offspring 0 1 1 1 1 0 1 1
Figure 6 : Two point crossover

From two point crossover, one can imagine three point crossover, four point,
five point, and so on. The logical conclusion of this is uniform crossover. In
uniform crossover, each symbol in the offspring’s chromosome is chosen randomly
to be equal to the corresponding symbol of the chromosome of either Parent A or
Parent B as shown in Figure 7.

Parent A: 0 1 1 0 1 0 1 1 (the underlined symbols are

the ones chosen for the
Parent A: 1 0 1 1 1 0 0 0 offspring)

Offspring 0 0 1 0 1 0 0 1
Figure 7 : Uniform crossover

Irhamah Introduction to Genetic Algorithm – page 5

 Loft and Snow (2006) observed that if a pattern appears in the chromosomes
of both parents, then recombination will preserve that pattern. For example, both
parents above have 1’s as the third and fifth symbol in their chromosomes and 0 as
the sixth then in one point, two point, and uniform crossover the offspring has the
same pattern. Other crossovers that usually used for TSP and VRP are Partially
Match Crossover (PMX), Cycle Crossover (CX), Order Crossover (OX), Modified
Crossover (MOX), Order-Based Crosover (OBX) and Position-Based Crossover
(PBX) (Potvin, 1996). They can be described in the next sub sections.

2.1 Partially-Mapped Crossover (PMX)

For PMX operator, firstly we randomly select two cut points on both parents.
In order to create offspring, the substring between the two cut points in the first
parent replaces the corresponding substring in the second parent. Then, the inverse
replacement is applied outside of the cut points, in order to eliminate duplicates and
recover all cities. In Figure 8, the offspring is created by first replacing the subset {2,
3, 6} in parent 2 by the subset {5, 6, 4}. Hence, city 5 replaces city 2, city 6 replaces
city 3, and city 4 replaces city 6 (step 1). Since cities 4 and 6 are now duplicated in
the offspring, the inverse replacement is applied outside of the cut points. Namely,
city 2 replaces city 5, and city 3 replaces city 4 (step 2). In the latter case, city 6 first
replaces city 4, but since city 6 is already found in the offspring at position 4, city 3
finally replaces city 6. Multiple replacements at a given position occur when a city is
located between the two cut points on both parents, like city 6 in this example.

Parent 1 : 1 2 │ 5 6 4 │ 3 8 7
Parent 2 : 1 4 │ 2 3 6 │ 5 7 8
Offspring:
Step 1 : 1 4 │ 5 6 4 │ 5 7 8
Step 2 : 1 3 │ 5 6 4 │ 2 7 8

Figure 8 : Partially-mapped crossover

Irhamah Introduction to Genetic Algorithm – page 6

2.2 Cycle Crossover (CX)

The cycle crossover focuses on subsets of cities that occupy the same subset
of positions in both parents. Then, these cities are copied from the first parent to the
offspring (at the same positions), and the remaining positions are filled with the cities
of the second parent. In this way, the position of each city is inherited from one of
the two parents. However, many edges can be broken in the process, because the
initial subset of cities is not necessarily located at consecutive positions in the parent
tours. In Figure 9, the subset of cities {3, 4, 6} occupies the subset of positions {2, 4 ,
5} in both parents. Hence, an offspring is created by filling the positions 2, 4 and 5
with the cities found in parent 1, and by filling the remaining positions with the cities
found in parent 2.

Parent 1 : 1 3 5 6 4 2 8 7
Parent 2 : 1 4 2 3 6 5 7 8
Offspring: 1 3 2 6 4 5 7 8
Figure 9 : Cycle crossover

2.3 Modified Crossover

This crossover was proposed by Davis in 1985 as an extension of the one-

point crossover for permutation problems. A cut position is chosen at random on the
first parent chromosome. Then, an offspring is created by appending the second
parent chromosome to the initial part of the first parent (before the cut point), and by
eliminating the duplicates. An example is provided in Figure 10. The two parent
chromosomes are cut in the position between second and third positions. An
offspring is formed by using the chromosome of Parent 1 before the cut point, that is
the subset {1, 2} and from second parent (after cut position) {2, 3, 6, 5, 7, 8}, since
the city 2 is already on the offspring, the city 2 is discarded and replaced by 4.

Parent 1 : 1 2 │ 5 6 4 3 8 7
Parent 2 : 1 4 │ 2 3 6 5 7 8
Offspring: 1 2 4 3 6 5 7 8
Figure 10 : Modified crossover

Irhamah Introduction to Genetic Algorithm – page 7

2.4 Order Crossover (OX)

This crossover extends the modified crossover of Davis (1985) by allowing

two cut points to be randomly chosen from the parent chromosomes. In order to
create an offspring, the string between the two cut points in the first parent is first
copied to the offspring. Then, the remaining positions are filled by considering the
sequence of cities in the second parent, starting after the second cut point (when the
end of the chromosome is reached, the sequence continues at position 1). In Figure
3.12, the subset {5, 6, 4} in parent 1 is first copied to the offspring (step 1). Then,
the remaining positions are filled one by one after the second cut point, by
considering the corresponding sequence of cities in parent 2, namely subset {5, 7, 8,
1, 4, 2, 3, 6} (step 2). Hence, city 5 is first considered to occupy position 6, but it is
discarded because it is already included in the offspring. City 7 is the next city to be
considered, and it is inserted at position 6. Then city 8 is inserted at position 7, city 1
is inserted at position 8, city 4 is discarded, city 2 is inserted at position 1, city 3 is
inserted at position 2, and city 6 is discarded.

Parent 1 : 1 2 │ 5 6 4 │ 3 8 7
Parent 2 : 1 4 │ 2 3 6 │ 5 7 8
Offspring (step 1): - - 5 6 4 - - -
Offspring (step 2): 2 3 5 6 4 7 8 1

Figure 11 : Order crossover

2.5 Order-Based Crossover (OBX)

This crossover focuses on the relative order of the cities on the parent
chromosomes. First, a subset of cities is selected from the first parent. In the
offspring, these cities appear in the same order as in the first parent, but at positions
taken from the second parent. Then, the remaining positions are filled with the cities
of the second parent. In Figure 12, cities 5, 4 and 3 are first selected in parent 1, and
must appear in this order in the offspring. These cities occupy positions 2, 4 and 6,

Irhamah Introduction to Genetic Algorithm – page 8

respectively, in the offspring. The remaining positions are filled with the cities found
in parent 2.

Parent 1 : 1 2 5 6 4 3 8 7
Parent 2 : 1 4 2 3 6 5 7 8
Offspring: 1 5 2 4 6 3 7 8

Figure 12 : Order-based crossover

2.6 Position-Based Crossover

Here, a subset of positions is selected in the first parent. Then, the cities
found at these positions are copied to the offspring (at the same positions). The other
positions are filled with the remaining cities, in the same relative order as in the
second parent. The name of this operator is rather misleading, because it is the
relative order of the cities that is inherited from the parents (the absolute position of
the cities inherited from the second parent are rarely preserved). This operator can
be seen as an extension of the order crossover, where the cities inherited from the
first parent do not necessarily occupy the consecutive positions. In Figure 13,
position 3, 5 and 6 are first selected in parent 1. Cities 5, 4 and 3 are found at these
positions, and occupy the same positions in the offspring. The other positions are
filled one by one, starting at position 1, by inserting the remaining cities according to
their relative order in parent 2, namely the set {1, 2, 6, 7, 8}.

Parent 1 : 1 2 5 6 4 3 8 7
Parent 2 : 1 4 2 3 6 5 7 8
Offspring: 1 2 5 6 4 3 7 8

Figure 13 : Position-based crossover

Irhamah Introduction to Genetic Algorithm – page 9

4 Mutation

Mutation is a genetic operator that alters one or more gene values in a

chromosome from its initial state. This may produce entirely new gene values being
added to the gene pool. With these new gene values, the GA may be able to arrive at
a better solution than what was previously possible. Mutation is an important part of
the genetic search as helps to prevent the population from stagnating at any local
optima. Mutation occurs during evolution according to a user-definable mutation
probability rate. This probability rate should usually be set low. If it is set too high,
the search will turn into a primitive random search (Mitchell, 1996).

The primary purpose of mutation is to increase variation in a population.

Especially in the case where the initial population may be a small subset of all
possible solutions, mutation is the most important operator in GA. It is possible, for
example, that every instance of an essential bit might be zero in the initial
population. In such a case, crossover could never set that bit to one; mutation,
however, can set that bit to one. Mutation takes place after the crossover has been
performed. Mutation changes the new offspring randomly. For binary encoding, it
is done by switching a few randomly chosen bits from 1 to 0 or from 0 to 1 (Mitchell,
1996; Shen et al., 2001).

A variety of mutation methods are used in GAs, e.g. inversion, insertion,

displacement and reciprocal exchange mutation. Inversion mutation selects two
positions within a chromosome at random and then inverts the substring between
these two positions. Insertion is where individual is randomly selected and inserted
in a random position. Displacement is where a subtour is selected at random and
inserted in a random position. Whereas reciprocal exchange where we select two
positions at random and swap them (Gen and Cheng, 1997).

Another mutation operators are 2-opt, 3-opt and Or-opt. In the 2-opt operator,
two edges (a, b) and (c, d) are randomly selected from the tour and check if we can
connect these four nodes in a different manner that will give a lower cost. To do this
check if cab + ccd > cac + cdb. If this is the case, replace the edges (a, b) and (c, d)
with the edges (a, c) and (d, b). Note that it is assumed that a, b, c and d appear in

Irhamah Introduction to Genetic Algorithm – page 10

that specific order in the tour even if b and c are not connected. A 3-opt operator
looks at three random edges instead of two. If there are edges (a, b), (c, d) and (e, f),
check if cab + ccd + cef > cac + cbe + cdf. If it is, replace (a, b), (c, d) and (e, f) with
the edges (a, c), (b, e) and (d, f).

The Or-opt operator is similar to the 3-opt. A set of connected nodes are
randomly chosen and checking is made if the string can be inserted between two
other connected nodes to give a reduced cost. We can calculate this by finding the
total cost of the edges being inserted and the total cost of the edges being removed.
If the cost of the edges being removed is greater than the cost of those being inserted
the switch is made (Bryant, 2000).

5 Operator Settings in Genetic Algorithm

It has long been acknowledged that the choice of operator settings has a
significant impact upon GA performance. These settings come in the form of
components and parameters such as chromosome representation, population sizes,
crossover and mutation operators, crossover and mutation probability, replacement
techniques, etc. However, finding a good choice is somewhat of a black art. The
most commonly used method is empirically through hand tuning. This is where the
GA’s settings are determined by first experimenting with a selection of combinations
of settings and then selecting the combination that gives the best results.

The crossover and mutation operators are considered to be two of the main
conduits by which to control exploration and exploitation of the search space of GA.
This is partly why most of the work performed on finding good configurations for a
GA consists of keeping the other settings constant and just looking for good operator
rates. In one of earliest works of variation of settings in a GA, Davis (1989) suggests
the division of operators into two categories based on the roles they play. In the first
category are those operators that directly improve a chromosome, while in the second
category, operators that set the stage for such improvement.

Irhamah Introduction to Genetic Algorithm – page 11

 Different forms of schemata exist in solutions for different problems
(Starkweather et al., 1991) and different crossover or mutation operators are better at
exploring different forms of schemata. Where a crossover operator successfully
exploits wanted traits from a particular solution space, for a solution space of a
different problem, it may be that it is able to exploit the wanted traits, but at a
different rate of efficiency (thus explaining the need for different operator rates) or
unable to do so at all.

While classic GAs employ the use of settings that remain static throughout its
application run, some existing work was found that indicated that by varying these
settings appropriately during the GA’s application, improvements can be made to
overall performance of the GA in terms of both solution quality and the amount of
time required to find it. The adaptive variation approach, also known as online
approach (Spears, 1995) involves the utilization of information gained about the state
of the GA during the run of the GA, which may consist of information about the
population as a whole, individuals in the population or components of the GA, to
adaptively vary its settings there.

Irhamah Introduction to Genetic Algorithm – page 12