Advanced Artificial

Intelligence

Lecture 4:
Evolutionary Computing
Evolutionary Computing
Evolution is an optimization process.

The aim is to improve the ability of an

organism to survive in dynamically
changing and competitive
environments.

Advanced AI - 4: Evolutionary Computing 2

In this sense, an evolutionary algorithm (EA)
is a search for an optimal solution to a given
problem
Main components of an EA:
1. an encoding of solutions to the problem as a
chromosome;
2. a function to evaluate the fitness, or survival strength
of individuals;
3. initialization of the initial population;
4. selection operators; and
5. reproduction operators.
6. Stop condition

Advanced AI - 4: Evolutionary Computing 3

Evolutionary Computation Paradigms
Genetic algorithms model genetic evolution.

Genetic programming, based on genetic

algorithms, but individuals are programs.

Evolutionary programming, derived from the

simulation of adaptive behavior in evolution (i.e.
phenotypic evolution).

Evolution strategies, geared toward modeling

the strategic parameters that control variation in
evolution, i.e. the evolution of evolution.
Advanced AI - 4: Evolutionary Computing 4
Genetic Algorithms
Directed search algorithms based on the mechanics of
biological evolution
Local Search method
Developed by John Holland, University of Michigan
(1970’s)
To understand the adaptive processes of natural systems
To design artificial systems software that retains the
robustness of natural systems

Advanced AI - 4: Evolutionary Computing 5

Inspired by natural evolution
Living things evolved into more successful organisms
– offspring exhibit some traits of each parent

Provide efficient, effective techniques for optimization

and machine learning applications
Widely-used today in business, scientific and
engineering circles

Advanced AI - 4: Evolutionary Computing 6

Simple Genetic Algorithm
{
initialize population;
evaluate population;
while TerminationCriteriaNotSatisfied
{
select parents for reproduction;
perform recombination and mutation;
evaluate population;
}
}

Advanced AI - 4: Evolutionary Computing 7

The GA Cycle of Reproduction
children
reproduction modification
modified
parents children
population evaluation
evaluated children
deleted
members

discard

Advanced AI - 4: Evolutionary Computing 8

Population
population

Chromosomes could be:

Bit strings (0101 ... 1100)
Real numbers (43.2 -33.1 ... 0.0 89.2)
Permutations of element (E11 E3 E7 ... E1 E15)
Lists of rules (R1 R2 R3 ... R22 R23)
Program elements (genetic programming)
... any data structure ...

Advanced AI - 4: Evolutionary Computing 9

Reproduction
children
reproduction

parents

population

Parents are selected at random with

selection chances biased in relation to
chromosome evaluations.

Advanced AI - 4: Evolutionary Computing 10

Chromosome Modification
children
modification
modified children

Modifications are stochastically triggered

Operator types are:
Mutation
Crossover (recombination)

Advanced AI - 4: Evolutionary Computing 11

Mutation: Local Modification
Before: (1 0 1 1 0 1 1 0)
After: (0 1 1 0 0 1 1 0)

Before: (1.38 -69.4 326.44 0.1)

After: (1.38 -67.5 326.44 0.1)

Causes movement in the search space

(local or global)
Restores lost information to the population

Advanced AI - 4: Evolutionary Computing 12

Crossover: Recombination
*
P1 (0 1 1 0 1 0 0 0) (0 1 0 0 1 0 0 0) C1
P2 (1 1 0 1 1 0 1 0) (1 1 1 1 1 0 1 0) C2

Crossover is a critical feature of genetic

algorithms:
It greatly accelerates search early in evolution of a
population
It leads to effective combination of schemata
(subsolutions on different chromosomes)

Advanced AI - 4: Evolutionary Computing 13

Evaluation
modified
evaluated children
children
evaluation

The evaluator decodes a chromosome and assigns

it a fitness measure
The evaluator is the only link between a classical
GA and the problem it is solving
Advanced AI - 4: Evolutionary Computing 14
Deletion
population
discarded members

discard

Generational GA:
entire populations replaced with each iteration
Steady-state GA:
a few members replaced each generation

Advanced AI - 4: Evolutionary Computing 15

An Abstract Example

Distribution of Individuals in Generation 0

Distribution of Individuals in Generation N

Advanced AI - 4: Evolutionary Computing 16

Keep a population of individuals that are
complete solutions (or partial solutions)
Explore solution space by having these
individuals “interact” and “compete”
interaction produces new individuals
competition eliminates weak individuals
After multiple generations a strong individual
(i.e., solution) should be found
“Simulated Evolution” via a form of
Randomized Beam Search

Advanced AI - 4: Evolutionary Computing 17

Mechanisms of evolutionary change:
Alteration (Crossover): the (random) combination
of 2 parents’ chromosomes during reproduction resulting in
offspring that have some traits of each parent

Alteration requires genetic diversity among

the parents to ensure sufficiently varied offspring

Advanced AI - 4: Evolutionary Computing 18

Mechanisms of evolutionary change:
Natural selection: the fittest survive in a competitive
environment resulting in better organisms
individuals with better survival traits
generally survive for a longer period of time
this provides a better chance for reproducing

and passing the successful traits on to offspring

over many generations the species improves

since better traits will out number weaker ones

Advanced AI - 4: Evolutionary Computing 19

 Genetic Algorithm
Create initial random population

Evaluate fitness of each individual

yes
Termination criterion satisfied? stop
no
Select parents according to fitness

Recombine parents to generate offspring

Mutate offspring

Replace population by new offspring

Advanced AI - 4: Evolutionary Computing 20
Genetic Algorithm
1. Let s = {s1, …, sN} be the current population
2. Let p[i] = f(si)/SUMjf(sj) be the fitness probabilities
3. for k = 1; k < N; k += 2
• Parent1 = randomly pick si with prob. p[i]
• Parent2 = randomly pick another sj with prob. p[j]
• Randomly select 1 crossover point, and swap
strings of parents 1 and 2 to generate two children
t[k] and t[k+1]
4. for k = 1; k ≤ N; k++
• Randomly mutate each position in t[k] with a
small probability
5. New generation replaces old generation: s = t
Advanced AI - 4: Evolutionary Computing 21
x example: selection
2

String Init. X Value Fitness Probi Cumulative Random Selected

Population f(x)=x2 Prob. Number
1 01101 13 169 0.14 0.14 0.13 1
2 11000 24 576 0.49 0.63 0.53 2
3 01000 8 64 0.06 0.69 0.61 2
4 10011 19 361 0.31 1 0.92 4

Sum 1170 1
Average 293 0.25
Max 576 0.49

Advanced AI - 4: Evolutionary Computing 22

X example: crossover
2

String Mating Pool Crossover point Offspring after X value Fitness

Crossover F(x)= x2
1 01101 4 01100 12 144
2 11000 4 11001 25 625
3 11000 2 11011 27 729
4 10011 2 10000 16 256

Sum 1754
Average 439
Max 729

Advanced AI - 4: Evolutionary Computing 23

X example: mutation
2

Advanced AI - 4: Evolutionary Computing 24

Some Issues in Genetic Algorithm
Solution is only as good as the evaluation
function (often hardest part)

Advanced AI - 4: Evolutionary Computing 25

Initialization: Seeding the Population
• Initialization sets the beginning population
of individuals from which future generations
are produced

• Issues:
– size of the initial population
experimentally determined for problem
– diversity of the initial population (genetic diversity)
 a problem resulting from lack of diversity is premature

convergence to a non-optimal solution

Advanced AI - 4: Evolutionary Computing 26

Initialization: Seeding the Population
How is a diverse initial population generated?
uniformly random: generate individuals
randomly from a solution space with uniform distribution
grid initialization: choose individuals
at regular "intervals" from the solution space
non-clustering: require individuals to be a predefined
"distance" away from those already in the population
local optimization: use another technique (e.g. HC)
to find initial population of local optima; doesn't ensure
diversity but guarantees solution to be no worse than the
local optima

Advanced AI - 4: Evolutionary Computing 27

Evaluation: Ranking by Fitness

Evaluation ranks the individuals using some

fitness measure that corresponds with the
quality of the individual solutions

For example, given individual i:

classification: (correct(i))2
TSP: 1/distance(i)

Advanced AI - 4: Evolutionary Computing 28

Selection: Finding the Fittest
• Choose which individuals survive and possibly
reproduce in the next generation
• Selection depends on the evaluation/fitness
function
– if too dependent, then, like greedy search, a non-
optimal solution may be found
– if not dependent enough, then may not converge to a
solution at all
• Nature doesn't eliminate all "unfit" genes;
they usually become recessive for a long period
of time, and then may mutate to something useful
Advanced AI - 4: Evolutionary Computing 29
Advanced AI - 4: Evolutionary Computing 30
Selection Techniques

• Deterministic Selection
– relies heavily on evaluation/fitness function
– converges fast
• Two approaches:
– next generation is parents and their children
• parents are the best of the current generation
• parents produce children and survive to next generation
– next generation is only the children
• parents are the best of the current generation
• parents are used to produce children only
• parents don't survive

Advanced AI - 4: Evolutionary Computing 31

Selection Techniques
Proportional Fitness Selection
each individual is selected proportionally to their
fitness score
even the worst individual has a chance to survive
helps prevent “stagnation” in the population
Two approaches:
rank selection: individual selected with a probability
proportional to its rank in population sorted by fitness
proportional selection: individual selected with a
probability: Fitness(individual) / ∑ Fitness for all
individuals

Advanced AI - 4: Evolutionary Computing 32

Alteration: Producing New Individuals
• Alteration is used to produce new
individuals
• Crossover for vector representations:
– Pick pairs of individuals as parents and
randomly swap their segments
– also known as "cut and splice"

• Parameters:
– number of crossover points
– positions of the crossover points
Advanced AI - 4: Evolutionary Computing 33
Alteration: Producing New Individuals

1-point Crossover
pick a dividing point in the parents' vectors
and swap their segments
Example
given parents: 1101101101 and 0001001000
crossover point: after the 4th digit
children produced are:
1101 + 001000 and 0001 + 101101

Advanced AI - 4: Evolutionary Computing 34

1-point Crossover

Advanced AI - 4: Evolutionary Computing 35

Alteration: Producing New Individuals

• N-point Crossover
– generalization of 1-point crossover
– pick n dividing points in the parents' vectors
and splice together alternating segments
• Uniform Crossover
– the value of each element of the vector is
randomly chosen from the values in the
corresponding elements of the two parents
• Techniques also exist for permutation representations
Advanced AI - 4: Evolutionary Computing 36
N-point Crossover

uniform Crossover
Advanced AI - 4: Evolutionary Computing 37
Alteration: Producing New Individuals
• Alteration is used to produce new individuals
• Mutation
– randomly change an individual
– e.g. TSP: two-swap, two-interchange
• Parameters:
– mutation rate
– size of the mutation

Advanced AI - 4: Evolutionary Computing 38

Genetic Algorithms as Search
Problem of Local Maxima
individuals get stuck at pretty good but not optimal
solutions
– any small mutation gives worse fitness
– crossover can help get out of a local maximum
– mutation is a random process, so it is possible that we
may have a sudden large mutation to get these
individuals out of this situation

Advanced AI - 4: Evolutionary Computing 39

 GA is a kind of hill-climbing search
 Very similar to a randomized beam search
 One significant difference between GAs and
HC
is that, it is generally a good idea in GAs to “fill
the local maxima up with individuals”
 Overall, GAs have less problems with local
maxima than HC or neural networks

Advanced AI - 4: Evolutionary Computing 40

Hill-climbing
Calculus based approach

Advanced AI - 4: Evolutionary Computing 41

But…
?

Advanced AI - 4: Evolutionary Computing 42

But… ?

?
?

? ?

Advanced AI - 4: Evolutionary Computing 43

More realistically? ?

? ?
?
?
?

? ?

? ?
•Noisy?
•Discontinuous? ?

?
Advanced AI - 4: Evolutionary Computing 44
Genetic algorithms

Cope with non-linear functions

Cope with large numbers of variables efficiently
Cope with modelling uncertainties
Do not require knowledge of the function

Advanced AI - 4: Evolutionary Computing 45

The Traveling Salesman Problem
Find a tour of a given set of cities so that
each city is visited only once
the total distance traveled is minimized

Advanced AI - 4: Evolutionary Computing 46

Representation of Individuals
Solutions represented as a vector of values
Traveling salesperson problem (TSP)
 Tour can be represented as a sequence of cities visited

Advanced AI - 4: Evolutionary Computing 47

Representation
Representation is an ordered list of city
numbers known as an order-based GA.

1) London 3) Dunedin 5) Beijing 7) Tokyo

2) Venice 4) Singapore 6) Phoenix 8) Victoria

CityList1 (3 5 7 2 1 6 4 8)
CityList2 (2 5 7 6 8 1 3 4)

Advanced AI - 4: Evolutionary Computing 48

Crossover
Crossover combines inversion and
recombination:
* *
Parent1 (3 5 7 2 1 6 4 8)
Parent2 (2 5 7 6 8 1 3 4)

Child (2 5 7 2 1 6 3 4)

This operator is called the Order1 crossover.

Advanced AI - 4: Evolutionary Computing 49

Mutation
Mutation involves reordering of the list:

* *
Before: (5 8 7 2 1 6 3 4)

After: (5 8 6 2 1 7 3 4)

Advanced AI - 4: Evolutionary Computing 50

TSP Example: 30 Cities
120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

Advanced AI - 4: Evolutionary Computing 51

Solution i (Distance = 941)
TSP30 (Performance = 941)

120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

Advanced AI - 4: Evolutionary Computing 52

Solution j(Distance = 800)
44
62 TSP30 (Performance = 800)
69
67 120
78
64 100
62
54
80
42
50
40 y 60
40
38 40
21
35
67 20
60
60 0
40 0 10 20 30 40 50 60 70 80 90 100
42 x
50
99

Advanced AI - 4: Evolutionary Computing 53

Solution k(Distance = 652)
TSP30 (Performance = 652)

120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

Advanced AI - 4: Evolutionary Computing 54

Best Solution (Distance = 420)
42
38 TSP30 Solution (Performance = 420)
35
120
26
21
35 100
32
7
80
38
46
44 y 60
58
60 40
69
76
20
78
71
69 0
67 0 10 20 30 40 50 60 70 80 90 100
62 x
84
94

Advanced AI - 4: Evolutionary Computing 55

Overview of Performance
TSP30 - Overview of Performance

1800

1600

1400

1200
e
1000 c
n
a
t
800 s
i
D
600

400

200

0 Best
31 29 27 25 23 21 19 17 15 13 11 9 7 5 3 1
Worst
Generations (1000)
Average

Advanced AI - 4: Evolutionary Computing 56

Benefits of Genetic Algorithms
Concept is easy to understand
Modular, separate from application
Supports multi-objective optimization
Good for “noisy” environments
Always an answer; answer gets better with time
Inherently parallel; easily distributed

Advanced AI - 4: Evolutionary Computing 57

Benefits of Genetic Algorithms (cont.)

Many ways to speed up and improve a GA-based

application as knowledge about problem domain is
gained
Easy to exploit previous or alternate solutions
Flexible building blocks for hybrid applications
Substantial history and range of use

Advanced AI - 4: Evolutionary Computing 58

Some GA Application Types
Domain Application Types
Control gas pipeline, pole balancing, missile evasion, pursuit

Design semiconductor layout, aircraft design, keyboard

configuration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation

Robotics trajectory planning

Machine Learning designing neural networks, improving classification

algorithms, classifier systems
Signal Processing filter design

Game Playing poker, checkers, prisoner’s dilemma

Combinatorial set covering, travelling salesman, routing, bin packing,

graph colouring and partitioning
Optimization

Advanced AI - 4: Evolutionary Computing 59