Professional Documents
Culture Documents
Introduction To Genetic Algorithms: Dr. Rajib Kumar Bhattacharjya
Introduction To Genetic Algorithms: Dr. Rajib Kumar Bhattacharjya
Bhattacharjya/CE/IITG
7 November 2013
References
2 R.K. Bhattacharjya/CE/IITG
D. E. Goldberg, ‘Genetic Algorithm In Search,
Optimization And Machine Learning’, New York:
Addison – Wesley (1989)
7 November 2013
Introduction to optimization
3 R.K. Bhattacharjya/CE/IITG
Global optima
Local optima
Local optima
Local optima
Local optima
X 7 November 2013
Introduction to optimization
4 R.K. Bhattacharjya/CE/IITG
Multiple optimal solutions
7 November 2013
Genetic Algorithms
5 R.K. Bhattacharjya/CE/IITG
7 November 2013
Principle Of Natural Selection
6 R.K. Bhattacharjya/CE/IITG
“Select The
Best, Discard
The Rest”
7 November 2013
An Example….
7 R.K. Bhattacharjya/CE/IITG
Favorable characteristic
propagated through generations
of giraffes.
Initial Population of
animals
7 November 2013
Simple Genetic Algorithms
11 R.K. Bhattacharjya/CE/IITG
Start
Initialize
population
Evaluate Solutions
T=0 NO
Optimum
Solution?
Selection
YES
T=T+1 Stop Crossover
Mutation
7 November 2013
Simple Genetic Algorithm
12 R.K. Bhattacharjya/CE/IITG
function sga ()
{
Initialize population;
Calculate fitness function;
7 November 2013
GA Operators and Parameters
13 R.K. Bhattacharjya/CE/IITG
Selection
Crossover
Mutation
7 November 2013
Selection
14 R.K. Bhattacharjya/CE/IITG
7 November 2013
Functions of Selection operator
15 R.K. Bhattacharjya/CE/IITG
7 November 2013
Fitness function
16 R.K. Bhattacharjya/CE/IITG
7 November 2013
Assigning a fitness value
17 R.K. Bhattacharjya/CE/IITG
Considering c = 0.0654
23 h
7 November 2013
Selection operator
18 R.K. Bhattacharjya/CE/IITG
They are:
Tournament selection
Roulette wheel selection
Proportionate selection
Rank selection
Steady state selection, etc
7 November 2013
Tournament selection
19 R.K. Bhattacharjya/CE/IITG
7 November 2013
Tournament selection
20 R.K. Bhattacharjya/CE/IITG
22 32 Selected
22 25
22 25
13 +30 25
32 22
32 40
32 22
25 13 +30
40 22
Best solution will have two copies
3
21%
4
7 November 2013
18%
Rank selection
22 R.K. Bhattacharjya/CE/IITG
New
Good Good
offspring
New
Bad
offspring
7 November 2013
How to implement crossover
24 R.K. Bhattacharjya/CE/IITG
The crossover operator is used to create new solutions from the existing solutions
available in the mating pool after applying selection operator.
This operator exchanges the gene information between the solutions in the
mating pool.
0 1 0 0 1 1 0 1 1 0
7 November 2013
Encoding Methods
26 R.K. Bhattacharjya/CE/IITG
Most common method of encoding is binary coded.
Chromosomes are strings of 1 and 0 and each position
in the chromosome represents a particular characteristic
of the problem
Decoded value 52 26
7 November 2013
Encoding Methods
27 R.K. Bhattacharjya/CE/IITG
h (d,h) = (8,10) cm
Chromosome = [0100001010]
d 7 November 2013
Crossover operator
28 R.K. Bhattacharjya/CE/IITG
Solution 1
Solution 2
Child 1
Child 2
7 November 2013
Binary Crossover
29 R.K. Bhattacharjya/CE/IITG
7 November 2013
Binary Mutation
31 R.K. Bhattacharjya/CE/IITG
Mutation operator changes a 1 to 0 or vise versa, with a mutation
probability of .
The mutation probability is generally kept low for steady convergence.
A high value of mutation probability would search here and there like a
random search technique.
7 November 2013
Source: Deb 1999
Elitism
32 R.K. Bhattacharjya/CE/IITG
7 November 2013
Nature to Computer Mapping
33 R.K. Bhattacharjya/CE/IITG
Nature Computer
Population Set of solutions
Individual Solution to a problem
Fitness Quality of a solution
Chromosome Encoding for a solution
Gene Part of the encoding solution
7 November 2013
An example problem
34 R.K. Bhattacharjya/CE/IITG
7 November 2013
An example problem
35 R.K. Bhattacharjya/CE/IITG
Initialize Calculate decoded Calculate real Calculate objective
population value value function value
Fitness
Initial population Decoding Selection: Proportionate selection
calculation
Sol No Binary DV x f F Relative Expected Actual
String value Fitness count count
Random generation of
Matting pool New population
crossover site
Mutation
Sol No Population Population DV x f F
after crossover after mutation value
7 November 2013
Real coded Genetic Algorithms
38 R.K. Bhattacharjya/CE/IITG
more computation
lower accuracy
hamming cliff
7 November 2013
Real coded Genetic Algorithms
39 R.K. Bhattacharjya/CE/IITG
7 November 2013
Real coded Genetic Algorithms
40 R.K. Bhattacharjya/CE/IITG
P1 8 6 3 7 6
P2 2 9 4 8 9
C1 8 6 4 8 9
C2 2 9 3 7 6
7 November 2013
Real coded Genetic Algorithms
41 R.K. Bhattacharjya/CE/IITG
Linear Crossover
0.1 0.3 0.1 0.3 0.7 0.2 0.5 0.1 0.2 0.1 0.3 0.1 0.3 0.7 0.5 0.5 0.1 0.2
7 November 2013
Real coded Genetic Algorithms
43 R.K. Bhattacharjya/CE/IITG
0.5 0.7 0.7 0.5 0.2 0.8 0.3 0.9 0.4 0.5 0.7 0.7 0.5 0.2 0.5 0.4 0.5 0.3
0.1 0.3 0.1 0.3 0.7 0.2 0.5 0.1 0.2 0.1 0.3 0.1 0.3 0.7 0.5 0.4 0.5 0.3
7 November 2013
Real coded Genetic Algorithms
44 R.K. Bhattacharjya/CE/IITG
0.1 0.3 0.1 0.3 0.6 0.2 0.5 0.1 0.2 0.3 0.5 0.4 0.4 0.4 0.5 0.4 0.5 0.3
7 November 2013
Simulated binary crossover
45 R.K. Bhattacharjya/CE/IITG
Developed by Deb and Agrawal, 1995)
7 November 2013
Random mutation
46 R.K. Bhattacharjya/CE/IITG
7 November 2013
Normally distributed mutation
47 R.K. Bhattacharjya/CE/IITG
7 November 2013
Polynomial mutation
48 R.K. Bhattacharjya/CE/IITG
7 November 2013
Multi-modal optimization
49 R.K. Bhattacharjya/CE/IITG
7 November 2013
After Generation 200
50 R.K. Bhattacharjya/CE/IITG
7 November 2013
Multi-modal optimization
51 R.K. Bhattacharjya/CE/IITG
Simple modification of Simple Genetic Algorithms can capture all the optimal
solution of the problem including global optimal solutions
Basic idea is that reduce the fitness of crowded solution, which can be
implemented using following three steps.
Sharing function
Niche count
Modified fitness
7 November 2013
Hand calculation
52 R.K. Bhattacharjya/CE/IITG
Maximize
7 November 2013
Distance table
53 R.K. Bhattacharjya/CE/IITG
dij 1 2 3 4 5 6
1 0 0.254 0.73 1.302 0.127 0.191
2 0.254 0 0.476 1.048 0.127 0.063
3 0.73 0.476 0 0.572 0.603 0.539
4 1.302 1.048 0.572 0 1.175 1.111
5 0.127 0.127 0.603 1.175 0 0.064
6 0.191 0.063 0.539 1.111 0.064 0
7 November 2013
Sharing function values
54 R.K. Bhattacharjya/CE/IITG
sh(dij) 1 2 3 4 5 6 nc
1 1 0.492 0 0 0.746 0.618 2.856
2 0.492 1 0.048 0 0.746 0.874 3.16
3 0 0.048 1 0 0 0 1.048
4 0 0 0 1 0 0 1
5 0.746 0.746 0 0 1 0.872 3.364
6 0.618 0.874 0 0 0.872 1 3.364
7 November 2013
Sharing fitness value
55 R.K. Bhattacharjya/CE/IITG
7 November 2013
Solutions obtained using modified
56
fitness value R.K. Bhattacharjya/CE/IITG
7 November 2013
Evolutionary Strategies
57 R.K. Bhattacharjya/CE/IITG
7 November 2013
Two members ES: (1+1) ES
58 R.K. Bhattacharjya/CE/IITG
7 November 2013
Two members ES: (1+1) ES
59 R.K. Bhattacharjya/CE/IITG
7 November 2013
Two members ES: (1+1) ES
60 R.K. Bhattacharjya/CE/IITG
Rechenberg postulate
The ratio of successful mutations to all the mutations
should be 1/5. If this ratio is greater than 1/5, increase
mutation strength. If it is less than 1/5, decrease the
mutation strength.
7 November 2013
Two members ES: (1+1) ES
61 R.K. Bhattacharjya/CE/IITG
7 November 2013
Matlab code
62 R.K. Bhattacharjya/CE/IITG
7 November 2013
63
7 November 2013
Some results of 1+1 ES
64 R.K. Bhattacharjya/CE/IITG
4.5
10
20 0
0
4
3.5 50 10 0
10 0
Optimal Solution is
50
10
20 50 X*= [3.00 1.99]
2.5
Y
20
20 0
1 5 10 20
2
Objective function value f
2
5
2
1.5 50 10 = 0.0031007
10
20
0
20
1
50
20 10 0
0
0.5 10 0
20 0
0 20 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
X
7 November 2013
Multimember ES
65 R.K. Bhattacharjya/CE/IITG
ES
7 November 2013
Multimember ES
66 R.K. Bhattacharjya/CE/IITG
ES
Through selection
Through mutation
7 November 2013
Multimember ES
67 R.K. Bhattacharjya/CE/IITG
ES
Offspring
Through selection
Through mutation
7 November 2013
Multi-objective optimization
68 R.K. Bhattacharjya/CE/IITG
Comfort
Price
7 November 2013
Multi-objective optimization
69 R.K. Bhattacharjya/CE/IITG
• Minimize weight
• Minimize deflection
7 November 2013
Multi-objective optimization
70 R.K. Bhattacharjya/CE/IITG
7 November 2013
Multi-objective optimization
71 R.K. Bhattacharjya/CE/IITG
7 November 2013
Vector Evaluated Genetic Algorithm (VEGA)
72 R.K. Bhattacharjya/CE/IITG
f1 P1
f2 P2
Crossover
f3 P3
and
f4 P4 Mutation
…
…
fn Pn
7 November 2013
Weighted based genetic algorithm (WBGA)
74 R.K. Bhattacharjya/CE/IITG
Sharing function
7 November 2013
Multiple objective genetic algorithm (MOGA)
75 R.K. Bhattacharjya/CE/IITG
5 60
4.5
50
4
3.5
40
30
f2
x2
2.5
2
20
1.5
1 10
0.5
0
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x1
f1
Maximize f1
Multiple objective genetic algorithm (MOGA)
77 R.K. Bhattacharjya/CE/IITG
7 November 2013
Multiple objective genetic algorithm (MOGA)
78 R.K. Bhattacharjya/CE/IITG
7 November 2013
NSGA
79 R.K. Bhattacharjya/CE/IITG
7 November 2013
NSGA II
80 R.K. Bhattacharjya/CE/IITG
7 November 2013
Non-dominated sorting
81 R.K. Bhattacharjya/CE/IITG
1
4
Feasible Region
5
Objective 2 (Minimize)
Objective 2 (Minimize)
4
1
6 2
3
2
3 5
Infeasible Region
7 November 2013
Calculation crowding distance
82 R.K. Bhattacharjya/CE/IITG
7 November 2013
Crowded tournament operator
A solution ݅ wins a tournament with another solution
83 R.K. Bhattacharjya/CE/IITG
݆,
If the solution ݅ has better rank than ݆, i.e. ݎ < ݎ
7 November 2013
Replacement scheme of NSGA II
R.K. Bhattacharjya/CE/IITG
ݎଵ ݎଵ
ݎଶ ݎଶ
P ݎଷ
ݎଷ Selected
Rejected
ݎସ
84
Q
ݎହ
Rejected based on crowding
Non dominated sorting distance
7 November 2013
Initialize population of size N
Selection
No
Crossover
Termination Mutation
Criteria? Yes
Calculate objective function of the
new population
Pareto-optimal solution
Combine old and new population
7 November 2013
Constraints handling in GA
86 R.K. Bhattacharjya/CE/IITG
55
45
35
f(X)
25
15
-5
0 2 4 6 8 10 12
X
݂()ݔ
7 November 2013
Constraints handling in GA
87 R.K. Bhattacharjya/CE/IITG
55
45
35
f(X)
25
15
-5
0 2 4 6 8 10 12
X
݂ ݔ+ܴ ݃ ݔ
݂()ݔ
7 November 2013
Constraints handling in GA
88 R.K. Bhattacharjya/CE/IITG
55
45
35
f(X)
25
15
݂௫ + ݃ ݔ
5
-5
0 2 4 6 8 10 12
X
݂ ݔ+ܴ ݃ ݔ
݂()ݔ
7 November 2013
Constrain handling in GA
89 R.K. Bhattacharjya/CE/IITG
Minimize ݂ ܺ
Deb’s approach
ܺ ݂=ܨ If ܺ is feasible
7 November 2013
90 R.K. Bhattacharjya/CE/IITG
THANKS
7 November 2013