Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010
NEURAL NETWORK OPTIMIZATION BASED ON IMPROVED DIPLOIDIC
GENETIC ALGORITHM
KE-YONG SHAO, FEI LI, BEI-YAN JIANG, NA WANG, HONG-YAN ZHANG, WEN-CHENG LI
School of Electrical and Information Engineering, Daqing Petroleum Institute, Heilongjiang Daqing 163318, China
E-MAIL: shaokeyong@tom.com, lifei96485@ tom.com
Abstract:
In this paper, a kind of improved method of diploid
genetic algorithm (DGA) without considering the dominant and
recessive of the allele is given directed at the disadvantages of
DGA which are easy to fall into premature convergence and
have low efficiency in late period local searching. Improved the
genetic operation process by imitating the reproductive
processes of diplont and adopting the process of gametes
recombination and homologous chromosomes chiasma. United
the advantages of genetic algorithm and neural network, a new
neural network structure contacted with the diploid genetic
algorithm closely is designed. This scheme combines the strong
global search capability of genetic algorithm and self-learning
ability of neural network. Then applied the method to the
complex multi-peak function optimization. Simulation results
show that the improved algorithm can keep the population
diversity and repressed the premature convergence effectively.
The neural network optimization based on diploid genetic
algorithm increased the convergence speed and accuracy, and
ensured the global optimal.
Keywords:
Neural network; Genetic algorithm; Diploid; Function
optimization
1. Introduction
Artificial neural network (ANN) has strong ability of
nonlinear mapping approximation and parallel information
processing, as well as better capability of robustness and fault
tolerance. Which has been applied widespread in data mining,
industrial control system and especially in the complex
nonlinear object modeling [1-5]. There is numbers of
algorithms to train network, for example, BP learning
algorithm which is widely used in multilayer feed forward
neural network. But it belongs to the gradient descent
algorithm essentially, thus has some defects inevitably, such
as easily sinking into the local optimal values, training
slowly and indicator function having to be derivable and so
978-1-4244-6527-9/10/$26.00 © 2010 IEEE
1470
on [3-5].
Genetic algorithm (GA) is a kind of optimization
method by simulating biological evolution, and the
optimization process has nothing to do with gradient
information and continuity and derivability of optimal
function. Therefore, ANN and GA have complementary
advantages. However, the SGA has many shortcomings, such
as poor local search capability, premature convergence, etc
[6-8]. In order to improve the performance, researchers from
different aspects put forward lots of improved measures, such
as: ecologicniche techniques, adaptive GA, and so on [7-9].
Among them, Balgey put forward the DGA which has better
adaptability of adaptability and memory, but it doesn't allow
mutation operation to dominant value, thereby leads to
premature convergence. Hollosetin studied a diploid ternary
coding. In [9] He Fei proposed an fully dominant-recessive
diploid gene encoding method which plays an active role
about keeping the diversity. Yet, the gene encoding operation
is very troublesome, which not only increases the calculation,
but also has certain random.
This paper avoids dominance and recessive of gene,
makes alleles play equal role in individual performance,
improves the DGA by imitating the process of diploid
reproduction. Finally, uses strong global search capability and
high efficiency learning ability of DGA to optimize the
weights of NN. Simulation results show that the genetic
algorithm can effectively restrain premature convergence,
and achieve fast speed and high precision.
2. New type neural network structure
2.1. Back-Propagation neural network(BPNN)
Back-Propagation Neural Networkcomprised of an input
layer, a certain amount of hidden layers and an output layer is
a feed-forward network. If the number of neurons of input,
 Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010
hidden and output layer is nǃqǃm respectively, the network
can be expressed as BP(n, q, m). Using the network can
achieve the nonlinear mapping of n dimension input vector to
m dimension output vector. Normally, increasing the number
of hidden layer do not exert a severe influence on the
precision and expression ability of network. The basic
structure of BPNN has three layers of which the nodes
employing S-type effect function. The structure is shown in
Figure 1.
wij v jk
ˊ 
u ˊ ˊ  y
ˊ
Figure 1. Three layer BPNN structure diagram
2.2. Improved diploid genetic algorithm
Diploid structure provide redundancy memory ability
that makes organism not easy to forget knowledge learn from
former environmental. Therefore, they have strong capability
in adapting the environmental changing and dynamic
tracking.
The genotypes of fully dominant-recessive diploid gene
encoding are 0+, 0-, 1+ and 1- and 1 and the phenotypes are
0 and 1. Obviously, we need to convert binary number to
quaternary number first. Then the binary coding length is
twice as traditional methods. as a result, the calculation
amount is doubled. Similarly, the amount of calculation is
also twice when decoded.
Because of the shortcoming of large calculation in
traditional diploid genetic algorithm, we saves the dominance
and recessive of diploid as well as the genetic manipulation
closely related, makes the allele interact directly. According
to the impact mechanism of biological chromosomes, the
golden section method which has fuzzy properties is given by
simplifying the process. Namely, we determine the action
extent of alleles based on the golden section law exists
extensively in the nature.
Assuming the population scale is M, each individual xi
has two homologous chromosomes, denoted by xi1 and xi2,
each chromosomes contains n genes. Then, the golden section
method is:
1471
xi = 0.618xi1 + 0.382xi2 (1)
The specific operation of improved genetic algorithm is
as follows:
Step1: Firstly, sort the parent-population by the
individual fitness.
Step2: Homologous chromosomes crossover.
Cross the two homologous chromosomes for all
individuals respectively. The two chromatids are named
gamete after the operation of crossover.
Step3: Gametes recombination
Select a parent individual according to the sort order,
and match the two gametes of this individual with other two
gametes from another parent individual selected randomly.
Then, four new offspring individuals are generated (see
Figure 2). In addition, set the former M/5 parent individuals
have two chances to reproduce.
Figure 2. Schematic diagram of gametes recombination
Step4: Select one from six
Calculate the fitness of parent individuals and offspring
individuals. Select a high fitness individual from these six
individuals, then join it in the elite population.
Step5: When the number of the elite population reaches
to 2M, stop hybridization, otherwise return to step3.
Step6: Select M individuals of higher fitness from the
elite population, then offspring-populations is generated.
The improved algorithm makes the groups search a
larger solution space in the initial stage of evolution,
improves the capability of searching from global scope;
Reduces the probability of destroying optimum mode in the
later stage of evolution, while maintains a certain diversity of
the population, and strengthens the search capability in the
local range.
2.3. Diploid neural network(DNN)
In the process of diploid genetic algorithm and neural
network are integrated, correspond each chromatid with one
weight, namely two homologous connection are generated
between two neurons. The structure is shown in Figure 3.
 Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010

(1) Feedforward adjustment of HL to OL

wij v jk ∂J (k ) ∂J ∂O3l ∂IN 3l
= ⋅ ⋅
∂v jl (k ) ∂O3l ∂IN 3l ∂v jl
wij' v 'jk
ˊ ∂J (9)
= ⋅ g ' ( IN 3l ) ⋅ O2 j
ˊ ˊ  ∂O3l
u ˊ
ˊ  y
(2) Backward adjustment of HL to OL
∂J (k ) m
∂J ∂O3 s ∂IN 3s ∂O2 j ∂IN 2 j
=¦ ⋅ ⋅ ⋅ ⋅
∂v jl (k ) s =1 ∂O3 s ∂IN 3 s ∂O2 j ∂IN 2 j ∂v 'jl
'

m
∂J
=¦ ⋅ g ' ( IN 3 s ) ⋅ v js ⋅ g ' ( IN 2 j ) ⋅ O3l (k − 1)
s =1 ∂O3 s
Figure 3. Three layer DNN structure diagram
m

The DNN have all the characteristics of conventional = ¦ δ js ⋅ O3l (k − 1) (10)

s =1
feed forward neural network, Although the nodes unchanged,
but because it makes the connection weights of the neural where, δ = ∂J ⋅ g ' ( IN ) ⋅ v ⋅ g ' ( IN ) .
js 3s js 2j
network multiplied, the DNN possesses stronger robustness ∂O3 s
and tolerance.
Assuming a DNN of three-layer structure, denoted by (3) Feed forward adjustment of IL to HL
NN (n, q, m), the input layer activation function is: ∂J ( k ) m
∂J ∂O3 s ∂IN 3 s ∂O2 j ∂IN 2 j
=¦ ⋅ ⋅ ⋅ ⋅
g1( x) = x (2) ∂wij (k ) s =1 ∂O3s ∂IN 3 s ∂O2 j ∂IN 2 j ∂wij
And the activation function of hidden and output layers m
∂J
is: =¦ ⋅ g ' ( IN 3 s ) ⋅ v js ⋅ g ' ( IN 2 j ) ⋅ O1i
s =1 ∂O3 s
e x − e−x (3)
g ( x) = x m
e + e −x = ¦ δ js ⋅ O1i (11)
The feed forward propagation process of the network is s =1
as follows:
(4) Backward adjustment of IL to HL
(1) Input layer(IL):
q ∂J ( k ) m
∂J ∂O3 s q ∂IN 3 s
IN 1i = U i + ¦ wij' O2 j (k − 1) =¦ ⋅ ⋅¦ ⋅
j =1
∂wij ( k ) s =1 ∂O3 s ∂IN 3 s t =1 ∂O2t
'

O1i = IN1i , i = (1,2, …, n) (4) ∂O2t ∂IN 2t ∂O1i ∂IN1i

⋅ ⋅ ⋅
(2) Hidden layer(HL): ∂IN 2t ∂O1i ∂IN1i ∂wij'
n m
IN 2 j = ¦ wij O1i + ¦ v 'jl O3l (k − 1)
q
m
∂J
=¦ ⋅ g ' ( IN 3 s ) ⋅ ¦ vts ⋅ (12)
s =1 ∂O3 s
i =1 l =1
t =1
O 2 j = g ( IN 2 j ) , j = (1,2, … , q ) (5)
g ' ( IN 2 t ) ⋅ wit ⋅ g ' ( IN1i ) ⋅ O2 j ( k − 1)
(3) Output layer(OL):
q
IN 3l = ¦ v jl O2 j 3. Diploidic GA neural network(DGANN)
j =1

Yl = O3l = g ( IN 3l ) , l = (1,2, …, m) (6)

3.1. Encoding
Take indicator function as J = f (Y1 , Y2 , …, Ym ) , search
the network weights based on gradient descent method, The
The binary coding with intuitive and easy operation has
back propagation process of the network is as follows:
stronger search ability than decimal encoding. Furthermore,
All the weights are adjusted on the basis of formula(7).
the requirements of accuracy is much more valuable when
w(k + 1) = w(k ) + Δw(k ) (7) optimizing function or identifying system off-line. The
∂J (k ) (8) algorithm which put forward in this paper not only ensure
Δw(k ) = −η
∂w(k ) accuracy, but also has the characteristic of high efficiency, so

1472
 Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010

we choose binary coding in this paper. Quit searching process when meet any one of the two
conditions, go to Step7.
3.2. Genetic operators Step4: Carry out the improved selection operation.
Step5: Conduct diploid reproduction operation to the
selected population (including crossover and recombination
(1) Selection: This paper using ratio selection operator, process), and get offspring population.
namely, the probability of choosing any individual is Step6: Execute the mutation operation to population, go
proportional to its fitness. to Step2.
In order to avoid the reduction of population diversity Step7: Put the optimization results as the initial weights
and premature convergence which caused by too much of DNN at the end of DGA evolution.
selection pressure, this paper presents a limited selection Step8: Continue to optimize by DNN until meet the
strategy. That is to say, however the fitness, the time of precision requirement.
individual copy shall not exceed the given election limit.
(2) Flowchart of DGANN is as follows:
for the purpose of preventing to eliminate the highest
fitness individual, we have to adopt the optimal choice
strategy. Namely, the best individual can directly get into the Begin
next generation without genetic operation.
In the case of the number of the population is Initialize population
insufficient after selection, one part of the balance generate
new individual randomly to adding new genes, the other part
leave to the optimal individuals to increase the ratio of Calculate fitness
excellent genes in population.
(2) Crossover: Crossover is the genetic recombination of
Yes
two individuals. It is an important symbol of genetic Meet termination conditions
algorithm which is distinguished from other traditional
optimization method. In order to improve the computational No
efficiency and stability, unlike previous genetic algorithms, Genetic
this paper presents a cross-over strategies of homologous operation
chromosomes (see section 2.2).
(3) Recombination: This paper introduces a new concept No
of gametes recombination (see section 2.2) by imitating Reach limitied step
diploid organisms reproduction process for improving the
traditional algorithm. Yes

(4) Mutation: The variation of individual is a operation severe

No
that to change some of the gene randomly to form a new Meet required precision
variation
individual. It can improve partial searching ability of GA, and
maintain the diversity of population and prevent premature Yes
phenomenon. DNN optimize

3.3. Algorithm steps

End

(1) Steps of DGANN algorithm are as follows:

Step1: Initialize the control parametersˈgenerate the
initial population randomly.
Step2: Empty the elite population, and calculate the
individual fitness.
Step3: Judge whether meet the termination conditions of
algorithm. Here we use two end conditions:
1) The best fitness reach the set value.
2) The best fitness achieve a lower value, at the same
time the evolution step number over a certain limit value.
Figure 4. The flowchart of DGANN algorithm
4. Simulation
In this paper BPNN algorithm and DGANN algorithm is
used to optimize two typical test functions, to verify the
feasibility and effectively of neural network based on diploid
genetic algorithm. The comparison and analysis of simulation
results by Matlab are as follows:

1473
 Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010

Function 1: relatively simple, and 10-2 (based on BPNN) or 10-3 (based on

f1 = 25 + x 2 − 10 cos(2πx) + y 2 − 10 cos(2πy ) DGANN). The algorithm of DGANN includes 200 steps GA
Where, [ x, y ] ∈ [−2, 2] ; optimization and 800 steps NN optimization. The search
operation is considered failed if the algorithm hasn’t met the
Function 2:
given precision when iterations equal 1000. Under these
f 2 = 10 + ( x 2 + y 2 ) 0.25 *{sin[50( x 2 + y 2 ) 0.1 ]2 + 1} conditions, compare the average iterations, the average
Where, [ x, y] ∈ [−1, 1] . convergence time and the average optimal solutions in 100
Function 1 obtains the global minimum 5 at [0, 0], with times. The simulation results is shown in Table 1 and Table 2.
24 local optimum solution around (see Figure 5). function 2
obtains the global minimum 10 at [0, 0], with countless local TABLE 1. OPTIMIZATION PERFORMANCE CONTRAST FOR FUNCTION 1
optimal values around (see Figure 6). If conventional Average
Satisfaction Average
algorithm is adopted to optimize these two functions, it will Algoristhm Precision
Rate iterations
convergence
fall into local optimum or approximate optimum easily. So time

these functions can fully evaluate the search performance of BPNN 1 2 1000 --
algorithm and show its effectiveness clearly. DGANN 1e-3 92 90.4 0.64s

Average x Average y Average goal

1.2203 -1.0138 11.5380

60
0.0001 -0.0001 4.9995
40

20
TABLE 2. OPTIMIZATION PERFORMANCE CONTRAST FOR FUNCTION 2
0
Satisfaction Average Average
2 Algoristhm Precision
Rate iterations convergence time
1
BPNN 1e-1 0 1000 --
2
0
1 DGANN 1e-2 76 240.6 1.71s
-1 0
-1 Average x Average y Average goal
-2 -2

-0.4853 0.5164 11.3167

Figure 5. The image of function 1 -0.0041 0.0018 10.0052

As we know from Tables 1 and Table 2, traditional
BPNN algorithm is inefficient in searching extreme values of
multimodal functions, often gets local optimums. While, the
success rates of DGANN is higher and average iterations is
13 also significantly reduced. The data fully prove the
12 superiority and practicality of the improved algorithm in
11 multi-peak optimization problems.
10 Figure 7 and Figure 8 shows the performance curves of
1
BPNN and DGANN optimizing function 2. It is thus clear
0.5 1 that the improved algorithm has higher convergence speed,
0 0.5 better precision and a greater probability of escaping from
-0.5
0 local optimums.
-0.5
-1 -1

Figure 6. The image of function 2

In order to eliminate the influence of random factors in

single operation, the program runs 100 times for each
function. And the maximum iterations is 1000 to ensure
accuracy. The required precision is 1 for function 1 which is

1474
 Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, Qingdao, 11-14 July 2010

12
Acknowledgements
11.95

11.9
This paper is supported by fund of colleges and
11.85 universities youth academic backbone support project:
optimal value

11.8 (1152G001).
11.75

11.7 References
11.65

11.6 [1] L. N. Xu. “Neural Network Control”, Beijing:

11.55
Electronics Industry Press, 2004.
0 100 200 300 400 500 600 700 800 900 1000
[2] M. Y. Li, Y. L. Du. “Complex Neural Network Adaptive
step
Temperature Control System Based on Genetic
Figure 7. The performance curve of BPNN for optimizing Function 2 Algorithm”, Control Theory and Applications, Vol. 21,
No. 2, pp. 242-246, 2004.
11
[3] G. j. Yang, P. Y. Cui, L. L. Li. “Applications and
10.9 Implementation of Genetic Algorithm in Neural
10.8 Network Control System”, System Simulation, Vol. 13,
10.7 No. 5, pp. 567-570, 2001.
optimal value

10.6 [4] Qing Wang, G. F. Ma, Man Mi. “A Neural Network

10.5
Control Method Based on Genetic Algorithm”, System
10.4
Simulation, Vol. 18, No. 4, pp. 1070-1072, 2006.
10.3
[5] X. R. Shi, M. L. Zhang. “A Quasi-human Intelligent
Control Method Based on Neural Network and Genetic
10.2
Algorithm”, System Simulation, Vol. 16, No. 8, pp.
10.1
1835-1838, 2004.
10
0 100 200 300 400 500 600 700 800 900 1000 [6] Wei Pan, H. Z. Diao, Y. W. Jing. An Improved Real
step Numbers Adaptive Genetic Algorithm [J]. Control and
Decision, 2006, 21 (7): 792-800.
Figure 8. The performance curve of DGANN for optimizing Function 2 [7] C. Z. Chen, Nan Wang. “Adaptive Methods and
Excitation Selection in Crossover and Mutation of
5. Conclusions Genetic Algorithm”, Control Theory and Applications,
Vol. 19, No. 1, pp. 41-43, 2002.
[8] Ling Wang, Xuan Huang, D. Z. Zheng. “A Class of
In this paper a neural network structure DGANN is
Genetic Algorithms with Selection Strategy and Its
designed with combined advantages of diploid genetic
algorithm and neural network. And a method of DNN Performance Analysis”, Control and Decision, Vol. 19,
weights adjustment based on back-propagation algorithm is No. 11, pp. 1290-1297, 2004.
presented. An improved genetic algorithm is adopted for the [9] Fei He. “Reserch on Completely Recessive Diploid
training of neural networks, which combines the nonlinear Genetic Algorithm with Multi-objective Optimization”,
mapping ability of neural network and fast global Beijing University of Technology, 2002.
convergence of genetic algorithm, and has good robustness
and self-learning ability. Homologous chromosomes
crossover and gametes recombination is employed by
imitating the reproductive processes of diplont to improve the
conventional genetic operation. The proposal is applied to
optimize the complex functions and the simulation results
show that the improved algorithm not only can maintain the
diversity of gene population, effectively inhibit the premature
convergence of the algorithm, but also increase the
convergence speed and optimization accuracy.

1475