Professional Documents
Culture Documents
Optimization Problem Formulation and Solution Techniques
Optimization Problem Formulation and Solution Techniques
Optimization Problem Formulation and Solution Techniques
OPTIMIZATION PROBLEM
FORMULATION AND SOLUTION
TECHNIQUES
5.1 Introduction
There are many cases in practical applications where the variables of optimization are
not continuous. Some or all of the variables must be selected from a list of integer or
discrete values. For example, structural members may have to be designed using
selected from the commercially available ones. Therefore, considerable interest was
shown for discrete variable engineering optimization problems since the late 1960s and
early 1970s. However, at that time even optimization methods for simpler continuous
nonlinear programming (NLP) problems were still in the process of development. In the
1970s and 80s, a major effort was put into development and evaluation of such
especially for large scale problems, several reliable algorithms are now available for
augmented Lagrangian methods. In recent years, the focus has shifted back to
Optimization Problem Formulation and Solution Techniques 129
continuous variables in their formulation. Among the methods for discrete variable
nonlinear optimization problems, the following techniques have been most commonly
random search techniques have also been applied to discrete optimization problems.
The purpose of this chapter is to define the basic formulation of the design
optimization problem, and survey the most relevant techniques to structural design
It is not just in civil engineering that the search for minimum weight is the main goal,
quantity of material is an important factor in most design fields. Everyone naturally tries
produce better designs has been severely limited by the techniques available for design
programs to evaluate design parameters. Now the question arises why minimum weight
design for steel frameworks built for domestic and residential activities. This question
Clients specify their requirements through a brief. It is essential for effective design to
understand the intentions of the client: the brief is the way in which the client expresses
and communicates these intentions. As far as the designer is concerned, the factors
Optimization Problem Formulation and Solution Techniques 130
which are most important are intended use, budget cost limits, time to completion and
quality. Once these are understood, a realistic basis for producing the design will be
established.
The time taken to realise a steel building from concept to completion is generally less
than that for a reinforced concrete alternative (Owens et al.,1992). This reduces time–
related building costs, enables the building to be used earlier and produces an earlier
return on the capital invested. To gain full benefit from the manufacturer and
cladding and finishes of the building must have similar attributes. In addition, because a
repetition of dimensions, shapes and details will streamline the manufacturing process
Typical cost breakdown is investigated by many authors, e.g. Owens et al. (1992). This
and internal cladding, sunscreens, etc. This stage may cost about 52% of the total
cost.
2. Finishes: this involves ceilings and floors, etc. This stage is calculated as 20% of the
3. Services: this entails electric facilities, lifts plumbing and sprinklers, etc. It is
The designer of a steel framework should aim to achieve minimum overall cost. This
is a balance between the capital cost of the framework and the improved revenue from
early occupation of the building through fast design, fabrication and erection. For a
domestic and residential building, the cost of welding and connections becomes the
Now the question arises what the structural designer could do to provide the client
with an economic design presupposing the intend use of the building as a domestic and
residential activities. Assuming the location of the building is fixed, the cost of
foundation piling can be determined depending of the structural system and the bearing
capacity of the soil. In addition, the cost of finishes and services depends on the
intended use of the building and this can be easily determined. From this discussion, it
becomes clear that for a domestic and residential construction, the minimum weight of
construction becomes a major task for a structural designer. From this point of view,
many researches, among them Grierson and Pak (1993), Adeli and Kumar (1995),
Huang and Arora (1997), Jenkins (1997), Saka (1998) and Camp et al. (1998), have
constraints.
The formulation of the optimization will be addressed and the concept of genetic
has been formulated in terms of some parameters and restrictions. The parameters
chosen to describe the design of a structure are known as design variables while the
Optimization Problem Formulation and Solution Techniques 132
permissible changes of certain design variables denoted by a vector x = {x1 , x2 ,…, xn}.
The design variables in the structural optimization problem might be the cross–sectional
area, the node position, the second moment of inertia, etc. In other words, they are the
parameters that control the geometry of the optimized structure. The design variable can
take either continuous or discrete variables. A continuous variable is one that takes any
value in the range of the variation in its region. A discrete variable is one that takes only
x = ( x1T , x T2 , x Tj ,Λ x TJ ) , j = 1, 2, Λ J
xi , j ∈ D j and (5.1)
Dj = (d ,d ,..., d ).
j, 1 j, 2 j, λ
indicates the number of design variables in each sub–vector and λ is the number of
In structural steelwork design problem, the material design variables and sectional
properties from catalogue are often discrete. In the present study, the standard sections
of universal beams, universal columns and circular hollow sections suggested by the BS
Although the discrete variable problem appears to be easier to solve than the
continuous one (since fewer possible solutions exist), in general, it is more difficult to
solve except in some trivial cases. This is due to the fact that the discrete design space is
The notation of optimization also implies that there are some merit function or functions
that can be improved and can also be used as a measure of effectiveness of the design.
The objective function, merit function, and cost function are names of the function F(x)
being optimized and this function measures the effectiveness of the design. This
follows:
buckling loads, vibration frequency and cost or any combination of these can be used as
objective function. The multicriteria function has different ways commonly used for
reducing the number of functions to one. The first way is simply to generate a composite
objective function that replaces all the objectives. The second way, most common in the
function, for instance the total weight of the structure, and to consider this function to be
the goal of the optimization task. Then, imposed limits, like stresses in each member,
nodal displacements and critical buckling load, etc, are prescribed. The third way is
5.4.3 Constraints
The limits, which take values for the design variables, are known as side constraints.
The side constraints are divided into two types. The first type, commonly used in the
Gs ( x)
~ ≤ 1 , s =1, 2, Λ , s s (5.3)
Gs ( x)
~
where G s ( x ) and G s ( x ) are the calculated and limited values of constraints and s s is
In the design optimization problem, not all constraints are functions of one term
G s,1 ( x ) G s, 2 ( x ) G s, ss ( x )
~ + ~ + ... + ~ ≤1 (5.4)
G s,1 ( x ) G s, 2 ( x ) G s, ss ( x )
From the above sections, the final formulation of the optimization problem can be
mathematically represented by
Optimization Problem Formulation and Solution Techniques 135
Minimize F(x)
G ( x)
subjected to: ~ s ≤ 1 , s =1, 2, Λ , s s
Gs ( x)
x = ( x1T , x T2 , x Tj , Λ x TJ ) , j = 1, 2, Λ J (5.5)
xi , j ∈ D j and
Dj = (d ,d ,...d ).
j, 1 j, 2 j, λ
example, a nonlinear function F(x) of two design variables x1 and x2 with three
nonlinear constraints G1(x), G2(x),and G3(x) can depicted as shown in Figure 5.1.
G1(x)
x1 G2(x)
F(x)
G3(x)
x2
It is important to highlight some of the features of the discrete nonlinear problem (5.5).
First, any of the inequality constraints may not be active at the optimum point because
the constraint surface may not pass through any of the discrete points, i.e. in numerical
calculations only a point closest to the constraint boundary may be found. Second, there
process. Thus, local optimality of the solution point can not be assured unless an
exhaustive search is performed. Third, the size of discreteness and nature of the discrete
values may govern the behaviour of some of the numerical algorithms as well as the
final solution of the problem. Fourth, the design problem is highly non–linear problem
due to the nature of design variables and the relationships between the constraint
functions and design variables. Fifth, constraints have different formulation for different
members of the structure. For example, a structure has beams, columns, and a bracing
system. The constraints that control the design of beams are different from those of
bracing systems or columns. Moreover, the set of catalogue sections for beams are
different from those of bracing systems or columns. Sixth, the computational effort
needed to reach satisfactory results increases with the complexity of the treated design
A review of the methods for discrete variable optimization was recently presented by
Bremicker et al. (1990), Vanderplaats and Thanedar (1991) and Arora et al. (1994).
Several algorithms for discrete optimization problems were developed, among them
branch and bound method, penalty function approach, rounding–off, cutting plane,
methods. It is observed that some of the methods for discrete variable optimization use
the structure of the problem to speed up the search for the discrete solution. This class of
methods is not suitable for implementation into a general purpose application (Arora et
al., 1994). The branch and bound method, simulated annealing, and genetic algorithm
Optimization Problem Formulation and Solution Techniques 137
are the most used methods. Herein, the literature review will be focused on these
The branch and bound method BBM is perhaps the most widely known–method for
mixed–discrete optimization problems. The method was originally developed for LP,
however it is quite general and can be applied to nonlinear discrete and mixed variable
point for the problem assuming all variables to be continuous. Then, each variable is
assigned a discrete value in sequence and the problem is solved again in the remaining
variables. The process of assigning discrete values to variables need not start from a
continuous optimum point although this approach may reduce the number of times the
problem needs to be re–solved to obtain a feasible discrete point and subsequently the
optimum solution. It can be seen that the number of times the problem needs to be re–
solved increases exponentially with the number of variables. Several procedures have
been devised to reduce this number. The first use of the branch and bound method is
attributed to Land and Doig (1960) for linear problem. Other attempts, to use BBM to
solve integer LP problems related to the plastic design of frames, made by Reinschmidt
(1971).
BBM was combined with exterior penalty functions and SQP methods to treat the
mixed–discrete NLP problem. John et al. (1988) combine BBM with sequential
linearization for discrete optimal design of trusses. Hajela and Shih (1990) used BBM to
Salajegheh and Vanderplaats (1993) used BBM for optimizing trusses with discrete
sizing and shape variables. Large storage space was needed and an exponential growth
Optimization Problem Formulation and Solution Techniques 138
problems.
the discrete variables are forced to take discrete values from the specified set. The
logical structure for the set of solutions is that of a tree for each variable. Initially an
optimum point is obtained by treating all design variables as continuous. If this solution
is discrete, then the process is terminated. If one of the desired variables is not discrete,
Now two sub–problems are defined, one with the constraint x i , j ≤ d j , λ and the
other with x i , j ≤ d j , λ+1 . This process called branching. It basically eliminates some
portion of the continuous feasible region, which is not feasible for the discrete problem.
It does not, however, eliminate any of the discrete feasible solutions. The two sub–
problems are solved again, and the optimum solutions are stored as nodes of the tree
containing optimum values of the variables, the objective function and the appropriate
bounds on the variables. This process of branching and solving continuous problems is
continued until a feasible solution is obtained. The cost function corresponding to this
solution becomes an upper bound on the optimum solution. From this point, all the
nodes of the tree that have cost function values higher than the established upper bound
are eliminated from further consideration. Such nodes, known as fathomed nodes, when
their lowest point is reached and no further branching is necessary from them. This
process is known as bounding. The process of branching and bounding is repeated from
each of the unfathomed nodes. From each node, at most two new nodes may originate.
A better upper bound for the optimum objective function is established when a feasible
Optimization Problem Formulation and Solution Techniques 139
discrete solution is obtained with a value of objective function less than the current
upper bound. The nodes may be fathomed in any of the following three ways: (1) a
feasible discrete solution to the continuous solution with the cost function value higher
than the current upper bound, (2) an infeasible continuous problem and (3) the optimal
value of the objective function for the continuous problem higher than the upper bound.
This method has been used successfully, however, for problems with a large
number of discrete design variables, the number of sub–problem nodes becomes large
making the method inefficient (Arora et al., 1994). This drawback is oppressive for
developed can be found for the BB in Mesquita and Kamat (1987), Ringertz (1988),
Simulated annealing (SA) is one of the techniques, which do not require derivatives of
the problem functions because it does not use any gradient or Hessian information. The
programming problems.
occurrence. Elperin (1988) applied the SA to design a ten–bar truss where member
cross–sectional area were to be selected from a set of discrete values. Kincaid and
Padula (1990) used SA for minimizing the distortion and internal forces in a truss
structures using SA. One year later, the same framework was studied using the filtered
Optimization Problem Formulation and Solution Techniques 140
simulated annealing algorithm by May and Balling (1992). Recently, Leite and Topping
5.6.3.1 Background
The famous naturalist Charles Darwin defined natural selection or survival of the fittest
in his book (Darwin, 1929) as the preservation of favourable individual differences and
variations, and the destruction of those that are injurious. In nature, individuals have to
those features that make an individual more suitable to compete are preserved when it
reproduces, and those features that make it weaker are eliminated. Such features are
controlled by units called genes, which form sets known as chromosomes. Over
subsequent generations not only the fittest individuals survive, but also their genes
which are transmitted to their descendants during the sexual recombination process,
In the late 60s, John H. Holland became interested in the application of natural
that allowed computer programs to mimic the process of evolution. This technique
became popular after the publication of his book (1975). He renamed this technique
using the term genetic algorithm (GA). The main goals of the research of Holland and
natural systems.
Optimization Problem Formulation and Solution Techniques 141
Basically, the central thrust of the research on genetic algorithms (GAs) has been
due to its robustness and the balance between efficiency and efficacy necessary for
survival in many different environments. GAs are search algorithms, which are based on
the mechanics of natural selection and survival of the fittest, and unlike many
with an associated fitness value, into a new population (i.e., the next
reproduction and survival of the fittest and after naturally occurring genetic
The GA–based techniques accept discrete and/or continuous design variables and
therefore are very versatile. GAs are different from most optimization techniques in
many ways:
• GAs work on a coding of the design variables–binary bit string representation is one
of such coding, rather than the design variables themselves. This characteristic
• GAs proceed from several points in the design space to another set of design points.
• GAs work on the function evaluations alone and do not require any of the function
toward the optimum, the derivative also directs the search process towards a local
optimum, and
highly exploitative search. To this extent, GAs should not be considered as a variant
The general features of the theory of the GA are widely accepted and applied,
which result in good solutions for different types of problems in different disciplines.
Holland (1975).
1. Evolution is a process that operates on chromosomes rather than on the living beings
they encode.
2. Natural selection is the link between chromosomes and the performance of their
3. The process of reproduction is the point at which evolution takes place. Mutations
may cause the chromosomes of biological children to be different from those of their
parents.
individuals who will function well in their environment is contained in the gene
pool, the set of chromosomes carried by the current individuals, and in the structure
GAs are implicit enumeration procedures. A set of randomly created design alternatives
and cross among themselves, with bias allocated to the most fit individuals. A
results in progenies that are more fit than their parents. Therefore, if a measure which
indicates the fitness of a generation is also the desired goal of a design process,
The technique for encoding solutions may vary from problem to problem and from
carried out using bit strings, 0 and 1. A major task is the encoding of different design
variable. The member section is then represented by a bit string. Each bit–string is then
merged to form chromosomes, which represent a design set. In the present study, the
possible cross–sections of each design variable are presented in binary strings. The bit–
string is associated with a position in the table and its corresponding sectional
in Figure 5.2. This frame has two design variables x1 and x2, which represent the
columns and the beam girder respectively. The variable x1 takes a position out of 32 UCs
x2
x1 x1
table and thus the properties of this section can be selected. The cross section, selected
from the catalogue, can be represented in the binary code according to the number of the
available cross sections. The string length of each design variable λnv should be
evaluated by
λ nv = 2n . (5.7)
For instance, there are 64 types of UBs, so the number of bits required to distinguish the
range is 6. A part of the encoded variables for UBs are listed in Table 5.1. Similar
encodings for the utilised UCs and CHS can be drawn. Hence, for the given framework
shown Figure 5.2, the chromosomes given in Figure 5.3 represent an individual in the
population. This can be read as a design in which the design variable x1 takes the
position of number 8 in the table of UCs while x2 takes the position of number 6 in the
table of UBs.
0 0 1 1 1 0 0 0 1 0 1
x1 x2
GAs, and also real numbers and other alphabets have been proposed by Wright (1991).
The term “floating” may seem misleading since the position of the implied decimal
point is at a fixed position, and the term fixed point representation seems to be more
optimized, may have a point at any position along the string. This means that even when
the point is fixed for each gene, it is not necessarily fixed along the chromosomes.
Therefore, some variables could have a precision of 32 decimal places, while others are
integers. As Eshelman and Schaffer (1993) point out, many researchers in the GA
Optimization Problem Formulation and Solution Techniques 146
community agreed to use real coded GAs for numerical optimization despite the fact
that there are theoretical arguments that seem to show that small alphabets should be
more effective than large alphabets. Muhlenbein and Schilierkamp–Voosen (1993) also
Over other encodings, bit strings have several advantages that can be summarised as
follows:
2. They are theoretically tractable, in that their simplicity makes it easy to prove
theorems.
3. Performance theorems have been proved for bit string chromosomes that
4. Just about anything can be encoded in bit strings, so one–point crossover and
In a simple GA, one starts with a randomly created set of designs. From this set, new
and better designs are reproduced using the fittest members of the set. The entire process
are conceived, born and raised until they are ready to reproduce. A simple GA is
Reproduction is an operation where an old string is copied into the new population
according to the string fitness. Here, fitness is defined according to the objective
function value. More fit strings, i.e. those with smaller objective function values, receive
Optimization Problem Formulation and Solution Techniques 147
algorithmic form in a number of ways. Perhaps the easiest way is to create a biased
roulette wheel (see DeJong, 1975) where each current string in the population has a
roulette wheel slot sized in proportion to its fitness. Consequently, each time we require
offspring, a simple spin of the weighted wheel yields the reproduction candidate. In this
way, more highly fit strings have a higher number of offspring based on the probability
Fi
Pisel = Np
(5.8)
Fj
j =1
where Fi is the value of the objective function of the individual i–th and N p is the
Once a string has been selected for reproduction, an exact replica of the string is
made. This string is then entered into a mating pool, a tentative new population, for
Other selection schemes can be used, among them stochastic remainder selection
Ranking selection is presented by Baker (1985) in which the population is sorted from
best to worst, and each individual is copied as many times as possible. According to a
selection in which the population is shuffled and then is divided into groups of ng
elements from which the best individual, i.e. the fittest, will be chosen. The number of
NP
N= . (5.9)
ng
themselves (Arora et al., 1994). Simple crossover may proceed in two steps. First,
members of the newly reproduced strings in the mating pool are mated at random.
Second, each pair of strings undergoes crossover. Several ways of performing crossover
are used in the literature, the most simple one termed one–point crossover Goldberg
(1989). This can be illustrated as follows: an integer position b along the string as
indicated in Figure 5.4a is selected uniformly at random between 1 and the string length
less one [1, λnv − 1 ]. Two new strings termed children are created by swapping all
characters between positions b+1 and λnv of the parents inclusively. Figure 5.4b shows
In order to find an effective search, Eshelman et al. (1989) and Syswerda (1989)
use different types of crossover such as segment crossover, uniform crossover, shuffle
Mutation is the third step in simple GA, and this step safeguards the process from
crossover. In terms of binary string, this step corresponds to selecting a few members
of the population, determining at random a location on the strings, and switching the 0
children as given in Figure 5.5. The 2nd bit was randomly selected over the child 1
Optimization Problem Formulation and Solution Techniques 149
while it is the 6th bit over the child 2. Then, the procedure is to change the 1 to a 0 and
Cross–point Cross–point
Parent 1 0 0 0 1 0 1 1 1 1 0 1 0 1 1 1 0 Parent 2
Child 1 1 0 1 1 0 1 1 1 0 0 0 0 1 1 1 0 Child 2
Cross–points Cross–points
Parent 1 0 0 0 1 0 1 1 1 1 0 1 0 1 1 1 0 Parent 2
Child 1 0 0 1 0 1 1 1 1 1 0 0 1 0 1 1 0 Child 2
1 0 1 1 0 1 1 1 0 0 0 0 1 1 1 0
1 1 1 1 0 1 1 1 0 0 0 0 1 0 1 0
It is observed that the mutation is a random walk through the string space. When
The foregoing three steps are repeated for successive generations of the
population until no further improvement in the fitness is attainable. The member in this
Figure 5.6 illustrates the flowchart for a simple GA linked to a structural design
problem. At the beginning, all the necessary data – GA parameters and structural
geometry – will be read and the process of the GA will start for the first generation.
The initial population will be generated randomly. Then, the objective function
regarded as the weight of the structure as well as the constraint functions, which are
reflected on the design criteria requested by BS 5950, are computed. At this stage, the
average, maximum and the fittest design are obtained. Convergence criteria described
later are also checked. The GA process is terminated if the convergence is achieved.
Otherwise, the GA process resumes. By creating the mating pool and applying the GA
operators, the next population is created. The GA process will proceed until either the
Start
Design set i
New generation No
Design set =Np? New design
Yes
Yes
Convergence Stop
occurred?
No
Store the best individuals, and impose them into the next
generation and carry out the crossover and mutation
Figure 5.6. Flowchart for genetic algorithm linked to structural design problem
Optimization Problem Formulation and Solution Techniques 152
GAs have traditionally been applied to unconstrained problems as they have no built–in
method to handle constraints. Constraints can be classified as two types: explicit and
implicit. Explicit constraints are those that can be checked without a system simulation.
Cost is often one example of an explicit constraint. Implicit constraints require a system
simulation i.e. analysis and design checks. For example, cross sections have design
criteria as requested by the code of practice, therefore, a system simulation must be run
before this information can be ascertained. Several approaches have been used to handle
constraints including:
Specialised operators work only for explicit constraints, and are useful for those
problems such as the travelling salesman problem. The second and third approach can
approach, eliminating those designs from the population that violate one or more
constraints, can be very ineffective for large problems that have few viable solutions
compared with the number of infeasible ones. The most prevalent technique for coping
with constraint violations is to penalise a population member for one or more violations.
The main difficulty in applying penalty functions is that they are generally problem
dependent. Different techniques of employing penalty functions are used in the literature
among them Moe (1973), Fletcher (1975), Haftka and Starnes (1976), Shin et al. (1990),
Hajela and Yoo (1995), Huang and Arora (1997), and Camp et al. (1998). Generally, the
problems attempted using GAs are all of the constrained optimization type, and
Optimization Problem Formulation and Solution Techniques 153
This can be dealt with using a penalty–based transformation method (Hajela and Yoo,
−
Minimize F ( x , r ) = F ( x ) + P(r ,G ( x ) , H ( x )) (5.10)
−
where F is the modified objective function that also contains the penalty term P ,
which brings the constraint functions into the problem and r is called a penalty
multiplier. The way in which the penalty parameters and the constraint functions are
combined and the rules for updating the penalty parameters specify the particular
method.
In the present work, the design optimization problem has been attacked differently
because careful consideration must be given to the selection of the penalty function, and
in the present context, the "exact" penalty function is used. This results in the following
definition of the fitness function combined with the simple "exact" penalty function:
where C is a constant evaluated at each generation. The technique used for penalty
optimization. In the present study, three criteria are used and if any of them are satisfied,
• If the fittest design has not changed for 30 successive generations, or if the
difference between the fittest design F cu of the current generation and that of 30
generations before is very small value C cu . This could be expressed in the form
F cu − F cu −30
cu
≤ C cu . (5.12)
F
• As we proceed with more generation the population gets filled by more fit
individuals, with perhaps a very small deviation from the fitness of the best
individuals. Consequently, the average fitness comes very close to the fitness of the
best design. This could result in another convergence criterion such that the
percentage difference between the average fitness F av of the current population and
the current fitness of the best design F cu reaches a very small value C av . This can
be expressed by
F cu − F av
cu
≤ C av . (5.13)
F
• The simplest one is when a total allocated number of generations ( gen max = 200 )
are reached.