MULTIDISCIPLINARY SYSTEM

DESIGN Optimization
CS492

Department of Computer Sciences

 Faculty of Information Technology and Computer Sciences
 Yarmouk University,  Irbid-Jordan
Optimization
An optimization is the act of achieving
the best possible result under given
circumstances.
Primary objective may not be optimize
absolutely but to compromise effectively
& thereby produce the best formulation under a
given set of restrictions .
CURSE OF DIMENSIONALITY

Looks complicated …

 
 Consider problem with n design variables
 Sample each variable with m samples
 Number of computations required: mn
 
 
 
 
 
 
 
 
 

Take 1 s per computation,

 
 
 
 

10 variables, 10 samples:
 
 
 
 
 

total time 317 years!

 
 
 
 
 
 
 
 
 
 
 
Parallel computing
 Still, for large problems,
optimization requires lots of
computing power
 Parallel computing
 OPTIMIZATION IN THE DESIGN
PROCESS
Conventional design process:
Optimization-based design process:
Identify: Collect data to describe the
1. Design variables system
Collect data to describe the
2. Objective function system
3. Constraints Estimate initial design
Estimate initial design
Analyze the system
Analyze the system
Check performance criteria
Check the constraints

Is design
Does satisfactory?
the design satisfy Done
convergence criteria? Done
Change design based on
experience
Change the/ design
heuristics / wild
using an
guessesmethod
optimization
 OVERVIEW
 Optimization deals with betterment and improvement.
 Optimization is defined as “the process of achieving the most favorable system
condition on the basis of a metric or set of metrics [Merriam-Webster, 1998].

 Optimization means the maximizing or minimizing of a given function possibly

subject to some type of constraints and controlled by decision variables.
 OVERVIEW
 Broadly,
 The efforts and processes of making a decision, a design or a system as perfect,
effective or functional as possible.
 Narrowly,
 The specific methodology, techniques and procedures used to decide on the one
specific solution in a defined set of possible alternatives that will best satisfy a
selected criterion.

“The main aim of OPTIMIZATION is to construct a model that can be easily

understood and that provides good solutions in a reasonable amount of computing time”
 OVERVIEW
 Optimization, in engineering interpretation, is
referred as the process of finding appropriate
solutions with the intention of finding the best
solutions to the system design problem.
 Optimization is a computational design
method which helps us select an optimal design
among a number of (or an infinite set of )
possible options, such that a certain requirement
or a set of requirements is best satisfied.
 OVERVIEW
 When constraints have been established, it is necessary to determine if there are
any solutions to the problem that simultaneously satisfy all of the constraints.
Any such solution is called a feasible solution.

 A key step in the formulation of any optimization problem is the assignment of

performance measures that are to be optimized.

“The success of any optimization result is critically dependent

on the selection of meaningful performance measures”
 OPTIMIZATION------WHY?????
YES
EVALUATI
DESIGN ANALYSIS
ON FINE

HOW TO MODIFY THE

DESIGN????

MODIFY NO

WITH
THE WITH
EXPERIENCE….TRI

DESIGN
AL and ERROR
OPTIMIZATION
METHODS
 Why Optimization is necessary?

Reduce
the cost

Save the optimization

Safety &
time reduce the
error

Innovation
3 reproducibility &
efficiency
 OPTIMIZATION------WHY?????

Is there one aircraft which is the

fastest, most efficient, quietest, most
inexpensive, most light weight ??????
OPTIMIZATION------WHY?????
 OVERVIEW
 Informally, but rigorously, we can say that design optimization involves:
 1. The selection of a set of variables to describe the design alternatives.
 2. The selection of an objective (criterion), expressed in terms of the design
variables, which we seek to minimize or maximize.
 3. The determination of a set of constraints, expressed in terms of the design
variables, which must be satisfied by any acceptable design.
 4. The determination of a set of values for the design variables, which minimize (or
maximize) the objective, while satisfying all the constraints.

 This definition of optimization suggests a philosophical and tactical approach

during the design process.
 OVERVIEW
 It is not a phase in the process but rather a pervasive viewpoint.

 Philosophically, optimization formalizes what humans (and designers) have

always done.

 Operationally, it can be used in design, in any situation where analysis is used,

and is therefore subjected to the same limitations.
 OPTIMIZATION MODEL
DESIGN VARIABLES
 A design variable is a decision variable or a control variable that can be changed
by designers during a design process. It control the value of the objective
function.

 A design variable have an impact on the performances of a design.

 Different combinations of design variables represent different designs.

 Design variables should be as independent of each other as possible.

Optimization is the process of choosing the design

variables that yield an optimum design
 OPTIMIZATION MODEL
DESIGN VARIABLES
 Design variables are also known as design parameters and will be represented by the
vector x. They are the variables in the problem that we allow to vary in the design
process.
 During design space exploration or optimization we change the entries of x in some
rational fashion to achieve a desired effect.
 OPTIMIZATION MODEL
DESIGN CONSTRAINTS
 Few practical engineering optimizations problems are unconstrained.
 We can not optimize our objectives infinitely because we have limited resources.
 Constraints on the design variables are called bounds and are easy to enforce.
 Constraints are restrictions or requirements imposed to a design.
 A constraint function is expressed in a mathematical form in terms of design variables.
 gi(d)<0, i=1,2,…,n
 hj(d)=0, j=1,2,…,n
 mk(d)>0, k=1,2,…,n
 OPTIMIZATION MODEL
DESIGN CONSTRAINTS
 Constraints that must be satisfied are called rigid constraints or absolute goals.
 Physical variables often are restricted to be nonnegative; for example, the amount of a
given material used in a system is required to be greater than or equal to zero.

 Soft constraints are those constraints that are negotiable to some degree.
 These constraints can be viewed as goals that are associated with target values.
 OPTIMIZATION MODEL
Constraints vs. Objectives
 It can be difficult to choose whether a condition is a constraint or an objective.

 For example: should we try to minimize cost, or should we set a constraint

stating that cost should not exceed a given level.

 The two approaches can lead to different designs.

 Sometimes, the initial formulation will need to be revised in order to fully

understand the design space.

 In some formulations, all constraints are treated as objectives (physical

programming).
 OPTIMIZATION MODEL
OBJECTIVE FUNCTION
 In general, an optimization problem consists of an objective function which
needs to be minimized or maximized, have some constraints and controlled by
design variables.

 Objective function is a “measure of goodness” that enables us to compare two

designs quantitatively.
 OPTIMIZATION MODEL
OBJECTIVE FUNCTION
 The objective function is the goal of the optimization.

 For example, to minimize the mass of some structure or to maximize the speed of
an object can be an objective function.

“If we select the wrong goal, it doesn’t matter how good

the analysis is, or how efficient the optimization method
is”
 OPTIMIZATION MODEL
OBJECTIVE FUNCTION FORMULATION
 Optimization techniques are used to find a set of design parameters,
x = [x1, x2, .., xn], that can in some way be defined as optimal.

 In a simple case this might be the minimization or maximization of some system

characteristic that is dependent on x.

 In a more advanced formulation the objective function, f(x), to be minimized or

maximized, might be subject to constraints in the form of equality constraints
 CONVERGENCE CRITERIA
 Algorithm has converged when….
 No change in the objective function is obtained
OR
 The maximum number of iterations is reached.
Historical development
 Isaac Newton (1642-1727) :The development of differential
calculus methods of optimization.
 Joseph-Louis Lagrange (1736-1813) :Calculus of variations,
minimization of functionals, method of optimization for
constrained problems.
 Augustin-Louis Cauchy (1789-1857) :Solution by direct
substitution, steepest descent method for unconstrained
optimization.
 George Bernard Dantzig (1914-2005):Linear programming
and Simplex method (1947).
 Albert William Tucker (1905-1995):Necessary and sufficient
conditions for the optimal solution of programming problems,
nonlinear programming.
Statement of an optimization problem
 An optimization problem can be stated as follows:

To find X =
which minimizes f(X)

Subject to the constraints

gi(X) ≤ 0 , i = 1, 2, …., m
lj(X) = 0, j = 1, 2, …., p
where X is an n-dimensional vector called the design vector,
f(X) is called the objective function, and gi(X) and lj(X) are
known as inequality and equality constraints, respectively.
 Classification of optimization
Based on Constraints
◦ Constrained optimization
◦ Unconstrained optimization
 Advanced Optimization Techniques
 Hill climbing
Hill climbing is a graph search algorithm where the current
path is extended with a successor node which is closer to the
solution than the end of the current path.
• Simple hill climbing
• Steepest ascent hill climbing
 Simulated Annealing

In the simulated annealing method, each point of the

search space is compared to a state of some physical
system, and the function to be minimized is interpreted
as the internal energy of the system in that state.
Genetic Algorithm:
GAs belong to a class of methods called Evolutionary Algorithms (EA)
that are inspired by the processes of natural selection.

•GAs are different from more traditional optimization techniques

because they search from a population of points rather than a single
point.

Ant Colony Optimization:

An ACO algorithm is an artificial intelligence technique based on the
pheromone-laying behavior of ants; it can be used to find solutions to
exceedingly complex problems that seek the optimal path through a
graph.

•Ant colony optimization algorithms have been used to produce near-

optimal solutions to the traveling salesman problem.

•The ant colony algorithm can be run continuously and can adapt to
changes in real time.