Optimization Techniques

Introduction to Optimization

Dr. Nasir M Mirza

Email: nasirmm@yahoo.com
1

Books
Recommended Books:
1.

Kalyanmoy Deb, optimization for engineering design,

algorithms and examples, Prentice Hall (2005).

2.

M. Asghar Bhatti Practical Optimization Methods With

Mathematics Applications, Springer-Verlag N.Y, Inc., 2000.

nasir m mirza

Slide

Optimization
Optimization is derived from the Latin word
optimus, the best and characterizes the
activities involved to find the best.
People have been optimizing for ages, but the
roots for modern day optimization can be
traced to the Second World War.

nasir m mirza

Slide

Operational Research
Operational
Operational Research
Research was
was defined
defined by
by the
the Operational
Operational
Research
ResearchSociety
Societyof
ofGreat
GreatBritain
Britainas
asfollows:
follows:
"Operational
"Operational research
research isis the
the application
application of
of the
the methods
methods of
of science
science
to
to complex
complex problems
problems arising
arising inin the
the direction
direction and
and management
management of
of
large
large systems
systems of
of men,
men, machines,
machines, materials
materials and
and money
money inin
industry,
industry, business,
business, government,
government, and
and defense.
defense. The
The distinctive
distinctive
approach
approach isis to
to develop
develop aa scientific
scientific model
model of
of the
the system,
system,
incorporating
incorporating measurements
measurements of
of factors
factors such
such as
as chance
chance and
and risk,
risk,
with
with which
which to
to predict
predict and
and compare
compare the
the outcomes
outcomes of
of alternative
alternative
decisions,
decisions, strategies
strategies or
or controls.
controls. The
The purpose
purpose isis to
to help
help
management
managementdetermine
determineits
itspolicy
policyand
andactions
actionsscientifically."
scientifically."

nasir m mirza

Slide

Use of Computers
Without computers, Operation Research and optimization
would not be what they are today.
Earlier mathematical models (such as calculus, Lagrange
multipliers) relied on sophistication of technique to solve
the problem.
Methods of mathematical optimization (e.g., Linear
Programming) rely far less on mathematical
sophistication than they do on an unusual adaptability
to the mode of solution inherent in the modern digital
computer.
The simplicity of these methods of mathematics coupled with
their iterative processes makes them very useful.

nasir m mirza

Slide

Computers and Linear Programming

Consider the case of Linear Programming
The first large scale computer became available in
1946 at the University of Pennsylvania, USA.
This was just one year before the development of
simplex method.
The simplex method for linear programming consists
only of a few steps and these steps require only the
most basic mathematical operations which a computer
is well suited to handle.
However, these steps must be repeated over and over
before one finally obtains an answer.
The first successful computer solution of a LP problem
was in January 1952 on the National Bureau of
Standards SEAC computer in USA.
nasir m mirza

Slide

Need for Optimization

Optimization problems arise naturally in many different
disciplines. In engineering design problems often goal is
to maximize or minimize certain parameter or a variable.
For example optimization algorithms are often used in
aerospace design activity to minimize overall weight.
For chemical engineers design and operation of a
process plant should have optimal rate of production.
A mechanical engineer designs a component to
minimize the cost or to maximize the component life.
An electrical engineer designs a network to achieve
minimum time for communications.
nasir m mirza

Slide

Need for Optimization

A structural engineer designing a multi-story building
must choose materials in the building with a safe
structure and an economical as well.
A portfolio manager for a large mutual fund company
must choose investments that generate the largest
possible rate of return for its investors
A plant manager in a manufacturing facility must
schedule the plant operations such that the plant
produces products that maximize company's revenues.
A scientist in a research laboratory may be interested in
finding a mathematical function that best describes an
observed physical phenomenon.
nasir m mirza

Slide

Need for Optimization

All these situations have the following three

things in common.

There is an overall goal, or objective, for the activity.

For the structural engineer, the goal may be to
minimize the cost of the building,
for the portfolio manager it is to maximize the rate
of return,
for the plant manager it is to maximize the
revenue, and
for the scientist, the goal is to minimize the
difference between the prediction from the
mathematical model and the physical observation.
nasir m mirza

Slide

Need for Optimization

In addition to the overall goal, there are
constraints, that must be met.
The structural engineer must meet safety
requirements dictated by applicable building
standards.
The portfolio manager must keep the risk of major
losses below levels determined by the company's
management.
The plant manager must meet customer demands
and work within available work force and raw
material limitations.
For the laboratory scientist, there are no other
significant requirements.
nasir m mirza

Slide

10

Need for Optimization

In all situations, choices available to meet the goals and
requirements.
The choices are known as optimization variables.
For example, from safety point of view, it does not matter
whether a building is painted purple or pink, and therefore the
color of a building would not represent a good optimization
variable.
On the other hand, the height of one story could be a possible
design variable because it will determine overall height of the
building, which is an important factor in structural safety.

The variables that do not affect the goals are clearly not

important.

nasir m mirza

Slide

11

Optimization
All engineering tasks involve either minimization or
maximization of an objective function.
It will be very difficult to discuss formulation of each
type of engineering optimization problem in one course.
However a designer can learn different types of
optimization techniques and latter choose optimal
algorithm for his or her problem.
First we shall learn optimal problem formulation.

nasir m mirza

Slide

12

Optimization Problem
Some define the formulation of problem as
taking statements,
defining general goals and requirements of a given
activity, and
converting them into a series of well-defined
mathematical statements.

Others say the formulation of Opt. problem is:

1. Selecting one or more optimization variables,
2. Choosing an objective function, and
3. Identifying a set of constraints.

nasir m mirza

Slide

13

 The objective function and the constraints must

all be functions of one or more optimization

variables.

Let us have examples on this.

nasir m mirza

Slide

14

Maximizing Area
EXAMPLE 1: Find the dimensions of the rectangular
garden of greatest area that can be fenced off (all four sides)
with 300 meters of fencing.
SOLUTION

The garden will be in the shape of a rectangle. The perimeter of it

is to be 300 meters. Lets make a picture of the garden, labeling
the sides.
y
x

x
y

Since we know the perimeter is 300 meters, we can now construct

an equation based on the variables contained within the picture.
x + x + y + y = 2x + 2y = 300
nasir m mirza

(Constraint Equation)
Slide

15

Maximizing Area

- CONTINUED

Now, we wish to maximize is area. Therefore, we will need

an equation that contains a variable representing area.
A = xy

(Objective Equation)

Now we will rewrite the objective equation in terms of A

(the variable we wish to optimize) and either x or y. Since it
doesnt make a difference which one we select, we will
select x.
2x + 2y = 300; This is the constraint equation.
y = 150 x
2y = 300 2 ;
Now we substitute 150 x for y in the objective equation so
that the objective equation will have only one independent
variable.
nasir m mirza

Slide

16

Maximizing Area

-CONTINUED

The objective equation is given as

A = xy

Replace y with 150 x.

A = x(150 x)

Finally we get

A = 150x x2

Now we will graph the resultant function;

6000
5000
Area (A)

4000
3000
2000
1000
0
0

50

100

150

nasir m mirza

Slide

17

Maximizing Area

-CONTINUED

Since the graph of the function is obviously a parabola, then the

maximum value of A (along the vertical axis) would be found at the only
value of x for which the first derivative is equal to zero.
The area function is:

A = 150x x2

Differentiating we get: A = 150 2x

Let the derivative equal to zero. 150 2x = 0;

x = 75

Therefore, the slope of the function equals zero when x = 75.

This is the x-value for where the function is maximized. Then,
2x + 2y = 300;

2(75) + 2y = 300;

y = 75

So, the dimensions of the garden will be 75 m x 75 m.

nasir m mirza

Slide

18

Minimizing Cost
EXAMPLE 2:

A rectangular garden of area 75 square feet is to be surrounded on three sides

by a brick wall costing $10 per foot and on one side by a fence costing $5 per
foot. Find the dimensions of the garden such that the cost of materials is
minimized.
SOLUTION

Below is a picture of the garden. The red side represents the side that is fenced.
y
x

x
y

The quantity that we will be minimizing is cost. Therefore, our objective

equation will contain a variable representing cost, C.

nasir m mirza

Slide

19

Minimizing Cost
EXAMPLE 2: -CONTINUED

C = (2x + y)(10) + y(5)

C = 20x + 10y + 5y
C = 20x + 15y

(Objective Equation)

Now we will determine the constraint equation. The only piece of information
we have not yet used in some way is that the area is 75 square feet. Using this,
we create a constraint equation as follows.
75 = xy

(Constraint Equation)

Now we rewrite the constraint equation, isolating one of the variables therein.
75 = xy
75/y = x
nasir m mirza

Slide

20

Minimizing Cost

EXAMPLE 2: -CONTINUED

Now we rewrite the objective equation using the substitution we just acquired
from the constraint equation.
The objective equation is:

C = 20x + 15y

Replace x with 75/y:

C = 20(75/y) + 15y;

Simplify,

Now we use this

equation to
sketch a graph of
the function.

Cost (C)

C = 1500/y + 15y

2000
1800
1600
1400
1200
1000
800
600
400
200
0
0

50

100

150

y
nasir m mirza

Slide

21

Minimizing Cost

EXAMPLE 2: -CONTINUED

It appears from the graph that there is exactly one relative

extremum, a relative minimum around x = 10 or x = 15.
To know exactly where this relative minimum is, we need
to set the first derivative equal to zero and solve
The object function is:

C = 1500/y + 15y

Differentiating we get:

C = -1500/y2 + 15

Set the function equal to 0.

-1500/y2 + 15 = 0;
15y2 = 1500
y2 = 100;
nasir m mirza

y = 10.
Slide

22

Minimizing Cost

EXAMPLE 2: -CONTINUED

Therefore, we know that cost will be minimized when y =

10. Now we will use the constraint equation to determine
the corresponding value for x.
The constraint equation is:

75 = xy

Replace y with 10:

75 = x(10)

Let us solve for x:

7.5 = x

So the dimensions that will minimize cost,

are x = 7.5 ft and y = 10 ft.
nasir m mirza

Slide

23

Minimizing Surface Area

EXAMPLE 3:

(Volume) A canvas wind shelter for the beach has a back, two
square sides, and a top. Find the dimensions for which the
volume will be 250 cubic feet and that requires the least possible
amount of canvas.
SOLUTION

Below is a picture of the wind shelter.

x
y
x

The quantity that we will be maximizing is surface area.

Therefore, our objective equation will contain a variable
representing surface area, A.
nasir m mirza

Slide

24

Minimizing Surface Area

EXAMPLE 3: -CONTINUED

Sum of the areas of the sides:

A = 2x2 + 2xy

A = xx + xx + xy + xy

(Objective Equation)

Now we will determine the constraint equation. The only piece of

information we have not yet used in some way is that the volume
is 250 ft3. Using this, we create a constraint equation as follows.
250 = x2y

(Constraint Equation)

Now we rewrite the constraint equation, isolating one of the

variables therein.
250 = x2y;
nasir m mirza

250/x2 = y
Slide

25

Minimizing Surface Area

EXAMPLE 3: -CONTINUED

Now we rewrite the objective equation using the substitution we just acquired
from the constraint equation.
A = 2x2 + 2xy

The objective equation is:

Replace y with 250/x2 :

A = 2x2 + 2x(250/x2)

Simplify.

4000
3500

+ 500/x

Now we use this

equation to sketch a
graph of the function.

3000
Area (A)

A=

2x2

2500
2000
1500
1000
500
0
-5

15

25

35

45

x
nasir m mirza

Slide

26

Minimizing Surface Area

EXAMPLE 3: -CONTINUED

It appears from the graph that there is exactly one relative extremum, a
relative minimum around x = 5. To know exactly where this relative
minimum is, we need to set the first derivative equal to zero and solve (since
at this point, the function will have a slope of zero).
A = 2x2 + 500/x
Differentiating we get

A = 4x 500/x2

Then set the function equal to 0. 4x - 500/x2 = 0

4x = 500/x2
4x3 = 500
x3 = 125
x=5

nasir m mirza

Slide

27

Minimizing Surface Area

EXAMPLE 3: -CONTINUED

Therefore, we know that surface area will be minimized when

x = 5. Now we will use the constraint equation to determine
the corresponding value for y.
The constraint equation:

250 = x2y

Replace x with 5.
250 = (5)2y;

y = 10

So the dimensions that will minimize surface area,

are x = 5 ft and y = 10 ft.

nasir m mirza

Slide

28