8 Optimization Techniques

2
Chapter
Classical
Optimization
Techniques

2.1 IntroductIon
In the present chapter, we shall discuss the classical optimization techniques with necessary
and sufficient conditions for obtaining the optimum solution of unconstrained single and
multivariable optimization problems. Constrained multivariable problems with equality and
inequality constraints have also been discussed in detail with examples. The classical optimization
techniques are very useful to obtain the optimal solution of problems involving continuous and
differentiable functions. Such types of techniques are analytical in nature to obtain maximum and
minimum points for unconstrained and constrained continuous objective functions. For equality
constrained problems we use Lagrange’s multiplier method and for inequality constrained, the
Kuhn-Tucker conditions, for getting optimum solutions.

2.2 unconstraIned optImIzatIon problems

2.2.1  Single Variable Optimization Problems
Let f (x) be a continuous function of single variable x defined in interval [a, b].

Local Maxima
A function f (x) with single variable is said to have a local (relative) maxima at x = xo if
f (x0) ≥ f (x0 + h)
for all sufficiently small positive and negative values of h.

Local Minima
A function f (x) with single variable is said to have a local (relative) minima at x = xo if
f (x0) £ f (x0 + h)
for all sufficiently small positive and negative values of h.
 Classical Optimization Techniques 11

Step 1: Differentiate f (x1, x2) partially with respect to x1 and x2, we get
∂f ∂f
and
∂x1 ∂x2
Step 2: For extreme points, we have
∂f
=0
∂x1
∂f
and =0
∂x2
Solving these equations we get some points as (a1, b1), (a2, b2),…etc.
Step 3: Differentiate again partially to get
∂2 f ∂2 f ∂2 f
r= , s = and t=
∂x12 ∂x1∂x2 ∂x22
2
Step 4: At (a1, b1), calculate r and rt – s
Case I: If rt – s2 > 0 and r < 0 then f (x1, x2) is maximum at (a1, b1).
Case II: If rt – s2 > 0 and r > 0 then f (x1, x2) is minimum at (a1, b1).
Case III: If rt – s2 < 0 then f (x1, x2) has neither maxima nor minima at (a1, b1).
Case IV: If rt – s2 = 0 then f (x1, x2) has point of inflexion at (a1, b1).

2.3.2  Working Rule to Find the Extreme Points of Functions of n-Variables
Let us consider u = f (x1, x2, x3, …, xn) as a function of x1, x2, x3, …, xn.

Necessary Conditions
∂f ∂f ∂f
= 0, = 0, …, =0
∂x1 ∂x2 ∂xn

Sufficient Conditions
The Hessian matrix at P for n variables will be
È ∂2 f ∂2 f º ∂2 f ˘
Í 2 ˙
Í ∂x1 ∂x1∂x2 ∂x1∂xn ˙
Í ∂2 f
Í ∂2 f º ∂2 f ˙˙
Í ∂x ∂x ∂x22 ∂x2 ∂xn ˙
H= Í 2 1 ˙
ͺ º º º ˙
ͺ º º º ˙
Í ˙
Í ∂2 f ∂2 f º ∂2 f ˙
Í ∂x ∂x ∂xn ∂x2 ˙
Î n 1 ∂xn2 ˚
12 Optimization Techniques

Its leading minors are defined as

∂2 f
H1 = ,
∂x12

∂2 f ∂2 f
H2 = ∂x12 ∂x1∂x2
,
∂2 f ∂2 f
∂x2 ∂x1 ∂x22

∂2 f ∂2 f ∂2 f
∂x12 ∂x1∂x2 ∂x1∂x2
∂2 f ∂2 f ∂2 f
H3 = ,
∂x2 ∂x1 ∂x22 ∂x2 ∂x3
∂2 f ∂2 f ∂2 f
∂x3∂x1 ∂x3∂x2 ∂x32

Hence, the following cases will arise:

Case I: If H1, H2, H3,… are positive (i.e., H is positive definite) then f (x1, x2, x3,…, xn) has
minimum at P.
Case II: If H1, H2, H3,… are alternately negative, positive, negative (i.e., H is negative definite)
then f (x1, x2, x3,…, xn) has maximum at P.
Case III: H1 and H3,… are not of same sign and H2 = 0 (i.e., semidefinite or indefinite) then
f (x1, x2, x3,… xn) has a saddle point at P.

solved examples
Example 1: Assume the following relationship for revenue and cost functions. Find out at what
level of output x, where x is measured in tons per week, profit is maximum.
R(x) = 1000x – 2x2 and C (x) = x3 – 59x2 + 1315x + 5000.
Solution: The profit function is
P (x) = R(x) – C (x)
= 1000 x – 2x2 – x3 + 59 x2 – 1315x – 5000
= – x3 + 57x2 – 315x – 5000 … (1)
Differentiating both sides of (1) with respect to x, we get
dP
= – 3x2 + 114x – 315 … (2)
dx
For maxima and minima, we have
dP
=0
dx
 Classical Optimization Techniques 13
fi – 3x2 + 114x – 315 = 0
fi x = 3, 35.
Differentiating both sides of (2) again with respect to x, we get
d 2P
= – 6x + 114
dx 2
d 2P
At x = 3, = 96 > 0, i.e., P is minimum at x = 3.
dx 2
d 2P
At x = 35, = 96 < 0, i.e., P is maximum at x = 35.
dx 2
Hence, the profit is maximum at x = 35 tons per week.
Example 2: The profit P earned, by a company, on some item is function of its units produced
say x and is given by
P = 800x – 2x2
If the company’s expenditure or interest, rent and salary of the staff be Rs. 1 lakh, show that
the company will always be in loss.
Solution: Given that
P = 800x – 2x2 … (1)
Differentiate both sides of (1) with respect to x, we get
dP
= 800 – 4x … (2)
dx
For maxima and minima, we have
dP
=0
dx
800 – 4x = 0
x = 200
Differentiate both sides of (2) again with respect to x, we get
d 2P
= – 4 (negative)
dx 2
So, profit (P) is maximum for (x = 200).
The net profit = P – expenditure
= 800(200) – 2(200)2 – 1,00,000
= – 20,000.
Hence, the company will always be in loss.
Example 3: Assuming that the petrol burnt (per hour) in driving a motor boat varies as the
cube of its velocity, show that the most economical speed when going against a current of c
3c
kmph is km per hour.
2
14 Optimization Techniques

Solution: Let the velocity of the boat be v km per hour, the current’s velocity is c km per hour
and the relative velocity of the boat is (v – c) km per hour.
a
If the total distance travelled ‘a’ km then the time taken = hour.
v-c
According to given the petrol burnt in one hour is proportional to v3, i.e. = lv3 where l is a
suitable positive constant.
The petrol burnt for distance ‘a’ is given by
a
P = lv3. … (1)
v-c
Differentiate both sides of (1) with respect to v, we get

dP È 3v 2 (v - c) - v3 ˘
= la Í ˙ … (2)
dv ÎÍ (v - c ) 2 ˙˚

È 2v3 - 3cv 2 ˘
= la Í 2 ˙
ÍÎ (v - c) ˙˚
For maxima and minima, we have
dP
=0
dv
È 2v3 - 3cv 2 ˘
fi la Í 2 ˙
=0
ÍÎ (v - c) ˙˚
fi 2v3 – 3cv2 = 0
3c
fi v = 0,
2
Differentiate both sides of (2) again with respect to v, we get
d 2P È (6v 2 - 6vc)(v - c) 2 - (2v3 - 3cv 2 ).2(v - c) ˘
= la Í ˙
dv 2 ÍÎ (v - c ) 4 ˙˚

d 2P È 6v(v - c)2 - 2v 2 (2v - 3c) ˘

= la Í ˙
dv 2 ÍÎ (v - c )3 ˙˚
d 2P
At v = 0, = 0, i.e., P = 0 fi no petrol is burnt.
dv 2
3c d 2 P 3c
At v= , is positive, i.e., P is minimum at v = .
2 dv 2 2
3c
Hence, the most economical speed is v = km per hour.
2
 Classical Optimization Techniques 15
Example 4: A beam of length l is supported at one end. If w is the uniformly distributed load per
1 1
unit length, the bending moment M at a distance x from the end is given by M = lx – wx2.
2 2
Solution: Given the bending moment is
1 1
M= lx – wx2 … (1)
2 2
Differentiate both sides of (1) with respect to x, we get
dM 1
= l – wx
dx 2
For maxima, we have
dM
=0
dx
1
fi l – wx = 0
2
l
fi x=
2w
Differentiate both sides of (2) with respect to x, we get
d 2M
= – w (negative)
dx 2
l
So the bending moment M is maximum at x = from the end.
2w
2
l 1 Ê l ˆ 1 Ê l ˆ l2
At x= , M= lÁ ˜ - wÁ = .
2w 2 Ë 2w ¯ 2 Ë 2w ˜¯ 8w

Example 5: Find the maximum and minimum value of y = 3x5 – 5x3.

Solution: Given that y = 3x5 – 5x3 … (1)
Differentiate both sides of (1) with respect to x, we get
dy
= 15x4 – 15x2 … (2)
dx
For maxima and minima, we have
dy
=0
dx
fi 15x4 – 15x2 = 0
2
fi 15x (x – 1) (x + 1) = 0
x = 0, 1, – 1.
Differentiate both sides of (2) again with respect to x, we get
d2y
= 60x3 – 30x
dx 2
16 Optimization Techniques

d2y
At x = 0, = 0 i.e., x is a point of inflexion.
dx 2
d2y
At x = 1, = 30 > 0, i.e., y is minimum at x = 1.
dx 2
d2y
At x = –1 = – 30 < 0, i.e., y is maximum at x = –1.
dx 2
Ê 1ˆ
Example 6: In a submarine telegraph cable, the speed of signaling varies as x2 log Á ˜ where
Ë x¯
x is the ratio of the radius of the cube to that of the covering. Show that the greatest speed is
attained when this ratio is 1: e.
Solution: Let u be the speed of signaling, then
Ê 1ˆ
u = x2 log Á ˜ , l > 0, x π 0
Ë x¯
2
u = – lx log x … (1)
Differentiate both sides of (1) with respect to x, we get
du
= – 2lx log x – lx2(1/x)
dx
= – l [2x log x + x] … (2)
For maxima and minima, we have
du
=0
dx
fi – l[2x log x + x] = 0
1
fi log x = –
2
fi x = e–1/2 = 1/ e.
Differentiate both sides of (2) again with respect to x, we get
d 2u
= – l[2x.(1/x) + 2 log x]
dx 2
= – l [2 log x + 3]
d 2u
A x = 1/ e, = – 2l (negative), i.e., u is maximum.
dx 2
Hence, u is maximum, ratio for x = 1: e.
Example 7: A rectangular sheet of metal has four equal square portions removed at the corners
and the sides are then turned up so as to form an open rectangular box. Show that when the
volume contained in the box is maximum, the depth will be
1
[(a + b) – (a2 – ab + b2)1/2]
6
Where a and b are sides of the original dimensions of the rectangular.
 Classical Optimization Techniques 17
Solution: Let x be the length of each side of the squares removed at the corners. Then the
dimensions of the box will be (a – 2x), (b – 2x) and x. Let V be the volume of the box, then
we have
V = (a – 2x) (b – 2x) x
= 4x3 – 2x2 (a + b) + abx … (1)
Differentiate both sides of (1) with respect to x, we get
dv
= 12x2 – 4x (a + b) + ab … (2)
dx
For maxima and minima, we have

dv
=0
dx
fi 12x2 – 4x (a + b) + ab = 0

4(a + b) ± 16(a + b)2 - 48ab

fi x=
24
1
fi x= [(a + b) ± (a2 – ab + b2)1/2]
6
Differentiate both sides of (2) again with respect to x, we get
d 2V
= 24x – 4 (a + b)
dx 2
1 d 2V
At x = [(a + b) ± (a2 – ab + b2)1/2], 2
= ± (a - b)2 + ab
6 dx
1 d 2V
If x = [(a + b) – (a2 – ab + b2)1 / 2] then is negative, i.e., V is maximum.
6 dx 2

Example 8: The efficiency of a screw jack is given by h = tan a cot (a + f) where f is constant.
p f 1 - sinf
Prove that the efficiency is maximum at a = - and h = .
4 2 1 + sinf

Solution: The efficiency of a screw jack is

h = tan a cot (a + f) … (1)
Differentiate both sides of (1) with respect to a, we get
dh
= – tan a cosec2 (a + f) + sec2a cot (a + f)
da
= sec2a cot (a + f) – tana cosec2 (a + f)
1
= [sin (a + f) cos (a + f) – sin a cos a] … (2)
cos a sin 2 (a + f )
2
18 Optimization Techniques

For maxima and minima, we have

dh
=0
da
1
fi [sin(a + f) cos (a + f) – sin a cosa] = 0
cos a sin 2 (a + f )
2

fi sin 2(a + f) – sin 2a = 0

fi 2(a + f) = p – 2a
p f
fi a= -
4 2
By (2), we have
dh
cos2a sin2 (a + f) = [sin (a + f) cos(a + f) – sina cosa]
da
1
= [sin 2(a + f) – sin 2a]
2
= cos (2a + f) cosf. … (3)
Differentiate both sides of (3) again with respect to a, we get
d 2h d dh d
cos2a sin2 (a + f) + {cos2a sin2 (a + f)}. = [cos (2a + f) cos f]
da 2
da da da
d 2h d dh
cos2a sin2 (a + f)+ {cos2a sin2 (a + f)}. = – [sin(2a + f) cos f]
da 2
da da
p f dh d 2h
At a= - , = 0 then is negative, i.e., h is maximum.
4 2 da da 2
p f
At a= - the value of h by (1), we have
4 2
sin(2a + f - sin f )
h = sin(2a + f - sin f )

sin(2a + f - sin f )
h = sin(2a + f - sin f )

1 - sin f
h= .
1 + sin f
Example 9: Show that the right circular cylinder of given surface (including its ends) and
maximum volume is such that its height is equal to twice its radius.
Solution: We know that
V = pr2h … (1)
2
and S = 2prh + 2pr (let constant according to given)
fi 2prh = 2k 2p – 2pr2
k2 - r2
h= … (2)
r
 Classical Optimization Techniques 19
From (1) and (2), we have
V = pr (k2 – r2) … (3)
Differentiate both sides of (3) with respect to r, we get
dV
= p (k2 – 3r2) … (4)
dr
For maxima and minima, we have
dV
=0
dr
fi p (k2 – 3r2) = 0
k
fi r= … (5)
3
Differentiate both sides of (4) with respect to r, we get
d 2V
= – 6pr (negative) i.e., V is maximum.
dx 2
Using (2) and (5), we have
hr = k2 – r2
fi h = 2r.

Example 10: A person being in a boat ‘a’ miles from the nearest point of the beach, wishes to
reach as quickly as possible, a point ‘b’ miles from that point along the sea shore. The ratio of
his rate of walking to his rate of rowing is sec a. Prove that he should land at a distance b – a
cot a from the place to be reached.
Solution: Let the person stand at A and reach on point B. Let C be the point where AC and
BC meet.

Let BC = b and let him row the distance AP in the boat and land at P where PB = x.
If t is the time of the journey, then
AP BP
t= +
vr vw
where AP = a 2 + (b - x)2 and BP = x
vr
Given that = sec a.
vw
20 Optimization Techniques

1 È 2 x ˘
so t=
Í a + (b - x) 2 + … (1)
vr Î seca ˙˚
Differentiate both sides of (1) with respect to x, we get

dt 1 ÈÍ - (b - x) 1 ˘˙
= + … (2)
dx vr Í a 2 + (b - x)2 seca ˙
Î ˚
For maxima or minima, we have
dt
=0
dx

1 ÈÍ - (b - x) 1 ˘˙
fi + =0
vr Í a 2 + (b - x)2 seca ˙
Î ˚

fi – (b – x) seca + a 2 + (b - x)2 = 0

fi (b – x) seca = a 2 + (b - x)2
fi (b – x)2 sec2a = a2 + (b – x)2
fi (b – x)2 tan2 a = a2
fi b–x = a cot a
fi x = b – a cot a
By (2), we have
dt
vr seca a 2 + (b - x)2 = – (b – x) sec a + a 2 + (b - x)2 … (3)
dx
Differentiate both sides of (3) again with respect to x, we get

vr seca a 2 + (b - x)2
d 2t d
+
dx 2 dt
{
vr sec a a 2 + (b - x) 2 } = seca – (b - x)
a 2 + (b - x)2
2
dt d t
At x = b – a cot a, = 0 and is positive.
dx dx 2
i.e., t is minimum at x = b – a cot a.
a 2 b2
Example 11: Prove that the minimum radius vector of the curve + = 1 is of length
(a + b). x2 y 2

Solution: It is given that

a 2 b2
+ =1 … (1)
x2 y 2
Changing the variable co-ordinate Cartesian to polars by taking x = r cosq, y = r sin q in (1),
we get
a2 b2
+ =1 … (2)
r 2 cos 2 q r 2 sin 2 q
 Classical Optimization Techniques 21
or r2 = a2 sec2 q + b2 cosec2 q
Let R = a2 sec2 q + b2 cosec2 q … (3)
Differentiate both sides of (3) with respect to q, we get
dR
= 2a2 sec q. sec q tanq + 2b2 cosec q (– cosecq cot q)
dq
dR
or = 2a 2 sec2 q tan q – 2b2 cosec2 q cot q … (4)
dq
For maxima and minima, we have
dR
=0
dq
fi 2a2 sec2 q tanq – 2b2 cosec2q cotq = 0
b2
fi tan4 q = 2
a
b
fi tanq = .
a
Differentiate both sides of (4) with respect to q, we get
d 2R
= 2a2 [sec2q sec2 q + 2sec2 q tanq. tanq] – 2b2 [cosec2 q (– cosec2 q) – 2 cosec2 q cot2 q]
dq 2
= 2a2 [sec4 q + 2sec2 q tan2 q] + 2b2 [cosec4q + 2cosec2q cot2 q] > 0
b
i.e., R or r2 is minimum at tan q =
a

b a
fi sinq = and cosq =
a+b a+b
By equation (2), we have
a2 b2
+ =1
2Ê a ˆ 2Ê a ˆ
r Á r Á
Ë a + b ˜¯ Ë a + b ˜¯
fi r2 = a (a + b) + b (a + b)
= (a + b)2
b
At tanq = , the value of r is (a + b).
a

Example 12: ADC generator has integral resistance R ohms and has an open circuit voltage of
V volts. Find the lead resistance r for which the power delivered by the generator is maximum.
Solution: We know that the ohm’s law
V = i (R + r)
V
fi i=
R+r
22 Optimization Techniques

V 2r
The power generated P = i2 r = … (1)
( R + r )2
Here V, R being constant.
Differentiate both sides of (1) with respect to r, we get
dP È ( R + r ) 2 .1 - r.2( R + r ) ˘
= V2 Í ˙
dr ÍÎ ( R + r )4 ˙˚
È ( R + r ) - 2r ˘
= V2 Í 3 ˙
Î (R + r) ˚
È R-r ˘
= V2 Í 3˙
… (2)
Î (R + r) ˚
For maxima and minima, we have
dP
=0
dr
È R-r ˘
fi V2 Í 3˙
=0
Î (R + r) ˚
fi R=r
Differentiate both sides of (2) again with respect to r, we get
d 2P È 3
2 ( R + r ) .( -1) - ( R - r ).3( R + r )
2˘
= V Í ˙
dr 2 ÍÎ ( R + r )6 ˙˚
È - 4 R + 2r ˘
= V2 Í 4 ˙
Î (R + r) ˚
d 2P -V 2
At R = r, = (negative) i.e., P is maximum for r = R.
dr 2 8R3
V2
Hence, Pmax =
4R
Example 13: A given quantity of metal is to be cast into a half cylinder, i.e., with rectangular
base and semicircular ends. Show that in order to have minimum surface area, the ratio of the
height of the cylinder to the diameter of semicircular ends is p:p + 2.
Solution: Suppose the volume of the half cylinder is
1
V= pr2h … (1)
2
where r and h are the radius and height of the half cylinder respectively
Surface area of rectangular base = 2rh
Curved surface = prh
Two semicircular ends = pr2
 Classical Optimization Techniques 23
The total surface area
S = 2rh + prh + pr2
= rh (2 + p) + pr2 … (2)
From equation (1), we have
2V
h=
p r2
r.2V
Then S= 2
(2 + p) + pr2
pr
2V
= (2 + p) + pr2 … (3)
pr
Differentiate both sides of (3) with respect to r, we get
dS 2V
= 2pr – (p + 2) … (4)
dr p r2
For maxima and minima we have
dS
=0
dr
2V
fi 2pr – (p + 2) = 0
p r2
Using equation (1), we have
2pr – h (p + 2) = 0
h p
fi = … (5)
2r p +2
Differentiate both sides of (4) again with respect to r, we get
d 2S 4V
2
= 2p – (p + 2)
dr p r3
= 6 p (positive) i.e., S is maximum.
Hence, surface area S is minimum at
h
=
p
2r p +2
Example 14: Find the extreme points of the function u = x2 + 4y2 + 4z2 + 4xy + 4xz + 16yz.
Solution: Given that
u = x2 + 4y2 + 4z2 + 4xy + 4xz + 16yz … (1)
Differentiate partially both sides of (1) with respect to x, y and z respectively, we get
∂u
= 2x + 2y + 2z
∂x
∂u
= 8y + 4x + 16z
∂y
∂u
and = 8z + 4x + 16y
∂z
24 Optimization Techniques

For extreme points, we have

∂u ∂u ∂u
= 0, = 0 and =0
∂x ∂y ∂z
fi 2x + 2y + 2z = 0 or 2(x + y + z) = 0
8y + 4x + 16z = 0 or 4(x + 2y + 4z) = 0
and 8z + 4x + 16y = 0 or 4(x + 4y + 2z) = 0
Solving above these expression, we get x = 0, y = 0, z = 0.
Let the point P be (0, 0, 0)
Differentiate partially (2) and we get
∂2u ∂2u ∂2u ∂2u ∂2u ∂2u ∂2u ∂2u
= 2, = 8, = 8, = 4, = 4, = 16, = 16, = 4,
∂x 2 ∂y 2 ∂z 2 ∂ x∂ y ∂y ∂x ∂y ∂z ∂z ∂y ∂ x∂ z
∂2u
= 4.
∂z ∂x
The Hessian matrix of u (x, y, z) is

È2 4 4 ˘
H = Í4 8 16˙
Í ˙
ÍÎ4 16 8 ˙˚
The leading minors of H are
2 4 4
2 4
H1 = |2| = 2, H2 = = 0 and H3 = 4 8 16 < 0.
4 8
4 16 8
Here H1 and H3 are not of same sign and H2 = (i.e., semi-definite). Hence u has a saddle
point at (0, 0, 0).
Example 15: Find the maximum and minimum value of u, where u = sin x sin y sin (x + y).
Solution: Given that
u = sin x sin y sin (x + y) … (1)
Differentiate partially (1) with respect to x and y both sides respectively, we get,
∂u
= sin y [sin x cos (x + y) + cos x sin (x + y)] … (2)
∂x
∂u
and = sin x [sin y cos (x + y) + cos y sin (x + y)] … (3)
∂y
For maxima and minima, we have
∂u ∂u
= 0, and =0
∂x ∂y
sin y [sin x cos (x + y) + cos x. sin (x + y)] = 0
and sin x [sin y cos (x + y) + cos y, sin (x + y)] = 0
 Classical Optimization Techniques 25
Solving above these expressions, we get
tan (x + y) = – tan x … (4)
and tan (x + y) = – tan y … (5)
Now, we have
tan 2 x = – tan x = tan (p – x)
2x = p – x
x = p /3 = y
Also
sin y = 0 fi y = 0
and sin x = 0 fi x = 0
Thus, the stationary points are (0, 0), (p/3, p/3).
Differentiate partially (2) and (3) again, we get
∂ 2u
r= = 2 sin y cos (2x + y)
∂x 2
∂ 2u
s= = sin 2 (x + y)
∂x ∂y
∂ 2u
and t= = 2 sin x cos (2y + x)
∂y 2
At (0, 0), we get r = 0, s = 0, t = 0.
rt – s2 = 0, i.e., u has a saddle point at (0, 0).
Now at (p /3. p /3), we get r = 2 sin (p /3) cos p = – 3 .
3
s = sin (4p /3) = – sin (p /3) = –
2
and t = 2 sin (p /3) sin p = – 3
9
fi rt – s2 = is positive and r < 0.
4
Hence, u is maximum at (p /3, p /3).
Example 16: Find the extreme points f (x1, x2) = 20x1 + 26x2 + 4x1x2 – 4x 12 – 3x 22.
Solution: Given that
f (x1, x2) = 20x1 + 26x2 + 4x1x2 – 4x 12 – 3x 22 … (1)
Differentiate partially (1) with respect to x1 and x2 both sides respectively, we get
∂f
= 20 + 4x2 – 8x1 … (2)
∂x1
∂f
and = 26 + 4x1 – 6x2 … (3)
∂x2
For extreme points, we have
∂f ∂f
=0 and =0
∂x1 ∂x2
26 Optimization Techniques

fi 20 + 4x2 – 8x1 = 0
And 26 + 4x1 – 6x2 = 0
Solving these, we get x1 = 7, x2 = 9.
Differentiating again partially (2) and we get
∂2 f
r= = –8
∂x12
∂2 f
s= =4
∂x1∂x2
∂2 f
and t= = –6
∂x22
We have rt – s2 = (– 8) (– 6) – (4)2 = 32.
At (7, 9), rt – s2 > 0 and r < 0, i.e., f is maximum at (7,9).
Example 17: Find the point (x1, x2, x3) at which is functions
f (x1, x2, x3) = – x12 – x22 – x 32 + x1 x2 + x1 + 2x3 has optimum values.

Solution: Given that

f (x1, x2, x3) = –x12 – x22 – x 32 + x1 x2 + x1 + 2x3 … (1)
Differentiate partially (1) with respect to x1, x2 and x3 both sides respectively, we get
∂f
= – 2x1 + 1
∂x1
∂f
= – 2x2 + x3
∂x2
∂f
and = – 2x3 + x2 + 2 … (2)
∂x3
For extreme points, we have
∂f ∂f ∂f
= 0, = 0 and =0
∂x1 ∂x2 ∂x3
fi – 2x1 + 1 = 0
– 2x2 + x3 = 0
and – 2x3 + x2 + 2 = 0
Solving these we get x1 = ½, x2 = 2/3, x3 = 4/3.
Ê 1 2 4ˆ
Let the point P be = Á , , ˜.
Ë 2 2 3¯
Differentiate partially (2) again and we get
∂2 f ∂2 f ∂2 f ∂2 f ∂2 f ∂2 f ∂2 f
= –2, = –2, = –2, = 0, = 0, = 0, = 0,
∂x12 ∂x22 ∂x32 ∂x1∂x2 ∂x2 ∂x1 ∂x1∂x3 ∂x3 ∂x1
∂2 f ∂2 f
= 1, =1
∂x2 ∂x3 ∂x3 ∂x2
28 Optimization Techniques

At (a, a), we get r = 2, s = 1, t = 2.

Then rt – s2 = 3 > 0
Since at (a, a), rt – s2 > 0 and r > 0, then u is minimum at (a, a).
a3 a3
The minimum value of u = a. a + + = 3a2.
a a
Example 19: Find the extreme points of the function f (x1, x2) = – x31 + 2x32 + 3x21 + 12x22 + 24.
And determine their nature also.
Solution: Given that
f (x1, x2) = x31 + 2x32 + 3x21 + 12x22 + 24 … (1)
Differentiate partially (1) with respect to x1 and x2 both sides respectively, we get
∂f
= 3x12 + 6x1 … (2)
∂x1
∂f
and = 6x 22 + 24x2 … (3)
∂x2
For extreme points, we have
∂f ∂f
= and =0
∂x1 ∂x2
fi 3x12 + 6x1 = 0
And 6x 22 + 24x2 = 0
Solving these, we get x1 = 0, – 2, x2 = 0, – 4.
Differentiating again partially (2) and we get
∂2 f
r= = 6(x1 + 1)
∂x12
∂2 f
s= =0
∂x1∂x2
∂2 f
and t= = 12(x2 + 2)
∂x22
At (0, 0) rt – s2 = 72(x1 + 1) (x2 + 2) = 72 > 0 and r > 0 i.e., f is minimum at (0, 0).
At (0, – 4) rt – s2 = 72(x1 + 1) (x2 + 2) = –144 < 0 fi no extreme point, i.e., f has a saddle point
at (0, – 4).
At (– 2, 0) rt – s2 = 72(x1 + 1) (x2 + 2) = –144 < 0 fi no extreme point, i.e., f has a saddle point
at (–2, 0).
At (– 2, – 4) rt – s2 = 72(x1 + 1) (x2 + 2) = 144 > 0 and r < 0 i.e., f is maximum at (– 2, – 4).
Example 20: Find the dimension of a box of the largest volume that can be inscribed in a
sphere of radius 3 meters.
Solution: Let the volume of the box be
V = 2x. 2y. 2z fi V = 8xyz … (1)
 Classical Optimization Techniques 29
Changing the variables Cartesian to spherical polar co-ordinates as x = r sin q cosf, y = r sinq
sinf, z = r cosq and using r = 3 (given).
V = 216 sin2q cosq cosf sinf
= 108 sin2q cosq sin 2f … (2)
Differentiate partially (2) with respect to q and f both sides respectively, we get
∂V
= 108 sin 2 f [sin2 q (–sin q) + 2 sinq cos2 q]
∂q
= 108 sin 2 f sinq {–sin2 q + 2 cos2 q} … (3)
∂V
and = 216 cos 2 f sin 2 q cosq … (4)
∂f
For maxima and minima, we have
∂V ∂V
= 0 and =0
∂q ∂f
fi 108 sin 2 f sin q{–sin2 q + 2 cos2 q} = 0
i.e., sinq = 0 or tanq = 2 or sin f = 0
i.e., q = 0, tan-1 2 , f = 0.
and 216 cos 2 f sin2 q cos q = 0
i.e., q = 0, p /4.
Because q = 0 or f = 0 gives V = 0, so we take only q = tan-1 2 and f = p /4.
Differentiate partially (3) again and we get
∂ 2V
2
= 108 sin 2 f [sinq (– 4 cos q sin q – 2 sinq cosq) + cos q(2 cos2 q – sin 2 q)]
∂q
È 2 ˘
1 Ê Ê 1 ˆ Ê 2ˆ ˆ˙
2
-1 ∂ 2V Í 2Ê 1 2ˆ
At (tan 2 , p/4), = 108 -6 ¥ ¥ ˜+ Á 1Á ˜ -Á ˜ ˜
∂q 2 Í 3 ÁË 3 3¯ 3 ÁË Ë 3 ¯ Ë 3 ¯ ˜¯ ˙
Î ˚
432
= -
3
∂2V
= 216 cos 2 f[sin2q(– sinq) + cos2q (2 sinq)]
∂q ∂f
= 216 cos 2 f sinq [2 cos2q – sin2q]
∂2V
At (tan– 1 2 , p /4), =0
∂q ∂f
∂ 2V
2
= – 432 sin 2 f[sin2q cosq]
∂f
2
∂ 2V Ê 2ˆ Ê 1 ˆ 864
At (tan– 1 2 , p/4), = – 432 ¥ Á ˜ ¥ ÁË ˜ =–
∂f 2
Ë 3¯ 3¯ 3 3
32 Optimization Techniques

4. At x = 400.
6. At x = 0 give point of inflection; at x = 1 give maxima and value is 12; at x = 2 give
minima and value is – 11.
7. At x = 6/5 f (x) give maxima.
8. Diameter = 3.93 meters, length = 4.12 meters.
10. At x = 1 give point of inflexion, at x = 2 give local maxima and x = 3 give local minima.
12. y = x = C/4, i.e., square.
2pc3
13. At h = C/ 3 volume = .
9 3
15. Speed 40 km/hour.
17. The intensity is maximum for tan q = 2 height = 25 2 meters.
20. Height = 40.423 mm, length = 216.154 mm, width = 129.154 mm and maximum volume
= 1128.5 cm3.
Ê a aˆ
21. At (0, 0), (0, a) and (a, 0) u is neither maxima nor minima. At Á , ˜ = u is minimum if
Ë 3 3¯
a < 0 and u is maximum if a > 0.
22. At (1, 2) u is minimum and at (– 1, – 2) u is maximum.
Ê 4 8ˆ
23. At (0, 0) f is minimum and Á - , - ˜ f is maximum.
Ë 3 3¯
24. At ( )
2 , - 2 u is maximum.
Ê p p pˆ
25. At Á , , ˜ f is maximum.
Ë 3 3 3¯
26. At x = y = z = (2 v0)1/3 give minima.
Ê 2 1 ˆ
27. At Á - , - , 1˜ u is minimum.
Ë 3 3 ¯
28. At (0, 0) u give saddle point.

2.4 constraIned multIvarIable optImIzatIon problems wIth

equalIty constraInts
The optimization problem of a continuous and differentiable function subject to equality
constraints:
Optimize (Max or Min) Z = f (X)
Subject to constraints (s.t.) gj (X) = 0 ; j = 1, 2, 3, …, mg
Ê x1 ˆ
Á x2 ˜
Á ˜
where X = Áº˜
Áº˜
Á ˜
ÁË x ˜¯
n
 Classical Optimization Techniques 33
Here m is less than or equal to n. There are several methods for solving this type of problem.
Here we discuss only two methods:
1. Direct substitution method
2. Lagrange multipliers method

2.4.1  Direct Substitution Method
In this method, the value of any variable from the constraint set is put into the objective function.
The problem reduces to unconstrained optimization problem and can be solved by unconstrained
optimization method.

2.4.2  Lagrange Multiplier Method
Consider a general problem with n variables and m equality constraints:
Optimize Z = f (X)
s.t. gj(X) = 0 ; j = 1, 2, 3,…, m (m < n)
X≥0

Ê x1 ˆ
Áx ˜
Á 2˜
where X = Á º˜
Á º˜
Á ˜
ÁË x ˜¯
n

Now we define a function

m
L(x1, x2,…, xn, l1, l2,…, lm) = f (X) + Â ljgj (X) … (1)
j =1
Here, l1, l2, …, lm are known as Lagrange’s undetermined multipliers.
The necessary conditions for extreme of L are

∂L ∂f
m
∂g j
∂xi
=
∂xi
+ Âl j
∂xi
=0 … (2)
j =1

∂L Ê i = 1, 2,º n ˆ
and = 0; Á … (3)
∂l j Ë j = 1, 2,º m˜¯
Solving equation (4) and (5), we get
Ê x1* ˆ Ê x1* ˆ
Á *˜ Á *˜
Á x2 ˜ Á x2 ˜
Á ˜ Áº ˜
X = Á º ˜ and l* = Á ˜
Áº ˜ Áº ˜
Á *˜ Á *˜
Ë xm ¯ Ë xm ¯
34 Optimization Techniques

The sufficient condition for the function to have extreme at point X *, is that the values of k
obtained from equation
È L11 - k L12 º L1n g11 g 21 º g m1 ˘
ÍL
Í 21 L22 - k º L2 n g12 g 22 º g m 2 ˙˙
ͺ º º º º º º º ˙
Í ˙
ͺ º º º º º º º ˙
Í Ln1 Ln 2 º Lnn - k g1n g2n º g mn ˙ = 0
Í ˙
Í g11 g12 º g1n 0 0 º 0 ˙
ͺ º º º º º º º ˙
Í ˙
ͺ º º º º º º º ˙
Í ˙
Î g m1 gm2 º g mn 0 0 º 0 ˚
must be of the same sign. If all the eigen values of k are negative then it is a maxima and if all the
eigen values k are positive then it is a minima. But if some eigen values are zero or of different
∂2 L ∂g j
sign then that is a saddle point. In above Lij and gij denoted by and respectively.
∂xi ∂x j ∂xi

2.5. constraIned multIvarIable optImIzatIon problems wIth

InequalIty constraInts
Let us consider a problem
Optimization (Max or Min) Z = f (X) … (1)
s.t. gj(X) £ 0 ; j = 1, 2, 3,…, m … (2)
Ê x1 ˆ
Áx ˜
Á 2˜
where X = Á º˜
Á º˜
Á ˜
ÁË x ˜¯
n
The inequality constraints in equation (2) can be converted into equality constraints by adding
slack variables as
gj (X) + yj2 = 0; j = 1, 2, 3,…, m
Now the problem becomes
Optimize (Max or Min) Z = f (X)
s.t. Gj (X, Y) = gj (X) + yj2 = 0; j = 1, 2, 3,…, m
Ê x1 ˆ Ê y1 ˆ
Áx ˜ Áy ˜
Á 2˜ Á 2˜
where X = Á º˜ and Y = Á º ˜
Á º˜ Áº ˜
Á ˜ Á ˜
ÁË x ˜¯ ÁË y ˜¯
n m
Now it can be solved by Lagrange’s multipliers method.
36 Optimization Techniques

∂L
=0
∂l j
fi gj (X) + yj2 = 0; j = 1, 2, 3,…, m … (6)
∂L
and =0
∂y j
fi 2lj yj = 0; j = 1, 2, 3,…, m … (7)
From (6) and (7), we have
lj gj (X) = 0
fi lj = 0
or g j (X) = 0.
Case I: If g j (X) = 0 at the optimum point then it is called the active constraint and we can
find optimum solution.
Case II: If l j = 0 at the optimum point then it is called inactive constraints.
Note: If the given optimization problem is a minimization problem with constraints of the form
g j (X) ≥ 0 then l j £ 0 but if the problem is a maximization problem with constraints of the form
gj (X) £ 0 then l j £ 0. Let us consider some maximization or minimization problems given in
the following terms.
(i) Maximize Z = f (X)
s.t. gj (X) £ 0 ; j = 1, 2, 3,…, m
For the function f (X) to have maxima, we have
∂f
m
∂g j
∂xi
+ Âl j
∂xi
= 0; i = 1, 2, 3,…, m
j =1

lj g j (X) = 0; j = 1, 2, 3,…, m
and lj £ 0.
(ii) Maximize Z = f (X)
s.t. g j (X) ≥ 0; j = 1, 2, 3,…, m
For the function f (X) to have maxima, we have
∂f
m
∂g j
∂xi
+ Âl j
∂xi
= 0; i = 1, 2, 3,…, m
j =1

lj gj (X) = 0; j = 1, 2, 3,…, m
and lj ≥ 0.
(iii) Minimize Z = f (X)
s.t. g j (X) £ 0; j = 1, 2, 3,…, m
For the function f (X) to have maxima, we have
∂f
m
∂g j
∂xi
+ Âl j
∂xi
= 0; i = 1, 2, 3, …, m
j =1
lj gj (X) = 0; j = 1, 2, 3, …, m
and lj £ 0.
 Classical Optimization Techniques 37
(iv) Minimize Z = f (X)
s.t. gj (X) ≥ 0; j = 1, 2, 3, …, m
For the function f(X) to have maxima, we have
∂f
m
∂g j
∂xi
+ Âl j
∂xi
; i = 1, 2, 3, …, m
j =1

lj gj (X) = 0; j = 1, 2, 3, …, m
and lj £ 0.

solved examples
Example 21: Find the optimum solution of the following constrained multivariable problem
minimize Z = x12 + (x2 + 1)2 + (x3 – 1)2 s.t. x1 + 5x2 – 3x3 = 6.
Solution: Given that
Z = x12 + (x2 + 1)2 + (x3 – 1)2 … (1)
and x1 + 5x2 – 3x3 = 6
x1 + 5 x2 + 6
fi x3 = … (2)
3
Using (1) and (2), we get
1
Z = x12 + (x2 + 1)2 +
(x1 + 5x2 – 9)2 … (3)
9
Differentiate partially both sides of (3) with respect to x1 and x2 respectively, we get
∂Z 2
= 2x1 + (x1 + 5x2 – 9)
∂x1 9
∂Z 10
and = 2(x2 + 1) + (x1 + 5x2 – 9)
∂x2 9
For maxima and minima, we have
∂Z ∂Z
=0 and =0
∂x1 ∂x2
2
fi 2x1 +(x1 + 5x2 – 9) = 0
9
10
and 2(x2 + 1) + (x1 + 5x2 – 9) = 0
9
Solving these, we get
2
x1 = and x2 = 1
5
Differentiate again partially (4) and we get
∂2 Z 2 20
r= 2
=2+ =
∂x1 9 9
38 Optimization Techniques

∂2 Z 50 68
t= =2+ =
∂x22 9 9
∂2 Z 10
and =
∂x1 x2 9
2
Ê2 ˆ Ê 20 ˆ Ê 68 ˆ Ê 10 ˆ
At ÁË 5 , 1˜¯ , rt – s2 = Á ˜ Á ˜ - Á ˜ = 1260 > 0 and r > 0
Ë 9 ¯Ë 9 ¯ Ë 9 ¯
Ê2 ˆ 28
i.e., Z is minimum at Á , 1˜ the minimum value of is .
Ë5 ¯ 5
Example 22: Find the dimensions of a box of large volume that can be inscribed in a sphere
of radius a.
Solution: Suppose x, y and z are the dimensions of the box with respect to origin O and
OX,OY,OZ are reference axes. The volume of box is
V = 8xyz … (1)
Given that the box is to be inscribed in a sphere of radius ‘a’ i.e.,
x 2 + y2 + z2 = a2 … (2)
Eliminating z from (1) and (2), we get
V = 8xy (a2 – x2 – y2 )1/2 … (3)
Differentiate partially (3) with respect to x and y both sides respectively, we get
∂V È 1 ˘
= 8y Í x ◊ (a 2 - x 2 - y 2 )-1/ 2 (- 2 x) + (a 2 - x 2 - y 2 )1/ 2 ˙
∂x1 Î 2 ˚
È x2 ˘
= 8y Í - 2 2 2 1/ 2
+ (a 2 - x 2 - y 2 )1/ 2 ˙
Î (a - x - y ) ˚
È (a 2 - 2 x 2 - y 2 ) ˘
= 8y Í 2 2 2 1/ 2 ˙
… (4)
Î (a - x - y ) ˚

∂V È (a 2 - x 2 - 2 y 2 ) ˘
and = 8x Í 2 2 1/ 2 ˙
… (5)
∂y 2
Î (a - x - y ) ˚
For maxima and minima, we have
∂V ∂V
= 0 and =0
∂x1 ∂y
È (a 2 - 2 x 2 - y 2 ) ˘
fi 8y Í 2 2 2 1/ 2 ˙
=0
Î (a - x - y ) ˚
È (a 2 - x 2 - 2 y 2 ) ˘
and 8x Í 2 2 2 1/ 2 ˙
=0
Î (a - x - y ) ˚
40 Optimization Techniques

The necessary conditions for extreme L are

∂L ¸
= 2 x + 4l = 0 Ô
∂x
Ô
∂L Ô
= 2 y + 2 yl = 0
∂y ÔÔ
˝ … (4)
∂L Ô
= 2 z + 2l = 0
∂z Ô
∂L Ô
= 4 x + y 2 + 2 z - 14 = 0 Ô
∂l Ô˛
Solving these we get x = –2l, z = –l and l = –1
fi x = 2, z = 1 and y = ± 2, putting x = –2l, z = –l, y = 0 in (2), we get l = –1.4.
Here (2, 2, 1, –1), (2, – 2, 1, –1) and (– 2l, 0, – l, l) or (2.8, 0, 1.4, –1.4) are extreme points.
Differentiate again partially (4) and we get

∂2 L ∂2 L ∂2 L ∂2 L ∂2 L ∂2 L ∂2 L ∂2 L
= 2, = 0, = 0, = 0, = 2 + 2l, = 2, = 0,
∂x 2 ∂x ∂y ∂x ∂z ∂y ∂x ∂y 2 ∂y ∂z ∂z ∂x ∂z ∂y

∂2 L ∂g ∂g ∂g
= 0, = 2, = 4, = 2y, = 2.
∂z 2
∂x ∂y ∂z
The sufficient condition for the extreme point is

∂2 L ∂2 L ∂2 L ∂g
-k
∂x 2 ∂x ∂y ∂x ∂z ∂x
∂2 L ∂2 L ∂2 L ∂g
-k
∂yx∂x ∂y 2 ∂y ∂z ∂y
H= =0
∂2 L ∂2 L ∂2 L ∂g
-k
∂z∂∂x ∂z ∂y ∂z 2 ∂z
∂g ∂g ∂g
0
∂x ∂y ∂z

2-k 0 0 4
0 2 + 2l - k 0 2y
i.e., H= =0
0 0 2-k 2
4 2y 2 0
or 4 (2 – k) [–10 + 5k – 10l – 2y2 + y 2k] = 0
At (2, 2, 1, –1), equation (2) we have
(2 – k) (–10 + 5k + 10 – 8 + 4k) = 0
fi k = 2, 8/9
 Classical Optimization Techniques 41
i.e., values of k are positive then there is a minima.
At (2, – 2, 1, –1), equation (2) we have
(2 – k) (–10 + 5k + 10 – 8 + 4k) = 0
fi k = 2, 8/9
Also values of k are positive then there is a minima.
At (2.8, 0, 1.4, –1.4), from (2) we have
(2 – k ) (–10 + 5k + 14 – 0 + 0) = 0
fi k = 2, – 4/5.
i.e., value of k are positive and negative (neither maxima and nor minima) i.e., saddle point.
1 2
Example 24: Minimize f (x) = (x1 + x22 + x32)
2
s.t. g1 (x) = x1 – x2 = 0
and g 2 (x) = x1 + x2 + x3 – 1 = 0 by Lagrange multiplier method.
Solution: It is given that
1
Minimize f (x) = (x12 + x22 + x32) … (1)
2
s.t. g1(x) = x1 – x2 = 0 … (2)
and g 2 (x) = x1 + x2 + x3 – 1 = 0 … (3)
The Lagrangian function L is
1
L(x1, x2, x3; l1, l2) = (x12 + x22 + x32) + l1(x1 – x2) + l2 (x1 + x2 + x3 – 1) … (4)
2

The necessary conditions for extreme of L are

∂L ¸
= x1 + l1 + l2 = 0 Ô
∂x1
Ô
∂L Ô
= x2 - l1 + l2 = 0 Ô
∂x2
Ô
∂L Ô
= x3 + l2 = 0 ˝ … (5)
∂x3 Ô
∂L Ô
= x1 - x2 = 0 Ô
∂l1 Ô
∂L Ô
= x1 + x2 + x3 - 1 = 0 Ô
∂l ˛
Solving these, we get
1 1
x1 = x2 = x3 = ; l1 = 0, l 2 = –
3 3
 Classical Optimization Techniques 43
x + y ≥ 80,
x + y + z ≥ 120.
g1 = x – 40, g2 = x + y – 80, g3 = x + y + z – 120 … (6)
Using equations (4), (5) and (6), we have
L = x2 + y2 + z2 + 20x + 10y + l1(x – 40 – y12) + l2(x + y – 80 – y22)
+ l3 (x + y + z – 120 – y32) … (7)
The Kuhn-Tucker necessary conditions for minimization of L (with g j (X) ≥ 0) are
∂L ∂L ∂L
= 0, = 0, =0
∂x ∂y ∂z
l j g j = 0; j = 1, 2, 3
l ≥ 0; j = 1, 2, 3
Differentiate partially (7) and we get
∂L ¸
= 2 x + 20 + l1 + l2 + l3 = 0 Ô
∂x Ô
∂L Ô
= 2 y + 10 + l2 + l3 = 0 ˝ … (8)
∂y Ô
∂L Ô
= 2 z + l3 = 0 Ô
∂z ˛
l1g1 = l1( x - 40) = 0 ¸
Ô
l 2 g 2 = l 2 ( x + y - 80) = 0 ˝ … (9)
l 3 g3 = l 3 ( x + y + z - 120) = 0 Ô˛

l 1, l 2, l 3, ≥ 0 … (10)
If l1 π 0, l2 π 0, l3 π 0 from (9) we have
( x - 40) = 0 ¸
Ô
( x + y - 80) = 0 ˝
( x + y + z - 120) = 0 Ô˛
Solving these we get
x = 40, y = 40, z = 40.
Using values of x, y and z. From equation (6), we get
l1 = –10, l2 = –10, l3 = – 80
Hence, the condition lj £ 0 from equation (10) is satisfied.
Hence, the optimum solution is
x = y = z = 40.
and minimize f = (40)2 + (40)2 + (40)2 + 20(40) + 10(4)
= 6000.