CE 771 Lecture 4 Interaction PDF

9/2/20
CE 771: Optimization in Civil

Engineering
Multi variable optimization: Unconstrained

Problems and solutions
AvijitMaji©
HESSIAN MATRIX
AvijitMaji©
1
9/2/20
Positive, negative Definite

• If A is 𝑛×𝑛 diagonal matrix
𝑑! ⋯ 0
A= ⋮ ⋱ ⋮
0 ⋯ 𝑑"
Then A is :
• Positive definite if and only if 𝑑# > 0 for i=1,2,……,n
• Negative definite if and only if 𝑑# < 0 for i=1,2,……,n
1st Semester of 2020-21 Academic year CE 771: Optimization in Civil Engineering (Online course) AvijitMaji© 3

• Positive semidefinite if and only if 𝑑# ≥ 0 for 𝑖 =1,2,……,n
• Negative semidefinite if and only if 𝑑# ≤ 0 for 𝑖 =1,2,……,n
• Indefinite if and only if 𝑑# > 0 for some indices 𝑖 , 1 ≤ 𝑖 ≤ 𝑛 and
negative for other indices
2
9/2/20
Converting matrix into diagonal matrix

−14 −28 −44
• Hessian matrix = −7 −14 −23
9 18 29
• Corresponding diagonal matrix after transformation-
−1 0 0
0 0 0
0 0 2
• Thus eigen values are -1,0,2 which clearly states the hessian matrix is
indefinite that the particular solution would be a saddle point

• Let A is 𝑛×𝑛 symmetric matrix, and let A𝑘 be the submatrix of A
obtained be taking the upper left-hand corner k×k submatrix of A
and ∆$ = det 𝐴$ , the kth principal minor of A. Then
• Positive definite if and only if ∆$ > 0 for k=1,2,……,n
• Negative definite if and only if (−1)$ ∆$ < 0 for k=1,2,……,n
• Positive semidefinite if ∆$ > 0 for k=1,2,……,n-1 and ∆" = 0
• Negative semidefinite if and only if (−1)$ ∆$ > 0 for k=1,2,……,n-1
and ∆" = 0
3
9/2/20
Example 1
Find the solution of the multivariable unconstrained optimization
problem :
𝑓 𝑥, 𝑦 = 𝑥2 – 2𝑥𝑦 + 4𝑦2
Solution
Necessary condition for the optimum point
#$
#%
= 2𝑥 − 2y = 0 ∴𝑥=y
#$
#&
= −2𝑥 + 8y = 0 ∴ x = 4y
Possible decision variable value 𝑥 ∗ , 𝑦 ∗ = (0,0)
To find whether the solution found corresponds to a maximum or a minimum we
apply the sufficiency conditions to𝑓 𝑥, 𝑦 . The second-order partial derivatives of f
at 𝑥 ∗ , 𝑦 ∗ are given by :
#! $ #! $
= 2, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ ) = 8, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ )
#% ! #&!
#! $ #! $
#%#&
= −2, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ ) #(#%
= −2, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ )
4
9/2/20
Solution Contd.
Sufficiency conditions:
2 −2
• Hessian matrix :
−2 8
• Principal determinants:
)
#! $ #! $ #! $ #! $
#% !
=2 >0 𝑎𝑛𝑑 #% ! #&!
− #%#&
= 12 (> 0)
the Hessian matrix of f is Positive definite at 𝑥 ∗ , 𝑦 ∗ . Hence the point 𝑥 ∗ , 𝑦 ∗

corresponds to the minimum of f and 𝑓*+% = 0
Example 2
Find the solution of the multivariable unconstrained optimization
problem
𝑓 𝑥, 𝑦 = 𝑥 % − 3𝑥 & + 𝑦 % − 3𝑦 &
10
5
9/2/20
Solution
𝜕𝑓
= 3𝑥 ) − 6𝑥 = 0
𝜕𝑥
#$
#&
= 3𝑦 ) − 6y = 0
Possible decision variable value 𝑥 ∗ , 𝑦 ∗ = 0,0 , 0,2 , 2,0 , (2,2)
To find whether the solution found corresponds to a maximum or a
minimum we apply the sufficiency conditions to𝑓 𝑥, 𝑦 . The second-order
partial derivatives of f at 𝑥 ∗ , 𝑦 ∗ are given by :
# !$ # !$
! = 6x − 6, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ ) = 6y − 6, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ )
#% #& !
# !$ # !$
#%#&
= 0, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ ) #(#%
= 0, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ )
11
6𝑥 − 6 0 −6 0
q Hessian matrix for (0,0): 0 6𝑦 − 6
=
0 −6
)
#! $ #! $ #! $ #! $
= −6 < 0 𝑎𝑛𝑑 − = 36 (> 0)
#% ! #% ! #&! #%#&
the Hessian matrix of f is negative definite at 𝑥 , 𝑦 . Hence the point 𝑥 ∗ , 𝑦 ∗ ∗ ∗
= (0,0) corresponds to maximum of f and fmax = 0
6𝑥 − 6 0 −6 0
qHessian matrix for (0,2): =
0 6𝑦 − 6 0 6
)
#! $ #! $ #! $ #! $
= −6 < 0 𝑎𝑛𝑑 − = −36 (< 0)
#% ! !
#% #& ! #%#&
the Hessian matrix of f is indefinite at 𝑥 , 𝑦 . Hence the point 𝑥 ∗ , 𝑦 ∗ = (0,2)
∗ ∗
corresponds to the saddle point
12
6
9/2/20
6𝑥 − 6 0 6 0
0 6𝑦 − 6 0 −6
)
#! $ #! $ #! $ #! $
=6 >0 𝑎𝑛𝑑 − = −36 (< 0)
#% ! #% ! #&! #%#&
the Hessian matrix of f is indefinite at 𝑥 ∗ , 𝑦 ∗ . Hence the point 𝑥 ∗ , 𝑦 ∗ = (2,0)
6𝑥 − 6 0 6 0
0 6𝑦 − 6 0 6
)
#! $ #! $ #! $ #! $
#% !
=6 >0 𝑎𝑛𝑑 #% ! #&!
− #%#&
= 36 (> 0)
the Hessian matrix of f is positive definite at 𝑥 ∗ , 𝑦 ∗ . Hence the point 𝑥 ∗, 𝑦 ∗ =
(0,2) corresponds to minimum of f and fmin = -4
13
Example 3
Example: Find the solution of the multivariable unconstrained
optimization problem
𝑓 𝑥, 𝑦 = 𝑥 % − 12𝑥𝑦 + 8𝑦 %
14
7
9/2/20
Solution
𝜕𝑓
= 3𝑥 ) − 12𝑦 = 0
𝜕𝑥
𝜕𝑓
= −12x + 24𝑦 ) = 0
𝜕𝑦
Possible decision variable value 𝑥 ∗ , 𝑦 ∗ = 0,0 , (2,1)
To find whether the solution found corresponds to a maximum or a minimum we
apply the sufficiency conditions to 𝑓 𝑥, 𝑦 . The second-order partial derivatives of f
at 𝑥 ∗ , 𝑦 ∗ are given by
#! $ #! $
#% !
= 6x, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ ) #&!
= 48y, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ )
#! $ #! $
= −12, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ ) = −12, 𝑎𝑡 (𝑥 ∗ , 𝑦 ∗ )
#%#& #(#%
15
Solution Contd.
0 −12
• Hessian matrix for (0,0):
−12 0
&
" !# " !# " !# " !#
"$ ! = 0 𝑎𝑛𝑑
"$ ! "% !− "$"%
<0
the Hessian matrix of f is indefinite at 𝑥 ∗ , 𝑦 . Hence the point 𝑥 ∗ , 𝑦 ∗ = (0,0)
∗
12 −12
• Hessian matrix for (2,1):
−12 48
&
" !# " !# " !# " !#
>0 𝑎𝑛𝑑 − >0
"$ ! !
"$ "% ! "$"%
the Hessian matrix of f is positive definite at 𝑥 ∗ , 𝑦 ∗ . Hence the point 𝑥 ∗ , 𝑦 ∗ = (2,1)
corresponds to the minimum of f with fmin = -8
16
8
9/2/20
Example 4
• Find the solution of the multivariable unconstrained optimization
problem :
𝑓 𝑥, 𝑦 = 4𝑥 & − 4𝑦 %
17
Solution
• Necessary conditions for the optimum point
#$
#%
= 8𝑥 = 0
#$
#&
= −12𝑦 ) = 0
Decision variable values: (x, y) = (0, 0)
• Sufficiency conditions:
8 0
Hessian matrix 𝐴 =
0 −24𝑦
)
#! $ #! $ #! $ #! $
Principal determinants 𝐴, = #%! > 0 and 𝐴) = #%! #&! − #%#&
= 0 at (0,0)
• Hence this is a case of Positive Semidefinite
18
9
9/2/20
Example 5
• Find the solution of the multivariable unconstrained optimization
problem :
𝑓 𝑥, 𝑦 = 3𝑦 % − 3𝑥 &
19
Solution
• Necessary conditions for the optimum point
"#
= −9𝑥 = 0
"$
"#
= 9𝑦 & = 0
"%
Decision variable values: (x, y) = (0, 0)
• Sufficiency conditions:
−9 0
Hessian matrix 𝐴 = It is not a positive definite nor positive semidefinite
0 18𝑦
Lets check for Negative definite and negative semidefinite
9 0
Hessian matrix −𝐴 =
0 −18𝑦
20
10
9/2/20
Solution Contd.
)
#! $ #! $ #! $ #! $
Principal determinants 𝐴, = #%! > 0 and 𝐴) = #%! #&! − #%#&
= 0 at (0,0)
• Hence this is a case of Negative Semidefinite
21
Thank you
22
11

CE 771 Lecture 4 Interaction PDF

Uploaded by

Copyright:

Available Formats

You might also like

CE 771 Lecture 4 Interaction PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CE 771 Lecture 4 Interaction PDF

Uploaded by

Copyright:

Available Formats

9/2/20

CE 771: Optimization in Civil

Multi variable optimization: Unconstrained

Positive, negative Definite

Positive, negative Definite

Converting matrix into diagonal matrix

• Corresponding diagonal matrix after transformation-

Positive, negative Definite

the Hessian matrix of f is Positive definite at 𝑥 ∗ , 𝑦 ∗ . Hence the point 𝑥 ∗ , 𝑦 ∗

You might also like