Professional Documents
Culture Documents
Slides Multivariate Calculus Wima 2018
Slides Multivariate Calculus Wima 2018
Rüdiger Frey
ruediger.frey@wu.ac.at
http://statmath.wu.ac.at/˜frey
Winter 2018
1 / 27
Multivariate Calculus and optimization
2 / 27
Multivariate Calculus and optimization
4
2
2
0
0
y
y
-2
-2
-4
-4
-4 -2 0 2 4 -4 -2 0 2 4
x x
0.25
0.4
0 0.05 0.1 0.15 0.2
Z
4
4
2
4 2
4
0 2
2
Y 0
0 Y 0
-2
X -2
X
-2
-2
-4 -4
-4 -4
3 / 27
Multivariate Calculus and optimization
4 / 27
Multivariate Calculus and optimization
A useful counterexample
5 / 27
Multivariate Calculus and optimization
Examples
∂f (x) ∂f
= 3x12 x2 + 2x1 x22 + 1, = x13 + 2x12 x2 + 2x2
∂x1 ∂x2
∂f (x) ∂f (x)
= 2a11 x1 + 2a12 x2 = (2Ax)1 and = (2Ax)2
∂x1 ∂x2
7 / 27
Multivariate Calculus and optimization
8 / 27
Multivariate Calculus and optimization
R(x,z)
where it holds that limkzk→0 kzk = 0.
Idea. The function f can be approximated locally around x by the affine
mapping y 7→ f (x) + ∇f (x)t (y − x).
9 / 27
Multivariate Calculus and optimization
Chain rule
d d
f (φ(t)) = (∇f (x + tv ))t v , in particular f (φ(0)) = (∇f (x))t v .
dt dt
10 / 27
Multivariate Calculus and optimization
Second derivatives
11 / 27
Multivariate Calculus and optimization
12 / 27
Multivariate Calculus and optimization
Example
13 / 27
Multivariate Calculus and optimization
1
f (y ) − f (x) = ∇f (x)t (y − x) + (y − x)t Hf (x)(y − x) + R2 (x, y − x)
2
R2 (x,z)
where limkzk→0 kzk2
= 0.
Idea. f can be approximated locally around x ∈ U by the quadratic
function
1
y 7→ f (x) + ∇f (x)t (y − x) + (y − x)t Hf (x)(y − x) .
2
Locally, this is a better approximation than the first order Taylor
approximation.
14 / 27
Multivariate Calculus and optimization
15 / 27
Multivariate Calculus and optimization
Example
An easy computation gives for the Hessian that Hf (x) = 2A. Hence we
need to check the definiteness of A.
Approach via eigenvalues. The characteristic polynomial of A is
P(λ) = λ2 + 2λ; the equation P(λ) = 0 has solutions (eigenvalues)
−2 and 0. Hence, A is negative semidefinite and the function is
concave.
16 / 27
Multivariate Calculus and optimization
Example
Problem. Let f (x1 , x2 ) = x12 − x22 − x1 x2 − x13 . Find the largest domain in
which f is concave.
We look for the maximal domain where A(x) is negative semidefinite. Now
a b
a symmetric 2 × 2 matrix A = is negative semidefinite if and only
b c
if a ≤ 0, c ≤ 0 and det(A) ≥ 0. This gives the conditions
m1 = 2 − 6x1 ≤ 0 and m2 = det(A) = 12x1 − 5 ≥ 0 (as c < 0). Therefore
the solution is
17 / 27
Multivariate Calculus and optimization
Optimization problems
18 / 27
Multivariate Calculus and optimization
19 / 27
Multivariate Calculus and optimization
20 / 27
Multivariate Calculus and optimization
21 / 27
Multivariate Calculus and optimization
22 / 27
Multivariate Calculus and optimization
23 / 27
Multivariate Calculus and optimization
∇f (x ∗ ) = Qx ∗ − b = 0, Hf (x ∗ ) = Q positive semi-definite;
∇f (x ∗ ) + λ∗ ∇h(x ∗ ) = 0.
25 / 27
Multivariate Calculus and optimization
∇f (x ∗ )0 y = λ∗ ∇h(x ∗ )0 y = 0.
26 / 27
Multivariate Calculus and optimization
Examples
27 / 27