Professional Documents
Culture Documents
Chapter 3
Chapter 3
(y, f (y))
(x, f (x))
• f is concave if −f is convex
• f is strictly convex if dom f is convex and
convex:
• affine: ax + b on R, for any a, b ∈ R
• exponential: eax, for any a ∈ R
• powers: xα on R++, for α ≥ 1 or α ≤ 0
• powers of absolute value: |x|p on R, for p ≥ 1
• negative entropy: x log x on R++
concave:
• affine: ax + b on R, for any a, b ∈ R
• powers: xα on R++, for 0 ≤ α ≤ 1
• logarithm: log x on R++
extended-value extension f˜ of f is
• dom f is convex
• for x, y ∈ dom f ,
f (y)
f (x) + ∇f (x)T (y − x)
(x, f (x))
f is twice differentiable if dom f is open and the Hessian ∇2f (x) ∈ Sn,
2 ∂ 2f (x)
∇ f (x)ij = , i, j = 1, . . . , n,
∂xi∂xj
Strict convexity
• If and only if domf is convex and for any x , y ∈ domf , x 6= y
f (y ) > f (x ) + ∇f (x )T (y − x ).
Strict convexity
• If and only if domf is convex and for any x , y ∈ domf , x 6= y
f (y ) > f (x ) + ∇f (x )T (y − x ).
convex if P 0
least-squares objective: f (x) = kAx − bk22
f (x, y)
T 1
2 y y
∇2f (x, y) = 3 0
y −x −x 0
2 2
1 0
convex for y > 0 y 0 −2 x
1 1
∇2f (x) = diag(z) − zz T
(zk = exp xk )
1T z (1T z)2
to show ∇2f (x) 0, we must verify that v T ∇2f (x)v ≥ 0 for all v:
2
zk ) − ( k v k zk ) 2
P P P
T 2 ( k zk vk )( k
v ∇ f (x)v = ≥0
( k zk ) 2
P
2 2
P P P
since ( k v k zk ) ≤( k zk vk )( k zk ) (from Cauchy-Schwarz inequality)
Qn 1/n n
geometric mean: f (x) = ( k=1 x k ) on R ++ is concave
α-sublevel set of f : Rn → R:
Cα = {x ∈ dom f | f (x) ≤ α}
epigraph of f : Rn → R:
epi f
f (E z) ≤ E f (z)
prob(z = x) = θ, prob(z = y) = 1 − θ
examples
X
m
f (x) = − log(bi − aTi x), dom f = {x | aTi x < bi, i = 1, . . . , m}
i=1
examples
is convex
examples
• support function of a set C: SC (x) = supy∈C y T x is convex
• distance to farthest point in a set C:
f (x) = sup kx − yk
y∈C
λmax(X) = sup y T Xy
kyk2 =1
composition of g : Rn → R and h : R → R:
f (x) = h(g(x))
examples
• exp g(x) is convex if g is convex
• 1/g(x) is convex if g is concave and positive
composition of g : Rn → Rk and h : Rk → R:
examples
Pm
• i=1 log gi (x) is concave if gi are concave and positive
Pm
• log i=1 exp gi(x) is convex if gi are convex
is convex
examples
• f (x, y) = xT Ax + 2xT By + y T Cy with
A B
0, C≻0
BT C
g is convex if f is convex
examples
• f (x) = xT x is convex; hence g(x, t) = xT x/t is convex for t > 0
• negative logarithm f (x) = − log x is convex; hence relative entropy
g(x, t) = t log t − t log x is convex on R2++
• if f is convex, then
T T
g(x) = (c x + d)f (Ax + b)/(c x + d)
f (x)
xy
(0, −f ∗(y))
Sα = {x ∈ dom f | f (x) ≤ α}
a b c
• f is quasiconcave if −f is quasiconvex
• f is quasilinear if it is quasiconvex and quasiconcave
is quasiconvex
∇f (x)
x
1 1 T
Σ−1 (x−x̄)
f (x) = p e− 2 (x−x̄)
(2π)n det Σ
Suppose f : R → R is increasing and convex on its domain (a, b). Let g denote
its inverse. Is g(·) a convex function, a concave function, or both, or neither?
on {x ∈ Rn |T (x , ω) > 0, 0 ≤ ω ≤ 2π}.
2. f (x ) = − log det(A0 + x1 A1 + x2 A2 + ... + xn An ) where Ai ∈ Sm , on
{x |A0 + x1 A1 + x2 A2 + ... + xn An ≻ 0}
Pr
3. f (x ) = i=1 |x |[i] where |x |[i] is the i − th largest absolute value in the
components of the vector x .
Pm aiT x +bi
Pm T
ai x +bi
4. f (x ) = − log(− log( i=1
e )) on {x | i=1
e ) < 1}.