Pascal Gourdel Pascal Gourdel: Weierstrass's Theorem

The one The one
dimensional dimensional
results results
Pascal Pascal
Gourdel Gourdel The main starting point of existence results is Weierstrass’s
Existence Existence Theorem and its corollaries.
results results
Weierstrass’s
Theorem 1 (Weierstrass)
First order
condition The one dimensional results Theorem
coercivity
Convex and First order If the domain is compact, and if f is continuous then f is
concave condition
functions bounded and the extremum are reached.
Pascal Gourdel1 Convex and
concave
functions
Université Paris 11
October 18, 2008 The proof is based on the notion of maximizing sequence
(respectively minimizing). When the domain is not bounded,
numerous problems can be solved using the coercivity
condition.
Pascal Gourdel (Université Paris 1) The one dimensional results October 18, 2008 1 / 26 Pascal Gourdel (Université Paris 1) The one dimensional results October 18, 2008 3 / 26
coercivity coercivity
The one The one

results results
Let f : R → R be a continuous function, and let us consider
Pascal
Gourdel two optimisation problems.
Pascal
Gourdel
Definition 2
The function f : R → R is said coercive if
Existence Existence
results results

Weierstrass’s
Theorem min f (x) min f (x) Weierstrass’s
Theorem lim f (x) = lim f (x) = +∞.
coercivity (P1 ) (P2 ) coercivity x→−∞ x→+∞
x ∈ [a, b] x ∈R
First order First order
condition condition
The previous example
Convex and We can apply Weierstrass’s Theorem in order to prove that Convex and
f (x) = x 8 − x 7 + x 6 + x 3 (1 + sin x) + 4x, corresponds to a
concave concave
functions Sol(P1 ) is nonempty but what about Sol(P2 ) ? functions
coercive function.
Let us consider by example Exercise 1

Let us assume that f : R → R is coercive and continuous,
f (x) = x 8 − x 7 + x 6 + x 3 (1 + sin x) + 4x show that f has a lower bound and that there exists at least
a minimum.
It is obvious that maxa∈R f (x) has no solutions.
The one The one
results results
Pascal
By extension, we can adapt this definition for example for Pascal Proposition 1
Gourdel Gourdel
functions defined on [0, +∞[ by requiring that Let −∞ ≤ a < b ≤ ∞, we let I = [a, b] ∩ R and we consider
Existence Existence
results results f : I → R differentiable on I, if we assume that the problem
Weierstrass’s
lim f (x) = +∞. First order
Theorem
coercivity
x→+∞ condition
maxx∈I f (x) has a local solution x ∈ R, then
if x ∈ ]a, b[, then f 0 (x) = 0,
Necessary condition
First order Alternative
1
condition We can also use a change of variable in order to study a formulation in terms
if x = a, then f 0 (x) ≤ 0,
of multipliers
2
Convex and
concave
problem defined on ]1, +∞[. qualification
functions Convex and

concave
3 if x = b, then f 0 (x) ≥ 0.
functions

min f (x) min g(y ) = f (1 + exp(y ))
(P1 ) (P2 ) Exercise 2
x ∈ ]1, +∞[ y ∈R
1) Prove Proposition 1.
x ∈ Sol(P1 ) if and only if ln(x − 1) ∈ Sol(P2 ) 2) Let us fix some real number a ∈ R. We assume that f (a)
y ∈ Sol(P2 ) if and only if ey + 1 ∈ Sol(P1 ) exists. Show that a is a solution (even a global solution) of
maxx∈[a,a] f (x), can we deduce a condition on f 0 (a) ?
Alternative formulation in terms of multipliers
The one The one

results results
Pascal Pascal We can state Proposition 1 under the following version.
Gourdel Gourdel
Proposition 2
Existence Let us focus on the previous problem when the domain is Existence
results results
bounded (a and b finite). The problem maxx∈[a,b] f (x) can Let −∞ < a < b < ∞, we let I = [a, b] and we consider
condition be formulated as condition
f : I → R differentiable on I, we assume that the problem
Necessary condition Necessary condition
(P1 ) has a local solution x, then there exist multipliers

Alternative
  Alternative
formulation in terms
of multipliers
 max f (x)  max f (x) formulation in terms
of multipliers
qualification
(P) x −b ≤0 i.e (P1 ) g (x) ≤ 0 qualification λ1 ≥ 0 and λ2 ≥ 0 such that
 1
Convex and
a−x ≤0 g2 (x) ≤ 0 Convex and

concave
functions
concave
functions f 0 (x) = λ1 g10 (x) + λ2 g20 (x)
with the notations
g1 (x) = x − b and λ1 g1 (x) = λ2 g2 (x) = 0.
g2 (x) = a − x
Exercise 3
Deduce Proposition 2 from Proposition 1.
The one The one
results results Definition 3
Pascal Pascal
Gourdel
Be careful, if we consider the optimization problem, Gourdel Let us consider the following domain, the set O is open and
Existence Existence gi are continuous on O.
results results
max 3x
First order (P) (1) First order

x ∈O
condition g1 (x) = x 2 ≤ 0 condition 

g1 (x) ≤ 0
Necessary condition Necessary condition

Alternative Alternative
of multipliers It is easy to prove that x = 0 is the unique solution of (P) formulation in terms
of multipliers 
 ...
qualification qualification
but nevertherless the system gp (x) ≤ 0

Convex and Convex and
concave  0 concave
functions
 f (x) = λ1 g10 (x) functions
W say that the Slater condition is satisfied for this list of
λ ≥0 constraints if there exists a feasible point x̂ such that x̂ ∈ O,
 1
λ1 g1 (x) = 0 and for each i = 1, . . . , p,
has no solution. 
 either gi is an affine function and gi (x̂) ≤ 0
or
gi is a convex function and gi (x̂) < 0

The one The one

results results
Pascal Pascal
Gourdel Gourdel
Existence Exercise 4 Existence

results results
First order
Let f : [a, b] → R, let us consider the optimization problems First order
Exercise 5
condition condition
Necessary condition
  Necessary condition Let us consider again the problem
Alternative
 max f (x) ∈ R  max f (x) Alternative
(P1 ) x −b ≤0 (P2 ) (x − b)3 ≤ 0

of multipliers of multipliers

qualification qualification max 3x
a−x ≤0 a−x ≤0 (P)
g1 (x) = x 2 ≤ 0
 
concave concave
functions functions
1) Compare the two problems. Show that this system does not satisfy the Slater condition.
2) Show that the Slater condition is satisfied for (P1 ).
3) Show that the Slater condition is NOT satisfied for (P2 ).
The one The one
In this part, f is a real-valued function defined on U convex
results
Proposition 3
results
subset (interval) of R.
Pascal Pascal
Gourdel Gourdel
Definition 4
Let x be a local solution of the optimization problem:
Existence Existence
results  results The function f is convex (resp. concave) if for all (x, y ) ∈ U × U
First order
max f (x) First order
and for all λ ∈ [0, 1],


condition
 x ∈O condition


Necessary condition
Convex and
Alternative
(P) g1 (x) ≤ 0 concave f (λx + (1 − λ)y ) ≤ λf (x) + (1 − λ)f (y ).
of multipliers
functions
...


qualification 
 Definitions
gp (x) ≤ 0

Convex and
concave
(respectively ) f (λx + (1 − λ)y ) ≥ λf (x) + (1 − λ)f (y ).
functions
Let us assume that the Slater condition is satisfied and that The function f is strictly convex if for all (x, y ) ∈ U × U such
each function is differerentiable on O, then, there exists that x 6= y and for all λ ∈ ]0, 1[,
(λi )pi=1 ∈ Rp+ such that f (λx + (1 − λ)y ) < λf (x) + (1 − λ)f (y ).
f 0 (x) = λ1 g10 (x) + . . . + λp gp0 (x)

A function f is convex if and only if −f is concave.
λi gi (x) = 0 for all i = 1, . . . , p Consequently, the results obtained for convex functions can
be translated in terms of concave functions. These notions
can be defined locally.
The one The one

dimensional Definition 5 dimensional
results results
Pascal The epigraph and the hypograph (denoted by epi(f ) and hypo(f )) Pascal
Gourdel Gourdel
of a function f are the sets (see figure 1) defined by
Existence Existence
results results
epi(f ) = {(x, t) ∈ U × R | t ≥ f (x)}. Theorem 6
condition condition
The three following properties are equivalent:
Convex and hypo(f ) = {(x, t) ∈ U × R | t ≤ f (x)}. Convex and
concave concave (i) f is convex (resp. concave);
functions functions
Definitions Definitions (ii)Pfor all k ≥ 2, (xi ) ∈ U k P
and λ ∈ S k ,
f ( i=1 λi xi ) ≤ (resp. ≥) ki=1 λi f (xi );
k
(iii) The epigraph (resp. the hypograph) of f is a convex

E(f ) subset of R2 .
Cf
H(f )
Figure: graph, epigraph and hypograph
The one The one
Proposition 4
results results
Pascal
Examples: Pascal if f is convex (resp. concave) and if x < x 0 < x 00 are
Gourdel Gourdel
elements of I, then (figure 2),
Existence
1 Let us recall that an affine function is a function R → R, Existence
results such that results
f (x) − f (x 0 ) f (x) − f (x 00 ) f (x 00 ) − f (x 0 )
First order
f (x) = αx + β. First order ≤ ≤
condition condition x − x0 x − x 00 x 00 − x 0
concave Any affine function is both convex and concave, but not concave .
functions functions
Definitions
strictly. It can be easily proved that if f is convex and Definitions b
concave on U, then it is the restriction on U of an affine
function.
2 |.| is a convex function. b
3 If C is a nonempty convex convex subset of R, the
distance to C defined by dC (x) = inf{|x − c| such that
c ∈ C} is convex. b
O x x′ x′′
Figure: Monotonicity of the slope of a convex function

The one The one

results Corollary 7 results
Pascal
Gourdel
Pascal
Gourdel
Corollary 8
Let f be a convex function on some interval U,
Existence 1) if x1 < x2 < x3 < x4 < x5 are elements of U, then Existence Let f be a convex and differentiable function on an open
results results
interval containing a and b. If x1 < x2 are elements from
condition f (x1 ) − f (x2 ) f (x2 ) − f (x3 ) f (x3 ) − f (x4 ) f (x4 ) − f (x5 ) condition ]a, b[, then
≤ ≤ ≤
Convex and x1 − x2 x2 − x3 x3 − x4 x4 − x5 Convex and f (x2 ) − f (x1 )
concave concave f 0 (a) ≤ ≤ f 0 (b).
functions functions x2 − x1
Definitions Definitions
2) The right and left derivatives exist on the interior of U and In particular f is lipschitz on [a, b]
moreover if x1 < x2 are interior points, then
f (x2 ) − f (x1 )
fr0 (x1 ) ≤ ≤ f`0 (x2 ). Corollary 9
x2 − x1
3) The right and left derivatives are non decreasing on the Let f be a differentiable function on the interval U, f is
interior of U, and more over if x is an interior point, convex on U if and only if f 0 is non decrasing on U.
f`0 (x1 ) ≤ fr0 (x1 ).
The one The one
results results
Pascal Pascal
Gourdel Gourdel Exercise 6
Existence
Corollary 10 Existence 1) Show that the function f defined by f (x) = x 4 is strictly
results results
Let f be a twice differentiable function on the interval U, f is convex on R but nevertheless the second derivative is not
condition
convex if and only if for all x ∈ U, f 00 (x) ≥ 0. condition always positive on R since f 00 (0) = 0.
concave concave
functions
Be careful, f may be discontinuous on the boundary of the functions 2) Let f be a twice differentiable function on the interval U.
Show that if for all x ∈ U, f 00 (x) > 0, then f is strictly convex
Definitions Definitions
domain. For example,

on U.

1 if x = 0
f (x) = 3) Let f be a C 2 function on U a neighborhood of x, and
0 if x > 0
assume that f 00 (x) > 0. Show that there exists V a (possibly
is convex on R+ . smaller) neighborhood of x such that f is strictly convex on
V.
The one In the following picture, f 00 (0) = 0.

dimensional
results
Pascal
Gourdel
Existence
results 0,16
First order
condition
Convex and 0,12

concave
functions
Definitions
0,08
0,04
-0,12 -0,08 -0,04 0 0,04 0,08 0,12
Figure: f (x) = x 2 (1 + sin(1/x)), g(x) = 2x 2
Pascal Gourdel (Université Paris 1) The one dimensional results October 18, 2008 26 / 26

Pascal Gourdel Pascal Gourdel: Weierstrass's Theorem

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Pascal Gourdel Pascal Gourdel: Weierstrass's Theorem

Uploaded by

Copyright:

Available Formats

The one The one

The one The one

Let us consider by example Exercise 1

functions Convex and

Alternative formulation in terms of multipliers

The one The one

(P1 ) has a local solution x, then there exist multipliers

The one The one

Existence Exercise 4 Existence

(P1 ) x −b ≤0 (P2 ) (x − b)3 ≤ 0

f 0 (x) = λ1 g10 (x) + . . . + λp gp0 (x)

The one The one

(iii) The epigraph (resp. the hypograph) of f is a convex

Figure: graph, epigraph and hypograph

Figure: Monotonicity of the slope of a convex function

The one The one

domain. For example,

The one In the following picture, f 00 (0) = 0.

Convex and 0,12

-0,12 -0,08 -0,04 0 0,04 0,08 0,12

Figure: f (x) = x 2 (1 + sin(1/x)), g(x) = 2x 2

You might also like