Introduction to Mathematical Optimization

Insoon Yang

Department of Electrical and Computer Engineering

Seoul National University

CORE
Control + Optimization Research Lab

1 / 11
Mathematical optimization

A mathematical optimization problem has the form

min f0 (z)
z∈Rn (1)
s.t. fi (z) ≤ bi , i = 1, . . . , m,

where
z := (z1 , . . . , zn ): optimization variable
f0 : Rn → R: objective function
fi : Rn → R: constraint functions

Note: It can also contain equality constraints (Why?)

2 / 11
Example

What is the optimization variable?

What is the objective function?
What are the constraints?
3 / 11
Optimal solution

Definition
A vector z ? ∈ Rn is called an optimal solution of (1), if it achieves the
smallest objective value among all vectors satisfy the constraints, i.e.,

f0 (z ? ) ≤ f0 (y)

for any y ∈ Rn such that fi (y) ≤ bi , i = 1, . . . , m.

4 / 11
Two important classes
Linear programming (LP):
f0 and fi ’s are linear
min c> z
z∈Rn
s.t. Gz ≤ w
Az = b

Quadratic programming (QP):

f0 is convex quadratic and fi ’s are linear
1 >
min z Hz + q > z + r
z∈Rn 2
s.t. Gz ≤ w
Az = b,
where H = H >  0.
These are convex optimization problems!
5 / 11
Convex functions
Definition
A function f : Rn → R is said to be convex provided that dom f is a
convex set and

f (θx + (1 − θ)y) ≤ θf (x) + (1 − θ)f (y) ∀x, y ∈ dom f, ∀θ ∈ [0, 1].

Figure: Graph of a convex function. The chord (i.e., line segment) between any
two points on the graph lies above the graph.

Convexity is probably the most well-understood nonlinearity.

6 / 11
Convex Sets

Definition
A set C is said to be convex if the line segment between any two points in
C lies in C, i.e.,

θx1 + (1 − θ)x2 ∈ C ∀x1 , x2 ∈ C, ∀θ ∈ [0, 1].

Every affine set is convex.

Which of the examples in Figure 2 are convex?

Figure: Some simple convex and nonconvex sets.

7 / 11
Convex optimization problems

The standard form of convex optimization problems is

min f0 (x)
s.t. fi (x) ≤ 0, i = 1, . . . , m
a>
i x = bi , i = 1, . . . , p,

where f0 , . . . , fm are convex functions. Note that

the objective function must be convex
the inequality constraint functions must be convex
the equality constraint functions hi (x) = a>
i x − bi must be affine

Note: More generally, a mathematical optimization problem is said to be

convex if its cost function is convex and its feasible set is convex.

8 / 11
Why do we care about convex optimization problems?

For some classes of convex optimization problems, we have a mature

(software) technology to find an optimal solution (e.g., least squares,
linear programs).

For general convex optimization problems, there are very effective

methods for solving them (e.g., interior-point methods).

We can use convex optimization to study nonconvex optimization

problems.

Note: Recognizing convex optimization problems can be difficult. But

once the skill of recognizing or formulating convex optimization problems
is developed, you will find that surprisingly many problems can be solved
via convex optimization.

9 / 11
Why do we care about linear and quadratic optimization
problems?

Linear optimization ⊂ Quadratic optimization ⊂ Convex optimization

LP and QP are the most tractable classes of convex optimization

problems: well established theory and algorithms

The solution technology for LQ and QP is almost mature

(many commercial solvers, e.g., CPLEX).

Many MPC problems can be formulated as LP and QP!

10 / 11
Example

What is the type of this MPC problem?

11 / 11