Piecewise Linear Convex Objective Functions

Graphical Representation and Solution

Optimization Methods and Applications

Minati De

Department of Mathematics, Indian Institute of Technology Delhi, India.

Lecture 3: Introduction to LP (contd.)

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

General Form

Minimize c T x
Subject to ai T x ≥ bi , i ∈ M1 ,
ai T x ≤ bi , i ∈ M2 ,
ai T x = bi , i ∈ M3 ,
xj ≥ 0, j ∈ N1 ,
xj ≤ 0, j ∈ N2 .

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Table of Contents

1 Piecewise Linear Convex Objective Functions

2 Graphical Representation and Solution

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Convex Function

A function f : Rn → R is called convex if for every x, y ∈ Rn , and

every λ ∈ [0, 1], we have

f (λx + (1 − λ)y ) ≤ λf (x) + (1 − λ)f (y ).

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Concave Function

A function f : Rn → R is called concave if for every x, y ∈ Rn , and

every λ ∈ [0, 1], we have

f (λx + (1 − λ)y ) ≥ λf (x) + (1 − λ)f (y ).

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Why Convex (or concave) Functions are important in

Optimization?

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Why Convex (or concave) Functions are important in

Optimization?

Local and Global minimum

A vector x is a local minimum of f if f (x) ≤ f (y ) for all y in the
vicinity of x.
A vector x is a global minimum of f if f (x) ≤ f (y ) for all y .
Question: For a convex function, can there be a local minimum
that fail to be global minimum?

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Convex Function

Theorem
Let f1 , . . . , fm : Rn → R be convex functions. Then the function f
defined by f (x) = maxi=1,...,m fi (x) is also convex.

Proof?

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Piecewise Linear Convex Function

Definition
A function of the form maxi=1,...,m (ciT x + di ) is called a piecewise
linear convex function.
Example: The absolute value function f (x) = |x| = max{x, −x}.

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Piecewise Linear Convex Objective Function

A generalization of LP, where the objective function is piecewise

linear and convex:

Minimize maxi=1,...,m (ciT x + di )

Subject to Ax ≥ b

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Piecewise Linear Convex Objective Function

Minimize maxi=1,...,m (ciT x + di )

Subject to Ax ≥ b

Note that maxi=1,...,m (ciT x + di ) is equal to the smallest number z

that satisfies z ≥ ciT x + di for all i.

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Piecewise Linear Convex Objective Function

Minimize maxi=1,...,m (ciT x + di )

Subject to Ax ≥ b

is equivalent to

Minimize z
Subject to z ≥ ciT x + di , i = 1, . . . m
Ax ≥ b

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Piecewise Linear Convex Objective Function

Minimize maxi=1,...,m (ciT x + di )

Subject to Ax ≥ b

is equivalent to

Minimize z
Subject to z ≥ ciT x + di , i = 1, . . . m
Ax ≥ b

What are the decision variables?

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Example 1: Problems involving absolute values

P
Minimize i=1,...,n ci |xi |
Subject to Ax ≥ b

Q1: Is the objective function piecewise linear?

Q2: If yes, how to obtain the LP formulation?

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Example 1: Problems involving absolute values

P
Minimize i=1,...,n ci |xi |
Subject to Ax ≥ b

Observation
|xi | is the smallest number zi that satisfies xi ≤ zi and −xi ≤ zi .

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Example 1: Problems involving absolute values

P
Minimize i=1,...,n ci |xi |
Subject to Ax ≥ b

is equivalent to
P
Minimize i=1,...,n ci zi
Subject to Ax ≥ b
xi ≤ zi , i = 1, . . . n
−xi ≤ zi , i = 1, . . . n.

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Example 2: Chebyshev approximation problem

Chebyshev approximation problem:

Minimize maxki=1 |aiT x − bi |

Here x ∈ Rn is the variable, and a1 , . . . , ak ∈ Rn , b1 , . . . , bk ∈ R

are parameters that specify the problem instance.

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Chebyshev approximation problem

Chebyshev approximation problem in LP form:

Minimize t
Subject to aiT x − t ≤ bi , i = 1, . . . k,
−aiT x − t ≤ −bi , i = 1, . . . k.

with variables x ∈ Rn and t ∈ R.

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

Table of Contents

1 Piecewise Linear Convex Objective Functions

2 Graphical Representation and Solution

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

What does the feasible region of an LP look like?

M. De Optimization Methods and Applications

 Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

What does the feasible region of an LP look like?

M. De Optimization Methods and Applications

Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

M. De Optimization Methods and Applications

Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

M. De Optimization Methods and Applications

Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

M. De Optimization Methods and Applications

Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

M. De Optimization Methods and Applications

Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

M. De Optimization Methods and Applications

Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

M. De Optimization Methods and Applications

Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

M. De Optimization Methods and Applications

Piecewise Linear Convex Objective Functions
Graphical Representation and Solution

M. De Optimization Methods and Applications