Slide-1 Convex Set

Convex Optimization Theory and
Applications
Topic 1 - Convex Set
Li Li
Department of Automation
Tsinghua University
Fall, 2009-2022.
-1-
1.0. Outline 纲要
1.-1. Preliminary 预备知识
1.1. Definition and Examples 定义和例子
1.2. Generalized Inequalities 广义不等式
1.3. Operations that Preserve Convexity 保凸运算
1.4. Separating Hyperplane Theorem 凸集分离
1.5. Some Quiz Problems 测试问题
-2-
非正式地说，向量 x 的范数 x p 是这个向量的某种长度度
i1 i ，写成向
n
量，以第二范数/欧几里得范数为例， x 2 
2
x

2 T
x
量形式为 2 x x。
正式地说，向量 x 的范数 x p 是满足以下四个条件的任何函

数 f : n
 ：
 非负性：对于所有 x  ，有 f ( x)  0
n
 正定性：当且仅当 x  0 的时候 f ( x)  0
 齐次性：对于所有 x  n
, t  ， f (tx) | t | f ( x)
 三角不等式：对于所有 x, y  n
， f ( x  y )  f ( x)  f ( y )
-3-
mn
仿射子空间（affine subspaces）：给定矩阵 A  以及向
量b  m
，仿射子空间为集合{x  n
: Ax  b}
仿射变换（ affine transformation ）：从 n 到 m 的变换
y  A x c被称为仿射变换，其中 y  m ，A  mn ，c  m 为
常向量
仿射函数（affine function）
：仿射函数为仿射变换的一种特
殊情況，当仿射变换中的 m  1时， AT  n ， y, c 都为常数；
如果进一步，将此处 y  Ax  c 中的截距 c 去掉，剩下的部分
y  Ax 被称为线性函数（linear function）
-4-
为什么我们从凸集开始
布尔巴基学派认为“数学结构勾画了数学体系内部的关系，
数学结构简称为结构，结构由若干集合，定义在集合上或集
合间的一些关系，以及一组作为条件的公理组成”
--《数学辞海》编辑委员会《数学辞海（第一卷）》
希望大家在学习的过程中，能逐渐熟悉相对抽象的性质和
定理，同时又掌握足够多的相对具象的实例
-5-
点Point，在我们课中，如无特殊说明，指欧式空间中的点
线Line：through given two distinct points x1 and x 2 , Set C

consists of any point of the form x  1 x1   2 x2 , 1 , 2  R .
仿射集Affine set：Set C contains the line through any two

distinct points in the set C
It is easy to show that every affine set is convex
-6-
The solution set of any linear equations is an affine set.
Let S be the solution of a linear equation. By definition, we

have S  {x  n : Ax  b}.
Let x1 , x2  S  Ax1  b and Ax2  b .
To prove: A[ x1  (1   ) x2 ]  b,   , let us check
A[ x1  (1   ) x2 ]   Ax1  (1   ) Ax2   b  (1   )b  b
Thus S is an affine set.
-7-
清华大学《凸优化》课程 2022年作业题
(Inversely) Every affine set can be expressed as the solution

set of system of linear equations
-8-
凸集 Convex Set，Set C is a convex set, if for any x, y  C ,
and   0,1 , we have  x  1    y  C
典型的凸集包括：
 点 Point，线 Line，射线 Ray，面 Hyperplane，半空间 Half
Space
 锥 Cone
 球 Ball，多面体 Polyhedral P1  {x | Ax  b}
-9-
球 Ball、超平面 Hyperplane、半平面 Half Space 都是凸集
作为超平面、半平面的交集的可行域是多面体 Polyhedron，
多面体是凸集
基于球的概念定义边界点 Boundary Point，顶点 Extreme

Point
- 10 -
球 Ball，开球 OpenBall，闭球 Closed Ball
- 11 -
开球和闭球都是凸集
- 12 -
超平面 Hyperplane
- 13 -
超平面比原空间的维度要低一维
半平面 Halfspace
- 14 -
半平面 Half Space 是凸集
- 15 -
边界点 Boundary Point
- 16 -
顶点/极点Extreme Point，如果凸集内的一点不在凸集内任
何不同的两点的连线上，则称该点为该凸集的顶点
- 17 -
多面体Polyhedron
凸集的交集也是凸集
- 18 -
A polyhedron is intersection of a finite number of halfspaces

and hyperplanes
- 19 -
多面体Polyhedron是凸集可以用下面的方法来证明：
大前提：线性规划问题的约束条件为若干个线性等式或者
不等式，而这些集合都是凸集
小前提：凸集的交集也是凸集
结论：作为这些凸集交集的线性规划问题定义域也是凸集
（我们约定空集为凸集）
- 20 -
锥 Conic (nonnegative) combination of x1 and x 2 : any point
of the form x  1 x1   2 x2 , with 1 ,  2  0 .
凸锥 Convex cone C : set that contains all conic combinations

of points in the set and convex. Thus, y  C if and only if there
is a solution of 1 ,  2  0 for y  1 x1   2 x2 .
- 21 -
锥 Cone 的性质
A set  n
is a cone if
x   x  ,   0
A set is a convex cone if is a cone and is convex.
If 1 and 2 are cones then  1 2 is a cone.
A cone is a 正常锥proper cone if

 is convex
 is closed
 is solid, i.e. it has non-empty interior
 is pointed, i.e. if x  and  x  then x  0
- 22 -
 1.1. Definition and Examples 定义和例子
- 23 -
尖锥 Pointed Cone 和多面体锥 Polyhedral Cone
- 24 -
- 25 -
Are they cones? Are they convex/proper/pointed cones?

Remember that a proper cone contains no line.
- 26 -
凸组合Convex Combination of x1 , , xk : any point x of the
form x  1 x1  2 x2    k xk ,with 1   2    k  1, i  0 .
凸包Convex Hull：The convex hull of a set  is defined to

be the intersection of all convex sets that contain  .
凸组合和凸包在某种意义上等价：The convex hull conv S

is the set of all convex combinations of points in S .
- 27 -
我们可以证明这两个定义的等价性
- 28 -
- 29 -
- 30 -
- 31 -
- 32 -
单纯形Simplex：We call the convex hull of any set of n  1
points in R n which do not lie on a hyperplane a simplex.
多胞形Polytope：A bounded polyhedron is called a polytope.
- 33 -
半正定矩阵Positive Semidefinite Matrix
n
The set of real symmetric n n matrices is denoted S . A
matrix A S is called positive semidefinite if x Ax  0 for
n T
all x  R , and is called positive definite if x Ax  0 for all

n T
nonzero x  R . The set of positive semidefinite matrices is

n
n
S
denoted  and the set of positive definite matrices is denoted
n n
S S
by   .  is a closed, convex, pointed, and solid cone.
There are several equivalent conditions for a matrix to be

positive (semi)definite.
In the follows, we will focus on semipositive matrices but the

conclusions can be extended to positive matrices.
- 34 -
Proposition 1.1: The following statements are equivalent:
n
1) matrix A S is positive semidefinite
2) for all x  R , x Ax  0
n T
3) all the eigenvalues of A are nonnegative
4) all principal minors of A are nonnegative
5) there exists a factorization A  BT B
- 35 -
Proof: 1) and 2) are equivalent by definition. 3) and 4) are
obvious according to Lemma 1.1 below.
Define A  BT B . For any x  R , x Ax  x B Bx  ~ x ~

x 0,
n T T T T
where ~x  Bx . So we can derive 5) from 1).

On the other side, based on on 5), there exist A  P T
P  QT
Q ,
where P   Q ,  is the diagonal matrix containing all the
1/ 2 T
eigenvalues of A . This indicates 5) leads to 1).
Corollary 5.1: Every psd (pd) matrix A has a unique psd (pd)
square root A1 / 2 such that A1/ 2 A1/ 2  A and every pd matrix is
nonsingular.
- 36 -
Proposition 1.2: The following statements are equivalent:
n
1) matrix A S is positive
2) for all x  R , x Ax  0
n T
3) all the eigenvalues of A are positive
4) all principal minors of A are positive
5) there exists a factorization A  BT B (any else condition

on B ?)
- 37 -
Lemma 1.1: Any a real Hermitian matrix A can be
diagonalized by an orthogonal matrix.
Proof:
1) all the eigenvalues and eigenvectors are real, notice that
 ( A)   ( AT )
2) the eigenvectors of different eigenvalues are orthogonal.
Suppose 1 1  X  AX 1,  X
2 2  AX 2, 1  2, X 1 X 2  0 , because
T
1 X1T X 2  X1T AX 2  X 2T AX 1  2 X 2T X1 , 1  2 X1T X 2  0

3) if  is a r th multiple root of the characteristic equation
of A , we can prove that rank ( A  I )  n  r . Thus,  has r
linear independent eigenvectors.
- 38 -
4) based on 2) and 3), we can see that A has n linear
independent eigenvectors. Via Gram-Schmidt orthogonalization,
we can make these eigenvectors orthogonal. So it is proved.
Another Proof:
According to the Schur triangularization theorem there is a
unitary matrix U such that U AU  R , where R is upper
T
triangular. But, moreover, we have R T

 (U T
AU ) T
 U T
AU  R .
Therefore, R is both upper and lower triangular. This makes
R a diagonal matrix.
- 39 -
半正定锥 Positive Semidefinite Cone：The set of all PSD
matrices A  Rnn forms a convex cone.
x y
A
For example, let y z  , we have
- 40 -
Is the following set convex?
x1  0 , x1 x2  1
- 41 -
Yes
 x1 1
1  0
 x2 
- 42 -
x2  x12
- 43 -
Yes
1 x1 
x  0
 1 x2 
- 44 -
x12  x22  1
- 45 -
Yes
1  x1 x2 
 x  0
 2 1  x1 
- 46 -
x2  x12 or x1  0 , x2  0
- 47 -
Yes, but cannot be written into a SDP (LMI) form, because
not basic semialgebraic (a semialgebraic set is a subset S of
real n -dimensional space defined by a finite sequence of
polynomial equations and inequalities; or any finite union of
such sets.)
- 48 -
1  2 x1  x12  x22  2 x13  0
- 49 -
No because not connected
- 50 -
1  2 x1  x12  x22  2 x13  0 and x1  0.5
- 51 -
Yes
1 x1 0 
x 1 x2   0
 1
 0 x2 1  2 x1 
- 52 -
x14  x24  1
- 53 -
Yes, but cannot be written into a SDP (LMI) form.
- 54 -
半正定矩阵和椭球/椭球面 Ellipse：Any a PSD matrix A is
corresponding to a special ellipse x Ax  I .
T
Suppose A can be diagonalized by U as U AU   . We

T
have the ellipse rewritten as the standard ellipse form y T

y  I ,
with y  Ux .
- 55 -
清华大学《凸优化》课程 2021年期末考试题
n n
如果一个定义在 R 上的集合 C 和任意一条 R 中的直线的
交集是凸集，请问该集合是否是凸集？并给出理由。
- 56 -
布尔巴基学派将数学结构分为三个元结构，分别是序结构、
代数结构和拓扑结构。其中序结构即由集合及在其上规定的
序关系/偏序关系Partial Ordering组成的数学结构。
Properties of partial order
 Additive property: If x y and u v , then x  u y  v

 Transitive property: If x y and y z , then x z
 Non-negative scaling: If x y , then  x  y for all   0
 Reflexive property: x x
 Antisymmetric property: If x y and y x , then x  y
- 57 -
特别对于一些凸集，我们也可以定义相应的序结构。比如
对于凸锥，有了射线而不是直线，则有了方向！
Every proper cone K in R n induces a partial ordering  K
defining 广义不等式generalized inequalities on R n
a  K b  a b  K
positive orthant R n : standard coordinatewise ordering or

componentwise inequality
x  R n y  xi  yi

Löwner partial order：We can also define the partial order

A  B , where matrix H  A  B  S n is positive, or
equivalently for all x  R n
, x T
( A  B) x  0 .
- 58 -
集合中极小/极大元
- 59 -
1.3. Operations Preserve Convexity 保凸运算
除了上述常见实例，如何分析手头的集合是否凸集？
- 60 -
- 61 -
- 62 -
设 A  mn ，证明如果 S  m 是一个凸集，那么
A1 ( S )  {x  n : Ax  S}也是一个凸集。
证明：我们取 x, y  A1 ( S ) ，那么会有 Ax  S , Ay  S 。取

t [0,1]，那么会有 A[tx  (1  t ) y]  tAx  (1  t ) Ay  S（因为 S 是
凸的）。这就可以推出 tx  (1  t ) y  A1 ( S ) ，也就证明完毕
了。
A1 ( S ) 一般称为原象集（pre-image）
。
- 63 -
，那么所有满足不等式 i 1 xi Ai
k
给定 A1 , , Ak , B  n

B的
点构成的集合为凸集。
这个问题要想解释清楚，使用常规的凸集证明方式即可，
但是有一种更加有趣的证明方法。
  ， f ( x)  B   i 1 xi Ai 那么这个时候我们会发
k
设f : k n
现
 k

f ( )  {x : f ( x)  }   x : B   xi Ai 0 
1 n n
 i 1 
而这个就是我们关注的那个集合。这个集合当然是凸集，
因为它是一个凸集在线性映射下的原象集。
- 64 -
- 65 -
1.4. Separating Hyperplane 凸集分离
分离超平面
- 66 -
支撑超平面
- 67 -
- 68 -
- 69 -
- 70 -
- 71 -
- 72 -
- 73 -
- 74 -
超平面分离定理：Suppose C and D are two convex sets in
R n that do not intersect, i.e., C D   . Then there exist a
a  Rn , a  0 and b  R such that aT x  b for all x  C and
aT x  b for all x  D . In other words, the affine function aT x  b
is nonpositive on C and nonnegative on D . The hyperplane
x | aT x  b is called a separating hyperplane for the sets C and
D , or is said to separate the sets C and D .
The separating hyperplane we constructed above satisfies the

stronger condition that aT x  b for all x  C and aT x  b for
all x  D . This is called strict separation of the sets C and D .
Simple examples show that in general, disjoint convex sets need
not be strictly separable by a hyperplane (even when the sets are
Closed).
- 75 -
不能用超平面严格分离两个不相交的非空闭凸集的例子
 ,
C  x  R2 | x1  0  
D  x  R2 | x1 x2  1
- 76 -
For any a convex 2n -sided polyhedron in a R 2 plane.
Suppose we color any n vertices into red, the rest n vertices
into blue. Please prove or disprove there must be a point on the
plane so that the sum of its distance from all the red points is
equal to the sum of its distance from all the blue points.
- 77 -
At least one perpendicular foot of a point inside a convex

polygon to each edge of this convex polygon falls on an edge of
this convex polygon.
- 78 -
只需要证明离给定点最近的那条边的确满足投影点在边上
的要求就可以了。假设上图中的红色虚线是给定点到所有边
的垂线段中最短的一个，但垂足却在边的外面。我们立即发
现，存在某个边对应的蓝色垂足（请问为什么？），由于灰
色直角三角形中斜边大于直角边，蓝色垂线段显然要比红色
线段更短（注意：蓝线的垂线不一定要求落在边上），这就
与红色线段是所有垂线段中最短相矛盾。
这个命题对更高维的情形也都是成立的：对于给定凸多面
体和它内部的一点，总能找到其中一个面使得，给定点在这
个面上的投影恰好就落在这个面上。
- 79 -
A Gömböc is a convex three-dimensional homogeneous body
which, when resting on a flat surface, has just one stable and
one unstable point of equilibrium. Its existence was conjectured
by Russian mathematician Vladimir Arnold in 1995 and proven
in 2006 by Hungarian scientists Gábor Domokos and Péter
Várkonyi. The Gömböc shape is not unique; it has countless
varieties, most of which are very close to a sphere and all have
very strict shape tolerance (about 0.1 mm per 10 cm). The most
famous solution has a sharpened top and is shown on the right.
Its shape helped to explain the body structure of some turtles in
relation to their ability to return to equilibrium position after
being placed upside down.
- 80 -
The shape of the Indian Star Tortoise resembles a Gömböc.

This turtle rolls easily without relying much on its limbs.
- 81 -
Suppose you are given a collection of squares and equilateral
triangles (of unit side). You are asked to form convex polygons
by sticking the squares and triangles together with their sides
aligned. These polygons must also have unit sides (so that for
example sticking two squares together to form a domino does
not count). How many distinct polygons can be formed in this
way (including polygons formed by a single square or triangle)?
- 82 -
- 83 -
我们的学习要求：
 了解基本概念
 形成足够直觉
 完成简单题目（需要综合分析，代数和拓扑的基础知识）
我们是有选择性的介绍部分定理的详细证明，目的有两个：
其一，熟悉相关概念的性质；其二，学习相关的证明思路，
提高数学能力。我们的考试一般不涉及证明。
- 84 -
1.7. References
[1] S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge
University Press, 2004. http://www.stanford.edu/~boyd/cvxbook/
http://www.ee.ucla.edu/~vandenbe/cvxbook
[2] D. Bertsimas, J. N. Tsitsiklis, Introduction to Linear Optimization,
Athena Scientific, 1997.
[3] Gömböc http://en.wikipedia.org/wiki/G%C3%B6mb%C3%B6c
[4] http://domino.research.ibm.com/Comm/wwwr_ponder.nsf/Challenges/
April2008.html
- 85 -

Slide-1 Convex Set

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Slide-1 Convex Set

Uploaded by

Copyright:

Available Formats

Convex Optimization Theory and

1.1. Definition and Examples 定义和例子

1.2. Generalized Inequalities 广义不等式

1.3. Operations that Preserve Convexity 保凸运算

1.4. Separating Hyperplane Theorem 凸集分离

1.5. Some Quiz Problems 测试问题

非正式地说，向量 x 的范数 x p 是这个向量的某种长度度

正式地说，向量 x 的范数 x p 是满足以下四个条件的任何函

线Line：through given two distinct points x1 and x 2 , Set C

仿射集Affine set：Set C contains the line through any two

Let S be the solution of a linear equation. By definition, we

A[ x1  (1   ) x2 ]   Ax1  (1   ) Ax2   b  (1   )b  b

Thus S is an affine set.

(Inversely) Every affine set can be expressed as the solution

基于球的概念定义边界点 Boundary Point，顶点 Extreme

A polyhedron is intersection of a finite number of halfspaces

凸锥 Convex cone C : set that contains all conic combinations

A set is a convex cone if is a cone and is convex.

If 1 and 2 are cones then  1 2 is a cone.

A cone is a 正常锥proper cone if

Are they cones? Are they convex/proper/pointed cones?

凸包Convex Hull：The convex hull of a set  is defined to

凸组合和凸包在某种意义上等价：The convex hull conv S

多胞形Polytope：A bounded polyhedron is called a polytope.

all x  R , and is called positive definite if x Ax  0 for all

nonzero x  R . The set of positive semidefinite matrices is

There are several equivalent conditions for a matrix to be

In the follows, we will focus on semipositive matrices but the

3) all the eigenvalues of A are nonnegative

4) all principal minors of A are nonnegative

5) there exists a factorization A  BT B

Define A  BT B . For any x  R , x Ax  x B Bx  ~ x ~

where ~x  Bx . So we can derive 5) from 1).

eigenvalues of A . This indicates 5) leads to 1).

3) all the eigenvalues of A are positive

4) all principal minors of A are positive

5) there exists a factorization A  BT B (any else condition

1 X1T X 2  X1T AX 2  X 2T AX 1  2 X 2T X1 , 1  2 X1T X 2  0

triangular. But, moreover, we have R T

1  2 x1  x12  x22  2 x13  0

1  2 x1  x12  x22  2 x13  0 and x1  0.5

Suppose A can be diagonalized by U as U AU   . We

have the ellipse rewritten as the standard ellipse form y T

Properties of partial order

 Additive property: If x y and u v , then x  u y  v

positive orthant R n : standard coordinatewise ordering or

Löwner partial order：We can also define the partial order

证明：我们取 x, y  A1 ( S ) ，那么会有 Ax  S , Ay  S 。取

The separating hyperplane we constructed above satisfies the

At least one perpendicular foot of a point inside a convex

The shape of the Indian Star Tortoise resembles a Gömböc.

You might also like