The Canonical and Dictionary Forms of Linear Programs

Recall the general optimization problem:

Minimize f(x) subject to xD

(1)

The real valued function f is called the objective function, and D

the constraint set, or decision set. We choose to minimize cost.
The problem (1) is called feasible if D is non-empty. The decision
x* D solves (1) or is optimal if x D implies that f(x) f(x*).
In this case f(x*) is the optimal value.
Some Examples:
To get started we will look at some simple examples of linear
programs.
P1: Minimize z

= x1
x1

+ x2
+ x4
+ x2
+ x3
+ x5
=1
x2
x3
+ x4
+ x6
=5
x1 3x2
+ x3
+ x4
+ x7 = 4
x1 0 x2 0 x3 0 x4 0 x5 0 x6 0 x7 0

Is there a feasible solution for this linear program; if so what?

Does the linear program have an optimal solution? Is there more
than one optimal solution?
Now we look at this one.

Canonical Forms Revised 10/11/05

R. Van Slyke

p. 1/10

P2: Minimize z
= x1
+ x2
+ x4
z
+ x2
x3
+ x5
=1
x1
x3
+ x4
+ x6
=5
x2
x3
+ x4
+ x7 = 4
x1 3x2
x1 0 x2 0 x3 0 x4 0 x5 0 x6 0 x7 0
Is there a feasible solution for this linear program; if so what?
Does the linear program have an optimal solution? Is there more
than one optimal solution?
How about this one?
P3: Minimize z
= x1
x2
+ x4
z
x2
+ x3
+ x5
=1
x1
x2
x3
+ x4
+ x6
=5
+ x3
+ x4
+ x7 = 4
x1 3x2
x1 0 x2 0 x3 0 x4 0 x5 0 x6 0 x7 0
Is there a feasible solution for this linear program; if so what?
Does the linear program have an optimal solution? Is there more
than one optimal solution?
How about this one?
P4: Minimize z
= x1
x2
+ x4
z
+ x2
+ x3
+ x5
=1
x1
x2
x3
+ x4
+ x6
=5
+ x3
+ x4
+ x7 = 4
x1 +3x2
x1 0 x2 0 x3 0 x4 0 x5 0 x6 0 x7 0

Canonical Forms Revised 10/11/05

R. Van Slyke

p. 2/10

Is there a feasible solution for this linear program; if so what?

Does the linear program have an optimal solution? Is there more
than one optimal solution?
Finally, how about this?
P5: Minimize z
= x1
x2
+ x4
z
+ x2
+ x3
+ x5
=0
x1
x2
x3
+ x4
+ x6
=5
+ x3
+ x4
+ x7 = 4
x1 +3x2
x1 0 x2 0 x3 0 x4 0 x5 0 x6 0 x7 0
Is there a feasible solution for this linear program; if so what?
Does the linear program have an optimal solution? Is there more
than one optimal solution?
The ideas illustrated by these examples, pretty much define
Dantzigs Simplex Method1.
The Canonical Form:
These problems are all in what George Dantzig [1963] called
canonical form. In matrix notation the canonical form can be
written:

Min z = cN xN
AxN + IxB = b
s.t.
xN 0 xB 0
Where cN an n-dimensional row vector, A is an mxn matrix, I is an
mxm identity matrix are given data. b is a non-negative mdimensional column vector. The decision variables xN and xB are n1

Recently the Simplex Algorithm was named one of the ten most important algorithms of
the century.
Canonical Forms Revised 10/11/05
R. Van Slyke
p. 3/10

dimensional and m-dimensional column vectors respectively. The

solution obtained by setting the non-basic variables, xN, to 0, xB
to b, is called the basic solution corresponding to the canonical
form. The basic solution is feasible because b 0. The variables
xN are called the non-basic variables and xB the basic variables.
In classical mathematics, where, often, the variables are not
required to be non-negative, the xN are called independent
variables, and the xB dependent variables because the xN can be
arbitrarily set, and the linear equations solved by assigning the
appropriate values of the basic variables, xB.
The Dictionary Format:
Strum [1972] and Chvtal [1983] advocate another format called
the dictionary format that is very useful. (See also [Vanderbei,
2001].) It is probably best shown by simply converting P1-P5 to
their corresponding dictionary formats D1-D5.
D1: Minimize z

x5 = 1 x1 x2 x3
x2 + x3 x4
x6 = 5
x7 = 4 x1 +3x2 x3 x4
+ x4
z = 0 + x1 + x2
x j 0 j = 1,...,7

D2: Minimize z

Canonical Forms Revised 10/11/05

R. Van Slyke

p. 4/10

x5 = 1 x1 x2 + x3
x6 = 5
x2 + x3 x4
x7 = 4 x1 +3x2 + x3 x4
z = 0 + x1 + x2
+ x4
x j 0 j = 1,...,7
D3: Minimize z

x5 = 1 x1 + x2 x3
+ x2 + x3 x4
x6 = 5
x7 = 4 x1 +3x2 x3 x4
+ x4
z = 0 + x1 x2
x j 0 j = 1,...,7

D4: Minimize z

x5 = 1 x1 x2 x3
x6 = 5
+ x2 + x3 x4
x7 = 4 x1 3x2 x3 x4
z = 0 + x1 x2
+ x4
x j 0 j = 1,...,7

D5: Minimize z

x5 = 0 x1 x2 x3
+ x2 + x3 x4
x6 = 5
x7 = 4 x1 3x2 x3 x4
+ x4
z = 0 + x1 x2
x j 0 j = 1,...,7

D2: Minimize z
Canonical Forms Revised 10/11/05

R. Van Slyke

p. 5/10

x5 = 1 x1 x2
x6 = 5
x2
x7 = 4 x1 +3x2
z = 0 + x1 + x2
x j 0 j = 1,...,7

+ x3
+ x3 x4
+ x3 x4
x3 + x4

Let us now return to the pair P4,D4 which we left in the middle
of. Let us work with the dictionary representation.
D4: Minimize z

x5 = 1 x1 x2 x3
x6 = 5
+ x2 + x3 x4
x7 = 4 x1 3x2 x3 x4
z = 0 + x1 x2
+ x4
x j 0 j = 1,...,7

An initial basic feasible solution can be read off by making the

variables: x1, x2, x3, and x4 all zero. Then we can set x5, x6, and x7
equal to their corresponding constant terms, 1, 5, and 4
respectively. Similarly z would be 0, for this feasible solution.
Now lets see if we can improve from that. We can make small
changes in the variables x1, x2, x3, and/or x4 and adjust for these
changes using x5, x6, and/or x7. For example, we could make x3 = 1.
This would make x5 = 0, x6 = 6, and x7 = 3. Unfortunately, z
remains unchanged, so we have not gained anything. Setting x1 to
one would only make things worse; z would increase from 0 to 1.
Increasing x2 looks more promising. If we increase it to 1, z
decreases to 1, an improvement. If we can do better by
increasing x2 by 1, why not increase it further? The problem is, as
we increase x2 past 1, x5 starts to go negative, and therefore,
Canonical Forms Revised 10/11/05

R. Van Slyke

p. 6/10

infeasible. So we increase x2 to 1, and decrease x5 = 0, x6 = 7, and

x7 = 3. This suggests interchanging the roles of x2 and x5; that is
to put x2 on the left of the equal signs and x5 on the right.
One way to do this is to solve x2 in terms of x2 using the first
equation. That is, x2 = 1 x1 x5 x3. Substituting this into the
other equations, you get the new dictionary:

x2 = 1 x1 x5 x3
x6 = 6 x1 x5
x4
x7 = 1 +2 x1 3x5 +2 x3 x4
z = 1 +2 x1 + x5 + x3 + x4
x j 0 j = 1,...,7
Now if we try the same trick as before, we find that since all the
coefficients in the z expression are non-negative there is no way
to improve z. In fact, since the coefficients for z for the nonbasic variables are positive there is only one optimal solution. So,
for this problem, in one step, we obtained an optimal basic
feasible solution. In general, it can take many iterations, but,
when done carefully, it will always terminate with an optimal
solution, or a solution like P6/D6. The place where care has to be
taken is with dictionaries like D5, where one of the constant
terms is 0.
The canonical form seems rather special. But by using various
devices that we will soon discuss, a very wide variety of
optimization problems can be put in this format.

Canonical Forms Revised 10/11/05

R. Van Slyke

p. 7/10

The Simplex Method:

Based on the examples above we can build up an algorithm for
solving linear programs called the simplex method. The simplex
algorithm consists of several steps:
1.

2.

3.

We look at the coefficients of the objective function (z) to

find one that is negative. If there is none, we have an
optimal solution (Examples P1/D1, P2/D2). Otherwise, let
the variable, xs be the non-basic variable with a negative
coefficient that we choose.
We increase xs until the first basic variable, xr, starts to go
negative. If this never happens; that is no basic variable
starts to go negative, then the value of the objective is
unbounded below (Example P3/D3).
We interchange the roles of xr and xs, making xs basic, and
xr non-basic at value 0. The value of z decreases unless we
have a case like Example P5/D5 where one of the constants
is zero. In this case the value of z in the new dictionary is
the same as the old. This situation is called degeneracy, and
we will just assume for the time being that it doesnt
happen. Later we will show how to deal with it.

As long as the value of z keeps on decreasing we cannot repeat a

dictionary. There is a finite number of dictionaries, therefore
the algorithm is finite.
Each feasible dictionary corresponds to an extreme point of the
decision space. By looking at the coefficients in the z equation we
see if we are at a local minimum or not. If all the coefficients are
non-negative, then we are at a local minimum. But the decision
space is convex and the objective linear, so, by the Theorem from
Section 2.4 of The General Optimization Notes, the extreme
Canonical Forms Revised 10/11/05

R. Van Slyke

p. 8/10

point is globally optimal (of course it is easy to see this directly

from the dictionary). By the same theorem, and the fact that the
linear objective is also concave, we need only look at extreme
points. So we go to the next dictionary, which is another extreme
point, with a smaller value of the objective
REFERENCES:
Chvtal, Vaek, Linear Programming, Freeman, 1983.
Dantzig, George B., Linear Programming and Extensions, Princeton
University Press, 1963.
Strum, J. E., Introduction to Linear Programming, Holden-Day,
1972.
Vanderbei, Robert J., Linear Programming: Foundations and
Extensions, 2nd ed., Kluwer, 2001.

Canonical Forms Revised 10/11/05

R. Van Slyke

p. 9/10

EXERCISES:
1.

Show that A = { x = ( x1 , x2 ,..., xn ) | a1 x1 + a2 x2 + ... + an xn = b} is an affine

set for given constants a1, , an.

2.

Show that an affine set is convex.

3.
a)

4.

Show that the intersection of an arbitrary collection

of affine sets is affine.
b)
Show that the intersection of an arbitrary collection
of convex sets is convex.
c)
Show that the intersection of an arbitrary set of
conesall with the same vertexis a cone with the
common vertex.
Show that A = { x = ( x1 , x2 ,..., xn ) | ai ,1 x1 + ai ,2 x2 + ... + ai ,n xn = bi , i = 1,..., m} is

5.

affine.
Show that

C = { x = ( x1 , x2 ,..., xn ) | x j 0, j = 1,..., n} is a convex cone

with vertex 0.
6.

Show that

K = {( x = ( x1 , x2 ,..., xn ) | ai ,1 x1 + ai ,2 x2 + ... + ai ,n xn = bi , i = 1,..., m, x j 0, j = 1,..., n}

is convex.
7.

A function f(x)=f(x1, x2, , xn) is linear iff f(x+y) = f(x) +

f(y) for arbitrary vectors x, and y, and scalars and .
a)

Show that f(x) = c1x1 + c2x2 ++cnxn is linear.

b)

Show that if f(x) is linear, it is both convex and

concave.

Canonical Forms Revised 10/11/05

R. Van Slyke

p. 10/10