Exact methods

Branch & Bound

Michel Bierlaire

Optimization: principles and algorithms

Branch & Bound

In the absence of optimality conditions, enumeration is the only way to find the
optimal solution.
However, it is most of the time impossible to perform explicitly due to the curse
of dimensionality.
The branch & bound method is some sort of implicit enumeration technique, that
attacks the huge set of feasible solutions using a “divide and conquer” strategy.
Combinatorial optimization

min f (x)
x

subject to
x ∈ F,
where F is a large set containing a finite number of elements.
Divide

F
Divide

F1 F2 ··· FK −1 FK
Divide

Problem P Problem Pk
min f (x) min f (x)
x x
subject to subject to
x ∈ F, x ∈ Fk ,
Solution: x .∗
Solution: xk∗ .
Conquer

Theorem 26.1
Consider i such that
f (x ∗ ) ≤ f (xi∗ )
f (xi∗ ) ≤f (xk∗ ), k = 1, . . . , K . ∃j such that x ∗ ∈ Fj
Optimality of xj∗ : f (xj∗ ) ≤ f (x)∀x ∈ Fj
Then, x ∗ ∈ Fj ⇒ f (xj∗ ) ≤ f (x ∗ )
f (x ∗ ) ≤ f (xi∗ ) ≤ f (xj∗ ) ≤ f (x ∗ )
f (x ∗ ) = f (xi∗ ),

and xi∗ is solution of P.

Divide

F1 ··· FM FM+1 ··· FK

Divide

Problem P Problem Pk , k ≤ M Problem Pk , k > M

min f (x) min f (x) min f (x)
x x x
subject to subject to subject to

x ∈ F. x ∈ Fk . x ∈ Fk .

Solution: x ∗ . Solution: xk∗ . Lower bound:

ℓ(Pm ) ≤ f (xk∗ ).
Conquer
Corollary 26.2
Consider i such that

f (xi∗ ) ≤ f (xk∗ ), k = 1, . . . , M,

and ℓ(Pk ) ≤ f (xk∗ ) for each k > M

Therefore, f (xi∗ ) ≤ f (xk∗ ) for all k
f (xi∗ ) ≤ ℓ(Pk ), k = M+1, . . . , K . The previous theorem applies.
Then,

f (x ∗ ) = f (xi∗ ),

and xi∗ is solution of P.

Summary

When the feasible set is large, one way to solve the problem is to partition it into
smaller subsets
Solve each problem independently. The best solution found is the optimal
solution of the original problem.
The most important result says that it is not necessary to solve all problems.
If good lower bounds can be found, it is sufficient to prove the optimality.
And for MILP, we know how to calculate lower bound: by solving the relaxation
problem.
 Exact methods
Branch & Bound for integer optimization

Michel Bierlaire

Optimization: principles and algorithms

Branch & Bound for integer optimization

We illustrate the idea of the B&B method in the context of integer optimization.
Example

Problem P
min x1 − 2x2
subject to
−4x1 + 6x2 ≤ 9
x1 + x2 ≤ 4
x1 , x2 ≥ 0
x1 , x2 ∈ N.

Upper bound
(0,0) is feasible: U=0
Example

0
0 1 2 3 4
Lower bound

Solve the relaxation. Erase the integrality constraints and call the problem R(P).
Lower bound

min x1 − 2x2
subject to
−4x1 + 6x2 ≤ 9
x1 + x2 ≤ 4
x1 , x2 ≥ 0
x1 , x2 ∈ N.
Lower bound

0
0 1 2 3 4
Divide

◮ Optimal solution of R(P) : (1.5, 2.5)

◮ ℓ(P) : −3.5

P1 P2
min x1 − 2x2 min x1 − 2x2
s.c. s.c.
−4x1 + 6x2 ≤ 9 −4x1 + 6x2 ≤ 9
x1 + x2 ≤ 4 x1 + x2 ≤ 4
x1 , x2 ≥ 0 x1 , x2 ≥ 0
x1 , x2 ∈ N x1 , x2 ∈ N
x2 ≤ 2 x2 ≥ 3
Divide

0
0 1 2 3 4
Tree representation

Upper bound
U =0
Tree
P[−3.5]

P1 [?] P2 [?]

P2 is infeasible.
P1

Mention that the feasible set of P1 is exactly the same as P.

Problem P1

0
0 1 2 3 4
P1: lower bound

Write the relaxed problem by hand.

P1: lower bound

Problem P1 Relaxation R(P1 )

min x1 − 2x2
subject to

−4x1 + 6x2 ≤ 9
x1 + x2 ≤ 4
x1 , x2 ≥ 0
x2 ≤ 2
x1 , x2 ∈ N.
P1: lower bound

0
0 1 2 3 4
P1: divide
◮ Optimal solution of R(P1 ) : (0.75, 2)
◮ ℓ(P1 ) : −3.25

P11 P12
min x1 − 2x2 min x1 − 2x2
s.c. s.c.
−4x1 + 6x2 ≤ 9 −4x1 + 6x2 ≤ 9
x1 + x2 ≤ 4 x1 + x2 ≤ 4
x1 , x2 ≥ 0 x1 , x2 ≥ 0
x1 , x2 ∈ N x1 , x2 ∈ N
x2 ≤ 2 x2 ≤ 2
x1 ≤ 0 x1 ≥ 1
P1: divide

0
0 1 2 3 4
Tree representation

U =0

P[−3.5]

P1 [−3.25] P2 [?]

P11 [?] P12 [?]

P11: lower bound

Write the relaxed problem by hand.

Mention that x1 = 0.
P11: lower bound

Problem P11 Relaxation R(P11 )

min x1 − 2x2
subject to

−4x1 + 6x2 ≤ 9
x1 + x2 ≤ 4
x1 , x2 ≥ 0
x2 ≤ 2
x1 ≤ 0
x1 , x2 ∈ N.
P11: lower bound

0
0 1 2 3 4
P11: lower bound

◮ Optimal solution of R(P11 ) : (0, 1.5)

◮ ℓ(P11 ) : −3
P12: lower bound

Write the relaxed problem by hand.

P12: lower bound

Problem P12 Relaxation R(P12 )

min x1 − 2x2
subject to

−4x1 + 6x2 ≤ 9
x1 + x2 ≤ 4
x1 , x2 ≥ 0
x2 ≤ 2
x1 ≥ 1
x1 , x2 ∈ N.
P12: lower bound

0
0 1 2 3 4
P12: lower bound

◮ Optimal solution of R(P12 ) : (1, 2)

◮ ℓ(P12 ) : −3
Integer solution
It is the optimal solution of P12 .
Feasible solution for P
Better upper bound: U = −3.
Tree

Clean the tree by removing subproblem P1 because its lower bound is not better
than the upper bound.
Then show that we have found any optimal solution
Tree representation
U = −3

P[−3.5]

P1 [−3.25] P2 [?]

P11 [−3] P12 (−3)

Summary

The branch & bound method for integer optimization applies the divide and
conquer concept progressively.
Branching: taking a fractional variable, and branch on it.
Lower Bound: solving the relaxation
Upper bound: best feasible solution found so far.
 Exact methods
Gomory cuts

Michel Bierlaire

Optimization: principles and algorithms

Gomory cuts

It is possible to shrink the constraint polyhedron so that its vertices are integer
In that case, the simplex algorithm will solve the problem.
In practice, additional constraints must be added.
They must shrink the feasible set of the relaxation, but not of the original
discrete optimization problem.
These constraints are called “valid inequalities” when they are designed by the
modeler, and “cutting planes” or “cuts” when they are generated by an
algorithm.
Feasible set of integer optimization

3 3

2 2

1 1
0 0
0 1 2 3 4 5 0 1 2 3 4 5
Feasible set of integer optimization

3 3

2 2

1 1
0 0
0 1 2 3 4 5 0 1 2 3 4 5
Simplex tableau
B −1 A B −1 b
c T − cBT B −1 A −cBT B −1 b
We have an optimal tableau for the relaxed problem.
Upper part: a modified version of the constraints
Decompose the upper part into basic and non basic variables
B −1 Ax = B −1 b,
B −1 BxB + B −1 NxN = B −1 b,
xB + B −1 NxN = B −1 b.
Basic variables are first, so that xi = (xB )i , i ≤ m. Consider row i and call αij
the element of the tableau
X
xi + αij xj = (xB∗ )i .
j non basic
Rounding the constraint

X
xi + αij xj = (xB∗ )i .
j non basic

x is feasible, it is non negative.

Rounding down all coefficients αij , the value cannot increase
Valid inequality for all feasible solutions of the relaxation
X X
xi + ⌊αij ⌋xj ≤ xi + αij xj = (xB∗ )i .
j non basic j non basic
Rounding again the constraint

X
xi + ⌊αij ⌋xj ≤ (xB∗ )i .
j non basic
$ %
X
xi + ⌊αij ⌋xj ≤ ⌊(xB∗ )i ⌋.
j non basic
x is integer

$ %
X
xi + ⌊αij ⌋xj ≤ ⌊(xB∗ )i ⌋.
j non basic
$ %
X X
xi + ⌊αij ⌋xj = xi + ⌊αij ⌋xj
j non basic j non basic
X
xi + ⌊αij ⌋xj ≤ ⌊(xB∗ )i ⌋,
j non basic

Valid inequality for all feasible solution of the integer problem: it is called a
Gomory cut
Solution of the relaxation

X
xi + ⌊αij ⌋xj ≤ ⌊(xB∗ )i ⌋.
j non basic

If x is solution of the relaxation:

∗

All non basic variables are 0

xi∗ ≤ ⌊(xB∗ )i ⌋ = ⌊xi∗ ⌋,

This is verified only if xi∗ is integer

Therefore, if i is chosen such that xi is fractional, it generates a cut that excludes
x ∗.
Examples

3 3

2 2

1 1

0 0
0 1 2 3 4 5 0 1 2 3 4 5
Examples

3 3

2 2

1 1

0 0
0 1 2 3 4 5 0 1 2 3 4 5
Summary

The branch & bound was cutting the constraint polyhedron along the axes of
coordinates.
Gomory cuts exploit the structure of the simplex tableau to generate valid
inequalities, that exclude (if well chosen) the optimal solution of the relaxation.
It means that, at each iteration, the polyhedron is made strictly smaller.
Eventually, all the vertices will be integer.