Professional Documents
Culture Documents
Optimization of NLP Problems
Optimization of NLP Problems
Benders Decomposition
Paul I. Barton
Process Systems Engineering Laboratory
Massachusetts Institute of Technology
Motivation
Two-stage Stochastic MINLPs & MILPs
s
min
x1 ,...,xs , y
∑w f (xh , y )
h h
h=1
PI PII PIII
Pump Network
Oil Refinery
Pharmaceutical Product Launch
2
Two Different Decomposition
Philosophies
s s
min
x1 ,...,xs , y
∑w f (xh , y )
h h
min
x1 ,...,xs , y1 ,..., ys
∑w f (xh , yh )
h h
h=1 h=1
Master Problem
min f (x, y) min η
x, y y,η
Projection
min φ ( y) min φ ( y)
y y∈Y
Dualization
s.t. φ ( y) = inf f (x, y) s.t. φ ( y) = sup inf f (x, y) + λ T g(x, y)
x∈X ,g ( x, y )≤0
λ ≥0 x∈X
y ∈Y ∩ V , 0 ≥ inf µ T g(x, y), ∀µ ∈ M
V = {y : ∃x ∈ X , g x, y ≤ 0} ( ) x∈X
4
Generalized Benders Decomposition
Subproblems generated via restriction and relaxation
k
objPrimal = min f (x, y ( k ) )
k
objFeas = min z
x x
x ∈X x ∈ X , z ∈Z ⊂ {z ∈ m : z ≥ 0}
Restriction
- Fixing y
Problem Feasibility
Primal Problem
Problem
Projection and
Dualization
Relaxation
- Finite subset of constraints
Relaxed Master
Master Problem Problem
min η
y,η
5
Generalized Benders Decomposition
Relaxed master problem with separability in x and y
Relaxed Master
Problem
Property P!
min η
y,η
( )
T
s.t. η ≥ objPrj imal + f 2 ( y) − f 2 ( y ( j ) ) + λ j ⎡ g 2 ( y) − g 2 ( y ( j ) ) ⎤ , ∀λ j ∈T k
⎣ ⎦
( )
T
0 ≥ objFeas
i
+ µi ⎡ g 2 ( y) − g 2 ( y (i) ) ⎤ , ∀µ i ∈S k
⎣ ⎦
* A.M. Geoffrion. Generalized Benders decomposition. Journal of Optimization Theory and Applications, 10(4):237–260, 1972.
6
Generalized Benders Decomposition
Algorithm flowchart
UB, LB
Initialization
Yes
Optimal solution or
LB ≥ UB?
infeasibility indication
No
Primal Problem
Yes
Primal Feasible?
No
Update LB
Feasibility
Update UB
Problem
Feasibility Cut
Optimality
Cut
Relaxed Master
Problem
7
Generalized Benders Decomposition
GBD and scenario-based stochastic programs
∑ w ( f (x ) + c )
s
T
min h h h h
y
x1 ,...,xs , y
h=1
9
Gaps Due to Dual and Convex Relaxations
Duality gap
10
Nonconvex Generalized Benders
Decomposition: Overview
Primal Problem
(Nonconvex NLP)
Restriction
- Fixing integer realization
Convex relaxation
- Convexifying nonconvex functions
Restriction
Lower Bounding - Fixing integer realization
Primal Bounding Feasibility
Problem (Convex Problem Problem
MINLP/MILP) (Convex NLP/LP) (Convex NLP/LP)
Projection and
GBD
Dualization
Relaxation
- Finite subset of constraints
Relaxed Master
Master Problem Problem
(MISIP) (MILP)
11
Nonconvex Generalized Benders
Decomposition
Lower bounding problem via convexification
∑w ∑ w (u )
s
min f (xh , y )
h h min (x ,q ) + c T
y
x1 ,...,xs , y x1 ,...,xs ,q1 ,...,qs , y h f ,h h h h
h=1 h=1
s.t. g h (xh , y) ≤ 0, h = 1,...,s s.t. ug ,h (xh ,qh ) + Bh y ≤ 0, h = 1,...,s
xh ∈ X h , h = 1,...,s (xh ,qh ) ∈ Dh , h = 1,...,s
y ∈Y ⊂ {0,1} y
n
y ∈Y
Relaxed Master
Problem (MILP)
min η
η, y
s.t. η ≥ obj ( j)
PBP ( ( ) )(
⎛ s
+ ⎜ ∑ wh chT + λh( j )
⎝ h=1
T ⎞
)
Bh ⎟ y − y ( j ) ,
⎠
∀j ∈T k ,
⎛ ⎞
+ ⎜ ∑(µ ) B ⎟ ( y − y ),
s
T
0 ≥ objFP
(i) (i)
h h
(i)
∀i ∈S k ,
⎝ h=1⎠
∑ (1)
yr − ∑ yr ≤ |{r : yr(t ) = 1} | −1, ∀t ∈T k S k ,
r ∈Ξ r ∈Ξ( 0 )
y ∈Y , η ∈
13
Nonconvex Generalized Benders
Decomposition
Primal problem
Primal Problem
(nonconvex NLP)
(l )
objPP = min wh f h (xh , y (l ) )
h xh
s.t. g h (xh , y (l ) ) ≤ 0,
wh f h (xh , y (l ) ) ≤ UBDh(l ) ,
xh ∈ X h
Yes
PBP Feasible?
Update UB,
No
PBUpper
Update LB
Feasibility Update
Subproblems
PBUpper
(Convex NLP/LP)
Finite convergence to a
Feasibility Cut
Optimality global optimum with a given
Relaxed Master Cut
Problem
tolerance is guaranteed.
New Integer (MILP)
Realization
15
Example
Sarawak Gas Production System (SGPS)
Daily production
- 4 billion scf
Annual revenue
- US $5 billion
(4% of Malaysia’s GDP)
16
Infrastructure Expansion of
SGPS Under Uncertainty
Goal
Optimal network design and long-term operation policy to
maximize profitability
satisfy product-specific constraints for all addressed
uncertainty scenarios
17
Challenges and Novel
Formulation
Challenges Novel Formulation
Strict specifications on Generalized pooling model
gas products: to track gas flow qualities
H2S, CO2, GHV, etc. throughout the system
Large uncertainty in the Two-stage stochastic
system framework to address
quality and capacity uncertainty explicitly
of reservoirs Stochastic pooling problem
customer demands formulation combines
etc. generalized pooling model
18
Stochastic Pooling Problem
⎧
( )
s
⎪ y,xmin c1 y + ∑ c2,h
T T
xh + c3,h T
qh + c4,h
T
uh
1 ,...,xs ,
⎪ q1 ,...qs , h=1 Economic objective
⎪u1 ,...us
⎪ Nonconvex bilinear terms
⎪ model gas qualities
⎪s.t. uh,l.t = xh,l qh,t , ∀(l,t) ∈Ω, ∀h ∈{1,...,s}
⎪ Mass balances
⎪ A 2,h
(equ)
xh + A3,h(equ)
qh + A 4,h
(equ)
uh = bh(equ) , ∀h ∈{1,...,s}
⎪
⎪
⎨ A1,h y + A2,h xh + A3,h qh + A4,huh ≤ bh , ∀h ∈{1,...,s}
⎪
⎪ A 2,h xh + A3,h qh + A 4,huh ≤ bh , ∀h ∈{1,...,s} Flow constraints
⎪
⎪ x hL ≤ x h ≤ xUh , qhL ≤ qh ≤ qUh , ∀h ∈{1,...,s}
⎪
⎪
⎪ By ≤ d, Topology constraints
⎪
⎪
⎪ x ∈
nx
, q ∈
nq
, u ∈
nu
,∀h ∈{1,...,s}, y ∈{0,1}
ny
⎩ h h h
Ω ⊂ {1,..., nx } × {1,..., nq }, nu =| Ω |
19
Computational Study
Implementation Issues
Platform
- CPU 2.83 GHz, Memory 1 GB, Linux, GAMS 22.8.1
Solvers
- LP and MILP solver : CPLEX
- Global NLP solver: BARON
- Local NLP solver: SNOPT
20
Computational Study
SGPS design problem A
The stochastic problem contains 38 binary variables and 93s continuous variables
(s represents total number of scenarios).
1000
10
6
BARON
NGBD
Nonconvex
EI (Estimated)
800
MINLP with
10
4
600
almost
Seconds
Seconds
a hundred
thousand
2
10 400
0 variables
10 200
-2
10 0
5 10 15 20 0 200 400 600 800 1000
Number of Scenarios
Number of Scenarios
(a) Solver times with different methods (b) Solver times with NGBD for more scenarios
21
Computational Results: MILP
Recourse
36 million BVs!
176 million CVs!
Hardware: 3.2 GHz Intel Xeon CPU with 12 GB RAM on Windows
Software: GAMS 23.6 / CPLEX 12.2
Tolerance: 0.001 (relative) for both full model and NGBD
22
Computational Study
- Summary of studied nonconvex problems
(all solved to global optimality with given tolerances)
Bilinear,
SGPS C 146,410/20 4,234.8 quadratic, power (B)
Bilinear,
Pump 50,578/18 2,794.8 quadratic, cubic (B)
Note: [1] Problems were run with different CPUs, GAMS systems and relative termination tolerances ε:
(A) CPU 2.83 GHz , GAMS 28.1, ε=10-2 ; (B) CPU 2.83 GHz, GAMS 23.4, ε=10-3; (C) CPU 2.66 GHz, GAMS 23.5, ε=10-2 .
[2] Enhanced NGBD with tighter lower bounding problems employed.
23
Conclusions & Future Work
Classical decomposition ideas (GBD) can be
extended to certain problems with nonconvex
recourse
Algorithm running time tends to grow linearly with no. scenarios
Applied to hard nonconvex optimization problems in optimal
design & operation of energy systems under uncertainty
Continuous complicating variables?
Multi-stage problems?
How to close the relaxation gap?
24
Acknowledgments
Dr. Xiang Li, Dr. Arul Sundaramoorthy, MIT
Prof. Asgeir Tomasgard, NTNU (Norway)
Statoil (Norway)
Novartis
BP
25
Generalized Benders Decomposition
Principle of projection
Mapping
* A.M. Geoffrion. Generalized Benders decomposition. Journal of Optimization Theory and Applications, 10(4):237–260, 1972.
26
Generalized Benders Decomposition
Optimality cuts
Optimality
Feasibility
27
Generalized Benders Decomposition
Optimality
Optimality
Feasibility
Mapping
28
Nonconvex Generalized Benders
Decomposition
Primal bounding problem and feasibility problem
( ) = min
(k )
obj (k )
= min wh u f ,h (xh ,qh ) + c y T (k ) objFP wh || z h ||
PBPh h h xh ,qh ,zh
xh ,qh
29
Nonconvex Generalized Benders
Decomposition
Master problem
Master Problem
(MISIP)
min η
η, y
( )
s s
s.t. η ≥ ∑ w c + λ Bh y + ∑
T T
inf ⎡ whu f ,h (xh ,qh ) + λhT ug ,h (xh ,qh ) ⎤,
h h h ( xh ,q )∈X ×Q ⎣ ⎦
h=1 h=1 h
∀λ1 ,..., λs ≥ 0,
s s
0 ≥ ∑ µ Bh y + ∑
T
h
inf µ hT ug ,h (xh ,qh ),
( xh ,q )∈X ×Q
h=1 h=1 h
∀µ1 ,..., µs ∈ M ,
y ∈Y , η ∈
30
Haverly’s Pooling Problem
32
Global Optimization via Branch-and-Bound
- Nonconvex optimization
local maximum
local minimum
global minimum
ubd
lbd
34
Global Optimization via Branch-and-Bound
- Branch
35
Global Optimization via Branch-and-Bound
- Branch, and bound
ubd1
lbd1
ubd2
lbd2
36
Global Optimization via Branch-and-Bound
- Branch, bound and fathom
ubd1
lbd1
FATHOM
ubd2
lbd2
37
Two-Stage Stochastic
Programming Framework
Scenario 1
Probability 1
Scenario 2
Probability 2
Probability 3
Scenario 3
Goal
Probability s
- Maximize
Design Decisions expected
- Binary variables to net present
determine whether to Scenario s
value
develop potential gas Realization of
wells/fields, gas uncertainty
platforms and pipelines
- Reservoir quality
- Customer demand Operational Decisions
- Gas price
- Continuous variables to
determine rates of the gas
flows throughout the network
in long-term operation
38
The Different Design Results
Uncertain CO2 content in gas from M1
Supply Chain
Primary
Production
PI PII PIII
Distribution
Center
Retailer
Customer
Research and Development (R&D)
41
Pharmaceutical Business Processes
Supplier
Supply Chain
Integrated
Production
new molecular entity
PI PII PIII
Distribution
Center
Retailer
Customer
Research and Development (R&D)
42
Pharmaceutical Business Processes
new facility?
demand profile
PI PII PIII
*Stonebraker, J.S. How Bayer makes decisions to develop new drugs. Interfaces, 32, 77, 2002.
43
Uncertainty in the Problem
IF/IP PI fail/pass
IF IP
IIF/IIP PII fail/pass
PI outcomes IIIF/IIIP PIII fail/pass
IIF IIP
PII outcomes
outcomes of interest in the
IIIF IIIP product-launch planning
PIII outcomes
*Colvin, M; Maravelias, C.T. A Stochastic programming approach for clinical trial planning in new drug development. Comput. Chem.
Eng., 32, 2626-2642, 2008.
44
Proposed Approach
Two-stage stochastic programming
First stage decisions
» When to build a new facility
» When to expand an existing facility
Second stage decisions
» When, where, and how much to produce, store, and supply
product launch
Problem Description
s8
s9 (IIIP,IIIF)
here-and-now decisions (IIIP,IIIP) wait-and-see decisions
(First stage decisions) (Second stage decisions)
45
Model, Size, and Solution Method
Mixed-integer linear programming (MILP)
model
First-stage variables binary, recourse problem MILP
46
Gaps Due to Dual and Convex
Relaxations
Relaxed
dual gap 1
Duality gap
Convex
relaxation gap
47
Enhanced NGBD with More Dual Information
Dualization
Relaxed Dual of
Primal Problem Primal Problem
(Nonconvex NLP) Relaxation
(Nonconvex NLP) Primal
Dual
Information
Restriction
- Fixing integer realization
Relaxation
The Original Problem
(Nonconvex MINLP)
Convexification
- Convexifying nonconvex functions
Restriction
Lower Bounding - Fixing integer realization
Primal Bounding Feasibility
Problem (Convex Problem Problem
MINLP/MILP) (Convex NLP/LP) (Convex NLP/LP)
Projection and
Dualization
Relaxation
Enhanced
- Finite subset of constraints
Relaxed Master
Master Problem
(MISIP) Problem
(MILP)
48
The Enhanced Decomposition Strategy
– Relaxed Dual of Primal Problem
Relaxed Dual of
Primal Problem
(NLP)
The relaxed dual of the primal problem is usually much more difficult than the primal
problem
49
The Enhanced Decomposition Strategy
–Enhanced Relaxed Master Problem
Enhanced Relaxed Master
Problem (MILP)
: Optimal solution of Problem (DPPh) with : Optimal solution of Problem (DPPh) with
50
The Enhanced Decomposition Strategy
–Enhanced Relaxed Master Problem
Enhanced Relaxed Master
Problem (MILP)
: Optimal solution of Problem (DPPh) with : Optimal solution of Problem (DPPh) with
51
The Enhanced Decomposition Strategy
–Enhanced Relaxed Master Problem with Primal Dual Multicuts
52
The Enhanced Decomposition Strategy
–Enhanced Relaxed Master Problem with Primal Dual Multicuts
Primal Dual
Multicuts
53
Enhanced NGBD Algorithm Flowchart
UB, PBUpper, Global solution or
Initialization
LB
End
infeasibility indication
Yes
LB ≥ Yes
PBUpper ≥
PBUpper?
UB?
No
No
Primal Bounding
Subproblems Primal
Subproblems
(Convex NLP/LP)
(Nonconvex NLP)
Yes
PBP Feasible?
Update UB,
No
PBUpper
Update LB
Feasibility Update
Subproblems
PBUpper
(Convex NLP/LP)
Solve DPP?
Feasibility Cut
Yes
Optimality
No
Enhanced Relaxed Cut
Master Problem
Relaxed Dual of
New Integer (MILP)
Primal Problem
Realization
Optimality Cuts (Nonconvex NLP)
from solving DPP
54
What is polygeneration?
Polygeneration is a novel conversion process
Multiple feedstocks and multiple products
Multiple processes with strong synergy in a single plant
Flexible product portfolios in response to market conditions
Electricity
Coal
Petroleum
products
Biomass
Polygeneration
Systems Chemicals
55
Process Flowsheet
56
Case Study
Problem Sizes and Optimization Results
Case 1 Case 2
57
Case 1 Results
Multicut
Enhanced
BARON NGBD Enhanced
NGBD
NGBD
Total Time --- [1] 73004.91 11985.4 3065.11
Tolerance: 10-2
58
Case 1 Results
NGBD Enhanced
NGBD
Objective
Objective
Iteration Iteration
Multicut
Enhanced NGBD
Objective
Multicut
Enhanced
BARON NGBD Enhanced
NGBD
NGBD
Total Time --- [1] --- 48430.99 15824.95
Tolerance: 10-2
60
Case 2 Results
Objective
Objective
Iteration Iteration
Tolerance: 10-2
61
Case 1 Results
Multicut
Enhanced
BARON NGBD Enhanced
NGBD
NGBD
Total Time --- [1] --- 21323.92 16447.34
Tolerance: 10-3
62
Case 1 Results
Objective
Objective
Iteration Iteration
Tolerance: 10-3
63
More Case Study Results
Scenarios Addressed in the Stochastic Formulation
Scenario No. 1 No. 2 No. 3 No. 4 No. 5
Probability 0.036 0.238 0.451 0.238 0.036
M1 quality1 1.90 2.62 3.34 4.06 4.78
1 Quality means the CO2 mole percentage in gas.
64
Fixed Capacity
Process Flowsheet
Group 1 Group 2
Group 3