Data memberane in geoquest data

Robust Optimization
Aojie Hong
Remus Hanea
Agenda
Optimization Problems Sequential Decision Making
• What is a sequential decision making
Gradient-based Methods problem?
• Optimization without uncertainty • Modeling of sequential decision
• Optimization under uncertainty making problems
• Optimization over an ensemble o myopic model
• EnOpt algorithm o naïve model
• Examples o dynamic programming model
o a decline curve example • Closed-loop Optimization
o a CRM example o Workflow
o a reservoir simulation example o a reservoir simulation example
Recap
2
Optimization Problems
3
Definition of Optimization Problems
max 𝐽 (𝑼 )
𝑼 ∈ 𝜴𝑼
• is the objective function – what we want to achieve (e.g., oil recovery of

field).
• is the vector containing control (or decision) variables – what actions we
can do (e.g., injection rate of a water injection well).
• is the feasible space of the control variables – what alternatives we have
(e.g., the feasible range of the injection rate of a water injection well is
from 0 bbl/day to 1,000 bbl/day).
4
Solving Optimization Problems
In general, there are two categories of methods for solving optimization problems.
• Gradient-based methods
o Need to calculate (approximately) the gradients of the objective function with respect to the
control variables.
o Can be trapped in a local optimum (sensitive to the starting point).
o Less computationally demanding.
• Non-gradient-based methods
o Do not need to calculate (approximately) the gradients of the objective function with respect to
the control variables.
o Have a higher chance to find the global optimum.
o More computationally demanding.
5
Idea behind Gradient-based Methods
(Cases without Uncertainty)
6
0.15
0.1
J
0.05
0
0 5 10 15
u 7
Gradient-based Methods
0.15
0.1
J
0.05
0
0 5 10 15
u 9
(Cases with Uncertainty)
10
Optimization under Uncertainty
For a risk-neutral decision maker, the objective is to maximize the mean over the objective
functions of individual realizations in an ensemble.
• is the mean objective function.

• is the realization in an ensemble.
11
0.3
Realization 1
Realization 2
Ensemble Mean
0.2
J
0.1
0
0 5 10 15
u 12
• Each green circle presents one
0.3
Realization 1 forward modeling (e.g., reservoir
Realization 2 simulation).
Ensemble Mean
0.2 • Forward modeling can be
computationally demanding.
J
• If we want to make the

0.1
optimization process faster, fewer
forward modeling should be run.
0 • A common practice is to use only
0 5 10 15 one forward model with the means
u of model parameters (mean
model).
13
Optimization Using Mean Model
If we use the mean model, the optimization problem is reduced to
• is the mean model.

• is the realization in an ensemble.
14
Optimization Using Mean Model
Optimal control maximizing mean model
0.3
Realization 1 result: Is it maximizing ensemble mean?
Realization 2 Jensen’s inequality:
Ensemble Mean
0.2 Mean Model
for non-linear objective function
J
0.1
0
0 5 10 15
u 15
Idea behind Ensemble-based Optimization
16
Ensemble-based Optimization
• Each green circle presents one
0.3
Realization 1 forward modeling (e.g., reservoir
Realization 2 simulation).
Ensemble Mean
0.2 • Forward modeling can be
computationally demanding.
J
• If we want to make the

0.1
optimization process faster, fewer
forward modeling should be run.
0 • The idea behind ensemble-based
0 5 10 15 optimization methods is to using
u forward modeling more
effectively.
17
0.3
Realization 1
Realization 2
Ensemble Mean
0.2
J
0.1
0
0 5 10 15
u 18
0.3
Realization 1
Realization 2
Ensemble Mean
0.2
J
The idea behind ensemble-

based optimization methods
0.1 is to using forward modeling
more effectively:
Evaluating different control
0 values on different
realizations to explore a wider
0 5 10 15
area of the control variables
u space. 19
The Ensemble-Based Optimization Method
(EnOpt)
20
EnOpt Algorithm
1. Define a starting point of control values .
2. Evaluate on realization for to get and calculate .
3. Generate random control values () around .
4. Evaluate on realization for to get .
𝑢1
5. Calculate
6. Calculate the cross-covariance matrix (sensitivity) .

𝑢2
7. Use the normalized as an approximation of the gradient .
21
EnOpt Algorithm
8. Use the approximated gradient to update the control values .
9. Evaluate on realization for to get and calculate .
10. Compare and :
o If , the mean objective value is increased! Use as the starting point in Step 1 to start a new iteration.
o If , the current step size is too large. Reduce , recalculate in Step 8, and repeat Steps 9 – 10.
22
EnOpt Implementation
(Simple Production Model Example)
23
A Simple Production Model Example
A production model is used to model the oil and water production from a field with one producer and
one water injector. (The production model is given in the Jupyter Notebook.) Our uncertainty about the
reservoir is modeled using an ensemble of different settings of the production model parameters.
Injector Producer
• The field will be produced for 60 months.

• The control variable is the water injection rate for the 60 months (constant over the 60 months after
it has been decided at time 0). That is, there is only one control variable.
• The objective is to maximize the net present value (NPV) of the production.
• What water injection rate should be used for the 60 months? Python: EnOpt_CRM_1Control.ipynb
24
A Simple Production Model Example
• For the same context, suppose we now have a “smart” injection well, whose injection rate can vary
monthly for the 60 month. That is, there are now 60 control variables (one for each month).
• What water injection rate should be used for the 60 months?
Python: EnOpt_CRM_nControl.ipynb
25
Sequential Decision Making
26
The essential of sequential decision making is learning over time.
Current Decision
Prior Decision Additional Decision Additional Decision

Information to make Information to make Information to make …
() () () () () ()
Time 0 1 2 …
Past Current Future
The myopic model ignores future decisions and information.
Current Decision

() () () () () ()
Time 0 1 2 …
Past Current Future
Initial Uncertainty
𝐷0 𝑈∨ 𝐼 0
The naïve model ignores future information/learning.
Current Decision

() () () () () ()
Time 0 1 2 …
Past Current Future
Initial Uncertainty
𝐷0 𝐷1 𝐷2 𝐷 𝑒𝑛𝑑 𝑈∨ 𝐼 0
…
The farsighted model preserves the full decision-info structure.
Current Decision

() () () () () ()
Time 0 1 2 …
Past Current Future
Dynamic programming is based on the farsighted model.
Evolution of Uncertainty (Learning over Time)
𝐷0 𝐼 1∨𝐼 0 𝐷1 𝐼 2∨𝐼 0 : 1 𝐷 𝑒𝑛𝑑 𝑈∨ 𝐼 0 : 𝑒𝑛𝑑
…
The myopic and naïve models can lead to sub-optimal decision policies and lower
the value of sequential decisions, but the farsighted model is computationally
demanding.
Low 𝐷0 𝑈∨ 𝐼 0
Myopic
Computational Burden
Decision Value
𝐷0 𝐷1 𝐷2 𝐷 𝑒𝑛𝑑 𝑈∨ 𝐼 0
Naïve …
𝐷0 𝐼 1∨𝐼 0 𝐷1 𝐼 2∨𝐼 0 : 1 𝐷 𝑒𝑛𝑑 𝑈∨ 𝐼 0 : 𝑒𝑛𝑑
Farsighted …
High
A Sequential Drilling Example
A B
• Reservoirs A and B are disconnected.

• The chance of getting oil from Reservoir A is 10%
and that of Reservoir B is also 10%.
• Because of their similarity, the NPVs of the two
reservoirs are the same ($600 million) if oil is found.
• The cost of a well is $100 million. 32
Alternative 1: to drill only Reservoir A.
NPV (M$)
oil
600
10%
Res. A
EV = -30
dry
90% -100
EV = 0
walk
0
33
Alternative 2: to drill only Reservoir B.
NPV (M$)
oil
600
10%
Res. B
EV = -30
dry
90% -100
EV = 0
walk
0
34
Alternative 3: to drill Reservoirs A and B simultaneously.
Res. A is …
joint
oil dry
oil 7% 3%
Res. B is…
dry 3% 87%
35
Alternative 3: to drill Reservoirs A and B simultaneously.
A(oil)&B(oil) NPV (M$)
600 + 600
7%
A(oil)&B(dry)
600 – 100
Res. A&B 3%
EV = -60 A(dry)&B(oil)
3% -100 + 600
A(dry)&B(dry)
EV = 0 87% -100 – 100
walk
0
36
Summary for the case without information.
Alternative EV (M$)
0. walk away 0 EVWOI
1. to drill Reservoir A only -30
2. to drill Reservoir B only -30
3. to drill both simultaneously -60
37
Alternative 4: to drill one first and then decide whether the other
one should be drilled based on the outcome of the first one.
Given Res. A is …
Likelihood
oil dry
oil 70% 3.3%
Res. B is…
dry 30% 96.7%
38
Alternative 4: to drill one first and NPV (M$)
then decide whether the other one
oil
should be drilled based on the 600 + 600
outcome of the first one. drill B 70%
EV = 990 dry
oil 600 – 100
30%
10% EV = 990 walk
600 + 0
To drill oil
EV = 9 -100 + 600
A first drill B 3.3%
EV = -176.7
dry
dry -100 – 100
96.7%
90% EV = -100 walk
-100 + 0
39
Summary for all cases.
Alternative EV (M$)
0. walk away 0 EVWOI
1. to drill Reservoir A only -30
2. to drill Reservoir B only -30
3. to drill both simultaneously -60
4. to drill sequentially 9 EVWI (free)
VOI = 9 – 0 = $9 million
After the first well has been drilled, the cost of knowing
the outcome of the first well is zero. Thus, we should go
for the alternative of sequential drilling. 40
Closed-Loop Optimization
41
Closed Loop
Optimization
Jansen, J. D., Brouwer, R., & Douma, S. G. (2009). Closed loop reservoir management. SPE reservoir simulation symposium.
Closed loop optimization is based on the (?) model.
What What What What
injection injection injection injection
rate should rate should rate should rate should
be used? be used? be used? be used?
Will collect Will collect Will collect end
production data. production data. production data.
Year 0 Year 10 Year 20 Year 30 Year 40
Prior
EnOpt on the prior: the optimal , , , and .
Implement the optimal to the field.
43
What What What
injection injection injection
𝑝𝑟𝑖𝑜𝑟 rate should rate should rate should
𝐼 0 :10 be used? be used? be used?
Production data () have

Will collect Will collect end
been collected production data. production data.
EnKF: Posterior 1
EnOpt on the posterior 1:
the optimal , , and .
Implement the optimal to the field.
44
What What
injection injection
𝑝𝑟𝑖𝑜𝑟 rate should rate should
𝐼 0 :10 𝐼 𝑝𝑜𝑠𝑡
10: 20
1
be used? be used?
Production data () have Production data () have

Will collect end
been collected been collected production data.
EnKF: Posterior 2
the optimal , and .
Implement the optimal
to the field. 45
What
injection
𝑝𝑟𝑖𝑜𝑟 rate should
10: 20
1
𝐼 𝑝𝑜𝑠𝑡
20 :30
2
be used?
Production data () have Production data () have Production data () have

end
been collected been collected been collected

EnKF: Posterior 3
the optimal .
Implement the optimal
to the field.
46
𝑝𝑟𝑖𝑜𝑟
10: 20
1
𝐼 𝑝𝑜𝑠𝑡
20 :30
2
𝐼 𝑝𝑜𝑠𝑡 3
30 : 40
Production data () have Production data () have Production data () have

end
been collected been collected been collected
47
Value of Information in
Closed-Loop Optimization
48
VOI Calculation Using Ensemble-Based Methods
(only one decision point)
Step 1: Frame the decision problem.
What production
strategy should
be used?
end
Data () will be collected.
Year 0 Year 20 Year 40
Prior Posterior
What is the value of the future data ()?

49
Step 2: Build the initial (or prior) ensemble of realizations.
iR1 iR2 iR3
50
Step 3: Perform robust optimization on the initial ensemble and
calculate Expected Value WithOut Information (EVWOI).
iR1 iR2 iR3
Robust Optimization
Max Average NPV = EVWOI 51

Step 4: Assess and model the measurement errors.
Measurement error
52
Step 5: Simulate information from all initial realizations.
If iR1 is the truth Simulate data
iR1
iR2
iR3
53
Step 6: Update the initial ensemble using the simulated information.
If iR1 is the truth Simulate data Update initial ensemble
iR1 Updated
Ensemble 1
iR2 Updated
Ensemble 2
iR3 Updated
Ensemble 3 54
Step 7: Perform robust optimization on each updated ensemble.
If iR1 is the truth Simulate data Update initial ensemble Robust opt
Updated Optimal
iR1 Controls
Ensemble 1
Updated Optimal
iR2 Controls
Ensemble 2
Optimal
iR3 Updated
Ensemble 3 Controls
55
Step 8: Apply each optimal set of controls on corresponding initial realization
to calculate Value With (free) Information (VWI) of the realization.
Apply on iR1
Updated Optimal and calculate
iR1 Controls
Ensemble 1 NPV (VWI(iR1))
Apply on iR2
Updated Optimal and calculate
iR2 Controls
Ensemble 2 NPV (VWI(iR2))
Apply on iR3
Optimal
iR3 Updated and calculate
Ensemble 3 Controls NPV (VWI(iR3))
56
Step 9: Calculate Expected Value With (free) Information (EVWI).
EVWI (free) =
Step 10: Calculate VOI for a risk-neutral decision maker.
VOI = EVWI – EVWOI
57
• When there is only one decision point, the VOI calculated using the ensemble-based methods
is consistent with the definition of VOI.
• However, for sequential decision making, the closed-loop optimization (EnKF + EnOpt) is
based on the naïve model and may lead to sub-optimal sequential decisions.
o VOI generally refers to the value of information that leads to the optimal sequential
decisions (i.e., decisions solved using the farsighted (dynamic programming) model).
o We use “special VOI” to refer to the value of information that leads to sub-optimal
sequential decisions (i.e., decisions solved using the naïve or myopic model).
o Special VOI ≤ VOI
58
Special VOI Calculation for Closed Loop Optimization
(multiple decision points)
Step 1: Frame the decision problem.
What What What
production production production
strategy strategy strategy
should be should be should be
used? used? used?
Data () will be Data () will be Data () will be end

collected. collected. collected.
Year 0 Year 10 Year 20 Year 30
What is the value of the future data (, , and )?

59
Step 2: Build the initial (or prior) ensemble of realizations.
iR1 iR2 iR3
60
Step 3: Perform robust optimization on the initial ensemble and
calculate Expected Value WithOut Information (EVWOI).
iR1 iR2 iR3
Robust Optimization
Max Average NPV = EVWOI 61

Step 4: Assess and model the measurement errors.
Measurement error
62
Step 5 – 8: Repeat CLO for all initial realizations.
Simulate data for
If iR1 is the truth next 10 years Update initial ensemble Robust opt Apply controls
Optimal at immediate
iR1 1st Updated
Controls decision point
Ensemble
on iR1.
Update 1st updated Simulate data for

Repeat same Apply controls Robust opt ensemble next 10 years
procedure for at immediate Optimal 2nd Updated
remaining decision point Controls Ensemble
decision points. on iR1.
Apply optimal controls

Optimal Controls at
on iR1 and calculate
Repeat for other initial realizations (iR2,
All Decision Points iR3, …) and calculate VWI(iR2), VWI(iR3),
63 …
NPV (VWI(iR1)).
Step 9: Calculate Expected Value With (free) Information (EVWI).
EVWI (free) =
Step 10: Calculate VOI for a risk-neutral decision maker.
VOI = EVWI – EVWOI
64
Summary
65
Summary
• Value can only be created (or destroyed) through our decisions.
• Uncertainty is inseparable from decision making.
• Probability is the language of uncertainty and probabilistic thinking and understanding is required
for high-quality decision making.
o We are all biased and our biases must be avoided/mitigated.
o Bayes’ rule should be used to update our uncertainty when additional information becomes available.
• The Ensemble Kalman Filter (EnKF) is practical and robust for updating reservoir simulation
models, but care must be taken.
• The ensemble-based optimization algorithm, EnOpt, is efficient in tuning many control variables
simultaneously for expected value maximization, but it may be trapped in a local maximum.
• Closed-loop optimization – a combination of ensemble-based history matching and optimization – is
based on the naïve model for sequential decision making and can result in sub-optimal sequential
decisions.
66

Data memberane in geoquest data

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Data memberane in geoquest data

Uploaded by

Copyright:

Available Formats

Robust Optimization

• is the objective function – what we want to achieve (e.g., oil recovery of

• is the mean objective function.

• If we want to make the

• is the mean model.

• If we want to make the

The idea behind ensemble-

6. Calculate the cross-covariance matrix (sensitivity) .

• The field will be produced for 60 months.

Prior Decision Additional Decision Additional Decision

Prior Decision Additional Decision Additional Decision

Prior Decision Additional Decision Additional Decision

Prior Decision Additional Decision Additional Decision

• Reservoirs A and B are disconnected.

Production data () have

Production data () have Production data () have

Production data () have Production data () have Production data () have

Year 0 Year 10 Year 20 Year 30 Year 40

Production data () have Production data () have Production data () have

Year 0 Year 10 Year 20 Year 30 Year 40

What is the value of the future data ()?

iR1 iR2 iR3

iR1 iR2 iR3

Max Average NPV = EVWOI 51

If iR2 is the truth Simulate data

If iR1 is the truth Simulate data

If iR2 is the truth Simulate data Update initial ensemble

If iR1 is the truth Simulate data Update initial ensemble

Step 10: Calculate VOI for a risk-neutral decision maker.

VOI = EVWI – EVWOI

Data () will be Data () will be Data () will be end

Year 0 Year 10 Year 20 Year 30

What is the value of the future data (, , and )?

iR1 iR2 iR3

iR1 iR2 iR3

Max Average NPV = EVWOI 61

Update 1st updated Simulate data for

Apply optimal controls

Step 10: Calculate VOI for a risk-neutral decision maker.

VOI = EVWI – EVWOI

You might also like