Inde 311 Stochastic Models and Decision Analysis

IndE 311
Stochastic Models and Decision Analysis

UW Industrial Engineering
Instructor: Prof. Zelda Zabinsky
Decision Analysis-1
Operations Research
The Science of Better
Decision Analysis-2
Operations Research
Modeling Toolset
311
Queueing
Theory
Simulation
Inventory
Theory
Forecasting
310
Markov
Chains
PERT/
CPM
Decision
Analysis
Stochastic
Programming
Markov
Decision
Processes
Dynamic
Programming
Game
Theory
Network
Programming
Linear
Programming
Integer
Programming
Nonlinear
Programming
312
Decision Analysis-3
IndE 311
Decision analysis
Decision making without

experimentation
Decision making with
experimentation
Decision trees
Utility theory
Markov chains
Modeling
Chapman-Kolmogorov equations
Classification of states
Long-run properties
First passage times
Absorbing states
Queueing theory
Basic structure and modeling

Exponential distribution
Birth-and-death processes
Models based on birth-and-death
Models with non-exponential
distributions
Applications of queueing
theory
Waiting cost functions
Decision models
Decision Analysis-4
Decision Analysis
Chapter 15
Decision Analysis-5
Decision Analysis
Decision making without experimentation
Decision making criteria
Decision making with experimentation

Expected value of experimentation
Decision trees
Utility theory
Decision Analysis-6
Decision Making without

Experimentation
Decision Analysis-7
Goferbroke Example
Goferbroke Company owns a tract of land that may contain oil

Consulting geologist: 1 chance in 4 of oil
Offer for purchase from another company: $90k
Can also hold the land and drill for oil with cost $100k
If oil, expected revenue $800k, if not, nothing
Payoff
Alternative
Oil
Dry
1 in 4
3 in 4
Drill for oil

Sell the land
Chance
Decision Analysis-8
Notation and Terminology

Actions: {a1, a2, }
The set of actions the decision maker must choose from
Example:
States of nature: {1, 2, ...}

Possible outcomes of the uncertain event.
Example:
Decision Analysis-9
Notation and Terminology

Payoff/Loss Function: L(ai, k)
The payoff/loss incurred by taking action a i when state k occurs.
Example:
Prior distribution:
Distribution representing the relative likelihood of the possible
states of nature.
Prior probabilities: P( = k)
Probabilities (provided by prior distribution) for various states of
nature.
Example:
Decision Analysis-10
Decision Making Criteria

Can optimize the decision with respect to several criteria
Maximin payoff
Minimax regret
Maximum likelihood
Bayes decision rule (expected value)
Maximin Payoff Criterion
For each action, find minimum payoff over all states of nature
Then choose the action with the maximum of these minimum
payoffs
State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
Min
Payoff
Minimax Regret Criterion
For each action, find maximum regret over all states of nature
Then choose the action with the minimum of these maximum
regrets
(Payoffs)
State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
(Regrets)
State of Nature
Action
Oil
Dry
Max
Regret
Drill for oil

Sell the land
Maximum Likelihood Criterion
Identify the most likely state of nature

Then choose the action with the maximum payoff under that state of
nature
State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
0.25
0.75
Prior probability
Bayes Decision Rule

(Expected Value Criterion)
For each action, find expectation of payoff over all states of nature
Then choose the action with the maximum of these expected
payoffs
State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
0.25
0.75
Prior probability
Expected
Payoff
Sensitivity Analysis with

Bayes Decision Rule
What is the minimum probability of oil such that we choose to drill

the land under Bayes decision rule?
State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
1-p
Prior probability
Expected
Payoff
Decision Making with

Experimentation
Goferbroke Example (contd)

State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
0.25
0.75
Prior probability
Option available to conduct a detailed seismic survey to obtain a

better estimate of oil probability
Costs $30k
Possible findings:
Unfavorable seismic soundings (USS), oil is fairly unlikely
Favorable seismic soundings (FSS), oil is fairly likely
Posterior Probabilities
Do experiments to get better information and improve
estimates for the probabilities of states of nature. These
improved estimates are called posterior probabilities.
Experimental Outcomes: {x1, x2, }
Example:
Cost of experiment:
Example:
Posterior Distribution: P( = k | X = xj)

Based on past experience:
If there is oil, then
the probability that seismic survey findings is USS = 0.4 =
P(USS | oil)
the probability that seismic survey findings is FSS = 0.6 =
P(FSS | oil)
If there is no oil, then
the probability that seismic survey findings is USS = 0.8 =
P(USS | dry)
the probability that seismic survey findings is FSS = 0.2 =
P(FSS | dry)
Bayes Theorem
Calculate posterior probabilities using Bayes theorem:
Given P(X = xj | = k), find P( = k | X = xj)
P( k | X x j )
P(X x j | k ) P( k )
P(X x
i
| i ) P( i )

We have
P(USS | oil) = 0.4
P(USS | dry) = 0.8
P(oil | USS) =
P(oil | FSS) =
P(dry | USS) =
P(dry | FSS) =
P(FSS | oil) = 0.6

P(FSS | dry) = 0.2
P(oil) = 0.25
P(dry) = 0.75

Optimal policies
If finding is USS:
State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
Expected
Payoff
Posterior probability
If finding is FSS:
State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
Posterior probability
Expected
Payoff
The Value of Experimentation

Do we need to perform the experiment?
As evidenced by the experimental data, the experimental outcome
is not always correct. We sometimes have imperfect information.
2 ways to access value of information

Expected value of perfect information (EVPI)
What is the value of having a crystal ball that can identify true
state of nature?
Expected value of experimentation (EVE)
Is the experiment worth the cost?
Expected Value of Perfect Information

Suppose we know the true state of nature. Then we will
pick the optimal action given this true state of nature.
State of Nature
Action
Oil
Dry
Drill for oil
700
-100
Sell the land
90
90
0.25
0.75
Prior probability
E[PI] = expected payoff with perfect information =

Expected Value of Perfect Information

Expected Value of Perfect Information:
EVPI = E[PI] E[OI]
where E[OI] is expected value with original information
(i.e. without experimentation)
EVPI for the Goferbroke problem =
Expected Value of Experimentation

We are interested in the value of the experiment. If the
value is greater than the cost, then it is worthwhile to do
the experiment.
Expected Value of Experimentation:
EVE = E[EI] E[OI]
where E[EI] is expected value with experimental
information.
E [EI ] E [value | experiment al result j ] P (result j )
j

Expected Value of Experimentation:
EVE = E[EI] E[OI]
E [EI ] E [value | experiment al result j ] P (result j )

j
EVE =
Decision Trees
Decision Tree
Tool to display decision problem and relevant
computations
Nodes on a decision tree called __________.

Arcs on a decision tree called ___________.
Decision forks represented by a __________.

Chance forks represented by a ___________.
Outcome is determined by both ___________ and ____________.

Outcomes noted at the end of a path.
Can also include payoff information on a decision tree branch

Decision Tree
Analysis Using Decision Trees

1. Start at the right side of tree and move left a column at a time. For
each column, if chance fork, go to (2). If decision fork, go to (3).
2. At each chance fork, calculate its expected value. Record this
value in bold next to the fork. This value is also the expected value
for branch leading into that fork.
3. At each decision fork, compare expected value and choose
alternative of branch with best value. Record choice by putting
slash marks through each rejected branch.
Comments:
This is a backward induction procedure.

For any decision tree, such a procedure always leads to an optimal
solution.

Decision Tree Analysis
Painting problem
Painting at an art gallery, you think is worth $12,000

Dealer asks $10,000 if you buy today (Wednesday)
You can buy or wait until tomorrow, if not sold by then, can be yours
for $8,000
Tomorrow you can buy or wait until the next day: if not sold by then,
can be yours for $7,000
In any day, the probability that the painting will be sold to someone
else is 50%
What is the optimal policy?
Drawer problem
Two drawers
One drawer contains three gold coins,
The other contains one gold and two silver.
Choose one drawer

You will be paid $500 for each gold coin and $100 for each silver
coin in that drawer
Before choosing, you may pay me $200 and I will draw a randomly
selected coin, and tell you whether its gold or silver and which
drawer it comes from (e.g. gold coin from drawer 1)
What is the optimal decision policy? EVPI? EVE? Should you pay
me $200?
Utility Theory
Validity of Monetary Value Assumption

Thus far, when applying Bayes decision rule, we
assumed that expected monetary value is the
appropriate measure
In many situations and many applications, this
assumption may be inappropriate
Choosing between Lotteries

Assume you were given the option to choose from two
lotteries
Lottery 1
50:50 chance of winning $1,000 or $0
Lottery 2
Receive $50 for certain
.5
.5
1
$1,000
$0
$50
Which one would you pick?
Choosing between lotteries

How about between these two?
Lottery 1
Lottery 2
.5
.5
1
Or these two?
Lottery 1
Lottery 2
$0
$400
.5
.5
1
$1,000
$1,000
$0
$700
Utility
Think of a capital investment firm deciding whether or
not to invest in a firm developing a technology that is
unproven but has high potential impact
How many people buy insurance?
Is this monetarily sound according to Bayes rule?
So... is Bayes rule invalidated?

No- because we can use it with the utility for money
when choosing between decisions
Well focus on utility for money, but in general it could be utility
for anything (e.g. consequences of a doctors actions)
A Typical Utility Function for Money

u(M)
4
3
What does
this mean?
2
$100 $250
$500
$1,000
M
Decision Makers Preferences

Risk-averse
u(M)
Avoid risk
Decreasing utility for money
Risk-neutral
M
u(M)
Monetary value = Utility

Linear utility for money
Risk-seeking (or risk-prone)
M
u(M)
Seek risk
Increasing utility for money
Combination of these
M
u(M)
M
Constructing Utility Functions

When utility theory is incorporated into a real decision
analysis problem, a utility function must be constructed
to fit the preferences and the values of the decision
maker(s) involved
Fundamental property:
The decision maker is indifferent between two alternative
courses of action that have the same utility
Indifference in Utility
Consider two lotteries
p
1-p
$1,000
$0
$X
The example decision maker we discussed earlier

would be indifferent between the two lotteries if
p is 0.25 and X is
p is 0.50 and X is
p is 0.75 and X is
Goferbroke Example (with Utility)

We need the utility values for the following possible
monetary payoffs:
Monetary
Payoff
Utility
u(M)
45
-130
-100
60
90
670
700

Goferbroke Example
u(0) is usually set to 0. So u(0)=0
We ask the decision maker what value of p makes
him/her indifferent between the following lotteries:
p
1-p
700
-130
The decision makers response is p=0.2

So

Goferbroke Example
We now ask the decision maker what value of p makes
p
1-p
700
0
90

So

Goferbroke Example
We now ask the decision maker what value of p makes
p
1-p
700
0
60

So
Goferbroke Example (with Utility)

Decision Tree
Exponential Utility Functions
One of the many mathematically prescribed forms of a closed-form

utility function
u(M ) R 1 e
M
R
It is used for risk-averse decision makers only

Can be used in cases where it is not feasible or desirable for the
decision maker to answer lottery questions for all possible outcomes
The single parameter R is the one such that the decision maker is
indifferent between
0.5
0.5
R
-R/2
and
(approximately)

Inde 311 Stochastic Models and Decision Analysis

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Inde 311 Stochastic Models and Decision Analysis

Uploaded by

Copyright:

Available Formats

IndE 311

Stochastic Models and Decision Analysis

Decision making without

Basic structure and modeling

Decision making with experimentation

Decision Making without

Goferbroke Company owns a tract of land that may contain oil

Drill for oil

Notation and Terminology

States of nature: {1, 2, ...}

Notation and Terminology

Decision Making Criteria

Maximin Payoff Criterion

Drill for oil

Sell the land

Minimax Regret Criterion

Drill for oil

Sell the land

Drill for oil

Maximum Likelihood Criterion

Identify the most likely state of nature

Drill for oil

Sell the land

Bayes Decision Rule

Drill for oil

Sell the land

Sensitivity Analysis with

What is the minimum probability of oil such that we choose to drill

Drill for oil

Sell the land

Decision Making with

Goferbroke Example (contd)

Drill for oil

Sell the land

Option available to conduct a detailed seismic survey to obtain a

Goferbroke Example (contd)

Goferbroke Example (contd)

P(FSS | oil) = 0.6

Goferbroke Example (contd)

Drill for oil

Sell the land

Drill for oil

Sell the land

The Value of Experimentation

2 ways to access value of information

Expected Value of Perfect Information

Drill for oil

Sell the land

E[PI] = expected payoff with perfect information =

Expected Value of Perfect Information

Expected Value of Experimentation

Goferbroke Example (contd)

E [EI ] E [value | experiment al result j ] P (result j )

Nodes on a decision tree called __________.

Decision forks represented by a __________.

Outcome is determined by both ___________ and ____________.

Goferbroke Example (contd)

Analysis Using Decision Trees

This is a backward induction procedure.

Goferbroke Example (contd)

Painting at an art gallery, you think is worth $12,000

Choose one drawer

Outcome is determined by both _ and __.