Base Rate Neglect and the

Gambler’s Fallacy
Business Decisions
SOC 138/ECO 148

1
From Reason to Error....

2
 Class Overview
■ Three Components:

Part 1: Models of Rational and
Irrational Decision Making Behavior

Part 2: Pathologies in Individual
Decision Making

Part 3: Decision Making in Social
Contexts

3
 Back to Bayes
■ As we saw previously, the core principle
of rational judgment is Bayes’ Theorem:

Probability of Probability of
data given H H before data

p  D∣H  p H 
p H∣D=
p D∣H  p  H  p  D∣¬H  p¬H 

Probability of Probability of NOT

H after data Probability of
data given H before data
NOT H
4
 Base Rate Neglect
■ Thinking
in accordance with Bayes’
Theorem is not easy

Must properly integrate evidence with
prior knowledge
■ Often,
decision makers focus only
on evidence at hand, underweighting
previous experience

This is called “base rate neglect”

5
 What’s the Trouble With Base Rate Neglect?

■ Ratio form of Bayes’ Theorem:


Posterior odds = prior odds x likelihood ratio:
p  H∣D  p  H  p  D∣H 
=
p  ¬H ∣D  p  ¬H  p  D∣¬H 
■ Where prior odds are heavily skewed,
substantial evidence is needed to overcome
them

If prior odds were 100:1, for instance, likelihood
ratio must be 1:100 just to break even!
■ If we neglect base rates, our judgments may be
wildly inaccurate

6
 Example – Detecting Terrorists
■ Background assumptions:

1 in 10,000,000 travelers is a terrorist (~600 million
travelers/year in US)

Screening process is 99% accurate (99 to 1)
■ Bayes’ Theorem given a positive test:
1 99 99 100
= ≈ =0.00001
9999999 1 9999999 10000000

Odds are still about 100,000 to 1 against!
■ If we neglected base rates…

Would assume posterior of 99% (99 to 1 for)

Would be wrong by a factor of about 10,000,000!
➔
If used directly, about 100,000 false alarms for every actual
“hit”
➔
(In real life, base rates and test accuracies likely even
lower than in this "toy" scenario. Thus, things are worse
than the example implies....) 7
 Example – Eyewitness Testimony
"A cab was involved in a hit-and-run
accident. Two cab companies serve
the city: the Green, which operates
85% of the cabs, and the Blue, which
operates the remaining 15%. A
witness identifies the hit-and-run cab
as Blue. When the court tests the
reliability of the witness under
circumstances similar to those on the
night of the accident, he correctly
identifies the color of the cab 80% of
the time and misidentifies it the other
20%. What’s the probability that the
cab involved in the accident was Blue,
as the witness stated?"

p  Says Blue∣Is Blue  p  Is Blue 

p  Is Blue∣Says Blue  =
p  Says Blue∣Is Blue  p  Is Blue   p  Says Blue∣Is Green  p  Is Green 
0.80.15
= ≈0.41
0.80.150.20.85
8
 Example – Eyewitness Testimony
"Pseudo-solution" Effect? Base-rate Neglect
(0.8*0.15=0.12)
"A cab was involved in a hit-and-run
accident. Two cab companies serve
the city: the Green, which operates
85% of the cabs, and the Blue, which
operates the remaining 15%. A
witness identifies the hit-and-run cab
as Blue. When the court tests the
reliability of the witness under
circumstances similar to those on the
night of the accident, he correctly
identifies the color of the cab 80% of
the time and misidentifies it the other
20%. What’s the probability that the
cab involved in the accident was Blue,
as the witness stated?"

p  Says Blue∣Is Blue  p  Is Blue 

p  Is Blue∣Says Blue =
p  Says Blue∣Is Blue  p  Is Blue  p  Says Blue∣Is Green  p  Is Green 
0.80.15
= ≈0.41
0.80.150.20.85
9
 Do People Always Neglect Base Rates?
■ Base rate neglect is common, but not
universal

“Conservatism” in predictions

Persistence of stereotypes
■ The jury is still out – but some guidelines:

Evidence overweighted when
➔
Evidence presented in terms of “accuracy” (confusion
of the inverse)
➔
Evidence seems causally connected
➔
Prior knowledge seems causally disconnected

Evidence underweighted when
➔
Prior knowledge is of a salient causal or categorical
form
➔
Evidence seems weak, causally disconnected
10
 Example – Predicting Firm Failure
"Imagine that you are asked to
evaluate the 5-year viability of a new
consulting firm, ConsultInc. Based
on your prior research, you have
established that 20% of similar firms
in this area survive for at least 5
years, with 80% going out of
business within this period.
However, your research also reveals
that 90% of non-surviving firms had
asset/liability ratios which were
worse than ConsultInc, as opposed
to 50% of surviving firms. What is
the probability that ConsultInc will
survive for at least 5 years?"
p  A/L∣Survive  p  Survive 
p  Survive∣A/L  =
p  A/L∣Survive  p  Survive  p  A/L∣¬Survive  p  ¬Survive 
0.50.2
= ≈0.56
0.50.20.10.8
11
 Example – Predicting Firm Failure
"Conservatism"
"Imagine that you are asked to
evaluate the 5-year viability of a new
consulting firm, ConsultInc. Based
on your prior research, you have
established that 20% of similar firms
in this area survive for at least 5
years, with 80% going out of
business within this period.
However, your research also reveals
that 90% of non-surviving firms had
asset/liability ratios which were
worse than ConsultInc, as opposed
to 50% of surviving firms. What is
the probability that ConsultInc will
survive for at least 5 years?"
p  A/L∣Survive p  Survive
p  Survive∣A/L  =
p  A/L∣Survive  p  Survive   p  A/L∣¬Survive  p  ¬Survive 
0.50.2
= ≈0.56
0.50.20.10.8
12
 “Base Rates” More Broadly: Underweighting Prior
Information

■ Prior knowledge may take many forms:


Known rates

Scientific theories

Personal experience
■ “Base rate” neglect encompasses failing to use
any relevant prior information
■ Application: Scams and Marketing

Attempt to convince buyer that past experience is
irrelevant

Use of innovation (“New breakthrough!”)

Attack on credibility of existing data sources (“What
they don’t want you to know!”)

13
 Independent Events
■ Two events are independent if and only if
p(AB)=p(A)p(B)

Intuition: the probability of event A given event
B is just the probability of event A (follows from
p(AB)=p(A|B)p(B))
■ Examples:

Tosses of a fair coin

Rolls of a fair die

(Arguably) small deviations in price sequences

14
 The Gambler’s Fallacy
■ Fact: Probability of an independent event
sequence is the product of the event
probabilities
■ Gambler’s Fallacy: p(A|A…A)<p(A)

That is, repeated occurrences of an independent
event make it less likely in the future

Violates probability theory, since future realizations
of an independent process cannot depend on past
realizations
■ Belief linked to the idea of probability as “self-
correcting”

In some cases, misunderstanding of law of large
numbers

15
 Some Examples
■ “If a lottery number wins today, it is less likely to win
tomorrow.”

Agree: 38% Disagree: 62%
■ "Suppose an unbiased coin is flipped 10 times, and each time
the coin lands on Heads. If you had to bet $100 on the next
toss, what would you choose?"

Heads: 35% Tails: 23% No Pref: 42%
■ "Which of the following sequences of Heads (H) and Tails (T)
has alternations more typical of a random sequence of coin
flips?"

HTHHHTTTTHTHHTTTHHHTH [43%] (10 switches; 50%>x)

HTHTHTTTHHTHTHTTHHHTH [57%] (14 switches; 4%>x)

16
 Examples – Gambling in the Real World
■ Clotfelter and Cook (1993)

Players in Maryland daily
numbers game avoid placing
bets on numbers which have
recently won
■ Metzger (1985)

Race-track gamblers more
likely to bet on “favorites”
after a series of races won by
“long shots”
■ Application: “Progressive
slots”

Jackpot not only makes
potential payoff larger, but
also emphasizes how long it
has been since the game was
last won
17
 Summary
■ Two pathologies of judgment:

Base Rate Neglect: Underweighting prior
knowledge
➔
Focus on evidence in hand, neglecting what we knew
beforehand
➔
Can lead to dangerous mistakes where priors are
strong

Gambler’s Fallacy: Superstitious belief in
dependence of sequences
➔
Idea that certain numbers will “come due,” or luck
will “run out”
➔
Seen in real-world gambling behavior
■ Next time: associative thinking and
representativeness
18
Associative Thinking and
Representativeness
Business Decisions
SOC 138/ECO 148

1
 Associative Thinking, Conditioning, and Reinforcement

■ Much of our everyday cognition is based on

associative reasoning, rather than normative
procedures of judgment (e.g., Bayes’ Theorem)
■ Associations are linkages between stimuli,
judgments, and behaviors, generally brought
about by observations of co-presence

Creation of associations between sets of stimuli

Creation of associations between behavior and
aversive/reinforcing stimuli (conditioning)
■ Such cognition is “low-level,” and not always
conscious

2
 Positive/Negative Association
■ Reinforcement is both
automatic and “superstitious”
■ Implication: positive and
negative associations can be
created which have no rational
basis

Requirement: co-presence of
reinforcing/aversive stimulus
and target of interest

Subjects are not aware of
conditioning process, and
cannot later discern its effects
■ Classic application: use of
celebrities, models, and the
like to sell unrelated products 3
 Examples
■ Razran’s (1940) “luncheon technique”

Subjects shown political messages, some while
eating

Those seen while eating later rated higher (test-
retest), unlike those seen while not eating
➔
Reverse results obtained when noxious odors introduced

Subjects did not recall which were seen while eating
■ McCall and Belmont (1996), Feinberg (1990)

Subjects spend more when credit card insignias
present

Only works, however, for subjects with a positive
history of credit card usage

4
 Associations of Power and Authority

■ Symbols of power generate associations with

authority, and greater compliance

Bickman (1974), Bushman (1988): persons stopped on
the street were more likely to obey random orders from
someone dressed as a private security guard than as a
“regular person”

Lefkowitz, Blake, and Mouton (1955): more people follow
a suited jaywalker than one in ordinary clothes
■ Those who are seen wielding power and
authority in one domain are often perceived as
having it in others

Collected quotations of famous persons

Recruitment of specialists into leadership roles
5
 Example – the TV Doctor
■ 1970s adds for Sanka
decaffeinated coffee featured
actor Robert Young

Why? Because he was well-
known as “Marcus Welby,
M.D.” on the eponymous TV
show
■ Ads warned of the potential
health risks of caffeine

Although an actor in reality,
the association with medical
authority via fiction influenced
many people
■ Imperfect ability to separate
fictional roles from those
who play them 6
 Causal Association
■ Co-presence can also lead to the
development of causal associations

“Post hoc ergo propter hoc” (“After this
therefore because of this”)

In some cases, temporal ordering is not even
necessary (“Whenever B happens, A is there;
therefore, A must cause B!”)
➔
Subjects are not usually attentive to cases in which A
is present without B occurring….
■ (We will return to this idea later, when we
study pattern recognition and the
regression effect!)

7
 Example – Medical Marketing
■ Cold medication

The common cold is viral, and cannot currently be cured
(although symptoms can be alleviated)

Marketers sometimes exploit limited knowledge of this
by implying that their products cure colds

“I took X Cold-Away for 5 days, and my cold vanished!”
(But, of course, colds do go away on their own….)
■ Causal association combines with
positive associations in alternative
medicine

“When I’m around X, I feel good. Therefore, X will cure
my illness!” (Let’s hope that X isn’t “wild parties”….)

“X sounds good/is pleasant. Therefore it must be good
for me.” (“Natural” is a standard example; but arsenic is
natural too.)
8
 Representativeness
■ A common heuristic: group people or
situations into categories based on perceived
similarity

Similarity generally based on matching salient
features
■ Can result in many fallacies:

Base rate neglect (“But he looks like an axe
murderer!”)

Conservatism in probability judgments (“That must
have been a fluke; I know her kind.”)

Fallacious imputation (“But everyone who dresses
like that loves Band X!”)

Conjunctive fallacy (see example)

9
 Example – Stereotypes

Data from Kahneman and Tversky (1983):

"Linda is 31 years old, single, outspoken, and very bright. She majored in
philosophy. As a student, she was deeply concerned with issues of
discrimination and social justice, and also participated in antinuclear
demonstrations."

Linda is a bank teller. [14%]

Linda is a bank teller and is active in the feminist movement. [86%]

Similar results for stereotypes with other

professions – lawyers, musicians, engineers, etc.

10
 How Similar is Similar Enough? The Contrast Model

■ Note that the representativeness heuristic implies

association based on similarity
■ “Similarity” defined in terms of surface features

More easily observed features are likely to be more
salient (even if less diagnostic)

Features are often more salient if “typical” of the group
(I.e., usually present among group members)
■ Focus on matches rather than mismatches

Raw number of features compared is underweighted
■ Matches to alternative categories often ignored

Features may not be diagnostic (I.e., may be present
among non-group members as well)

11
 Example – Checklists for Detecting Malfeasance

■ Authorities (governmental, familial, etc.)

sometimes employ “checklists” to detect hidden
malfeasance
■ Items found on a “Behavioral and Physical Signs
of Drug and Alcohol Use” checklist:

Change in sleeping patterns

Bloodshot eyes

Changes in grades

Furtive or secretive behavior

Locked doors

Refusal to do chores
■ How many of these are also found among
adolescents who are not using drugs/alcohol?
12
 Scenarios as Representative Cases
■ As we have already seen, humans tend
naturally to think in terms of “stories” or
“scenarios”

Where scenarios can be seen as “typical” of one or
another case, representativeness may also come
into play

Tversky and Kahneman (1982): “As the amount of
detail in the scenario increases, its probability can
only decrease steadily, but its representativeness
and hence its apparent likelihood may increase.
The reliance on representativeness, we believe, is a
primary reason for the unwarranted appeal of
detailed scenarios and the illusory sense of insight
that such constructions often provide.”

13
 Representativeness and Conservatism/Base Rate Neglect

■ Similarly, where some evidence seems

strongly associated with (“representative of”) a
given hypothesis, it is more likely to be noted
■ Priors based on stereotypes or detailed case
experience

The richer the case, the more compelling the
experience, even if the actual evidence is weak
■ Evidence in hand based on “typical” signs

Clinical signs and diagnosis

14
 Summary
■ Associative mechanisms are behind much of
our thought and behavior

Can be unconscious

May have no valid empirical rationale
■ Representativeness: associations based on
similarity to idealized types

Leads to many errors in judgment

Behind common phenomena such as racial
stereotypes
■ Next time: memory and hindsight bias (but you
knew that all along, right?)

15
Memory and Hindsight Bias

Business Decisions
SOC 138/ECO 148

1
 More Low-Level Processing
■ As with association formation, mnemonic
processing is a “low-level,” largely
unconscious phenomenon
■ Thus, as one might expect, actual
mnemonic processes are quite far from
their idealized versions

Memory is not a “tape” which can be
rewound

Not all information is stored in memory

Retrieval is a haphazard process

2
 The Importance of Remembering
■ Good decision making requires memory

“What happened when we did this last time?

“How often has this happened?”

“Where did I leave those incredibly insightful
readings from Business Decisions?”
■ Where memory is distorted, decision
making suffers

“No one will ever want a Pentium on the desktop”

“This bull market will never end!”

3
 How Do We Remember
■ Computational model

Working memory
➔
Short duration storage

Long-term memory
➔
Long-term storage of “durable
traces”

Retrieval process
➔
Based on associations with traces
■ Traces and reconstruction

Limited information (including sense
perceptions, cognitions, and
associations) is stored in “traces”

When a memory is retrieved, the
traces are used to reconstruct the
original data

Elements of the reconstruction may
be “saved” as new traces
(sometimes “overwriting” the
originals)
4
 Cognitive Structure Example

Sample Mental Model from Coffey et al. (2004) 5

 What Do We Remember?
■ More likely to be remembered

Recent events

Events specifically looked-for

Events which are consistent with past experience
(assuming we have past experience)

Events causing emotional arousal

General trends
■ Less likely to be remembered (ceteris paribus)

Distant events

Events not looked-for

Atypical events

Specific counts, quantities, or details

6
 Examples – Reporting on Our Experience

■ Meeting attendance – Freeman, Romney, and

Freeman

Regular attendees report who usually comes…
whether or not they are present!

Newcomers have high overall error rates, but are
not biased
■ Personal histories – Pixley

Informants disagree on when marriages occurred,
how many children they have, etc.

Survey context difficult – need special instruments
to aid retrieval

7
 Confabulation
■ Remembering is a creative act: traces are combined with
other information in reconstruction to produce memories

When this process produces an incorrect memory, we call it
“confabulation”
■ Confabulated memories are no different from other
memories

Can be extremely vivid, detailed

Feel “real” – little if any link between confidence and accuracy
■ Confabulation can be produced through repeated
demands

Interrogation (especially torture) or any repeated questioning
■ Confabulated memories will be “saved” as the real thing

Hence, memory is also a destructive act

Repeated high-pressure questioning can permanently erase
knowledge
8
 Example – Hypnosis and Extreme Confabulation

■ Myth: hypnosis produces perfect (or even

improved) recall
■ Reality: hypnosis produces confabulation
■ Examples:

Satanic ritual abuse

UFO abduction

Multiple personality disorder

Recovered memories of childhood trauma
■ Suggestion: don’t sign up for this one….

9
 Hindsight Bias
■ If memory is really a collection of “filled in” traces,
and if traces gradually decay with time, then
shouldn’t memory be biased towards recently
acquired experience?

Absolutely – this manifests as the hindsight bias
■ Hindsight: our memories of the past are distorted
so as to be more consistent with present
experiences and beliefs

Current stimuli can act as cues for reconstruction
➔
Loftus and Palmer’s (1974) “smashed” cars

Present beliefs can color our evaluation of the past
➔
Memories of conservatism/liberalism (Markus, 1986)

10
 Example – Believing in Interventions
■ Conway and Ross (1984)

Students randomly allocated to study skills
program or control condition

Grades/study skills assessed before and after
program

Program produced no improvement; but students
believed it to be effective
➔
The secret? Program participants misremembered their
original academic performance as being worse than it was!
■ Devious management consulting practices

Rather than trying to produce improvements,
some consultants may just make the past look
bad!

11
 “Creeping Determinism”
■ The difficulty of separating the present from the
past can make the past look more certain than it
was
■ Example: Fischhoff’s prospective historical
studies

Subjects asked to make predictions regarding political
events (e.g., Nixon’s 1972 visit to China)

Later recall of predictions biased towards actual
outcome
■ Risks in managerial decision making

Overestimation of managerial (and firm) agency

Overzealous allocation of rewards/blame

Difficulty in recognizing patterns of similar events
12
 Summary
■ Memory is essential for decision making

Require past experience to make judgments regarding
the future
■ Memory is constructive, not passive

Minimal information stored in traces

Traces retrieved via association are reconstructed into
memories

Memories are then often re-written into traces
■ Constructive nature of memory leads to many
biases

Confabulation: remembering what never was

Hindsight bias: remembering things as they are now
■ Next time: Pattern recognition and illusionary
expertise
13
Illusionary Expertise and
Pattern Recognition
Business Decisions
SOC 138/ECO 148

1
 The Brain as a Pattern-Recognition Device

■ Evolutionary prerogative: detect important

patterns in the environment

Signs of food, water, shelter, predation

Learning of speech, social codes
■ The problem: imperfect calibration

The brain sees patterns in data, whether accidental
or not

Evolution based on a “good enough” model –
sophisticated probabilistic reasoning not high on
the agenda
■ Our inheritance: superstitious learning

Affects performance in business, politics, and
everyday life

2
 Belief in the “Law of Small Numbers”
■ Tversky and Kahneman (1971): many people
act as though they believe in a “law of small
numbers”

Error in extrapolating from small samples

Underappreciation for the effects of random
variation
■Numerous consequences:

Underestimation of risk in small-
sample extrapolation

Overestimation of trend stability

Overconfidence in forecasting

Interpretation of chance deviations
as causally significant

3
 Example – Buy-and-Hold, Versus Cut-and-Run

■ The rise and fall of investment strategies


When market was increasing in the booms of the
1980s and 1990s, buy-and-hold strategies were
widely advocated

Later, with falling markets, buy-and-hold became
less prominent
■ Part of the reason? Extrapolation from small
samples

When market goes up, many think it will continue
forever

When market goes down, similar reasoning holds
■ A moral: don’t get taken by “new economy”
hype

4
 Post-hoc Assessments of Probability
■ When a salient event happens,
we often consider the probability
of seeing that event by chance

“What are the odds of my old friend
Josie calling right after I thought
about her?”

In many cases, however, we should
consider populations of similar
events, e.g., “What are the odds of
some old friend calling right after I
thought of him or her, eventually?”
■ “Paradox of the Dart Board”

The dart must land somewhere, but
each exact location is infinitely
improbable!

5
 Example – Professional Psychics
■ Standard mentalist trick: throw a vague
statement to a large audience, and see who
“bites”

“I’m getting the name `Howard’; is there a `Howard’
here?”
■ Exploits post-hoc probability assessment

Audience members ask, “What is the chance of
knowing that his name was Howard?”
➔
Post-hoc assessment: the event was not defined until after
the match was made, then retroactively calculated

Should ask, “What is the chance of someone in the
audience having some connection to the name
`Howard’?”

6
 Overfitting and Post-hoc Theorizing
■ A probability model (formal or informal) can be
described in terms of the number of free
parameters (adjustable quantities) it contains

Overfitting occurs when free parameters are
adjusted to make a model match a specific instance,
at the expense of overall predictive power

In general, the more free parameters, the greater the
danger of overfitting
■ The problem of post-hoc theorizing

After the fact, one can nearly always find a theory
which fits

Need rigorous statistical controls to minimize bias

7
 Example – “New Coke” Conspiracy Theories

■ The Odd Saga of “New Coke”:


In 1982, Coca-Cola executives try to
fight Pepsi via a sweeter beverage

“New Coke” launched in 1985 –
provokes consumer fury

Three months later, the original is
returned to market
■ After the fact, some claim it was
“obviously” a plot

Apparent marketing blunder, quick
turnaround, subsequent success seen
as signals that it all must have been
planned

Stories can seem compelling after the
fact – but no matter what happened, one
could have come up with a good story!
8
 Seeing Patterns in Data
■ Without training, it is hard to tell when
“unusual” patterns are present
■ Random events

For rare events, “clusters” happen more frequently
than we realize

Overestimation of significance for rare disease
“outbreaks,” clusters of accidents
■ Binary sequences

Real random sequences have more runs than we
think

Alternation probabilities of 0.6-0.7 judged most
“random” (Wagenaar, 1970; Gilovich et al., 1985)

Example we saw earlier (from DE 2):
➔
HTHHHTTTTHTHHTTTHHHTH [43%] (10 switches; 50%>x)
➔
HTHTHTTTHHTHTHTTHHHTH [57%] (14 switches; 4%>x)
9
 Example – The “Hot Hand”
■ Common belief: basketball players’
shots evidence positive dependence
■ Empirical evidence (Gilovich et. al,
1985):

No hot hand found in Philadelphia 76’ers
floor shots, Boston Celtics free throws, or
Cornell varsity team members in controlled
tests

Players and observers overwhelmingly
believed hot hand was present
■ Why the persistance of belief?

Good stories (remember scenario thinking?)

Perception of alternating series as more
random than real randomness

10
 Convincing Ourselves – Illusionary Expertise

■ Where we try to predict random sequences, we

can fall prey to illusionary expertise

Ambiguity of phenomenon – can’t tell whether we’ve
“got it right”

Post-hoc theorizing – we can always find a good
story

Law of small numbers – temporary success seems
predictive

Intermittent reinforcement – sometimes we guess
correctly
■ Convinced of our abilities, we may bet heavily
on them

Worse still, further experience can solidify our initial
impressions!

11
 Example – Technical Trading
■ Efficient market hypothesis: price sequences
should be random walk

By rational expectations, any predictive power of
past prices should already be incorporated into
current price

Future price changes are thus independent of past
ones
■ Technical trading: practice of trading based on
patterns in price sequences

Idea: somehow, we can predict the market based on
its past behavior!

But are we just seeing patterns where there are
none?

12
13
14
 Technical Trading, Cont.
■ A real-world example:

Price and volume data
from NASDAQ.com
(10/28/02), RHAT

Fit to data, ACF, PACF
indicates a random
walk
■ Note similarity in
appearance to artificial
sequences!

But can you resist
trying to extend that
sequence?

15
 Summary
■ Humans naturally see patterns in randomness
■ Several common fallacies to avoid

Post hoc probability assessments which define events
themselves after the fact

Overfitting in post hoc theorizing

Unwarranted extrapolation from small samples

Perception of patterns in random data
■ Implication: illusionary expertise

We can be lead to believe we have predictive powers
when we don’t
■ Next time: I predict we’ll cover the regression
effect

16
The Regression Effect

Business Decisions
SOC 138/ECO 148

1
 A “Tall” Tale
■ Sir Francis Galton

Credits:
➔
Biologist
➔
Introduced the correlation coefficient

More questionable activities:
➔
Founder of eugenics movement
➔
Made a “beauty map” of the British
Isles
■ Discovery: height of children is
more moderate than their
parents

1885, “Regression Towards
Mediocrity in Hereditary Stature”
■ Little did he know that his
discovery would have
implications for classes such
as this one….
2
 Regression to the Mean
■ For two imperfectly correlated variables, extreme
values on one variable tend to be matched with
less extreme values on another

“Extremity” here is defined relative to the population mean

Variables may be entirely different, the same measure at
different points in time, etc.
■ Galton’s height observation as regression

Child’s height is correlated with parent’s height (but only
somewhat)

Very tall parents tend to have shorter children; very short
parents tend to have taller children

(Very tall children tend to have shorter parents, too!)

3
 A Visual Explanation
Regression
of x on y
Son’s
Height
74
(in) Regression
72 of y on x
70 y
68
66
64
62
60
58
x
56
Father’s
52 54 56 58 60 62 64 68 70 72 74 76 78 80 82 Height (in)

4
 Some Contexts in Which Regression Will Occur

■ When predicting one variable from another


Predictions will regress (xy  yx)

Example: SAT Math vs SAT Verbal scores
■ When an outcome depends partially on
intrinsic factors, and partially on chance

Successive outcomes will exhibit regression

Example: sales performance
■ When measurements contain error

Multiple measurements (test-retest) will regress

Example: MMPI scores

5
 Example: Discoveries and Inventions
■ Number of
scientific
discoveries in one
year is imperfectly
correlated with the
next

Correlation: ≈0.28
■ Result: regression
to the mean

Years with many
discoveries
followed by years
with fewer
6
 How Much Regression Will There Be?
■ Total extent of regression between two standardized
measurements depends on their correlation

For 0, equal to 1-, where  is the Pearson product-moment
correlation coefficient

Example: if the correlation between performance in subsequent
quarters is 0.5, then we would expect second quarter
performance to be equal to the average of first quarter
performance and the overall mean
■ Perfectly correlated measures will not regress at all

Whatever you got before, you’ll get again
■ Uncorrelated measures will regress completely

All information from first measurement is “lost”

y − y y − y 1 x− x y − y 1 x− x y − y 3 x− x y − y x− x

=0 = = = =
y y 4 x y 2 x y 4 x y x

1 1 3
 =0 = = =  =1 7
4 2 4
 Confusions to Avoid
■ Regression to the mean is not the gambler’s
fallacy!

In the gambler’s fallacy, we assert dependence
between independent events (“chance is self-
correcting”)

The regression effect states (in effect) that “unusual
events are unlikely to happen” (and therefore
unlikely to repeat)
➔
Not unlikely because it happened last time!
■ Regression towards the mean is not absolute
for positively correlated variables

Daughters of tall mothers are still taller than
average, just less tall (on average) than their (tall)
mothers
8
 Measurement
■ Most measurement contains error

Observations equal signal plus noise
■ Where this is so, repeated measures will regress to the
mean

A particularly high SAT score will tend to be followed by a
more moderate score
■ Similarly, measures on one variable will predict
regressively on others

A group of high-SAT students will not be as extreme on GPA,
and vice versa
■ One must be careful not to leap to causal explanations
for such effects

Inevitable artifacts of the correlation structure

9
 Illusionary Trends
■ Poor Horace Secrist

In 1933, wrote The Triumph of Mediocrity in Business

Found that businesses with exceptional profits had lower
profits the next year; the reverse was true of businesses with
especially poor performance

Concluded that strong companies were getting weaker, weak
companies getting stronger

Hotelling and others pointed out that this was an illusion.…
same data “showed” that high performers had been previously
mediocre, and hence that performance was becoming more
extreme
■ General cautionary tale: be careful of identifying
“trends towards mediocrity” – you may merely
have found the regression effect!

10
 Careers and Performance Evaluation
■ Evaluation of performance over time is quite
vulnerable to regression effects

“Getting lazy” and “turning it around”

Unrealistic expectations of greatness
➔
“Junior stars” lose their shine (“second-year jinx”)

Overinterpretation of poor performance
➔
“Bounce-back” effects
■ The “Sports Illustrated Jinx”

Supposedly, being featured is bad luck

Evidence: those who are featured are having great
seasons, but don’t do as well subsequently

Regression effect – those who are featured tend to have
supranormal performances

11
 Praise and Blame
■ A common experimental finding: reward is a
more effective motivator than punishment in
typical training contexts
■ Rarely believed – seems to contradict
experience

When people do well and we praise them, they tend
to do worse next time

When people do badly and we blame them, they
tend to do better next time
■ As Kahneman points out, this “experience” is
based on the regression effect

Very difficult to counteract, in practice

12
 The “Lost Cause” Scam
■ A classic family of scams, based on the regression
effect
■ The setup:

Some of your organizational units are failing

Scam artist claims to be able to improve performance

Offers to intervene for worst of the failing units – you only pay
for those which show improvement
■ What a bargain! Or is it?

Performance at two points in time imperfectly correlated

By regression effect, we expect for worst units at time one to
show an average improvement

Net result: scam artist gets paid, no matter how ineffectual
■ Applicable in many contexts

Firms, schools, individuals, etc.

13
 Quack Medicine
■ Much like the “Lost Cause” scam, ignorance of the
regression effect can make us vulnerable to quack
medical treatments
■ Scenario:

You are very sick

You try Reginald Jones’ Powdered Prairie Wigs™

Later, you feel somewhat better
■ Is it the Prairie Wigs™? Or is it regression?

We tend to seek treatment when at our worst

The course of illness is uneven; thus, we may see some
improvement due to the regression effect
■ It also works in reverse: consider failed “health
maintenance” strategies

14
 Assessing Interventions
■ The “Lost Cause” scam raises a tricky question: just
how can you tell whether your intervention works?
■ First rule: don’t look only at extreme cases

The extremes are where you will see the most regression
■ Second rule: evaluate change relative to the expected
regression

Take the reliability of your measures into account

Don’t trust an intervention which seems to have “perverse”
effects
■ Third trick: look for changes in population mean

If the intervention really works, it should have the net effect of
pulling the total average upwards

15
 Summary
■ Regression effects are inevitable wherever we
find imperfect correlations
■ Misunderstanding of this effect can lead to
many errors in judgment

Perceptions of convergence within a population

Belief in the efficacy of useless interventions

Superstitious career-related beliefs
■ Avoiding this trap requires understanding and
anticipating regression to the mean
■ Next time: confirmation bias

16
Confirmation Bias

Business Decisions
SOC 138/ECO 148

1
 Learning from Evidence
■ In order to learn from
evidence, we must
consider the
associations between
features and hypotheses H True H False
■ All four cells of the 2x2
table are needed for
rational inference

Must know when features Feature ? ?
are present/absent under Present
the hypothesis, versus its
negation
■ However, we don’t Feature ? ?
always think this way…. Absent

2
 Confirmatory Search
■ A common heuristic when seeking out/processing
information: focus on a given hypothesis, and
determine whether there is evidence which confirms it

Note that attempts are not made to find disconfirming
evidence
■ Examples

Watson’s (1960) number guessing experiments
➔
Subjects jumped to conclusions about the latent rule, because
they did not attempt to disconfirm their theories

Snyder and Cantor’s (1979) job suitability studies
➔
Given descriptions with conflicting information, subjects sought
out elements consonant with the job description (even though
this lead them to view the same person as introverted or
extroverted, depending on the frame!)

3
 The Confirmation Bias
■ Confirmation bias: a tendency to seek out and be more
sensitive to evidence for a focal hypothesis, versus A)
evidence against the hypothesis, or B) evidence
relating to competing hypotheses

Can result from the confirmatory search heuristic
■ “When I believe it, I’ll see it”

Confirmatory judgment is not comparative

Produces vulnerability to problem re-framing (focal hypothesis
is advantaged)

Same evidence can be interpreted as supporting competing
theories (which violates a basic principle of rational inference)
■ Incorporates several other well-known biases as
special cases

4
 Example: The Teddy Bear of Doom
■ The Warning: a friend's computer has been infected with a
virus, and yours is likely infected as well!
■ The Instructions: check for the virus file – if it's present, you
must eliminate it at once

Distinctive filename (jdbgmgr.exe) and incriminating teddy bear icon:

■ The Result: you find the file (with the icon); are you infected?
■ The Reality: the “virus” is a hoax

jdbgmgr.exe is a system file, present on most Windows installations

Exploitation of confirmation bias: victims search for confimation of virus
warning, don't think of whether file might have been present all along

■ The Moral: don't forget to look for disconfirming evidence!

5
 Special Case: Pseudodiagnosticity
■ Pseudodiagnosticity: the error of treating the likelihood
of data given a single hypothesis as being able to
differentiate between two alternative hypotheses
Both
parts
p  H 1∣D  p  D∣H 1  p  H 1  needed
=
p  H 2∣D  p  D∣H 2  p  H 2 
■ Special case of the confirmation bias: we seek
information on our focal hypothesis, without regard to
alternatives

Connection with representativeness – pseudodiagnostic data
reveals how representative the data is of the focal hypothesis

6
 Example – Predicting Default
Characteristic Defaulters Non-defaulters
Ratio of Monthly Obligation to Income >0.4 85% (1)
Ratio of Monthly Obligation to Income ≤0.4 15% (1)
Credit Card Debt > $20,000 (2) (3)
Credit Card Debt ≤ $20,000 (2) (3)

"Assume that you must predict whether or not a loan applicant with a
obligation/income ratio of 0.32 and a total credit card debt of $32,000 will
default. Given the resources available to you, you can obtain additional
information to aid your decision from any one of the call pairs marked (1)-
(3) in the table. Which pair will you choose?"
(1) Ratio for non-defaulters [34%]
(2) Debt for defaulters [49%] Pseudodiagnostic
(3) Debt for non-defaulters [17%] options [66%]

7
 Special Case: Self-Serving Bias
■ Just as we tend to look for confirmation of
preconceived ideas in other contexts, we tend to
employ confirmatory search when evaluating ideas
about ourselves

“Self-serving bias”: the tendency to seek out or be attentive to
information which confirms our positive self-image
■ Self-serving bias sometimes seen as distinct
phenomenon, but can be seen as a special case of
confirmation

Note that persons with highly negative self-images seek out
information which confirms their negative views, which is
consistent with this interpretation

Since most people think well of themselves, we see the above
only in rare cases (e.g., the clinically depressed)

8
 Example – Personal Validation
■ Many people evaluate the efficacy of pop psychology,
astrology, and the like by personal validation – how
true it “feels” to them
■ Such validation can be reliably manipulated

Sunderberg (1955) found that college students would endorse
the following as being a tailored personal description:
➔
“You appear to be a cheerful, well-balanced person. You may
have some alteration of happy and unhappy moods, but they are
not extreme now. You have few or no problems with your health.
You are sociable and mix well with others. You are adaptable to
social situations. You tend to be adventurous. Your interests are
wide. You are fairly self-confident and usually think clearly.”

Hyman (1977) found that a mix of 75% positives to 25%
negatives produced the highest acceptance rates

9
 Confirmation Bias and Cold Reading
■ “Cold Reading”: the art of producing an acceptable
character assessment, based on little or no
individuating information

Widely used by mentalists, astrologers, professional psychics,
pop psychologists, etc.
■ Basic tactics

Invoke authority as a “reader”

State numerous claims which are true of most people

Use many positive descriptors

Force the subject to make sense of the claims
■ How does it work?

Once authority has established the reader’s efficacy as the
focal hypothesis, subject begins a confirmatory search; with
enough claims (particularly positive ones), the subject is
bound to find some which he or she regards as “hits”

10
 “Confirmation Marketing”: the Case of SRI’s VALS2
System
■ VALS was a market demographics analysis tool
produced by the Stanford Research Institute in the
1990s

Intended to predict consumer behavior

Based on statistical analysis of large-scale questionnaires
■ VALS was promoted online via a web site

The site allowed you to “try out” the system, get predictions
for yourself based on your responses
■ Intentionally or otherwise, the marketing materials for
VALS illustrate the use of confirmation bias in
marketing

(Note that I make no claims regarding the efficacy of the VALS
system itself; these comments pertain only to SRI’s online
marketing of the system in the mid-1990s.)

11
 Examples of VALS Profiles
■ Based on questionnaire responses, subjects are
assigned to one of several VALS “profiles”

Intended to classify individuals by purchasing habits
■ From the VALS2 online system (SRI, 1996):

“Actualizers”
➔
“Actualizers tend to be successful, sophisticated, active, `take-
charge’ people with high self-esteem and abundant resources.
They are interested in growth and seek to develop, explore, and
express themselves in a variety of ways, sometimes guided by
principle, and sometimes by a desire to have an effect, to make a
change.”

“Fulfilleds”
➔
“Fulfilleds generally have a moderate respect for the status quo
institutions of authority and social decorum, but they are open-
minded to new ideas and social change. …they look for
durability, functionality, and value in the products they buy.”

12
 The Initial Pitch
■ Establish authority

“The motivations and demographic characteristics this
questionnaire asks about are very strong predictors of a
variety of consumer preferences in products, services, and
media.”

Based on “…a $1.5 million development effort and several
large national surveys of consumer opinion…”
■ Place the burden of sense-making on the subject

“The way to think about the data is as if it paints a portrait. …
the details we provide should form an overall portrait that, as a
whole, reflects your individual preferences pretty well.”

“A few aspects of your life will be in the portrait directly…
many others will be in the portrait indirectly…and a few parts
will seem outright strange. Thus, it’s the overall picture that
you should focus on when considering how the data relates to
you.”

13
 Encouraging Confirmatory Search
■ Make many predictions

Two VALS types are assigned, with no rules for integrating
them

For each type, numerous descriptive elements and product
choices are predicted

Many predictions are mundane, and apply to most people
■ Play up the hits, play down the misses

“You may find that some of the details in your VALS type
description…are not like you at all. But don’t judge too
quickly. The details that are literally unlike you make identify a
basic tendency that you express some other way.”

“In fact, chances are that many of the items in your VALS
type’s data tables will not apply directly to your life.”

14
 Dealing with Malcontents
■ Imply that this is only a sample of your abilities

“You may find that some of your favorite activities or interests
do not appear in the data tables. Understand, however, that
we are only listing a few illustrative examples out of a vast
data set.”
■ If we can’t predict you, it’s because you are unusual…

“Alternatively, we may just not have data on your favorite
activities or interests available to us.”
■ …and, if all else fails, use the bandwagon

“Questionnaires such as VALS are one of the primary ways
that businesses come to understand your individual
preferences, needs, and interests.”

15
 Summary
■ Rather than seeking evidence diagnostically, we fall
prey to confirmatory search

We look for evidence which confirms our focal hypothesis,
rather than evidence which compares it to alternatives

Numerous special cases, including pseudodiagnosticity and
self-serving bias
■ Confirmation bias can make us vulnerable to certain
sales tactics

Burden of sense-making placed on the customer

Vast numbers of predictions are made – some will fit

Hits taken as proof of efficacy, misses ignored
■ Next time: anchoring and availability

16
Anchoring and the Availability
Bias
Business Decisions
SOC 138/ECO 148

1
 The Anchoring and Adjustment Heuristic
■ Begin with a reference value, then adjust this
value up or down

Reference value often taken from the immediate
environment

A&A as a form of feedback learning: make an initial
guess, and then revise based on apparent error
■ Heuristic works well when good reference
values are available, and when many
opportunities for adjustment exist
■ Heuristic encounters difficulty when reference
values are misleading, minimal adjustment
possible

Poor feedback regarding the nature of error also a
problem

2
 Anchoring on Question Components

■ When asked a difficult question, the first place

we look for anchors is the question itself

Can a component of the question provide a clue to
the answer?

What about the response options?
■ Examples

Conservatism or base rate neglect (anchor on prior
or likelihood, respectively)

Response scale bias
➔
Subjects tend to answer towards the “middle” of any
response scale
➔
Probability assessments using log-units much lower than
those using standard units

3
 Example – Rapid Calculation
"Without using a calculator or writing it out, take a few seconds
to estimate the sum of the integers from 1000 to 1 [1 to 1000],
i.e. 1000+999+998+996+995+...+1 [1+2+3+4+5+6+...
+1000]"

1% trimmed mean estimate of 1000+...+1: 9,939,597

1% trimmed mean estimate of 1+...+1000: 294,708
True value: 500,500

(Note that the difference in intuitive estimates was an

order of magnitude larger than the size of the sum
itself! Anchoring on the first few numbers can
dramatically alter one's estimates.)

4
 A Sneaky Response Scale Trick
■ Beware any survey which does not provide the
same scale to all participants
■ A fictitious example: “How much would you pay
for SneakerPoofs™?”

$65 $300

$0 $100 $0 $1000

■ Responsible polling/marketing firms will not do

this, but keep your eyes open

Can occasionally happen by accident in complex
questionnaires or multi-wave studies

Major problem where comparing different studies

5
 Anchoring on Contextual Cues
■ A second source of anchors: cues in the
problem environment

Can we find other, related problems and/or their
solutions?

May include previous questions, behavior of other
decision makers, etc.
■ Examples

What is a “reasonable” price for a house? Look to
the market (even if your house isn’t comparable)

“Just mentioning effects”: in surveys, aspects of
previous questions can affect later responses

6
 Robustness of the Anchoring Effect
■ Surely, we wouldn't be affected by anchors
which are obviously wrong...right?
■ Quattrone et al. (1984) find effects for

Average price of a college textbook (anchor: $7128.53)

Number of top 10 Beatles records (anchor: 100,025)

Average temperature in San Francisco (anchor: 558°F)
■ Effect does not go away with absurd anchors

Extent of effect seems to level off, however

7
 Example – Estimating US Income

"Was the median US individual income in the year 2018 greater or

less than $500 [$5,000,000]?"

"What was the median US individual income in the year 2018?"

[$500 Anchor]
1% Trimmed Mean = $37,755; Median = $38,000
[$5,000,000 Anchor]
1% Trimmed Mean = $468,725; Median = $64,320

(Trimmed mean under larger anchor was an order of

magnitude higher – even the median was almost twice as
large in the second case. Note that the real value is
around $32,000 per year!)

8
 Anchoring and Marketing
■ Anchoring and adjustment has a range of
marketing applications – and pitfalls
■ Price-setting

Where prices are subject to negotiation, anchoring one’s
trading partner on an especially high/low price may lead to a
better deal

In addition to price scale tricks, can provide biased pricing
cues from other goods in environment, other potential deals
■ Valuation

For complex goods, valuation is difficult

Your good may be made to appear more favorable by strategic
comparison
■ Two sample pitfalls: self-delusion and failure to
change

Studies which give your product an unfair edge may impress
management today, but set you up for failure tomorrow

You can’t always tweak yesterday’s good for today’s market
9
 Example – Real Estate
■ Northcraft and Neal (1987) – study of list price
effect on agents’ appraisals and recommended
sales prices

Houses initially appraised

Subjects (experienced real estate agents) given
information on homes, including list prices

List prices manipulated up or down

Agents’ appraisals moved (significantly) in the
direction of the manipulation (changes on the order
of $14,000, or 10%)

Agents did not indicate that they had used list
prices

10
 Anchoring in Strategic Contexts
■ It’s not always obvious how we should choose
in a complex game, especially one with
continuous strategy sets
■ A heuristic solution: anchor on a plausible
choice based on a simple model of your
opponent, and then adjust

In repeated play, you may hope to converge to an
optimal strategy

On the other hand, your behavior may be
exploitable

11
 Example – The “Beauty Pageant” Game
■ Oddly-named game works like so:

N players

Each chooses a number in [0,100]

Player whose choice is closest to half the mean
choice wins
■ Subgame perfect Nash equilibrium: everyone
chooses 0
■ Real play:

Subjects tend to “anchor” on the interval, initially
choose numbers around 25 or so

In iterated play, choices converge to 0 (adjustment)

12
 Availability Bias
■ In many cases, we must reason from examples
in memory
■ Not all memories created equal

Some are more plentiful, accessible than others

These memories are said to be more “available”
■ Reasoning biased towards more available
memories
■ Connection with A&A: we anchor on highly
available cases, only later attempting to adjust
for our biases

13
 Example – Vividness and Culpability
■ Famous study by Reyes et al. (1980) on
culpability in a drunk driving case

Manipulation of the evidence: “pallid” accounts
versus “vivid” ones
➔
“On his way out of the door as he was leaving a party, the
defendant staggered against a serving table, knocking a
bowl to the floor.”
➔
“On his way out of the door as he was leaving a party, the
defendant staggered against a serving table, knocking a
bowl of green guacamole dip onto the floor splattering
guacamole on the expensive white shag carpet.”

Side with the vivid accounts judged stronger by
mock jurors, particularly 48 hours later
■ Interpretation: vivid accounts more memorable
and hence more available to later reasoning

14
 Summary
■ When we don’t know what to do, we often start by
“anchoring” on options suggested by the
environment, and then attempting to “adjust” this
initial choice or evaluation

Anchors can come from questions, response options,
others’ behaviors, etc.
■ Anchors provide an example of bias in judgment
towards highly available stimuli

We reason with what we remember, and memory is
biased
■ Next time: actuarial reasoning and clinical
judgment

15
Actuarial Versus Clinical
Judgment
Business Decisions
SOC 138/ECO 148

1
 The Story of Cassandra
■ In Greek myth, Cassandra was
a prophetess who was cursed
such that no one would ever
believe her (though she always
spoke truth)

Recommended against letting the
Wooden Horse into Troy, and we
know how well that worked out
■ Cassandra as metaphor for
today’s topic:

What I will tell is backed up by a
vast number of studies, but has
made little impact on many
decision makers

In part, this is because the
message is not one that many
decision makers want to hear
2
 Actuarial Reasoning and Clinical Judgment
■ Two competing practices for decision making:

“Actuarial reasoning”: originally referred to
accounting/insurance practices, but is now often
used generically for decision making based on
formal (especially statistical) models

“Clinical judgment”: originally referred to expert
judgments by clinicians (doctors/psychologists),
but is now often used generically for judgment
based on intuition or personal expertise
■ The latter is overwhelmingly preferred by
laypersons and professionals alike – but
evidence suggests that the former is nearly
always superior

3
 A Stable Pattern of Findings
■ Meehl, in 1986: “There is no controversy in social
science which shows such a large body of qualitatively
diverse studies coming out so uniformly in the same
direction as this one. When you are pushing 90
investigations predicting everything from the outcome
of football games to the diagnosis of liver disease, and
when you can barely come up with half a dozen studies
showing even a weak tendency in favor of the clinician,
it is time to draw a practical conclusion.”
■ That was in 1986; by 1995, Grove and Meehl had found
over 150 studies showing the same result

4
 Formal Decision Models
■ OK, so what is an actuarial model?
■ Basic idea:

Start with a set of outcomes and
treatments/covariates

Construct a mathematical framework linking
treatments/covariates to outcomes

Estimate model parameters based on data

Use estimated model to make predictions for new
cases

Make decisions based on predicted outcomes
■ Key elements: systematic, data-driven, explicit
about assumptions, precise in making
predictions, “honest” (in the sense of error
assessment)
5
 Linear Models
■ A special (but important) case of actuarial
reasoning: models of the form
Y = 0 1 X 1  2 X 2⋯ n X n

Standard regression model is an example of this type
(with some statistical add-ons)

Extremely simple starting point (more complex models
are available)
■ Performance: when placed against expert
predictions, linear models are almost always more
accurate

This is even true when the expert has access to the
linear model him/herself – most won’t use them

6
 Example: Real Estate Valuation
Hypothetical model for predicting selling prices in a given
area:
Model Terms: y≡ Building Price (in $100,000) Coefficients:  0 =0.5
x 1 ≡ Building Age (in Years)  1=−0.02
x 2≡ Building Size (in Square Feet)  2 =0.001
x 3≡ Lot Size (in Acres)  3=2

Predicted price for a 12 year old, 2000sq ft house on a 0.25

acre lot:
y= 0 1 x 12 x 2 3 x 3
=0.5−0.02 x 10.001 x 22 x 3
=0.5−0.02120.001200020.25
=2.76 (or, $276,000)

7
 Linear Models Based on Expert Judgment

■ Another tweak: fit a linear model to a series of

expert judgments, and then use that model
against the experts themselves in a new round
of predictions

Again, the model wins
■ How can this be?

Expert judgments are usually close to linear
(despite what we tend to think) on average, but
highly variable – linear models pick up on the
general tendencies but throw out the noise

When models are based on multiple experts,
integration of expertise results (many heads are
better than one!)

8
 Linear Models With Random Weights
■ The final insult: Dawes and Corrigan (1974)
reanalyzed previous studies with linear models
using weights with correct sign but random
magnitude

The models still won!
■ The critical implication: it’s not the weights which
make the difference!

Key to predictive superiority is threefold: ability to
integrate multiple features, consistency in estimation,
and correct direction of effect (positive or negative)

This is encouraging: it means that even if we choose
weights intuitively, a linear model we construct will
probably do better than our own raw judgment (so long
as we get the directions of effect right)

9
 Why Do Linear Models Work So Well?
■ The world isn’t always We often imagine the world like this…
as nonlinear as we think

And we’re not good at
dealing with nonlinearity,
anyway
■ Consistency is more
important than flexibility
■ Even where the world is …but this is more common.
complicated, the linear
approximation often
does pretty well

Main effects tend to
dominate, curvature often
modest relative to noise

10
 Another Take on the Relative Success of Linear
Models
■ In many cases, the success of linear models
versus expert judgment lies not in the efficacy of
the model, but in the relative failure of the experts

The Einhorn survival study – linear model predictions
weak but significant, no significance for clinician’s
assessments
■ Experts try to “get fancy” in accounting for a wide
range of possibilities, but often just wind up being
inconsistent

Linear model is unexciting and approximate, but it does
the same thing each time

Situational errors and variation: even mere averaging of
experts' own judgments on the same item over time
produces better results!
■ Sometimes, we just don’t know how to predict the
future – and the linear models help make the most
of what we have
11
 The Inevitable Objections
■ But linear models are obviously wrong in a lot of cases!

This is true – but the problem is, so are we; the models are just
less wrong, on average, than our trained intuition

As Einhorn put it, we must “accept error to make less error”
■ But I can think of a special case when the expert’s
intuition would obviously beat the model!

Probably so…but for every such “hit,” there are many, many
“misses” in which the expert’s intuition was incorrect
■ You must have used the wrong experts!

These studies have been done with top experts in their fields
(medicine, business, science, etc.)

If the other experts couldn’t do it, what makes you think that
your favorite will somehow succeed?
■ But how can you say that you trust a model over your
own mind?

We trust tools to outperform us on a regular basis in other
domains (e.g., I cannot outrun my car); why should this be any
different?
12
 Do It Yourself: Steps to Building a Simple Decision
Model
■ We’ve focused on prediction, but the Dawes and
Corrigan result highlights the value of these models
for choice as well
■ Here’s a crude, but effective, strategy:

Identify the attributes of each choice

Decide whether each is positive or negative, on average

Assign a subjective weight to each attribute – when in
doubt, keep weights similar in size

Calculate the linear combination of attributes given the
weights

Choose the option with the highest total value
■ This model is not complex, but it will help you
integrate multiple features consistently

Try it – you’ll be surprised at how it can differ from your
intuition!

13
 Using the Model In Practice
■ When building the model, try to think of each feature
comparatively

How much more is this house worth to me with three
rooms, rather than two?

Come up with “exchange rates”: how many square feet
does it take to make up for not having a formal dining
room?
■ After computing the initial preference ordering, try
robustness testing

Don’t be afraid to “tweak” the parameters – often, you’ll
find that the results don’t change much

Where some factors prove decisive, you can often focus
your attention on those
■ Where the results seem counterintuitive, try to
understand them – you may discover an
inconsistency in your thinking
14
 Summary
■ When making decisions in the real world, we must
often choose between our intuition and a formal
procedure
■ Where the two have been compared, the formal
procedures almost always win

The superiority of actuarial reasoning depends more on
consistency and ability to integrate information than on
modeling details

Simple linear models usually do quite well
■ You can use this in your own life

It’s easy to get started with linear models – Excel is your
friend

Use formal procedures to make key decisions – you may
become a Cassandra, but you are the one who has to live
with the outcomes!

15
 Debiasing

Business Decisions
SOC 138/ECO 148

1
Continuing the Theme...

2
 So, We’re Not So Smart After All
■ As we have seen throughout the past several
lectures, real people deviate from the perfectly
rational ideal
■ We use heuristics instead of optimal
procedures
■ Our basic cognitive processes of association,
learning, and memory are imperfect
■ We suffer from a variety of fallacies and biases

Confirmation bias, Hindsight bias, Self-serving bias,
Availability bias, Gambler’s fallacy, Base rate
neglect, Pseudodiagnosticity, Conservative
probability judgments, belief in a “Law of Small
Numbers,” etc., etc., etc.

3
 And, It's Worse Than It Seems
■ In addition to being subject to various biases, we do
not adequately account for error in decision making
■ A common finding: Overconfidence

When asked to predict accuracy of own judgments, both
experts and laypersons tend to systematically overestimate
accuracy

Perception of accuracy increases with case familiarity, total
amount of information reviewed, and number of judgments
made – real accuracy increases less quickly (and often
levels off)

Calibration can be very poor: Christiansen-Szalanski and
Bushyhead (1981) found physicians who were 90%
confident in pneumonia diagnoses were only about 15%
accurate!
■ Overconfidence + Error = Poor decision making
4
 …But, Is That the Whole Story?
■ Perhaps we can think (more)
rationally, even if we often don’t

Some contexts (familiar ones,
perhaps?) may be easier than others

Rational behavior might be learned
through practice or experience
■ Thepicture painted by the previous
research could be overly pessimistic
– but is it?

5
 The Scientific Question: Robustness
■ Mostdeviations observed in
controlled, laboratory studies

Strengths: Replicated, exogenous
factors can be ruled out, mechanisms
examined

Weaknesses: May not generalize, if
laboratory tasks are too different from
real ones
■ The Big Question: Are these findings
robust, or do they go away if we
change the setting and/or task
slightly?
6
 The Practical Question: Improved Decision Making

■ The standard findings may be fair

characterizations of how most people make
decisions, but it does not follow that we could
not learn to do better

After all, most people can’t do calculus, either, but it
can be mastered with a year or two of training!
■ The Big Question: Are there things we can do
to minimize or even eliminate bias in the field?

7
 Debiasing
■ The idea: once a bias has been replicated
experimentally, what does it take to get rid of it
(i.e., to “debias” the subjects)?
■ Two kinds of approaches:

Modify the person: Provide training, education, etc.
to reduce mistakes
➔
E.g. education about biases, feedback training,
training in improved decision procedures

Modify the environment: Alter the decision problem
of situation in ways that reduce opportunities for
error
➔
E.g. stronger incentives, clearer instructions,
warnings about bias
■ The question, then: do any of these ever work?
8
 Minimizing Hindsight
■ The Good News:

Requiring subjects to systematically address how
they would have explained outcomes which did not
occur reduces (but does not eliminate) hindsight
bias
■ The Bad News:

Incentives, education, problem clarification,
warnings, etc. does not seem to work
■ Overall:

Reducing the bias requires a thought process
(counterfactual reasoning) which is uncommon in
everyday practice; training in this might be a useful
intervention, but the bias itself is likely to prove
robust

9
 Disconfirming Confirmation
■ The Good News:

Cautioning subjects that stereotype-confirming
questions might be taken as evidence of prejudice has
eliminated stereotype-related confirmation bias in at
least one study
■ The Bad News:

Education about the confirmation bias, general and
specific instructions to seek disconfirmation, etc. do not
seem to work
■ Overall:

Situations in which we are in danger of appearing biased
may help us avoid bias; encouraging decision makers to
think of themselves as being “watched” may prove an
effective intervention in some cases

10
 Deflating Overconfidence
■ The Good News:

Systematic calibration training does seem to greatly
reduce overconfidence within the domain being trained
■ The Bad News:

Other manipulations (including incentives, general
education, explanation of the effect, etc.) do not seem to
work, or are equivocal
■ Overall:

Those in fields with extensive feedback (weather
forecasters, for instance) may become very well-
calibrated for the problems on which they have such
feedback, and training may help others attain this goal
as well; unfortunately, there is little evidence that
anything else works

11
 Overall
■ Clearly, some biases can be reduced in some
cases by training, improved problem clarity,
and stronger incentives

Training subjects in the use of normative strategies
and providing feedback may enable near-optimal
behavior for specific, pre-specified problem
domains
■ In general, there is little evidence that most
biases can be eliminated generically

Some reflect “deep” cognitive mechanisms, and are
unlikely to be easily overcome

Learning tends to be domain-specific, and does not
easily generalize to other problem areas

12
 Beyond Debiasing
■ Another approach: use formal procedures to
supplement or replace fallible judgment/choice
behavior

Systematic policies to detect/combat poor decision making
practices, a la Armstrong's forecasting audit, or Soll et al.’s
checklists

Formal mathematical/statistical models to implement precise
rules of judgment/choice, a la actuarial judgment
■ Pros:

Less subject to human limitations (your firm's computers
constantly increase in speed and storage capacity – do you?)

Explicit assumptions, subject to external review
■ Cons:

Often requires technical expertise and advanced preparation

A “hard sell” in many environments
■ Moving beyond intuition and experience is difficult,
but it can be worth the price 13