Professional Documents
Culture Documents
CH04 Causation, Correlation & Probability Problems 2
CH04 Causation, Correlation & Probability Problems 2
Behavioral Finance
Special Topics in
Finance
Behavioral Finance
• Invisible correlation
• The nationwide house price collapse leading to the credit crisis
7
Probability Solutions
Q1: Two cards are picked at random from a deck (without
replacement). What is the probability that the first card is red and the
second is black?
Half of the cards in a deck are red, so the probability that the first
card is red is: ½
Now we’ve removed a (red) card, so there are 51 cards left, of
which 26 are black. So the probability that the second card is
black is: 26/51
Hence the probability that the first card is red and the second is
black is:
½ * 26/51 = 0.255 (i.e., slightly higher than ¼)
Q2: Suppose you toss a coin 3 times, and each time it comes up
heads. What is the probability that the 4th toss of the same coin
will also be heads? Coin tosses are independent events. Unless
the coin is mis-weighted, the probability that the 4th toss is heads
8
is still ½.
The Juror’s Fallacy— Base Rates and False Positives
You are a member of a jury. A taxi driver is accused of having run down a
pedestrian on a stormy night and having fled the scene of the accident. The
prosecutor, in asking for a conviction, bases his whole case on a single witness, a
lady who saw the accident from her window a little way away. The witness
testifies that she saw the pedestrian struck by a blue taxi and then saw that taxi
drive away from the scene. The accused works for a taxi company whose taxis are
all blue.
1. There are only two taxi companies in this town. The whole fleet of one
company is green; the other has only blue cabs. Eighty-five percent (85%) of all
taxis on the road that night were green, and only fifteen percent were blue.
2. The single witness has undergone a number of vision tests in conditions similar
to those of the night of the accident. She has been shown to be able to identify
the two colors correctly about 80% of the time; i.e., out of all of the blue &
green cabs she saw during the vision test, she got the color right 4 out of 5
times.
Let’s try to answer this question. We’ll start by assuming that there are 1,000 cabs in this
town. From the
information on the previous page, we know that 850 of the cabs are green, and 150 are blue.
There are two possible scenarios, given that the witness identified the cab as blue:
We know that the witness only gets the cab color right 80% of the time.
- She will incorrectly report 170 of the green cabs as blue (20% x 850). This is called a “false
positive.”
- She will correctly report 120 of the blue cabs as blue (80% x 150).
Hence, of the 1,000 cabs on the road that night, she will report 290 as blue (170 + 120). But
only 120 of these cabs are truly blue.
So she will correctly report a blue cab only 41% of the time (120/290 = 0.41).
1
1
The Juror’s Fallacy
Base Rates and False Positives
Total number of
cabs
Base 1,000
rate 15% 85%
With 1,000 cabs on the road, only 150 of them are blue, and the witness will correctly identify the
color of either cab only 80% of the time.
The witness’s 20% false positive rate, along with the fact that far more of the cabs were green
than blue that night, results in a high probability that the cab was green, but that she misreported 8
it as blue.
The Juror’s Fallacy— Base Rates and False Positives
There are two significant sources of confusion here:
• Failing to take into account the 20% false positive rate. Since
the witness incorrectly identifies 20% of the green cabs as
blue, and since most of the cabs on the road that night were
green, it is more likely that she saw a green cab and thought it
was blue, than that she correctly identified a blue cab.
Any time you have a low base rate, and a non-trivial false
positive rate, the potential for error becomes surprisingly high. 13
Correlation / Causation error
▪ Young investors who are doing well and making lots of
money tend to ignore the base rate of profit & loss success
(the S&P500, perhaps, if they are investing in US equities)
▪ If all equity prices are going up, then you would have to
work hard not to make money.
5 BBB
• In 2002 – 2006, the proportion of mortgage loans made to “subprime” borrowers (no job and/or
no savings and/or no credit history) increased from around 5% to 35% of all US mortgages
• For the first time in history, more than 1 trillion dollars in subprime mortgages were packaged
into MBS and sold to investors
• Investors assumed that subprime mortgage default rates would be similar to prime
default rates. In fact, they were about 20 times higher
• Investors assumed that house prices would continue to rise, so that even if borrowers
defaulted, each loan would be paid off in full by selling the house(Exit opportunity). House
prices across the country started to fall in late 2005
• Investors believed that, even if house prices declined in some parts of the US, this would not
be a country-wide phenomenon, so geographically diversified pools of mortgages would be
safer. By 2009, house prices across the US had declined by 50% on average, with at least some
degree of decline in every US state.
• Investors significantly underestimated correlations between (1) house price decreases across
the US, and (2) subprime mortgage default rates across the US
We have 4 million innocent people in the Washington area (and one guilty
sniper). 10 of them (including the guilty one) own all three of the items
mentioned above. So:
(a) P( an innocent man would own all these items) is 9 / 4,000,000 (0.0002%)
(b) P ( a man who owned all these items would be innocent) is 9 / 10 (90%)
P(O/I) P(I/O)
• Put simply:
Most people who are admitted to Harvard are brilliant and most
astronauts live in the USA.
P(Brilliance|Admission)>P(Admission/ Brilliance)
P(USA|Astronaut)>P(Astronaut /USA)
However, this does not imply that if you are brilliant you will be
admitted to Harvard or if you live in the USA you are an Astronaut.
When this occurs you are:
falsely assuming that
P(a|b)=P(b|a) While P(a|b) and P(b|a) are unrelated.
Please take a look at the information on
the following slide, which was part of
an education campaign, a few years
ago, from what is now called
Partnership for Drug-Free Kids.
Ask yourself why we have included this
public service announcement in the
course materials. Once you’ve
answered that question, please move
on to the subsequent slide.
17
MARIJUANAANDYOURTEEN’S MENTALHEALTH
Depression. Suicidal Thoughts. Schizophrenia.
If you have outdated perceptions about marijuana, you might be putting your teen at risk. New
research is giving us better insight into the serious consequences of teen marijuana use,
especially how it impacts mental health.
Did you know that young people who use marijuana weekly have double the risk of
depression later in life?1 And that teens aged 12 to 17 who smoke marijuana weekly are
three times more likely than non-users to have suicidal thoughts?2 And if that’s not bad
enough, marijuana use in some teens has been linked to increased risk for schizophrenia in
later years.3 Today’s teens are smoking a more potent drug4 and starting use at increasingly
younger ages during crucial brain development years.5 Still think marijuana’s no big deal?
Remember, you are the most important influence in your teen’s life when it comes to drugs,6 so
tell your teen the facts about marijuana. Teens who learn about the risks from their parents are
less likely to smoke marijuana or use other drugs than teens who don’t.
Let your teens know you don’t want them using marijuana. Their mental health may depend on it.
Signed,
PARE NTS | THE ANTI - DRUG | 1 – 8 0 0 – 7 8 8 – 2 8 0 0
www.theantidrug.com
More Confusion of the Inverse
WEIGHTWATCHERS– DOYOUKNOWTHERISKS?
Obesity. Diabetes. Heart Disease.
If you haveoutdated perceptions about attending WeightWatchers, youmight beputting yourself at risk. Newresearch is
giving us better insight into the serious consequencesof attending WeightWatchers, andespecially howit impacts your
physical health.
Did you know that people whoattend Weight Watchers weekly have double the risk of obesity?
Andif that’s not badenough, attending Weight Watchershas beenlinked to diabetes, heart disorders, andstrokes in later
years.
Today’spopulation has moreaccess to WeightWatchersthan ever before, as they continue to opennewoffices domestically
and worldwide.
Still think Weight Watchers is no big deal?
27
Probabilities
• Discuss a model that predicts how we tent to
distort event probabilities: o over- weight low
probabilities, and under-weight high ones.
28
Subjective Probability
Imagine that a is forcing you to play Russian roulette.
However, you are allowed to purchase one bullet from the
loaded gun (which has 6 chambers).
• How much would you be willing to pay to reduce the
number of bullets in the gun from four to three?
• How much would you pay to reduce the number of
bullets from one to zero?
• What about if you are told that the gun is currently fully
loaded? How much would you be willing to pay to remove
one bullet this time?
29
Subjective Probability & the Certainty Effect
Imagine that you are forced to play Russian roulette. However, you are allowed to purchase one
bullet from the loaded gun (which has 6 chambers).
• How much would you be willing to pay to reduce the number of bullets in the gun from four to
three?
• How much would you pay to reduce the number of bullets from one to zero?
• What about if you are told that the gun is currently fully loaded? How much would you be
willing to pay to remove one bullet this time?
When offered this thought experiment, respondents are typically willing to pay
significantly more to reduce the number of bullets from one to zero – thus guaranteeing
their survival – than to reduce their survival probability from two-thirds to one-half.
Even though the objective probability reduction is the same 1/6th in both cases, the
move from probable to certain survival is more emotionally appealing.
Similarly, individuals tend (not surprisingly) to indicate higher willingness to pay for the
possibility of survival by removing one bullet from a fully loaded gun, than for the
equivalent reduction in probability in the middle of the range.
30
Probability Weighting Function
subjective response On this graph, “true” probabilities are on the
weights horizontal axis, while our “subjective weight”
100% interpretation is reflected in the dark blue
Actual curve, with values on the vertical axis.
A* probabilities • Look at the actual change in probability
from 100% to 83% (marked A on the
horizontal). This probability decrease of
1/6 (17%) takes us from certain death to
merely probable with the removal of one
B* bullet from the fully loaded gun. Our
subjective response ( A* on the vertical
axis) to this change is noticeably larger
Subjective than the objective probability change. A
Probability
“Weights”
• Our emotional response to the change
from certainty to probability (A) is also
Probabilities
significantly greater than our response to
0% the change from 50% to 33% probability
0% 33% 50% 83% 100% (distance B* on the vertical), even
though it has the same 17% reduction B
B A in true probability terms.
-17% -17%
31
Subjective Probability & the Certainty Effect
weights YES!
100% Further examination of the graph
helps us to see, more broadly, that
we tend to be more sensitive to
MAYBE… probability changes that take us
from certainty to probability, than
we are to probability changes in the
middle of the range.
50% Small
child’s At the extreme, imagine the small
Subjective child’s weighting function: when
Probability asking for a treat (an icecream; an
Weights afternoon at the park) she
understands the responses “yes”
Probabilities
and “no”, corresponding to 100%
0% (certainly) and 0% (certainly not).
0% 25% 50% 75% 100% All other probabilities are viewed
generically as “maybe.”
NO!
32
Probability Weighting:
Glossary of characteristics
Voluntariness — people are less anxious about risks that they voluntarily
expose themselves to than those that they are required to engage in
• Smokers…
34
Probability Weighting – Insurance Example
Probabilistic Insurance
• People would rather eliminate risk than reduce it, even if the probability of
a catastrophe is diminished by an equal amount in both cases
• From a purely statistical perspective, if you are willing to pay $100 for annual
flood insurance on all days, you should be willing to pay about $43 (= 3/7 x
100) for the “flood insurance only on MWF” option.
• Most people, however, will pay much less for the partial insurance, since it
includes a level of uncertainty that people are not willing to accept.
Another way of saying this: we are willing to pay a premium to move from
uncertainty to certainty, relative to a simple change in probabilities.
36
Relative Probabilities
As Adam set out on his habitual morning walk across the park, he pondered the troubling
information that he read in the morning’s newspaper: that “over-the-counter painkillers such as
ibuprofen can double the risk of a heart attack.” The article noted that millions “..depend on such
drugs to relieve the symptoms of arthritis, headaches, and other common ailments…[but] now
sufferers face the dilemma of whether to continue taking some of the most commonly-used
painkillers after they were found to carry similar risks to other drugs which have already been
withdrawn.”
Numbers of
Relative probability Heart attacks Absolute probability
(per 1,000
heart attacks people per year)
6 1,000
100% relative
risk increase
3
500
0.6% absolute
risk increase
0
People not People 0 People not People
taking taking taking taking
ibuprofen ibuprofen ibuprofen ibuprofen
12
Relative vs Absolute Probabilities: Answer the following
Why do newspapers report the relative risk change (100%), rather than the absolute
risk change (0.3%), in stories of this type?
Newspapers are in the business of selling dramatic stories. “Risk doubled” will
attract more eyes and sell more papers than “risk increased from 3 in 1,000 to 6 in
1,000”
With so many different risks that we face on a daily basis, why is Adam suddenly so
concerned about heart attacks, just because he read about them in the paper? He
has known for many years that heart attacks run in his family…
While we cannot spend all day thinking of every possible risk that we face, we are
susceptible to risks that are suddenly made especially salient to us – for example
because we read about them in the paper and there is an element of the story
that we feel applies to us personally
In Adam’s case, the “base rate” should be slightly different than 0.3%. Why?
Hugh’s family is predisposed to heart attacks. His base rate should therefore be
the probability of getting a heart attack in a given year based on his family
history, which may be higher than the risk within the overall population*
14
Adam’s age, relative to the overall population, is also a factor
The Monte Hall Problem (or “Where is the goat?”)
You are on a game show, and you are given the choice of three doors. Behind
one door is a car; behind the others, goats. You pick a door, say #1, and the
host (who knows what is behind each door) opens another door, say #3, to
show you a goat. He then says to you: “Do you want to pick door #2?”
Should you switch?
15
The Monte Hall Problem Discussion
There are lots of ways to explain this one, and many hours and column inches have been wasted on
arguments over the correct answer. The answer is YES, you should switch. Here’s why:
• In your initial selection, suppose you • In your initial selection, suppose you picked
picked a door with a goat behind it. the door with the car behind it. The
The probability of this scenario is 67%, probability of this scenario is 33%, since
since 2 out of 3 doors have goats only 1 of the 3 doors has a car behind it,
behind them, and your door selection and your door selection is completely
is completely random. random
• The host then opens the other door • The host then opens one of the other two
that has a goat behind it doors, to show a goat.
• If you now switch to the remaining • If you now switch to the remaining door,
door, you will get the car. you will get the other goat.
• The probability of this scenario is 67%, • The probability of this scenario is 33%,
since two out of the three doors have since only one of the three doors had a car
goats behind them behind it
In summary: if your initial door choice was a goat, you should switch;
but if your initial door choice was the car, you should not.
Your initial door choice was random; therefore, 2/3 of the time you would have picked a goat.
So your probability of winning the car is 2/3 if you switch, and only 1/3 if you don’t. 16
Assuming you want the car, you should switch.
What have we learned?
In this section, we have reviewed a number of common traps that arise from correlation and
causation errors:
46
Section Summary
• Why we are more likely to take risks in scenarios that involve our
areas of interest or expertise – and more likely to underweight
those risks under such circumstances
• How very subjective we are about probabilities, and how this affects
our decision- making in many important areas such as health, and
wealth.
47
A Final Thought
48