The Lady Tasting Coffee:

A Case Study in Experimental Design

The History of Experimentation
Experimentation characterizes modern science.
Galileo (1564-1642) reportedly dropped balls of
various masses from the Leaning Tower of Pisa.
Assuming the story of Galileo’s Pisa experiment
is true: How many balls did he drop? How
many times did he repeat the comparison?
What were his independent and dependent
variables? How did he measure the time to
impact? We don’t know the answers to these
questions…
Take Home Message: Experimental design
was haphazard prior to the 1920’s.
 Ronald Aylmer Fisher
• Considered by some scientists to be the father of
modern statistics .
• Poor eyesight; did a lot of math in his head without
paper or pencil.
• In 1919, he began working as a statistician at the
Rothamsted Agricultural Experiment Station in the
United Kingdom.
• Published many papers and wrote several books on
experimental design and evolution.
 Ronald Aylmer Fisher
1890-1962

http://www-groups.dcs.st-and.ac.uk/~history/Mathematicians/Fisher.html
At Rothamsted, Fisher recognized problems with
some of the agricultural experiments
Same field, same treatment, but
plant performance is uneven...

Thin
Thick
Growth
Fisher’s Solution:
Growth
Replicate and
randomize to spread
variation evenly
among treatments.

Source of Picture: http://www.ipm.iastate.edu/ipm/icm/files/images/uneven-corn-VS6.jpg

 Lessons Learned at Rothamsted
Experiments at Rothamsted prior to Fisher
generally involved two fields (containing
hundreds of plants), each receiving a treatment.
Example: two levels of nitrogen (N) fertilizer

Field with Field with

High N Low N
Problem: So much variability exists within a field
itself that it is difficult or impossible to tease out
the effect the treatment.
 Fisher’s Solution at Rothamsted
– Old Problematic Design: One large field receiving high
nitrogen (N), one large field receiving low nitrogen (N).
(Today this design is sometimes called “pseudoreplication” if
the experimenter attempts to say that the sample size is the
number of plants.)

– New Improved Design: Many small plots, randomly

receiving high N or low N; plots can also be blocked to
help tease out the variation due to location and local
conditions.
Hurlbert, S. H. (1984). Pseudoreplication and the design of ecological field experiments. Ecological monographs 54(2): 187-211.
Examples of Correct & Incorrect Ways
to Randomize Treatments
Correct Ways: Incorrect Ways:
• Use a random • Haphazardly decide which experimental
number table. units should receive which treatments.
(Problem: too tempting for experimenter to bias.)
• Pick treatments
from a hat. • Use a net to grab the goldfish in an
ecology study. (Problem: might pick just the
• Flip a coin. easiest to catch, sickly animals.)
• Alternate treatments (every other one).
(Problem: that’s systematic, not random; who knows
what other factors vary in the same systematic way.)
• Assign people to drug study on the basis
of their last name. (Problem: could be related to
a person’s ancestry.)
 Fisher, Randomization, Replication & Blocking
• No replication (or pseudoreplication) (Rothamsted, pre-Fisher):
Field with
Field with
Low N
High N

• Replicated with complete randomization:

Treatments are applied to plots
Field broken
rather than to an entire field;
up into
this improves replication &
smaller plots
interspersion of treatments.

• Replicated, randomized and blocked design: Dashed rectangle

is a block
Field broken
up into Plots are blocked by
smaller plots location or other
& plots are condition; treatments
grouped. are applied randomly to
plots within blocks.
Another of Fisher’s Contributions to Statistics:
The Analysis of Variance (ANOVA)
Allows scientists to mathematically partition variation
among different sources (treatments, blocks, plots, for
example).
Some of Fisher’s contributions to the field of statistics grew out of
his experience with spatial agricultural experiments at
Rothamsted.
From: Sokal, Robert R., & F.James Rohlf, Biometry: The Principles and
Practice of Statistics in Biological Research, San Francisco: W.H. Freeman.
 Why do these two
plants differ in
growth? Is it
because of block,
treatment, or
extraneous
variation within
plots?

At Rothamsted, Fisher saw firsthand that the purpose of good experimental design
is not to eliminate variation entirely, but rather to try to ensure that extraneous
variation is spread evenly among treatments. In the case of ANOVA, the
experimental design can enable the variation to be partitioned mathematically
during analysis.

Variation in growth of plants can be partitioned into different sources of variation:

1. Variation in soil moisture, texture, etc. within a plot.
2. Variation between treatments (high N and low N).
3. Variation in soil moisture, texture, sunlight, etc., among blocks.
 The Design of Experiments (1935)
One of the first chapters of this textbook written by Fisher
is the essay, “Mathematics of a Lady Tasting Tea.”
 A lady tasting tea
Can she tell whether the milk was added before or after the tea?

Afternoon tea during study abroad experience by University of Pittsburgh at Bradford students at the
University of Sussex in Brighton, Great Britain. Copyright © Janelle Elmquist. Used with permission.
 So, you think statistics is boring . . .
Statisticians and the history of
statistics are far from boring.
Other interesting trivia on Fisher:
-Charming but had a terrible temper
(and a big ego)
-Smoked a pipe & argued
professionally in the 1950’s that
smoking did not cause cancer
-Supported eugenics
Picture taken from:
Parascandola, M. (2004). "Two approaches to etiology:
the debate over smoking and lung cancer in the
1950s." Endeavour 28(2): 81-86.
 Take Home Messages
• The 1920’s was a rich time for the development of concepts
of modern experimental design.
• Fisher was one of a number of statisticians who greatly
affected the development of modern statistics.
• Fisher’s experience at Rothamsted Agricultural Experiment
Station influenced his vision of experimental design and
helped him develop the concept of ANOVA .
• Fisher’s essay on a lady tasting tea eloquently outlines some
important issues in experimental design.
 To learn more, read the biographies of
statisticians as you learn their techniques
The Student’s t-test
Student is the pseudonym of William Sealy Gosset, a
contemporary of Fisher who worked for Guiness, the Irish
brewery.
Other techniques
Many statistical techniques are named after interesting
historical people:
Bayes, Bernoulli, Cochran, Cox, Kolmogorov, Mann, Pearson, Smirnov, Tukey, Whitney, Wilcoxon
to name just a few

You are more likely to remember specific statistical techniques if

you know about the people who created them. Don’t be
afraid to look at the original works published by these famous
statisticians.
Examples of statistical techniques or tests named after people important in the
history of statistics. Names below include: Cochran, Cox, Friedman, Gosset,
Kolmogorov, Kruskal, Mann, Smirnov, Spearman, Wallace, Whitney, & Wilcoxon.
 Recommended Reading

• Salsburg, D. 2002. The Lady Tasting Tea: How Statistics Revolutionized Science in
the Twentieth Century. Henry Holt and Company, NY.
• Stigler, S. M. 1999. Statistics on the Table: The History of Statistical Concepts and
Methods. Harvard University Press, Cambridge, MA.