Bootstrap Methods and Their Applications.

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/37434447
Bootstrap Methods and Their Application
Article in Technometrics · January 1997

DOI: 10.2307/1271471 · Source: OAI
CITATIONS READS
4,359 21,218
2 authors, including:
D. V. Hinkley
University of California, Santa Barbara
120 PUBLICATIONS 13,450 CITATIONS
SEE PROFILE
All content following this page was uploaded by D. V. Hinkley on 26 September 2015.
The user has requested enhancement of the downloaded file.

Bootstrap Methods and their
Application
Anthony Davison
c
2006
A short course based on the book
‘Bootstrap Methods and their Application’,
by A. C. Davison and D. V. Hinkley
c
Cambridge University Press, 1997
Anthony Davison: Bootstrap Methods and their Application, 1

Summary
◮ Bootstrap: simulation methods for frequentist inference.

◮ Useful when
• standard assumptions invalid (n small, data not normal,
. . .);
• standard problem has non-standard twist;
• complex problem has no (reliable) theory;
• or (almost) anywhere else.
◮ Aim to describe
• basic ideas;
• confidence intervals;
• tests;
• some approaches for regression

Motivation
Motivation
Basic notions
Confidence intervals
Several samples
Variance estimation
Tests
Regression

Motivation
AIDS data
◮ UK AIDS diagnoses 1988–1992.
◮ Reporting delay up to several years!
◮ Problem: predict state of epidemic at end 1992, with
realistic statement of uncertainty.
◮ Simple model: number of reports in row j and column k
Poisson, mean
µjk = exp(αj + βk ).
◮ Unreported diagnoses in period j Poisson, mean
X X
µjk = exp(αj ) exp(βk ).
k unobs k unobs
◮ Estimate total unreported diagnoses from
period j by replacing αj and βk by MLEs.
• How reliable are these predictions?
• How sensible is the Poisson model?
Motivation
Diagnosis Reporting-delay interval (quarters): Total

period reports
to end
Year Quarter 0† 1 2 3 4 5 6 ··· ≥14 of 1992
1988 1 31 80 16 9 3 2 8 ··· 6 174
2 26 99 27 9 8 11 3 ··· 3 211
3 31 95 35 13 18 4 6 ··· 3 224
4 36 77 20 26 11 3 8 ··· 2 205
1989 1 32 92 32 10 12 19 12 ··· 2 224
2 15 92 14 27 22 21 12 ··· 1 219
3 34 104 29 31 18 8 6 ··· 253
4 38 101 34 18 9 15 6 ··· 233
1990 1 31 124 47 24 11 15 8 ··· 281
2 32 132 36 10 9 7 6 ··· 245
3 49 107 51 17 15 8 9 ··· 260
4 44 153 41 16 11 6 5 ··· 285
1991 1 41 137 29 33 7 11 6 ··· 271
2 56 124 39 14 12 7 10 263
3 53 175 35 17 13 11 306
4 63 135 24 23 12 258
1992 1 71 161 48 25 310
2 95 178 39 318
3 76 181 273
4 67 133

Motivation
AIDS data
◮ Data (+), fits of simple model (solid), complex model
(dots)
◮ Variance formulae could be derived — painful! but useful?
◮ Effects of overdispersion, complex model, . . .?
500
400
+ ++
300
Diagnoses
+ ++ +
+ ++ + +
+ + +
+ + +
200
++
+
++ +
+++
100
+
++
+
++ +
++++
0
1984 1986 1988 1990 1992

Time

Motivation
Goal
Find reliable assessment of uncertainty when

◮ estimator complex
◮ data complex
◮ sample size small
◮ model non-standard

Basic notions
Motivation
Basic notions
Several samples
Variance estimation
Tests
Regression

Basic notions
Handedness data
Table: Data from a study of handedness; hand is an integer measure

of handedness, and dnan a genetic measure. Data due to Dr Gordon
Claridge, University of Oxford.
dnan hand dnan hand dnan hand dnan hand

1 13 1 11 28 1 21 29 2 31 31 1
2 18 1 12 28 2 22 29 1 32 31 2
3 20 3 13 28 1 23 29 1 33 33 6
4 21 1 14 28 4 24 30 1 34 33 1
5 21 1 15 28 1 25 30 1 35 34 1
6 24 1 16 28 1 26 30 2 36 41 4
7 24 1 17 29 1 27 30 1 37 44 8
8 27 1 18 29 1 28 31 1
9 28 1 19 29 1 29 31 1
10 28 2 20 29 2 30 31 1

Basic notions
Handedness data
Figure: Scatter plot of handedness data. The numbers show the mul-
tiplicities of the observations.
1
8
7
1
6
5
hand
1 1
4
1
3
2211
2
1 1 2 2 15534 11
1
15 20 25 30 35 40 45
dnan

Basic notions
Handedness data
◮ Is there dependence between dnan and hand for these

n = 37 individuals?
◮ Sample product-moment correlation coefficient is θb = 0.509.
◮ Standard confidence interval (based on bivariate normal
population) gives 95% CI (0.221, 0.715).
◮ Data not bivariate normal!
◮ What is the status of the interval? Can we do better?

Basic notions
Frequentist inference
◮ Estimator θb for unknown parameter θ.

iid
◮ Statistical model: data y1 , . . . , yn ∼ F , unknown
◮ Handedness data
• y = (dnan, hand)
• F puts probability mass on subset of R2
• θb is correlation coefficient
◮ Key issue: what is variability of θb when samples are
repeatedly taken from F ?
◮ Imagine F known — could answer question by
• analytical (mathematical) calculation
• simulation

Basic notions
Simulation with F known
◮ For r = 1, . . . , R:
iid
• generate random sample y1∗ , . . . , yn∗ ∼ F ;
• compute θbr using y1∗ , . . . , yn∗ ;
◮ Output after R iterations:
θb1∗ , θb2∗ , . . . , θbR

∗
◮ Use θb1∗ , θb2∗ , . . . , θbR b

∗ to estimate sampling distribution of θ
(histogram, density estimate, . . .)

◮ If R → ∞, then get perfect match to theoretical calculation
(if available): Monte Carlo error disappears completely
◮ In practice R is finite, so some error remains

Basic notions
Handedness data: Fitted bivariate normal model
Figure: Contours of bivariate normal distribution fitted to handedness data;

b1 = 28.5, µ
parameter estimates are µ b2 = 1.7, σ
b1 = 5.4, σ
b2 = 1.5, ρb = 0.509.
The data are also shown.
10
0.020
8 1
0.015
6 1
hand
0.010
4 1 1
2 2211 0.005
1 1 2 2 15534 11
0 0.000
10 15 20 25 30 35 40 45
dnan

Basic notions
Handedness data: Parametric bootstrap samples
Figure: Left: original data, with jittered vertical values. Centre and
right: two samples generated from the fitted bivariate normal distribu-
tion.
10
10
10
8
8
Correlation 0.509 Correlation 0.753 Correlation 0.533
6
6
hand
hand
hand
4
4
2
2
0
0
0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50
dnan dnan dnan

Basic notions
Handedness data: Correlation coefficient
Figure: Bootstrap distributions with R = 10000. Left: simulation from

fitted bivariate normal distribution. Right: simulation from the data by
bootstrap resampling. The lines show the theoretical probability density
function of the correlation coefficient under sampling from a fitted bivariate
normal distribution.
3.5
3.5
3.0
3.0
1.0 1.5 2.0 2.5
1.0 1.5 2.0 2.5

Probability density
Probability density
0.5
0.5
0.0
0.0
−0.5 0.0 0.5 1.0 −0.5 0.0 0.5 1.0

Correlation coefficient Correlation coefficient

Basic notions
F unknown
◮ Replace unknown F by estimate Fb obtained
• parametrically — e.g. maximum likelihood or robust fit of
distribution F (y) = F (y; ψ) (normal, exponential, bivariate
normal, . . .)
• nonparametrically — using empirical distribution function
(EDF) of original data y1 , . . . , yn , which puts mass 1/n on
each of the yj
◮ Algorithm: For r = 1, . . . , R:
iid
• generate random sample y1∗ , . . . , yn∗ ∼ Fb ;
• compute θbr using y1 , . . . , yn ;
∗ ∗
◮ Output after R iterations:
θb1∗ , θb2∗ , . . . , θbR

∗

Basic notions
Nonparametric bootstrap
iid
◮ Bootstrap (re)sample y1∗ , . . . , yn∗ ∼ Fb, where Fb is EDF of
y1 , . . . , yn
• Repetitions will occur!
◮ Compute bootstrap θb∗ using y1∗ , . . . , yn∗ .
◮ For handedness data take n = 37 pairs y ∗ = (dnan, hand)∗
with equal probabilities 1/37 and replacement from original
pairs (dnan, hand)
◮ Repeat this R times, to get θb∗ , . . . , θb∗
1 R
◮ See picture
◮ Results quite different from parametric simulation — why?

Basic notions
Handedness data: Bootstrap samples
Figure: Left: original data, with jittered vertical values. Centre and
right: two bootstrap samples, with jittered vertical values.
8
8
Correlation 0.509 Correlation 0.733 Correlation 0.491
7
7
6
6
5
5
hand
hand
hand
4
4
3
3
2
2
1
1
10 15 20 25 30 35 40 45 10 15 20 25 30 35 40 45 10 15 20 25 30 35 40 45
dnan dnan dnan

Basic notions
Using the θb∗

◮ b
Bootstrap replicates θbr∗ used to estimate properties of θ.
◮ Write θ = θ(F ) to emphasize dependence on F
◮ Bias of θb as estimator of θ is
iid
β(F ) = E(θb | y1 , . . . , yn ∼ F ) − θ(F )
estimated by replacing unknown F by known estimate Fb:

iid
β(Fb) = E(θb | y1 , . . . , yn ∼ Fb ) − θ(Fb )
◮ Replace theoretical expectation E() by empirical average:

R
X
β(Fb) ≈ b = θb∗ − θb = R−1 θbr∗ − θb
r=1

Basic notions
◮ Estimate variance ν(F ) = var(θb | F ) by
1 X b∗ b∗ 2
R
v= θ −θ
R − 1 r=1 r
◮ Estimate quantiles of θb by taking empirical quantiles of
θb1∗ , . . . , θbR
∗
◮ For handedness data, 10,000 replicates shown earlier give
b = −0.046, v = 0.043 = 0.2052

Basic notions
Handedness data
Figure: Summaries of the θb∗ . Left: histogram, with vertical line showing θ.
b
Right: normal Q–Q plot of θb∗ .
Histogram of t
2.5
2.0
0.5
1.0 1.5
Density
t*
0.0
0.5
−0.5
0.0
−0.5 0.0 0.5 1.0 −4 −2 0 2 4

t* Quantiles of Standard Normal

Basic notions
How many bootstraps?

◮ Must estimate moments and quantiles of θb and derived
quantities. Nowadays often feasible to take R ≥ 5000
◮ Need R ≥ 100 to estimate bias, variance, etc.
◮ Need R ≫ 100, prefer R ≥ 1000 to estimate quantiles
needed for 95% confidence intervals
R=199 R=999
4
4
2
2
Z*
Z*
0
0
−2
−2
−4
−4
−4 −2 0 2 4 −4 −2 0 2 4
Theoretical Quantiles Theoretical Quantiles

Basic notions
Key points
◮ Estimator is algorithm
• applied to original data y1 , . . . , yn gives original θb
• applied to simulated data y1∗ , . . . , yn∗ gives θb∗
• θb can be of (almost) any complexity
• for more sophisticated ideas (later) to work, θb must often be
smooth function of data
◮ Sample is used to estimate F
• Fb ≈ F — heroic assumption
◮ Simulation replaces theoretical calculation
• removes need for mathematical skill
• does not remove need for thought
• check code very carefully — garbage in, garbage out!
◮ Two sources of error
• statistical (Fb 6= F ) — reduce by thought
• simulation (R 6= ∞) — reduce by taking R large (enough)

Motivation
Basic notions
Several samples
Variance estimation
Tests
Regression

Normal confidence intervals

.
◮ If θb approximately normal, then θb ∼ N (θ + β, ν), where θb
has bias β = β(F ) and variance ν = ν(F )
◮ If β, ν known, (1 − 2α) confidence interval for θ would be
(D1)
θb − β ± zα ν 1/2 ,
where Φ(zα ) = α.
◮ Replace β, ν by estimates:
. .
β(F ) = β(Fb) = b = θb∗ − θb
. . X
ν(F ) = ν(Fb) = v = (R − 1)−1 (θbr∗ − θb∗ )2 ,
r
giving (1 − 2α) interval θb − b ± zα v 1/2 .

◮ Handedness data: R = 10, 000, b = −0.046, v = 0.2052 ,
α = 0.025, zα = −1.96, so 95% CI is (0.147, 0.963)

◮ Normal approximation reliable? Transformation needed?
◮ b
Here are plots for ψb = 12 log{(1 + θ)/(1 b
− θ)}:
1.5
Transformed correlation coefficient
1.5
1.0
1.0
Density
0.0 0.5
0.5
−0.5
0.0
−0.5 0.0 0.5 1.0 1.5 −4 −2 0 2 4

Transformed correlation coefficient Quantiles of Standard Normal


◮ Correlation coefficient: try Fisher’s z transformation:
ψb = ψ(θ)
b = 1
2
b
log{(1 + θ)/(1 b
− θ)}
for which compute
1 X b∗ b∗ 2
R
X R
bψ = R−1 ψbr∗ − ψ,
b vψ = ψ −ψ ,
r=1
R − 1 r=1 r
◮ (1 − 2α) confidence interval for θ is

n o n o
1/2 1/2
ψ −1 ψb − bψ − z1−α vψ , ψ −1 ψb − bψ − zα vψ
◮ For handedness data, get (0.074, 0.804)

◮ But how do we choose a transformation in general?

Pivots
◮ Hope properties of θb1∗ , . . . , θbR
∗ mimic effect of sampling from
original model.
◮ Amounts to faith in ‘substitution principle’: may replace
unknown F with known Fb — false in general, but often
more nearly true for pivots.
◮ Pivot is combination of data and parameter whose
distribution is independent of underlying model.
iid
◮ Canonical example: Y1 , . . . , Yn ∼ N (µ, σ 2 ). Then
Y −µ
Z= ∼ tn−1 ,
(S 2 /n)1/2
for all µ, σ 2 — so independent of the underlying
distribution, provided this is normal
◮ Exact pivot generally unavailable in nonparametric case.

Studentized statistic
◮ Idea: generalize Student t statistic to bootstrap setting
◮ Requires variance V for θb computed from y1 , . . . , yn
◮ Analogue of Student t statistic:
θb − θ
Z=
V 1/2
◮ If the quantiles zα of Z known, then
!
θb − θ
Pr (zα ≤ Z ≤ z1−α ) = Pr zα ≤ 1/2 ≤ z1−α = 1 − 2α
V
(zα no longer denotes a normal quantile!) implies that

Pr θb − V 1/2 z1−α ≤ θ ≤ θb − V 1/2 zα = 1 − 2α
so (1 − 2α) confidence interval is (θb − V 1/2 z1−α , θb − V 1/2 zα )

◮ Bootstrap sample gives (θb∗ , V ∗ ) and hence

θb∗ − θb
Z∗ =
V ∗1/2
◮ b V ):
R bootstrap copies of (θ,
(θb1∗ , V1∗ ), (θb2∗ , V2∗ ), ..., (θbR
∗
, VR∗ )
and corresponding
θb1∗ − θb θb2∗ − θb θbR
∗ −θ b
z1∗ = ∗1/2
, z2∗ = ∗1/2
, ..., ∗
zR = ∗1/2
.
V1 V2 VR
◮ Use ∗ ∗
z1 , . . . , zR
to estimate distribution of Z — for example,
∗ < · · · < z∗
order statistics z(1) (R) used to estimate quantiles
◮ Get (1 − 2α) confidence interval
θb − V 1/2 z((1−α)(R+1))
∗
, θb − V 1/2 z(α(R+1))
∗

Why Studentize?
D
◮ Studentize, so Z −→ N (0, 1) as n → ∞. Edgeworth series:
Pr(Z ≤ z | F ) = Φ(z) + n−1/2 a(z)φ(z) + O(n−1 );
a(·) even quadratic polynomial.

◮ Corresponding expansion for Z ∗ is
Pr(Z ∗ ≤ z | Fb) = Φ(z) + n−1/2 b

a(z)φ(z) + Op (n−1 ).
a(z) = a(z) + Op (n−1/2 ), so

Typically b
Pr(Z ∗ ≤ z | Fb) − Pr(Z ≤ z | F ) = Op (n−1 ).

D
◮ If don’t studentize, Z = (θb − θ) −→ N (0, ν). Then
z z z
Pr(Z ≤ z | F ) = Φ 1/2 +n−1/2 a′ 1/2 φ 1/2 +O(n−1 )
ν ν ν
and
z z z
Pr(Z ∗ ≤ z | Fb) = Φ +n −1/2 ′
a
b φ 1/2 +Op (n−1 ).
νb1/2 νb1/2 νb
Typically νb = ν + Op (n−1/2 ), giving
Pr(Z ∗ ≤ z | Fb ) − Pr(Z ≤ z | F ) = Op (n−1/2 ).
◮ Thus use of Studentized Z reduces error from Op (n−1/2 ) to

Op (n−1 ) — better than using large-sample asymptotics, for
which error is usually Op (n−1/2 ).

Other confidence intervals

◮ Problem for studentized intervals: must obtain V , intervals
not scale-invariant
◮ Simpler approaches:
• Basic bootstrap interval: treat θb − θ as pivot, get
θb − (θb((R+1)(1−α))
∗ b
− θ), θb − (θb((R+1)α)
∗ b
− θ).
• Percentile interval: use empirical quantiles of θb1∗ , . . . , θbR

∗
:
θb((R+1)α)
∗
, θb((R+1)(1−α))
∗
.
◮ Improved percentile intervals (BCa , ABC, . . .)
• Replace percentile interval with
θb((R+1)α
∗
′) , θb((R+1)(1−α
∗
′′ )) ,
where α′ , α′′ chosen to improve properties.

• Scale-invariant.

Handedness data
◮ 95% confidence intervals for correlation coefficient θ,

R = 10, 000:
Normal 0.147 0.963

Percentile −0.047 0.758
Basic 0.262 1.043
′ ′′
BCa (α = 0.0485, α = 0.0085) 0.053 0.792
Student 0.030 1.206
Basic (transformed) 0.131 0.824

Student (transformed) −0.277 0.868
◮ Transformation is essential here!

General comparison
◮ Bootstrap confidence intervals usually too short — leads to

under-coverage
◮ Normal and basic intervals depend on scale.
◮ Percentile interval often too short but is scale-invariant.
◮ Studentized intervals give best coverage overall, but
• depend on scale, can be sensitive to V
• length can be very variable
• best on transformed scale, where V is approximately
constant
◮ Improved percentile intervals have same error in principle
as studentized intervals, but often shorter — so lower
coverage

Caution
◮ Edgeworth theory OK for smooth statistics — beware
rough statistics: must check output.
◮ Bootstrap of median theoretically OK, but very sensitive to
sample values in practice.
◮ Role for smoothing?
.. .
10
.
..
....
........
.........
5
.............
T*-t for medians
.
.............................
..........
0
................
............
..............
........
-5
.
.........
. ......
-10
. .
-2 0 2
Quantiles of Standard Normal

Key points
◮ Numerous procedures available for ‘automatic’ construction

of confidence intervals
◮ Computer does the work
◮ Need R ≥ 1000 in most cases
◮ Generally such intervals are a bit too short
◮ Must examine output to check if assumptions (e.g.
smoothness of statistic) satisfied
◮ May need variance estimate V — see later

Several samples
Motivation
Basic notions
Several samples
Variance estimation
Tests
Regression

Several samples
Gravity data
Table: Measurements of the acceleration due to gravity, g, given as

deviations from 980,000 ×10−3 cms−2 , in units of cms−2 × 10−3 .
Series
1 2 3 4 5 6 7 8
76 87 105 95 76 78 82 84
82 95 83 90 76 78 79 86
83 98 76 76 78 78 81 85
54 100 75 76 79 86 79 82
35 109 51 87 72 87 77 77
46 109 76 79 68 81 79 76
87 100 93 77 75 73 79 77
68 81 75 71 78 67 78 80
75 62 75 79 83
68 82 82 81
67 83 76 78
73 78
64 78

Several samples
Gravity data
Figure: Gravity series boxplots, showing a reduction in variance, a shift

in location, and possible outliers.
100
80
g
60
40
1 2 3 4 5 6 7 8
series

Several samples
Gravity data
◮ Eight series of measurements of gravitational acceleration g
made May 1934 – July 1935 in Washington DC
◮ Data are deviations from 9.8 m/s2 in units of 10−3 cm/s2
◮ Goal: Estimate g and provide confidence interval
◮ Weighted combination of series averages and its variance
estimate
P8 8
!−1
y × n i /s 2 X
i=1 i
θb = P 8 2
i
, V = ni /s2i ,
i=1 n i /s i i=1
giving
θb = 78.54, V = 0.592
and 95% confidence interval of θb ± 1.96V 1/2 = (77.5, 79.8)

Several samples
Gravity data: Bootstrap

◮ Apply stratified (re)sampling to series, taking each series as
a separate stratum. Compute θb∗ , V ∗ for simulated data
◮ Confidence interval based on
θb∗ − θb
Z ∗ = ∗1/2 ,
V
whose distribution is approximated by simulations
θb1∗ − θb θbR
∗ −θ b
z1∗ = 1/2
, . . . , zR
∗
= 1/2
,
V1 VR
giving
(θb − V 1/2 z((R+1)(1−α))
∗
, θb − V 1/2 z((R+1)α)
∗
)
◮ For 95% limits set α = 0.025, so with R = 999 use
∗ , z∗
z(25) (975) , giving interval (77.1, 80.3).

Several samples
Figure: Summary plots for 999 nonparametric bootstrap simulations.

Top: normal probability plots of t∗ and z ∗ = (t∗ − t)/v ∗1/2 . Line on
the top left has intercept t and slope v 1/2 , line on the top right has
intercept zero and unit slope. Bottom: the smallest t∗r also has the
smallest v ∗ , leading to an outlying value of z ∗ .
. .
.. .. .
5
81 ....
.. ............
.... ...........
.... ...
......
........ .....................
.........
..... .............
..........
80
0
....
.......
....
..
....... ...
... .............
....
.
.... .....
......... ........
............
-5
79
.
........
z*
t*
...........
.........
......
-10
....
78
..
...........
.
.
....
.....
-15
....
77
...
.. .
-2 0 2 -2 0 2
Quantiles of Standard Normal Quantiles of Standard Normal
... ..
0.1 0.2 0.3 0.4 0.5 0.6 0.7
0.1 0.2 0.3 0.4 0.5 0.6 0.7
. .. .. . ...
.. ................... . .....................
. . .................................................................. . . .. .. ................... .... ..
. .... . .. .. ... ... .. . ........................................ .
.. ....................................................................................... . . ..........................
....................
. . ................................................................. .............. ....... .
... . . . . . ......................................................... ..
. . . . .. . ................................................................ ....
. . .. .
. . . .............................................. . .
. . . . ..
sqrt(v*)
sqrt(v*)
.. ...... ..... ..... ........... . . . .. ...................................... . .

. ..... . .. ........................ . . . . ................................. ....
. . ............................. ........ . . . . . . . ...
... . . ........... ..... .................. .. .. .. . ... .. .................................... .
. . . .. . . ..... ...... .... . .. . . . ...... ..... .... .
. . . . . ..
. . . .. . . . . . . ... ... .....
.
. .
. . . .
. .
. .
77 78 79 80 81 -15 -10 -5 0 5
t* z*

Several samples
Key points
◮ For several independent samples, implement bootstrap by

stratified sampling independently from each
◮ Same basic ideas apply for confidence intervals

Variance estimation
Motivation
Basic notions
Several samples
Variance estimation
Tests
Regression

Variance estimation
Variance estimation
◮ Variance estimate V needed for certain types of confidence

interval (esp. studentized bootstrap)
◮ Ways to compute this:
• double bootstrap
• delta method
• nonparametric delta method
• jackknife

Variance estimation
Double bootstrap
◮ Bootstrap sample y1∗ , . . . , yn∗ and corresponding estimate θb∗

◮ Take Q second-level bootstrap samples y1∗∗ , . . . , yn∗∗ from
y1∗ , . . . , yn∗ , giving corresponding bootstrap estimates
θb1∗∗ , . . . , θbQ
∗∗ ,
◮ Compute variance estimate V as sample variance of

θb1∗∗ , . . . , θbQ
∗∗
◮ Requires total R(Q + 1) resamples, so could be expensive

.
◮ Often reasonable to take Q = 50 for variance estimation, so
need O(50 × 1000) resamples — nowadays not infeasible

Variance estimation
Delta method
◮ Computation of variance formulae for functions of averages
and other estimators
◮ b estimates ψ = g(θ), and θb ∼. N (θ, σ 2 /n)
Suppose ψb = g(θ)
◮ Then provided g′ (θ) 6= 0, have (D2)
b = g(θ) + O(n−1 )
E(ψ)
b = σ 2 g′ (θ)2 /n + O(n−3/2 )
var(ψ)
b = . 2 ′ b2
◮ Then var(ψ) σ
b g (θ) /n = V
◮ Example (D3): θb = Y , ψb = log θb
.
b =
◮ Variance stabilisation (D4): if var(θ) S(θ)2 /n, find
.
b =constant
transformation h such that var{h(θ)}
◮ Extends to multivariate estimators, and to
ψb = g(θb1 , . . . , θbd )

Variance estimation

Variance estimation
Nonparametric delta method

◮ Write parameter θ = t(F ) as functional of distribution F
◮ General approximation:
n
. 1 X
V = VL = 2 L(Yj ; F )2 .
n
j=1
◮ L(y; F ) is influence function value for θ for observation at y

when distribution is F :
t {(1 − ε)F + εHy } − t(F )
L(y; F ) = lim ,
ε→0 ε
where Hy puts unit mass at y. Close link to robustness.
◮ Empirical versions of L(y; F ) and VL are
X
lj = L(yj ; Fb), vL = n−2 lj2 ,
usually obtained by analytic/numerical differentation.
Variance estimation
Computation of lj
◮ Write θb in weighted form, differentiate with respect to ε
◮ Sample average:
1X X

θb = y = yj = wj yj
n wj ≡1/n
Change weights:
wj 7→ ε + (1 − ε) n1 , wi 7→ (1 − ε) n1 , i 6= j
so (D5)
y 7→ y ε = εyj + (1 − ε)y = ε(yj − y) + y,
P
giving lj = yj − y and vL = n12 (yj − y)2 = n−1 n n s
−1 2
◮ Interpretation: lj is standardized change in y when increase

mass on yj

Variance estimation
Nonparametric delta method: Ratio

◮ Population F (u, x) with y = (u, x) and
Z Z
θ = t(F ) = x dF (u, x)/ u dF (u, x),
sample version is
Z Z
θb = t(Fb) = x dFb(u, x)/ u dFb(u, x) = x/u
◮ Then using chain rule of differentiation (D6),

b j )/u,
lj = (xj − θu
giving
!2
1 X b j
xj − θu
vL = 2
n u

Variance estimation
Handedness data: Correlation coefficient

◮ Correlation coefficient
P may be written as a function of
averages xu = n −1 xj uj etc.:
xu − x u
θb = n o1/2 ,
(x2 − x2 )(u2 − u2 )
from which empirical influence values lj can be derived

◮ In this example (and for others involving only averages),
nonparametric delta method is equivalent to delta method
◮ Get
vL = 0.029
for comparison with v = 0.043 obtained by bootstrapping.
◮ b — as here!
vL typically underestimates var(θ)

Variance estimation
Delta methods: Comments

◮ Can be applied to many complex statistics
◮ Delta method variances often underestimate true variances:
b
vL < var(θ)
◮ Can be applied automatically (numerical differentation) if

algorithm for θb written in weighted form, e.g.
X
xw = wj xj , wj ≡ 1/n for x
and vary weights successively for j = 1, . . . , n, setting

X
wj = wi + ε, i 6= j, wi = 1
for ε = 1/(100n) and using the definition as derivative

Variance estimation
Jackknife
◮ Approximation to empirical influence values given by
lj ≈ ljack,j = (n − 1)(θb − θb−j ),
where θb−j is value of θb computed from sample
y1 , . . . , yj−1 , yj+1 , . . . , yn
◮ Jackknife bias and variance estimates are

1X 1 X
2
bjack = − ljack,j , vjack = ljack,j − nb2jack
n n(n − 1)
◮ Requires n + 1 calculations of θb
◮ b with
Corresponds to numerical differentiation of θ,
ε = −1/(n − 1)

Variance estimation
Key points
◮ Several methods available for estimation of variances

◮ Needed for some types of confidence interval
◮ Most general method is double bootstrap: can be expensive
◮ Delta methods rely on linear expansion, can be applied
numerically or analytically
◮ Jackknife gives approximation to delta method, can fail for
rough statistics

Tests
Motivation
Basic notions
Several samples
Variance estimation
Tests
Regression

Tests
Ingredients
◮ Ingredients for testing problems:

• data y1 , . . . , yn ;
• model M0 to be tested;
• test statistic t = t(y1 , . . . , yn ), with large values giving
evidence against M0 , and observed value tobs
◮ P-value, pobs = Pr(T ≥ tobs | M0 ) measures evidence
against M0 — small pobs indicates evidence against M0 .
◮ Difficulties:
• pobs may depend upon ‘nuisance’ parameters, those of M0 ;
• pobs often hard to calculate.

Tests
Examples
◮ Balsam-fir seedlings in 5 × 5 quadrats — Poisson sample?
0 1 2 3 4 3 4 2 2 1
0 2 0 2 4 2 3 3 4 2
1 1 1 1 4 1 5 2 2 3
4 1 2 5 2 0 3 2 1 1
3 1 4 3 1 0 0 2 7 0
◮ Two-way layout: row-column independence?
1 2 2 1 1 0 1
2 0 0 2 3 0 0
0 1 1 1 2 7 3
1 1 2 0 0 0 1
0 1 1 1 1 0 0

Tests
Estimation of pobs
◮ Estimate pobs by simulation from fitted null hypothesis
model Mc0 .
◮ Algorithm: for r = 1, . . . , R:
c0 ;
• simulate data set y1∗ , . . . , yn∗ from M
• calculate test statistic t∗r from y1∗ , . . . , yn∗ .
◮ Calculate simulation estimate
1 + #{t∗r ≥ tobs }
pb =
1+R
of
c0 ).
pbobs = Pr(T ≥ tobs | M
◮ Simulation and statstical errors:
pb ≈ pbobs ≈ pobs

Tests
Handedness data: Test of independence

◮ Are dnan and hand positively associated?
◮ Take T = θb (correlation coefficient), with tobs = 0.509; this
is large in case of positive association (one-sided test)
◮ Null hypothesis of independence: F (u, x) = F1 (u)F2 (x)
◮ Take bootstrap samples independently from
Fb1 ≡ (dnan1 , . . . , dnann ) and from Fb2 ≡ (hand1 , . . . , handn ),
then put them together to get bootstrap data
(dnan∗1 , hand∗1 ), . . . , (dnan∗n , hand∗n ).
◮ With R = 9, 999 get 18 values of θb∗ ≥ θ, b so
1 + 18
pb = = 0.0019 :
1 + 9999
hand and dnan seem to be positively associated
◮ To test positive or negative association (two-sided test),
b gives pb = 0.004.
take T = |θ|:
Tests
c0
Handedness data: Bootstrap from M
1
8
2.5
7
1.0 1.5 2.0

Probability density
2 1 1
65
hand
4
1
3
0.5
2 2
2
2 1 1102 3 4 3 1 0.0
1
15 20 25 30 35 40 45 −0.6 −0.2 0.2 0.6

dnan Test statistic

Tests
Choice of R
◮ Take R big enough to get small standard error for pb,

typically ≥ 100, using binomial calculation:

1 + #{t∗r ≥ tobs }
var(b
p) = var
1+R
. 1 pobs (1 − pobs )
= 2
Rpobs (1 − pobs ) =
R R
.
so if pobs = 0.05 need R ≥ 1900 for 10% relative error (D7)
.
◮ Can choose R sequentially: e.g. if pb = 0.06 and R = 99, can
augment R enough to diminish standard error.
◮ Taking R too small lowers power of test.

Tests
Duality with confidence interval
◮ Often unclear how to impose null hypothesis on sampling

scheme
◮ General approach based on duality between confidence
interval I1−α = (θα , ∞) and test of null hypothesis θ = θ0
◮ Reject null hypothesis at level α in favour of alternative
θ > θ0 , if θ0 < θα
◮ Handedness data: θ0 = 0 6∈ I0.95 , but θ0 = 0 ∈ I0.99 , so
estimated significance level 0.01 < pb < 0.05: weaker
evidence than before
◮ Extends to tests of θ = θ0 against other alternatives:
6 I 1−α = (−∞, θα ), have evidence that θ < θ0
• if θ0 ∈
• if θ0 ∈6 I1−2α = (θα , θα ), have evidence that θ 6= θ0

Tests
Pivot tests
◮ Equivalent to use of confidence intervals
◮ Idea: use (approximate) pivot such as Z = (θb − θ)/V 1/2 as
statistic to test θ = θ0
◮ Observed value of pivot is zobs = (θb − θ0 )/V 1/2
◮ Significance level is
!
θb − θ
Pr ≥ zobs | M0 = Pr(Z ≥ zobs | M0 )
V 1/2
= Pr(Z ≥ zobs | F )
.
= Pr(Z ≥ zobs | Fb)
◮ Compare observed zobs with simulated distribution of
Z ∗ = (θb∗ − θ)/V
b ∗1/2 , without needing to construct null
hypothesis model M c0
◮ Use of (approximate) pivot is essential for success
Tests
Example: Handedness data

◮ Test zero correlation (θ0 = 0), not independence; θb = 0.509,
V = 0.1702 :
θb − θ0 0.509 − 0
zobs = 1/2 = = 2.99
V 0.170
◮ Observed significance level is
1 + #{zr∗ ≥ zobs } 1 + 215
pb = = = 0.0216
1+R 1 + 99990.3
Probability density
0.1 0.20.0
−6 −4 −2 0 2 4 6
Test statistic

Tests
Exact tests
◮ Problem: bootstrap estimate is

c0 ) 6= Pr(T ≥ t | M0 ) = pobs ,
pbobs = Pr(T ≥ tobs | M
so estimate the wrong thing

◮ In some cases can eliminate parameters from null
hypothesis distribution by conditioning on sufficient
statistic
◮ Then simulate from conditional distribution
◮ More generally, can use Metropolis–Hastings algorithm to
simulate from conditional distribution (below)

Tests
Example: Fir data

iid
◮ Data Y1 , . . . , Yn ∼ Pois(λ), with λ unknown
◮ Poisson model has E(Y ) = var(Y ) = λ: base test of
overdispersion on
X .
T = (Yj − Y )2 /Y ∼ χ2n−1 ;
observed value is tobs = 55.15
◮ Unconditional significance level:
c0 , λ)
Pr(T ≥ tobs | M
P
◮ Condition on value w of sufficient statistic W = Yj :
c0 , W = w),
pobs = Pr(T ≥ tobs | M
independent of λ, owing to sufficiency of W
◮ Exact test: simulate from P multinomial distribution of
Y1 , . . . , Yn given W = Yj = 107.
Tests
Example: Fir data
Figure: Simulation results for dispersion test. Left panel: R = 999 val-
ues of the dispersion statistic t∗ obtained under multinomial sampling:
the data value is tobs = 55.15 and pb = 0.25. Right panel: chi-squared
plot of ordered values of t∗ , dotted line shows χ249 approximation to
null conditional distribution.
.
0.04
80
...
dispersion statistic t*
.
....
0.03
......
..........
.
.....
......
60
......
.....
0.02
..
.........
.
......
......
.....
40
......
0.01
.
........
.
..
.....
...
.
..
0.0
20
20 40 60 80 30 40 50 60 70 80
t* chi-squared quantiles

Tests
Handedness data: Permutation test

◮ Are dnan and hand related?
◮ Take T = θb (correlation coefficient) again
◮ Impose null hypothesis of independence:
F (u, x) = F1 (u)F2 (x), but condition so that marginal
distributions Fb1 and Fb2 are held fixed under resampling
plan — permutation test
◮ Take resamples of form
(dnan1 , hand1∗ ), . . . , (dnann , handn∗ )
where (1∗ , . . . , n∗ ) is random permutation of (1, . . . , n)
◮ Doing this with R = 9, 999 gives one- and two-sided
significance probabilities of 0.002, 0.003
◮ Typically values of pb very similar to those for
corresponding bootstrap test

Tests
Handedness data: Permutation resample
3.0
8
1.0 1.5 2.0 2.5

7
Probability density
65
hand
4 3
0.5
2
0.0
1
15 20 25 30 35 40 45 −0.6 −0.2 0.2 0.6

dnan Test statistic

Tests
Contingency table
1 2 2 1 1 0 1
2 0 0 2 3 0 0
0 1 1 1 2 7 3
1 1 2 0 0 0 1
0 1 1 1 1 0 0
◮ Are row and column classifications independent:
Pr(row i, column j) = Pr(row i) × Pr(column j)?
◮ Standard test statistic for independence is
X (yij − ybij )2 yi· y·j
T = , ybij =
ybij y··
i,j
.
◮ Get Pr(χ224 ≥ 38.53) = 0.048, but is T ∼ χ224 ?

Tests
Exact tests: Contingency table

◮ For exact test, need to simulate distribution of T
conditional on sufficient statistics — row and column totals
◮ Algorithm (D8) for conditional simulation:
1. choose two rows j1 < j2 and two columns k1 < k2 at
random
2. generate new values from hypergeometric distribution
of yj1 k1 conditional on margins of 2 × 2 table
y j 1 k1 y j 1 k2
y j 2 k1 y j 2 k2
3. compute test statistic T ∗ every I = 100 iterations, say

◮ Compare observed value tobs = 38.53 with simulated T ∗ —
.
get pb = 0.08

Tests
Key points
◮ Tests can be performed using resampling/simulation

◮ Must take account of null hypothesis, by
• modifying sampling scheme to satisfy null hypothesis
• inverting confidence interval (pivot test)
◮ Can use Monte Carlo simulation to get approximations to
exact tests — simulate from null distribution of data,
conditional on observed value of sufficient statistic
◮ Sometimes obtain permutation tests — very similar to
bootstrap tests

Regression
Motivation
Basic notions
Several samples
Variance estimation
Tests
Regression

Regression
Linear regression
◮ Independent data (x1 , y1 ), . . ., (xn , yn ) with
yj = xTj β + εj , εj ∼ (0, σ 2 )
◮ b leverages hj , residuals
Least squares estimates β,
yj − xTj βb .
ej = ∼ (0, σ 2 )
(1 − hj )1/2
◮ Design matrix X is experimental ancillary — should be
held fixed if possible, as
b = σ 2 (X T X)−1
var(β)
if model y = Xβ + ε correct

Regression
Linear regression: Resampling schemes

◮ Two main resampling schemes
◮ Model-based resampling:
yj∗ = xTj βb + ε∗j , ε∗j ∼ EDF(e1 − e, . . . , en − e)
• Fixes design but not robust to model failure

• Assumes εj sampled from population
◮ Case resampling:
(xj , yj )∗ ∼ EDF{(x1 , y1 ), . . . , (xn , yn )}
• Varying design X but robust

• Assumes (xj , yj ) sampled from population
• Usually design variation no problem; can prove awkward in
designed experiments and when design sensitive.

Regression
Cement data
Table: Cement data: y is the heat (calories per gram of cement) evolved
while samples of cement set. The covariates are percentages by weight
of four constituents, tricalciumaluminate x1 , tricalcium silicate x2 , tet-
racalcium alumino ferrite x3 and dicalcium silicate x4 .
x1 x2 x3 x4 y
1 7 26 6 60 78.5
2 1 29 15 52 74.3
3 11 56 8 20 104.3
4 11 31 8 47 87.6
5 7 52 6 33 95.9
6 11 55 9 22 109.2
7 3 71 17 6 102.7
8 1 31 22 44 72.5
9 2 54 18 22 93.1
10 21 47 4 26 115.9
11 1 40 23 34 83.8
12 11 66 9 12 113.3
13 10 68 8 12 109.4

Regression
Cement data
◮ Fit linear model
y = β0 + β1 x1 + β2 x2 + β3 x3 + β4 x4 + ε
and apply case resampling

.
◮ Covariates compositional: x1 + · · · + x4 = 100% so X
almost collinear — smallest eigenvalue of X T X is
l5 = 0.0012
◮ Plot of βb1∗ against smallest eigenvalue of X ∗T X ∗ reveals
that var∗ (βb∗ ) strongly variable
1
◮ Relevant subset for case resampling — post-stratification of
output based on l5∗ ?

Regression
Cement data
10
10
.
. .
. ..
5
5
. . . . ..... .
. . .
. . . .. ..... .................................................................... . . . .
...... .. ........ ..
beta1hat*
beta2hat*
. . .... . .
.. .. .............................................................. . .. .................................. .............................. .
........
. . .. ... .. ................................................................. . . ................
.. . .. ... ... ................................................................................
.. . .... ................ .. . . ... ..................... . ... .
0
0
. . ... ..... . . . . .......... ............... ..
.. .. ... ..
. .. . .
-5
-5
. .
.
.
. .
-10
-10
1 5 10 50 500 1 5 10 50 500
smallest eigenvalue smallest eigenvalue

Regression
Cement data
Table: Standard errors of linear regression coefficients for cement data.

Theoretical and error resampling assume homoscedasticity. Resampling
results use R = 999 samples, but last two rows are based only on those
samples with the middle 500 and the largest 800 values of ℓ∗1 .
βb0 βb1 βb2

Normal theory 70.1 0.74 0.72
Model-based resampling, R = 999 66.3 0.70 0.69
Case resampling, all R = 999 108.5 1.13 1.12
Case resampling, largest 800 67.3 0.77 0.69

Regression
Survival data
dose x 117.5 235.0 470.0 705.0 940.0 1410
survival % y 44.000 16.000 4.000 0.500 0.110 0.700
55.000 13.000 1.960 0.320 0.015 0.006
6.120 0.019
◮ Data on survival % for rats at different doses

◮ Linear model:
log(survival) = β0 + β1 dose
• ••
4
50
••
•
2
•
40
•
log survival %
survival %
•
30
0 • •
•
20
-2
• •
•
10
-4
•
• ••
• • • • •
0
200 600 1000 1400 200 600 1000 1400

dose dose

Regression
Survival data
◮ Case resampling
◮ Replication of outlier: none (0), once (1), two or more (•).
◮ Model-based sampling including residual would lead to
change in intercept but not slope.
0.0
• •
• •• •
-0.004
•
estimated slope
• •
1 • • •• •
• ••
1 ••• • 1 •••••• ••• •••
111 1 11 1 ••• • •••••••• •••••
1111 1
1
11
11111111 11
1
11111
1
1 1 11
11 1111111111 1 1111
•
0 1
11 11
1 1 11 1 1 11 1 1 1
0 0 01000 00 0
-0.008
0 0 0 0
0 000 0 00000 0 0 0 0 0
0 0 00 0 000
0
0 0000
0 00 000 00
0000
00 0
-0.012
0.5 1.0 1.5 2.0 2.5 3.0

sum of squares

Regression
Generalized linear model

◮ Response may be binomial, Poisson, gamma, normal, . . .
yj ∼ mean µj , variance φV (µj ),
where g(µj ) = xTj β is linear predictor; g(·) is link function.
◮ b fitted values µ
MLE β, bj , Pearson residuals
yj − µ
bj .
rP j = 1/2
∼ (0, φ).
{V (b
µj )(1 − hj )}
◮ Bootstrapped responses
yj∗ = µ µj )1/2 ε∗j
bj + V (b
where ε∗j ∼ EDF(rP 1 − rP , . . . , rP n − rP ). However
• possible that yj∗ 6∈ {0, 1, 2, . . . , }
• rP j not exchangeable, so may need stratified resampling

Regression
AIDS data
◮ Log-linear model: number of reports in row j and column k
follows Poisson distribution with mean
µjk = exp(αj + βk )
◮ Log link function
g(µjk ) = log µjk = αj + βk
and variance function
var(Yjk ) = φ × V (µjk ) = 1 × µjk
◮ Pearson residuals:
Yjk − µbjk
rjk =
µjk (1 − hjk )}1/2
{b
◮ Model-based simulation:
∗ 1/2
Yjk =µ bjk ε∗jk
bjk + µ
Regression
Diagnosis Reporting-delay interval (quarters): Total

period reports
to end
Year Quarter 0† 1 2 3 4 5 6 ··· ≥14 of 1992
1988 1 31 80 16 9 3 2 8 ··· 6 174
2 26 99 27 9 8 11 3 ··· 3 211
3 31 95 35 13 18 4 6 ··· 3 224
4 36 77 20 26 11 3 8 ··· 2 205
1989 1 32 92 32 10 12 19 12 ··· 2 224
2 15 92 14 27 22 21 12 ··· 1 219
3 34 104 29 31 18 8 6 ··· 253
4 38 101 34 18 9 15 6 ··· 233
1990 1 31 124 47 24 11 15 8 ··· 281
2 32 132 36 10 9 7 6 ··· 245
3 49 107 51 17 15 8 9 ··· 260
4 44 153 41 16 11 6 5 ··· 285
1991 1 41 137 29 33 7 11 6 ··· 271
2 56 124 39 14 12 7 10 263
3 53 175 35 17 13 11 306
4 63 135 24 23 12 258
1992 1 71 161 48 25 310
2 95 178 39 318
3 76 181 273
4 67 133

Regression
AIDS data
◮ Poisson two-way model deviance 716.5 on 413 df —
indicates strong overdispersion: φ > 1, so Poisson model
implausible
◮ Residuals highly inhomogeneous — exchangeability
doubtful
500
6
400
4
2
Diagnoses
+ ++
300
+ + +
+ ++ ++ + rP
0
+ + +
+++
200
++ -2
++
++++ +
100
+
-4
++
+++
+++++
-6
0
1984 1986 1988 1990 1992 0 1 2 3 4 5

skewness

Regression
AIDS data: Prediction intervals
◮ To estimate prediction error:

∗
• simulate complete table yjk ;
∗
• estimate parameters from incomplete yjk
• get estimated row totals and ‘truth’
∗ X b∗ X
b∗+,j = eαbj
µ eβk , y+,j
∗
= ∗
yjk .
k unobs k unobs
• Prediction error ∗
y+,j b∗+,j
−µ
∗1/2
µ
b+,j
studentized so more nearly pivotal.
◮ Form prediction intervals from R replicates.

Regression
AIDS data: Resampling plans
◮ Resampling schemes:
• parametric simulation, fitted Poisson model
• parametric simulation, fitted negative binomial model
• nonparametric resampling of rP
• stratified nonparametric resampling of rP
◮ Stratification based on skewness of residuals, equivalent to
stratifying original data by values of fitted means
◮ Take strata for which
µ
bjk < 1, 1≤µ
bjk < 2, µ
bjk ≥ 2

Regression
AIDS data: Results

◮ Deviance/df ratios for the sampling schemes, R = 999.
◮ Poisson variation inadequate.
◮ 95% prediction limits.
poisson negative binomial
6
6
5
600
4
4
3
3
2
500
1
1
0
Diagnoses
0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5
400
deviance/df deviance/df
nonparametric stratified nonparametric
+ ++
6
300
+ +
5
+ ++ + +
+
4
+
+ ++ +
3
200
+
2
2
1
1989 1990 1991 1992

0
0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5
deviance/df deviance/df

Regression
AIDS data: Semiparametric model

◮ More realistic: generalized additive model
µjk = exp {α(j) + βk } ,
where α(j) is locally-fitted smooth.
◮ Same resampling plans as before
◮ 95% intervals now generally narrower and shifted upwards
100 200 300 400 500 600
600
500
Diagnoses
Diagnoses
+ ++ 400
+ +
+ ++ ++ + +
+++++ +
+++ + ++
300
+++++ + + + +
+ + + ++ +
++ +
++++ + ++ +
++++
200
+
0
1984 1986 1988 1990 1992 1989 1990 1991 1992

Regression
Key points
◮ Key assumption: independence of cases

◮ Two main resampling schemes for regression settings:
• Model-based
• Case resampling
◮ Intermediate schemes possible
◮ Can help to reduce dependence on assumptions needed for
regression model
◮ These two basic approaches also used for more complex
settings (time series, . . .), where data are dependent

Regression
Summary
◮ Bootstrap: simulation methods for frequentist inference.

◮ Useful when
• standard assumptions invalid (n small, data not normal,
. . .);
• standard problem has non-standard twist;
• complex problem has no (reliable) theory;
• or (almost) anywhere else.
◮ Have described
• basic ideas
• confidence intervals
• tests
• some approaches for regression

Regression
Books
◮ Chernick (1999) Bootstrap Methods: A Practicioner’s
Guide. Wiley
◮ Davison and Hinkley (1997) Bootstrap Methods and their
Application. Cambridge University Press
◮ Efron and Tibshirani (1993) An Introduction to the
Bootstrap. Chapman & Hall
◮ Hall (1992) The Bootstrap and Edgeworth Expansion.
Springer
◮ Lunneborg (2000) Data Analysis by Resampling: Concepts
and Applications. Duxbury Press
◮ Manly (1997) Randomisation, Bootstrap and Monte Carlo
Methods in Biology. Chapman & Hall
◮ Shao and Tu (1995) The Jackknife and Bootstrap. Springer
View publication stats


Bootstrap Methods and Their Applications.

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bootstrap Methods and Their Applications.

Uploaded by

Copyright:

Available Formats

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Bootstrap Methods and Their Application

Article in Technometrics · January 1997

The user has requested enhancement of the downloaded file.

Anthony Davison: Bootstrap Methods and their Application, 1

◮ Bootstrap: simulation methods for frequentist inference.

Anthony Davison: Bootstrap Methods and their Application, 2

Anthony Davison: Bootstrap Methods and their Application, 3

Diagnosis Reporting-delay interval (quarters): Total

Anthony Davison: Bootstrap Methods and their Application, 5

1984 1986 1988 1990 1992

Anthony Davison: Bootstrap Methods and their Application, 6

Find reliable assessment of uncertainty when

Anthony Davison: Bootstrap Methods and their Application, 7

Anthony Davison: Bootstrap Methods and their Application, 8

Table: Data from a study of handedness; hand is an integer measure

dnan hand dnan hand dnan hand dnan hand

Anthony Davison: Bootstrap Methods and their Application, 9

Anthony Davison: Bootstrap Methods and their Application, 10

◮ Is there dependence between dnan and hand for these

Anthony Davison: Bootstrap Methods and their Application, 11

◮ Estimator θb for unknown parameter θ.

Anthony Davison: Bootstrap Methods and their Application, 12

Simulation with F known

θb1∗ , θb2∗ , . . . , θbR

◮ Use θb1∗ , θb2∗ , . . . , θbR b

(histogram, density estimate, . . .)

Anthony Davison: Bootstrap Methods and their Application, 13

Handedness data: Fitted bivariate normal model

Figure: Contours of bivariate normal distribution fitted to handedness data;

Anthony Davison: Bootstrap Methods and their Application, 14

Handedness data: Parametric bootstrap samples

Anthony Davison: Bootstrap Methods and their Application, 15

Handedness data: Correlation coefficient

Figure: Bootstrap distributions with R = 10000. Left: simulation from

1.0 1.5 2.0 2.5

−0.5 0.0 0.5 1.0 −0.5 0.0 0.5 1.0

Anthony Davison: Bootstrap Methods and their Application, 16

◮ Output after R iterations:

θb1∗ , θb2∗ , . . . , θbR

Anthony Davison: Bootstrap Methods and their Application, 17

Anthony Davison: Bootstrap Methods and their Application, 18

Handedness data: Bootstrap samples

Anthony Davison: Bootstrap Methods and their Application, 19

Using the θb∗

estimated by replacing unknown F by known estimate Fb:

◮ Replace theoretical expectation E() by empirical average:

Anthony Davison: Bootstrap Methods and their Application, 20

◮ Estimate variance ν(F ) = var(θb | F ) by

◮ Estimate quantiles of θb by taking empirical quantiles of

◮ For handedness data, 10,000 replicates shown earlier give

b = −0.046, v = 0.043 = 0.2052

Anthony Davison: Bootstrap Methods and their Application, 21

−0.5 0.0 0.5 1.0 −4 −2 0 2 4

Anthony Davison: Bootstrap Methods and their Application, 22

How many bootstraps?

Anthony Davison: Bootstrap Methods and their Application, 23

Anthony Davison: Bootstrap Methods and their Application, 24

Anthony Davison: Bootstrap Methods and their Application, 25

Normal confidence intervals

giving (1 − 2α) interval θb − b ± zα v 1/2 .

Normal confidence intervals

−0.5 0.0 0.5 1.0 1.5 −4 −2 0 2 4