CLEC 23 16.69 19.

5
ple standard deviation
 = it really is; but they can compare it to sb or to an eyeball
of the bootstrap distribution, sb what Inferent

1/(r−1) ri=1 (θ̂i∗ −θ̂ ∗ )2 . estimate of standard deviation from a histogram of the bootstrap theory, to
The data for one combination of CLEC, class of service, and • confidence intervals: a quick-and-dirty interval, the bootstrap
Remuestreo en el Currículo de Estadística de
period are shown in Table 1 and Figure 1. Both samples are
positively skewed. The mean CLEC repair time is nearly double
percentile interval, is the range of the middle 95% of the
bootstrap distribution,
distribution, and see that it measures how the sample mean varies
due to random sampling. • draw (a

could be just chance.

Pregrado
that for ILEC, suggesting discrimination, though the difference • bias: the bootstrap bias estimate is θ̂ ∗ − θ̂ .
the
Resampling
only
is
practical
also
way
important
to do
in practice.
inference—when
It often
it is
provides
too dif-
and
• comput
mean, m
The one-sided permutation test p-value is 0.0171, well above Summary statistics of the bootstrap distributions are ficult to derive formulas, or the data are stored in a way
the 1% cutoff mandated by the PUC. In comparison, the pooled Tim C. HESTERBERG (Poster resumen: Andrey Duvan thatRinco
make Torres)
calculating the formulas impractical; a longer ver-
Observed SE Mean Bias The dis
t-test p-value is 0.0045, about four times too small. The permu-
CLEC 16.50913 3.961816 16.53088 0.0217463
sion of this article (Hesterberg 2014) and (Chamandy 2015) see Figure
tation test gives the correct answer, with nearly exact Type 1 contains examples from Google, from my work and oth- Howeve
ILEC 8.41161 0.357599 8.40411 -0.0075032
error rates; this was recognized as far back as Fisher (1936), ers. In other cases, resampling provides better accuracy than from the p
Introducción
who used t-tests as an approximation because perturbation tests The CLEC SE is larger primarily due to the smaller sam-
were computationally infeasible then. The t-test is inaccurate ple size and secondly to the larger sample sd in the original
formula methods. For one simple example, consider confi- idea is to
dence intervals for the variance of the CLEC population. lieu of the
Los
becausemétodos de toremuestreo
it is sensitive skewness whentienen
the sampleenorme
sizes dif-potencial en la
data. Bootstrap educación
percentile intervalsestadística.
are (7.73,
√ 9.13) Se
for utilizan
ILEC and constantemente como herramientas para motivar los
fer. Using t-tests for 10,000 Verizon fairness tests would result (10.1, 25.4) for CLEC. For comparison, s/ n = 0.36 for
s
ILEC
2
= 380.4, the bootstrap SE for s 2
is 267, and the 95% per-
métodos clásicos, la evidencia experimental sugiere que los estudiantes aprenden mejor utilizando centile
estos
interval
métodos
is (59, 932).
[2].
The classical normal-based in- • draw sa
in about 400 false positive results instead of the expected 100, and 4.07 for CLEC, and standard t intervals are (7.71, 9.12) and
El enfoque principal de este artículo es el bootstrap, donde hay una variedad de métodos
resulting in large monetary penalties. Similarly, t confidence in- (8.1, 24.9). The distribution appears approximately normal for
y problemas
terval is ((n − 1)s 2competitivos
/χ 2
, (n − que
1)s 2 son
/χ 2 más) = sutiles
(228, y
762). •
menos
comput
22,0.975 22,0.025
conocidos. Se espera
tervals are inaccurate. We will poder
see how proporcionar una mejor
inaccurate, and explore comprensión
the ILEC sample but not for dethelas ideas
smaller CLECclave y los
sample, méritos
It assumes
suggest- de that
los diferentes
(n − 1)s 2 /σ 2métodos
∼ χ 2 (n − [1].1), but for long-tailed The dist
alternatives, using the bootstrap. ing that t intervals might be reasonable for the ILEC mean but distributions the actual variance of s 2 is far greater than for see Figure
not the CLEC mean.
Una muestra
1.2 One-Sample Bootstrapde Bootstrap Validación distribuciones
The bootstrap separates the concept of a standard error—the La idea detrás Bootstrap
2000 to estimate
standard deviation of a how large
sampling standard errors the
distribution—from would
com-be with that • Bootstrap resamples are chosen randomly from the original
Let θ̂ be a statistic calculated from a sample of n iid obser- Lo explicamos√ ahora con
I θ̂ estadística
vations (time series and calculada a partir
other dependent datade areuna mues-
beyond the mon formula sample n forConversely,
s/ size. estimating the una
this SE serie
alsoofanswers de
a sample ejemplos
a common
mean. question sample.
traofde about bootstrapping—why
This visuales,
separation uno help
should donde las
students we sample with
cosas
understand the same size as the
generalmente
the concept.
scope thisobservaciones de nonparametric
article). In the ordinary iid. bootstrap,
on original
extensivedata—because
Basedfuncionan experience by doing so the
interviewing job
bien y tres con problemas. Nos di-standard errorsI reflect the
candidates,
I Calculamos
we draw n observations θ̂∗withcon n observaciones
replacement con
from the original data
attest that aactual
betterdata,
wayrather than
to teach a hypothetical
about larger or smaller
SEs is needed—too many dataset.
3.1 Sample Mean: Large Sample Size
to create
remplazo a bootstrap
de sample
los or resample,
datos originales. and calculate the rigimos a dos preguntas:
∗ do not understand SEs, and even confuse SEs in other contexts Figure 6 shows a population, the sampling distribution for
statistic θ̂ for this sample (we use ∗ to denote a bootstrap quan-
I
tity).Lo
Werepetimos
repeat that many muchas
times, say veces,
r = 10,000 digamos que
(we use 10,000 with I ¿Qué
the formula tan
for preciso
the SE of a es
sampleel bootstrap
mean.
3. VARIATION IN BOOTSTRAP DISTRIBUTIONS teórico (ex- the mean with n = 50, four samples, and the corresponding
r noted
unless = 10000, otherwise).obteniendo
The bootstrapla distribución
statistics de
comprise the haustivo)?. bootstrap distributions. Each bootstrap distribution is centered
1.3 Two-Sample
I We claimed
Bootstrap above that the bootstrap distribution usually pro- at the statistic x̄ from the corresponding sample rather than at
bootstrap
bootstrap Figure
distribution. Figure1.2 shows bootstrap distributions of ¿Con qué precisión la implementación de
vides useful information about the sampling distribution. We the population mean μ. The spreads and shapes of the bootstrap
Monte
elaborateCarlo
For a two-sample on thatse
bootstrap, weaproxima
now with a seriesadraw
independently oflavisual
trampa
bootstrap teó-one distributions vary a bit but not a lot.
examples,
samples rica?.
with
wherereplacement from each
things generally worksample,
well andand
threecompute a
with problems. We These observations inform what the bootstrap distributions
statistic
I that compares
address two the samples. For the Verizon data, we
questions:
La muestra original se elige
draw a sample of size 1664 from the ILEC data and 23 from the
aleatoriamente may be used for. The bootstrap does not provide a better esti-
de la población. mate of the population parameter, because the bootstrap means
CLEC data, • and
Howcompute
accuratethe difference in means x̄1 − x̄2bootstrap?
Figure 5. Bootstrap
. The Figura world.
2: LaThedistribución
bootstrap distribution
bootstrapis obtained by drawing
se obtiene al ex-repeat
is the theoretical (exhaustive) are centered at x̄, not μ. Similarly, quantiles of the boot-
I Las
bootstrap • How muestras
distribution
accurately de
(see online Bootstrap
doessupplement) se
the Monte isCarlo eligen
centered al azar
at the
the statistic
implementation ap-of interest
traer for each, and
muestras collectingde
repetidas those
una statistics. Thequantiles
estimación distribution
de la po- is cen
strap distributions are not useful for estimating of
observedde statistic; it is used
la muestra
proximate the for confidence
original.
theoretical intervals and standard
bootstrap? blación,
the samplingcomputando
distribution. La estadística
Instead, de interés
the bootstrap para
distributions
errors. It is skewed like the CLEC distribution; t intervals would
are useful
cada uno, fory estimating
la the spread
recopilación de and shape
esas of the sampling
estadísticas.
not be appropriate.
Both reflect random variation: 374 Statistics and the Undergraduate Curriculum
distribution.
For comparison, the permutation test pools the data and splits
The right column shows additional bootstrap distributions
the pooled• dataThe into two groups using
is sampling without re- from the
original sample chosen randomly for the first sample, with r = 1000 or r = 104 resamples. Using
Media de muestra: Tamaño de muestra grande
placement, before taking the difference in means. The sampling
population. more resamples reduces random Monte Carlo variation, but does
is consistent with the null hypothesis of no difference between
groups, and the distribution is centered at zero.

1.4 Bootstrap t-Distribution

Figure 1. Normal quantile plot of ILEC and CLEC repair times.
Figura 1: Diagrama de cuantiles
It is not surprising that t procedures are inaccurate for skewed
data with a sample of size 23, or for the difference when one
sample is that small. More surprising is how bad t confidence
intervals are for the larger sample, size 1664. To see this, we
bootstrap t statistics.

372 Statisticscada
Usamos and the Undergraduate
distribución Curriculum
para estimar ciertas
cosas sobre el muestreo r correspondiente.
∗
 
θ̂ − θ̂¯∗
1
Pr
I Error estándar: r −1 i=1 i i
I Intervalos de confianza: rango del 95 % me-
dio del distribución
 ∗  de arranque.
I sesgo: θ̂i − θ̂¯i∗

Bootstrap de dos muestras

Para un bootstrap de dos muestras, extraemos
muestras de bootstrap de forma independiente
con el reemplazo de cada muestra, y calcula-
mos una estadística que compara las muestras.

La distribución bootstrap se centra en la esta-

dística observada; Se utiliza para intervalos de
confianza y errores estándar. Está sesgada co- Figura 3:6. Distribución
Figure Bootstrap
Bootstrap distribution para nla= media,
for the mean, 50. The left n column
=is,50.
α La
shows
= P ( columna
δ̂ ∗ population
the
≤ q α ). deand
Then la four
izquierda muestra
samples. The la población
middle column shows the y
∗
mo la distribución CLEC; Los intervalos t no se- sampling distribution for X̄, and bootstrap distributions of X̄ for
cuatro muestras. La columna del medio muestra4 la distribución de muestreo each sample, with r = 10 4
. The right column
para shows
X̄ y more
las bootstrap distributions
distribuciones de
∗
for the first sample,
∗ three with r = 1000 and two with r = 10 . α/2 = P (θ̂ − θ̂ < qα/2 )
rían apropiados. arranque de X̄ para cada muestra, con r = 104.
≈ P (θ̂ − θ < qα/2 ) = P (θ̂ − qα/2 < θ ).
376 Statistics and the Undergraduate Curriculum Similarly for the other tail. The resulting confidence interval is
Intervalo percentil Intervalos de confianza
(θ̂ − q1−α/2 , θ̂ − qα/2 ) = (2θ̂ − Q1−α/2 , 2θ̂ − Qα/2 ), (3)
∗
Qα is the intervalo
whereNingún quantile of the
es muy preciso. Solo .son pre-
bootstrap distribution of θ̂
Referencias This interval is the mirror image of the bootstrap percentile
cisos de primer orden y son pobres
interval; it reaches as far above θ̂ as the bootstrap percentile
en mues-
[1] Tim C Hesterberg. What teachers should tras
interval pequeñas,
reaches below. Fortienden a ser
example, for demasiado
the CLEC mean, theestre-
know about the bootstrap: Resampling samplechos.
mean El error
is 16.5, theestándar de bootstrap
percentile interval es
is (10.1, 25.4) =dema-
16.5 + (−6.4, 8.9), and the reverse percentile interval n−1is 16.5 +
q
in the undergraduate statistics curriculum. (−8.9,siado
6.4) = pequeño, por10.1)
2 · 16.5 − (25.4, un =factor
(7.6, 22.9).n , por lo que
The American Statistician, 69(4):371–386, el intervalo
Reversing works wellt con bootstrap
for a pure translationSE esbut
family, demasiado
those
2015. are rare in practice.por
estrecho Moreese
common are cases like Figure 9, where
factor.
the spread of the bootstrap distribution depends on the statistic.
[2] Nathan Tintle, Beth L Chance, George W Then El intervalo
a good de percentiles
interval needs to be asymmetric noines the una buena al-
same direc-
tion asternativa,
the data, seemientras
Section 4.5.que trata percentile
The reverse mejor poblaciones
interval
Cobb, Allan J Rossman, Soma Roy, Todd
sesgadas,
is asymmetrical in themenos preciso
wrong direction! para muestras
Its coverage accuracy in pe-
Swanson, and Jill VanderStoep. Introduc- Figurequeñas.
10 is terrible. It also suffers from theexponenciales,
same small-sample el in-
Para poblaciones
tion to statistical investigations. John Wiley Figura
Figure 10. 4:Confidence
Intervalo de one-sided
interval confianza missde probabilidades
probabilities for nor- narrowness issues as the percentile interval.
tervalo
Hall (1992) percentil
called the es menos
bootstrap percentile preciso
interval que
“the wrongel inter-
& Sons, 2015. unilaterales
mal de fallas
and exponential para95%
populations. población
confidenceexponencial.
interval, the ideal
noncoverage is 2.5% on each side. The intervals are described at the pivot,valo t estándar
backward”; para
the reverse n ≤ 34.
percentile interval uses that same
beginning of Section 4.4. For the normal population noncoverage prob- wrong pivot in reverse. δ̂ is the wrong pivot because it is not even