Analysis of Extreme Values in Education
J Beirlant, Katholieke Universiteit Leuven, Heverlee, Belgium
G Dierckx, Hogeschool-Universiteit Brussel, Brussel, Belgium
K Luwel and P Onghena, Katholieke Universiteit Leuven, Heverlee, Belgium
ã 2010 Elsevier Ltd. All rights reserved.
Goals of Extreme Value Theory
Extreme value methodology has been constructed with
the specific aim to estimate extreme quantiles and/or
small exceedance probabilities corresponding to rare
events. Classical applications are the calculation of return
periods of floods or other catastrophic events, or the
prediction of large claims in insurance port-folios. In
educational studies applications are less prominent but
also here the statistical modeling of rare events can be of
interest. We present an application to the modeling of
reaction times in a numerosity test when applying the
choice/no-choice method. This method involves testing
participants under two types of conditions: (1) a choice
condition in which participants can freely choose which
strategy to use and (2) several no-choice conditions in
which participants are required to apply a given strategy
on all problems. Asking all participants to use a given
strategy on all trials in the no-choice conditions excludes
possible selection efforts and/or individual preferences,
resulting in unbiased estimates of the speed and accuracy
of the strategies under consideration. It also becomes
possible to investigate participants’ strategy adaptiveness
by comparing their actual strategy choices under the
choice condition with their ideal strategy choices based
on the no-choice performance data.
The application is taken from Luwel et al. (2003) where
the authors wanted to extend the applicability of the
choice/no-choice method toward the domain of numer-
osity judgment. They aimed at developing a reliable and
valid method for measuring the adaptiveness of numer-
osity judgment strategies, based on the procedure for
strategy identification in the individual response-time
patterns. Participants were instructed to determine, as
quickly and accurately as possible, different numerosities
of colored blocks that were presented in a 7 by 7 grid on a
computer screen. This task allows for two main strategies:
an addition strategy in which the different colored blocks
in the grid are added and a subtraction strategy in which
the number of empty squares is determined and then
subtracted from the total number of squared in the grid.
In contrast to the addition strategy which only involves
one solution step, the subtraction strategy is a two-
step strategy and therefore, cognitively more demanding.
The experiment was performed under three conditions:
one choice (C) condition and two different no-choice
(NC) conditions: (1) In the C-condition participants
were free to use either the addition or the subtraction
strategy, (2) in one NC-condition (i.e., the forced addition
or FA-condition) all trials had to be solved by means of the
addition strategy, and (3) in the other NC-condition (the
forced subtraction or FS-condition) the use of the subtrac-
tion strategy was required on all trials. Thirty-seven stu-
dents at the University of Leuven (Belgium) participated
in this study. Their mean age was 21 years with ages
ranging from 18 to 26 years. Each participant was tested
individually and ran three different sessions. The presen-
tation order of the different conditions was counterba-
lanced over participants with the important restriction
that C-condition was always presented first. As a conse-
quence all participants were randomly divided over one of
both presentation orders: C/FA/FS or C/FS/FA.
In Figure 1 the reaction times were plotted as a func-
tion of the number of blocks (from 1 to 49 blocks) for
each of the three conditions: (1) FA-condition; (2) FS-
condition and (3) C-condition. Some participants show
extreme long reaction times. Whereas some statisticians
would qualify these as outliers, the right skewness of
the reaction times distribution (as a function of the
block size) is always apparent and appears stronger for
larger block sizes in the FA- and C-conditions, and
for smaller block sizes in the FS-condition. Extreme
value methodology provides models that allow one to
model this right-skewness and the largest reaction times.
More specifically, it tries to answer questions such as: what
is, for a given number of blocks, the extreme reaction time
level that occurs only once in 100 trials.
The Extreme Value Modeling Approach
The Basic Model
Here we present the extreme value theory (EVT) approach
for pure random samples X1, X2, . . . , Xn. The population
distribution is assumed to satisfy the domain of attraction
condition which expresses that for large sample sizes n the
statistical behavior of the maximum of such samples can be
well approximated by a nondegenerate distribution which
then is shown to be necessarily an extreme value distribu-
tion with cumulative distribution function
25
26 Statistics

60 60

50 50
Reaction time

Reaction time
40 40

30 30

20 20

10 10

0 0
0 10 20 30 40 50 0 10 20 30 40 50
(a) Number of blocks (b) Number of blocks

50

40
Reaction time

30

20

10

0
0 10 20 30 40 50
(c) Number of blocks
Figure 1 The reaction time (RT) as a function of the number of blocks for 37 participants: (a) addition; (b) substraction, and (c) choice.

exp½ð1 þ gyÞ1=g  ½1 Tail Estimation

The parameter g, called the extreme value index (EVI),
characterizes the tail decay of the underlying distribution
F : a positive value of g corresponds to heavy-tailed or
Pareto-type distributions for which the density decays as a
power law, while g ¼ 0 typically indicates an exponentially
decreasing tail. Finally, g < 0 indicates a distribution F with
a finite right endpoint. This general condition is quite
broad so that it does not really lead to restrictions in
practice.
It can be shown that this framework is equivalent to
assuming the following:
The POT–GPD assumption
For large enough threshold values t, the excesses, or peaks over
threshold (POTs) X  t (with X > t) approximately follow a
generalized Pareto distribution (GPD) with cumulative distribu-
tion function 1  ð1 þ sg yÞ1=g .
In case g ¼ 0 the GPD is to be interpreted as the
exponential distribution with cumulative distribution
function 1  ey/s for y > 0. This discussion makes it
clear that the goal of tail estimation is carried out under a
nonstandard model which involves the choice of a thresh-
old t. Of course, the estimation of the EVI parameter g is
an important step toward the goal of tail estimation.
The POT approach to estimate g and s > 0 simply con-
sists of using for instance the maximum likelihood
approach when fitting the GPD to the excesses X  t for
those data X which are larger than t. Quite often, the
maximum likelihood estimation procedure leads to appro-
priate estimates g^k and s^k which show a stable behavior as
a function of the threshold t. We refer to this method as
the GP-POT approach.
Most commonly, t is chosen as one of data points itself,
say as the (k + 1)-largest observation, and then one lets k
vary between 2 and the full sample size n. In Figure 2 the
POT estimators for the extreme value index of the RT
values in our case study are shown for each of the three
conditions as a function of k ¼ 2, . . . , 37 with the number of
blocks equal to 30. Remark that in case of the addition
strategy the estimated values are clearly positive, while for
the other two conditions values around zero are predomi-
nant. In Figure 3 the POT estimators of g obtained with
k ¼ 17 are shown for each condition as a function of the
number of blocks. Comparing with Figure 1 we remark that
the appearance of high outliers in reaction times leads to
predominant positive EVI estimates. Figure 4 shows the
corresponding maximum likelihood estimates of the scale
parameter s. While the POT estimates of the extreme value
index do not seem to show trends with the number of blocks,
the s estimates grow linearly for the addition strategy, while
 Analysis of Extreme Values in Education 27

1.0 1.0
POT estimator for g

0.5

POT estimator for g

0.5

0.0
0.0
−0.5
−0.5
−1.0

−1.0
10 20 30 10 20 30
(a) k (b) k

1.0
POT estimator for g

0.5

0.0

−0.5

−1.0
10 20 30
(c) k
Figure 2 POT estimator for the extreme value index of the reaction times as a function of k ¼ 2.37 (37 participants) for 30 blocks for: (a)
addition, (b) subtraction, and (c) choice.

2.0 2.0

1.5 1.5
POT estimator for g

POT estimator for g

1.0 1.0

0.5 0.5

0.0 0.0
−0.5 −0.5
−1.0 −1.0
10 20 30 40 50 10 20 30 40 50
(a) Number of blocks (b) Number of blocks

2.0

1.5
POT estimator for g

1.0

0.5

0.0

−0.5

−1.0
0 10 20 30 40 50
(c) Number of blocks
Figure 3 POT estimator of g using the k ¼ 17 largest RTs out of 37 observations for the extreme value index of the reaction times
as a function of the number of blocks for: (a) addition, (b) subtraction, and (c) choice.
28 Statistics

10 10

8
POT estimator for s

POT estimator for s

6 6

4 4

2 2

0 0
10 20 30 40 50 10 20 30 40 50
(a) Number of blocks (b) Number of blocks

10

8
POT estimator for s

0
0 10 20 30 40 50
(c) Number of blocks
Figure 4 POT estimator of s using the k ¼ 17 largest RTs out of 37 observations for the extreme value index of the reaction times
as a function of the number of blocks for: (a) addition, (b) subtraction, and (c) choice.

they decline linearly for the subtraction strategy. For the Pareto-Type Distributions
C-condition the scale estimates again follow the pattern of
In the special case g > 0, the underlying distribution is
the original data plot as discussed in Figure 1 with a change
member of the class of Pareto-type distributions. In this
point at about 25 blocks.
case the POT–GPD assumption is equivalent to the
Given estimators g^k and s^k , how can one now estimate
assumption that the excesses X/t (rather than X  t,
the exceedance probability px of obtaining an observation
X > t) for large enough t approximately follow a simple
larger than a given large value x? To this end the condi-
Pareto distribution (PD) with cumulative distribution
tional probability of an excess being larger than x (where
function 1  y1/g, y > 1. Again, using the (k + 1)-largest
we restrict attention to the excess data and which is
observation as the threshold t, this leads to the following
computed along the postulated GPD excess model) is to
tail estimators that can be used in case g > 0:
be multiplied with the probability that an observation is
larger than the threshold. For the latter we assume that k x 1=g^
the threshold is situated deep enough in the sample so p^þ
x ¼ ½4
n t
that the sample proportion can be used here as an esti-
mate. This fraction equals k/n if the threshold t equals the Remark that this is completely similar to [2] with the
(k + 1)th largest observation. This leads to GPD replaced by the simple PD and the excess x  t
k x  t 1=g^ replaced by x/t. Also
p^x ¼ 1 þ g^ ½2
n s^  g^
k
Moreover, an extreme quantile qp which denotes the q^þ
p ¼t ½5
np
outcome level which is exceeded with a chosen probability p
(typically smaller than 1/n) can be estimated by equat-
Here maximum likelihood estimation fitting the strict PD
ing [2] to p and solving for x:
based on the excesses X/t (X > t) leads to the Hill (1975)
ðk=ðnpÞÞg^  1 estimator which equals the average of the logarithms of
q^p ¼ t þ s
^ ½3
g^ the excesses.
 Analysis of Extreme Values in Education 29

The use of the GPD in the general case g 2 R, respec-
tively the PD in case g > 0, as a consequence of theoretical
probabilistic results concerning the distribution of max-
ima, should of course be validated in practice. One can
validate the use of the fitted parametric models through
goodness-of-fit methods. For this purpose for instance
quantile plotting techniques can be used.
Quantile Plotting
An alternative view to the above material, which allows
one to support an extreme value analysis graphically, con-
sists of plotting the ordered data against the corresponding
theoretical quantiles of the postulated model. For instance
in the specific case of Pareto-type distributions (g > 0), the
model can be evaluated through the ultimate linearity of
a Pareto quantile plot where the logarithm of the j th

smallest observation is plotted against thestandard expo-
nential theoretical quantile log 1  nþ1 j
ð1  j  nÞ.
In Figure 5 the Pareto quantile plots for the RTs
to count 30 blocks at each of the three conditions are
shown. These plots confirm the Pareto-type hypothesis
for the corresponding reaction times as indeed above a
certain level linearity becomes apparent. In Figure 6 for
each condition the Hill estimators for g > 0 at k ¼ 17 are
given as a function of the block size.
From Figures 1–6 the problems arising with application
of extreme value methods become apparent. First the
choice of k is quite important. The higher the value k (or,
the lower the threshold t) the higher the bias of the estima-
tor since the models are only valid for threshold values
approaching the endpoint of the underlying distribution.
The smaller the k the larger the variance of the estimators
because then these are based on fewer data. Also theoreti-
cally one can show that optimal k values are different for
different estimators (such as the GP-POT and the Hill
estimator). Also when only few data are available, such as
is the case here with only 37 observations per number of
blocks and per condition, the methods can be quite sensitive
and yield different conclusions. In fact, the GP-POT
approach here indicates a more nuanced picture con-
cerning the signs and values of the EVI estimates, while
the positive Hill estimates taken at k ¼ 17 indicate EVI
values around 0.2 for all numbers of blocks and all condi-
tions. Also the GP-POT approach offers a more flexible
framework offering two parameters in order to model the
exceedances. In Figure 7 we offer the GP quantile plots for
the RTs corresponding to the counts of 30 blocks based on
k ¼ 17 exceedances for each of the three conditions. From

3.0
4.0
2.8
3.5
2.6
log Xn−j + 1,n
log Xn−j + 1,n

3.0 2.4

2.2
2.5
2.0
2.0
1.8
0 1 2 3 0 1 2 3
(a) log((n +1)/j) (b) log((n +1)/j )

3.2

3.0

2.8
log Xn−j + 1,n

2.6

2.4

2.2

2.0

0 1 2 3
(c) log((n+1)/j )
Figure 5 Pareto QQ plots for the reaction times to count 30 blocks. The data are based on results for 37 participants: (a) addition,
(b) subtraction, and (c) choice.
30 Statistics

1.0 1.0

0.8 0.8

Hill estimator
Hill estimator

0.6 0.6

0.4 0.4

0.2 0.2

0.0 0.0
0 10 20 30 40 50 0 10 20 30 40 50
(a) Number of blocks (b) Number of blocks

1.0

0.8
Hill estimator

0.6

0.4

0.2

0.0
0 10 20 30 40 50
(c) Number of blocks
Figure 6 Hill estimator values when using the k ¼ 17 largest RTs out of 37 observations for the extreme value index of the reaction
times as a function of the number of blocks: (a) addition, (b) subtraction, and (c) choice.
Quantiles of generalized Pareto
Quantiles of generalized Pareto

20
6

15
4
10

2
5

0 0
0 10 20 30 40 50 0 2 4 6 8 10
(a) Excesses over treshold of reaction time (b) Excesses over treshold of reaction time

8
Quantiles of generalized pareto

0
0 2 4 6 8 10 12
(c) Excesses over treshold of reaction time
Figure 7 Generalized Pareto QQ plots for the excesses of the reaction times to count 30 blocks over the threshold corresponding to
k ¼ 17. The estimated g and s values for k ¼ 17 are used: (a) addition ðg^¼ 0:395; s^¼ 4:186Þ, (b) subtraction ðg^¼ 0:131; s^¼ 2:048Þ,
and (c) choice ðg^¼ 0:125; s^¼ 2:214Þ.
 Analysis of Extreme Values in Education 31

10 using the GPD as an excess model with s substituted by

the estimated regression function allows one to estimate a
8 high quantile curve as a function of the number of blocks
POT estimator for σ

for the addition condition. In Figure 8 the estimated

6 linear s function is plotted with the original simple s
estimates as shown in Figure 4, next to the extreme
4
quantile curve [3] corresponding with p ¼ 0.01 with the
2 original data under the addition strategy.

0
(a) 10 20 30 40 50 A Literature Review

60 In the last few decades extreme value methods in regres-

sion, multivariate, and time series settings have been
50 developed and studied. More refined models designed
Reaction time

40 to account for temporal dependence and multivariate

data in extreme value analysis include Poisson process
30
models, Markov chain models, and extreme value copulas.
20 Another important issue is the choice of the threshold
defining the excesses to which to fit the (generalized) PD.
10
Recently developed techniques offer a solution through
0 stabilizing the estimates over a large range of thresholds.
0 10 20 30 40 50
(b) Number of blocks See also: Exploratory Data Analysis; Goodness-of-Fit
Figure 8 (a) The estimated linear s-curve and (b) the 0.99 Testing; Nonparametric Statistical Methods; Order
quantile curve based on regression modeling with the original Statistics.
data for the addition strategy data.

this we infer that the GP model appears accurate apart from
the largest observation for each of the cases.
Extensions
Our case study motivates the use of regression models
which allow one to retain the flexible modeling of the
Bibliography
Davison, A. C. and Smith, R. L. (1990). Models for exceedances over
high thresholds (with comments). Journal of the Royal Statistical
Society, Series B 52, 393–442.
Hill, B. M. (1975). A simple general approach to inference about the tail
of a distribution. Annals of Statistics 3, 1163–1174.
Luwel, K., Verschaffel, L., Onghena, P., and De Corte, E. (2003).
Analysing the adaptiveness of strategy choices using the choice/no-
choice method: The case of numerosity judgement. European
Journal of Cognitive Psychology 15, 511–537.

underlying distribution tails as proposed by the maximum

domain of attraction framework. This then allows one to
use more data points in order to estimate only a limited
number of parameters, and hence keep a decent number
of degrees of freedom. Davison and Smith (1990) provide
an interesting regression modeling approach within the
GP-POT modeling framework.
In the case of the addition strategy, we used maximum
likelihood estimation to fit the GP distribution with para-
meters g and s ¼ s0 + s1  (the number of blocks). For
each number of blocks the 17 largest data points were
used so that we used 49  17 excess data. This leads to the
estimates g^¼ 0:096; s^0 ¼ 0:167, and s^1 ¼ 0:098. Then
Further Reading
Beirlant, J., Goegebeur, Y., Segers, J., and Teugels, J. (2004). Statistics
of Extremes – Theory and Applications. Wiley Series in Probability and
Statistics. New York: Wiley.
Coles, S. G. (2001). An Introduction to Statistical Modelling of Extreme
Values. London: Springer.
de Haan, L. and Ferreira, A. (2006). Extreme Value Theory: An
Introduction. New York: Springer.
Reiss, R. D. and Thomas, M. (2001). Statistical Analysis of Extreme
Values. Basel: Birkhäuser Verlag.
Weissman, I. (1978). Estimation of parameters and large quantiles
based on the k-largest observations. Journal of the American
Statistical Association 73, 812–815.