Chapter 6: Order Statistics

Random variable are denoted by uppercase (mayúscula) letters (like X, Y, Z , etc)

mathematical variables are denoted by lowercase letters (minúscula) (like x, x, z , etc )
observations are denoted by lowercase (minúscula) letters (like x, x, z , etc )
parameters of the distribution are usually denote by greek letters (such as α, β, θ, η, etc)
Statistical Inference, August 21, 2020

cdf: cumulative distribution function (función de distribución acumulativa)

pdf: probability density function or pdf or density function (función de densidad de
probabilidad)
f X (·): the pdf of X. The suffix (subscript) indicates that f X (·) is the density of X
d
X  Y means that X and Y have identical distribution.
The set {x : f X (x) > 0} is called support set.

The set of random variables X1 , X2 , . . . , X n is said to be a random sample of size n from

a population with density function f X (x) if

1. X1 , X2 , . . . , X n are independent,

2. X1 , X2 , . . . , X n have identical detrition which is same as that of X.

Consequently, the joint pdf has the form

f X1 ,X2 ,...,X n (x1 , x2 , . . . , x n ),  f X (x1 ) f X (x2 ) · · · f X (x n ).

Let X1 , X2 , . . . , X n denote a random sample from a population with continuous cdf

FX . First let FX be continuous, so that the probability is zero that any two or more of
these random variables have equal magnitudes. In this situation there exists a unique
ordered arrangement within the sample. Suppose that X(1) denotes the smallest of the set
X1 , X2 , . . . , X n ; X(2) denotes the second smallest; ... and X(n) denotes the largest. That is

X(1)  min{X1 , X2 , . . . , X n },

X(2)  min {X1 , X2 , . . . , X n } − {X(1) } ,



X(3)  min {X1 , X2 , . . . , X n } − {X(1) , X(2) } , . . . ,



1
 X(n−1)  min {X1 , X2 , . . . , X n } − {X(1) , X(2) , . . . , X(n−2) } ,


X(n)  min {X1 , X2 , . . . , X n } − {X(1) , X(2) , . . . , X(n−2) , X(n−1) }



 max{X1 , X2 , . . . , X n }.

Then
X(1) < X(2) < · · · X(n)
Statistical Inference, August 21, 2020

denotes the original random sample after arrangement in increasing order of magnitude,
and these are collectively termed the order statistics of the random sample X1 , X2 , . . . , X n .
The rth smallest, 1 ≤ r ≤ n, of the ordered X ’s, X(r) , is called the rth-order statistic. Some
authors use alternative notation

X1:n < X2:n < · · · < X n:n

to denote the order statistics. Further, by taking Yi  X(i) , i  1, . . . , n one can also define
order statistics as
Y1 < Y2 < · · · < Yn .

Some familiar applications of order statistics, which are obvious on reflection, are as
follows:

1. X(n) , the maximum (largest) value in the sample, is of interest in the study of foods
and other extreme meteorological phenomena.

2. X(1) , the minimum (smallest) value, is useful for phenomena where, for example,
the strength of a chain depends on the weakest link.

3. The sample median, defined as X([n+1]/2) for n odd and any number between X(n/2)
and X(n/2+1) , for n even, is a measure of location and an estimate of the population
central tendency.

4. The sample midrange, defined as (X(1) +X(n) )/2, is also a measure of central tendency.

5. The sample range X(n) − X(1) is a measure of dispersion.

6. In some experiments, the sampling process ceases after collecting r of the observa-
tions. For example, in life-testing electric light bulbs, one may start with a group of
n bulbs but stop taking observations after the rth bulb burns out. Then information
is available only on the first r ordered ”lifetimes” X(1) < X(2) < · · · X(r) , where r ≤ n.
This type of data is often referred to as censored data.

2
 7. Order statistics are used to study outliers or extreme observations, e.g., when so-
called dirty data are suspected.

8. Order statistics are used extensively in statistical inference.

For example, if a random sample of five light bulbs is tested, the observed failure times
might be (in months) (x1 , . . . , x5 )  (5, 11, 4, 100, 17). Now, the actual observations would
Statistical Inference, August 21, 2020

have taken place in the order x 3  4, x1  5, x2  11, x5  17, and x4  100. But the
"ordered" random sample in this case is (y1 , . . . , y5 )  (4, 5, 11, 17, 100).
The joint distribution of the ordered variables (that is Y1 , . . . , Yn ) is not the same as the
joint distribution of the unordered variables (that is X1 , X2 , . . . , X n ). Note that because of
ordering

1. the random variables Y1 , . . . , Yn are not independent,

2. the except is same cases they are not identically distribution,

3. the random variable Yj and Yj , 1 ≤ i , j ≤ n may have different distribution and

different from that of X.

The joint density of ordered statistics Y1 , . . . , Yn are obtained by applying transforma-

tion on X1 , X2 , . . . , X n . Let us consider the case of three variables, that is n  3. In this
case
f X1 ,X2 ,X3 (x 1 , x2 , x3 ),  f X (x1 ) f X (x2 ) f X (x 3 ),

where a < x 1 , x2 , x3 < b, and the transformation

Y1  X(1)  min{X1 , X2 , X3 }, Y2  min{{X1 , X2 , X3 } − {X(1) }}, Y3  max{X1 , X2 , X3 }.

Let us do some mathematics to compute inverse transformation. Note that whenever I do

mathematics I use lowercase letters. There are six possible inverse transformations

x 1  y1 , x 2  y2 , x 3  y3 ,

x 1  y2 , x 2  y1 , x 3  y3 ,

x 1  y1 , x 2  y3 , x 3  y2 ,

x 1  y3 , x 2  y2 , x 3  y1 ,

x 1  y2 , x 2  y3 , x 3  y1 ,

3
 x 1  y3 , x 2  y1 , x 3  y2 ,
One can check that for the transformation of n variables there are n! inverse transforma-
tions. Further, one can check that

J(x1 , x2 , x3 → y1 , y2 , y3 )  1,

J(x 1 , x2 , x3 → y2 , y1 , y3 )  1,
J(x 1 , x2 , x3 → y1 , y3 , y2 )  1,
Statistical Inference, August 21, 2020

J(x 1 , x2 , x3 → y3 , y2 , y1 )  1,
J(x 1 , x2 , x3 → y2 , y3 , y1 )  1,
J(x1 , x2 , x3 → y3 , y1 , y2 )  1.
Now, the joint density of Y1 , Y2 , Y3 is derived as

fY1 ,Y2 ,Y3 (y1 , y2 , y3 )  f X (y1 ) f X (y2 ) f X (y3 )(1) + f X (y2 ) f X (y1 ) f X (y3 )(1)
+ f X (y1 ) f X (y3 ) f X (y2 )(1) + f X (y3 ) f X (y2 ) f X (y1 )(1)
+ f X (y2 ) f X (y3 ) f X (y1 )(1) + f X (y3 ) f X (y1 ) f X (y2 )(1)
 6 f X (y1 ) f X (y2 ) f X (y3 )
 3! f X (y1 ) f X (y2 ) f X (y3 ), a < y1 < y2 < y3 < b

where the last line has been obtained because the product is commutative.
Note that we can write the joint density of Y1 , Y2 , Y3 as

(function of y1 only) × (function of y2 only) × (function of y2 only).

But only the factorization of the joint density is not sufficient for independence. You have
to look at the support set of fY1 ,Y2 ,Y3 (y1 , y2 , y3 ). If the support set of the joint density is not
a cartesian product, the random variables are not independent. In this case the support
set of fY1 ,Y2 ,Y3 (y1 , y2 , y3 ) can not be written as cartesian product and therefore Y1 , Y2 , Y3
are not independent.
Now you can easily state the joint density of Y1 , . . . , Yn . This is Theorem 6.5.1 of the
book.
If X1 , X2 , . . . , X n is a random sample from a population with continuous pdf f X (x),
then the joint pdf of the order statistics Y1 , . . . , Yn is

fY1 ,Y2 ,...,Yn (y1 , y2 , . . . , y n ),  n! f X (y1 ) f X (y2 ) · · · f X (y n ),

where a < y1 < y2 , · · · < y n < b.