See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/305976319

‘Wald's Lemma' for sums of order statistics of i.i.d. random variables

Article  in  Advances in Applied Probability · September 1991

DOI: 10.1017/S0001867800023764

CITATIONS READS
22 66

2 authors:

F. Thomas Bruss James B. Robertson

Université Libre de Bruxelles University of California, Santa Barbara
116 PUBLICATIONS   1,293 CITATIONS    34 PUBLICATIONS   350 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Algorithms and combinatorial questions View project

Optimal sequential selection View project

All content following this page was uploaded by F. Thomas Bruss on 25 August 2016.

The user has requested enhancement of the downloaded file.

'Wald's Lemma' for Sums of Order Statistics of i.i.d. Random Variables
Author(s): F. Thomas Bruss and James B. Robertson
Source: Advances in Applied Probability, Vol. 23, No. 3 (Sep., 1991), pp. 612-623
Published by: Applied Probability Trust
Stable URL: http://www.jstor.org/stable/1427625
Accessed: 23-08-2016 13:32 UTC

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
http://about.jstor.org/terms

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted
digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about
JSTOR, please contact support@jstor.org.

Applied Probability Trust is collaborating with JSTOR to digitize, preserve and extend access to Advances
in Applied Probability

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
 Adv. Appl. Prob. 23, 612-623 (1991)
Printed in N. Ireland

( Applied Probability Trust 1991

'WALD'S LEMMA' FOR SUMS OF ORDER STATISTICS OF

I.I.D. RANDOM VARIABLES

F. THOMAS BRUSS,* University of California, Los Angeles

JAMES B. ROBERTSON,** University of California, Santa Barbara

Abstract

Let X1, X2, , X,, be positive i.i.d. random variables with known distribution
function having a finite mean. For a given s > 0 we define N,, = N(n, s) to be the
largest number k such that the sum of the smallest k Xs does not exceed s, and
M, = M(n, s) to be the largest number k such that the sum of the largest k X's does
not exceed s. This paper studies the precise and asymptotic behaviour of E(N,),
E(M,), N,, M,, and the corresponding 'stopped' order statistics X(Nn) and
X(n --M,,+) as n -- oo, both for fixed s, and where s = s, is an increasing function of n.
STOPPING TIMES; EMPIRICAL DISTRIBUTION FUNCTION; PROPHET INEQUALITY

1. Introduction

Let us first motivate the research by the following problem.

The baker's problem. As a baker opens his shop in the morning he finds n clients in
front of the door, all waiting for fresh breakfast rolls. The baker has a supply of s
rolls. If the customers' demands X1, - - - , X,, are i.i.d. with common distribution
function F, what is the expected number of demands he can serve completely(!) if
he serves the smallest demands first, the demands in order of arrival (FCFS), or the
largest demands first?

This formulation is easy to remember and catches the essential nature of the
problem, although we confine our attention to continuous Xi's. More interesting
forms of such questions of 'stopping times' arise, for example, in connection with
resource-dependent branching processes and their extinction probabilities as a
function of the social behaviour of particles (Bruss (1983)). Other examples where
these problems become important include strategies to select intervals under a sum
constraint and a no-recall condition as studied by Samuels and Steele (1981) and
Coffman et al. (1987). However, in what follows we shall confine our attention to
the theoretical part of the problem.
Received 17 July 1989; revision received 29 May 1990.
* Present address: Departement Wiskunde, F 733, Vrije Universiteit Brussel, B-1050 Brussels,
Belgium.
** Postal address: Department of Statistics and Applied Probability, University of California, Santa
Barbara, CA 93106, USA.
Research supported by the grant 'Automatic Decision Strategies' (No. ST2J-0227-C) of the European
Community.

612

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
'Wald's lemma' for sums of order statistics 613

The intrinsic difficulty of the problem of the

largest demands first is, of course, the fact that we
variables which, by their ordering, are no longer
The demand of the nth customer is denoted by
paper that {X1, X2, - - -, X,} are positive i.i.d
distribution function F having a finite mean.
increasing order statistics of X1, X2, -- , -X,. An
random variables

(1.1) N, = N(n, s) = max {k - n XI,, + X2,n + + Xk,n , S),

N,,=O if XI,,,>s,
M,= M(n, s) = max {k-n: +Xn+-1, n-, +-- + Xn -k+1,n S),
(1.2)
M,,=O0 if X,,n,>s

is to define them via the empirical distribution function F, associated with the

sample X1, X2,- - , X,,. Indeed if F is continuous, then they are determined by
(1.3) N,, = nF,(X,,,),
(1.4) M, = n(1 - F (X,,nM,)).
Here we set Xo,,, = 0. Moreover, for general F we have
xkn S

(1.5) N,, < k o x

n

(1.6) M, < k x dF,,(x) >-, 1- Xn-k+l,n

It is tempting to use the actual distribution function F in place of the sample

distribution F,. Thus for a given parameter a (replacing sin) we define parameters r
and v (replacing XN,, and N,,n respectively) by the equations

(1.7) x dF(x) = a

(1.8) v = F(r).
Our aim is to find conditions on the distribution function F under which this

replacement is valid and to assess the approximations E(XN,,,) - r and E(N,,n) - v

In Section 2 we consider the case where s,,/n tends to a finite limit as n --> oo.
remarked by Coffman et al. (1987), convergence in distribution, N,,In --> v
relatively easy application of the law of large numbers. Under certain continuit
conditions on F we strengthen this to almost sure convergence and also show th

Xu,--, -r a.s. as n -- oo (implying that E(N,,)/n -- F(r) as n - oo).

In Section 3 we consider the case where s is fixed as n -* oo. This is more delica
and our proof does still not cover all distributions, but the extension of the nic

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
614 F. THOMAS BRUSS AND JAMES B. ROBERTSON

result of Coffman et al. is again significant. Th

(F(x) = Ax in a neighborhood of zero) and here
through a more general analysis. In addition to s
suppose that for some 0 < A < 1, lim supx,0o
sumptions we show that N, -- nF(t,) where tn is
prove that E(Nn) - nF(tn), and even E(Nn) -5 nF
In Section 4 we interpret the results and make
we show there the analogy between Wald's lem

2. The large-sample, large-supply case (n -- oo,

In this section we allow the supply s to de

limn,,ssn/n = a. We consider first the Nn-prob

Xk'S, i.e.
different = fox
treatment. dF(x). Each of the cases a

Theorem 2.1. If lim,~. s,/n - M, then

lim Nn/n = 1 a.s., lim E(N,)/n =
n---oo n---oo

Proof. (i) a> y

numbers S,nn -
eventually and E(N
(ii) a = y: Set Sk =

XNN + 1,,n, if Sn > sn,,

= , 9if Sn Sn.
We now show that if a = then

(2.1) n N, >=n - - - 1. Yn

The first inequality is clear by the definition of Nn. The second inequality is also
clear if S,, 5 s, since then N, = n and the right-hand side is less than n. Therefore we
suppose that S,, > s,. Then N, + 1 < n and since the Xk,n are increasing in k we have

+ Sn - SN+-S Sn - Sn
XNnN+1,n XNn+ ,n
thus

Sn - Sn

XN+1,n

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
'Wald's lemma' for sums of order statistics 615

We next note that if S,, > s, we have (again because

XNn+l,n
N, +1 > > Y + o(1).

This shows that y, is bounded away from 0 as n -

in (2.1) by n we have

Sn Sn
N, n n 1
1~ =21-
n yn n - - ----- 1 a

since s,/n-- p and since by

bounded away from 0. We als
bounded by 0 and 1). This com

Finally we suppose that a < K

results.

Theorem 2.2. Suppose that F

function on [a, b] where 0

Y = fx dF(x), let r e (a, b) be defined by the equation a = fTx dF(x). Let s,, s2,
be a sequence
distribution such F.that
function limn_
Let N, sn/n by
be defined = a, and
(1.1). let X1, X2, " be i.i.d. with
Then

1 1

lim
n---o n--=i n n-= XNn
n

Proof. Since ar < K, there e

(2.2) x dF(x) > a = lim

n-o n

The law of large numbers implies that

1n m
(2.3)
n k=1
- E Xkl{X

where 1{A} is the i

that XN,, - m for s

generality, that F i
otherwise we may t

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
616 F. THOMAS BRUSS AND JAMES B. ROBERTSON

Let L = lim sup,,. XN,,,. It follows that

f L XNn'n
x dF(x) = lim sup x dF(x)

= lim sup XNnn (F(Xn,,) - F(x)) dx + aF(XN,,,)

= lim sup (F,(XN,,) - F,,(x)) dx + aF,(Xxn,n))

XNnn

= lim sup x dFn,(x)

limsup-= x dF(x).
n ---noo fl a

Here the third equality above uses the Glivenko-Cantelli theorem which a
F, converges uniformly to F with probability 1, and we have used the cont

throughout. The above inequality clearly implies that lim sup X,,,=X
similar way we can show that r - lim inf,,n XN,+l,n.
We next show that lim,,~o (XN+1,,, - XN,,n) =0 a.s., which will com
proof of the first equation of the theorem. Let e > 0 and let A1, A2, - - -
partition of [a, b] into intervals of positive lengths less that e/2. Since F
increasing on each Aj, we have P(XeAj)>0 for all ljr. Then
Borel-Cantelli lemma, P(Xn e Ai i.o.) = 1. Thus for almost all w with resp
there exists an integer I(o) such that for all n I(o) and all 1 4 k - n, Xk+l
Xk,n(o) < E (since Xk,n(w) and Xk+l,n(() are in the same or adjacent
(XN,+l,n - XN,,n)(Ow) = XN,(w)+l,n(w) - XN,(w),n(W) < E, implying lim,,,o
XN,,,)(w) = 0 as desired.
The second equation of the theorem follows from the first equation, E
(1.3), and the Glivenko-Cantelli theorem.
The third equation of the theorem follows at once from the second equa
N,/n is uniformly bounded by 1. This completes the proof.
For bounded random variables the M,-problem can be treated in the sam
the N,-problem. The role of r is now played by 0, defined by fbx dF(x)
under the same conditions as in Theorem 2.2 (except that b <oo), we
following result.
Theorem 2.3. We have

1 1

n- n--cl n n-~c n
lim XnM
The proof goes throug
unlike the N,-problem,
restriction, and we have

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
 'Wald's lemma' for sums of order statistics 617

3. The large-sample, small-supply case (n - oo, s fixe

In this section we fix s, let n - 0oo, and study the qu

N, and E(N,) are asymptotic to nF(t,) as n --oo, w
equation n fn x dF(x) = s. The corresponding problem

as we clearly have M,,-->[s/m], where m--oo

{x:F(x)> 1} and [-] is the greatest integer part fun
inf {x: F(x) > 0} > 0 is similarly easy. We therefore s
function is continuous and strictly increasing in a nei
Coffman et al. (1987) have shown the above r
neighborhood of zero where A, a > 0 are fixed param

for more general F, is similar to theirs. If t E Rj

sample, we set

(3.1) , = #{k:Xk t} = nF,(t),

(3.2) = Xl{X t} = n x dFl(x). k=1O

Thus St is the sum of t

were proved by Coffma
Lemma 3.1.

( 2
(3.3) P[,t - nF(t)l z a] <- 2 exp 4nF(t)) if 0 5 a 5 2n
t[a 2
(3.4) P $ - nf xdF(x) a 2 exp
We next state some preliminary results

Lemma 3.2. Fix F, s, n. For E > 0 let

also implies that sl/s2 > 1 - E). For i = 1,

(3.5) n x dF(x) = si.

Let t,,, and ,, be defined as in (3.1) and (3.2) w

Define the sets

Di = Di,, = [I|,, - nF(t;,,)l = (nF(ti,,))'],

Ei = Ei,, = [1Sti, - Sil - ti,,(nF(ti,,))-].
Then if nF(t,,,) > 1,

(3.6) P(D,) _ 2 exp (-14(nF(ti,,))2r-1)

(3.7) P(E,) - 2 exp (-(nF(t,,))2r-1).

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
618 F. THOMAS BRUSS AND JAMES B. ROBERTSON

Further we have

(3.8) nti,nF(ti,n) Si,

(3.9) F(t,)F(t,,,)
sS1?
Finally for all 0 < A < 1

(3.10) nti,,,F(ti,,n) - si
Proof. If nF(t,,,) > 1 and r < 1, we have

(nF(ti,,))' < 2nF(ti,,), ti,,,(nF(ti,,))' - 4nti,,F(ti,,).

Thus the conditions for Lemma 3.1 are satisfied. Equations (3.6) and (3.
follow from Equations (3.3) and (3.4).
Using (3.5) it is straightforward to see that

nt,,,F(t,,,) Ln x dF(x) = si.

This proves Equation (3.8). Using it with i = 1 we obtain

s2-S1= nf xdF(x) - n xdF(x)

o2,n ftl,n
> ntl,,(F(t2,n) - f(tl,n))

- 1 (F(t2,n)- F(t-l,))
F(t2,n)
= S1l S
F(t1,,)

Equation (3.9) follows from this.

Suppose now that 0 < A <1. We have from (3.5)

nF(ti,,) = (t)
fix dF(x)
F(ti,,)si

f x dF(x)

S F(tin)Si

Si

- Att,,,(1 - F(At;,,)/F(ti,,))
This proves Equation (3.10) and completes the proof of the lemma.

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
'Wald's lemma' for sums of order statistics 619

We are now ready for the main result of this section.

Theorem 3.3. In addition to the notation of Lemma 3.2

and N,:

nf x dF(x) = s, N, = nF,(X,,n).

We suppose that for some 0 < A < 1

F(Qx)
(3.11) lim sup < 1.
x-.o+ F(x)
Then

(3.12) N, - nF(t,) in probability as n - oo.

Proof. Equation (3.5) implies that ti,,--- 0 as n-- oo. By (3.11) and
nti,,F(t~,,) is bounded in n. Thus since r < 1, we have ti,n(nF(ti,n))r' 0 as n -,
us choose n sufficiently large so that tl,n(nF(t,,))r <l(s - sl). Then
complement of E1 we have S1 <s. This implies that Nt, - 5Nn. In the complem

D1 we have Nt, -nF(tl,n) - (nF(tl,,))r. Thus in the complement of D1 U

inequalities N, - Nt,,? nF(t,n) - (nF(tl,,))r must hold. Using (3.9) yields
N,, N,, F(tl,,,)
nF(t.) nF(tI,n) F(t.)

s
= (1 - (nF(tI,n))r- ) s.
Equation (3.8) implies that nF(t,n,)--oo and n - oo. We may therefore take n
sufficiently large so that

(1 - (nF(tI,))r-n) s 1 - E.
Thus

N. >
nF(t,) -
Similarly by considering D2 and E2 we may take n sufficiently large so that in the

complement of D2 U E2 we have N,, - N,,n - nF(t2,_) - (nF(t2,n))r and

N, N, F(t2,,,)
nF(t.) nF(t2,,,) F(t,)

= (1 + (nF(t2,n)r-f) s
=1+e.

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
620 F. THOMAS BRUSS AND JAMES B. ROBERTSON

In the light of the above inequalities it remains

P(Ei,,,)-- 0 as n -- oo. This follows from (3.6) and
Theorem 3.4. The sequence N,,(nF(t,)) is unifo
conditions of Theorem 3.3 E(N,) - nF(t,).

Proof. The second statement follows from the f

ity and uniform integrability imply convergence in

(3.13) N_ - An +-. tn

From Equations (1.3) and (3.1) we have

N, =wenFn(X.,,,)
From this and Equation (1.5) conclude _5 N,. + n[F,(X.,,,) - F,(t,)].
XNn~n

nsn x dF,,(x)

X Nn,n

ntn A XNn,n

n ntn[Fn,(XNu,,) - F,(t,)]+

=t.(Nn - NJt).
This proves Equation (3.13). Consequently
N,_ _, s
nF(t,) = nF(t,) nt,F(t,)

and using sin = o" x dF(x) 5 tF(t,) we obta

N,, < NS " + 1.

nF(t,) nF(t,)

Thus it is sufficient to prove that Zn,- l/(nF(t,)) is uniformly integrable. Since iit,
is the sum of n i.i.d. indicator functions with 'success' probability p = F(t,), it
follows that E(Z,,)= 1 and
Var (Z)-nF(t,,)(1 - F(t,,)) < 1
Var (Z,) = - .
(nF(t,,))2 - nF(t,)
Thus using (3.8) and Chebychev's inequality we have
P(Z,, > a + 1) - P(IZ, - 1I > a)
t1

as a2

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
'Wald's lemma' for sums of order statistics 621

where M is some constant (recall t,,-- 0). Consequently,

all m > 0

Z,, ndP = mP(Z,, > m) + P(Z,, i>x) dx

[Z,,>m]M
mM r" dx
2 +M

2M

Since this last expression converges to zero uniformly in n as m-->, {(Z,} is

uniformly integrable as desired.

We conclude this section with a discussion of condition (3.11).

Sufficient conditions for (3.11)

1. Condition (3.11) of Theorem 3.3 is fairly general. The example treated by
Coffman et al. (1987) (F(x) - Ax ') can be seen to satisfy this condition by direct
computation.
2. There are many examples of F satisfying (3.11) but not the Coffman et al.
(1987) condition, such as F(x) - -x log x or F(x) = exp (-l1x).
3. If the density f(0) = lim supx-,o+ (F(x)/x) is such that 0 <f(0) < oo, then using
L'Hospital's rule shows that limsupx,o+ (F(Ax)/F(x)) = A and hence (3.11) is
satisfied. It is easy to see that the same is true if any of the derivatives F'j)(0) is finite
and different from 0.
4. If F is convex in a neighborhod of zero, then for all x in this neighborhood

F(Ax) F(Ax)I(Ax)
F(x) F(x)/x

which again implies condition (3.11).

5. The family of distribution functions satisfying (3.11) is also close un
operation of taking convex combinations, i.e. if F and G satisfy (3.
05 fl - 51, then OF + (1 - 0)G also satisfies (3.11).
More general conditions. The distribution function F(x) - -A/log x is an
that does not satisfy the condition (3.11). For this example f(0) = 00. An
that does not satisfy (3.11) and for which f(0) = 0 is given by
fX dt

F(x) - -A g dt
Numerical calculations lead us to conjecture that in these cases we still have

(3.14) lim t,(nF(t,))'

n-.aoo
= 0 for some < r < 1.

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
 622 F. THOMAS BRUSS AND JAMES B. ROBERTSON

If this condition holds, then the proof of Theorem 3.3 can

N, - nF(tn). It seems difficult to verify this slightly weaker c
implicit functions of F. We have not been able to find a di
does not satisfy (3.14).

4. Remarks

Here we collect some additional remarks.

An inequality for E(N,). The inequality (3.13) can be improved as follows.

Lemma 4. 1.

E(N,) nF(t,,) Vn.

Proof. We first show that

SN - St s-St
tn tn

(4.1) N, n<5 "+ N -S _ _+ Vn.

The second inequality follows d

see that the first inequality holds, suppose first that SN ---,. This implies that
XN,,, -t,,, and SNn - Stn is therefore a sum of terms which are not less than t,.
Consequently

tN
N. , S _ sN- - St
which implies the first inequality. On the other hand if SNn < S,, then (Nt - N,)t, >
St - SNn because each term in the difference on the right-hand side is at most t,.
Thus the final inequality always holds. Finally, taking expectations in (4.1) yields

_ s E(S,)
E (Nn) 5 E( Nt,) +^ = E(,t) = nF(t,).
tn tn

Example. If F is uniform on [0, 1], assuming ar < = 1 we obtain

o 2
fxdF(x)= - a 2 r = 2-a.
Corresponding to Section 2, we take s, = na with a fixed. Then Theo

that XN,,, - V and N,-- nVV-a. Corresponding to Section 3, for

oa, = s/n and t, = Vsn. Since the conditions of Theorem 3.3 are clea
we have N, - - . Moreover, Lemma 4.1 shows that E(N,) ?5 / fo
The connection with Wald's lemma. Wald's lemma (Wald (1947)) sta

is a stopping time on an i.i.d. sequence X, XI, X2, ... such that E(N)<
E( Xk) = E(N)E(X).
k=1

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
All use subject to http://about.jstor.org/terms
 'Wald's lemma' for sums of order statistics 623

From this we can get an easy estimate for the nu

the FCFS rule for the baker's problem, namely E
is not too big in relation to s.
Since in the preceding problems the sequence X
lemma is not applicable, but our way of thinking
stopping time, but N, + 1 is a stopping time on X
XN, = r, SNn can be thought of as being a sum of
coincides with S, which is the sum of all Xk such

have S,__ = SN- s < S,. If r' were not random, we could apply Wald's lemma to
obtain E(SNn+) = E(Nn + 1)E(X I X 5 r'). This heuristic result agrees with the
results in Section 2. However, in Section 3 the factor E(XIX 5 XNJ) is not easy to
calculate, and so we derived an expression for E(N,) directly.

Acknowledgements

Part of the research leading to this paper, supported by the European Com-
munity, was carried out by the authors at the University of Arizona where the first
author was holding a research fellowship of the Alexander-von-Humboldt Founda-
tion. We thank the foundations for their support and the University of Arizona for
its hospitality. We are also grateful to the referee for many constructive comments.

References

BRUSS, F. T. (1983) Resource dependent branching processes (abstract). Stoch. Proc. Appl. 16, 36.
COFFMAN, E. G., FLATTO, L. AND WEBER, R. R. (1987) Optimal selection of stochastic integrals
under a sum constraint. Adv. Appl. Prob. 19, 454-473.
SAMUELS, S. M. AND STEELE, J. M. (1981) Optimal selection of a monotone sequence from a random
sample. Ann. Prob. 9, 937-947.
WALD, A. (1947) Sequential Analysis. Dover, New York.

This content downloaded from 164.15.133.2 on Tue, 23 Aug 2016 13:32:09 UTC
View publication stats
All use subject to http://about.jstor.org/terms