Scheduling for Multiple Flows Sharing a Time-Varying

Channel: The Exponential Rule

Sanjay Shakkottai Alexander L. Stolyar

Coordinated Science Laboratory Bell Labs, Lucent Technologies
and Department of Electrical 600 Mountain Avenue
and Computer Engineering Murray Hill, NJ 07974
University of Illinois stolyar@research.bell-labs.com
at Urbana-Champaign
shakkott@uiuc.edu

December 23, 2000

Abstract
We consider the following queueing system which arises as a model of a wireless
link shared by multiple users. Multiple ows must be served by a \channel" (server).
The channel capacity (service rate) changes in time randomly and asynchronously with
respect to dierent ows. In each time slot, a scheduling discipline (rule) picks a ow
for service based on the current state of the channel and the queues.
We study a scheduling rule, which we call the exponential rule, and prove that this
rule is throughput-optimal, i.e., it makes the queues stable if there exists any rule which
can do so. In the proof we use the uid limit technique, along with a separation of time
scales argument. Namely, the proof of the desired property of a \conventional" uid
limit involves a study of a dierent uid limit arising on a \ner" time scale.
In our companion paper [12] it is demonstrated that the exponential rule can be used
to provide Quality of Service guarantees over a shared wireless link.

 Part of this work was carried out when this author was an intern at Bell Labs, Lucent Tech., Murray Hill,

NJ 07974

1
1 Introduction
The primary motivation of this work is the problem of scheduling transmissions of multiple
data users ( ows) sharing the same wireless channel (server). The unique feature of this
problem is the fact that the capacity (service rate) of the channel varies with time randomly
and asynchronously for dierent users. The variations of the channel capacity are due to
dierent (and random) interference levels observed by dierent users, and also due to fast
fading [3, 14], of a radio signal received by a moving user. At the very high level, the problem
is: How to schedule transmission of dierent users in a rational way, so that the wireless
channel is utilized eciently? We will refer to this general problem and the corresponding
queueing model as the variable channel scheduling problem (model).
The variable channel scheduling problem arises, for example, in the 3G CDMA High Data
Rate (HDR) system [4], where multiple mobile users in a cell share the same CDMA wireless
channel. On the downlink (the link from cell base station to users), time is divided into xed
size (1:67 msec) time slots. This slot size is short enough so that each user's channel quality
stays approximately constant within one time slot. In each time slot, one user is scheduled
for transmission. Each user constantly reports to the base station its \instantaneous" channel
capacity, i.e., the rate at which data can be transmitted if this user is scheduled for transmis-
sion. In HDR system (and in the generic variable channel model as well) a \good" scheduling
algorithm should take advantage of channel variations by giving some form of priority to users
with instantaneously better channels.
In this paper, we study a scheduling algorithm which explicitly uses information on the
state of the channel and the queues. We call it the Exponential (EXP) rule. Our main result
is that
The EXP rule is throughput optimal, i.e., it renders queues stable in any system for which
stability is feasible at all, with any other rule.
The specic variable channel scheduling model we study is the same as that in [2] (and its
extended version [1]), where a scheduling rule called Modied Largest Weighted Delay First
(M-LWDF) was proposed and proved to be throughput optimal. The Exponential rule was
introduced in [1], but not studied analytically.
In a companion paper [12], we study the EXP rule using simulations, and show that
this policy, in conjunction with a token queue mechanism allows us to support a mixture of
real-time and non-real time data over HDR with high eciency.
As in [2], our main tool for proving stability results is the uid limit technique [11, 6,
5, 13, 7]. However, in this paper, the use of this technique is much less conventional. To
prove the desired property of a \conventional" uid limit process, we use a \separation of
time scales" argument which leads to the analysis of another uid limit, on a \ner" (space
and time) scale. We believe, this approach and the constructions we use, are of independent
interest.
In the next section we introduce the precise model and formulate the main result. At the

2
end of that section, we describe the layout of the rest of the paper.

2 Variable Channel Scheduling Model. Main Result

2.1 The Model and Notations
The system consists of N input ows (of discrete customers) which need to be served by a
single channel (or server). We will denote by N = f1; : : : ; N g; both the set of ows and its
cardinality. Each ow has its own queue where customers wait for service.
The channel operates in discrete time. A time interval [t; t + 1), with t = 0; 1; 2; : : : , we
will call the time slot t. There is a nite set of channel states M = f1; : : : ; M g; and the
channel state is constant within each time slot. Associated with each state m 2 M is a xed
vector of data rates (m; : : : ; mN ), where all mi are strictly positive integers. The meaning of
1
mi is as follows. If in a given time slot t the channel is in state m and all service (in this time
slot) is allocated to queue i, then mi type i customers are served from those already present
at time t (or the entire queue i content at t, whichever is less). Note that what we call a
\channel state" here is actually a collection of channel states with respect to individual ows.
However, the service in any time slot may be split P according to a (generally speaking
random) stochastic vector  = ( ; : : : ; N ), i  0; 8i, i i = 1. If in a given time slot t
1
the channel is in state m and a \split" vector  is chosen, then for each queue i, bi mic type
i customers are served from those already present at time t (or the entire queue i content at
t, whichever is less). Here and below b
c denotes the integer part, and d
e - the ceiling of a
number.
The random channel state process m is assumed to be an irreducible discrete time Markov
chain with the (nite) state space M . (See Feller [9] for the denitions of irreducibility,
aperiodicity, and ergodicity of countable discrete time Markov chains.) The (unique) stationary
distribution of this Markov chain we denote by  = ( ; : : : ; M ).
1

Denote by Ai(t) the number of type i customers arrived in time slot t. We will adopt a
convention that all arrivals in the time slot t actually happen at time t, but those arrivals are
not available for service until time slot t + 1. We assume that
The aggregate arrival process A = f(A (t); : : : ; AN (t)); t = 1; 2; : : : g can be described by a
1
nite number of regenerative processes with nite mean regeneration cycles.
Let us denote by i; i = 1; : : : ; N , the mean arrival rate for ow i, i.e., the mean number
of type i customers arriving in one time slot.
In addition, we will assume that the average input rates converge to their mean rates
exponentially fast. Namely, we make the following

Assumption 2.1 For any i 2 N and any  > 0, there exists a constant a = a( ) > 0 such
3
that the following estimate holds. Uniformly on the initial state of the input process, for all
suciently large n,
1 X
n
Prfj n Ai(t) ? ij   g < e?an :
1

To simplify notation, let us assume that each input process Ai is an ergodic (discrete time)
Markov chain with countable state space, and the input processes are mutually independent.
(Extension of the proofs to more general input ows described above is straightforward.) In
this case, Assumption 2.1 is satised, for example, if all Ai (t) are i.i.d. random variables with
a nite exponential moment, or if each Ai (
) is a nite state Markov chain.
The random process describing the behavior of the entire system is S = (S (t); t =
0; 1; 2; : : : ), where
S (t) = f(Ui (t); : : : ; UiQi t (t)); i = 1; : : : ; N ; m(t) g;
1 ( )

Qi(t) is the type i queue length at time t, and Uik (t) is the current delay of the k-th type i
customer present in the system at time t. (Within each: type, the customers are numbered
in the order of their arrivals.) We will denote by Wi (t) = Ui (t) the delay of ow i at time t
1
(with Wi(t) = 0 if Qi (t) = 0 by convention).

2.2 A Scheduling Rule. Throughput Optimality.

A mapping H which takes a system state S (t) in a time slot into a xed probability distribution
H (S (t)) on the set of stochastic vectors , will be called a scheduling rule, or a queueing
discipline. So, if we denote by Di(t) the number of type i customers served in the time slot t,
then according to our conventions, for each time t,
Qi (t + 1) = Qi(t) ? Di(t) + Ai(t); 8i;
where Di (t) = minfQi (t); bi(t)mi t cg and (t) is chosen randomly according to the distri-
( )

bution H (S (t)).
Our assumptions imply that with any scheduling rule, S is a discrete time countable
Markov chain. To avoid trivial complications, we make an additional (not very restrictive)
technical assumption that we will only consider scheduling rules H such that the Markov
chain S is aperiodic and irreducible. By stability of the Markov chain S (and stability of the
system) we understand its ergodicity, which (in the case of aperiodicity and irreducibility) is
equivalent to the existence of a stationary distribution.
A scheduling rule H we will call universally stable, or throughput optimal, if it makes a
system stable if the stability is feasible at all with any other rule. More precisely, a rule H is
throughput-optimal if for any xed system, the existence of a rule (possibly dependent on the
system) which makes it stable, implies that the system is also stable under the rule H .

4
2.3 Exponential Rule. Main Result
Let an arbitrary set of positive constants ; : : : ; N , a ; : : : ; aN , and positive constants  and
1 1
 2 (0; 1) be xed. The following two related rules we will call Exponential.
The Exponential (Queue length) rule (EXP-Q) chooses for service in time slot t a single queue
 a Q (t) 
i i
i 2 i(S (t)) = arg max  (t) exp
i i i
;
 + [Q(t)]
P
where i(t)  mi t and Q(t) =: (1=N ) i aiQi (t). Similarly, the Exponential (Waiting time)
( )

rule (EXP-W) chooses for service a queue

 a W (t) 
i i
i 2 i(S (t)) = arg max i  i (t) exp ;
i  + [W (t)]
P
where W (t) =: (1=N ) a W (t).
i i i

Remark 2.1 1. Formally speaking, in the denition of the EXP rule, we also need to specify
a \tie-breaking" convention. For example, we can assume the the queue i = maxfj : j 2
i(S (t))g is chosen.
2. Note that without loss of generality we can assume 1 = 1. We will use this convention
later in the paper.

The main result of this paper is the following

Theorem 2.1 An EXP rule (either EXP-Q or EXP-W), with any xed set of positive pa-
rameters  ,  2 (0; 1), and i ; ai; i 2 N , is throughput optimal.

2.4 Layout of the Rest of the Paper

In the next section we discuss the necessary and sucient condition for a system to be stable.
This condition is closely related to Static Service Split (SSS) scheduling rules. We study the
properties of SSS rules which are needed for the proof of our main result. In Section 4 we
rst introduce preliminaries of the uid limit technique, and then prove Theorem 2.1. The
key element of the proof is a local uid limit argument.

3 Necessary and Sucient Stability Condition: Static

Service Split Rule
From this point on, we consider a xed system, with a xed set of the parameters.
5
 Suppose a stochastic matrix P= (mi ; m 2 M; i = 1; : : : ; N ) is xed, which means
that mi  0 for all m and i, and i mi = 1 for every m. Consider a Static Service Split
(SSS) scheduling rule [2], parameterized by the matrix . When the channel is in state m,
the SSS rule chooses for service a (single) queue i with probability mi. Clearly, the vector
v = (v ; : : : ; vN ) = v(), where
1
X
vi = m mimi ;
gives the long term average service rates allocated to dierent ows. This observation makes
the following necessary and sucient stability condition very intuitive. The proof of this result
is available in [2].

Theorem 3.1 For a given set of system parameters, a scheduling rule H under which the
system is stable exists if and only if there exists a stochastic matrix  such that
i < vi(); 8i : (1)

In the rest of this paper, we assume that the system parameters are such that stability
of the system is feasible, i.e., condition (1) holds for some  and, consequently, the SSS rule
associated with this  makes system stable.
An SSS rule associated with a stochastic matrix  we will call maximal if the vector

v( ) is not dominated by v() for any other stochastic matrix .
(We say that vector v is (1)

dominated by vector v if vi  vi for all i, and the strict inequality vi < vi holds for
(2) (1) (2) (1) (2)

at least one i.)

Remark 3.1 We note that any xed maximal SSS rule can not be throughput optimal, be-
cause throughput-optimality requires that a rule makes stable any system for which stability
is feasible. (And the necessary and sucient condition for such feasibility is given by Theo-
rem 3.1.)

We next present a very useful characterization of a maximal SSS rule. The following
result is proved in [2].

Theorem 3.2 Consider a maximal SSS rule associated with a stochastic matrix . Suppose
in addition that all components of v  = v( ) are strictly positive. Then there exists a set of
strictly positive constants i ; i = 1; 2; : : : ; N such that
mi > 0 implies i 2 arg max
j
j mj : (2)

The theorem says that basically a maximal SSS rule simply chooses for service at any
time t the queue i for which imi t is maximal. It does not specify what to do in case of a tie
( )

(when j mj is same for multiple queues); as a result the same set of fig may (and typically
will) correspond to dierent maximal SSS rules.
6
3.1 A \Diagonal Drift" Maximal SSS Rule
In this section, we show that there exists a maximal SSS rule which keeps the (weighted)
dierence of the arrival rates and service rates equal. Let us dene
 
G = y = (y ; : : : ; yN ) j
1 0  yi  max m; ai(i ? yi) = aj (j ? yj )
i;m i
8i; j 2 N
Gb = fy 2 G j 9 SSS rule parameterized by  s:t: v()  yg
where faig are arbitrary positive constants. Then, we have the following simple result.

Lemma 3.1 There exists a unique maximal element y in Gb, and a maximal SSS rule  such
that y  = v( ).

Roughly speaking, each maximal SSS rule  described in the lemma, provides the max-
imal absolute value negative drift along the diagonal aiQi = aj Qj ; 8i; j .

Proof. The existence of a matrix  such that condition (1) holds, immediately implies
that the set Gb is non-empty. Since Gb is a completely ordered bounded set, we can consider
y = sup Gb, where the supremum is component-wise. Using compactness of the set of possible
matrices  and continuity of the mapping v(), we easily establish the existence of  such
that y  v(), and therefore y 2 Gb. The equality y = v() must hold. Indeed, if the
strict inequality yi < vi() would hold for at least one i, we could always \adjust" elements
of the matrix  to produce another matrix  such that yi < vi () for all i, which would
contradict the maximality of y. The same argument shows the maximality of the SSS rule
associated with .

4 Throughput Optimality of the Exponential Rule

In this section, we prove that the exponential rule is throughput-optimal. The detailed proof
will be for the EXP-Q only. The proof for the EXP-W is virtually same, but requires a slight
adjustment which we will sketch at the end of this section.
Consider the system we xed earlier in this paper. We remind that for this system the
necessary and sucient condition described in Theorem 3.1 holds. Suppose now that this
system operates under an EXP-Q scheduling rule with a xed set of parameters  > 0,
 2 (0; 1), and ai > 0, i > 0 for i 2 N . Without loss of generality, we assume that = 1.
1
Also, just to make the proof more readable, we put  = 1=2. (The proof for any 0 <  < 1 is

7
obtained by trivial modications.) Then the EXP-Q rule is given by
0 1
i 2 i(S (t)) = arg max  (t) exp @ aiQqi (t) A :
i i i
 + Q(t)

From this point on, let us x a matrix  (and the corresponding maximal SSS), as in
Lemma 3.1, with the constants ai being the parameters of the EXP-Q rule. Let us denote
 =: a (y ?  ) = a (v () ?  ); 8i 2 N :
1 1 1 i i i (3)
For this matrix , let us x a corresponding set of positive constants fig, as in Theorem 3.2.
Without loss of generality we assume  = 1. 1

Let us dene bi by
iebi = i ; i 2 N :
Note that b = 0 (since  = 1 by the convention adopted earlier).
1 1

To prove Theorem 2.1, it will suce to prove the following \narrower" statement.

Lemma 4.1 The xed system described above is stable.

The proof will use the uid limit technique. In the next subsection we describe prelim-
inaries needed to use the technique. In the following subsection, we apply the technique to
prove Lemma 4.1. Our application of the uid limit technique is not straight forward. It
involves a separation of time scales argument: namely, it requires the analysis of uid limit
processes on two dierent time scales.

4.1 Fluid Limit Technique Preliminaries

Let us dene the norm of the state S (t) as follows:
X
N
kS k  Qi (t) :
i
Let S n denote a process S with an initial condition such that kS n (0)k = n. In the analysis
( ) ( )

to follow, all variables associated with a process S n will be supplied with the upper index
( )

(n).
The following theorem is a corollary of a more general result of Malyshev and Menshikov
[10].

8
Theorem 4.1 Suppose there exist  > 0 and an integer T > 0 such that for any sequence of
processes S (n) ; n = 1; 2; : : : , we have
lim sup E [ n1 kS n (nT )k]  1 ?  :
( )
(4)
n!1
Then S is ergodic.

It was shown by Rybko and Stolyar [11] that an ergodicity condition of the type (4)
naturally leads to a uid-limit approach to the stability problem of queueing systems. This
approach was further developed by Dai [6], Chen [5], Stolyar [13], and Dai and Meyn [7]. As
the form of (4) suggests, the approach studies a uid process s(t) obtained as a limit of the
sequence of scaled processes n S n (nt); t  0. At the heart of the approach (in its standard
1 ( )

form) is a proof that s(t) starting from any initial state with norm ks(0)k = 1 reaches 0 in
nite time T and stays there.
The rst thing we need to do is to dene what the scaling n S n (nt) means in our setting. 1 ( )

In order for this scaling to make sense, we will need an alternative denition of the process.
To this end, let us dene the following random functions associated with the process
S (t). Let Fi n (t) be the total number of type-i customers that arrived by time t  0,
n
( ) ( )

including the customers present at time 0; and F^i n (t) be the number of type-i customers
( )

that were served by time t  0. Obviously, F^i n (0) = 0 for all i. As in [11] and [13], we
( )

\encode" the initial state of the system; in particular, we extend the denition of Fi n (t) to ( )

the negative interval t 2 [?n; 0) by assuming that the customers present in the system in its
initial state S n (0) arrived in the past at some of the time instants ?(n ? 1); ?(n ? 2); : : : ; 0,
( )

accordingP to their delays in the state S (0). By this convention Fi n (?n) = 0 for all i and ( )

n, and Ni Fi n (0) = n. Also, denote by Gmn (t) the total number of time slots before time
=1
( ) ( )

t (i.e., among the slots 0; 1; : : : ; t ? 1), when the channel was in state m; and by G^ min (t) ( )

the number of time slots before time t when the channel state was m and the channel was
allocated to serve queue i. Then the following relations obviously hold:
Qin (t)  Fi n (t) ? F^i n (t); t  0; i = 1; 2; : : : ; N;
( ) ( ) ( )
(5)
Finally, let
s1 X
e (t) 
Qin
( )
a i Qi n
( )
(t) ? bi N aj Qjn (t)
( )

It is clear that the process S n = (S n (t); t  0) is a projection of the process X n =

( ) ( ) ( )

(F n ; F^ n ; G n ; G^ n ; Q n ), where
( ) ( ) ( ) ( ) ( )

F n = (Fi n (t); t  ?n; i = 1; 2; : : : ; N ) ;

( ) ( )

F^ n = (F^i n (t); t  0; i = 1; 2; : : : ; N );
( ) ( )

9
 G n = (Gmn (t); t  0; m 2 M ) ;
( ) ( )

G^ n = (G^ min (t); t  0; m 2 M; i = 1; 2; : : : ; N );

( ) ( )

Q n = (Qin (t); t  0; i = 1; 2; : : : ; N );
( ) ( )

In other words, a sample path of X n uniquely denes the sample path of S n .

( ) ( )

Let us also adopt the convention

Y (t) = Y (btc); for Y = S n ; Fi n ; F^i n ; Gmn ; G^ min ; Qin
( ) ( ) ( ) ( ) ( ) ( )

with t  ?n for Y = Fi n and t  0 for all other functions. This convention allows us to view
( )

the above functions as continuous-time processes dened for all t  0 (or t  ?n), but having
constant values in each interval [t; t + 1).
Now consider the scaled process x n = (f n ; f^ n ; g n ; g^ n ; q n ), where
( ) ( ) ( ) ( ) ( ) ( )

f n = (fi n (t); t  ?1; i = 1; 2; : : : ; N ) ;

( ) ( )

f^ n = (f^i n (t); t  0; i = 1; 2; : : : ; N );
( ) ( )

g n = (gmn (t); t  0; m 2 M ) ;
( ) ( )

g^ n = (^gmin (t); t  0; m 2 M; i = 1; 2; : : : ; N );
( ) ( )

q n = (qi n (t); t  0; i = 1; 2; : : : ; N ) ;
( ) ( )

q~ n = (~qi n (t); t  0; i = 1; 2; : : : ; N ) ;
( ) ( )

and the scaling is dened as

z n (t) = n1 Z n (nt) :
( ) ( )

From (5) we get:

qi n (t)  fi n (t) ? f^i n (t); t  0; i = 1; 2; : : : ; N:
( ) ( ) ( )
(6)

The following lemma states the convergence to and basic properties of a \ uid limit"
process, and is a variant of Theorem 4.1 in [6] or Theorem 7.1 in [13].

Lemma 4.2 The following statements hold with probability 1. For any sequence of processes
X n , there exists a subsequence X k ; fkg  fng, such that for each i; 1  i  N and
( ) ( )

m 2 M,
(fi k (t); t  ?1) ) (fi (t); t  ?1)
( )
(7)

(fi k (t); t  0) ! (fi (t); t  0) u:o:c:

( )
(8)

10
 (f^i k (t); t  0) ! (f^i (t); t  0) u:o:c:
( )
(9)
(qi k (t); t  0) ! (qi(t); t  0) u:o:c:
( )
(10)
(~qi k (t); t  0) ! (aiqi (t); t  0) u:o:c:
( )
(11)
(gmk (t); t  0) ! (gm(t); t  0) u:o:c:
( )
(12)
(^gmik (t); t  0) ! (^gmi(t); t  0) u:o:c:
( )
(13)
where the functions fi are non-negative non-decreasing right continuous with left limits (RCLL)
in [?1; 1), the functions fi ; f^i ; gm; g^mi are non-negative non-decreasing Lipschitz-continuous
in [0; 1), functions qi are continuous in [0; 1), \)" signies convergence at continuity points
of the limit, and \u.o.c." means uniform convergence on compact sets, as k ! 1. The lim-
iting set of functions
x = (f; f;^ g; g^; q)
also satises the following properties:
X
N
fi(0)  1; (14)
i=1
and for all i; 1  i  N and m 2 M ,
fi(t) ? fi (0) = i t; t  0; (15)
f^i(0) = 0; (16)
f^i(t)  fi(t); t  0 ; (17)
gm(t) = m t; t  0; (18)
qi (t) = fi(t) ? f^i (t); t  0 ; (19)
g^mi(0) = 0; (20)
X
N
g^mi (t) = gm(t); (21)
i=1
for any interval [t1 ; t2]  [0; 1),
X
f^i(t2 ) ? f^i (t1)  mi(^gmi(t ) ? g^mi(t )) ;
2 1 (22)
m2M
if qi (t) > 0 for t 2 [t1 ; t2 ]  [0; 1), then
X
f^i(t2 ) ? f^i(t1 ) = mi(^gmi (t ) ? g^mi(t )) ;
2 1 (23)
m2M

11
Proof. It follows from the strong law of large numbers that, with probability 1 for every i,
(fi n (t) ? fi n (0); t  0) ! (it; t  0) u:o:c:
( ) ( )

So, to prove (8), (14), and (15) it suces to choose a subsequence fkg  fng such that for
every i, lim fi k (0) exists, and denote the limit by fi (0). Since all fi k are non-decreasing, we
( ) ( )

can always choose a further subsequence such that (7) holds.

The properties (12) and (18) follow from the ergodicity of the channel state process.
Also, for any xed 0  t  t , for every i, m, and any n, we have (using the notation
1 2
  maxm;j m): j
X
f^i n (t ) ? f^i n (t ) 
( )
2
( )
1 mi(^gmin (t ) ? g^min (t ) + 1=n)  (t ? t + 1=n) :
( )
2
( )
1 2 1
m2M
From this inequality we deduce the existence of a subsequence (of the subsequence already
chosen) such that the convergences (9) and (13) take place, and (22) holds.
The relations (16), (17), (20), and (21), follow from the corresponding relations which
trivially hold for the prelimit functions (with index (n)). The convergence (10) and identity
(19) trivially
p follow from identity (6). Convergence (11) follows from (10) as the (scaled by
1=n) : term goes to zero u.o.c., as n ! 1.
Suppose, qi(t) > 0 for t 2 [t ; t ]  [0; 1). Let us x  2 (0; mint2 t ;t qi(t)). The
1 2 [ 1 2]

Lipschitz continuity of qi(
), along with u.o.c. convergence of qi k to qi, implies that (with ( )

probability 1) the sequence X k is such that for all suciently large k, the following inequal-
( )

ities hold:
min Q k (t) > k > max
t2 bt kc;t k [ m i
m : ( )

1 2 +1]

The latter property implies that if the queue i was chosen for service anywhere in the interval
[bt kc; t k + 1] when the channel state was m, then exactly mi type i customers were served.
1 2
So, we must have
X
jF^i k (kt ) ? F^i k (kt ) ?
( )
2
( )
1 mi(G^ mik (kt ) ? G^ mik (kt ))j  2 max
( )
2
( )
1
m i
m :
m2M
Scaling the last inequality by k and taking the limit k ! 1 we get (23).

Since some of the component functions included in x, namely fi(
), f^i(
), gm(
), g^mi(
),
qi(
), are Lipschitz in [0; 1), they are absolutely continuous. Therefore, at almost all points
t 2 [0; 1) (with respect to Lebesgue measure), the derivatives of all those functions exist. We
will call such points regular.

12
4.2 More Preliminaries
Let arbitrary  > 0 and T > 0 be xed. For each n, let us cover the interval [0; nT ] with
PTn =: bnT=n c + 1 equal non-overlapping n -long intervals [(i ? 1)n ; in ), 1  i  PTn.
1
4
1
4
1
4
1
4

Dene for each j = 1; 2; : : : ; N , and each m = 1; 2; : : : ; M ,

j , the number of arrivals from ow j in the time interval [(i ? 1)n ; in ),

Ai;n 1 1
4 4

Bmi;n, the number of (full) time-slots that the channel is in state m in the time interval
[(i ? 1)n ; in ).
1
4
1
4

Let us denote
[ ( Ai;nj  )
Ejn(T;  ) =  n ? j  >  ;
1
iPTn
[  Bmi;n m 
4
1

Gnm(T;  ) =
1
n
iPTn
 ? > :
1
4

Also, let Q denote the set of strictly positive rational numbers.

+

Lemma 4.3 The following properties hold:

0 1
[ \[ n A
1 1
Pr @ Ej (T;  ) = 0 8j 2 N ; (24)
0 ;T 2Q k1 n1 k
+ =1 =
1
[ \ [
Pr @ Gnm (T;  )A = 0 8m 2 M : (25)
;T 2Q+ k=1 n=k
Equivalently, with probability 1, for all rational T > 0 and  > 0, there exists nite k such
that for all n > k,
 i;n 
max  Aj ? j  <  ; (26)
j 2N; iPTn  n
1
1
4 
 Bi;n 
max  m ? m <  :
m2M; iPTn  n
1
1 (27)
4

Proof Fix any rational T > 0 and  > 0. Fix any j 2 N . According to Assumption 2.1 on
the arrival process, the following large deviations estimate holds. There exists a = a(; j ) > 0
and a k = k (; j ) such that for all n  k , uniformly on 1  i  PTn,
1 1
 i;n  ! 1

Pr  j ? j  >  < exp(?n a)

A
(28)
1
4

n 1
4

13
Hence, for all n  k ,1 ?

Pr Ejn(T;  ) < PTn exp(?n a) ; 1
4

and therefore X ? n

Pr Ej (T;  ) < 1 :
n
By Borel-Cantelli lemma, it follows that
\
1 [
1 !
Pr Ejn(T;  ) =0:
k=1 n=k
This implies property (24), since the rst union in (24) is over a countable set.
The proof of (25) is essentially same: a large deviation estimate analogous to (28) follows
from the fact that the channel state process m is a nite irreducible Markov chain (see [8]).

4.3 Proof of the Main Result

We now in position to prove Lemma 4.1 and therefore our main result, Theorem 2.1.

Theorem 4.2 With probability 1, a limiting set of functions, as in Lemma 4.2, satises the
following additional condition. At every regular point t > 0,
max a q (t) > 0 implies (max
i i i
a q (t))0  ? ;
i i i
where
 =: a (y ?  ) > 0
1 1 1

is dened by (3).
P there exists T > 0 such that, with probability 1, a limiting set of functions is
Consequently,
such that i qi (t) = 0 for any t  T .

Proof. The subset of outcomes (i.e., elements of the underlying probability space) for which
the statements of both Lemma 4.2 and Lemma 4.3 hold, has probability 1. Consider this
subset. Suppose statement of the theorem does not hold. Then there exists an outcome within
the specied subset, such that a subsequence of scaled processes converges to a \ uid limit"
(i.e., a set of limiting functions as in Lemma 4.2) satisfying the: following property. For some
xed regular point t > 0 and a constant  <  , we have h(t) = maxi q~i (t)  maxi aiqi (t) > 0
1
and h0(t) > ? . Let us prove that this assumption leads to a contradiction.
1

If the assumption holds, then there exist constants  > 0,  > 0, and  2 ( ;  ), such
2 1
that
h(s) >  ; 8s 2 [t; t + ] ;
14
and
h(t + ) ? h(t) > ? :
 2

p
Forpeach n, let us now divide the interval [t; t + ] into np intervals, each of length pn .
(Since n may not be an integer, we should divide p into, say, d ne intervals. To avoid trivial
complications and heavy notation, we assume that n is integer. It will be clear that we do
not lose the correctness of the argument.) Note that in the \unscaled
p time" (i.e. on the time
scale of the original process S ), each subinterval is of length  n.
Let us denote
h n (t) =: max
( )
q~ n (t) ;
i i
( )

and x any constant  2 ( ; ). From the Dirichlet principle, for all suciently large n, in
2
at least one of the subintervals (of length pn ), the average rate of change of h n (:) is greater ( )

p
than or equal to (?). We pick such a subinterval [s n ; s n + = n] for each n. Let us choose
( ) ( )

a further subsequence of the sequence of indices fng (which we will still denote by fpng), such
that s n ! s, for some xed s 2 [t; t + ]. Obviously, the right end-point s n + = n of the
( ) ( )

subinterval also converges to s. Let us recall the notation

1 1 q
n
( ) e
qei (t)  n Qi (nt)  n [ai Qi (nt) ? bi Q n (nt) ] ;
n
( ) n ( ) ( )

where X
Q n (t)  N1 aj Qjn (t) :
( ) ( )

From the subsequence fng, we choose a further subsequence such that the order of values
of qei n (s n ); i 2 N , remains the same. For example, without loss of generality let us assume
( ) ( )

that
1 qe n (s n )  : : :  qeNn (s n ) :
( ) ( ) ( ) ( )

Finally, for each i 2 N , consider the following processes:

:p n n p
q i (x) = n[qei (s + x= n) ? qe (s )]; x 2 [0;  ] ;
n
( ) ( ) ( ) n n ( ) ( )
1

and choose a subsequence such that for each i,

n (0) ! qi(0) ;
q i
( )

where q (0) = 0 (by our construction), and each other qi(0) is either nite non-positive or
1
?1. Let us consider only the case when all qi(0) are nite. (If not, it is easy to observe
that the ows pwith qi(0) = ?1 receive no service at all in the (unscaled) time interval
[ns n ; ns n + n]. So, the same argument, restricted to the remaining subset of ows
( ) ( )

applies.)

15
 p We notice that a process qin (
) is obtained from
( )
p the process Qin (
) by the time \speedup"
( )

of n and the \space" scaling by the factor 1= n (in addition to the centering). Thus, it is
very natural that as n ! 1, the sequence of processes fqin (
); i 2 N g, all dened in the
( )

interval [0; ], should converge (over a subsequence of fng to another - \local" - uid limit.
The Proposition 1 formulated below formalizes this observation.
Let us dene the following functions, all dened x 2 [0; ], and associated with the
(unscaled) time interval [ns n ; ns n + pn ]:
( ) ( )
pn f n (x) is the number of type i customers arrived in the (unscaled) interval [ns n ; ns n +
( )
pnx ];i
( ) ( )

p n
pnnxf];bi (x) is the number of type i customers served in the (unscaled) interval [ns ; ns +
( )
n n ( ) ( )

pn g n (x) is the number of (complete) time slots in the (unscaled) interval [ns n ; ns n +pnx],
( )
 i ( ) ( )

in
pn bg n (xthe
which channel was in state m; p
 mi ) is the number of (complete) time slots in the (unscaled) interval [ns n ; ns n + nx],
( ) ( ) ( )

in which the channel was in state m and the service was allocated to queue i.
Proposition 1. There exists a further subsequence of fng such that the following additional
properties hold for each i 2 N and m 2 M :

(f in (x); 0  x  ) ! (ix; 0  x  ) u:o:c:

( )
(29)

(gin (x); 0  x  ) ! (m x; 0  x  ) u:o:c:

( )
(30)

(bgmin (x); 0  x  ) ! (gbmi(x); 0  x  ) u:o:c:

( )
(31)
X
(fbi (x); 0  x  ) ! (fbi(x); 0  x  ) = (
n
mibgmi (x); 0  x  ) u:o:c: (32)
( )

(qin (x); 0  x  ) ! (qi(x); 0  x  ) = (qi(0) + ai(ix ? fbi(x)); 0  x  ) u:o:c: ;

( )

(33)
where all the functions  bgmi (
) and fbi(
) are non-decreasing Lipschitz continuous with value
0 at x = 0, and all functions qi (
) are Lipschitz continuous, and in addition
X
N
b
gmi (x) =  mx :
i=1

The proof of Proposition 1 is completely analogous to the proof of Lemma 4.2. The
convergence properties (29) and (30) trivially follow from the fact that we consider an element
of probability space for which the properties (26) and (27) hold.
16
 Let us denote
h
( )n (x) = max q n (x); x 2 [0; ] ;
i  i
( )

and
x 2 [0; ] :
h(x) = max q (x);
i  i
It follows from Proposition 1 that h(
) is a Lipschitz continuous function, and from our
construction that
h( ) ? h(0)  ? :

A point x 2 (0; ) we will call regular if the derivatives of all the functions h(
), qi(
),
gbmi (
), and  fbi (
), exist in this point. Almost all points (with respect to Lebesgue measure)
of the interval (0; ) are regular.
We will now show that in each regular point x 2 (0; ), h0 (x)  ? . This will imply
that h() ? h(0)  ? , which is the desired contradiction.
For each n consider the (unscaled) time interval [ns n ; ns n + n], and consider how ( ) ( )
p
the coecient of i(t) (in the EXP-Q rule) behaves in this interval. Obviously, multiplying
the coecients of all i(t) by the same positive function of time (not necessarily a constant),
does not change the EXP-Q scheduling rule. Therefore, the following functions ~i n (
) can be ( )

regarded as the coecients of i(t). For every n, i 2 N , and x 2 [0; ], we have
0 n p n
1
~i n (x) =: i exp @ aiQi (ns q + nx) ?pnq~ (s ) A
n ( ) n ( ) ( ) ( )
( ) 1

 + Q n (ns + nx)( )

0 q q 1
B p p p
aiQin (ns n + nx) ? bi Q n (ns n + nx) ? nq~ n (s n ) + bi Q n (ns n + nx) C
( ) ( )

B C
( ) ( )

pn  p + q Q n ns pnx 
( ) ( ) ( ) ( )

= i exp @ A ( )
1

( + )
n n
0 1 0 q n n p 1
B n (x)
C B bi Q ns n nx C
( )

q ( ( )+ )
( )

= i exp @ q n n p A exp @ q n n p A :
 i

pn + Q ns ( )
nx
( ( )+  Q ns
) nx pn +
( )
( ( )+ )
n n
Note that as n ! 1 the convergence
( ) ( )
p
Q n (ns n + nx) ?! q(s)
n
is uniform on x 2 [0; ], where
X
q(s) =: N1 aiqi(s) > 0 :
i

17
Therefore, we have
p
(~ i n (x); 0  x  ) ! (i exp(qi (x)= q(s)); 0  x  ) u:o:c: :
( )
(34)

Now, consider any regular point x 2 (0; ). Consider the subset of ows I   N for
which qi (x) is maximal, i.e. qi(x) = h(x). Also, let us denote
M  = fm 2 M j mi > 0 for at least one i 2 I g :
For every m 2 M  , let us pick an element i(m) 2 I  for which mi > 0. Observe that
the value of i m mim will be the same regardless of which of those elements we pick. From
( ) ( )
the form of the EXP-Q scheduling rule and the uniform convergence (34) we can make the
following observation. There exists a small  > 0 such that for any z 2 (x; x +  ) and any
1 1
 > 0, we have the following estimate for all suciently large n,
2

X
i (fbi (z) ? fbi (x)) =
n
( ) n
( )

i2I 
X X
i mi(gmin (z) ? gmin (x)) 
( ) ( )

i2I 
X Xm
imi(gmin (z) ? gmin (x)) 
( ) ( )

m2M  i2I 
X
m (1 ?  2 )(z ? x)i m mim  ( ) ( )
m2M 
X X
m(1 ?  )(z ? x)
2 mi imi =
m2M 
Xi2I


(1 ?  )(z ? x)
2 ivi( ) :
i2I 
Since  > 0 is arbitrary, this implies that for any z 2 (x; x +  ),
2 1

X X
i(fbi (z) ? fbi(x))  (z ? x) ivi () ;
i2I  i2I 
and therefore X X
ifbi(x) 
0
ivi() :
i2I  i2I 
Then, fbi(x)  vi() for at least one i 2 I  , and therefore q0i(x)  ? holds for this i.
0
Point x is regular. By denition, in any regular point the derivatives h0 (
) and q0i(
) for all
i 2 I  are all equal. Thus
0 
 h (x)  ? ; (35)
and we are done.

18
Proof of Lemma 4.1.
The results of this section imply the following property.
There exists T < 1 such that, with probability 1, any subsequence of processes X n , has a
( )

further subsequence (still denoted X n ), such that

( )

X n
qi (T ) ! 0 :
( )

i
This in particular means that w.p.1, any sequence of processes X n is such that
( )

lim 1 kS n (nT )k = 0 ;
( )
n!1 n
which, along with the (easily veried) uniform integrability of the family fkS n (nT )k=ng,
( )

implies
lim E [ 1 kS n (nT )k] = 0 :
( )
n!1 n
This veries the condition (4), and we are done.

4.4 Proof for the EXP-W Rule: A Sketch

The proof for the EXP-W rule requires an additional preliminary step, which goes after
Lemma 4.2. This step is analogous to the one in [2] for the M-LWDF rule. (But its proof for
the EXP-W is much simpler than that for the M-LWDF.) Namely, it needs to be shown that
with probability 1 each limiting set of functions, described in Lemma 4.2, is such that by a
nite time T > 0 all the work present at time 0 has been served, i.e.,
1

t ? wi(t) > 0; 8t  T ; 8i ;
1

where wi is the \)" limit of the (scaled) delay process (1=n)Wi n (nt).
( )

After this step, the rest of the proof for EXP-W essentially repeats the proof for EXP-
Q, because for t  T the linear relation (Little's law) qi (t) = i wi(t) exists for both the
1
\conventional" and \local" uid limits.

5 Conclusions
In this paper, we have studied the scheduling problem in a system with multiple ows served
by a single channel with capacity varying in time randomly and asynchronously with respect
to dierent ows. This problem arises, for example, in wireless communications.
We have proved that the EXP scheduling rule is throughput optimal. The rule also shows
good performance in \practical" situations (see [12]). This motivates an important subject
of the future work: a more detailed study of the EXP rule properties, beyond throughput
optimality.
19
References
[1] M. Andrews, K. Kumaran, K. Ramanan, A. L. Stolyar, R. Vijayakumar, P. Whiting
\CDMA Data QoS Scheduling on the Forward Link with Variable Channel Conditions,"
Bell Laboratories Technical Report, April, 2000.
[2] M. Andrews, K. Kumaran, K. Ramanan, A. L. Stolyar, R. Vijayakumar, P. Whiting,
\Scheduling in a Queueing System with Asynchronously Varying Service Rates," 2000.
(Submitted.)
[3] W. C. Jakes, \Microwave Mobile Communications," Wiley-Interscience, 1974.
[4] P.Bender, P.Black, M.Grob, R.Padovani, N.Sindhushayana, A.Viterbi, \CDMA/HDR:
A Bandwidth Ecient High Speed Wireless Data Service for Nomadic Users," IEEE
Communications Magazine, July 2000.
[5] H. Chen, \Fluid Approximations and Stability of Multiclass Queueing Networks: Work-
conserving Disciplines," Annals of Applied Probability, Vol. 5, (1995), pp. 637-665.
[6] J. G. Dai, \On the Positive Harris Recurrence for Open Multiclass Queueing Networks:
A Unied Approach Via Fluid Limit Models," Annals of Applied Probability, Vol. 5,
(1995), pp. 49-77.
[7] J. G. Dai and S. P. Meyn, \Stability and Convergence of Moments for Open Multiclass
Queueing Networks Via Fluid Limit Models," IEEE Transactions on Automatic Control,
Vol. 40, (1995), pp. 1889-1904.
[8] A. Dembo and O. Zeitouni, \Large Deviations, Techniques and Applications," Springer-
Verlag, 1998.
[9] W. Feller, \An Introduction to Probability Theory and its Applications," Wiley, 1950.
[10] V.A. Malyshev and M.V. Menshikov, \Ergodicity, Continuity, and Analyticity of Count-
able Markov Chains," Transactions of Moscow Mathematical Society, Vol. 39, (1979), pp.
3-48.
[11] A.N. Rybko and A.L. Stolyar, \Ergodicity of Stochastic Processes Describing the Op-
eration of Open Queueing Networks," Problems of Information Transmission, Vol. 28,
(1992), pp. 199-220.
[12] S. Shakkottai and A. L. Stolyar, \A Study of Scheduling Algorithms for a Mixture of
Real and Non-Real Time Data in HDR," Bell Laboratories Technical Report, October,
2000.
[13] A.L. Stolyar, \On the Stability of Multiclass Queueing Networks: A Relaxed Sucient
Condition via Limiting Fluid Processes," Markov Processes and Related Fields, 1(4),
1995, pp.491-512.
[14] A. J. Viterbi, \CDMA. Principles of Spread Spectrum Communication," Addison-Wesley,
1995.

20