Oblivious AQM and Nash Equilibria: D. Dutta, A. Goel and J. Heidemann

Oblivious AQM and
Nash Equilibria
D. Dutta, A. Goel and J. Heidemann
USC/ISI USC USC/ISI

IEEE INFOCOM 2003 - The 22nd Annual Joint Conference of the
IEEE Computer and Communications Societies

Presented By Sharon Mendel
Game Theory in Networks Seminar 25/01/2006
Before we Begin
Todays Internet - Motivation
Oblivious AQM & Nash Equilibria
Oblivious AQM & Nash Equilibria 3
Active Queue Management ??
A congestion control protocol (e.g. TCP)
operates at the end-points and uses the
drops or marks received from the Active
Queue Management policies (e.g. Drop-
tail, RED) at routers as feedback signals to
adaptively modify the sending rate in
order to maximize its own goodput.
Before we Begin
Transport Control Protocol
TCP is the dominating transport layer protocol in
the Internet and accounts for over 90% of the
total traffic.
The TCP Protocol is well defined, robust and
congestion-reactive (thus stable).
The end-to-end congestion control mechanisms
of TCP have been a critical factor in the
robustness of the Internet.
It is widely believed that if all users deployed
TCP, networks will rarely see congestion
collapses and the overall utilization of the
network will be high.
Before we Begin
Todays Internet
There are indications that the amount of non-
congestion-reactive traffic is on the rise.
Most of this misbehaving traffic does not use TCP.
e.g. Real media, network games, other real time
multimedia applications.
The unresponsive behavior can result in both
unfairness and congestion collapse for the
Internet.
The network itself must now participate in
controlling its own resource utilization.
Before we Begin
* Some of the previous slides:
Promoting the Use of End-to-End Congestion Control in the Internet,
S. Floyd and K. Fall, IEEE/ACM Transactions on Networking Vol. 7 1999.
The Papers Motivation
TCP (and in fact, any transport protocol) does not
guarantee good performance in the face of
aggressive, greedy users (who are willing to violate
the protocol to obtain better performance).
Protocol Equilibrium A protocol which leads
to an efficient utilization and a somewhat fair
distribution of network resources (like TCP does),
and also ensure that no user can obtain better
performance by deviating from the protocol.
If protocol equilibrium is achievable, then it
would be a useful tool in designing robust
networks.
Before we Begin
More Introduction
Oblivious AQM and Nash Equilibria
Oblivious AQM
Oblivious (stateless) AQM scheme a
router strategy that does not
differentiate between packets belonging
to different flows.
Stateful schemes e.g. Fair-Queuing.
Stateful Schemes offer good
performance, but oblivious schemes are
easier to implement.
More Introduction
Popular AQM Schemes (1)
Congestion avoidance is achieved
through packet dropping.
1. Drop-Tail Buffers as many packets as it
can and drops the ones it can't buffer:
Distributes buffer space unfairly among
traffic flows.
Can lead to global synchronization as all TCP
connections "hold back" simultaneously,
hence networks become under-utilized.
More Introduction
D
T
Popular AQM Schemes (2)
2. RED Random Early Dedication - Monitors
the average queue size and drops packets
based on statistical probabilities:
If the buffer is almost empty, all incoming packets
are accepted; As the queue grows, the probability
for dropping an incoming packet grows; When the
buffer is full, the probability has reached 1 and all
incoming packets are dropped.
Considered more fair than tail drop - The more a
host transmits, the more likely it is that its packets
are dropped.
Prevents global synchronization and achieves lower
average buffer occupancies.
More Introduction
R
E
D
Oblivious AQM and Nash Equilibria
The paper studies the existence
and quality of Nash equilibria
imposed by oblivious AQM
schemes on selfish agents:
1. Existence
2. Efficiency
3. Achievability
More Introduction
Content
Introduction
The Model
Existence
Efficiency
Achievability
Summary and Future Work
The Model
The Markovian Internet Game
The Internet Game
Players The end-to-end selfish traffic agents.
Each player has a strategy which is to control the
average rate he tries to push through the network.
Users Performance Metric goodput.
Rules set by the AQM policies (AQM schemes in routers).
Nash Equilibrium No selfish agent has any incentive
to unilaterally deviate from its current state.
Oblivious AQM scheme leads to a protocol equilibrium
only if it imposes a Nash equilibrium on the selfish users.
Papers focus AQM schemes that guarantee
bounded average buffer occupancy regardless of the
total arrival rate.
The Model
The Markovian Internet Game
The agents generate Poisson traffic.
Does not accurately model Internet traffic, yet a
reasonable first step.
Each user controls its own offered load = the average
Poisson traffic rate.
The system is modeled as a M/M/1/K queue:
Average service time - Without loss of generality
assumed to be unity.
Buffers capacity - K=B.
No assumptions are made on the selfish protocol
(i.e. TCP, AIMD etc).
The Model
The M/M/1/K Internet Game
n Number of users players.
i
The Poisson average arrival rate of player i.
U
i
Utility function of player i.
i
Goodput , U
i
=
i
.
p AQM router drop probability due an average
aggregated load (offered load) of and an average
service time of unity.
A symmetric Nash equilibrium - Ensures that every agent
has the same goodput at equilibrium.
Unless mentioned otherwise, quantities such as the
rates, goodput and throughput are averages (Poisson
traffic sources).
The Model
Nash Equilibrium Conditions (1)
1. No agent can increase their goodput, at Nash
equilibrium, by either increasing or decreasing their
throughput:

2. A symmetric Nash equilibrium:

3. Oblivious AQM scheme, hence functions of router
state (drop probability, queue length) are independent
in i:

n
and j i
j i j i

= = = ,
The Model
0 s
c
c
i
i
i
d
d
i
i
=
c
c
nd
p
dp
=
1
Nash Equilibrium Condition :
Nash Equilibrium Satisfying Condition
Necessary and sufficient
( ) p U
i i i
= = 1
Utility Function in Nash Equilibrium :
R
E
D
D
T
Nash Equilibrium Conditions (2)
Nash
,
Nash

,

p
Nash
- The

aggregate throughput (offered
load), goodput, drop probability respectively.
The Nash equilibrium imposed by an AQM
scheme is efficient if the goodput of any selfish
agent is bounded below when the throughput
(offered load) of the same agent is bounded
above:
( )
2
1
1
c
c p
Nash
Nash Nash
s
>
p
Nash
is bounded

c
1
s c
2
some constants
The Model
Nash Equilibrium Efficiency Condition :
Existence
Are there oblivious AQM schemes that
impose Nash equilibria on selfish
users?

Drop-Tail Queuing
p - Drop Probability = Probability to find a full system.
From Queuing Theory:
Theorem 1: There is NO Nash Equilibrium for
selfish agents and routes implementing Drop-Tail
queuing.
Proof: applying the condition for Nash equilibrium:
( )
1
1
1
+

=
B
B
p

Existence
( ) ( )

|
.
|
\
|
=
|
.
|
\
|
= = =
i i
i i i
p p U 1 1
0 0 > >
c
c
|
.
|
\
|
+
|
.
|
\
|
c
c
=
c
c
i
i i
i i
i
QED.
2
i i
i
= |
.
|
\
|
c
c ( )
( )
1
1
1
1
+
= =
B
B
p
+ + + +
=
2
1
1
1
Random Early Detection
Approximated steady state RED model:

From Queuing Theory, at steady state:
RED Router, at steady state:
Steady State:

( )
th th
th q
p
l
min max
min
max
p =
0
1
If l
q
<min
th
If min
th
s l
q
s max
th
Otherwisw

Faster network simulation
with scenario pre-filtering
Tech. Rep., USC/ISI Tech
Report 550, November 2001.
l
q
Queue length
p Drop Probability
Aggregated Offered Load
( )
( ) p
p
l
q

=
1 1
1
l
q
s max
th
( )
( )
th th
th
p
p
p
p
min max
min
1 1
1
max
|
|
.
|
\
|

=

Existence
RED and Nash Equilibria
Theorem 2: RED Does NOT impose a Nash
equilibrium on uncontrolled selfish agents.
Proof: applying the condition for Nash equilibrium:

Summary:
RED punishes all flows with the same drop probability.
The nature of the drop function is considerably gentle.
Misbehaving flows can push more traffic and get less hurt (marginally).
There is no incentive for any source to stop pushing packets.
( )
( )
|
|
.
|
\
|
+
|
.
|
\
|
=

=
q
q
q
l
l
p
p
p
l
1
1
1
1 1
1

( )
|
|
.
|
\
|
+
|
.
|
\
|
= =
q
q
i
i i
l
l
p
1
1

0
1 1
>
|
|
.
|
\
|
+ c
c
|
.
|
\
|
+
|
.
|
\
|
c
c
|
|
.
|
\
|
+
=
c
c
q
q
i
i i
i q
q
i
i
l
l
l
l

QED.
0 > |
.
|
\
|
c
c
i
i
( )
0
1
1
1
2
>
+
|
.
|
\
|
=
|
|
.
|
\
|
+ c
c
|
.
|
\
|

d
dl
l l
l
q
q
i
q
q
i
i
RED is oblivious:
Nash
Equlibria
does not
exist.
Existence
Virtual Load RED
Virtual Load RED model:

Theorem 3: VLRED imposes a Nash
Equilibrium on selfish agents if
Proof:
Throughput at Nash equilibrium is independent
of min
th
:
th th
th vq
l
min max
min
p =
0
1
If l
vq
<min
th
If min
th
s l
vq
s max
th
Otherwise

l
vq
M/M/1 Queue length when
facing the same load :
=
1
vq
l
1 max 1 min + s
th th
( )
2
1
2
max 4 1
, 1
1
2
+
+ +
= <
+
=
n
n n
t
t
t
th Nash
Existence
Efficiency
If an Oblivious AQM scheme can impose a
Nash equilibria, is that equilibria efficient, in
terms of achieving high goodput and low
drop probability?
Number Of Flows
N
o
r
m
a
l
i
z
e
d

D
a
t
a

Total Offered Load
Goodput
min
th
= 0
max
th
=50
n 1,..,50
Example - Efficiency of VLRED
Efficiency

Proof: Applying Nash
equilibrium satisfying and
efficiency conditions:

Theorem 4: VLRED
is not efficient
imposing a Nash Equilibrium
on selfish agents
|
o
+
=
1
p
, - some constants
( )
( )
o
p n
d
dp
=
|
.
|
\
|
=
1
1
1
2
2
l
vq
2 n/
n = (l
vq
2
)
th
th
vq
l
max 1
max
+
s The goodput falls to zero asymptotically.
QED.
Efficient Nash AQM scheme
Assume Total
desirable offered load
at Nash equilibrium:

Oblivious mechanism can ensure an efficient
Nash equilibria under selfish behavior of users.

=
1 2
1 4
1
1
2
d
p
dp
n
Efficiency
Goodput
Number Of Flows
N
o
r
m
a
l
i
z
e
d

D
a
t
a

Total Offered Load
Drop Probability

+
=
1 1
1 1
3
1
1 p
ENAQM drop probability
is bounded
Achievability
How easy is it for players (users) to reach
the equilibrium point? or
How can we ensure that agents actually
reach the Nash equilibrium state?
Ensuring a Nash equilibrium by
an Oblivious AQM
*
i
Offered load at equilibria when the number of agents is i.
i
=
*
i
-

*
i-1
= i

some constant
p = (
*
i
) non decreasing and convex.
Applying Nash equilibrium satisfying and efficiency conditions:

From convexity of p
*
i
:

From efficiency:

The equilibrium imposed by any oblivious AQM strategy is
(very) sensitive to the number of agents, thus making it
impractical to deploy in the Internet.
Agents need the help of the router to reach the equliibria.
( )
i
i
i
i
c f
i
c
2
'
1
s s
Achievability
c
1
, c
2
- some constants
( ) ( ) ( )
*
1
'
1
*
2
'
2
*
1
'
1
*
1
*
...

A + + A + A + >
i i i
f f f p p
c i i
i i
i
s = A

=
+
= 1
1
1
o
c - constant
( ) c +
= A
2
i
i
The sensitivity
coefficient falls
faster than the
inverse quadric.
Overview
Now what? Future Work
Some further work
Overview
Introduction Todays Internet.
The proposed model Markovian (M/M/1/K) Game
Existence
Drop tail and RED cannot impose a Nash equilibra.
VLRED imposes a Nash equilibra.
But the equilibrium points do not have a very high utilization.
Efficiency - ENAQM imposes an efficient Nash equilibra.
Achievability - Equilibrium points in oblivious AQM
strategies are very sensitive to the change in the
number of users.
It may be hard to deploy oblivious schemes that do have Nash
equilibria without the explicit help of a protocol.
Now What ? - Future Work
VLRED Explore why the Nash equilibria do not
result in good network utilization.
Conjecture VLREDs drop function becomes very
harsh as we reach equilibria.
Study gentler versions of VLRED and determine
whether such modification can still impose Nash
equilibria.
Design protocols which lead to efficient network
operation, such that no user has any incentive
to unilaterally deviate from the protocol can it
be done ? The Protocol Equilibrium Question.
Some Further work (1)
Towards Protocol Equilibrium with Oblivious
Routers D. Dutta, A. Goel and J. Heidemann,
IEEE INFOCOM 2004.
In this paper, we show that if routers used EWMA
to measure the aggregate rate, then the best
strategy for a selfish agent to minimize its losses is to
arrive at a constant rate. Even though the protocol
space is arbitrary, our scheme ensures that the best
greedy strategy is simple, i.e. send with CBR. Then,
we show how we can use the results of an earlier
paper to enforce simple and efficient protocol
equilibria on selfish traffic agents
Some Further work (2)
Pricing Differentiated Services A Game -Theoretic Approach, E.
Altman, D. Barman, R. El Azouzi, D. Ros, B. Tuffin, (accepted for
publication in Computer Networks, 2005).
The goal of this paper is to study pricing of differentiated services and
its impact on the choice of service priority at equilibrium. We consider
both TCP connections as well as non controlled (real time) connections.
We first study the performance of the system as a function of the
connections parameters and their choice of service classes. We then
study the decision problem of how to choose the service classes. We
model the problem as a noncooperative game. We establish conditions
for an equilibrium to exist and to be uniquely defined. We further provide
conditions for convergence to equilibrium from non equilibria initial
states. We finally study the pricing problem of how to choose prices so
that the resulting equilibrium would maximize the network benefit
The paper (Oblivious AQM and Nash Equilibria) restricted itself to
symmetric users and symmetric equilibria and the pricing issue was not
considered. In this framework, with a common RED buffer, it was shown
that an equilibrium does not exist. An equilibrium was obtained and
characterized for an alternative buffer management that was proposed,
called VLRED. We note that in contrast to (Oblivious AQM and Nash
Equilibria), since we also include in the utility of CBR traffic a penalty for
losses we do obtain an equilibrium when using RED
Thank You !
Appendixes
Stateful Schemes Fair Queuing
From Queuing Theory
Nash Equilibrium Conditions
Approximated Steady State RED Model
VLRED Model, Theorem 3 - Nash Equilibrium Existence -
Proof
Efficient Nash AQM Drop Probability
Stateful Schemes
Fair Queuing - Nagles Algorithm
Gateways maintain separate queues for
packets from each individual source.
The queues are serviced in a round-robin
manner.
Nagles algorithm, by changing the way
packets from different sources interact,
does not reward, nor leave others
vulnerable to, anti-social behavior.
Introduction - Appendix
Analysis and Simulation of a Fair Queuing Algorithm,
A.Demers, S. Keshav and S.J. Shenker, ACM SIGCOMM, 1989.
From Queuing Theory
M/M/1/K Queuing system - Poisson arrivals, Exponentially distributed
service times, one server and finite capacity buffer:

PASTA - Poisson Arrivals See Time Averages
For a queuing system, when the arrival process is Poisson and
independent of the service process:
The probability that an arriving customer finds i customers in the
system Equals The probability that the system is at state i.
i
Probability that the system is in state i, Using birth-death model:

Block Probability -
K i
i
i
s
|
|
.
|
\
|
=
0
t
t
( )

t =
= =
+
,
1
1
1 K
K
K B
P
2 K
K-
1
1 0

Nash Equilibrium Condition
1.
2.
3.
4.
5.
Substituting 2,3 in 5 we obtain:

n
and j i
j i j i

= = = ,
d
d
i
i
=
c
c
( ) p U
i i i
= = 1
0 1 =
c
c
=
c
c
i
i
i
i
p
p
nd
p
dp
=
1
Nash Equilibrium Satisfying Condition :
The Model Appendix
0 s
c
c
i
i
i

Approximated Steady State RED
Model
Standard RED
Transfer Function
Avg. Buffer Occupancy
D
r
o
p

p
r
o
b
.

Approximated RED
Transfer Function
D
r
o
p

p
r
o
b
.

Normalized Offered Load
Approximated RED
Buffer Occupancy
Normalized Offered Load
A
v
g
.

B
u
f
f
e
r

O
c
c
u
p
a
n
c
y

Faster network simulation
with scenario pre-filtering
Tech. Rep., USC/ISI Tech
Report 550, November 2001.
Existence - Appendix
Example:
min
th
= 10 max
th
=20
n 1,..,50 P
max
= 0.3
VLRED Model, Theorem 3 - Nash
Equilibrium Existence - Proof (1)
Virtual Load RED model:

Proof of Theorem 3: VLRED imposes a Nash Equilibrium
on selfish agents if
Assume the drop probability can be written as a continues
function for all l
vq
<max
th
:

Nash equilibrium condition:

th th
th vq
l
min max
min
p =
0
1
If l
vq
<min
th
If min
th
s l
vq
s max
th
Otherwise

l
vq
M/M/1 Queue length when
facing the same load :
=
1
vq
l
1 max 1 min + s
th th
|
|
.
|
\
|
|
.
|
\
|
|
|
.
|
\
|
=
2
1 1 min max
1
min max
min
1
th th th th
th
d
dp
p
|
|
|
|
.
|
\
|
=
|
|
.
|
\
|
|
.
|
\
|
|
|
.
|
\
|
th th
th
th th
n
min max
min
1
1
1 1 min max
1
2

VLRED Model, Theorem 3 - Nash
Equilibrium Existence - Proof (2)
Substituting and simplifying we obtain:

This is true if (by Substituting n=1)

=
1
t
0 max ) 1 (
2
= + +
th
n n t
( )
th
th
t
n
n n
t min ,
2
1
2
max 4 1
2
>
+
+ +
=
1 max 1 min + s
th th
Efficient Nash AQM Drop
Probability
Total desirable offered load at Nash equilibrium:

Substituting y
2
=1- and then integrating:

k is determined so when there is one user, the drop
probability is zero as long as his offered load is less than
unity
The offered load at Nash equilibria is bounded drop
probability at equilibria is bounded (proved by substitution).
( )
y
y
k p k
y
y
p
y
dy
y
dy
p
dp
+
= +
+
=
|
|
.
|
\
|
+
=
1
1
1 log
1
1
log 1 log
1 1 2
1
1
Efficiency - Appendix

=
1 2
1 4
1
1
2
d
p
dp
n

+
=
1 1
1 1
3
1
1 p
k - arbitrary constant
to be determined
3 1 = k

Oblivious AQM and Nash Equilibria: D. Dutta, A. Goel and J. Heidemann

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Oblivious AQM and Nash Equilibria: D. Dutta, A. Goel and J. Heidemann

Uploaded by

Copyright:

Available Formats

Oblivious AQM and

Oblivious AQM & Nash Equilibria 39

Oblivious AQM & Nash Equilibria 41

You might also like