Queueing Theory I

Summary
Littles Law
Queueing System Notation
Stationary Analysis of Elementary Queueing
Systems
M/M/1
M/M/m
M/M/1/K

Littles Law
a(t): the process that counts the number of arrivals up to t.
d(t): the process that counts the number of departures up to t.
N(t)= a(t)- d(t)
N(t)
a(t)
Time t
Area (t)
Average arrival rate (up to t)
t
= a(t)/t
Average time each customer spends in the system T
t
= (t)/a(t)
Average number in the system N
t
= (t)/t
d(t)
Littles Law
t t t
N T =
Taking the limit as t goes to infinity
| | | | E E
N T
=
Expected number of
customers in the system
Expected time in the system
Arrival rate IN the system
N(t)
a(t)
Time t
Area (t)
d(t)
Generality of Littles Law
Littles Law is a pretty general result
It does not depend on the arrival process distribution
It does not depend on the service process distribution
It does not depend on the number of servers and buffers
in the system.
| | | | E E
N T
=
Queueing
Network

Aggregate
Arrival rate
Specification of Queueing Systems
Customer arrival and service stochastic models
Structural Parameters
Number of servers
Storage capacity
Operating policies
Customer class differentiation (are all customers
treated the same or do some have priority over others?)
Scheduling/Queueing policies (which customer is
served next)
Admission policies (which/when customers are
admitted)
Queueing System Notation
A/B/m/K/N
Arrival Process
M: Markovian
D: Deterministic
Er: Erlang
G: General
Service Process
M: Markovian
D: Deterministic
Er: Erlang
G: General
Number of
servers m=1,2,
Storage Capacity K=
1,2,
(if then it is omitted)
Number of customers
N= 1,2,
(for closed networks
otherwise it is omitted)
Performance Measures of Interest
We are interested in steady state behavior
Even though it is possible to pursue transient results, it is a
significantly more difficult task.
E[S] average system time (average time spent in the
system)
E[W] average waiting time (average time spent waiting
in queue(s))
E[X] average queue length
E[U] average utilization (fraction of time that the
resources are being used)
E[R] average throughput (rate that customers leave the
system)
E[L] average customer loss (rate that customers are lost
or probability that a customer is lost)
Recall the Birth-Death Chain Example
At steady state, we obtain

0
0 1

1
2

j-2
j-1

j-1

j-1
j

j+1
0 0 1 1
0 t t + =
0
1 0
1

t t

=
In general
( )
1 1 1 1
0
j j
j j j j j

t t t
+ +
+
+ =
0
1 0
1 1
...
...
j
j
j

t t

+
+
| |
=
|
|
\ .
Making the sum equal to 1
0 1
0
1 1
...
1 1
...
j
j j

t

=
| |
| |
+ =
|
|
|
|
\ .
\ .

Solution exists if
0 1
1 1
...
1
...
j
j j
S

=
| |
= + <
|
|
\ .

M/M/1 Example
Meaning: Poisson Arrivals, exponentially distributed service
times, one server and infinite capacity buffer.

0 1

j-1

Using the birth-death result
j
= and
j
=, we obtain
0
, 0,1, 2,...
j
j
j

t t

| |
= =
|
\ .
Therefore
0
1
1
1
j
j

=
| |
| |
=
+ |
|
|
\ .
\ .

for / = <1
0
1 t =
( ) , 1, 2,... 1
j
j
j t = =
M/M/1 Performance Metrics
Server Utilization
Throughput
| | ( )
0
1
1 1 1
j
j
E
U
t t

=
= = = =

Expected Queue Length

| |
( )
0
1
1
j
j
E
R
t t

=
= = = =

| | ( ) ( )
{ }
0 0 0
1 1
j
j
j
j j j
d
E j j
X
d

= = =
= = = =

( ) ( )
( ) ( )
0
1
1 1
1 1
j
j
d d
d d

=


= = =
` `


) )

M/M/1 Performance Metrics

Average System Time
Average waiting time in queue
| | | | | | | |
1
E E E E
S S X X

= =
| | | | | | | | | | | | E E E E E E
S W W S Z Z
= + =
| |
( ) ( )
1 1
1 1
E
S


= =

| |
( ) ( )
1 1
1 1
E
W


= =

M/M/1 Performance Metrics Examples
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
5
10
15
20
25
30
35
40
rho
D
e
l
a
y

(
t
i
m
e

u
n
i
t
s
)

/

N
u
m
b
e
r

o
f

c
u
s
t
o
m
e
r
s
=0.5
[]

[W]

[S]

PASTA Property
PASTA: Poisson Arrivals See Time Averages
Let
j
(t)= Pr{ System state X(t)= j }
Let a
j
(t)= Pr{ Arriving customer at t finds X(t)= j }
In general
j
(t) a
j
(t)!
Suppose a D/D/1 system with interarrival times equal to 1 and
service times equal to 0.5
0 0.5 1.0 1.5 2.0 2.5 3.0
a a a a
Thus
0
(t)= 0.5 and
1
(t)= 0.5 while a
0
(t)= 1 and a
1
(t)= 0!
Theorem
For a queueing system, when the arrival process is Poisson
and independent of the service process then, the probability
that an arriving customer finds j customers in the system is
equal to the probability that the system is at state j. In other
words,
( ) ( ) ( )
{ }
Pr , 0,1,...
j j
a X j j
t t t
t = = =
Proof:
( ) ( )
( ) { }
0
limPr |
,
j
t
a X j a
t t t
t t
A
=
+ A
Arrival occurs in interval (t,t)
( )
( ) { }
( ) { }
0
Pr ,
,
lim
Pr
,
t
X j a
t t t
t
a
t t t
A
=
+ A
=
+ A
( )
{ } ( ) { }
( ) { }
( )
{ }
( )
0
Pr Pr
,
lim Pr
Pr
,
j
t
X j a
t t t
t
X j
t t
a
t t t
t
A
=
+ A
= = = =
+ A
M/M/m Queueing System
Meaning: Poisson Arrivals, exponentially distributed service
times, m identical servers and infinite capacity buffer.

0 1

2
2

m
m

m+1
m

3

m

1
m

if 0
and =
if
j j
j j m
m j m

s s

=

>

M/M/m Queueing System

Using the general birth-death result
0
1
, if
!
j
j
j m
j

t t

| |
= <
|
\ .
Letting =/(m) we get
0
, if
!
j
m
j
m
j m
m m

t t

| |
= >
|
\ .
( )
0
0
if
!

if
!
j
j
m j
m
j m
j
m
j m
m

t
t

<

>

( )
1
0
1
1
1
! !
j
m j
m
j j m
m m
j m

t

= =
| |
=
+ + |
|
\ .

( ) ( )
( )
1
1
0
1
1
! ! 1
j m
m
j
m m
j m

t

=
| |
=
+ + |
|

\ .

To find
0
M/M/m Performance Metrics
Server Utilization
| |
{ }
1
1
Pr
m
j
j
E j m X m
U
t

=
= + >

( )
1
0
1
! !
j
m j
m
j j m
m m
j m
j m

t

= =
| |
=
+ |
|
\ .

( )
( )
( )
( )
( )
1
0
2
! ! 1 1
j m
m
j
m m
m m
m j

t

=
| |
=
+ + |
|

\ .

( )
( )
( )
( )
( )
( )
( )
( )
1 1 1 1
1
0
2
1
! ! ! ! 1 1
1 1
j m m m
m
j
m m m m m
m
m j
m m

t

=
| |
=
+ + + |
|


\ .

( ) ( )
( )
1
0
1
1
! ! 1
j m
m
j
m m
m
j m

t

=
| |
=
+ + |
|

\ .

0
0
1
m m

t
t
= = =
M/M/m Performance Metrics
Throughput
Expected Queue Length
| |
1
1
m
j j
j j m
E j m
R
t t

= =
= + =

| |
( )
1
0
0
1
...
! !
j
m
m
j
j
j
j j m
m m
E j
j j
X
j m

t t

=
= =
| |
= = =
+ |
|
\ .

| |
( )
( )
0
2
!
1
m
m
E m
X
m

t

= +

Using Littles Law

| | | |
( )
( )
0
2
1 1
!
1
m
m
E E m
S X
m

t


| |
= = + |
|

\ .
Average Waiting time in queue
| | | |
1
E E
W S

=
M/M/m Performance Metrics
Queueing Probability
{ }
( )
( )
0
0
Pr
! ! 1
m
m j
Q j
j m j m
m m
P X m
m m
t
t t

= =
= > = = =

Erlang C Formula
Example
Suppose that customers arrive according to a Poisson
process with rate =1. You are given the following two
options,
Install a single server with processing capacity
1
= 1.5
Install two identical servers with processing capacity
2
= 0.75 and

3
= 0.75
Split the incoming traffic to two queues each with probability 0.5 and
have
2
= 0.75 and
3
= 0.75 serve each queue.

C
Example
Throughput
It is easy to see that all three systems have the same
throughput E[R
A
]= E[R
B
]= E[R
C
]=
Server Utilization
| |
1
1 2
1.5 3
A
E U

= = =
| |
2
1 4
0.75 3
B
E U

= = = Therefore, each server is 2/3 utilized

| |
2
0.5 1 2
2 0.75 3
C
E U

= = =

Therefore, all servers are similarly loaded.

Example
Probability of being idle
0
1
1
1
3
A

t

= =
For each server
0
2
1
1
2 3
C

t

= =
1
2
4
1
4
3
1
5
2 3
2 1
3

| |
| |
|
|
\ .
|
= =
+ +
|
| |

|
|
\ .
\ .
( ) ( )
( )
1
1
0
1
1
! ! 1
j m
m
j
m m
j m

t

B
=
| |
=
+ + |
|

\ .

Example
Queue length and delay
| |
1
1
2
1.5 1
A
E X


= = =

For each queue!
| |
( )
( )
0
2
12
! 5
1
m
B
m
E m X
m

t

= + =

| |
1
2
/ 2 0.5
2
/ 2 0.75 0.5
C
E X


= = =

| | | |
1
2
A A
E E S X

= =
| | | |
1
2 4
C C
E X E X = =
| | | |
1 12
5
B B
E E S X

= =
| | | |
1
4
C C
E X E X

= =
M/M/ Queueing System
Special case of the M/M/m system with m going to

0 1

2
2

m
m

m+1
(m+1)

3

and = for all
j j
j j =
0
!
j
j
j

t t =
Let =/ then, the state probabilities are given by
0 0
1
1 1
!
j
j
e
j

t t

=
| |
+ = =
|
\ .

!
j
j
e
j

t

=
System Utilization and Throughput
| |
0
1 1 E e
U

t

= =
| | E
R
=
M/M/ Performance Metrics
Expected Number in the System
| |
( )
1
0 0 1
! ! 1
j j
j
j j j
E j j e e
X
j j


t

= = =
= = = =

Using Littles Law
| | | |
1 1 1
E E
S X


= = =
No queueing!
Number of busy servers
M/M/1/K Finite Buffer Capacity
Meaning: Poisson Arrivals, exponentially distributed service
times, one server and finite capacity buffer K.
Using the birth-death result
j
= and
j
=, we obtain
0
, 0,1, 2,...
j
j
j K

t t

| |
= =
|
\ .
Therefore
0
1
1
1
j
K
j

=
| |
| |
=
+ |
|
|
\ .
\ .

for / =
0
1
1
1
K

( )
1
1
, 1, 2,...
1
j
j
K
j K

t

= =

0 1

K-1

M/M/1/K Performance Metrics
Server Utilization
Throughput
| |
( ) ( )
0
1 1
1 1
1 1
1 1
K
K K
E
U


t

+ +

= = =

Blocking Probability
| |
( )
0
1
1
1
1
K
K
E
R

t

= = <

( )
1
1
1
K
B K
K
P

t

= =

Probability that an arriving customer

finds the queue full (at state K)
M/M/1/K Performance Metrics
Expected Queue Length
| |
( ) ( )
{ }
1 1
0 0 0
1 1
1 1
j
K K K
j
j
K K
j j j
d
E j j
X
d

t

+ +
= = =

= = = =


( ) ( )
( )
1
1 1
0
1 1 1
1 1 1
K
K
j
K K
j
d d
d d

+
+ +
=


= = =
` `


) )

( )
( ) ( )
( )
( )
1
2 1
1
1 1 1
1
1
K
K
K
K

+
+
| |
+
= =
|
|

\ .
1
1
1 1
K
K
K
K



+
| |
=
|

\ .
System time
| |
( )
| | 1
K
E E
S X
t =
Net arrival rate (no losses)
M/M/m/m Queueing System
Meaning: Poisson Arrivals, exponentially distributed service
times, m servers and no storage capacity.
Using the birth-death result
j
= and
j
=, we obtain
0
1
, 0,1, 2,...
!
j
j
j m
j

t t

| |
= =
|
\ .
Therefore
0
0
1
1
!
j
m
j
j

t

=
| |
| |
=
|
|
|
\ .
\ .
for / =
1
0
0
!
j
m
j
j

t

=
| |
=
|
\ .

0
, 1, 2,...
!
j
j
j m
j

t t = =

0 1

2
2

m-1
(m-1)

m
m

3

M/M/m/m Performance Metrics
Throughput
Blocking Probability
| |
( )
!
0
/ !
1
1 j
m
m
m
j
j
m
E
R

t
=
| |

| = = <
|
\ .

!
0
/ !
j
m
B m
m
j
j
m
P

t
=
= =

Probability that an arriving customer

finds all servers busy (at state m)
Erlang B Formula
M/M/1//N Closed Queueing System
Meaning: Poisson Arrivals, exponentially distributed service
times, one server and the number of customers are fixed to N.
N

0 1

(N-1)

2

2

N-1

(N-2)

1
N

Using the birth-death result,
we obtain
( )
0
!
, 1, 2,...
!
j
j
N
j N
N j
t t = =

( )
1
0
0
!
!
N
j
j
N
N j

t

=
(
=
(

M/M/1//N Closed Queueing System

Response Time
Time from the moment the customer entered the queue until it
received service.
| |
( )
| |
0
1 E E
S X
t =
N

0 1

(N-1)

2

2

N-1

(N-2)

For the queue, using Littles law we get,
| |
( )
0
1
1 E
N X
t

In the thinking part,

| |
( )
( ) ( )
0
0 0
1
1
1
1 1
N
N
E
S
t

t t

= =

Therefore