Stochastic Queueing Systems

Civ1540 – Urban Operations Research

Fall 2023

Amer Shalaby, Ph.D., P.Eng.

1
Starter

 In many service systems, “queues” form when demand for service

exceeds capacity of service over some period of time. Examples
include:

 Airport check-in counter

 Automated teller machines (ATMs)
 Toll booths
 Merging highways
 Elevator systems
 Bus stops
 Intersections
 etc.

2
Starter

 Queuing Theory, aka Theory of Congestion, is the

study of:
 Waiting lines (i.e. queues)
 Delays
 Interaction between service demand and service capacity
 Dynamic behavior of congested systems

 Four main characteristics of any queuing system

 the manner in which customers arrive
 the manner in which customers are served
 the priority determining the order of service
 the number and configuration of servers in the system
Queueing Process and Queueing System
Queueing System
Servers
C
Arrival point Departure point
at the system C from the system

Queue C
Source
of users/ C C C C C C C
customers
C
C
C

Arrival
process
Size of Queue discipline and Service process Number of servers
user source Queue capacity
Queueing System (cont’d)

 A Queueing system consists of

 Service facility with one or more identical parallel servers
 User/customer source
 Queue space

 Fundamental processes and parameters of Queueing systems

 Arrival process
 Type of process, arrival (demand) rate and inter-arrival times
 Departure (service) process
 Type of process, departure rate (service capacity or service rate), and inter-
departure (service) times
 Queue capacity (finite vs. infinite)
 Queue discipline (regime)
 FIFO (aka FCFS), LIFO, priorities

5
Applications of Queueing Analysis

 Queueing analysis is useful for

 Analyzing the performance of service systems
 Setting level-of-service (LOS) standards
 Congestion pricing
 Economic analysis involving trade-offs among operating costs, capital
investments and LOS
 Suppose we want to decide how many bank tellers we should hire and the
customer space require

6
Notations and Terminology

 Each queueing system is usually described by a code as follows

__ / __ / __

The first space is for a letter indicating the distribution of inter-arrival time
The second space is for a letter indicating the distribution of service time
The third space is for the number of servers

 Standard code letters

 M: exponential (M stands for memoryless)
 D: deterministic
 Ek: kth-order Erlang
 G: general distribution

 Examples
 M/M/1, M/G/2, etc.
7
Notations and Terminology (con’d)

 Consider a queueing system which you start observing at time 0

every day

 For the ith customer to arrive

 Wq(i) = total time spent in the queue by the ith customer (obviously a RV
because it changes from day to day in a non-deterministic manner)

 W(i) = total time spent in the system by the ith customer

 S(i) = service time for the ith customer

 Note that W(i) = Wq(i) + S(i)

8
Notations and Terminology (con’d)

 At a particular time t
 Nq(t) = number of customers waiting in the queue at time t
 N(t) = number of customers in the “system” at time t, which includes
customers in the queue and those at the servers

 Define
 n = mean arrival rate of new customers when N(t) = n
 n = mean service rate per busy server when N(t) = n

9
Notations and Terminology (con’d)

 If n and n are constants  and  respectively, then

 Expected inter-arrival time = 1 / 
 Expected service time = 1 /  = E(S)

 Two important states of queueing systems

 Transient state
 Here the state of the system at time t is influenced by the state of the system
at time 0 or before that time
 Steady state
 Here the state of the system at time t is independent of the initial state of the
system (system has “warmed up”)

 Stochastic Queueing theory provides useful results for the steady

state

10
Values of Interest at Steady State

 Given
  = expected arrival rate
  = expected service rate per server
 Unknowns
 E[L] = L = expected number of users in queueing system = lim E[N(t)]
as t
 E[Lq] = Lq = expected number of users in queue = lim E[Nq(t)] as t
 E[W] = W = expected waiting time per customer in the queueing system
= lim E[W(i)] as i
 E[Wq] = Wq = expected waiting time per customer in the queue = lim
E[Wq(i)] as i

 We have 4 unknowns, so we need 4 equations to solve

 Little’s Law provides the 1st equation

11
Little’s Law
 Consider a queueing system, where you observe the arrival and
departure processes for a period of time T
 Define
 A(t) = cumulative arrivals to the system up to time t
 C(t) = cumulative service completions (i.e. departures) up to time t

Number
of Users
A(t)

C(t)

Time
T
 N(t) = number of users in the system at time t = A(t) – C(t)
12
Little’s Law (cont’d)

 Obviously, the area between the 2 “curves” is the total customer time (i.e.
total time in the system spent by all users up to time T). This can be
T
written as  N(t) dt
0
 Define LT as the expected number of users in the system at any instant
between 0 & T. It is the total customer time between 0 & T divided by T
T T

 N (t )dt  N (t )dt
A(T ) 0
LT  0
   T WT
T T A(T )
 As T  , the state of the system is independent of initial conditions (i.e.
it is in steady state). Therefore, at steady state, we have the relationship

L  W ……… 1

 This is known as “Little’s Law”, after J. Little (1961)

13
Other Equations
 Now, if we observe the previous system at the entrance of the system and
at the exit from the queue, we get
L q  Wq ……… 2
 We also know that
1 ……… 3
W  W  E (S )  W 

q q

 Note that equations 1-3 are applicable to any queueing system

 Now, we have 3 equations and 4 unknowns. If we can determine one of the
4 unknowns, we can use the 3 equations to determine the remaining 3
unknowns. E[L] can be determined easily for some queueing systems using
the following equation

E[ L]  L   n  Pn
0
 Where, Pn is the probability that n customers are in the system at a random
point in time during steady state
 Now, let’s consider one specific and commonly encountered class of
queueing systems: Birth-and-Death Queueing Systems

14
Birth and Death Systems

 Characteristics
 m parallel, identical servers
 Infinite queue capacity
 Whenever n users are in the system (in queue plus in service), the
arrival process is Poisson with arrival rate of n per unit of time
 Whenever n users are in the system, the departure process is Poisson
with departure (service) rate of n per unit of time
 FCFS discipline
 We are now interested in determining E[L] for the B&D queueing
systems, then use equations 1-3 to determine the 3 other unkowns
 We know that

E[ L]  L   n  Pn
0
 we need to determine Pn under steady state
 Let’s do that for one special B&D system: M/M/1 with constant  and

15
Illustrative Example

 A bank teller has the following characteristics

  = 10 customers per hour
  = 15 customers per hour

 On average a customer arrives every six minutes and it takes an

average time of 4 minutes to serve one customer

 If the arrivals and service times were deterministic, then no queue

would form and the server would be idle for 2 minutes every 6
minutes (i.e. 33% of the time)

 But because the arrival and service processes for bank tellers are
typically stochastic, we might have something like

16
Illustrative Example

Time Customer Customer Remarks

Arrives Departs
10:00 Server idle
10:01
10:02 #1 Enters server immediately
10:03
10:04 #2 Starts queue
10:05 #1 1 done, 2 enters server, no queue
10:06
10:07
10:08
10:09
10:10 #3 Starts queue
10:11
10:12 #2 2 done, 3 enters server
10:13
10:14 #3 3 done, server idle
10:15
:
: #4 Enters server immediately 17
M/M/m with FCFS and Infinite Queueing Capacity

 Define
 n = number of users in the system
 m = number of parallel and identical servers
  = user arrival rate
  = service rate of each server

 When n < m
 Rate of arrivals = 
 Overall rate of departures = ???
 Now let’s write the state balance equations from the state transition
diagram

18
M/M/m with FCFS and Infinite Queueing Capacity (cont’d)

 When n < m (cont’d)

 From cut -, P0=P1  P1 = (/)P0
 From cut -, P1=P2(2)  P2 = (/2)P1 = 0.5*(/)2P0
 In general,
( /  ) n
Pn  P0 for n  0,1,2,....m  1
n!
 When n  m
 Rate of arrivals = 
 Overall rate of departures = ??
 The state transition diagram can be drawn as

19
M/M/m with FCFS and Infinite Queueing Capacity (cont’d)

 When n  m (cont’d)
 It can be shown that
( /  ) n
Pn  n  m P0 for n  m, m  1, m  2,....
m m!
 By taking the summation of Pn from n=0 to , and equating that to 1,
we get an expression for P0. We will find that the condition for
steady state for this case is
 /m < 1
 It can be shown that

P0 ( /  ) m ( / m )
Lq 
m!(1   / m ) 2
The 3 other quantities can now be derived

20
M/M/

 This is a special case of M/M/m, where we assume to have a very

large number of identical servers such that no user has to wait in
queue (i.e. each arriving user will find a server available)
 The state transition diagram is

( /  ) n
Pn  P0 for n  0,1,2,.......
n!

1
P
n 0
n 1  P0 
e / 

21
M/M/ (cont’d)

By Substituting into Pn ,
( /  ) n e  (  /  )
Pn  for n  0,1,2,.......
n!

 This is a remarkable result

 It states that the number of users in the system at a random point in time
during steady state (which is also the number of busy servers at SS) is
Poisson distributed with parameter (/)
 The 4 values of interest
 E[L] = /
 E[W] = 1/
 E[Lq] = 0 and E[Wq] = 0

22
M/M/1, FCFS with a finite queue capacity k

 This means that new customers arriving when the system is at

capacity are “lost”

Recall that
Pn   n P0 for n  0,1,2,......., k
k
1 
 Pn  1
n 0
 P0 
1   k 1

Substituti ng into Pn
 n (1   )
Pn  for n  0,1,2,......., k
1   k 1 0
Now we can get expressions for L, Lq ,W ,Wq (check Table 4 - 1)

23
M/M/1, FCFS with a finite queue capacity k (cont’d)

 Remarks
 Note that steady state for this system is always reached, even when
>1
 Little’s Law should be applied with caution. We must count only the
customers who actually join the system
 ’ =  (1 – Pk), where
 ’ is the effective arrival rate discounting customers who get lost
 Pk is the probability that the queue is saturated (i.e. full). It is also the
fraction of potential facility users who get turned away

24
Other Queueing Systems

 M/M/m, FCFS with a finite queue capacity k

 Can write the state balance equations from the state transition
diagram and obtain closed form expressions for Pn, E[L], E[Lq], E[W]
and E[Wq]
 See Table 4-1
 M/M/m, FCFS with a finite queue capacity m
 This is a special case of the above system
 Since k=m, there is no queueing space available (i.e. customers
arriving while all servers are full are turned away)
 The probability of a full system (i.e. all servers occupied) is Pm and is
known as the “Erlang’s Loss Formula”, which is widely tabulated for
different values of /
( /  ) n
Pn  n! for n  0,1,2,......, m
m
( /  ) i

i 0 i!
25
Other Queueing Systems (cont’d)

 M/M/m, FCFS with a finite queue capacity m (cont’d)

 The above equation turns out to be the exact same expression for Pn of
M/G/m systems with k=m

 Variations and extensions of B&D queueing systems

 Many extensions of the previous models
 Most common is arrival rates and service rates that depend on the state
of the system
 Some (not all) lead to closed-form expressions

26
M/G/1 Queueing System

 Characteristics
 Arrival process is Poisson with arrival rate 
 Departure process is “General” with service rate 
 Service time, S, has a general pdf fS(s) with mean 1/ and variance S2
 One server and infinite queueing capacity

 Recall that for M/M/1 systems

 At any time instant when n>0, the probability of a “departure” (i.e.
service completion) in the next t is equal to .t

 For M/G/1 systems

 At any time instant when n>0, the probability of a departure in the next
t depends on how long ago the customer currently served has entered
the server (i.e. how long ago he/she has been in service)
 Therefore we need to consider 2 random variables for M/G/1 systems
 N (number of customers currently in the system)
 Time since the current service began
27
M/G/1 Queueing System (cont’d)

 To simplify the analysis of this system, we focus on instants in time,

which we are going to call “epochs”
 An epoch is the time instant immediately after a service completion
 By focusing on the epochs, all we need to know is the number of
customers in the system at each epoch in order to determine the
transition probability that at the next epoch there will be 0,1,2,….,n
customers in the system

Number of Customers t1, t2, …, t6 are epochs

in the system

t1 t2 t3 t4 t5 t6 time
28
M/G/1 Queueing System (cont’d)

 We can draw the state-transition diagram at the epochs, where a

state is defined as the number of customers present at the time
when a service is completed

 We are now interested (as before) in the 4 quantities under steady

state. It can be shown that

   2 2 2
L*    s
2(1   )

29
M/G/1 Queueing System (cont’d)

 L* in the previous equation is the expected number of customers in

the system at a random epoch under steady state

 It turns out that this is also equal to L, the expected number of

customers in the system at a random point in time under steady
state

 We can also say that the probability of having n customers in the

system at a random epoch under steady state is equal to the
probability of having n customers in the system at a random point in
time under steady state

 We can now derive equations for E[L], E[W], E[Lq] and E[Wq]
 Note that the condition for steady state for this queueing system is  < 1
which is the same condition for M/M/1

30
M/G/1 Queueing System (cont’d)

P0  1  
 2  2 s2
L
2(1   )

L 2  2 s2
1
W  
  2 (1   )

 2  2 s2 [(1 /  2 )   s2 ]
1
Wq  W   
 2 (1   ) 2(1   )

 2  2 s2
Lq  Wq 
2(1   )

31
M/G/1 Queueing System (cont’d)

 Note the following

 The equations do not require the knowledge of fS(s). All we need to

know is E[S] (which is equal to 1/) and Var[S]

 Reducing Var[S] by making the service consistent improves the service

 The condition for steady state is  < 1

 You can derive the equations for M/M/1 from the above equations by
using Var[S] = 1/2 (try it!)

 Can show that

 E[B] = 1 / ( - )

32
Queueing Systems with Priorities

 Priority systems are common. Examples include

 Police dispatching system
 Emergency medical rooms

 In a priority system, the system chooses the next customer to be

served from the highest-priority class present

 Two general types of priority

 Preemptive priority
 Non-preemptive priority

33
Queueing Systems with Priorities (cont’d)

 Preemptive priority
 The system interrupts the service to any customer being served at the
service facility upon arrival of a customer from a higher-priority class

 Non-preemptive priority
 Any service cannot be interrupted once it started, even if a customer
from a higher-priority class arrives during service

 Note that
 In preemptive priority, the “ejected” customer may resume service later
(i) from the point his service was interrupted, (ii) from scratch, or (iii) with
some priority
 Different classes may have different priority rules
 E.g. preemptive priority for high-priority classes
 Different classes may have different queue disciplines
34
M/G/1 with r priority classes

1 x 1

2 xxx 2

k-1 x k-1
Service
Facility
k xx k

k+1 xxx k+1

r xx r

35
 Characteristics
 r classes of customers, with class 1 having the highest priority and class
r having the lowest priority
 Poisson arrivals for each class k with rate k
 Service time for each class k is Sk which has a general pdf fSk(sk) with
first moment E[Sk]=1/k and second moment E[Sk2]
 FIFO for each class k
 Infinite queue capacity for each class

 Define k = k / k

 Assume for now that  = 1 + 2 + ….. + r < 1

36
 Define E[Wqk] as the expected wait time in queue of a customer from class
k who has just arrived at the system while under steady state

 Where
 W0 = expected remaining time in service of the customer who occupies the
server when the new customer (from class k) arrives

 Lqi = expected number of customers from class i who are already waiting in
queue at the instant when the new customer (from class k) arrives

 Mi = expected number of customers of class i who will arrive while the newly
arrived customer (from class k) is waiting in queue
37
 E[ S i2 ] i .E[ S i2 ]
W0 |i   (why?)
2.E[ Si ] 2

W0 

L qi  i .W qi

M i  i .W qk
 Now substitute the above equations into the equation of E[Wqk] 38
 k k 1
W qk  W 0    i .W qi  W qk   i
i 1 i 1
k
W 0    i .W qi
W qk  i 1
k 1
for k  1,2,......., r
1   i
i 1

 By solving the above equation recursively for k=1, k=2, ……., we obtain

 Now, we can obtain expressions for Lqk, Wk and Lk 39

 Note that we have assumed thus far that 1 + 2 + ….. + r < 1

 If for the top x classes, we have 1 + 2 + ….. + x < 1 and the class
x+1 brings the sum of s to more than 1 (i.e. 1+2+…..+x+x+1  1)
 Then the customers in classes 1 through x experience steady-state
conditions, while those in classes x+1 through r suffer unbounded in-
system (or waiting) times
 This means that
 customers in each class k of classes 1 through x occupy the server a
fraction k of the time under steady state
 customers in class x+1 occupy the server a fraction 1-ax of the time
 customers in classes x+2 through r do not have any access to the server

 Note that the book uses the letter p instead of x 40

 Now we can generalize the previous equation for E[Wqk] as
follows:

 p E[ Si2 ] E[ S p21 ]
  i  (1  a p )
 i 1 2 E[ Si ] 2 E[ S p 1 ]
W qk  for k  p
 (1  ak 1 )(1  ak )
 for k  p


 Note that
 The above equation reduces to the previous one if p=r
 E[Wqk] does not depend on the customers in classes lower than k except for
the contributions of these customers to the numerator of the equation
41