STOCHASTIC MODELLING AND

ITS APPLICATIONS
Stochastic
process
 A stochastic process or sometimes random
process (widely used) is a collection of random variables,
representing the evolution of some system of random
values over time. This is the probabilistic counterpart to a
deterministic process . Instead of describing a process
which can only evolve in one way, in a stochastic or
random process there is some indeterminacy: even if the
initial condition is known, there are several directions
in which the process may evolve.
Mathematical
Representation
Given a probability space and a measurable
space, an S-valued stochastic process is a
collection of S-valued random variables on ,
indexed by a totally ordered set T ("time"). That is,
a stochastic process X is a collection

where each is an S-valued random variable

on . The space S is then called the state
space of the process.
Real life example of stochastic
process
 Stochastic
Modelling
A method of financial modeling in which one or more variables within the model
are random. Stochastic modeling is for the purpose of estimating the probability
of outcomes within a forecast to predict what conditions might be like under
different situations. The random variables are usually constrained by historical
data, such as past market returns.
Real life application

The Monte Carlo Simulation is an example of a stochastic

model used in finance.
When used in portfolio evaluation, multiple simulations
of the performance of the portfolio are done based on
the probability distributions of the individual stock
returns.
 A statistical analysis of the results can then help
determine the probability that the portfolio will provide
the desired performance.
 stochastic modelling as applied to the
insurance industry, telecommunication , traffic
control etc
 telecommunicatio
When
n messages flow from a source to a
destination (end-to-end) through a network, parts
of a message or the whole message may be
dropped due to unavailable resources (buffer
capacity) to store the messages. The probability
of delivering a message with some data loss is
termed as loss probability. The time between the
source sending a message and the destination
receiving it is called latency or delay.
The message flow (will be called traffic
henceforth) and the network conditions are ex-
tremely stochastic in nature.
Other applications of stochastic processes in
communications include coding theory,
 Token rings
 Consider N independent and identical users that are
arranged logically in the form of a ring
 In this model at most one user is allowed to generating a
message over the cable or ring.

Wiring center

D
C
When a user with a message to transmit
receives the free token, the user holds on to
the token and transmits the message onto
the ring or cable.
Frame circles the ring and is removed
by the transmitting station.
Each station interrogates passing frame, if
destined for station, it copies the frame into
local buffer.
Once the user completes transmission,
the busy token is converted into a free
token and passed along the ring.
Re-inserting token on the
ring
Choices:
1. After station has completed transmission
of the frame.
2. After leading edge of transmitted frame
has returned to the sending station
 A A A A

t=0, A begins frame t=90, return t=400, transmit t=490, reinsert

of first bit last bit token

A A A A

t=0, A begins t=400, last bit of t=840, return of first t=1240, reinsert
frame frame enters bit token
ringNetworks: Token Ring and FDDI 11
In probability theory, a continuous-time
Markov chain (CTMC) is a mathematical
model which takes values in some finite or
countable set and for which the time
spent in each state takes non-negative
real values and has an exponential
distribution.
It is a continuous-time stochastic
process with the Markov property which
means that future behaviour of the model
depends only on the current state of the
model and not on historical behaviour.
To model the system as a CTMC, one
could assume that the packets are
generated according to a Poisson process,
the length of the packets are
exponentially distributed.
The propagation time is also exponentially
distributed. Since all the users are
identical, a CTMC of the form {(X(t), Y (t), t
≥ 0} model where X(t) is the number of
messages in the network and Y (t) is the
status of the token (free or busy) at time t.

Using the steady state distribution of
the CTMC, performance measures such
 Traffic
Modelsthrough the networks can be classified
Traffic flowing
into several types. Depending on the network segment,
all messages are broken down into either packets or
cells.
Packets:
The length or size of a packet ranges anywhere from
60 bytes to 1500 bytes and generally follows a
bimodal distribution.
ATM Cells:
The length of ATM cells is fixed at 53 bytes.
 Hierarchical
Networks
Telecommunication networks are typically hierarchical in nature. Some
frequently used stochastic models for traffic flow are explained in this
section.

Traffic can be classified into four level

1. Application Level
• The traffic generated by an application, say, http or telnet
or ftp which can vary significantly based on the protocols
they follow
2. Source Level :

Each workstation or computer can be thought of

as a source that generates traffic. This traffic
comprises of the traffic generated by different
applications that are running on the source.
Therefore the traffic that flows on a link that
exits the computer is a mixture of the different
applications. The process of mixing is known as
multiplexing.
3. Aggregate
Level
Several computer, printers, etc are connected together to
form a local area network (LAN). The traffic on a LAN pipe
is the aggregated traffic that is multiplexed from all the
sources.

4. Backbone Level
The LANs are connected together by means of a backbone
(say, the Internet backbone), and this forms the Metropolitan
Arean Networks (MANs) or the Wide Area Networks
(WANs). The traffic on a MAN/WAN pipe is the combination
of the traffic from several LANs.
 Fluid-flow Traffic Models
In the fluid-flow models it is assumed that traffic is in the
form of fluid which flows through a pipe at different rates at
different times. For example, fluid flows at rate r(1) bytes
per second for a random amount of time t1, then flows at rate
r(2) bytes per second for a random amount of time t2, and so
on. This behaviour can be captured as a discrete stochastic
process that jumps from one state to another whenever the
traffic flow rate changes. This can be formalized as a
stochastic process {Z(t), t ≥ 0} that is in state Z(t) at time t.
Fluid flows in the pipe at rate r(Z(t)) at time t.
 Aggregate Dynamic Stochastic Model For ATS
 Air traffic control can be simplified using stochastic
modelling.
 Here we assume the aircrafts arriving at an airport as
a Poisson distribution and compute the average delay
incurred due to constraints of landing aircraft
 we assume that each aircraft in Centre i independently
travels to Centre j (or leaves the airspace for j = 0) between
time-steps k and k + 1 with probability pij[k]. We denote
the total number of aircraft that flow from Centre i to
Centre j between times k and k + 1 by Uij[k]. For small
enough ∆T, it can be shown that the conditional distribution
for the flow Uij[k] given the Centre count si[k] is well-
approximated by a Poisson random variable, with mean
pij[k] si[k] .
 Now that we have characterized the flows of aircraft in
our model, the state variable update can be specified by
accounting for the number of aircraft entering and leaving
each Centre i between times k and k + 1:

1) si[k  1 ]  si[k]-  (Uij[k)   (Uji[k]

 This update rule defines the temporal evolution of our

aggregate stochastic model.
 In our application of the aggregate model, it is not
Equation 1 that we propagate forwards in time.
 Instead, we propagate expectations and variances of
the si[k], using equations that are derived from Equation
1, and that have a very simple structure
 the conditional expectation for the number of aircraft in
Center i is

E( s[k1]| s[k])  si[k]pij[k]  (  ( sj[k]pji [k]

ëi[k] ) ),
 which is a linear function of the time-k Centre counts. Finally, by
taking the expectation with respect to the time-k Centre counts
s[k], given the initial Centre counts s[0], we find that
E( si[k + 1] | s[0] ) = E( si[k]|s[0]) -
(E(sj[k]| s[0])pji[k] 
 (E(si[k]| s[0])pij[k])   ëi[k] )
 Thus, we see that the expected number of aircraft in Centre i
at time k+1givens[0] can be written as a linear function of the
expected Centre counts at time k given s[0].
 Disturbances
:
 The U.S. ATS is subject to disturbances that change rates of aircraft
flow in parts of the network.
 Many of these flow-altering disturbances, which are often
inclement weather events in parts of the airspace, cannot
accurately be predicted in advance.
 Furthermore, although the disturbance event may directly affect only
a small part of the airspace, the resulting changes in flows and
Sector/ Centre counts may propagate throughout the network.
 Since our model for the U.S. ATS is stochastic, we can naturally
in- corporate stochastic disturbances that alter flows in the model.
 By computing the expected behaviour and variability of Centre
counts and flows in the model, regions of the airspace that may be
prone to capacity excesses due to the weather events can be
identified.
 In turn, the model may suggest improved methods for managing
traffic flow in response to weather disturbances.
 Given that a particular set of disturbances has occurred, we can
calculate statistics of Centre counts with our basic model, using the
appropriate set of model parameters (which are modified from their
nominal values based on the particular disturbances that have
occurred).
 In turn, we can calculate statistics of Centre counts without prior
knowledge of the disturbances, by scaling the predicted statistics for
each set of disturbances with the probability that these disturbances
occur, and then summing these scaled statistics.
 In this way, the dynamics of an ATS that is subject to stochastic
disturbances can be modelled and analyzed. One possible shortcoming
of this approach for modelling stochastic disturbances is the
computational complexity resulting from the large number of
disturbances that may need to be considered. (For example, if there are
10 different weather events that may or may not be present on a given
day, we must consider 2^10 = 1024 possible combinations of
disturbances.)
 Given certain special conditions on the location of disturbances, the
computational complexity can sometimes be reduced by considering
the change in the system’s dynamics due to each disturbance
separately, and then combining these individual responses.
Wireless Network Models
 One of hottest research topics in telecommunications is wireless
communications technology and a survey paper would certainly be
incomplete without describing some of the on-going research work in
mobile communications. However, the field is relatively new and most
of the techniques are not well-established. Therefore only a brief
summary of some of the current papers in the area of stochastic
models in wireless networks are presented here.
 Almost all the forementioned traffic models, performance analysis,
flow control, congestion control, etc do not make any assumptions
about whether the networks are at least partially wireless or not. It is
to be noted that mobile communications where the users (sources and
destinations) are mobile are called wireless communication here.
Since the sources and destinations are not static an important problem
is to locate the users to send and receive messages.
Awduche et al describe location management issues
that involve tracking compo-nents that maintain
dynamic data on the locations of mobile stations
through a distributed database. The main focus is on a
search component that prescribes the manner in
which the wireless network is to be paged so as to
determine the location of mobile stations whose
whereabouts are unknown. The methods used are
based on search theory where a stochastic sequential
framework that systematically determines the
location of mobile stations situated within a group of
cells. Search algorithms are hence developed.
A Poisson-arrival location model (PALM) was
introduced in which customers arrive according to a
non-homogeneous Poisson process and move
independently through a general location state space
according to a location stochastic process. That was
extended to a version of PALM to study communicating
mobiles on a highway. Leung et al stress the need for
combining tele-traffic theory and vehicular traffic
theory. Their numerical results indicate that both the
time-dependent behaviour and the mobility of vehicles
play important roles in determining the system
performance.
•Other Topics
One of the most critical factor that will enable QoS provisioning in
high- speed networks is pricing. F.P. Kelly and colleagues have
developed some optimal pricing models .

• ATM switch design and router design involve significant amount of

stochastic modeling, particularly queueing. All the multiclass
scheduling policies (polling, static priority, waited fair queueing, etc)
can be implemented on the currently available switches and routers.
• All the models considered here were unicast where traffic flows from a
single source to a single destination. There are interesting stochastic
models for multicasting (single source and a few destinations like an
Internet classroom with students globally located) and for broadcasting
(single source and all nodes as destinations) applications.
• Several scenarios in telecommunication networks (such as client-server
systems) can be modeled as Queueing Networks. Walrand [76]
provides several applications of Queueing Networks in