UNIT-4

ANALYSIS OF ALTERNATIVES-II
 RELIABILITY
• Reliability engineering consists of the systematic application
of time-honoured engineering principles and techniques
throughout a product lifecycle and is thus an essential
component of a good Product Lifecycle Management (PLM)
program.
• The goal of reliability engineering is to evaluate the inherent
reliability of a product or process and pinpoint potential areas
for reliability improvement.
• Realistically, all failures cannot be eliminated from a design, so
another goal of reliability engineering is to identify the most
likely failures and then identify appropriate actions to mitigate
the effects of those failures.
 RELIABILITY
• The reliability evaluation of a product or process can include a
number of different reliability analyses.
• Depending on the phase of the product lifecycle, certain types of
analysis are appropriate.
• As the reliability analyses are being performed, it is possible to
anticipate the reliability effects of design changes and corrections.
• The different reliability analyses are all related, and examine the
reliability of the product or system from different perspectives, in
order to determine possible problems and assist in analyzing
corrections and improvements.
 RELIABILITY
• Reliability engineering can be done by a variety of engineers,
including
– reliability engineers,
– quality engineers,
– test engineers,
– systems engineers or
– design engineers

• In highly evolved teams, all key engineers are aware of their

responsibilities in regards to reliability and work together to help
improve the product.
 RELIABILITY
• The reliability engineering activity should be an ongoing process
starting at the conceptual phase of a product design and continuing
throughout all phases of a product lifecycle.
• The goal always needs to be to identify potential reliability
problems as early as possible in the product lifecycle.
• While it may never be too late to improve the reliability of a
product, changes to a design are orders of magnitude less
expensive in the early part of a design phase rather than once the
product is manufactured and in service.
 RELIABILITY- DEFINITION
• Reliability is a broad term that focuses on the ability of a product to
perform its intended function.
• Mathematically speaking, assuming that an item is performing its
intended function at time equals zero, reliability can be defined as
the probability that an item will continue to perform its intended
function without failure for a specified period of time under stated
conditions.
 Importance of Reliability
There are a number of reasons why reliability is an important product
attribute, including:

Reputation.
✔ A company's reputation is very closely related to the
reliability of its products.
✔ The more reliable a product is, the more likely the company is
to have a favourable reputation.

Customer Satisfaction.
✔ While a reliable product may not dramatically affect customer
satisfaction in a positive manner, an unreliable product will negatively
affect customer satisfaction severely.
✔ Thus high reliability is a mandatory requirement for customer
satisfaction.
 Importance of Reliability
Warranty Costs.
– If a product fails to perform its function within the warranty
period, the replacement and repair costs will negatively affect
profits, as well as gain unwanted negative attention.
– Introducing reliability analysis is an important step in taking
corrective action, ultimately leading to a product that is more
reliable.

Repeat Business.
– A concentrated effort towards improved reliability shows
existing customers that a manufacturer is serious about its
product, and committed to customer satisfaction.
– This type of attitude has a positive impact on future business.
 Importance of Reliability
Cost Analysis.
✔ Manufacturers may take reliability data and combine it with other cost
information to illustrate the cost-effectiveness of their products.
✔ This life cycle cost analysis can prove that although the initial cost of a
product might be higher, the overall lifetime cost is lower than that of
a competitor's because their product requires fewer repairs or less
maintenance.
Customer Requirements.
✔ Many customers in today's market demand that their suppliers have
an effective reliability program.
✔ These customers have learned the benefits of reliability analysis from
experience.
Competitive Advantage.
✔ Many companies will publish their predicted reliability numbers to
help gain an advantage over their competitors who either do not
publish their numbers or have lower numbers
AVAILABILITY
 AVAILABILITY
• Availability, inherent (Ai) The probability that an item will operate
satisfactorily at a given point in time when used under stated
conditions in an ideal support environment.
• It excludes logistics time, waiting or administrative downtime, and
preventive maintenance downtime.
• Availability is defined as the probability that the system is operating
properly when it is requested for use.
• In other words, availability is the probability that a system is not
failed or undergoing a repair action when it needs to be used.
• At first glance, it might seem that if a system has a high availability
then it should also have a high reliability.
 AVAILABILITY
Representation
• The simplest representation of availability(A) is a ratio of the
expected value of the uptime of a system to the aggregate of the
expected values of up and down time,

• Another equation for availability(A) is a ratio of the Mean Time

Between Failure (MTBF) and Mean Time To Repair (MTTR),
 AVAILABILITY
If we define the status function as

therefore, the availability A(t) at time t > 0 is represented by

Average availability must be defined on an interval of the real line. If

we consider an arbitrary constant C>0, then average availability is
represented as
 AVAILABILITY
Limiting (or steady-state) availability is represented by

Limiting average availability is also defined on an interval [0,C] as

Availability is the probability that an item will be in an operable and

committable state at the start of a mission when the mission is called for at a
random time, and is generally defined as uptime divided by total time
(uptime plus downtime).
 AVAILABILITY
Methods and techniques to model availability

Reliability Block Diagrams or Fault Tree Analysis are developed to

calculate availability of a system or a functional failure condition
within a system including many factors like:

• Reliability models
• Maintainability models
• Maintenance concepts
• Redundancy
• Common cause failure
• Diagnostics
• Level of repair
• Repair status
 AVAILABILITY
• Dormant failures
• Test coverage
• Active operational times / missions / sub system states
• Logistical aspects like; spare part (stocking) levels at different
depots, transport times, repair times at different repair lines,
manpower availability and more.
• Uncertainty in parameters

Furthermore, these methods are capable to identify the most

critical items and failure modes or events that impact availability.
 Definitions within systems engineering

Availability, inherent (Ai)

The probability that an item will operate satisfactorily at a given point in
time when used under stated conditions in an ideal support environment.
It excludes logistics time, waiting or administrative downtime, and
preventive maintenance downtime. It includes corrective maintenance
downtime. Inherent availability is generally derived from analysis of an
engineering design:
• The impact of a repairable-element (refurbishing/remanufacture isn't
repair, but rather replacement) on the availability of the system, in which
it operates, equals mean time between failures
MTBF/(MTBF+ mean time to repair MTTR).
• The impact of a one-off/non-repairable element (could be
refurbished/remanufactured) on the availability of the system, in which it
operates, equals the mean time to failure
(MTTF)/(MTTF + the mean time to repair MTTR).

It is based on quantities under control of the designer.

 Definitions within systems engineering

Availability, achieved (Aa)

• The probability that an item will operate satisfactorily at a
given point in time when used under stated conditions in an
ideal support environment (i.e., that personnel, tools, spares,
etc. are instantaneously available).
• It excludes logistics time and waiting or administrative
downtime.
• It includes active preventive and corrective maintenance
downtime.
 Definitions within systems engineering

Availability, operational (Ao)

• The probability that an item will operate satisfactorily at a given
point in time when used in an actual or realistic operating and
support environment.
• It includes logistics time, ready time, and waiting or administrative
downtime, and both preventive and corrective maintenance
downtime.
• This value is equal to the mean time between failure (MTBF) divided
by the mean time between failure plus the mean downtime (MDT).
• This measure extends the definition of availability to elements
controlled by the logisticians and mission planners such as quantity
and proximity of spares, tools and manpower to the hardware item.
 Basic example
If we are using equipment which has a mean time to failure (MTTF)
of 81.5 years and mean time to repair (MTTR) of 1 hour:

MTTF in hours = 81.5 × 365 × 24 = 713940 (This is a reliability

parameter and often has a high level of uncertainty!)
Inherent availability (Ai) = 713940 / (713940+1) = 713940 / 713941
= 99.999860%
Inherent unavailability = 1 / 713940 = 0.000140%
Outage due to equipment in hours per year = 1/rate = 1/MTTF =
0.01235 hours per year.
 Availability Classifications

✔ The definition of availability is somewhat flexible, depending on

what types of downtimes are considered in the analysis.
✔ As a result, there are a number of different classifications of
availability.
The availabilities can be obtained directly from simulation or can be
indirectly calculated with values returned from analysis:

• Point (instantaneous) availability

• Average up-time availability (mean availability)
• Steady state availability
• Operational availability
 Point Availability
Point, or instantaneous, availability is the probability that a system
(or component) will be operational at any random time, t. This is
very similar to the reliability function in that it gives a probability
that a system will function at the given time, t. Unlike reliability, the
instantaneous availability measure incorporates maintainability
information.
At any given time t, the system will be operational if the following
conditions are met:

• It functioned properly during time t with probability R(t), or,

• It functioned properly since the last repair at time u, 0 < u < t, with
probability:
 Point Availability

• With m(u) being the renewal density function of the system.

The point availability is the summation of these two probabilities,
or:
 Mean Availability

• The mean availability is the proportion of time during a

mission or time-period that the system is available for use. It
represents the mean value of the instantaneous availability
function over the period (0, T):
 Steady State Availability

• The steady state availability of the system is the limit of the

instantaneous availability function as time approaches infinity.
• The instantaneous availability function approaches the steady
state value very closely at time approximate to four times the
MTBF:
 Operational Availability

• Operational availability is a measure of availability that

includes all experienced sources of downtime, such as
administrative downtime, logistic downtime, etc.
• The equation for operational availability is:

where the operating cycle is the overall time period of operation

being investigated and uptime is the total time the system was
functioning during the operating cycle.
 Operational Availability

• The operational availability is the availability that the

customer actually experiences.
• It is essentially the a posteriori availability based on actual
events that happened to the system.
• The previous availability definitions are a priori estimations
based on models of the system failure and downtime
distributions.
• In many cases, operational availability cannot be controlled by
the manufacturer due to variation in location, resources and
other factors that are the sole province of the end user of the
product.
 MAINTAINABILITY

Maintainability is with which a product can be

maintained in order to:
• correct defects or their cause
• repair or replace faulty or worn-out components without
having to replace still working parts
• prevent unexpected working conditions
• maximize a product's useful life
• maximize efficiency, reliability, and safety
 MAINTAINABILITY

• Meet new requirements,

• Make future maintenance easier, or
• Cope with a changed environment.
In some cases, maintainability involves a system of continuous
improvement - learning from the past in order to improve the
ability to maintain systems, or improve reliability of systems
based on maintenance experience.
 RELATION BETWEEN RELIABILITY,
MAINTAINABILITY AND AVAILABILITY

Table below displays the relationship between reliability, maintainability and

availability. Please note that in this table, an increase in maintainability implies
a decrease in the time it takes to perform maintenance actions
MARKOV MODELS
 Markov model
• a Markov model is a stochastic model used to model
randomly changing systems.
• It is assumed that future states depend only on the current
state, not on the events that occurred before it (that is, it
assumes the Markov property).
• Generally, this assumption enables reasoning and
computation with the model that would otherwise be
intractable.
• For this reason, in the fields of predictive modelling and
probabilistic forecasting, it is desirable for a given model to
exhibit the Markov property.
 Markov model
Contents
1. Introduction
2. Markov chain
3. Hidden Markov model
4. Markov decision process
5. Partially observable Markov decision process
6. Markov random field
7. Hierarchical Markov models
8. Tolerant Markov model
9. Markov-chain forecasting models
 Introduction
There are four common Markov models used in different situations,
depending on whether every sequential state is observable or not,
and whether the system is to be adjusted on the basis of
observations made:

Markov models
System state is fully System state is
observable partially observable
System is Hidden Markov
Markov chain
autonomous model
Partially observable
System is Markov decision
Markov decision
controlled process
process
 Markov chain
The simplest Markov model is the Markov chain.
• It models the state of a system with a random variable that
changes through time.
• In this context, the Markov property suggests that the
distribution for this variable depends only on the distribution
of a previous state.
 Markov Chains
• Markov chains, named after Andrey Markov, are mathematical
systems that hop from one "state" (a situation or set of values) to
another.
• For example, if you made a Markov chain model of a baby's
behaviour, you might include "playing," "eating", "sleeping," and
"crying" as states, which together with other behaviours could form
a 'state space': a list of all possible states.
• In addition, on top of the state space, a Markov chain tells you the
probability of hopping, or "transitioning," from one state to any
other state.
e.g., the chance that a baby currently playing will fall asleep
in the next five minutes without crying first.
 Markov Chains
A simple, two-state Markov chain is shown below.

With two states (A and B) in our state space, there are 4 possible
transitions (not 2, because a state can transition back into itself). If
we're at 'A' we could transition to 'B' or stay at 'A'. If we're at 'B' we
could transition to 'A' or stay at 'B'. In this two state diagram, the
probability of transitioning from any state to any other state is 0.5.
 Markov Chains
✔Of course, real modellers don't always draw out Markov chain diagrams. Instead
they use a "transition matrix" to tally the transition probabilities.
✔ Every state in the state space is included once as a row and again as a column,
and each cell in the matrix tells you the probability of transitioning from its row's
state to its column's state.
✔ So, in the matrix, the cells do the same job that the arrows do in the diagram.

If the state space adds one state, we add one row and one column, adding one
cell to every existing column and row. This means the number of cells grows
quadratically as we add states to our Markov chain.
 Hidden Markov model

• A hidden Markov model is a Markov chain for which the state is

only partially observable.
• In other words, observations are related to the state of the system,
but they are typically insufficient to precisely determine the state.
• Several well-known algorithms for hidden Markov models exist.
• For example, given a sequence of observations, the Viterbi
algorithm will compute the most-likely corresponding sequence of
states,
– the forward algorithm will compute the probability of the sequence of
observations, and
– the Baum–Welch algorithm will estimate the starting probabilities, the
transition function, and the observation function of a hidden Markov model.
Hidden Markov model
 Hidden Markov model-Example

Drawing balls from hidden urns

• Probabilistic parameters of a hidden Markov model (example)
X — states
y — possible observations
a — state transition probabilities
b — output probabilities
• In its discrete form, a hidden Markov process can be visualized as a
generalization of the urn problem with replacement (where each item
from the urn is returned to the original urn before the next step).
• Consider this example: in a room that is not visible to an observer there is
a genie.
• The room contains urns X1, X2, X3, ... each of which contains a known mix
of balls, each ball labeled y1, y2, y3, ... .
Hidden Markov model-Example
 Hidden Markov model-Example
• The genie chooses an urn in that room and randomly draws a ball
from that urn.
• They then put the ball onto a conveyor belt, where the observer can
observe the sequence of the balls but not the sequence of urns
from which they were drawn.
• The genie has some procedure to choose urns; the choice of the urn
for the n-th ball depends only upon a random number and the
choice of the urn for the (n − 1)-th ball.
• The choice of urn does not directly depend on the urns chosen
before this single previous urn; therefore, this is called a Markov
process.
• The Markov process itself cannot be observed, only the sequence of
labeled balls, thus this arrangement is called a "hidden Markov
process"
 Markov decision process

• A Markov decision process is a Markov chain in which state

transitions depend on the current state and an action vector
that is applied to the system.
• Typically, a Markov decision process is used to compute a
policy of actions that will maximize some utility with respect
to expected rewards.
• It is closely related to reinforcement learning, and can be
solved with value iteration and related methods.
Markov Decision Process-definition
 Markov Decision Process
A Markov Decision Process (MDP) model contains:
• A set of possible world states S.
• A set of Models.
• A set of possible actions A.
• A real valued reward function R(s,a).
• A policy the solution of Markov Decision Process.
Markov Decision Process
 Markov Decision Process
What is a State?
A State is a set of tokens that represent every state that the agent can be
in.
What is a Model?
• A Model (sometimes called Transition Model) gives an action’s effect in a
state. In particular, T(S, a, S’) defines a transition T where being in state S
and taking an action ‘a’ takes us to state S’ (S and S’ may be same).
• For stochastic actions (noisy, non-deterministic) we also define a
probability P(S’|S,a) which represents the probability of reaching a state S’
if action ‘a’ is taken in state S.
• Note Markov property states that the effects of an action taken in a state
depend only on that state and not on the prior history.
 Markov Decision Process
What is Actions?
An Action A is set of all possible actions. A(s) defines the set of
actions that can be taken being in state S.
What is a Reward?
A Reward is a real-valued reward function. R(s) indicates the reward
for simply being in the state S. R(S,a) indicates the reward for being
in a state S and taking an action ‘a’. R(S,a,S’) indicates the reward
for being in a state S, taking an action ‘a’ and ending up in a state S’.
What is a Policy?
A Policy is a solution to the Markov Decision Process. A policy is a
mapping from S to a. It indicates the action ‘a’ to be taken while in
state S.
 Markov Decision Process
Let us take the example of a grid world:
 Markov Decision Process
• An agent lives in the grid. The above example is a 3*4 grid. The grid
has a START state(grid no 1,1).
• The purpose of the agent is to wander around the grid to finally
reach the Blue Diamond (grid no 4,3).
• Under all circumstances, the agent should avoid the Fire grid
(orange color, grid no 4,2).
• Also the grid no 2,2 is a blocked grid, it acts like a wall hence the
agent cannot enter it.
• The agent can take any one of these actions: UP, DOWN, LEFT,
RIGHT
• Walls block the agent path, i.e., if there is a wall in the direction the
agent would have taken, the agent stays in the same place.
• So for example, if the agent says LEFT in the START grid he would
stay put in the START grid.
 Markov Decision Process
First Aim:
To find the shortest sequence getting from START to the Diamond.
Two such sequences can be found:
RIGHT RIGHT UP UP RIGHT
UP UP RIGHT RIGHT RIGHT
• Let us take the second one (UP UP RIGHT RIGHT RIGHT) for the
subsequent discussion.
• The move is now noisy. 80% of the time the intended action works
correctly.
• 20% of the time the action agent takes causes it to move at right angles.
For example, if the agent says UP the probability of going UP is 0.8
whereas the probability of going LEFT is 0.1 and probability of going RIGHT
is 0.1 (since LEFT and RIGHT is right angles to UP).
 Markov Decision Process
The agent receives rewards each time step:-
• Small reward each step (can be negative when can
also be term as punishment, in the above example
entering the Fire can have a reward of -1).
• Big rewards come at the end (good or bad).
• The goal is to Maximize sum of rewards.
 Partially observable Markov decision process

• A partially observable Markov decision process (POMDP)

is a Markov decision process in which the state of the
system is only partially observed.
• POMDPs are known to be NP complete, but recent
approximation techniques have made them useful for a
variety of applications, such as controlling simple agents
or robots.
 Markov random field

• A Markov random field, or Markov network, may be considered to

be a generalization of a Markov chain in multiple dimensions.
• In a Markov chain, state depends only on the previous state in time,
whereas in a Markov random field, each state depends on its
neighbours in any of multiple directions.
• A Markov random field may be visualized as a field or graph of
random variables, where the distribution of each random variable
depends on the neighbouring variables with which it is connected.
• More specifically, the joint distribution for any random variable in
the graph can be computed as the product of the "clique potentials"
of all the cliques in the graph that contain that random variable.
• Modelling a problem as a Markov random field is useful because it
implies that the joint distributions at each vertex in the graph may
be computed in this manner.
 Markov random field

• In the domain of physics and probability, a Markov random field

(often abbreviated as MRF), Markov network or undirected
graphical model is a set of random variables having a Markov
property described by an undirected graph.
• In other words, a random field is said to be a Markov random field if
it satisfies Markov properties.
• A Markov network or MRF is similar to a Bayesian network in its
representation of dependencies; the differences being that
Bayesian networks are directed and acyclic, whereas Markov
networks are undirected and may be cyclic.
 Markov random field
• Thus, a Markov network can represent certain dependencies that a
Bayesian network cannot (such as cyclic dependencies; on the other
hand, it can't represent certain dependencies that a Bayesian
network can (such as induced dependencies).
• The underlying graph of a Markov random field may be finite or
infinite.

An example of a Markov random field.

Each edge represents dependency. In this
example: A depends on B and D. B
depends on A and D. D depends on A, B,
and E. E depends on D and C. C depends
on E.
 Markov random field
Given an undirected graph, a set of random variables indexed by
form a Markov random field with respect to if they satisfy the local
Markov properties:
 Hierarchical Markov models
• Hierarchical Markov models can be applied to categorize human
behaviour at various levels of abstraction.
• For example, a series of simple observations, such as a person's
location in a room, can be interpreted to determine more complex
information, such as in what task or activity the person is
performing.
• Two kinds of Hierarchical Markov Models are the Hierarchical
hidden Markov model and the Abstract Hidden Markov Model.
• Both have been used for behaviour recognition and certain
conditional independence properties between different levels of
abstraction in the model allow for faster learning and inference.
 Hierarchical Hidden Markov model (HHMM)
• The hierarchical hidden Markov model (HHMM) is a statistical
model derived from the hidden Markov model (HMM).
• In an HHMM each state is considered to be a self-contained
probabilistic model.
• More precisely each state of the HHMM is itself an HHMM.
• HHMMs and HMMs are useful in many fields, including pattern
recognition.
• In the hierarchical hidden Markov model (HHMM) each state is
considered to be a self-contained probabilistic model.
• More precisely each state of the HHMM is itself an HHMM.
• This implies that the states of the HHMM emit sequences of
observation symbols rather than single observation symbols as is
the case for the standard HMM states.
Hierarchical Hidden Markov model (HHMM)

Illustration of the structure of a HHMM.

Gray lines show vertical transitions. The
horizontal transitions are shown as black
lines. The light gray circles are the internal
states and the dark gray circles are the
terminal states that returns control to the
activating state. The production states are
not shown in this figure.
 HHMM
• When a state in an HHMM is activated, it will activate its own
probabilistic model, i.e. it will activate one of the states of the
underlying HHMM, which in turn may activate its underlying HHMM
and so on.
• The process is repeated until a special state, called a production
state, is activated.
• Only the production states emit observation symbols in the usual
HMM sense.
• When the production state has emitted a symbol, control returns to
the state that activated the production state.
• The states that do not directly emit observations symbols are called
internal states.
 HHMM
• The activation of a state in an HHMM under an internal state is
called a vertical transition. After a vertical transition is completed a
horizontal transition occurs to a state within the same level.
• When a horizontal transition leads to a terminating state control is
returned to the state in the HHMM, higher up in the hierarchy, that
produced the last vertical transition.
• Remember that a vertical transition can result in more vertical
transitions before reaching a sequence of production states and
finally returning to the top level.
• Thus the production states visited gives rise to a sequence of
observation symbols that is "produced" by the state at the top level.
 HHMM
• The methods for estimating the HHMM parameters and
model structure are more complex than for the HMM and the
interested reader is referred to (Fine et al., 1998).
• The HMM and HHMM belong to the same class of classifiers.
• That is, they can be used to solve the same set of problems.
• In fact, the HHMM can be transformed into a standard HMM.
• However, the HHMM utilizes its structure to solve a subset of
the problems more efficiently.
 Abstract Hidden Markov Model.
• Hidden Markov Models, HMM's, are mathematical models of
Markov processes whose state is hidden but from which
information can leak via channels.
• They are typically represented as 3-way joint probability
distributions.
• We use HMM's as denotations of probabilistic hidden-state
sequential programs, after recasting them as “abstract” HMM's, i.e.
computations in the Giry monad D, and equipping them with a
partial order of increasing security.
• However to encode the monadic type with hiding over state X we
use DX→D 2 X rather than the conventional X→DX.
 Tolerant Markov model
• A Tolerant Markov model (TMM) is a probabilistic-algorithmic
Markov chain model.
• It assigns the probabilities according to a conditioning context
that considers the last symbol, from the sequence to occur, as
the most probable instead of the true occurring symbol.
• A TMM can model three different natures:
– substitutions,
– additions or
– deletions.
Successful applications have been efficiently implemented
in DNA sequences compression.
 Markov-chain forecasting models
• Markov-chains have been used as a forecasting methods for several
topics, for example price trends, wind power and solar irradiance.
• The Markov-chain forecasting models utilize a variety of different
settings, from discretizing the time-series to hidden Markov-models
combined with wavelets and the Markov-chain mixture distribution
model (MCM).
Queuing Network Optimization
 Queuing Network Optimization
• Queueing theory is the mathematical study of waiting lines, or
queues.
• A queueing model is constructed so that queue lengths and waiting
time can be predicted.
• Queueing theory is generally considered a branch of operations
research because the results are often used when making business
decisions about the resources needed to provide a service.
Single queuing nodes
• Simple description and analogy A queue, or queueing node can be
thought of as nearly a black box.
• Jobs or "customers" arrive to the queue, possibly wait some time,
take some time being processed, and then depart from the queue
Queuing Network Optimization

A black box. Jobs arrive to, and depart from, the queue.
 Queuing Network Optimization
• The queueing node is not quite a pure black box, however, since
there is some information we need to specify about the inside of
the queuing node.
• The queue has one or more "servers" which can each be paired
with an arriving job until it departs, after which that server will be
free to be paired with another arriving job

A queueing node with 3 servers.

Server a is idle, and thus an arrival is
given to it to process. Server b is
currently busy and will take some time
before it can complete service of its
job. Server c has just completed
service of a job and thus will be next to
receive an arriving job.
 Queuing Network Optimization

• An analogy often used is that of the cashier at a supermarket.

There are other models, but this is one commonly
encountered in the literature.
• Customers arrive, are processed by the cashier, and depart.
Each cashier processes one customer at a time, and hence
this is a queueing node with only one server.
• A setting where a customer will leave immediately if the
cashier is busy when the customer arrives, is referred to as a
queue with no buffer (or no "waiting area", or similar terms).
• A setting with a waiting zone for up to n customers is called a
queue with a buffer of size n.
 Queuing Network Optimization
Birth-death process:
• The behaviour / state of a single queue (also called a "queueing node")
can be described by a birth-death process, which describes the arrivals
and departures from the queue, along with the number of jobs (also
called "customers" or "requests", or any number of other things,
depending on the field) currently in the system.
• An arrival increases the number of jobs by 1, and a departure (a job
completing its service) decreases k by 1.

Birth / death process. The values in the circles represent the state of the birth-death
process, k. The system transitions between values of k by "births" and "deaths" which
occur at rates given by various values of λi and μi, respectively. For a queueing system, k
is the number of jobs in the system (either being serviced or waiting if the queue has a
buffer of waiting jobs). Further, for a queue, the arrival rates and departure rates are
generally considered not to vary with the number of jobs in the queue so that we consider
a single average rate of arrivals/departures per unit time to the queue. Hence, for a queue,
this diagram has arrival rates of λ = λ1, λ2, ..., λk and departure rates of μ = μ1, μ2, ..., μk
 Queuing Network Optimization

A queue with 1 server, arrival rate λ and departure rate μ.

 Queuing Network Optimization
• The steady state equations for birth-and-death process are as
follows. (Here Pn denotes the steady state probability to be in state
n.)
 Queuing Network Optimization
Example analysis of an M/M/1 queue
Consider a queue with 1 server and the following variables:
λ: the arrival rate (the expected time between each customer arriving, e.g. 30
seconds);
μ: the reciprocal of the mean service time (the expected number of consecutive
service completions per the same unit time, e.g. per 30 seconds);
n: the parameter characterizing the number of customers in the system;
Pn: the probability of being n customers in the system in steady state.
• Further, let En represent the number of times the system enters state n, and Ln
represent the number of times the system leaves state n.
• For all n, we have |En − Ln| ∈ {0, 1}, that is, the number of times the system
leaves a state differs by at most 1 from the number of times it enters that
state, since it will either return into that state at some time in the future (En =
Ln) or not (|En − Ln| = 1).
• When the system arrives at a steady state, the arrival rate should be equal to
the departure rate.
 Queuing Network Optimization
Note:
In the notation, the M stands for Markovian; M/M/1 means that
the system has a Poisson arrival process, an exponential service time
distribution, and one server.
 Queuing Network Optimization
Simple two-equation queue
• A common basic queuing system is attributed to Erlang, and is a
modification of Little's Law. Given an arrival rate λ, a dropout rate σ, and a
departure rate μ, length of the queue L is defined as:

• Assuming an exponential distribution for the rates, the waiting time W can
be defined as the proportion of arrivals that are served. This is equal to
the exponential survival rate of those who do not drop out over the
waiting period, giving:

• The second equation is commonly rewritten as:

• The two-stage one-box model is common in epidemiology

 Queuing Network Optimization

Service disciplines
Various scheduling policies can be used at queuing nodes:
• First in first out Also called first-come, first-served (FCFS), this
principle states that customers are served one at a time and that
the customer that has been waiting the longest is served first

First in first out (FIFO) queue example.

 Queuing Network Optimization
• Last in first out
– This principle also serves customers one at a time, but the
customer with the shortest waiting time will be served first. Also
known as a stack.
• Processor sharing
– Service capacity is shared equally between customers.
• Priority
– Customers with high priority are served first. Priority queues can
be of two types, non-preemptive (where a job in service cannot
be interrupted) and preemptive (where a job in service can be
interrupted by a higher-priority job). No work is lost in either
model.
 Queuing Network Optimization
• Shortest job first
– The next job to be served is the one with the smallest size
• Preemptive shortest job first
– The next job to be served is the one with the original smallest
size
• Shortest remaining processing time
– The next job to serve is the one with the smallest remaining
processing requirement.
• Service facility
– Single server: customers line up and there is only one server
– Several parallel servers–Single queue: customers line up and
there are several servers
– Several servers–Several queues: there are many counters and
customers can decide going where to queue
 Queuing Network Optimization

• Customer's behavior of waiting

– Balking: customers deciding not to join the queue if it is too long
– Jockeying: customers switch between queues if they think they
will get served faster by doing so
– Reneging: customers leave the queue if they have waited too
long for service
• Arriving customers not served (either due to the queue having no
buffer, or due to balking or reneging by the customer) are also
known as dropouts and the average rate of dropouts is a significant
parameter describing a queue.
 Queuing Network Optimization

Queueing networks
• Networks of queues are systems in which a number of queues are
connected by what's known as customer routing. When a customer
is serviced at one node it can join another node and queue for
service, or leave the network.
• For networks of m nodes, the state of the system can be described
by an m–dimensional vector (x1, x2, ..., xm) where xi represents the
number of customers at each node.
• The simplest non-trivial network of queues is called tandem
queues. The first significant results in this area were Jackson
networks, for which an efficient product-form stationary
distribution exists and the mean value analysis which allows
average metrics such as throughput and sojourn times to be
computed.
 Queuing Network Optimization

• If the total number of customers in the network remains constant

the network is called a closed network and has also been shown to
have a product–form stationary distribution in the Gordon–Newell
theorem.
• This result was extended to the BCMP network where a network
with very general service time, regimes and customer routing is
shown to also exhibit a product-form stationary distribution.
• The normalizing constant can be calculated with the Buzen's
algorithm, proposed in 1973.
• Networks of customers have also been investigated, Kelly networks
where customers of different classes experience different priority
levels at different service nodes.
 Queuing Network Optimization

Routing algorithms
• In discrete time networks where there is a constraint on which
service nodes can be active at any time, the max-weight scheduling
algorithm chooses a service policy to give optimal throughput in the
case that each job visits only a single person service node.
• In the more general case where jobs can visit more than one node,
backpressure routing gives optimal throughput.
• A network scheduler must choose a queuing algorithm, which
affects the characteristics of the larger network
TIME SERIES AND REGRESSION
MODELS
 TIME SERIES AND REGRESSION MODELS
Describe relationships and make predictions from time series data
• Time series regression is a statistical method for predicting a future
response based on the response history (known as autoregressive
dynamics) and the transfer of dynamics from relevant predictors.
• Time series regression can help you understand and predict the
behavior of dynamic systems from experimental or observational
data.
• Time series regression is commonly used for modeling and
forecasting of economic, financial, and biological systems.
• You can start a time series analysis by building a design matrix (Xt),
which can include current and past observations of predictors
ordered by time (t).
• Then, apply ordinary least squares (OLS) to the multiple linear
regression (MLR) model
 TIME SERIES AND REGRESSION MODELS
yt=Xtβ+ut
• To get an estimate of a linear relationship of the response (yt) to the
design matrix. β represents the linear parameter estimates to be
computed and (ut) represents the innovation terms.
• The residual terms can be extended in the MLR model to include
heteroscedasticity or autocorrelation effects.
Other models that capture dynamics more explicitly include:
– Autoregressive integrated moving average with exogenous predictors
(ARIMAX)
– Regression models with ARIMA time series errors
– Distributed lag models
The choice of model depends on your goals for the analysis and the
properties of the data.
Evaluation of large scale Models
 Evaluation of large scale Models

• What is Monitoring and Evaluation (M&E)?

Monitoring is the collection and analysis of information about a
project or programme, undertaken while the project/programme is
ongoing.
Evaluation is the periodic, retrospective assessment of an
organisation, project or programme that might be conducted
internally or by external independent evaluators.
 Evaluation of large scale Models

A broader way of thinking about M&E is Action Research

• Action Research is a term for a variety of methodologies that at

their core are cycles of planning, action and reflection.
• This is a useful approach when thinking about how to integrate your
M&E into on-going plans and activities.
• There are many Action Research methodologies which could be
used as part of your M&E.
• A good overview and resources can be found at the Ashridge Centre
for Action Research Particular methods that you may find useful are
Co-operative Inquiry and Appreciative Inquiry.
Evaluation of large scale Models
 Evaluation of large scale Models
1. WHY DO M&E ?
• Monitoring and evaluation can sometimes seem like an
unaffordable luxury, an administrative burden, or an unwelcome
instrument of external oversight. But if used well, M&E can become
a powerful tool for social and political change.

Doing M&E can help you assess what difference you are making and
can provide vital intelligence, for example to help you:
• Assess and demonstrate your effectiveness in achieving your
objectives and/or impacts on people’s lives;
• Improve internal learning and decision making about project design,
how the group operates, and implementation i.e. about success
factors, barriers, which approaches work/ don’t work etc;
 Evaluation of large scale Models
• Empower and motivate volunteers and supporters;
• Ensure accountability to key stakeholders (e.g. your community,
your members/supporters, the wider movement, funders,
supporters);
• Influence government policy;
• Share learning with other communities and the wider movement;
• Contribute to the evidence base about effectiveness and limits of
community action
 Evaluation of large scale Models
2. AGREEING SOME GUIDING PRINCIPLES
It is useful to develop some guiding principles to ensure that your
M&E is relevant, useful, timely, and credible.
Some examples might include making sure the M&E and/or
information you collect is:

• focused and feasible in relation to your available resources so that it

supports rather than diverts resources from action (i.e. make sure
you focus information collection on what you ‘need to know’, not
on what would be ‘nice to know’);
• useful and timely information to improve group learning, group
decision making, and project design;
• useable by, and/or comparable to, data collected by other
stakeholders so it contributes to the wider evidence base;
 Evaluation of large scale Models

• credible, valid and reliable to the extent possible within your

available resources;
• sensitive to unequal power relations when you collect information
(i.e. ensure that you listen to people who might be marginalised in
the community or do not have a strong voice);
• ethical e.g. in relation to data consent and protection.
 Evaluation of large scale Models

3. DECIDING WHICH PROGRAMMES/PROJECTS YOU NEED TO

MONITOR

• It is important to decide and prioritise the programmes or projects

need to be monitor as it is unlikely that will have the resources to
monitor all the interventions at the same time.
• So it is necessary to think about which programmes or projects
need to assess; over what time period; and whether it is an on-
going activity which requires monitoring or a completed activity
which requires evaluation.
 Evaluation of large scale Models
4. DECIDING WHO TO INVOLVE IN THE DIFFERENT STAGES OF YOUR
M&E
• To ensure M & E is relevant to stakeholders it is important to
consider their information needs, as well as our own.
• Therefore it is needed to identify the key internal and external
stakeholders, and decide how to involve them in the design,
implementation, analysis and/or communication of findings.
Examples of people you might want to include are
(a) people directly involved in the projects
(b) stakeholders in the wider community (geographic or
community of interest) such as specific groups of residents, specific
networks, community groups, the wider movement, and/or
(c) external stakeholders e.g. funders, local and national policy
makers. It might also be possible to work in partnership University
departments.
 Evaluation of large scale Models

5. DECIDING THE KEY ISSUES AND QUESTIONS YOU WILL WANT TO

INVESTIGATE
The next key step is to identify the issues.
These often include:
Issues and questions to internal group
• Organisational capacity/group processes – how well are you
working together in relation to the following?:
– needed resources (human, financial, technical)
– leadership and vision
– management (e.g. clarity about aims, objectives, roles &
responsibility; adaptability)
– cost effectiveness
– sustainability (e.g. finance and/or volunteer burn out)
 Evaluation of large scale Models

• Joint working – how well are you working with others, for example
– in relation to partnerships, the wider movement, alliances,
coalitions
– disseminating or sharing good practice and techniques
Issues and questions to external group
• Relevance/acceptability - how relevant are our projects to different
demographic sections of the community?
• Effectiveness – Is the project is achieving objectives (e.g. in relation
to attitudes & values; behaviours; public support; community
capacity/local resilience; the wider movement; improved policies &
increased democratic space)? What internal or external factors are
facilitating/constraining progress?
 Evaluation of large scale Models

• Impact - what is the impact on people’s lives (e.g. in relation to the

ultimate changes in people’s lives or environment as a result of our
initiatives)?
• Contribution/attribution - what are all the contribution have made to
outcomes and impacts (in relation to other factors/actors)?
Decision is needed, whether or not to monitor or evaluate all of
these questions or just some. This is likely to require balancing
information needs with available resources.
 Evaluation of large scale Models
6. CLARIFYING YOUR AIMS, OBJECTIVES, ACTIVITIES AND PATHWAYS
TO CHANGE
Definitions
• In order to assess progress you need to know what you are trying to
achieve and how: that is, your aims, objectives and planned
activities.
Evaluation of large scale Models
 Evaluation of large scale Models

A simplified impact chain looks like this:

In practice your impact chain is unlikely to be linear: there may be

multiple outcomes and impacts and there may be interactions and
feedback loops between different parts of the pathway.
CHANGE ASSUMPTIONS
A change pathway/ impact chain can be useful because it reveals the
interrelationships between activities, outcomes and impacts and
therefore, also the change assumptions or theory about how the
change will be achieved. These assumptions are often implicit rather
than explicit so it may not even be aware of them.
 Evaluation of large scale Models
Examples of change assumptions
 Evaluation of large scale Models

7. IDENTIFYING WHAT INFORMATION YOU NEED TO COLLECT

Generally you are likely to need information to:
• Track and assess what has changed (both intended and
unintended);
• Understand the reasons for changes - i.e. what
factors/organisations/individuals have facilitated/constrained
change (including your contribution);
• Interpret the changes i.e. people’s perceptions and experiences of
change.
 Evaluation of large scale Models

The information you collect might either be

• Quantitative information expressed in numerical terms as numbers
and ratios for example. This information will allow you to answer
‘what’, ‘how many’ and ‘when’ questions.
• Qualitative information is expressed through descriptive prose and
can address questions about ‘why’ and ‘how’, as well as
perceptions, attitudes and beliefs.
 Evaluation of large scale Models
Examples of Indicator themes Note: these are example of possible indicator
themes, rather than precise indicators, and are by no means a complete list
Evaluation of large scale Models
Evaluation of large scale Models
Evaluation of large scale Models
 Evaluation of large scale Models

8. DECIDING HOW YOU WILL COLLECT THE INFORMATION INTERNAL

MONITORING
Generally it is preferable to collect data from a range of internal and
external sources. Some useful information collection methods for
internal monitoring include:
• Internal records to track project activities, processes and output
indicators;
• Keeping records of relevant secondary information to track changes
in outcomes and impacts and accompany internal records, such as
policy changes, media coverage, relevant surveys/databases
• Periodic group workshops, discussions, focus groups (including
group ratings/ranking exercises and/or other visual techniques such
as time lines, mapping, diagrams; other diagnostic tools)
 Evaluation of large scale Models

• Periodic surveys (e.g. to assess attitudes, event feedbacks and/or

behaviour change)
• Use of automated household or community tools, modelled data or
conversion ratios e.g. carbon calculators
Example of indicators & information collection methods
Evaluation of large scale Models
 Evaluation of large scale Models
9. ASSESSING YOUR CONTRIBUTION/INFLUENCE
Randomised controls
• For some academics and policy makers, the only objective way to
assess attribution is through surveys over a defined period of time,
which compare changes in the communities taking part in the
project to changes in communities who are not taking part in the
project (either a randomised or purposively selected control group).
• Any differences in outcomes or impacts can then be argued to have
been caused by the project.
• Alternatively, there might be able to make some comparisons of the
outcome and impact indicators with data from long-term local
and/or national surveys/databases if similar samples and indicators
have been used.
 Evaluation of large scale Models

Retrospective assessments
• One alternative to using longitudinal surveys and randomised
controls is to select a random sample of people from both a
‘project’ and ‘control’ community and ask them to retrospectively
rate or rank the influence or impact of a range of selected
factors/organisations/individuals, including the project, on any
observed outcomes or impacts.
• Another alternative is to periodically (e.g. every few years)
commission an independent evaluator/facilitator to explore our
contribution to observed outcomes and impacts, such as through
focus groups, group workshops and/or interviews with a range of
internal and external stakeholders (e.g. from organisation,
community, local council, media, government etc).
 Evaluation of large scale Models

10. ANALYSING AND USING THE INFORMATION

Information is only useful if it is analysed and put to good use. A key
purpose of monitoring is to support internal decision making and
planning, so you need to ensure you periodically analyse, assess,
and actually use the information you collect.
Analysis
Some tips for analysing information include:
• Qualitative information Identify categories, themes & data (this is
called ‘coding’) Interpret findings in relation to research question
Watch out for unintended results & data that does not fit your
expectations
• Quantitative information Calculate simple totals, averages, and
percentages, and statistical tests (if appropriate)
• General Check credibility, validity and reliability Invite periodic
external verification/evaluation
 Evaluation of large scale Models
Using and Communicating the Data
When you have analysed the data you could hold periodic internal
meetings and/or organise a specific evaluation workshop to share,
discuss and interpret findings. You should use the data to answer
your initial key questions. You might also draw out learning for ot
Internal learning
• What is working well and not so well, and why?
• What have we learnt about how to achieve change i.e. how plausible
were our change assumptions?
• What changes do we need to make to our change strategy,
understanding of change, and ways of working?
 Evaluation of large scale Models

Learning for other stakeholders

• What are the lessons for other practitioners –e.g. other
communities, local authorities, intermediary or support
organisations?
• What are the lessons for national policy? her stakeholders.

11. COMMUNICATING THE DATA

Depending on the purpose of your M&E it may also be
important to communicate the data to relevant stakeholders.
This might involve:
• Deciding key audiences e.g. community group, community,
donors, policy makers and the media
 Evaluation of large scale Models

• Tailoring and packaging the data to key stakeholders/audiences

• Converting data into graphs, pie charts etc
• Drawing out key lessons for key stakeholders/audience

12. ETHICS AND DATA PROTECTION

It is very important that you gain informed consent from research
respondents/participants, ensure their anonymity in the
communication of research findings, and respect data protection
laws.