Download as ppt, pdf, or txt
Download as ppt, pdf, or txt
You are on page 1of 114

Risk and opportunity

Part 1

Tor Stlhane
Torbjrn Skramstad
Time Topics
09:00 Risk and opportunity
What is it and why do we need to manage it?
Why is opportunity assessment important?
Why should we worry about risk and opportunity in SPI?
The SWIR model and how to use it.
Exercise Application of the SWIR model
09:50 Coffee break
10:00 Assessment and brainstorming
The human bias
Qualitative assessment
Simple brainstorming techniques
Some important diagrams for risk assessment
Exercise Build an event tree
10:50 Coffee break
11:00 Simple risk and opportunity assessment
Risk and opportunity management - barriers and enablers.
The ROP - Risk and Opportunity Pattern
Exercise Application of the ROP in SPI
11:50 Coffee break
12:00 Leverage as a decision tool
Extended risk and opportunity assessment
The ALARP and GALE concepts when is enough really enough?
The CORAS model quantitative and qualitative assessment
Exercise Application of the GALE concept
Important things to remember summing it up
12:30 Lunch

EuroSPI 2006 - Part 1 2


Contents of part 1
What is risk and what is opportunity
Why should we care
Assessing risk and opportunity
Risk and opportunity in SPI the SWIR
and the SWIRO models
More on assessment
Brainstorming techniques

EuroSPI 2006 - Part 1 3


Risk and opportunity
Risk and opportunity have three things in
common:
They are concerned with events that may
or may not happen in the future.
The events are identifiable but their effect
are uncertain, although less uncertain than
the probabilities.
The outcome of the events can be
influenced by our actions
EuroSPI 2006 - Part 1 4
What is risk
A risk is something that can be a problem in
the future. It is defined by two parameters
The consequences - C. What will happen
if the risk becomes a problem?
The probability - p. What is the probability
that the risk will become a problem?

The risk R is defined as R = C*p


EuroSPI 2006 - Part 1 5
What is opportunity
An opportunity is something that can be
beneficial in the future. It is defined by two
parameters
The value - V. What will happen if the
opportunity becomes a reality?
The probability - p. What is the probability
that the opportunity will be realized?

The opportunity O is defined as O = V*p


EuroSPI 2006 - Part 1 6
Why should we care -1
Risks may turn into problems. We can reduce or
avoid future problems by reducing their
consequences or their probabilities. This can be
done by
Changing the way we work to
Replace a high risk activity with a low risk activity.
Remove the risk possibility
Adding risk avoidance activities to the way we
work

EuroSPI 2006 - Part 1 7


Why should we care - 2
Opportunities may turn into benefits. We can
increase future benefits by increasing their
probabilities. This can be done by
Changing the way we work replace a low
opportunity activity with a high opportunity
activity.
Adding opportunity enabling activities to
the way we work

EuroSPI 2006 - Part 1 8


Assessing risk and opportunity
Both risk and opportunity is defined by value
and probability.
Experience and data are important for two
reasons. They can:
Be used to estimate values and
probabilities.
Serve as an anchor for assessment e.g.
How bad can it get?
EuroSPI 2006 - Part 1 9
Risk and improvement
All SPI activities implies change and all
changes carries their own risks and
opportunities.
We will present two relevant models called
SWIR and SWIRO respectively.
The purpose of these models is to identify
risks and opportunities in SPI work.

EuroSPI 2006 - Part 1 10


The SWIR model -1
The SWIR model is the SPI version of the
SWOT model.
SWOT Strengths, Weaknesses,
Opportunities and Threats.
SWIR Strengths, Weaknesses,
Improvements and Risks.

EuroSPI 2006 - Part 1 11


The SWIR model - 2
Strengths Weaknesses
Were shall we win? What are our weak sides?

Improvements Risks
Where shall we improve What can go wrong?
ourselves? Which opportunities can
we loose

EuroSPI 2006 - Part 1 12


The SWIR components - 1
Strengths we need to know and understand
our strong sides so that we
do not destroy them in the SPI process
can build on them and improve them
Weaknesses must be known so that we
understand what we are up against.
Improvements what we want to achieve. They
must be discussed and understood together with
our strengths and weaknesses.

EuroSPI 2006 - Part 1 13


The SWIR components - 2
Risks potential problems that we have to
cope with. They can stem from:
Our weak sides
Changes that are a necessary part of the SPI
process.
Threats to our strong side things that must
be kept the way they are.

EuroSPI 2006 - Part 1 14


The SWIRO model - 1
The SWOT model includes opportunities but
ignore improvements
The SWIR model includes improvements but
ignores opportunities.
It might be a good idea to merge these two
models so that we have a unified
presentation of strengths, weaknesses,
risk, opportunities and improvements.

EuroSPI 2006 - Part 1 15


The SWIRO model - 2
Strengths Weaknesses
Where shall we win? What are our weak
sides?
Improvements Risks
Where shall we improve What can go wrong?
ourselves?
Current opportunities New opportunities
Which opportunities do Which new opportunities
we have now? will the change open
up?
EuroSPI 2006 - Part 1 16
A caveat
None of the presented models SWOT,
SWIR or SWIRO will help us to assess
the risks and opportunities.
The models are just used to get a complete
picture of the situation.
Assessment is the logical next step.

EuroSPI 2006 - Part 1 17


Exercise
You are considering the introduction of an
ISO conform process into your company.

Fill in the SWIR or SWIRO diagram.

EuroSPI 2006 - Part 1 18


Assessment - 1
Even though assessment is a subjective
activity it is not about throwing out any
number that you like.
To be useful, an assessment must be
Based on relevant experience.
Anchored in real world data.
The result of a documented and agreed-
upon process.
EuroSPI 2006 - Part 1 19
Assessment - 2
Risk and opportunity assessment is critically
dependent on the persons who participate,
their experience and their knowledge.
Experiments have shown that people have
some biases which implies that we need to
be careful when we look at the identified
risk events and their assessed
consequences and probabilities.

EuroSPI 2006 - Part 1 20


The human bias
Two human biases are important:
Omission bias - most persons prefer doing
nothing instead of an action if the
consequences have equal values.
Status quo bias - people assign a larger
risk to change than to maintaining status
quo. This bias increases if the change
action has the potential to create victims.

EuroSPI 2006 - Part 1 21


Qualitative assessment
We can assess consequences, probabilities
and benefits qualitatively in two ways. We
can use:
Categories e.g. High, Medium and Low
Numbers e.g. values from 1 to 10.

EuroSPI 2006 - Part 1 22


Categories 1
When using categories, it is important to
give a short description as to what each
category implies. E.g. it is not enough to
say High consequences. We must relate
it to something already known, e.g.
Project size
Company turn-over
Company profit
EuroSPI 2006 - Part 1 23
Categories 2
Two simple examples:
Consequences: we will use the category
High if the consequence will gravely
endanger the profitability of the project.
Probability: we will use the category Low
if the event can occur but only in extreme
cases.

EuroSPI 2006 - Part 1 24


Impact and probability - 1
Impact

Probability H M L

H H H M
M H M L
L M L L
EuroSPI 2006 - Part 1 25
Impact and probability - 2
The multiplication table is used to rank risks
and opportunities. It can not tell us how
large they are.
We should only use resources on risks and
opportunities that are above a certain,
predefined level.

EuroSPI 2006 - Part 1 26


Numbers as categories -1
We can use numbers instead of names. This
does not make the assessment more
precise but will free us from the need to
define a multiplication table in order to
identify risks.
In principle we can use any numbers. The
best solution is, however, to just assign
number to the three aforementioned
categories

EuroSPI 2006 - Part 1 27


Numbers as categories 2
The following values are often used in
practice, both for consequences, benefits
and probabilities:
10 high
4 medium
1 low
Thus, a medium consequence and a low
probability will give a risk of 4*1 = 4.
EuroSPI 2006 - Part 1 28
Numbers as categories 3
Impact

Probability H / 10 M/3 L/1

H / 10 H / 100 H / 30 M / 10
M/3 H / 30 M/9 L/3
L/1 M / 10 L/3 L/1
EuroSPI 2006 - Part 1 29
Simple brainstorming techniques
Brainstorming is an efficient way to use the
creative abilities that each person have.
In its simplest form, people just generate
ideas and a person registers the ideas on
a whiteboard or a flip-over.
We can, however, use techniques to do
better.

EuroSPI 2006 - Part 1 30


Brainstorming and risks - 1
We can use previous experiences to answer
questions such as
Can this really happen; e.g. has it
happened before?
Can we describe a possible cause -
consequence chain for the event?
How bad can it get?
How often has this happened in the past?
EuroSPI 2006 - Part 1 31
Brainstorming and risks - 2
We can use techniques such as:
Affinity diagrams post it notes
Cause consequence diagrams, such as
Ishikawa diagrams also called fishbone
diagrams
Event trees
Cause consequence networks

EuroSPI 2006 - Part 1 32


Ishikawa diagram
Resources Planning

Wrong
personnel Estimation

Loose key Follow-up


personnel
Too late
delivery
Changes Tool X is not
working
Misunderstandings
Reuse problems

Requirements Development

EuroSPI 2006 - Part 1 33


Event trees
Found in unit test
Found in
Coding error integration test
Found in
systems test

Not found in
unit test
Not found in
integration test Delivered
Not found in to customer
systems test

EuroSPI 2006 - Part 1 34


Cause consequence diagram

E4
C1
C6 E1
C2 E7
E5
Acc E2
C3
E6

E3 E8
C4 C7

E6

C5

EuroSPI 2006 - Part 1 35


Change and risk
Changes can introduce risks. The main
reasons are that:
Any effect of a change is related to the
future and can thus not be certain
It is difficult to completely understand the
effect of changes in a complex,
sociological system

EuroSPI 2006 - Part 1 36


Change and opportunities
Changes can create new opportunities. The
opportunities are mostly
Indirect effects of what we do to achieve
our goals e.g. a new tool that can be
used in several ways
Additional effects of having achieved the
goals e.g. less need for rework frees
resources for developing a new product.

EuroSPI 2006 - Part 1 37


Risk and opportunity in SPI
Risk and opportunity are important in SPI.
We need to consider:
Cost related to the change.
Benefit, which is its planned purpose
Risk related to the change, since we are
going to work in a new way.
New opportunities that are opened up by
the changes
EuroSPI 2006 - Part 1 38
Exercise
You want to study the effect of document
inspection on the number of defects
delivered to the customer.

Build an event tree for the starting event


A defect has been introduced in high level
design

EuroSPI 2006 - Part 1 39


Next session
The next session will focus on
How to do simple risk and opportunity
assessment.
The introduction of barriers and enablers
into risk and opportunity assessment
How to use leverage to prioritize our
actions

EuroSPI 2006 - Part 1 40


Risk and opportunity
Part 2

Tor Stlhane
Torbjrn Skramstad
Contents of part 2
Simple risk assessment
Simple opportunity assessment
The total picture risk and opportunity
The risk and opportunity pattern
Barriers, enablers and leverage
Extended risk analysis
Extended opportunity analysis
Risk and regret
EuroSPI 2006 - Part 1 42
Simple risk assessment
In order to a simple risk assessment we
need to identify:
Dangerous events
Each events
consequence C
probability p
Possible barriers changes or controls
Person responsible for each risk - Resp.
EuroSPI 2006 - Part 1 43
Simple risk table

Event C p R Barriers Resp

EuroSPI 2006 - Part 1 44


Events
We start by identifying dangerous events.
The simple way to do this is to use
brainstorming.

The process is simple just sit down and


envisage your worst nightmares related to
the activities under consideration.
Be realistic only consider things that you
believe can happen.
EuroSPI 2006 - Part 1 45
Barriers
Barriers can be realized through:
Prevention we change our process so
that the event cannot occur.
Mitigation we can
change the process in order to reduce the
events probability or consequences.
define activities that will reduce the problems
if the event occurs.

EuroSPI 2006 - Part 1 46


Prevention barriers Handling barriers
Prevent risk from becoming Prevent event from having
a problem bad consequences

Barrier 6
Barrier 1

Barrier 2

Barrier 3

Barrier 5
Barrier 4
Risk Prob. Event

Reduction barriers
Reduce effect
of event

EuroSPI 2006 - Part 1 47


Simple opportunity assessment
In order to assess opportunities, we need to
identify:
The event that opens up opportunities -
enablers
Each opportunitys
realizable value V
probability - p
The activity needed to realize the value
Person responsible for each opportunity
EuroSPI 2006 - Part 1 48
Simple opportunity table

Enabler

Opportunity V p O Enabling Resp.


activity

EuroSPI 2006 - Part 1 49


Enablers
Any action e.g. a change can create an
opportunity enabler. Each enabler opens
up a set of opportunities.
Further actions are needed in order to
realize value.
Both enablers, opportunities and enabling
actions can be identified through
brainstorming.

EuroSPI 2006 - Part 1 50


Opportunity and risk
Assessing consequences and value:
H High. Will have large impact
M Medium. Should not be ignored
L Low. Can be ignored
Assessing Probability:
H High. Will happen quite often
M Medium. Will happen now and then
L Low. Will almost never happen
EuroSPI 2006 - Part 1 51
The total picture - 1
The total picture of the situation shows the
risks and the benefits that stem from a
planned change.
This is not a mechanism that can be used to
identify the best solution.
It is, however, an important input when we
want to make a decision.

EuroSPI 2006 - Part 1 52


The total picture - 2
The total picture shows risks, benefits and
opportunities. Risk can be shown in two
ways:
1. Unmitigated risks
2. Mitigated risks include the effect of risk
reduction activities, e.g. barriers. This
can be done by
Modifying the risk assessment
Indicate how the risk will move in the
diagram
EuroSPI 2006 - Part 1 53
Costs and benefits
Reduced number of
H MMI-related defects

B M
L
p L M H
Extra work needed for
L MMI-specification

C M
H
EuroSPI 2006 - Part 1 54
Unmitigated risks
Reduced number of
H MMI-related defects

B M
L
p L M H
Extra work needed for
L MMI-specification

C M
Large disagreements Partnership does not
H between designers and work
MMI experts

EuroSPI 2006 - Part 1 55


The mitigation effect
Reduced number of
H MMI-related defects

B M
L
p L M H
Extra work needed for
L MMI-specification

2
C M
Large disagreements Partnership does not
H between designers and work
MMI experts 1

EuroSPI 2006 - Part 1 56


Including opportunities
Reduced number of Better MMI
MMI-related defects requirements will
H Better MMI for existing reduce imp. costs
products
B
M
L
p L M H
Extra work needed for
L MMI-specification
2

C M
Large disagreements Partnership does not
H between designers and work
1
MMI experts
EuroSPI 2006 - Part 1 57
The tyranny of either or
All too often we are confronted by the
statement that we can get only get X if
we are willing to suffer Y.
This is the wrong attitude. The right attitude
is that we will
1. Do what is needed to get X
2. Perform activities that will remove or
reduce the bad effects of Y.
EuroSPI 2006 - Part 1 58
The risk and opportunity pattern
A pattern is a description of a standard way
to solve a common problem. The Risk and
Opportunity Pattern ROP is a way to
analyze and manage risk and opportunity.
ROP has two components:
A set of assessment and management
activities
A process that describe an activity
sequence
EuroSPI 2006 - Part 1 59
The ROP process
ROP consists of the following activities:
1. Define the job and its borders
2. Perform a risk assessment
3. Perform an opportunity assessment
4. Implement the identified barriers
5. Do the job while
controlling risks and preventing problems
searching for opportunity enables and harvesting
benefits

EuroSPI 2006 - Part 1 60


ROP activities risk part
Define the job and its borders. We
cannot consider everything only what is
inside the defined borders.
Perform a risk assessment.
Implement the barriers identified in the
previous step.
Do the job - control risks and prevent
problems.
EuroSPI 2006 - Part 1 61
Exercise
Your company consider buying a new test
administration tool. Management is unsure
whether this is a wise investment.

Use the risk part of ROP to help


management in their decision.

EuroSPI 2006 - Part 1 62


Barriers and enablers
Barriers and enablers will define actions that
will help us to
Avoid problems barriers
Reap benefits enablers

Identification of barriers and enablers is,


however, not enough. We also need to
assess how effective they are.
EuroSPI 2006 - Part 1 63
Leverage
Leverage is a prioritizing mechanism:

Leverage = (Benefit Cost) / Cost

Leverage will prioritize activities with


Large net benefits
Small costs

EuroSPI 2006 - Part 1 64


Extended risk table -1
We can use cause consequence chains or
event trees for a risk to identify the best
place to insert a barrier.
For each barrier, we need to assess:
Cost - the cost of implementing it. We will
use the scale H = 10, M = 3 and L = 1.
E how effective is the barrier? We will
use the scale h = 1.0, m = 0.5 and l = 0.2
EuroSPI 2006 - Part 1 65
Extended risk table - 2

Event C p R Barrier Cost E L Resp.

EuroSPI 2006 - Part 1 66


Barrier leverage

Leverage = (C*p*E Cost) / Cost

The leverage will prioritize barriers which:


Have low costs Cost is small
Have high efficiency E is large
Attack important risks C*p is high

EuroSPI 2006 - Part 1 67


Barrier example
Event Cons p R Mitigation E Cost Resp
L
.
Partnership does Do a thorough research John
not work business on selected partners
10 3 30 0.5 10 0.5
conflicts business goals

Customers do not State the conditions and Pete


prioritize project consequences of customer
participation 10 3 30 participation in the 1.0 3 9.0
contract

EuroSPI 2006 - Part 1 68


Some comments on barriers
It is important to remember that:
Each risk will usually need a different barrier a
barrier that works against one risk can be
valueless against another risk.
It is important to consider the three main barrier
strategies:
Prevent the risk from becoming a problem
Control the problem to avoid the consequences
Reduce the consequences

EuroSPI 2006 - Part 1 69


Extended opportunity table - 1
Even if an opportunity arises, nothing will
really happen if we do not do something to
realize it.
An enabler is an event that will help us to
reap a benefit.
Just as barriers, the activities linked to an
enabler have costs and effectiveness.
Thus, we can compute the leverage and
use this as a basis for our decisions.
EuroSPI 2006 - Part 1 70
Extended opportunity table - 2
Enabler

Opportu V p O Action Cost E L Resp.


nity

EuroSPI 2006 - Part 1 71


Opportunity leverage
Leverage = (V*p*E Cost) / Cost

The enabling activity leverage will prioritize


activities which:
Have low costs Cost is small
Have high efficiency E is large
Enable valuable opportunities V*p is
high
EuroSPI 2006 - Part 1 72
Enabler - example
Enabler Better understanding of how MMI requirements are implemented
and adapted

Opportunity Action Resp


Value p O E Cost L

Better MMI Use new Peter


requirements, knowledge to
which will make better
reduce imp. 10 10 100 MMI 1 3 32
costs requirements
spec
Use MMI more Redesign user Brian
actively to interface for
create more 10 3 30 products A and 1 10 2
popular B
products

EuroSPI 2006 - Part 1 73


An alternative presentation - 1
We have earlier used the cost-benefit
diagram to show benefits, opportunities,
costs and risks.
By including the efficiency of barriers and
enabling actions, we get a better picture of
the overall situation.
Since we already have performed the
necessary multiplications, we can use a
one-dimensional representation.
EuroSPI 2006 - Part 1 74
An alternative presentation - 2
The alternative representation is just a representation. It is
thus just one of several inputs to a decision.

100 30 10 10 30 100
Costs and
Benefits and
risks
opportunities

EuroSPI 2006 - Part 1 75


A small example - 1
We have the following assessed values:
Cost: C = medium, p = high, Cost = 30.0
Benefit: V = high, p = high, Benefit = 100.0
Risks
R1: C = medium, p = low, barrier efficiency = medium,
R1 = 1.5
R2: C = high, p = low, barrier efficiency = low, R2 =
8.0
Opportunities
O1: V = medium, p = high, enabling activity efficiency
= medium, O1 = 15.0
O2: V = high, p = high, enabling activity efficiency is
low, O2 = 20.0

EuroSPI 2006 - Part 1 76


A small example - 2

Cost R2 R1 O1 O2 Benefit

100 30 10 10 30 100
Costs and
Benefits and
risks
opportunities

EuroSPI 2006 - Part 1 77


Regret and risk - 1
Instead of just looking at cost and value of
an opportunity, we can include risk and
regret in the leverage expression.
Regret is the, often indirect, cost of skipping
or ignoring an opportunity.

Priority = (Value + Regret) / (Cost + Risk)

EuroSPI 2006 - Part 1 78


Regret and risk - 2
Just as cost, value and risk, regret has to
bee assessed, for instance on a scale
from 1 to 10 or just using three values
such as 10, 3 and 1.
As should be expected
High regret and low risk will give high
priority.
Low regret and high risk will give low
priority
EuroSPI 2006 - Part 1 79
Next session
The next session will focus on
Two risk assessment concepts ALARP
and GALE
How to use the GALE method
Quantitative assessment and the CORAS
model
Summing up - some important things to
remember
EuroSPI 2006 - Part 1 80
Risk and opportunity
Part 3

Tor Stlhane
Torbjrn Skramstad
Contents
ALARP and GALE
Using GALE
How to do risk assessment with GALE
A small example
Quantitative assessment
The CORAS model
A small example
Important things to remember
EuroSPI 2006 - Part 1 82
ALARP and GALE
There are two competing principles in the
assessment of risk:
ALARP As Low As reasonably Possible-
We have done all that is reasonable to
prevent problems and dangers.
GALE Globally At Least Equivalent. E.g.
introducing a new process will not
increase the risks compared to what it is
today.
EuroSPI 2006 - Part 1 83
ALARP
ALARP requires that we analyze each risk
separately and then implement mitigation
activities.
A reasonable goal is to reduce each risk
until the extra mitigation costs exceed the
value of the risk reduction achieved.
All that we have seen up till now fits into an
ALARP policy .

EuroSPI 2006 - Part 1 84


GALE
GALE requires us to look at the total risk of
a change. In this way we can start by
attacking the cheapest risk or the risk with
the largest leverage.
The problem with the GALE principle is that
we need to perform arithmetic on risks.
E.g. we need to decide how many medium
risks we need before we have a large risk

EuroSPI 2006 - Part 1 85


ALARP vs. GALE - 1
There is no such thing as the right risk
principle. It is always a matter of company
choice and company policy.
The two principles will lead to different
prioritization of mitigation activities.
ALARP each risk is reduced as much as
possible.
GALE we need to be below the present
risk level.
EuroSPI 2006 - Part 1 86
ALARP vs. GALE - 2
The one important thing with using the
GALE principle is that it forces us to ask
What is the current risk level?
All too often we act as it the current way of
doing things is risk free and all risk stems
from changes.
This stance is enforced by the human
tendency to underestimate the risk of
status quo.
EuroSPI 2006 - Part 1 87
Using GALE
Important points
GALE is a method for risk analysis.
Benefits must be included elsewhere
We need to look at both our current risk
and the risk resulting from the proposed
changes.
Always perform a sensitivity analyses.

EuroSPI 2006 - Part 1 88


Risk status quo vs. change
In many cases, maybe even in most of
them, we do risk assessment because we
want to compare two or more alternatives,
e.g.:
Status quo no changes
One or more changes - improvements

EuroSPI 2006 - Part 1 89


Event identification
All significant dangerous events must have
been identified.
There must be a minimal overlap between
the dangerous events .
There must be a maximum of commonality
between the dangerous events considered
for the status quo and for the system after
the proposed changes

EuroSPI 2006 - Part 1 90


The three event sets
The previous rules split the dangerous
events into three sets dangerous events
that:
Apply both to the status quo and to the
new system.
Are unique to the status quo
Are unique to the new system

EuroSPI 2006 - Part 1 91


GALE and risk assessment - 1
GALE uses the following parameters for risk
assessment:
FE the event frequency
PE the probability that the event will lead
to an accident
S the severity score of an event

EuroSPI 2006 - Part 1 92


GALE and risk assessment - 2
We can compute individual and
accumulated risk indices:

IE = FE + PE + S
IGR = log Sumi(10I)

IE is the risk index for a hazardous event


IGR is the global risk index
EuroSPI 2006 - Part 1 93
The GALE scoring scheme
The scoring scheme of GALE
Focuses on deviations from current
average. This is reasonable, given that it is
mainly concerned with comparing status
quo to a new situation.
Must be tailored to each situation. The
next slide shows an example from road
safety. We need a scheme adapted to SPI.

EuroSPI 2006 - Part 1 94


Road safety - frequency score for
event
Frequency
Occurrences / year on M42 ATM section FE
classification
Very frequent 10000 Hourly 6
Frequent 1000 A few times a day 5
Probable 100 Every few days 4
Occasional 10 Monthly 3
Remote 1 Annually 2
Improbable 0.1 Every 10 years 1
Incredible 0.01 Every 100 years 0

EuroSPI 2006 - Part 1 95


SPI and GALE
We need a special scoring scheme for
development projects. For events that can
lead to problems we need to consider:
How often does the event occur - FE?
If the event occurs, what is the probability
that it will cause a real problem - PE?
If the problem occurs, how severe will the
consequences be S?
EuroSPI 2006 - Part 1 96
SPI goals
Based on the GALE parameters, we can
also identify possible SPI goals:
S: reduce the consequences reduction
and handling barriers
FE: reduce the number of event
occurrences problem opportunities
PE: reduce the probability that the event
will cause a problem prevention barriers
EuroSPI 2006 - Part 1 97
Frequency score for event
Frequency
Occurrences per project FE
class
Very frequent 200 Every project 6
Frequent 100 Every few projects 5
Probable 40 Every 10th project 4
Occasional 10 Every 100th project 3
Remote 1 A few times in the companys
2
lifetime
Improbable 0.2 One or two times during the
1
companys lifetime
Incredible 0.01 Once in the companys
0
lifetime
EuroSPI 2006 - Part 1 98
Probability score for event
Classification Interpretation PE
Probable It is probable that this event, if it
occurs, will cause a problem 3

Occasional The event, if it occurs, will


occasionally cause a problem 2

Remote There is a remote chance that this


event, if it occurs, will cause a 1
problem
Improbable It is improbable that this event, if it
occurs, will cause a problem 0

EuroSPI 2006 - Part 1 99


Severity score for event
Severity
Interpretation S
class
Severe The portion of occurring problems that
have serious consequences is much 2
larger than average
Average The portion of occurring problems that
have serious consequences is similar 1
to our average
Minor The portion of occurring problems that
have serious consequences is much 0
lower than average

EuroSPI 2006 - Part 1 100


Sensitivity analysis
The global risk index is made of many
indices. Each index will have a certain
degree of uncertainty connected to it.
Usually, a few indices will have a large
influence on the result while the rest will
have but little influence.
Paretos rule applies - we need to identify
the few important indices.

EuroSPI 2006 - Part 1 101


A small example - 1
Status quo After process
improvement

Event S FE PE S FE PE

Too late delivery 1 1 5 3 1 4 3

Too high cost 2 1 5 3 2 4 3

Low customer satisfaction 3 1 4 3 0 3 2

Low developer satisfaction 4 1 4 2 0 3 2

Too low product quality - 5 1 4 2 0 3 2

EuroSPI 2006 - Part 1 102


A small example - 2
Status quo:
I1 = 9, I2 = 9, I3 = 8, I4 = 7, I5 = 7
After SPI activity:
I1 = 8, I2 = 9, I3 = 5, I4 = 5, I5 = 5

IGR = log Sum(10I)

Status quo: log Sum(10I) = 9.3


After SPI activity: log Sum(10I) = 9.0
EuroSPI 2006 - Part 1 103
A small example - 3
We see from the results that the risk
reduction is small from 9.3 to 9.0.
We also see that the main reason for this is
that we have increased quality but
increased the cost.
The main result from the GALE process is
that we need to find ways to increase the
quality without increasing our development
cost.
EuroSPI 2006 - Part 1 104
Quantitative assessment -1
In some cases, we can use numerical
values. This occurs if we can use
Experience to identify the cost of a
problem e.g. correcting an error or
loosing a customer.
Old data to identify a probability e.g. the
probability of missing a defect during
inspection.

EuroSPI 2006 - Part 1 105


Quantitative assessment - 2
Quantitative risks and opportunities give us
real values.
The usefulness of this is, however, limited
since it is difficult to find real values for all
risks and opportunities.
It is not obvious how we can compare
qualitative and quantitative risks or
opportunities

EuroSPI 2006 - Part 1 106


The CORAS model
CORAS was developed as a framework for
assessment of security risks.
What should concern us here, however, is
how they related the qualitative risk
categories, not to absolute values, but to
the companys turn-over.

EuroSPI 2006 - Part 1 107


The CORAS consequence table
Consequence values
Category Insignificant Minor Moderate Major Catastrophic

Measured
related to 0.0 0.1% 0.1 1.0% 1 5% 5 10% 10 100%
income

Reduce the
Measured resources of one Close down
No impact on
loss due to or more departments or Out of
business. Lost profits
impact on departments business business
Minor delays
business Loss of a couple sectors
of customers

EuroSPI 2006 - Part 1 108


The CORAS frequency table - 1
As we will see on the next slide, CORAS
allows us to interpret frequency in two
ways:
The number of incidents per year
The failing portion of demands
We will use the second interpretation but
instead of focusing on a system, we
related it to the number of projects, e.g.
SPI projects.
EuroSPI 2006 - Part 1 109
The CORAS frequency table - 2
Frequency values
Almost
Category Rare Unlikely Possible Likely
certain
Number of
Unwanted
1/100 1/100 1/50 1/50 - 1 1 - 12 > 12
incidents per
Year

Number of
Unwanted
1/1000 (1/500) 1/50 (1/25) 1/1
incidents per
Demand

Each Every
Unwanted Each five Each tenth
Interpretation thousand second
incident times the time the
of number of time the time the
never system is system is
demands system is system is
Occurs used used
used used

EuroSPI 2006 - Part 1 110


A small example
We have a company with 10 developers and an
estimated yearly turnover of NOK 10 millions.
We decide that the consequences of a late
delivery is medium, which gives a
consequence of 1 5% or NOK 100 000 to 500
000.
We decide that the event is likely to occur, which
gives us a p-value of 0.04
The expected loss is thus 4 000 to 20 000.

EuroSPI 2006 - Part 1 111


Exercise
Your company has decided to change
development process.
List all important events
Find the risk index for each event for
Status quo
The new development process

EuroSPI 2006 - Part 1 112


Important things to remember - 1
The most important things to remember:
Risk assessment is by its nature subjective.
Use group techniques and include all
stakeholders
Use simple techniques so that you do not
exclude one or more stakeholders
Anchor it in experience and available data will,
however, improve the quality
Subjective values like High must be anchored
in each companys reality. One companys
High may be another companys Low.

EuroSPI 2006 - Part 1 113


Important things to remember - 2
Include the effect of choosing status quo in
all SPI risk analyses.
Always include opportunities
Consider the three barrier categories
prevention, handling and reduction
Rank risks and opportunities according to
their leverage
The results from a risk assessment is just
one of several inputs to a decision

EuroSPI 2006 - Part 1 114

You might also like