Risk Analysis in Maritime Transportation: Principles, Frameworks and Evaluation

Department of Applied Mechanics
Risk analyses are widely used tools for

Risk analysis in maritime
Aalto-DD 107/2015
Floris Goerlandt
decision support. Nonetheless, the risk
analysis discipline has received much
criticism. Calls have been made for
increased focus on foundational issues, both transportation: principles,
in the general discipline and in application
areas. frameworks and evaluation
Risk analysis in maritime transportation: principles, frameworks and evaluation

Answering these calls, this thesis
investigates and proposes a number of risk
analysis principles, addressing concepts and
terminology, risk model reliability, risk Floris Goerlandt
models and prediction, risk model use and
the consideration of uncertainty and bias.
The principles are used as a basis for
developing two risk analysis frameworks:
one for policy-oriented and one for
operational maritime risk analysis. Finally,
the evaluation of risk models and analyses is
addressed, focusing on the issue of
credibility of a risk analysis.
Principles, frameworks and evaluation are

investigated and exemplified through
applications concerning accidental risk of
maritime transportation.
ISBN 978-952-60-6313-3 (printed) BUSINESS +

9HSTFMG*agdbdd+
ISBN 978-952-60-6314-0 (pdf) ECONOMY

ISSN-L 1799-4934
ISSN 1799-4934 (printed) ART +
ISSN 1799-4942 (pdf) DESIGN +
ARCHITECTURE
Aalto University
Aalto University
School of Engineering SCIENCE +
Department of Applied Mechanics TECHNOLOGY
www.aalto.fi
CROSSOVER
DOCTORAL
DOCTORAL DISSERTATIONS
DISSERTATIONS
Aalto University publication series
DOCTORAL DISSERTATIONS 107/2015

transportation: principles, frameworks
and evaluation
Floris Goerlandt
A doctoral dissertation completed for the degree of Doctor of

Science (Technology) to be defended, with the permission of the
Aalto University School of Engineering, at a public examination held
at the lecture hall 216 at Otakaari 4 (K1 building) on 30 October
2015 at 12.
Aalto University
School of Engineering
Research Group on Maritime Risk and Safety
Supervising professor
Prof. Pentti Kujala
Thesis advisors
Prof. Pentti Kujala
Dr. Jakub Montewka
Preliminary examiners
Prof. Emer. Bernardus Johannes Maria Ale
Delft University of Technology, the Netherlands
Dr. Xiaobo Qu
Griffith University, Australia
Opponents
Prof. Terje Aven
University of Stavanger, Norway
Aalto University publication series

DOCTORAL DISSERTATIONS 107/2015
© Floris Goerlandt
ISBN 978-952-60-6313-3 (printed)

ISBN 978-952-60-6314-0 (pdf)
ISSN-L 1799-4934
ISSN 1799-4934 (printed)
ISSN 1799-4942 (pdf)
http://urn.fi/URN:ISBN:978-952-60-6314-0
Unigrafia Oy
Helsinki 2015
Finland
Abstract
Aalto University, P.O. Box 11000, FI-00076 Aalto www.aalto.fi
Author
Floris Goerlandt
Name of the doctoral dissertation
Publisher School of Engineering
Unit Department of Applied Mechanics
Series Aalto University publication series DOCTORAL DISSERTATIONS 107/2015
Field of research Marine Technology and Naval Architecture
Manuscript submitted 29 April 2015 Date of the defence 30 October 2015
Permission to publish granted (date) 18 May 2015 Language English
Monograph Article dissertation (summary + original articles)
Abstract
Risk analyses are widely used tools for supporting decision making. Nonetheless, many
criticisms have been raised against the discipline of risk analysis, e.g. technical analyses having
a narrow focus, poorly examined claims of the ability of accurately measuring risk and lack of
standards for quality assurance and risk analysis evaluation. In response to these criticisms,
calls have been made for increased focus on these and other foundational issues, both in the
general risk analysis discipline and in the various application areas.
This thesis answers these calls for research addressing the underlying concepts and
principles of risk analyses, which are approached through applications focusing on the
accidental risk of maritime transportation. Focusing first on a set of foundational issues
underlying waterway risk analyses, it is established that many different definitions,
perspectives and scientific approaches co-exist in the application area. Through two case
studies of reliability of maritime risk models, previous research claiming that risk models
provide unreliable decision support, are confirmed for some maritime applications, thus
confirming the need for focusing on risk related principles.
Subsequently, a set of principles is presented, addressing concepts and terminology, risk and
prediction, risk model use and the consideration of uncertainty and bias. A framework is
introduced to communicate the scientific principles adhered to in a specific risk analysis.
Following this, the principles are translated in two risk analysis frameworks: one for policy-
oriented and one for operational risk analysis; the first leading to a quantitative and the second
to a qualitative risk characterization. In both, risk is understood as a concept referring to the
possible but uncertain occurrence of a situation where something of human value is at stake.
Risk models are used as putting forward an argument based on available evidence, as a tool
for communication between stakeholders and as a platform for thinking. Uncertainties and
value-laden biases are assessed, and some tools for communicating these are introduced. Both
frameworks are illustrated by extensive case studies. The first concerns accidental risk of oil
spills from tanker collisions in the Gulf of Finland. The second focuses on a risk-informed ship-
ship collision alert system.
A final issue addressed in the thesis concerns the evaluation of a risk analysis, i.e. principles
and criteria for establishing credibility. An integrated framework for this is developed,
addressing model use, model plausibility, value-related validity and process-related validity.
Specific evaluation criteria are proposed and a selection of these is applied in the presented
case studies.
Keywords Risk analysis, foundational issues, waterway risk, collision alert system
ISBN (printed) 978-952-60-6313-3 ISBN (pdf) 978-952-60-6314-0
ISSN-L 1799-4934 ISSN (printed) 1799-4934 ISSN (pdf) 1799-4942
Location of publisher Helsinki Location of printing Helsinki Year 2015
Pages 167 urn http://urn.fi/URN:ISBN:978-952-60-6314-0
Preface
This thesis is based on work carried out at the Maritime Risk and Safety re-
search group at the Department of Applied Mechanics in Aalto University. The
work has been conducted within the RescOp and WINOIL projects, which are
co-funded by the European Union, the Russian Federation and the Republic of
Finland, and the project FAROS, funded by the European Union under the
FP7-program. Financing for finalizing the thesis has been received from Me-
renkulun Säätiö. This financial support is gratefully acknowledged.
First, I would like to warmly thank my supervisor and advisor, prof. Pentti
Kujala. His support and encouragement was important during the entire peri-
od of my doctoral studies. I particularly appreciate how Pentti has allowed me
the freedom to pursue my own research interests, over and beyond the needs
from the projects I have been involved in. I also sincerely thank my second
advisor, dr. Jakub Montewka. His support, guidance and devoted enthusiasm
for research have been very valuable during the entire process. I am also
thankful to my final co-author Vladimir Kuzmin for his contributions to the
research. Special thanks also go to prof. emer. Petri Varsta for reading the the-
sis draft, and for providing comments which helped to more clearly articulate
the flow of ideas. I also want to express my gratitude to prof. emer. B.J.M.
(Ben) Ale and dr. Xiaobo (Bob) Qu who acted as the pre-examiners of this the-
sis and provided useful remarks, and to prof. Terje Aven for accepting to be
opponent despite numerous other commitments and activities.
I would also like to thank my department colleagues, especially but not only
the current and former post-graduate students from the Maritime Risk and
Safety research group, for the peer support, practical help and for creating a
pleasant work environment. The secretaries deserve a special mention for solv-
ing many practical daily problems, as well as the staff at the Kotka Maritime
Research Centre for facilitating the projects in which the research was done.
Finally, I am thankful to my family and friends, especially to my wife Hanna
for the continuous support. I particularly appreciate her dedication to and care
for the increasingly busy household during these thesis years. I thank my chil-
dren Leila, Sonja, Boris and Alma for the happiness and joy they have brought
to my life, and for tidying up their toys before going to bed.
Espoo, Wednesday, 20 May 2015
Floris Goerlandt
1
Contents
List of Publications .................................................................................. 5

Author’s Contribution.............................................................................. 7
Original features ...................................................................................... 9
Special terms .......................................................................................... 11
Abbreviations .......................................................................................... 13
1. Introduction................................................................................. 15
1.1 Motivation: Risk Analysis as an unsettled scientific discipline15
1.2 Objectives and structure of the thesis ...................................... 16
1.3 Limitations ............................................................................... 17
2. Background .................................................................................. 19
2.1 Scientific approaches to Risk Analysis..................................... 19
2.2 Reliability, validity and evaluation of risk analysis ................ 22
2.3 State of the art in maritime transportation Risk Analysis ...... 24
3. Analysing Risk Analysis: the need for clarifying principles ....... 27
3.1 An analysis of foundational issues through applications ....... 27
3.2 Analysing reliability: two case studies .................................... 30
3.3 Research questions ................................................................. 32
4. Risk and model-based risk analysis: principles ......................... 33
4.1 Terminological-conceptual basis ............................................ 33
4.2 Risk Analysis: modelling, prediction and uncertainties/biases38
4.3 Implications of the adopted principles to Risk Analysis ........ 42
5. Frameworks for risk analysis...................................................... 43
5.1 Policy-oriented risk analysis: accidental risk in a waterway .. 43
5.2 Operational risk analysis: risk in collision avoidance context 46
6. Evaluating risk models and analyses .......................................... 49
6.1 Evaluation (not validation) of a risk model/analysis ............. 49
6.2 Evaluation of a model-based risk analysis: framework .......... 50
7. Conclusions and future work .......................................................57
Bibliography .......................................................................................... 59
3
4
List of Publications
This doctoral dissertation consists of a summary and of the following publica-

tions which are referred to in the text by their Roman numerals.
PI. Goerlandt, Floris; Montewka, Jakub. 2015. Maritime transportation risk

analysis: review and analysis in light of some foundational issues. Reliability
Engineering and System Safety, volume 138, pages 115-134. ISSN 0951-8320.
DOI 10.1016/j.ress.2015.01.025.
PII. Goerlandt, Floris; Kujala, Pentti. 2014. On the reliability and validity of
ship-ship collision risk analysis in light of different perspectives on risk. Safety
Science, volume 62, pages 348-365. ISSN 0925-7535. DOI
10.1016/j.ssci.2013.09.010.
PIII. Goerlandt, Floris; Montewka, Jakub. 2015. A framework for risk analysis
of maritime transportation systems: a case study for oil spill from tankers in a
ship-ship collision. Safety Science, volume 76, pages 42-66. ISSN 0925-7535.
DOI 10.1016/j.ssci.2015.02.009.
PIV. Goerlandt, Floris; Montewka, Jakub; Kuzmin, Vladimir; Kujala, Pentti.

2015. A risk-informed ship collision alert system: framework and application.
Safety Science, volume 77, pages 182-204. ISSN 0925-7535. DOI
10.1016/j.ssci.2015.03.015.
5
Author’s Contribution
Publication I: Maritime transportation risk analysis: review and analysis in

light of some foundational issues
The author developed the idea, prepared the framework for analysis, carried
out the analysis and was the main contributor to the manuscript. Montewka
provided valuable comments and suggestions.
Publication II: On the reliability and validity of ship-ship collision risk anal-
ysis in light of different perspectives on risk
The author developed the idea, carried out the analyses for the case study, and
was the main contributor to the manuscript. Kujala provided valuable com-
ments and suggestions.
Publication III: A framework for risk analysis of maritime transportation

systems: a case study for oil spill from tankers in a ship-ship collision
The author developed the framework, constructed the model, carried out the
analyses and evaluations for the case study, and was the main contributor to
the manuscript. Montewka provided valuable comments and suggestions.
Publication IV: A risk-informed ship collision alert system: framework and

application
The author developed the idea, prepared the expert elicitations, implemented
the model, carried out the analyses and evaluations for the case study, and was
the main contributor to the manuscript. Montewka assisted in the expert elici-
tation and contributed to the manuscript. Kuzmin assisted in the expert evalu-
ation of the developed model and contributed to the manuscript. Kujala pro-
vided valuable comments and suggestions.
7
Original features
This thesis aims to contribute to selected foundational issues within the appli-
cation area of maritime transportation Risk Analysis. Focus is on the princi-
ples underlying risk analysis applications as tools tool for informing decisions,
on proposing frameworks for building applications, and on principles and
methods for evaluating these. The applications concern policy-oriented mari-
time waterway risk analysis and operational risk analysis in a collision avoid-
ance context. The following features of this thesis are believed to be original:
1. A detailed analysis is conducted of adopted risk definitions, perspec-

tives and scientific approaches to risk analysis in applications address-
ing accidental risk in maritime waterways. Dependencies between defi-
nitions, perspectives and approaches are identified. [PI]
2. The reliability of policy-oriented and operational maritime transporta-

tion risk analysis applications is systematically studied. [PII, PIV]
3. A two-stage policy-oriented risk analysis framework for maritime

transportation systems is developed, including a risk-theoretical basis
linked with the applied measurement tools. Tools for contextualizing
the quantitative risk picture are proposed. [PIII]
4. A two-stage risk analysis framework for ship collision alert systems in

an operational context is developed, including a risk-theoretical basis
linked with the applied measurement tools. A framework for opera-
tionalizing the construct “ship-ship collision risk” is proposed. [PIV]
5. A Bayesian Network model for oil spill risk analysis for tanker colli-
sions is developed, focusing on the potential occurrence of spills of var-
ious sizes [PIII]
6. A Fuzzy Expert System based model is developed for providing risk-

informed ship-ship collision alerts in an open sea area [PIV]
7. Principles and methods for evaluating policy-oriented and operational

risk analysis applications in maritime transportation systems are pro-
posed [PIII, PIV]
9
Special terms
maritime transportation an activity in which ships transport goods from

one location to another over sea or waterway areas
operational risk analysis a risk analysis which is continuously performed

to support decision making in an ongoing operation which may require
fast decision making and action
policy-oriented risk analysis a risk analysis which is performed at a given

point in time or is performed periodically, to support a decision in a policy
context which has implications to investments and societal concerns
principle a fundamental proposition that serves as a foundation for a system

of beliefs or for a chain of reasoning
Risk Analysis the scientific risk discipline, concerned with developing con-
cepts, principles, frameworks and models for analysing risk (Aven, 2012a)
risk analysis A specific case study in which risk is analysed (Aven, 2012a)
risk analysis evaluation a process of building confidence in a given risk

analysis, a process of appraising or valuing the analysis and its results
risk analysis framework a basic structure, a set of ideas, mechanisms and

tools that provide support for performing a risk analysis
risk concept the ‘general idea’ of what risk is as a constituent of thought
risk definition the result of an intellectual activity to delineate a meaning of

the risk concept
risk perspective a way to describe risk, a systematic manner to analyse and

make statements about risk
reliability the extent to which a measurement procedure leads to the same

result when the measurement is repeated
scientific approach to Risk Analysis a set of principles adhered to as a

basis for performing a risk analysis, in particular whether realist, con-
structivist or proceduralist foundations are adopted
validity the extent to which a measurement procedure adequately describes

the concept one intends to describe
11
Abbreviations
BN Bayesian Network
BSQ background situational quality
CAS collision alert system
COLREGS international regulations for preventing collisions at sea
DCPA distance at closest point of approach
FES fuzzy expert system
FSA formal safety assessment
FSQ foreground situational quality
MF membership function
QRA quantitative risk analysis
SQ situational quality
TCPA time to closest point of approach
VTS vessel traffic service
13
1. Introduction
1.1 Motivation: Risk Analysis as an unsettled scientific discipline
Risk analyses are widely used for decision support. In the maritime application
area, many risk analysis approaches and models have been proposed for vari-
ous purposes. This includes regulatory decision making concerning ship de-
sign and equipment (IMO, 2008, 2007), ship design (Klanac and Varsta, 2011;
Papanikolaou, 2009), prevention and mitigation of maritime transportation
accidents in waterways (Li et al., 2012; Özbaş, 2013) and operational collision
avoidance (Statheros et al., 2008; Tam et al., 2009). Much research has been
dedicated to developing advanced mathematical methods for risk analysis, and
many applications for specific problems have been presented in the literature.
Nonetheless, risk analysis, and especially quantitative risk analysis (QRA),
has been widely criticised, see Table 1.
Table 1. Summary of some main criticisms concerning risk analysis
Criticism References
Narrow focus of technical analyses: lack of inclusion of public percep- Kermisch (2012), Kunreuther and
tion and consent to procedure. Slovic (1996), Wolff (2006)
Abuse of risk analyses to serve interests of business and government Shrader-Frechette (1993), O’Brien
agencies, by suppressing uncertainty and adjusting assumptions to (2000), Aven (2011)
meet predefined risk acceptance criteria.
Causal accident theories underlying technical analyses are simplistic Adams (1995), Leveson (2012),
in complex systems, due to risk compensation and feedback. Stringfellow (2010)
Little academic scrutiny about efficacy for engineering safer systems. Rae et al. (2014)
Poorly examined claims about ability to accurately measure risk. Rae et al. (2014)
Lack of scientific understanding of and standards for quality assurance Cumming (1981), Rae et al.
/ evaluation of risk analysis: “peer review” lacks evaluation criteria. (2014)
In response to these criticisms, there has been and is ongoing research to

strengthen the foundations of Risk Analysis. This amongst other concerns cre-
ating meta-theoretical frameworks for classifying different risk problems
(Klinke and Renn, 2002; Rosa, 1998), attempts at clarifying a number of con-
ceptual foundations of the field (Aven and Renn, 2009; Johansen, 2010; Rosa,
2010), proposing alternative scientific platforms for risk analysis (Aven, 2011;
Hermansson, 2005) and proposals for evaluating risk analyses (Busby and
Hughes, 2006; Rosqvist and Tuominen, 2004).
Nonetheless, these and many issues need further research (Aven, 2012a),
and calls have been made to extend the research on foundational issues to the
application areas (Aven and Zio, 2014). The need for more research on Risk
Analysis per se has also been voiced in the maritime application area, as focus
on these issues has been rather limited in the maritime field (Psaraftis, 2012).
15
Introduction
1.2 Objectives and structure of the thesis
The overall objective of this thesis is to contribute to some foundational issues

within the application area of maritime transportation risk analysis. Focus is
on principles underlying a risk analysis as a tool for informing decisions, on
developing frameworks for performing risk analyses, and on principles and
methods for evaluating risk analysis applications. Focus is on policy-oriented
waterway risk analysis (i.e. supporting societal decisions) and operational risk
analysis in a collision avoidance context (i.e. supporting decisions in an ongo-
ing operation). The specific objectives are:
Objective 1: Investigate which definitions, perspectives and scientific ap-

proaches to Risk Analysis have been adopted in maritime wa-
terway risk analysis applications, and their relations.
Objective 2: Investigate the reliability of selected waterway risk analysis
applications and risk models for ship collision alert systems.
Objective 3: Present a set of principles as a basis of subsequently developed
risk analysis frameworks. These address terminology and
adopted understanding of key concepts, risk modelling in rela-
tion to prediction, risk model use and the consideration of un-
certainty and bias. These principles intend to clarify the
adopted scientific approach to risk analysis.
Objective 4: Propose frameworks for waterway risk analysis and risk analy-
sis in a collision avoidance context, and present applications to
illustrate the rationale.
Objective 5: Contribute to the research on the evaluation of risk analyses,
addressing both the principles and practical tools.
The links between these objectives, the publications (PI to PIV) and the
structure of the thesis summary (section numbers) are shown in Figure 1.
Figure 1. Relations between research objectives, publications and structure of the thesis
Objective 1 primarily aims to identify a need to focus on the underlying prin-

ciples of risk analyses in the maritime application area. Based on this, Objec-
16
Introduction
tive 3 focuses on proposing a conceptual basis for Risk Analysis, and on the
need for systematically considering uncertainties and biases in applications.
Objective 2 supports the need to focus on underlying principles by perform-
ing two reliability analyses of waterway risk applications and risk models for
collision alert systems. The findings lead to the questions of how risk models
may be justifiably used, which addressed in Objective 3 and how risk analyses
can be evaluated, addressed in Objective 5.
In Objective 4, frameworks are proposed which translate the principles ad-
dressed in Objective 3 to practical methods and tools for performing a risk
analysis. Objective 5 is closely linked to Objective 3 and Objective 4.
In the summary, Section 2 provides a background for the research topics. A
general introduction is given to different foundational views on Risk Analysis,
and the concepts of reliability, validity and evaluation are outlined. A brief
review is given of previous research on these issues, focusing on the maritime
application area. Section 3 summarizes the main research results related to the
diffuse state of the scientific maritime risk analysis application area (PI), the
results of the research addressing reliability (PII and PIV), and formulates a
number of research questions. Section 4 establishes the principles underlying
the frameworks of PIII and PIV. Section 5 outlines the main features of the
frameworks presented in PIII and PIV, and briefly shows the developed appli-
cations. Section 6 combines the approaches to risk analysis evaluation pre-
sented in PIII and PIV. Section 7 concludes.
1.3 Limitations
Maritime Risk Analysis is a very broad research area. This thesis focuses only
on Risk Analysis in maritime transportation, and in particular on waterway
risk analysis applications and on risk analysis in a collision avoidance context.
While the results may be more widely applicable also to other problems, this is
not explicitly claimed or further addressed.
While a coherent set of principles of Risk Analysis is addressed in this thesis,
many other foundational issues require attention. Much more research is
needed on topics such as understanding the concepts underlying the disci-
pline, the implications of and ways to address uncertainty in complex systems,
the role of values in Risk Analysis, ways to represent and communicate risk
analysis results, the role of risk science in societal decision making, and many
more, see Psaraftis (2012) and Aven and Zio (2014).
It is acknowledged that other risk analysis frameworks for the addressed ap-
plication areas may be feasible. No claims are made that the presented frame-
works are the only possible ones. The ambitions are more modest to devise
frameworks with clearly articulated underlying principles, answering calls to
this effect (Aven and Zio, 2014; Psaraftis, 2012).
Given the very limited research dedicated to how to evaluate risk analyses,
the principles and tools for this should be considered as an exploratory at-
tempt to contribute to this research area.
17
2. Background
2.1 Scientific approaches to Risk Analysis
A central issue in this thesis is the differing views on the scientific basis of Risk
Analysis2. This has been addressed in the Risk Analysis literature, but no re-
search is known in the maritime application area where this is addressed. Be-
cause this is important for the objectives of Section 1.2, a brief outline is given.
There are three prototypical approaches: realist, constructivist and proce-
duralist. While variations exist (see PI), these prototypes are useful to under-
stand the philosophical schisms underpinning the discipline.
2.1.1 Realist approaches
Risk realists typically consider risk as a physically given attribute of a system,

which can be characterized by objective facts. Work is performed under the
presumption that the quantities resulting from technical analyses are a repre-
sentation or an approximation of an absolute truth. Risk management deci-
sions are considered rational to the extent they are based on the objective,
non-personal factors of technical analysis (Bradbury, 1989; Shrader-Frechette,
1991; Thompson and Dean, 1996). Thus, realist approaches to Risk Analysis
rely exclusively or primarily on data and models from engineering or natural
sciences. Expert judgment, if applied, is seen as an objective representation of
an underlying truth, totally determined by evidence. This view is succinctly
expressed by Kaplan, who understands risk as a set of scenarios, probabilities
and consequences (1997, p. 414):
This idea of “objectifying” the so called subjective probability has major implica-
tions. It resolves the historical controversies, and it shows us the how to put
Risk Analysis on a totally solid conceptual foundation. It opens the way to what
we can call “evidence-based” risk assessment and “evidence-based” decision
2
It is worth reflecting on the fact that various authors have expressed concerns about Risk
Analysis being a science at all. Two editorials in the first edition of the Risk Analysis journal
express serious reservations to its scientific nature (Cumming, 1981; Weinberg, 1981). More
recently, arguments have been made that it is scientific “when understood as consisting pri-
marily of (i) knowledge about risk-related phenomena, processes, events, etc., and (ii) con-
cepts, theories, frameworks, approaches, principles, methods, and models to understand,
assess, characterize, communicate, and manage risk, in general and for specific applications
(the instrumental part).” (Hansson and Aven, 2014, p. 1181)
19
Background
making. In regulatory and public decision making it shows us how, quantitative-

ly, to “Let the Evidence Speak!” not the opinions, personalities, moods, politics,
positions, special interests, or wishful thinking!
This reification of risk has several implications. First, understanding risk as a

kind of physical quantity, the focus in risk analyses is on the calculated num-
bers (often probabilities). From this, there typically is a strong link between
the risk quantification and decision criteria. This can be either through risk
acceptance criteria, or through mathematical procedures such as maximizing
expected utility. Kaplan (1997) and De Rocquigny et al. (2008) propose such
risk-based decision frameworks. Second, a sharp distinction is claimed to exist
between the facts of technical analysis and the non-epistemic values3 inherent
in decision making (Bradbury, 1989; Shrader-Frechette, 1991). Finally, contex-
tual dimensions such as controllability, fear, the voluntariness of exposure and
emotions with respect to the possible occurrences are seen as accidental and
not part of the risk concept per se (Thompson and Dean, 1996).
2.1.2 Constructivist approaches
Risk constructivists reject the idea that risks exist as some absolute quantity
independently of the people assessing and experiencing them. Their starting
premise is that risk primarily involves social processes. Hence, their focus
shifts from (probabilistic) quantifications to the assessor/perceiver of risk,
expert as well as layperson (Bradbury, 1989; Shrader-Frechette, 1991; Thomp-
son and Dean, 1996). Several variations of this view exist. One manifestation is
clearly expressed by Otway and Thomas (1982, p. 70):
It is clear that truths do not exist independently of people, whether taken to be

individuals, significant social groups in the general public, professional or polit-
ical/industrial groups. […] Once the criterion of an absolute truth is abandoned,
then surely no one can avoid the inference that people see the world differently
and that these differences emerge from different experiences of differently con-
structed social worlds.
Another constructivist view is expressed by Aven and Renn, defining risk as

“the uncertainty about and severity of the events and consequences/outcomes
of an activity with respect to something that humans value” (2009, pp. 8–9):
In our concept, risk does not exist independent of the assessor, as the uncertain-
ties need to be assessed by somebody. […] Risk according to our definition re-
quires a mental construction of the uncertainty (knowledge) dimension […]
‘Uncertainty’ is not real, it is a construct of a human imagination to cope with
potential future outcomes that can become real. […] Emphasizing the subjective
3
Non-epistemic values are values of a moral, political or aesthetic nature, i.e. values which
have no relevance to determining whether a claim is true but stem from a reflective consider-
ation of what is good in a given context (Wandall, 2004).
20
Background
and constructive nature of uncertainty does not imply, however, that these con-
structs are arbitrary or not subject to scientific scrutiny.
Some constructivists adhere to a broad, polythetic understanding of risk, and

include perceptual factors such as controllability, fear, the voluntariness of
exposure, emotions and procedural issues such as liability and trust in the risk
concept, apart from probabilities and events/consequences (Kermisch, 2012;
Rayner, 1992; Slovic, 1999; Wolff, 2006). Thus, these characteristics are all
part of the risk concept, but none of these are essential for describing risk.
Another feature of many constructivist approaches to Risk Analysis is the re-
jection of a complete separation between facts and values, stressing the value-
ladenness of risk analyses: it is argued that problem framing, choice of risk
metrics and the types of assumptions made are no value-neutral exercises
(Bohnenblust and Slovic, 1998; Bradbury, 1989; Hansson, 2010; Shrader-
Frechette, 1991).
The subjective nature of risk has implications for how risk is managed. First,
the risk decision process does not exclusively rely on the quantitative results of
technical analysis. Perceptual and other qualitative factors are considered as
well in the decision (Bohnenblust and Slovic, 1998; Bradbury, 1989). Second,
constructivist approaches do not typically apply strict risk acceptance criteria
or mathematical decision procedures such as maximization of expected utility:
a process of looking beyond such mathematical tools is needed (Aven and Vin-
nem, 2005; Hartford, 2009).
2.1.3 Proceduralist approaches
Some authors have argued that in democratic societies, the fairness of the dis-
tribution of risks and benefits related to a given activity are important aspects
of consistent risk management, which is often linked to the fairness of risk
analysis and decision making procedures (Fiorino, 1989; Hermansson, 2005).
The requirement of including risk-affected parties in the risk analysis pro-
cess has been argued for on ethical grounds (Shrader-Frechette, 1991), and
several mechanisms have been suggested for public participation in decision
making (Fiorino, 1990; Hermansson, 2010; Renn et al., 1993).
In procedural approaches, different stakeholder groups such as scientists,
domain experts, risk-affected lay persons and decision makers take part in a
deliberative process. Thus, risk is characterized and decision options evaluated
by explicitly accounting for facts, expert judgments and stakeholder values.
Such an approach to analysing risk is known as an analytic-deliberative pro-
cess, in which rationality is defined through adherence to democratic princi-
ples (Douglas, 2009; Shrader-Frechette, 1991; Stern and Fineberg, 1996).
21
Background
2.2 Reliability, validity and evaluation of risk analysis
2.2.1 Outline of the concepts
Reliability and validity relate to measurement methods, addressing two essen-

tially different properties. Reliability is concerned with the question in how far
a measurement leads to the same result when the measurement is repeated,
whereas validity concerns the question to what extent the measurement ap-
propriately describes the concept one intends to describe (Carmines and
Zeller, 1979; Trochim and Donnely, 2008). Figure 2 illustrates the concepts
using the analogy of a target. A valid measurement results in an outcome close
to the bulls-eye, either directly or as an average of repeated measurements. A
reliable measurement results in outcomes which are closely clustered, while
unreliable measurement involves large scatter.
Figure 2. Illustration of the concepts of reliability and validity in a measurement context
Risk analysis evaluation concerns the process of building confidence in a giv-

en risk analysis. It can be defined as “[…] an independent peer review process
consolidating the decision-maker’s confidence in the results […] of risk as-
sessment”4 (Rosqvist, 2003). The importance of such quality control proce-
dures has been raised in an editorial of the first edition of the Risk Analysis
journal, finding however that these are not well developed (Cumming, 1981).
The relation between validity and the evaluation process warrants a com-
ment. Validity only concerns the question to what extent the measurement
appropriately describes the concept one intends to describe. The evaluation of
a risk analysis goes beyond this, with a focus on methodological characteristics
of the risk analysis as a measurement procedure. Thus, validity is only one
aspect of evaluation.
2.2.2 Challenges to reliability, validity and evaluation
Reliability is challenging to assess because risk analyses have unique results in

the sense that these are usually not repeated for the same problem, such that
little empirical evidence exists concerning measurement reliability. In the
analogy of Figure 2, this means that only one measurement point is made.
4
Different terminology is used: quality control / quality assurance / qualification (Apostolakis
et al., 1983; Rosqvist and Tuominen, 2004; Suokas and Rouhiainen, 1989), verification (Gra-
ham, 1995), credibility assessment (Busby and Hughes, 2006) and validation (Rosqvist, 2010)
22
Background
Validity is a challenging issue because there are different views on the risk
concept (Aven, 2012b; Kaplan, 1997; Kermisch, 2012; Slovic, 1999), and hence
on what a risk analysis should measure. This inevitably leads to different views
on whether a given measurement is valid. In the analogy of Figure 2, this
means that there is disagreement on where the bulls-eye is located5.
Evaluating risk analyses is challenging due to diverging views on the appro-
priate scientific foundations for Risk Analysis, as outlined in Section 2.1 . Ad-
herence to a realist, constructivist or proceduralist approach has repercussions
on the methodological characteristics to which the risk analysis as a measure-
ment procedure should comply. This is further addressed in Section 6.
Another essential difficulty in determining the quality of a risk analysis is the
impossibility of a comparison with experiments or data. If, as understood in
this thesis, risk refers to future occurrences, this is a genuinely metaphysical
problem. Empirical measurements of future (non-)occurrences are impossible
to make: “the most powerful method of science – experimental observation –
is inapplicable to the estimation of overall risk” (Weinberg, 1981, p. 5).
Finally, many different tools and modelling approaches are used for measur-
ing risk. This complicates the evaluation of specific risk analyses. While some
general issues can be stated, the procedures to evaluate a specific risk analysis
are likely to differ for different specific measurement tools (Rosqvist, 2003).
2.2.3 Previous research on reliability, validity and evaluation
Despite the centrality of reliability, validity and evaluation in any measure-

ment procedure, there has been very limited research dedicated to these issues
in Risk Analysis, as found by e.g. Rae et al. (2014).
Suokas (1985) evaluates the reliability and validity of two methods for as-
sessing the risk of an industrial plant and two machinery installations. Suokas
and Kakko (1989) report on reliability exercises in the nuclear industry, show-
ing that the reported estimates for accident frequency typically vary within
several orders of magnitude. Amendola et al. (1992) report on reliability stud-
ies of chemical risk analysis, concluding that variations of multiple (up to six)
orders of magnitude occur for risk estimates of the same top event. Laheij et al.
(2003) perform a benchmark reliability study of five methods for chemical risk
analysis, finding discrepancies of about one order of magnitude.
Aven and Heide (2009) provide a theoretical discussion on the reliability and
validity of risk analysis. They propose a number of reliability and validity crite-
ria, which are applied to evaluate different approaches for quantifying risk,
namely traditional statistical analysis, the ‘probability of frequency’ approach
(Kaplan, 1997) and the ‘predictive Bayesian’ approach (Apeland et al., 2002).
5
For example, if risk is understood as a combination of uncertainties, events and conse-
quences (Aven and Renn, 2009), measurement validity involves an argumentation that the
focus is on the right events and consequences, and that the associated uncertainties are ap-
propriately assessed. If a polythetic understanding of risk is adopted (Kermisch, 2012; Slovic,
1999), measurement validity involves an argumentation that the facets considered in the
measurement (e.g. probabilities, events, emotional reactions, etc.) are the appropriate ones
for the specific problem, and that these are adequately measured.
23
Background
The different approaches have different validity criteria, which relates to the
different underlying commitments to the risk concept.
While some research on the reliability of Risk Analysis has been performed
in industrial applications as indicated above, no systematic work is known in
the maritime transportation application area.
Concerning the evaluation of risk analyses, Suokas and Rouhiainen (1989)
identify four possibilities to do this: i) carrying out a complete parallel analysis
of the same activity, ii) carrying out a parallel analysis on some parts of the
same activity, iii) comparing the analysis with descriptions of accidents in the
corresponding system and with personal experience, and iv) examining the
process behind the analysis. Rouhiainen (1992) presents a checklist approach
for evaluating risk analyses in an industrial system. Macgill et al. (2000) pro-
pose another checklist approach for the water treatment industry, which fol-
lows a two-level hierarchy. The top level contains five characteristics: observa-
tion, method, output, peer review and validity. These are further divided in
sub-characteristics, which are assessed using an ordinal scale.
Rosqvist (2003) proposes four methodological quality characteristics for
FSA: transparency, completeness, credibility and fairness. Completeness is
assessed by inspecting the model scope in relation to the scope of the study.
Credibility is assessed through three criteria: i) sensitivity analyses for dealing
with parameter uncertainty, ii) model uncertainty and direction of bias and iii)
adequacy of the recommendations in light of risk results and decision rules.
Busby and Hughes (2006) perform interviews to identify which norms a risk
analysis should conform to, concerning both the object system to which it is
applied, as well as the social system it intends to serve. A broad set of norms is
identified by regulators, consultants and researchers.
Rosqvist (2010) finds that a key element in the evaluation of a risk analysis is
the different roles of the experts, risk analysts and decision makers. The do-
main expert focuses on the availability of knowledge, the risk analyst on the
appropriateness of data, experts and methods, and the decision maker on the
relevance of the inferences in light of the adopted decision criteria.
2.3 State of the art in maritime transportation Risk Analysis
2.3.1 Research on Risk Analysis in maritime application area
Most of the work concerning principles and methods for maritime Risk Analy-
sis addresses the Formal Safety Assessment (FSA) method, adopted by the
International Maritime Organization (IMO, 2002). Several reviews and com-
mentaries have been made, often critically discussing some of its deficiencies
and sometimes proposing methods and ways of improving the process. Work
by Wang (2006, 2002, 2001), Guedes Soares and Teixeira (2001), Skjong and
Wentworth (2001), Devanney (2008), Kontovas and Psaraftis (2009), Peder-
sen (2010), Psaraftis (2012) and Yang et al. (2013) belongs to this category.
Wang et al. (2004) focus on the possibility of using advanced mathematical
tools such as approximate reasoning, artificial neural networks and optimiza-
24
Background
tion in maritime risk analyses. Rosqvist and Tuominen (2004) and Busby and
Hughes (2006) address the issue of the credibility of FSA studies. Montewka et
al. (2014b) propose a risk perspective for FSA, which highlights the knowledge
and understanding about the system behaviour.
Several authors focus on risk acceptance or cost effectiveness criteria for use
in risk cost benefit analyses in maritime applications. These either provide a
general overview of risk acceptance and cost-effectiveness principles (Skjong
et al., 2007; Trbojevic, 2006; Vanem, 2012), propose new criteria (Puisa and
Vassalos, 2012), discuss the extension to environmental risk (Psaraftis, 2008)
or perform analyses to propose numerical values for the decision criteria (Eide
et al., 2009; Kontovas et al., 2010; Psarros et al., 2011; Yamada, 2009).
Despite the significant number of papers written about FSA-related issues,
there is no research known which specifically focuses on the underlying scien-
tific methods for Risk Analysis in the maritime application area, which is the
topic of PI.
2.3.2 Waterway risk analysis applications
Many methods and applications of waterway risk analysis have been present-
ed. Review papers by Li et al. (2012) and Özbaş (2013) outline the rationale of
some of the most influential models, in particular traffic flow, traffic simula-
tion, and event tree models. Mazaheri et al. (2014) review available models
addressing the risk of ship grounding from a risk management perspective.
Various models have been proposed for waterway risk analysis, based on
traffic flow theory (Fowler and Sorgård, 2000; Mulyadi et al., 2014; Pedersen,
1995), traffic simulation (Almaz, 2012; Merrick et al., 2000; Sormunen et al.,
2014; Ulusҫu et al., 2009), ordinal logistic regression and a vessel conflict
technique (Debnath, 2009), Bayesian Networks (Hänninen et al., 2014;
Klemola et al., 2009; Montewka et al., 2014a), risk indicators (Qu et al., 2011),
fuzzy logic (Hu et al., 2007; Wang et al., 2014) and artificial force fields (Mon-
tewka et al., 2011; Xiao, 2014).
While many different models are developed, no research is known concern-
ing the underlying principles adhered to in maritime risk analysis applications.
This is the primary research objective of PI. In PIII, a framework for maritime
transportation risk analysis is developed in which the risk-theoretical basis is
elaborated upon, and an application of oil spill risk analysis in a waterway is
presented.
2.3.3 Risk analysis applications in a ship collision avoidance context
In a ship collision avoidance context, there are two main classes of risk analy-
sis applications.
A first class concerns methods for automatic collision avoidance and route
planning. Various methods have been proposed for this purpose, see Staheros
et al. (2008), Tam et al. (2009) and Campbell et al. (2012) for reviews. These
methods rely on optimization and path planning algorithms for automatizing
the collision avoidance actions of ships. Among the recent approaches, Tsou
25
Background
and Hsueh (2010) apply ant colony algorithms, Szlapczynski (2011) proposes
evolutionary sets and Xu et al. (2014) apply multi-objective optimization.
A second class concerns methods for collision alert systems. Such risk analy-
sis applications have a more restricted aim of only providing alerts to ship nav-
igators and/or personnel in Vessel Traffic Service (VTS) centres, without pro-
posing routing alternatives or automated collision avoidance actions. Hilgert
and Baldauf (1997) propose heuristic criteria to categorize collision risk, re-
fined by Baldauf et al. (2011) with fast-time simulation techniques. Kao et al.
(2007) and Wang (2010) propose fuzzy ship domains. Other approaches in-
clude fuzzy expert systems (Bukhari et al., 2013; Lee and Rhee, 2001; Ren et
al., 2011), domain theory and fast-time simulation (Zhang et al., 2012), Demp-
ster-Shafer evidence theory (Li and Pang, 2013) and neural networks (Simsir
et al., 2014). It is this class of applications which are addressed in this thesis.
While many different collision alert models have been presented, there is
lack of research focusing on the risk-theoretical basis of such applications.
Furthermore, no theoretical frameworks have been developed for operational-
izing of the construct “ship-ship collision risk”. PIV addresses these issues
while presenting a model for a risk-informed collision alert system.
2.3.4 Research on risk analysis evaluation in maritime application area
In the maritime application area, research on evaluative peer review is limited,

and often framed in an FSA-context.
NRC (1998) presents an evaluation methodology for a waterway risk analysis
study, focusing on the adequacy of data and methodologies, the transparency
and limitations of the analysis and the consistency between the analysis and
the provided recommendations. Rosqvist and Tuominen (2004) propose four
methodological quality characteristics: transparency, completeness, credibility
and fairness. In the risk analysis phase, the completeness is assessed by in-
specting the model scope in relation to the scope of the study. Credibility is
assessed through three criteria. First, sensitivity analyses dealing with parame-
ter uncertainty are reviewed. Second, the model uncertainty and direction of
bias is considered. Third, the adequacy of the recommendations in light of the
risk results and the decision rules is assessed. Busby and Hughes (2006) per-
form interviews to identify which norms a risk analysis should conform to. A
broad set of norms is identified by regulators, consultants and researchers,
and a framework is outlined on how these could be integrated. Psaraftis (2012)
proposes review criteria of FSA studies, including adherence to procedure and
expertise of the FSA team, adequacy of data, assumptions and scope, transpar-
ency, the assessment of sensitivity and uncertainty.
Given the importance of establishing quality control mechanisms in scientific
research and the limited work focusing on risk analysis evaluation, this is more
elaborately addressed in PIII and PIV. Based on these, an integrated frame-
work for model-based risk analysis is presented in Section 6.
26
3. Analysing Risk Analysis: the need for
clarifying principles
This chapter addresses Objective 1 and Objective 2 of Section 1.2. Section 3.1
focuses on some foundations of waterway risk analysis applications. In Section
3.2, the reliability of risk analyses is investigated, based on case studies from
waterway risk analysis and risk models in a collision avoidance context. Sub-
sequently, research questions are defined in Section 3.3.
The main novelties in this chapter are:
x the analysis of waterway risk analysis applications in light of founda-
tional issues provides new insights in the commitments to Risk Anal-
ysis made in this research community [PI];
x the classification of scientific approaches to Risk Analysis (Figure 3)
is a new way to communicate the principles one adheres to [PI];
x the systematic research on reliability is unprecedented in the mari-
time transportation application area [PII, PIV].
3.1 An analysis of foundational issues through applications
In PI, the state of the art in waterway risk analysis applications is analysed in
light of a number of foundational issues. The main purpose of this research is
to identify how this research community has approached Risk Analysis and to
support calls for intensified focus on foundations of Risk Analysis. The analy-
sis focuses on three issues: definitions, perspectives and scientific approaches.
Risk definitions provide insight in how the authors understand the risk con-
cept. Risk perspectives encompass which elements are considered in the risk
description, i.e. into which measurement tools are applied, whether only
events or events and consequences are considered, and whether the risk de-
scription contains elements to convey the confidence in the analysis. The sci-
entific approach concerns the adherence to realist, constructivist or procedur-
alist foundations, providing insight in whether the focus is on a true risk, the
types of considered evidence, the extent of uncertainty treatment, the in-
volvement of stakeholders and the inclusion of contextual attributes.
Risk definitions are classified using a categorisation proposed by Aven
(2012b), see Table 2. The scientific approaches are classified in eight classes
shown in Figure 3, refining the prototypical realist, constructivist and proce-
duralist approaches. These are more elaborately addressed in PI.
27
Analysing Risk Analysis: the need for clarifying principles
Table 2. A classification of risk definitions (Aven, 2012b)
Risk definition classes Abbreviation

D1 Risk = Expected value R = EV
D2 Risk = Probability of an (undesirable) event R=P
D3 Risk = Objective uncertainty R = OU
D4 Risk = Uncertainty R=U
D5 Risk = Potential/possibility of a loss R = PO
D6 Risk = Probability and scenarios / (severity of) consequences R = P&C
D7 Risk = Event or consequence R=C
D8 Risk = Consequences/damage/severity + uncertainty R = C&U
D9 Risk = Effect of uncertainty on objectives R = ISO
The main conclusions of PI are as follows.

First, it is found that many different risk definitions are used in the applica-
tion area. Risk is often not explicitly defined, and when provided, it is usually
adopted as if no alternatives exist. Probability-based definitions, especially D1
(R=EV) and D6 (R=P&C) are dominant. Definitions based on possibility D5
(R=PO) and uncertainty D8 (R=C&U) are found as well, albeit minimally. This
confirms research that the Risk Analysis discipline faces challenges related to
the applied terminology (Aven and Zio, 2014; Kaplan, 1997).
Figure 3. Spectrum of scientific approaches to Risk Analysis
Second, as clear from Figure 4, definitions do not necessarily provide insight

into the scientific approach to Risk Analysis. This justifies earlier claims that
underlying risk definitions, deeper philosophical disputes exist regarding the
nature of the risk concept and the appropriate principles underlying analyses:
The claim that there are competing conceptions of risk implies that […] compet-
ing interpretations reflect philosophical differences that are long-standing and
systematically linked. Such disputes will not be settled merely by stipulating
28
definitions for the disputed terms. Generally speaking, stipulative definitions for
risk will be useful only when the discourse community […] already possesses a
shared conception of risk (Thompson and Dean, 1996, p. 363).
Third, it is found that there is a significant relation between the applied defi-
nition and the adopted risk perspective, see Table 3. For definitions based on
probability (D1, D2 and D6), risk descriptions typically apply probabilities to
describe risk. In such cases, uncertainties beyond the probabilities are typically
not assessed. In contrast, the application where risk is defined through uncer-
tainty (D8) also applies probabilities to measure risk, but a systematic uncer-
tainty evaluation accompanies the risk description. Furthermore, in applica-
tions where risk is defined through possibility (D5) or as an event (D7), more
risk descriptions use indicators and fuzzy numbers. This means that a com-
mitment to a certain definition to a large degree determines how one describes
it, as asserted e.g. by Slovic (1999).
Figure 4. Relation between risk definitions (Table 2) and scientific approaches to Risk Analysis
(Figure 3) in maritime waterway risk analysis applications analysed in PI
Table 3. Relation between risk definition and risk perspective, abridged from PI
Risk definition ID P f* Ps Px IQU IQL F A C C* UQU UQL UAH B

D1: R=EV M6 x x x x [x]
M11 x x x
M18 x x x
M25 x x x x
M48 x x x
D2: R=P M34 x x x
M54 x x x x x
D5: R=PO M17 x x x
M43 x x x x
D6: R=P&C M9 x x x x
M12 x x x x
M26 x x x x
M37 x x x x
M51 x x x x x x
D7: R=C M17 x x x
M43 x x x x
D8: R=C&U M56 x x x x x x
ID = identification number, Pf* = frequentist probability, Ps = subjective probability, Px = modelled probability,
IQU = quantitative indicator, IQL = qualitative indicator, F = fuzzy number, A = event, C = consequence, C * =
consequence [should be, but is not, included], UQU = quantitative measure of evidential uncertainty, UQL =
qualitative measure of evidential uncertainty / strength of knowledge, UAH = alternative hypothesis-based
epistemic uncertainty, B = bias, x = found in risk description, [x] = mentioned, not systematically analysed
29
Fourth, it is found that within the waterway risk analysis application area , a
wide spectrum of scientific approaches to Risk Analysis co-exists. Most appli-
cations adhere to a form of the realist prototype (I-III). Constructivist ap-
proaches (V-VII) are found as well, but less frequently. The proceduralist ap-
proach (IV) constitutes a minority in the application area.
Finally, the lack of systematic uncertainty treatment in most applications is
apparent. Of the 58 investigated applications, only six considered uncertainty:
three provided a quantitative uncertainty analysis (addressing model parame-
ters or structure), while in three cases, the risk model is accompanied with a
qualitative description of uncertainties or biases. This lack of uncertainty
treatment contrasts the findings in the example applications of PI from p. 128
onwards, where several important uncertainties can be identified.
The above findings support calls for intensified research and discussion on
the appropriate underlying principles of maritime risk analysis applications.
3.2 Analysing reliability: two case studies
In PII, the reliability of selected waterway risk analysis applications is investi-

gated. In PIV, the inter-methodological reliability of four risk models for a ship
collision alert system is assessed. The main purpose of this research is to pro-
vide some evidence to the claims in the theoretical analysis by Aven and Heide
(2009) that risk analyses are generally not reliable tools for informing deci-
sions. These authors propose following reliability criteria:
R1. Degree to which a risk analysis method produces the same results at
reruns of the method.
R2. Degree to which a risk analysis produces identical results when con-
ducted by different analysis teams, using the same methods and data.
R3. Degree to which risk analyses produce identical results when conduct-
ed by different analysis teams with the same scope and objective, but
no restrictions on method and data.
In PII, three waterway risk analyses methods are applied to a case study in
the Gulf of Finland (GoF). The analysis focuses on the question how likely
ship-ship collisions are in different locations in the sea area. The methods de-
termine the likeliness of collision using probabilities or using quantitative in-
dicators, see Table 4. In methods M2 and M3, which apply probabilities to
measure risk, the accuracy of the calculated numbers can be assessed. Howev-
er, comparisons between the calculated probabilities (M2 and M3) and the
quantitative indicators (M1) can only be performed by inspecting the rank or-
der of the risk measures in the waterway areas.
Table 4. Summary of perspectives and scientific approaches, methods applied in PII
ID Reference Risk perspective Scientific approach to Risk Analysis

M1 Qu et al., 2011 R ~ (IQU → A) II – moderate realist
M2 Weng et al., 2012 R ~ (Pf*, A, C*) I – strong realist
M3 Goerlandt and Kujala, 2011 R ~ (Pf*, A, C*) I – strong realist
Abbreviations of risk perspectives, see Table 3. Rationale of scientific approaches, see Figure 3.
30
In Figure 5, the results of the R3-reliability criterion for these methods are
shown, by means of pairwise comparisons of the risk measures as determined
by the three methods in 16 defined areas of the investigated TSS area6.
The diagonal provides information about the considered test cases R3.i to
R3.vi. Below the diagonal, each matrix element shows a comparison between
the calculated risk measures for the corresponding test case. Above the diago-
nal, three correlation coefficients for the risk measures are shown: Spearman
rank ρ, Kendall’s τ and (if meaningful) Pearson’s r. These provide a mathemat-
ical appreciation of the strength of the relation between the metrics7.
Figure 5 clearly shows that the inter-methodological reliability of the risk
analyses is low. For instance, comparing R3.iii (M2) with R3.v (M3) results in
ρ = 0.47 and τ = 0.32. This means that the rank order of the risk measures in
the different waterway areas is poorly retained across the methods.
Figure 5. Inter-methodological (R3)-reliability of selected models for waterway risk analysis, test
cases (shown on the diagonal, together with the axis labels) from PII
Table 5 summarizes the results for the methods of Table 4. For the determin-
istic methods M1 and M3, R1-reliability is fulfilled. For the probabilistic meth-
od M2, R1-reliability is high. R2-reliability of the methods depends on the pa-
rameter choices within each method: reliability in terms of accuracy and rank
order vary from low to high. R3-reliability is low.
Table 5. Summary results of the reliability scores for risk analyses investigated in PII
Rank order
Method Reliability criterion Accuracy of metric
retention
M1 R1 N/A Y
R2 N/A L-H
R3 N/A L
M2 R1 H H
R2 M-H M-H
R3 L L
M3 R1 Y Y
R2 L-M L-H
R3 L L
R1, R2, R3 = criteria by Aven and Heide (2009), N/A = not applicable, Y | L | M | H = yes, low, medium, high
6
TSS: an area where ship traffic is regulated, requiring vessels to follow certain sea lanes.
7
In PII, particularly on p. 356, it is elaborated how to read the figure.
31
In PIV, five risk models for collision alert systems are compared as part of an
evaluation of the proposed risk model. Figure 6 shows the time histories of the
four test scenarios for these methods, shown as videos in PIV. The sequences
show the low inter-methodological (R3) reliability for the considered methods,
for all considered scenarios.
Figure 6. R3-reliability of selected models for risk-informed collision alert systems, test scenari-
os from PIV. ARPA = industry standard, e.g. Baldauf et al. (2011). RICAS = PIV. M1 = Mou
et al. (2010). M2 = Wang (2010). M3 = Bukhari et al. (2013)
The analyses performed in PII and PIV confirm the theoretical discussions
by Aven and Heide (2009), providing some empirical evidence about the relia-
bility of risk analysis in the maritime transportation area. Compared with Aven
and Heide (2009), who focus on probability-based risk perspectives, it is
shown that also perspectives applying indicators (IQU or IQL) or fuzzy numbers
(F) do not lead to a reliable risk characterization.
3.3 Research questions
Based on the findings of Section 3.1 and 3.2, the following research questions
are formulated, which are addressed in the following Sections:
1. Given the conceptual/terminological disarray in the application area,
what is a coherent terminological basis for maritime transportation
risk analysis?
2. If risk models and analyses are unreliable, how can these be used?
3. How do the above principles translate to practical frameworks?
4. How can we establish credibility of a risk model/analysis, i.e. how can
risk analyses be evaluated?
32
4. Risk and model-based risk analysis:
principles
This chapter focuses on Objective 3 as identified in Section 1.2, with corre-

sponding research questions 1. and 2. of Section 3.3. The main novelties are:
x the definitions of risk and related concepts constitute a novel, coher-
ent set of principles to approach Risk Analysis [PIII, PIV]8;
x the discussion on the relation between risk modelling and prediction,
from which the proposed model uses are established [PIII, PIV]9.
4.1 Terminological-conceptual basis
When terminologies are disputed, definitions are important. In this section,

the adopted definitions and understanding of the concepts in the frameworks
for risk analysis, presented in PIII and PIV, are introduced. The underlying
rationale and some implications are outlined as well, which is instructive be-
cause there are different views on definition as an intellectual activity10 and
because different types of definitions have a different aim11.
4.1.1 On the logic of definitions
A definition is the result of an intellectual activity for setting out the meaning
of symbols or words (Moore, 2009). The definiendum is the symbol being de-
fined, whereas the definiens is the (group of) symbol(s) explaining the mean-
8
It is acknowledged that the various definitions exist in some form through work of previous
authors, as referenced. The novelty is modest in the sense of integrating various ideas into a
coherent set (e.g. the relation between risk and situations, the relation between risk and
different types of uncertainty and the distinction between uncertainty, acceptance and bias).
The aim is to propose a coherent web of related terms and to clarify how these can be under-
stood. While briefly presented in PIII and PIV, the discussion in Section 4.1 and 4.2 is more
elaborate than in these papers. The discussion in Section 4.1.3 is not given in the papers.
9
The issue of the non-predictive nature of the risk models is briefly presented in PIII and PIV,
but is elaborated upon in Section 4.2.
10
There are essentialist, linguistic and nominalist views on definition. Generally, essentialists
maintain that definitions (should) provide a causal explanation of the thing defined; linguists
view definitions as historical reports of word usage; and nominalists see definitions as syntac-
tic or semantic rules for assigning names to things. (Moore, 2009)
11
Moore (2009) mentions three aims: lexical (historically reporting of the actual usage of a
term), stipulative (deliberately assigning a meaning to an old term in a new way) and precising
(stipulating additional features to an established usage of the word, for adding precision).
33
Risk and model-based risk analysis: principles
ing of the definiendum. In the cases of Table 2, the method of definition is ana-
lytical12 and intensional13: the phenomenon referred to by the term is broken
down into its constituent elements, which are the attributes or characteristics
shared by the class to which the term is applied.
Swartz (1997a) outlines the two main tenets of the Classical Theory of Defini-
tion: i) a 'proper' intensional definition states in the definiens the logically
necessary and sufficient conditions for applying the definiendum, and ii) there
are intensional definitions for each term we use. These tenets are controver-
sial, and it has been argued that there are many phenomena for which there
are no necessary conditions common to all members of the class14. Rather than
an absolute set of necessary conditions, there is “a complicated network of
similarities overlapping and criss-crossing: sometimes overall similarities,
sometimes similarities in detail” (Wittgenstein, 1953, pp. §66, p.27e). The im-
plication of this argument is that there are terms for which it is impossible to
construct a precise intensional definition (Moore, 2009), which is compatible
with Hansson’s (2000) argument that it is a myth to seek a single, all-
compassing risk definition and with the view that, sometimes, “a concept with
blurred edges” is exactly what we need (Wittgenstein, 1953, pp. §71, p.29e).
Considering the above, risk is in this thesis understood as a polythetic con-
cept. Thus, while there are general features of the concept which allow one to
parse and interpret when the risk concept applies, there is no unique set of
necessary conditions. Hence, which features are in focus depends on the appli-
cation context and is a choice requiring justification.
Definitions for and the adopted understanding of key concepts are provided
in the next section. While these may contribute to clarifying terminology in
Risk Analysis (one of the main research issues identified by Aven and Zio
(2014)), these should not be construed to resolve the ongoing debates over the
essence of these concepts. We concur with the view of Moore (2009):
[…] we will limit our ambitions to fashioning precising definitions […] while
drawing upon the many and varied meanings that previous thinkers have as-
cribed to these terms, we will attempt to be precise or particularize [original
emphasis] their meanings within the bounded context of the conceptual frame-
work of which they will be integral parts. What is important here is not that we
succeed in contriving definitions upon which everyone in the scholary commu-
nity can agree (an impossible task). Rather, what is essential is that we be clear
in our own minds [original emphasis] – and make clear to those to whom we
will communicate the fruits of our labours – what we mean […].
12
In analytical definition, the defeniens sets out the individually necessary and jointly suffi-
cient conditions for the correct application of the definiendum. Other methods exist, e.g.
denotative (citing examples of the object class) and synonymous (providing another word
with the same general sense as the definiendum) (Swartz, 1997).
13
Versus extensional, which lists examples of objects belonging to the class (Swartz, 1997)
14
Wittgenstein (1953) uses the “game” concept as illustration, referring to card games, board
games, ball-games, Olympic games, etc. Finding no necessary conditions common to all these
activities, he speaks of “family resemblances”. An activity having sufficient of these similari-
ties can justifiably be taken as a “family member”. See also Slovic (1999) and Kermisch (2012).
34
4.1.2 Definition of risk, related concepts and measurement tools
Definitions and a brief outline of the adopted understanding of the key con-
cepts underlying the frameworks of PIII and PIV are summarized in Table 6.
Selected characteristics of the concepts as understood in this thesis are listed,
allowing further insight in how these are understood. Definitions of key meas-
urement tools applied in the frameworks are summarized in Table 7.
Table 6. Concepts, definitions and brief outline of adopted understanding 15
Definiendum Definiens
Further outline of the adopted understanding
Reference(s)
Acceptance A cognitive attitude in which an assessor presupposes a premise for specific reasons
in the deliberation
x Under voluntary control of assessor, action-oriented rather than truth-oriented
x Varies with context, can be based on non-epistemic values
Elliott and Willmes (2014)
Belief A cognitive attitude in which an assessor considers a premise true
x Is not under voluntary control of assessor, truth-oriented
x Is independent of context, not based on non-epistemic values
Elliott and Willmes (2014)
Bias The qualitative difference between what one believes to be the truth and an imper-
fect representation of this truth which is accepted in a given context
x Has an epistemological-normative connotation: relates to knowledge and values
x Directional, in the sense that conservative (overestimating) and optimistic (un-
derestimating) biases exist
Rosqvist and Tuominen (2004) and Elliott and Willmes (2014)
Consequence A specific type of event which is causally connected to another event, i.e. under
conditions of constant conjunction, temporal succession and spatial propinquity
Solberg and Njå (2012)
Event A specific (defined) state of the world and how that state changes or develops over a
time interval
Likeliness A qualitative, argumentative appreciation of how much more one believes that an
event will occur, compared to other events or to its opposite non-occurrence
van der Helm (2006)
Possibility16 Something we can think of, which we suppose to be grounded in factual reality
(factually epis- x Has philosophical-ontological connotation: relates to what can or could exist
temological) x Is of binary nature: either something is possible or not, with nothing in between
x Can only be described in hypothetical terms, either because the premises are
not yet corroborated or because the premises are not yet known
van der Helm (2006)
Risk A concept which refers to the possible but uncertain occurrence of a situation where
something of human value is at stake
x A “bridge” between possible futures and a present time
x Used for exploring future possibilities and for coping with identified possibilities
x Possibility is understood in factually epistemological, uncertainty in ontic sense
x Refers to future situations which we believe could become a reality
x Depending on the context, the focus of the possible situation can be an event, or
both events and consequences
Hansson (1999) and Solberg and Njå (2012)
Situation A contextual whole consisting of a set of circumstances
x Has a complex structure, including focus, foreground, background and horizon.
x It includes objects, events, agents, their relations, the background on which all
these appear, and a qualitative experienceable unity.
Brown (2012)
Uncertainty The inherent variation associated with a considered physical system or environment
(aleatory) Levin (2005)
Uncertainty The lack of knowledge about evidential elements e1, … en which stand in an eviden-
(epistemic, tial relation to an assessor’s statement about the possible occurrence of an outcome
evidence) Levin (2005)
15
Behind all these concepts lay complex philosophical questions. Focus is on providing clarity
through distinction rather than through definition, a view shared by e.g. van der Helm (2006).
16
This is one of the four types of possibility identified by Bloch (1995). The other three are the
‘formally possible’ (all we can think of, even if nonsensical or contradictory), the ‘possibile
according to the object’ (an characteristic of an object, which entails that it can have certain
implications) and the ‘objectively real possible’ (the determinism present in the object-world).
35
Uncertainty A metaphysical limit to human knowledge, rooted in the relation between time and
(ontic) states of affairs
x A characteristic of the world and how it works, cannot be fully eradicated by in-
creasing knowledge
x The result of the inaccessibility of the future for human considerations
Uncertainty A cognitive attitude in which an assessor simultaneously entertains different rival
(epistemic, beliefs concerning an outcome which is indeterminate at the time of consideration
outcome) Levin (2005)
Table 7. Measurement tools and their definitions
Definiendum Definiens
Reference(s)
Alternative hypothesis- An expression of epistemic uncertainty, particularly related to model struc-
based epistemic uncer- ture, by weighing multiple plausible hypotheses related to a phenomenon
tainty assessment Zio and Apostolakis (1996) and Aven and Zio (2011)
Fuzzy number The degree to which a specific instance belongs to a certain category, i.e.
the degree of similarity between the instance and the category
Mendel (1995) and Bilgiҫ and Türkşen (1998)
Indicator A measure of a system characteristic, used as a proxy for inferring the likeli-
ness of occurrence of events and/or consequences
Davies et al. (2006) and Beasley et al. (2010)
Probability The fraction of time a specified outcome occurs in an in principle infinite
(Frequentist) number of repeated tests
Watson (1994) and Aven and Reniers (2013)
Probability The degree of belief of an assessor based on evidence available to him/her
(Subjective) Watson (1994) and Aven and Reniers (2013)
Qualitative measure of A linguistic or numerical measure on an ordinal or categorical scale indicat-
evidential uncertainty or ing the degree of lack of knowledge or the strength of knowledge for making
strength of knowledge a measurement or a statement
Flage and Aven (2009), Kloprogge et al. (2011) and (Aven, 2013)
4.1.3 Application of risk-conceptual basis in a risk analysis
The adopted use of the concepts is illustrated in Figure 7 for the framework of
PIII and in Figure 8 for the framework of PIV. A clear distinction is made be-
tween the concepts and their measurement, a fundamental requirement of
measurement theory (Trochim and Donnely, 2008). In the risk identification
stage, the risk concept is used to explore the space of possible occurrences
which are relevant to the considered decision problem. In the risk measure-
ment stage, a statement is made about how likely (qualitative measurement) or
how probable (quantitative measurement) these occurrences are. An evidence
assessment moderates the measurement.
In Figure 7 (top), the risk concept is used to identify relevant events A and
consequences C in a possibly occurring situation. The main situational quali-
ties SQ, i.e. the features relevant for conceptualizing the events A and conse-
quence C, are identified. These constitute the foreground of the situation, but
prior events, further consequences and additional situational qualities
(marked in gray) are conceivable as a background of the situation. The focus of
the situation is the consequence C.
The situation is delineated using the available evidence (broadly construed),
which is conditional to the decision context in two ways. First, the available
resources (time, money, expertise,…) limit the strength of the evidence base,
which indirectly affects the identified events. Second, the decision context and
stakeholder values guide the risk analysis to focus on those parts of the situa-
tion which are valued in the decision making process. Thus, the risk analysis
and decision context are not independent (Vareman and Persson, 2010).
36
In Figure 7 (bottom), the situational qualities, events and consequences are

measured. This is further explained in PIII, from p. 44 onwards.
In Figure 8 (top), the risk concept is used to identify relevant events A in a
possibly occurring situation. A is the focus of the situation, but prior events,
further consequences and situational qualities are conceivable as a situational
background. The assessor uses knowledge about the system states available to
him to interpret a current situation based on its situational qualities. From
this, an interpretation is made about the possible event occurrence.
In Figure 8 (bottom), the situational qualities are measured and combined to
provide an indication of the possible event occurrence. This is further ex-
plained in Section 5.2 and PIV, from p. 5 onwards.
Figure 7. Use of concepts of Table 6 and Table 7 in risk identification and risk measurement
stage, framework of PIII (policy-oriented risk analysis)
37
Figure 8. Use of concepts of Table 6 and Table 7 in risk identification and risk measurement
stage, framework of PIV (operational risk analysis)
4.2 Risk Analysis: modelling, prediction and uncertainties/biases
PII and PIV provide evidence regarding the unreliability of risk models and
analyses, confirming limited available research from other industries. Because
reliability is a prerequisite for accuracy (Crawford, 2001), it seems implausible
that accurate risk estimation is possible17. This leads to the question how risk
models can justifiably be used. One central issue in this context is the relation
of the model with prediction.
In Risk Analysis, it is a quite common conception that risk analyses make
predictions, e.g. Apeland et al. (2002) and Rae et al. (2014)18. Solberg and Njå
(2012) make more reservations, finding that the laws of nature can be used to
make predictions, but that due to ontic uncertainty (defined in Table 6), these
are not certain. Apostolakis (2014) asserts that QRA models do not predict the
future, but presents no justification for this view.
In PIII and PIV, the starting premise is taken that risk models in socio-
technical systems are non-predictive tools. This is considered next.
17
This may be contrasted with the finding in PI that many waterway risk analyses focus on an
accurate quantification of an underlying true risk and present the results as such.
18
Various authors differ also on what is predicted: some focus on predicting observable quan-
tities or events (Apeland et al., 2002, p. 94), others on predicting risk (Rae et al., 2014, p. 4).
38
4.2.1 Prediction: definition and criteria
Hodges and Dewar (1992) define a predictive use of a model as follows: i) a

statement about an observable or potentially observable quantity or event is
produced, ii) the modelled situation is such that predictive accuracy can be
measured, and iii) the predictive accuracy of the model in the situation has
been measured. The last two requirements significantly restrict the common
usage of the word “prediction”, which only accounts for the first requirement.
The restricted definition is adopted, for two reasons. First, it allows a clear
distinction between different types of model uses: predictive model uses re-
quire a warrant of predictive accuracy, whereas non-predictive model uses do
not. Second, the clarification regarding model use has implication for how val-
idation is understood. For predictive models, validation consists of tests to
measure the accuracy, justifying that the model in itself provides warranted
conclusions. For non-predictive model use, validation (or better: evaluation)
requires different standards for quality assurance (Hodges and Dewar, 1992)19.
Hodges and Dewar (1992) list four criteria for prediction to be possible. The
situation being modelled must: i) be possible to observe and measure, ii) ex-
hibit constancy of structure in time, iii) exhibit constancy across variations in
conditions not specified in the model, and iv) permit the collection of ample
data to make tests concerning the model accuracy20.
Condition iv) is violated for risk models in complex socio-technical systems
such as the maritime transportation system: if risk refers to future occurrenc-
es, data collection is impossible. Moreover, models for socio-technical systems
generally fail to meet the constancy-criteria ii) and iii) (Oreskes, 1998; Scher
and Koomey, 2011)21. Hence, they cannot be used for prediction.
This instability of open systems (criteria ii) and iii)) is an underlying reason
why accident modelling using linear causal relations is controversial. Several
rival theories exist regarding the mechanisms by which accidents occur in
complex socio-technical systems (Heinrich, 1931; Hollnagel, 1998; Leveson,
2004; Perrow, 1984; Rasmussen, 1997; Reason, 1990), see Qureshi (2007) for
a summary. Several of these argue against linear cause-effect modelling, which
is understandable because certain causal relations may not be stable in open
systems. Where causal relations can justifiably assumed to be stable, models
based on linear causation can be used to obtain insight in the system.
Linear causality models are especially controversial when models are used
for accident prevention related purposes (Hänninen, 2014). One argument for
this is that certain risk mitigating interventions may results in feedback within
19
The common usage of the term “prediction” provides no way to distinguish use cases or
different quality standards for different uses.
20
Criteria i) and iv) are self-explanatory. Criterion ii) ensures that the model is predictive for
the same conditions as those in the validation tests. Criterion iii) ensures that the model re-
mains predictive for conditions differing from those in the validation tests.
21
Taking the application of PIII, criterion ii) and iii) may be violated because different ships
begin trade in a given sea area, because new regulations require new safety devices or proce-
dures, etc. In the application of PIV, criterion ii) and iii) may be violated due to unexpected
actions of one of the vessels or because other COLREG regulations induce a different situa-
tional interpretation, e.g. because a vessel is “engaged in fishing”.
39
the system: people can adapt to the changes, possibly nullifying or reducing
the intended effect (Adams, 1995). Another argument is the phenomenon
known as risk migration: the introduction of a risk mitigating measure to ad-
dress one problem in the system may introduce other, unexpected conse-
quences in another part of the system (Grabowski et al., 2000).
4.2.2 Uses of non-predictive models
The above elaboration is made to clarify the intended use of models in the risk
analysis frameworks of PIII and PIV. Risk models, if not predictive, are pri-
marily useful heuristically. Non-predictive models are representations for
providing insight and guiding further inquiry, but are not susceptible to proof
(Oreskes et al., 1994). This is in agreement with the adopted understanding of
the risk concept outlined in Section 4.1.2: it is used for exploring possible oc-
currences, not to uncover an underlying truth.
Hodges (1991) identifies a number of uses for non-predictive models, taken
as a basis for the risk analysis frameworks of PIII and PIV.
In PIII, the model is intended for informing a policy decision. The model
conveys an argumentation based on available evidence, provides a basis for
communication between stakeholders, and serves as an aid to thinking.
In PIV, the intended model use is in an operational context. The model pro-
vides alerts to distinguish the urgency of collision avoidance actions for differ-
ent vessels in the area, aimed at enhancing situation awareness and for sup-
porting operators’ judgments. It can provide a basis for communication be-
tween ship officers and VTS operators, serving as an aid to thinking.
In both frameworks, it is essential that the models do not lead to a decision
in and by themselves: different mechanisms are applied for looking beyond the
model. In PIII, this is an extensive evidence and assumption assessment
scheme, leading to an informed judgment on risk. In PIV, information from
other sources such as VHF radio, radar and ECDIS is used to judge the risk
level and plan appropriate actions22.
4.2.3 Accounting for uncertainty and bias
In PI, it is found that only a small minority of waterway risk analysis applica-
tions explicitly account for uncertainty. In the framework of PIII, uncertainty
and bias are given important roles through an evidence assessment, alterna-
tive hypothesis and an assumption effect assessment. In PIV, evidential uncer-
tainties and biases are assessed through a qualitative evidence assessment.
Additional uncertainties, requiring further research before applying the model
in operational practice, are identified as well. The need for broadly considering
uncertainty and bias is discussed in PI, p. 127.
The main argumentation for considering uncertainty is provided by Douglas
(2009) that scientists have a responsibility to consider the consequences of
22
VHF: Very High Frequency, ECDIS: Electronic Chart Display and Information System
40
error. If evidence is poor and if this may lead to foreseeable changes to the
conclusions of an inquiry, these uncertainties need to be made explicit23.
Biases differ from uncertainty as they may carry a normative content, see
Table 6. It has been argued that in applied and policy-oriented sciences, non-
epistemic values can have a direct role in inferential choices.
Two arguments support this. The first is the “gap argument”: if available evi-
dence is insufficient for supporting a claim, moral or social values can be used
to fill the evidential gap (Brown, 2014; Wandall, 2004)24, by e.g. preferring
conservative choices. The second goes significantly beyond this by arguing that
certain models ought to be based on non-epistemic values. The reasoning be-
hind this is rooted in the intended aim and use of the model (Diekmann and
Peterson, 2013)25. This is in line with the pragmatist functionalist view on in-
quiry, where facts and values have different functional roles, are jointly neces-
sary and rationally revisable (Brown, 2014)26.
The main issue is that if values which are believed to be controversial are re-
lied on, these should be made explicit (Hermansson, 2012; Wandall, 2004).
4.2.4 Model uses in relation to uncertainties and biases
While in both frameworks the risk models have similar uses (as a platform
for thinking and an aid to communication), the relation between the risk mod-
els and the uncertainties/biases in the risk analysis context differ on an im-
portant point.
In the policy-oriented risk model use (PIII), the entire model is subject to
scrutiny for a subsequent judgment. In the operational risk model use (PIV),
the model is used as a kind of ‘black box’ during operations, with only the
model output informing an operator’s judgment. This is due to the different
time scales for decision making in policy-oriented versus operational contexts.
In the former, the evidence assessment is an integral part of the risk meas-
urement (for both risk analysis stages). In the latter, the consideration of un-
certainties is a separate process, performed because of the argumentation pre-
sented in Section 4.2.3. This is also illustrated in Figure 7 and Figure 8.
Consequently, the uncertainties beyond the model are in focus during the
model use in the policy-oriented framework, whereas in the operational
23
This ”error argument” is one of the primary reasons why science is not (as often thought),
value-free: non-epistemic values are needed to consider the consequences of error and to
identify which uncertainties are relevant to assess (Douglas, 2009; Rudner, 1953). This is also
one reason why a strict separation between risk analysis and risk management is untenable
(Vareman and Persson, 2010)
24
In the traditional ”gap argument”, non-epistemic values have a secondary role in the space
fixed by the evidence: only when evidence is insufficient, values have a role (Brown, 2014).
25
This is argued as follows: : i) models ought to be developed with one or several goals in
mind, ii) sometimes one of these goals ought to be a non-epistemic goal, iii) the extent to
which a non-epistemic goal is accurately reflected in a model depends on the influence of
non-epistemic values. Hence, some models ought to be influenced by non-epistemic values.
26
There is ongoing debate on the appropriate roles for values in science and Risk Analysis, see
e.g. Douglas (2009), Brown (2014) and Aven and Zio (2014). The view adopted in this thesis is
in line with the analyses provided in Diekmann and Peterson (2013) and Brown (2014).
41
framework, uncertainties need to be addressed and as much as possible elimi-

nated during model development.
In both cases, the biases need to be in line with stakeholder values. In the
policy-oriented risk model, conservative choices are preferred because a pre-
cautionary decision maker is assumed. In the operational risk model, different
values need to be considered. Conservative model choices are preferred as
raising the alarm earlier rather than later is considered beneficial from an ac-
cident prevention viewpoint. However, a balance needs to be sought such that
the number of alerts raised in practice is kept sufficiently low. This last point is
also a user value: it is good not to be “disturbed” by alarms too frequently.
4.3 Implications of the adopted principles to Risk Analysis
In light of the scientific approaches to Risk Analysis outlined in Section 2.1 and
3.1 (Figure 3), the frameworks of PIII and PIV can be understood as primarily
“precautionary constructivist with uncertainty evaluation”.
Understanding risk as somebody’s interpretation concerning possible but
uncertain future situations, it is constructivist: risk does not exist inde-
pendently from the people assessing and experiencing it. A wide evidence base
is allowed: data, models, judgments, assumptions and non-epistemic values.
The precautionary nature of the approach is clear in PIII by making the val-
ue-laden biases underlying the risk model construction explicit. In PIV, the
value-ladenness is inherent in the model due to certain conservative parameter
choices. Uncertainties are broadly assessed beyond the model. In PIII, these
are an integral part of the process of informing the decision. In PIV, these are
used to indicate which parts of the model would benefit most from additional
evidence-seeking efforts, and for identifying additional questions which would
benefit from research efforts before implementing the model in practice.
In both frameworks, the focus is not on the produced numbers per se. While
the numbers translate available evidence in numerical form to allow further
mathematical inferences, the model-based results should be seen in a wider
knowledge-seeking effort. The models suggest a certain risk level, but it is an
assessor or a group of assessors who deliberate on the model outcome to form
a judgment about the risk, upon which is acted.
In this sense, the scientific approach may also have a procedural element as
outlined in Section 2.1 and 3.1 (Figure 3), as it is possible to include different
stakeholder groups in the process of deliberative judgment.
42
5. Frameworks for risk analysis
This chapter focuses on Objective 4 as identified in Section 1.2, with corre-

sponding research question 3. of Section 3.3. The main novelties are:
x a 2-stage maritime transportation risk analysis framework [PIII]27;
x tools for evaluating the strength of evidence and for assessing the ef-
fect of assumptions [PIII]28;
x a 2-stage framework for collision alert systems [PIV];
x a framework for operationalizing “ship-ship collision risk” [PIV];
x the risk models in the example applications [PIII, PIV].
5.1 Policy-oriented risk analysis: accidental risk in a waterway
5.1.1 Framework
Understanding risk as in Section 4, the proposed framework consists of two

stages, see Figure 9. The first is oriented towards expert-review, the second
towards decision makers. It is discussed in detail in PIII, from p. 43 onwards.
Figure 9. Proposed framework for risk analysis of maritime transportation systems. Symbols of
the risk perspectives RP1 and RP2 are explained in PIII, from p.43 onwards
27
The framework is inspired by Aven (2010) and Aven (2013) but is based on a modified risk
perspective (including biases), adopts a different method for assessing the evidence and for
selecting alternative hypotheses, and is expanded with a second stage.
28
These tools are inspired by Kloprogge et al. (2011), Zio and Apostolakis (1996) and Aven
(2013), but are modified and extended in light of the adopted risk perspective.
43
Frameworks for risk analysis
In the first stage, a risk model is developed using Bayesian Networks (BN).
This (non-predictive) model does not lead to a risk characterisation by itself,
but functions as i) an argumentation based on available evidence, ii) a tool for
communication between stakeholders, and iii) an aid to thinking.
Model functions i) and ii) are closely connected with the qualitative evidence
assessment, in which a set of qualities of the evidence underlying the model
construction are judged by an assessor. This provides insight behind the quan-
tification, indicating which parts of the model are well-supported and which
are not, which are conservative and optimistic and which to prioritize for fur-
ther refinement. Together with a sensitivity analysis, this evidence assessment
is used for selecting alternative hypotheses for the most important elements,
where importance is qualitatively mapped in terms of sensitivity, strength of
evidential support and direction of bias. The alternative hypotheses are im-
plemented in the risk model and allow insight in the stability of the risk metric
in light of plausible alternative submodels in the Bayesian Network.
Model function iii) takes the model as its object. The fundamental idea is
that the model does not directly provide insight in the actual situation being
modelled. Rather, it provides insights by revealing key features of its own as-
sumptions, leading to a reflection on behalf of the model user. Thus, the asses-
sor deconstructs the model by identifying assumptions, the effect of which on
the model results are argumentatively assessed. Acknowledging the instability
of open systems as outlined in Section 4.2.1, these assumptions include the
causal relations between the variables in the Bayesian Network model (the
arcs) and the parameterization of the probability tables underlying the varia-
bles (the nodes). Assumptions can also relate to factors or phenomena not
considered in the model. Assumptions are assessed regarding the magnitude
and direction of deviation and consequence range where the assumption effect
occurs. The strength of justification indicates how plausible the ratings are.
In the second stage, the results of the first stage are taken as evidence for a
deliberative uncertainty judgment. An assessor expresses his degree of belief
of the consequences considering the evidence of the first stage. Due to the in-
accuracy of these beliefs, subjective uncertainty intervals are used. A qualita-
tive assessment of the global strength of evidence accompanies these judg-
ments, providing insight in the confidence the assessor has about these.
The decision making does not follow directly from the risk quantification,
but relies on a broad evaluation process where other relevant factors (costs,
public and socio-economic concerns) are considered as well.
Compared with the state of art, the framework takes a clearer position in the
spectrum of scientific approaches to Risk Analysis, see Figure 3. The risk anal-
ysis is primarily “precautionary constructivist with uncertainty evaluation”, as
it is the result of a series of judgments by an assessor, broadly considering un-
certainties. The precautionary character follows from the explicit attention
given to value-laden biases. The framework can also be proceduralist if the
judgments in the second stage are made by a wider stakeholder group.
Furthermore, the evidence and assumption assessment schemes contribute
to ongoing research on how to characterize uncertainty in risk analyses. Rec-
44
ognizing the previous developments in risk research for tools for characteriz-
ing the strength of evidence, uncertainties and biases, e.g. (Aven, 2013; Flage
and Aven, 2009; Rosqvist and Tuominen, 2004), the developed tools add to
the available methods for this purpose.
5.1.2 Application
In PIII, the framework is applied to a case study addressing the accidental risk
of ship-ship collisions with oil tankers in the open sea area of the Gulf of Fin-
land. The focus is on the consequences, in particular on the probability of oil
spills exceeding certain sizes. Such information is useful to inform response
capacity and fleet organization planning, and for ecological risk analysis.
Throughout the analysis, a precautionary decision maker is assumed. Details
about the model and analysis are found in PIII, from p. 52 onwards.
The BN model integrates ship traffic, tanker layout and vessel dimension da-
ta, accident data, ship collision damage models and expert judgments.
The evidence assessment shows that the encounter situation (described
through ship traffic and vessel data) is based on strong evidence, whereas the
impact situation (mainly based on expert judgment) involves higher evidence
uncertainty. The consequence models are based on accepted engineering prin-
ciples, but are poorly empirically confirmed and involve a conservative bias.
The parameter sensitivity analysis on the BN-model, together with the evi-
dence assessment, is used to identify BN-variables which are prioritized for
considering alternative hypotheses. When implemented, the risk model is used
to calculate the model-based risk levels, using the alternative hypotheses.
The assumption assessment is used to reflect on the effect of the assump-
tions on the model results, focusing on the magnitude and direction of devia-
tion and the consequence range where these occur.
Figure 10. Results of the case study using the proposed framework. PIII, p. 51 onwards
45
In the second risk analysis stage, the information from the first stage is used
to make a deliberative uncertainty judgment, using interval probabilities. A
colour code represents the overall strength of evidence on which this judgment
is based.
5.2 Operational risk analysis: risk in collision avoidance context
5.2.1 Framework
Understanding risk as in Section 4, the proposed framework for a risk-

informed collision alert system (CAS) takes a current situation as its basis for
identifying the possibility of ship-ship collision between the “own” and the
“target” vessel. The CAS intends to enhance situational awareness, the lack of
which has been found to be an important contributing factor to ship collision
accidents (Baldauf et al., 2011; Chauvin et al., 2013; Grech et al., 2002).
Figure 11 illustrates the theoretical framework for operationalizing ship-ship
collision risk, explained in detail in PIV, from p. 184 onwards.
Figure 11. Outline of the theoretical framework for operationalizing ship-ship collision risk,
based on PIV
In regards the collision risk, the assessor relies on a set of situational quali-
ties (SQs) of a present time to make an interpretation about the possibility of
occurrence of a collision event (the focus in a possible future situation). As
outlined in Table 6, situations have a complex structure with a foreground,
background, focus and horizon. The main challenge in devising a risk-
informed CAS is to identify which SQs are primarily relied on for judging the
possible collision occurrence. In the framework of PIV, based on expert elicita-
tion, it is taken that four mechanisms underlie the risk judgment. These are
the COLREGs29, which (amongst other) guide the interpretation into a certain
encounter type (overtaking, head-on or crossing), the interpretation of the
imminence of accident occurrence, of the deviation from a reference level and
of the presence of ambiguity.
29
COLREGS: international regulations for preventing collisions at sea
46
For different encounter types, the current situation is operationalised

through a set of foreground situational qualities (FSQs). These are influenced
by certain background situational qualities (BSQs).
The judgmental character of the risk analysis is clear from the fact that the
SQs are interpretations by an assessor based on the actual system states 30.
Furthermore, the risk interpretation is contextual in the sense that the SQs can
be operationalized differently for different navigational settings. For example,
in open sea navigation, the “deviation from reference level” can be that a target
vessel navigates closer to the own vessel than would normally be expected in
similar encounters, or that the target vessel performs an unexpected turning
manoeuvre. In ice convoy navigation, the “deviation from reference level”
could be that a target vessel ahead in the convoy suddenly drops its speed.
The risk is measured using a two-stage approach, see Figure 12.
Figure 12. Framework for risk analysis in operational collision avoidance context
In the first stage, a fuzzy expert system (FES) based risk model is construct-
ed. This model converts system states to measurements of SQs, which are fur-
ther combined into a qualitative risk measure. A FES is a modelling tool which
can map complex, nonlinear input-output relations in an intuitive and rela-
tively simple manner. An interpretation of the main elements of a FES is given
in light of the adopted risk perspective in PIV, from p. 186 onwards.
In the second stage, the risk indicator (i.e. the model output) is considered in
a wider context and other information sources are consulted. These include
visual observation, information from other technological sources (ECDIS and
radar) and communication through VHF radio 31 . Together, these lead to a
judgment about the likeliness of ship-ship collision. Based on this judgment, a
decision is made about the need to perform collision-avoidance actions.
Thus, the risk model suggests a risk level rather than declaring the situation
to be of a certain definite severity. The model is not intended to be used in an
automated collision avoidance algorithm, unlike many models suggested in the
literature (Statheros et al., 2008; Tam et al., 2009).
The reason for this is that the model only accounts for the main SQs. Howev-
er, other elements in the situation, e.g. the presence of a recreational vessel, a
third vessel in the encounter or a modified navigational status of one of the
encountering vessels as signalled by lights and shapes, are not modelled. These
SQs may affect the risk judgment and the subsequent decision making.
30
E.g., a distance (an actual system state) can be interpreted as ’close’ by an assessor.
31
ECDIS: electronic chart display and information system, VHF: very high frequency
47
5.2.2 Application
In PIV, the framework is applied to a case study for an open sea area. The re-
sulting CAS model is schematically shown in Figure 13.
Figure 13. Schematic overview of the developed risk-informed CAS, based on PIV
The structure is defined through an expert elicitation process, identifying five

structural clusters corresponding to different types of encounters: overtaking,
crossing, head-on, ambiguous and encounters with an unexpected turn.
In each of these, different SQs are relied on for interpreting the possible col-
lision occurrence. The Analytical Hierarcy Process (Saaty, 1980) is used to elic-
it which FSQs are relied on for making the risk interpretation, and which BSQs
influence the interpretation of the FSQs.
The discretization and parameterization of the baseline categories of the SQs,
i.e. the membership functions (MFs) of the variables of the FES, is based on an
expert elicitation procedure proposed by Cornelissen et al. (2003). These base-
line MFs are modified into situational MFs based on the values of BSQs.
Using the rule base, these situational MFs are linked to an appropriate ordi-
nal risk level (safe, caution, warning and alarm), which are given an interpre-
tation relevant for navigational operations, see PIV, p. 188.
As outlined in Section 4.3, uncertainties and biases need to be addressed. In
PIV, the evidential qualities of the data, expert judgments, models and as-
sumptions are qualitatively rated separately for the model structure, content,
discretization and parameterization.
Other uncertainties requiring attention (and possibly leading to modifica-
tions to the model) before implementing the system in practice include: i) the
plausibility of the reduction of actual encounters to pairwise encounters, ii) the
number of alarms raised by the CAS compared with currently applied meth-
ods, iii) the existence of possible risk-compensating behaviour and risk migra-
tion due to the use of the model, and measures to counteract these effects.
48
6. Evaluating risk models and analyses
This chapter addresses Objective 5 of Section 1.2, and research question 4. of

Section 3.3. It provides an integrated view on the issue of evaluation of risk
models and analyses, which combines the approaches taken in PIII and PIV.
The reason for combining these in an overarching framework is that while the
risk models are used differently (in PIII the entire model is subject to scrutiny
in its use phase, in PIV only the output is used), the main characteristics of
evaluating the risk model are very similar. The main novelties are:
x a discussion on the nature of evaluation in risk analysis32;
x a framework including criteria to establish credibility of risk models
and analyses, based on the principles of Section 4 [PIII,PIV]33.
6.1 Evaluation (not validation) of a risk model/analysis
Debates about the nature of validation in science cannot be seen separate from
an underlying adherence to fundamental theories of knowledge: what is
knowledge and what constitutes confirmation of a knowledge claim? Barlas
and Carpenter (1990) describe two major paradigms in the philosophy of sci-
ence addressing these questions: the logical empiricist/reductionist school and
the functionalist/holistic school34.
According to the former paradigm, knowledge is an entirely disinterested,
universal, asocial, acultural, value-free “truth”. Utilizing the metaphor “mirror
of nature”, such philosophies consider knowledge as the reflection of nature on
an “unclouded mirror”. The latter paradigm takes knowledge to be socially
justified belief rather than a product of mirroring nature. Knowledge is a type
of assertion warranted because of the arguments given to support it rather
than because of an absolute external standard. Knowledge is socially, cultural-
ly and historically embedded, lacking neutral foundations.
32
This issue is not covered explicitly in the publications, but is elaborated upon to understand
the principles underlying the evaluation framework.
33
The framework is mainly based on the ideas discussed in PIII, p. 53 onwards. It is extended
to account for the different model uses of PIII and PIV, more clearly distinguishing model
structure, content, discretization, parameterization and behavior.
34
Clearly, the reduction to two paradigms is a rough simplification as many theories exist, see
e.g. Lemos (2007) for an overview. The analysis by Barlas and Carpenter (1990) is followed as
it is considered sufficiently distinctive for the current purposes, and because it is broadly in
line with the realist-constructivist distinction made by e.g. Bradbury (1989), Shrader-Frechette
(1991), Thompson and Dean (1996) and Hermansson (2012), see Section 2.1 and Publication I.
Evaluating risk models and analyses
Under a logical empiricist/reductionist philosophy of knowledge, validation

is seen as a strictly formal, algorithmic, “confrontational” process. As the mod-
el is taken as an objective and absolute representation of the modelled system,
attempts at revealing its truth are focused on confronting the model with em-
pirical “facts”35. In a functionalist/holistic view, validation becomes a semi-
formal, conversational process. Models are not true or false, but can be more
or less useful: model validation is a process of building confidence in the mod-
el’s usefulness. Hence, this becomes a conversational matter, where argumen-
tative justification rather than algorithmic procedures take centre stage.
In Section 4.1, a constructivist understanding of the risk concept has been
adopted, where risk is not an objective reality existing as an inherent part of a
system, but rather something attributed to a system by an assessor in terms of
a possibility. Due to violation of the constancy requirements and the empirical
testability of risk models in complex socio-technical systems such as the mari-
time transportation system (see Section 4.2), it is proposed to use risk models
heuristically, as a guide in inquiry rather than as a reflection of an underlying
truth. The intended use of risk models is argumentatively and suggestively, as
a basis for communication and as an aid to thinking (both in the policy-
oriented and operational risk analyses).
From this, it is evident that for the developed risk analysis frameworks of
PIII and PIV, validation is to be understood in line with the functional-
ist/holistic view. Focus is on conversationally establishing that the models can
be used as intended. Correspondingly, model evaluation is taken as a forma-
tive, rather than summative process, i.e. the evaluation should contribute to
the reflection of the risk analysts and reviewers of risk analyses (peers and
stakeholders) rather than to declare the analysis to be of a certain standard, a
view shared by Busby and Hughes (2006).
Furthermore, in Section 4.2.3, it is argued that a broad uncertainty and bias
evaluation is needed. Thus, rather than being validated, risk models are eval-
uated. As noted by Oreskes (1998), evaluation implies an assessment in which
both positive and negative results are highlighted, where the grounds on which
a model is declared good enough for its purpose are articulated while openly
acknowledging uncertainties36.
6.2 Evaluation of a model-based risk analysis: framework
6.2.1 General outline
In the frameworks of PIII and PIV, the risk model and the risk analysis are
distinguished, the former being a part of the latter. Mechanisms are required
35
An approach along these lines in the context of waterway risk analysis is taken e.g. by Friis-
Hansen and Simonsen (2002), Weng et al. (2012) and Mulyadi et al. (2014), where the model
output is compared with historical data as (the only) validation method.
36
Oreskes et al. (1994) provide further definitions, clarifying the distinctions. Verification
(from Latin verus: true) is the process of establishing truth. Validation (from Latin validus:
strong) does not necessarily establish truth, but is a process of establishing legitimacy. Evalua-
tion (from French évaluer: to find the value of) is a process of appraising or valuing.
50
for looking beyond the model-based results. In PIII, this is performed through
an evidence and assumption assessment. In PIV, other means of observation
are used during the model use phase, while an evidence assessment and uncer-
tainty identification process highlight further research directions in the model
development stage.
A central issue in the evaluation is the intended use of the model, see Section
4.2.2 and 4.2.3. In PIII, the model is a communication tool which puts forward
an argumentation, which is scrutinized through an evidence and assumption
assessment scheme to arrive at a risk judgment. In PIV, the model suggests an
alert level to the system user, who seeks other information to judge the risk.
Figure 14 shows the conceptual framework for evaluating policy-oriented
and operational risk analyses, combining the ideas of the risk model/analysis
evaluations performed in PIII and PIV. Table 8 contains the specific evaluation
criteria for the different aspects.
Figure 14. Framework for evaluating risk models and analyses, integrating ideas of PIII and PIV
The framework integrates a number of ideas from various sources and disci-
plines37. The focus is on the intended practical use of the model (item i.), fol-
lowing ideas of Hodges and Dewar (1992). Certain further usability criteria are
identified: the interpretability of the numbers (Aven, 2011b) and the focus on
observable quantities (Aven and Heide, 2009)38. The evaluation of the model
37
These include operations research (Hodges and Dewar, 1992), social sciences (Trochim and
Donnely, 2008), systems dynamics modelling (Forrester and Senge, 1980), and modeling with
expert systems (Pitchforth and Mengersen, 2013). Relevant sources from within Risk Analysis
are included as well (Aven and Heide, 2009; Rosqvist and Tuominen, 2004). While various
elements of the framework are known in other research communities, the framework has to
the best of the author’s knowledge not been presented in an integrated manner in the Risk
Analysis discipline.
38
In Aven and Heide (2009), this concerns whether uncertainties are assessed about observa-
bles, or about model parameters. The assignment of subjective uncertainty intervals to model
51
qua model (item ii.) combines ideas from Forrester and Senge (1980),
Trochim and Donnely (2008) and Pitchforth and Mengersen (2013). The con-
sideration of uncertainties and biases (item iii.) is included in the evaluation
framework based on ideas from Rosqvist and Tuominen (2004) and Aven and
Heide (2009). The importance of procedural aspects of the risk analysis (item
iv.) is based on Rosqvist and Tuominen (2004) and Aven and Heide (2009).
6.2.2 Specific evaluation criteria
Table 8 lists the four identified evaluation aspects of Figure 14, considered
separately for policy-oriented and operational uses as needed. Each aspect is
further split into different evaluation criteria, which are given a name and
which are further concretized by formulating a question which aids in estab-
lishing credibility. The sources on which the proposed criteria are based are
indicated as well.
The focus is on the model use (item i.), which implies that different criteria
are applied for the models of PIII and PIV. Both models are intended as aids to
communication and can be evaluated by ascertaining that these indeed assist
in communication. The model of PIII is used as a policy-support tool, and can
be evaluated in terms of its ability to assist decision making, whether the rec-
ommendations are sensitive to the identified model scope, structure, content,
discretization and parameterization and whether the implementation of the
recommendations improves system performance. The model of PIV is used to
provide alerts, and can be evaluated by ascertaining if the alerts are timely and
parsimonious. It can also be evaluated by identifying the model’s ability to
assist decision making and by reducing the number of collisions.
The evaluation of the model qua model (item ii.) focuses on model structure,
content, discretization, parameterization and behavior. Considering these as-
pects in turn creates a platform for systemizing the formative, conversational
process of argumentative reflection about the model as an adequate tool for
aiding an assessor in making a risk judgment. For each element, various crite-
ria/tests are identified, see Table 8. These aspects can be understood in terms
of the concept of the operationalization of a construct, see Trochim and Don-
nely (2008). This concerns how the mental construct (the conceptual level) is
transformed to a model construct (the measurement level). This is split into
translational validity (how well the mental construct is translated in a model)
and criterion validity (how well the model scores in certain tests). Translation
validity consists of face and content validity, whereas criterion validity consists
of concurrent, convergent and behaviour validity39. These are outlined below
and concretized in specific criteria in Table 8.
parameters has been criticized by e.g. Apostolakis (1990) because of the difficulty in explain-
ing what exactly the assessor is uncertain about and what the uncertainty interval means.
39
Trochim and Donnely (2008) and Pitchforth and Mengersen (2013) also consider discrimi-
nant validity, i.e. the degree to which the measurement/model appropriately differs from
measurement/models describing another system. As this is considered of too little added
value in an evaluation exercise, this is not retained here.
52
Face validity refers to a subjective interpretation of the relevance and quality

of the model in relation to the concept it intends to measure. For example,
when considering the tanker oil spill risk, it can be checked that the tank sizes
are considered in the model.
Content validity concerns a more careful judgment about the relevance and
completeness of the elements included in the model in relation to the system it
intends to describe. For example, each variable can be considered in turn,
where its underlying rationale and its relations to other variables are scruti-
nized, which may expose assumptions relevant to consider.
Concurrent validity refers to how the model compares with other models for
the same system. For example, the elements and relations considered in an oil
spill risk model for a waterway area can be compared with other models for
this purpose. Such a comparison can elucidate discrepancies, which can in-
crease the credibility of the model, or highlight uncertainties and assumptions.
Convergent validity addresses how the model compares with other models
for similar systems. For example, the elements considered in a ship collision
risk model can be compared with a risk model for road traffic collisions. This
can increase, on a more distal level, the confidence of the model’s plausibility.
Behaviour validity concerns a number of specific tests, e.g. questioning
whether the model response is qualitatively in line with expectations and
whether the model behaviour is sensitive to elements to which the system is
expected to be sensitive to.
All above listed model evaluation aspects can be used to increase the confi-
dence in the model as an appropriate representation of the addressed phe-
nomenon. Simultaneously, these can also be used to highlight uncertainties
and biases beyond the model, in line with item iii. of the risk analysis evalua-
tion framework.
As mentioned above, in the framework of PIII, these uncertainties and biases
(considered by alternative hypotheses, evidence assessment or an assumption
assessment) are used in and alongside the model to arrive at a risk judgment.
The biases in the model need to be in accordance with stakeholder values. In
the framework of PIV, the uncertainties need consideration before implement-
ing the model in practice while the biases need to be in line with the non-
epistemic aims of the model, i.e. stakeholder values need to be appropriately
reflected. This last issue relates to the need to provide sufficient and timely
alerts, while minimizing the number of alerts raised, as these are often consid-
ered a nuisance (Baldauf et al., 2011).
The procedural aspects of the risk analysis (item iv.) concern whether the ev-
idence and the model construction are transparent. Finally, it can be evaluated
whether the expert elicitation (subjective probabilities in PIII, fuzzy member-
ship functions in PIV) are based on accepted expert elicitation procedures.
Relevant guidelines in Cornelissen et al. (2003) and Aven and Heide (2009)
are referred to.
53
Table 8. Evaluation framework and criteria, elements from Figure 14
i. Model use and further implications Ref. Tested

Policy-oriented PIII PIV
OBS Observability Does the analysis focus on observable system qualities
AH Y N/A
or on model parameters?
INT Interpretability Can the produced numbers be given an interpretation? A1 Y N/A
COM Aid to com- Does the model help the communication between as-
HD N N/A
munication sessors/stakeholders in the deliberation phase?
PS Policy If the model scope, structure, content, discretization or
sensitivity parameterization is changed, do the policy recommenda- FS N N/A
tions change significantly?
SI System Are the provided results helpful to decision makers?
FS N N/A
improvement When applied, is the system performance improved?
Operational
TA Timeliness of Are the alerts provided in a timely manner?
- N/A Y
alerts
PA Parsimony of Is the number of alerts adequate? Is the number of
- N/A N
alerts unnecessary alerts minimal?
INT Interpretability Can the produced numbers be given an interpretation? A1 N/A Y
COM Aid to com- Does the model assist in the communication between
HD N/A N
munication operators to form a risk judgment?
SI System Are the alerts considered useful in decision making?
FS N/A N
improvement Does the model help to reduce the number of collisions?
ii. Model per se
Structure
FV Face Does the model structure seem like a plausible repre- TD
Y Y
validity sentation of the conceptual construct? PM
CV Content Do the relationships between model elements and
TD
validity variables correspond to how the corresponding factors Y N
PM
in the conceptual construct are related?
CCV Concurrent Does the model structure correspond to that of another TD
Y Y
validity model for the same problem in the same system? PM
CVV Convergent Does the model structure correspond to that of a TD
N N
validity model for a comparable problem in a similar system? PM
BA Boundary Are the model elements aggregated in an appropriate
adequacy manner? Are the structural relations at the interfaces FS N N
between submodels adequate?
Content
FV Face Do the model elements seem to adequately represent TD
Y Y
validity the conceptual construct? PM
CV Content Does the model contain the most relevant factors rele- TD
Y Y
validity vant to the conceptual construct? PM
CCV Concurrent Does the model (or its submodels) contain the same TD
Y Y
validity elements as another model for the same problem? PM
CVV Convergent Does the model (or its submodels) contain similar ele- TD
N N
validity ments as a model for a similar system? PM
Discretization (of nodes of a BN or MFs of a FES)
FV Face Is each variable discretized into states which look plau- TD
N N
validity sible to experts? PM
CV Content Is each variable discretized into states which correspond TD
N N
validity to the full range the variable can adopt? PM
CCV Concurrent In another model for the same problem in the same TD
N N
validity system, are similar variables similarly discretized? PM
CVV Convergent In a model for a comparable problem in a similar system, TD
N N
validity are similar variables similarly discretized? PM
DC Dimensional Are all possible states included in the discrete states of FS
Y Y
consistency the variable? PM
Parameterization (probabilities, parameters of MFs, other factors)
FV Face Do the parameters in the model look plausible to the TD
N N
validity experts? PM
CV Content Do the parameters adequately reflect the background TD
N N
validity knowledge? PM
CCV Concurrent In another model for the same problem in the same TD
N N
validity system, do the parameters have comparable values? PM
CVV Convergent In a model for a comparable problem in a similar system, TD
N N
validity do the parameters have similar values? PM
DC Dimensional Are the units for the parameters in the model dimension-
FS Y Y
consistency ally compatible?
Behaviour tests
BST Behaviour Is the model sensitive to structural relations and/or pa-
FS Y N
sensitivity rameters to which the conceptual system would also be?
QFT Qualitative Does the model qualitatively respond to hypothesized
FS Y Y
features variable states as the conceptual system would?
ECT Extreme Does the model respond to hypothesized extreme varia-
FS Y N
conditions ble states as the conceptual system would?
CCV Concurrent When the model is run, are the output states comparable TD
Y Y
validity to those of other models? PM
54
iii. Value-related validity

Uncertainty
AH Alternative Are alternative structural relations, discretizations or AH
Y/N N
hypotheses parameterizations assessed by alternative hypotheses? ZA
AS Assumptions Are assumptions in and beyond the model identified and AH
Y Y
their effects on the model output assessed? A2
EVI Evidence Is the evidential strength assessed and areas for im- AH
Y Y
provement identified? FA
OUT Outcome Are additional outcome uncertainties identified? Are the AH
Y N
effects on the model outcome assessed? FA
Bias
VAL Stakeholder When value-laden choices are needed in the analysis,
RT Y N
values are stakeholder values appropriately reflected?
EVI Evidence Are evidential biases assessed? RT Y Y
OUT Outcome Is the effect of evidential biases on the model outcome
RT Y N
assessed?
iv. Process
EE Expert Is the expert elicitation based on accepted procedures
AH Y Y
elicitation and guidelines?
TR Transparency Is the model construction and underlying evidence
RT N N
transparent?
A1 = Aven (2011b) | A2 = Aven (2013) | AH = Aven and Heide (2009) | FA = Flage and Aven (2009) | FS =
Forrester and Senge (1980) | HD = Hodges and Dewar (1992) | PM = Pitchforth and Mengersen (2013) | RT
= Rosqvist and Tuominen (2004) | TD = Trochim and Donnely (2008) | ZA = Zio and Apostolakis (1996) | Y
= assessed in the Publication | N = not assessed in the Publication
6.3 Examples from the risk analysis applications
In Table 8, it is shown which evaluation tests have been applied to the applica-
tions developed in PIII and PIV. For PIII, examples are found from p. 54 on-
wards. For PIV, this is from p. 192 onwards.
While the framework is constructed with BNs and FESs as modelling tools, it
is plausible to assume that the framework (or elements thereof) can also be
useful for evaluating risk analyses using other modelling techniques. This is
clear from the discussions made in the appendix of PI from p. 128 onwards,
where various criteria of the evaluation framework are applied to maritime
waterway risk models which apply different modelling tools.
For example, the interpretability of the risk numbers has been considered in
the three example applications discussed in PI from p. 128 onwards, where it is
found that the produced numbers cannot always be meaningfully interpreted.
This can be problematic, as it may be difficult to explain what exactly has been
measured in the analysis.
In one of the applications (PI, p. 130), the dimensional consistency test is
applied, finding that the model discretization is made in such a way that the
risk seems larger in certain sea areas than in others. However, upon applica-
tion of this test, it is found that the reason for this discrepancy in risk levels
may be significantly affected by the choice of the size of the sea areas. Hence,
the measured parameters are not dimensionally comparable for the areas. This
leads to uncertainty about the conclusions made in the analysis.
In another application (PI, p. 131), the content validity criterion is applied to
the model structure of the vessel conflict operator, where it is argued that the
structural relations between TCPA and DCPA are incorrectly accounted for in
the model40. This is supplemented by a qualitative features test, which shows
40
TCPA: time to closest point of approach, DCPA: distance at closest point of approach
55
that two scenarios, which should lead to a different risk level, are not discrimi-
nated as such by the model. Hence, these tests are found helpful to highlight
uncertainties in the model.
Overall, these examples and the examples shown in PIII and IV show that
the evaluation criteria of the framework can increase confidence in the risk
models, and/or elucidate uncertainties in risk models. Depending on the in-
tended model use in the risk analysis, this can act as a basis for forming an
informed judgment about risk (PIII) or to identify focus areas for further re-
search (PIV).
56
7. Conclusions and future work
The overall aim of this thesis has been to contribute to a number of founda-
tional issues of Risk Analysis within the application area of maritime transpor-
tation, answering calls to extend such research to application areas.
PI has shown that Risk Analysis faces a number of challenges to its scientific
foundations. Unlike earlier research, which has addressed these issues on a
theoretical level, this has been approached through analysing applications and
through case studies. This is believed to be more directly beneficial for spur-
ring research and discussion on foundational issues within the specific appli-
cation areas in focus in this thesis.
The analysis of risk definitions and perspectives and scientific approaches in
PI confirms the existence of a rather chaotic situation in the discipline, as ar-
gued e.g. in Aven (2012a). Many risk definitions co-exist, while these do not
necessarily provide insight in the adopted scientific approach (realist, con-
structivist or proceduralist). Most work in the waterway risk analysis applica-
tions applies some form of the prototypical realist approach, with a focus on
the calculated probabilities/numbers, which are based primarily or exclusively
on data and mathematical/engineering models. Constructivist and procedural-
ist approaches are found as well, but represent a minority. PI also shows the
close relation between the adopted definition and perspective, and illustrates
the lack of uncertainty treatment in applications.
The case studies of PII and PIV provide evidence for the theoretical discus-
sions in Aven and Heide (2009) that risk analyses in general are not reliable
tools for informing a decision.
The above findings, corresponding to Objectives 1 and 2 as stated in Section
1.2, lead to the need to clarify a number of principles. First, a terminological-
conceptual basis is needed, clarifying how risk is understood, how it relates to
other concepts, and which implications this understanding has. Second, a rea-
soned argument is needed how risk models, if not reliable, can be used. Third,
consideration is needed as to how to evaluate risk models and analyses.
The above issues have been addressed in Objective 3 to 5, covered in PIII and
PIV. Two frameworks for risk analysis have been proposed, for policy-oriented
maritime transportation risk analysis and for risk analysis in a collision avoid-
ance context. The former is a quantitative risk analysis framework applying
Bayesian Networks as a modelling tool; the latter a qualitative framework ap-
plying Fuzzy Expert Systems. A risk analysis evaluation framework integrating
the approaches to evaluation in PIII and PIV is presented.
57
Conclusions and future work
Future research on risk analysis in maritime transportation can take several

directions, both concerning applications and related to more theoretical issues.
First, the presented frameworks can be applied to other accident types, and
models for the presented applications further improved. For the oil spill risk
model, more advanced collision damage and oil outflow models could be used,
to shed light on the temporal progression of the oil spill, considering subse-
quent hull damage. For the collision alert system, research can be performed
to strengthen the evidence base, to address the uncertainties mentioned in
Section 5.2.2, and to develop alert systems in other navigational environments
(port waters, harbour approaches, specific environmental conditions).
Second, additional frameworks for risk analysis in maritime transportation
can be developed, based on other modelling tools and approaches. For exam-
ple, PI shows that relatively little work has been performed to define maritime
transportation risk indicators. Further research on defining appropriate indi-
cators and on how to integrate these in a framework to monitor and mitigate
transportation risk could be undertaken. It is also worth exploring the possibil-
ities of new systemic accident theories in the context of waterway risk analysis.
The merits and challenges of proposed frameworks could be investigated.
Third, the research on principles and frameworks for risk analysis could be
extended to other maritime applications. Psaraftis (2012) identifies a number
of issues requiring research in the context of Formal Safety Assessment. The
feasibility of applying relatively recently proposed uncertainty-based risk per-
spectives (Aven, 2013; Aven, 2011) in the ship design process and in the con-
text of goal-based standards could be investigated.
Fourth, there is much room for additional research and discussion on the
evaluation/validation of risk analysis. As found in Section 2.2.3, very little sys-
tematic research has been dedicated to this issue. The contributions made in
this thesis could be discussed, further developed/adjusted and applied.
Finally, there is a long list of foundational issues in the general Risk Analysis
discipline which could be approached within the maritime application area.
Aven and Zio (2014) list 10 main scientific issues needing further research,
including concepts/terminology, methods for representing uncertainty, causal-
ity in a risk analysis context, the relation between risk analysis and values, how
to represent risk information and how to use risk analyses in different decision
making contexts. These authors furthermore identify more than 20 questions
which would benefit from future scientific attention. Many of these are rele-
vant as well to the maritime application area and correspond to the issues
identified by Psaraftis (2012) in the context of Formal Safety Assessment.
It is clear that the list of future research is very extensive, which corresponds
to the position taken in the introduction that Risk Analysis is an unsettled sci-
entific discipline.
The primary motivation of the research performed in this thesis is to estab-
lish that the maritime application area would, as the general discipline, benefit
from increased attention to and scientific research on foundational issues. To
the extent the proposed principles, frameworks and tools for evaluation con-
tribute to this end, the general aims of the work have been achieved.
58
Bibliography
Adams, J., 1995. Risk. Routeledge.

Almaz, O.A., 2012. Risk and performance analysis of ports and waterways: the
case of Delaware river and bay (PhD Dissertation). Rutgers University, De-
partment of Industrial and Systems Engineering, New Brunswick, New Jer-
sey.
Amendola, A., Contini, S., Ziomas, I., 1992. Uncertainties in chemical risk as-
sessment: results of a European benchmark exercise. J. Hazard. Mater. 29,
347–363.
Apeland, S., Aven, T., Nilsen, T., 2002. Quantifying uncertainty under a pre-
dictive, epistemic approach to risk analysis. Reliab. Eng. Syst. Saf. 75, 93–
102.
Apostolakis, G., 2014. Beware of the assumptions: decision making and statis-
tics. Presented at the PSAM12, Probabilistic Safety Assessment and Man-
agement, Honolulu, Hawaii, USA.
Apostolakis, G., 1990. The concept of probability in safety assessments of
technological systems. Science 250, 1359–1364.
Apostolakis, G., Garrick, B.J., Okrent, D., 1983. Quality, peer review, and the
achievement of consensus in probabilistic risk analysis. Nucl. Saf. 24, 792–
800.
Aven, T., 2013. Practical implications of the new risk perspectives. Reliab. Eng.
Syst. Saf. 115, 136–145. doi:10.1016/j.ress.2013.02.020
Aven, T., 2012a. Foundational issues in risk assessment and risk management.
Risk Anal. 32, 1647–1656.
Aven, T., 2012b. The risk concept—historical and recent development trends.
Reliab. Eng. Syst. Saf. 99, 33–44. doi:10.1016/j.ress.2011.11.006
Aven, T., 2011a. Selective critique of risk assessments with recommendations
for improving methodology and practise. Reliab. Eng. Syst. Saf. 96, 509–
514. doi:10.1016/j.ress.2010.12.021
Aven, T., 2011. Quantitative risk assessment - the scientific platform, 1st ed.
Cambridge University Press.
Aven, T., 2011b. Interpretations of alternative uncertainty representations in a
reliability and risk analysis context. Reliab. Eng. Syst. Saf. 96, 353–360.
Aven, T., 2010. Some reflections on uncertainty analysis and management.
Reliab. Eng. Syst. Saf. 95, 195–201. doi:10.1016/j.ress.2009.09.010
Aven, T., Heide, B., 2009. Reliability and validity of risk analysis. Reliab. Eng.
Syst. Saf. 94, 1862–1868. doi:10.1016/j.ress.2009.06.003
Aven, T., Reniers, G., 2013. How to define and interpret a probability in a risk
and safety setting. Saf. Sci. 51, 223–231. doi:10.1016/j.ssci.2012.06.005
Aven, T., Renn, O., 2009. On risk defined as an event where the outcome is
uncertain. J. Risk Res. 12, 1–11.
Aven, T., Vinnem, J.E., 2005. On the use of risk acceptance criteria in the off-
shore oil and gas industry. Reliab. Eng. Syst. Saf. 90, 15–24.
doi:10.1016/j.ress.2004.10.009
59
Bibliography
Aven, T., Zio, E., 2014. Foundational issues in risk assessment and risk man-
agement. Risk Anal. 34, 1164–1172.
Aven, T., Zio, E., 2011. Some considerations on the treatment of uncertainties
in risk assessment for practical decision making. Reliab. Eng. Syst. Saf. 96,
64–74. doi:10.1016/j.ress.2010.06.001
Baldauf, M., Benedict, K., Fischer, S., Motz, F., Schröder-Hinrichs, J.-U., 2011.
Collision avoidance systems in air and maritime traffic. Proc. Inst. Mech.
Eng. Part O J. Risk Reliab. 225, 333–343. doi:10.1177/1748006X11408973
Barlas, Y., Carpenter, S., 1990. Philosophical roots of model validation: two
paradigms. Syst. Dyn. Rev. 6, 148–166.
Beasley, M.S., Branon, B.C., Hancock, B.V., 2010. Developing key risk indica-
tors to strengthen enterprise risk management. The Committee of Sponsor-
ing Organizations of the Treadway Commission.
Bilgiҫ, T., Türkşen, B.İ., 1998. Measurement of membership functions: theo-
retical and empirical work, in: International Handbook of Fuzzy Sets and
Possibility Theory. Kluwer Academic, Norwell.
Bloch, E., 1995. The principle of hope. The MIT Press.
Bohnenblust, H., Slovic, P., 1998. Integrating technical analysis and public
values in risk based decision making. Reliab. Eng. Syst. Saf. 59, 151–159.
Bradbury, J.A., 1989. The policy implications of differing concepts of risk. Sci.
Technol. Hum. Values 14, 380–399.
Brown, M.J., 2014. Values in science beyond underdetermination and induc-
tive risk. Philos. Sci. 80, 829–839.
Brown, M.J., 2012. John Dewey’s Logic of Science. HOPOS J. Int. Soc. Hist.
Philos. Sci. 2, 258–306.
Bukhari, A.C., Tusseyeva, I., Lee, B.-G., Kim, Y.-G., 2013. An intelligent real-
time multi-vessel collision risk assessment system from VTS view point
based on fuzzy inference system. Expert Syst. Appl. 40, 1220–1230.
Busby, J.S., Hughes, E.J., 2006. Credibility in risk assessment: a normative
approach. Int. J. Risk Assess. Manag. 6, 508–527.
Campbell, S., Naeem, W., Irwin, G.W., 2012. A review on improving the au-
tonomy of unmanned surface vehicles through intelligent collision avoid-
ance manoeuvres. Annu. Rev. Control 36, 267–283.
Carmines, E.G., Zeller, R.A., 1979. Reliability and validity assessment, Quanti-
tative applications in the social sciences. SAGE Publications, Thousand
Oaks, California.
Chauvin, C., Lardjane, S., Morel, G., Clostermann, J.-P., Langard, B., 2013.
Human and organisational factors in maritime accidents: analysis of colli-
sions at sea using the HFACS. Accid. Anal. Prev. 59, 26–37.
Cornelissen, A.M.G., van den Berg, J., Koops, W.J., Kaymak, U., 2003. Elicita-
tion of expert knowledge for fuzzy evaluation of agricultural production sys-
tems. Agric. Ecosyst. Environ. 95, 1–18.
Crawford, J., 2001. What’s wrong with the numbers? A questioning look at
probabilistic risk assessment. J. Syst. Saf. 37, 1–11.
Cumming, R.B., 1981. Is risk assessmet a science? Risk Anal. 1, 1–3.
Davies, J., Finlay, M., McLenaghen, T., Wilson, D., 2006. Key risk indicators -
their role in operational risk management and measurement, in: The Ad-
vanced Measurement Approach to Operational Risk. Risk Books, London,
pp. 215–246.
Debnath, A.K., 2009. Traffic-conflict-based modeling of collision risk in port
waters (PhD Dissertation). National University of Singapore, Singapore.
De Rocquigny, E., Devictor, N., Tarantola, S. (Eds.), 2008. Uncertainty in in-
dustrial practice - a guide to quantitative uncertainty management. Wiley,
New Jersey.
Devanney, J., 2008. Formal Safety Assessment in Wonderland.
Diekmann, S., Peterson, M., 2013. The role of non-epistemic values in engi-
neering models. Sci. Eng. Ethics 19, 207–218.
60
Bibliography
Douglas, H., 2009. Science, Policy and the value-free ideal. University of Pitts-
burgh Press, Pittsburgh, PA.
Eide, M.S., Endresen, Ø., Skjong, R., Longva, T., Alvik, S., 2009. Cost-
effectiveness assessment of CO2 reducing measures in shipping. Marit. Pol-
icy Manag. Flagship J. Int. Shipp. Port Res. 36, 367–384.
doi:10.1080/03088830903057031
Elliott, K., Willmes, D., 2014. Cognitive attitudes and values in science. Philos.
Sci. 80, 807–817.
Fiorino, D.J., 1990. Citizen participation and environmental risk: A survey of
institutional mechanisms. Sci. Technol. Hum. Values 15, 226–246.
Fiorino, D.J., 1989. Technical and democratic values in risk analysis. Risk
Anal. 9, 293–299.
Flage, R., Aven, T., 2009. Expressing and communicating uncertainty in rela-
tion to quantitative risk analysis (QRA). Reliab. Risk Anal. Theory Appl. 2,
9–18.
Forrester, J.W., Senge, P.M., 1980. Tests for building confidence in system
dynamics models. TIMS Stud. Manag. Sci. 14, 209–228.
Fowler, T.G., Sorgård, E., 2000. Modeling ship transportation risk. Risk Anal.
20, 225–244.
Friis-Hansen, P., Simonsen, B.C., 2002. GRACAT: software for grounding and
collision risk analysis. Mar. Struct. 15, 383–401.
Grabowski, M., Merrick, J.R.W., Harrald, J.R., Mazzuchi, T.A., van Dorp, J.R.,
2000. Risk modeling in distributed, large-scale systems. IEEE Trans. Syst.
Man Cybern. - Part Syst. Hum. 30, 651–660.
Graham, J.D., 1995. Verifiability isn’t everything. Risk Anal. 15, 109.
Grech, M.R., Horberry, T., Smith, A., 2002. Human error in Maritime Opera-
tions: Analyses of Accident Reports Using the Leximancer Tool. Proc. Hum.
Factors Ergon. Soc. Annu. Meet. 46, 1718–1721.
Hänninen, M., 2014. Bayesian networks for maritime traffic accident preven-
tion: benefits and challenges. Accid. Anal. Prev. 73, 305–312.
Hänninen, M., Mazaheri, A., Kujala, P., Montewka, J., Laaksonen, P., Sal-
miovirta, M., Klang, M., 2014. Expert elicitation of a navigation service im-
plementation effects on ship groundings and collisions in the Gulf of Fin-
land. Proc. Inst. Mech. Eng. Part O J. Risk Reliab. 228, 19–28.
Hansson, S.O., 2010. Risk: objective or subjective, facts or values. J. Risk Res.
13, 231–238.
Hansson, S.O., 2000. Seven myths of risk. Presented at the Stockholm thirty
years on. Progress achieved and challenges ahead in international environ-
mental co-operation, Swedish Ministery of the Environment, Stockholm,
Sweden, pp. 1–15.
Hansson, S.O., 1999. A philosophical perspective on risk. Ambio 28, 539–542.
Hansson, S.O., Aven, T., 2014. Is risk analysis scientific? Risk Anal. 34, 1173–
1183.
Hartford, D.N.D., 2009. Legal framework considerations in the development
of risk acceptance criteria. Struct. Saf. 31, 118–123.
Heinrich, H.W., 1931. Industrial accident prevention: a scientific approach.
McGraw-Hill, New York.
Hermansson, H., 2012. Defending the conception of “objective” risk. Risk
Anal. 32.
Hermansson, H., 2010. Towards a fair procedure for risk management. J. Risk
Res. 13, 501–515.
Hermansson, H., 2005. Consistent risk management: three models outlined. J.
Risk Res. 8, 557–568.
Hilgert, H., Baldauf, M., 1997. A common risk model for the assessment of
encounter situations on board ships. Dtsch. Hydrogr. Z. 49, 531–542.
doi:10.1007/BF02764347
61
Bibliography
Hodges, J.S., 1991. Six (or so) things you can do with a bad model. Oper. Res.
39, 355–365.
Hodges, J.S., Dewar, J.A., 1992. Is it you or your model talking? A framework
for model validation (Research Report No. R-4114-AF/A/OSD), The RAND
Publication Series. RAND, Santa Monica, CA.
Hollnagel, E., 1998. Cognitive reliability and error analysis method (CREAM).
Elsevier Science.
Hu, S., Fang, Q., Xia, H., Xi, Y., 2007. Formal safety assessment based on rela-
tive risks model in ship navigation. Reliab. Eng. Syst. Saf. 92, 369–377.
IMO, 2008. Formal Safety Assessment - FSA Crude oil tankers (No. MEPC
58/INF.2). International Maritime Organization.
IMO, 2007. Formal safety assessment - consolidated text of the guidelines for
formal safety assessment (FSA) for use in the IMO rule-making process.
IMO, 2002. Guidelines for formal safety assessment (FSA) for use in the IMO
rule-making process. International Maritime Organization -
MSC/Circ.1023-MEPC/Circ.392, London, UK.
Johansen, I.L., 2010. Foundations of risk assessment (No. ROSS (NTNU)
201002). Norwegian University of Science and Technology - Department of
Marine Technology, Department of Production and Quality Engineering,
Trondheim, Norway.
Kao, S.-L., Lee, K.-T., Chang, K.-Y., Ko, M.-D., 2007. A fuzzy logic method for
collision avoidance in Vessel Traffic Service. J. Navig. 60, 17–31.
Kaplan, S., 1997. The words of risk analysis. Risk Anal. 17, 407–417.
Kermisch, C., 2012. Vers une définition multidimensionelle du risque. VertigO
- Rev. Électronique En Sci. Environ. 12. doi:10.4000/vertigo.12214
Klanac, A., Varsta, P., 2011. Design of marine structures with improved safety
for environment. Reliab. Eng. Syst. Saf. 96, 75–90.
Klemola, E., Kuronen, J., Kalli, J., Arola, T., Hänninen, M., Lehikoinen, A.,
Kuikka, S., Kujala, P., Tapaninen, U., 2009. A cross-disciplinary approach
to minimizing the risks of maritime transport in the Gulf of Finland. World
Rev. Intermodal Transp. Res. 2, 343–363.
Klinke, A., Renn, O., 2002. A new approach to risk evaluation and manage-
ment: risk-based, precaution-based, and discourse-based strategies. Risk
Anal. 22, 1071–1094.
Kloprogge, P., van der Sluijs, J.P., Petersen, A.C., 2011. A method for the anal-
ysis of assumptions in model-based environmental assessments. Environ.
Model. Softw. 26, 289–301.
Kontovas, C.A., Psaraftis, H.N., 2009. Formal safety assessment: a critical re-
view. Mar. Technol. 46, 45–59.
Kontovas, C.A., Psaraftis, H.N., Ventikos, N.P., 2010. An empirical analysis of
IOPCF oil spill cost data. Mar. Pollut. Bull. 60, 1455–1466.
Kunreuther, H., Slovic, P., 1996. Science, values and risk. Ann. Am. Acad. Pol.
Soc. Sci. 545, 116–125.
Laheij, G.M.H., Ale, B.J.M., Post, J.G., 2003. Benchmark risk analysis models
used in the Netherlands, in: Safety and Reliability: Proceedings of ESREL
2003. Maastricht, the Netherlands, pp. 993–997.
Lee, H.J., Rhee, K.P., 2001. Development of collision avoidance system by us-
ing expert system and research algorithm, in: Proceedings of International
Shipbuilding Programming Conferencee. pp. 197–212.
Lemos, N., 2007. An introduction to the theory of knowledge, Cambridge In-
troductions to Philosophy. Cambridge University Press.
Leveson, N., 2004. A new accident model for engineering safer systems. Saf.
Sci. 42, 237–270.
Leveson, N.G., 2012. Engineering a safer world: systems thinking applied to
safety, Engineering Systems. The MIT Press, Cambridge, Massachusetts.
Levin, R., 2005. Uncertainty in risk assessment - contents and modes of com-
munication (Doctoral Thesis). Royal Institute of Technology, Sweden.
62
Bibliography
Li, B., Pang, F.-W., 2013. An approach of vessel collision risk assessment based
on the D-S evidence theory. Ocean Eng. 74, 16–21.
Li, S., Meng, Q., Qu, X., 2012. An overview of maritime waterway quantitative
risk assessment models. Risk Anal. 32, 496–512.
Macgill, S.M., Fewtrell, L., Kay, D., 2000. Towards quality assurance of as-
sessed waterborne risks. Water Res. 34, 1050–1056.
Mazaheri, A., Montewka, J., Kujala, P., 2014. Modeling the risk of ship
grounding - a literature review from a risk management perspective. WMU
J. Marit. Aff. 13, 269–297.
Mendel, J.M., 1995. Fuzzy logic systems for engineering: a tutorial. Proc. IEEE
83, 345–375.
Merrick, J.R.W., van Dorp, J.R., Harrald, J., Mazzuchi, T.A., Spahn, J.E.,
Grabowski, M., 2000. A systems approach to managing oil transportation
risk in Prince William Sound. Syst. Eng. 3, 128–142.
Montewka, J., Ehlers, S., Goerlandt, F., Hinz, T., Tabri, K., Kujala, P., 2014a. A
framework for risk assessment for maritime transportation systems—A case
study for open sea collisions involving RoPax vessels. Reliab. Eng. Syst. Saf.
124, 142–157. doi:10.1016/j.ress.2013.11.014
Montewka, J., Goerlandt, F., Kujala, P., 2014b. On a systematic perspective on
risk for formal safety assessment (FSA). Reliab. Eng. Syst. Saf. 127, 77–85.
doi:10.1016/j.ress.2014.03.009
Montewka, J., Krata, P., Goerlandt, F., Mazaheri, A., Kujala, P., 2011. Marine
traffic risk modelling – an innovative approach and a case study. Proc. Inst.
Mech. Eng. Part O J. Risk Reliab. 225, 307–322.
doi:10.1177/1748006X11399988
Moore, J.W., 2009. The logic of definition (No. DRDC Toronto TN 2009-082).
DRDC Toronto.
Mulyadi, Y., Kobayashi, E., Wakabayashi, N., Pitana, T., 2014. Development of
ship sinking frequency model over subsea pipeline for Madura Strait using
AIS data. WMU J. Marit. Aff. 13, 43–59.
NRC, 1998. Review of the Prince William Sound, Alaska, risk assessment
study. Committee on risk assessment and management of marine systems,
Marine Board, National Research Council.
O’Brien, M., 2000. Making better environmental decisions. The MIT Press,
Cambridge.
Oreskes, N., 1998. Evaluation (not validation) of quantitative models. Environ.
Health Perspect. 106, 1453–1460.
Oreskes, N., Shrader-Frechette, Kristin, Belitz, K., 1994. Verification, valida-
tion, and confirmation of numerical models in the Earth Sciences. Science
263, 641–646.
Otway, H., Thomas, K., 1982. Reflections on risk perception and policy. Risk
Anal. 2, 69–82. doi:10.1111/j.1539-6924.1982.tb01368.x
Özbaş, B., 2013. Safety risk analysis of maritime transportation: review of the
literature. Transp. Res. Rec. J. Transp. Res. Board 2326, 32–38.
Papanikolaou, A. (Ed.), 2009. Risk-based ship design: methods, tools and ap-
plications. Springer.
Pedersen, P.T., 1995. Collision and grounding mechanics, in: Proceedings of
the Danish Society of Naval Architects and Marine Engineers. pp. 125–157.
Perrow, C., 1984. Normal accidents: living with high-risk technologies. Basic
Books.
Pitchforth, J., Mengersen, K., 2013. A proposed validation framework for ex-
pert elicited Bayesian Networks. Expert Syst. Appl. 40, 162–167.
Psaraftis, H.N., 2012. Formal Safety Assessment: an updated review. J. Mar.
Sci. Technol. 17, 390–402.
Psaraftis, H.N., 2008. Environmental risk evaluation criteria. WMU J. Marit.
Aff. 7, 409–427.
63
Bibliography
Psarros, G., Skjong, R., Vanem, E., 2011. Risk acceptance criterion for tanker
oil spill risk reduction measures. Mar. Pollut. Bull. 62, 116–127.
Puisa, R., Vassalos, D., 2012. Robust analysis of cost-effectiveness in formal
safety assessment. J. Mar. Sci. Technol. 17, 370–381.
Qureshi, Z., 2007. A review of accident modelling approaches for complex so-
cio-technical systems, in: Proceedings of the Twelfth Australian Workshop
on Safety Critical Systems and Software and Safety-Related Programmable
Systems. pp. 47–59.
Qu, X., Meng, Q., Suyi, L., 2011. Ship collision risk assessment for the Singa-
pore Strait. Accid. Anal. Prev. 43, 2030–2036.
doi:10.1016/j.aap.2011.05.022
Rae, A., Alexander, R., McDermid, J., 2014. Fixing the cracks in the crystal
ball: a maturity model for quantitative risk assessment. Reliab. Eng. Syst.
Saf. 125, 67–81.
Rasmussen, J., 1997. Risk management in a dynamic society: a modelling
problem. Saf. Sci. 27, 183–213.
Rayner, S., 1992. Cultural theory and risk analysis, in: Krimsky, S., Golding, D.
(Eds.), Social Theories of Risk. Praeger, Westport, pp. 83–115.
Reason, J., 1990. The contribution of latent human failures to the breakdown
of complex systems. Philos. Trans. R. Soc. Lond. Ser. B 327, 475–484.
Renn, O., Webler, T., Rakel, H., Dienel, P., Johnson, B., 1993. Public participa-
tion in decision making: A three-step procedure. Policy Sci. 26, 189–214.
Ren, Y., Mou, J., Yan, Q., Zhang, F., 2011. Study on Assessing Dynamic Risk of
Ship Collision, in: Multimodal Approach to Sustained Transportation Sys-
tem Development: Information, Technology, Implementation. Presented at
the International Conference on Transportation Information and Safety,
ASCE, Wuhan, China, pp. 2751–2757. doi:10.1061/41177(415)346
Rosa, E.A., 2010. The logical status of risk - to burnish or to dull. J. Risk Res.
13, 239–253.
Rosa, E.A., 1998. Metatheoretical foundations for post-normal risk. J. Risk
Res. 1, 15–44.
Rosqvist, T., 2010. On the validation of risk analysis - a commentaty. Reliab.
Eng. Syst. Saf. 95, 1261–1265.
Rosqvist, T., 2003. On the use of expert judgment in the qualification of risk
assessment (Doctoral Thesis). Helsinki University of Technology, Espoo,
Finland.
Rosqvist, T., Tuominen, R., 2004. Qualification of Formal Safety Assessment:
an exploratory study. Saf. Sci. 42, 99–120.
Rouhiainen, V., 1992. QUASA: a method for assessing the quality of safety
analysis. Saf. Sci. 15, 155–172.
Rudner, R., 1953. The scientist qua scientist makes value judgments. Philos.
Sci. 20, 1–6.
Saaty, T.L., 1980. The Analytic Hierarchy Process. McGraw-Hill, New York.
Scher, I., Koomey, J.G., 2011. Is accurate forecasting of economic systems pos-
sible? Clim. Change 104, 473–479.
Shrader-Frechette, K.S., 1993. Burying uncertainty: risk and the case against
geological disposal of nuclear waste. University of California Press.
Shrader-Frechette, K.S., 1991. Risk and rationality: philosophical foundations
for populist reforms. University of California Press, Berkely.
Simsir, U., Amasyali, M.F., Bal, M., Çelebi, U.B., Ertugrul, S., 2014. Decision
support system for collision avoidance of vessels. Appl. Soft Comput. 25,
369–378.
Skjong, R., Vanem, E., Endresen, Ø., 2007. Risk evaluation criteria (SAFEDOR
Report No. SAFEDOR-D-4.5.2-2007-10-24-DNV-RiskEvaluationCriteria-
rev-3.0).
Skjong, R., Wentworth, B.H., 2001. Expert judgment and risk perception, in:
Proceedings of the 11th International Offshore and Polar Engineering Con-
64
Bibliography
ference. The International Society of Offshore and Polar Engineers, Sta-

vanger, Norway, pp. 537–544.
Slovic, P., 1999. Trust, emotion, sex, politics, and science: surveying the risk
assessment battlefield. Risk Anal. 19, 689–701.
Soares, C.G., Teixeira, A.P., 2001. Risk assessment in maritime transportation.
Reliab. Eng. Syst. Saf. 74, 299–309.
Solberg, O., Njå, O., 2012. Reflections on the ontological status of risk. J. Risk
Res. 15, 1201–1215.
Sormunen, O.-V.E., Goerlandt, F., Häkkinen, J., Posti, A., Hänninen, M., Mon-
tewka, J., Ståhlberg, K., Kujala, P., 2014. Uncertainty in maritime risk anal-
ysis: Extended case study on chemical tanker collisions. Proc. Inst. Mech.
Eng. Part M J. Eng. Marit. Environ. doi:10.1177/1475090213515640
Statheros, T., Howells, G., McDonald-Maier, K., 2008. Autonomous ship colli-
sion avoidance navigation concepts, technologies and techniques. J. Navig.
61, 129–142.
Stern, P.C., Fineberg, H.V., 1996. Understanding risk: informing risk in a
democratic society. National Academy Press, Washington, D.C.
Stringfellow, M.V., 2010. Accident analysis and hazard analysis for human and
organizational factors (PhD Dissertation). Massachusetts Institute of Tech-
nology, Boston, Massachusetts.
Suokas, J., 1985. On the reliability and validity of safety analysis (Doctoral
Thesis). Tampere University of Technology, Tampere, Finland.
Suokas, J., Kakko, R., 1989. On the problems and future of safety and risk
analysis. J. Hazard. Mater. 21, 105–124.
Suokas, J., Rouhiainen, V., 1989. Quality control in safety and risk analysis. J.
Loss Prev. Process Ind. 2, 67–77.
Swartz, N., 1997. Definitions, dictionaries and meaning [WWW Document].
URL http://www.sfu.ca/~swartz/definitions.htm (accessed 2.20.15).
Szlapczynski, R., 2011. Evolutionary sets of safe ship trajectories: a new ap-
proach to collision avoidance. J. Navig. 64, 169–181.
Tam, C., Bucknall, R., Greig, A., 2009. Review of collision avoidance and path
planning methods for ships in close range encounters. J. Navig. 62, 455–
476.
Terndrup Pedersen, P., 2010. Review and application of ship collision and
grounding analysis procedures. Mar. Struct. 23, 241–262.
Thompson, P.B., Dean, W.R., 1996. Competing conceptions of risk. Risk
Health Saf. Environ. 7, 361–384.
Trbojevic, V.M., 2006. Risk criteria for the shipping industry. Qual. Reliab.
Eng. Int. 22, 31–40.
Trochim, W., Donnely, J.P., 2008. The research methods knowledge base, 3rd
ed. Atomic Dog Publishing.
Tsou, M.-C., Hsueh, C.-K., 2010. The study of ship collision avoidance route
planning by ant colony algorithm. J. Mar. Sci. Technol. 18, 746–756.
Ulusҫu, Ö.S., Özbaş, B., Altiok, T., Ilhan, O., 2009. Risk analysis of the vessel
traffic in the Strait of Istanbul. Risk Anal. 29, 1454–1472.
Van der Helm, R., 2006. Towards a clarification of probability, possibility and
plausibility: how semantics could help futures practice to improve. Fore-
sight 8, 17–27.
Vanem, E., 2012. Ethics and fundamental principles of risk acceptance criteria.
Saf. Sci. 50, 958–967.
Vareman, N., Persson, J., 2010. Why separate risk assessors and risk manag-
ers? Further external values affecting the risk assessor qua risk assessor. J.
Risk Res. 13, 687–700.
Wandall, B., 2004. Values in science and risk assessment. Toxicol. Lett. 152,
265–272.
Wang, J., 2006. Maritime risk assessment and its current status. Qual. Reliab.
Eng. Int. 22, 3–19.
65
Bibliography
Wang, J., 2002. Offshore safety case approach and formal safety assessment of
ships. J. Safety Res. 33, 81–115.
Wang, J., 2001. The current status and future aspects in formal ship safety
assessment. Saf. Sci. 38, 19–30.
Wang, J., Lu, M., Liu, Y., Zhang, H., Zou, W., Cheng, L., 2014. Safety assess-
ment of shipping routes in the South China Sea based on the fuzzy analytic
hierarchy process. Saf. Sci. 62, 46–57.
Wang, J., Sii, H.S., Yang, J.B., Pillay, A., Yu, D., Liu, J., Maistralis, E., Saajedi,
A., 2004. Use of advances in technology for maritime risk assessment. Risk
Anal. 24, 1041–1063.
Wang, N., 2010. An intelligent spatial collision risk based on the quaternion
ship domain. J. Navig. 63, 733–749.
Watson, S.R., 1994. On the meaning of probability in probabilistic safety anal-
ysis. Reliab. Eng. Syst. Saf. 45, 261–269.
Weinberg, A.M., 1981. Reflections on risk assessment. Risk Anal. 1, 5–7.
Weng, J., Meng, Q., Qu, X., 2012. Vessel Collision Frequency Estimation in the
Singapore Strait. J. Navig. 65, 207–221.
Wittgenstein, L., 1953. Philosophical investigations. Basil Blackwell, Oxford.
Wolff, J., 2006. Risk, fear, blame, shame and the regulation of public safety.
Econ. Philos. 22, 409–427.
Xiao, F., 2014. Ships in an artificial force field: a multi-agent system for Nauti-
cal Traffic and Safety (Doctoral Thesis). Delft University of Technology,
Delft, NL.
Xu, Q., Zhang, C., Wang, N., 2014. Multiobjective Optimization Based Vessel
Collision Avoidance Strategy Optimization. Math. Probl. Eng. 2014, 9.
doi:http://dx.doi.org/10.1155/2014/914689
Yamada, Y., 2009. The cost of oil spills from tankers in relation to weight of
spilled oil. Mar. Technol. 46, 219–228.
Yang, Z.L., Wang, J., Li, K.X., 2013. Maritime safety analysis in retrospect.
Marit. Policy Manag. 40, 261–277. doi:10.1080/03088839.2013.782952
Zhang, J., Yan, X., Chen, C., Sang, L., Zhang, D., 2012. A novel approach for
assistance with anti-collision decision making based on the International
Regulations for Preventing Collisions at Sea. Proc. Inst. Mech. Eng. Part M
J. Eng. Marit. Environ. 226, 250–259.
Zio, E., Apostolakis, G.E., 1996. Two methods for the structured assessment of
model uncertainty by experts in performance assessments of radioactive
waste repositories. Reliab. Eng. Syst. Saf. 54, 225–241.
66
Errata
Publication I
On p. 131, Section A.3., first paragraph: “Finally, the collision probability ܲ௫ ሺ‫ܣ‬ሻ
is mathematically derived from the fitted distribution ݂ሺሺ‫ܣ‬Ͷሻሻ [...]” should
read “Finally, the collision probability ܲ௫ ሺ‫ܣ‬ሻ is mathematically derived from
ᇱ ሻ […]”.
the fitted distribution ݂ሺ‫ܥ‬௠௔௫
Publication IV
On p. 187, Section 4.3.1., fourth paragraph: “As the measurement procedure

does not necessarily result in the same calculated risk level for each vessel,
maximum risk is adopted level for both vessels […]” should read “As the meas-
urement procedure does not necessarily result in the same calculated risk level
for each vessel, the maximum risk level is adopted for both vessels […]”.
On p. 188, Section 4.3.2., second paragraph: “The elements of ‫ ݓ‬reflect relative

importance […]” should read “The elements of ‫ ݓ‬reflect the relative im-
portance […]”.
67
Risk analyses are widely used tools for

Aalto-DD 107/2015
Floris Goerlandt
decision support. Nonetheless, the risk
analysis discipline has received much
criticism. Calls have been made for
increased focus on foundational issues, both transportation: principles,
in the general discipline and in application
areas. frameworks and evaluation

Answering these calls, this thesis
investigates and proposes a number of risk
analysis principles, addressing concepts and
terminology, risk model reliability, risk Floris Goerlandt
models and prediction, risk model use and
the consideration of uncertainty and bias.
The principles are used as a basis for
developing two risk analysis frameworks:
one for policy-oriented and one for
operational maritime risk analysis. Finally,
the evaluation of risk models and analyses is
addressed, focusing on the issue of
credibility of a risk analysis.
Principles, frameworks and evaluation are

investigated and exemplified through
applications concerning accidental risk of
maritime transportation.
ISBN 978-952-60-6313-3 (printed) BUSINESS +

9HSTFMG*agdbdd+
ISBN 978-952-60-6314-0 (pdf) ECONOMY

ISSN-L 1799-4934
ISSN 1799-4934 (printed) ART +
ISSN 1799-4942 (pdf) DESIGN +
ARCHITECTURE
Aalto University
Aalto University
School of Engineering SCIENCE +
Department of Applied Mechanics TECHNOLOGY
www.aalto.fi
CROSSOVER
DOCTORAL
DOCTORAL DISSERTATIONS
DISSERTATIONS

Risk Analysis in Maritime Transportation: Principles, Frameworks and Evaluation

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Risk Analysis in Maritime Transportation: Principles, Frameworks and Evaluation

Uploaded by

Copyright:

Available Formats

Department of Applied Mechanics

Risk analyses are widely used tools for

Risk analysis in maritime transportation: principles, frameworks and evaluation

Principles, frameworks and evaluation are

ISBN 978-952-60-6313-3 (printed) BUSINESS +

ISBN 978-952-60-6314-0 (pdf) ECONOMY

Risk analysis in maritime

A doctoral dissertation completed for the degree of Doctor of

Dr. Jakub Montewka

Aalto University publication series

ISBN 978-952-60-6313-3 (printed)

Espoo, Wednesday, 20 May 2015

List of Publications .................................................................................. 5

This doctoral dissertation consists of a summary and of the following publica-

PI. Goerlandt, Floris; Montewka, Jakub. 2015. Maritime transportation risk

PIV. Goerlandt, Floris; Montewka, Jakub; Kuzmin, Vladimir; Kujala, Pentti.

Publication I: Maritime transportation risk analysis: review and analysis in

Publication III: A framework for risk analysis of maritime transportation

Publication IV: A risk-informed ship collision alert system: framework and

1. A detailed analysis is conducted of adopted risk definitions, perspec-

2. The reliability of policy-oriented and operational maritime transporta-

3. A two-stage policy-oriented risk analysis framework for maritime

4. A two-stage risk analysis framework for ship collision alert systems in

6. A Fuzzy Expert System based model is developed for providing risk-

7. Principles and methods for evaluating policy-oriented and operational

maritime transportation an activity in which ships transport goods from

operational risk analysis a risk analysis which is continuously performed

policy-oriented risk analysis a risk analysis which is performed at a given

principle a fundamental proposition that serves as a foundation for a system

risk analysis evaluation a process of building confidence in a given risk

risk analysis framework a basic structure, a set of ideas, mechanisms and

risk concept the ‘general idea’ of what risk is as a constituent of thought

risk definition the result of an intellectual activity to delineate a meaning of

risk perspective a way to describe risk, a systematic manner to analyse and

reliability the extent to which a measurement procedure leads to the same

scientific approach to Risk Analysis a set of principles adhered to as a

validity the extent to which a measurement procedure adequately describes

BSQ background situational quality

CAS collision alert system

COLREGS international regulations for preventing collisions at sea

DCPA distance at closest point of approach

FES fuzzy expert system

FSA formal safety assessment

FSQ foreground situational quality

QRA quantitative risk analysis

TCPA time to closest point of approach

VTS vessel traffic service

1.1 Motivation: Risk Analysis as an unsettled scientific discipline

Table 1. Summary of some main criticisms concerning risk analysis

In response to these criticisms, there has been and is ongoing research to

1.2 Objectives and structure of the thesis

The overall objective of this thesis is to contribute to some foundational issues

Objective 1: Investigate which definitions, perspectives and scientific ap-

Objective 1 primarily aims to identify a need to focus on the underlying prin-

2.1 Scientific approaches to Risk Analysis

2.1.1 Realist approaches

Risk realists typically consider risk as a physically given attribute of a system,

making. In regulatory and public decision making it shows us how, quantitative-

This reification of risk has several implications. First, understanding risk as a

2.1.2 Constructivist approaches

It is clear that truths do not exist independently of people, whether taken to be

Another constructivist view is expressed by Aven and Renn, defining risk as