Cogn Tech Work (2015) 17:249–267
DOI 10.1007/s10111-014-0294-y
ORIGINAL ARTICLE
A practical approach to assess risk in aviation domains for safety
management systems
P. C. Cacciabue • M. Cassani • V. Licata •
I. Oddone • A. Ottomaniello
Received: 11 January 2014 / Accepted: 14 September 2014 / Published online: 2 October 2014
 Springer-Verlag London 2014
Abstract The introduction of a safety management sys-
tem (SMS) in the real operational environment has become
a key factor in a proactive view of flight safety. The deep
cultural change and effort that the application of an SMS
requires, at economical and organisational level, has led to
a delay in the definition of shared mandatory guidelines by
the international civil aviation authorities. At present, the
majority of attempts to develop an SMS are the result of the
efforts of single organisations and operators. This paper
describes a methodology aimed at practical and straight-
forward implementations of risk assessment processes and
able to tackle real problems. The application process might
be used as guideline for the analysis of critical activities
resulting from retrospective and prospective assessments of
operational environments. The results obtained for a sam-
ple case, based on a real ‘‘change situation’’ occurring in a
medium size flight operator, demonstrate how the process
is carried out and how to highlight and deal with the
underlining critical threats and hazards that require
immediate action in order to guarantee acceptable safe
operational levels. These include the definition of correc-
tive actions and the identification of responsible persons to
monitor the activities designated to mitigate the associated
risk.
A. Ottomaniello is currently working at SW Italia, Safety Department
P. C. Cacciabue (&)
Dip. di Ing. Aerospaziale, Politecnico Milano, Milan, Italy
e-mail: pietrocarlo.cacciabue@polimi.it
M. Cassani
V. Licata
KITE Solutions, SRL, Laveno Mombello, VA, Italy
I. Oddone
A. Ottomaniello
Safety Department, Air Dolomiti, Dossobuono di Villafranca,
Verona, Italy
Keywords Aviation safety
Safety management system

Risk analysis
Management of change
1 Introduction
National and international authorities responsible for imple-
menting an orderly development of civil aviation moved from
a traditional attitude towards safety, based essentially on
compliance with legal requisites, to a proactive approach,
which identifies an elaborate and comprehensive structure,
defined ‘‘safety management system’’ (SMS), as the foun-
dation of a new contemporary approach, based on proactive
and constantly evolving attitudes of the organisations
involved. In particular, the International Civil Aviation
Organization (ICAO), the Federal Aviation Administration
(FAA), the European Aviation Safety Agency (EASA) and
the European Commission (EC) included SMS in their stra-
tegic plans (e.g., ACRP 2012; IATA 2010; FAA 2010; ICAO
2012; EASA 2011, 2012). The implementation of a SMS and
associated manual of implementation are gradually becoming
mandatory requirements for all Organisations involved in
commercial aviation in order to obtain or maintain their
commercial activities.
In essence, the SMS is a fully comprehensive and
integrated approach towards safety based on the assessment
of risks and the concept of management. It aims at applying
three basic steps of prevention, control and containment of
consequences derived from hazardous events, non-com-
pliance occurrences and incidents that may occur in the life
of a system and during production processes. All types of
technical, commercial and financial operations of an
Organisation are involved and affected by a SMS devel-
opment process. The success of a SMS entails primarily the
support of the top management which must be constantly
123
250
fed through to the front line by key figures of the organi-
sation (Stolzer et al. 2010; Liou et al. 2008).
The SMS has been widely utilised as technique to assess
safety (Byrom 1994) and comply with regulatory require-
ments (Kirchsteiger et al. 1998) in many technologically
advanced domains, such as energy production, oil and gas
industry and process systems. The methodologies and
techniques for hazard assessment and risk management are
widely established and are commonly applied (Hudson
2000). It is probably for this reason that, in the commercial
aviation environment, the manuals promoting SMS at
institutional level (e.g., ICAO 2012) do not discuss specific
techniques to be put in place for effectively implementing
risk assessment and evaluation processes. These guide-
books are primarily aimed at managers, rather than tech-
nical experts, as they focus on detailed discussions about
motivation and benefit derived from implementing accurate
and extensive safety analysis and prevention measures.
In these manuals and guidebooks, the specific techniques
to be used to assess levels of risks are neither presented nor
discussed, under the assumption that they are part of the
basic academic and practical training of safety analysts.
However, this is not always the case and in many circum-
stances the crucial area of the evaluation of probabilities or
frequencies of occurrence of events and hazards and the
process of estimation of consequences of occurrences are
insufficiently dealt with. The documentation about these
techniques and tools exists and is equally well established
(UK-CAA 2010; NASA 2011), but it represents an enor-
mous amount of knowledge and information that need to be
adequately filtered for implementation into SMS processes.
The recent request of authorities for the implementation
of SMS in the commercial aviation justifies the several
revamped initiatives aimed at further training safety analysts
to the in-depth study of methods of risk analysis. Moreover,
there is a clear need for the formulation of methodological
stepwise approaches, dedicated specifically to aviation,
which enables to combine the process of implementation of
risk analysis with the selection of the most appropriate
techniques, to be applied on a case-by-case base.
Indeed, the obligation of implementing SMS and risk
based studies has led to the request by the organisations for
tools and instruments that enable a rapid and as-easy-as-
possible implementation of a response to the authority
requirement. This generates the danger of choosing the
‘‘painless’’ way out offered by techniques that only appar-
ently seem to solve the problem of assessing risk in a simple
and straightforward manner. Unfortunately, the assessment
of risk requires the consideration of a number of boundary
and complementary conditions, such as the evaluation and
propagation of uncertainties and the availability of historical
data, which are extremely sensitive and can invalidate the
whole study when shallow and too superficial methods are
123
Cogn Tech Work (2015) 17:249–267
applied. On the contrary, by employing consolidated meth-
ods, based on good knowledge of data and in-depth literature
reviews, the overall process of risk assessment becomes
more complex from the numerical implementation per-
spective but certainly more acceptable from the statistical
perspective. In this way, the implementation of adequate
methodologies leads to the generation of reliable results that
enable a knowledgeable and consistent risk informed deci-
sion making (RIDM) process about safety (Tamasi and
Demichela 2011; Schindler and Cassani 2013).
The objective of this paper is to propose a methodo-
logical framework that offers Safety Managers a relatively
simple and practical stepwise approach for implementing
risk assessment and risk evaluation, that avoids a specu-
lative and opportunistic use of data and information and
suggests the implementation of standard techniques to
reduce the uncertainty, enabling to frame the overall
approach in a more consolidated safety perspective. The
proposed methodology is called Risk Assessment Meth-
odology for Company Operational Processes (RAMCOP).
This paper firstly examines the general features of risk
assessment methodologies and discusses the major key
features and limitations of some recently proposed and
recognised tools especially from the perspective of data
and uncertainty assessment. Then the flow chart of the
RAMCOP approach and the stepwise implementation
procedure are presented in detail. A specific practical
application shows how the methodology may be imple-
mented in response to specific need of an organisation and
in accordance to authority requirements. Finally, ways
forward and possible implementation of the tool to other
transportation domains and technologies are discussed in
the conclusive section of the paper.
2 Key features of a risk assessment tool
2.1 Authority requirements
The latest updates to the technical annexes and regulations
adopted by aviation national and international bodies
require that all associated service providers develop a SMS.
EASA has developed a Safety Plan (EASA 2012) con-
taining very important guidelines for harmonising the
European norms and standards related to air transport
safety and aimed at a continuation of the work already done
by ICAO. This Safety Plan describes a standard manage-
ment mode, at a European level, supporting the imple-
mentation of safety programmes at national level, and
identifies three main thematic areas, namely:
1. Reporting issues that concern an aviation system as a
whole, often associated to the root cause and
Cogn Tech Work (2015) 17:249–267
retrospective analysis of sequences of events, actually
occurred and reported to the authorities.
2. Operational issues and problems linked directly to
flight operations.
3. Emerging issues that include all aspects related to
novel operations, changes or new regulations for which
there are still no data to be parsed.
The first thematic area concerns essentially the data
collection, storage and analysis of real occurrences leading
to retrospective assessment of the safety state of an orga-
nisation. The second and third thematic areas require the
performance of prospective analyses for the evaluation of
the risk associated to every-day and new operational pro-
cesses and their acceptability. Moreover, with reference to
the operational issues, all EASA member states have to
focus on five major safety related problems encountered in:
• runway excursion,
• mid-air collision,
• controlled flight into terrain,
• loss of control in flight and
• ground collision.
The definition of these thematic areas and major issues,
identified by EASA, are therefore very useful for the def-
inition of the agenda of a valuable and acceptable SMS at a
national level, even if no specifications about means of
compliance are identified (Ayres et al. 2013).
2.2 SMS as risk informed decision making process
2.2.1 Role of the safety manager in the decision making
process
Even if the IACO requirements for a SMS (ICAO 2012) do
not detail explicitly the need to carry out a RIDM process,
the way in which ICAO proposes to the aviation commu-
nity how to carry out safety assessment and preservation
reflects the guidelines proposed originally by US-NRC in
1995 and consolidated in the following years in a series of
publications about RIDM (NRC 1995, 1998, 2002, 2003,
2009). The recently published handbook on RIDM by
NASA (2010) and the various reports published by the
International Atomic Agency (IAEA 2005, 2011) represent
further reference documentation for handling the way in
which risk analysis methods can be implemented in order
to offer decision makers a structured and orderly amount of
information to enable adequately informed management
choices.
In this realm, the role of the Safety Manager is partic-
ularly delicate and important as he/she has to interact with
different actors of the SMS. In particular, within the
organisation the safety manager operates at three levels:
251
1. The Safety Manager, as leader of the safety team and
major developer of the SMS content and documenta-
tion, should have sufficient knowledge and expertise in
risk analysis to guide the selection of the appropriate
means of implementation to carry out the technical
steps of a SMS.
2. The knowledge of aviation practices is equally impor-
tant for the safety manager in order to enable the
interaction with the primary actors of the domain, e.g.,
pilots, maintenance operators, ground staff, air traffic
controllers etc., who are the providers of crucial field
data and reporting that are the fundamental sources of
information about safety.
3. At a more managerial level, the Civil Aviation
Authorities consider the Chief Executive Officer
(CEO) of an organisation as the decision maker
responsible for the implementation of SMS. Usually,
in this activity, the CEO is supported by the Safety
Manger and safety team. Consequently, the Safety
Manager should be able to provide the CEO with the
appropriate and synthetized information enabling sus-
tainable and consolidated decisions.
Therefore, the Safety Manager, while in charge of pro-
viding the CEO with the necessary information to make the
choices in terms of safety, should equally be able to carry
out technical assessment of risk and acquire trust and
acceptance from the field data providers. This imposes on
the figure of the safety manager a very large competence
based on appropriate knowledge and leadership at all levels
of the organisation.
In particular, on the RIDM process, which consists in the
essential and final step of the SMS implementation, the
problem of managing uncertainty is very relevant in con-
junction with the question about data and with the moral
issue of what is the best criterion to set a limit on safety
levels. The latter essentially concentrate on the criteria uti-
lised by the organisation to aim at certain safety perfor-
mances. In general, there are a variety of principles that can
be used by the decision maker (Ersdal and Aven 2008).
Amongst them, the most commonly applied methods and
criteria are: Cost-Benefit Analysis, multivariate analysis and
the As Low As Reasonably Practicable (ALARP) principle.
These criteria are applied once the problem of handling
the uncertainty associated with information are resolved
and accounted for. These issues are briefly discussed in the
following section.
2.2.2 Assessment of uncertainty and field data from real
occurrence reporting
The assessment of uncertainty and the availability of
sound and consolidated bodies of data are crucial
123
252
aspects for any risk assessment process. These two
elements are correlated and have a strong impact on the
possibility to provide the decision makers with consol-
idated and relatively sound information on which to
formulate the final decision about the implementation of
new barriers or to continue operating as usual or to stop
operations and reconsider the whole system from a
safety perspective.
In particular, the treatment of the uncertainties associ-
ated to the formulation of the risk assessment, also called
epistemic uncertainties, the performance of sensitivity
analysis to prioritise the results and the process of propa-
gation of the uncertainties throughout the overall risk
assessment process are essential contributor to the gener-
ation of a consolidated body of information.
Strictly correlated to the problem of uncertainty is the
issue of real data collected and analysed within the orga-
nisation. The availability of a rich and reliable database of
reports, coupled with associated analyses and statistical
study of the root causes of occurrences, enables the
reduction of epistemic uncertainties and therefore
strengthens the results of a risk assessment.
A key step in reducing the uncertainties is therefore the
availability of a data collection system about real occur-
rences reported by the primary actors within the organi-
sation that enables the safety team to derive a variety of
data for risk assessment from the analysis of such reports.
This process of data collection and analysis is well iden-
tified in the regulations that refer to SMS. It requires a
comprehensive approach of retrospective analysis that
enables to identify root causes of events. Moreover, a
shared taxonomy is needed between retrospective and
prospective methods for the definition of data and for the
assessment of hazards and consequences. The problem of
consolidated and integrated taxonomies and data collec-
tion/analysis has been discussed for many years with well
documented results in the literature especially in relation to
human contribution to incidents (e.g., ICAO 1987, 1993,
1997; Siu 1994; Hollnagel 1998). More recently, methods
have been proposed that tackle these issues in a more
dynamic and time dependent perspective, implementing
techniques such as Event Time Line building, Event
Sequence Diagrams and Dynamic Event Tree Analysis
(e.g., Mosleh et al. 2004; Cacciabue 2004; Roelen and
Wever 2005).
2.3 Living characteristics of SMS
In general, the operational aspects tackled by a SMS
comprise the performance of prospective and retrospective
studies, the routine implementation of safety audit and the
definition of how to manage emergencies and security
issues.
123
Cogn Tech Work (2015) 17:249–267
The prospective and retrospective methods must be
integrated and correlated: information collected from the
analysis of real occurrences (retrospective study) enable
the exploitation of data transfer to the benefit of prospec-
tive studies performed about new situations, requiring
appropriate assessment, before engaging in novel activities.
The osmosis between the two types of analysis is repre-
sented the Risk Matrix (RM) which offers an immediate,
synthetic and comprehensive vision of the level of safety in
which the organization is located.
The process of safety audit has the goal of continually
assessing and ensuring the conformity of the safety levels
and measures implemented within an organisation with
respect to the norms and standards defined by the national
and international authorities. The audit processes are the
most obvious link between quality and safety bodies of an
organisation. Their implementation requires the use of
techniques that go beyond the scope of this paper and will
not be further discussed.
Emergency management and security assessment are
very specific chapters of the SMS, which focus on man-
aging the ‘‘after-incident’’ and events of intentional mali-
cious actions. The methodologies and techniques for
emergency management and security assessment involve
media communication and interaction with general public
and with family members of persons involved in an acci-
dent, sharing tasks and responsibilities within an organi-
zation, specific training of personnel, etc. As for the case of
the safety audit, these methods will not be discussed
hereafter, as they go beyond the scope of this paper.
However, it is important to note that also in the case of
emergency and security management, it is essential to
adapt and develop new approaches in order to cope with
continuously changing situations and regulations.
This short review of the basic constituents of an SMS
shows the need to implement a ‘‘living’’ process that is able
to generate a constantly adjourned and updated image of
the safety state of an organisation, correlated to its opera-
tional life. From the risk analysis point of view, this is a
well-established approach that is operative in domains such
as nuclear power plant and energy production in general
(Ranlöf et al. 2012).
2.4 Agility and user friendliness
In addition to respecting authority requirements, basic rules
of RIDM processes and being a ‘‘living’’ appraisal of safety
state of an organisation, a ‘‘tool’’ that supports safety
assessment must show two fundamental aspects: agility and
user friendliness.
Agility is necessary as the instrument must be applied to
many different studies of prospective and retrospective
nature, with many different specific aspects, due to the
Cogn Tech Work (2015) 17:249–267
need to respond to the requirements of the authorities, as
well as to the actual processes implemented or planned by
the organisation. Moreover, a variety of questions face the
analysts for the definition of the overall safety state and,
consequently, the instrument must be able to tackle dif-
ferent issues and different perspectives with simple alter-
native approaches.
The ability to rapidly adapt to different types of analyses
and perspectives, i.e., the agility of the tool, can be ade-
quately exploited only if the user can simply access the
means and ways for performing the intended assessments.
In other words, the interface with the user should be as easy
and friendly as possible, so as not to create an obstacle to
the selection and implementation of the best means of
compliance for assessing risk levels.
2.5 General considerations
The analyses of safety levels and responses of installations
and aeronautical structures require the ability for an orga-
nisation to prevent, recover and contain the consequences
arising from potential dangers caused by different sources,
such as random events, technical failures or human errors.
The outcome of these analyses enable the definition of the
states of safety of an organisation. Such ‘‘states’’, measured
in terms of likelihood and severity of occurrences, must be
within values identified and accepted by the authority,
which are framed in a RM.
Analytically, the probabilistic risk analysis represents a
perspective estimate of possible evolutionary processes, or
‘‘occurrences’’, that develop following an initiating event
or an unsafe operational state or danger. This involves the
intervention of safety systems and/or protective barriers,
established by the organisation with the aim to recover
normality or to limit the consequences. Other important
barriers aim at minimising the probability of occurrence of
the initiating events.
In order to assess the risk associated with a certain ini-
tiating event, it is therefore necessary to adopt a procedure
for the evaluation of: (a) the consequences, through the
analysis of the barriers that come into play during the
evolution of potential occurrences associated with specific
initiating events; and (b) the probabilities of the occur-
rences, associated with the success or failure of each of the
barriers. This process involves the combination of a qual-
itative analysis of possible evolutions from an initiating
event or danger, and a quantitative assessment of the
likelihood of occurrence and severity of the consequences,
in terms of damage to people, environment and systems or
facilities. The results of these studies are organised in
accordance to the principles of RIDM and in consideration
for the problems of real data and uncertainties previously
discussed.
253
3 Methodologies utilised in the proposed approach
3.1 Reference methodologies
Several tools or methods exist for prospective risk evalu-
ation. These are combined and correlated by means of
appropriate methodologies to assess quantitatively the
probability of occurrence and the severity of the conse-
quences for the evaluation of the acceptability of the
associated risk within the RM.
In the domain of aviation, two well-known and estab-
lished methodologies are usually proposed for the imple-
mentation of SMS: Bow-Tie and ARMS. Bow-Tie has
become, since several decades, a very popular, structured
and consolidated approach for assessing risk (Fig. 1, left).
The success of the Bow-Tie diagram is that it is easy to
understand, even for non-specialists, and enables a rapid
qualitative risk assessment. The idea is simple: to combine
causes and consequences associated to an initiating event
or dangerous situation. The origin of the Methodology
Bow-Tie is not entirely known, but it can be assumed that it
is an evolution of the Cause-Consequences diagrams
(Nielsen 1971). The process of application of the meth-
odology involves the systematic identification of major
threats, the assessment of associated hazards and the defi-
nition of specific control measures and recovery means that
need to be implemented and maintained to reduce and
control risks. The process is iterative and Bow-Tie is often
developed by a team of experts and safety analysts.
Aviation Risk Management Solutions (ARMS) is a
much more recent methodology and results from the work
of a working group of experts in aeronautical, industrial
and academic domain, established with the primary goal to
develop an approach applicable to all aviation organiza-
tions (ARMS 2011). The methodology tackles both types
of retrospective and prospective operational risk assess-
ment. The ‘‘Event Risk Classification’’ (ERC) method
implies the analysis of the events that actually happened
within an organisation. Whereas, the ‘‘Safety Issue Risk
Assessment’’ (SIRA) generates, in a perspective mode, a
log for the continuous assessment of risks and control
actions, through safety barriers, providing a means for
safety supervision (Fig. 1, right).
3.1.1 Advantages and limitations of Bow-Tie and ARMS
The methodology ARMS and Bow-Tie are very similar,
with some minor but significant differences. Firstly, the
methodology ARMS, in contrast to the Bow-Tie, implies
the specific choice of Expert Judgement (EJ) as means for
the assessment of the probabilities associated with barriers.
In addition, to make ARMS easier and quick, this judgment
is limited to the group of safety analysts applying the
123
254
Fig. 1 The Bow-Tie and ARMS methodologies
method. Similarly, EJ is used for the estimation of the
severity.
The advantages and simplifications introduced by the
ARMS methodology lead to the possibility for the user to
rapidly determine the value of risks associated to hazards
that need to be assessed. These aspects coincide also with
the most important drawbacks of the methodology. In
particular, two critical issues have to be handled extremely
carefully, namely:
1. Firstly, the extensive use of the judgement of the safety
analyst, and possibly safety team, is exposed to the
threat of transforming the implementation of ARMS in
a simple exercise of combination of probabilities and
severities aimed at satisfying the need of the organi-
sation to ‘‘fit’’ within the acceptable areas of the
adopted RM.
2. Secondly, the lack of explicit consideration for the
treatment of uncertainties and for the sensitivity
analysis may result in the evaluation of risks that are
logical but not substantially credible. In other words,
avoiding the treatment of epistemological aspects of
the data utilised invalidates the predictive nature of the
risk estimates and may represent a bias, rather than a
support, to the decision making process.
These two critical issues make the straightforward
implementation of ARMS particular problematic and
require a well thought implementation. This is particularly
true in relation to the application of ARMS predictive
method SIRA than can lead to too simplistic and unreliable
results. Alternatively, a more formalised process can be
developed that guides the safety manager and team in the
evaluation of hazards and risks in such a way to avoid the
traps discussed above.
3.1.2 Goals of the proposed methodology
The methodology proposed in this paper is essentially
based on the Bow-Tie/ARMS approach, but it offers the
user to implement more accurate methods for evaluating
the probabilities of certain events, when the simple
123
Cogn Tech Work (2015) 17:249–267
judgements of the Safety Manager or safety team are
deemed insufficient.
The first issue to tackle in order to consolidate the risk
assessment approach is to implement a formalised method
for EJ. The extensive use of people with recognised
experience and proven knowledge of phenomena and
processes is the best way to proceed. These experts are
called to express their assessment in relation to precise
questions, either in terms of possible threats and hazards
associated to certain operations, or in terms probability of
occurrence of some event or seriousness of the conse-
quences. The transformation of such judgements in quan-
titatively meaningful values for the purposes of risk
assessment requires the application of sophisticated statis-
tical analysis tools which enable to account also for the
uncertainties of different nature.
Another way to tackle the above mentioned critical
issues is to implement ‘‘classical’’ methods for assessing
probabilities and severity of certain sequences deemed
particularly relevant. These methods will not be discussed
here as they widely described in many reference books and
publications on risk analysis. The use of consolidated
methods is particularly relevant when aspects associated to
human reliability and human–machine interactions are
concerned. Many different approaches are available in the
literature for the assessment of the likelihood of success or
failure of a procedure. In particular, in the proposed
methodology two well-known and broadly applied methods
are utilised, namely: Tecnica Empirica Stima Errori Ope-
ratori (TESEO) for the assessment of the likely failure of a
procedure through the judgement of experts and a pre-
established formulation policy of success or failure (Bello
and Colombari 1980); and Technique for Human Error
Rate Prediction (THERP) for structured assessment of
specific procedures (Swain and Guttmann 1983).
The issue associated to the management of uncertainties
can only be resolved by recognising its relevance and
implementing a set of procedures and processes for
ensuring that the uncertainties are adequately assessed and
then they are ‘‘propagated’’ affecting the final results. In
other words, the epistemic uncertainties associated to the
Cogn Tech Work (2015) 17:249–267
various aspect of data handling are accounted for and
propagated thorough the risk assessment process so as to
offer the decision maker a more valid vision of safety.
The process of implementing the proposed methodology
will be described in detail in the next section. In the fol-
lowing section, some of the methods implemented for
improving the evaluation of probabilities of events and the
overall RM applied in an SMS of an organisation are dis-
cussed. Particular reference is made to the RM utilised in
the case study discussed in the last section of this paper.
3.1.3 Assessing and categorising probabilities of events
and severity of occurrences
3.1.3.1 Evaluation of probabilities and severity of occur-
rences In order to organise the probabilities in discrete
levels, it must be considered that the norms and standards,
discussed earlier, do not define a precise scale of proba-
bilities to be implemented in a RM. The proposed meth-
odology has selected a scale based on seven levels of
probability values. This discretisation is rather detailed and
makes reference to a set of frequencies derived from
practical observations and reports of events for large
operators in the civil aviation domain: frequent/very high
probability (P5), likely/high probability (P4), possible/
medium probability (P3), possible under certain circum-
stances/low probability (P2), unlikely/very low probability
(P1), remote/rare probability (P0); and extremely remote/
extremely rare probability (Pe). The highest level of
probability (P5) corresponds to events that occur several
times during daily operations, with an average probability
of occurrence of 7.3 9 10-3 and a frequency of about
1/140 flights, whereas the level of lowest probability (Pe)
corresponds to events never occurred in past history of
aviation, which is associated to a probability of occurrence
of 2.0 9 10-8 and a frequency of about 1/(50 9 106)
flights (Air Dolomiti 2011).
In order to identify the severity of an occurrence, the
usual approach based on EJ is utilised. In particular, the
proposed methodology offers a variety of natures of dam-
age to be accounted for defining how severe a certain
occurrence may be. The reason for offering this variety of
severities is to enable the safety assessment to be really
proactive and to consider the seriousness of certain situa-
tions, attitudes, decisions or facts even before a tangible
and measurable damage occurs.
The methodology enables to consider 11 different types
of severity from the standard one, i.e., ‘‘injury’’ to persons,
to ‘‘equipment damage’’, to other potentially serious con-
sequences of less tangible nature, but, possibly, of even
greater impact, such as for example ‘‘process breach’’, or
‘‘safety awareness ignorance’’ by managers (Air Dolomiti
2011).
255
In order to organise the severity, selected for each case
study, a number of discrete levels of severity need to be
assigned. The proposed methodology considers 6 levels of
severity, corresponding to: None, Minor, Low, Medium,
High, and Extreme.
Table 1 shows the complete matrix of combination of
levels versus nature of severity to be considered for risk
analysis.
3.1.3.2 Reference Risk Matrix Combining the severity
and probability/frequency scales, the resulting Risk Matrix
applied for safety studies is based on a 7 9 6 areas of
different acceptance (Fig. 2).
This RM is extremely important both from the qualita-
tive and quantitative perspective. The relevant qualitative
aspects are that the subdivision of the risk levels in a 7 9 6
matrix offers the user a more complete variety of levels of
risk that can consequently be assessed in a more accurate
way. Than the classical 5 9 5 matrix.
Moreover, the possibility to consider 11 different types
of severity measures enables the user to carry out an
extremely more articulated evaluation of safety states than
the classical measure of damage to define severity. In
particular, the use of these different estimates of severity
covers the entire organisation structure from the highest
level of management to the front line actors and the
material and immaterial measures of safety. This may lead
to very accurate analysis and in depth assessment of the
levels of safety at just culture and practical levels.
Focusing on the actual RM (Fig. 2), it is possible to
note:
• areas with values ‘‘A’’ correspond to unacceptable risk
and require immediate mitigation actions otherwise
operations have to be stopped;
• areas with values ‘‘B’’ are of high risk and require short
term improvement;
• areas with values ‘‘C’’ are considered of acceptable risk
which must, anyway, be accompanied by long term
improvement;
• areas with values ‘‘D’’ are of low risk and require
simple monitoring by the safety team; and finally
• areas with values ‘‘E’’ are of negligible severity and
acceptable in terms risk.
Finally, it is important to note that the RM shown in
Fig. 2 represents a reference matrix based on real data of
flight frequency for the organisation under study. There-
fore, the results obtained in risk studies, such as the one
shown in the final section of this paper have an intrinsic
validity and meaning. Obviously, this matrix when applied
to other organisations with different characteristics, such as
flight type, number and frequency, need to be adapted
accordingly.
123
 256

Table 1 Nature and levels severity utilised by the methodology

123
Severity S5 S4 S3 S2 S1 S0
nature Extreme High Medium Low Minor None

ICAO Accident Serious incident Occurrence with minor injuries Occurrence with discomfort None None
incident and minor damage to aircraft
definition
Injury Multiple fatalities and/or Fatalities and/or permanent Serious but non-permanent injuries Injuries requiring medical No or minor injuries (first aid None
permanent disabilities with disability with serious illness or (e.g., loss time injury) first aid treatment only treatment)
serious illness or impairments impairments
Property or [20 Mio EUR 400.000 EUR to 20 Mio. EUR 10.000 EUR to 400.000 EUR 300 EUR to 10.000 EUR \300 EUR None
A/C
damage
cost
Reputation Fundamental change in the Extended national/international Short-term nation-wide negative Negative local media None None
and public public perception of quality negative media coverage media coverage coverage
confidence airline
Customer Extensive shut down of More than 40 flights cancelled, Between 1 and 40 flights Between 2 and 5 flights One flight rescheduled or None
impact services for an extended rescheduled or delayed. cancelled, rescheduled or rescheduled or delayed. delayed. Small number of
period. All customers Thousands of customers delayed. Hundreds of customers Many customers affected discomforts
affected affected affected
Operational Fleet grounding for extended Brief fleet grounding up to 2 days Aircraft grounding more than Aircraft grounding 4–48 h Aircraft delay \4 h None
impact period 2 days
Equipment Loss of critical equipment, Major damage, results in major Minor damage, leads to Minor damage, potential No adverse consequences None
shutdown of organization slowdown and/or downtime organizational slowdown and/or slowdown and/or
minor downtime downtime
Compliance Significant disruption to Substantial fine and disruption to Substantial fine but no disruption No fine and no disruption to Minor breaches by individual None
scheduled services over an scheduled services to scheduled services scheduled services staff members
extended period of time
Process Several steps of flight critical No steps of documented process Majority of steps of documented Contiguous steps of Some single steps of None
breach process not followed or flight followed or process non- process not followed or process documented process not documented process not
critical process non-existent existent unknown followed or process partly followed
unclear
Know-how Dramatic loss resulting in fully Heavy loss resulting in Worrying loss resulting in Loss resulting in noticeable Slight loss that can be easily None
loss new build-up requiring more substantial build-up and/or substantial build-up and/or build-up and/or renewal absorbed within the existing
than 2 years renewal requiring 1–2 years renewal requiring up to 1 year requiring 3–6 months organization
Safety Intolerable total absence of Unusually high level of safety Unacceptable attitude toward Generally acceptable Sound attitude towards safety None
awareness safety awareness demanding awareness ignorance needing safety awareness needing attitude toward safety awareness with occasional
ignorance immediate dismissal immediate correction or immediate correction or awareness with occasional and isolated misjudgement
dismissal dismissal warning blackouts
Cogn Tech Work (2015) 17:249–267
Cogn Tech Work (2015) 17:249–267 257

Fig. 2 Generic risk matrix

Fig. 3 Risk assessment

methodology for company
operational processes—flow
chart

123
258
4 Risk assessment for managing company operational
processes
4.1 Flow chart of RAMCOP
The methodology that respects the key features discussed
in the previous sections is named: RAMCOP. It consists of
a practical approach that offers a simple operational pro-
cedure for implementing the prerequisites of SMSs set by
the Authorities (Fig. 3).
The methodology, initially described elsewhere (De
Grandis et al. 2012), contains three essential Steps, namely:
Develop the case; Conduct Risk Assessment; Revise
design. For each Step, a detailed process of analysis and
evaluation is performed, following a very classical process
for risk assessment. In essence, Step 1, ‘‘Problem statement
and data sources’’ implies:
1.1
The definition of the risk problem under study. For each
system, process and procedure involved, the causal and
consequential barriers, human factors and non-technical
issues are identified.
1.2
The selection of available methods and models and the
definition of the reference RM.
1.3
The identification of threats and hazards to be studied.
1.4
The mining of data sources from reports about real
events and other means of information.
Step 2 entails the assessment of the risks associated to
the hazards, selected in Step 1, from a qualitative and
quantitative perspective. Step 2A (‘‘Qualitative Event
Assessment’’) performs a qualitative analysis, usually by
means of the Event Tree method, and carries out a first
screening based on the analyst’s judgement. Step 2B
implements the ‘‘Quantification’’ process by the evaluation
of probabilities of each occurrence, by the estimation of the
severity of consequences, and by the assessment of the
risks and plans for further safety measures, in the case of
need. This process, performed for all selected hazards,
generates the overall picture of safety related issues and
risks. A process of preliminary evaluation of uncertainties
is also carried out and an overall uncertainty is evaluated in
association to the risk of each sequence.
As a result of Step 2, either:
1. All assessed risks are acceptable (Riskocc B Accept-
able) and consequently the risk analysis is completed
and recommendations are developed before ending the
process. Or
123
Cogn Tech Work (2015) 17:249–267
2. Some of the sequences and associated risks are
associated to a risk zone that resides between the
totally unacceptable and acceptable level (Accept-
able \ Riskocc \ Unacceptable). This intermediate
area requires the decision of whether or not the
assessed risks of occurrence can be accepted, as is, or
if a re-evaluation of the design is necessary, as it is
done in Step 3, for a further reduction of risks. Or
3. Some of the sequences and associated risks reside in a
risk zone that is unacceptable (Riskocc C Unaccept-
able) and consequently it is necessary to re-evaluate
design and safety measures in order to further reduce
risks and reach the ‘‘green’’ or ‘‘yellow’’ areas.
In Step 3 (‘‘Re-evaluation of design’’) the actual design
of safety measures must be revised by including further
causal and consequential barriers, and the overall safety
assessment process must be repeated, from Step 2, so as to
ascertain that the introduction of new safety barriers has
not induced other ‘‘new’’ hazards and risks that need ade-
quate assessment.
The iterative process is completed when all assessed
hazards and risks are considered acceptable and the overall
set of recommendations is developed, on exit of Step 2.
4.2 Practical implementation: the Overall Risk
Assessment Table
The methodology RAMCOP described in Fig. 3 is quite a
standard process that implements the principles of risk
analysis and risk management. In the framework of a SMS
for an organisation operating in the aviation domain, it is
necessary to explore an enormous variety of situations and
conditions for ensuring that all processes involved in
everyday operations are adequately accounted for. For this
reason, ARMS (2011) is proposed as the methodology to
apply, for an initial assessment of risks. This implies that
the judgement of the safety manager and safety team of the
organisation are taken as principal source of information.
In a second instance, the consideration for more sophisti-
cated and alternative methods may be utilised in the case of
situations and events of particular importance.
In practice, the safety evaluation is performed in three
phases of implementation (Fig. 4). During these phases,
the safety analyst and safety team gradually fill an Overall
Risk Assessment Table that contains the key elements of the
SMS analysis and represents the practical implementation
of the RAMCOP methodology (Table 3).
The Overall Risk Assessment Table (ORAT) may be
compared with the outcome of ARMS/BOW-TIE (Fig. 1).
The first two main columns (Treats and Hazards/UOS)
coupled with a qualitative evaluation of the variety of
Cogn Tech Work (2015) 17:249–267
Fig. 4 Phases of development
of the RAMCOP
possible incidental sequences and associated consequences
are equivalent to the left-end side of a Bow-Tie analysis.
The remaining columns are equivalent to the right-end side
of the Bow-Tie. The first two columns can actually be
expanded in a more articulated structure, in the cases of a
complex activity to be studied. This process is represen-
tative of Phase 1 of implementation of the methodology.
The next three main columns of the ORAT table (Inci-
dent sequence description, Existing control and Outcome
Pre-Mitigation) refer to Phase 2 of the implementation of
the methodology. For each specific hazard under study
(UOS), all possible incidental evolutions are described and
the associated consequences are identified with their
probabilities of occurrence and overall distribution of
uncertainty. The existing control measures (barriers) that
contribute to reduce either severity of consequences or
probability of occurrence are identified. The overall risk
can therefore be assessed, in accordance to the reference
RM, in terms of severity and probability of each sequence.
This step completes Phase 2 of implementation.
The need of further mitigation is dealt with in Phase 3
and it is shown in the last four main columns of the ORAT
table (Additional mitigation required, Outcome Post-Miti-
gation, Actions and Owners, and Monitoring and Review
requirements). This phase starts by identifying additional
barriers that may be put into operation. The overall risk
assessment process is repeated, leading to new values of
259
severity and/or probability, and hence new values of risks.
When this process is successfully completed and all risks
are deemed, at least, acceptable, the actions to carry out for
implementing the new safety measures and the subsequent
process of monitoring and reviewing requirements are
made operative.
The ORAT table represents the practical implementation
of the RAMCOP methodology. A table of this nature must
be generated for each UOS to be studied. Moreover,
depending on the specific activity or process, it may be
convenient to split the table in several different sub-tables,
associated, for example, to each specific consequence
resulting from the same hazard, or to the combination of
different threats leading to the same UOS, or else, to the
implementation of different mitigation measures.
This simplification and optimisation of the process rep-
resents a further step of implementation of the methodology
that cannot be formalised in a procedure, as it requires
experience and engineering understanding of the way in
which a risk assessment evaluation may be carried out.
5 A case study: Electronic Flight Bags for flight
management
The RAMCOP methodology has been applied in the case
study presented hereafter with the aim of showing how the
123
260
methodology can be implemented in detail. It is possible to
see the degree of knowledge required in order to identify
the appropriate methods and techniques to be used on a
case by case level.
The case study refers to the introduction of new ways
and processes for flight management in a medium-size
airline: the use of Electronic Flight Bag (EFB) technology.
It is a typical case of prospective study (change manage-
ment), i.e., a situation of a new configuration that occurs
within an organisation, and requires a thorough safety
assessment before implementation.
The three phases of the methodology are applied in
sequence and, in this case, the knowledge and experience
of the safety team have been fully exploited, as many of the
data necessary for the study have been derived from
piloting practice.
In the development of the case study, the evaluation of
epistemic uncertainties and the propagation through the
event tree of the prospective study has not been discussed,
as the aim of the sample case is to show how the main step
of the methodology are actually implemented in a realistic
application.
5.1 EFB-Phase 1: problem statement, data sources,
threats, and hazards
5.1.1 Problem statement
The implementation of the EFB technology is an opera-
tional change for an airline that aims at improving flight
efficiency and effectiveness by replacing the paper copies
of information and support material, typically carried on
board by the pilots in their ‘‘bags’’, with electronic files
contained in laptops or integrated within the flight control
panel as part of the flight management system. The
implementation and use of the EFB is an obvious important
change in the organization, as it is expected that there will
be an overall return in the management of the operations,
reducing the time spent in preparation of the different
routes and loads, as well as in optimizing flight times, and
eventually reducing the overall cost. This would also
impact on the overall efficiency of the operations, reducing
the amount of time loss in correlating the work of ground
and flying staff and decreasing possible human errors in
assessing critical flight data (Cassani et al. 2012).
In particular, it is expected that the use of EFB will
remove ‘‘human errors’’ in relation to the evaluation of
routes and in preparing landing and take-off procedures, as
well as in setting a variety of crucial data, such as decision
on key speeds of go-non-go or rotation (take-off speed).
However, it is important to consider that the introduction of
the EFBs may induce other types and modes of human
error which must be defined and studied in order to perform
123
Cogn Tech Work (2015) 17:249–267
a complete assessment of the new risk scenarios to be
evaluated.
The problem was initially tackled by expert judgment,
supported by the familiarity with company rules and
accepted behaviours (culture), and by the availability of
data and past experience of other organizations willing to
share their historic data and past experience on EFB.
Moreover, the implementation of the EFB within the
operational processes of an airline follows a precise pro-
cess of progressive inclusion. An initial step of imple-
mentation is performed with the usage of electronic maps
inserted in hardware systems, e.g., I-pads, tablets, portable
PC, etc. to be carried on board by the pilots. This implies
that the EFB system cannot be utilised during the flight
critical phases of take-off and landing, but can be exten-
sively connected to the control system while the aircraft is
on ground and during the pre-flight phase. This step of
utilisation was considered a ‘‘static usage’’ of the EFB.
In a second step of implementation, the EFB is fully
integrated within the control system of the aircraft, so as to
enable its usage during the whole operational period. In
this case therefore, in addition to the usage for the maps
and airport plans and procedures, the EFB can extensively
be utilised to support the calculation of critical speeds. This
step of utilisation was considered a ‘‘dynamic usage’’ of the
EFB.
These two steps of implementation have required the use
of specific methods in support of risk evaluation. The
RAMCOP methodology has been extensively applied.
5.1.2 Threats, hazards and potential outcome
In order to carry out the preliminary analysis of the ‘‘static
and dynamic usage’’ of the EFB, a set of workshops and
brain storming meetings have been performed, accounting
also for the configuration of the hardware carried on board
and the EFB usage for the pre-flight phase. This process
defined the set of threats and hazards or undesirable
operational states that may derive from the usage of the
EFB system. The threats and hazards were further framed
within the usual activities and processes implemented in
the company as Standard Operating Procedures (SOP), i.e.,
a table of threats and hazards was developed considering
the pre-flight steps and activities carried out by the pilots
on board. These can be formalized in three phases: the
cockpit preparation; the final cockpit crew preparation and
the dynamic cockpit crew management. The ‘‘cockpit
preparation’’ can be further subdivided in Cockpit Power
up, Walk Around (external inspection), Cockpit Prepara-
tion by pilot in command (CM 1—captain), Cockpit
Preparation by pilot (CM 2—first officer). The ‘‘final
cockpit crew preparation’’ is essentially associated to the
operations carried out in the cockpit prior to leaving the
Cogn Tech Work (2015) 17:249–267 261

parking area. The ‘‘dynamic cockpit crew management’’
refers essentially to the take-off phase and focuses on the
setting of the aircraft critical speeds.
A number of threats affecting flight operation, such as
pilot workload, distraction and misunderstanding in some
operation, as well as lack of familiarity with the EFB and
technology used on board, or even the improper setting and
updating of data and Information Technology support may
be the relevant causes/precursors of possible hazards.
In summary, Table 2 shows the threats, hazards and
potential outcomes identified in Phase 1 of the study: 14
UOSs associated to the ‘‘static usage’’ of the EFB of
cockpit preparation and 2 UOSs occurring during the
dynamic cockpit management and take-off operations.
5.2 EFB-Phase 2: quantification of severity, probability
and initial assessment of risk
5.2.1 Methods for risk assessment
The quantification phase of the methodology begins by
combining the different hazards identified in the qualitative
assessment process and associating these to the potential
outcome resulting from the brain storming process, in

Table 2 Threats, hazards and potential incidental sequences of the EFB introduction
Activity and threats Hazard/UOS Potential incidental sequence

Cockpit preparation: cockpit power up; walk around; cockpit preparation CM 1; cockpit preparation CM 2
Excessive workload of CM2 due to number of task 1. Software initialisation not completed Flight diversion or cancellation
to carry out during cockpit preparation 2. Maps not available
Improper/inadequate loading of software 3. Improper selection of portrait Flight diversion or cancellation
2. Maps not available CFIT (Controlled Flight Into Terrain)
Loss of separation
Lack of adjournment of software 3. Improper selection of portrait Flight diversion or cancellation
2. Maps not available CFIT
Loss of separation
Lack of familiarity with PC handling, time pressure 4. Improper storage of PC Damage to cables/PC and fire/smoke
on CM2 in cabin
Damage to cables/PC and flight
delay/cancel
Final cockpit crew preparation
Pilot workload 5. Pilots unable to locate maps CFIT
6. Loss of SA (Situational Awareness) Loss of separation
Out of charge batteries 7. No charts on show CFIT
6. Loss of SA Loss of separation
Diversion/alternated/delay
No updated paper maps or 8. Flying with wrong maps or without maps CFIT
Missing paper maps 6. Loss of SA Loss of separation
Diversion/alternated/delay
No airfield Sketch. Lack of familiarity with airfield, 9. No coordinates for cross-check with FMS (Flight Runway incursion
worsen by visibility problems. Management System). Impossible to see taxiway Ground collision with aircraft/
10. Getting lost on airfield infrastructures
Flight cancellation
No SID (Standard Instrumental Departure) 11. Missing performance CFIT
No/wrong SID, bad weather 13. No info/news on obstacles Loss of separation (ground/flight)
14. Flying wrong departure Mid Air Collision (MAC)
No approach for emergency chart 12. Missing information in the case of emergency CFIT
6. Loss of SA Loss of control in flight/loss of
separation
Dynamic cockpit crew management
Pilot workload, distraction, misunderstanding etc. 15. Inadequate speed for take-off Tail strike
16. Inadequate speed for aborted take-off Loss of control
Runway overrun/runway excursion

123
262 Cogn Tech Work (2015) 17:249–267

terms of consequences and probability of occurrence. This

of compliance with

(‘‘how and what to

activity and means
Describe monitoring

assigned actions
leads to the definition of the hazards that need to be studied
Monitoring and

and auditing
in order to ‘‘position’’ each of them on the RM.

monitor’’)
review req.
In order to assess the severity of the consequences, some
initial work of definition of the nature and reference units
of severity was carried out. In particular, the injuries to

for mitigating risk

Actions and owners

(‘‘who should do
passengers and crew members and the damage to property
that are planned

and identify the

actors involved
Describe actions
or aircraft resulting from potential incidents were utilized,
in accordance to the first two types of severity measures

what’’)
defined in Table 1.
The quantification, in terms of probabilities, required
the usage of a variety of methods. Although direct EJ
Rcons.1

Rcons.2

Rcons.3

Rcons.i

……
……
Outcome (post-mitigation)

Risk

application of the safety analysts carrying out the study is

usually applied, as discussed in the previous section, in this
Probab.

pcons.1

pcons.2

pcons.3

pcons.i

……
……
case, given the complexity of operations and the relevance
human factors, it was decided to combine EJ with more
Severity

consolidated methods. For the ‘‘static usage’’ of the EFB,

Scons.1

Scons.2

Scons.3

Scons.i

……
……

the TESEO approach was chosen for calculating the

probability of unsuccessful performance of procedures
reduction

badd.barr.1

badd.barr.2

badd.barr.3

badd.barr.i
Type of

carried out by pilots. Moreover, in some cases TESEO was

……
……
barr.
Add. mitigation

combined with the information derived from the existing

Barrier1

Barrier2

Barrier3

database of reports on incidents and near-misses existing in

Barrieri
required

Type of
Phase 3

barriers

the company.
……
……
Add.

Add.

Add.

Add.

For assessing probabilities of human error and sequen-

Rcons.1

Rcons.2

Rcons.3

Rcons.i

ces associated to the ‘‘dynamic usage’’ of the EFB and the

……
……
Risk

two UOSs identified in Phase 1, an extension the THERP

Outcome (pre-mitigation)

method was considered (Cacciabue and Cassani 2012). In

pcons.1 = f(a,

pcons.2 = f(a,

pcons.3 = f(a,

pcons.i= f(a,
pcons.1)

pcons.2)

pcons.3)

practice, the procedure of critical speed evaluation for

pcons.i)
Probab.

……
……

take-off has been analysed evaluating different possibilities

of pilot errors, rather than utilising the simple binary
Severity

alternative of success or failure, typical of the standard

Scons.1

Scons.2

Scons.3

Scons.i

……
……

THERP approach.
reduction

5.2.2 Selection of sequences and quantification of risk

abarr.1

abarr.2

abarr.3
Prob.

abarr.i
Existing control

……
……

For the calculation of the risks associated to the 16 UOSs,

Barrier1

Barrier2

Barrier3
Barriers

Barrieri

……
……

Table 2 has been firstly rearranged in such a way that, for

each UOS, all associated sequences are identified and
Phase 2

without

pUOS

pUOS

pUOS

pUOS
control

pcons.1/

pcons.2/

pcons.3/

evaluated in terms of risk level, i.e., severity and proba-

pcons.i/
Prob.

……
……

bility of occurrence. This is in line with the methodology

Incident sequence

approach shown in Table 3. The resulting table that groups

Consequences
description

all UOSs and associated sequences is rather complex and

Cons.1

Cons.2

Cons.3

Cons.i

cannot be shown here for sake of space. As for example,

……
……

the sequences associated to the UOS n. 2 (‘‘Maps not

Table 3 Table of risk assessment

available’’) are: Flight diversion or cancellation, CFIT

of pUOS as a

pUOS = f(pi)
Description of

function of
calculation
Hazard UOS

Description

UOS and
probability

(Controlled Flight Into Terrain), and Loss of separation.

pthr.

Similarly, the sequences associated to the UOS n. 6 (‘‘Loss

and

of SA-Situational Awareness)’’ are: CFIT, Loss of sepa-

ration, and Diversion or Alternated or Delay.
Prob.

……
……
pth1

pth2

pth3

pthi

The risk levels of all occurrences have been calculated

Description

utilising the methods described in the previous section and

Threath1

Threath2

Threath3

Threathi
Phase 1

Threats

considering the mitigation measures existing within the

……
……

organisation. An overall table of risks has been developed.

123
Cogn Tech Work (2015) 17:249–267
This process implies a very detailed and accurate analysis
of each sequence and goes beyond the scope of this paper
(Mariani 2012; De Col 2012). The results obtained have
been streamlined by selecting, for each UOS, the sequence
showing the highest risk, i.e., the highest value of com-
bined probability and severity according to the adopted
reference RM. The results, summarising the outcome of
Phases 1 and 2, prior to the implementation of the further
mitigation measures, are presented in the first two main set
of columns of Table 4.
5.3 EFB-Phase 3: additional mitigation and final safety
assessment and actions
5.3.1 The risks assessed
The results of Phase 2 of analysis have shown that only one
UOS, i.e., UOS n. 10, ‘‘Getting lost on airfield’’, can be
accepted with no further mitigation, as it is located in the
area ‘‘Low—Monitor’’ of the reference RM (Table 4).
The additional mitigation measures required for further
reducing risk have been identified in the availability of
Paper maps on board, as a back-up to the EFB system, and
a more intensive and focused Pilot Training. It has been
assumed that for each of these two further barriers a
reduction of a factor 10 of the probability of occurrence
could be adopted. No consequential barriers reducing the
severity of the consequences have been adopted.
The choice of the additional ‘‘barrier’’ requires a more
careful discussion in order to explain that the introduction
of this barrier, reinstalls the risks associated to human
errors in the management of paper form documentation.
However, the availability paper copies of maps is seen as a
second level barrier or backup measure to offer redun-
dancy, to be utilised only when the EFB technology fails.
As a result of this post mitigation the overall risk
associated to the 16 UOS has been further reduced. The
risk levels, after the post mitigation, are shown in the last
set of columns of Table 4. For readability purposes the last
two columns of the standard table associated to Phase 3 of
the RAMCOP methodology (‘‘Actions and owners’’ and
‘‘Monitoring and review requirements’’) have not been
shown. A graphical representation of the overall risk values
for all 16 UOSs, after the implementation of all mitigation
measures, is shown in Fig. 5.
5.3.2 Discussion and acceptability of risks
The results of the prospective analysis with the RAMCOP
methodology and the specific methods utilised for the
various sequences studied has led to a number of consid-
263
erations in relation to the introduction of the new EFB
system on board of the aircrafts of the Company.
Firstly, it is noticeable (Fig. 5) that no hazards have
resulted in the neither ‘‘red’’ nor ‘‘orange’’ areas that
requires immediate action and possibly the stoppage of
operations. Moreover, 50 % of the hazards identified and
studied have resulted in the acceptable area of the matrix
and they do not require, at present, further mitigation
actions. The remaining 50 % of the hazards, i.e., eight
hazards, require careful consideration and possibly long
term improvement.
In particular, the risk levels associated to UOSs 15 and
16 (inadequate speed for take-off and inadequate speed for
aborted take-off) are the worse cases. This was expected as
they are associated to the dynamic phase and may need the
implementation of further protective barriers.
Similarly, sequences associated to UOSs 6 and 12 are in
the ‘‘yellow’’ area and those risks need to be evaluated. As
in the previous case, the loss of SA (UOS 6) and the
absence of information in the case of an emergency (UOS
12) are crucial hazards that affect dynamic conditions, i.e.,
flight management. Possibly, a common improvement or
safety measure can be implemented that is able to tackle all
the hazards generated in dynamic conditions.
UOSs 7, 9, 13 and 14 are also in the ‘‘yellow’’ area of
the RM. However, they relate to extreme consequences and
consequently it is possible to improve the risk only by
mitigating the severity of consequences, as the probability
of occurrence is already at the lowest possible level (below
minimum).
This is a typical situation in which particular attention
needs to be dedicated to the decision making process of the
management, as two critical decision processes may be
generated. On the one side the extreme consequences
envisaged in the case of encounter of the specific hazards
require a very careful consideration about the opportunity
to increase further the barriers and mitigation measures, as
they would surely contribute to reduce even further the
already very low probability of occurrence. However, this
would not actually ‘‘improve’’ the risk index, as the lowest
position in the RM is already reached in terms of proba-
bility. Therefore the new barriers could be deemed not
necessary.
On the other side, there could be an even more para-
doxical situation in which the fact that the catastrophic
conditions are associated to very low probabilities could
favour a reduction of barriers or safety measures, as the
effect in terms of probability would not be noticed for the
same reason as above. But this time the criticality of the
decision making would be even more inadequate than
before.
123
 Table 4 Overall risk table for the EFB case study
264

Phase 1 Phase 2 Phase 3

123
Hazard/UOS no. Incident sequence description Existing control Outcome (pre-mitigation) Add. mitig. Outcome (post-mitigation)
required
Consequences Barriers Severity Probab. Risk Type of Severity Probab. Risk
barriers

1 Software initialisation not Flight div. or canc. M-Q; TCAS; EGPWS Low Likely Acc. with Paper maps Low Possible Low
completed mitig. Monitor
2 Maps not available Loss of separation M-Q; TCAS; EGPWS High Remote Acc. with Paper maps High Ext. Low
mitig. remote Monitor
3 Improper selection of portrait Flight cancellation or delay Training; SOP/EOP; Low Likely Acc. with Paper maps Low Possible Low
TCAS; EGPWS mitig. Monitor
4 Improper storage of PC Damage to cables/PC and flight M-Q; SOP High Unlikely Acc. with Paper maps; High Ext. Low
delay/cancel mitig. training remote Monitor
5 Pilots unable to locate maps Loss of separation Training; SOP/EOP; High Unlikely Acc. with Paper maps; High Ext. Low
TCAS; EGPWS mitig. training remote Monitor
6 Loss of SA Loss of separation Training; SOP/EOP; High Low High Paper maps; High Remote Acc. with
TCAS; EGPWS training mitig.
Long term
7 No charts on show CFIT M-Q; TCAS; EGPWS Extreme Ext. Acc. with Paper maps; Extreme Ext. Acc. with
remote mitig. remote mitig.
Long term
8 Flying with wrong maps or Loss of separation Training; SOP/EOP; High Unlikely Acc. with Paper maps; High Ext. Low
without maps TCAS; EGPWS mitig. training remote Monitor
9 No coordinates for X-check Ground collision with aircraft, ATC; SOP/EOP; training Extreme Ext. Acc. with Training Extreme Ext. Acc. with
with FMS vehicles or infrast remote mitig. remote mitig.
Long term
10 Getting lost on airfield Runway incursion ATC; EOP; training Medium Unlikely Low – Medium Unlikely Low
Monitor Monitor
11 Missing performance Mid Air Collision (MAC) EOP; training; EGPWS High Remote Acc. with Training High Ext. Low
mitig. remote Monitor
12 Missing info. in the case of Loss of control in flight ATC; EOP; training; High Unlikely Acc. with Paper maps; High Remote Acc. with
emergency EGPWS mitig. Training mitig.
Long term
13 No info/news on obstacles CFIT ATC; EOP; EGPWS Extreme Ext. Acc. with Training Extreme Ext. Acc. with
remote mitig. remote mitig.
Long term
14 Flying wrong departure CFIT ATC; EOP; TCAS; Extreme Ext. Acc. with Paper maps; Extreme Ext. Acc. with
EGPWS remote mitig. training remote mitig.
Cogn Tech Work (2015) 17:249–267

Long term
Cogn Tech Work (2015) 17:249–267 265

6 Conclusions

Long term

Long term
Acc. with

Acc. with
This paper has discussed a methodology aimed at sup-

mitig.

mitig.
Outcome (post-mitigation)
porting safety analysts and safety managers in the practical
Risk
implementation of SMSs and, in particular, in the assess-

Unlikely
ment of risks in modern and complex aviation organiza-
Probab.

tions. The methodology is embedded in an instrument that

Low

enables the performance of a rapid and relatively easy

sequence of steps for the application of conventional
Medium
Severity

High
methods.
The methodology implements well-established meth-
ods, in addition to the sole use of the analysts’ EJ as it
Type of barriers

proposed in other methodologies. Moreover, it accounts for

epistemic uncertainties and contains a simplified sensitiv-
Add. mitig.

ity analysis. This ensures that the results and information

Training

Training
required
Phase 3

obtained are credible and present a measure of certainty

that enables the decision maker to reach knowledgeable
and consolidated opinions about what should or should-not
be done in order to maintain a safe and efficient operational
Acc. with

Acc. with

environment.
mitig.

mitig.

A real application case study has been discussed: the

Risk
Outcome (pre-mitigation)

implementation on board of modern aircrafts of a new

electronic system supporting flight management, the so
Unlikely
Possible
Probab.

called EFBs. This is a typical chase of a change manage-

ment situation, i.e., a novel state/condition encountered by
an organization, requiring prospective type of risk and
Medium
Severity

safety assessment.
High

The proposed methodology has been developed as part

of a more general framework, developed in a EU supported
SOP; training

SOP; training

research action aimed at studying the management of

Existing

system changes in aviation (McDonald et al. 2012). The

Barriers
Phase 2

control

methodology, called RAMCOP, enables the implementa-

tion of the overall risk assessment and supports the ade-
quacy of the SMS of an organisation in responding to the
requirements of the safety authorities.
The linear and simple procedure described in the paper
Runway excursion
Incident sequence

enhances the relevance of the safety manager’s knowl-

Consequences

edge and experience, as well the importance of having a

description

Tail strike

sound repository of information about historical events

and occurrences in order to compile the picture of the
company culture and attitudes towards safety. The sub-
stantial use of expert judgment, applied in order to define
Inadequate speed for aborted take-

severity and frequency of occurrences, requires a very

careful and well thought and balanced exploitation of
Inadequate speed for take-off

expertise, field analysis, information and existing com-

pany data.
The principles that govern the approach described in
this paper have been implemented and utilized by AirDo-
Table 4 continued

Hazard/UOS no.

lomiti in the recently developed company SMS, which has

been reviewed and accepted by the Italian National Civil
off

Aviation Authority. The generality of the approach enables

Phase 1

to consider its possible implementation to other areas of

15

16

transport as well in other technological domains. In

123
266
Fig. 5 Final assessment of risk
for all 16 UOS—EFB case
study Probability S5
Level Extreme
P5
Frequent
P4
Likely
P3
Possible
P2
Low
P1
Unlikely
P0
Remote
Pe 7, 9, 13,
Extr. Remote 14
particular, its use in the ship and rail areas requires simply
the straightforward application of the equivalent concepts
with respect to the specific technologies and procedures. In
the area of automotive transport, especially for the human
reliability assessment, the type and variety of existing
models and the enormous amount of research work asso-
ciated to personal factors and environmental impact on
driver performance should be adequately accounted for.
Another important aspect that requires consideration when
attempting to extrapolate the method to other areas or
domains is the influence of time in the overall assessment
of risk. In particular, in the automotive environment time is
a crucial variable to consider from the aspect of dynamic
reliability of human–machine interaction as well from the
perspective of implementation on board of hazard identi-
fication and risk perception for interactive systems sup-
porting driver decision making.
Finally, possibility to implement the method in rapid
and fast running software tools, supported by large dat-
abases of historical outputs, makes realistic its potential
application in industrial development for on board decision
support systems for all transportation domains.
Acknowledgments The authors are in debt with the members of the
research team of the MASCA Project. The MASCA project has
received funding from the European Commission Seventh Framework
Programme (FP7/2007–2013) under Grant agreement No. 266423. An
important support work of data collection and analysis has been
carried out in two Master Theses of the Politecnico of Milan. The
authors are grateful the C. Mariani and A. De Col for their detailed
and skilled work of data analysis.
123
Cogn Tech Work (2015) 17:249–267
Severity Level
S4 S3 S2 S1 S0
High Medium Low Minor None
1, 3
15
16 10
6, 12
2, 4, 5,
8, 11
References
Air Dolomiti (2011) Air Dolomiti Safety Management Manual
Airport Cooperative Research Program (ACRP) 2012. ACRP Report
74. Application of Enterprise Risk Management at Airports
ARMS (2011) The ARMS methodology for operational risk assess-
ment in aviation organisations. http://www.easa.eu.int/essi/docu
ments/Methodology.pdf.visited2011.12.28
Ayres M, Shirazi H, Carvalho R, Hall J, Speir R, Arambula E, David
R, Gadzinski J, Caves R, Wong D, Pitfield D (2013) Modelling
the location and consequences of aircraft accidents. Saf Sci
51:178–186
Bello GC, Colombari C (1980) The human factors in risk analyses of
process plants: the control room operator model, TESEO. Reliab
Eng Syst Saf 1:3–14
Byrom NT (1994) The essential elements of a safety management
system. In: Cacciabue PC, Gerbaulet I, Mitchison N (eds) Safety
management systems in the process industry. EUR 15743 EN,
22–30
Cacciabue PC (2004) Human error risk management for engineering
systems: a methodology for design, safety assessment, accident
investigation and training. Special Issue on HRA Data Issues and
Errors of Commission. Reliab Eng Syst Saf 83:229–240
Cacciabue PC, Cassani M (2012) Modelling motivations, tasks and
human errors in a risk-based perspective. Cog Tech Work
14(3):229–241
Cassani M, Licata V, Baranzini D, Corrigan S, De Grandis E,
Ottomaniello A (2012) Integrated data management for handling
hazard of change situations: a sample case of operational
implementation. In: Proceedings of PSAM-11—ESREL 2012
Helsinki Finland June 25–29
De Col A (2012) Il safety management system come strumento di
valorizzazione e certificazione della sicurezza in campo aeronau-
tico: analisi prospettiche in accordo alle line guida dall’Autorità
ed alle esperienze Aziendali. Master Thesis (in Italian) Politec-
nico di Milano
Cogn Tech Work (2015) 17:249–267
De Grandis E, Oddone I, Ottomaniello A, Cacciabue PC (2012)
Managing risk in real contexts with scarcity of data and high
potential hazards: the case of flights in airspace contaminated by
volcanic ash. In: Proceedings of PSAM-11—ESREL 2012,
Helsinki, Finland, June 25–29
EASA—European Aviation Safety Agency (2011) Notice of proposed
amendment—explanatory note. Authority, Organisation and
Operations Requirements for Aerodromes. NPA 2011-20 (A)
EASA—European Aviation Safety Agency (2012) European aviation
safety plan 2012–2015—Final report
Ersdal G, Aven T (2008) Risk informed decision-making and its
ethical basis. Reliab Eng Syst Saf 93:197–205
FAA (2010) SMS notice of proposed rulemaking (NPRM) for 14 CFR
Part 121 Certificate Holders. SMS NRPM for 14 CFR Part 121
Hollnagel E (1998) Cognitive reliability and error analysis method.
Elsevier, London
Hudson P (2000) Safety management and safety culture. The Long,
Hard and Winding Road. In: Pearse W, Gallagher C, Bluff L
(eds) (2001) Proceedings of the first national conference on
occupational health and safety management systems. Crown
Content, Melbourne, Australia, 3–32
IAEA—International Atomic Energy Agency (2005) Risk informed
regulation of nuclear facilities: overview of the current status
IAEA-TECDOC-1436. Vienna, Austria
IAEA—International Atomic Energy Agency (2011) A framework
for an integrated risk informed decision making process INSAG-
25. Vienna, Austria
IATA—International Air Transport Association (2010) IOSA stan-
dards manual. Reference No: 6361-03. IATA, Montreal, Canada
ICAO—International Civil Aviation Organisation (1987) Acciden/
incident reporting manual. Second edition—DOC 9156-AN/900,
Montreal, Canada
ICAO—International Civil Aviation Organisation (1993) Investiga-
tion of human factors in accidents and incidents. Human Factors
Digest, No 7, ICAO Circular 240-AN/144 Montreal, Canada
ICAO—International Civil Aviation Organisation (1997) Accident/
incident reporting manual-ADREP 2000 draft. ICAO Report,
Montreal, Canada
ICAO—International Civil Aviation Organisation (2012) Safety
management manual. Doc 9859 AN/474. Third Edition Mon-
treal, Canada
Kirchsteiger C, Christou MD, Papadakis GA (1998) Risk assessment
and management in the context of the Seveso II Directive.
Industrial Safety Series, n. 6. Elsevier, Amsterdam
Liou JJH, Yen L, Tzeng GH (2008) Building an effective safety
management system for airlines. J Air Transp Manag 14:20–26
Mariani C (2012) Risk analysis in take-off procedure with Electronic
Flight Bag. Master Thesis, Politecnico di Milano
McDonald N, Ulfvengren P, Ydalus M, Oder E (2012) A method-
ology for managing system change. In: Proceedings of PSAM-
11—ESREL 2012, Helsinki, Finland June 25–29
267
Mosleh A, Dias A, Eghbali G, Fazen K (2004) An integrated
framework for identification, classification, and assessment of
aviation system hazards. In: Proceedings of PSAM-07—ESREL
2004, Berlin, Germany June 14–18
NASA (2010) NASA risk informed decision making handbook.
NASA/SP-2010-576 (Vol. 1)
NASA (2011) Probabilistic risk assessment procedures guide for
NASA managers and practitioners NASA-SP-2011-3421
Nielsen DS (1971) The cause/consequence diagram method as a basis
for quantitative accident analysis. Danish Atomic Energy
Commission RISO-M-1374
NRC—US Nuclear Regulatory Commission (1995) Final policy
statement ‘use of probabilistic risk assessment (PRA) methods in
nuclear regulatory activities. 60 FR 42622, Washington
NRC—US Nuclear Regulatory Commission (1998) An approach for
plant-specific, risk-informed decision making: technical specifi-
cations. RG 1.177, Washington
NRC—US Nuclear Regulatory Commission (2002) An approach for
using probabilistic risk assessment in risk-informed decisions on
plant-specific changes to the licensing basis. RG 1.174,
Washington
NRC—US Nuclear Regulatory Commission (2003) Fleming KN.
Issues and recommendations for advancement of PRA technol-
ogy in risk-informed decision making. NUREG 6813
NRC—US Nuclear Regulatory Commission (2009) Drouin M, Parry
G, Lehner J, Martinez-Guridi G, LaChance J, Wheeler T.
Guidance on the treatment of uncertainties associated with PRAs
in risk-informed decision making. NUREG 1855, vol. 1
Ranlöf L, Knochenhauera M, Hultqvist G (2012) Chasing a moving
target: applying safety goals on a living PSA. In: Proceedings of
international conference PSAM 11-ESREL 2012, 25–29 June,
Helsinki
Roelen ALC, Wever R (2005) Accident scenarios for an integrated
aviation safety model. National Aerospace Laboratory NLR,
Amsterdam, The Netherlands, NLR-CR-2005-560
Schindler J, Cassani M (2013) Using an integrated simulation
environment for the risk based design of advanced driver
assistance systems. Transp Res Part F 21:269–277
Siu N (1994) Risk assessment for dynamic systems: an overview.
Reliab Eng Syst Saf 43(1):43–73
Stolzer AJ, Halford CJ, Goglia JJ (2010) Safety management systems
in aviation. Ashgate, London
Swain AD, Guttmann HE (1983) Handbook of reliability analysis
with emphasis on nuclear plant applications. Nuclear Regulatory
Commission NUREG/CR-1278, Washington
Tamasi G, Demichela M (2011) Risk assessment techniques for civil
aviation security. Reliab Eng Syst Saf 96(8):892–899
UK-CAA (2010) Guidance on the conduct of hazard identification,
risk assessment and the production of safety cases. UK Civil
Aviation Authority, ISBN 978 0 11792 488 8
123