Cogn Tech Work (2015) 17:219–236
DOI 10.1007/s10111-014-0297-8
ORIGINAL ARTICLE
Development and Implementation of a Safety Management
System in a Lean Airline
Pernilla Ulfvengren • Siobhán Corrigan
Received: 7 February 2014 / Accepted: 14 September 2014 / Published online: 22 October 2014
 Springer-Verlag London 2014
Abstract All stakeholders within the aviation sector are
currently facing immense system changes due to implica-
tions from the future Single European Sky concept and the
new requirement for a Safety Management System (SMS).
At the same time, the airline industry is under great
financial pressure. So, there are no margins for failure to
adapt and comply with these system changes. Yet, the
reported success rate of organisational change in industry is
low. The MAnaging System Change in Aviation (MA-
SCA—EU FP7 2010–2013) project addressed these
industrial needs. MASCA was driven by industrial-based
case studies. The change case reported on in this paper is
an ongoing development and implementation of an SMS in
a major European Airline. The overall objective in this
industry case was to develop a SMS that will demonstrate
safety performance to comply with new regulations. A new
approach for human factors and safety was developed in
earlier research. As part of this a system and process ana-
lysis, a concept called the System Change and Operations
Evaluation (SCOPE) model was developed. The approach
taken in this case was to apply the SCOPE to enhance core
functionalities and further develop the airline SMS. Taking
an action research approach, researchers worked closely
with the airline’s safety department to support their
development of the SMS and to study that system change.
The MASCA research objective was to develop a theory
for change which included a methodology to evaluate
P. Ulfvengren (&)
Department of Industrial Economics and Management,
KTH Royal Institute of Technology, Stockholm, Sweden
e-mail: pernilla.ulfvengren@indek.kth.se
S. Corrigan
Centre for Innovative Human Systems (CIHS),
School of Psychology, Trinity College, Dublin, Ireland
change to support implementation. This case contributed to
the multi-case study in MASCA in which a framework for
evaluating change emerged, called the Structured Enquiry
(SE). The SE was applied to provide complementary rec-
ommendations to the proposed enhanced SMS. The SE
provided support to the fact that, even if all the pieces are
in place required for a compliant SMS, many other
essential areas need to be addressed to make all the pieces
work together, such as information and knowledge cycles
and social relations building teams and trust. Results show
benefits of combining SCOPE and SE in system change in
aviation in order to encompass identified essential com-
ponents for safety performance and increasing the chances
for a successful change.
Keywords Aviation
Safety
Change management

Safety performance
Safety Management System (SMS)
1 Introduction
All aviation stakeholders in the world are facing
immense system changes as a result of implications from
the future Single European Sky (SES) concept and, more
urgent, new requirements for Safety Management Sys-
tems (SMS). The whole airline industry is at the same
time under great financial pressure. For many, their key
priority is survival. So, there are no margins for failure
to adapt and comply with these new technical and reg-
ulatory requirements. Yet, the academic literature indi-
cates that only a minority of major change initiatives
have a positive outcome (Dent and Goldberg 1999;
Kotter 1995). Due to this, there is an industrial need to at
least improve chances for successful change (McDonald
2014). In addition, the SES concept needs to become
123
220
operational without compromising safety. Although avi-
ation is an ‘‘almost totally safe transportation system’’
(Amalberti 2001), an analysis of the safety trend shows
that it is no longer getting safer (Learmont 2009). This
implies that the methodologies for safety management
across the system that brought aviation to this level are
not going to take the air transportation system (ATS)
beyond this plateau and that a new approach to safety
and change is needed. The European Commission has set
a target to reduce accident rate by 80 % (EC 2011), and
the new regulatory requirement of a SMS is intended to
contribute to this goal. In this paper, research from the
MAnaging System Change in Aviation project (MA-
SCA—EU FP7 2010–2013) is presented. MASCA
addressed the industrial need to manage system change,
including risk and performance, in an integrated way. In
MASCA, a theory of change was developed based on a
multi-case study among aviation industrial partners. This
paper presents one of those industrial-based case studies.
The case reported on here focuses on an ongoing
development and implementation of a SMS in a major
European Airline. In this research, it is argued that avi-
ation safety in the future depends both on the organisa-
tional capabilities to adapt to the ongoing system changes
as well as the industry and research communities’ ability
to develop and successfully implement new approaches
for safety management.
1.1 Industrial partner objectives and needs
The goal of this ongoing work is to develop and to
implement a SMS that enables the airline to allow reliable
and relevant indicators drive sustainable change to improve
the operational process and demonstrate safety perfor-
mance. The airline’s objectives in MASCA were:
Allow safety performance indicators (SPIs) to drive
actual and sustainable change
• Develop a process for change linked to SPIs.
• Develop a methodology for developing SPIs.
• Align a SPI-driven change process with existing risk
management process.
Develop a Safe-Lean concept for the airline.
• Improve ways to control and balance safety with
production.
• Integrate Lean with SMS structures and processes,
where appropriate.
• Develop an integrated risk model for data analysis for
integrated performance monitoring.
The top priority is that it all needs to work well
together! (Head of Safety in Airline 2012).
123
Cogn Tech Work (2015) 17:219–236
1.2 Objective
The overall objective in this industry case is to develop
an SMS with core functionalities that will demonstrate
safety performance. These functionalities should be
integrated with the existing airline structures and pro-
cesses and ensure compliance with the new regulations.
In addition, another research objective is to apply an
initial framework intended to support a successful
implementation of a system change including an evalu-
ation of the change occurring and suggest direction
towards increased chance for a success. The objective
was also to study the changes occurring during the
development both in research and in-house initiatives as
a contribution to the MASCA multi-case study for
building a theory of change.
1.2.1 Core functionalities of an SMS
The new regulation was effective from October 2014. The
new regulation is based on ICAO’s safety management
manual (SMM) (ICAO 2012), which is intended to provide
guidance to develop a regulatory framework and guidance
material for the implementation of a SMS. Airlines will
need to apply for renewed Airline Operations Certificate
(AOC), based on their new SMS. Compliance will initially
be based on a checklist, called the Compliance Checklist
(CCL).
The basic framework of the SMS consists of four main
components:
• Safety policy and objective
• Safety risk management
• Safety assurance
• Safety promotion
The SMS shall, as a minimum:
• identify safety hazards
• ensure the implementation of remedial action necessary
to maintain agreed safety performance
• provide for continuous monitoring and regular assess-
ment of the safety performance
• aim at a continuous improvement of the overall
performance of the SMS
A SMS is part of the strategy towards the new per-
formance-based regulatory environment in aviation.
Safety performance is defined as the ‘‘effectiveness of
the SMS, how well it performs and manages to improve
the system’’ (ICAO 2008). SPIs reflect safety perfor-
mance measurement exclusively. The SPIs of an SMS
should not reflect safety measurement. ‘‘The safety per-
formance of an SMS is not directly related to the
Cogn Tech Work (2015) 17:219–236
quantification of high-consequence outcomes (safety
measurement) but rather to the quantification of the low-
consequence process (safety performance measurement)’’
assumed to reduce latent conditions (ICAO 2012). Safety
performance for service providers is proposed to be
expressed in terms of compliance to set safety require-
ments, safety performance targets and SPIs. This needs
to be agreed between authorities and each provider.
As prescribed in the SMM, the safety risk manage-
ment process (in the SMM, the term safety risk man-
agement is meant to differentiate this function from the
management of financial risk, legal risk, economic risk
and so forth) is essential and the core functionalities are
to: describe the process (operational process description),
identify hazards (hazard list), make risk analysis (risk
analysis methods), assess risk (risk matrix or the like)
and finally take action for required risk mitigations
(reduce risk). The last step of the safety risk management
is risk mitigation which ultimately should improve the
system. If this intended change is successful, the process
demonstrates safety performance and a full cycle of
change. Hence, safety performance depends on change
capability. However, it has been shown in earlier aviation
research (McDonald 1999, 2001; McDonald et al. 2000)
that a full cycle of change is rarely demonstrated. Risk
management often ends with risk assessment and sug-
gested recommendations from safety and quality area
representatives for each production area or from the
safety department. The risk mitigation strategy is decided
by higher level area management and short-term or
simple solutions are sometimes prioritised over complex
analysis and long-term benefits.
One new requirement is to have a Safety Review
Committee (SRC). The SRC function is to ensure resource
allocation and assessment of the effectiveness and effi-
ciency of risk mitigation strategies (ibid). In this group,
strategic decision-making is made. So aligned with an
integrated management approach, information on safety
risk needs to be integrated and presented as one common
picture of the safety performance. This is argued to facil-
itate a holistic view and systemic strategic decision-making
(ibid.).
It is recommended to have a safety information system
supporting the SRC. The available risk data need to be
analysed in order to produce relevant and accurate safety
information. The following risk analysis methods have
been suggested to enhance the functionality of using indi-
cators in safety performance improvement according to the
SMM (ibid.): statistical analysis, trend analysis, normative
comparisons, simulation and testing, expert panel and cost-
benefit analysis.
221
1.2.2 Integrated management
The previous generation of safety regulation was based on
the requirement for an independent quality or safety system
with separate accountability to the manager accountable for
the operational organisation. The new SMS is supposed to
become the management system, integrated with other
dominant management structures and processes. In a brief
from the Civil Aviation Authority (CAA), the SMS was
changed to MS only (EU-Ops 1.035 and 1.037), and the
transition from a Quality Management System simply to a
Management System. There are a number of clear benefits
of an integrated management listed in the SMM (ICAO
2012):
• reduction of duplication and therefore of costs
• reduction of overall organisational risks and an increase
in profitability
• balance of potentially conflicting objectives
• elimination of potentially conflicting responsibilities
and relationships
There are two novel approaches in ICAO’s latest ini-
tiative. The first is the integration of the management of
safety into the overall management system of the organi-
sation, and second that this should be a performance-based
regulation, capable of demonstrating its implementation
and effectiveness in terms of measurable operational out-
comes, which are related to safety. This creates a paradox
for regulation. It has to prescribe what has to be done to
assure that safety is properly addressed. But, actually
assuring this requires a broader focus than just safety-
related functions. Since safety performance depends on the
integrated performance of the whole system, regulators
cannot prescribe what is beyond their remit, which relates
to those aspects that are (specifically) related to the safety
of the system (Ulfvengren et al. 2009).
The envisioned regulatory SMS is still clearly advo-
cating an integrated approach (ICAO 2008). Hence, the
aviation industry needs to manage system change with
maintained safety, including aspects of risk and perfor-
mance in an integrated management system. This requires
high level safety knowledge and understanding of how
safety is produced by the overall complex socio-technical
system (Rasmussen et al. 1994). Understanding how a
socio-technical system works is the key to manage it more
effectively. But this understanding is something that safety
research is still struggling with: What builds and improve
safety? How can safety be measured? What does safety
cost? An indication of this may be that ICAO has published
three versions of the SMM (2006, 2008, 2012). It has taken
years only to develop a manual, describing only the theory.
123
222
The theory is based on traditional Human Factors (HF)
models. Most traditional HF models, like the SHELL
model (Hawkins 1972; Hawkins and Orlady 1993), focus
on the individual operator and identification of what affects
the human performance that may contribute to human
errors (Norman 1993; Reason 1990; Wickens et al. 2004).
However, none of the traditional approaches is adequate to
address complex service operations like aviation and what
really impacts on system safety. As argued earlier, a new
approach is needed to move beyond the current safety
plateau.
The logic of the new regulation for an SMS (ICAO
2012) has been derived from systems engineering and
quality management (Stolzer et al. 2008). To apply the
quality management approach and process control to safety
management may seem logical, but it is an oversimplifi-
cation in many ways. In practice, the integration between
quality and safety is a real challenge, not only for every
airline in the world, but also for researchers in this field.
Quality models are based on manufacturing industry, and
most aviation processes are service production. Quality
models are based on high failure rate and lots of opera-
tional data whereas safety data are extremely rare in an
ultra-safe system. Safety models are loosely described and
lack description of the underlying system and process
mechanism. For any performance management, it is
essential to have capabilities for measuring and monitoring
the process performance, but safety does not lend itself
easily to traditional indicators or performance measures
compared to, i.e. manufacturing industry.
An indicator is an observable measure that provides
insights into a concept that is difficult to measure directly
(Harms-Ringdahl 2008), such as safety. It is very difficult
to develop new measures and indicators for safety or safety
performance (Mawdsley 2010; Rignér et al. 2009a, b).
Existing safety models do not support how to measure and
confirm safety (Reiman and Pietikäinen 2012). The
model(s) of safety in an organisation reflect the under-
standing of how safety emerges and will also determine
what indicators that are identified to be essential for
improved safety (Wreathall 2008). If an indicator is valid
or not, it has more to do with the causal model that is being
assumed than with the indicator per se (Hopkins 2008). If
relevant measures are not identified, the danger is that SPIs
will be monitoring that do not adequately reflects the state
of the system. Without a good safety model and numerous
data, it is doubtful whether measures will support antici-
pation and prevention of complex system failures.
1.3 Lean quality management in safety management
Performance-based management systems in other indus-
tries have been developed driven by the need to compete in
123
Cogn Tech Work (2015) 17:219–236
a continuously changing world long before aviation. Lean
production is a frequently used term for describing indus-
trial strategies with common underlying principles and
concepts such as TQM (Total Quality Management),
Concurrent Engineering, JIT (Just In Time) or Kaizen.
Along with these principles or concepts, there are tools and
methods, for example: Six Sigma, 5S and standards like
ISO 9000 (Bessant 2003; Ward and Brito 2007). One of the
key objectives is continuous improvement. Just as safety
management, the logic of quality, Lean production, ideas
behind continuous improvement and the tools are all easy
to understand but proven really difficult to implement
(Bessant 2003).
Lean includes local workplace improvement and per-
sonal safety, but it is not comprehensive for the demands of
an SMS and system safety (Rasmussen et al. 1994). Lean
holds many tools, processes and methods essential for
change. Common tools are the PDCA (Plan, Do, Check,
Act) wheel as a change process, continuous improvement,
people involvement, a visual scoreboard, process model-
ling and Ishikawa diagram to support problem identifica-
tion and solutions generation, etc. But, as has been argued,
Lean is not developed as a comprehensive approach for
aviation or other similar complex safety–critical socio-
technical systems where people play the key coordination
role in the process. Lean initiatives do not have safety or
human factors perspective (Womack and Jones 2003;
Womack et al. 1991). In this current research, the‘‘com-
pleteness’’ of Lean for sustainable system change in avia-
tion (and elsewhere?), and its lack of inclusion of Human
Factors and safety models is questioned.
In another EU-project called HILAS, there was an
industrial-based case study (Ward et al. 2010) where such
an approach was taken and integrated with a Lean initia-
tive. It was proven to be more effective as a Performance
Management initiative than a Lean approach alone. The
reason for this was argued to be that Lean analyses tend to
be weak in relation to people and safety functions in
complex systems. The most essential HILAS complement
to this Lean initiative was the implementation of the HI-
LAS developed OPM/KSM (operational process model/
knowledge space model) (Leva et al. 2011).
1.3.1 A process and system analysis approach to safety
performance
In earlier EU funded research (e.g. ADAMS, Human
Factors in Aircraft Dispatch and Maintenance, 1994–1999)
projects (a summary accounting for this evidence can be
found in McDonald 1999, 2001; McDonald et al. 2000),
four key issues were identified that raised serious questions
to the traditional Human Factors models and how they
dealt with safety: (1) Double standard describing the gap
Cogn Tech Work (2015) 17:219–236
between strategic management views of operations and the
real ‘‘normal’’ operations as performed by operators, (2)
WIPIDO, meaning Well Intended People In Dysfunctional
Organisations, describes how operators do their best to
overcome an actual or perceived lack of support in oper-
ations, (3) a strong professional culture seems to com-
pensate for weaknesses in the organisation and (4) cycles of
stability describes that even after failures, organisations do
not easily learn and change. It was shown that the reasons
for completing tasks were humans ability to coordinate:
tasks, dependencies between processes and process uncer-
tainty, and by coping and compensating for suboptimal
conditions. The operational process was central, and in
order to improve operations, a better understanding of an
improved process control was needed.
The whole idea with process control is to identify
undesired outcome, produced by the process, by measuring
its output and improve the process input until the desired
outcome is achieved. Modelling of the process is com-
monly used in industry to identify influencing mechanisms
and variables on the outcome. Hence, the first step in
gaining control over an organisation is to know and
understand the basic process for production (Juran 1988;
Deming 1986). In aviation, the production is delivered by a
complex socio-technical system. If safety is to be main-
tained, or even improved, relevant mechanisms influencing
these processes need to be identified and improvements
may be accomplished only by changing the processes’
functionality. Process modelling is the process of collecting
information about the activities that an organisation needs
to perform in order to achieve a certain goal (ibid.). It is
used to identify and define the activities and events that
constitute the production process and further to highlight
deficiencies and weaknesses in the flow of information and
resources, ‘‘as-is’’ in order to rectify these. The modelling
needs to be able to project to a future system, a ‘‘to-be’’
state. This enables current performance to be compared to
projected demands, benefits as well as anticipating poten-
tial risks in the new system design.
One main concept in MASCA is the System Change and
Operations Evaluation (SCOPE) process model which in a
generic but systemic manner describes process activities
and models their underlying causal logic (Bunderath et al.
2008; Morrison 2009; Leva et al. 2011). It is a tool which
allows an analyst to interrogate operational processes using
a series of questions designed to highlight the most
important relating to information, material and human
inputs. It describes the relationship between different tasks
in the process, the goals, contextual factors and key process
enablers. The core of the SCOPE model (Fig. 1) is the
ability of the operator to coordinate: tasks and resources,
task and process dependencies, as well as uncertainty in the
process.
223
Fig. 1 Schematic model of process analysis (McDonald 2014)
In relation to safety, another HILAS research effort
focused on the development of a prospective risk analysis.
It was developed for anticipating the effect of change and
to explore risk of operations in advance of the occurrence
of incidents so that the reduction in risk may be achieved
proactively (Cooke and McDonald 2009). In aviation, it is
essential for understanding both current and future risk in a
more proactive safety management. To explore risk, it was
suggested to involve the analysis of data along with the use
of the SCOPE process model for anticipating the effects or
outcomes of the operational processes. This combination of
a process model and data would also support a direct link
between the low level measures and the high level KPIs
(Key Performance Indicators) of the organisation. Data
would also need to flow between departments aligned with
interdependent operational processes from various pro-
duction areas, delivering value to a common operational
process. What this emphasises is the need for the integra-
tion of all performance data and the avoidance of data silos.
These components are believed to be underlying and clo-
sely linked to many of the core functionalities in the SMS
discussed. It is argued that a functional process analysis of
socio-technical systems like the SCOPE model process
analysis (Fig. 1) is required to enhance current SMS
functionalities.
Many of the discussed functionalities for safety man-
agement discussed so far depends on human and social
processes, i.e. like trust for information and data sharing
across departments, developing a common holistic risk
picture, data analysis used to produce risk information and
competence for understanding what produces safety. So,
123
224
Fig. 2 Schematic architecture of overall system analysis (McDonald
2014)
even though the model for process analysis (Fig. 1) illus-
trates relevant mechanisms in the process, it needs to be
complemented by other system components essential for
supporting underlying mechanisms for system change, see
more in McDonald (2014) in this issue. In MASCA, the
SCOPE process analysis tool and method was further
developed into an overall system analysis framework
(Fig. 2). The developed theory of change in MASCA
(McDonald 2014) argues that without addressing the full
scope of both technical system processes and human pro-
cesses in a systematic way, chances for successful system
change is not better than average (Dent and Goldberg 1999;
Kotter 1995). According to the MASCA, framework ana-
lysis of complex socio-technical processes needs to address
three complementary systems (Fig. 2): System 1 is the
functional processes (operational and management), Sys-
tem 2 is the knowledge and information system and System
3 is the social relations and cohesion (team and trust).
In MASCA, the Structured Enquiry (SE) has also been
developed to make an evaluation of change that encom-
passes the overall system in Fig. 2. This framework is part
of the overall SCOPE concept. The purpose of the SE is to
synthesise analysis from the three key system levels that
can derive a suggested strategy and recommendations
aligned with the organisations’ objectives. The framework
is structured around these seven concepts (McDonald 2014,
in this issue): the goal of the system change initiative,
operational processes, management processes, teams, trust,
information and knowledge cycles. The enquiry structure is
not a simple question and answer process. It is to be
applied to a dossier of knowledge built up about the ini-
tiative under evaluation at its current state of play in order
123
Cogn Tech Work (2015) 17:219–236
to draw inferences about current status and prospects for
the future. Such inferences should be powerful enough to
drive recommendations that are sufficiently cogent and
authoritative to seriously influence the plan for the next
stage of the initiative. The close link between system
change and safety performance makes such a framework
relevant not only for implementation of any system but,
especially, for further development of functionalities in a
SMS.
2 Methods
The approach taken in this research is to apply the new
system and process analysis (1) to further develop the core
functionalities of the airline’s SMS by applying the SCOPE
model and (2) to increase the chances for a successful
system change, by applying the SE to provide comple-
mentary recommendations to the proposed enhanced SMS
and company objectives.
This chapter presents the approaches adopted to ensure
an ongoing process of enquiry and participatory research
(i.e. action-based research) within the airline over a period
of 10 years. This approach allowed the researchers to
develop a rich picture narrative of the change as it evolved
that could be analysed together with other cases in the
MASCA project for developing a theory of change.
2.1 Action research
Service operators like airlines do not have research and
development departments like large design and manufac-
turing companies. Researchers in academia do not have
easy access to industries like airlines due to lack of internal
research activities or resources. Airlines also have reasons
for strong integrity, especially in relation to sensitive
information on safety issues. However, in order to combine
research and practice in this field, collaboration is required.
Collaborative research and development can provide an
effective innovation process. However, for this to work, the
collaboration has to deliver benefits to meet the short-term
operational or organisational goals of the participating
industry, as well as the medium to long-term research goals
of a research project. The challenge is to develop longer-
term collaboration between research institutes and opera-
tional organisations around more strategically formulated
goals. These are reasons for the methodological choice for
this case study. The approach adopted is action-based
research. Even though this paper primarily focuses on the
MASCA project, the foundations of the research/industry
collaboration was established at the commencement of the
HILAS project in 2005. In HILAS, a mix of researchers
and industrial partners became over time organised in
Cogn Tech Work (2015) 17:219–236 225

clusters, called HILAS centre of excellences. One core and one responsible for training of safety office. The last of
team of these constellations was the same airline safety the four interviews had a new role responsible for change
department and researchers from two universities. There- management in the airline.
fore, this case study is the culmination of almost 10 years
of research collaboration with this airline. In this, research
2.2.1 Workshops and observations
has been identifying challenges, developing interventions
to deal with those challenges (e.g. training, safety semi-
As part of the first phase of the project, two workshops
nars), providing scientific rigour to understanding some of
were conducted with the purpose of gathering information
those challenges and building up a working relationship
on ongoing change initiatives linked to the development
based on mutual trust and respect. An excellent relationship
and implementation of the SMS. One was with the Human
of research and industry collaboration was established over
Factors project group, with representatives from all pro-
the project where the lead researcher developed a role of
duction areas and safety and training. The need for a
mentor and advisor to the safety management team (e.g. by
common human factors model was discussed as well as
taking part in critical meetings, facilitating high level
the need for particular human factors theory and appli-
strategic safety seminars, undertaking joint presentation at
cation directed to the various production areas. The other
industry and academic conferences). In time, the long
was an initial meeting with the safety office group
collaboration led to relations in the team that allowed
responsible for working with and developing various SMS
informal discussions to take place on a needs basis, by
functionalities and supporting technologies. MASCA
email, phone, on Skype or face to face. This type of rela-
researchers received access to review a range of safety
tionship is of vital importance particularly when working in
documents and safety reports and were given training on
the domain of safety management and risk. Due to the
and direct access to the IT-support under development.
nature of this methodological approach which has been
Field studies consisted of using the new IT, studying
ongoing and evolving, it is beyond the scope of this paper
documents with descriptions of data and risk index
to highlight all of the methods and interventions under-
structures. Enquiries were made in discussions with the
taken. Therefore, the following provides an overview of the
developers of the IT-system, as well as with the people
key methodological approaches applied as part of the
responsible at the safety office. These activities made it
MASCA project (2010–2013).
possible to observe how new processes were developed
and how different users slowly gained knowledge on the
2.2 Interventions
development that had taken place. This provided the
researchers with an invaluable insight into everyday safety
In this MASCA case, a total of 34 interviews were con-
and risk management as well as organisational processes
ducted with key staff (some repeated) from operational
linked to this tool. The second half of the project included
level staff (10), to middle management (12) through to
a series of ten work meetings and/or workshops with the
senior management (8). The objectives of the various semi-
particular focus on SPI-driven change, integrated risk,
structured interviews changed as the research evolved. In
both linking the airlines SMS development to the need for
the first phase, 17 interviews were conducted in the airline
a process model to identify a common operational process
with top managers for safety, quality and operations
delivering outcome of relevance to flight safety and
development, as well as nominated persons (NP) for
identifying SPIs.
ground, technical, flight and security operations. An initial
MASCA template was designed with the basic concept of • Annual report
change management focusing primarily on the airlines • Organisational Charts
overall strategic approach, the process of change, compe- • Manuals: Airline Operation System (AOS), Airline
tence and human resources. Special interest was how to Operation Procedures (AOP)
manage change due to external demands such as regula- • List of responsibilities, standards regulatory require-
tions. This initial phase provided clarification of key stra- ments, safety programme
tegic goals, current change initiatives in the airline, lessons • Lean description/workshop material
learnt from previous change initiatives and readiness for • Airline operational concept
change. Another four interviews were directly focused on • Leader seminar presentations including SPI/KPI’s,
change management and the Lean initiative for under- vision, strategy, Goals.
standing how the Lean structure within the airline could be • Human Factors group project final report
integrated with an SMS. Three specially trained and • Training material
selected people, so called Lean navigators were inter- • ICAO manuals
viewed, one from ground, one from technical operations • EASA-Ops regulatory documents

123
226
2.2.2 Safety management seminars series, the MASCA
approach
Four seminars were held around each of the four pillars of
the SMS as described by ICAO (2012). In the first seminar
on Safety Policy, the focus was an integrated management
system, safety production models and the MASCA new
approach to address Human Factors issues. The second
seminar was more generally on Safety Risk Management
and included state of art from the MASCA on process
descriptions relation to hazards identification, SPIs, reac-
tive versus proactive, contributory factors and antecedents
were discussed. The third seminar on Safety Assurance
focused on the MASCA change management system and
the proposed process view of the airline’s SMS as a full
cycle of change. In the final seminar, Safety promotion was
addressed including safety culture, training and commu-
nication. The safety seminars as such were discussed as
part of the SMS training required, and the process views
was proposed as the way of visualising the process and
identify each and everyone’s role in this process for
understanding and supporting implementation of SMS.
These safety seminars provided a very useful platform for
management to further explore a more integrated approach
to SMS.
2.2.3 Training the trainers for ground safety
In a later phase, a particular initiative on safety training for
ground operations was performed. In preparation, six
interviews consisting of three ground handlers and three
ground operations managers were conducted. The purpose
of the interviews was to gain domain knowledge on ground
operations processes and risk to understand the particular
needs for safety training in this context. MASCA
researchers then participated in the development of a
ground handling safety training. The development of the
training programme was an iterative process of develop-
ment and implementation involving MASCA researchers
and safety and training personnel from the airline. Access
was given to previous safety trainings for ground opera-
tions, and the objective was to integrate this with the the-
oretical and conceptual framework from MASCA and state
of art knowledge on Human Factors and SMS as well as the
operational requirements from the ground handling section
of the airline. The key objective of the training programme
for the ground handling section of the airline was to
improve airside safety by supporting a collaborative and
‘‘common’’ safety mindset between the staff and manage-
ment. From the outset, the most important learning objec-
tive that the training focused on was—ensuring that
everyone needs to work together—safety challenges are
not the responsibility only of quality, front-line staff or
123
Cogn Tech Work (2015) 17:219–236
management; they are everyone’s responsibility and can
only be addressed by working together. The final training
resulted in a 2-h programme that was highly interactive and
guided the trainees (operational management) to firstly
look at the current safety and human factor related chal-
lenges they face and how to work together with the com-
pany to improve safety operations.
2.3 Narratives and structured enquiry
The overall purpose of the vast information collected
during the interventions was to understand both the change
initiatives and to identify factors that may have facilitated
success and to document past and current experiences of
change, how things happen. Stories were written iteratively
along the project. The first narrative, in MASCA, was
written halfway into the project using a first version of a
change evaluation framework. This task was based on the
researchers’ experiences so far. This was also validated
with four extra complementary interviews. These were
conducted with the head of safety and head of quality and
two persons in the safety office which were main contacts
in the airline during the project. The interview discussed
the MASCA project and change cases progression
according to a simple template for a story builder, a nar-
rative. The second version of the narrative updated the
initial one of latest progress in the company developments
but also most recent MASCA findings and current devel-
opment of the evaluation framework. The SE was used as a
template and the narratives already documented as well as
current understanding of the change case formed the bases
for the evaluation.
3 The airline case study
The airline has followed the guidelines from EASA/ICAO
and spent years in preparation and has a compliant SMS
and is in an ongoing implementation phase. The airline is
known to have a generally flat hierarchy and just culture
which also supports the idea of an SMS. For the airline, an
essential part of their progress has been their ability to
continuously develop and implement in-house systems and
apply contributions from research projects like HILAS and
MASCA. This became an iterative process which addres-
sed the challenges in ongoing industrial and theoretical
developments as well as academic support to these chal-
lenges. Many of the tools and organisational structures
required for SMS functions exist in the airline today as a
result of many years development. Below follows a
description of the various change initiatives, mainly during
MASCA, in which researchers participated and studied to
learn about change.
Cogn Tech Work (2015) 17:219–236
3.1 In-house change initiatives for developing an SMS
Within the course of MASCA, the airline’s SMS devel-
opments have been reviewed, and the overall logic has
improved where applicable, based on the theoretical
framework of consolidated knowledge from earlier
research. When the MASCA project commenced in 2010,
the airline had just introduced a prototype indicator
matrix for integrated safety that was developed as part of
the previous research project HILAS (2005–2009). This
established an initiative to use data in an integrated way
both across the different production areas, but also
between safety and quality. The airline started a simple
development of a new safety information system, con-
sisting of 13 files of data from various sources. The air-
line’s existing IT-systems were not suited for other
analysis purposes than for the specific use it had been
developed for originally. Technology was fragmented due
to data set characteristics, industrial domain and various
production areas. This meant that, in addition to numerous
data sources, there were several different platforms and
data formats in the company. This made it time-con-
suming to collect a sample data for analysis (Rignér et al.
2009a).
3.1.1 Safety information system: VMA
The excel files soon became a prototype for the new safety
information system. The airline teamed up with Vision
Monitor and developed an aviation application, VMA. The
development of VMA, IT-tool, was not only providing a
solution to the data management problem but also allowed
for data management originating from a large number of
different sources throughout an organisation to deliver real-
time data and safety information for management. It pro-
vided a dashboard view of current and historical safety
performance data. The data and underlying systems are the
same as before but VMA enabled communication by
visualisation of a common picture for managers from dif-
ferent departments. The more holistic picture of risk in
operations is presented using a spider diagram for both the
whole airline’s and for each production area’s SPIs and risk
indexes. The trends of each indicator and it target values
are also available. The tool fulfils many of the requirements
for a safety information system described in the SMM
(ibid.). It has a user-friendly interface for data entry and
query; it monitors and measures safety trends or perfor-
mance to ascertain latent factors underlying safety defi-
ciencies; it provides the capability of transforming large
amounts of safety data into useful information that supports
decision-making both for strategic decisions as well as for
operations management.
227
3.1.2 Safety review committee: FSQB
The VMA application was developed to support both the
safety department but also operations management. The
airline implemented an organisational structure compliant
to the safety review committee, SRC (ICAO 2012), called
the Flight Safety Quality Board (FSQB). The objective was
to initiate integration at a management level for strategic
decisions based on common risk information. This is a step
towards an integrated management system approach. The
functionality of the FSQB was clearly enabled by the
VMA. The FSQB is a group consisting of the head of
safety and the top managers from the main production
areas: Flight Operations, Technical Operations, Ground
Operations and Security. They meet every 4 weeks to
discuss safety issues. This is supported by a bottom-up
process for safety information aggregation, from data
sources and up through Safety Action Group (SAG) and to
the FSQB. With this integrating process, the SPI’s and Key
Performance Indicators (KPI’s) from the various depart-
ments are openly presented and discussed.
3.1.3 How VMA and FSQB developed into improved
functionality
Initially, the system was introduced during a financial crisis
in the company. This put additional pressure and stress on
the managers. A strong and trusted position of the head of
safety had fostered enough support in quite influential
group to get the development approved. The SMS devel-
opments were initially considered sensitive, in example
regarding sharing data and visualising data openly in
FSQB. This group was initially not open to new ideas or
critiques from others with respect to comparing operations
and risk information. Despite the Lean introduction, the
idea of continuous improvement at a management level and
management processes were not considered analogue to
Lean initiatives in their own operations. This has been
critiqued and seems like very un-Lean thinking. SMS goals
for an integrated management do not appear to be shared
by all top managers and commercial units. SMS is still
much the safety office system. There was little under-
standing of the benefits to show ‘‘my data’’ to others.
Managers protected their area and workers from any
questioning of ‘‘outsiders’’. One contributing factor was
that the tool initially did not fit each production area’s need
for indicators equally well. For example, Flight operations
had more than 40 indicators for safety and ground had only
5. The developments was like following a moving target of
IT-developments, learning from users and increased
understanding of SMS functionalities as it took form. But
as the VMA improved and the use of it increased, the
stakeholders understanding grew, and with this, their desire
123
228
Fig. 3 Safety risk management process of the SMS visualised as a
full cycle of change required for demonstrating safety perfor-
mance and sustainable change
for further developments and adaptation for their particular
needs also grew. Involvement and suggestions for
improvements increased. This gave highly relevant new
design requirements for, i.e. processes supporting new
functionalities. The IT-development enabled the develop-
ment of SMS core functionalities. An example is that
Ground Operation today has 15 indicators linked to their
operations.
3.2 SCOPE applied to SMS core functionalities
The general core functionalities in an SMS should follow
the cycle of change (Fig. 3): to describe the process
(operational process description), identify hazards (hazard
list), make risk analysis (risk analysis methods), assess risk
(risk matrix or the like) and finally take action for required
risk mitigations (reduce risk). The last step of the safety
risk management is risk mitigation which ultimately should
improve the system. Delimitation was that risk analysis and
risk assessment methods were not part of this case. The
airline uses Airline Risk Management Solutions (ARMS)
as their methodology for risk assessment. Focus related to
risk management has been on performance management
and SPIs.
3.2.1 SCOPE for system description
As a first step, in a risk management process is to under-
stand the process delivering safety-related outcomes. In the
SMM, this core functionality is called a system description.
SCOPE process model makes it possible to map the current
situation (AS-IS) and identify the weaknesses as well as to
propose, in the model, improvements to the process. When
this future system (TO-BE) is modelled, it allows for
123
Cogn Tech Work (2015) 17:219–236
anticipating risk due to the suggested change in operations.
Even a simple simulation or process walk-through may
reveal unanticipated knock-on effects like conflicting tasks
or competing resources.
3.2.2 SCOPE model applied for hazard identification
and indicator development for VMA
SCOPE may add value to SMS functionalities such as
hazard identification and safety performance indicator
development. The development of indicators needs to be
derived from an understanding of the underlying mecha-
nisms influencing the functionality of the process deliver-
ing the desired outcome. A performance measure such as
an indicator is logically an outcome from some related
process. SCOPE modelling and analyses of these processes
may support identifying high risk processes and possible
indirect measures of safety indicators. When changes are
made in processes, then SCOPE modelling will support
identification of potential bottlenecks and identification of
influencing factors that should be monitored to make sure
the change is an actual improvement of the process as well
as to monitor the effective new process. SCOPE ideally
will support validation of existing SPIs/KPIs and devel-
opment of new indicators as well as defining the logic
behind existing SPIs and KPIs. When need for improve-
ment is identified using these indicators, the root cause
analysis may also start from the common understanding of
partly shared processes as well as anticipating possible
contributors to the problem. When solutions are designed,
these changes may be introduced in the model and used as
a proactive anticipation of knock-on effects and propaga-
tion of these in the system and distributed across various
production areas’ processes. Indicators need to be inter-
preted with people that have expertise in the real processes
and that understand what may have influenced the indica-
tor. Therefore, stakeholders need to collaborate in cases of
identified need of change and participate in the modelling.
Dependencies of risk and causal links will become trans-
parent in a collaborative learning effort. This integrative
approach using SCOPE would also enhance the integrating
initiative and functionality of the already implemented
FSQB and their use of VMA.
The main airline’s operations are Flight Operations,
Technical Operations, Ground Operations and Security.
These operations are interdependent for the overall safety
production process and outcome within the airline. Ways to
combine safety risk after identified dependencies between
production areas as well as various safety production
models for person and system safety need to be developed.
To understand, the respective logic has been discussed as
fundamental to all process control. For improved aviation
safety, it is the operational processes that need to deliver
Cogn Tech Work (2015) 17:219–236
desired outcome. The main production areas have different
logic, and main influences that affect risk due to the dif-
ferences in operations globally. This emphasises the need
for integrating performance data from various production
areas with a common operational process model in which
propagation of risk from one area to another could be
analysed.
3.2.3 SCOPE integrated with Lean for safety
The airline in this case study is today Europe’s most
punctual airline and a very good example for others in the
airline business with their many safety activities. But at the
same time, they are struggling for survival and were at one
time literally only hours from filing bankruptcy. One of the
airlines core business strategies to survive has been to
implement ‘‘Lean’’ quality management company wide,
primarily as a cost-cutting measure. It is anticipated that
when Lean has been fully implemented, it will also develop
into a continuous improvement culture. At the same time,
the safety department developed and implemented tools
and organisational structures, for managing safety perfor-
mance. The airline’s SMS is directed towards safety per-
sonnel and senior managers of the main production areas
and the safety office. The Lean strategy has been imple-
mented top-down across the whole organisation. They were
developed in parallel which means that the Lean initiative
is not part of the safety initiative, and the Lean initiative is
not concerned with system safety.
Lean includes local workplace improvement and per-
sonal safety, but it is not comprehensive for the demands
of an SMS and system safety (Rasmussen et al. 1994). It
is argued, in this paper, that an integration of Lean (pro-
cess-oriented approach) and Human Factors (human-ori-
ented approach) could provide aviation stakeholders with
a better management approach, which would achieve the
dual objectives of operational effectiveness and flight
safety. To keep balance between protection and produc-
tion, the airline should be able to manage Lean and safe
performance in an integrated system. This research sug-
gests an integration of Safety and Lean to address system
safety performance. Mediating models and methodologies
still need to be developed. A first step for integrating
Safety and Lean could be to apply SCOPE modelling
instead of existing process maps and waste reduction
strategy towards safety performance. Lean structures and
processes could be applied to general change management
in the processes.
3.2.4 SPI action process
Within the VMA, there is a functionality that triggers a
change process. The current change process is to delegate
229
to area managers to solve it as they see appropriate. The
SRC, here FSQB, meet regularly and discuss risk infor-
mation available. The role of the FSQB is to ensure
resource allocation and assessment of the effectiveness and
efficiency of risk mitigation strategies. In this group, stra-
tegic decision-making is made. It is part of manager’s task
to be able to ‘‘go fix it’’ and work strategically for system
and operational improvements in their production area.
This was true before SMS and before Lean. However, there
is still an outspoken industrial need for a systematic change
process, responding to strategic decisions and SPIs. One
early initiative in MASCA was to support a structure and
process that may be triggered by an SPI exceedance, when
trends and boundaries being routinely monitored are
observed to go beyond acceptable levels. This initiative
included a mix of applying both competence of the airline
Lean process modelling and the MASCA SCOPE
approach. This resulted in the SPI action process and is
linked to the functionality and use of the VMA by the
FSQB and other managers. The logic of the SPI action
process linked to VMA is in short:
1. Identify an owner for each SPI for all production areas.
2. Each SPI owner then need to understand the process
producing the SPI and identify factors affecting the
process as well as a SPI process timeframe.
3. SPI owners identify levels of data ‘‘upstream’’ for
antecedents for each main process.
The process is linked to a VMA email service when the
SPI action process needs to activated. A remaining and
urgent issue is to develop an improvement process for each
SPI so that SPI’s actually may drive change.
3.2.5 Challenges to support strategic decision-making
The previous SPI action process is triggered when an SPI
exceeds a threshold. This needs to be developed further
given the risk that managers run on everything and that the
FSQB has no way of determining what is urgent and what
is not. Despite domain knowledge and understanding of
one’s production area and possibly enhanced understanding
of others with integrating initiatives like VMA, it is not
always sufficient for strategic decisions. The availability of
rich and a large amount of data is a prerequisite for data
analysis. Even if the amount of data is rich, the analysis of
collected data may still not be utilised to its full potential.
By allowing know-how from operations and from domain
knowledgeable managers in FSQB in using SCOPE mod-
elling in combination with VMA, much of these approa-
ches may be applied. The challenge here is related to the
very few risk data produced. It is difficult to get significant
results. This could be resolved by using normal operational
data, directly from flight data monitoring as well as other
123
230 Cogn Tech Work (2015) 17:219–236

operations performance systems, not necessarily linked to

risk data. With support from the SCOPE process model,
potential links between normal operations data and risk
data may be identified. In the SCOPE supported indicator
development process, it would be possible to identify also
antecedent data that could be more frequent and numerous
as well as more proactive. If this was resolved and VMA
got the statistical functionality to give information on
significant trends, declining and improved, it would greatly
enhance the performance management aspect of safety
performance and also for regulators to assess if targets have
been met or not.

3.3 A safety management process delivering safety

performance

The main functionality of the SMS is to demonstrate safety
performance. The SMS primarily requires a risk manage-
ment and safety assurance process. The safety assurance
process includes safety performance measurement and
monitoring, change management and continuous
improvement of the SMS. The risk management process
includes description of the process, hazards identification,
risk analysis, risk assessment and finally action for required
risk mitigations. The safety assurance requirement inte-
grates performance management and change management.
In order to review and analyse this functionality and
identify gaps, the core functionalities and recent develop-
ment is mapped out as a performance management
framework. It may be described in terms of both a struc-
tural and a procedural part (Folan and Browne 2005). The
proposed framework is a consolidation of existing func-
tionalities in the airline as well as parts developed within
MASCA using SCOPE. The structural part, with a typo-
logical performance measurement framework, includes
organisational structures. The procedural part is described
with a step-by-step process. Aligned with the SCOPE
approach and the notion that also an SMS has a function,
the procedural part will be visualised as a full cycle of
change (Fig. 3) and represents a functional process that
delivers safety performance in operations.
The structural part of the performance management
framework includes:
• Enabling technologies
• Data collection tools, VMA data management tool,
risk assessment tools and models, statistical models
and SCOPE process modelling tool.
• Organisational structures
• FSQB, SAG, SPI action process, SCOPE SPI process.
• Data and information structures
Fig. 4 A simple representation of a Safe-Lean performance man-
agement process. Strategic decision making in the Safety Review
Committee (SRC) is enabled by a Safety Information System, Lean
and SCOPE model for system and process analysis. These are
together supporting a Risk information production (1–4) as well as a
System change and evaluation process (5–8)
• List of indicators, targets, risk index, definitions,
logic and risk models, policies.
The procedural part builds on the safety risk manage-
ment process. The overall process (Fig. 4) integrates a SRC
(FSQB), a safety information system (VMA), SCOPE and
Lean to support the Integrated Safety Management process
from step 1–8. The process may be divided into two main
functionalities for safety performance which are the Risk
information production and the Change and evaluation
process.
3.3.1 Risk information production
1. Data is collected and risk assessed (operations and
systems data sources, safety personnel in production
areas and safety department).
2. Data is aggregated to appropriate level of decision-
making (various levels available in VMA).
3. Strategic decision-making using VMA in SRC/SAG.
Allocation of resources and appointments of responsi-
ble group.
4. Data is disaggregated downstreams in an SPI action
process and as feedback to operations (use of an
integrated system consisting of VMA and SCOPE.
3.3.2 System change and evaluation process
5. An integrated safe-Lean initiative using SCOPE
modelling to analyse process as well as to anticipate
effects of change including Lean initiatives.

123
Cogn Tech Work (2015) 17:219–236
6. Identify solutions and prepare change implementation.
7. Coordinate and evaluate change with other ongoing
change initiatives.
8. Evaluation of change and continuous improvement of
the performance management framework including
revising and developing SPIs in VMA using SCOPE.
3.4 Structured enquiry of the development
and implementation of SMS
This section provides a summary of the most pertinent out-
comes of the SE applied to the ongoing and suggested
development of the SMS with respect to what has been
learned in the case study. Much of the background in this
Enquiry has been reported on here already, Sects. 3.1–3.3.
The framework is structured around the seven concepts: the
goal of the system change initiative, operational processes,
management processes, teams, trust, information and
knowledge. The findings have been structured under these
same areas.
3.4.1 Goals with Safety Management System
A non-negotiable goal for the airline was to be compliant to
the new regulation, EASA-OPS, by fulfilling the compli-
ance check list (CCL) for an SMS by October 2014. The
meaning of compliance may be discussed. At this early
stage of implementation of the new regulation, the
approval of new AOC for providers is based on submitted
SMS documentation according to the compliance check
list, CCL. The CCL is not the same as proven safety per-
formance as discussed in this research nor in the SMM
(ICAO 2012). Regulators and providers have not yet
agreed on exactly what to assess in terms of ‘‘safety per-
formance’’ (i.e. indicators, targets and requirements) the
effectiveness of the SMS meaning how well it performs
and manages to improve the system or when this is to be
done. It is reasonable to believe that the long-term goal
from both regulators and providers are to reach the func-
tional objectives of the SMS which ultimately is to improve
safety performance in airlines and other providers.
The airline needs to take seriously the fact that the CCL
may not be sufficient in the long run and develop an
integrated management system that may demonstrate
safety performance when operationalised. If not enough
resources are put into the implementation of the SMS, it
will be an isolated system for the safety department with
little engagement form others in the organisation. This will
not develop into a fully functioning integrated management
system. In addition, regulators seem to lag behind in the
airlines SMS development. If there is a gap between the
understanding of compliance and what will be assessed at
231
next regulatory, audit regulators may have to delay audit,
or even worse, to lower requirements for compliance when
it comes to a defining a fully functioning SMS.
Recommendations for change are to: discuss priority
with top managers in relation to what expectations they and
regulators may have on the future SMS; prioritise and
apply sufficient resources for development of a training
concept and conduct SMS training company wide; develop
a plan to reach that expectations and goal; continue to
develop an integrated management process that is capable
to demonstrate safety performance.
3.4.2 Operational processes
There are three main production processes, in relation to
operational safety performance: Flight Operations, Tech-
nical Operations (Part-145 and Part-M) and Ground
Operations. Since all areas contribute both to their own
process quality and also share risk contribution to the
overall flight safety, the integrated risk need to be under-
stood in addition to the particular risk in a single produc-
tion area. A common weakness in many organisations is
that different production areas work in ‘‘silos’’. This may
foster perceived conflicting interests and cultural differ-
ences between ‘‘us and them’’. In aviation, this is more than
a cultural aspect since the main production areas contribute
to the overall flight safety. In addition, if the relative
contributions from the different areas are not understood, it
will also be difficult to prioritise resources for safety.
Recommendations for change:
• Support each production area and its operators by
providing well-functioning processes, i.e. conditions
for coordinating tasks.
• Develop appropriate human factors and safety logic as
well as process understanding for each production area,
if relevant.
• Develop safety training and SMS training for enabling
participation in process improvement and safety
performance.
• Develop a process model representing common oper-
ational processes, describing interdependent opera-
tional processes from different operations for common
understanding.
• Participate in process modelling to support the SMS
functions for understanding shared and propagated risk.
3.4.3 Management processes
The whole case of the SMS is management process
development and implementation. SMS is a management
system and intended to become ‘‘the’’ Management System
(MS) (ICAO 2012). To work in silos may be even more
123
232
common in management than in operations, since opera-
tions actually need to work together to get the job done.
Management must have their own strategy and resources
and perceive conflicting interests which may duplicate
costs and sub-optimise an overall goal. In the SRC using
the VMA, there has been an improvement of integrating
people and data from various production areas. However,
there is still no common operational concept (operational
process description) representing integrated risk. Interde-
pendencies between processes and intersections of pro-
cesses are not fully understood. There is a risk that the new
VMA system has little impact and that indicators will be
monitored as now, but when it becomes required to
improve one or the other, there is going to be local solu-
tions within production areas without a clear understanding
of how the system change will affect other areas where the
operational processes involved interact. There will be no
effective drive from indicators to improve the overall
operations successfully.
Since Lean was implemented, there is support for pro-
cess improvement in operations. However, no formal
change process linked to the particularities of safety per-
formance has been implemented. It is argued in this
research that Lean is not encompassing system safety nor
have sufficient process model methodologies for complex
socio-technical systems. There has been no integration of
Lean and safety management other than at local safety
level. There is also a risk that Lean management is driving
change rather than safety management.
The integrated management processes suggested in
MASCA is recommended. It has a shared critical path of a
full cycle of change. This full cycle of change may be
described as a 8-step process (see Fig. 4). This process is
designed and centred around the risk information produc-
tion unit of the flight safety quality board (SRC) using
Vision Monitor and SCOPE methodology for a common
operational concept. It supports integration of Lean and
Safety where appropriate. To strengthen the strategic
decisions and capability to prioritise among needs for
improvement, VMA should also be upgraded with statis-
tical analysis capability (significant trends, variance and
correlations).
Another recommendation is to further develop a change
process that links SPIs to process and teams to deal with
this from process analysis and solution identification as
well as anticipating the process after making changes and
finally evaluation of the change implemented building on
Lean training they have and complement with safety
management and human factors.
Finally, it is recommended to develop support and
training for managers and teams involved in taking actions
after an SPI has exceeded a threshold. Training should
contain elements of both risk management, change
123
Cogn Tech Work (2015) 17:219–236
management and performance management in relation to
safety.
3.4.4 Team structure
There are various groups and teams linked to the organi-
sational structure and role. The safety department and
production areas have different roles and have both divided
and shared reports and risk analysis tasks responsibilities.
The safety office is a focused group that works towards all
national bases and operations. Nominated persons see the
whole operations, not only safety information. Nominated
persons often delegate the actions that need to be taken
when an SPI exceeds a limit. Operators at various levels
and in the different areas are experts groups of their
operations.
If the SMS is to function as a whole, the roles in the
SMS need to be clear. Implementing an SMS will change
roles and accountabilities and stretch knowledge bases. For
example, there will be new relationships between safety,
production areas, and change initiatives like Lean, new
accountabilities for outcomes of change initiatives, and
new know-how to manage change process.
The safety office is a team and the SRC has an estab-
lished meeting structure of accountable persons. External
experts are invited openly, i.e. MASCA researchers. The
airline has a flat organisation, and the SMS requires
intensive transverse coordination. There is no team struc-
ture per indicator standing ready to act on its related safety
information. The existing groups of people have potential
to become functional (mix) teams in an SMS with new
constellations.
Potential risk or impact is that if people do not have an
understanding of their role in the SMS or even lack a sense
of the purpose and benefits of the SMS, it will be very
challenging to get people to work together to solve issues
that easily falls outside the shared or integrated functions.
Functions and roles are evident, but cross department teams
have not been clearly defined in the SMS. Without clear
teams, there is always risk for duplication of work that goes
on in parallel. In safety critical systems, the lack of func-
tional teams may also contribute to less communication or
learning which may block safety information.
It is recommended that each person in the SMS full
cycle of change should be mapped in the 1–8 steps to
identify their role and also identify who should be in teams
to deliver service from one step to another. The mapping
would also support analysis of where there are people
missing due to either lack of person, competence or
information. With this mapping and gap analysis, effective
transverse team structure could be formed for safety per-
formance and change in the SMS. This mapping of people
and teams along critical paths is as relevant in the
Cogn Tech Work (2015) 17:219–236
integrated management as well as in shared or interde-
pendent operations.
3.4.5 Trust
Relations between departments were at a low point during
near bankruptcy when there was, for example, fear over
losing jobs and reduction in funding. The airline’s ambi-
tious change management programme required cuts for
many levels of staff, but yet a few managers still received
their bonuses. This resulted in a climate of mistrust among
operational staff. Most people in the airline have known
each other and worked alongside each other for quite a long
time. There are several nationalities in the airline that
sometimes will have very different cultures which affect
trust. The head of safety that introduced SMS was very
trusted and known to make things happen. As reported in
the development of the FSQB and introduction of VMA,
trust has not always been strong.
There is a risk that people central to the SMS process
may not buy into the need for further development and
commitment in order to implement SMS successfully and
efficiently. This may affect other essential mechanisms: To
share risk information is essential for identifying systemic
risk. To collaborate is essential to come up with holistic
solutions. To allow weaknesses to show is the only way to
support improvement with relevant stakeholders to mini-
mise unanticipated risk following a weaker local change.
Training could reduce competence gaps and increase a
common understanding and buy-in. Training should consist
of both understanding operational and management pro-
cesses and their relation to safety performance. This may
include explicit training on safety models and SMS as a full
cycle of change, i.e. step 1–8 and try to agree upon the
process. It is recommended to continue to allow influence
from production areas to form their VMA and SPIs so they
are working in their environment so that benefits and good
examples increase. It has been shown that as the VMA
usage mature, it is also being questioned and users come up
with ideas for further adaptation and improvements. It is
also recommended to continue to improve the SAG and
SRC functions when it comes to apply decisions into
operational improvements.
3.4.6 Information cycle
The new VMA provides comprehensive risk information
in near real time, compared to common quarterly reports.
For some data, it may take up to 2–3 days to get risk
assessments of reports done and uploaded to VMA. Via
VMA there is access to all non-normal data from the
production areas. VMA aggregates data to give overall
risk visualisation. All nominated persons have full access
233
to VMA. VMA allows all in SAG and SRC to share a
common view of risk and discuss openly their relations,
priority and resource allocation. There is an ongoing
iterative user centred development of the VMA, especially
revising SPIs used in the application (to match the oper-
ations considered relevant in relation to production area’s
risk and flight safety). VMA may collect data from any
variety of data sources for further improvements. VMA
will deliver information that directs attention to one or
several areas with an SPI exceeding a limit or has a rapid
change rate.
There is no formal process developed for validating the
existing SPIs or their targets as well as indexes. If the
airline risk index is very constant, is it good or bad? To
control for business driven change, commercial and safety
data should be analysed jointly. How should Lean KPIs be
compared or prioritised with SPIs? What is Lean enough in
relation to safety? VMA is available to all airline person-
nel, but the information produced is so far only used by
VMA developers and FSQB. For example, the ground
operations safety office representatives are not yet using
VMA. They report events from operations independently to
CAA. There is still room for improvement in the overall
risk information production field.
With insufficient risk information, there is a risk that
strategic decisions may be made based on inadequate or
incomplete information. Resources may be allocated to an
SPI area that did not need ‘‘fixing’’, i.e. since its’ deviation
from threshold was only temporary and not significantly
deteriorated from normal operations. If initiatives are
directed based on SPI but without understanding of how
that SPI is produced or what the contributing and depen-
dent processes are, there is risk that changes will miss their
targets and only make local change without fixing the
problem.
It is recommended to develop processes for improved
risk information production to enhance the quality of the
risk information. There is for example a lack of support for
prioritisation among several SPIs. Better support from
statistical functionalities in the SMS would facilitate
comparable values and define thresholds for significant
trends. This may require increased statistical competence
in management and the safety office.
3.4.7 Knowledge cycle
All relevant personnel have basic training in the principles
of EASA-ops and SMS. All relevant safety department
personnel is getting basic familiarisation training with the
new software system, VMA. The SRC or FSQB structure
around data is designed to include domain knowledge and
subjective measures to complement data in VMA. When-
ever action is taken as a response to SPI values, the
123
234
production area experts are the ones that identify cause and
suggest solution based on the operators and their own
expertise about the operations. Production areas have basic
training and implemented Lean; process mapping knowl-
edge and methods exist within the production areas since
Lean has been implemented there and process mapping is a
common Lean tool. safety office have only had basic Lean
concept training in relation to their routines and not
incorporated Lean in SMS or Lean in the production areas,
which the safety department is supporting.
It is not clear that there is a common understanding
among those playing a key coordination role in SMS. The
domain knowledge is key for safety performance, but there
is no systematic approach to support analysis of operations
and identifying solutions. There is no clear link between
Lean and the SMS. In the Lean training, there is only very
general change management training. To make the link
between safety and quality is not easy for industry. It is still
a challenge for researchers both in Human Factors and
Safety Science.
Even if the domain knowledge may be excellent, but the
holistic picture for complex problem solutions is hard to
acquire as an individual. There is a risk this results in
locally and short-term satisfactory solutions rather than
sustainable change. Tasks may be duplicated in Lean and
SMS (unnecessary time and money), and solutions may not
be systemically solved.
System Change and Operations Evaluation may be
applied as a common process analysis methodology. This
would increase understanding of relevant process steps and
also give hints on who is involved where. It would mix
operational expertise with data. This could strengthen the
inference of results of statistical data analysis based on
statistics to validate process. SMS safety seminar series—
presenting integrated framework and functional change
focus—needs to be repeated and improved. Different
conceptualisation—from compliance to full cycle of
change and safety performance concepts—needs to be
discussed and visualised. This could strengthen the
understanding of a system that works well together and see
the various roles everyone have and what and to who they
deliver in the process of the full cycle of change to dem-
onstrate safety performance.
4 Discussion
The majority of enabling processes for safety performance
improvement may be summarised in terms of developing
ways of integrating data and people for improved system
analysis. However a joined-up organisation is not easy to
develop. Sectional or departmental interests mean that
priorities are not always fully shared; there often appears
123
Cogn Tech Work (2015) 17:219–236
to be a competition for power or influence that tempers
collaboration, but above all, sharing information, or failure
to do so, due to either trust or lack of methods, across
organisational boundaries, is a major barrier to create an
integrated management system. With an integrated
approach, a common operational picture could be devel-
oped. The particular context of safety critical systems
seems to have some positive influence of people’s moti-
vation to share data, in the name of safety. But safety
science in itself is struggling with developing new
approaches for safety management. So even if industry
were willing to do more, the knowledge is yet not easily
accessible for industry. In order to improve safety, also the
organisational change capability need to improve since
this is what demonstrated safety performance really
means.
In this longitudinal case study, the reality of the chal-
lenges of organisational change as well as safety manage-
ment challenges are clearly present. The approach to apply
state of the art research such as the SCOPE model and SE
to development and implementation of an SMS has given
numerous new insights to the airline and raised new
research challenges from risk assessment and data man-
agement to the need for training and knowledge creation.
The functional process and system analysis capability
that SCOPE and SE provide has been shown to have
potential to contribute to essential enhancements for a
majority of the core functionalities in the SMS as well as
the airline’s objectives to allow SPIs to drive actual and
sustainable change and develop a Safe-Lean concept for
the airline. In addition, the need for an integrated risk and
performance framework was identified to support strategic
decision-making. This research also had the purpose to
develop a strategy for successful implementation of this
SMS. For this, the SE was applied to the case study and
many change initiatives that contributed to the Integrated
Safety Management concept, directly or indirectly. Func-
tional process analysis requires a process. So, if SCOPE is
well suited for the functionalities of the SMS to improve
operational process and control risk in those, a similar
framework should fit a functional process analysis also of
the organisational process of the SMS. The approach taken
to conceptualise the SMS as an extended safety risk man-
agement process considered as a full cycle process; it
opened up for the possibility of making such analysis. The
SE is based on the same principles as SCOPE and is merely
an extension for the specific use of managing and evalu-
ating change in complex socio-technical systems.
Potential benefits of SCOPE in an SMS:
• SCOPE modelling supports the system description and
identification of relevant indicators for performance
monitoring.
Cogn Tech Work (2015) 17:219–236
• SCOPE supports analysis of root causes and proposed
solutions with a broader and more both process and
human-centred analysis of socio-technical processes than
regular process modelling and human factors models.
• For system change, SCOPE may be applied after
indicators have triggered an alert, and an SPI action
process is initiated. There is a defined link between the
measures to a process model that in turn links to the
operations for identifying the problem and support
solution identification.
• The suggested change, future state of the operational
process may be modelled in SCOPE to anticipate
knock-on effects on dependent processes or tasks.
• In the evaluation of change, SCOPE may be used to
validate and revise SPIs as well as identifying targets
for controlling safety performance.
• In the identification of need for change, SCOPE may be
used as a knowledge transformation method that support
development of a common picture of needs and an
agreement in a process of change among those involved.
• The knowledge transfer among different stakehold-
ers has the objective to increase integration and
potential team building.
• It also supports processes for knowledge transfor-
mation, learning across departments and different
levels of the organisation from production to
management.
• This in turn may foster trust as well as relations
essential for successful change teams and imple-
menting change.
Main recommendations for suggested change:
• Continue to develop an integrated process for risk
information production functionalities, supporting stra-
tegic decision-making with regard to business, safety
and the balance of those information.
• Discuss with top managers and argue for SMS priority
and the actual expectations regulators and managers
have on future SMS. Plan a strategy to reach that goal.
• Supply sufficient resources for development of a training
concept and conduct the training company wide.
• Make contact and start dialogue with regulators to
develop a performance and compliance structure for
safety performance assessment.
• Each person in the SMS full cycle of change should be
mapped in the 1–8 steps to identify their role and also
who should be in teams to deliver service from one step
to another.
• Training could reduce competence gaps and increase a
common understanding and buy-in. Training should
consist of both understanding operational and organi-
sational processes and their relation to safety
235
performance. This may include explicit training on
safety models and SMS as a full cycle of change.
• To strengthen the strategic decisions and capability to
prioritise among needs for improvement, VMA should
also be upgraded with statistical analysis capability
(significant trends, variance and correlations).
The case study resulted in an integrated performance
management framework including new and existing tools
and structures, exceeding that what is required by authori-
ties and their CCL. Another part of the framework is the
supporting process, in which the structures and tools are
functions that together is argued to drive actual and sus-
tainable safety performance in a full cycle of change. As a
complement to improve the chances for a successful system
change and implementation of such SMS, the SE resulted in
a set of recommendation for further development and
implementation of the SMS. The SE provided support to the
fact that even if all the pieces are in place for an advanced
SMS functionality, required by authorities including a
conceptual process of how it should function, still many
other essential areas need to be addressed. To address also
information and knowledge cycles and social relations,
building teams and trust is essential to make all the pieces to
work well together. This shows complementary benefits of
combining SCOPE and SE in system change in aviation.
This case contributes theoretically to industrial man-
agement by integrating traditional approaches in manage-
ment and process control to human factors and safety
research in the development of tools for functional process
analysis of complex socio-technical systems.
The industrial contribution from this case is a conceptual
framework for an integrated SMS, which has all the core
functionalities of a compliant SMS. The concept includes
existing structures and tools within the airline’s SMS as
well as innovative functionalities developed using SCOPE.
In addition to this concept, the developed SE was applied
to the developments and implementation of the Safe-Lean
concept SMS resulting in suggestions and recommenda-
tions for the way forward.
The SE is assessed to have captured the main experi-
ences from the development of the SMS and also provided
new insights that had not yet been identified in the past
ongoing research. The systematic approach and need to
consider all the seven areas relevant for system change in
socio-technical system did address all outstanding issues,
known to the MASCA researchers in the development and
implementation of the airline SMS including the developed
Safe-Lean concept. This validates its’ use for others that
are implementing a SMS in a complex socio-technical
industrial setting.
The Safe-Lean concept needs to be complemented with
an integrated risk model. Very few high-consequence data
123
236
exist, fortunately for safety and unfortunately for empirical
statistical studies. However, what can be monitored is the
‘‘normal variation’’ of operational performance data. With
the SCOPE modelling for developing SPIs, there will also
be a need to identify antecedents in measurable normal
operations data. Monitoring this data will increase the
understanding of the underlying system mechanisms. Fur-
ther research initiatives are required for developing and
integrated risk model that may be used to anticipate or
prevent complex system failures. For this, further work
with data mining and association rules of normal opera-
tional data will be used.
Acknowledgments The above-mentioned research has received
funding from the MASCA project in the European Commission’s
Seventh Framework Programme FP7/2007–2013 under Grant
Agreement 26642, the HILAS project in the European Commission’s
Sixth Framework Programme FP6/2005–2009 under Grant Agree-
ment No: 516181 and ‘‘Integrated performance management sys-
tems’’ Project Number: 2010-02783 from VINNOVA—Swedish
Governmental Agency for Innovation Systems.
References
ADAMS Human Factors in Aircraft Dispatch and Maintenance. Under
the EU BRITE-EURAM Programme. Contract No. BRPR-CT95-
0038, Project No: BE95-1732. Brussels: European Commission
(1994–1999), Amalberti R (2001) The paradoxes of almost totally
safe transportation systems. Saf Sci 37(2001):109–126
Bessant J (2003) High-involvement innovation. Wiley, Chichester, UK
Bunderath M, McDonald N, Grommes P, Morrison R (2008) The
operational impact to the maintainer. In: Proceedings of IET
conference, London
Cooke M, McDonald N (2009) Prospective risk analysis. HILAS
deliverable. Project NO. 516181.EC—6th FP
Deming WE (1986) Out of the crisis. MIT Press, Cambridge
Dent E, Goldberg S (1999) Challenging ‘‘resistance to change’’.
J Appl Behav Sci 35:25–41
European Commission (2011) Setting up an Aviation Safety
Management System for Europe. Communication from the
commission to the council and the EUR. Parliament, Brussels,
25.10.2011 COM(2011) 670 final
Folan P, Browne J (2005) A review of performance measurement:
towards performance management. Comput Ind 56(7):663–680
Harms-Ringdahl L (2008) Dimensions in safety indicators. Saf Sci.
doi:10.1016/j.ssci.2008.07.019
Hawkins FH, Orlady HW (ed) (1993) Human factors in flight, 2nd
edn. Avebury Technical, England
Head of Safety in Airline (2012) Personal communication
HILAS (2005–2009) Human Integration into the Lifecycle of
Aviation Systems. Priority no: 1.4—Aeronautics and Space
Research Area 3. Priority title: improving aircraft safety and
security, IP8. proposal/contract no: 516181. Brussels: EC
Hopkins A (2008) Reply to comments. Saf Sci. doi:10.1016/j.ssci.
2008.07.020
ICAO (2006) Safety management manual (SMM), 1st edn. Draft
document 9859
International Civil Aviation Organisation (2008) Safety management
manual (SMM), 2nd edn. Montreal, Canada: Doc 9859
International Civil Aviation Organisation (2012) Safety management
manual (SMM), 3rd edn. Montreal, Canada: Doc 9859
123
Cogn Tech Work (2015) 17:219–236
Juran JM (1988) Juran’s quality control handbook, 4th edn. McGraw-
Hill Companies, New York
Kotter JP (1995) Leading change: why transformation efforts fail.
Harv Bus Rev 73(2):59–67
Learmont D (2009) Year in review. In: Keynote speak at EASS 2009,
the European Aviation Safety Seminar, March, Nicosia, Cyprus
Leva C, Ulfvengren P, Corrigan S, Zon R, Baranzini D, McDonald N,
Licata V (2011) Deliverable: D 2.1 Review of requirements for
change. Deliverable to EC as part of MASCA project FP 7-AAT-
2010-4.3-4.: Grant agreement no 266423
Mawdsley D (2010) Measuring safety. IATA IOSA SMS Workshop
22(10):2010
McDonald N (ed) (1999) Human-centred management for aircraft
maintenance ADAMS-WP4A-D2 (April 1999). Deliverable to
the European Com., ADAMS project. Dept of Psychology,
Trinity College Dublin
McDonald N (2001) Human systems and aircraft maintenance. Air
Space Eur 3(3/4):221–224
McDonald N (2014) The evaluation of change. Cogn Tech Work.
doi:10.1007/s10111-014-0296-9 (this issue)
McDonald N, Corrigan S, Cromie S, Daly C (2000) Safety
management systems and safety culture in aviation maintenance
organisations. Saf Sci 34:151–176
Morrison R (2009) Operational process modell/knowledge space
model HILAS deliverable and draft manuscript to book. Project
number 516181. Funded by European Commission—6th FP
Norman DA (1993) Things that make us smart. Addison-Wesley,
New York
Rasmussen J, Pejtersen AM, Goodstein LP (1994) Cognitive systems
engineering. Wiley, Hoboken
Reason J (1990) Human error. Cambridge University Press, New
York
Reiman T, Pietikäinen E (2012) Leading indicators of system
safety—monitoring and driving the organizational safety poten-
tial. Saf Sci 50(10):1993–2000
Rignér J, Ulfvengren P, Kay A (2009a) Measuring safety perfor-
mance—strategic risk data. In the EASS, 21st annual European
aviation safety seminar, proceedings, Nicosia, Cyprus, 16–18
March 2009
Rignér J, Ulfvengren P, Cooke M, Leva C, Kay A (2009b) Study of
safety performance indicators and contributory factors as part of
an airline systemic safety risk data model. In: Proceedings of the
17th world congress on ergonomics, IEA 2009, Beijing 9–14
August 2009
Stolzer AJ, Halford CD, Goglia JJ (2008) Safety management systems
in aviation. Ashgate, Farnham
Ulfvengren P, Rignér J, Ydalus M, Mårtensson L (2009) SAS HILAS
participation and experiences. HILAS report, deliverable and
draft manuscript to book in preparation. Project number 516181.
Funded by EC—Sixth Framework Programme
Ward Y, Brito M (2007) Lean-safe operations for the aviation
industry. In: Proceedings of EurOMA (European Operations
Management Association), June 2007
Ward M, McDonald N, Morrison R, Gaynor D, Nugent T (2010) A
performance improvement case study in aircraft maintenance
and its implications for hazard identification. Ergonomics
53(2):247–267
Wreathall J (2008) Leading? Lagging? Whatever! Saf Sci. doi:10.
1016/j.ssci.2008.07.031
Wickens CD, Lee JD, Liu Y, Gordon Becker SE (2004) An
introduction to human factors engineering, 2nd edn. Prentice
Hall, Pearcon
Womack JP, Jones DT (2003) Lean thinking: banish waste and create
wealth in your organisation, 2nd edn. Free Press, New York
Womack JP, Jones DT, Roos D (1991) The machine that changed the
world: the story of lean production. Harper Perennial, New York