See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/367361831

Resilience Engineering's synergy with Threat and Error Management - an

operationalised model

Preprint · January 2023

CITATIONS READS

0 38

2 authors, including:

Andrew Mizzi
Griffith University
5 PUBLICATIONS 7 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Flight Crew Briefings with Resilience Engineering Capacities – The STC Model View project

How Performance Variability Influences Work-As-Done in the Cockpit View project

All content following this page was uploaded by Andrew Mizzi on 25 January 2023.

The user has requested enhancement of the downloaded file.

 Resilience Engineering’s synergy with Threat and Error Management
– an operationalised model.

Andrew Mizzi, Griffith University, Nathan, Queensland, Australia.

E-mail: andrew.mizzi@griffithuni.edu.au
(ORCID 0000-0002-3094-8535)
Pete McCarthy, Cranfield University, Cranfield, United Kingdom.
E-mail: pete.mccarthy@cranfield.ac.uk
(ORCID 0000-0003-0021-4503)

Keywords: flight crew, briefing, resilience engineering, capacity, team cognition, aviation.

1. Introduction
Recently, recognition of the variability of work in complex systems and its influence on human
performance has been published in a guidance manual by the International Civil Aviation Organization,
which drives state aviation regulations [1]. Additionally, new regulatory requirements around Crew
Resource Management (CRM) have been introduced requiring training into resilience by the supranational
regulator in Europe [2]. These improvements to the understanding of human influence within socio-
technical systems require improved models to operationalise the theory. The discipline of Resilience
Engineering has adopted four potentials required for resilience in an organisation, first presented by
Hollnagel. They are described as the potential to anticipate, to monitor, to respond and to learn [3]. Whilst
the incorporation of resilience in pilot training is only emerging, the Threat and Error Management (TEM)
model has been widely adopted throughout the industry [4]. It has three basic concepts: the anticipation
of threats, the recognition of errors and recovery from unintended aircraft states [5]. The constant
identification and mitigation of threats by the flight crew engages a sense of unease which fosters resilience
and prevents complacency. However, despite its wide adoption, confusion can exist in the way that TEM
is integrated into CRM training, and how crew can use TEM tools in practice [6]. This paper investigates
the synergy which is present between the resilient potentials of anticipate, monitor, and respond [7] and
Threat and Error Management. A briefing model concept is presented which can extend operational
resilience by engaging crews’ capacity to anticipate threats and errors, monitor for errors and respond with
countermeasures, so to recover from errors and undesired aircraft states.

2. Threat and Error Management

Threat and error management (TEM) is a proactive approach to safety in the aviation industry. The
TEM model (Figure 1, [8]) is based on the premise that threats and errors are inevitable on any flight, and
that it is important to have processes in place to detect and correct these before they lead to an incident or
accident. Through applying the TEM framework, crew are compelled first to identify potential threats to
the safe operation of a flight and implement strategies to mitigate those threats. If those threats aren’t
managed or are mismanaged, errors can occur which require detection and management to resolve. If errors
are led to continue an undesired aircraft state can occur, leading to an incident or accident. There are several
strategies that can be used to manage threats and errors in aviation. These include the proactive
implementation of strategies to mitigate against anticipated threats such as weather, the use of checklists
and standard operating procedures to counter latent threats, the use of automation to reduce the workload
on pilots, and the execution of go-arounds or diversions when the situation require recovery actions. By
2 Resilience Engineering’s synergy with Threat and Error Management – an operationalised model

implementing these strategies, it is possible to

significantly reduce the risk of accidents and improve
the overall safety of aviation operations.
There are several applications of the TEM model
in practice. It has become the foundation of many
human factors training programs in airlines. The
International Civil Aviation Organization (ICAO)
adopted the model into its human factors curriculum
[9], prompting to the further adoption by state
regulators and airlines. However, MacLeod has
mentioned that there is confusion in the way that it
can be deeply integrated into CRM training [6], by
arguing that there is no ‘pure’ airline training
implementation of TEM aside from mentioning its
key concepts of threat, errors, and outcomes. Deep
integration of TEM into workflows can be further
developed, as further explained in this paper. The Line
Oriented Safety Audit (LOSA) program was
developed in conjunction with TEM and provides a
systemic observation tool for airlines [10]. A
behavioural audit methodology studies flight crew
TEM in normal line flight operations which has
proven successful [11] and recognised as a proactive
measure of flight safety [12]. The TEM model has also
been operationalised into safety reporting. The United
States’ Aviation Safety Action Program (ASAP)
identifies parts of the model to describe the event at
the level of threats and errors, and a TEM based
approach has shown to increase the quality of reports
[13].
It is important to recognise that Threat and Error Figure 1 The model of flight crew threat and error management
Management principles focus is based on a model of
error management which doesn’t account for the complexity of systems involved in its application,
particularly when applied in incident investigation (ASAP) and safety assurance (LOSA). A greater focus in
systems thinking is needed. As the industry’s journey into applying the fundamentals of Safety-II continues,
the utilisation of the mature TEM model with newer modes of thinking is needed to achieve further
improvement in flight safety. Resilience engineering provides needed guidance.

3. Resilient Potentials
Resilience engineering is a field of study that focuses on improving safety by helping people cope with
complexity to achieve success [14]. The goal of resilience engineering is to design systems that can adapt
and recover quickly when faced with unexpected challenges or failures. Resilience is achieved by the system,
in this case the flight crew and aircraft state, sustaining its required operation in both expected and
unexpected conditions [7]. In the special issue of Reliability Engineering and System Safety, Nemeth and Herrera
[15] describe three core values of resilience engineering: finding resilience, assessing resilience, and creating
resilience. The topic of creating resilience has remained foundational to the term resilience engineering, thus
finding ways to operationalise the capacities which remerge in resilient work is key to future development.
3 Resilience Engineering’s synergy with Threat and Error Management – an operationalised model

Figure 2 Required capacities of a resilient system

There are several key potentials developed by Woods and Hollnagel and modelled in Figure 2 [16] that
contribute to the resilience of a system. The first potential is the ability to anticipate and prepare for potential
failures. This requires having attention in place to identify and assess potential threats, and planning
strategies to mitigate or prevent those threats from occurring. The next potential is the ability to monitor for
signals of system degradation, in order to adapt and recover from failures when they do occur. This then
requires having systems and processes in place to quickly identify and respond to failures, as well as having
the necessary resources and capabilities to address degraded systems or recover from undesired aircraft
states. In addition to these technical capacities, resilience engineering also recognises the importance of
human factors in the resilience of a system. This includes the need for effective communication and
collaboration among crew, as well as the need for clear roles and responsibilities to ensure that everyone
knows their part in the response to a failure. By understanding and improving these capacities, it is possible
to significantly improve the resilience of complex systems.
The requirement for team training in resilience is emerging in aviation. The European Aviation Safety
Agency (EASA) has modified its regulatory framework in Crew Resource Management (CRM) to include
‘Resilience Development’ [2]. Training topics required under the regulations involve mental flexibility and
performance adaptation, requiring crew to draw the right conclusions from both positive and negative
experiences. The normative language involved in the regulations however may see these training
categorisations turned into levels of accountability, in a similar vein as CRM [17, 18]. As argued by Dekker
and Bergström [19], “the language responsibilises pilots not only for the performance of the system, but in
a sense even for their beliefs in resilience theory”. It’s therefore key for airlines to develop ways to
encompass it wisely and ethically into their training, policies, and procedures, and for regulators to follow
ICAO guidance on human performance [1]. Utilising the synergistic attributes between the resilient
potentials and TEM may provide the required assistance to galvanise these regulations into practice.

4. Discussion: Synergistic Attributes

In design, the threat and error management model’s actions can be closely aligned to the resilient
potentials, allowing for the two concepts to be operationalised into flight decks and every-day work with
greater ease (Figure 3). One important way that TEM contributes to resilience is through its focus on
proactive identification and assessment of potential threats. These two actions call upon the crews’ capacity
to learn and anticipate. By identifying and assessing potential threats in advance, it is possible to implement
strategies to mitigate or prevent those threats from occurring. This proactive approach helps to broaden
the crew’s capacity and acceptable boundaries in dealing with disturbances to their flight, which in turn
increases the overall resilience of the system. Another way in which TEM enhances resilience is through its
focus on error detection and correction. By having processes in place to quickly identify and correct errors,
4 Resilience Engineering’s synergy with Threat and Error Management – an operationalised model

Figure 3 The TEM model (simplified flow diagram) overlaid with the Resilient Potentials (Venn
diagram)

it is possible to prevent those errors from cascading into more serious failures and disturbances to the flight
are quickly recovered back to a crews’ acceptable norm.
4.1 Anticipate for Threats and Errors
The TEM model categorises threats into two categories. Environmental threats are associated to adverse
weather, ATC, the airport, and environmental operational pressures. Conversely, airline threats are related
to the aircraft, airline operational pressures, cabin, dispatch, ground, maintenance, and documentation. A
mismanaged threat is defined “as a threat that is linked to or induces flight crew error” [5]. This associates
strongly into the anticipation potential from resilience engineering, where the anticipation of threats (based
on previously learned experience and information) is made to mitigate and monitor. Hollnagel delineates
the difference between anticipation and monitoring [20] by explaining that anticipation deals with that
which is beyond the immediate event horizon, either something that lies further into the future or
something that has no immediate relation or impact on the flight.
4.2 Monitor Errors and Responding Actions
Monitoring then, according to Hollnagel is about observing whether the flight state is changing in a way
that requires a response or an intervention. This can be monitoring for threats which were first anticipated,
or where TEM actions to manage subsequently led to an error. Errors are actions (or inactions) which lead
to a deviation from intentions or expectations. Errors are categorised into aircraft handling errors,
procedural errors, and communication errors. Klinect et al. [5] explain that threats come ‘at’ the crew -
which crew can work to anticipate, whereas errors come ‘from’ the crew - which crew should monitor for.
The development of monitoring cues is essential for resilient performance in everyday work.
Error management is a driving force behind TEM and subsequently, the resilient potentials encompass
error anticipation and monitoring, whilst response potential manages error ‘capture’ actions.
4.3 Respond with Actions to Errors and Undesired Aircraft States
The response to errors, if successful may lead to an inconsequential outcome. However, if the response is
insufficient or absent, can result in an undesired aircraft state. These are categorised into aircraft handling,
ground handling and incorrect aircraft configurations. The response potential is crucial in resilient
performance for being able to handle disturbances to the flight.
It should also be strongly noted that whilst the TEM model is linear, the capacity for crew to learn should
take place throughout, allowing new experience gained to be further disseminated, creating new
understanding and anticipation of threats.

5. Example Application
5 Resilience Engineering’s synergy with Threat and Error Management – an operationalised model

Figure 4 The STC model

An example of how Threat and Error Management can be deeply integrated into pilots’ work is through
planning and briefing models. An example is the ‘STC’ briefing model [21], which is constructed around a
foundation of the resilient potentials of ‘anticipate’, ‘monitor’ and ‘respond’. The briefing is broken into
four segments or steps. The ‘situation’ is a pre-briefed segment conducted prior to the main ‘strategic’ plan
(S), ‘tactical’ plan (T), and ‘contingency’ plan (C) modules. It is designed to identify threats-first and foster crew
communication through a framework of briefing an overview of ‘where we are’ (the situation), and then a
more detailed ‘what our plan is’ (strategic), ‘how we’ll execute the plan’ (tactical) and ‘what we’ll do outside
the plan’ (contingency) (Figure 4). The three key resilient potentials are structured as integral pillars to the brief
with the situation and strategic modules anticipating threats and establishing a broad strategic plan. Then,
the strategic and tactical modules establishing the boundaries of the strategic plan, and monitoring cues
requiring tactical intervention. Lastly, the tactical and contingency modules priming responses in deviations
outside the boundaries of anticipated work.
The STC briefing model combines the concepts of threat and error management with the potentials
needed for resilience. However, as depicted in Figure 4, it’s recognised that each of the three core modules
of the STC brief does not align solely with the three resilient potentials of anticipate, monitor, and respond.
Instead, there is an overlap between the three resilient potentials and the four briefing modules. Through
the briefing process, the crew anticipate threats and strategy with their strategic plan, which sets monitoring
cues and boundaries through the strategic and tactical plans. This builds upon the crews’ capacity to respond
through the tactical and contingency plan. An improved conversational flow is developed between each
module, strengthening resilient potentials. It maintains foundational strength of the resilient potentials,
particularly where the strategic and tactical modules are foundationally constructed of two resilient potentials
whilst the situation is anticipated, and contingencies responded with.
This example of how the resilient potentials can be applied directly into the every-day work of pilots
could show improvements to threat awareness and mitigation strategies and improve communication
between crew. This is particularly important as airlines recover from the COVID-19 pandemic and re-train
their flight crew [22]. This concept of work directly operationalises Carroll & Malmquist’s recommendation
that the attributes of resilience performance be operationalised directly into flight crews’ procedures [23].

6. Future Development
The future for resilience engineering in aviation is strong. Supra-national regulators such as EASA are
implementing early editions of regulation requiring training into resilience. Future evolutions of this
regulation, plus from other regulators are anticipated.
Given the widespread adoption of threat and error management in aviation, future research, and
development into how the resilient potentials can be operationalised into training, procedures and practices
6 Resilience Engineering’s synergy with Threat and Error Management – an operationalised model

would assist airlines in meeting their regulatory obligations along with fostering resilient performance
amongst their flight crew. This is particularly needed in training evaluation, such as competency based
training and evidence based training.
An exciting evolution of resilience has been made by American Airlines’ Learning and Improvement
Team (LIT) [24, 25]. They have utilised the success of their Line Oriented Safety Audit program as a vehicle
to develop a Safety-II audit program. LIT proficiencies were developed from the resilient potentials specific
to the program’s needs and a series of observations and interviews conducted to gather data. LIT offers
evolutionary development from Safety-I’s TEM and LOSA approach into Safety-II flight safety auditing.
One further example of this shift to a Safety II approach is the Learning Review Safety Audit [26]. Flight
crew performance in a normal line environment is disclosed by the flight crew, facilitated by individuals
who have been specially selected and trained to conduct Operational Learning Reviews (OLRs) – these
facilitators utilise TEM and systems thinking to identify resilient behaviours from the context being
provided by the flight crew – this can then be analysed alongside data sources such as flight data, creating
a much wider understanding of “work as done” in the operation.

7. Conclusion
TEM largely contributes to the resilience of a system through its emphasis on human factors. By training
and supporting crew members to effectively anticipate, monitor and respond to threats and errors, TEM’s
synergy with the resilient potentials helps to improve the overall performance and decision-making skills
of flight crews. This improved performance can be critical in the response to an undesired aircraft state, as
it helps to ensure that corrective actions are taken at the appropriate time to lead to inconsequential
outcomes or minimise their impact in recovery. Future development and integration between the two
concepts of TEM and resilience is recommended to regulators and airlines, and Safety-II audit programs
encouraged to develop into maturity and widespread use.

8. References

1. ICAO: Manual on Human Performance (HP) for Regulators, https://www.icao.int/safety/OPS/OPS-

Section/Documents/Advance-unedited.Doc.10151.alltext.en.pdf, (2021).

2. European Aviation Safety Agency: Annex II to Decision 2015/022/R,

https://www.easa.europa.eu/en/downloads/19874/en, (2015).

3. Hollnagel, E.: The four cornerstones of resilience engineering. In: Nameth, C., Hollnagel, E., and
Dekker, S. (eds.) Resilience Engineering Perspectives: Preparation and Restoration. pp. 117–133. Ashgate
(2009). https://doi.org/10.1201/9781315244389-7 .

4. Dekker, S., Lundström, J.: From Threat and Error Management (TEM) to resilience. Human Factors
and Aerospace Safety. 261–273 (2006).

5. Klinect, J., Merritt, A.: Defensive Flying for Pilots: An Introduction to Threat and Error Management.
(2006).

6. MacLeod, N.: Crew Resource Management Training: A Competence-based Approach for Airline
Pilots. CRC Press, Boca Raton, FL (2021). https://doi.org/10.1201/9781003138839 .
7 Resilience Engineering’s synergy with Threat and Error Management – an operationalised model

7. Hollnagel, E.: Epilogue: RAG. The resilience analysis grid. In: Pariès, J., Wreathall, J., Hollnagel, E.,
and Woods, D.D. (eds.) Resilience Engineering in Practice. pp. 275–296. Ashgate, Surrey, UK (2011).
https://doi.org/10.1201/9781317065265 .

8. Helmreich, R., Klinect, J., Wilhelm, J.: Models of Threat, Error, and CRM in Flight Operations. In:
Proceedings of the Tenth International Symposium on Aviation Psychology. , Columbus, Ohio (1999).

9. ICAO: Human Factors Training Manual, https://elibrary.icao.int/, (1998).

10. Klinect, J., Murray, P., Merritt, A., Helmreich, R.: Line Operations Safety Audit - Definition and
Operating Characteristics. In: Proceedings of the 12th International Symposium on Aviation Psychology.
pp. 663–668. , Dayton, OH (2003).

11. Klinect, J.: Line Operations Safety Audit - A Cockpit Observation Methodology for Monitoring
Commercial Airline Safety Performance, (2005).

12. Dismukes, R.K., Berman, B.A., Loukopoulos, L.: The Limits of Expertise. Ashgate, Hampshire,
England (2007). https://doi.org/10.4324/9781315238654 .

13. Harper, M.: The aviation safety action program : assessment of the threat and error management
model for improving the quantity and quality of reported information, http://hdl.handle.net/2152/ETD-
UT-2011-05-3061, (2011).

14. Woods, D.D., Hollnagel, E.: Prologue: Resilience Engineering Concepts. In: Hollnagel, E., Woods,
D.D., and Leveson, N. (eds.) Resilience Engineering: Concepts and Precepts (2006).
https://doi.org/10.1201/9781315605685 .

15. Nemeth, C.P., Herrera, I.: Building change: Resilience Engineering after ten years. Reliab Eng Syst
Safe. 141, 1–4 (2015). https://doi.org/10.1016/j.ress.2015.04.006 .

16. Woods, D.D., Hollnagel, E.: Epilogue: Resilience Engineering Precepts. In: Hollnagel, E., Woods,
D.D., and Leveson, N. (eds.) Resilience Engineering: Concepts and Precepts. pp. 347–358 (2006).
https://doi.org/10.1201/9781315605685-30 .

17. Cook, R.I., Nemeth, C.P.: “Those found responsible have been sacked”: some observations on the
usefulness of error. Cognition Technology Work. 12, 87–93 (2010). https://doi.org/10.1007/s10111-010-
0149-0 .

18. Dekker, S.W.A., Nyce, J.M., Myers, D.J.: The little engine who could not: “rehabilitating” the
individual in safety research. Cognition Technology Work. 15, 277–282 (2013).
https://doi.org/10.1007/s10111-012-0228-5 .

19. Dekker, S.: Foundations of Safety Science. CRC Press, Boca Raton, FL (2019).
https://doi.org/10.4324/9781351059794 .

20. Hollnagel, E.: Safety-II in Practice. Routledge, New York, NY (2018).

https://doi.org/10.4324/9781315201023 .

21. Mizzi, A., McCarthy, P.: Flight Crew Briefings with Resilience Engineering Capacities - The STC
Model, https://doi.org/10.13140/RG.2.2.13499.08480, (2023).

22. Mizzi, A., Lohmann, G., Junior, G.C.: The Role of Self-Study in Addressing Competency Decline
Among Airline Pilots During the COVID-19 Pandemic. Hum Factors. 00187208221113614 (2022).
https://doi.org/10.1177/00187208221113614 .
 8 Resilience Engineering’s synergy with Threat and Error Management – an operationalised model

23. Carroll, M., Malmquist, S.: Advancing Resilient Performance. In: Nameth, C.P. and Hollnagel, E.
(eds.) Advancing Resilient Performance. pp. 85–95 (2022). https://doi.org/10.1007/978-3-030-74689-
6_7 .

24. American Airlines: Trailblazers into Safety - II: American Airlines’ Learning and Improvement
Team, https://www.skybrary.aero/sites/default/files/bookshelf/5964.pdf, (2020).

25. American Airlines: Charting a New Approach: What Goes Well and Why at American Airlines,
https://skybrary.aero/sites/default/files/bookshelf/AA%20LIT%20White%20Paper%20II%20-
%20SEP%202021.pdf, (2021).

26. McCarthy, P.: The Application of Safety II in Commercial Aviation – The Operational Learning
Review (OLR). In: Harris, D. and Li, W.-C. (eds.) Engineering Psychology and Cognitive Ergonomics.
Cognition and Design. HCII 2020. pp. 368–383. Springer, Cham (2020). https://doi.org/10.1007/978-3-
030-49183-3_29 .

View publication stats