[Type text]

Draft Introduction to Maintenance Threat Error Management

[Type text]

Preface 1. Threat and Error Management (TEM) is an overarching safety concept regarding aviation operations and human performance. TEM is not a revolutionary concept, but one that has evolved gradually, as a consequence of the constant drive to improve the margins of safety in aviation operations through the practical integration of Human Factors knowledge. 2. TEM was developed as a product of collective aviation industry experience. Such experience fostered the recognition that past studies and, most importantly, operational consideration of human performance in aviation had largely overlooked the most important factor influencing human performance in dynamic work environments: the interaction between people and the operational context (i.e., organisational, regulatory and environmental factors) within which people discharged their operational duties. TEM Background 3. The origin of TEM can be traced down to the origin of Line Operations Safety Audits (LOSA). A partnership between the University of Texas Human Factors Research Project (UT) and Delta Airlines in 1994 has developed a line audit methodology utilising jump-seat observations on scheduled flights. Both parties agreed that in order for the audit to be productive and show realistic and un-obscured results, confidentiality of the findings with no regulatory or organisational jeopardy to the flight crews should be guaranteed. 4. The initial observation forms of the audit were designed by the University of Texas researchers to evaluate Crew Resource Management (CRM) behaviour on the flight deck. The form was lately enhanced to address error and its management as well as type of error committed. This method enabled the observers to identify the origin of the error, the response to the error, was the error detected and by whom, together with the outcome of the error. 5. The first full scale TEM-based LOSA was conducted at Continental Airlines in 1996. Together with the original CRM indicators (leadership, communication, and monitoring/cross-checking) the new concept of TEM was used to identify most frequent threats. This method provided a clear picture of the most common errors and threats, both that were well managed and the more problematic and mismanaged. 6. The recognition of the influence of the operational context in human performance led to the conclusion that the study and consideration of human performance in aviation operations must not be an end in and on itself. With regard to the improvement of margins of safety in aviation operations, the study and consideration of human performance without context addresses only part of the larger issue. TEM therefore aims to provide a principled approach to the broad examination of the dynamic and challenging complexities of the operational context in human performance, for it is the influence of these complexities that generates the consequences that directly affect safety. TEM Framework 7. The TEM framework is a conceptual model that assists in understanding, from an operational perspective, the inter-relationship between safety and human performance in dynamic and challenging operational contexts. 1 Note: This article is a draft which contains articles extracted from other references which shall be included in the final paper

[Type text]

Draft

[Type text]

8. The TEM framework focuses simultaneously on the operational context and the people discharging operational duties in such a context. The framework is descriptive and diagnostic of both human and system performance. It is descriptive because it captures human and system performance in the normal operational context, resulting in realistic descriptions. It is diagnostic because it allows quantifying the complexities of the operational context in relation to the description of human performance in that context, and vice-versa.

9. The TEM framework can be used in several ways. As a safety analysis tool, the framework can focus on a single event, as is the case with accident/incident analysis; or it can be used to understand systemic patterns within a large set of events, as is the case with operational audits. The TEM framework can be used to inform about licensing requirements, helping clarify human performance needs, strengths and vulnerabilities, thus allowing the definition of competencies from a broader safety management perspective. Subsequently the TEM framework can be a useful tool in On the-Job Training (OJT). The TEM framework can be used as guidance to inform about training requirements, helping an organisation improve the effectiveness of its training interventions, and consequently of its organisational safeguards. The TEM framework can be used to provide training to quality assurance specialists who are responsible for evaluating facility operations as part of certification. 10. Originally developed for flight deck operations, the TEM framework can nonetheless be used at different levels and sectors within an organisation, and across different organisations within the aviation industry. It is therefore important, when applying TEM, to keep the user's perspective in the forefront. Depending on "who" is using TEM (i.e. front-line personnel, middle management, senior management, flight operations, maintenance, air traffic control), slight adjustments to related definitions may be required. The Components of the TEM Framework 11. There are three basic components in the TEM framework, from the perspective of their users they have slightly different definitions: threats, errors and undesired (aircraft) states. The framework proposes that threats and errors are part of everyday aviation operations that must be managed by the aviation professionals, since both threats and errors carry the potential to generate undesired states. The undesired states carry the potential for unsafe outcomes thus undesired state management is an essential component of the TEM framework, as important as threat and error management. Undesired state management largely represents the last opportunity to avoid an unsafe outcome and thus maintain safety margins in aviation operations.

Threats - generally defined as events or errors that occur beyond the influence of the line personnel, increase operational complexity, and which must be managed to maintain the margins of safety. Errors - generally defined as actions or inactions by the line personnel that lead to deviations from organisational or operational intentions or expectations. Unmanaged and/or mis-managed errors frequently lead to undesired states. Errors in the operational context thus tend to reduce the margins of safety and increase the probability of an undesirable event.

Undesired states - generally defined as operational conditions where an unintended situation results in a reduction in margins of safety. Undesired states that result from ineffective threat 2 Note: This article is a draft which contains articles extracted from other references which shall be included in the final paper

[Type text]

Draft

[Type text]

and/or error management may lead to compromised situations and reduce margins of safety aviation operations. Often considered the last stage before an incident or accident. TEM: Definitions and Examples 12. The Threat and Error Management (TEM) framework focuses simultaneously on the operating environment and the humans working in that environment. Because the framework captures performance in its natural or normal operating context, the resulting description is realistic, dynamic, and holistic. Because the TEM taxonomy can also quantify the specifics of the environment and the effectiveness of performance in that environment, the results are also highly diagnostic. Threats and their Management 13. Pilots have to manage various complexities in the operating environment on a typical day of flying. In TEM, such complexities are known as threats. Threat Definition 14. Threats are defined as events or errors that: occur outside the influence of the flight crew (i.e., not caused by the crew); increase the operational complexity of a flight; and require crew attention and management if safety margins are to be maintained.

15. Threat management can be broadly defined as how crews anticipate and/or respond to threats. A mismanaged threat is defined as a threat that is linked to or induces flight crew error. Some of the common tools and techniques used in commercial aviation to manage threats and prevent crew errors include reading weather advisories, turning weather radar on early, thorough walk-around during pre-departure, correct use of procedures to diagnose unexpected aircraft malfunctions, briefing an alternate runway in case of a late runway change, briefing cabin crew as to acceptable times and reasons for interruptions, and loading extra fuel when the destination airport is in question due to poor weather or restricted access. Errors and their Management 16. From the TEM perspective, error is a crew action or inaction that leads to a deviation from crew or organizational intentions or expectations. Put simply, threats come at the crew, while errors come from the crew. Flight crew errors can be the result of a momentary slip or lapse, or induced by an expected or unexpected threat. For example, a late runway change might induce a procedural shortcut that results in further error, just as a gate agent interruption could distract the flight crew from completing a checklist, causing them to miss an incorrect flaps setting for takeoff. Other errors are more deliberate. Known as intentional noncompliance errors in the TEM taxonomy, these errors are often proven shortcuts used by flight crews to increase 3 Note: This article is a draft which contains articles extracted from other references which shall be included in the final paper

[Type text]

Draft

[Type text]

operational efficiency even thought they are in violation of Standard Operating Procedures. High rates of noncompliance at an airline can often indicate systemic over-procedualization. Error Definition 17. Errors are defined as flight crew actions or inactions that: lead to a deviation from crew or organizational intentions or expectations; reduce safety margins; and increase the probability of adverse operational events on the ground or during flight.

18. Error management is now recognized as an inevitable part of learning, adaptation, and skill maintenance; hence, a primary driving force behind TEM is to understand what types of errors are made under what circumstances (i.e., the presence or absence of which threats) and how crews respond in those situations. For example, do crews detect and recover the error quickly, do they acknowledge the error but do nothing, perhaps because they believe it is inconsequential or will be trapped later, or do they only see the error when it escalates to a more serious undesired aircraft state? This is the heart of error management: detecting and correcting errors. However, approximately 45% of the observed errors in the LOSA Archive were errors that went undetected or were not responded to by the flight crew, which gives credence to an important point for effective error management: An error that is not detected cannot be managed. 19. An error that is detected and effectively managed has no adverse impact on the flight. On the other hand, a mismanaged error reduces safety margins by linking to or inducing additional error or an undesired aircraft state. Just how common are mismanaged errors and when do they occur? The LOSA Archive provides some insight, as shown in the quiz below. Undesired Aircraft States and their Management 20. Unfortunately, not all errors are well managed. Sometimes they lead to another error or a safety-compromising event called an undesired aircraft state (UAS). Undesired Aircraft State Definition 21. An undesired aircraft state (UAS) is defined as a position, speed, attitude, or configuration of an aircraft that: results from flight crew error, actions, or inaction; and clearly reduces safety margins

22. As with errors, UASs can be managed effectively, returning the aircraft to optimally safe flight, or mismanaged, leading to an additional error, undesired aircraft state, or worse, an incident, or 4 Note: This article is a draft which contains articles extracted from other references which shall be included in the final paper

[Type text]

Draft

[Type text]

accident. The last quiz sheds light on the prevalence and mismanagement of undesired aircraft states in the LOSA Archive. TEM Tools & Techniques 23. The principles of TEM are not new to aviation. In fact, Orville and Wilbur Wright no doubt practiced threat and error management when they took their first controlled flight with the Wright Flyer in 1903. Since then, various tools and techniques have been developed over the past century to help flight crews manage threats, errors, and undesired aircraft states. 24. Some toolsthe hard safeguardsare associated with aircraft design, and include automated systems, instrument displays, and aircraft warnings. The Traffic Collision Avoidance System (TCAS), which provides flight crews with visual and audio warnings of nearby airplanes to prevent midair collisions, is a good example of a hard TEM safeguard. Even with the best designed equipment however, these hard safeguards are not enough to ensure effective TEM performance. 25. Other toolsthe soft safeguardsare very common in aviation (and other high-risk industries). They include regulations, standard operating procedures, and checklists to direct pilots and maintain equipment; and licensing standards, checks, and training to maintain proficiency. With the hard and soft safeguards in place, the last line of defence against threat, error, and undesired aircraft states, is still, ultimately, the flight crew. Checklists only work if flight crews use them; the autopilot only works when engaged in the correct mode. 29. Therefore, TEM tools work best when pilots adopt TEM techniques. The TEM philosophy stresses three basic concepts: anticipation, recognition, and recovery. The key to anticipation is accepting that while something is likely to go wrong, you cant know exactly what it will be or when it will happen. Hence, a chronic unease reinforces the vigilance that is necessary in all safety-critical professions. Anticipation builds vigilance, and vigilance is the key to recognizing adverse events and error. Logically, recognition leads to recovery. In some cases, particularly when an error escalates to an undesired aircraft state, recovering adequate safety margins is the first line of action: Recover first, analyse the causes later. 26. While hard and soft safeguards help support pilots to best anticipate, recognize and recover from threats, errors, and undesired aircraft states, there is arguably no better way to manage these events in multi-pilot cockpits than through effective crew coordination. Many of the best practices advocated by Crew Resource Management (CRM) can be considered TEM countermeasures. Planning countermeasuresplanning, preparation, briefings, contingency managementare essential for managing anticipated and unexpected threats. Execution countermeasuresmonitor/cross-check, taxiway/runway management, workload and automation managementare essential for error detection and error response. Review/Modify countermeasuresevaluation of plans, inquiryare essential for managing the changing conditions of a flight, such as undesired aircraft states.

27. Initial research in the LOSA Archive has supported links between TEM and CRM. 5 Note: This article is a draft which contains articles extracted from other references which shall be included in the final paper

[Type text]

Draft

[Type text]

For example, crews that develop contingency management plans, such as proactively discussing strategies for anticipated threats, tend to have fewer mismanaged threats; crews that exhibit good monitoring and cross-checking usually commit fewer errors and have fewer mismanaged errors; and finally, crews that exhibit strong leadership, inquiry, workload management are typically observed to have fewer mismanaged errors and undesired aircraft states than other crews.

Maintenance Threat Error Management System (MTEMS ) Definition 28. A generic term for a system in which the investigation of maintenance errors is carried out within the organisational context in which they have occurred which takes full account of the concept of a Just Culture.

29. An effective MTEMS is usually a requirement for an MRO to achieve a Part 145 or equivalent NAA Regulatory approval. The MEDA system developed by Boeing was the first widespread version of a MEMS and remains popular and widely imitated. Following the original MEDA initiative, the UK CAA has been especially active in this area and the description MEMS is also used by that agency as a specific description of its own system. Maintenance Error- Definition 30. The unintended failure to carry out a maintenance task in accordance with the requirements of that task and/or not working in accordance with the principles of good maintenance practice. Description 31. Aviation industry studies have found that the origin of as many as 20% of all in flight engine shutdowns can be traced to maintenance error.

32. Typical maintenance errors include:

Electrical wiring discrepancies. Loose objects left in airplane. Incorrect installation of components. Fitting of wrong parts. Inadequate lubrication. Access panels, fairings, or cowlings not secured. Fuel or oil caps and fuel panels not secured.

33. For installation errors on engines, one specific study found the following types of error: 6 Note: This article is a draft which contains articles extracted from other references which shall be included in the final paper

[Type text]

Draft Boroscope plug not refitted Engine Driven Pump (EDP) drive shaft seal not fitted Engine attachment bolts incorrectly fitted Anti Ice valves locked out Fire bottle squibs not fitted Fuel pipe not secured Magnetic Chip Detectors (MCDs) not fitted Prop spinner fitment not completed Fuel Control Unit (FCU) controls not fitted

[Type text]

34. The circumstances in which maintenance error occurs are the focus of human factors methodology Conclusion: Applications of TEM 35. TEM is both a philosophy of safety and a practical set of techniques. Originally designed to simultaneously capture performance and the context in which it occurs, TEM has demonstrated its usefulness in many settings. 36. Training: The application of MTEM in the training of aircraft technicians and engineers will help them to be more vigilant and improve situational awareness. 37. Line Operations Safety Audits (LOSA): Considered a best practice for normal operations monitoring and aviation safety by both ICAO and the FAA, TEM-based LOSAs continue to provide valuable diagnostic information about an airlines safety strengths and vulnerabilities. 38. Incident Reporting: Several US airlines now use TEM as the conceptual structure for their incident reporting systems. Reporting forms prompt pilots to report the threats that were present, the errors they may have made, how the event was managed, and how the event may have been avoided or handled better. Even pilots who have not had training in TEM are able to complete the reporting form, a fact that speaks to the intuitive nature of the TEM framework.

39. TEM has proved its utility in many safety management applications. As organizations and individuals continue to adopt TEM as a way to understand and enhance their performance, we hope that you too will see the utility of the TEM framework and find ways to incorporate TEM techniques into your own personal philosophy of safety.

7 Note: This article is a draft which contains articles extracted from other references which shall be included in the final paper