See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/257036217

A framework for identifying and analyzing sources of resilience and

brittleness: A case study of two air taxi carriers

Article  in  International Journal of Industrial Ergonomics · May 2012

DOI: 10.1016/j.ergon.2011.12.001

CITATIONS READS

24 260

2 authors:

Tarcisio Abreu Saurin Guido Carim Junior

Universidade Federal do Rio Grande do Sul Griffith University
208 PUBLICATIONS   3,280 CITATIONS    24 PUBLICATIONS   124 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Adaptações Estruturais e Funcionais no Ambiente de Trabalho em Decorrência da Pandemia do novo Corona vírus: Estudo no Hospital Moinhos de Vento View project

Projeto de Interação Acadêmica intitulado Capacitação em Produção Enxuta 2019-2021 View project

All content following this page was uploaded by Guido Carim Junior on 18 December 2017.

The user has requested enhancement of the downloaded file.

This article appeared in a journal published by Elsevier. The attached
copy is furnished to the author for internal non-commercial research
and education use, including for instruction at the authors institution
and sharing with colleagues.
Other uses, including reproduction and distribution, or selling or
licensing copies, or posting to personal, institutional or third party
websites are prohibited.
In most cases authors are permitted to post their version of the
article (e.g. in Word or Tex form) to their personal website or
institutional repository. Authors requiring further information
regarding Elsevier’s archiving and manuscript policies are
encouraged to visit:
http://www.elsevier.com/copyright
 Author's personal copy

International Journal of Industrial Ergonomics 42 (2012) 312e324

Contents lists available at SciVerse ScienceDirect

International Journal of Industrial Ergonomics

journal homepage: www.elsevier.com/locate/ergon

A framework for identifying and analyzing sources of resilience and brittleness:

A case study of two air taxi carriers
Tarcisio Abreu Saurin*, Guido Cesar Carim Junior
Programa de Pós-Graduação em Engenharia de Produção/Federal University of Rio Grande do Sul, Av. Osvaldo Aranha, 99, 5. andar, 90035-190 Porto Alegre-RS, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: The increasing interest in resilience engineering (RE) has led to a demand for frameworks that undertake
Received 17 June 2011 safety assessments from such a standpoint. However, the few existing frameworks have drawbacks, such
Received in revised form as not analyzing the sources of resilience (SRs) and the sources of brittleness (SBs) side-by-side. More-
1 November 2011
over, they limit themselves to investigating resilience in pre-determined units of analysis (e.g., teams),
Accepted 12 December 2011
neglecting the fact that resilience might be in any element of a socio-technical system. This article
Available online 12 January 2012
introduces a framework for identifying and analyzing SRs and SBs jointly, which do not constrain the
identification process to any specific unit of analysis within the investigated system. The sources should
Keywords:
Resilience engineering
be identified and analyzed across five categories: the opposite SR or SB; the risk from the SB; the
Brittleness effectiveness of the SR; those originating from either internal processes or the external environment;
Adaptations those arising from formal or informal practices. A case study of two air taxi carriers illustrates the
Complex systems application of the framework.
Air taxi carriers Relevance to industry: Resilience engineering (RE) is an emerging safety management paradigm con-
cerned with normal work, rather than emphasizing learning from incidents. The proposed framework
allows the identification and analysis of the most salient sources of resilience and brittleness. It can be
applied for investigating resilience at any unit of analysis within a socio-technical system, supporting the
identification of strengths and weaknesses from the RE perspective.
Ó 2011 Elsevier B.V. All rights reserved.

1. Introduction improvements due to the adaptations themselves; (c) the absence

Some paradigms that still are strongly rooted in industry have
been subjected to increasing criticism, such as the one that regards
the human being as the weak link in a complex system and who,
because of this, should be removed from the front-line and be
substituted by automation (Hollnagel and Woods, 2005). Besides
contributing to keeping alive what is probably the most detri-
mental assumption about the cause of accidents (i.e., humans are to
be blamed for accidents), this paradigm also denies the fact that,
most of the time, it is front-line operators’ ability to adapt that
keeps badly-designed systems operational (Dekker, 2006). Never-
theless, adaptations often create new hazards that are not recog-
nized by organizations, either because they are not detected or
because identifying them is unsettling for a number of reasons,
such as: (a) the overt recognition of adaptations might expose
management flaws that create the need for workers to adapt;
(b) constraining adaptations might compromise productivity
* Corresponding author. Tel.: þ55 (0) 51 3316 3491; fax: þ55 (0) 51 3316 4007.
E-mail addresses: saurin@ufrgs.br, tasaurin@yahoo.com.br (T.A. Saurin).
of overt recognition of adaptations makes it easier to blame
operators when things go wrong (Dekker, 2007).
The impossibility of having full control over and full knowledge
of complex systems (Cilliers, 1998) has also not been explicitly
taken into account when designing the safety management prac-
tices currently dominant in the industry. Thus, the limits and
systemic impacts of safety practices are not usually assessed, which
contributes to establishing a false feeling of safety and control.
While it is psychologically comfortable, this feeling is detrimental
to safety culture (Hollnagel, 2004).
In academic studies, safety management innovations have
been investigated by a number of communities of practice (Saleh
et al., 2010). These communities may be regarded as groups of
researchers who often also involve practitioners, and who share
a research agenda and a vocabulary. They approach a given subject
from a perspective that partially differs from and partially overlaps
with the perspectives adopted by other communities of practice
(Hoffman and Militello, 2009). This study is aligned with a commu-
nity of practice deemed resilience engineering (RE), which has rela-
tionships with other communities that to a greater or lesser extent
have implications for safety. Examples of such communities are those

0169-8141/$ e see front matter Ó 2011 Elsevier B.V. All rights reserved.
doi:10.1016/j.ergon.2011.12.001
 Author's personal copy
T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324
of high reliability organizations, cognitive systems engineering,
complex systems and safety culture.
RE might be defined as a safety management paradigm that aims
at identifying, analyzing and improving the resilience of systems
(Nemeth et al., 2009). For its part, resilience is the ability of a system
to adjust its functioning prior to, during, or following changes and
disturbances, so that it can sustain the operations required under
both expected and unexpected conditions (Hollnagel, 2011). Even
though the application of the concept of resilience to safety is fairly
recent, it has been used over years in other contexts, such as in
psychology, ecology and physics. In all of those areas, the purpose of
investigating resilience is to reach a better understanding of
a system’s ability to survive, adapt and recover, whether the system
is the human mind, an ecosystem, a material or an organization
(Hollnagel et al., 2006).
The increasing interest in RE has demanded frameworks for
identifying and analyzing sources of resilience (SRs) and their
opposite, sources of brittleness (SBs). However, some assessments
of resilience have not adopted any explicit analytical framework
(e.g., Gomes et al., 2009; Morel et al., 2009), which makes it difficult
to compare and generalize the results of different studies. Other
studies, while adopting clear analytical frameworks, limit them-
selves to investigating resilience in some pre-determined units of
analysis, such as the safety management system (Saurin and Carim
Junior, 2011; Hale and Heijer, 2006) and the behavior of small teams
(Furniss et al., 2011). While searching for resilience in specific
elements of a socio-technical system allows a deep investigation, it
also neglects the fact that resilience might be in any element of that
system. Another drawback of the existing frameworks (e.g., Furniss
et al., 2011) is that they do not consider resilience and brittleness
side-by-side. This means that they do not recognize the SBs that
demand the existence of the SRs, and so it is not questioned
whether the SRs could be unnecessary if the SBs were eliminated.
Moreover, it is not common for studies on RE to have a traceable
line from concrete observations to high-level resilient principles
(Furniss et al., 2011).
In this context, this study introduces a framework for identifying
and analyzing SRs and SBs which can be applied in socio-technical
systems, without constraining the identification process to any
specific unit of analysis within the selected system. Thus, it does not
matter whether the sources are in a team within the larger socio-
technical system selected or whether they are in the system’s
management routines. An effort should be made to identify the
sources wherever they emerge, within the boundaries established
for the investigation. The application of the framework is illustrated
by a case study of two air taxi carriers in Brazil. This service is
usually demanded either when clients need air transportation to
destinations not served by regular commercial flights or simply
to suit the passenger’s convenience (Sheehan, 2003). The choice
of this sector was driven by the fact that, both the operation
of an aircraft and the aviation industry as a whole, are widely
regarded as having strong characteristics of complex systems
(Perrow, 1984), and so they more emphatically demand that
RE principles be applied. Moreover, from 1999 to 2008, air taxi
carriers accounted for 23.2% of all aircraft accidents in Brazil
(CENIPA, 2010).
2. Principles of RE
There are several studies that have proposed characteristics and
design principles of resilient systems (Nemeth et al., 2009;
Hollnagel et al., 2008; Hollnagel et al., 2006). In this paper, the
principles proposed by Costella et al. (2009) are adopted, since they
compile the principles established by previous studies:
313
(a) Top management commitment: this implies that safety is
a core organizational value, rather than a temporary priority.
The adoption of this principle sets up a barrier against
production pressures on safety;
(b) Learn from both incidents and normal work (learning): RE
emphasizes learning from the analysis of normal work, while it
does not neglect learning from incidents. According to this
principle, monitoring the implementation of procedures is as
important as designing procedures, since the former contrib-
utes to reducing the gap between work as imagined by
managers and work as performed by front-line operatives;
(c) Increase flexibility (flexibility): since an underlying assumption
of RE is that human errors are inevitable, work system design
must be error-tolerant and recognize that variability manage-
ment is as important as variability reduction. In line with this,
designers should aim at increasing the variability that leads to
positive outcomes and decreasing the variability that leads to
unwanted events. This principle also implies that people at the
front-line (particularly first-level supervisors) are able to make
important decisions without having to wait unnecessarily for
management instructions;
(d) Be aware of system status (awareness): everyone in the system
should be aware both of their own current status and the status
of the defenses in relation to the limit of the loss of control. In
particular, awareness is critical both for anticipating the
changing nature of risks and for assessing the trade-off
between safety and production.
It is possible to trace a parallel between the principles put forward
by Costella et al. (2009) and the four cornerstones of RE proposed by
Hollnagel (2011), who elaborated on each of them in detail. The
principle of learning equates to the Hollnagel (2011) proposal that
resilient systems must know what has happened, i.e., they must know
how to learn the right lessons from the right experience. The principle
of flexibility is dealt with by Hollnagel (2011) to the extent that he
argues that resilient systems must know what to do, i.e., how to
respond to disruptions and disturbances by adjusting normal func-
tioning. The principle of awareness is implicit in the Hollnagel’s idea
that resilient systems must know what to look for (how to monitor
performance) and what to expect (how to anticipate future threats).
While top management commitment was not made explicit by
Hollnagel (2011), it is still maintained as a RE principle in this article
because it is a major prerequisite if, as desired, there is to be learning,
awareness and flexibility.
3. Framework for identifying and analyzing SBs and SRs
3.1. Who should apply the framework
In the case studies reported by this article, the framework was
applied by two researchers, who hold post-graduation degrees on
ergonomics and safety management. Nevertheless, the framework
was designed to be also applied by ergonomics practitioners. In fact,
whether researchers or practitioners are applying the framework, the
following qualifications are desirable: (a) to be acquainted with the
framework’s underlying theoretical background, especially on RE; (b)
possessing basic technical knowledge of the domain investigated,
although a stage of the application process may be specifically
designed for this purpose, if necessary (e.g., the researcher or prac-
titioner could attend training events in the company to be investi-
gated, before starting the investigation); and (c) preferably, either the
researcher or practitioner should not be an employee in the company
under investigation, since this may bias the data collection and
analysis process (e.g., he/she may erroneously take some issues
for granted, neglecting a deep investigation).
 Author's personal copy
314 T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324
3.2. How to identify SBs and SRs
Taking as a basis both the previously mentioned definition
of resilience proposed by Hollnagel (2011) and the perspective
on resilience by Furniss et al. (2011), a SR is defined in this study as
follows: any element of the socio-technical system (e.g., a manage-
ment routine) that supports performance adjustment, so the system
can respond to and keep functioning regardless of poor behavior,
poor design and poor circumstances, both during expected and
unexpected situations. As to the SBs, they are any element that plays
against performance adjustment, pushing the system towards its
limits (e.g., SBs can be poor behavior, poor design, etc.).
Collecting data to identify the sources should start with
a description of the socio-technical system, comprising its four sub-
systems (Hendrick and Kleiner, 2001): technical, social, work
system design and external environment. Since defining the
boundaries of any system depends on the objectives of the inves-
tigation (Cilliers, 2005), two questions (what are the objectives?
What are the boundaries?) should be explicitly answered before
starting data collection.
As to content, the description should give visibility to the gap
between the practices laid down and those that are really used,
emphasizing the contextual conditions that explain the use of
informal practices. Also, the description should include an overview
of the history of the system (Cilliers, 1998), since this might help to
understand its current status. The research strategy should take
a qualitative emphasis, such as case studies, action research and
ethnography. Data collection techniques should be similarly guided.
Thus, adequate sources of data are several types of interviews,
observations and document analysis, since they make it easier, in
comparison with quantitative approaches, both to understand the
context and to access multiple perspectives, which are critical issues
when investigating complex systems (Page, 2007; Crandall et al.,
2006; Anderson et al., 2005; Snook, 2000). Of course, quantitative
methods may be useful as well, especially for deepening the analysis
of data originated from qualitative methods (e.g., prioritizing SRs
and SBs, investigating the intensity of the relationships among the
sources). As an example of using quantitative methods in studies on
RE, Ferreira et al. (2011) used factor analysis to analyze data from
a questionnaire, which was designed to identify measurable aspects
of resilience in rail engineering planning.
In addition to the data collected to describe the socio-technical
system, other data should be collected specifically to assess the use
of RE principles. In order to do this, a protocol should be designed
that expands each RE principle into less abstract characteristics
and, for each of them, defines sources of data that enables them to
be assessed. This protocol should be domain specific, using tech-
nical language and sources of data that ought to be expected in that
context. Also, it is recommended that the protocol be designed after
characterizing the socio-technical system. Thus, areas of particular
interest for investigating SBs and SRs may well have already been
identified.
Based on the data collected in the previous stages, both the SBs
and SRs should be identified. Although the extraction of the sources
from the data collected is not a purely mechanistic process, guid-
ance may be provided, such as: (a) to pay attention to pieces of
evidence (e.g., excerpts from interviews, descriptions and analysis
of incidents) that match the definition of SR and SB previously
mentioned; (b) to pay attention to informal working practices,
since they are often associated with adjustments of performance
due to either unforeseen or undesired variability (i.e., resilience);
(c) to pay attention to humanetechnology interaction, since this is
a well-known moment where resilience and brittleness become
observable; and (d) to be open-minded to search for SBs and SRs
wherever they may emerge in the investigated system, whether in
terms of hierarchical levels, departments and/or management
routines. Moreover, some SBs and SRs will be less subtle than
others, and so a number of them may be quite obvious for analysts
who are qualified to apply the framework.
It is further proposed that the SBs be identified and stated before
the SRs, since the latter are often an adaptive response to the
former. Due to both the large volume of data that might have been
generated and to the multiple viewpoints from which the data
might be interpreted, it is desirable that at least two researchers
codify the sources based on the raw data. This same approach has
been used by other authors in similar studies, such as by O’Connor
et al. (2008), in the codification of non-technical skills. In order to
minimize bias, ideally both researchers should analyze the raw data
separately and, later, they should meet to agree on what the sources
are and how they should be stated.
3.3. How to analyze the SBs and SRs
After being identified, the sources should be analyzed and
classified based on a set of categories. Similarly to what was sug-
gested with regard to identifying the sources, at least two
researchers should be involved in such classification. As to the SBs,
the categories proposed are as follows:
(a) the opposite SR: the SBs frequently trigger adaptive responses,
either at individual, team or organizational level. So, the lack of
an opposite SR is usually undesirable;
(b) the risk associated with the SB: assessing the risks supports
prioritizing the sources, and is based on assessing the proba-
bility of their occurrence and severity (Fig. 1). In order to esti-
mate these parameters, the assumption should be made that
they are independent of the adaptive responses to the SBs e the
impacts of the SRs will be analyzed as a specific category. Of
course, the risk assessment might be based on data collected in
the previous stages of the study. With regard to probability,
how regular the incidence of the source is (e.g., in all flights,
rarely, etc.) should be considered. As to the severity, the
immediate events triggered by the source should be taken into
account. Ultimately, all SBs might contribute to an accident
and, if this is considered as the benchmark impact, it is not
possible to discriminate the severity of each SB;
(c) the origins of the SBs, considering two dimensions. On the one
hand, it is necessary to assess whether the sources originate
either from formal or informal work organization. SBs arising
from the formal work organization are typically associated with
failures in documented management routines. Informal SBs are
more subtle and are not clearly connected to any management
routine. Both informal and formal sources are equally undesir-
able, since many formal SBs indicate the ineffectiveness of the
organization of work laid down, while many informal SBs point
out to interactions that are difficult to control and anticipate. On
the other hand, whether the source comes from the internal
Severity
Low Moderate High
High
Probability Moderate
Low
Low risk
Moderate risk
High risk
Fig. 1. Matrix for assessing the risk of the SBs.
 Author's personal copy
T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324
processes or from the external environment should be assessed.
The existence of external origins is worse, since the organization
has a lesser degree of control over them.
As to the SRs, the proposed categories are presented below:
(a) the opposite SB: the lack of an opposite SB means both that the SR
is not an adaptive response and that it has not had any detri-
mental systemic impact. However, it should not be taken for
granted that the SR will continue to be of this nature in the future;
(b) the effectiveness of the SR: this assessment takes into account
the limitations, shortcomings and new hazards introduced by
the source. Three levels of performance are proposed, referred
to as high, moderate and low effectiveness;
(c) the origins of the source, adopting the same categories that
were applied to assess the origins of the SB. The SRs that
originated from the formal organization of work are usually
connected to documented management routines. Informal SRs
emerge from adaptations undertaken by groups or individuals,
normally without the consent of higher hierarchical levels and
without being incorporated into the documented management
routines. It is preferable that the highest possible portion of the
SRs arises from the formal organization of work, which points
out the availability of resources both for maintaining and
improving them. The existence of informal SBs normally indi-
cates that the adaptive capacity has been used to fill out gaps in
badly-designed systems, rather than being used to improve
good designs. As to the other category to assess the origin of the
sources (internal versus external), it is preferable that as few
sources as possible spring from the external environment.
For both the SBs and the SRs, an analysis could also be made of
how they are related to RE principles. Nevertheless, the framework
does not prescribe this analysis as mandatory, since the impact of
the sources on the principles tends to be diffused over all of them.
This is consistent with the systemic nature of RE.
3.4. Recommendations for designing an action plan to deal with the
SRs and SBs
The action plan should be based on the assigned classification
for each source. Thus, some actions could be prioritized, such as: (a)
controlling the high risk SBs and the low effective SRs; (b) checking
whether the informal SRs are worth being formalized or if they are
doing more harm than good; (c) checking what the company can
do, both to minimize the impacts of the external SBs and to be less
dependent on the external SRs; and (d) considering either existing
or potential threats to the highly effective SRs, and thus not taking
them for granted.
Another objective of the action plan should be to support the
comprehension and monitoring of the interactions among the
sources, since major vulnerabilities in a complex system usual-
ly arise from interactions among its elements (Perrow, 1984).
Although some interactions might be quite obvious (e.g., the rela-
tionship between opposite SBs and SRs), their understanding and
monitoring can be supported through the use of existing methods,
such as concept maps and performance measurement systems.
4. Research method
4.1. Defining the boundaries of the socio-technical systems
investigated
The framework was applied in two air taxi carriers in Brazil: one
of them was based in the State of Rio Grande do Sul (company A)
315
and the other in the State of São Paulo (company B). By choosing
two companies, it was possible to investigate the framework in
markedly different organizational environments, and thus allowed
a contrasting inter-case comparison to be made. Moreover, these
two companies were selected because, differently from other
companies that had been contacted by the research team, they
allowed full access to the company facilities and staff, as required by
the data collection techniques.
The boundaries of the systems under study were geographi-
cally defined by the physical facilities of both companies as well as
by the airplanes themselves. Thus, everything that occurred
outside these boundaries was considered to be in the domain of
the external environment. This included, for example, the actions
of regulators, suppliers, air traffic control and the infrastructure of
airports. Thus, these actors were relevant to this study only to
the extent that they contributed for the existence of SBs and
SRs that had an impact on the operations of the companies
investigated.
4.2. Identifying and analyzing the SBs and SRs
In company A, the field study took four months, which involved
eight visits by one researcher to the company facilities e each visit
took approximately 3 h. In company B, the field study took three
months, which involved three visits e each one taking approxi-
mately 8 h. In both companies, the field study was preceded by
a meeting, in which the researcher introduced the objectives and
methodological procedures of the research to the company direc-
tors and staff.
In line with the proposed framework, in both companies the
field study started by describing the sub-systems of the socio-
technical system. With regard to the external environment, a liter-
ature review was undertaken on the legal environment shared by
all airlines in Brazil (Brasil, 2008). Thus, the minimum legal, tech-
nical and managerial requirements that should be complied with
by an air taxi carrier were identified. Characterizing other external
environment elements (e.g., political, educational, cultural and
economic ones) is particularly difficult for airlines, since they are
exposed to the environment of all destinations where they have
operations.
As to the social sub-system, workers were characterized according
to their marital status, age, sex, job, remuneration schemes, length of
employment in the company, academic background and previous
professional experiences. For the technical sub-system, emphasis was
placed on the characterization of the facilities (e.g., buildings for
administrative work and for aircraft maintenance) and aircraft (e.g.,
how and when they were purchased; average daily operation time,
operational costs; required training for pilots).
The work organization sub-system was characterized accord-
ing to three categories: (a) departments and hierarchical levels;
(b) flight, maintenance and administrative processes; (c) tasks
carried out by each worker. Identifying the differences between
work laid down and real work was based on multiple sources of
data, such as documents (e.g., standard operating procedures),
observations (e.g., during real flights) and semi-structured inter-
views. The interviews included questions such as: what are your
activities? Are you able to remember procedures that are difficult
to comply with? Why? Could you give examples of adapting
procedures? How often do you interact with other departments
and colleagues? How are your activities influenced by those of
your colleagues?
According to the framework, the next stage of the field study
emphasized the assessment of the RE principles. Thus, a data
collection protocol was designed, based on breaking down
each RE principle into less abstract characteristics. The sources of
 Author's personal copy
316 T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324
data were defined, to a large extent, based on insights obtained
during the characterization of the socio-technical system. That
stage of the research had pointed out both issues likely to
be associated with SRs and SBs (e.g., procedures and training)
and available sources of data that were linked to those issues
(e.g., standard operations procedures, meetings for planning
flights).
An extract of the protocol, dedicated to the assessment of the
principle of flexibility, is presented in Appendix A. As a whole, the
protocol includes thirty-two recommendations for observations,
sixty questions to be used in interviews and fifteen recommenda-
tions on documents to be consulted. Regardless of differences in the
organizational design of both companies, the vast majority of data
sources were available in both companies, thus indicating that the
protocol seems to have some generalizability for air taxi carriers.
Some of the staff interviewed while characterizing the socio-
technical system were interviewed again during the assessment
of the RE principles.
In addition to the very nature of the adopted sources of
evidence, which were in line with ethnographic approaches, it is
worth stressing other aspects of the data collection process that
made it easier for the identification of the informal work organi-
zation, such as: (a) the small size of both companies (15 employees
in company A and 11 in company B), which tended to give visibility
to informal practices, besides making it easier to interview almost
the entire staff of each company (67% in company A and 80% in
company B); (b) the fact that the researcher who collected the raw
data in the field was a domain expert; and (c) the social climate in
each company. In company A, employees regarded the interviews
as a welcome opportunity to express dissatisfaction with their
working environment, and thus not being concerned with
revealing poor informal practices. The opposite situation occurred
in company B, as the employees regarded the interviews as an
opportunity to express their satisfaction and pride, while also not
being concerned with pointing out shortcomings associated with
informal practices e discussing these issues was seen as positive to
reinforce a safety culture.
With the exception of one employee in company A, who did not
authorize recording, all the remaining interviews were tape-
recorded, transcribed and analyzed based on techniques of
content analysis. Therefore, the researcher who collected the raw
data moved on to identifying the words, statements and phrases
used and their relationship to the objectives of the study.
The lists of SBs and SRs drawn up by this researcher were later
refined after thoroughly discussing them with another member of
the research team. In this study, as opposed to the ideal procedure
for applying the framework (see Section 3.2), only this researcher
coded the sources directly from the raw data. This drawback
happened both because the other researcher was not directly
involved in the data collection process (his role was to support the
design of the project) and because he was not a domain expert.
Nevertheless, his support was important to state the SBs and SRs
as precisely as possible, besides acting as a critical backup to
the analysis undertaken by the researcher who collected the raw
data.
It is also worth noting that, in company B, there was a work-
shop shortly before the study ended in order to present and
discuss the results with company representatives. Company B
representatives did not consider it necessary to draw up a formal
action plan, probably because they were satisfied with their
current performance and because most of the detected short-
comings could not be “solved” with simplistic measures that
could be detailed in the short-term. A similar event was not
conducted in company A, due to the lack of interest shown in this
by its staff.
Fig. 2. One of company A’s airplanes.
5. Results
5.1. Company A
5.1.1. Main characteristics of the socio-technical system of company
A
This company had been operating for forty years, and had been
managed by a number of different owners over this period. Its
facilities are located within a hangar, which houses the aircraft and
both the administrative and operational staff. The fleet comprises
three airplanes (Fig. 2). All of them were manufactured in the 70s
and they can carry a maximum of nine passengers. The company
has permission to undertake flights in Brazil and in some South
American countries, and can provide transportation services for
passengers and goods. Maintaining and housing of third parties’
airplanes are other services provided by company A.
The workforce is, on average, forty years old, and consists of
fifteen people; seven of them are pilots. Four pilots also have
administrative jobs the existence of which is required by regula-
tions, such as having someone in charge of flight safety. This
multifunctional characteristic is permitted by regulations and, to
a great extent, is a result of the seasonality of the demands for
flights.
In line with the framework, differences between work laid down
and real work were investigated. For instance, it was detected that
the assignment of pilots to flights often differs from what was
planned. This usually happens in order to ensure that all pilots have
a similar number of monthly flight hours. This number has a strong
impact on pilots’ remuneration, since there is a variable portion,
which is dependent on the number of hours flown.
The high turnover rate of pilots is another characteristic of this
company, since the typical length of a pilot’s stay in the company is
from six months to one year. According to the interviews, pilots are
usually interested in working for this company as a means to
accumulate hours of flying-time and, therefore, increasing the
possibility of their being hired by major commercial airlines, in
which working conditions are regarded as better than in the air taxi
sector.
5.1.2. SBs in company A
Two out of the thirteen SBs identified in this company (Fig. 3)
were chosen to illustrate the application of the framework. SB1
concerns both the obsolescence and low reliability of the airplanes’
navigation instruments, which is a particularly serious drawback
when pilots do not have any visual references. In such situations,
 Author's personal copy

T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324 317

SBs Corresponding SRs Risk Origin

(1) Obsolescence and unreliability of
navigation instruments
(2) Lack of guidelines on how to refuse a
flight on account of adverse weather
conditions
(3) All the airplanes are technologically
obsolescent,but some more so than others
(4) The automatic pilot of all the airplanes
is permanently out of order
(5) Pressure for production, coming from
top management, which can lead to flight
sales and filights being made under unsafe
conditions
(6) Lack of documented and and up-to-date
procedures
(7) Relationship difficulties between pilots
(8) Four of the seven pilots do not work
full-time for the company and are only
present to make the flights
(9) Incident reports do not occur by means
of a system formally designed for this
(10) The most important airports for the
company are in poor condition
(11) Inversions in the hierarchy, since staff
who are heads of administrative posts, are
subordinates during flights
(12) High turn-over among the staff who
remain in the company, on average from six
months to one year , as they are aiming to
accumulate flying hours to get jobs in
regular airline transport companies
(13) Half the remuneration of the crew is
variable, as it depends on monthly flying
hours. This encourages flights in unsafe
conditions, so as to improve remuneration
GPS devices were added to the control panels of all High risk: high probability and Internal and formal
airplanes, as a means of partial compensation for the high severity
unreliable navigation instruments
The more experienced pilots use informal rules and teach High risk: high probability and Internal and formal
them to the less experienced moderate severity
If possible, the flights are made in the airplane in best High risk: high probability and Internal and formal
condition; in addition, there are periodic training flights to moderate severity
simulate emergencies
The co-pilots, when they perform the functions of pilot High risk: high probability and Internal and formal
monitoring, also control the airplane by means of the yoke, a high severity
task which the automatic pilot would normally do
Crews try to refuse flights with support from other staff, High risk: moderate Internal and informal
who also issue a report on the decision probability and high severity
The pilots hold informal meetings to exchange ideas on the High risk: high probability and Internal and formal
procedures and define standards moderate severity
The pilots try to refuse a flight when they do not feel Low risk: moderate probability Internal and informal
comfortable with the other crew appointed and low severity
The three pilots who have a fulltime commitment to the Moderate risk: high Internal and informal
company do administrative tasks which the other pilots probability and low severity
should do
The reports of unsafe situations are made informally Moderate risk: high Internal and formal
between the crews probability and low severity
Crews exchange information about the operational Moderate risk: high External and informal
conditions of the airports the company uses probability and low severity
None Low risk: moderate probability Internal and formal
and low severity
None High risk: high probability and Internal and informal
moderate severity
None High risk: high probability and Internal and formal
moderate severity

Fig. 3. SBs in company A.

the flight crew divides its attention span between the navigation
instruments and a GPS device, which was added to the control
panel in order to partially compensate for the unreliable instru-
ments. However, the pilots reported that, sometimes, they forget to
check the GPS and, as a result, the flight paths differ from those
that were planned.
This SB has been associated with incidents. According to the
report of a pilot, one of those events occurred during the final
approach at the destination airport, when the air traffic controller
requested a curve to be made in a specific direction. While the pilot
was doing the maneuver, based only on the information provided
by the navigation instruments, he was alerted by the controller that
the airplane was going in the wrong direction (in fact, they were on
a collision course with another plane), which prompted an imme-
diate correction. After this call, the crew checked the GPS and
noticed that the information displayed on this device differed from
that displayed on the instruments, and that it was consistent
with the controller’s request. According to the report of a first-
officer, “[...] this type of situation is frequent, rather than being an
exception”.
This SB is known to all staff and top management. However,
according to the director, there are no plans to substitute the
instruments, due to the costs involved e this will happen only
when new airplanes are acquired. Thus, the origin of this SB was
classified as internal and formal, since the processes for equipment
acquisition and improvement have not been brought into effect.
Although none of the interviewees were able to pinpoint when this
SB began, all of them reported that these problems existed in the
three airplanes, there being slight variations in terms of intensity.
Thus, this SB was regarded as of high probability and of high
severity.
It can be argued that this SB has relationships with all RE
principles. Top management commitment is clearly low, as it
tolerates a serious drawback that it is fully aware of. Learning has
not occurred, as incidents like the one previously mentioned have
not triggered improvements, but rather brittle adaptations (e.g., the
GPS) that have side-effects of their own. The principle of being
flexible has also been hindered, as pilots are overloaded by simul-
taneously having to monitor the GPS and the navigation instru-
ments, thus decreasing their cognitive capacity to undertake other
tasks. The principle of awareness is also compromised to the extent
that pilots do not have a clear picture of the real status of
operations.
The second SB chosen to illustrate the use of the framework is
concerned with the lack of procedures to support pilots’ decision-
making on whether or not to take off, due to the meteorological
conditions. As an example of the impact of this SB, there was
a report on a flight in which the crew made the decision to return to
the airport a few minutes after taking-off, as a result of bad
weather. The captain of this flight was requested to give an account
of his decision to top management, who was upset as the company
could not charge the client. Moreover, the passengers fell sick
during the flight and, according to the flight coordinator, they no
longer bought services from company A. This is likely not to have
been an isolated event, as all crew members reported that they
either have been pressed to fly under unfavorable meteorological
 Author's personal copy

318 T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324

conditions or they have witnessed fellow colleagues being
submitted to this type of pressure. Similarly to what happened in
the example previously mentioned, the crew often resigns itself to
accepting such pressures.
This SB has relationships with all RE principles. Refusing work
due to lack of safety largely depends on both top management
support and clear guidelines on how to do this in practical ways
(e.g., does the refusal have to be documented and investigated?).
The lack of formal procedures for supporting the decision-making
on take-off hinders the principle of learning, to the extent that
there is no benchmark to compare the assumptions made by pilots
with a set of assumptions supposedly agreed upon at the corporate
level. As to the principle of awareness, this is compromised since
pilots have no guidelines to assess whether the meteorological
conditions justify a refusal to take-off. Awareness on safety limits is
also biased by the fact that there are few flights and the pilots’
remuneration depends on the number of hours flown. Moreover,
the information on meteorological conditions has low reliability
and it is simply not available in some of the places where the
company operates.
Concerning the principle of flexibility, one of its core dimensions
is hindered, as the front-line operators have no real autonomy to
make a key decision which impacts on safety. Indeed, there were
many pieces of evidence that indicated there are strong production
pressures. According to reports by the general director and pilots,
there have been situations in which pilots were substituted because
they refused to fly in circumstances they regarded as unsafe. Also,
the department responsible for selling flights seems to be partic-
ularly under pressure, especially during periods of low demand
from clients. For example, the coordinator of this department
reported that he has already been pressed by top management to
sell flights bound for airports that were not fully compatible with
operational characteristics of the company’s fleet.
This SB has also been classified as formal and of internal origin,
since the company has staff that are, at least formally, responsible
for designing procedures, such as the ones on refusing unsafe
flights. This SB was regarded as of moderate severity and of high
probability, since it takes place during all flights and periods of the
year.
5.1.3. SRs in company A
Ten SRs were identified in this company (Fig. 4). The two chosen
to illustrate the application of the framework are the SRs opposite
the two SBs detailed in the previous item. SR1, associated with the
use of the GPS, corresponds to the adaptation mentioned above
which compensates for the unreliable navigation instruments. Of
course, this has a number of limitations as an adaptive response,
such as: the control panel was not designed to hold a GPS, and
therefore its location was improvised in a place that does allow
perfect visibility; as the GPS does not present all the information
displayed on the navigation instruments, the pilots still need to
check the latter. Attenuating the overload generated by monitoring

SRs Corresponding SBs Effectiveness of the SRs Origin

(1) GPS equipment was installed in all the
airplanes, with the aim of compensating, partially,
for the failures in navigation instruments
(2) Given the lack of formal guidelines on refusing
flights due to adverse weather conditions, the most
experienced pilots use informal rules and teach the
less experienced
(3) If possible, flights are made in the airplane in
the best condition, thus avoiding those with the
graetest technological obsolescence
(4) When the co-pilots perform the functions of
pilot monitoring,they also control the airplane by
means of the yoke, a task which should normally
done by the automatic pilot
(5) When they judge that conditions are unsafe, the
pilots try to refuse flights with the support of other
staff, by requesting that the latter issue a report on
the decision
(6) The pilots hold informal meetings to exchange
ideas on the procedures and to define operational
standards
(7) The pilots try to refuse flights when they do not
feel comfortable with the crew appointed
(8) The three pilots who are full-time in the
company undertake administrative tasks which the
other pilots should do
Obsolescence of the navigation
instruments
Lack of guidelines for refusing to
fly on acount of adverse weather
conditions
All the airplanes are obsolescent
technologically, but some more
so than others
The automatic pilot of all the
airplanes is permanently out of
order
Pressures for production, coming
from top management, which can
lead to flight sales and flights in
unsafe conditions
Lack of documented and up-to-
date procedures
Difficulties in relationships
between the pilots
Four of the seven pilots do not
work full-time for the company,
and are only physically present
when they are going to fly
Low effectiveness: the pilots´ attention is divided Internal and formal
between two sources of incomplete and imprecise
information
Moderate effectiveness: although the informal Internal and informal
rules seem adequate, sometimes they are not
followed due to organizational pressures to have
flights undertaken
Low effectiveness: the company only has three Internal and informal
airplanes and this means that it is difficult to
avoid those in the poorest condition, which are
more susceptible to break-downs and
emergencies during flights
Low effectiveness: the lack of an automatic pilot Internal and informal
adds to the co-pilots’ workload, resulting in
symptoms of somnolence and tiredness
Moderate effectiveness: despite actions like these, Internal and informal
the various reports of flights being undertaken in
unsafe conditions indicate that in general, the
organizational pressures predominate
Low effectiveness: as the meetings are informal, Internal and informal
they do not result in written documents which can
be consulted during flights and form a basis for
training events
Low effectiveness: due to the small number of Internal and informal
pilots, the refusal to work with certain colleagues
is frequently innocuous besides not tackling the
roots of the problem
Low effectiveness: the pilots who remain on full- Internal and formal
time duty in the company are overloaded with
administrative tasks besides which there are no
strategies to guarantee that non-full-time pilots
kep themselves fully informed about subjects of
interest for the safety of operations

(9) Incident reports are made informally between Incident reports are not compiled
the crews by means of a formally designed
system for this
(10) There is exchange of information between the The most important airports for
crews about operational conditions at the airports the company are in poor
used by the company condition
Moderate effectiveness: due to the informality as Internal and informal
well as to the four pilots not working full-time for
the company, it is possible that not all staff are
informed of incidents
Moderate effectiveness: for the same reasons as Internal and informal
cited in FR 8

Fig. 4. SRs in company A.

 Author's personal copy
T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324
two devices is the fact that the fairly low speed of the aircraft
reduces the time pressure on decision-making.
This SR supports the implementation of two RE principles:
flexibility, to the extent that the use of the GPS is an example
of variability management rather than variability elimination;
awareness, since, regardless of the shortcomings of the adaptation,
pilots regard that using the GPS reduces uncertainty on the
airplane’s position, in comparison with the situation without this
device. As to the origin, this source was classified as internal
and formal, since people from all hierarchical levels reached
a consensus on the appropriateness of using the GPS.
The second SR discussed in this item is a response to the non-
existence of guidelines to support decision-making on whether or
not to take-off due to meteorological conditions. In this case, the
adaptation concerns the development, by flight crew, of informal
rules on take-off decisions. These rules are informally transmitted
from the more experienced staff to the less experienced, taking into
account factors such as the technological limitations of the fleet, the
conditions of the runways and passenger comfort. For example,
a pilot reported that, if there is a flight towards the south and there
is evidence of a polar air mass coming from the south, the flight
should be canceled. However, if this flight is to the north and it is
not necessary to return to the base within a short timeframe, the
flight can take place, since the polar air mass is coming from
the south.
This SR is associated with two RE principles. On the one hand,
learning is shown to the extent that the informal rules are a result
of crew members’ past experiences. On the other hand, the prin-
ciple of awareness is implemented, as the rules help support the
informal risk assessment that pilots make when deciding whether
or not to take-off. This SR was classified as both internal and
informal, as it results from spontaneous initiatives by staff, without
being made known to higher-ranking members of staff. The effec-
tiveness of this SR was considered to be moderate, since informal
rules are not usually the only input for decision-making. Indeed,
a number of other factors, such as organizational pressures, might
reduce the importance of this SR.
5.2. Company B
5.2.1. Main characteristics of the socio-technical system of
company B
This company was founded in 2005 with the primary purpose of
providing air transport services to its owners. They are also direc-
tors of the holding company that controls it. However, when there
is no demand from the owners, the aircraft are made available to
provide services to external customers.
The fact that, in practice, the company does not aim to make
a profit, is acknowledged by staff as the fundamental factor for
safety. This point becomes clear from a report of the head of
maintenance: “[...] not prioritizing profit facilitates the prioritiza-
tion of safety. Our sole objective is to transport the directors safely”.
Since top management is the main client, the resources for safety
are usually made available without great difficulties.
The company owns two jet aircraft which have not been in
service for very long (Fig. 5). Control of the aircraft is, for most of the
flight, maintained by the autopilot and a flight management
computer. One of the aircraft, which can carry nine passengers and
two crew members, is used for destinations within Brazil. The other
one, which can carry ten passengers and two crew members, is
used for international flights or for long distances within Brazil.
The staff consists of eleven people, six of whom are pilots.
Similar to Company A, the average age of staff is forty. Two pilots
have qualifications in engineering (aeronautical and electrical)
and have worked in this role in a major aircraft factory. All crew
319
Fig. 5. One of company B’s airplanes.
members also have administrative positions, such as head of
maintenance and of operations.
The crew receives fixed salaries with no variable remuneration
for hours flown. According to the reports obtained, this form of
remuneration reduces the pressures on flights taking place in
unsafe conditions. According to one commander, “the pilot should
expect to earn for waiting and not for flying. The opposite of most
air taxi companies that I know and have worked with occurs here”.
The high salary compared to other air taxi companies, is an
attraction of the company. Thus, there is no need for staff to have
other sources to complement their income and they can devote
themselves fully to the company.
Although the organogram indicates there is a director general
(one of the owners), he is located at some physical distance from
the headquarters, which leads to staff considering that the heads of
maintenance and of operations are the top level managers for day-
to-day business. In fact, it was clear that both have authority and
are the channel of communication between other staff and top
management.
5.2.2. SBs in company B
The SBs identified in company B are listed in Fig. 6. SB1 was
chosen to illustrate the application of the framework, referring to
pressures from the owners. According to a pilot’s report, illustrating
these pressures, during a recent flight, one of the owners requested
a landing at an airport, which was not the one previously requested.
The reason for the request was that the cost of landing at this other
airport would be lower, despite its offering poorer safety condi-
tions. In this case, the commander did not follow the director’s
advice, and landed at the airport that he considered safer. It is
worth pointing out that, shortly after the conclusion of this study,
there was a serious accident involving a private aircraft at the
airport at which the owners would have preferred to land, which is
well-known in Brazilian aeronautic circles for being difficult to land
at, especially at night.
Besides the choice of destination airports, there were signs of
pressures that influenced the decision of whether or not to take-off.
According to some reports, there were situations in which the pilots
did not consider it safe to undertake the flight, but due to requests
from the owners or members of their families, the decisions were
re-evaluated.
This source adversely affects two RE principles: flexibility,
because the pressures from top management are a threat to the
autonomy of the pilots; top management commitment, taking the
origin of the pressures into consideration. The probability of
the source manifesting itself is low, since, according to the pilots’
 Author's personal copy

320 T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324

SBs Corresponding SRs Risk Origin

(1) Pressure by the owners to undertake flights in
conditions which the pilots consider unsafe
(2) Flights should be undertaken to destinations not
operated before or not operated regularly
(3) The company´s two airplanes have very
different flight performances (e.g., autonomy of
flight, performance in take-offs and landings, etc.)
(4) Difficulty of reserving slots at the airport where
the company´s operational base is situated
(5) There is no crisis management plan
(6) The staff does not take part in strategic decisions
made by the directors of the holding company (e.g.,
technical support on purchase and sale of airplanes,
choice of airports of preference)
The pilots’ decisions on whether or not to Moderate risk: low probability and Internal and
undertake flights, are usually communicated to high severity informal
the heads of operations and maintenance, who,
then, pass these on to the owners
The pilots seek information from colleagues of Moderate risk: moderate probability External and formal
other companies or cancel flights and moderate severity
The pilots usually refuse to fly the airplane which Moderate risk: high probability and Internal and formal
has the lower performance low severity
The lack of slots can cause delays to flights, a Low risk: moderate probability and External and formal
problem which is anticipated by the flight low severity
coordinator and communicated in good time to
passengers
Each pilot carries on him, cards with phone High risk: high probability and high Internal and formal
numbers to be called in case incidents occur severity
Informally, the staff give their opinions to the Moderate risk: high probability and Internal and formal
heads of maintenance and operations, who pass low severity
them on to the owners

Fig. 6. SBs in company B.

reports, this occurs on relatively rare occasions, such as when the
owners travel to sign trade agreements or to tourist destinations
where there are airports which have a poor infrastructure. The
severity is high, to the extent that it may lead to undertaking a flight
under precarious safety conditions. The source was classified as
internal and informal in origin, since it is associated with owners’
behaviors, and has no links to any formal organizational routine.
5.2.3. SRs in company B
In this company, eleven SRs (Fig. 7) were identified. SRs 1, 2 and
3 represent a counterpoint to SBs 1, 2 and 3, listed in Fig. 6. Indeed,
although pressures from the owners sometimes occur, staff has
a level of autonomy probably higher than in other air taxi compa-
nies, such as company A. In part, this autonomy is due to the owners
(top management) not being physically present in day-to-day
business, but they do monitor at a distance, in the main office of
the holding company. Furthermore, the decisions on choosing
airports, as well as undertaking the flight or not, are usually
communicated by the pilots to the heads of maintenance and of
operations, who then communicate them to the owners. This
strategy reduces the pressure on pilots, to the extent that their
decisions are backed by higher hierarchical levels, and have greater
impact on the owners.
The head of operations and the head of maintenance reported
that they hardly ever depend on top management to conduct their
activities. These staff members, due to their great experience, are
seen as reference points by colleagues and the owners, who dele-
gate the main day-to-day decisions to them. In the words of
one of the commanders, “the company’s heads of maintenance
and operations run the company and we do our work with their
assistance. They are very open to suggestions”. “For me, top
management are the heads of maintenance and operations,” is how
one of the co-pilots sums this up.
Thus, it is clear that the heads cited are the main interlocutors
with top management, and are active both in requesting resources
and in absorbing pressure from top management. There were also
signs that they seek to take decisions by consensus with the other
employees. One example of refusing a flight illustrates autonomy
and decentralization of decision-making. On being consulted about
a possible charter flight, the flight coordinator, even before
submitting a budget to the client, consulted the pilots about the
viability of the operation of aircraft at the airport concerned. The
pilots concluded that, in the vicinity of the runway, there were trees
and hills which made the operation difficult, especially at night and
on rainy days. The final decision was to refuse the flight because,
given the circumstances, the risk was not acceptable. This kind of
attitude is appreciated by the group, as shown in a report from one
of the commanders on the episode cited: “I thought that was great,
because what I’ve seen a lot is that a pilot does things [wrong ones]
to keep his job. Here we put our cards on the table: if it can work, it
works; if it’s not on, no way”.
SR1 operationalizes two RE principles: (a) flexibility, in that
front-line employees make important decisions without depending
on higher ranks, (b) awareness, since decentralized decision-
making requires that all employees understand the safety limits,
and do not simply follow orders from higher levels. Regarding the
origin, the source was classified as internal and informal, since the
autonomy granted to the heads of operations and maintenance
responsibilities do not arise from documented responsibilities, but
rather emerged naturally as a result of physical distance from top
management. This SR was classified as moderately effective,
because examples were still detected when the pressure from the
owners overrides the autonomy of the crews.
6. Discussion and conclusions
6.1. Overall assessment of companies A and B
Figs. 8 and 9 summarize the results of the assessment for both
companies. In these figures, some sources were identified by the
initial “S”, which refers to those that are associated with both
resilience and brittleness. In Fig. 8, the arrows located in the lower
right corner represent the SBs that do not have corresponding SRs.
These SBs exert a pressure which is detrimental to all the other
sources, as they push them to the zone of high risk and low effec-
tiveness. In Fig. 9, the arrows located in the upper left corner,
represent the SRs that do not have any opposite SBs. In this case,
those SRs push all the other sources to the zone of low risk and high
effectiveness. The different sizes of the arrows show the relative
intensity of the risks and effectiveness. Thus, in Fig. 8, SB11 has
a lower risk than SB12 and SB13. In Fig. 9, SR8 has a lower effec-
tiveness than SR7, SR9, SR10 and SR11.
Given the results of the assessment in both companies, there are
pieces of evidence that safety in company A has deteriorated
substantially more than in company B, such as: (a) the higher
percentage of SBs in company A (54% of all sources, against 17% in
company B); (b) the higher percentage of SRs with low effective-
ness in company A (60% of all SRs, against 18% in company B);
(c) the fact that, unlike company A, company B has five SRs without
corresponding SBs, which highlights the proactive nature of these
 Author's personal copy

T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324 321

SRs Corresponding SBs Effectiveness of the SRs Origin

(1) The pilots’ decisions on whether or not to undertake Pressure by the owners to
flights, are usually communicated to the heads of operations undertake flights in conditions
and maintenance, who, then, pass these on to the owners which the pilots consider unsafe:
flights which should be
(2) The pilots seek information from colleagues of other undertaken to destinations not
companies or cancel flights to airports that are never or operated before or not operated
rarely operated regularly; the company´s two
airplanes have very different
(3) The pilots usually refuse to fly the airplane which has flight performances
the lower performance
(4) Anticipation of the lack of slots at airports and Difficulty of reserving slots at the
communication to clients in good time, which minimizes airport where the company´s
upsets operational base is situated
(5) Each pilot carries on him, cards with phone numbers to Lack of a crisis management
be called in case incidents occur plan
Sources (1), (2) and (3): moderate Internal and informal
effectiveness. There is evidence that the
pilots usually refuse to fly when
conditions are unsafe. In addition, the
company seems to have a participative
culture and one in which staff enjoy good
relationships with each other and with the
heads of department
Moderate effectiveness: the fact of the Internal and informal
operational base being at the critical
airport facilitates access to information on
slots; however, the company does not
have the power to avoid this brittleness
Low effectiveness: the pilots’ strategy has Internal and informal
a very limited impact

(6) The heads of maintenance and operations have good The staff does not take part in Low effectiveness: the heads probably Internal and informal
relationships with the pilots, which encourages the latter to strategic decisions taken by the register the opinions of the other staff
give their opinions on the company´s strategic decisions directors of the holding company members. However, the informality does
(e.g. technical support on not ensure that feedback will be given to
purchase and sale of airplanes, those who offered opinions
choice of airports of preference)
(7) Autonomy of the pilots to define who will be the crew None High effectiveness: this seems to be a Internal and informal
members and to define schedules and days of work string source of satisfaction for the pilots,
as it contributes to having more cohesive
teams
(8) The level of automation of the airplanes permits the None Moderate effectiveness: although the Internal and formal
pilots to intervene manually at any stage of the flight, thus definition of the ideal level of automation
providing a certain flexibility is a complex question, the pilots appear to
be satisfied with the current level
(9) The real objective of the company is to transport the None High effectiveness: since making a profit Internal and informal
owners, not to obtain profits is not emphasized, resources for safety
are made available more easily
None High effectiveness: the crews recorded Internal and informal
their satisfaction with the fact of, in
(10) Four of the six crew are experienced commanders,
general, having at least one experienced
which periodically enables two of them to fly together
commander on each flight which enriches
feedback on individual performance
None High effectiveness: this form of Internal and formal
(11) To the contrary of what is typical in the air taxi sector,
remuneration does not induce flights
the monthly remuneration of the crew is fixed, and thus
being made in unsafe conditions, which is
does not depend on monthly hours of flight
what happens in company A

Fig. 7. SRs in company B.

SRs. Indeed, it could be argued, preferably with the benefit of
hindsight, that the better performance of company B should have
been expected, since it is not as profit-driven as company A.
However, what could not be anticipated without using the frame-
work are the specific SBs and SRs, as well as their characteristics.
In line with the conclusions on the relative performance of both
companies, there was a serious accident in company A, towards the
end of the data collection period. An airplane made a forced landing
and the passengers suffered minor injuries. The research team had
access both to the pilots’ written and verbal reports, even though
the investigation conducted by governmental agencies had not yet
been concluded by the time this article has been written. Never-
theless, based on the data available, it was possible to see that at
least four sources (SB5, SB6, SB7, SR6) interacted in a non-linear
way in this event. A key event in this accident was the decision
made by the pilot (he was also the general-director), during a stop-
over, not to refuel the aircraft. According to the data obtained, this
decision was made because of the high fuel prices in that location,
which underscores the manifestation of SB5 (production pressures
from top management).
As a result of that decision, during the next leg of the flight, one
of the engines stopped due to a lack of fuel. At this moment, the lack
of standard operations procedures (SB6) made it impossible to
check what the rules to be applied in this situation should be. Also,
in the final minutes of the flight, there was a discussion between
the pilot and the co-pilot, who were arguing about the proper
emergency procedures to be adopted. Apparently, the pilot was
not aware of the procedures, since, due to difficulties in his
inter-personal relationships with his colleagues (SB7), he did not

SR7, SR9, SR10, SR11

SR8

Fig. 8. Summary of the assessment in company A. Fig. 9. Summary of the assessment in company B.
 Author's personal copy
322 T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324
attend the informal meetings in which crew members exchanged
information on procedures (SR6). It is also worth noting this
company went bankrupt a few months after this accident. Even
though this position might be biased by hindsight, it seemed that
company A had been working at the edge of chaos, a state in which
socio-technical systems operate with maximum functionality and
maximum complexity, exploiting as many as possible of the avail-
able resources (Dekker, 2011).
On reviewing both companies, only four out of the twenty-one
SRs (all in company B), were classified as being highly effective.
This means that the adjustments of performance can also be brittle
and, even worse than that, they can introduce new hazards.
Although performance adjustment is a core characteristic of resil-
ience (Hollnagel et al., 2008), adjustments do not take place in
a vacuum and, due to the explosion of interactions that happen
when anything changes in a complex system (Dekker, 2011), it is
not reasonable to expect that adjustments will lead only to bene-
ficial interactions. Indeed, only either the highly effective SRs or the
SRs without opposing SBs seem to act as valuable counter-gradients
to the process of drifting into failure, while all the other SRs and SBs
are in fact pushing the organization towards failure.
From a broader perspective, this insight indicates that resilience
is not intrinsically positive and that, as illustrated by the case study
results, being thoroughly familiar with the nature of resilience is
critical in order to assess how good it is. Indeed, it could be argued
that the low effective SRs are simply ways of masking inefficiencies,
rather than being a constructive form of resilience. In this respect, it
is worth stressing the importance of identifying whether the
sources arise internally or externally. With regard to the external
origins (e.g., the lack of slots in the main airport used by company
B), the tolerability of brittle adjustments might be higher.
Two results shared by both companies concern the facts that the
majority of the SBs had a formal origin (61% in company A and 83%
in company B) and that most of the SRs were informal in origin (80%
in company A and 82% in company B). These results are consistent
with previous studies (Hollnagel and Woods, 2005) that stress
employees’ informal behavior at the sharp end as being essential to
ensuring badly-designed systems are kept operational.
The aforementioned percentages show substantial differences
in comparison with those found by Saurin and Carim Junior (2011),
who identified SBs and SRs in an electricity distributor. In the study
conducted by those authors, 71% of the SRs and 87% of the SBs were
formal in origin. The higher portion of formal sources detected in
that study, especially concerning the SRs, was possibly due to the
emphasis placed on the investigation of the formal health and
safety management system, even though informal sources had also
been detected.
6.2. Assessment of the framework
As a noteworthy characteristic, the framework brought to the
surface informal practices that had been incorporated into the
routine and that, because they helped to keep the system working,
employees regarded either as being intrinsically good or accepted
with resignation. The proposed SBs and SRs analytical categories
were shown to support the understanding of the impacts of these
practices. It also points out which sources (i.e., the high risk SBs and
the low effectiveness SRs) could be prioritized for monitoring. For
example, in both companies, the results showed that the produc-
tion pressures, especially those coming from top management,
should be monitored. This could be done, for example, by applying
surveys to the sharp end staff, similarly to what happens in some
nuclear power plants (EPRI, 2006).
The classifications assigned to each source based on the
proposed categories (SB, SR, risk, effectiveness and origin) should
be interpreted as the result of the appearance of the sources in the
period of the assessment e it could be compared with a snapshot of
the sources. Indeed, due to the dynamic nature of the sources, it is
more appropriate to say that for the time being they are in a certain
classification, rather than saying that they belong to it. This
perspective is especially important to analyze the SRs that do not
have opposite SBs. For example, the level of automation in the
airplanes of company B was considered as an SR. However, the
humanemachine interaction in a cockpit is complex and subjected
to unforeseen events, thus it is likely that some brittleness had not
yet been detected.
This insight is related to what is likely to be the main drawback
of the framework, since it does not totally capture the evolving
nature of the sources and their interactions. In particular, the
framework is limited in terms of capturing the interactions among
the SRs themselves and among the SBs themselves. In fact, the
interactions between the SRs and the SBs are at least partially
captured, since they are often opposite to each other. Of course, an
additional effort of data collection and analysis, based on fairly
simple methods, could help to deal with this drawback. For
example, a survey could be given to company’s employees, so they
could point out how intense the relationships are among the
sources. Based on this, models of relationships among the sources
could be developed, supporting the anticipation and monitoring of
the interactions.
Nevertheless, it should be stressed that the modeling of inter-
actions in a complex system may be tricky, since their full antici-
pation is impossible if the system is really complex (Cilliers, 2005).
In a complex system, the SBs and SRs are continuously coupling and
decoupling with all elements of such system e i.e., the interactions
are dynamic, rather than static. Thus, while efforts to model and
monitor the interactions should not be neglected, a concurrent
emphasis should be placed on giving as much visibility as possible
to each individual source, either by organizational (e.g., incident
reports, formal or informal social encounters involving crew
members) or technological means (e.g., alarms).
The visibility of the individual sources might, as a beneficial
side-effect, work as a means to anticipate and control interactions.
The assumption is that undesired interactions will be more likely
and visible when individual SBs and ineffective SRs occur close to
each other in space and time. For example, in the case of company
A, simultaneity might mean undertaking a flight using the GPS as
an improvised backup, with the autopilot out of service, while
bound for an airport with poor infrastructure, and for which
the decision to take-off was made by pilots who were not aware of
the tacit rules that allowed unfavorable meteorological conditions
to be anticipated. Even though each of these SBs does not normally
interact with the others, the system dynamics might trigger
undesired and unforeseen interactions, thus quickly changing
loosely-coupled interactions into tightly-coupled ones (Perrow,
1984) e indeed, this is what happened in the accident in
company A already mentioned. Moreover, the small size of the
companies in this study, which have few workers and equipment
(i.e., there were no significant buffers), tends to favor such
couplings being tight, even though this characteristic make them
more visible as well.
The definition of the level of abstraction of the sources has also
emerged from the field studies as a relevant issue concerning the
limitations of the framework. Although the word “source” has been
used, there was no tracking of the deeper organizational origins of
either the SB or the SR, albeit the available data allows inferences
regarding this issue. A parallel might be traced between this charac-
teristic of the framework and the identification of root causes
of accidents, since there are no rigid stop rules (Rasmussen et al.,1994).
According to Snook (2000) if the investigation has a scientific purpose
 Author's personal copy

T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324 323

(in contrast, for example, to a judicial one), it is reasonable to stop the
investigation when something new, from this perspective, is found. In
this study, the abstraction level of the sources identified in companies
A and B, provided a sound basis on which the theoretical and practi-
cal discussion herein reported were undertaken.
Another possible perspective is that the stated abstraction level
represents the status of each source during the assessment period,
which, in turn, is the result of a dynamic process in which the
sources are continuously changing. This means that, if the assess-
ment were repeated in the future, many sources would be identi-
fied at another level of abstraction. Indeed, every SB and SR is likely
to have had predecessors and they will likely generate new sources
in the future. Also, the different abstraction levels are a result of the
fact that the sources were not sought in any specific organizational
locus. Thus, it is natural that some sources refer to operational
issues (e.g., the lack of autopilot in company A’s airplanes), while
others are concerned with more strategic ones (e.g., in company B,
workers complained about not being consulted by top manage-
ment on strategic decision-making).
The framework is also limited in terms of completeness, as the
sources identified should be regarded as a portion of all existing SBs
and SRs. To some extent, this drawback is a result of the fairly little
time spent in data collection (one researcher, who spent about 24 h,
distributed over several visits, in the premises of each company).
Moreover, as happens in the analysis of any complex system, it is
impossible for knowledge of it to be complete.
Also, it is worth noting that it is difficult to understand each
source in itself and that there is room for deeper investigations,
either by collecting additional data or by using support from the
literature. For instance, what can be cited is the inversion of hier-
archical levels that sometimes occurs in company A, in which
a director works as a co-pilot, being directed by people who are his
subordinates during day-to-day activities. The implications of this
sort of inversion are not well-known, there being reports that it has
already been detected as a contributing factor in major accidents
in complex systems, such as a friendly fire event in the Iraq war,
reported by Snook (2000).
6.3. Opportunities for future studies
Some research opportunities arising from this study should be
stressed, such as: (a) investigating how the framework might be
used in association with existing methods of safety assessment in
complex systems; (b) developing SBs and SRs monitoring systems;
(c) using existing methods (e.g., concept maps) or developing new
ones both to model and monitor the interactions among the sour-
ces; and (d) developing means to anticipate the temporal evolution
of the sources (i.e., how will they look within a certain timeframe?).
Appendix A. Extract of the protocol to assess the principle of
flexibility.

Characteristics of the principles Sources of data

Operators are encouraged to use the Analysis of documents
procedures with discretion, rather a) General operations manual
than enforcing compliance at any cost.
b) Standard Operating Procedures
Adaptations of procedures are not
necessarily seen as negative. c) Checklists
The reasons underlying adaptations d) Incident reports
are investigated, with the aim of
improving the system Interviews (pilots, directors and responsible for flight safety)
a) What do you do when the procedures are not applicable?
b) Have you been involved in the design of procedures?
c) How strict are the procedures?
d) Is the design for procedures guided by a set of principles? If so, please state them
e) What is the reaction of workers and managers when the adaptations of procedures
become public?

Observations
a) Meetings for planning flights, including briefings before taking-off
b) In-flight observations, checking how procedures are followed and identifying adaptations
c) Any formal or informal meeting in which flight safety is in focus
e) Debriefing after landing, comparing the planning with the real flight
The organization, teams and individuals Analysis of documents
should be capable of dealing with a) Incident reports
variability, including unforeseen events
b) The accident prevention plan required by Brazilian regulations
c) Documents for internal communications (minutes of meetings, leaflets, newspapers, etc.).

Interviews (pilots, directors, responsible for flight safety, flight coordinator)

a) How often are incidents reported?
b) How often operations are disrupted due to behaviors of other stakeholders
(e.g., air traffic control)?
c) Do you take into account safety concerns when you sell a flight
(e.g., does the destination have a safe airport)?
d) Could you describe the crisis management program of this company?
e) How is the process of continuous improvement, especially concerning procedures?

Observations: the same as proposed for assessing the previous characteristic

(continued on next page)
 Author's personal copy

324 T.A. Saurin, G.C.C. Junior / International Journal of Industrial Ergonomics 42 (2012) 312e324

(continued )

Characteristics of the principles Sources of data

The responsibilities and autonomy of each Analysis of documents
job is stated in procedures. However, it a) General operations manual
is acceptable and predictable that the
b) Incident reports
hierarchy can change during emergencies
c) The accident prevention plan required by Brazilian regulations

Interviews (pilots; responsible for flight safety, flight coordinator, directors)

a) What are your formal and informal responsibilities?
b) To what extent are the formal responsibilities carried out in practice?
c) What types of decisions are usually dependent on fellow workers and higher ranks?
d) Could you refuse a flight due to lack of safety? Please cite an example.
e) On which grounds would you refuse a flight?
f) Is the flight safety department involved in determining responsibilities and
autonomy of each job?

Observations
a) Observe the process of selling flights
Training includes both technical and Analysis of documents
non-technical skills a) Crew Resource Management training manual
b) Training program
c) The accident prevention plan required by Brazilian regulations
d) Training records of pilots

Interviews (pilots; responsible for flight safety, flight coordinator, directors)

a) What was your introductory training in this company like and to what extent
was safety emphasized?
b) How does periodic training work?
c) Does the training deal with both emergency and unexpected situations?
d) Does training include both technical and non-technical skills?

Observations
a) Observe the facilities in which training takes place
b) Observe real training sessions taking place

References Hendrick, H.W., Kleiner, B.M., 2001. Macroergonomics: An Introduction to Work

System Design. Human Factors and Ergonomics Society, Santa Monica.
Hoffman, R., Militello, L., 2009. Perspectives on Cognitive Task Analysis: Historical
Anderson, R., Crabtree, B., Steele, D., Mcdaniel, R., 2005. Case study research: the
Origins and Modern Communities of Practice. Taylor and Francis, New York.
view from complexity science. Qualitative Health Research 15 (5), 669e685.
Hollnagel, E., 2004. Barriers and Accident Prevention. Ashgate, London.
Brasil, 2008. http://www.planalto.gov.br/ccivil_03/LEIS/L7565.htm.
Hollnagel, E., 2011. The scope of resilience engineering. In: Hollnagel, E., Paries, J.,
Centro de Prevenção e Investigação de Acidentes Aeronáuticos (CENIPA), 2010.
Woods, D., Wreathall, J. (Eds.), Resilience Engineering in Practice: A Guidebook.
http://www.cenipa.aer.mil.br/cenipa/paginas/normas/Panorama19992008.pdf.
Ashgate, London, pp. xxixexxxix.
Cilliers, P., 1998. Complexity and Postmodernism: Understanding Complex Systems.
Hollnagel, E., Nemeth, C.P., Dekker, S., 2008. Resilience Engineering Perspectives:
Routledge, London.
Remaining Sensitive to the Possibility of Failure, vol. 1. Ashgate, Burlington.
Cilliers, P., 2005. Complexity, deconstruction and relativism. Theory Culture &
Hollnagel, E., Woods, D., 2005. Joint Cognitive Systems: Foundations of Cognitive
Society 22 (5), 255e267.
Systems Engineering. Taylor & Francis, Boca Raton.
Costella, M.F., Saurin, T.A., Guimarães, L.B.M., 2009. A method for assessing health
Hollnagel, E., Woods, D., Leveson, N., 2006. Resilience Engineering: Concepts and
and safety management systems from the resilience engineering perspective.
Precepts. Taylor & Francis, London.
Safety Science 47 (8), 1056e1067.
Morel, G., Amalberti, R., Chauvin, C., 2009. How good micro/macro ergonomics may
Crandall, B., Klein, G., Hoffman, R., 2006. Working Minds: A Practitioner’s Guide to
improve resilience, but not necessarily safety. Safety Science 47 (2), 285e294.
Cognitive Task Analysis. The MIT Press, Cambridge.
Nemeth, C., Hollnagel, E., Dekker, S., 2009. Resilience Engineering Perspectives:
Dekker, S., 2011. Drift into Failure: From Hunting Broken Components to Under-
Preparation and Restoration, vol. 2. Ashgate, Burlington.
standing Complex Systems. Ashgate, London.
O’Connor, P., Dea, A., Flin, R., Belton, S., 2008. Identifying the team skills required by
Dekker, S., 2007. Just Culture: Balancing Safety and Accountability. Ashgate, London.
nuclear power plant operations personnel. International Journal of Industrial
Dekker, S., 2006. The Field Guide to Understanding Human Error. Ashgate,
Ergonomics 38, 1028e1037.
Burlington.
Page, S., 2007. The Difference: How the Power of Diversity Creates Better Groups,
Electric Power Research Institute (EPRI), 2006. Business Performance Indicators for
Firms, Schools and Societies. Princeton University Press, Princeton.
Nuclear Asset Management. EPRI, Palo Alto.
Perrow, C., 1984. Normal Accidents: Living with High-risk Technologies. Princeton
Ferreira, P., Wilson, B., Ryan, B., Sharples, S., 2011. Measuring resilience in the
University Press, Princeton.
planning of rail engineering work. In: Hollnagel, E., Paries, J., Woods, D.,
Rasmussen, J., Pejtersen, A., Goodstein, L., 1994. Cognitive Systems Engineering.
Wreathall, J. (Eds.), Resilience Engineering in Practice: A Guidebook. Ashgate,
John Wiley & Sons, New York.
London, pp. 146e156.
Saleh, J., Marais, K., Bakolas, E., Cowlagi, R., 2010. Highlights from the literature on
Furniss, D., Back, J., Blandford, A., Hildebrandt, M., Borberg, H., 2011. A resilience
accident causation and system safety: review of major ideas, recent contri-
markers framework for small teams. Reliability Engineering and Systems Safety
96 (1), 2e10. butions, and challenges. Reliability Engineering and System Safety 95,
1105e1116.
Gomes, J.O., Woods, D., Carvalho, P.V.R., Huber, G.J., Borges, M.R.S., 2009. Resilience
Saurin, T.A., Carim Junior, G., 2011. Evaluation and improvement of a method for
and brittleness in the offshore helicopter transportation system: the identifi-
assessing HSMS from the resilience engineering perspective: a case study of an
cation of constraints and sacrifice decisions in pilots’ work. Reliability Engi-
electricity distributor. Safety Science 49 (2), 355e368.
neering and Systems Safety 94, 311e319.
Sheehan, J., 2003. Business and Corporate Aviation Management: On Demand Air
Hale, A.R., Heijer, T., 2006. Is resilience really necessary? The case of railways. In:
Transportation. McGraw-Hill, New York.
Hollnagel, E., Woods, D., Leveson, N.G. (Eds.), Resilience Engineering: Concepts
Snook, S., 2000. Friendly Fire: The Accidental Shootdown of U.S. Black Hawks over
and Precepts. Ashgate, London, pp. 115e137.
Northern Iraq. Princeton University Press, Princeton.

View publication stats