Safety Science 47 (2009) 1056–1067

Contents lists available at ScienceDirect

Safety Science
journal homepage: www.elsevier.com/locate/ssci

A method for assessing health and safety management systems

from the resilience engineering perspective
Marcelo Fabiano Costella a, Tarcisio Abreu Saurin b,*, Lia Buarque de Macedo Guimarães b
a
UNOCHAPECÓ (Regional University of Chapecó), Rua Quintino Bocaiuva, 390-D, Chapecó, SC, CEP 89801-080, Brazil
b
DEPROT/UFRGS (Industrial Engineering and Transportation Department, Federal University of Rio Grande do Sul), Av. Osvaldo Aranha, 99,
5. andar. Porto Alegre, RS, CEP 90035-190, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: This article introduces a method for assessing health and safety management systems (MAHS) that has
Received 12 June 2008 two innovative characteristics: (a) it brings together the three main auditing approaches to health and
Received in revised form 8 November 2008 safety (HS) – the structural approach (which assesses the system prescribed), the operational approach
Accepted 21 November 2008
(which assesses what is really happening on the shop-floor) and the performance approach (which
assesses the results of performance indicators); (b) it emphasizes the resilience engineering perspective
on HS, which takes into consideration four major principles (flexibility, learning, awareness, and top man-
Keywords:
agement commitment). Such principles underlie seven major assessment criteria, which in turn are
Health and safety management systems
Resilience engineering
divided into items (e.g. hazard identification from a resilience perspective is an item that belongs to
Health and safety audits the criteria of production processes). The items are sub-divided into statements, which are the require-
Manufacturing industry ments that should be assessed based on interviews, analysis of documents and direct observations.
Within the 112 requirements proposed, 38 of them have clear links with at least one out of the four resil-
ience engineering principles adopted. The remaining requirements are based on traditional assumptions
underlying the so-called best practices of HS management. The results of the assessment for each item
are expressed by a score on a scale of compliance with the established requirements, ranging from 0%
to 100%. The specific score within that scale is obtained from tables used to assess applications for the
Brazilian national quality award. The MAHS was tested in a case study that was carried out in a factory
that manufactures automobile exhaust systems, located in Brazil.
Ó 2008 Elsevier Ltd. All rights reserved.

1. Introduction The socio-technical vision served as a basis for the development

Health and safety (HS) management best practices are well
known among the leading companies in the area. Nevertheless,
best practices are generally not applied systematically but rather
normally amount to fragmented actions, in addition to which the
companies which use them have reached a performance plateau
(Amalberti, 2006; Mitropoulos et al., 2005).
Thus, advances are needed in terms of concepts and methods to
ameliorate this state of affairs. The traditional HS management
best practices analyze people, technology and the work context
separately by means of focusing on a sociological, technological
or organizational approach. The socio-technical approach brings
these ways of focusing together and emphasizes their interfaces,
and thus furnishes an analysis which is closer to the complex real-
ity of mutual interactions and adaptations between people, tech-
nology and work (Clegg, 2000).
* Corresponding author. Tel.: +55 51 3223 8009; fax: +55 51 3308 4007.
E-mail addresses: costella@nostracasa.com.br (M.F. Costella), saurin@ufrgs.br
(T.A. Saurin), lia@producao.ufrgs.br (L.B. de Macedo Guimarães).
of principles and concepts of cognitive systems engineering (CSE),
which is a cross-disciplinary approach for the design of complex
socio-technical systems, being concerned with the study of how
joint cognitive systems (a human machine ensemble that cannot
be separated) perform, rather than cognition as a mental process
(Hollnagel and Woods, 2005). Differently from behaviourist ap-
proaches, CSE takes into account all the complexity that exists in
the organizational environment in which human behaviour takes
place. In CSE, humans are no longer regarded as simply determin-
istic input–output devices but as goal-oriented creatures who ac-
tively select their goals and seek relevant information
(Rasmussen, 1983) – i.e. both adaptations and the role of the con-
text are core issues. A set of key CSE principles for the design of safe
and adaptive work systems was proposed by Jens Rasmussen (Ras-
mussen et al., 1994; Rasmussen, 1997), whose ideas have been fur-
ther developed under the label of resilience engineering (RE),
which is a term adopted by a number of studies to refer to CSE
applications on safety management (Hollnagel, 2006; Hollnagel
and Woods, 2006; Leveson et al., 2006). This approach has been
adopted in different domains (e.g. surgery and firefighting), even

0925-7535/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ssci.2008.11.006
 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067
though some studies use neither the label of CSE nor RE, such as
that by Montagna and Ferrari (2006).
The challenge for HS management in the context of RE is to
draw up prevention strategies which adequately address complex,
dynamic and unstable systems. In particular, strategies are needed
which adequately take account of system variations which cannot
be totally foreseen at the design stage. Therefore the challenge is to
construct dynamically stable systems, with a view to ensuring that
adaptations, despite their being necessary at any given moment,
allow for the system to remain under control (Hollnagel, 2006).
Although RE has been mostly studied in the context of complex
systems of high risk, such as in the aviation, petro-chemical and
nuclear power industries (Hollnagel, 2006), its concepts also tend
to be beneficial for manufacturing industry, especially for med-
ium-sized and large industries positioned in highly competitive
supply chains, such as the automobile sector. This occurs since
manufacturing also possesses characteristics of complex systems,
such as high interdependence between processes (most strongly
felt in environments of just-in-time production), the huge number
of variables and various trade-offs to be managed (Christoffersen
and Woods, 1999).
Given that all control systems tend to deteriorate over time or
become obsolete as a consequence of changes, the continuous per-
formance measurement is essential for HS management, whether
or not under the RE paradigm. Such measurement can occur at dif-
ferent levels, such as individual workstations, individual manage-
ment processes or at the level of the HS management system
(HSMS) as a whole. A particular type of measurement, which is
dealt with in this study, is auditing. In fact, auditing goes beyond
measurement, since it explains performance and builds on mea-
surement to identify gaps between current and desired perfor-
mance, to identify where there are problems and needs, and to
provide information that can be used in developing actions plans
to improve performance (Chiesa et al., 1996). The European Foun-
dation for Quality Management (EFQM, 1995) adds that auditing
means a comprehensive, systematic and regular review of an orga-
nization’s activities and results referenced against a model of busi-
ness excellence.
Currently, the structural approach is the one most used to audit
HSMS, as it is based on the analysis of documents which prove that
the organization is meeting certain HS requirements which it itself
has defined or which are defined by standards, like OHSAS 18001
(occupational health and safety assessment systems). The struc-
tural emphasis is typical of most current safety audit tools, and is
concerned with assessing if there is a documented HSMS (Ahmad
and Gibb, 2004). As to the operational approach, it verifies if the
documented HSMS has in fact been implemented in practice, by
means of observations and interviews with the company’s opera-
tional and management staff. There is also the approach of auditing
by performance, based on analyzing the results of normally reac-
tive performance indicators (Cambon et al., 2006).
The literature indicates a series of limitations to the existing
models for auditing HSMS, such as (Bluff, 2003; Le Coze, 2005):
(a) non-specification of the context in which the HSMS implicit
in the models is valid or more effective; (b) focus on relation to
imminent risks of accidents, so giving little importance to the la-
tent hazards to HS in the long term; (c) the items assessed are sta-
tic, with there being no guidelines for their continuous adaptation
to the dynamic conditions of modern work environments – this is
partly a result of not making explicit what HS principles underlie
the audit model; and (d) strategic and cultural dimensions which
interfere in HS management are not considered.
Although there are no models for auditing which explicitly em-
brace the principles of RE, some studies have re-interpreted mod-
els which originally did not take account of RE, with a view to
verifying the extent to which its principles were indirectly borne
1057
in mind (Hale et al., 2006). Other auditing models, such as the Du-
pont system for process safety management (Dupont, 2006), fo-
cuses on behaviour-based safety. This has been criticized for not
emphasizing the capacity of workers to adapt and for not having
an impact on incidents which occur without there being any safe
or unsafe behaviour by workers on the shop-floor (Hopkins,
2006). Other auditing models do not adopt explicit assumptions
about what HS management philosophy should underpin the sys-
tem, as for example ISRS – international safety rating system (Eis-
ner and Leger, 1988), CHASE – complete health and safety
evaluation (Chase, 2006) and MISHA – method for industrial safety
and health activity assessment (Kuusisto, 2001).
Considering this context, this study aims to present a method
for assessing HSMS (MAHS) with the focus on RE. Such a method
takes account of the three main approaches to auditing HSMS
and was tested in a case study in a factory which manufactures
motor vehicle exhaustion systems. Even though the MAHS might
be considered as an audit tool, according to the definition of an
audit that was previously given, the ‘‘A” of the acronym MAHS
stands for assessment rather than auditing. This choice was made
since the term ‘‘audit” is often narrowly interpreted as looking for
non-compliance with and deviations from regulations and stan-
dards, such as OHSAS 18001.
2. Resilience engineering
Resilience engineering is a paradigm for safety management
that focuses on how to help people to cope with complexity under
pressure to achieve success (Resilience Engineering Network,
2008). Due to this fact, a distinctive feature of RE is its emphasis
on understanding how success is obtained, how people learn and
adapt themselves by creating safety in an environment which
has faults, hazards, trade-offs and multiple objectives (Hollnagel
and Woods, 2006). Wreathall (2006) also associates resilience with
the ability of an organization to keep, or recover quickly to, a stable
state, allowing it to continue operations during and after a major
mishap or in the presence of continuous significant stresses. Thus,
resilience includes both the property to avoid failures and losses, as
well as the property to respond effectively after these have
occurred.
The applications of RE are particularly suitable for high risk sys-
tems with complex characteristics, such as (Christoffersen and
Woods, 1999): (a) the high degree of inter-connection between
the components of the system, the consequence of which is the
very great difficulty which the operator has to foresee the effects
of his actions and the rapid propagation of the errors; (b) uncer-
tainty and variability. Under such conditions of complexity, the
attributes associated with RE grow in importance.
Considering that there is no one set of RE principles which is
widely accepted in academic circles and also that there are differ-
ences in the terminology adopted by different authors, for the pur-
poses of this article, there was a need to endeavour to compile a set
of principles which would serve as a reference for the assessment
method now put forward. It is worth stressing that RE principles
may be used at any level of aggregating the cognitive system, rang-
ing from the focus of a single worker at his workstation to the focus
of the organization as a whole. Thus, based on various studies (Ras-
mussen, 1997; Hollnagel and Woods, 2005; Hale and Heijer, 2006;
Wreathall, 2006; Saurin et al., 2008) four principles were identified
which have interfaces with each other and do not possess strictly
defined limits:
(a) Top management commitment: this implies demonstrating
a devotion to HS above or to the same extent as the com-
pany’s other objectives.
1058 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067
(b) Increase flexibility (flexibility): a basic assumption of RE is
that human errors are inevitable because of individual and
organizational pressures (e.g. workload and cost) (Rasmus-
sen et al., 1994). Therefore, work system design must be
flexible, recognizing that variability management is as
important as variability reduction. In fact, design should
support the natural human strategies for coping with haz-
ards, rather than enforce a particular strategy. This means
that no more than what is absolutely essential should be
specified in the design of jobs (Clegg, 2000). This implies
studying what people actually do and then considering
whether it is possible to support that through design (Hol-
lnagel and Woods, 2005). For instance, a mechanism to com-
ply with this principle is to design error-tolerant boundaries
(Rasmussen et al., 1994). Wreathall (2006) also emphasizes
that flexibility requires that people at the working level (par-
ticularly first-level supervisors) should be able to make
important decisions without having to wait unnecessarily
for management instructions.
(c) Learn from both incidents and normal work (learning): RE
emphasizes understanding normal work rather than just
learning from incidents, in order to learn and to disseminate
successful working strategies. Nevertheless, learning
requires an organizational environment that encourages
the reporting of incidents and recognizes adaptive strategies,
although it will not tolerate culpable behaviours (Wreathall,
2006). Also, learning must take into account the way proce-
dures are implemented. In fact, monitoring the implementa-
tion of procedures should be considered as important as
devising procedures, since it may contribute to reducing
the gap between work as imagined by managers and work
as performed by front-line operatives. The smaller this gap,
the greater the evidence that learning is taking place (Wre-
athall, 2006; Hale and Heijer, 2006).
(d) Be aware of system status (awareness): this principle
implies that actors should be aware both of their own cur-
rent status and the status of the defences in the system. This
is critical for anticipating future changes in the environment
that may affect the system’s ability to function (Resilience
Engineering Network, 2008). Awareness is also important
for the assessment of the trade-offs between production
and safety (Hale and Heijer, 2006; Hollnagel and Woods,
2005). Rasmussen et al. (1994) suggest two broad
approaches for implementing this principle: performance
measurement based on proactive indicators and the design
of visible boundaries of performance.
Besides these four principles, another which permeates them
can be highlighted. This is proactiveness which refers to anticipat-
ing problems, needs or changes, and which leads to actions being
drawn up which directly alter the surroundings of the environ-
ment. In terms of HS, proactiveness refers to anticipating hazards
and control measures so as to interrupt the evolutionary course
of incidents.
Of course, the RE principles presented overlap somewhat with
principles of other safety management paradigms (and even with
generic management principles, such as the well-known idea of
top management commitment) and they are fully in line with
the general principles for the design of socio-technical systems
established by previous studies, such as that by Clegg (2000). For
example, there is an overlap with the perspective of safety culture
taken by Reason (1997), since he agrees with the stance that safety
culture might be engineered and managed and it should encom-
pass four subcomponents: a reporting culture, a just culture, a flex-
ible culture and a learning culture. In fact, Reason (2001) included
safety culture principles in a checklist aimed at assessing what he
referred to as the institutional resilience of airlines. Nevertheless,
this article assumes that the characteristic of RE which distin-
guishes it most from other paradigms is that it emphasizes the po-
sitive side of safety (i.e. understanding how adaptive strategies
ensure safe and productive work), although it does not neglect
learning based on incidents. Moreover, it is based on the assump-
tion that resilience is a property of a system that may be con-
sciously designed and managed, even though the development of
a strong RE framework in terms of concepts, principles and meth-
ods is still an on-going process (Woods and Hollnagel, 2006).
3. Research method
The selection of the elements assessed by the MAHS was made
based on the standards OHSAS 18001 and ILO-OSH 2001 (guide-
lines on occupational safety and health management systems), as
well as based on a review of the literature which covered three
areas: HSMS, RE and HSMS audits. An exploratory case study to as-
sess a HSMS conducted in an agricultural equipment factory also
contributed to constructing the audit model. The elements as-
sessed by MAHS can be classified into criteria and items. The for-
mer correspond to the large categories of assessment and the
items to the sub-categories, which, for their part, consist of the
requirements to be assessed. Thus, 28 items and seven criteria
were established.
Since the framework of the MAHS was defined, it was applied
and assessed by means of a case study, carried out over a 3 months
period in 2007. The case study took place in the Brazilian company
which is the market leader for replacing automobile exhaust sys-
tems. It was chosen because of the ease of access which the
researchers had to it, as well as on account of its size (450 employ-
ees) and integration with a highly competitive supply chain, which
were indicative of characteristics of complexity. The company pro-
duces exhaust pipes and accessories, such as protective caps,
crankcase guards, tow bars and trailers for motor vehicles, ranging
from automobiles to tractors. The main clients are the sales outlets
for spare parts for all models, there being 12,000 clients in Brazil
and Latin America.
Throughout 2005, 2006 and 2007, the company went through
financial difficulties as a result of an investment in a new line of
products that produced a disappointing return. This negatively
influenced HS in terms of there being a shortage of resources for
the area. Despite this, the company holds an ISO 9001 certificate.
The workforce consists predominantly of men from the rural zone,
which requires substantial investment in training events.
The company’s portfolio boasts more than 850 products, with
most of its components being manufactured in the same plant in
which the final products are assembled. This contributes to the fre-
quent change of dies and tools, to the existence of functional lay-
outs and difficulties in standardizing processes, which makes the
production process more complex.
Three examiners applied the MAHS in the case study: one of
them was the first author of this article, who was designated as
the lead examiner (LE) and is a certified auditor of quality manage-
ment systems by the Brazilian foundation for the national quality
award. The other two examiners were selected in accordance with
their knowledge of the area of HS. One of these examiners, deemed
examiner 1 (E1), is a civil engineer and was completing a special-
ization course in safety engineering. The other, deemed examiner
2 (E2), is a business administrator and was completing a master’s
degree in industrial engineering with emphasis on ergonomics.
Due to the disparity of knowledge about RE among the examiners,
the LE fostered a levelling up of these concepts using the MAHS
assessment instrument which was fully discussed. The three exam-
iners took part in all the stages of the case study jointly. Taking into
 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067 1059

account only the collection of field data, the examiners made a nected, since the objectives must be consistent with the policy.
weekly 4-h visit over 2 months, a total of eight visits and 32 h With regard to RE, a requirement of this item is to emphasize con-
per examiner in the company. The activities undertaken by the tinuous improvement, in order not to be complacent about the cur-
examiners throughout the case study correspond to the application rent situation, even when safety performance is good (Hale and
of all the stages of the MAHS, which are described in the following Heijer, 2006).
section of this article. At the end of the case study, interviews were (1.2) HSMS planning: this item has requirements such as estab-
conducted with some of the company’s representatives who had lishing priorities for HS planning and establishing tasks, resources,
accompanied the study, so that they could express their opinions deadlines and responsibilities for achieving the HS objectives.
about the MAHS and the application procedures. (1.3) Structure and responsibility: this encompasses bureau-
After concluding the case study reported in this article, another cratic issues, such as defining HS responsibilities throughout all
application of MAHS was undertaken in a small-sized school for hierarchical levels of the organization and establishing mecha-
training pilots for civil aviation (Carim Jr. et al., 2008). On the nisms to let all those involved in it be aware of their HS
one hand, this study indicated that all items assessed by MAHS responsibilities.
were relevant and potentially necessary for the school of aviation. (1.4) Documentation and records: this has requirements such as
On the other hand, various terms and requirements established by a way of easily finding out and tracking documents which are dis-
MAHS were unfamiliar to the aviation school’s instructors and tributed to company personnel.
managers and therefore needed to be clarified (for example, the (1.5) Legal requirements: this is a core issue to any HSMS and, at
meaning of personal protective equipment was not clear, nor were MAHS, there are also requirements related to preparedness for and
they familiar with concepts of HSMS). In fact, this is a reflection of response to emergency, since this is mandatory in accordance with
the fact that the MAHS was designed for the context of manufac- Brazilian regulations.
turing industry, so that adaptations of terms, among others, may (1.6) Top management commitment: this item has require-
be necessary to make it viable to apply it in other contexts. ments such as top management monitoring of HS metrics and
managing production pressures on HS.
4. Method for assessing health and safety management systems (2) Production processes
(MAHS) (2.1) Hazard identification from a traditional perspective: this
checks whether there are mechanisms to identify those hazards
4.1. Criteria and items of the MAHS that are easily observable and that are usually emphasized by reg-
ulations, such as physical, chemical and biological hazards.
In Fig. 1, the criteria and items of MAHS are presented as well as (2.2) Hazard identification from an RE perspective: this item
the association between each item of the MAHS and both the RE checks whether there are mechanisms for identifying organiza-
principles and the requirements of OHSAS 18001, with additions tional hazards (e.g. production pressures, monotony, excessive la-
from standard ILO-OSH 2001. bour division), which could be broadly understood as all hazards
An overview of the scope of each item is presented below: that make the traditional hazards riskier than they should be.
(1) HSMS planning (2.3) Risk assessment: this item checks whether there are
(1.1) HSMS policy and objectives: policy and objectives were mechanisms for prioritising risks based on their severity and
put together in a single item because they are closely inter-con- probability.

RE principles OHSAS 18001 requirements plus ILO-OSH 2001

OHS Management Program(s)

Emergency preparedness and

identification, risk assessment

Legal and other requirements

Structure and responsability

Performance measurement
Document and data control

Procurement e contracting
corrective and preventive
Training, awareness and

Continual improvement
nonconformances, and

Management systems
Records and records
Accidents, incidents,

Management review

Worker participation
Planning for hazard

Operational control
Top management

Consultation and
and risk control

Documentation
communication

and monitoring
commitment

management
competence
Awareness

OHS Policy

Objectives

integration
Flexibility
Learning

response

actions

Audit

Criteria and itens

1.1 — HSMS policy and objectives
1.2 — HSMS Planning
1.3 — Structure and responsability
1 — HSMS Planning
1.4 — Documentation and records
1.5 — Legal requirements
1.6 — Top management commitment
2.1 — Hazard identification from traditional perspective
2.2 — Hazard identification from RE perspective
2 — Production
2.3 — Risk assessment
processes
2.4 — Hazard responses from traditional perspective
2.5 — Hazard responses from RE perspective

3 — People 3.1 — Workers' participation

management 3.2 — Training and competence
4.1 — Management systems integration
4.2 — Management of change
4 — General safety
4.3 — Maintenence
factors
4.4 — Procurement and contracting
4.5 — External environment
5 — Planning of 5.1 — Reactive indicators
performance 5.2 — Proactive indicators
monitoring 5.3 — Internal audit
6.1 — Incident investigations
6.2 — Real work investigations
6 — Feedback and
6.3 — Preventive actions
learning
6.4 — Corrective actions
6.5 — Management review and continuous improvement
7.1 — Reactive performance
7 — Performance
7.2 — Proactive performance

Fig. 1. Association between the items of the MAHS and the principles of RE and OHSAS 18001 elements, with additions from ILO-OSH 2001.
1060 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067
(2.4) Hazard responses from a traditional perspective: this
checks whether there is an action plan that is consistent with haz-
ard identification (traditional focus) and risk assessment. For in-
stance, it may be appropriate to check whether the handling of
manual materials is properly dealt with by organizational and
technological preventive measures.
(2.5) Hazard responses from RE perspective: this is a core item
of MAHS since it checks whether there are action plans for dealing
with the hazards identified at item 2.2. For instance, a check is
made on how the difference between real and prescribed work is
managed, whether there are initiatives to design error-tolerant
performance boundaries and whether there are formal guidelines
for carrying out sacrificial judgments of production in favour of HS.
(3) People management
(3.1) Workers’ participation: this is based on the assumption
that workers’ participation concerning HS issues should imply they
learn and become aware of the boundaries of safe performance.
(3.2) Training and competence: this item emphasizes that
workers should receive training on non-technical skills (e.g. com-
munication, error detection and recovery, development of proac-
tive attitudes). It also requires the integration of HS and
production training.
(4) Generic safety factors
(4.1) Management systems integration: this requires the inte-
gration of HS, quality and environmental management systems.
(4.2) Management of change: this item has strong connections
with RE, since it checks whether the organization has mechanisms
for anticipating and managing any changes in the work environ-
ment, taking into account their HS implications as early as possible.
(4.3) Maintenance: since maintenance errors are well-known
factors that contribute to mishaps in complex systems (Reason
and Hobbs, 2003), this item has requirements such as that all risk
management tasks mentioned in criteria 2 also be extended to
maintenance.
(4.4) Procurement and contracting: from a RE perspective, this
item is important because it implies anticipating HS issues during
the procurement and contracting of any resources, such as people,
machinery and materials.
(4.5) External environment: in line with the socio-technical ap-
proach underlying RE, this item has requirements such as that the
organization have mechanisms for being made aware of threats to
and opportunities for HS imposed by the external environment,
which in turn include socioeconomic, educational, political, cul-
tural and legal aspects (Hendrick and Kleiner, 2001). For instance,
this item checks whether the organization maintains either an
adversary or collaborative relationship with government agencies
which are responsible for enforcing HS regulations.
(5) Planning of performance monitoring
(5.1) Reactive indicators: this item checks what the HS reactive
metrics adopted (e.g. frequency accident rates) are, why they are
used, how they are collected, how they are analyzed and how the
organization learns from the metrics. Also, this requires the exis-
tence of both personal and process reactive safety metrics, which
are essential since excellent personal safety performance does
not necessarily imply excellence in process safety performance
(Hopkins, 2009). Indeed, while poor personal safety means that
individual accidents take place, poor process safety means that
hazardous technologies have been damaged.
(5.2) Proactive indicators: this has similar requirements to the
previous item, and emphasizes proactive metrics (e.g. amount of
hours dedicated to HS training). This item also requires mechanisms
to monitor the trade-off between safety and production. As to item
5.1, both personal and process proactive metrics are required.
(5.3) Internal audits: this item is demanded by all major HSMS
standards and it checks how HS internal audits are undertaken,
such as the regularity and use of multiple sources of evidence.
(6) Feedback and learning
(6.1) Incident investigations: this is a well-known item of any
HSMS. However, at MAHS the requirements explicitly state that
any situation of lack of safety should be investigated from a sys-
tems perspective.
(6.2) Real work investigations: in sharp contrast with other
HSMS audits, MAHS requires real work to be regularly audited with
the aim of understanding workers’ adaptive strategies and, conse-
quently, to reduce the gap between real and prescribed work. For
instance, such audits of real work could be undertaken in a similar
way to audits that are conducted in behaviour observation pro-
grams. However, rather than the enforcement of rules, which is
the usual focus in behaviour-based safety programs, audits of real
work from an RE perspective should emphasize both an under-
standing of adaptations to and the identification of workerś de-
grees of freedom.
(6.3) Preventive actions: this item stresses that preventive ac-
tions should be documented and monitored as well as aimed at
closing the gap between real and prescribed work.
(6.4) Corrective actions: this item has similar requirements to
item 6.3, though emphasizing corrective actions.
(6.5) Management review and continuous improvement: at
MAHS, distinctive requirements of this item are that learning and
continuous improvement should be based on an understanding
of adaptations and successful performances, based on data pro-
vided by item 6.2.
(7) Performance
(7.1) Reactive performance: this item assesses the results and
tendency of reactive HS metrics, as well as how they are bench-
marked against external competitors and how they are dissemi-
nated throughout the organization.
(7.2) Proactive performance: this item is similar to item 7.1,
though emphasizing proactive metrics.
4.2. MAHS assessment tool
Applying MAHS takes place by means of assessing each item
based on a series of questions about the organization’s management
practices. Throughout the questionnaire, in each item, what are
made explicit are the type of assessment approach (performance,
structural or operational) and the sources of evidence recommended
for assessing each requirement, such as: interviews with managers,
interviews with representatives of the HS department, interviews
with workers, analysis of documents and records, and direct obser-
vation. With regard to the interviews with workers, it is recom-
mended that it be carried out in groups with a view to reducing the
time taken to collect data, when compared to individual interviews.
The group interview approach has been adopted with good results in
ergonomics assessment studies by the laboratory responsible for
this research, with typically 30% of the total number of workers being
interviewed, in groups of, at most, eight workers (Saurin and Gui-
marães, 2008). It is recommended that the sample of workers inter-
viewed is diversified, including both the most experienced and those
who have been recently hired. The sample should also cover people
from various departments (should this not be possible, focus on the
departments at greatest safety risks) and the interview should be
conducted without the presence of the HS managers and staff.
It is important to emphasize that the scripts to be followed in the
interviews correspond to the very requirements laid down in the
detail of the guidelines for the MAHS. For example, in order to hold
the interviews with the workers, it is necessary for the examiner to
select all the paragraphs which contain the indication that the
interviews with workers are a recommended source of evidence.
The requirements in each paragraph were drawn up in accor-
dance with the point of view of the examiner, with a view to facilitat-
ing his/her work. For example, in item 1.1, paragraph ‘a’, the exact
 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067 1061

words are: ‘‘to highlight what HS objectives have been established by
the company”. If this text had the format set by the HSMS standards,
the text might be: ‘‘the company should establish objectives for HS”.
Beside each requirement related to RE, the principle which is
being assessed is presented. Among the 112 paragraphs, 38 (34%)
are directly related to RE. As an example, the following presents
the requirements linked to item 3.1 (workers’ participation). The
complete questionnaire can be found in the study by Costella
(2008) and other examples of requirements are presented in the
Appendix A.
3.1 Workers’ participation
Approach: operational.
Sources of evidence: interview with representatives from the
HS department (requirements: a, b) interview with workers
(requirements: a, b):
(a) to highlight what the participative approach of the workers
is like. To check what the workers’ degree of involvement is in
improving everyday safety at work, by highlighting the modalities
of participation, whether they are more active or passive;
(b) to highlight if the workers’ opinions are observed in the con-
text of the work process design (awareness and learning).
4.3. Summary of the sources of evidence of the MAHS
Fig. 2 presents a summary of the sources of evidence required
by MAHS, the sources being divided in accordance with the evalu-
ation approach.
From Fig. 2, it can be seen that the MAHS items are evaluated
based on, at least, two sources of evidence (for example, the item

Evidence sources
Structural Performance Operational

Interviews with managers

Interviews with sample of

Performance indicators

Interviews with OSH

Documentation and

Interviews with top

Direct observation
records analysis

representatives
management
analysis

workers
Items
1.1 — HSMS policy and objectives
1.2 — HSMS Planning
1.3 — Structure and responsability
1.4 — Documentation and records
1.5 — Legal requirements
1.6 — Top management commitment
2.1 — Hazard identification from traditional perspective
2.2 — Hazard identification from RE perspective
2.3 — Risk assessment
2.4 — Hazard responses from traditional perspective
2.5 — Hazard responses from RE perspective
3.1 — Workers' participation
3.2 — Training and competence
4.1 — Management systems integration
4.2 — Management of change
4.3 — Maintenance
4.4 — Procurement and contracting
4.5 — External environment
5.1 — Reactive indicators
5.2 — Proactive indicators
5.3 — Internal audit
6.1 — Incident investigations
6.2 — Real work investigations
6.3 — Preventive actions
6.4 — Corrective actions
6.5 — Management review and continuous improvement
7.1 — Reactive performance
7.2 — Proactive performance

Fig. 2. Association between the MAHS items and the sources of evidence for evaluation.
1062 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067

of external environment factors) and, at most, five sources of evi-
dence (for example, hazards response from RE perspective).
Fig. 2 also indicates that the MAHS reconciles in a balanced way
the structural and operational approaches, given that 25 of 28
items adopt both the sources of evidence linked to documentation
(typical of the structural approach) as well as the sources linked to
interviews or direct observation (typical of the operational
approach).
4.4. Attribution of scores to the items assessed
MAHS includes the possibility that a score be attributed to each
item, based on the scoring system adopted by the Brazilian founda-
tion for the national quality award (FPNQ, 2006). The scoring tables
adopted for such an award have been widely tested in the practice
of assessing quality management systems in Brazil, and a substan-
tial number of professionals are familiar with them (FPNQ, 2006).
Thus, for each item among the criteria from 1 to 6, the percentage
should be defined which best approximates the reality observed, in
accordance with the assessment factors being focused on (sub-di-
vided into suitability, proactiveness, refinement and innovation)
and being applied (sub-divided into advertising, continuity and
integration). The items of criterion 7 of the MAHS (results) should
be assessed based on the factors of relevance, current level and
tendency, also defined by the FPNQ in order to assess the results
of the quality management system.
To attribute the scores, with a view to assessing the focus and
application of the management practices (Fig. 3), the following se-
quence should be strictly followed: (a) choose the line from Fig. 3
which is best adjusted to the situation observed; (b) choose the
column from Fig. 3 the description of which best approximates
the situation observed; (c) identify the inter-section of the column
with the line selected by means of the letters and numbers located
in the margins of the scoring table, from which one obtains the cor-
responding percentage.
Should there be characteristics in distinct score bands, one
should always opt for the lower score band. For example, when
we consider the following performance for a given item in relation
to the focus: the management practices are adequate for most of
the requirements of the item (line C); almost all the practices are
refined (line E); meeting almost all the requirements is proactive
(line E); innovation is present in some practices (line E). In this
case, the score will be determined by line C, given that in the lines
above, at least, one of the factors would not be met. It is worth
stressing that that the coverage of each assessment factor should
be determined in accordance with the note in Fig. 3: some (less
or equal to 50%), most (greater than 50%), almost all (greater than
75%) and all of them (100%).
4.5. Steps for applying mahs
Six major steps are needed to apply MAHS:
(a) definition of the team of examiners, which should comprise
at least two people, external or internal to the company.
(b) Initial meeting with the following objectives: mutual intro-
ductions by the company team and the examiners; explana-
tion of the main objectives, of the items and criteria, as well
as of the MAHS assessment tool; negotiation of the schedule
of the assessment process; checking the level of knowledge
about the concepts of RE.
(c) Application of the assessment tool: the examiners should
certify that all the items have been answered, and should
also be familiar with the profile of the company in order to
better understand the management practices adopted and
to estimate how much the company’s characteristics have

Dissemination, 1 2 3 4 5 6
continuity and • Management • Management practices • Management practices • Management practices • Management • Management
practices presented presented are presented are presented are practices presented are practices presented
integration
are not disseminated. disseminated in some disseminated by most of disseminated by the disseminated in almost are disseminated in all
• Use not reported. areas, processes, the main areas, main areas, processes, all areas, processes, areas, processes,
Suitability, • No evidence of products and/or by the processes, products products and/or by the products and/or by the products and/or by the
integration. pertinent interested and/or by the pertinent pertinent interested pertinent interested pertinent interested
proactiveness,
parts. interested parts. parts. parts. parts.
refinement and • Continued use in some • Continued use in most • Continued use in • Continued use in all • Continued use in all
innovation management practices. of the management almost all management management practices. management
• No evidence of practices. practices. • Presented most of the practices.
integration. • Some evidence of • Presented most of the evidence expected • Presented all
integration. evidence expected from from integration. evidence expected
integration. from integration.
• Management practices are suitable for all requisites of
the item.
F • Meeting all requisites is pro-active. 10% 30% 50% 70% 90% 100%
• All practices are refined.
• A inovação está presente em algumas práticas.
• Management practices are suitable for all requisites of
the item.
E • Meeting almost all the requisites is pro-active. 10% 30% 50% 70% 80% 90%
• Almost all the practices are refined.
• A inovação está presente em algumas práticas.

• Management practices are suitable for almost all

requisites of the item.
D • Meeting most of the requisites is pro-active. 10% 30% 50% 60% 70% 70%
• Most practices are refined.

• Management practices are suitable for most of the

requisites of the item.
C • Meeting some of the requisites is pro-active. 10% 30% 40% 50% 50% 50%
• Some practices are refined.

• Management practices are suitable for some requisites of

B the item. 10% 20% 30% 30% 30% 30%

• Management practices are unsuitable for the requisites of

A the item or are not reported. 0% 0% 0% 0% 0% 0%

Notes: Coverage: some (less or equal to 50%), most (more than 50%), almost all (more than 75%) and all (100%).

Fig. 3. Framework of the Brazilian foundation for the national quality award for assessment of the factors in focus and being applied.
 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067 1063

in common with the characteristics of complex systems, the Table 1

context which gives potential to the utility of the MAHS. A Scores obtained in the case study, in accordance with the items of the MAHS.

sequence is proposed for applying the assessment instru- Items of the MAHS Score (%)
ment based on the approach (structural, operational and 1.4 – Documentation and records 70
by performance) and on the respective sources of evidence. 1.5 – Legal requirements 70
Initially, the structural approach may be analyzed, which is 2.4 – Hazard responses from traditional perspective 50
restricted to analyzing documents and records described in 3.2 – Training and competence 50
4.3 – Maintenance 50
the sources of evidence for each item of the MAHS. By start- 2.1 – Hazard identification from traditional perspective 40
ing in this way, it is possible to become familiar with the 3.1 – Workers’ participation 30
HSMS idealized by the company in order to check, later, if 5.1 – Reactive indicators 30
it is being put into practice, besides obtaining support to 5.3 – Internal audit 30
6.1 – Incident investigations 30
enrich the interviews. Next, what is suggested is the analysis
6.5 – Management review and continuous improvement 30
following the approach by performance, which consists of 1.2 – HSMS Planning 20
analyzing records which contain the results of the indicators 6.4 – Corrective actions 20
collected by the company. After the structural and by perfor- 1.1 – HSMS policy and objectives 10
mance assessment, the operational approach of the HSMS 2.5 – Hazard responses from RE perspective 10
7.1 – Reactive performance 10
can be evaluated. This phase consists of direct observation
1.3 – Structure and responsibility 0
and of interviews with those involved in the HSMS. It is 1.6 – Top management commitment 0
important to emphasize that the sequence suggested may 2.2 – Hazard identification from RE perspective 0
not prove to be the most appropriate because of the partic- 2.3 – Risk assessment 0
4.1 – Management systems integration 0
ularities of each company or because of the availability of
4.2 – Management of change 0
the interviewees. It is worth pointing out that the sequence 4.4 – Procurement and contracting 0
suggested was not followed rigidly during the case study, 4.5 – External environment 0
but rather than it can be understood as a proposal of an ideal 5.2 – Proactive indicators 0
situation, which was established based on what was learned 6.2 – Real work investigations 0
6.3 – Preventive actions 0
during the case study.
7.2 – Proactive performance 0
(d) Definition of the scoring of each examiner: after collecting
Average score 20
the data, each examiner, individually, should undertake
their evaluation, the product of which is a given score based
on the tables already quoted.
(e) Meeting to reach consensus on defining the final score: are no incentives for participation, which is voluntary. The workers
under the coordination of the lead examiner, with the aim reported that they valued participating in the QCC a lot and believe
of such a meeting being to analyze the divergences of eval- that they add value to HS. Generally speaking, the degree of work-
uation among the examiners. In extreme cases, in which ers’ participation is limited, given that the only channels of direct
there is no consensus, the lead examiner should decide the participation are the QCC and a monthly meeting of a safety com-
scoring, and register this fact. In the minutes of the meeting, mittee required by regulations. Both of which require management
the lead examiner should record the items which suffered approval in order to implement suggestions, which, normally, also
modifications, report on the discussions and the new scores demand resources. In this item, management practices showed
obtained by means of consensus. themselves to be adequate to some requirements (50%, 1 requisite
(f) Preparation and presentation of the assessment report: the of a total of 2), without refinement, innovation and proactiveness
lead examiner should present the final report on applying (line B of Fig. 3). At the same time, the practices were seen to have
the MAHS to the company’s representatives who took part spread to all the main areas, and demonstrated continuity and evi-
in the first meeting, to top management and to representa- dence of integration with other areas, mainly by means of the QCC
tives of the workers. (column 3 of Fig. 3). Thus the result was 30%, corresponding to in-
ter-section B3 of Fig. 3.
With regard to the individual scores which the examiners
5. Main results of the case study attributed to each item and the scores from the consensus meeting,
among the 28 items, in 13 of them, the scoring was identical
5.1. Scores obtained in each item assessed among the three examiners and, in 10 items, the scoring was equal
for, at least, two examiners. Thus, the scoring was different for the
The scores obtained in the case study are presented in Table 1, three examiners in only 5 items, which needed greater effort to ob-
which indicates that the company investigated aimed, first and tain consensus, although this was always obtained. With the
foremost, to try to fulfil the legal requirements, which demand full exception of only one item, the scoring of the consensus was equal
documentation and records. to the scoring attributed by the lead examiner.
As an example of the argumentation process needed to attribute The fairly good uniformity in the attribution of scores is indica-
scores, the case of item 3.1 (workers’ participation) is presented. In tive of the consistency of the MAHS assessment tool. According to
this case, it was observed that those in charge generally pass on the examiners E1 and E2, the fact of the requirements being organized
workers’ demands to the production manager and to the safety in paragraphs facilitated the attribution of scores. Examiners E1
specialists, there not being any formal system for collecting sug- and E2 reported that the complexity of the assessment of the items
gestions. However, the safety specialists report a low take-up of related to RE was greater than for the other items, in addition to
the workers’ suggestions because they are related to reducing steps which they also reported the difficulty of assessing two aspects
or excluding requirements of the procedures, which, according the of the scoring system (dissemination and integration of the man-
safety specialists’ report, are aimed at excluding preventive mea- agement practices), because of the greater subjectivity perceived.
sures. Besides this, there are the quality control circles (QCC), It is also worth stressing that examiner E2, who had less experi-
which suggest projects, including in the area of HS. However, the ence in HS, showed greater disparity in scoring for the items re-
QCC meetings are held outside normal working hours and there lated to legislation and the structural aspects of the HSMS.
1064 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067
5.2. Analysis of meeting Re principles
5.2.1. Top management commitment
With regard to the principle of top management commitment,
what was detected was the lack of giving value to HS in compari-
son with other management functions. One piece of evidence for
this is the subordination of the HS area to a management unit little
related to this issue (information technology) and the lack of struc-
ture for the HS department, in which the safety specialists are
overloaded with bureaucratic tasks and dedicate little time to the
activities of planning and control.
As another piece of evidence that HS is not among the priorities
of top management, one of the directors reported that, after the
investments made in implementing and maintaining ISO 9001, re-
sources will be invested in environmental management and, after
implementing ISO 14001, the focus will be on HS. Neither is there
any formal assessment of the HS department, its performance only
being questioned if there is a serious or fatal accident. In fact, the
workers reported that top management only takes an interest in
HS when an accident happens. Moreover, the comment was also
made by representatives of the HS department that, due to the
company’s financial difficulties in recent years, the large-scale
changes needed have not been turned into concrete reality, with
only demands needing minor financing being carried out.
5.2.2. Learning
With regard to the principle of learning, observations are not
undertaken to assess how safety procedures are met and by means
of these to monitor the differences between real work and pre-
scribed work. The assessment made it clear that the main function
of the procedures is bureaucratic, as they do not form a document
linked to the dynamics of real work. On being hired by the com-
pany, the workers receive training from the HS department based
on the content of the procedures. This training is later reviewed,
with the addition of more details of the function by the officer-
in-charge in the sector where the worker will work. Thereafter,
the worker gets used to carrying out duties with the help of a more
experienced colleague who works alongside the newly contracted
person for approximately 1 month, until the latter acquires suffi-
cient knowledge and experience to operate the machine alone.
According to the report of the workers interviewed, after initial
training, they do not consult the procedures again, but only follow
the verbal instructions given by the officer-in-charge. Should they
have any doubts, the procedure is not consulted, but rather the
officer-in-charge is. Therefore, the worker does not acquire the ha-
bit of checking the written procedures, given that he is instructed
not to waste time on this but rather to ask the officer-in-charge,
who, for his part, does not conduct audits to check if the proce-
dures are in fact being complied with. Although this may indicate
that the differences between prescribed and real work are great, it
was not possible to specify the magnitude of this difference, for
two reasons: (a) it is not part of the scope of the MAHS to measure
the gap between real and prescribed work, but rather its scope in-
cludes verifying if the company has this concern and if it seeks to
close the gap between real and prescribed work and (b) deep
knowledge of each process would be needed to undertake this
measurement.
Apart from this, moments that would be propitious for learning
such as carrying out preventive actions and a critical analysis of the
HSMS have not been duly exploited. With regard to corrective and
preventive actions, they do not emphasize investigation of the
causes of the deviations and are neither fully broadcast in the fac-
tory nor formally recorded. Such actions normally arise from in-
tense inspection by the safety specialists instead of coming from
the initiative of others who may intervene. With regard to the crit-
ical analysis of HSMS, this is conducted without being based on
reliable and full information, which limits the participants’ under-
standing of the safety performance, and consequently, of the
causes of the problems. It is worth stressing that critical analysis
of HS is analyzed superficially by top management as part of the
critical analysis of the quality management system.
Still with regard to the principle of learning, two other opportu-
nities have been wasted: (a) in the investigations of accidents,
since the company operates under the paradigm of the culture of
blaming the worker, which is felt to be insisted on during the pro-
cess of investigation and (b) in the management of changes, a very
faulty process in the company, since it does not include alterations
in training or procedures should there be changes (for example,
technologies). Thus, changes occur and their impact on HS is per-
ceived late on, which characterizes reactive learning.
5.2.3. Flexibility
Performance in relation to the principle of flexibility is ambigu-
ous. As an example of this, in 2006, at the request of the operator of
an overhead crane, members of the HS department inspected the
main supporting cable of the bridge and issued a report declaring
that there was the grave and imminent risk of the cable snapping,
and requesting the immediate stoppage of the equipment, without
consulting the production supervisors. This attitude generated
huge upsets in production on that day and friction between the
production and safety staff because the former were not consulted.
The positive fact of this example was that the safety staff’s action
prevailed over the pressures from production staff. At the same
time, this was seen as a circumstantial event, since there are no
guidelines for managing the trade-off between safety and produc-
tion. Although top management supported the stoppage decision,
they also held that there should be better communication between
HS and production.
Another ambiguity arises from the fact that, both in the proce-
dures and in the discourse of the production manager, it is recog-
nized that a very quick rhythm can generate accidents, although
there are no mechanisms to identify what characterises an exces-
sive rhythm. For example, in the procedure of the pressing opera-
tion there is a recommendation not to accelerate production with a
view to making up for lost time after a stoppage, the aim of which
is to prevent accidents. In this example, at least under the struc-
tural approach, the principle of flexibility is fairly well satisfied, be-
cause of not ceding to the pressures of production, in favour of
safety. Besides this, according to the representative from the pro-
duction planning and control department, there are no individual
production targets in each workstation. There is only the concern
that the factory as a whole meets the global production targets.
The workers who declared that they did not feel pressurised into
attaining production targets confirmed this.
The deficiencies already cited in the management of changes
also indicate that the company is not prepared to respond flexibly
to the dynamic environment in which it finds itself. The lack of def-
inition of responsibilities regarding HS is also detrimental from the
point of view of flexibility, since, especially during emergencies, it
may not be clear what the channels of assistance are nor do those
who intervene feel obliged to act in favour of HS.
In operational terms, the lack of flexibility is evident in the non-
existence of fail-safe devices in machines with the highest risks,
which would make the limits error-tolerant. According to a tradi-
tional HS approach, a factor which was emphasized by the manag-
ers interviewed was the consideration of requirements concerning
attention, concentration and discipline when selecting skilled
workers who operate machines of greatest risk, such as the presses
in the stamping department, the folding machines and milling.
Nevertheless, this criterion could be less important should the ma-
chines of greatest risk have fail-safe devices, as is the case of a sin-
gle machine which has a system of sensors which switches the
 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067
machine off when something or someone gets close to the hazard-
ous parts.
During the interviews with the managers, situations were also
reported of refusal to work on machines shortly after an accident,
as the member of staff was afraid. According to the production
manager, in this case, the worker was transferred to another func-
tion. Similarly, the company has demonstrated concern for staff
who presents health complaints, by immediately arranging for
them to have medical attention and transferring the worker to
activities with less risk of accident. These facts display the auton-
omy of production management to deal with aspects of HS, which
is one dimension of the principle of flexibility.
5.2.4. Awareness
Although the examples of re-allocation of functions are positive
from the point of view of flexibility, they are indicative of the lack
of meeting the principle of awareness, since they are evidence of a
belated awareness in which the workers and managers become
aware of the risk after tragic events. Another piece of evidence that
the principle of awareness is characterised by great reactiveness
arises from a change of attitudes with regard to safety on the occa-
sion of hiring new welders coming from a large multi-national
company. Such welders were accustomed to wearing protective
spectacles under the welder’s mask and demanded that they
should only work under the same conditions. The company sup-
plied the requested equipment, and the other welders, slowly but
surely, also began to use the same equipment. Currently, most
welders use this double protection, which occasioned the reduc-
tion of eyes injuries.
Although, as has already been commented on, there is a system-
atic effort to train the workers, as well as there being a few channels
for their participation, the potential of these mechanisms to support
the principle of awareness could be better exploited. With regard to
the training, it does not tackle management skills, which would fa-
vour hazard identification and control, such as critical observation
of their own and their colleagues’ work, as well as communication
of hazards. As to participation, the existing channels do not include
opportunities for work enrichment, which would lead the worker to
get to know the nature of their activities in more depth.
Nor is HS planned from the early conception of products and
processes, whether because of the lack of support and release of re-
sources by top management or whether because of the lack of
awareness of the hazards. There is a procedure relating to the
development of new dies and tools in the company itself, it having
been foreseen in this procedure that there was a need for approval
by the HS during a pilot test of the new pieces of equipment. Nev-
ertheless, this posture is reactive for it does not require the partic-
ipation of operators and safety specialists from the initial steps of
the design for new pieces of equipment.
6. Conclusions
The method for assessing health and safety management sys-
tems (MAHS) proposed in this article contributes to filling gaps
regarding the assessment of the HSMS. One of these gaps concerns
the deficiency of the current audits of HSMS when it comes to rec-
onciling the structural, operational and by performance ap-
proaches in a single audit model. Besides this, the MAHS also
contributes to adopting explicitly the focus of RE on HS. With re-
gard to the practical application of the MAHS, what can be stressed
is its use to re-structure the HSMS, and its presenting the main po-
sitive and negative points as well as the action priorities. These are
assessed from a focus which, although still little known in the
industry, makes it possible for there to be wide-ranging assess-
ment of the HSMS performance.
1065
Indeed, key characteristics of the MAHS are the broad scope of
hazard identification and its proactiveness, since the established
requirements may be interpreted as guidelines for implementing
improvement measures consistent with the RE approach. The dif-
ferences between the MAHS and standard-based audits were evi-
dent in the case study. Before this study, the only audit in the
company was based on the ISO 9001 standard, which despite
encompassing HS issues mostly relied on the structural approach.
Due to this narrow focus, the ISO 9001 audit indicated good HS
performance, which was in sharp contrast with the results ob-
tained by the MAHS, which indicated low HS performance.
Regarding the MAHS scope of hazard identification, the results
of the case study also indicated that a positive feature was the
detection of failures from strategic issues (e.g. lack of top manage-
ment commitment) to operational ones (e.g. lack of safeguards in
some equipment). The consideration of generic safety factors was
also beneficial, since this provided a broad perspective of responsi-
bilities for HS, which was strongly limited to the HS staff. Even
though the company performed poorly in the items encompassed
by generic safety factors, the simple questioning of these issues
provided insights for the company staff. For instance, it was real-
ized to what extent the financial crisis the company was going
through was impacting on HS and how the lack of external pres-
sures (e.g. lack of enforcement of regulations by government agen-
cies) in favour of safety contributed to a state of complacency.
Nevertheless, a limitation of the MAHS in terms of scope is that
it looks for the application of RE principles only in the items
encompassed by the assessment tool. Of course, this implies lim-
ited flexibility, since RE principles might be applied in other
dimensions of the organization that are not encompassed by the
HSMS (e.g. product design). Also, in its current version, the MAHS
does not provide tools for systematically analyzing the correlations
among the items assessed, which could be important because of
the systems approach underlying RE. The case study indicated that
such correlations exist, such as with regard to the following items:
(a) planning the monitoring of performance and reactive and pro-
active performance and (b) hazard identification, risk assessment
and hazard response. For example, since proactive indicators have
not been planned, the result of the proactive indicators will also be
non-existent. A characteristic such as this indicates that it is possi-
ble that the improvement in the result of an assessment based on
MAHS may occur in large increments. These correlations also indi-
cate which items should be prioritized since they are prerequisites
for other items.
Another limitation of the MAHS is the need for both company
representatives and examiners to know about RE, a difficulty which
could be tackled by means of drawing up training mechanisms for
examiners (for example, games and manuals with examples of good
practices), with a view to facilitating and making the assessment
process uniform. The difficulty of assessment also springs, as it also
does for other audits of management systems, from the MAHS not
indicating how the requirements should be met, and thus what is
required is an auditor with great experience. In fact, since RE is
not yet a fully established discipline, as well as assuming that it is
not being widely disseminated in the industry, it can be considered
that there is not even a sufficient base of knowledge in the literature
for establishing prescriptive requirements.
Concerning the length of time necessary for applying the MAHS,
it took approximately 96 man-hours, without computing the time
needed for data analysis. The fact of MAHS having been applied
over 4 weeks (one-day visit per week), was considered by company
staff as an advantage in relation to the audits of ISO 9001, which
are generally concentrated into a single week, which facilitates
the company’s circumstantial preparation to receive the auditors.
It is appropriate to recognize that the scenario of the case study
was a limiting factor for the validation of the MAHS. If the company
1066 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067
studied had possessed more advanced HS management practices, it
would probably have been possible to have identified a larger
number of good examples of meeting RE principles, which would
have contributed to the refinement of the model.
Among the opportunities for future studies arising from what
have been presented in this article, adapting and applying the
MAHS in other contexts stands out, principally in organizations
acknowledged as examples of excellence in HS management,
which have a real interest in improving their performance in this
area and which possess characteristics of complex systems.
Appendix A
Extracts from the MAHS assessment tool
1.6 – Top management commitment
Approach: operational.
Sources of evidence: interview with top management represen-
tatives (requirements a, b, c, d, e, f), interview with production
management representatives (requirements a, b, c, d, e, f), inter-
view with representatives from the HS department (requirements
a, b, c, d, e, f) and interview with workers (requirements a, b, c, e):
(a) to highlight if top management participates in HS efforts and
interacts with the interested parties, thus demonstrating
commitment and seeking opportunities to develop the
HSMS (top management commitment and learning);
(b) to highlight if top management is aware of the satisfaction,
motivation and well-being of the workers (top management
commitment and awareness);
(c) to highlight if top management ensures there are resources
for HS (top management commitment);
(d) to highlight if top management shows concern for improv-
ing HSMS performance and if there is oversight regarding
the level of the safety performance indicators, especially
the proactive ones (top management commitment and
awareness);
(e) to highlight if top management demonstrates commitment
both to HS and other business functions, such as production,
marketing and sales (top management commitment);
(f) to check the position of top management with regard to the
management of the pressures of production in relation to
safety (top management commitment and flexibility).
2.5 – Hazard response from an RE perspective
Approach: structural and operational.
Sources of evidence: direct observation (requirements a, d),
analysis of the procedures relative to HS (requirements a, b, d, e),
interview with representatives from top management (c, d), inter-
view with the production manager (requirements a, b, c, d, e),
interview with representatives from the HS department (require-
ments a, b, c, d, e) and interview with workers (requirements a,
c, d):
(a) to observe how the difference between real and prescribed
work is managed. To describe how the monitoring and mod-
ification of the HS procedures are carried out and how the
adaptations carried out by the workers in relation to the
HS procedures are managed (flexibility);
(b) to highlight how the procedures indicate what the limits of
safe work are and how to detect faults and regain control. In
addition, to highlight what the mechanisms are so that the
limits are made visible, respected and/or error-tolerant
(flexibility);
(c) to check how workers and supervisors on the shop-floor
have autonomy to take decisions which influence safety,
such as to stop production should there be an imminent risk
of accidents (flexibility);
(d) to highlight when judgments of sacrificing production can
be observed (stopping production in favour of HS). To high-
light if the procedures include guidelines regarding how to
make these judgments and if there is any indication of the
most likely situations or workstations in which such judg-
ments may become necessary (flexibility);
(e) to check if the planned preventive actions take into account the
systemic focus, the macro ergonomic point of view and the
organizational hazards, not only the physical ones (learning).
References
Ahmad, K., Gibb, A., 2004. Towards effective safety performance measurement –
evaluation of existing techniques and proposals for the future. In: Rowlinson, S.
(Ed.), Construction Safety Management Systems. Routledge Published, pp. 425–
442.
Amalberti, R., 2006. Optimum system safety and optimum system resilience:
agonistic or antagonistic concepts? In: Hollnagel, E., Woods, D., Leveson, N.
(Eds.), Resilience Engineering: Concepts and Precepts. Ashgate, London, pp.
238–256.
Bluff, L., 2003. Systematic Management of Occupational Health and Safety. National
Research Centre for Occupational Health and Safety Regulation, Australian
National University. Working Paper 20.
Cambon, J., Guarnieri, F., Groeneweg, J., 2006. Towards a new tool for measuring
safety management systems performance. In: Proceedings of the 2nd
Symposium on Resilience Engineering, Juan-les-Pins, France, November 8–10,
2006, France.
Carim Júnior, G., Silva, M., Saurin, T.A., 2008. Auditoria de saúde e segurança no
trabalho sob o enfoque da engenharia de resiliência: estudo de caso em uma
empresa de aviação civil. In: XV Simpósio de Engenharia de Produção, Bauru,
Brasil. Anais. Universidade Estadual Paulista Júlio de Mesquita Filho. <http://
www.simpep.feb.unesp.br/anais.php>.
Chase – Complete Health and Safety Evaluation, 2006. Chase Evaluation and Audit
System. <http://www.hastam.co.uk/chase.htm> (accessed February 2006).
Chiesa, V., Coughlan, P., Voss, C., 1996. Development of a technical innovation audit.
Journal of Product Innovation Management 13, 105–136.
Christoffersen, K., Woods, D., 1999. How complex human–machine system fail:
putting ‘‘human error” in context. In: Karwowski, W., Marras, W.S. (Eds.), The
Occupational Ergonomics Handbook. CRC Press, Boca Raton, FL, pp. 585–600.
Clegg, C., 2000. Sociotechnical principles for system design. Applied Ergonomics 31,
463–477.
Costella, M.F., 2008. Método de avaliação de sistemas de gestão de segurança e
saúde no trabalho (MASST) com enfoque na engenharia de resiliência. Porto
Alegre, Tese (Doutorado em Engenharia de Produção) – Programa de Pós-
Graduação em Engenharia de Produção, PPGEP/ UFRGS.
Dupont (DuPont Safety Resources), 2006. <http://www.dupont.com/safety>
(accessed March 2006).
EFQM – European Foundation for Quality Management, 1995. Self-assessment,
1995 Guidelines. EFQM, Brussels.
Eisner, H.S., Leger, J.P., 1988. The international safety rating system in South African
mining. Journal of Occupational Accidents 10, 141–160.
FPNQ – Fundação para o Prêmio Nacional da Qualidade, 2006. Critérios de
excelência: o estado da arte da gestão para a excelência do desempenho.
<http://www.fpnq.org.br/criterios_2006.htm> (accessed October 2006).
Hale, A.R., Heijer, T., 2006. Is resilience really necessary? The case of railways. In:
Hollnagel, E., Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts
and Precepts. Ashgate, London, pp. 115–137.
Hale, A.R., Guldenmund, F., Goossens, L., 2006. Auditing resilience in risk control
and safety management systems. In: Hollnagel, E., Woods, D., Leveson, N. (Eds.),
Resilience Engineering: Concepts and Precepts. Ashgate, London, pp. 270–295.
Hendrick, H., Kleiner, J., 2001. Macroergonomics: An Introduction to Work System
Design. Human Factors and Ergonomics Society, Santa Monica.
Hollnagel, E., 2006. Resilience: the challenge of the unstable. In: Hollnagel, E.,
Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts and Precepts.
Ashgate, London, pp. 8–17.
Hollnagel, E., Woods, D., 2005. Joint Cognitive Systems: An Introduction to Cognitive
Systems Engineering. Taylor and Francis, London.
Hollnagel, E., Woods, D., 2006. Resilience engineering precepts. In: Hollnagel, E.,
Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts and Precepts.
Ashgate, Epilogue, London, pp. 326–337.
Hopkins, A., 2006. What are we to make of safe behaviour programs? Safety Science
44, 583–597.
Hopkins, A., 2009. Thinking about process safety indicator. Safety Science 47, 460–
465.
Kuusisto, A., 2001. Safety management systems: audit tools and reliability of
auditing. Doctor of Technology Thesis, Tampere University of Technology, VTT –
Technical Research Centre of Finland.
Le Coze, J., 2005. Are organisations too complex to be integrated in technical risk
assessment and current safety auditing? Safety Science 43, 613–638.
 M.F. Costella et al. / Safety Science 47 (2009) 1056–1067
Leveson, N.G., Marais, K., Saleh, J.H., 2006. Archetypes for organizational safety.
Safety Science 44, 565–582.
Mitropoulos, P., Abdelhamid, T., Howell, G., 2005. Systems model of construction
accident causation. Journal of Construction Engineering and Management 131
(7), 816–825.
Montagna, A., Ferrari, E., 2006. Improving risk management in complex systems:
from reliability to collective competence. In: Proceedings of Triennial Congress
of the International Ergonomics Association, 16, Maastricht, Netherlands, Delft
University of Technology (CD-ROM).
Rasmussen, J., 1983. Skill, rules and knowledge; signals, signs, and symbols, and
other distinctions in human performance models. IEEE Transactions on Swems,
Man and Cybernetics SMC-13 (3).
Rasmussen, J., 1997. Risk management in a dynamic society: a modeling problem.
Safety Science 27 (2/3), 183–213.
Rasmussen, J., Petersen, A., Goodstein, L., 1994. Cognitive Systems Engineering. John
Wiley & Sons, New York.
Reason, J., 1997. Managing the Risks of Organizational Accidents. Ashgate,
Burlington, p. 252.
1067
Reason, J., 2001. Score your safety culture. Flight Safety Australia, January–February.
Reason, J., Hobbs, A., 2003. Managing Maintenance Error. Ashgate, Burlington, p.
183.
Resilience Engineering Network, 2008. Resilience engineering. <http://
www.resilience-engineering.org> (accessed April 2008).
Saurin, T.A., Guimarães, L.B.M., 2008. Ergonomic assessment of suspended scaffolds.
International Journal of Industrial Ergonomics 38 (2), 238–246.
Saurin, T.A., Formoso, C.T., Cambraia, F., 2008. An analysis of construction safety
best practices from the cognitive systems engineering perspective. Safety
Science 46, 1169–1183.
Woods, D., Hollnagel, E., 2006. Resilience engineering concepts. In: Hollnagel, E.,
Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts and Precepts.
Ashgate, Prologue, London, pp. 1–6.
Wreathall, J., 2006. Properties of resilient organizations: an initial view. In:
Hollnagel, E., Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts
and Precepts. Ashgate, London, pp. 258–268.