10 1016@j Ssci 2006 07 003 PDF

Safety Science 45 (2007) 329353
www.elsevier.com/locate/ssci
Review
The eectiveness of occupational health and safety

management system interventions:
A systematic review
Lynda S. Robson a,*, Judith A. Clarke a, Kimberley Cullen a,
Amber Bielecky a, Colette Severin a, Philip L. Bigelow a,b,
Emma Irvin a, Anthony Culyer a,c, Quenby Mahood a
b
a
Institute for Work and Health, 481 University Ave., Ste. 800, Toronto, ON, Canada M5G 2E9
Department of Public Health Sciences, University of Toronto, 6th Floor, Health Sciences Building,
155 College Street, Toronto, ON, Canada M5T 3M7
c
Department of Economics and Related Studies, University of York, Heslington,
York Y01 5DD, England, UK
Received 23 December 2005; received in revised form 12 May 2006; accepted 6 July 2006
Abstract
A variety of OHSMS-based standards, guidelines, and audits has been developed and disseminated over the past 20 years. A good understanding of the impact of these systems is timely. This
systematic literature review aimed to synthesize the best available evidence on the eects of OHSMS
interventions on employee health and safety and associated economic outcomes. Eight bibliographic
databases covering a wide range of elds were searched. Twenty-three articles met the studys relevance criteria. Thirteen of these met the methodological quality criteria. Only one of these 13 original
studies was judged to be of high methodological quality; the remainder had moderate limitations.
The studies results were generally positive. There were some null ndings but no negative ndings.
In spite of these promising results, the review concluded that the body of evidence was insucient to
make recommendations either in favour of or against OHSMSs. This was due to: the heterogeneity
Corresponding author. Tel.: +1 416 927 2027; fax: +1 416 927 4167.
E-mail addresses: lrobson@iwh.on.ca (L.S. Robson), jclarke@iwh.on.ca (J.A. Clarke), kcullen@iwh.on.ca (K.
Cullen), abielecky@iwh.on.ca (A. Bielecky), cseverin@iwh.on.ca (C. Severin), pbigelow@iwh.on.ca (P.L.
Bigelow), eirvin@iwh.on.ca (E. Irvin), aculyer@iwh.on.ca, ajc17@york.ac.uk (A. Culyer), qmahood@iwh.on.ca
(Q. Mahood).
0925-7535/$ - see front matter 2006 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ssci.2006.07.003
330
L.S. Robson et al. / Safety Science 45 (2007) 329353
of the methods employed and the OHSMSs studied in the original studies; the small number of
studies; their generally weak methodological quality; and the lack of generalizability of many of
the studies.
2006 Elsevier Ltd. All rights reserved.
Keywords: Management system; Systematic literature review; Audit; Intervention; Eectiveness
Contents
1.
2.
3.
4.
5.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1. Dening OHSMSs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2. Voluntary and mandatory OHSMS initiatives . . . . . . . . . . . . . . . . . . . . . . . . .
1.3. Background literature and research justication . . . . . . . . . . . . . . . . . . . . . . . .
1.4. Conceptual framework of the review. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1. Literature search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2. Selection of studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3. Quality appraisal (QA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4. Data extraction (DE). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5. Evidence synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1. Publications identied through the literature search and quality assessment . . . .
3.2. Quality of the literature investigating OHSMSs . . . . . . . . . . . . . . . . . . . . . . . .
3.3. Findings on the implementation and eectiveness of voluntary OHSMSs . . . . . .
3.3.1. Description of the studies on the implementation and eectiveness of
voluntary OHSMSs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.2. Findings on the implementation of voluntary OHSMSs . . . . . . . . . . . . .
3.3.3. Findings on the intermediate outcomes of voluntary OHSMSs . . . . . . . .
3.3.4. Findings on the nal outcomes of voluntary OHSMSs . . . . . . . . . . . . . .
3.4. Findings on the implementation and eectiveness of mandatory OHSMSs . . . . .
3.4.1. Description of the studies on the implementation and eectiveness
of mandatory OHSMSs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.2. Findings on the implementation and eectiveness of the IC legislation
in Norway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.3. Findings on the eectiveness of simple OHSMS legislation in Canada . . .
3.5. Evidence synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.1. Evidence for the eectiveness of voluntary OHSMS interventions . . . . . .
3.5.2. Evidence for the eectiveness of mandatory OHSMS interventions . . . . .
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1. Identifying and addressing research gaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2. Strengths of the review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3. Limitations of the review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
331
331
332
333
334
335
335
335
336
337
337
338
338
338
340
340
341
341
342
342
342
342
343
344
344
346
347
347
348
348
349
350
350
331
1. Introduction
The concept of an occupational health and safety management system (OHSMS) has
become common over the past 20 years. A variety of OHSMS-based standards, guidelines,
and audits have been developed (British Standards Institution, 1996, 1999; Dalrymple et al.,
1998; Frick et al., 2000; Gallagher et al., 2003; Grayham and del Rosario, 1997; HSE, 1997;
ILO, 2001; Standards Australia and Standards New Zealand, 1997) within the public, private and not-for-prot sectors and many have been adopted by workplaces. Some countries,
including Canada, are in the process of developing management standards for occupational
health and safety. An understanding of the impact of these systems is therefore timely.
This systematic literature review investigates the eectiveness of mandatory and voluntary OHSMS interventions on employee health and safety and associated economic
outcomes. The scope of the topic was developed through formal and informal consultations with representatives of employers, labour, and several public and not-for-prot institutions. A secondary aim of the review is to characterize the content and methodology of
the existing research literature on OHSMSs. The purpose here is to identify gaps and
weaknesses in the literature, whose addressing through future research could help inform
the primary research question.
There are no other systematic literature reviews on the topic of OHSMSs, although some
narrative reviews exist (Frick et al., 2000; Gallagher et al., 2003; Saksvik and Quinlan, 2003;
Walters, 2002). A systematic literature review uses explicit, thorough methods to identify,
select, appraise and synthesize a set of research studies on a well-dened topic. This methodology makes the ndings of a systematic review less vulnerable to the biases of a single
researcher than those of a narrative review. Systematic reviews aid decision-makers by sifting through an enormous literature to nd the high-quality studies and to synthesize them.
This paper will rst discuss some of the concepts and background literature relevant
to the review.
1.1. Dening OHSMSs
There is no consensus on what an OHSMS is and its scope is potentially wide. Some
denitions are simply too vague to be helpful in determining which literature should be
included in a systematic review: e.g., the denition used by the International Labour Organization (ILO, 2001):
A set of interrelated or interacting elements to establish OSH policy and objectives,
and to achieve those objectives.
It is not clear from the denition whether the management system includes only management components or technical/operational components as well. This problem of
demarcating the scope of a management system has also been noted by Nielsen (2000):
OHSM systems are not, of course, a well-dened set of management systems.
Indeed there are not clear boundaries between OHS activities, OHS management,
and OHSM systems.
332
OHSMSs, commonly understood, are distinguishable from traditional OHS programs

by being more proactive, better internally integrated and by incorporating elements of
evaluation and continuous improvement. Some OHSMS documents (e.g., Chemical
Industries Association, 1995; HSE, 1997; ILO, 2001) explicitly ascribe their basic source
as the Plan-Do-Check-Act model of continuous quality improvement made famous by
W. Edwards Deming (Tortorella, 1995). In contrast, traditional OHS programs have relatively less activity corresponding to the Check and Act domains of the Deming Model.
Furthermore, action tends to be in response to workplace accidents, legislation, or
enforcement, rather than proactive.
Redinger and Levine (1998) considered what constitutes an OHSMS in detail. After
reviewing 13 publicly available management system documents for occupational health
and safety, environment, or quality, they selected four of the most comprehensive from
which to construct an integrative, universal OHSMS model containing 27 elements. The
16 primary elements of their model are:
management commitment and resources,

employee participation,
occupational health and safety policy,
goals and objectives,
performance measures,
system planning and development,
OHSMS manual and procedures,
training system,
hazard control system,
preventive and corrective action system,
procurement and contracting,
communication system,
evaluation system,
continual improvement,
integration,
management review.
1.2. Voluntary and mandatory OHSMS initiatives

OHSMS initiatives are either mandatory or voluntary. Mandatory OHSMSs arise from
government legislation and their use is enforced through inspections, nes, etc. Voluntary
OHSMSs arise through private enterprise, employer groups, government and its agencies,
insurance carriers, professional organizations, standards associations and are not directly
linked to regulatory requirements. Their use is not required by governments; instead,
incentives are sometimes oered by governments or insurance carriers to organizations
that voluntarily adopt particular OHSMSs.
Many voluntary OHSMSs, especially those marketed through commercial industries, are
most frequently observed in large companies. They are characterized by being more thoroughly specied, and as a result, are considered to be too complex for the majority of (smaller) employers (Frick and Wren, 2000). Voluntary OHSMS schemes marketed through
public agencies, however, target not only large companies but also smaller ones (Frick
and Wren, 2000). These schemes either involve simpler OHSMSs or have a menu of options,
333
including simple ones, for companies of dierent sizes or at dierent stages of OHSMS
development. Mandatory OHSMSs are simpler in terms of the demands placed on organizations, since they are intended for all or most workplaces, including small ones.
1.3. Background literature and research justication
There is a body of research on the correlates of low injury rates in organizations (e.g.,
Cohen, 1977; Habeck et al., 1998; LaMontagne et al., 1996; Reilly et al., 1995; Shannon
et al., 1996). These studies suggest which potential elements of an OHSMS are important
by identifying those that are correlated with low injury rates. Other studies (Mearns et al.,
2003; Simard and Marchand, 1994) have developed researcher-dened measures of
OHSMSs and shown that a more developed OHSMS is correlated with a lower injury rate.
While these studies are a valuable contribution to the literature, they cannot tell us what
the likely eect is of a particular type of OHSMS intervention in a particular type of workplace. They are limited by their cross-sectional design and the lack of an OHSMS intervention variable.
A separate stream of research and practice related to safety management systems in
high hazard and high reliability operations, such as in the nuclear, chemical process,
and airline, rail and marine transportation industries, has also developed (e.g., Figuera
and Lopez, 2003; Hale, 2003; IAEA, 2005; SAMRAIL Consortium, 2004). The concept
of a safety management system overlaps with that of an OHS management system, but
is generally distinct. For instance, the scope of concern for a safety management system,
unlike that of an OHSMS, extends beyond workers to include the physical work environment and the surrounding community. Furthermore, the scope of an OHSMS covers a
broad range of workers health concerns, in contrast to that of a safety management system
which focuses on preventing traumatic injuries related to the loss of control of processes.
Many OHS practitioners presume that OHSMS interventions will be eective in lowering
injury and illness. After all, OHSMS standards and guidelines synthesize expert knowledge,
much of which is consistent with the research cited in the rst paragraph of this section.
However, the eectiveness of interventions cannot be presumed in all cases. Indeed, the failure rate of quality management systems has been documented as ranging from 67% to 93%
(Gardner, 2000). There is reason to expect that the failure rate of OHSMSs would be at least
as high. Typically, the level of management commitment to high product or service quality is
higher than to employee health and safety. The eectiveness of mandatory OHSMS strategies
has also been doubted by Quinlan and Mayhew (2000) in light of the current trends of the
globalization of business, the casualization of the labour force and declining unionization.
Some criticisms of OHSMSs have also emerged in more academic circles. Suggested deleterious eects of OHSMSs in general or of particular types of OHSMSs have included:
the weakening of external regulatory approaches (Bennett, 2002); a false sense of security
derived from the presence of a formal OHSMS (Gallagher et al., 2003); the development of
blame-the-worker attitudes (Nichols and Tucker, 2000; Wokutch and VanSandt, 2000);
and a shift in the power balance away from workers and toward management (Lund,
2004; Nichols and Tucker).
The success of OHSMSs is likely to be dependent on the nature of the intervention,
characteristics of the workplace and characteristics of the external environment. This
review synthesizes the research results on the eectiveness of a wide range of OHSMS
interventions under a wide range of conditions.
334
1.4. Conceptual framework of the review

The conceptual framework underlying the review is depicted in Fig. 1. The review
included interventions directed at developing an OHSMS in one or more workplaces. It
therefore included studies of extra-workplace initiatives arising from legislation, or voluntary programs arising through the government, its agencies, insurance carriers, groups of
employers, etc. It also included studies of workplace-level initiatives, through which a
workplace might attempt to improve its OHSMS, using either a scheme developed externally or internally.
A minimalist operational denition of an OHSMS intervention was adopted to screen
the research literature. For a study to be included, an intervention was required to address
two or more of the 27 elements in the Redinger and Levine (1998) universal OHSMS
framework, with at least one of these being a management element.
While the primary focus of the review was on the eectiveness of OHSMSs, evidence
was also sought about implementation at the workplace level. There were two reasons
for this. First, it was anticipated that for some mandatory initiatives there might be measures only of the OHSMS rather than of its eects in workplaces. Second, implementation
information would allow one to distinguish between two possible explanations for an
absence of eect, namely poor intervention content or poor implementation of the
intervention. It is possible for a well-conceived intervention to fail through poor
implementation.
One could measure implementation in extra-workplace OHSMS initiatives (e.g., legislative initiatives) through indicators of extra-workplace implementation (e.g., labour
inspector orders pertaining uniquely to the legislation). In this review, however, implementation was considered only at the workplace level; specically, a change in the state of the
workplaces OHSMSs.
Final outcomes were identied by considering the ultimate purpose of OHSMS interventions. This would be improved employee health and safety for many stakeholders.
For others, the associated economic benets were also of interest. On these grounds,
two types of nal outcomes were studied in the review: OHS outcomes and economic
outcomes. Examples of nal OHS outcomes are changes in rates of employee injury or illness. Examples of economic outcomes are changes in workplace workers compensation
premium rates and workplace productivity.
INTERVENTION
IMPLEMENTATION
Workplace or extraworkplace initiative
Change in workplace
OHSMS
INTERMEDIATE
OHS OUTCOMES
FINAL OHS
OUTCOMES
(e.g., safety climate)
(e.g., injury rates)
ECONOMIC
OUTCOMES
(e.g., firm insurance
premiums)
Fig. 1. Conceptual framework for the review on OHSMSs.
335
Intermediate OHS outcomes were considered to be outcomes of secondary interest,

though they are proxies potentially for nal OHS outcomes. They are changes in mediators between the OHSMS and nal OHS outcome, such as safety climate; employee
knowledge, beliefs, values or perceptions; employee behaviours; or OHS hazards.
2. Methods
The methodological steps were: (1) search of the literature; (2) selection of relevant
studies by applying inclusion and exclusion criteria; (3) appraisal of the quality of the
research evidence in the studies; (4) extraction of the higher quality evidence from the studies; and (5) a synthesis of the higher quality evidence.
2.1. Literature search
Eight electronic databases, abstracting primarily peer-reviewed research journal articles,
were searched from their inception until July 2004: MEDLINE (from 1966), EMBASE
(from 1980), PsycInfo (from 1887), Sociological Abstracts (from 1963), CCInfoWeb (consisting of NIOSHTIC-2, HSELINE, and OSHLINE), Safety Science and Risk Abstracts
(SSRA, from 1981), EconLit (from 1969), and American Business Inform (ABI, from
1918). Since the search terms and language of the databases were found to dier signicantly, the terms used in the search were customized for each database.
The search strategy combined two sets of keywords using an AND strategy. The rst
set of keywords focused on a set of 30 OHSMS terms (e.g., health and safety management
system(s), safety management system(s), systematic occupational health and safety management, OHS program(s), OHSAS 18001, OHSMS, BS8800, International Safety Rating
System(s), safety program(s), occupational health/safety standard(s), safety and health
legislation). The keywords in the second set included 35 evaluation or OHS eect terms
(e.g., evaluat(ing/ion/ions/e/es/ed), program/me evaluat(ing/ion/ions/e/es/ed), implementat(ing/ion/ed/es), eect(s), impact(s), climate, culture, perception(s), behavio(u)r, workplace injur(y/ies), injury, occupational health, occupational exposure, occupational
accident, compensation cost(s), compensation claims cost(s), time loss/lost). The full
search strategy is available in Robson et al. (2005) or from the authors. The terms within
each group were combined using an OR strategy. When possible, the titles, abstracts,
case registry, and subject headings were searched for keywords. The search strategy was
simplied for CCInfoWeb and ABI, which had simpler search capacities.
After merging the citations from the electronic search of the eight databases and removing duplicates, 4837 remained for inclusion in the review. Four additional sources were
used to identify citations of potential relevance: the reference lists of all journal articles
meeting the eligibility criteria; the reference lists of pertinent reviews published in the form
of books, book chapters, or journal articles; bibliographies requested from experts in the
area of OHSMSs; and the personal les of the authors.
2.2. Selection of studies
Titles, key words and abstracts of each article were independently screened by two
reviewers. Full text articles were retrieved for those studies appearing to meet the eligibility
criteria, and for those where the information in the title, abstract, and key words was
336
insucient for exclusion. Upon retrieval of the full text article, the eligibility of a study
was determined again through a pair of reviewers. A consensus method was used to
resolve any disagreements between the reviewers about inclusion. A third reviewer was
used if the pair could not reach a consensus on eligibility.
The full set of inclusion and exclusion criteria used to determine the eligibility of studies
is described below:
Publication type. Only peer-reviewed journal articles were included.
Population of interest. Workplaces could be located anywhere in the world.
Nature of intervention. An OHSMS intervention, initiated at either the workplace level
or extra-workplace level was required. An OHSMS intervention was identied by one
of three means:
(i) directly, by a term synonymous with OHSMS or mention of specic types of
OHSMS (e.g., safety and health management system, OHSAS 18001);
(ii) indirectly, by mention of OHSMS legislation or other extra-workplace OHSMS
initiatives (e.g., European Framework Directive 89/391, Internal Control); or
(iii) indirectly, by a term suggestive of OHSMS, and a description of its components
that demonstrated that it was an OHSMS (e.g., comprehensive occupational health
and safety program). In the case where terms were merely suggestive of an
OHSMS, a description of the OHSMS had to have been reported or referenced
and to have qualied as that of an OHSMS (i.e., two or more system elements
(Redinger and Levine, 1998) were specied, at least one of which was in the management domain rather than the activity/technical domain).
Multi-faceted management system interventions were included if they had an occupational health and/or safety component (e.g., a safety, health and environmental management system) with primary prevention as the main focus. Management system
interventions focusing on disability or health services were excluded. Extra-workplace
or workplace initiatives designed to address isolated aspects of OHSMSs or particular
risks (e.g., needle-stick injuries in a health-care facility) were not included.
Type(s) of evidence. Studies had to examine either OHSMS implementation or the
eectiveness of OHSMS interventions.
Implementation studies were required to have a quantitative measure of change in the
level or intensity of the OHSMS. Eectiveness studies were required to have a quantitative
measure of one of the following outcomes: intermediate OHS outcomes (e.g., changes in
knowledge, beliefs, values, perceptions, behaviours, hazards, or risks); nal OHS outcomes (e.g., changes in injury/illness statistics or employee quality of life); or economic
outcomes (e.g., changes in the costs associated with employee illness/injury).
Studies were required to make a comparison of outcomes with the presence and absence
of an OHSMS intervention, or between OHSMS interventions of dierent intensities.
2.3. Quality appraisal (QA)

Studies meeting the eligibility criteria were assessed for methodological quality using a
process developed by the authors based on previous work (Abenhaim et al., 2000; Cote
337
et al., 2001; Drummond and Jeerson, 1996; Franche et al., 2005; Jadad, 1998; Kuhn et al.,
1999; Schulpher et al., 2000; Tompa et al., 2004; van Tulder et al., 2003; Zaza et al., 2000).
The method developed for this review emphasized parsimony with an aim to streamlining
the consensus procedure. The questions focused on internal validity.
The ve quality appraisal questions were:
(i) Are you condent that the means of selecting and maintaining the sample minimized
bias?
(ii) Are you condent that the potential confounders were adequately considered, and
then either well controlled or appropriately discounted as a source of bias?
(iii) Are you condent that the measurement methods did not introduce bias to the
corresponding ndings?
(iv) Were appropriate statistical tests applied to the data?
(v) Are you condent that there are no additional potential sources of bias in the estimate of implementation/eectiveness not already captured in the previous questions?
Multiple choice response options were provided (e.g., yes, partially, no, unclear/
unknown from information provided).
Reviewers gave an overall summary rating of the quality of evidence provided by the
study. They selected one of the following four categories:
very low (serious limitations),

low (major limitations),
moderate (moderate limitations), and
high (no or minor limitations).
The methodological quality of the evidence of each study was rated independently by
two reviewers, who then met for consensus. If consensus could not be reached an additional reviewer was consulted. The quality assessment was carried out separately for
implementation, intermediate, nal OHS and economic outcomes in each study.
2.4. Data extraction (DE)
Evidence rated during the quality appraisal step as moderate or high was extracted for
the evidence synthesis. Pairs of reviewers independently extracted data from the included
studies, using a standard form, and then met to reach consensus. Data were extracted on
the research question, intervention, study design, study population, results, and statistical
analyses.
2.5. Evidence synthesis
Many systematic reviewers choose an explicit algorithm at the outset of the study for
later translation of the ndings into a summary statement about the level or strength of
evidence they provide. (Briss et al., 2000; Franche et al., 2005; GRADE Working Group,
2004; Kuhn et al., 1999; Tompa et al., 2004; van Tulder et al., 2003). Criteria for these
algorithms are customarily based on study design, quality of research, consistency of
the results, and number of studies.
338
This review did not adopt an explicit algorithm at the outset, since there is a lack of
consensus in the eld as to which synthesis algorithm is best. In addition, it was thought
premature to base an algorithm upon a newly developed quality assessment tool. Instead,
a summary statement was synthesized in the style of a traditional narrative review, after
considering the same aspects of evidence as do the algorithms. Such an approach is permissible in best-evidence syntheses (Slavin, 1995) and has been used in other systematic
reviews in this eld (see special issue of American Journal of Preventive Medicine, 2000).
3. Results
3.1. Publications identied through the literature search and quality assessment
Following a review of titles and abstracts, and initial screening of full papers where necessary, 23 studies were identied which met the inclusion criteria. Their citations were
found through the following sources:
bibliographic databases (18),

citations in reviews (3),
authors personal les (1),
suggestion by external reviewer of interim report (1).
The 23 eligible studies were then assessed for methodological quality. Thirteen contained
evidence given an overall rating of moderate or high and proceeded to the next step in
the review, data extraction. The other ten studies, whose evidence received an overall rating
of low or very low, (Anonymous, 1993, 1994; Bolton and Kleinsteuber, 2001; Dotson,
1996; Eisner and Leger, 1988; Everley, 1997; Kjellen et al., 1997; Lanoie, 1992; Mikkelsen
and Saksvik, 2004; Nichols, 1990) were excluded from further review. (It should be noted
that this overall rating was determined from the point of view of this reviews question
about OHSMS eectiveness. The studies excluded at this stage were not necessarily low
quality studies from the point of view of their own research questions.)
An overview of the characteristics of the 13 studies proceeding to data extraction is
shown in Table 1. Seven were of voluntary OHSMS interventions, all of which took place
in English-speaking countries. Many of the voluntary interventions were unique to the
workplace(s) under study. Six studies were of mandatory OHSMS interventions, with four
dealing with the Norwegian Internal Control legislation introduced in 1992. The other two
were initiatives of Canadian provincial legislatures in the late 1970s.
While the studies of mandatory OHSMSs were of large numbers of workplaces, the
studies of voluntary OHSMSs were of small numbers of workplaces and often of just
one workplace. The most common eect studied was implementation but there were also
some ndings on intermediate OHS outcomes, nal OHS outcomes and economic
outcomes.
3.2. Quality of the literature investigating OHSMSs
The methodological quality of the OHSMS intervention literature was generally judged
to be weak. Only the study of implementation by LaMontagne et al. (2004) was assessed as
high quality.
Table 1
Summary of the 13 studies providing the highest quality evidence
Authors (year of publication)
Voluntary interventions
Alsop and LeCouteur (1999)
Edkins (1998)
Country
Industrial sector
Number of organizations
in analysis
Type of data
Organizations own; based

on international and
national standards
Organizations own
Australia
Public administration
1 organization;
multiple sites
Economic
United States
Manufacturing
1 corporation;
multiple sites
Australia
Transportation, air
1 airline; 2 sites
Implementation
Final OHS
Economic
Intermediate OHS
United States
Australia
Manufacturing
Manufacturing
15 work sites
16 companies
Implementation
Implementation
Walker and Tait (2004)

Yassi (1998)
Organizations own;
based on
other programs in industry
Researcher
Organizations
own; adapted
from AS/NZS 4804
HSE-based
Organizations own
Britain
Canada
Mixed
Health care
24 enterprises
1 hospital
Implementation
Economic
Mandatory interventions
Dufour et al. (1998)
Quebec LSST (1979)
Canada
Manufacturing
Economic
Lewchuk et al. (1996)

Saksvik and Nytro (1996)
Ontario Bill 70 (1979)

Internal Control (1992)
Canada
Norway
Manufacturing; retail
All
All rms in most

manufacturing sectors
636 workplaces
2092 enterprises
Nytro et al. (1998)

Saksvik et al. (2003)
Torp et al. (2000)

Norway
Norway
Norway
All
All
Motor vehicle repair services
11822092 enterprises
11822092 enterprises
267 garages
LaMontagne et al. (2004)

Pearse (2002)
Final OHS
Implementation
Intermediate OHS
Final OHS
Implementation
Implementation
Intermediate OHS
Final OHS
Bunn et al. (2001)
OHSMS interventiona
Abbreviations used in column: AS/NZS 4804, Australian/New Zealand Standard; HSE, Health and Safety Executive (British government).
339
340
The prevalence of methodological weakness was mainly a consequence of the experimental designs typically employed. Only three of the 23 eligible studies (Edkins, 1998;
LaMontagne et al., 2004; Yassi, 1998) employed a comparison group of any kind; the
remaining studies used after-only, cross-sectional, beforeafter or time series designs.
The quality assessment process rated studies against four main criteria concerned with
the studys internal validity: sampling, confounding, measurement, and statistics. The
most common weakness in the 23 eligible studies was poor evaluation of and control
for confounders, with 11 studies failing to meet the corresponding criterion at least partially. The second most common weakness, found in six studies, concerned potential bias
introduced through measurement methods.
The diculty in meeting the confounding criterion was inherent in the studies with no
comparison group. In some cases, the publications lacked explicit consideration of potential confounders. In single workplace studies, co-interventions were a major concern. For
example, some authors mentioned the existence of coincident interventions such as cost
control (Bunn et al., 2001) or restructuring (Kjellen et al., 1997). Bolton and Kleinsteuber
(2001) stated that many other variables, including budgetary considerations and the
scope and nature of programmatic tasks and activities could have contributed to the
injury rate trend they observed. For other single workplace studies, co-intervention was
a concern simply because the authors omitted any explicit consideration and discounting
of it. No doubt in some cases, there were no major, relevant changes, but the lack of any
report caused the study to fail on the criterion concerned with confounding.
Insucient information was also a reason why some studies failed to satisfy the criterion regarding measurement methods. In other studies, self-report methods were used to
measure OHSMS implementation without any consideration of the bias arising when
people have a vested interest in portraying compliance.
3.3. Findings on the implementation and eectiveness of voluntary OHSMSs
3.3.1. Description of the studies on the implementation and eectiveness of voluntary
OHSMSs
The reviews search for relevant literature resulted in 14 studies of voluntary OHSMSs,
seven of which remained after quality assessment (Alsop and LeCouteur, 1999; Bunn et al.,
2001; Edkins, 1998; LaMontagne et al., 2004; Pearse, 2002; Walker and Tait, 2004; Yassi,
1998). Only one study (Bunn et al., 2001) examined multiple outcomes (implementation,
nal OHS outcomes, and economic outcomes). The other studies focused on only one
implementation (LaMontagne et al., 2004; Pearse, 2002; Walker and Tait, 2004), intermediate OHS (Edkins, 1998), or economic (Alsop and LeCouteur, 1999; Yassi, 1998).
The studies involved a variety of interventions and samples. Four studies (Alsop and
LeCouteur, 1999; Bunn et al., 2001; Edkins, 1998; Yassi, 1998) reported on OHSMSs developed and implemented in a single organization respectively, a municipal government, an
international manufacturer, a regional airline, and a hospital. The study by Alsop and
LeCouteur (1999) concerned an OHSMS developed in the presence of and integrated with
management systems for quality and the environment. The intervention reported by Bunn
et al. (2001) consisted of the creation of a Health, Safety and Productivity Department,
which integrated all functions concerned with employee health, safety and associated productivity. Key features of the intervention were goal-setting, performance measurement
and senior management commitment. Edkins (1998) investigated a new system of hazard
341
identication, safety information management, and safety communication, overseen by a

new safety manager. The hospital intervention (Yassi, 1998) consisted of the systematic
identication, measurement, and control of OHS risks, as well as program evaluation.
The three other studies of voluntary OHSMSs involved interventions delivered to multiple organizations. Pearse (2002) described a community intervention with 20 small and
medium-sized metal fabrication companies. It consisted of the development and dissemination of OHSMS guidelines adapted from the Australian/New Zealand voluntary standard (AS/NZS 4804), group networking meetings and audits. LaMontagne et al.s (2004)
research group targeted large manufacturers that were likely to use hazardous substances.
Fifteen worksites completed the intervention, which included a baseline OHS program
assessment and tailored consultation and educational sessions for managers and committees. Sessions emphasized upper management commitment, employee participation,
improvement in the resources directed towards hazard control, and OHS training for all
employees. Walker and Tait (2004) determined the impact of 90-min consultation sessions
delivered by two non-prot information centres upon the OHSMSs of small- and mediumsized organizations.
3.3.2. Findings on the implementation of voluntary OHSMSs
The four studies of implementation all showed the sizeable development of OHSMSs,
as determined by audit methods. The Walker and Tait study (2004) showed marked selfreported changes in terms of a policy statement being present and a risk assessment having
taken place. Forty-six percent of organizations added a policy statement and 79% added a
risk assessment as a result of the intervention. It must be noted that these organizations
had requested assistance with establishing an OHSMS, and that these two OHSMS elements were emphasized in the information provided to the organizations.
The other three implementation studies reported positive changes in the scores from a
quantitative audit. Bunn et al. (2001) reported an increase from 63% to 79% of available
audit points over a 2 years period for the organization studied. Pearse (2002) and
LaMontagne et al. (2004) included data in their publications that allowed eect sizes to
be calculated (0.48 and 0.68, respectively). These are considered to be medium and mediumlarge, respectively (Cohen, 1977).
Only one of the four studies of implementation contained information relating these
changes in audit scores to changes in nal OHS or economic outcomes (Bunn et al.,
2001). Thus, the ultimate practical signicance of most of these reports remains unknown.
In the study by Bunn et al. (2001), described below, there were changes in nal OHS and economic outcome indicators, but it is dicult to attribute these solely to the changes in the
OHSMS because of the concurrent cost containment initiatives that were also underway.
3.3.3. Findings on the intermediate outcomes of voluntary OHSMSs
As for intermediate outcomes, Edkins (1998) reported greater positive changes in the
intervention group than in the comparison group, both for some self-reported measures
(e.g., safety culture index) and for some objective measures (condential hazard reports,
organizational actions). Thirteen new organizational actions in safety were attributed to
the intervention group, while none were attributed to the control group. As with the implementation ndings, the signicance of these results for nal outcomes remains unknown.
Furthermore, the reader is left wondering whether the large achievements in organizational actions were largely attributable to the personal qualities of the new safety manager
342
rather than the entirety of the new OHSMS. Indeed, the reader is told that many of the
actions addressed long standing issues.
3.3.4. Findings on the nal outcomes of voluntary OHSMSs
Bunn et al. (2001) was the only study of voluntary OHSMSs to report on nal OHS
outcomes. They found a 24% decrease in illness/injury frequency and a 34% decrease in
lost-time case rate over 3 years. It should be noted that management became accountable
to the board of directors for improvement in these safety indicators, coincident with the
intervention. One eect of this would have been to ensure critical management commitment for the OHSMS intervention. It could also have led to a shift in the reporting practices in the organization, such that incidents were less likely to be documented. The
authors refer to aggressive return to work programs, which could have contributed to
the reduction in lost-time case rate.
Workers compensation costs also decreased markedly in studies of voluntary
OHSMSs. Bunn et al. (2001) found a 13% decrease in workers compensation cost per
employee. Yassi (1998) and Alsop and LeCouteur (1999) found decreases of 25% and
52% in premium rates, respectively. However, in each of these studies, there were initiatives beyond primary prevention which likely contributed to these results. Yassi (1998)
mentioned a return to modied work program. Alsop and LeCouteur (1999) referred to
targeted and sustained eort in claims and injury management during the 2 years prior
to the OHS initiative. Bunn et al. (2001) listed various cost-cutting measures including
better case management and aggressive return to work programs.
3.4. Findings on the implementation and eectiveness of mandatory OHSMSs
3.4.1. Description of the studies on the implementation and eectiveness of mandatory
OHSMSs
Nine studies of mandatory OHSMS interventions were identied by the review. Six
remained after the quality assessment screen. Four of these were based on the Norwegian
regulations on Internal Control (IC) of health, safety, and the environment (Nytro et al.,
1998; Saksvik and Nytro, 1996; Saksvik et al., 2003; Torp et al., 2000). Two studies
focused on regulations in the Canadian provinces of Quebec and Ontario, respectively
(Dufour et al., 1998; Lewchuk et al., 1996).
The Canadian regulations were progressive for their time (the late 1970s), but in terms
of contemporary approaches to integrated health and safety management are more limited
and less system-oriented in their requirements. The Norwegian IC regulations, on the
other hand, incorporated systematic management concepts, which were increasingly being
found in best practice models in business at that time.
3.4.2. Findings on the implementation and eectiveness of the IC legislation in Norway
The IC studies demonstrated most clearly the eects of the legislation on the use of
OHSMSs. There was an increase from 8% to 47% of workplaces fully implementing the
IC requirements over the period 1 year to 7 years post-intervention (Nytro et al., 1998;
Saksvik and Nytro, 1996; Saksvik et al., 2003). Consistent with these ndings, Saksvik
and Nytro (1996) found that, in 1993, a sizeable portion of workplaces credited the legislation with improvements in various aspects of an OHSMS (e.g., 25% attributed clearer
lines of responsibility). Researchers also found an impact on awareness of health, safety
343
and the environment, an intermediate OHS outcome, with 39% of workplaces attributing
an increase to the legislation.
The attempts by Saksvik and Nytro (1996) and Torp et al. (2000) to nd any impact of
the legislation on other intermediate OHS outcomes and on nal OHS outcomes were less
convincing. Saksvik and Nytro (1996) carried out multi-variable regression analyses to
relate the degree of self-reported IC implementation after 1 year to self-reported changes
in absenteeism and injury rates. The model for absenteeism was statistically signicant, but
explained only 5% of the observed variance. The model for injury rates was not statistically signicant. These relatively weak results might have been partly due to the short
observation period and the crude measure of change used. Respondents were asked to
report on whether rates were lower/stable/higher in 1991 and the rst half of 1992, compared with 1990. (The legislation came into eect on January 1, 1992.)
Torp et al. (2000) took a similar analytic approach to survey data gathered 4 years postimplementation. They found that the majority of intermediate variables tested (e.g., satisfaction with HES activities in the garage, perceived physical working environment), had a
statistically signicant relationship with IC status. For the two nal OHS outcomes tested,
musculoskeletal symptoms and sick leave, the relationship was in the expected direction
but weak in both cases (standardized beta coecients of 0.026 and 0.048, respectively),
and statistically signicant in only the rst.
The evidence provided by Saksvik and Nytro (1996) and Torp et al. (2000) on the
eects of IC implementation is weak because of the cross-sectional designs. Whether
the IC implementation preceded the outcomes remains unknown and unmeasured factors
related to both the degree of IC implementation and nal OHS outcomes (e.g.,
management commitment to OHS) are alternative conceivable explanations for the
ndings.
3.4.3. Findings on the eectiveness of simple OHSMS legislation in Canada
Lewchuk et al. (1996) examined the eect of the 1979 introduction of Bill 70 legislation
in Ontario on lost-time injury claim rates. They analyzed claims data from 1976 to 1989 in
six manufacturing and two retail sectors. These two sectors were dierently aected by the
legislation, with the requirements for manufacturing being more intensive. The reviewers
therefore categorized the research design as two distinct time series.
Results from regression analyses of the manufacturing sector data (controlled for
employment, union status, time and sector) showed that Bill 70 had a signicant eect,
equivalent to an 18% decrease in lost-time injury rate. A separate analysis showed that
the eects were progressively larger for 1980, 1981, and 1982+, suggesting a phasing in
of implementation and perhaps a lag in eects. In contrast, in the retail sector, the eect
of the legislation was small and statistically not signicant.
An explanation for the dierences observed between sectors might be a dierence in the
OHSMS requirements, especially the requirement for a joint-health-and-safety committee
in only the manufacturing sector. Conicting with this interpretation, however, is the
observation from separate regression analyses that joint-health-and-safety committees
formed in 1980 or later appeared to have no eect on injury rates and yet Bill 70, introduced in 1979, appeared to have an eect. Thus, other unmeasured, industry-specic contextual factors or co-interventions might have played a role instead. For example, it seems
likely that the retail sector would have received less attention from the Ministry of
Labours inspectorate because of retails lower level of risk.
344
In the Quebec study (Dufour et al., 1998), pooled time series regression analyses based on
the 3 years of annual data across the 19 manufacturing sectors were used to develop six different specications of the authors theoretical equation explaining variation in total productivity growth. All of the specications had good explanatory power (R-squared values
from 0.54 to 0.77) and regression coecients were relatively consistent. The regression coefcient for the variable percentage of companies having prevention programs (the OHSMS
variable) was positive and statistically signicant. The authors interpreted this to mean that
the prevention programs reduced injuries, which in turn enhanced rm productivity.
However, there are plausible alternative explanations for the ndings. Although numerous potential confounding variables were included in the model, there remained the possibility of a common underlying factor (e.g., management competency) that could be
associated at the aggregate level with both (a) the more rapid development of a prevention
program in response to a legislative change and (b) productivity growth. Additionally, the
data on the prevalence of prevention programs in the study relied on the report of the prevention program by the workplace to the public authorities; its accuracy was unknown.
3.5. Evidence synthesis
Sections 3.3 and 3.4 described the ndings of the individual studies of voluntary and
mandatory OHSMSs. This section gives a higher level synthesis of each group of studies
and assesses the body of evidence for OHSMS eectiveness.
The heterogeneity of the studies precluded a quantitative pooling (i.e., meta-analysis),
especially because they are so few in number (van Tulder et al., 2003). The evidence was
therefore synthesized qualitatively. The following discussion focuses separately on voluntary and mandatory OHSMSs and assesses the evidence available for each.
3.5.1. Evidence for the eectiveness of voluntary OHSMS interventions
The seven studies on voluntary OHSMS interventions all showed positive eects (see
Table 2). One study also had some null ndings; there were no negative ndings. The positive eects included:
increased OHSMS implementation over time;
intermediate eects (e.g., better safety climate, increased hazard reporting by employees, more organizational action taken on OHS issues);
decreases in injury rates; and
decreases in disability-related costs (e.g., workers compensation costs, short- and longterm disability costs).
The falls in injury rates and workers compensation costs would be of practical importance to stakeholders (declines of 2434% in injury rates and of 1352% in workers compensation costs).
However, six of the seven studies had moderate methodological limitations. The one
high-quality study (LaMontagne et al., 2004) provided evidence for only implementation.
It is unknown how the implementation eects observed in this particular study would have
aected injury rates and economic costs.
Overall, the evidence provided by these seven studies was sparse. Only four studies had
ndings for any of intermediate OHS, nal OHS, or economic outcomes. Each involved a
Table 2
Summary of the evidence
Authors (year)
Method. quality
Study design
Voluntary interventions
Alsop and LeCouteur (1999)
Bunn et al. (2001)
Edkins (1998)
LaMontagne et al. (2004)
Pearse (2002)
Walker and Tait (2004)
Yassi (1998)
Moderate
Moderate
Moderate
High
Moderate
Moderate
Moderate
Time series
Time series
Non-randomized trial
Randomized trial
Before-after
After-only
Time series with
comparison group
1
1
2
15
16
24
1
Mandatory interventions
Dufour et al. (1998)
Moderate
19b
Lewchuk et al. (1996)

Saksvik and Nytro (1996)
Nytro et al. (1998)
Saksvik et al. (2003)
Torp et al. (2000)
Moderate
Moderate
Moderate
Moderate
Moderate
Time series,
aggregated data
Time series
Cross-sectional
Before-after
Time series
Cross-sectional
636
2092
11822092
11822092
267
Eects of the interventiona

Implementation
Intermediate
Final OHS
Economic
+
+
+
+/0
+
+
+
+
+
+
+, 0
+, 0
+, 0
Number of workplaces
in analysis
+ indicates that the OHSMS had a benecial eect; statistical signicance was not necessarily determined; indicates an eect in the opposite direction,
statistical signicance was not necessarily determined; 0 indicates that a statistical test was conducted and the results were not signicant (n.s.; p > 0.05).
b
For the Dufour study, the number of sectors is shown, since sector was the unit of analysis.
345
346
single enterprise, making the direct applicability of the results to other workplaces
uncertain.
The single enterprise studies must also be regarded cautiously from the point of reporting bias (one favouring positive results). Researchers are known to be reticent to publish
ndings that demonstrate an intervention has no signicant eect. Workplace representatives who champion interventions (and thus have a vested interest in their success) are
likely to have an even greater tendency not to publish such ndings. All four reports on
single workplace interventions appeared to have been authored by workplace champions.
The intervention failure rate seen in this small sample of studies (0%) is markedly dierent
than the rate of 6793% reported for quality management systems (Gardner, 2000). There
is no reason to expect the failure rate of OHSMSs to be markedly dierent than that for
quality, since many of the barriers to implementation can be presumed to be the same
(e.g., management commitment, culture change).
Generalizability is also an issue in the three multiple workplace studies, all of which
focused on implementation. The interventions in both Pearse (2002) and LaMontagne
et al. (2004) were delivered by researchers and showed high refusal rates in the recruitment
phase of 67% (Pearse, 2001) and 59%, respectively. The eects seen in the sample of rms
studied are likely to be a biased estimate of the eectiveness that would be seen if the intervention had been delivered to all eligible rms. Firms willing to participate in the study
would also have been likely to have had a higher degree of management commitment to
OHS, which would also be predictive of successful OHSMS implementation. A similar
consideration is at play for the Walker and Tait (2004) study. They reported on an intervention in the real world, but all of the workplaces in their sample had invited the intervention by seeking help from information centres.
In sum, there is insucient evidence in the published, peer-reviewed literature on the
eectiveness of voluntary OHSMSs to make recommendations either in favour of or
against them.
3.5.2. Evidence for the eectiveness of mandatory OHSMS interventions
The studies on mandatory OHSMS interventions also indicated consistently positive
eects, although there were also null ndings on some of the nal OHS outcomes (see
Table 2). The studies suggest that mandatory interventions result in
increased OHSMS implementation over time;
intermediate eects (e.g., increased HES awareness; improved employee perceptions of
the physical working environment and the psychosocial environment; and increased
workers participation in HES activities);
decreases in lost-time injury rates; and
increases in workplace productivity.
The size of the observed changes in OHSMS development (an increase from 8% to 47%
of all workplaces with full implementation of OHSMS regulations) and the decline in
injury rate (18%) observed are likely to be of practical importance to stakeholders.
On the other hand, all studies in this group had moderate methodological limitations.
These limitations arose largely from the simple study designs employed, especially the
two cross-sectional studies measuring nal OHS outcomes where the direction of causality
was uncertain (Saksvik and Nytro, 1996; Torp et al., 2000). In addition, it was hard
347
to exclude the possibility that there had been confounding or co-intervention in some studies. The most convincing nding was that of Saksvik et al. (2003), demonstrating the
implementation of IC regulations in Norway (an increase from 8% to 47% of workplaces
with full implementation). Unfortunately, there is a lack of credible evidence about how
the Norwegian achievements in compliance aected injury rates and other outcomes.
As well as internal validity weaknesses, two of the six studies were concerned with Quebec and Ontario legislation from the late 1970s and are therefore not applicable to developed countries now.
In sum, there is insucient evidence in the published, peer-reviewed literature on the
eectiveness of mandatory OHSMSs to make recommendations either in favour of or
against them.
4. Discussion
4.1. Identifying and addressing research gaps
The review identied a number of gaps in the research. The most important was the
lack of research whose explicit purpose was to study the eectiveness of voluntary and
mandatory OHSMS interventions on employee health, safety and economic outcomes.
Moreover, the studies were seldom suciently rigorous methodologically to allow for
great condence in the reported ndings. Their limitations also inhibit the ability to apply
the results to other workplaces.
The following were common limitations in the studies:
simple research designs (e.g., lack of comparison group, use of cross-sectional designs),
lack of consideration or control of confounding (through design or statistical
adjustments),
lack of information reported about measurement methods and potential biases in
measurement,
samples which prevented generalization of ndings (i.e., several single workplace
studies).
A similar limitation has been found in the quantity and quality of the evidence on the
closely related topic of safety management system eectiveness (Hale, 2003).
The scarcity of high-quality published research on the implementation/eectiveness of
OHSMSs may relate to the diculties in carrying out such research. Recruitment of workplaces to both intervention and controls is a challenge. Many workplaces are not willing to
make a commitment to a large intervention like an OHSMS, let alone its evaluation.
Among those that are, some might not want to risk being allocated to the comparison
group. Thus, large refusal rates and some withdrawal after allocation are to be expected.
Measurement presents conceptual, logistic and resource challenges because of the complexity of OHSMSs and their environment.
When conducting a controlled trial is not feasible, observational cohort studies are usually considered to be the next best option by those in the epidemiological eld. To answer
questions about OHSMSs using an epidemiological model, one would need to follow a
large sample of workplaces (the cohort) over time, measuring the introduction or upgrading of OHSMSs and then measuring outcomes of interest at the workplace level and
348
possibly at the worker-level too. However, such research designs are very expensive and
complex to implement, and as such, are rarely used. In Canada, such a research design
is employed by the national survey organization Statistics Canada (2005) to examine issues
such as organizational productivity, but thus far OHS is not within its scope.
4.2. Strengths of the review
The volume of studies published each year is more than most practitioners or researchers can easily keep track of or synthesize. This review has clearly eliminated a huge volume
of work in nding the relatively few studies of interest and to summarize their ndings. Its
use of explicit, systematic methods helps ensure that this summary is relatively objective in
its appraisal.
Within the parameters set by the review question and the included sources, the review
team feels condent that the search has been comprehensive and that it is unlikely that
there are other items in the peer-reviewed, published literature that would dramatically
alter the conclusions of the review. The review drew from a broad range of academic disciplines. The eight databases used in the search represent the elds of occupational medicine, occupational safety, risk management, management, occupational psychology, and
sociology.
The reviews extensive search of the current literature conrmed that no other systematic review has considered the eectiveness of OHSMSs. Until now there have only been
high-quality narrative reviews available (see Section 1). The present review therefore
makes a unique contribution to the research literature.
4.3. Limitations of the review
Resource and feasibility constraints limited consideration of the evidence to the published, peer-reviewed literature identied in eight databases. The usual expectation is that
the literature of highest quality is in peer-reviewed journals. Only a preliminary search and
screen of other literature, i.e., that which is not peer-reviewed and published, was
undertaken.
A simple search of thesis dissertations (through the Dissertations Abstract International database) revealed some sources which were eligible (Ford, 1998; Weems, 1998).
However, a cursory review of their quality suggested that they would not alter the conclusions of this review in any substantial way and might not in fact pass the quality assessment screen.
Other types of grey literature were omitted from the review too (e.g., government
reports, conference proceedings, unpublished reports by OHSMS vendors and users). This
research team identied two reports of this type (Hanson et al., 1998; European Agency
for Safety and Health at Work, 2002) during the course of the review. Accessing the grey
literature on OHSMSs in a systematic way would be a great challenge, given the variety of
potential sources and the proprietary nature of some of the knowledge.
Although a large set of search terms related to OHSMSs was used to search the bibliographic databases, it remains possible that it failed to capture some relevant studies. For
example, an intervention described as a climate intervention might not have been
detected if other terms like safety management system, safety program, safety
system, occupational safety standard were not used too (see Section 2.1). Such
349
omissions are not likely to be many. Not only were the results of the database searches
used as a source of potentially relevant citations, but also the following sources: the 26 eligible studies; recent reviews in journals and monographs, including Gallagher et al. (2003),
Frick et al. (2000), Hale (2003) and Kogi (2002); personal les; and reference lists
requested from experts in the eld.
There were a few specic limits placed on the scope of the review. First, only studies
that examined a clearly identiable voluntary or mandatory OHSMS intervention were
included. This excluded an ambitious U.S. study that compared state voluntary initiatives,
including prevention programs, while controlling for several other variables (Smitha et al.,
2001) and another interesting paper on performance indicators (Simpson and Gardner,
2001). In both cases, the potential OHSMS variable was not clearly dened. Also excluded
by this criterion were a few cross-sectional studies investigating the relationship between a
researcher-dened measure of OHSMSs and injury outcomes (e.g., Mearns et al., 2003;
Simard and Marchand, 1994), because there had been no intervention.
The review required the study to involve intervention on at least two OHSMS elements,
since the reviews focus was on systems. This excluded a substantial number of cross-sectional studies of the eectiveness of single OHSMS elements such as joint-health-andsafety committees (e.g., Cohen, 1977; Habeck et al., 1998; LaMontagne et al., 1996; Reilly
et al., 1995; Shannon et al., 1996).
By requiring studies to be quantitative in nature and published as a journal articles,
publications that were qualitative or theoretical in nature or published in a format outside
of a journal article were excluded. The case study of an Esso gas explosion (Hopkins,
2000), based on a Royal Commission on the event, included critical comments about
the OHSMS in use. The Frick et al. (2000) monograph contains further critique. Such
materials were not included in this review, although some of the issues raised were summarized in Section 1.
5. Conclusions
This systematic review found a relatively small quantity of published, peer-reviewed evidence involving OHSMS interventions, despite the fact that reviewers screened 4837 studies drawn from eight databases representing diverse disciplines. A qualitative synthesis of
the available research was used, because of the small number of studies and the heterogeneity of the studies characteristics.
The reviews synthesis of the evidence showed mostly favourable results. There were a
few null ndings, but no ndings of negative eects.
However, all but one of the studies included in the nal synthesis had moderate methodological limitations. The studies were seldom suciently rigorous to allow great condence in the reported ndings. In addition, many of the ndings concerned only OHSMS
implementation, with no accompanying results for outcomes. Finally, the characteristics
of workplace samples (e.g., single workplaces, large refusal rates) also prevent certainty
about the applicability of the ndings to other workplaces.
In conclusion, despite the generally positive results on the eectiveness of OHSMS
interventions in the published, peer-reviewed literature, the evidence is insucient to make
recommendations either in favour of or against particular OHSMSs. This is not to judge
these systems as ineective or undesirable; it is merely to say that it would be incautious to
judge either way in the present state of our research knowledge.
350
Acknowledgements
The Institute for Work and Health (IWH) is an independent, not-for-prot research
organization. It receives core funding from the Ontario Workplace Safety and Insurance
Board. The authors beneted from the assistance of others at IWH: methodological assistance from Jill Hayden, Selahadin Ibrahim and Victoria Pennick; library assistance from
Doreen Day and Dan Shannon; and administrative assistance from Krista Nolan. Several
external scientists gave generously of their time in reviewing an earlier report on this project: Kaj Frick, Clare Gallagher, Wayne Lewchuk, Michael Quinlan, Jos Verbeek, and
Stefan Zimmer. Finally, other external, local stakeholders helped orient LR to the subject
matter from their various viewpoints: Andy King, Ron Lovelock, John MacNamara, Ed
McCloskey, Ken Taylor.
References
Abenhaim, L., Rossignol, M., Valat, J.P., Nordin, M., Avouac, B., Blotman, F., Charlot, J., Dreiser, R.L.,
Legrand, E., Rozenberg, S., Vautravers, P., for the Paris Task Force, 2000. The role of activity in the
therapeutic management of back pain. Report of the International Paris Task Force on Back Pain. Spine 25
(Suppl. 4), 1S33S.
Anonymous, 1993. Best practice case study: Portland aluminium. Journal of Occupational Health & Safety
Australia & New Zealand 9, 375378.
Anonymous, 1994. Best practice case study: Herbert Adams Bakeries. Journal of Occupational Health & Safety
Australia & New Zealand 10, 275278.
Anonymous, 2000. American Journal of Preventive Medicine 18 (4S).
Alsop, P., LeCouteur, M., 1999. Measurable success from implementing an integrated OHS management system at
Manningham City Council. Journal of Occupational Health & Safety Australia & New Zealand 15, 565572.
Bennett, D., 2002. Health and safety management systems: liability or asset? Journal of Public Health Policy 23,
153171.
Bolton, F.N., Kleinsteuber, J.F., 2001. A perspective on the eectiveness of risk assessment by rst-line workers
and supervisors in a safety management system. Human and Ecological Risk Assessment 7, 17771786.
Briss, P.A., Zaza, S., Pappaioanou, M., Fielding, J., Wright-De Ageero, L., Truman, B.I., Hopkins, D.P., Mullen,
P.D., Thompson, R.S., Woolf, S.J., Carande-Kulis, V.G., Anderson, L., Hinman, A.R., McQueen, D.V.,
Teutsch, S.M., Harris, J.R., the Task Force on Community Preventive Services, 2000. Developing an
evidence-based Guide to Community Preventive Services methods. American Journal of Preventive
Medicine 18 (Suppl. 1), 3543.
British Standards Institution, 1996. BS 8800: guide to occupational health and safety management systems.
British Standards Institution, London.
British Standards Institution, 1999. OHSAS 18001: occupational health and safety management systems
specication. British Standards Institution, London.
Bunn III, W.B., Pikelny, D.B., Slavin, T.J., Paralkar, S., 2001. Health, safety, and productivity in a
manufacturing environment. Journal of Occupational and Environmental Medicine 43, 4755.
Chemical Industries Association, 1995. Responsible care management systems for health, safety and
environment. Chemical Industries Association, London.
Cohen, A., 1977. Factors in successful occupational safety programs. Journal of Safety Research 9, 168178.
Cote, P., Cassidy, J.D., Carroll, L., Frank, J.W., Bombardier, C., 2001. A systematic review of the prognosis of
acute whiplash and a new conceptual framework to synthesize the literature. Spine 26, E445E458.
Dalrymple, H., Redinger, C., Dyjack, D., Levine, S., Mansdorf, Z., 1998. Occupational health and safety
management systems: review and analysis of international, national, and regional systems and proposals for a
new international document. International Labour Organization, Geneva.
Dotson, K., 1996. An international safety and health measurement strategy: corporate programs, systems and
results. Journal of Occupational Health & Safety Australia & New Zealand 12, 669678.
Drummond, M., Jeerson, T., 1996. Guidelines for authors and peer reviewers of economic submissions to the
British Medical Journal. British Medical Journal 313, 275283.
351
Dufour, C., Lanoie, P., Patry, M., 1998. Regulation and productivity. Journal of Productivity Analysis 9, 233247.
Edkins, G.D., 1998. The INDICATE safety program: evaluation of a method to proactively improve airline
safety performance. Safety Science 30, 275295.
Eisner, H.S., Leger, J.P., 1988. The international safety rating system in South African mining. Journal of
Occupational Accidents 10, 141160.
European Agency for Safety and Health at Work, 2002. The use of occupational safety and health management
systems in the member states of the European Union: experiences at company level. European Agency for
Safety and Health at Work, Luxembourg.
Everley, M., 1997. Setting the standard. Health and Safety at Work 19, 1012.
Figuera, J., Lopez, L., 2003. Final report: review of the safety management and reporting practices. SAMRAIL
Consortium. Available from: <http://samnet.inrets.fr>.
Ford, A.C., 1998. Statistical analysis of the eectiveness of health and safety programs [dissertation]. University
of Louisville, Kentucky.
Franche, R.-L., Cullen, K., Clarke, J., Irvin, E., Sinclair, S., Frank, J., and the Institute for Work & Health
Workplace-Based RTW Intervention Literature Review Research Team, 2005. Workplace-based return-towork interventions: a systematic review of the quantitative literature. Journal of Occupational Rehabilitation
15, 607631.
Frick, K., Wren, J., 2000. Reviewing occupational health and safety management: multiple roots, diverse
perspectives and ambiguous outcomes. In: Frick, K., Jensen, P.L., Quinlan, M., Wilthagen, T. (Eds.),
Systematic Occupational Health and Safety Management: Perspectives on an International Development.
Pergamon, Amsterdam, pp. 1742.
Frick, K., Jensen, P.L., Quinlan, M., Wilthagen, T. (Eds.), 2000. Systematic Occupational Health and Safety
Management: Perspectives on an International Development. Pergamon, Amsterdam.
Gallagher, C., Underhill, E., Rimmer, M., 2003. Occupational safety and health management systems in
Australia: barriers to success. Policy and Practice in Health and Safety 1, 6781.
Gardner, D., 2000. Barriers to the implementation of management systems: lessons from the past. Quality
Assurance 8, 310.
GRADE Working Group, 2004. Grading quality of evidence and strength of recommendations. British Medical
Journal 328, 14901497.
Grayham, D.A., del Rosario, V.O., 1997. The management of health and safety at work regulations 1992. Journal
of the Royal Society of Health 117, 4751.
Habeck, R.V., Hunt, H., van Tol, B., 1998. Workplace factors associated with preventing and managing work
disability. Rehabilitation Counselling Bulletin 42, 98143.
Hale, A.R., 2003. Safety management in production. Human Factors and Ergonomics in Manufacturing 13, 185201.
Hanson, M.A., Tesh, K.M., Groat, S.K., Donnan, P.T., Ritchie, P.J., Lancaster, R.J., 1998. Evaluation of the
six-pack regulations 1992. Report No. 177/1998. HSE Books, Sudbury, UK.
Health and Safety Executive, 1997. Successful health and safety management: HSG65. Health and Safety
Executive, Sudbury, UK.
Hopkins, A., 2000. Lessons from Longford: the Esso Gas plant explosion. CCH Australia, Sydney.
International Atomic Energy Agency (IAEA), 2005. The management system for facilities and activities: draft
safety requirements, DS338 Draft 10. Available from: <http://www-ns.iaea.org/downloads/standards/drafts/
ds338.pdf>.
International Labour Oce, 2001. Guidelines on Occupational Safety and Health Management Systems.
MEOSH/2001/2(Rev.). International Labour Oce, Geneva.
Jadad, A., 1998. Randomised Controlled Trials: a Users Guide. BMJ Books, London.
Kjellen, U., Boe, K., Hagen, L., 1997. Economic eects of implementing internal control of health, safety and
environment: a retrospective case study of an aluminium plant. Safety Science 27, 99114.
Kogi, K., 2002. Work improvement and occupational safety and health management systems: common features
and research needs. Industrial Health 40, 121133.
Kuhn, M., Doucet, C., Edwards, N., Public Health Research, E.a.D.P.P., Ontario Ministry of Health, P.H.B.,
1999. Eectiveness of coalitions in heart health promotion, tobacco use reduction, and injury prevention: a
systematic review of the literature 19901998. Public Health Research, Education and Development Program,
Dundas (ON).
LaMontagne, A.D., Rudd, R.E., Mangione, T.W., Kelsey, K.T., 1996. Determinants of the provision of ethylene
oxide medical surveillance in Massachusetts hospitals. Journal of Occupational and Environmental Medicine
38, 155168.
352
LaMontagne, A.D., Barbeau, E., Youngstrom, R.A., Lewiton, M., Stoddard, A.M., McLellan, D., Wallace, L.M.,
Sorensen, G., 2004. Assessing and intervening on OSH programmes: eectiveness evaluation of the Wellworks2 intervention in 15 manufacturing worksites. Occupational and Environmental Medicine 61, 651660.
Lanoie, P., 1992. Safety regulation and the risk of workplace accidents in Quebec. Southern Economic Journal
58, 950965.
Lewchuk, W., Robb, A.L., Walters, V., 1996. The eectiveness of Bill 70 and joint health and safety committees
in reducing injuries in the workplace: the case of Ontario. Canadian Public Policy 22, 225243.
Lund, H.L., 2004. Strategies for sustainable business and the handling of workers interests: integrated
management systems and worker participation. Economic and Industrial Democracy 25, 4174.
Mearns, K., Whitaker, S.M., Flin, R., 2003. Safety climate, safety management practice and safety performance
in oshore environments. Safety Science 41, 641680.
Mikkelsen, A., Saksvik, P., 2004. The relationship between systematic OHS management and sick leave. Journal
of Occupational Health & Safety Australia & New Zealand 20, 169179.
Nichols, T., 1990. Industrial safety in Britain and the 1974 Health and Safety at Work Act: the case of
manufacturing. International Journal of the Sociology of Law 18, 317342.
Nichols, T., Tucker, E., 2000. OHS management systems in the UK and Ontario, Canada: a political economy
perspective. In: Frick, K., Jensen, P.L., Quinlan, M., Wilthagen, T. (Eds.), Systematic Occupational Health
and Safety Management: Perspectives on an International Development. Pergamon, Amsterdam, pp. 285
310.
Nielsen, K., 2000. Organization theories implicit in various approaches to OHS management. In: Frick, K.,
Jensen, P.L., Quinlan, M., Wilthagen, T. (Eds.), Systematic Occupational Health and Safety Management:
perspectives on an International Development. Pergamon, Amsterdam, pp. 99124.
Nytro, K., Saksvik, P., Torvatn, H., 1998. Organizational prerequisites for the implementation of systematic
health, environment and safety work in enterprises. Safety Science 30, 297307.
Pearse W., 2001. Implementing OHS management systems in small to medium fabricated metal product
companies. In: Proceedings of the First National Conference on Occupational Health and Safety
Management Systems, Crown Content, Melbourne, pp. 83100.
Pearse, W., 2002. Club zero: implementing OHSMS in small to medium fabricated metal product companies.
Journal of Occupational Health & Safety Australia & New Zealand 18, 347356.
Quinlan, M., Mayhew, C., 2000. Precarious employment, work-re-organisation and the fracturing of OHS
management. In: Frick, K., Jensen, P.L., Quinlan, M., Wilthagen, T. (Eds.), Systematic Occupational Health
and Safety Management: Perspectives on an International Development. Pergamon, Amsterdam, pp. 175
198.
Redinger, C.F., Levine, S.P., 1998. Development and evaluation of the Michigan occupational health and safety
management system assessment instrument: A universal OHSMS performance measurement tool. American
Industrial Hygiene Association Journal 59, 572581.
Reilly, B., Paci, P., Holl, P., 1995. Unions, safety committees and workplace injuries. British Journal of Industrial
Relations 33, 275288.
Robson, L., Clarke, J., Cullen, K., Bielecky, A., Severin, C., Bigelow, P., Irvin, E., Culyer, A., Mahood, Q., 2005.
The eectiveness of occupational health and safety management systems: a systematic review. Institute for
Work & Health, Toronto, Canada.
Saksvik, P.O., Nytro, K., 1996. Implementation of internal control (IC) of health, environment and safety (HES)
in Norwegian enterprises. Safety Science 23, 5361.
Saksvik, P.O., Nytro, K., Torvatn, H., 2003. Systematic occupational health and safety work in Norway: a
decade of implementation. Safety Science 41, 721738.
Saksvik, P.O., Quinlan, M., 2003. Regulating systematic occupational health and safety management: comparing
the Norwegian and Australian experience. Relations Industrielles/Industrial Relations 58, 3359.
SAMRAIL, 2004. Guidelines for the safety management system. SAMRAIL Consortium. Available from:
<http://samnet.inrets.fr>.
Schulpher, M., Fenwick, E., Claxton, K., 2000. Assessing quality in decision analytic cost-eectiveness models: a
suggested framework and an example of application. Pharmacoeconomics 17, 461477.
Shannon, H.S., Walters, V., Lewchuck, W., Richardson, J., Moran, L.A., Haines, T., Verma, D., 1996.
Workplace organizational correlates of lost-time accident rates in manufacturing. American Journal of
Industrial Medicine 29, 258268.
Simard, M., Marchand, A., 1994. The behaviour of rst-line supervisors in accident prevention and eectiveness
in occupational safety. Safety Science 17, 169185.
353
Simpson, I., Gardner, D., 2001. Using OHS positive performance indicators to monitor corporate OHS
strategies. Journal of Occupational Health & Safety Australia & New Zealand 17, 125134.
Slavin, R.E., 1995. Best evidence synthesis: an intelligent alternative to meta-analysis. Journal of Clinical
Epidemiology 48, 918.
Smitha, M.W., Kirk, K.A., Oestenstad, K.R., Brown, K.C., Lee, S.D., 2001. Eect of state workplace safety laws
on occupational injury rates. Journal of Occupational and Environmental Medicine 43, 10011010.
Standards Australia and Standards New Zealand, 1997. AS/NZS 4804: occupational health and safety
management systems general guidelines on principles, systems and supporting techniques. Standards
Australia; Standards New Zealand, Homebush (NSW); Wellington.
Tompa, E., Trevithick, S., McLeod, C., 2004. A systematic review of the prevention incentives of insurance and
regulatory mechanisms for occupational health and safety. Working paper #213. Institute for Work and
Health, Toronto.
Torp, S., Riise, T., Moen, B.E., 2000. Systematic health, environment and safety activities: do they inuence
occupational environment, behaviour and health? Occupational Medicine (Oxford) 50, 326333.
Tortorella, M.J., 1995. The three careers of W. Edwards Deming. Siam News. Available from: <http://
www.deming.org/theman/articles/articles_threecareers01.html>.
van Tulder, M., Furlan, A., Bombardier, C., Bouter, L., Editorial Board of the Cochrane Collaboration Back
Review Group, 2003. Updated method guidelines for systematic reviews in the Cochrane Collaboration Back
Review Group. Spine 28, 12901299.
Walker, D., Tait, R., 2004. Health and safety management in small enterprises: an eective low cost approach.
Safety Science 42, 6983.
Walters, D. (Ed.), 2002. Regulating health and safety management in the European Union: a study of the
dynamics of change. P.I.E.-Peter Lang, Bruxelles.
Weems, L.D., 1998. The Eect of Participation in the Occupational Safety and Health Administrations
Voluntary Protection Programs on Injury/Illness Rates [dissertation]. Louisiana State University, Louisiana.
Wokutch, R.E., VanSandt, C.V., 2000. OHS management in the United States and Japan: the DuPont and the
Toyota models. In: Frick, K., Jensen, P.L., Quinlan, M., Wilthagen, T. (Eds.), Systematic Occupational
Health and Safety Management: Perspectives on an International Development. Pergamon, Amsterdam, pp.
367390.
Yassi, A., 1998. Utilizing data systems to develop and monitor occupational health programs in a large Canadian
hospital. Methods of Information in Medicine 37, 125129.
Zaza, S., Wright-De Aguero, L.K., Briss, P.A., Truman, B.I., Hopkins, D.P., Hennessy, M.H., Sosin, D.M.,
Anderson, L., Carande-Kulis, V.G., Teutsch, S.M., Pappaioanou, M., 2000. Data collection instrument and
procedure for systematic reviews in the Guide to Community Preventive Services. Task Force on Community
Preventive Services. American Journal Preventive Medicine 18, 4474.

10 1016@j Ssci 2006 07 003 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

10 1016@j Ssci 2006 07 003 PDF

Uploaded by

Copyright:

Available Formats

Safety Science 45 (2007) 329353

The eectiveness of occupational health and safety

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

OHSMSs, commonly understood, are distinguishable from traditional OHS programs

management commitment and resources,

1.2. Voluntary and mandatory OHSMS initiatives

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

1.4. Conceptual framework of the review

Workplace or extraworkplace initiative

(e.g., safety climate)

(e.g., injury rates)

Fig. 1. Conceptual framework for the review on OHSMSs.

L.S. Robson et al. / Safety Science 45 (2007) 329353

Intermediate OHS outcomes were considered to be outcomes of secondary interest,

L.S. Robson et al. / Safety Science 45 (2007) 329353

2.3. Quality appraisal (QA)

L.S. Robson et al. / Safety Science 45 (2007) 329353

very low (serious limitations),

L.S. Robson et al. / Safety Science 45 (2007) 329353

bibliographic databases (18),

Organizations own; based

Walker and Tait (2004)

Quebec LSST (1979)

Lewchuk et al. (1996)

Ontario Bill 70 (1979)

All rms in most

Nytro et al. (1998)

Internal Control (1992)

LaMontagne et al. (2004)

L.S. Robson et al. / Safety Science 45 (2007) 329353

Bunn et al. (2001)

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

identication, safety information management, and safety communication, overseen by a

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

Lewchuk et al. (1996)

Eects of the interventiona

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

L.S. Robson et al. / Safety Science 45 (2007) 329353

You might also like