See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/235298780

Safety performance evaluation models: A review

Article  in  Disaster Prevention and Management · May 2007

DOI: 10.1108/09653560710739504

CITATIONS READS
19 2,914

3 authors, including:

Kazeem Adebiyi Adekojo Waheed

Ladoke Akintola University of Technology FUNAAB
52 PUBLICATIONS   206 CITATIONS    107 PUBLICATIONS   972 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Multi Criteria Decision Making Analysis View project

Eco-Buildings, Renewable Energy and Energy Management in Buildings View project

All content following this page was uploaded by Kazeem Adebiyi on 18 January 2017.

The user has requested enhancement of the downloaded file.

 The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0965-3562.htm

DPM
16,2 Safety performance evaluation
models: a review
K.A. Adebiyi
178 Department of Mechanical Engineering,
Ladoke Akintola University of Technology, Ogbomoso, Nigeria
O.E Charles-Owaba
Department of Industrial and Production Engineering, University of Ibadan,
Ibadan, Nigeria, and
M.A. Waheed
Department of Mechanical Engineering,
Ladoke Akintola University of Technology, Ogbomoso, Nigeria

Abstract
Purpose – Managing a safety programme and ensuring that change is in accordance with suitable
performance measures requires continuing improvement in the support of analytical power and
empirical information. This paper aims to consider different approaches and modeling efforts on
safety performance evaluation.
Design/methodology/approach – Review and synthesis of literature.
Findings – Ten major safety performance evaluation approaches are identified including expectation
function, risk assessment, statistical quality control, price deflation, engineering economic factor,
system analysis, artificial intelligence, and systems theory. Based on the approaches, quantitative and
qualitative models have been proposed. The quantitative models use measuring indicators such as
frequency, severity, percentages, relative weight and economic gains/loss of safety programme.
However, qualitative models employ hazard analysis and hazard operability.
Research limitations/implications – Several research questions remain to be answered in order
to completely improve and optimize the impact of these provisional safety performance measures.
Originality/value – This study offers a set of interesting lessons for academics, industry and safety
practitioners by providing guidelines that will assist in ensuring a correct focus to select an
appropriate safety performance evaluation model.
Keywords Health and safety, Performance measures
Paper type Conceptual paper

Disaster Prevention and Management
Vol. 16 No. 2, 2007
pp. 178-187
q Emerald Group Publishing Limited
0965-3562
DOI 10.1108/09653560710739504
Introduction
The campaign for safety is a long tradition (both at local and international levels) and
the demand for safety and health improvement was originally prompted by poor
conditions such as in lighting, ventilation and sanitation which arose as a result of
rapid industrial revolution in eighteenth century (Bannet, 1984). However, it took
centuries of experience to perfect discipline of safety which requires the knowledge of
management, ergonomics, psychology, industrial aesthetics, engineering to mention a
few (Harold and Moriarty, 1990). Safety may be regarded as an attribute of only
engineering set up, but, certainly entails more. The current trend in modern
technological societies emphasis is safety conscious attitude on the part of employers
of labour, individual employee, self employed, designers, importers, exporters,
suppliers and landlords to mention a few. Studies demonstrate that between 5 per cent Safety evaluation
and 15 per cent of accidents are caused by inherent job hazards and 85 per cent to 95 models
per cent are caused due to what employers do or fail to do (Encarta, 1999; Darby et al.,
2005). Also, it was reported that there exists immense correlation between safety and
productivity; and cost and suffering (Williams, 1984; Duignan, 2003; Fayad et al., 2003;
Inegbenebor and Olalekan, 2002).
Several approaches have been employed to safety performance evaluation. These 179
are statistical: expectation function, quality control; system analysis, engineering
economic factor, price deflation, systems theory, risk assessment, system dynamics,
etc. Also modeling efforts have been reported in literature. These include accident rate
model (Van Es, 2001); frequency co-efficent and injury co-efficient (Poltev, 1985); safety
programme efficiency index (Charles-Owaba and Adebiyi, 2001); justification model
(Fine, 1978) safety sampling model (Aggrawal, 1990); system-theoretic accident model
and process (STAMP) (Leveson, 2004)
Despite all these efforts, absolute safety for humans and property is still an illusion;
unsafe event persists, natural disasters ravage the earth; earthquakes, drought,
tornadoes and floods still claim lives. Factory machines and oil exploration facilities
are still claiming human lives or limbs. In the USA, about 6,500 American workers die
each year because of accidents (Encarta, 1999). Also, National Safety Council (2004)
reported that on an average day, 14 people are killed and more than 10,400 people are
disabled at work. In the UK, 1.6 million injury accidents and 27 million non-injury
accidents are being recorded annually (Phelps, 1999). In Nigeria, NISP (2000) reported
that over 11,000 people die from on-the-job accidents each year and a worker is injured
every 18 seconds in chemical industries in Nigeria. Buttressing this, Takala (1999)
reported that there is higher incidence of fatal work – related injuries in developing
countries than in established economy (11-23 versus 5.4 per 100,000 workers
respectively). Such injuries produce a major economic and social burden in developed
and developing countries (Fayad et al., 2003).
Considering the economic impacts of these accidents, Department of Labour
Force, (UK) survey estimates that annual costs of accidents to employers is between
£4.5 and £9.5 billion; to individual £5 billion and to the society between £11 and
£16 billion (Phelps, 1999). In USA, the National Safety Council reveals that the
death and disabling injuries cost is upward of $ 130 million annually (Harold and
Moriarty, 1990). However, NSC (2004) revealed that the cost to the USA economy of
the job – related injuries is estimated at more than $27 billion annually. Anibueze
(1992) reported that the costs of accidents to employers in petroleum industry in
Nigeria are upward of $25 million.

Safety performance evaluation

The primary purpose of safety evaluation is to determine the true basic cause of
accident for the express purpose of taking remedial action to prevent a recurrence and
to remedy the weakness in one or more of safety programme activities (Aggrawal,
1990). However, safety analysis is faced with problem of assessing the uncertain future
and quantifying safety output. But, the quote from Peters and Waterman reported by
Cox and Tait (1991) that “What gets measured gets done. Putting a measure on
something is tantamount to getting it done. It focuses management’s attention on that
DPM area. Information is simply made available and people respond to it” thereby puts
16,2 challenges to safety analyst.

Safety performance evaluation approaches

Several attempts have been employed to evaluate the performance of a safety
programme. No generally agreed model has been reported. The problem is in
180 quantification of safety output. The widely reported approaches are as follows.

Statistical approach
The statistical methods allow us to account for inherent variability and uncertainty in
making predictions to losses due to accidents. It provides a means of comparing any
given period with another period. Among the statistical methods being employed are
correlation, regression, analysis of variance and non-parametric contingency tables.
The measurement indicators are frequency co-efficient, severity co-efficient and days’
delay, severity and frequency of accident (Adekoya, 1999; Atambo, 1999; NISP, 2000;
Aggrawal, 1990; Poltev, 1985).

Expectation function
This approach looks into probability nature of accidents. In this approach the expected
number of accidents is compared with actual number of accidents to determine the
number of accidents saving due to safety programme.

Risk assessment approach

This is an acknowledgment that risk is an unavoidable feature of all decisions
concerning the future. Four risk analysis tools are currently popular (Ray, 2003). These
are hazard analysis (HA), failure modes and effect analysis (FMEA), fault tree analysis
(FTA) and probabilistic risk analysis (PRA).
According to Ray (2003) the risk assessment requires:
.
identification of risk;
.
estimation of risk, where identified risk are assessed and their importance,
likelihood, severity and impact are determined;
.
analysis and evaluation of risk, where acceptability of risk is determined and
action that can be taken to make risk more acceptable are evaluated.

Statistical quality control

The technique involves random sampling of work activities, evaluating the activities
against safety standards to detect unsafe condition and plotting the information on a
control chart. When control charts indicate significant deviation from the expected
plant safety level, immediate steps are taken to eliminate the assignable causes.

Price deflation
This compares the annual cost of preventing accidents with average cost of accident.
The problem here is getting an agreed value to injuries and life. An agreed model for
estimating the cost of deaths, injuries and properties damage resulting from accidents
has not been reported.
Questionnaire Safety evaluation
The questionnaire is prepared to get information from different departments of an
establishment especially on how the programme has affected accident occurrence in their
models
area. However, the actual numerical value may not be given by the respondents. Some
people might want undue credibility, hence giving faulty information. In the UK, the
employment department labour force employs this to obtain statistical data on accidents
and the estimated cost of accidents to individuals, employers and society (Phelps, 1999). 181
Engineering economic factor
This finds application in assessing the safety programme to plant and equipment in an
industrial set up. It compares the cost of an investment in safety programme to the cost
of damages that may result from accident. The worth of property damaged could be
evaluated at any point in time. The commonly used economic factor is “Present worth
factor”.

System analysis approach

The system approach demands a through examination of all accidents. It pays
attention to the capabilities and limitation of the working population. The approach
isolates every single operation in a job, examines the hazard of each area and indicates
what should be done. The important factors that are eminent in the analysis approach
are components, system behavior and organization (Cox and Tait, 1991).

Artificial intelligence (AI)

AI is already widely used as a component of safety-related systems, with varying
degrees of maturity (Firth, 1996). In general, usage falls into two categories: as serial
components, where the intelligent element performs an essential function within the
system, and without which the system will not perform; and as parallel components,
where it will act in a supervisory or advisory capacity to enhance an otherwise
independent system. However, its effect is pronounced in parallel system where it can
complement either a determinate system or a human component of the system. Aside
from the use of AI in system components, there has been significant progress in its use
during design and evaluation of safety-related system.

Systems theory
The systems theory is a useful way to analyze accidents, particularly systems
accidents. In this approach, accidents are seen as resulting from inadequate control or
enforcement of constraints on safety related behavior at each level of system
development and system operations control structure.

Safety evaluation modeling efforts

Several safety evaluation models have been reported in literature, both qualitative and
quantitative. Van Es (2001) employed a maximum likelihood estimator to develop
accident rate model to evaluate aircraft accident as:
Na 26
Ra ¼ *10 ð1Þ
Nf
DPM where
16,2 Ra ¼ accident rates
Na ¼ number of reported accidents
Nf ¼ number of flights
182 Poltev (1985) developed models to evaluate the frequency co-efficient and severity
co-efficient of injury accidents in manufacturing industry. The models are given as:
F ¼ ðTx1000Þ=P ð2Þ
where
F ¼ frequency co-efficient.
T ¼ number of injury accidents for a given period.
P ¼ total number of man-hours of exposure.
and
D
S¼ ð3Þ
T
S ¼ severity co-efficient.
D ¼ time loss.
T ¼ number of injury accidents.
Charles-Owaba and Adebiyi (2001) employed expectation function to develop a safety
programme performance model in terms of efficiency index of the programme. The
model is given as:
c:z
g:z ¼ ð4Þ
w:z
where
X
N

ai1 yix Qð yix Þ 2 yiz
i¼1
c¼ ð5Þ
X
N

ai1 yix Qð yix Þ 2 yi1
i¼1

X
M
bk1 I kz
k¼1
w:z ¼ ð6Þ
XM
bk1 I k1
k¼1
Given that: Safety evaluation
Qð yix Þ ¼ expected number of accidents in period x before safety programme for models
type i accident.
Yiz ¼ observed accident of type i during safety programme period z.
Yi1 ¼ number of accident of type i in targeted period.
183
ai1 ¼ unit cost of accident type i.
bk1 ¼ the unit value or cost of resource k.
Ikz ¼ the amount of resource k expended in safety programme in period z.
Ik1 ¼ the amount targeted or budgeted.
N ¼ number of identified types of accidents.
M ¼ number of resource type expended.
Fine (1978) proposed the following justification model:

½C
½E
P
J¼ ð7Þ
ðCFÞðDCÞ

where
C ¼ consequence of accident.
E ¼ exposure of hazard.
P ¼ probability of occurrence.
CF ¼ cost factor.
DC ¼ degree of correction.
Each factor in Fine model is determined by subjectively choosing an appropriate level
of rank.
Aggrawal (1990) used probability theory to develop a safety sampling model to
determine the expected number of safe work activities in a given plant or department
as:

N ¼ 4ð1 2 PÞ=Y 2 P ð8Þ

where:
N ¼ number of observation to be made to achieve a desired accuracy of safety.
P ¼ probability of accident occurrence.
Y ¼ safety accuracy (mostly taken as 10 per cent).
Adebiyi (1998) employed the price deflation with productivity principle to formulate
safety performance evaluation model in term of productivity of safety programme:
DPM X
I
 XM

16,2 hi1 K i 2 F ix  Rjx C j1

i¼1 i¼1
P :x ¼
X
I
 X
M
hi1 K i 2 F i1 Rj1 C j1
i¼1 i¼1
184 where:
P.x ¼ total productivity index in period x relative to the base period.
hi1 ¼ unit cost of an accident i prevented at the base period.
Ki ¼ maximum number of accident class i before the advert of safety
programme.
Fi1 ¼ number of accident class I recorded at the base period.
Fix ¼ accident class i recorded at the current period x.
Rjx ¼ quality of input resources j at the current period x.
Rji ¼ quantity of input resources j at the base period.
Cji ¼ unit cost of input resources j at the base period.
I ¼ number of identified classes of accidents.
M ¼ number of input resources type.
Leveson (2004) developed system-theoretic accident model and processes (STAMP)
based on systems-theory. In the model, systems are viewed as interrelated components
that are kept in a state of dynamic equilibrium by feedback loops of information and
control. The process leading up to an accident can be described in terms of an adaptive
feedback function that fails to maintain safety as performance changes over time to
meet a complex set of goals and values. The basic concepts in STAMP are constraints,
control loops and levels of control.

Discussion
Several safety performance evaluation models have been proposed by researchers
employing different approaches. The models can be classified as qualitative and
quantitative. The quantitative models are based on evaluating the frequency, severity
and economic loss resulting from accidents. The qualitative models are based on
evaluation of the potential system of risks and increasing of severity of the hazard. In
employing statistical method, Adekoya (1999) and Poltev (1985) used measurement
indicators such as frequency and severity of accidents to compare the accidents of a
given period and another, and drew inferences for designing safety system. Also,
Roelen et al. (2000) and Van Es (2001) used accidents rate to quantify the historical
safety performance to assist in the determination of safety target level. Similarly, Ames
(2002) employed percentages and relative weight in analyzing accidents from point of
view of personnel and materials problems putting into consideration the degree of
personnel injuries and amount of damage occurring to materials. The explanation of
additional risk assessment techniques has so far placed emphasis on equipment
design. They do not usually include the organizational and management flaws. The Safety evaluation
problem is that these risk assessment techniques are limited by focusing on a failure models
event and the roles of component failures. However, Rasmussen (1997), has argued that
major accidents are often caused not by a coincidence of independent failures but
instead reflect a systematic migration of organization behavior of boundaries of safe
behavior under pressure toward cost-effectiveness in an aggressive, competitive
environment. Also, Leveson (2004) reported that the risk assessment techniques are 185
firmly rooted in probabilistic analysis of a failure events and they have been
disappointing in application to management and cognitive complex human control
activities. However, the records of case histories of accidents entail more than that. Ives
(2002) reported that the failure events usually arise because of poor management
control and not because of unexpected failure of equipment.
The system analysis is synonymous to risk assessment. It employs the hazard
analysis and hazard operability techniques. Here the emphasis is on the work system
design with focus on identifying the contribution of causal factors to overall system
risk.
In employing expectation function approach, the efficiency index developed by
Charles-Owaba and Adebiyi (2001) measures the returns yielded by the efforts put in
safety programme. This does not address the accident causing actions as well as
prevention activities in safety programme. However Harold and Moriarty (1990) and
Ames (2002) said that there is need to include causes of accidents in safety analysis as
this will quickly point to remedial actions and prevention activities required.
Moreover, the safety-sampling model developed by Aggrawal (1990) determines the
expected number of safe work activities in a given department. Here the financial
implications of safety programme were not addressed to determine the actual gain/loss
on such investment. But, Fayad et al. (2003) reported that the economic burden of work
related injuries is a useful tool to convince workers and employers of the cost-benefit of
work safety and also the effectiveness of preventive interventions.
Adebiyi (1998) employed the price deflation approach with productivity principle to
formulate safety performance evaluation model for safety programme. Although the
model addressed the financial implication however, the dynamic nature of safety
system was not taken in consideration.

Conclusion
This study makes a contribution to the understanding of the mechanisms by which
safety programme performance is being evaluated. The study highlights the various
approaches and modeling efforts to safety performance evaluation content wise, the
study findings are not entirely surprising. However, research done would to provide
framework for identifying requirement safety evaluation approach for specific
situation.
In addition, the study will assist in ensuring a correct focus to select the appropriate
of the safety performance evaluation models to the safety programme management
effort.

References
Adebiyi, K.A. (1998), “Productivity measurement of a safety programme”, an unpublished MSc
thesis, Industrial & Production Engineering Department, University of Ibadan, Ibadan.
DPM Adekoya, A.B. (1999), “Safety evaluation of industrial facilities: a mathematical approach”,
unpublished MSc thesis, Industrial & Production Engineering Department, University of
16,2 Ibadan, Ibadan.
Aggrawal, J. (1990), Production Planning Control and Industrial Management, Cassels
Publishers, Delhi, pp. 937-1005.
Ames, J.S. (2002), “Aircraft accidents: method of analysis”, Report of 17th Safety Congress,
186 Passenger Safety, pp. 760-72.
Anibueze, E.P.C. (1992), “Promotion of safety awareness in the Nigerian petroleum industry”,
Industrial Engineering Digest, Vol. 2 No. 2, pp. 17-24.
Atambo, H. (1999), “Learning from accident is the basis for better safety”, ILO Africa Newsletter,
Vol. 3, available at: www.ilo.org
Bannet, D. (1984), Production System Design, Butterworth and Co. Publishers Ltd, London,
pp. 171-3.
Charles-Owaba, O.E. and Adebiyi, K.A. (2001), “On the performance of FRSC, Oyo State Sector
Command”, Nigerian Journal of Engineering Management, pp. 50-6.
Cox, S.J. and Tait, N.P. (1991), Reliability, Safety and Risk Management. An Integrated Approach,
Butterworth-Heinemann Limited, London.
Darby, T.F., Pickup, L. and Wilson, T.R. (2005), “Safety culture in railway maintenance”, Safety
Science, Vol. 43, pp. 39-60.
Duignan, T. (2003), “Good health and safety is good business”, Engineering Technology,
July/August, pp. 12-13.
Encarta (1999), Encarta Encyclopedias Microsoft Corporation, available at: www.encarta.com
Fayad, R., Nuwayhid, I., Tamin, H., Kassak, K. and Khogali, M. (2003), “Cost of work-related
injuries in insured work place in Lebanon”, Bulletin of World Health Organization, Vol. 81
No. 7, pp. 509-16.
Fine, W.T. (1978), “Mathematical evaluation for controlling hazards”, National Safety Council
News, available at: www.nsc.org
Firth, K. (1996), “Can we justify artificial intelligence in safety related system?”,
The Safety-Critical Systems Newsletter, Vol. 5 No. 3, available at: www.Safety-
criticalsystems.com
Harold, E.R. and Moriarty, B. (1990), System Safety Engineering and Management, John Wiley &
Sons, New York, NY.
Inegbenebor, A.O. and Olalekan, O.A. (2002), “A survey of safety practices in some
manufacturing industries in North-Eastern State of Nigeria”, Nigeria Journal of
Engineering Management, Vol. 3 No. 2, pp. 42-6.
Ives, S. (2002), “Targeting specific accidents in workplace”, Engineering Technology,
July/August, pp. 12-13.
Leveson, N. (2004), “A new accident model for engineering safer systems”, Safety Science, Vol. 42
No. 4, pp. 237-70.
National Safety Council (NSC) (2004), “Estimating the costs of unintentional injuries, available at:
www.nsc.org
Nigerian Institute of Safety Professionals (NISP) (2000), Professional Exam Preparatory Manual,
Nigerian Institute of Safety Professionals, Port Harcourt.
Phelps, G.R. (1999), Safety for Managers, Gower, Aldershot.
Poltev, M.K. (1985), Occupational Health and Safety in Manufacturing Industries, Mir Publishers,
Moscow.
Rasmussen, J. (1997), “Risk management in a dynamic society; a modeling problem”, Safety Safety evaluation
Science, Vol. 27 Nos 2/3, pp. 183-213.
Ray, P.S. (2003), “Fault tree in risk analysis for space programs”, Journal of System Safety, Vol. 39
models
No. 2, pp. 21-3.
Roelen, A.L.C., Pikaar, A.J. and Ovaa, W. (2000), “An analysis of the safety performance of air
cargo operation”, a report submitted to National Lucht-en Ruimtevaart labor
NLR-TP-2000-210, available at: www.nrl.com 187
Takala, J. (1999), “Global estimate of fatal occupational accidents”, Epidemiology, Vol. 10,
pp. 640-6.
Van Es, G.W.H. (2001), “A review of civil aviation accidents: air traffic management related
accidents 1980-1999”, paper presented at 4th International Air Traffic Management R&D
Seminar, New Mexico, 3-7 December 2001, pp. 1-10.
Williams, R.M. (1984), Safety Management Handbook for Plant Engineers, McGraw-Hill Book
Company, New York, NY.

Further reading
Jenkins, D. (2005), “Accident or heat related injury?”, Industrial Hygiene, available at: www.
industrial hygiene.org
Joiner, W. (2005), “Achieving the benefits of behaviour-based safety without pitfalls”, Industrial
Hygiene, available at: www.industrial hygiene.org
Read, V.D. (1999), “Safety and bottom line: poring the financial benefits of your safety
initiatives”, Proactive Accident and Incident Reporting and Investigation conference,
London, Goggle web directory, Mountain View, CA.

Corresponding author
K.A. Adebiyi can be contacted at: engradebiyi@yahoo.com

To purchase reprints of this article please e-mail: reprints@emeraldinsight.com

Or visit our web site for further details: www.emeraldinsight.com/reprints

View publication stats