ARTICLE IN PRESS

Building and Environment 41 (2006) 15841592 www.elsevier.com/locate/buildenv

Developing a model to measure the effectiveness of safety management systems of construction sites
Evelyn Ai Lin Teoa,, Florence Yean Yng Linga
a

Department of Building, School of Design and Environment, National University of Singapore, 4 Architecture Drive, Singapore 117566, Singapore Received 16 February 2005; received in revised form 6 May 2005; accepted 9 June 2005

Abstract In Singapore, the construction industry had implemented safety management system (SMS) and SMS auditing for about 10 years now, but the improvement in safety standard is not signicant. In response to the need to improve the effectiveness of SMS and SMS audit, the aim of the paper is to propose a method to develop and test the tools that auditors may use to assess the effectiveness of a construction rms SMS. The research methodology adopted in this study consists of 15 steps. Surveys were conducted; safety experts were consulted and invited to express their views, either through interviews or workshops. The Analytic Hierarchy Process (AHP) and Factor Analysis were used to assist in identifying the most crucial factors and attributes affecting safety. The model was developed by means of the multi-attribute value model (MAVT) approach. It was also subject to validation via site audits. Using the model, a Construction Safety Index (CSI) can be calculated. It is concluded that the CSI can act as an objective measure of different sites for management and appraisal purposes. r 2005 Elsevier Ltd. All rights reserved.
Keywords: Safety management system; Safety audit; Construction safety index; Singapore; Site accidents

1. Introduction: Arguments underpinning the work Safety management systems (SMS) are introduced into the construction industry as a formal system of managing site safety. Contractors are expected to manage site safety through formal SMS. It is important to evaluate how effective these systems are, so that deciencies can be corrected. This study was carried out in collaboration with the Occupational Safety Department, Ministry of Manpower (MOM) and according to the authority there is no audit protocol to evaluate the effectiveness of a construction sites SMS. The Singapore construction industry had implemented SMS and SMS auditing for about 10 years now, but the improvement in safety standard is not signicant. This is in contrast to the impact on the ship-building and ship-repairing industry, which experienced a steady
Corresponding author. Tel.: +65 6874 1008; fax: +65 6775 5502.

E-mail address: bdgteoal@nus.edu.sg (E. Ai Lin Teo). 0360-1323/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.buildenv.2005.06.005

improvement in terms of the safety performance indicators following the implementation of SMS and SMS audit. The lack of safety improvement may be due to a loose SMS governing structure and a lack of standard protocol for safety auditing. In Singapore, there is no regulation to govern the way in which the SMS policies are drafted by construction rms. More often than not, contractors believed that their SMS are sufcient whilst safety auditors think otherwise [1]. Past studies have discovered that the successful implementation of the SMS on construction sites can help to prevent accidents [25]. Therefore, it is essential to provide a comprehensive checklist of attributes that may affect the safety performance of worksites. There is also no standard protocol on how safety auditing is to be conducted. Each safety auditing rm has its own set of audit methodology, based on broad guidelines laid down in Singapore Code of Practice on Construction Safety Management System (CP79) and

ARTICLE IN PRESS
E. Ai Lin Teo, F. Yean Yng Ling / Building and Environment 41 (2006) 15841592 1585

Section 27(A)(2) of the Factories (Building Operations and Works of Engineering Construction) (Amendment) Regulations 1994. Besides stating that safety audits must include 14 main elements, there are no other guidelines and no standard checklists to audit the safety level or effectiveness of SMS. In practice, checklists used vary from company to company. Many safety audit companies have to undercut fees to win safety auditing contracts. The low fees may have led some companies to cut corners and spend less time on site, thereby reducing the quality of their safety audits to the minimum. To know whether a site has an effective SMS, the government requires the SMS to be audited every six monthly. The problem is that hitherto there is no standardised audit tool that can objectively and consistently assess the effectiveness of a rms SMS. There is also a large discrepancy between the auditing standards of different safety consultants. In response to the need to improve the effectiveness of SMS and SMS audit, the objective of this paper is to propose a method to develop and test the tools that auditors may use to assess the effectiveness of a construction rms SMS. This objective is important because the industry can adopt a standard auditing methodology and develop a benchmark for safety audit purposes. The objective of this research is to devise an audit protocol to evaluate the effectiveness of a construction sites SMS. This is achieved by developing and testing an assessment tool that calculates the Construction Safety Index (CSI) of a site. This is a quantitative score that indicates the effectiveness level of a construction sites SMS. A safety audit is a structured process of collecting independent information on the efciency, effectiveness and reliability of the total SMS and draw up plans for correction action. It is important in determining the effectiveness of SMS [5]. Kunju and Gibb [6] found that the advantage of having an active monitoring system before accidents occur is that the audit measures the success of the system implemented, and thus reinforces positive achievement. A proper measure of the safety performance is also found to be crucial for effective safety management [7]. Cox and Cox [8] found that safety auditing is one of the major elements of an SMS. It provides managers with further information and on compliance with standards. This promotes safe work practices that stemmed from reliable and continuing feedback through the safety level observed. Thus, a safety audit conducted in a timely manner will help to determine the strengths and weaknesses of the current safety program, and any problem areas that might adversely affect the success of the program will be identied. This allows rms to derive the maximum benet from the program consistent with the resources deployed [9].

2. SMS for the construction industry In Singapore, the MOM is the government regulatory body responsible for enforcement of legislations relating to safety. The main safety legislations that construction sites have to comply with are the Factories Act (Chapter 104) and the Factories (Building Operations and Work of Engineering Construction) (BOWEC) Regulations, 1994. The BOWEC Regulations require all construction worksites that have contract values of S$10 million (US$ 1 % S$1.70) or more to implement an SMS based on the Code of Practice on Construction Safety Management System (CP) 79. Worksites with contract values of less than S$10 million do not need to implement an SMS, but these sites are encouraged to do so. The CP79 consists of 14 main safety management elements as follows:

             

safety policy, safe work practices, safety training, group meetings, incident investigation and analysis, in-house safety rules and regulations, safety promotion, evaluation, selection and control of sub-contractors, safety inspections, maintenance regime for all machinery and equipment, hazard analysis, movement control and use of hazardous substances and chemicals, emergency preparedness, occupational health programmes.

Each element provides specic guidelines on how construction rms should organise and manage their sites to ensure safety of their personnel and the public. However, CP 79 is not meant to be a stringent set of rules and regulations because each construction project is different. The project manager has to carefully interpret the clauses in the code of practice and adapt the guidelines to the context of his project. Besides Singapore, SMS is also a requirement in many other countries. From the review of the SMSs used in Hong Kong [3,10], Australia [11], UK [4] and USA [5,12] it was found that Singapores CP 79 is very comprehensive and covers all levels of the construction project and organisation. However, the focus on the higher level issues like management commitment is inadequate as compared to Australia and the UK. The SMS approaches employed by the UK are not as extensive in terms of specic details, but they highlight the need for management participation where their opinions and reviews are given very high importance. Many studies have shown that management commitment

ARTICLE IN PRESS
1586 E. Ai Lin Teo, F. Yean Yng Ling / Building and Environment 41 (2006) 15841592

and involvement is the core element of any SMS [1316]. Without management support, it is difcult to enforce safe practices and inculcate a safety culture into the organisation.

3. Research methodology The research methodology is shown in Fig. 1. The rst step was to review the various SMS standards and guidelines of selected developed countries that are known to have relatively high safety standards. These countries include Hong Kong, Australia, UK and USA. In step 2, relevant attributes were selected and collated based on the literature review and study of various countries SMS. A questionnaire was then designed to survey practitioners perception of the importance of the collated attributes. The developed questionnaires were sent to 420 randomly selected general building contractors who were registered with the Building Construction Authority (BCA) of Singapore (step 3).
1. Compare SMS from different Countries 2. Streamline important factors

4. Review ASPA documents 5. Preliminary interviews with safety consultants 6. Develop preliminary framework

3. Survey questionnaire sent to contractors

7. Analyze survey results

8. Construct model 10. Importance weights of lower level attributes

In step 4, a review was done on selected award winning companies of the Annual Safety Performance Award (ASPA) documents to determine their best practices and SMS used within these organisations. In step 5, preliminary interviews were conducted with three safety auditors to nd out their auditing practices. A preliminary framework was developed (step 6) based on the literature review and the postal survey results (step 7). From the results a model to measure the effectiveness of SMS was invented (step 8). This model is based on multi-attribute value technique (MAVT) [17]. In step 8, all possible attributes to be incorporated into the model were identied through reviewing the CP79, checklists, tools and practices adopted in other countries. In steps 9 and 10, the importance weights for the factors and attributes of the CSI framework were determined. For the rst level factors and second level attributes, the weights were determined through Analytic Hierarchy Process (AHP) (step 9), where 30 industry experts were interviewed. Due to the relatively large amount of time needed to conduct AHP, and the large number of lower level attributes 4500, the lower level attributes weights were determined based on 5-point Likert Scale (step 10), where 1 not important; 3 neutral; and 5 very important/critical. Twelve industry experts were interviewed to obtain the importance weights. A rating method was developed (step11) and veried by nine industry experts. Thereafter the model was tested through three site audits (step 12). Based on the feedback, minor improvements were made (step 14) before the nal version was presented (step 15). The survey results (step 7) had been reported in Teo et al. [18]. This paper focused on step 8 onwards, describing the development of the proposed model and its validation.

9. Importance weights st of 1 level factors and 2nd level criteria

11. Rating method of each attribute

4. Model construction (step 8)

Design rating methods. Data collected through interviews with 5 experts

Determine the weights through AHP. Data collected through interviews with 30 experts

Determine the weights through 5point Likert Scale. Data collected through interviews with 12 experts

Analyze results. Calculate importance weights

Analyze results. Calculate importance weights

Analyze results. Refine rating methods.

12. Validate model

13. Obtain Industry Feedback on the model

From the literature review and survey results [18], the many attributes affecting safety were found. These were structured into an MAVT model. The MAVT approach to solving problems with multiple attributes is to develop a scoring model, where each attribute is assigned a weight to reect its importance, and each construction site is rated on a scale of 01 against all attributes. Thereafter, the weight is multiplied by the rating, and the product is summed for each alternative. The inputs to the model which need to be determined are as follows [17]: (1) (2) (3) (4) list of attributes that need to be evaluated; importance weights of attributes; the construction sites rating for each attribute; and an aggregation rule, to determine the score of each alternative.

14. Fine-tune and improve the model

15. Finalize 3P+I Model

Fig. 1. Research Methodology.

ARTICLE IN PRESS
E. Ai Lin Teo, F. Yean Yng Ling / Building and Environment 41 (2006) 15841592 1587

4.1. List of attributes The attributes that contractors and their construction sites need to achieve in order to ensure high level of site safety were identied through literature review and their relevance tested in an industry wide survey [18]. The signicantly important variables (identied through ttest) were input into the SPSS software and factor analysis was carried out, to ascertain if there is any further relationship among the many proposed safety strategies. Factor analysis is motivated by the fact that measured variables can sometimes be correlated in such a way that their correlation may be reconstructed by a smaller set of parameters, which could represent the underlying structure in a concise and interpretable form. The factor analysis produced four principal components, labelled as Policy Factor, Process Factor, Personnel Factor and Incentive Factor (3P + I). Each factor comprised several attributes. See Fig. 2 for the 3P + I model. The four factors and relevant attributes were organised into a hierarchy tree or value tree, where the goals at the top may be abstract, but lower down on the hierarchy, the goals are measurable, non-conicting, coherent and logical (see Fig. 3). Higher level objectives are usually the decision makers objectives in global

terms. These objectives need to be of the highest order and must collectively represent the decision makers total objectives. Each higher level objective is successively subdivided into twigs which are intermediate level objectives, and nally to lower level objectives. The value tree allows attributes to be presented in an orderly structure that helps in problem evaluation, and elicitation of importance weights for twigs. In this study, the highest level objective in the hierarchy is known as a factor. The four factors are: policy; process; personnel and incentive (see Fig. 3, Level 1). Second level attributes were the signicant attributes derived from the survey questionnaire, t-test and factor analysis. Each second level attribute was further opertionalised to lower level attributes until a measurable lowest level attribute was obtained. The nalised list contained 590 attributes and these make up the CSI checklist. 4.2. Importance weights of attributes There is a need to make a distinction between what are essential and what are desirable attributes in the 3P + I hierarchical framework which as mentioned earlier, contained 590 detailed attributes. This is because different attributes are of different importance with

Policy Factor (Po) 1. Safety related legislation and Codes of Practice 2. Safety Management System (SMS) Structure 3. OHSAS 18001 Certification 4. In-house Safety Rules and Regulations

Incentive Factor (I) 1. Safety incentive programs 2. Disincentive programs

Process Factor (Pr) 1. Hazard identification and analysis 2. Safe work process and safety measures 3. Communication and information transfer 4. Site Conditions and Housekeeping 5. Equipment, tools, plants, and hazard substances 6. Sub-contractor management

Personnel Factor (Pe) 1. Safety culture 2. Training and Competency 3. Work relationship and language barrier 4. Safety committee and safety organisation 5. Safety and health promotion

Fig. 2. 3P+I Model.

ARTICLE IN PRESS
1588 E. Ai Lin Teo, F. Yean Yng Ling / Building and Environment 41 (2006) 15841592

Effectiveness of SMS measured as CSI

01 Policy
Legislations & CP SMS Structure OHSAS 18001 Certification Rules and Regulations Haz identification and analysis Safe wk proc. & measures

02 Process
Comm and Info transfer Site Cond. and Housekeeping Eqm, tools, plants, & Haz subst Sub-contractor Management Safety culture

03 Personnel
Training and Competency Work reln and lang. barrier Safety Comand Safety Orgn Safety and health promotion

04 Incentives
Safety incentive programs Disincentive programs

Level 1

Level 2

Level 3 onwards
Fig. 3. 3P + I hierarchical framework.

respect to site safety. It is therefore necessary to nd out the degree of importance of each attribute by assigning them weights. The weight is important to decision makers because it expresses the importance of each attribute relative to the others. For those attributes being evaluated, a weight indicates what the decision makers are most concerned about in a quantitative way. There are several conventions to follow in assigning weights to attributes [17]. One convention is that the nal weight for each twig on the hierarchy tree is obtained by multiplying through the tree. The next convention is to normalise the weights, that is, to make weights sum to 1 at each level of the tree. This study adopted two methods to obtain the importance weights, using:

 

Saatys [19] AHP for higher level attributes (levels 1 and 2). Likert Scale for lower level attributes (level 3 onwards).

4.2.1. Importance weights for higher level attributes using AHP (step 9) The questionnaire to obtain the rst and second level weights using AHP. The weights of the four factors (Policy, Process, Personnel and Incentives) make up the rst level weights. The second level weights are the 17 sub-factors of the 3P + I model (see Fig. 3). The questionnaire consists of ve sections. They are (1) factors relating to site safety through policy, process, personnel and incentive aspects (level one weights); (2) factors relating to site safety through policy aspect (level two weights); (3) factors relating to site safety through process aspect (level two weights); (4) factors relating to personnel aspect (level two weights); and (5) factors

relating to site safety through incentive aspect (level two weights). Using Saatys [19] AHP technique, respondents were asked to compare each element or sub-factor against one another based on a 9-point scale using pairwise comparison method to indicate their relative importance. The measure of intensity of importance is determined by a scale of 1 as equal importance to 9 as absolute importance. Each element or sub-factor was pitted against one another in order to establish the importance weightage. For example, in the elements section where policy factor was compared against the process factor, a two-way scale of 19 in each direction indicates the relative importance over either the policy factor or the process factor. The selection of a number is done in accordance with the respondents experienced opinion and judgment for all construction projects s/he had been involved in. This is to minimise the possibility of a bias decision based on a particular project. To determine the weights using AHP, 30 experts with extensive experience in site safety were identied. They represent various stake holders in the construction value chain such as contractors, public-sector client, government safety department and safety auditing consultancy rm. All respondents have more than 5 years of working experience in the construction industry. They are considered subject matter experts because they have the necessary knowledge and working experience in handling construction projects. Data were collected through face-to-face interviews using the structured questionnaire. Each interview lasted for approximately 2 h. Respondents were instructed to refer to Fig. 3 showing the four factors (level 1) and the 17 sub-factors as the survey progressed in order to understand what they were comparing. The respondents were further

ARTICLE IN PRESS
E. Ai Lin Teo, F. Yean Yng Ling / Building and Environment 41 (2006) 15841592 1589

reminded that during the comparison of the variables, they had to relate them to the enhancement of SMS on construction worksites. The points were given as genuinely and honestly as possible based on respondents experience and no inuence over any variables were induced. The relative importance ratings from the 30 respondents were input into Expert Choice 2000 software. The programme makes use of the respondents data to crosscompare all variables to determine the weights and inconsistency ratios. Inconsistency ratio is a measure of the percentage of time decision makers are inconsistent in making judgement. The considered acceptable inconsistency ratio is approximately 10% or less but particular circumstance may warrant the acceptance of a higher value [19]. An inconsistency ratio of 100% is however unacceptable because the ratings are as good as random judgements. 14 of the 30 experts had inconsistency ratios above 15%. This was too high and their responses were discarded. Of the remaining 16, 14 Experts had low inconsistency ratios o5% and two had ratios between 10% and 15%. These two respondents (Experts 4 and 5) were given another chance to relook at their ratings and determine if they would like to change their decisions. Caution was taken to ensure that respondents do not change their previous decisions just to full the inconsistency ratio target. Eventually, one respondent did not change his rating (Expert 5) while another (Expert 4) made some adjustments on his own free will. The inconsistency ratio for Expert 5 on the section of policy aspect was 38%. This is considered very high and Expert 5 had chosen to keep this score. Nevertheless, Expert 5s data were included in the analysis of weightage because the higher than usual inconsistency ratio was due to his extreme judgement rather than a clerical error. Thus, Expert 5s ratings were accepted even though the inconsistency ratio was greater than 10%. According to Saaty [19], an accurate judgement is more important than consistently inaccurate judgement. The rst and second level weights were computed by averaging the weights for the 16 remaining respondents. As the 3P+I Model may be licensed, the actual weights are not shown in this paper. Nevertheless, the relative importances of the factors, in ascending order are:

to elicit the importance weights. A questionnaire showing all the lower level attributes was designed. Respondents were asked to rate the extent to which each attribute contributed to the effectiveness of SMS on construction sites on a 5-point scale where 1 not important, 3 neutral, 4 important and 5 very important (critical). To determine the importance of lower level attributes, 17 experts were randomly selected from the following types of organisations: clients (public and private); building contractors (local and foreign); safety auditing and consulting rms; and the MOM which is the safety regulatory body. Among the 17 respondents contacted, 12 expressed interest to take part in the questionnaire survey. They comprised four clients, two safety auditing and consulting rms, ve large building contractors and MOM. Data were collected using the structured questionnaire, through face-to-face interviews. All interviewees were senior management and had many years of experience in the construction industry. From the ratings of the 12 interviewees, mean importance weight for each lower level attribute was calculated. These importance weights were also normalised. For the same reasons as given above, the individual importance weights are not provided in this paper. 4.3. Rating the construction site for each attribute (step 11) The next element of the MAVT model is the rating method that auditors are required to use to rate the different attributes. The rating method was rst designed, and then veried with ve of the industry experts who participated in the AHP described earlier. The principles adopted in designing the rating method were ease of rating and objectivity during assessment. The ideal rating method is one that allows safety auditors to allocate points to the attributes in an objective and straight forward manner. This is to minimise the probability of having two auditors getting vastly different results when evaluating the same construction worksite at the same time. Before the actual design of the rating methods, the ve experts were interviewed on what appropriate rating methods could be adopted. Based on their feedback, the research team designed a set of rating methods, which were then shown to them. The ve experts agreed with the rating methods, which are now described. Four possible rating options were designed:

   

Personnel Factor, Incentive Factor, Process Factor, Policy Factor.

4.2.2. Importance weights for lower level attributes using Likert scale (step 10) Due to the large number of third and lower level attributes, it was not practical to use AHP to determine the weights. As such, the 5-point Likert Scale was used

   

0/1 which means 0 or 1, 01 which means fraction between 0 and 1, 0/1/NA which means 0 or 1 or not applicable, 01/NA which means fraction between 0 and 1 or not applicable.

ARTICLE IN PRESS
1590 E. Ai Lin Teo, F. Yean Yng Ling / Building and Environment 41 (2006) 15841592

4.3.1. 0/1 rating option In the 0/1 rating option, 0 means No or does not comply, and 1 means Yes or compliance. The 0/1 option is straightforward and objective, thus it is the most commonly used rating option. One example is attribute 01.01Is there a list of relevant legislation, standards and codes of practice that is monitored and updated periodically? under Policy Factor. If the list is found, then the rating would be 1. Otherwise, the site is rated 0 for this attribute. 4.3.2. 0 1 rating option The 01 rating option is normally applicable to an attribute that is assessed based on a set of samples. The rating is obtained by dividing the number of samples that complied with a specied standard or condition (NC number complied) by the total number of samples that were evaluated (NE number evaluated), i.e. NC/NE. For example, attribute 05.01(F)(iii)Are the following idling plant/machinery positioned properly to prevent collapse and obstruction under Process Factor. The NC would be the number of idling plant/machinery that are positioned properly and NE would be the number of idling plant/machinery that had been observed on site. If NC is 5 and NE is 10, then the rating would be 0.5. 4.3.3. NA rating option The NA option is to be used only when the attribute is not relevant in the context of that particular construction project. For example, attributes related to a specic type of work (e.g. demolition) will not be applicable to a site that does not have that particular type of work (new works). 4.4. Aggregation rule to calculate the CSI score After rating an attribute, its score is calculated using Eq. (1) below: Score S 1 Weight w1 Rate r1 (1)

where CSI is the total score for ith site, j is the attribute reference, and there are n number of attributes, wj is the weight assigned to j th attribute, rij is the rating given to P the ith site on the j th attribute, and means to sum the weighted scores over all the attributes from the 1st to the nth attribute. The assumption of the additive model is that the attributes are independent. This means that the contribution of an individual attribute to the total score is independent of other attribute values. The rating of one attribute should not be inuenced in any way by the values of the other attributes. To ensure independence, there is a need to minimise the degree of overlap or correlation among the attributes by combining or regrouping correlated attributes.

5. Model validation (step 12) Safety audits were carried out using the newly developed model to ensure that it is an objective tool to assess the effectiveness of a construction sites SMS. The objectives of the safety audits were: (i) to ascertain whether the items to be audited in model were comprehensible to auditors; (ii) to ascertain whether the rating method for each item was objective enough such that different auditors would award similar rating under similar circumstances; (iii) to make further improvements to the 3P + I model. 5.1. Safety audits and auditors Safety audits were conducted on three construction sites: sites 1, 2 and 3 by ve auditors. Each site was managed by a different main contractor. Sites 1 and 3 were managed by local contractors and site 2 was a joint-venture between a foreign contractor and a local contractor. Site 1 was shop-house construction project, site 2 was a library construction project and site 3 was a school construction project. Five safety inspectors from the MOMs Occupational Safety Department (OSD) were selected to participate in the trial as auditors (Auditors A, B, C, D and E). The auditors had at least 3 years of experience in construction and industrial safety. Prior to the commencement of the audits, the auditors were briefed on the purpose of the audits, the principles underpinning the 3P+I Model, and its usage. In order to ensure that the audits were not biased, the auditors were requested to minimise communication during the audits, in particular, communications that were related to safety issues on site. The audits were conducted in December 2003.

where S 1 is the score for Attribute 1, w1 is the relative importance of Attribute 1, r1 is the auditors assessment on Attribute 1 of a specic construction site. The additive method of aggregation was adopted to calculate the CSI score. In this method, the total construction safety score of each site is computed by multiplying the rating of an alternative for an attribute by the importance weight assigned to the attribute and then up summing the products over all the attributes. The value function is given in Eq. (2) below. CSIi
n X j 1

wj rij ,

(2)

ARTICLE IN PRESS
E. Ai Lin Teo, F. Yean Yng Ling / Building and Environment 41 (2006) 15841592 1591

5.2. Results and evaluation of trial audits After the audits were conducted, the ratings given by the ve auditors were input into the 3P+I Model. The CSI were calculated (see Table 1). To determine the objectivity of the rating methods proposed, the standard deviation of the ve auditors CSI scores was calculated for each site. As can be seen, the standard deviations are very low, with a maximum of 0.057 and average of 0.033. The highest standard deviation occurred in the rst audit, which was when the auditors were still getting used to the model. In comparison, the standard deviations of the second and third audits are signicantly lower. The implication of this nding is that the rating methods proposed in the 3P+I Model are objective and will enable effectiveness of the sites SMS to be empirically measured.

(ii) For attributes that are based on CP79, the descriptions were modied to be as closely related to the wordings used in CP79 as possible. (iii) Format of the model was modied to facilitate onsite usage. (iv) A database was created to maintain the linkage between the CSI attributes and the clauses in CP79. This will facilitate the evaluation of a sites compliance with CP79 when the model is being used.

8. Limitation of the model One of the limitations of this study is that the number of experts and respondents involved in this study is not large. In future, more experts will be interviewed to ensure that the results are generalisable. The study had been conducted primarily within the context of Singapore, and the interviews and surveys had focused on Singapores industry practitioners. Thus, the importance weights and attributes are strongly inuenced by the local environment and culture. In the event that the CSI is to be implemented or utilised in other countries, further research would be needed to modify the weights and attributes to suit the conditions and culture in that country. Another limitation is the large number of attributes (590) that must be evaluated on site. To further improve the user-friendliness of the model, a Personal Digital Assistant (PDA) can be used to replace the paper-based model. As compared to the paper-based checklist, the small and lightweight PDA is very easy to use on site. The PDA can also improve the ease of entering the ratings on-site, as the forms of the PDA can be designed to have various pull down menus and sorting functions. Furthermore, after each audit, auditors will only need to synchronise the ratings into the computer and there is no need to physically type in all the ratings to obtain the CSI score. The synchronisation capabilities of PDA will also prevent human errors during input of ratings into the computer.

6. Industry feedback on the model (step 13) After the model was developed and tested, it was presented to the MOM, its Advisory Committee on Occupational Safety and Health for the Construction Industry, and safety consultants and auditors at MOMs Safety Auditing Organisation Forum. Discussion is ongoing with the MOM on how the invention may be used industry wide. To facilitate the application of the model and CSI checklist, a manual has been prepared.

7. Improvements to the Model (step 14) and nalising the model (step 15) Even though the audits had shown that the 3P+I Model is an objective tool, the auditors also provided specic feedback on the comprehensibility of the attributes and usability of the model. The feedback was evaluated and appropriate changes were then made to further improve the usability model. Some of the changes that were made are listed below. (i) Numerous attributes were re-worded to improve their clarity.
Table 1 CSI scores and standard deviations from site audits Auditor A B C D E Standard deviation Site 1 0.590 0.460 0.539 0.583 0.596 0.057 Site 2 0.764 0.730 0.714 0.739 0.747 0.019 Site 3 0.776 0.743 0.720 0.741 0.714 0.024

9. Conclusion This paper described the development and testing of a model to calculate the CSI and its accompanying set of tools that can be used to audit the effectiveness of a construction rms SMS. A comprehensive list of 590 attributes that can be used to assess construction safety has been identied. The method used to determine the rst and second level weights using the AHP procedure was described. The 5-point Likert scale was used to determine the importance weights of lower level attributes. These weights were normalised before being input in the 3P+I model.

ARTICLE IN PRESS
1592 E. Ai Lin Teo, F. Yean Yng Ling / Building and Environment 41 (2006) 15841592

This paper also discussed the rating method and aggregation rule for all the components of the Model. The rating method was based on interviews with ve experts and the nalised rating method is made up of four possible rating options: 0/1, 01, 0/1/NA and 01/ NA. The aggregation rule used for the determination of the CSI is an additive aggregation method, which is based on the summation of weighted ratings. The model was also subject to validation via site audits. The main aim of the audit was to test the objectivity of the rating methods, which is a very important aspect of any audit tool. The trial audits, which were conducted by ve auditors on three different sites, showed that the model is objective because the standard deviation of the CSI ranged from 0.019 to 0.057. Furthermore, the CSI Checklist was also improved to ensure its usability and comprehensibility. The framework to calculate CSI developed and tested in this study is important because it can act as an objective measure of a sites SMS effectiveness for management and appraisal purposes. The model is also important because the construction industry had called for ways to ascertain the effectiveness of SMS and SMS audit. This is because there had been no standardised audit tool that can objectively assess the strength and weaknesses of the SMS consistently. With the model developed and tested in this study, the industry can adopt a standard auditing methodology for safety audit purposes to reduce the discrepancy between the auditing standards of different safety consultants. Ineffective SMS can be identied through low CSI scores. From there, steps can be taken to improve safety on site before fatalities occur. The proposed model is not intended to be the actual safety instrument. The intention of research was to respond to the need to ascertain the effectiveness of SMS and SMS audit. Admittedly, the model developed in this study will not solve all the safety problems on sites. What is more important in order to reduce the number of accidents is to have a proper framework to enhance safety on construction sites. The new framework developed in this study calls for combined effort to deal with safety risks at their source. All stakeholders must assume responsibility for identifying risks and take steps to prevent or mitigate them with safety culture deeply ingrained.

References
[1] Teo HP. Singapore Parliament Report, May 2004. [2] Hinze J, Gambatese JA. Factors that inuence safety performance of specialty contractors. Journal of Construction Engineering and Management 2003;129(2):15964. [3] Tam CM, Fung IWH, Chan APC. Study of attitude changes in people after the implementation of a new safety management system: the supervision plan. Construction Management and Economics 2001;19:393403. [4] Baxendale T, Jones O. Construction design and management safety regulations in practice-progress on implementation. Journal of Project Management 2000;18:3340. [5] Wilson Jr JM, Koehn E, editors. Safety management: problems encountered and recommended solutions. Journal of Construction Engineering and Management 2000;126(1):779. [6] Kunju AR, Gibb AGF. Measuring safety culture with SPMT eld-data. Journal of Construction Research 2003;4(1):2944. [7] Chin KS, Choi TW. Construction in Hong Kong: success factors in ISO9000 Implementation. Journal of Construction Engineering and Management 2003;126(6):599609. [8] Cox S, Cox T. Safety systems and people. London: Reed Educational and Professional publishing; 1996. [9] Lee H. Environmental, health and safety auditing handbook. 2nd ed. New York: McGraw-Hill; 1995. [10] Lingard H, Rowlinson S. Behaviour-based safety management in Hong Kongs construction industry. Journal of Safety Research 1997;28(4):24356. [11] Lingard H. The effect of rst aid training on Australian construction workers occupational health and safety motivation and risk control behaviour. Journal of Safety Research 2002; 33:20930. [12] Farrow S. Investing in safety: an analytical precautionary principle. Journal of Safety Research 2002;33:16574. [13] Mattila M, Hyttinen M, Rantanen E. Effective supervisory behaviour and safety at the building site. Internal Journal of Industrial Ergonomics 1994;13:8593. [14] Petersen KE, Guagnini E, Poucet A. knowledge based tools for reliability and safety analysis. CACHI89, European Symposium on Computer Application in the Chemical Industry, Erlangen, Germany, April 1989. [15] Petersen KE. Improving model quality, In: Cacciabue C, Papazoglou IA. editors, Probabilistic safety assessment and management96, vol. 3, ESREL96-PSAM-3. Crete, Greece, June 1996. Berlin: Springer; 1996, p. 1799803. [16] Levitt RE, Samelson NM. Construction safety management. New York: McGraw-Hill, Inc.; 1987. [17] Keeney RL, Raffa H. Decisions with multiple objectives: preferences and value tradeoffs. New York: Wiley; 1976. [18] Teo EAL, Ling FYY, Chong AFW. Framework for project managers to manage construction safety. International Journal of Project Management, 2005;23:32941. [19] Saaty TL. Fundamentals of decision making and priority theory with the AHP. Pittsburgh: RWS Publications; 1994.