Professional Documents
Culture Documents
Accreditation and ISO Certification: Do They Explain Differences in Quality Management in European Hospitals?
Accreditation and ISO Certification: Do They Explain Differences in Quality Management in European Hospitals?
1093/intqhc/mzq054
Advance Access Publication: 8 October 2010
Abstract
investment, or more robust evidence for their benefit on Most recently, Braithwaite et al. [12] have reported an inde-
structure, process or outcome of hospital care. ‘Considering pendent blinded assessment of 19 organizations representing
the amount of time and money spent on organisational approximately 5% of the Australian acute care health system,
assessment, and the significance of the issue to governments, concluding that, ‘accreditation results predict leadership beha-
it is surprising that there is no research into the cost- viours and cultural characteristics of healthcare organisations
effectiveness of these schemes’ [3]. but not organisational climate or consumer participation’.
Existing published literature shows increasing attention to Finally, Groene et al. [13] found in a study of 38 hospitals
these issues, but still no clear answers. A systematic review from eight countries that had undergone any type of external
by Greenfield and Braithwaite in 2007 [4] of accreditation assessment had a significantly higher compliance with the
research classified possible impacts in 10 categories: pro- World Health Organization Standards for Health Promotion
fessions’ attitudes to accreditation, promoting change, organ- in Hospitals than hospitals without such assessment (63.9%
izational impact, financial impact, quality measures, vs. 47.3%; P ¼ 0.012). In summary, there is evidence from
programme assessment, consumer views or patient satisfac- many countries and settings that organizations and systems
tion, public disclosure, professional development and sur- change in preparation for hospital accreditation and that
veyor issues. Of these, only promoting change and there is some consistency of findings and recommendations
professional development showed consistent positive associ- for improvement within and between countries. Until now
ation with accreditation; three categories (consumer views or there has been no published study to explore differences
patient satisfaction, public disclosure and surveyor issues) did between various modes of external assessment with hospital
not have sufficient studies to draw any conclusion, and the organization and systems in several countries. Compared
446
Accreditation, certification or neither
external assessment as ‘accreditation only’, ‘ISO only’, or In order to assess the effect of the country variable as a
‘neither ISO, nor accreditation’. The remainder of the hospi- possible confounding factor and a potential interaction
tals, which represent various combinations of approaches or between country and the mode of external assessment, we
were ‘in preparation’, were excluded. carried out a two-factorial ANOVA for these three modes of
external assessment.
Data analysis
Dependent variables were established based on hospital com- Results
pliance with 229 criteria, which were assessed and rated
according to a grading scale as follows: exceptional compli- Overall, site visits were carried out in 89 hospitals. Of these,
ance (indicative range .90%), extensive compliance (66 – 71 hospitals that complied with the inclusion criteria (accre-
90%), broad compliance (41 – 65%), minor compliance (15 – dited, certified or neither form of external assessment) were
40%) and negligible compliance (,15%). This assessment included in the analysis. Hospitals in preparation for accredi-
was converted for each criterion into a score from 4 to 0; tation or with combination of external assessment strategies
no weighting was applied. Invalid or null responses were were excluded from further analysis (Table 1).
excluded from calculations. Questions used to verify In Table 2, we present the aggregate and total scores and
responses to the initial questionnaire were excluded from the standard deviations for the six quality dimensions by the type
scoring. The results were clustered under six dimensions in of external assessment. The overall percentage scores for all
line with common chapters of accreditation standards: man- groups indicate that mean compliance is far from ‘exceptional’
agement, patients’ rights, patient safety, clinical organization, irrespective of external assessment. Nevertheless, a pattern
Belgium 1 0 0 1 0
Czech Republic 15 2 4 7 2
France 18 16 0 0 2
Ireland 6 3 0 1 2
Poland 15 0 3 6 6
Spain 29 10 3 11 5
UK 5 3 0 1 1
Total 89 34 10 27 18
a
In preparation, combination of external assessment or combination of external assessment with other quality management models
(excluded from the analysis).
447
Charles Shaw et al.
Management (28 criteria) 1.1 Governing body Mean (SD) 80.7 (16.7) 59.8 (15.7) 48.9 (19.8) 65.6 (23.0)
1.2 Quality Max value 112 112 112 112
management
1.3 Safety
management
1.4 Infection control % Score 72.0 53.4 43.7 58.6
1.5 Medication
management
Patients rights 2.1 Publication Mean (SD) 148.5 (27.8) 153.5 (31.4) 135.0 (28.6) 144.1 (33.1)
(53 criteria) 2.2 Patient records Max value 212 212 212 212
2.3 information
2.4 Consent to % Score 70.0 72.4 63.7 68.0
treatment
2.5 Privacy
2.6 Patient feedback
(accreditation or certification) with no external assessment assessment (accreditation, certification, neither) and the vari-
at all (column 2), differences were statistically significant ation attributable to the country effect.
for all groups and for the overall score—except for the For example, for the dimension ‘management’, external
dimension patients’ rights. The lack of statistical signifi- assessment is significantly associated with the variability in
cance (close to borderline) in this case may be influenced the management score (mean square, 1113; P ¼ 0.002);
by the small sample size. Less statistically significant however, country accounts for an even higher variability in
difference was evident when comparing accreditation and management scores (mean square, 1580; P , 0.001). There
certification (in the last column). Nevertheless, significant appears to be interaction between external assessment and
differences remain for the scores for management, patient country; the effect is not simply additive but country to
safety and clinical practice. some extent overlaps with the mode of external assessment.
In order to assess the effect of the country grouping vari- Similar results for the effect of external assessment and
able as a possible confounding factor we performed a two- country can be observed in the remaining dimensions and
factorial ANOVA. In Table 4, we report on the amount of for the total score. From this analysis, it appears that the
variation in the scores attributable to the mode of external effect of the country on the mean scores is substantial and
448
Accreditation, certification or neither
higher than the contribution of the mode of external assess- drafted into the cross-border directive. No previous studies
ment alone. have compared the relative impact of ISO certification and
hospital accreditation; if, again, this difference is found to be
consistent with a larger sample of hospitals (such as in the
Discussion successor project ‘Deepening our understanding of quality
improvement (DUQuE) [16]) then organizational develop-
These findings suggest that in this group of hospitals, those ment may be more effectively based on the model of health-
that have either ISO certification or accreditation are safer care accreditation rather than ISO certification.
and better than those which have neither—and that accredi- Given that many of the criteria reflect common elements
tation has more impact than ISO certification on hospital of standards used by healthcare accreditation programmes, it
management, patient safety and clinical practice. If this were is perhaps unsurprising that accredited hospitals appear
found to be true of European hospitals in general, this advantaged. Comparable criteria are not explicit in ISO 9001
by-product of the MARQuIS project would offer evidence which is designed for quality management systems in general,
for demanding all EC member states to define, assess and rather than being specific to hospitals. This difference may
improve compliance with published standards, as originally be reduced with the development and application of an
449
Charles Shaw et al.
Table 3 Test for statistical significance for differences in dimension and total scores by the mode of external assessment
Dimension Comparison
.................................................................................................................................................
1. All groups (accreditation vs. 2. Either forms of external 3. Accreditation vs.
certification vs. neither) (P-value) assessment vs. neither (P-value) certification (P-value)
.............................................................................................................................................................................
*P , 0.05.
interpretation document for ISO 9001 in healthcare such as This analysis of data from the MARQuIS study suggests
the CEN/TC 362 project of the Swedish Institute for that in the sample of hospitals the impact of ISO certifica-
Standards. This project set out to develop a health services tion on quality and safety may be less than with hospital
guide for the use of EN ISO 9001:2000 as a European stan- accreditation, but it appears that either system is better than
dard applicable to all EC member states [17]. no system. While this is in line with previous reports of the
In practice, many countries have no national accreditation data [19], caution in the interpretation of the findings is
programme; many hospitals in search of voluntary external required since statistical significance is limited by the small
assessment—whether for internal quality improvement or sample size and the selection of hospitals for external audit
for external marketing and contracting—thus face uncer- (from the upper and lower quartiles of respondents to the
tainty whether to contract accreditation services from initial MARQuIS questionnaire survey). The effect of the
another country, or ISO 9001 certification from registered country variable is clearly pronounced in this international
auditors. Where a national accreditation programme is avail- study and needs to be considered in the interpretation of the
able, hospitals may take a decision based on what they aim results. Nevertheless, despite the many cautions, the results
to achieve from participation, what it would cost and what of this study clearly indicate potentially fruitful lines of
external pressures exist as incentives from regulators, contrac- further enquiry.
tors and patients. ISO certification is well recognized and Given the need to advance the evidence base for external
accessible in many countries. assessment in healthcare, the International Society for
Although they share much in methodology, individual Quality in Healthcare has established a research website,
accreditation programmes in each country have much diver- hosted by Accreditation Canada, to gather published and
sity in the content of standards, assessment procedures and unpublished evidence of the impact of various forms of
thresholds for award. Likewise, although conceptually inter- external assessment [20]. In Europe, these will be further
national, ISO certification relies heavily on consistency explored in the successor project to MARQuIS, DUQuE
between individual registered auditors—especially in the which will examine among other issues the effect of external
interpretation of ISO 9001 in the healthcare setting [18]. pressure on the uptake of quality improvement by hospitals
450
Accreditation, certification or neither
and their impact on patient-level outcomes in a larger sample 11. Thornlow DK, Merwin E. Managing to improve quality: the
of hospitals [21]. relationship between accreditation standards, safety practices,
and patient outcomes. Health Care Manage Rev 2009;34:
262 –72.
Funding 12. Braithwaite J, Greenfield D, Westbrook J et al. Health service
accreditation as a predictor of clinical and organisational per-
This research was funded by the European Commission formance: a blinded, random, stratified study. Qual Saf Health
through its ‘Scientific Support to Policies’ action under the Care 2010;19:14–21.
Sixth Framework Programme for Research for the research 13. Groene O, Alonso J, Klazinga N. Development and validation
project ‘Methods of Assessing Response to Quality of the WHO self-assessment tool for health promotion in hos-
Improvement Strategies (MARQuIS)’ (SP21-CT-2004-513712). pitals: results of a study in 38 hospitals in eight countries.
Health Promot Int. 2010;25:221 –9.
14. Shaw CD, Kutryba B, Crisp H et al. Do European hospitals
References have quality and safety governance systems and structures in
place? Qual Saf Health Care 2009;18;i51 –6. doi:10.1136/
1. Shaw CD. External quality mechanisms for health care: qshc.2008.029306.
summary of the ExPeRT project on visitatie, accreditation,
EFQM and ISO assessment in European Union countries. Int J 15. Suñol R, Garel P, Jacquerye A. Cross-border care and healthcare
Qual Health Car 2000;12:169– 75. quality improvement in Europe: the MARQuIS research project.
Qual Saf Health Care 2009;18:i3–7. doi:10.1136/qshc.2008.
451