Download as pdf or txt
Download as pdf or txt
You are on page 1of 12

International Journal for Quality in Health Care 2008; Volume 20, Number 3: pp. 172 183 10.

1093/intqhc/mzn005
Advance Access Publication: 28 March 2008

Health sector accreditation research: a


systematic review
DAVID GREENFIELD AND JEFFREY BRAITHWAITE
Centre for Clinical Governance Research in Health, Faculty of Medicine, University of New South Wales, Sydney NSW, Australia

Abstract
Purpose. The purpose of this study was to identify and analyze research into accreditation and accreditation processes.
Data sources. A multi-method, systematic review of the accreditation literature was conducted from March to May 2007. The
search identied articles researching accreditation. Discussion or commentary pieces were excluded.
Study selection. From the initial identication of over 3000 abstracts, 66 studies that met the search criteria by empirically
examining accreditation were selected.
Data extraction and results of data synthesis. The 66 studies were retrieved and analyzed. The results, examining the
impact or effectiveness of accreditation, were classied into 10 categories: professions attitudes to accreditation, promote
change, organizational impact, nancial impact, quality measures, program assessment, consumer views or patient satisfaction,
public disclosure, professional development and surveyor issues.
Results. The analysis reveals a complex picture. In two categories consistent ndings were recorded: promote change and
professional development. Inconsistent ndings were identied in ve categories: professions attitudes to accreditation,

Downloaded from by guest on May 14, 2016


organizational impact, nancial impact, quality measures and program assessment. The remaining three categoriesconsumer
views or patient satisfaction, public disclosure and surveyor issuesdid not have sufcient studies to draw any conclusion.
The search identied a number of national health care accreditation organizations engaged in research activities.
Conclusion. The health care accreditation industry appears to be purposefully moving towards constructing the evidence to
ground our understanding of accreditation.
Keywords: accreditation, health care, systematic literature review, quality and safety

Introduction Many countries are embarking on accreditation programs without


any evidence that they are the best use of resources for improving
quality and no evidence about the effectiveness of different
Accreditation, quality and continuous improvement have systems and ways to implement them. [5]
become an intrinsic part of the discourse and activities of
health services. Internationally, dating from 1970s, health One response to the call for an empirically grounded base is
care accreditation programs and accrediting organizations a large-scale study currently in progress examining the
emerged and developed. There are now many national relationships between accreditation and organizational and
accreditation organizations and an international body, the clinical performance [9]. While the anecdotal literature con-
International Society for Quality in Health Care (ISQua), tains argument about the value and merits of accreditation,
which has enrolled to date members in over 70 countries. the evidence has not been assembled and reviewed. The
While it is fair to say that involvement in accreditation is purpose of this study was to identify and analyze the research
variable, in many parts of the world it now is an accepted literature on accreditation.
and important element in quality improvement activities.
Nevertheless, the evidence base for accreditation is thought
to be incomplete. In the accreditation literature, there have
Methods
been many calls for research into accreditation [1 8].
Search strategy
Commentators make the case that the evidence to support
the claims of accreditation programs is lacking. These com- The search, using a multi-method strategy, was conducted
ments are representative of the concerns expressed: between March and May 2007. A similar search strategy has

Address reprint requests to: David Greenfield, Centre for Clinical Governance Research in Health, Faculty of Medicine,
University of New South Wales, Sydney NSW 2052, Australia. Tel: 612 9385 1474; Fax: 612 9385 4926;
E-mail: d.greenfield@unsw.edu.au

International Journal for Quality in Health Care vol. 20 no. 3


# The Author 2008. Published by Oxford University Press in association with the International Society for Quality in Health Care;
all rights reserved 172
Health sector accreditation research

been used in other pertinent reviews [10 12]. In conducting agencies about accreditation research. The RWG-reported
the search, where available, citations, abstracts and complete research having been completed, in progress or being
references were downloaded into Endnote X.0.2, a database planned by their member agencies. The following agencies
referencing program, for subsequent assessment. are currently conducting one of more studies, but have yet to
report publicly about their research: Irish Health Services
Accreditation Board (IHSAB), the United Kingdom CHKS,
Selection criteria
Australian Council on Healthcare Standards (ACHS),
We included empirical work that systematically examined Australian General Practice Accreditation Limited (AGPAL),
accreditation or the accreditation process. The studies Haute Autorite de sante (HAS), Italian Society for Quality of
selected centered on how accreditation works, what it does, Health Care, JCAHO, Canadian Council on Health Services
the results achieved and accreditation surveyors and their Accreditation (CCHSA) and the Spanish accreditation organ-
processes. The selection was limited to contributions in ization Fundacion Avedis Donabedian (FAD).
English.
Snowballing via other databases
Electronic database search
The third search strategy involved a snowballing technique
The rst search strategy was a comprehensive interrogation of following up key materials (discussion papers, articles or
of three electronic bibliographic databases. The literature was reports) via the internet and search engines. The
examined from the Medline database from 1950, EMBASE Web-of-science, Google Scholar and Scirus internet search
from 1980 and nursing and allied health literature from engines were employed to locate documents. Literature from
CINAHL from 1982. 1966 or since the internets inception was searched. The
The initial search utilized the broad keywords quality, snowballing technique identied an additional seven docu-
quality assurance, quality indicators and quality of health ments. These included organizational reports and documents,
care. These keywords produced a large number and wide range and peer reviewed articles.
of references; the majority of which were not relevant to the

Downloaded from by guest on May 14, 2016


task. For example, a search in Medline with these terms
Final result of the systematic search
returned a result of over 2 900 000 items. After exploration and
testing two key search termsaccreditation and the Joint To summarize, the systematic search utilized three strategiesa
Commission on Accreditation of Healthcare Organizations search of the major health bibliographic databases, consultation
or JCAHOwere identied as relevant. Within each of the with accreditation agencies and the snowballing technique.
databases the search terms were searched separately. When This resulted in 66 documents meeting the search criteria.
combined the search identied 33 935 references.
Within the results obtained, a further narrowing was
undertaken by searching for those references associated with Findings
research, i.e. accreditation and research and JCAHO and
research. This identied 3921 references, which, with the The impact or effectiveness of accreditation programs has
removal of duplicates, left a total of 3029 references. been researched with a variety of foci and to differing
An analysis of abstracts of these references was conducted. degrees. The 66 documents were categorized under 10
This analysis identied 58 substantial studies that involved topics: professions attitudes to accreditation, promote
examining accreditation via one or more research methods. change, organizational impact, nancial impact, quality
The remainder were largely discussion or commentary pieces measures, program assessment, consumer views or patient
about accreditation or clinical research. satisfaction, public disclosure, professional development and
surveyor issues. These topics are summarized below.
Identification of additional materials from
accreditation agencies Professions attitudes to accreditation
The second search strategy comprised consultation with Some studies have examined the health professions attitudes
health sector accreditation agencies to identify additional towards and beliefs about accreditation; key results are pre-
materials. The strategy involved searching their websites and, sented in Table 1. These present contrasting views of pro-
where possible, through discussion with key agency person- fessional attitudes, with both support for and criticism about
nel. The search included 22 national agencies and ISQua. A accreditation programs expressed.
number of the agencies websites were in their native In an assortment of studies, health professionals sup-
language and were either relatively inaccessible or contained ported accreditation programs [13 18] or were in agreement
no articles which met the search criteria. While only one about their respective accreditation standards [19 24]. In
published document was retrieved from this search, it ident- one study, there was a high level of support for a proposed
ied where research into accreditation is underway. The accreditation program [15]. Accreditation programs were sup-
Research Working Group (RWG) of ISQua was rst con- ported for the following reasons: an accreditation program is
vened in October 2005 to collate information from member an effective strategy for assuring quality [14 16, 19],

173
D. Greenfield and J. Braithwaite

Table 1 Key results of professions attitudes to accreditation

Relevant studies Key ndings


(listed by
reference number)
.............................................................................................................................................................................

[13] A small majority of participants (medical technologists) preferred working in an accredited laboratory.
They experienced that accreditation improved the traceability of work and improved the procedures.
A large majority of participants considered that accreditation increased their workload. Two laboratories
did not think accreditation improved the quality of results. Concerns were accreditation increased
paperwork, decreased adaptability and perception that attention directed to processes rather than quality.
[14] Health professionals (incorporating nurse managers, nurses, carers and quality coordinators/managers)
generally supported the accreditation program for aged care facilities. However, health professionals also
expressed concern about a lack of consistency among assessors and the cost of the accreditation program
for aged care facilities.
[15] Stakeholders (incorporating doctors, hospital administrators, governmental ofcials and insurance
representatives) expressed a high level of support for an accreditation program (voluntary, standards
based approach, periodic external assessment and quality assurance measures). Concern was expressed
that the biggest obstacle was how to nance the accreditation program.
[16] Accredited organizations cited positive benets of the accreditation process. Most indicated that they
would reapply for accreditation. Accredited organizations discussed challenges complying with standards
and meeting the information requirements.
[17] A large majority of respondents agreed that the accreditation program had been of signicant benet to
their organization. The benets included improving communication, commitment to best practice,
information available for evaluation activities and quality care activities, improved structure for quality,

Downloaded from by guest on May 14, 2016


greater focus on consumers, supporting planned change and staff management and development.
Conversely, a number of respondents stated that the accreditation program was bureaucratic, created
increased workloads and stress for staff (particularly middle managers), consumed considerable resources
and they questioned the benet to their organization.
[18] Preparations for accreditation provided hospital staff with an opportunity to reect on the operation of
the organization. At the same time, staff experienced the accreditation process as bureaucratic.
[19] Participants (allied health deans) afrmed the role of accreditation as an effective system for measuring
quality in higher education.
[20] Dentistry practitioners and program directors agreed on the importance of most experiences and
activities required by current accreditation standards.
[21] Most laboratories thought accreditation had resulted in better laboratory performance with more
documentation and better health and safety training procedures. A signicant proportion of participants
(managers/clinicians) considered accreditation to be overly bureaucratic, inefcient and expensive.
A concern that accreditation covered the domains of other regulatory bodies was also expressed.
[22] Study demonstrated that general practitioners are extremely supportive of the preparation program for
accreditation, valuing the exible, learner-centered style of the teaching and learning.
[23] Medical fellowship faculty believed that the requirements and criteria were valid for determining quality
of faculty development fellowship programs. The accreditation process was also considered by faculty
staff to be time-consuming and they thought that it could be shortened.
[24] Purchasers had a keen interest in health plan accreditation and relied heavily on accreditation decisions
when choosing which plans to offer their beneciaries. Purchasers also wanted to understand the
strengths and weaknesses of the accreditation process for their own contracting purposes.
[25] Doctors were unaware or sceptical of accreditation. They held concerns about how safety and quality of
care should be measured. Doctors perceived themselves to be accountable within a professional
framework (self/patient/colleagues) not to the organizations in which they worked.
[26] Senior staff s principal concerns of an accreditation program were related to perceptions that the process
is unwieldy and it offers little value for patient care delivery for the resources required. Signicant levels
of negative feedback were received.
[27] The study examined the small hospital accreditation scheme in the United Kingdom. The program
(including visits by independent surveyors) was valued by respondents who were also keen to see it
continue to evolve.
(continued )

174
Health sector accreditation research

Table 1 Continued

Relevant studies Key ndings


(listed by
reference number)
.............................................................................................................................................................................

[28] Respondents (hospital administrators) explained that the largest factor contributing to rural hospital
deterrence to seeking accreditation was cost.
[29] Healthcare professionals (physicians, dentists, pharmacists, and nurses) had been facing many problems
with multidisciplinary process-related issues of an accreditation standard. Surveyors experienced
difculties in conveying the core quality improvement concepts to the professionals.

accreditation results in better organizational performance [13, [24, 27, 30]. Professionals from rural health services have
14, 17, 21] and enables collegial decision-making [17, 18] been asked their reasons for failing to participate in an
and accreditation provides a guide to external stakeholders to accreditation program [16, 28]. The most signicant barriers
how quality and safety is managed within an organization identied were cost [28], difculty in meeting standards and
[16, 18, 24]. However, all studies with one exception, which collecting data [16].
recorded improvements due to accreditation [21], did not
attempt to examine the impact of the programs.
Promote change
Other researches report health professionals critical per-
spectives on accreditation. These studies suggest that The activity of preparing and undergoing accreditation has
health professionals hold concerns about their respective been shown to promote change in health organizations [18,
accreditation programs, including the program is bureau- 31 33]; key results are presented in Table 2. A study in
cratic and time consuming for the organization to use Australia monitored for 2 years a group of 23 hospitals
[13, 14, 16 18, 21, 23, 25, 26], the program is perceived which applied for accreditation and then compared them

Downloaded from by guest on May 14, 2016


to add little value to patient care [25, 26], there are high with those which had not. The accredited hospitals showed
(direct and indirect) costs of the program [21, 26 28], signicant change in six areas, most notably in nursing
there is a perceived lack of consistency among assessors organization and safety [31]. Research in one organization
[14, 17] and there are problems with accreditation stan- revealed changes instigated by accreditation as it provided an
dards [29]. One further study, an Indian study examining opportunity for health professionals to reect on organiz-
stakeholder views about the proposed introduction of an ational practices. The organization then reportedly changed
accreditation program, identied caution about the pro- policy, decision-making behaviors and introduced a continu-
posed program [15]. ous quality program [18]. Similarly, participating in an accred-
The research into the views of health professionals about itation program and a randomized clinical trial led to
accreditation is patchy, with some professions or groups signicant improvements in both the dissemination and the
opinions examined and others unexplored. Two studies quality of clinical guidelines [33]. A review of the develop-
assessed the views of doctors about hospital accreditation ment of several accreditation programs noted their conver-
programs. Doctors considered such programs were not rel- gence and widespread impact on both individual
evant to them. Instead, they viewed their professional organizations and at a system level [32].
forums as more appropriate in addressing issues of practice
standards and quality [18, 23, 25]. Nurse managers were
most positive about and motivated to participate in an Organizational impact
accreditation process due to their organizational responsibil- The organizational impact of accreditation programs remains
ities [14, 18]. Health managers were reported to be positive unclear; key results are presented in Table 2. One study
about accreditation, identifying it as a strategic issue by failed to identify any differences between accredited and non-
which to inuence care processes [16, 18]. Allied health accredited (rehabilitation) programs [34]. Another study
deans also perceived accreditation favorably [19], as did lab- found improved outcomes when a (trauma) health service
oratory managers and medical technologists [13, 21], aged was accredited [35]. A review of accredited hospitals in
care service providers (incorporating nurse managers, France showed no signicant differences in accreditation
nurses, carers and quality coordinators/managers) [14], den- decisions according to their status and size [36]. However, a
tistry practitioners and program directors [20] and stake- trend was identied that larger hospitals received more
holders (including doctors, hospital administrators, numerous and serious recommendations.
governmental ofcials and insurance representatives) in a A study reported enhancements to patient care through
country where the introduction of a program was being three organizational strategies introduced as a result of parti-
considered [15]. Similarly, purchasers of health services cipating in an accreditation program [37]. The strategies were
regarded accreditation positively as it provided them with a a patient communication strategy, an evaluation strategy and
guide to the quality of the services they were examining a quality improvement strategy. A participative management

175
D. Greenfield and J. Braithwaite

Table 2 Key results of promote change and organizational impact

Relevant studies Key ndings


(listed by reference
number)
.............................................................................................................................................................................

[18] Preparations for accreditation provided hospital staff with an opportunity to reect on the operation
of the organization. It enabled the introduction of a continuous quality program, greater consideration
of exit surveys and improved procedure documentation.
[31] Accredited hospitals could be differentiated by signicant changes in six areas: administration and
management, medical staff organization, review systems, organization of nursing services, physical
facility and safety, hospital role denition and planning. Most affected were nursing organization and
physical facilities and safety; least change was found in areas most directly associated with medical
staff.
[32] Different countries have adopted similar accreditation programs. However, they were implementing
and adapting their accreditation programs to meet their specic policy needs.
[33] Hospital combining both a clinical trial and participation in an international accreditation program led
to a signicant improvement of both dissemination and quality of guidelines on perioperative diabetic
care.
[34] Survey of accredited and non-accredited (rehabilitation) programs suggested no signicant differences
in the organization and delivery of cognitive rehabilitation therapy.
[35] Development of a trauma program and commitment to meeting national guidelines through the
accreditation process appeared to be associated with improved outcome after injury.
[36] The study showed wide heterogeneity in the summaries on accreditation and in accreditation agency
decision-making for different size and status hospitals. Also provided initial insight into common

Downloaded from by guest on May 14, 2016


quality defects and priorities for hospitals.
[37] Described a program, developed to meet an accreditation standard, that helped focus a large acute
private hospital on patients.
[38] The manager was the most important entity in achieving a successful accreditation outcome.
Managers, who were perceived as participative, have more years of experience, had written more
self-studies, and whose faculty support the accreditation process, were likely to have more positive
accreditation outcomes.

style and an organizational support for the accreditation larger for small and rural organizations compared with
process have been shown to affect the outcome medium to large and urban locations.
positively [38].
Quality measures
Financial impact
Quality measures incorporate items dened as clinical indi-
The nancial costs of accreditation for organizations are an cators, quality indicators or clinical performance measures.
under-researched area. There are contrasting assessments The relationship between quality measures and accreditation
made in the few studies that have been conducted; key is complex; key results are presented in Table 4. In some
results are presented in Table 3. Three studies judged the work, there does not appear to be a direct relationship
costs to be high for individual organizations and ques- between the two. No relationship is generally found between
tioned whether accreditation was an appropriate use of a specied quality measure and an accreditation outcome
resources [14, 26, 39]. In examining an accreditation [42 45]. One study showed that improved compliance with
program in a developing country, one study found that the accreditation standards had little or no effect on clinical indi-
overall nancial viability of the program and costs for indi- cator performance [46]. A weak relationship between accredi-
vidual organizations was unsustainable [40]. Another study tation and quality measures was identied in one instance
noted the costs incurred in participating in accreditation, [47]. In a similar vein, a relationship did hold for health plan
and argued that these should be viewed as an essential scores when compared with accreditation [42]. However, in
investment [30]. the same study, accreditation and patient-reported measures
A recent work examined the costs for a specic health of quality and satisfaction were found to be unrelated.
service (methadone treatment sites). It concluded that there For some time, accreditation agencies have been develop-
were no signicant nancial differences for organizations of ing, implementing and monitoring quality measures in health
different size and location [41]. However, the study also care organizations. While not always an essential part of their
reported that per-patient accreditation costs were substantially respective accreditation programs, some quality measures

176
Health sector accreditation research

Table 3 Key results of nancial impact

Relevant studies Key ndings


(listed by
reference number)
.............................................................................................................................................................................

[14] Health professionals expressed concern about the cost of the accreditation program for aged care
facilities.
[26] Senior staff s principal concerns of an accreditation program were related to perceptions that the process
was unwieldy and it offered little value for patient care delivery for the resources required.
[30] A study of expenditures in accreditation argued that the costs should be seen as an essential investment
and demonstration of commitment to quality.
[39] Case study of a neuropsychiatric hospital which questioned whether the quality of care was improved by
the accreditation process and the costs constitute an appropriate use of resources.
[40] Serious resource constraints, both nancial and expertise, had undermined the ongoing viability of the
Zambian hospital accreditation program.
[41] Methadone treatment sites faced similar accreditation costs regardless of characteristics such as size and
location. Rural and smaller sites incurred a greater burden from accreditation. There was no signicance
difference in cost for a site regardless of accreditation outcome; nor did previous accreditation affect the
cost.
Accreditation preparation costs were the majority of the total expenditure, with the majority of
preparation in the nal months prior to survey. Preparation for accreditation was seen as labor intensive.

Table 4 Key results of quality measures

Downloaded from by guest on May 14, 2016


Relevant studies (listed Key ndings
by reference number)
.............................................................................................................................................................................

[42] A study to determine the characteristics of accredited plans, their performance on quality
indicators and the impact on enrolment. The results showed accreditation did not ensure
high-quality care. It is positively associated with some measures of quality, but it did not ensure
a minimal level of performance.
[43] During an accreditation survey, experienced surveyors failed to detect an error-prone medication
usage system (shown by an independent audit). This raised questions about the validity of
survey scores as a measure of safety.
[44] There was a potentially serious disjuncture between outcome measures and accreditation
evaluations. Data showed no relationship of substance, and a confusing pattern of minor and
sometimes conicting associations.
[45] No signicant relationships existed between categorical accreditation decisions (JCAHO) and
quality indicators.
[46] Those hospitals participating in an accreditation program improved their compliance with
accreditation standards; non-participating hospitals did not. However, there was no observed
improvement on the quality indicators.
[47] Analysis revealed a weak relationship between accreditation or certication status and the
indicators of quality of care (the characteristics examined were average cost per patient, per diem
bed cost, total staff hours per patient, clinical staff hours per patient, percent of staff hours
provided by medical staff bed turnover, and percent of beds occupied). Accredited or certied
hospitals were more likely to have higher values on specic indicators than hospitals without
accreditation.
[48] Clinical indicators were reported to have become an accepted part of hospital quality
improvement activities. The inclusion of clinicians in indicator development, along with regular
feedback had resulted in their extensive use, and many actions to improve patient care.
[49] An accrediting agency provided feedback to organizations on their quality indicator data.
A signicant majority of organizations responded to this feedback to improve both the
processes and outcomes of patient care.
(continued )

177
D. Greenfield and J. Braithwaite

Table 4 Continued

Relevant studies (listed Key ndings


by reference number)
.............................................................................................................................................................................

[50] A self-evaluation project allowed health services to monitor their own activities in order to asses
critical factors related to patient care. The project had the potential to be the starting point to
improve the quality of services and to compare national and international quality assurance
results.
[51] Quality indicator performance measures encouraged organizations to seek improvements in
patient care (data showed improvement in 15 of 16 inpatient measures).
[52] Including discharge instructions among other evidence-based heart failure core measures
appeared to reduce re-admission or mortality.
[53] Feedback to hospitals on their clinical indicator performance produced consistent improvement
over the study period. Hospitals initially with low levels of performance had greater
improvements than those with higher performance.
[54] A study examining the value of self-reporting standardized performance indicators for hospitals.
Symmetry of disagreement among original abstractors and re-abstractors identied eight
indicators whose differences in calculated rates were statistically signicant.
[55] A study comparing organizations with and without quality improvement activities: the effects of
self-reported quality improvement activities were often small and inconsistent, and in some
instances contrary to expectations.
[56] It was faulty to assume that clinical indicators derived from different measurement systems will
give the same rank order.
[57] Hospitals that participate in a quality improvement program were no more likely to show

Downloaded from by guest on May 14, 2016


improvement on quality indicators than were hospitals that did not participate.
[58] Medication errors were found to be common in a stratied random sample of organizations. A
signicant number (7%) of potentially harmful errors were identied. Accreditation of a facility
was not associated with a lower error rate.
[59] The association between quality of care and survival for acute myocardial infarction was
examined for accredited and non-accredited hospitals. Non-accredited hospitals displayed lower
quality than accredited hospitals. However, there was considerable variation in performance
among accredited hospitals.

have been shown to improve care outcomes in health organ- Program assessment
izations [48 54]. Similarly, participation in an accreditation
Accreditation programs have been assessed and researched
program and a randomized clinical trial promoted improve-
for their validity [17, 21, 23, 27, 43, 60 64]; key results are
ment in a quality measure, in this instance a clinical guideline
presented in Table 5. The ndings from this research are
[33]. However, another study found that the effects of
inconsistent. Accreditation programs in six studies were
quality improvement activities were often small and inconsist-
deemed to be credible [17, 21, 23, 27, 61, 62]. The validity
ent [55]. An important argument has been made that differ-
of accreditation programs was questioned in other cases and
ent quality measures, developed and implemented in
authors have argued for the need for improvements in or
different ways, should not be expected to promote similar
clarication of standards [21, 60, 63, 64]. In one instance,
outcomes [56].
the validity of an accreditation program as a measure of
Conicting ndings hold in comparing accredited and
patient safety was questioned, based on its failure to identify
non-accredited hospital quality indicator performance.
an error-prone medication usage system [43].
Quality indicator results from hospitals that voluntarily par-
Two descriptive studies examined the development or
ticipate with quality improvement organizations could not be
implementation of accreditation programs in developing
differentiated from those hospitals that do not participate
countries [40, 65]. In one, the milestones achieved and chal-
[57]. Similarly, no difference could be found between accre-
lenges facing the program were detailed [40]. In the other,
dited hospitals, non-accredited hospitals and nursing homes
an overview of the program and its development was
for medication errors [58]. However, another study revealed
presented [65].
that accredited hospitals performed better on a range of
An argument has been made in favor of specialized organ-
quality indicators than did non-accredited hospitals, albeit
izations that perform accreditation and establish standards
there was considerable variation of performance within the
for healthcare delivery, at least in the United States [66].
accredited hospitals [59].

178
Health sector accreditation research

Table 5 Key results of program assessment

Relevant studies Key ndings


(listed by
reference number)
.............................................................................................................................................................................

[17] A large majority of respondents agreed that the accreditation program had been of signicant benet
to their organization. The benets included improving communication, commitment to best practice,
information available for evaluation activities and quality care activities, improved structure for quality,
greater focus on consumers, supporting planned change and staff management and development.
[21] Most laboratories thought accreditation had resulted in better laboratory performance with more
documentation and better health and safety training procedures. However, a signicant proportion of
laboratories considered accreditation to be overly bureaucratic, inefcient and expensive. A concern
that accreditation covered the domains of other regulatory bodies was also expressed.
[23] Medical fellowship faculty believed that the requirements and criteria were valid for determining
quality of faculty development fellowship programs. However, the accreditation process was
considered by faculty staff to be time-consuming and they thought that it could be shortened.
[27] The study examined the small hospital accreditation scheme in the United Kingdom. The program
(including visits by independent surveyors) was valued by respondents who were also keen to see it
continue to evolve.
[40] Serious resource constraints, both nancial and expertise, had undermined the ongoing viability of the
Zambian hospital accreditation program.
[43] During an accreditation survey experienced surveyors failed to detect an error-prone medication usage
system (shown by an independent audit). This raised questions about the validity of survey scores as a
measure of safety.

Downloaded from by guest on May 14, 2016


[60] The study developed eight models by which to classify the process of evaluation criteria applied to the
nursing services adopted by accreditation programs.
[61] The study developed a framework and assessment tool which offers patients and other stakeholders a
credible measure of quality and safety at the practice level. The tool was developed through a process
bridging quality control and quality improvement.
[62] A protocol to evaluate the capacity of special care units to provide quality care was evaluated. The
standards used, their intent and the survey process, were considered sound by those who tested the
protocol and by those who were evaluated by it.
[63] The current design of a radiology accreditation phantom program was unsatisfactory for assessing
image quality in digital mammography.
[64] In examining the efcacy of accreditation standards applied to medical specialists the study concluded
that there is an imbalance between the setting of standards and their implementation.
[65] The accreditation program for hospitals in South Africa is deemed to be benecial.
[66] The uniqueness of each accrediting organization and their program and standards was noted.
Different organizations were better suited to accredit specic areas of health.

While leaving open the question as to whether such accredit- relationships [42, 67, 68]; key results are presented in
ing organizations are ensuring high-quality healthcare, the Table 6. An examination of the relationship between
authors express support for the range of specialized accredit- not-for-prot hospital accreditation scores and patient satis-
ing organizations and programs. The difculties experienced faction ratings found no association, either summatively or
by an accrediting organization in the United Kingdom have formatively [67]. Similarly, patient-reported measures of
also been examined. The study suggested that there was an quality and satisfaction between accredited and non-
imbalance between setting and implementing the standards accredited health plans could not be differentiated [42].
of the accreditation program [64]. Patients and health professionals views about compliance
with accreditation standards have been compared. While differ-
ing in specic details, the satisfaction rank-order correlations
Consumer views or patient satisfaction for the two groups were similar [69]. A survey of patients
Although the relationships between consumer views or during the accreditation of general practices showed patients
patient satisfaction and accreditation remain largely unex- scored practice issues (access, availability and information avail-
plored, the limited work that has taken place found no ability) lower than doctors interpersonal skills [68].

179
D. Greenfield and J. Braithwaite

Table 6 Key results of consumer views or patient satisfaction, public disclosure, professional development and surveyor
issues

Relevant studies Key ndings


(listed by
reference number)
.............................................................................................................................................................................

[42] A study to determine the characteristics of accredited plans, their performance on quality indicators
and the impact on enrolment. The results showed accreditation did not ensure high quality care. It is
positively associated with some measures of quality, but it did not ensure a minimal level of
performance.
[67] No relationships were identied between hospital accreditation scores and patient-satisfaction ratings,
suggesting a dissociation between them.
[68] As part of an accreditation program for general practices patient views were examined. Patients
considered that doctors need to improve interpersonal skills, access, availability and patient
information.
[70] The study showed a positive association between accreditation scores and public disclosure of
accreditation reports of hospitals.
[71] Students who attended an accredited paramedic program were more likely to achieve a passing score
on a national paramedic credentialing examination.
[72] There were signicant positive relationships between accreditation afliation of nursing programs and
membership in professional nursing organizations and between accreditation afliation of nursing
programs and major contributing factors used to select accrediting agencies.
[73] A re-accreditation program was having benecial effects (increasing the amount and type of activities)
on the continuing medical education activities of many participating general practitioners.

Downloaded from by guest on May 14, 2016


[74] Graduates from accredited nursing education programs did not perform better than those from
non-accredited programs.
[27] The study examined the small hospital accreditation scheme in the United Kingdom. The program
(including visits by independent surveyors) was valued by respondents who were also keen to see it
continue to evolve.
[29] Healthcare professionals (physicians, dentists, pharmacists, and nurses) had been facing many
problems with multidisciplinary process-related issues of an accreditation standard. Surveyors
experienced difculties in conveying the core quality improvement concepts to the professionals.
[75] A comparative study of how different accreditation agencies managed surveyors. Surveyors around the
world share many common features in terms of careers, training, work history and expectations.

Public disclosure Professional development


A study examined the impact of accreditation outcome and A link between accreditation programs and the development
public disclosure [70]; key results are presented in Table 6. of health professionals has been established; key results are
The work, conducted in Japan, yielded several ndings: presented in Table 6. The association has been shown to be
accreditation scores were positively related to public disclos- positive [71 73]. However, dissenting ndings have been
ure of hospital accreditation reports, public hospitals were noted [74].
signicantly more likely to publicly disclose than private hos- A study revealed that health professionals who received
pitals, larger hospitals were signicantly more likely to partici- training in an accredited education program were more likely
pate in public disclosure than smaller hospitals, hospitals in to pass a professional credentialing exam than their col-
rural areas were more likely to disclose than those in urban leagues in a non-accredited program [71]. Extending this
settings and disclosing hospitals scored higher than non- nding, an accreditation program had a small but benecial
disclosing hospitals on measures of patient focused care and impact on the ongoing professional education of health
efforts to meet community needs. Public disclosure was con- (medical) professionals [73]. Accreditation afliation of a
sidered by a majority of respondents as good for consumers health education program has been shown to have a positive
and hospitals; however, concern was expressed about the inuence on individuals seeking professional organization
public reaction to lower accreditation scores. A signicant membership [72].
number of hospitals who disclosed their accreditation reports The accreditation of a health program appears unrelated
perceived that disclosure provides incentives for improve- to professional performance [74]. No distinction between the
ment and increases the credibility of hospitals with their conduct of health professionals who trained or did not train
community. in an accredited program could be distinguished in the rst

180
Health sector accreditation research

year. Counter-intuitively, in the second year, the performance research program into accreditation. For this research the
of those who trained in a non-accredited program was Centre has industry partners the Australian Council on
assessed to be better than their colleagues from accredited Healthcare Standards and Ramsay Healthcare.
programs.

Surveyor issues
Funding
Research into accreditation surveyors is limited; key results
are presented in Table 6. One study examined the skills and This research was supported under Australian Research
qualities of surveyors and the challenges they faced when Councils Linkage Projects funding scheme ( project number
undertaking accreditation surveys [27]. A comparative study LP0560737).
examined the similarities and differences of surveyors across
six accreditation programs [75]. A more recent Thai study,
which also considered the opinions of health professionals,
demonstrated that both groups shared a similar prioritizing References
of concerns. The surveyors focused more attention on
care-related items than did the health professionals, who 1. Baker S, Dunn D. Accreditation: the hallmark of educational
were focused on multidisciplinary process-related problems quality. Radiol Technol 2006;78:123 30.
associated with the program [29]. 2. Fernandopulle R, Ferris T, Epstein A et al. A research agenda
for bridging the quality chasm. Health Aff 2003;22:178 90.

Discussion and conclusion 3. Mays G. Can Accreditation Work in Public Health? Lessons
from Other Service Industries. University of Arkansas for Medical
Sciences, 2004.
Although we searched in a multi-method, comprehensive
manner, electronic research indexing is generally problematic 4. vretveit J. Which Interventions Are Effective for Improving Patient
Safety? A Review of Research Evidence. Stockholm: Karolinska

Downloaded from by guest on May 14, 2016


and we may have missed some key literature. Searching rel-
evant accreditation bodies websites and personally contacting Institutet, Medical Management Centre, 2005.
agencies to gather data yielded additional references and 5. vretveit J, Gustafson D. Using research to inform quality pro-
information of value. grammes. BMJ 2003;326:759 61.
The necessity for an empirically grounded, comprehensive 6. Pomey M-P, Francois P, Contandriopoulos A-P et al. Paradoxes
evidence base for accreditation has long been recognized. of French accreditation. Qual Saf Health Care 2005;14:51 5.
Without this, the varying positive and negative views about
accreditation will remain anecdotal, inuenced by ideology or 7. Shaw C. External assessment of health care. BMJ
preferences, and driven by such biases. 2001;322:851 4.
This review of health care accreditation research literature 8. Shaw C. Evaluating accreditation. Int J Qual Health Care
reveals a complex picture. There are mixed views and incon- 2003;15:455 6.
sistent ndings. Only in two categories were consistent nd- 9. Braithwaite J, Westbrook J, Pawsey M et al. A prospective,
ings recorded: promote change and professional multi-method, multi-disciplinary, multi-level, collaborative,
development. Inconsistent ndings were identied in ve cat- social-organisational design for researching health sector accred-
egories: professions attitudes to accreditation, organizational itation. BMC Health Serv Res 2006;6:113.
impact, nancial impact, quality measures and program
10. Travaglia J, Braithwaite J. Engagement of Medical Practitioners in
assessment. In the remaining three categoriesconsumer Health Services Accreditation: Literature Review and Selected Citations.
views or patient satisfaction, public disclosure and surveyor Sydney: Centre for Clinical Governance Research, UNSW;
issueswe did not nd sufcient studies to draw 2007.
conclusions.
A positive note to emerge from this research has been the 11. Bak P, Bocker B, Muller WD et al. Certication and accredita-
tion systems as an instrument of quality management in the
identication of a number of national health care accredita- rehabilitation ( part 1): identication of most widely used
tion organizations and researchers presently engaged in systems. Phys Rehab Kur Med 2004;14:243 8.
empirical activities. They seem to be engaged in purposeful
work leading towards constructing an extensive evidence 12. Bak P, Bocker B, Muller WD et al. Certication and accredita-
base. tion systems as an instrument of quality management in the
rehabilitation ( part 2): characteristics of most widely used
systems. Phys Rehab Kur Med 2004;14:283 90.
Acknowledgements 13. Verstraete A, van Boeckel E, Thys M et al. Attitude of labora-
tory personnel towards accreditation. Int J Qual Health Care
The research was conducted by researchers in the Centre for 1998;11:27 30.
Clinical Governance Research at University of New South 14. Grenade L, Boldy D. The accreditation experience: views of
Wales, Sydney, Australia, and forms part of the Centres residential aged care providers. Geriaction 2002;20:5 9.

181
D. Greenfield and J. Braithwaite

15. Nandraj S, Khot A, Menon S et al. A stakeholder approach 32. Scrivens E, Klein R, Steiner A. Accreditation: what can we
towards hospital accreditation in India. Health Policy Plan learn from the Anglophone model? Health Policy
2001;16:7079. 1995;34:193 204.
16. Casey M, Klingner J. HMOs serving rural areas: experiences 33. Juul AB, Gluud C, Wetterslev J et al. The effects of a random-
with HMO accreditation and HEDIS reporting. Manag Care Q ised multi-centre trial and international accreditation on avail-
2000;8:4859. ability and quality of clinical guidelines. Int J Qual Health Care
Inc Leadersh Health Serv 2005;18:321 8.
17. Kreig T. An Evaluation of the ACHS Accreditation Program: Its
Effects on the Achievement of Best Practice. Sydney: University of 34. Mazmanian PE, Kreutzer JS, Devany CW et al. A survey of
Technology, 1996. accredited and other rehabilitation facilities: education, training
and cognitive rehabilitation in brain-injury programmes. Brain
18. Pomey MP, Contandriopoulos AP, Francois P et al.
Inj 1993;7:319 31.
Accreditation: a tool for organizational change in hospitals? Int
J Qual Health Care Inc Leadersh Health Serv 2004;17:113 24. 35. Simons R, Kasic S, Kirkpatrick A et al. Relative importance of
designation and accreditation of trauma centers during evol-
19. Baker SS, Morrone AS, Gable KE. Allied health deans and
ution of a regional trauma system. J Trauma 2002;52:827 34.
program directors perspectives of specialized accreditation
effectiveness and reform. J Allied Health 2004;33:247 54. 36. Daucourt V, Michel P. Results of the rst 100 accreditation pro-
cedures in France. Int J Qual Health Care 2003;15:463 71.
20. Casamassimo PS, Wilson S. Opinions of practitioners and
program directors concerning accreditation standards for post- 37. Sheahan M. Customer focus: patient, organisation and EQuIP
doctoral pediatric dentistry training programs. Pediatr Dent in collaboration. J Qual Clin Pract 1999;19:139 44.
1999;21:3548.
38. Peterson CA. Management, faculty, and accreditation outcomes:
21. Gough L, Reynolds T. Is clinical pathology accreditation worth a survey of physical therapy faculty and program directors.
it? A survey of CPA-accredited laboratories. Clin Perform Qual J Phys Ther Edu 2003;17:2231.
Health Care 2000;8:195 201.
39. Rockwell D, Pelletier L, Donnelly W. The cost of accreditation:
22. Macfarlane F, Tavabie A, Desombre T. Accredited professional one hospitals experience. Hosp Community Psychiatry
development: a qualitative study of the feasibility, acceptability 1993;44:151 5.

Downloaded from by guest on May 14, 2016


and practicality of a new scheme for CPD. Edu Prim Care
40. Bukonda N, Tavrow P, Abdallah H et al. Implementing a
2003;14:3029.
national hospital accreditation program: the Zambian experi-
23. Reznich CB, Mavis BE. Pilot test of family medicine faculty ence. Int J Qual Health Care 2003;14:7 16.
development fellowship accreditation guidelines. Fam Med
41. Zarkin G, Dunlap L, Homsi G. The costs of pursuing accredi-
2000;32:709 19.
tation for methadone treatment sites: results from a national
24. Scanlon D, Hendrix T. Health plan accreditation: NCQA, study. Eval Rev 2006;30:119 38.
JCAHO, or both? Manag Care Q 1998;6:5261.
42. Dean Beaulieu N, Epstein AM. National Committee on Quality
25. Stoelwinder J. A Study of Doctors Views on How Hospital Assurance health-plan accreditation: predictors, correlates of
Accreditation Can Assist Them Provide Quality and Safe Care to performance, and market impact. Med Care 2002;40:325 37.
Consumers. Monash University, Department of Epidemiology
43. Grasso BC, Rothschild JM, Jordan CW et al. What is the
and Preventive Medicine, 2004.
measure of a safe hospital? Medication errors missed by risk
26. Fairbrother G, Gleeson M. EQuIP accreditation: feedback from management, clinical staff, and surveyors. J Psychiatr Pract
a Sydney teaching hospital. Aust Health Rev 2000;23:153 62. 2005;11:268 73.
27. Hurst K. The nature and value of small and community hospi- 44. Grifth JR, Knutzen SR, Alexander JA. Structural versus out-
tal accreditation. Int J Qual Health Care 1997;10:94 106. comes measures in hospitals: a comparison of Joint
Commission and Medicare outcomes scores in hospitals. Qual
28. Brasure M, Stensland J, Wellever A. Quality oversight: why are
Manag Health Care 2002;10:2938.
rural hospitals less likely to be JCAHO accredited? J Rural
Health 2000;16:324 36. 45. Miller MR, Pronovost P, Donithan M et al. Relationship
between performance measurement and accreditation: impli-
29. Pongpirul K, Sriratanaban J, Asavaroengchai S et al.
cations for quality of care and patient safety. Am J Med Qual
Comparison of health care professionals and surveyors
2005;20:239 52.
opinions on problems and obstacles in implementing quality
management system in Thailand: a national survey. Int J Qual 46. Salmon J, Heavens J, Lombard C et al. The Impact of Accreditation
Health Care 2006;18:346 51. on the Quality of Hospital Care: KwaZulu-Natal Province Republic of
South Africa: Published for the U.S. Agency for International
30. Mihalik G, Scherer M, Schreter R. The high price of quality: a
Development (USAID) by the Quality Assurance Project,
cost analysis of NCQA accreditation. J Health Care Finance
University Research Co., LLC., 2003.
2003;29:38 47.
47. Hadley T, McGurrin M. Accreditation, certication, and the
31. Duckett S. Changing hospitals: the role of hospital accreditation.
quality of care in State hospitals. Hosp Community Psychiatry
Soc Sci Med 1983;17:1573 9.
1988;39:739 42.

182
Health sector accreditation research

48. Collopy BT. Clinical indicators in accreditation: an effective 62. Hampel MJ, Hastings MM. Assessing quality in nursing home
stimulus to improve patient care. Int J Qual Health Care dementia special care units: a pilot test of the Joint Commission
2000;12:211 6. protocol. J Ment Health Adm 1993;20:236 46.
49. Collopy BT, Williams J, Rodgers L et al. The ACHS Care 63. Huda W, Sajewicz AM, Ogden KM et al. How good is the ACR
Evaluation Program: a decade of achievement. J Qual Clin Pract accreditation phantom for assessing image quality in digital
2000;20:36 41. mammography? Acad Radiol 2002;9:764 72.
50. Gabriele P, Malinverni G, Bona C et al. Are quality indicators 64. Jain N, Willett KM. Maintaining standards in surgical training:
for radiotherapy useful in the evaluation of service efcacy in a how effective is accreditation by the specialist advisory commit-
new based radiotherapy institution? Tumori 2006;92:496 502. tee in the United Kingdom? J Bone Joint Surg
2006;88-B:111115.
51. Silver M, Geis M, Bateman K. Improving health care systems
performance: a human factors approach. Am J Med Qual 65. Whittaker S, Green-Thompson RW, McCusker I et al. Status of
2004;19:93102. a health care quality review programme in South Africa. Int J
Qual Health Care 2000;12:24750.
52. VanSuch M, Naessens J, Stroebel R et al. Effect of discharge
instructions on readmission of hospitalised patients with heart 66. Viswanathan HN, Salmon JW. Accrediting organizations and
failure: do all of the Joint Commission on Accreditation of quality improvement. Am J Manag Care 2000;6:1117 30.
Healthcare Organizations heart failure core measures reect
67. Heuer AJ. Hospital accreditation and patient satisfaction: testing
better care? Qual Saf Health Care 2006;15:414 7.
the relationship. J Healthcare Qual 2004;26:4651.
53. Williams SC, Schmaltz SP, Morton DJ et al. Quality of care in
68. Greco M, Sweeney K, Brownlea A et al. The practice accredita-
U.S. hospitals as reected by standardized measures, 2002-2004.
tion and improvement survey (PAIS). What patients think. Aust
N Engl J Med 2005;353:255 64.
Fam Physician 2001;30:1096 100.
54. Williams SC, Watt A, Schmaltz SP et al. Assessing the reliability
69. Durieux P, Bissery A, Gasquet I et al. Comparison of health
of standardized performance indicators. Int J Qual Health Care
care professionals self-assessments of standards of care and
2006;18:246 55.
patients opinions on the care they received in hospital: observa-
55. Borenstein J, Badamgarav E, Henning J et al. The association tional study. Qual Saf Health Care 2004;13:198 202.

Downloaded from by guest on May 14, 2016


between quality improvement activities performed by managed
70. Ito H, Sugawara H. Relationship between accreditation scores
care organisations and quality of care. Am J Med
and the public disclosure of accreditation reports: a cross sec-
2004;117:297304.
tional study. Qual Saf Health Care 2005;14:8792.
56. Gross P, Braun B, Kritchevsky S et al. Comparison of clinical
71. Dickison P, Hostler D, Platt TE et al. Program accreditation
indicators for performance measurement of health care quality:
effect on paramedic credentialing examination success rate.
a cautionary note. Clin Perform Qual Health Care 2000;8:202 11.
Prehosp Emerg Care 2006;10:224 8.
57. Synder C, Anderson G. Do quality improvement organisations
72. McCleish JM. Relationships between Accreditation Afliation,
improve the quality of hospital care for medicare beneciaries?
Denitions, and Tools Used to Assess Critical Thinking as a Learning
JAMA 2005;293:2900 7.
Outcome in Schools of Nursing. Ph.D. Thesis. Iowa State University
58. Barker K, Flynn E, Pepper G et al. Medication errors observed 2002.
in 36 health care facilities. Arch Intern Med 2002;162:1897 1903.
73. Tracey J, Arroll B, Richmond D. Changes in CME uptake
59. Chen J, Rathore SS, Radford MJ et al. JCAHO accreditation caused by reaccreditation. N Z Med J 1998;111:11820.
and quality of care for acute myocardial infarction. Health Aff
74. Gropper R. Educational outcomes and specialised accreditation.
2003;22:243 54.
Nurse Educ 1996;21:8 10.
60. Cunha I, Feldman L. Nursing service assessment: identication
75. Bohigas L, Brooks T, Donahue T et al. A comparative analysis
of process criteria in hospital accreditation programs. Rev Bras
of surveyors from six hospital accreditation programmes and a
Enferm 2005;58:6569.
consideration of the related management issues. Int J Qual
61. Gillon M, Buetow S, Wellingham J et al. A practical approach to Health Care 1998;10:7 13.
quality improvement: the experience of the RNZCGP practice
standards validation eld trial. N Z Med J 2003;116:pU682. Accepted for publication 18 January 2008

183

You might also like