Professional Documents
Culture Documents
EBM Theory and Practice PDF
EBM Theory and Practice PDF
EBM Theory and Practice PDF
www.elsevier.com/locate/ebhph
What is a guideline?
Clinical practice guidelines are defined as systematically developed statements to assist practitioner
and patient decisions about appropriate healthcare
for specific clinical circumstances.3
It is useful to make a distinction between guidelines and protocols as sometimes these terms are
used interchangeably (and erroneously). Protocols
are instructions on what to do in particular
1744-2249/$ - see front matter & 2005 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ehbc.2005.05.012
ARTICLE IN PRESS
Evidence-based guidelines: the theory and the practice
Box 1
309
Improve efficiency
Optimise value for money
Demonstrating adherence to guidelines may improve public image
Some systematic reviews include a meta-analysis. This is a statistical synthesis of the results of
the included studies, with the aim of producing a
single estimate of a treatment effect. Metaanalysis is only appropriate in certain circumstances and if carried out inappropriately can lead
to misleading, erroneous results. Readers keen to
ARTICLE IN PRESS
310
master the ability to critically appraise a systematic review and decide whether its results can be
applied to particular clinical circumstances are
directed elsewhere.10,11
Systematically reviewing the literature poses
several challenges for guideline developers. One
fundamental problem is that scientific evidence
about what to recommend is often lacking only a
small fraction of what is done in medicine has been
tested in appropriate, well-designed studies.2 Even
when well conducted systematic reviews already
exist, the leap from interpreting the results of a
systematic review to producing recommendations is
a complex one, which involves subjective value
judgements and an understanding of whether the
results obtained in the research setting might be
appropriate to extrapolate to everyday practice.
Producers of evidence-based guidelines should
make explicit links between the recommendations
contained in the guideline and the evidence that
has been used to inform the recommendations.
Important limitations identified in the retrieved
evidence (relating to the circumstances in which
a trial was conducted or the specific details of
the research participants or intervention, for
example) or absence of evidence should be made
explicit.
Although there are large databases of systematic
reviews and randomised controlled trials of therapy, it is much more difficult to find high quality
studies on diagnostic procedures, prognosis, and
aspects of healthcare whose study requires a
qualitative approach. Many published reports of
diagnostic tests suffer from poor methods and small
samples which make them prone to bias, and poor
quality of reporting which makes them difficult to
interpret.12
Guideline developers often refer to a hierarchy
of evidence and associated recommendations,
using numbers and letters to rank levels of
evidence and grades of recommendation. For
example, recommendations may be Grade A, if
based on evidence from meta-analysis of randomised controlled trials (Level 1a evidence) or
evidence from at least one randomised controlled
trial (Level 1b evidence). Unfortunately there has
been a proliferation of such hierarchies, with
different guideline developers using different
scales and assigning different weightings to different types of evidence. This causes confusion and
undermines the usefulness of such hierarchies.13
The same intervention might be assigned different
grades of recommendation in different international guidelines, thus failing to communicate a
message quickly and concisely in a way that is
universally understood.14
D. Swinglehurst
The GRADE Working Group has recently published
a system designed to overcome some of these
limitations, based on consideration of four key
elements: study design, study quality, consistency
(similarity of estimates of effect across studies) and
directness (the extent to which the people,
interventions and outcome measures are similar
to those of interest).15 These four components are
combined to make a judgement on the overall
quality of evidence, which is rated as high,
moderate, low or very low. Evidence is initially
categorised according to the study design (for
example RCT is high and an observational study
is low) but additional consideration of the quality,
consistency and directness results in shifts upwards
or downwards until the final evidence grading is
determined. The lowest quality of evidence for any
outcome that is critical to decision making then
provides the basis for the overall rating of the
evidence.15 Recommendations are then based on a
consideration of:
ARTICLE IN PRESS
Evidence-based guidelines: the theory and the practice
and other social and organisational variables.
Producing systematic reviews of this type of
intervention is particularly challenging; the interested reader might like to look out some specialist
articles in this area.17,18
Box 3
311
What are the guiding principles regarding how effective an intervention must be (compared to its
potential harms) before its recommendation is considered?
What values underpin decisions about which guideline developments to prioritise?
What is the ethical framework to which guideline developers are working in particular relating to
matters of distributive justice/rationing of limited resources?
Where disagreement between guideline developers exists, what explicit processes are being used to
resolve such disagreements?
ARTICLE IN PRESS
312
D. Swinglehurst
conclusions about cost effectiveness, priority setting and rationing. Whilst the outcome of this
process and the details of the final guidelines
remain uncertain, aiming to define and make
transparent the value judgements that underpin
NICE guidelines should be applauded.
When evidence-based guidelines are prepared,
they should be subjected to external review in
order to check them for validity, clarity and
applicability.20 Shekelle suggests that external
reviews should include people with clinical expertise in the relevant area and experts in systematic
review or guideline development, as well as
potential users of the guideline.
Box 4
ARTICLE IN PRESS
Evidence-based guidelines: the theory and the practice
practitioners and policy makers to help them assess
the process of guideline development and the
quality of the reporting of this process (as distinct
from the quality of the evidence underpinning the
recommendations).25 Like other appraisal instruments, the AGREE instrument (which can be downloaded from the AGREE website26) uses a checklist
that covers 23 items across six different quality
domains. The developers of the instrument recommend that four individuals use the instrument to
appraise a guideline, as this has been shown to
increase the reliability of the domains and yield a
more robust assessment.25 Many international
bodies are now using the AGREE instrument in the
adoption of standards for guideline development.
A summary of the AGREE instrument is shown in
Box 4.
Conclusion
No discussion of guidelines would be complete
without at least some mention of their implementation. Given the volume and complexity of guidelines, it is perhaps not surprising that major
difficulties arise when introducing clinical guidelines into routine daily practice.27 The evidence
base to guide decisions about which strategy of
guideline dissemination and implementation to use
is poor.28 The journey from the publication of
scientific evidence to its incorporation into guidelines and then to implementation in the real world
of professional practice is a long and complex one
that continues to perplex researchers and policy
developers worldwide. The evidence that guidelines result in improved health outcomes for
patients is far from clear cut, although a quality
evidence-based guideline has the potential to
achieve this, if as much attention is paid to the
dissemination and implementation phase as to its
original development. The implementation of
guidelines involves a complex interplay of sociocultural and organisational factors which extend
way beyond the strength of the evidence conveyed
in the guideline itself. Indeed, the implementation
of guidelines is a very good example of a complex
healthcare intervention.
Evidence-based guidelines are only one dimension in the processes that go on in a clinicians mind
as decisions are made in day-to-day practice.
Ethnographic research has shown that primary care
practitioners (the target audience for many national guidelines) rarely access guidelines when
observed in routine practice, but that their
decisions are informed by what have been de-
313
Acknowledgements
I would like to thank Professor Trisha Greenhalgh
for providing comments on a first draft of this
article.
References
1. National electronic Library for Health. Guidelines Finder.
2005. http://libraries.nelh.nhs.uk/guidelinesFinder/
2. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J.
Potential benefits, limitations, and harms of clinical guidelines. BMJ 1999;318:52730.
3. Clinical Practice Guidelines: directions for a new program.
Washington DC: National Academy Press, 1990
4. Grilli RG, Magrini N, Penna A, Mura G, Liberati A. Practice
guidelines developed by specialty societies: the need for a
critical appraisal. Lancet 2000;355:1036.
5. Keeley P. Clinical guidelines. Palliat Med 2003;17:36874.
6. Sackett DL, Rosenberg MC, Muir Gray JA, Haynes RB,
Richardson WS. Evidence based medicine: what it is and
what it isnt. BMJ 1996;312:712.
7. Systematic Reviews. London, UK: BMJ Publishing Group,
1995
8. The Cochrane Collaboration. The reliable source of evidence
in healthcare. http://www.cochrane.org/index0.htm
9. The Centre for Reviews and Dissemination, University of
York. http://www.york.ac.uk/inst/crd/
10. Greenhalgh T. How to read a paper: papers that summarise
other papers (systematic reviews and meta-analyses). BMJ
1997;315:6725.
11. Greenhalgh T, Donald A. Papers that summarise other papers
(systematic review and meta-analysis). In: Greenhalgh T,
Donald A, editors. Evidence Based Health Care Workbook.
London, UK: BMJ Publishing Group; 2000. p. 11129.
ARTICLE IN PRESS
314
12. Bandolier Extra. Evidence and Diagnostics. Bandolier.
http://www.jr2.ox.ac.uk/bandolier/Extraforbando/Diagnostic.
pdf
13. Upshur REG. Are all evidence-based practices alike? Problems in the ranking of evidence. CMAJ 2003;169:6723.
14. Schunemann HJ, Best D, Vist G, Oxman AD. Letters,
numbers, symbols and words: how to communicate grades
of evidence and recommendations. CMAJ 2003;169:67780.
15. GRADE Working Group. Grading quality of evidence and
strength of recommendations. BMJ 2004;328:14907.
16. The Cochrane Collaboration. The Cochrane Handbook for
Systematic Reviews of Interventions. http://www.cochrane.org/resources/handbook/
17. Pawson R, Greenhalgh T, Harvey G, Walshe, K. Realist
synthesis: an introduction. 4. ESRC Research Methods
Programme. http://www.ccsr.ac.uk/methods/publications/
documents/RMPmethods2.pdf
18. Medical Research Council. A framework for development
and evaluation of RCTs for complex interventions to improve health. London, UK: MRC; 2000 http://www.mrc.
ac.uk/prn/pdf-mrc_cpr.pdf.
19. Jackson R, Feder G. Guidelines for guidelines. BMJ 1998;
317:4278.
20. Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Developing
guidelines. BMJ 1999;318:5936.
21. Herrin J, Etchason JA, Kahan JP, Brook RH, Ballard DJ. Effect
of panel composition on physician ratings of appropriateness
of abdominal aortic aneurysm surgery: elucidating differences between multidisciplinary panel results and specialty
recommendations. Health Policy 1997;42:6781.
D. Swinglehurst
22. Coulter I, Adams A, Shekelle P. Impact of varying panel
membership on ratings of appropriateness in consensus
panels: a comparison of a multi- and single disciplinary
panel. Health Serv Res 1995;30:57791.
23. National Institute for Health and Clinical Excellence. Social
Value Judgements: guidelines for the institute and its
advisory bodies (draft for consultation). 2005. http://
www.nice.org.uk/pdf/CC_SVJ_0405.pdf
24. AGREE collaboration. 2002. http://www.agreecollaboration.
org/
25. The AGREE Collaboration. Development and validation of an
international appraisal instrument for assessing the quality
of clinical practice guideline: the AGREE project. Qual Saf
Health Care 2003;12:1823.
26. The AGREE Collaboration. AGREE instrument. AGREE . 2005.
http://www.agreecollaboration.org/instrument/
27. Grol R, Grimshaw J. From best evidence to best practice:
effective implementation of change in patients care. The
Lancet 2003;362:122530.
28. Grimshaw J, Thomas RE, MacLennan G, Fraser C, Ramsay CR,
Vale L, et al. Effectiveness and efficiency of guideline
dissemination and implementation strategies. Health Technology Assessment 2004;8.
29. Gabbay J, le May A. Evidence based guidelines or collectively contructed mindlines? Ethnographic study of
knowledge management in primary care. BMJ 2004;
329:10137.
30. Egger M, Davey Smith G. Principles of and procedures for
systematic reviews. In: Egger M, Davey Smith G, Altman DG,
editors. Systematic Reviews in Health Care, pp 23-42. 2001.