Critical Appraisal of

clinical guidelines
What is a Clinical Practice Guideline?

• A clinical practice guideline is a systematically developed

statement designed to help health care professionals make
decisions about appropriate health care for specific clinical
circumstances.
• In addition, guidelines can play an important role in health
policy formation
 The potential benefits of guidelines
are only as good as the quality of
the guidelines themselves.
Clinical
Appropriate methodologies and
Practice rigorous strategies in the guideline
Guideline development process are
important for the successful
implementation of the resulting
recommendations.
 • Validity.
• Reliability

Characteristic • Clinical applicability.

• Clinical flexibility.
of Good • Clarity.
Practice • Multidisciplinary process.
• Scheduled review.
Guidelines • Documentation.
• Estimated outcome.
• Strength of evidence.
Guidelines attempt to do this by:

Describing a range of generally accepted

approaches for the diagnosis, management, or
prevention of specific diseases or conditions.

Defining practices that meet the needs of most

patients in most circumstances.
The
Define selection criteria for
Search for guidelines
guidelines

process of
guideline Appraise guidelines critically

appraisal
- select an appraisal tool
- assess methodologic and Decide on acceptability of
content validity guideline (Should it be
adopted, adapted, or rejected?)
- assess quality of reporting
- score methodologic quality
Find Clinical Practice Guidelines:

1. Go to ClinicalKey and limit the search to "Guidelines".

2. When searching MEDLINE, click on "Additional Limits", and limit

by publication type (Practice Guideline).
 Cluzeau’s “Appraisal Instrument”
Tools for Cluzeau et al. (1999)
Measuring Appraisal of Guidelines for
Research & Evaluation (AGREE)
the Quality Cluzeau, F.A., J. Burgers and an
of Clinical international group of researchers
from 13 countries.
Practice
Guidelines AGREE II Brouwers et al. (2010)
AGREE
• The main objective was
to develop an appraisal
instrument to assess
clinical guidelines
• In November 2002 the
AGREE Collaboration
disseminate and
implement the AGREE
Instrument.
 The AGREE (Appraisal of Guidelines for
Research & Evaluation) Instrument is a tool
that assesses the methodological rigour
Appraisal and transparency in which a guideline is
developed and it is used internationally.
of Clinical
Practice The purpose of the AGREE Instrument is to
provide a framework for assessing the
Guidelines quality of clinical practice guidelines.
 • The AGREE Instrument is designed to
Appraisal of assess guidelines developed by local,
regional, national, or international
Clinical groups or affiliated government
organizations.
Practice • These include:
1) New guidelines
Guidelines 2) Existing guidelines
3) Updates of existing guidelines
 To assess the quality of CPGs:
how well a guideline is
developed and reported.

AGREE It does not assess the clinical

content or quality of evidence
of the recommendations.
 AGREE consists of 23 key items
organized in 6 domains.

AGREE Each domain is intended to

capture a separate dimension
of guideline quality.
 Each item is rated on Likert scale ranging
AGREE from (strongly agree) to (strongly disagree).
 Domain scores are calculated by
summing up all the scores of the
individual items in a domain and by
Domain Scores standardizing the total as a
percentage of the maximum
possible score for that domain.
 The six domain scores are
independent and should not be
aggregated into a single quality
Domain Scores score; there is no threshold score
that separates a “good” from a “not
good” guideline.
 • Overall assessment includes:
o the rating of the overall quality
Overall of the guideline
Assessment o whether the guideline would
be recommended for use in
practice.
 AGREE II represents an update of the
original tool to improve its usability
and methodological properties,
namely its validity and reliability.
AGREE II also includes 2 final overall
assessment items that require the
appraiser to make overall judgments
of the practice guideline and consider
how they rated the 23 items.
 Scope and Purpose

Stakeholder Involvement

AGREE Rigour of Development

Domains Clarity and Presentation

Applicability

Editorial Independence
 No. Domain Items
1-3
1 Scope and purpose

4-6
2 Stakeholder Involvement

AGREE 3 Rigour of Development

7-14

Items 4 Clarity and Presentation

15-17

18-21
5 Applicability

22-23
6 Editorial Independence
 Are the overall objectives of the
guideline specifically described?

Is the clinical question(s) covered

Scope and by the guideline specifically
Purpose described?
Are the patients to whom the
guideline is meant to apply
specifically described?
 Does the guideline development group
include individuals from all of the relevant
professional groups?
Have the patients' views and preferences
been sought?
Stakeholder
Involvement Have the target users of the guideline been
clearly defined?

Has the guideline been piloted among target

users?
 Were systematic methods used to search for
evidence?
Is the criteria for selecting the evidence clearly
described?
Were the methods used for formulating the
recommendations clearly described?
Rigour of Were the health benefits, side effects, and risks
Development considered in formulating the recommendations?
Is there an explicit link between the
recommendations and the supporting evidence?
Has the guideline been externally reviewed by experts
prior to its publication?
Is a procedure for updating the guideline provided?
 Are the recommendations specific and
unambiguous?

Are different options for management

of the condition clearly presented?
Clarity and
Presentation Are key recommendations clearly
identifiable?

Is the guideline supported with tools

for application?
 Have the potential organization
barriers in applying the
recommendation been discussed?
Have the potential cost implications
Applicability of applying the recommendations
been considered?
Does the guideline present key
review criteria for monitoring
and/or auditing purposes?
 Is the guideline editorially
independent from the
funding body?
Editorial
Independence Have conflicts of interest of
guideline development
members been recorded?
Calculating
Domain Scores

Domain scores are

calculated by summing up
all the scores of the
individual items in a
domain and by scaling the
total as a percentage of the
maximum possible score
for that domain.
Overall
Guideline
Assessment
AGREE User Manual
AGREE User Manual Link
References
• Himmelfarb, GWU (LINK)
• NIH (LINK)
• CA Guidelines (book chapter) (LINK)