Adherence of Clinical Practice Guidelines For Pharmacologic Treatments

Original Investigation | Medical Journals and Publishing
Adherence of Clinical Practice Guidelines for Pharmacologic Treatments

of Hospitalized Patients With COVID-19 to Trustworthy Standards
A Systematic Review
Karen E. A. Burns, MD, MSc (Epid); Matthew Laird, BMSc; James Stevenson, BSc; Kimia Honarmand, MD; David Granton, MD;
Michelle E. Kho, PT, PhD; Deborah Cook, MD; Jan O. Friedrich, MD; Maureen O. Meade, MD; Mark Duffett, PhD; Dipayan Chaudhuri, MD;
Kuan Liu, PhD; Frederick D’Aragon, MD; Arnav Agarwal, MD; Neill K. J. Adhikari, MD; Hayle Noh, BSc; Bram Rochwerg, MD;
for the Academy of Critical Care: Development, Evaluation, and Methodology (ACCADEMY)
Abstract Key Points

Question Do clinical practice guidelines
IMPORTANCE The COVID-19 pandemic created the need for rapid and urgent guidance for clinicians
(CPGs) that report on pharmacologic
to manage COVID-19 among patients and prevent transmission.
treatments of hospitalized patients with
COVID-19 meet the National Academy
OBJECTIVE To appraise the quality of clinical practice guidelines (CPGs) using the National Academy
of Medicine standards for
of Medicine (NAM) criteria.
trustworthiness?
EVIDENCE REVIEW A search of MEDLINE, EMBASE, and the Cochrane Central Register of Findings In this systematic review of 32
Controlled Trials to December 14, 2020, and a search of related articles to February 28, 2021, that CPGs of predominantly low quality, few
included CPGs developed by societies or by government or nongovernment organizations that reported funding sources or conflicts of
reported pharmacologic treatments of hospitalized patients with COVID-19. Teams of 2 reviewers interest, included a methodologist,
independently abstracted data and assessed CPG quality using the 15-item National Guideline described a search strategy or study
Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) instrument. selection process, or synthesized
evidence. Although 14 CPGs (43.8%)
FINDINGS Thirty-two CPGs were included in the review. Of these, 25 (78.1%) were developed by made recommendations or suggestions
professional societies and emanated from a single World Health Organization (WHO) region. Overall, for or against treatments, they
the CPGs were of low quality. Only 7 CPGs (21.9%) reported funding sources, and 12 (37.5%) reported infrequently rated the confidence in the
conflicts of interest. Only 5 CPGs (15.6%) included a methodologist, described a search strategy or quality of the evidence (6 [18.8%]),
study selection process, or synthesized the evidence. Although 14 CPGs (43.8%) made described potential benefits and harms
recommendations or suggestions for or against treatments, they infrequently rated confidence in the (6 [18.8%]), or graded the strength of
quality of the evidence (6 of 32 [18.8%]), described potential benefits and harms (6 of 32 [18.8%]), recommendations (5 [15.6%]).
or graded the strength of the recommendations (5 of 32 [15.6%]). External review, patient or public
Meaning The findings of this study
perspectives, or a process for updating were rare. High-quality CPGs included a methodologist and
suggest that few COVID-19 CPGs meet
multidisciplinary collaborations involving investigators from 2 or more WHO regions.
National Academy of Medicine
standards for trustworthy guidelines.
CONCLUSIONS AND RELEVANCE In this review, few COVID-19 CPGs met NAM standards for
trustworthy guidelines. Approaches that prioritize engagement of a methodologist and
multidisciplinary collaborators from at least 2 WHO regions may lead to the production of fewer, + Invited Commentary
high-quality CPGs that are poised for updates as new evidence emerges.
+ Supplemental content
Author affiliations and article information are
TRIAL REGISTRATION PROSPERO Identifier: CRD42021245239
listed at the end of this article.
JAMA Network Open. 2021;4(12):e2136263. doi:10.1001/jamanetworkopen.2021.36263
Open Access. This is an open access article distributed under the terms of the CC-BY License.
JAMA Network Open. 2021;4(12):e2136263. doi:10.1001/jamanetworkopen.2021.36263 (Reprinted) December 10, 2021 1/15
Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

JAMA Network Open | Medical Journals and Publishing CPGs for Pharmacologic Treatments of Hospitalized Patients With COVID-19
Introduction
Clinical practice guidelines (CPGs) should be systematically developed statements and
recommendations that articulate the roles for diagnostic tests and treatments to inform clinician and
patient decisions. The process for creating guidelines affects CPG quality. In turn, CPG quality affects
patient care, safety, and health care equality. In 2011, the National Academy of Medicine (NAM)
(formerly known as the Institute of Medicine)1 published a report stipulating that CPG
recommendations should be supported by a systematic review of the evidence and highlighted 8
criteria for assessing the trustworthiness of CPGs.
Many instruments and scorecards have been developed to evaluate CPG quality.2-11 The
Appraisal of Guidelines for Research and Evaluation (AGREE) II tool8 is the most widely used CPG
appraisal tool. The scope of the AGREE II tool targets all components of a CPG report, emphasizing
features that enhance its internal validity. The AGREE Recommendation Excellence tool,9 a
supplement to the AGREE II tool, highlights 9 items in 3 themes that focus on the quality of the CPG
recommendations and the justifications that underpin them. The AGREE Recommendation
Excellence tool ascertains whether CPGs are credible and implementable by assessing their internal
consistency. Notwithstanding, the AGREE II and AGREE Recommendation Excellence appraisal tools
do not directly address the NAM criteria for trustworthy CPGs or consider the perspectives of
different stakeholder groups involved in CPG development. More recently, the US Agency for
Healthcare Research and Quality developed the National Guideline Clearinghouse Extent of
Adherence to Trustworthy Standards (NEATS) instrument 11 (eMethods in Supplement 1) to provide
a standardized approach to assess CPG quality. The NEATS tool explicitly evaluates the NAM criteria
and assesses CPGs from a broad and multidisciplinary perspective.
The COVID-19 pandemic created the need for rapid and urgent guidance for clinicians to
manage COVID-19 among patients and prevent transmission, but methodological rigor has been
variable across CPGs.12 We systematically reviewed published CPGs reporting pharmacologic
treatments for hospitalized patients with COVID-19 and evaluated their quality and trustworthiness
using the NEATS instrument. We hypothesized that CPGs created and disseminated during the
pandemic have important methodological weaknesses that affect their quality and trustworthiness.
Methods
Study Design
We systematically reviewed published CPGs addressing pharmacologic treatments for hospitalized
patients with COVID-19. An ethics review was not obtained for this secondary analysis of
published data.
Data Sources and Searches

We searched MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials (to
December 14, 2020) and conducted a search of related articles (to February 28, 2021) to identify
updates. Two reviewers (M.L. and J.S.) independently screened all titles and abstracts of citations for
eligibility. Disagreements were resolved by consensus or in discussion with a third reviewer
(K.E.A.B.).
Study Selection
Eligible CPGs were investigator led, sponsored or produced by a national or international scientific
organization or government or nongovernment organization related to global health, and reported
on pharmacologic treatments of hospitalized patients with COVID-19 and its complications.
Pharmacologic interventions referred to treatments dispensed by hospital pharmacies, with an
identifiable molecular structure. We included all versions of published CPGs. We did not apply
language restrictions. We excluded CPGs produced for regional or local use (eg, hospital-based),

nonpharmacologic interventions, medications not dispensed by a hospital pharmacy (eg, herbal

remedies, homeopathic medications), and CPGs that focused on treatments for specific populations
(ie, obstetrical populations hospitalized with COVID-19).
Data Abstraction
Fifteen reviewers (K.E.A.B., M.L, J.S., K.H., D.G., M.E.K., D.C., J.O.F., M.O.M., M.D., D.C., F.D., A.A.,
N.K.J.A., and B.R.), working independently and in duplicate, abstracted data pertaining to CPG
publications (dates of submission, acceptance, and publication online and in print), geographical
representation of collaborators using World Health Organization (WHO) regions, CPG sponsorship
(professional society, government or nongovernment agency, or other), funding (monetary or
nonmonetary), and scope (international, national, state/province, or other). Reviewers recorded the
patient populations addressed (hospitalized, ward, intensive care unit, or other) and assessed
whether CPGs had a formal conflict of interest (COI) policy and declared COIs (financial, nonfinancial,
or both). They noted whether patient and/or public perspectives were sought or incorporated. For
each pharmacologic intervention, appraisers documented outcomes of interest and the direction of
the recommendation statement (for or against or no recommendation), the strength of the
recommendation, and the certainty of the evidence.
Quality Assessment
Working in pairs, reviewers appraised the quality of included CPGs using the NEATS instrument11
(eMethods in Supplement 1). Disagreements were resolved by consensus or in discussion with a third
reviewer (K.E.A.B.).
Data Synthesis and Analysis

We collated data in Excel, version 2016 (Microsoft Corporation), to characterize CPGs, NEATS scores,
and the direction of recommendation or suggestion statements (for, against, or no recommendation)
for each pharmacologic treatment described in included CPGs. The NEATS instrument includes 3
binary or categorical items reflecting adherence to the NAM standard: assessing disclosure of
funding (yes or no), multidisciplinary representation (yes, no, or unknown), and inclusion of a
methodologist on the guideline panel (yes, no, or unknown) in CPG panels and 12 Likert scales.
Reviewers rated adherence (on a scale ranging from 1 [low adherence] to 5 [high adherence]) to 12
NAM standards reflecting disclosure/management of COIs, inclusion of patient and public
perspectives, use of systematic review of the evidence (separate items for reporting search strategy,
study selection, and synthesis of the evidence), a process for making recommendations (separate
items for reporting grading or rating of the quality or strength of evidence, reporting benefits and
harms of recommendations, including evidence summary supporting recommendations, and rating
the strength of recommendations), generation of specific and unambiguous recommendations,
procurement of external reviews, and inclusion of a prespecified process to update the CPG.
We tabulated results to highlight the evolution of evidence over time for each pharmacologic
intervention addressed by 3 or more CPGs. We characterized and compared recommendations for
use of each specific pharmacologic intervention by assessing whether a recommendation or
suggestion was made for or against its use or whether no recommendation for its use was made. In
assessing the consistency between CPGs for each pharmacologic treatment, we prioritized the
direction of the recommendation over the strength of the recommendation.
We depicted quality scores for each element of the NEATS instrument for each included CPG
using a Coxcomb chart (Figure 1). We illustrated the quality ratings of all included CPGs using a heat
map (Figure 2). To summarize binary and categorical data in the NEATS assessment in both figures,
we assigned a score of 5 for yes and 0 for no or unknown responses.

Results
CPG Identification and Quality Assessment
From 2226 citations, the reviewers screened 51 full texts and identified 32 eligible CPGs.13-44
We excluded 19 CPGs (eFigure in Supplement 1).45-63 Of the 32 included CPGs, 3 (9.4%)15,38,40
reported on pharmacologic treatments for critically ill patients specifically, and the
remainder reported on treatments for hospitalized patients with COVID-19. Fifteen CPGs
(46.9%) were international15,16,20,21,23,24,27,31-34,36,38,43,44 and 17 (53.1%) were
national.13,14,17-19,22,25,26,28-30,35,37,39-42 Most CPGs (25 [78.1%]) had sponsorship from 1 or
more national societies, and few CPGs (3 [9.4%])14,23,34 had sponsorship from government,
nongovernment, or not-for-profit agencies (4 [12.5%]).13,32,38,44 Seven CPGs (21.9%)13,20,23,30-32,34
explicitly reported their funding sources (eTable 1 in Supplement 1). Guidelines predominantly
included authors from a single WHO region (20 [62.5%]) and mostly emanated from America or
Figure 1. Coxscomb Chart Depicting the National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Score of Included
Clinical Practice Guidelines
Roch
NEATS Instrument
werg
Disclosure of guideline funding sources

Ro
Disclosure and management of financial

ch
e
we
t al,
conflict of interests
rg
2020
Multidisciplinary group
44
et a
2020
Methodologist involvement
t al, 15
l,
32
Patient and public perspectives

20
Zhai et al,
20
ani e
Search strategy
Study selection
zz
Alha
Synthesis of evidence
20
al, 38 2020
Grading the quality or strength of evidence

2020
Benefits and harms of recommendations

Evidence summary supporting
20
recommendations
Abu-Raya et
20
Re
20
it
20
er Rating the strength of recommendations

l, 41
et
al, 2
al , 2 Specific and unambiguous articulation

et a
Th
3
et
20 of recommendations
ac
20
es
jak
20
hil
20 20 ExternaI review
rn
42 20
dre
e
8
Ba
Ram ,1
al,
ta
l Updating
et a
An
íre
l,
et
16
ze
t al rli
t wla
Cha
20
Raja , 22 Be
20
gopa 20
l et al 20
, 21 2 6 020
020 l,3 2
ka l et a
National Healt
h Commissio Chek
Republic of
China, 14 2020 n of the People's Ferreira et al
,35 2020
Flisiak et al, 17
2020
,27 2020
Moores et al Flisia
9 2020 0 k et al
al ,3 02 , 28 2
ta et 19 2 020
Meh t a l, Flis
e u s iak
hta tio Fo et a
Me ec t ie
Go
l, 37
20
I nf ta 20
lde
20
ty l, 40 20
cie
nb
l , 25
He
2020
20
So 20
er
20
21
nde
ta
n
g
lia 20
et
20
ue
Ita ,26
rso
Hend
et al 29
al,
Lla
, 30
ion ases
33
,
ne
c t
Se ise
Jin et al,
t al
20
erso
t al
dy l D
20
Li e
ar pica
,3
Kosio
b
m o
et al
Lo d Tr
20
13
20
an
,43 2
2020
020
20
Jin et al, 34 20
Different colors reflect the various components of the National Academy of Medicine [high adherence]) to 12 NAM standards and includes 3 binary or categorical questions
(NAM) score. Higher rays represent higher-quality scores as assessed using the NEATS (response options yes, no, and/or unknown).11
instrument, which measures adherence (on a scale ranging from 1 [low adherence] to 5

Europe (Table 1). Overall, few CPGs met most of the NAM standards for trustworthiness as assessed
by the NEATS instrument (Figure 1 and eTable 2 in Supplement 1).
Funding and Panel Composition

Eighteen CPGs (56.3%) explicitly disclosed funding information.13,15,18,20-24,26,27,30-32,34,38,39,42,44
Twenty CPGs (62.5%)13,15,17,18,20,21,23,24,27-30,32,34,35,37,38,40,43,44 included multidisciplinary guideline
panels, and 5 guideline panels (15.6%)13,15,32,34,44 included a methodologist.
Disclosure of COIs and Inclusion of Patient and Public Perspectives

We identified 12 CPGs (37.5%)13,15,17,20,23,27,28,32,37,40,42,44 with high adherence (score of 4 or 5) to the
NAM standard to disclose actual or potential financial COIs and report how COIs were incorporated
or managed in the CPG development process. Eight CPGs (25.0%)21,22,29-31,34,38,43 had intermediate
adherence (score of 3) and 12 CPGs (37.5%)14,16,18,19,24-26,33,35,36,39,41 had low adherence (score of 1
or 2) to this NAM standard.
Only 2 CPGs (6.3%)32,44 adhered (both with a score of 5) to the requirement to seek the views
of patients, surrogates, advocates, and/or the public who represent those that have experience with
the disease, its treatment, or its complications or those who could be affected by the CPG. These
Figure 2. Heat Map Depicting Quality of Included Clinical Practice Guidelines
of the People's Republic of China et al,14 2020

Lombardy Section Italian Society Infectious
Assessment
1 2 3 4 5 N Y UNK
and Tropical Diseases,26 2020
National Health Commission

Goldenberg et al,33 2020
Henderson et al,43 2021
Henderson et al,31 2020

Rochwerg et al,44 2020
Rochwerg et al,32 2020
Rajagopal et al,21 2020
Abu-Raya et al,38 2020

Alhazzani et al,15 2020
Andrejak et al,41 2020

Ramírez et al,22 2020
Chekkal et al,36 2020

Ferreira et al,35 2020
Moores et al,27 2020
Thachil et al,16 2020

Chawla et al,18 2020
Barnes et al,24 2020

Flisiak et al,37 2020
Kosior et al,29 2020

Mehta et al,39 2020
Mehta et al,19 2020

Reiter et al,23 2020
Berlit et al,42 2020

Zhai et al,20 2020
Llau et al,25 2020

Foti et al,40 2020
Jin et al,34 2020
Jin et al,13 2020
Li et al,30 2021
Disclosure of guideline
Y Y Y Y Y Y Y Y Y Y N Y N Y N Y Y N Y N Y Y Y N N N N N N N N N
funding sources
Disclosure and management
5 5 3 5 5 5 5 2 3 4 3 2 5 3 5 3 2 5 3 5 3 4 1 3 1 2 2 1 2 2 2 1
of financial conflict of interests
Multidisciplinary group Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N Y Y Y N N N Y N N N Y N N N N
Methodologist involvement Y Y Y N Y N N N N Y N N N N N N N N UNK N N N N N N N N N N N N N
Patient and public perspectives 5 5 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Search strategy 5 5 5 4 2 5 1 4 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1
Study selection 5 5 4 5 2 3 1 4 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Synthesis of evidence 5 5 5 5 4 2 3 1 3 1 1 2 1 1 1 1 1 1 1 1 2 1 1 1 1 2 1 1 2 1 1 1
Grading the quality or strength
5 5 4 5 5 5 2 2 3 2 2 1 3 1 3 2 1 3 1 1 1 1 2 1 1 1 1 1 1 1 1 1
of evidence
Benefits and harms
5 5 5 5 5 3 5 2 3 2 3 3 2 2 1 2 3 1 1 3 2 3 2 2 3 2 2 2 2 2 2 1
of recommendations
Evidence summary supporting
5 5 5 5 5 3 4 1 3 2 2 2 3 2 3 2 3 2 1 2 2 2 2 1 2 3 2 2 2 2 1 1
recommendations
Rating the strength
5 5 4 5 5 3 2 2 2 2 2 2 2 1 2 3 3 2 2 2 2 2 1 1 3 3 2 1 1 1 2 1
of recommendations
Specific and unambiguous
articulation of recommendations 5 5 5 4 5 4 5 5 2 2 3 4 3 4 3 4 4 3 3 2 3 2 4 3 3 3 3 2 4 2 2 1
ExternaI review 5 5 4 1 2 3 2 1 1 1 2 1 1 1 1 1 1 1 2 1 1 1 2 1 2 1 1 1 1 1 1 1
Updating 5 5 2 2 2 1 1 2 2 1 2 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1 2 1 1 2 1 1
The heat map depicts clinical practice guideline (CPG) quality. The CPGs are ordered from Standards [NEATS] questions) and 0 for no (N) or unknown (UKN) responses (2 NEATS
highest quality to lowest quality (left to right). We assigned a score of 5 for yes (Y) questions).
responses (3 National Guideline Clearinghouse Extent of Adherence to Trustworthy

individuals could be integrated into the CPG development group or engaged in other ways or at
various points in CPG development.
Inclusion of a Systematic Review of the Evidence

Six CPGs (18.8%)18,20,27,32,34,44 adhered (score of 4 or 5) to the requirement to describe their search
strategy in detail, including a listing of the databases searched, summary of the search terms used,
and the start and end date covered by the search. Five CPGs (15.6%)18,27,32,34,44 adhered (score of 4
or 5) to the requirement to describe the study selection, including the number of studies identified
and a summary of inclusion and exclusion criteria. Five CPGs (15.6%)15,27,32,34,44 adhered (score of 4
or 5) to the requirement to provide a synthesis of the evidence from the selected studies in the form
of a detailed description of the body of evidence or as evidence tables or both.
Table 1. CPG Authorship by WHO Regiona
WHO region
Source Africa Americas Southeast Asia Europe Eastern Mediterranean Western Pacific
Jin et al,13 2020 No Yes No No No Yes
National Health Commission of the No No No No No Yes
People's Republic of China,14 2020
Alhazzani et al,15 2020 No Yes No Yes Yes Yes
Thachil et al,16 2020 No Yes No Yes No Yes
Flisiak et al,17 2020 No No No Yes No No
Chawla et al,18 2020 No No Yes No No No
Mehta et al,19 2020 No No Yes No No No
Zhai et al,20 2020 No No No Yes No Yes
Rajagopal et al,21 2020 No Yes No Yes No No
Ramírez et al,22 2020 No No No Yes No No
Reiter et al,23 2020 No Yes No Yes No No
Barnes et al,24 2020 No Yes No No No No
Llau et al,25 2020 No No No Yes No No
Lombardy Section Italian Society No No No Yes No No
Infectious and Tropical Diseases,26 2020
Moores et al,27 2020 No Yes No Yes No No
Flisiak et al,28 2020b No No No Yes No No
Kosior et al,29 2020 No No No Yes No No
Li et al,30 2021 No No No No No Yes
Henderson et al,31 2020 No Yes No No No No
Rochwerg et al,32 2020 Yes Yes No Yes No Yes
Goldenberg et al,33 2020 No Yes Yes Yes Yes Yes
Jin et al,34 2020 No Yes No No No Yes
Ferreira et al,35 2020 No Yes No No No No
Chekkal et al,36 2020 Yes No No No No No
Flisiak et al,37 2020 No No No Yes No No
Abu-Raya et al,38 2020 No Yes No Yes No Yes
Mehta et al,39 2020 No No Yes No No No
Foti et al,40 2020 No No No Yes No No
Andrejak et al,41 2021 No No No Yes No No
Berlit et al,42 2020 No No No Yes No No
Henderson et al,43 2021 No Yes No No No No
Rochwerg et al,44 2020 Yes Yes Yes Yes Yes Yes
Abbreviations: CPG, clinical practice guideline; WHO, World Health Organization.

a
CPGs are presented in chronologic order based on first date of publication online or
in print.

Recommendations
Six CPGs (18.8%)15,20,27,32,34,44 adhered (score of 4 or 5) to the requirement to provide a grade or
rating of the level of confidence or certainty in the quality or strength of the evidence underpinning
each recommendation. Similarly, 6 CPGs (18.8%)15,23,27,32,34,44 adhered (all with a score of 5) to the
requirement to provide a clear description of the potential benefits and harms and link this
information to specific recommendations. The same 6 CPGs15,23,27,32,34,44 adhered (score of 4 or 5)
to the requirement to have an explicit link to a summary of the relevant evidence and link this
information directly to recommendations. Five CPGs (15.6%)15,27,32,34,44 adhered (score of 4 or 5) to
the standard to rate the strength of the recommendations (strong or conditional/weak) based on a
clear and well-described evidence-to-recommendation scheme that took into account the balance
between benefits and harms, available evidence, and their confidence in the underlying evidence
(quantity, quality, and consistency). Fourteen CPGs (43.8%)15,18,20,22-24,26,27,30,32,34,36,39,44 adhered
(score of 4 or 5) to the requirement to make specific and unambiguous recommendations that stated
which actions should or should not be taken in specific situations or populations, and, where
recommendations were vague or underspecified, clearly described the rationale for making
recommendations.
External Review and Plans for Updating

Only 3 CPGs (9.4%)32,34,44 adhered (score of 4 or 5) to the requirement to describe an external
review process by specifying (name and description) relevant stakeholders (ie, scientific and clinical
experts, organizations, agencies, patients, and representatives) and a process for external review.
Only 2 CPGs (6.3%; both with a score of 5)32,44 had a prespecified procedure to update the CPG that
included the time frame for updating, the process by which a decision would be made to update the
CPG, and a description of how the update would be conducted.
We depict the total NEATS score for each included CPG in a heat map in Figure 2. Common
features of the highest-quality CPGs (n = 5)15,27,32,34,44 were that they were multidisciplinary and
included collaborators from at least 2 WHO regions. Four of these 5 highest-quality CPGs15,32,34,44
included a methodologist in their guideline panel.
Direction of Recommendations for Pharmaceutical Interventions

Table 2 depicts the evolution and direction of recommendations over time for each pharmacologic
intervention reported by 3 or more CPGs in chronologic order. Clinical practice guidelines
consistently recommended or suggested use of supportive (ie, vasopressors, inotropes) and
prophylactic treatments (venous thromboembolism or deep venous thrombosis prophylaxis,
histamine receptor antagonists, or proton pump inhibitors) for hospitalized patients with COVID-19.
Notwithstanding, we noted relatively inconsistent recommendations for most pharmacologic
treatments identified (empirical antibiotics, azithromycin, corticosteroids, hydroxychloroquine or
chloroquine, lopinavir or ritonavir, remdesivir, tocilizumab, interferon, favipiravir, and oseltamivir) in
the included CPGs.
Clinical practice guideline recommendations evolved during the period of our review to
recommend or suggest the use of corticosteroids for hospitalized patients with COVID-19 (Table 2).
Conversely, CPGs evolved from largely recommending or suggesting use of hydroxychloroquine or
chloroquine, lopinavir or ritonavir, remdesivir, and tocilizumab to recommending or suggesting
against their use for hospitalized patients with COVID-19 during the period covered by our review.
Discussion
In this systematic review of CPGs evaluating pharmacologic treatments for hospitalized patients with
COVID-19, we found that few CPGs met the NAM standards for trustworthiness as assessed by the
NEATS instrument.1,11 Although nearly two-thirds of CPGs included multidisciplinary guideline panels,
fewer than 20% of CPG panels included a methodologist. Only 37.5% of CPGs had a detailed

Table 2. Direction of Recommendations for Pharmacologic Treatments of Hospitalized Patients With COVID-19a
Histamine Hydroxy-
VTE/DVT Empirical receptor Vasopressors/ Oral or IV chloroquine/ Lopinavir/
Source prophylaxis antibiotics Azithromycin antagonist/PPI inotropes corticosteroids choloroquine ritonavir Remdesivir Tocilizumab Interferon Favipiravir Oseltamivir
Jin et al,13 2020 R NR NR R R R NR R NR NR R NR NR
National Health Commission of the NR NR NR NR NR R R R NR R R NR NR
People's Republic of China,14 2020
Alhazzani et al,15 2020 NR R NR NR R R/NR/Xb NR X NR NR NR NR NR
Thachil et al,16 2020 R NR NR NR NR NR NR NR NR NR NR NR NR
Flisiak et al,17 2020 NR NR NR NR NR R R R R R NR R R
Chawla et al,18 2020 NR NR NR NR NR X X NR NR NR NR NR NR
Mehta et al,19 2020 R R NR R R X R R R NR NR NR R
Zhai et al,20 2020 R NR NR NR NR NR NR NR NR NR NR NR NR
Rajagopal et al,21 2020 R NR NR NR NR NR R NR R NR NR NR NR
Ramírez et al,222020 R NR NR NR NR NR NR NR NR NR NR NR NR
Reiter et al,23 2020 NR NR NR NR NR NR R NR NR NR NR NR NR
Barnes et al,24 2020 R NR NR NR NR NR NR NR NR NR NR NR NR
Llau et al,25 2020 R NR NR NR NR NR NR NR NR NR NR NR NR
JAMA Network Open | Medical Journals and Publishing
Lombardy Section Italian Society NR NR NR NR NR R R R R R NR NR R

Infectious and Tropical Diseases,26 2020
Moores et al,27 2020 R NR R NR NR NR NR NR NR NR NR NR NR

Flisiak et al,28 2020 R R X NR NR R R R R R NR X X
Kosior et al,29 2020 R NR NR NR NR NR NR NR NR NR NR NR NR
Li et al,30 2021 R NR NR NR NR NR NR NR NR NR NR NR NR
Henderson et al,31 2020 R NR NR NR NR R NR NR NR R NR NR NR
Rochwerg et al,32 2020 NR NR NR NR NR NR NR NR R NR NR NR NR
Goldenberg et al,33 2020 R NR NR NR NR NR NR NR NR NR NR NR NR
JAMA Network Open. 2021;4(12):e2136263. doi:10.1001/jamanetworkopen.2021.36263 (Reprinted)

Jin et al,34 2020 R NR NR NR R R/NR/Xc NR X R NR R R NR
Ferreira et al,35 2020 NR NR NR NR NR R NR NR NR NR NR NR NR
Chekkal et al,36 2020 R NR NR NR NR NR NR NR NR NR NR NR NR
Flisiak et al,37 2020 R X X NR NR R X X R R NR X X
Abu-Raya et al,38 2020 R NR NR NR NR R NR NR NR NR NR NR NR
Mehta et al,39 2020 R NR X NR NR R X X R X X NR NR
Foti et al,40 2020 R X NR NR R R NR NR NR NR NR NR NR
Andrejak et al,41 2021 NR NR NR NR NR R NR NR NR NR NR NR NR
Berlit et al,42 2020 NR NR NR NR NR R NR NR NR NR NR NR NR
Henderson et al,43 2021 R NR NR NR NR R NR NR NR X NR NR NR
Rochwerg et al,44 2020 NR NR NR NR NR R/NR/Xd X X X NR NR NR NR
Abbreviations: DVT, deep venous thrombosis; IV, intravenous; NR, no recommendation; PPI, proton pump corticosteroids. In adults with COVID-19 and respiratory failure (without adult respiratory distress syndrome)
inhibitor; R, recommend/suggest use for patients hospitalized or in the intensive care unit with COVID-19; VTE, undergoing mechanical ventilation, we suggest against the routine use of systemic corticosteroids.
venous thromboembolism; X, recommend/suggest against use for patients hospitalized or in the intensive care c
When the condition of patients with severe or critical COVID-19 deteriorates dramatically, low-dose
unit with COVID-19. glucocorticoids with a short course may be considered (grade 2B). We do not suggest glucocorticoids for
a
Clinical practice guidelines are presented in chronologic order based on first date of publication online or in print. patients with COVID-19 in general (grade 2B).
b d
For adults with COVID-19 and refractory shock, we suggest using low-dose corticosteroid therapy A strong recommendation for systemic corticosteroids in patients with severe and critical COVID-19. A
December 10, 2021

(shock-reversal) compared with no corticosteroids. In adults with COVID-19 and adult respiratory distress conditional recommendation against systemic corticosteroids in patients with nonsevere COVID-19.
syndrome undergoing mechanical ventilation, we suggest using systemic corticosteroids compared with no
CPGs for Pharmacologic Treatments of Hospitalized Patients With COVID-19
8/15
disclosure of actual or potential COIs. Few CPGs (6.3%) included patient and public perspectives.
Fewer than 20% of included COVID-19–related CPGs described their search strategy, a process for
study selection, or provided a synthesis of the evidence. Although nearly half of CPGs made
suggestions or recommendations for or against treatments, fewer than 20% of CPGs provided a
grade or rating of the level of confidence in or certainty with the quality or strength of the evidence,
offered a clear description of the potential benefits and harms with links to specific
recommendations, or rated the strength of the recommendations using a clear grading scheme.
Fewer than 10% of CPGs underwent external review and even fewer described a process for
updating. The overall quality of CPGs, as assessed by the NEATS score, was low. Multidisciplinary
panels that included a methodologist and collaborators from at least 2 WHO regions were features of
high-quality COVID-19 CPGs.
The rate at which CPGs pertaining to the management of COVID-19 in various settings
(outpatient, inpatient, or intensive care unit) have been published is unprecedented. During a
pandemic specifically, there is a high demand for early, systematically developed statements that
reflect best practices based on available evidence to guide the practice of health care professionals.
Nonetheless, strong methodologic standards for CPGs are essential to avoid promulgating useless or
potentially harmful treatments and wasting health care resources.64 Overall, most included CPGs in
our study failed to meet NAM standards and consequently were at increased risk of bias. Although
producing high-quality guidelines may be viewed as impractical during a pandemic, this review
identified features of high-quality COVID-19–related CPGs using the NEATS instrument. Although
most high-quality CPGs tended to be published later in the pandemic, a high-quality CPG in our
review was published in March 2020.15 Moreover, we noted that updates of CPGs published earlier
in the pandemic tended to be of higher quality than the parent documents (Figure 2). Improvement
of CPG quality over time may reflect accumulating knowledge, clinical experience, or lead-
time bias.
Our findings align with other assessments of nonpandemic and pandemic CPG quality.12,64-66
From 130 randomly selected CPGs from the National Guideline Clearinghouse, Kung et al65 found
that the median number of NAM standards satisfied was 8 of 18 (44.4% [IQR, 36.1%-52.8%]). The
authors noted that fewer than half of their included CPGs and one-third of CPGs produced by
subspecialty societies met more than 50% of the NAM standards.65 Similar to our study, others have
shown that fewer than half of CPGs provided information regarding COIs,12,65 few CPGs included
patients or patient representatives,12,64-66 and the CPGs rarely included a process for updating.64,66
The present review adds to the literature by documenting that fewer than 20% of CPGs included a
systematic review or adhered to the International Organization for Standardization to generate
recommendations for care. Although several CPGs in our review (14 of 32 [43.8%]) made suggestions
or recommendations for or against treatments, they infrequently provided a grade or rating of the
level of confidence or certainty regarding the quality or strength of the evidence (6 of 32 [18.8%]),
offered a clear description of the potential benefits and harms (6 of 32 [18.8%]), or rated the strength
of the recommendations using a clear grading scheme (5 of 32 [15.6%]). As such, the guidance
statements from most CPGs included in our review were not optimally informed by the key
dimensions of evidence on pharmacologic interventions for COVID-19. Contrary to a review of
oncology CPGs,66 most CPGs in our review did not undergo external peer review. Similar to our study,
an earlier review of 19 COVID-19–specific CPGs12 found that the overall quality of CPGs was poor;
lacked detail; had inconsistent recommendations, even for the same intervention; and did not
provide explicit linkage between the evidence and generating recommendations. Recently, Stamm
et al64 evaluated the quality of 188 general COVID-19 CPGs published from February 1 to April 27,
2020, using the AGREE II tool. The CPGs in this review were largely (83%) based on informal
consensus without clear evidence summaries and scored highest for scope and purpose (89%) and
lowest (25%) for rigor of development. The latter finding may relate to the paucity of evidence
available early in the pandemic. Unlike previous COVID-19 CPG reviews,12,64 we limited our review to
pharmacologic treatments for COVID-19, included CPGs with broad potential reach (authored by

societies and government or nongovernment organizations), and appraised quality using the NEATS
(vs AGREE II) instrument. Taken together, systematic reviews of CPG quality have identified that
most CPGs were of low overall methodologic quality and tended to make recommendations that
promoted more interventions as opposed to more effective interventions.
Instruments that appraise CPG quality provide stakeholders with a metric to evaluate and select
the most rigorously developed CPGs with the goal of improving patient care, safety, and outcomes.
The AGREE II checklist focuses on assessment of the quality and reporting of CPGs in 6 domains
(scope and purpose, stakeholder involvement, rigor of development, clarity of presentation,
applicability, and editorial independence) but does not address the clinical validity of CPG
recommendations. By contrast, the 15-item NEATS instrument assesses adherence to NAM
standards. The NEATS tool has been shown to have high interrater reliability (weighted κ = 0.73) and
external validity.11 To ensure consistency and reliability of judgments, 2 trained personnel assess each
CPG using the NEATS tool at the National Guideline Clearinghouse. Subsequently, these assessments
are shared with CPG developers to enhance the accuracy and completeness of NEATS quality
summaries. This feedback loop provides guideline developers with a benchmark to compare their
processes against the NAM standards and an opportunity to clarify their methods.67 Several
authors64,68 have noted that the additional rigor required to adhere to these standards may come at
the cost of increased complexity, expertise, money, and time to CPG completion, most of which are
in short supply during a pandemic. Future research is needed to compare appraisal tools, understand
how to create CPGs that are ready for implementation, and aid stakeholders (clinicians, patients, and
the public) to be informed CPG consumers.
Several strategies might enhance the development of trustworthy CPGs, even in the setting of
a pandemic. First, CPG panels should include participation of a methodologic expert (eg, an
epidemiologist, biostatistician, health services researcher). Their expertise adds to decisions
regarding study design and the potential for bias and influence on study findings, methods to
minimize bias in the conduct of systematic reviews, use of quantitative methods, conduct of
qualitative synthesis, and issues related to data collection and management.11 Second, approaches
that prioritize broad collaborations that engage multidisciplinary stakeholders who work together
and share expertise and resources and are from at least 2 WHO regions may be optimal and lead to
the production of fewer but higher-quality CPGs that are poised for updates as new evidence
emerges. This approach could not only limit duplication of efforts but also limit publication of
inconsistent recommendations. As opposed to de novo CPG development, local and regional groups
should consider appraising and adapting existing high-quality CPGs to their practice context using
the ADAPTE process.69 This 3-stage process includes start-up (assessment of skills and resources
required), adaptation (selection of specific questions and CPG retrieval, quality assessment,
selection, and compilation), and end stage (seeking opinions of decision makers affected by CPG,
CPG revision, and finalization).69 Inherent to the ADAPTE process is access to CPGs and availability
of local expertise in CPG appraisal. Adaptation to the clinical context is an important consideration,
because most CPGs in our review were sponsored by societies with infrastructure and expertise and
few were developed in low- and middle-income countries. Third, journal editors and peer reviewers
should mandate use of 1 or more CPG appraisal tools at the time of manuscript submission to ensure
publication of high-quality and trustworthy CPGs.
Strengths and Limitations

Our review has several strengths. Unlike prior reviews of COVID-19–related CPGs, we limited our
review to pharmacologic treatments for hospitalized patients with COVID-19, included CPGs with
broad potential reach (sponsored by societies and government or nongovernment organizations),
and appraised CPG quality using the NEATS (vs AGREE II) instrument. To our knowledge, this is the
first report to use the NEATS instrument to appraise CPG quality outside of the National Guideline
Clearinghouse. We prioritized use of the NEATS tool because it had undergone rigorous development
and testing and is aligned with NAM standards for trustworthy CPGs. We performed a

comprehensive search and reviewed citations, abstracted data, and assessed quality in duplicate to
limit ascertainment bias.
Our review also has some limitations. First, we limited our search to CPGs that were published
in peer-reviewed journals. Second, we did not contact CPG authors to verify the methodologic
aspects of their respective guidelines. Consequently, our assessment of methodologic expertise may
be an underestimate, limited by reporting of this information in CPGs. Third, we did not have specific
information pertaining to whether the included CPGs underwent peer review (regular, expedited,
or absent) or were appraised using a quality checklist or other tool by authors at the time of
submission. Notwithstanding, these points highlight the need for high publication standards even in
the unique circumstances posed by a pandemic.
Conclusions
Few COVID-19 CPGs met NAM standards for trustworthy guidelines. Approaches that prioritize engage-
ment of a methodologist and multidisciplinary collaborators from at least 2 WHO regions may lead to
the production of fewer, high-quality CPGs that are poised for updates as new evidence emerges.
ARTICLE INFORMATION
Accepted for Publication: September 14, 2021.
Published: December 10, 2021. doi:10.1001/jamanetworkopen.2021.36263
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Burns KEA
et al. JAMA Network Open.
Corresponding Author: Karen E. A. Burns, MD, MSc (Epid), Unity Health Toronto, St Michael’s Hospital, 30 Bond
St, 4-045 Donnelly Wing, Toronto, Ontario, Canada M5B 1W8 (karen.burns@unityhealth.to).
Author Affiliations: Interdepartmental Division of Critical Care Medicine, Department of Medicine, Temerty
Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada (Burns, Granton, Friedrich, Agarwal);
Departments of Critical Care and Medicine, Unity Health Toronto, St Michael’s Hospital, Toronto, Ontario, Canada
(Burns, Friedrich); Departments of Medicine, Critical Care Medicine, Pediatrics and Health Research Methods,
Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada (Burns, Kho, Cook, Meade, Duffett,
Chaudhuri, Agarwal, Rochwerg); Li Ka Shing Knowledge Institute, St Michael’s Hospital, Toronto, Ontario, Canada
(Burns, Friedrich); School of Medicine, Royal College of Surgeons, Dublin, Ireland (Laird, Stevenson); Department
of Critical Care Medicine, London Health Sciences Centre, London, Ontario, Canada (Honarmand); Department of
Medicine, Western University, London, Ontario, Canada (Honarmand); Physiotherapy and Division of Critical Care,
St Joseph’s Healthcare, Hamilton, Ontario, Canada (Kho); School of Rehabilitation Science, Faculty of Health
Science, McMaster University, Hamilton, Ontario, Canada (Kho); Hamilton Health Sciences, Hamilton, Ontario,
Canada (Meade); Dalla Lana School of Public Health and the Institute of Health Policy, Management, and
Evaluation, University of Toronto, Toronto, Ontario, Canada (Liu, Adhikari); Canadian Donation and Transplant
Research Program, Ottawa, Ontario, Canada (D’Aragon); Department of Anesthesiology, Université de Sherbrooke,
Sherbrooke, Quebec, Canada (D’Aragon); Department of Critical Care Medicine, Sunnybrook Health Sciences
Centre, Toronto, Ontario, Canada (Adhikari).
Author Contributions: Dr Burns had full access to all the data in the study and takes responsibility for the integrity
of the data and the accuracy of the data analysis.
Concept and design: Burns, Laird, Stevenson, Kho, Friedrich, Meade, Duffett, Adhikari, Noh, Rochwerg.
Acquisition, analysis, or interpretation of data: Burns, Laird, Stevenson, Honarmand, Granton, Kho, Cook, Friedrich,
Meade, Duffett, Chaudhuri, Liu, D’Aragon, Agarwal, Adhikari, Rochwerg.
Drafting of the manuscript: Burns, Laird, Stevenson, Noh.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Burns, Duffett, Liu, Rochwerg.
Administrative, technical, or material support: Burns, Laird, Stevenson, Cook, Friedrich, Chaudhuri, Agarwal,
Rochwerg.
Supervision: Burns, Meade, Rochwerg.

Conflict of Interest Disclosures: Dr Burns reported holding a career award from the Physician Services
Incorporated Foundation. Dr Kho reported receiving grants from Canada Research Chairs outside the submitted
work. Dr Adhikari reported serving as chair for COVID-19 guideline panels convened by the World Health
Organization. No other disclosures were reported.
Group Information: The Academy of Critical Care: Development, Evaluation, and Methodology (ACCADEMY)
collaborators are listed in Supplement 2.
Additional Contributions: David Lightfoot, MISt, Unity Health Toronto, St Michael’s Hospital, assisted in
conducting the literature searches. Mr Lightfoot did not receive remuneration for his assistance.
REFERENCES
1. Institute of Medicine. Clinical Practice Guidelines We Can Trust. National Academies Press; 2011:1-300.
2. Lohr KN, Field MJ. A provisional instrument for assessing clinical practice guidelines (Appendix B). In: Fields MJ,
Lohr KN, eds. Guidelines for Clinical Practice: From Development to Use. National Academies Press; 1992.
3. Hayward RS, Wilson MC, Tunis SR, Bass EB, Rubin HR, Haynes RB. More informative abstracts of articles
describing clinical practice guidelines. Ann Intern Med. 1993;118(9):731-737. doi:10.7326/0003-4819-118-9-
199305010-00012
4. Liddle J, Williamson M, Irwig L. Method for Evaluating Research Guideline Evidence. New South Wales Health
Department; 1996.
5. Cluzeau FA, Littlejohns P, Grimshaw JM, Feder G, Moran SE. Development and application of a generic
methodology to assess the quality of clinical guidelines. Int J Qual Health Care. 1999;11(1):21-28. doi:10.1093/
intqhc/11.1.21
6. Cluzeau F, Burgers J, Brouwers M, et al; AGREE Collaboration. Development and validation of an international
appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health
Care. 2003;12(1):18-23. doi:10.1136/qhc.12.1.18
7. Fervers B, Burgers JS, Haugh MC, et al. Predictors of high quality clinical practice guidelines: examples in
oncology. Int J Qual Health Care. 2005;17(2):123-132. doi:10.1093/intqhc/mzi011
8. Brouwers MC, Kho ME, Browman GP, et al; AGREE Next Steps Consortium. AGREE II: advancing guideline
development, reporting and evaluation in health care. CMAJ. 2010;182(18):E839-E842. doi:10.1503/cmaj.
090449
9. Brouwers MC, Makarski J, Kastner M, Hayden L, Bhattacharyya O; GUIDE-M Research Team. The Guideline
Implementability Decision Excellence Model (GUIDE-M): a mixed methods approach to create an international
resource to advance the practice guideline field. Implement Sci. 2015;10:36. doi:10.1186/s13012-015-0225-1
10. Brouwers MC, Spithoff K, Kerkvliet K, et al. Development and validation of a tool to assess the quality of clinical
practice guideline recommendations. JAMA Netw Open. 2020;3(5):e205535. doi:10.1001/jamanetworkopen.
2020.5535
11. Jue JJ, Cunningham S, Lohr K, et al. Developing and testing the Agency for Healthcare Research and Quality’s
National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) instrument. Ann Intern
Med. 2019;170(7):480-487. doi:10.7326/M18-2950
12. Dagens A, Sigfrid L, Cai E, et al. Scope, quality, and inclusivity of clinical guidelines produced early in the
COVID-19 pandemic: rapid review. BMJ. 2020;369:m1936. doi:10.1136/bmj.m1936
13. Jin YH, Cai L, Cheng ZS, et al; Zhongnan Hospital of Wuhan University Novel Coronavirus Management and
Research Team, Evidence-Based Medicine Chapter of China International Exchange and Promotive Association for
Medical and Health Care (CPAM). A rapid advice guideline for the diagnosis and treatment of 2019 novel
coronavirus (2019-nCoV) infected pneumonia (standard version). Mil Med Res. 2020;7(1):4. doi:10.1186/s40779-
020-0233-6
14. National Health Commission of the People’s Republic of China. Diagnostic and treatment protocol using
traditional Chinese medicine. Int J Acupuncture. 2020;14(1):7-12. doi:10.1016/j.acu.2020.04.002
15. Alhazzani W, Møller MH, Arabi YM, et al. Surviving sepsis campaign: guidelines on the management of critically
ill adults with coronavirus disease 2019 (COVID-19). Crit Care Med. 2020;48(6):e440-e469. doi:10.1097/CCM.
0000000000004363
16. Thachil J, Tang N, Gando S, et al. ISTH interim guidance on recognition and management of coagulopathy in
COVID-19. J Thromb Haemost. 2020;18(5):1023-1026. doi:10.1111/jth.14810
17. Flisiak R, Horban A, Jaroszewicz J, et al. Management of SARS-CoV-2 infection: recommendations of the Polish
Association of Epidemiologists and Infectiologists as of March 31, 2020. Pol Arch Intern Med. 2020;130(4):
352-357. doi:10.20452/pamw.15270

18. Chawla D, Chirla D, Dalwai S, et al; Federation of Obstetric and Gynaecological Societies of India (FOGSI),
National Neonatology Forum of India (NNF) and Indian Academy of Pediatrics (IAP). Perinatal-neonatal
management of COVID-19 infection: guidelines of the Federation of Obstetric and Gynaecological Societies of India
(FOGSI), National Neonatology Forum of India (NNF), and Indian Academy of Pediatrics (IAP). Indian Pediatr.
2020;57(6):536-548. doi:10.1007/s13312-020-1852-4
19. Mehta Y, Chaudhry D, Abraham OC, et al. Critical care for COVID-19 affected patients: position statement of the
Indian Society of Critical Care Medicine. Indian J Crit Care Med. 2020;24(4):222-241. doi:10.5005/jp-journals-
10071-23395
20. Zhai Z, Li C, Chen Y, et al; Prevention Treatment of VTE Associated with COVID-19 Infection Consensus
Statement Group. Prevention and treatment of venous thromboembolism associated with coronavirus disease
2019 infection: a consensus statement before guidelines. Thromb Haemost. 2020;120(6):937-948. doi:10.1055/s-
0040-1710019
21. Rajagopal K, Keller SP, Akkanti B, et al. Advanced pulmonary and cardiac support of COVID-19 patients:
emerging recommendations from ASAIO—a living working document. Circ Heart Fail. 2020;13(5):e007175. doi:10.
1161/CIRCHEARTFAILURE.120.007175
22. Ramírez I, De la Viuda E, Baquedano L, et al. Managing thromboembolic risk with menopausal hormone
therapy and hormonal contraception in the COVID-19 pandemic: recommendations from the Spanish Menopause
Society, Sociedad Española de Ginecología y Obstetricia and Sociedad Española de Trombosis y Hemostasia.
Maturitas. 2020;137:57-62. doi:10.1016/j.maturitas.2020.04.019
23. Reiter RJ, Abreu-Gonzalez P, Marik PE, Dominguez-Rodriguez A. Therapeutic algorithm for use of melatonin in
patients with COVID-19. Front Med (Lausanne). 2020;7:226. doi:10.3389/fmed.2020.00226
24. Barnes GD, Burnett A, Allen A, et al. Thromboembolism and anticoagulant therapy during the COVID-19
pandemic: interim clinical guidance from the anticoagulation forum. J Thromb Thrombolysis. 2020;50(1):72-81.
doi:10.1007/s11239-020-02138-z
25. Llau JV, Ferrandis R, Sierra P, et al. SEDAR-SEMICYUC consensus recommendations on the management of
haemostasis disorders in severely ill patients with COVID-19 infection. Rev Esp Anestesiol Reanim (Engl Ed). 2020;
67(7):391-399. doi:10.1016/j.redar.2020.05.007
26. Lombardy Section Italian Society Infectious and Tropical Diseases. Vademecum for the treatment of people
with COVID-19. Edition 2.0, 13 March 2020. Infez Med. 2020;28(2):143-152.
27. Moores LK, Tritschler T, Brosnahan S, et al. Prevention, diagnosis, and treatment of VTE in patients with
coronavirus disease 2019: CHEST Guideline and Expert Panel Report. Chest. 2020;158(3):1143-1163. doi:10.1016/j.
chest.2020.05.559
28. Flisiak R, Horban A, Jaroszewicz J, et al. Management of SARS-CoV-2 infection: recommendations of the Polish
Association of Epidemiologists and Infectiologists: annex No. 1 as of June 8, 2020. Pol Arch Intern Med. 2020;130
(6):557-558. doi:10.20452/pamw.15424
29. Kosior DA, Undas A, Kopeć G, et al. Guidance for anticoagulation management in venous thromboembolism
during the coronavirus disease 2019 pandemic in Poland: an expert opinion of the Section on Pulmonary
Circulation of the Polish Cardiac Society. Kardiol Pol. 2020;78(6):642-646. doi:10.33963/KP.15425
30. Li YH, Wang MT, Huang WC, Hwang JJ. Management of acute coronary syndrome in patients with suspected
or confirmed coronavirus disease 2019: consensus from Taiwan Society of Cardiology. J Formos Med Assoc.
2021;120(1 Pt 1):78-82. doi:10.1016/j.jfma.2020.07.017
31. Henderson LA, Canna SW, Friedman KG, et al. American College of Rheumatology clinical guidance for
multisystem inflammatory syndrome in children associated with SARS-CoV-2 and hyperinflammation in pediatric
COVID-19: version 1. Arthritis Rheumatol. 2020;72(11):1791-1805. doi:10.1002/art.41454
32. Rochwerg B, Agarwal A, Zeng L, et al. Remdesivir for severe COVID-19: a clinical practice guideline. BMJ.
2020;370:m2924. doi:10.1136/bmj.m2924
33. Goldenberg NA, Sochet A, Albisetti M, et al; Pediatric/Neonatal Hemostasis and Thrombosis Subcommittee of
the ISTH SSC. Consensus-based clinical recommendations and research priorities for anticoagulant
thromboprophylaxis in children hospitalized for COVID-19-related illness. J Thromb Haemost. 2020;18(11):
3099-3105. doi:10.1111/jth.15073
34. Jin YH, Zhan QY, Peng ZY, et al; Evidence-Based Medicine Chapter of China International Exchange and
Promotive Association for Medical and Health Care (CPAM); Chinese Research Hospital Association (CRHA).
Chemoprophylaxis, diagnosis, treatments, and discharge management of COVID-19: an evidence-based clinical
practice guideline (updated version). Mil Med Res. 2020;7(1):41. doi:10.1186/s40779-020-00270-8

35. Ferreira LL, Sampaio DL, Chagas ACP, et al. AMB guidelines: COVID-19. Rev Assoc Med Bras 2020. Rev. Assoc.
Med. Bras. 2020;66(suppl 2):17-21. doi:10.1590/1806-9282.66.S2.17
36. Chekkal M, Deba T, Hadjali S, et al. Prevention and treatment of COVID-19-associated hypercoagulability:
recommendations of the Algerian Society of Transfusion and Hemobiology. Transfus Clin Biol. 2020;27(4):
203-206. doi:10.1016/j.tracli.2020.09.004
37. Flisiak R, Parczewski M, Horban A, et al. Management of SARS-CoV-2 infection: recommendations of the Polish
Association of Epidemiologists and Infectiologists. Annex no. 2 as of October 13, 2020. Pol Arch Intern Med. 2020;
130(10):915-918. doi:10.20452/pamw.15658
38. Abu-Raya B, Migliori GB, O’Ryan M, et al. Coronavirus disease-19: an interim evidence synthesis of the World
Association for Infectious Diseases and Immunological Disorders (WAIDID). Front Med (Lausanne). 2020;7:
572485. doi:10.3389/fmed.2020.572485
39. Mehta Y, Chaudhry D, Abraham OC, et al. Critical care for COVID-19 affected patients: updated position
statement of the Indian Society of Critical Care Medicine. Indian J Crit Care Med. 2020;24(suppl 5):S225-S230.
doi:10.5005/jp-journals-10071-23621
40. Foti G, Giannini A, Bottino N, et al; COVID-19 Lombardy ICU Network. Management of critically ill patients with
COVID-19: suggestions and instructions from the coordination of intensive care units of Lombardy. Minerva
Anestesiol. 2020;86(11):1234-1245. doi:10.23736/S0375-9393.20.14762-X
41. Andrejak C, Cottin V, Crestani B, et al. Guide for the management of patients with respiratory sequelae after a
SARS-CoV-2 pneumonia: support proposals developed by the French-language Respiratory Medicine Society:
version of 10 November 2020 [in French]. Rev Mal Respir. 2021;38(1):114-121. doi:10.1016/j.rmr.2020.11.009
42. Berlit P, Bösel J, Gahn G, et al. “Neurological manifestations of COVID-19”—guideline of the German Society of
Neurology. Neurol Res Pract. 2020;2:51. doi:10.1186/s42466-020-00097-7
43. Henderson LA, Canna SW, Friedman KG, et al. American College of Rheumatology clinical guidance for
multisystem inflammatory syndrome in children associated with SARS-CoV-2 and hyperinflammation in pediatric
COVID-19: version 2. Arthritis Rheumatol. 2021;73(4):e13-e29. doi:10.1002/art.41616
44. Rochwerg B, Agarwal A, Siemieniuk RA, et al. A living WHO guideline on drugs for COVID-19. BMJ. 2020;
370:m3379. doi:10.1136/bmj.m3379
45. Chen X, Liu Y, Gong Y, et al; Chinese Society of Anesthesiology, Chinese Association of Anesthesiologists.
Perioperative management of patients infected with the novel coronavirus: recommendation from the Joint Task
Force of the Chinese Society of Anesthesiology and the Chinese Association of Anesthesiologists. Anesthesiology.
2020;132(6):1307-1316. doi:10.1097/ALN.0000000000003301
46. Andrejak C, Blanc FX, Costes F, et al. Guide for follow-up of patients with SARS-CoV-2 pneumonia:
management proposals developed by the French-language Respiratory Medicine Society. Version of 10 May 2020
[in French]. Rev Mal Respir. 2020;37(6):505-510. doi:10.1016/j.rmr.2020.05.001
47. Donders F, Lonnée-Hoffmann R, Tsiakalos A, et al; Isidog Covid-Guideline Workgroup. ISIDOG COVID-
Guideline Workgroup. ISIDOG recommendations concerning COVID-19 and pregnancy. Diagnostics (Basel). 2020;
10(4):243. doi:10.3390/diagnostics10040243
48. Tan SHS, Hong CC, Saha S, Murphy D, Hui JH. Medications in COVID-19 patients: summarizing the current
literature from an orthopaedic perspective. Int Orthop. 2020;44(8):1599-1603. doi:10.1007/s00264-020-
04643-5
49. Zhang Y, Coats AJS, Zheng Z, et al. Management of heart failure patients with COVID-19: a joint position paper
of the Chinese Heart Failure Association & National Heart Failure Committee and the Heart Failure Association of
the European Society of Cardiology. Eur J Heart Fail. 2020;22(6):941-956. doi:10.1002/ejhf.1915
50. Solé G, Salort-Campana E, Pereon Y, et al; FILNEMUS COVID-19 study group. Guidance for the care of
neuromuscular patients during the COVID-19 pandemic outbreak from the French Rare Health Care for
Neuromuscular Diseases Network. Rev Neurol (Paris). 2020;176(6):507-515. doi:10.1016/j.neurol.2020.04.004
51. Cardenas MC, Bustos SS, Enninga EAL, Mofenson L, Chakraborty R. Characterising and managing paediatric
SARSCoV-2 infection: learning about the virus in a global classroom. Acta Paediatr. 2021;110(2):409-422. doi:10.
1111/apa.15662
52. Miklowski M, Jansen B, Auron M, Whinney C. The hospitalized patient with COVID-19 on the medical ward:
Cleveland Clinic approach to management. Cleve Clin J Med. Published online November 3, 2020. doi:10.3949/
ccjm.87a.ccc064
53. Yang X, Liu Y, Liu Y, et al. Medication therapy strategies for the coronavirus disease 2019 (COVID-19): recent
progress and challenges. Expert Rev Clin Pharmacol. 2020;13(9):957-975. doi:10.1080/17512433.2020.1805315

54. Baller EB, Hogan CS, Fusunyan MA, et al. Neurocovid: pharmacological recommendations for delirium
associated with COVID-19. Psychosomatics. 2020;61(6):585-596. doi:10.1016/j.psym.2020.05.013
55. Fang F, Chen Y, Zhao D, et al; Chinese Pediatric Society and the Editorial Committee of the Chinese Journal of
Pediatrics. Recommendations for the diagnosis, prevention, and control of coronavirus disease-19 in children: the
Chinese perspectives. Front Pediatr. 2020;8:553394. doi:10.3389/fped.2020.553394
56. Dobesh PP, Trujillo TC. Coagulopathy, venous thromboembolism, and anticoagulation in patients with
COVID-19. Pharmacotherapy. 2020;40(11):1130-1151. doi:10.1002/phar.2465
57. Kronbichler A, Effenberger M, Eisenhut M, Lee KH, Shin JI. Seven recommendations to rescue the patients and
reduce the mortality from COVID-19 infection: an immunological point of view. Autoimmun Rev. 2020;19(7):
102570. doi:10.1016/j.autrev.2020.102570
58. Ferreira LL, Sampaio DL, Chagas ACP, et al. AMB guidelines: COVID 19. Rev Assoc Med Bras 2020. 2020;66(9):
1179. doi:10.1590/1806-9282.66.9.1179.
59. Aguilar RB, Hardigan P, Mayi B, et al. Current understanding of COVID-19 clinical course and investigational
treatments. Front Med (Lausanne). 2020;7:555301. doi:10.3389/fmed.2020.555301
60. Costa A, Weinstein ES, Sahoo DR, Thompson SC, Faccincani R, Ragazzoni L. How to build the plane while
flying: VTE/PE thromboprophylaxis clinical guidelines for COVID-19 patients. Disaster Med Public Health Prep.
2020;14(3):391-405. doi:10.1017/dmp.2020.195
61. Danthuluri V, Grant MB. Update and recommendations for ocular manifestations of COVID-19 in adults and
children: a narrative review. Ophthalmol Ther. 2020;9(4):853-875. doi:10.1007/s40123-020-00310-5
62. Chivukula RR, Maley JH, Dudzinski DM, Hibbert K, Hardin CC. Evidence-based management of the critically ill
adult with SARS-CoV-2 infection. J Intensive Care Med. 2021;36(1):18-41. doi:10.1177/0885066620969132
63. World Health Organization. Clinical management of severe acute respiratory infection (SARI) when COVID-19
is suspected: interim guidance. March 13, 2020. Accessed April 24, 2021. https://apps.who.int/iris/handle/
10665/331446
64. Stamm TA, Andrews MR, Mosor E, et al. The methodological quality is insufficient in clinical practice guidelines
in the context of COVID-19: systematic review. J Clin Epidemiol. 2021;135:125-135. doi:10.1016/j.jclinepi.2021.
03.005
65. Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet institute of medicine standards:
two more decades of little, if any, progress. Arch Intern Med. 2012;172(21):1628-1633. doi:10.1001/2013.
jamainternmed.56
66. Reames BN, Krell RW, Ponto SN, Wong SL. Critical evaluation of oncology clinical practice guidelines. J Clin
Oncol. 2013;31(20):2563-2568. doi:10.1200/JCO.2012.46.8371
67. Emergency Care Research Institute. US Agency for Healthcare Research and Quality launches new clinical
guideline assessment tool through contract to ECRI Institute. November 16, 2017. Accessed April 24, 2021. https://
www.ecri.org/press/ahrq-clinical-guideline-assessment-tool
68. Brouwers MC, Spithoff K, Lavis J, Kho ME, Makarski J, Florez ID. What to do with all the AGREEs? the AGREE
portfolio of tools to support the guideline enterprise. J Clin Epidemiol. 2020;125:191-197. doi:10.1016/j.jclinepi.
2020.05.025
69. Fervers B, Burgers JS, Voellinger R, et al. ADAPTE Collaboration. Guideline adaptation: an approach to
enhance efficiency in guideline development and improve utilization. BMJ Qual Saf. 2011;20(3):228-236. doi:10.
1136/bmjqs.2010.043257
SUPPLEMENT 1.
eMethods. NEATS Instrument
eFigure. Identification of Clinical Practice Guidelines (CPGs)
eTable 1. Characteristics of Included Clinical Practice Guidelines (CPGs)
eTable 2. Summary of NEATS Scores for the Included Clinical Practice Guidelines (CPGs)
SUPPLEMENT 2.
Nonauthor Collaborators. The Academy of Critical Care: Development, Evaluation, and Methodology
(ACCADEMY)

Adherence of Clinical Practice Guidelines For Pharmacologic Treatments

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Adherence of Clinical Practice Guidelines For Pharmacologic Treatments

Uploaded by

Copyright:

Available Formats

Original Investigation | Medical Journals and Publishing

Adherence of Clinical Practice Guidelines for Pharmacologic Treatments

Abstract Key Points

JAMA Network Open. 2021;4(12):e2136263. doi:10.1001/jamanetworkopen.2021.36263

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Data Sources and Searches

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

nonpharmacologic interventions, medications not dispensed by a hospital pharmacy (eg, herbal

Data Synthesis and Analysis

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Disclosure of guideline funding sources

Disclosure and management of financial

Patient and public perspectives

Grading the quality or strength of evidence

Benefits and harms of recommendations

er Rating the strength of recommendations

al , 2 Specific and unambiguous articulation

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Funding and Panel Composition

Disclosure of COIs and Inclusion of Patient and Public Perspectives

Figure 2. Heat Map Depicting Quality of Included Clinical Practice Guidelines

of the People's Republic of China et al,14 2020

and Tropical Diseases,26 2020

National Health Commission

Henderson et al,31 2020

Rajagopal et al,21 2020

Abu-Raya et al,38 2020

Andrejak et al,41 2020

Chekkal et al,36 2020

Thachil et al,16 2020

Barnes et al,24 2020

Flisiak et al,28 2020

Flisiak et al,17 2020

Kosior et al,29 2020

Mehta et al,19 2020

Berlit et al,42 2020

Llau et al,25 2020

Jin et al,13 2020

Methodologist involvement Y Y Y N Y N N N N Y N N N N N N N N UNK N N N N N N N N N N N N N

Patient and public perspectives 5 5 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Inclusion of a Systematic Review of the Evidence

Table 1. CPG Authorship by WHO Regiona

Abbreviations: CPG, clinical practice guideline; WHO, World Health Organization.

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

External Review and Plans for Updating

Direction of Recommendations for Pharmaceutical Interventions

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Lombardy Section Italian Society NR NR NR NR NR R R R R R NR NR R

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

JAMA Network Open. 2021;4(12):e2136263. doi:10.1001/jamanetworkopen.2021.36263 (Reprinted)

December 10, 2021

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Strengths and Limitations

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

Downloaded From: https://jamanetwork.com/ by Nguyen Tuan on 05/02/2022

You might also like