Online Submissions: http://www.journaltcm.
com J Tradit Chin Med 2018 June 15; 38(3): 339-350
info@journaltcm.com
Clinical practice guidelines for treating headache with Traditional
Chinese Medicine: quality assessment with the appraisal of guide-
lines for research and evaluation Ⅱ instrument
Luo Hao, Li Hui, Wang Yangyang, Yao Sha, Xu Wenjie
aa
Luo Hao, Yao Sha, School of Basic Medical Sciences, Guang-
zhou University of Chinese Medicine, Guangzhou 510000,
China
Li Hui, Wang Yangyang, Research Office of Standardiza-
tion, Second Affiliated Hospital of Guangzhou University of
Chinese Medicine, Engineering and Technology Research
Center of Standardization of Traditional Chinese Medicine,
Guangzhou 510000, China
Xu Wenjie, Scientific Research Office, Beijing Hospital of Tra-
ditional Chinese Medicine Affiliated to Capital University of
Medicine Sciences, Beijing 100000, China
Supported by the Special Research Project of Traditional
Chinese Medicine of Guangdong Hospital of Chinese Medi-
cine (No. YN2015MS22) and the Science planning project of
Guangzhou (No. 2014Y2-00040)
Correspondence to: Prof. Li Hui, Research Office of Stan-
dardization, Second Affiliated Hospital of Guangzhou Uni-
versity of Chinese Medicine, Engineering and Technology
Research Center of Standardization of Traditional Chinese
Medicine, Guangzhou 510000, China. lihuitcm@126.com
Telephone: +86-20-81887233-35945
Accepted: May 29, 2017
Abstract
OBJECTIVE: To critically appraise the methodologi-
cal quality of clinical practice guidelines for head-
ache produced over the last two decades, includ-
ing those covering specific interventions using Tra-
ditional Chinese Medicine.
METHODS: The guidelines on headache disorders
were obtained by searching a number of databas-
es, including PubMed, EMBASE, Web of Science,
Chinese Biomedical Literature Database, China Na-
tional Knowledge Infrastructure Database, China
JTCM | www. journaltcm. com
ISSN 0255-2922
© 2018 JTCM. This is an open access article under the CC BY-NC-ND license.
SYSTEMATIC REVIEW
TOPIC
Science and Technology Journal Database, and
Wanfang database, three guideline-related data-
bases [Guideline-International Network, National
Guideline Clearinghouse, and Medlive], and the re-
cords of organizations that develop guidelines. The
publication date was limited to the period from Jan-
uary 1996 to June 2015. The search terms "head-
ache", "headache disorders", "cephalalgia", "mi-
graine", "tension-type headache", "practice guide-
line", "consensus ", "statement", "regulation", and
"recommendation" were used in the "MeSH" and
"Free-text" fields. The guidelines were independent-
ly appraised by four researchers using the Apprais-
al of Guidelines for Research and Evaluation Ⅱ in-
strument.
RESULTS: A total of 23 guidelines published be-
tween 1998 and 2014 were reviewed. The overall
consistency of the four appraisers was good [inter-
class correlation coefficient 0.84; 95% confidence
interval (CI) 0.82-0.86]. The mean (standard devia-
tion) scores for scope and purpose, stakeholder in-
volvement, rigor of development, clarity of presen-
tation, applicability, and editorial independence
were 52.1 (18.0), 39.5 (17.1), 33.4 (21.0), 49.8 (21.9),
23.8 (19.3), and 24.2 (23.7). Only two guidelines
were recommended, 12 were recommended with
modification, and nine were not recommended.
CONCLUSION: Physical Traditional Chinese Medi-
cine therapies were recommended to treat head-
ache. The overall quality of headache guidelines
was low in China, but evidence-based guidelines
are gradually becoming mainstream. Guideline de-
velopers should carefully consider, in particular,
339 June 15, 2018 | Volume 38 | Issue 3 |
 Luo H et al. / Systematic Review
three domains: rigor of development, applicability,
and editorial independence.
© 2018 JTCM. This is an open access article under the
CC BY-NC-ND license.
Keywords: Practice guideline; Headache; Medicine,
Chinese Traditional; Appraisal of Guidelines for Re-
search and Evaluation Ⅱ instrument
INTRODUCTION
Headache is one of the most common neurological dis-
orders encountered by clinicians.1 According to the
World Health Organization (WHO), headaches are
among the five most common clinical disorders world-
wide. Evidence suggests that active headaches affect
more than 46% of the global population, and the life-
time prevalence of headache is more than 90%. Higher
prevalence of headaches is reportedly associated with
greater social, economic, and family burden.2-5
Traditional Chinese Medicine (TCM) has a long histo-
ry in treating headache disorders. Many treatments,
such as herbal TCM treatments and physical TCM
therapy, are effective in treating headaches and have
therefore been the focus of studies in modern medi-
cine.6 Several clinical practice guidelines (CPGs) for
treating headache with TCM have been developed.
Many studies have suggested that CPGs can improve
clinical practice and reduce healthcare costs,7,8 but sev-
eral different CPGs may be published on the same top-
ic. If these guidelines are conflicting, this may affect
the confidence of clinicians.9 To our knowledge, no
critical appraisal has been performed on the guidelines
covering TCM treatment for headaches. There are vari-
ous guideline assessment tools available, but the Ap-
praisal of Guidelines for Research and Evaluation
(AGREE) Ⅱ instrument is the most appropriate for
guideline appraisal.10
This study aimed to appraise the methodological quali-
ty of guidelines for the treatment of headache pro-
duced over the last two decades, and covering interven-
tions using TCM. We hoped to determine the accep-
tance level worldwide of the different interventions us-
ing TCM, and also obtain realistic efficacy data about
TCM treatments. By appraising and summarizing the
current CPGs for TCM headache treatment, we can de-
termine whether a further guideline for the treatment
of headache with TCM is needed.
METHODS
Information sources
We searched for guidelines on headache disorders in
PubMed, EMBASE, Web of Science, and four Chinese
academic electronic databases, the Chinese Biomedical
Literature Database (CBM), China National Knowl-
JTCM | www. journaltcm. com
edge Infrastructure Database (CNKI), China Science
and Technology Journal Database (VIP), and Wanfang
database. We also searched two international guideline
databases, the Guideline International Network (GIN)
and National Guideline Clearinghouse (NGC), as well
as a Chinese guideline database (Medlive). We used
"headache" as a search term on the websites of several
well-known organizations concerned with guideline de-
velopment: World Health Organization (WHO),
American Academy of Neurology (AAN), National In-
stitute for Health and Care Excellence (NICE), Scot-
tish Intercollegiate Guidelines Network (SIGN), and
European Academy of Neurology (EAN). We carried
out a manual search for gray literature to supplement
the electronic searches. The date of publication was re-
stricted to between January 1996 and June 5, 2015.
The terms "headache", "headache disorders", "cephalal-
gia", "migraine", "tension-type headache", "practice
guideline", "consensus", "statement", "regulation", and
"recommendation" were used in both the "MeSH" and
"Free-text" fields. We searched Chinese guidelines for
headache disorders manually, as these were published
in books and government documents. Using PubMed
and EMBASE as examples, the search process was:
For PubMed:
#1 "Headache"[Mesh] OR "Headache Disorders"
[Mesh]
#2 "cephalalgia" OR "headache" OR "headache disor-
ders" OR "migraine" OR "tension-type headache"
#3 "Practice Guideline" [Publication Type] OR "Prac-
tice Guidelines as Topic"[Mesh]
#4 "guideline" OR "consensus" OR "statement" OR
"recommendation"
#5 #1 OR #2
#6 #3 OR #4
#7 #5 AND #6
For EMBASE:
#1 'headache'/exp OR 'headache' OR 'cephalalgia' OR
'migraine' OR 'headache disorders' OR 'tension-type
headache'
#2 'practice guideline'/exp OR 'practice guideline' OR
'consensus' OR 'statement' OR 'recommendation'
#3 #1 AND #2
Inclusion and exclusion criteria
We included all Chinese and English language CPGs
that provided recommendations on the diagnosis, treat-
ment, and management of headache disorders, and
met the definition of guidelines, as described by the In-
stitute of Medicine.9
Guidelines were excluded if they met any of the follow-
ing criteria: (a) did not include TCM therapies for
headaches, including both herbal (Chinese herbal for-
mulas, Chinese patent medicine, and herbal extracts)
and physical TCM therapies (acupuncture, moxibus-
tion, massage, manipulation, and osteopathy); (b)
translations of international guidelines into Chinese,
adaptations of international guidelines for China, ab-
340 June 15, 2018 | Volume 38 | Issue 3 |
 Luo H et al. / Systematic Review
stracts or summary reports of international guidelines,
and other explanatory or evaluation reports of guide-
lines; and (c) beta versions or older versions of guide-
lines from the same source.
Literature screening and review data extraction
Using the inclusion and exclusion criteria, all records
were classified using Endnote (Version X7, Thomson
Reuters, USA), and duplicate studies were discarded.
Next, two of the authors (Luo Hao and Yao Sha) inde-
pendently screened all the search records by browsing
the title and abstract using the predefined inclusion
and exclusion criteria. We then obtained the full text of
possible guidelines to determine whether they were eli-
gible, and extracted the general characteristics of each
CPG included. Disagreements were resolved by consen-
sus between the two authors or by a third expert (Li
Hui).
Quality evaluation
Each eligible guideline was assessed using the AGREE
II instrument, which consists of 23 items across six do-
mains, and two overall assessment items. Each domain
assesses a different dimension of guideline quality.
Each item was scored from 1 (strong disagreement) to
7 (strong agreement). The standardized scores for each
domain were calculated as percentages using the follow-
ing formulae:
Maximum possible score = 7 (strongly agree) × No. of
items within a domain × No. of appraisers; Minimum
possible score = 1 (strongly disagree) × No. of items
within a domain × No. of appraisers; and Observed
score = Overall scores of all the appraisers:11,12
The overall guideline recommendation considered all
the domain scores. Using the standard criteria, a CPG
was "recommended" if the scores of five domains were
at least 60%; "recommended with modification" if the
scores of four domains were at least 30% and the score
of one other domain was at least 50%; and "not recom-
mended" if the scores of three domains were 30% or less.
Data collection process
A pre-designed table with two components (general
characteristics and scores of AGREE Ⅱ assessment)
was used to extract data from eligible headache guide-
lines. The general characteristics included title, year of
publication, country, organization, nature (Traditional
Chinese Medicine or modern medicine), type (evi-
dence-based or consensus-based guidelines), topic cov-
ered (syndromes of headache), grade of evidence, level
of recommendation, and economic assessment.
Synthesis of the results
Each eligible guideline was independently assessed by
four appraisers (Luo Hao, Wang Yangyang, Yao Sha,
and Xu WenJie). Prior to the formal assessment, we
conducted a pre-assessment by choosing five guide-
JTCM | www. journaltcm. com
lines. The intra-class correlation coefficients (ICCs)
were examined to assess the inter-rater reliability of the
four appraisers within each domain. The ICCs ranged
between 0 (completely unreliable) and 1 (completely
reliable). The reliability was considered poor where the
ICC was less than 0.40, and excellent where it was
more than 0.75.
Statistical analysis
We used mean and standard deviation (SD) values to
show the proportion of standardized scores for each do-
main in each guideline. We also calculated the overall
mean scores for each domain for all guidelines and the
Chinese guidelines. The consistency of the four apprais-
ers for each guideline and across all the guidelines was
measured using ICCs and 95% confidence intervals
(CIs). 13
Subgroup analyses were performed by year of publica-
tion, country, type (consensus-based or evidence-
based), grade of evidence, level of recommendation,
and economic assessment, using an independent t-test
or one-way analysis of variance. All the tests were
two-sided, and a P value of < 0.05 was considered sta-
tistically significant. All the statistical analyses used
SPSS version 19.0 (IBM Inc, New York).
RESULTS
Figure 1 shows a flowchart of the search and selection
process. A total of 13 630 articles were checked across
all possible sources. After removing duplicate articles,
the remaining 4427 articles were screened by assessing
the title and abstract, to give a shortlist of 96 articles
for screening by reviewing the entire text. A total of 24
guidelines met our inclusion and exclusion criteria, of
which 23 were evaluated using the AGREE Ⅱ instru-
ment, because NAPNP 200725,26 was in two parts.
The general characteristics of the eligible guidelines are
shown in Table 1. Seven guidelines (30.4%) were devel-
oped in China, six (26.1% ) in the USA, and five
(21.7% ) in the UK. Only four (17.4% ) were TCM
guidelines; the others were modern medicine guide-
lines touching on TCM treatments. In total, 18 guide-
lines (78.3% ) were evidence-based and the other five
(21.7% ) were consensus-based. Fourteen (60.9% )
guidelines focused on migraine. Only 12 (52.2% )
guidelines included the level of evidence and rated the
recommendations. Economic assessment data were on-
ly available in seven guidelines (30.4%).
The proportions of AGREE Ⅱ standard scores for the
eligible guidelines are shown in Table 2. The overall
consistency of the four appraisers was good [overall
ICC (95% CI), 0.84 (0.82-0.86); data not shown]. Af-
ter estimation of the scores for each domain and the
calculation of the global scores for each guideline, two
guidelines were "recommended", 12 were "recommend-
ed with modifications", and nine were "not recom-
341 June 15, 2018 | Volume 38 | Issue 3 |
 Luo H et al. / Systematic Review
Records identified through electronic Search in website and other resource
database searching (n = 3627)
Records after duplicates
removed (n = 2737)
Records after screened (n = 96)
Eligible guidelines after
screened (n = 24)
Figure 1 Flow diagram showing the search and selection process in the study
mended". The overall mean score showed that domain
1 was the highest of the six domains, and domains 5
and 6 were the lowest; the scores for the other domains
ranged from 39% to 50%. The mean score for the four
TCM guidelines was lower than the overall mean score
in each domain, although domains 5 and 6 were again
the lowest.
Table 3 shows the results of the subgroup analysis of
the eligible guidelines. The year of publication was not
statistically significant, although the mean scores for
the guidelines published in and after 2011 were better
than those for guidelines published before 2011. The
guidelines developed in China had significantly lower
scores than those developed in the USA for domains 1
and 6 (P < 0.05), and those developed in the UK for
all the domains except domain 2 (P < 0.01). Evi-
dence-based guidelines had higher quality scores than
consensus-based guidelines for all the domains (P <
0.05). The scores for guidelines with an evidence grade
were better, particularly in domains 1 to 4 (P < 0.05).
Recommended guidelines were significantly likely to
have higher scores in domains 4 (P < 0.05). Guidelines
with an economic assessment were significantly likely
to be better in all domains except domain 6.
Table 4 shows the recommendations on the use of
JTCM | www. journaltcm. com
(n = 68)
Records excluded by checking titles and
abstract (n = 2641)
Guidelines excluded by checking
full-text articles (n = 72)
TCM in the two "recommended" modern medicine
guidelines and two "recommended with modifica-
tion" TCM guidelines. Physical therapies were more
likely to be accepted by modern medicine. Herbal
therapies were not recommended in modern medi-
cine guidelines because of insufficient evidence, ad-
verse effects, or poor efficacy. Herbal formulas, Chi-
nese patent medicine, acupuncture, moxibustion,
massage and manipulation are, however, recommend-
ed in TCM guidelines.
We compared the guideline quality in this study with
previous related studies. Figure 2 shows comparisons
with the studies of Alonso-Coello et al,38 Chen et al,39
Wei et al,40 and Li et al,41 who assessed the quality of
international and Chinese guidelines in the last two
decades. The quality of the headache guidelines was
better than that of other domestic guidelines, but the
mean scores for the domestic headache guidelines
were poorer than international guidelines for do-
mains 3, 5, and 6. However, the mean domain
scores in international guidelines for domains 5 and
6 were still less than 30%.
We also compared the quality assessment results for
the four TCM guidelines with those from previous
studies (Figure 3). The scores for the TCM guide-
342 June 15, 2018 | Volume 38 | Issue 3 |
 Luo H et al. / Systematic Review

Table 1 General characteristics of the eligible guidelines

Grade of Level of Economic
Organization year Country Nature Type Topic
evidence recommendation assessment
PMDP 200614 China MM CB M No No No
PTDT 200715 China MM CB TTH No No No
CACM1 2008 16
China TCM CB M No No No
CACM2 2008 17
China TCM CB PH No No No
CACMS/CAAM 2011 18
China TCM EB M Yes Yes No
CASP 201119 China MM EB M Yes Yes No
CACMS 2011 20
China TCM EB M Yes Yes No
CHS 1998 21
Canada MM EB M Yes Yes Yes
AAN1 2000 22
USA MM EB M Yes Yes No
AAN2 200023 USA MM EB M Yes No No
AAN3 2000 24
USA MM EB M Yes Yes Yes
NAPNP 2007 25,26
USA MM EB M No No No
ICSI 2011 27
USA MM EB H Yes Yes Yes
ACP/ASIM 201228 USA MM EB M No No Yes
SIGN 2008 29
UK MM EB H Yes Yes Yes
EFNS 2009 30
UK MM EB M No Yes No
BASH 2010 31
UK MM EB H No No Yes
EFNS 201032 UK MM EB TTH Yes Yes No
NICE 2012 33
UK MM EB H Yes No Yes
ISSH 2012 34
Italy MM EB PH Yes Yes No
DHS 2012 35
Denmark MM EB H No No No
FSSMH 201436 France MM EB M No Yes No
LAPCMT 2013 37
Latin America MM CB M No No No
Notes: MM: modern medicine; TCM: Traditional Chinese Medicine; CB: consensus-based; EB: evidence-based; H: headache; PH; prima-
ry headache; M: migraine; TTH: tension-type headache; PMDP: Panel of Migraine Diagnosis and Prevention; PTDT: Panel of TTH Diag-
nosis and Treatment; CACM1/2: China Association of Chinese Medicine; CACMS/CAAM: China Academy of Chinese Medical Sciences/
China Association of Acupuncture-Moxibustion; CASP: Chinese Association for the Study of Pain; CACMS: China Academy of Chinese
Medical Sciences; CHS: Canadian Headache Society; AAN1/2/3: American Academy of Neurology; NAPNP: National Association of Pe-
diatric Nurse Practitioners; ICSI: Institute for Clinical Systems Improvement; ACP/ASIM: American College of Physicians/American Soci-
ety of Internal Medicine; SIGN: Scottish Intercollegiate Guidelines Network; EFNS: European Federation of Neurological Societies;
BASH: British Association for the Study of Headache; NICE: National Institute for Health and Clinical Excellence; ISSH: Italian Society
for the Study of Headaches; DHS: Danish Headache Society; FSSMH: French Society for the Study of Migraine Headache; LAPCMT:
Latin American Panel of Chronic Migraine Treatment.

lines in this study were lower than those reported by DISCUSSION

Choi et al 42 for all the domains but better than those
reported by Chen et al 43 in domains 2 and 3. They
were, however, lower than 30% in domains 3, 5, and
6. There are therefore no high quality TCM head-
ache guidelines in China, as the scores for all the
TCM guidelines were less than 40% in all domains.
As Figure 3 shows, the scores found in two studies
were higher than the others. After a thorough investi-
gation, we found that the authors of the two studies
had participated in the process of developing some of
the guidelines included but had not reported this as
a conflict of interest.44,45
Results of the appraisal of high quality CPGs
The ICC (95% CI) of the NICE headache guideline33
(0.59, 0.23-0.81) — one of the two recommended
guidelines — was the lowest of the eligible guidelines.
The four appraisers had different professional academic
backgrounds, and we believe that the structures of the
reporting guidelines and the miscellaneous content may
have led to diversity. We suggest that it is increasingly
important to have a uniform format for the reporting
guidelines, and also to develop alternative versions for
target users from different backgrounds, such as pa-
tients, clinicians, methodologists, and policy makers.

JTCM | www. journaltcm. com 343 June 15, 2018 | Volume 38 | Issue 3 |
 Table 2 AGREE Ⅱ domain scores and ICCs (95% CI) of the eligible guidelines
Scores of each domain (%)
Organization year ICC (95% CI) Domain 6: Overall
Domain 1: Domain 2: Domain 3: Domain 4: Domain 5:
editorial guideline
scope and purpose stakeholder involvement rigor of development clarity of presentation applicability
independence assessment
PMDP 200614 0.83 (0.68, 0.92) 34.7 33.3 4.7 18.1 5.2 0.0 NR
PTDT 200715 0.84 (0.70, 0.93) 23.6 11.1 2.1 18.1 2.1 0.0 NR
CACM1 200816 0.81 (0.64, 0.91) 19.4 15.3 4.7 20.8 2.1 0.0 NR
CACM2 200817 0.80 (0.61, 0.90) 18.1 13.9 5.7 25.0 1.0 0.0 NR
CACMS/CAAM 201118 0.89 (0.78, 0.95) 50.0 55.6 46.4 61.1 18.8 0.0 RM

JTCM | www. journaltcm. com

CASP 201119 0.89 (0.79, 0.95) 43.1 38.9 24.5 54.2 29.2 0.0 NR
CACMS 201120 0.89 (0.80, 0.95) 40.3 45.8 38.5 51.4 7.3 6.3 RM
CHS 199821 0.83 (0.67, 0.92) 48.6 41.7 41.7 47.2 16.7 18.8 NR
AAN1 200022 0.77 (0.57, 0.89) 65.3 37.5 40.6 48.6 18.8 22.9 RM
AAN2 200023 0.81 (0.63, 0.91) 70.8 43.1 40.6 63.9 20.8 22.9 RM
AAN3 200024 0.78 (0.59, 0.90) 38.9 15.3 9.4 3.8 20.8 0.0 NR
NAPNP 200725,26 0.62 (0.29, 0.82) 76.4 65.3 45.3 65.3 47.9 52.1 RM
ICSI 201127 0.81 (0.64, 0.91) 51.4 43.1 27.6 63.9 21.9 16.6 NR
ACP/ASIM 201228 0.81 (0.64, 0.91) 51.4 43.1 27.6 63.9 21.9 16.6 NR

344
SIGN 200829 0.65 (0.35, 0.84) 77.8 69.4 71.9 86.1 67.7 60.4 Recommended
EFNS 200930 0.84 (0.70, 0.93) 55.6 30.6 47.9 59.2 12.5 60.4 RM
BASH 201031 0.86 (0.73, 0.93) 61.1 45.8 21.9 56.9 36.5 10.4 RM
EFNS 201032 0.77 (0.56, 0.89) 58.3 33.3 46.4 59.7 17.7 52.1 RM
Luo H et al. / Systematic Review

NICE 201233 0.59 (0.23, 0.81) 88.9 76.4 83.3 91.7 78.1 77.1 Recommended
ISSH 201234 0.79 (0.61, 0.90) 55.6 29.2 45.8 69.4 25.0 37.5 RM
DHS 201235 0.72 (0.47, 0.87) 52.8 44.4 19.3 50.0 34.9 31.3 RM
FSSMH 201436 0.63 (0.31, 0.83) 61.1 51.4 35.4 52.8 26.0 43.8 RM
LAPCMT 201337 0.67 (0.37, 0.84) 48.6 33.3 21.9 29.2 14.6 18.8 NR
Overall mean (SD) score NA 52.1 (18.0) 39.5 (17.1) 33.4 (21.0) 49.8 (21.9) 23.8 (19.3) 24.2 (23.7) NA
Mean (SD) score of the
NA 31.9 (15.8) 32.6 (21.2) 23.8 (21.7) 39.6 (19.7) 7.3 (8.1) 1.6 (3.1) NA
Chinese guidelines
Notes: SD: standard deviation; NA: not applicable; NR: not recommended; RM: recommended with modification; MM: modern medicine; TCM: Traditional Chinese Medicine; CB: consensus-based; EB: evi-
dence-based; H: headache; PH: primary headache; M: migraine; TTH: tension-type headache; PMDP: panel of migraine diagnosis and prevention; PTDT: panel of TTH diagnosis and treatment; CACM1/2:
China Association of Chinese Medicine; CACMS/CAAM: China Academy of Chinese Medical Sciences/China Association of Acupuncture-Moxibustion; CASP: Chinese Association for the Study of Pain;
CACMS: China Academy of Chinese Medical Sciences; CHS: Canadian Headache Society; AAN1/2/3: American Academy of Neurology; NAPNP: National Association of Pediatric Nurse Practitioners; ICSI:
Institute for Clinical Systems Improvement; ACP/ASIM: American College of Physicians/American Society of Internal Medicine; SIGN: Scottish Intercollegiate Guidelines Network; EFNS: European Federa-
tion of Neurological Societies; BASH: British Association for the Study of Headache; NICE: National Institute for Health and Clinical Excellence; ISSH: Italian Society for the Study of Headaches; DHS: Dan-
ish Headache Society; FSSMH: French Society for the Study of Migraine Headache; LAPCMT: Latin American Panel of Chronic Migraine Treatment.

June 15, 2018 | Volume 38 | Issue 3 |

 Table 3 AGREE Ⅱ domain scores for guidelines by subgroups
Scores of each domain [% (SD)]
Variable n (%) Editorial
Scope and purpose Stakeholder involvement Rigor of development Clarity of presentation Applicability
independence
Year <2011 13 (56.5) 48.5 (19.8) 32.7 (16.3) 29.3 (22.4) 42.9 (23.7) 18.8 (17.9) 21.0 (23.1)
≥2011 10 (43.8) 58.0 (15.1) 48.3 (14.4) 38.8 (18.7) 58.9 (16.1) 30.3 (20.1) 28.3 (24.9)
P value 0.281 0.260 0.292 0.081 0.160 0.471
Country China 7 (30.4) 32.7 (12.5) 30.6 (17.4) 18.1 (18.4) 35.5 (19.1) 9.4 (10.6) 0.9 (2.4)
USA 6 (26.1) 57.3 (14.6) 40.7 (14.9) 35.0 (12.7) 49.5 (21.5) 22.8 (12.2) 20.8 (16.7)

JTCM | www. journaltcm. com

UK 5 (21.7) 68.3 (14.4) 51.1 (20.9) 54.3 (24.0) 70.7 (16.7) 42.5 (29.4) 52.1 (25.0)
P value (China vs USA) - 0.002 0.371 0.140 0.241 0.130 0.026
P value (China vs UK) - <0.001 0.700 0.005 0.009 0.006 <0.001
Nature TCM 4 (17.4) 31.9 (15.8) 32.6 (21.2) 23.8 (21.7) 39.6 (19.7) 7.3 (8.1) 1.6 (3.1)
MM 19 (82.6) 56.4 (15.7) 40.9 (16.4) 35.5 (20.1) 52.0 (22.2) 27.3 (19.3) 29.0 (23.3)
P value - 0.010 0.392 0.333 0.310 0.059 <0.001
Type CB 5 (21.7) 28.8 (12.8) 21.4 (11.0) 7.8 (8.0) 22.2 (4.8) 5.0 (5.6) 3.8 (8.4)
EB 18 (78.3) 58.6 (13.4) 44.5 (15.2) 40.5 (17.5) 57.5 (18.1) 29.0 (18.6) 29.9 (23.5)

345
P value - <0.001 0.004 0.001 <0.001 0.010 0.001
Evidence grade Yes 12 (52.2) 61.1 (15.0) 47.6 (15.5) 47.4 (15.5) 62.4 (14.3) 30.8 (22.0) 31.3 (24.8)
No 11 (47.8) 42.3 (16.3) 30.7 (14.7) 18.2 (14.7) 36.2 (20.8) 16.1 (12.9) 16.5 (20.7)
P value - 0.009 0.014 <0.001 0.003 0.067 0.140
Luo H et al. / Systematic Review

Recommendation level Yes 12 (52.2) 58.0 (12.1) 44.9 (13.2) 44.0 (10.9) 60.0 (10.6) 26.0 (16.6) 31.6 (22.6)
No 11 (47.8) 46.7 (21.7) 33.6 (19.4) 21.9 (23.6) 38.7 (25.8) 21.4 (22.6) 16.1 (23.1)
P value - 0.100 0.111 0.008 0.024 0.560 0.121
Economic assessment Yes 7 (30.4) 67.9 (14.8) 55.0 (14.8) 47.5 (22.4) 67.9 (15.7) 41.4 (24.2) 36.9 (25.9)
No 16 (69.6) 45.2 (15.0) 32.7 (13.4) 27.3 (17.6) 42.0 (19.6) 16.1 (10.3) 18.6 (21.1)
P value 0.003 0.002 0.030 0.006 0.033 0.088
Notes: SD: standard deviation; AGREE: Appraisal of Guidelines for Research and Evaluation Ⅱ instrument; TCM: Traditional Chinese Medicine; MM: modern medicine; CB: consensus-based; EB: evi-
dence-based.
The

were
tion,

tions
dence

These
coming
guidelines
low scores

all
dence-based

mainstream.
non-Chinese

sensus-based.
mendations.11
Domains with

"recommended"

suggest that there

June 15, 2018 | Volume 38 | Issue 3 |

evi-
Table 4, the four
gests that evi-
This trend sug-
had better scores
and summary, as
methods used for
of eight items
lines. It consists
main in the devel-
ered to be the
3, 5, and 6. The
were
of the headache
ble 2, the overall
As shown in Ta-

is no common,
findings
differed.
of recommenda-
and formulation
though the evi-
dence-based, al-
tion" guidelines
ed with modifica-
or "recommend-
In
more
guidelines are be-
guideline was con-
0.05). Only one
domains (P <
lines for all the
sus-based guide-
than the consen-
based guidelines
evidence-
lation of recom-
well as the formu-
evidence collec-
opment of guide-
most essential do-
main 3) is consid-
ment domain (do-
rigor of develop-
poor for domains
mean (SD) scores

grading
grading,
and appraises the
 Luo H et al. / Systematic Review

sensible, and trans- In a previous study, 87% of guideline developers were

Notes: NA: not applicable; NR: not recommended; SIGN: Scottish Intercollegiate Guidelines Network; NICE: National Institute for Health and Clinical Excellence; CACMS/CAAM: China Academy of Chi-
parent approach to found to have some form of affiliation with the phar-
Massage or manipulation
grading the quality maceutical industry, 58% had received financial sup-
Insufficient Evidence
Recommended

Recommended
(or certainty) of ev- port, and 38% had served as an employee or consul-
idence and the tant for a pharmaceutical company.46 Conflicts of inter-
NA
strength of the rec- est are often hard to detect, but evidence suggests that
ommendations. they can influence the recommendations in a guide-
The applicability line.49
domain (domain The results suggest that developers should focus on the
5) evaluates wheth- "rigor of development", "applicability", and "editorial
er the guideline independence" when developing TCM headache guide-
Physical therapy

provides advice on lines in the future. They should also use widely-recog-
Recommended
Recommended
Moxibustion

implementing rec- nized methods, such as the GRADE approach, to rate

NA
NA

ommendations. It the quality of the best available evidence and develop

considers the po- healthcare recommendations for the establishment of
tential resources, fa- CPGs.50 The potential advantages and disadvantages of
cilitators, and barri- treatments, such as patient and social values or prefer-
Table 4 Recommendations of two "recommended" modern medicine guidelines and two "recommended with modification" TCM guidelines

ers and provides ences, should be carefully considered and combined

monitoring or au- with the evidence to formulate the final recommenda-
tions.51 Before developing a new guideline, it is impor-
Recommended
Recommended
Recommended
Recommended

diting criteria de-

Acupuncture

rived from the key tant to list the funding sources, or provide formal con-
recommenda- flict of interest statements for all the authors for discus-
tions.11 However, sion and public disclosure.47,52,53
nese Medical Sciences/China Association of Acupuncture-Moxibustion; CACMS: China Academy of Chinese Medical Sciences.

most of the guide-

Discussion of the two TCM guidelines
lines included in
With the support of the World Health Organization/
this study did not
Western Pacific Regional Office (WHO/WPRO), the
discuss the applica-
CACMS/CAAM 2011 and CACMS 2011 were the
bility, or highlight
Herbal extract

first evidence-based guidelines on TCM to be adopted

the tools or meth-
in China. However, these two guidelines had poor
NR
NR
NA
NA

ods required to fa-

scores for domains 5 and 6. Table 4 shows that they rec-
cilitate or promote
ommended the use of herbal therapies to treat head-
the guidelines. The
ache. They differed from the two "recommended"
mean score (SD)
guidelines, which described the method for the extrac-
for domain 5 was
tion of herbs, acupuncture, massage, and manipulation
therefore the low-
Chinese patent medicine

in the prophylaxis of headache. We believe that there

est of the domains
Herbal therapy

are two main reasons for the difficulty in forming rec-

Recommended

[23.8 (19.3); Fig- ommendations for traditional Chinese treatments in

ure 2], which is
NA
NA
NA

the guidelines. First, it is difficult to conduct high-qual-

consistent with the
results of previous
studies.38-40,42
The editorial inde-
pendence domain
(domain 6) assesses
Herbal formula
ity randomized clinical trials because of the personal-
ized nature of treatment with TCM.54,55 Second, ran-
domized clinical trials are one of the major sources of
clinical evidence, but their current quality — although
better than before — remains unsatisfactory.56 The
AGREE Ⅱ instrument is a tool for assessing guideline

Recommended

the bias related to development, reporting, and evaluation.11 However, if a

conflict of interests TCM guideline is fully developed by this instrument,
NA
NA
NA

in the development we think that the clinical practice characteristics of

of guidelines. 11 In TCM may not be appropriately reflected. Different
our study, the methods should therefore be considered when develop-
mean score (SD) ing a guideline, and the existing guidelines on head-
for domain 6 was
18

ache should be critically appraised to develop

CACMS/CAAM 2011

24.2 (23.7), and high-quality evidence-based TCM guidelines for head-

Organization year

most of the guide- ache, suitable for use with individuals.57-60 We hope that
20
CACMS 2011

lines included did

NICE 201233
29

ongoing work will lead to the development of

SIGN 2008

not report any con- high-quality TCM guidelines and contribute to the in-
flicts of interest or corporation of effective traditional therapies into clini-
financial conflicts. cal practice.

JTCM | www. journaltcm. com 346 June 15, 2018 | Volume 38 | Issue 3 |
 Luo H et al. / Systematic Review

70
Mean scores of each domain (%) 60
50
40
30
20
10
0
ent lop e
ose ent ve ilit
y enc
pu
rp em pm ion lica
b nd
olv velo
nta
t p epe
e and er inv f d e
se Ap
al i
nd
cop old igo
ro pre ito
ri
keh S R ty of Ed
Sta lari
In our study Alonso 2010 C Chen 2012 Wei 2013 Li 2013
Figure 2 Comparison of assessed quality between headache guidelines and others

120
Mean scores of each domain (%)

100

80

60

40

20

0 t y
t urpo men
se op bilit ce
l ve men dpe velop t i o nvel Ap plica e p e nden
r inv
o e an
r of
d
resen
t a
al in
d
Scop h olde Rigo of p d itori
k e i t y E
Sta Clar
TCM Guildline Choi 2015 Chen 2014 Fang 2014 Yu 2011
Figure 3 Comparison of the assessment of quality of the TCM guidelines between this study and other TM studies

Strengths and limitations In conclusion, we found that modern medicine guide-

Our study has several strengths. First, the systematic re- lines did not recommend using Chinese herbal thera-
view of guideline quality covered a wide range of pies, but the two recommended guidelines, support-
guidelines on headache disorders, from around the ed by clinical evidence, suggest the use of physical
world. Second, our team included a methodologist TCM therapies for treating headache. The two TCM
and neurologist who had prior experience of develop- guidelines, which were "recommended with modifica-
ing guidelines. tion" following assessment, recommended the use of
The study also had some limitations, however. First, Chinese herbal and physical therapies to treat head-
we focused on guidelines that covered TCM therapies aches. Healthcare professionals should choose suitable
for headache and were published in Chinese or Eng- and high-quality CPGs to guide their clinical prac-
lish. Most of the guidelines included were therefore tice. The overall quality of TCM headache guidelines
published in Europe, America, and China. These was low in China, but evidence-based guidelines are
may not be representative of all the available guide- gradually becoming mainstream. Guideline develop-
lines on headache disorders. Second, the supplemen- ers should carefully consider the three most impor-
tary materials and background information on the tant domains, covering the rigor of development, ap-
guidelines could not be searched and assessed thor- plicability, and editorial independence.
oughly, because different organizations used different
measures to report the supporting information. This
ACKNOWLEDGMENTS
could lead to an underestimation of guideline quality
in certain domains. Third, the AGREE Ⅱ instru- We thank Yao Liang, Wei Dang, Wang Qi, Wang Xiao-
ment used to assess the overall quality of the guidelines qin, Evidence-Based Medicine Center, School of Basic
did not provide any criteria or instructions. This makes Medical Sciences, Lanzhou University, Lanzhou, Gan-
it difficult for assessors to reach a consensus on the su, China, for their general assistance and help in the
overall quality of the guidelines.10 revision of the manuscript.

JTCM | www. journaltcm. com 347 June 15, 2018 | Volume 38 | Issue 3 |
 Luo H et al. / Systematic Review
REFERENCES
1 Headache Classification Committee of the International
Headache Society. The International Classification of
Headache Disorders. 3rd ed (beta version). Cephalalgia
2013; 33(9): 629-808.
2 Vos T, Flaxman AD, Naghavi M, et al. Years lived with dis-
ability (YLDS) for 160 sequelae of 289 diseases and inju-
ries: 1990-2010: a systematic analysis for the Global Bur-
den of Disease Study 2010. The Lancet 2014; 384(9943):
582-582.
3 Stovner LJ, Hagen K, Jensen R, et al. The global bur-
den of headache: a documentation of headache preva-
lence and disability worldwide. Cephalalgia 2007; 27:
193-210.
4 Rasmussen BK, Jensen R, Schroll M, et al. Epidemiology
of headache in a general population — a prevalence study.
J Clin Epidemiol 1991; 44: 1147-1157.
5 Latinovic R, Gulliford M, Ridsdale L. Headache and mi-
graine in primary care: consultation, prescription, and re-
ferral rates in a large population. J Neurol Neurosurg Psy-
chiatry 2006; 77(3): 385-387.
6 Wang YY, Zhang YL. A review of the clinical research on
the treatment of headache in Chinese medicine in the past
10 years. Beijing Zhong Yi Yao Da Xue Xue Bao 1992; (1):
5-12.
7 Grimshaw JM, Russell IT. Effect of clinical guidelines on
medical practice: a systematic review of rigorous evalua-
tions. The Lancet 1993; 342(8883): 1317-1322.
8 Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J.
Clinical guidelines: potential benefits, limitations, and
harms of clinical guidelines. BMJ 1999; 318(7182):
527-530.
9 Graham R, Mancher M, Miller Wolman D, et al. Clinical
practice guidelines we can trust. Washington, DC: Nation-
al Academy of Sciences, 2011: 20, 50.
10 Vlayen J, Aertgeerts B, Hannes,K, Sermeus W, Ramaekers
D. A systematic review of appraisal tools for clinical prac-
tice guidelines: multiple similarities and one common defi-
cit. Int J Qual Health Care 2005; 17(3): 235-242.
11 Brouwers MC, Kho ME, Browman GP, et al. AGREE Ⅱ:
advancing guideline development, reporting and evalua-
tion in healthcare. Can Med Assoc J 2010; 182(18):
E839-E842.
12 AGREE. AGREE Next Steps Consortium (2009). The
AGREE Ⅱ Instrument UPDATE September, 2013. Avail-
able from URL: http://www.agreetrust.org/wp-content/up-
loads/2013/10/AGREE-II-Users-Manual-and-23-item-In-
strument_2009_UPDATE_2013.pdf.
13 Pan X, Ni Z. Application of intra-class correlation coeffi-
cient to reliability assessment. Hua Xi Yi Ke Da Xue Xue
Bao 1999; 30(1): 62-63, 67.
14 Li YS. The diagnosis and management of migraine: con-
sensus. Chin J Intern Med 2006; 45(8): 694-696.
15 Yu SY. The diagnosis and management of tension-type
headache: consensus. Chin J Neuro 2007; 40(7): 496-497.
16 China Association of Chinese Medicine. Guidelines for di-
JTCM | www. journaltcm. com
agnosis and treatment of common internal diseases in chi-
nese medicine: disease of modern medicine (migraine).
Beijing: China Press of Traditional Chinese Medicine
2008: 275-278.
17 China Association of Chinese Medicine. guidelines for di-
agnosis and treatment of common internal diseases in chi-
nese medicine: symptoms in chinese medicine (headache).
Beijing: China Press of Traditional Chinese Medicine
2008: 126-128.
18 Wu ZC. Evidence-based Guidelines of Clinical Practice
in Chinese Medicine: Acupuncture (Migraine), Beijing:
China Press of Traditional Chinese Medicine 2014,
144-162.
19 Yu SY. Guidelines for Diagnosis and Treatment of Mi-
graine. Zhong Guo Teng Tong Yi Xue Za Zhi 2011; 17
(2): 65-86.
20 China Academy of Chinese Medical Sciences, Evi-
dence-based Guidelines of Clinical Practice in Chinese
Medicine: Internal Medicine (Migraine), Beijing: China
Press of Traditional Chinese Medicine, 2011: 210-226.
21 Pryse-Phillips WE, Dodick DW, Edmeads JG, et al.
Guidelines for the non-pharmacologic management of
migraine in clinical practice. CMAJ 1998; 159(1):
47-54.
22 Ramadan NM, Silberstein SD, Freitag FG, et al. Evi-
dence-based guidelines for migraine headache in the
primary care setting: pharmacological management for
prevention of migraine. Headache 2000; 46(7): 1151-
1160.
23 Campbell JK, Penzien DB. Evidenced-based guidelines
for migraine headache: behavioral and physical treat-
ments. American Academy of Neurology. Available from
URL: http://tools.aan.com/professionals/practice/pdfs/gl
0089.pdf.
24 Silberstein SD. Practice Parameter: Evidence-based guide-
lines for migraine headache. Neurology 2000; 55(6):
754-762.
25 gunner kb, smith hd, ferguson le, et al. practice guideline
for diagnosis and management of migraine headaches in
children and adolescents: part one. J Pediatr Health Care
2007; 21(1): 327-332.
26 Gunner KB, Smith HD, Ferguson LE, et al. Practice
guideline for diagnosis and management of migraine head-
aches in children and adolescents: Part two. J Pediatr
Health Care 2007; 22(1): 52-59.
27 Institute for Clinical Systems Improvement. health care
guideline: diagnosis and treatment of headache 10th edi-
tion 2011. Available from URL: https://www.icsi.org/_as-
set/qwrznq/Headache.pdf.
28 Snow V, Weiss K, Wall EM, et al. Pharmacologic manage-
ment of acute attacks of migraine and prevention of mi-
graine headache. Ann Intern Med 2002; 137(10):
840-849.
29 Scottish Intercollegiate Guidelines Network. Diagnosis
and management of headache in adults 2008. Available
from URL: http://www.sign.ac.uk/pdf/sign107.
30 Evers S, Afra J, Frese A, et al. EFNS guideline on the drug
348 June 15, 2018 | Volume 38 | Issue 3 |
 Luo H et al. / Systematic Review
treatment of migraine-revised report of an EFNS task
force. Eur J Neurol 2009; 16(9): 968-981.
31 MacGregor EA, Steiner TJ, Davies PTG. Guidelines
for all healthcare professionals in the diagnosis and
management of migraine, tension-type headache, clus-
ter headache and medication-overuse headache. Avail-
able from URL: http://www.bash.org.uk/wp-content/up-
loads/2012/07/10102-BASH-Guidelines-update-2_v5-1-
indd.pdf.
32 Bendtsen L, Evers S, Linde M, et al. EFNS guideline on
the treatment of tension-type headache-Report of an EF-
NS task force. Eur J Neurol 2010; 17(11); 1318-1325.
33 National Institute for Health and Care Excellence. Diag-
nosis and management of headache in young people and
adults 2012. Available from URL: https://www.nice.org.
uk/guidance/cg150/update/CG150/documents/head-
aches-full-guideline-for-consultation2.
34 Sarchielli P, Granella F, Prudenzano M, et al. Italian
guidelines for primary headaches: 2012 revised version. J
Headache Pain 2012; 13(2): 31-70.
35 Bendtsen L, Birk S, Kasch H, et al. Reference pro-
gramme: diagnosis and treatment of headache disorders
and facial pain (2nd Edition). J Headache Pain 2012; 13
(suppl 1): 1-29.
36 Géraud G, Lantéri-Minet M, Lucas C, et al. French guide-
lines for the diagnosis and management of migraine in
adults and children. Clinical Therapeutics 2004; 26(8):
1305-1318.
37 Giacomozzi AR, Vindas AP, Jr SA, et al. Latin American
consensus on guidelines for chronic migraine treatment.
Arq Neuro-psiquiatr 2013; 71(7): 478-486.
38 Alonso-Coello P, Irfan A, SolàI, et al. The quality of clini-
cal practice guidelines over the last two decades: a system-
atic review of guideline appraisal studies. QualSaf Health
Care 2010; 19(6): e58.
39 Chen YL, Yao L, Xiao XJ, et al. Quality assessment of clin-
ical guidelines in China: 1993-2010. Chin Med J 2012;
125(20): 3660-3664.
40 Wei D, Wang XQ, Wu QF, et al. Quality evaluation on
chinese clinical practice guidelines in 2011. Chin J Ev-
id-based Med 2013; 13(6): 760-763.
41 Li N, Yao L, Wu QF, et al. Quality evaluation of clinical
practice guidelines published in journals of mainland Chi-
na during 2012-2013. Chin J Evid-based Med 2015; 15
(3): 259-263.
42 Choi TY, Choi J, Lee JA, Ji HJ, Bongki P, Myeong SL.
The quality of clinical practice guidelines in traditional
medicine in Korea: appraisal using the AGREE Ⅱ instru-
ment. Implement Sci 2015; 10(1): 104.
43 Chen H, Li GL, Xu WT, Xu B. Quality Assessment of
Clinical Practice Guidelines of Acupuncture in China.
Chin J Evid-based Med 2014; 14(6): 772-775.
44 Fang YG, Bai Y, Liu BY, Wang F, Xue W. Quality assess-
ment on guidelines of clinical practice in acupuncture and
moxibustion: a study based on AGREE Ⅱ. Zhong Guo
Zhen Jiu 2014; 34(6): 599-601.
45 Yu WY, Han XJ, Zhang H, et al. Study on quality assess-
JTCM | www. journaltcm. com
ment of TCM clinical practice guidelines based on
AGREE instrument. Shi Jie Ke Xue Ji Shu-Zhong Yi Yao
Xian Dai Hua 2011; 13(4): 596-600.
46 Papanikolaou GN, Baltogianni MS, Contopoulos-Ioanni-
dis DG, et al. Reporting of conflicts of interest in guide-
lines of preventive and therapeutic interventions. BMC
Med Res Methodol 2011; 1(1): 3.
47 Choudhry NK, Stelfox HT, Detsky AS. Relationships be-
tween authors of clinical practice guidelines and the phar-
maceutical industry. JAMA 2002; 287(5): 612-617.
48 Thompson DF. Understanding financial conflicts of inter-
est. N Engl J Med 1993; 329(8): 573-576.
49 Norris SL, Burda BU, Holmer HK, et al. Author's special-
ty and conflicts of interest contribute to conflicting guide-
lines for screening mammography. J Clin Epidemiol 2012;
65(7): 725-733.
50 Brozek JL, Akl EA, Alonso-Coello P, Lang D, Jaeschke R,
Williams JW; GRADE Working Group. Grading quality
of evidence and strength of recommendations in clinical
practice guidelines. Part 1 of 3. An overview of the
GRADE approach and grading quality of evidence about
interventions. Allergy 2009; 64(5): 669-677.
51 Guyatt G, Oxman AD, Akl EA, et al. (2011) GRADE
guidelines: 1. Introduction-GRADE evidence profiles and
summary of findings tables. J Clin Epidemiol 64(4):
383-394.
52 Schünemann HJ, Al-Ansary LA, Forland F, et al. Guide-
lines international network: principles for disclosure of in-
terests and management of conflicts in guidelines. Ann In-
tern Med 2015; 163(7): 548-553.
53 Campbell EG (2010) Public disclosure of conflicts of in-
terest: moving the policy debate forward. Arch Intern Med
170(8): 667.
54 Shekelle PG, Morton SC, Suttorp MJ, Buscemi N, Fries-
en C; Agency for Healthcare Research and Quality. Chal-
lenges in systematic reviews of complementary and alterna-
tive medicine topics. Ann Intern Med 2005; 142(12 Pt 2):
1042-1047.
55 Practice and Policy Guidelines Panel, National Institutes
of Health Office of Alternative Medicine. Clinical practice
guidelines in complementary and alternative medicine. An
analysis of opportunities and obstacles. Arch Fam Med
1997; 6(2): 149-154.
56 He J, Du L, Liu G, et al. Quality assessment of reporting
of randomization, allocation concealment, and blinding in
Traditional Chinese Medicine RCTs: a review of 3159
RCTs identified from 260 systematic reviews. Trials 2011;
12(1): 286-298.
57 Chen KJ, Jiang YR. Current status and problems in devel-
oping clinical guidelines for Chinese medicine and integra-
tive medicine. Zhong Xi Yi Jie He Xue Bao 2009; 7(4):
301-305.
58 Liu M, Yang J, Wang YP. Thoughts on the development
and Implementing of Evidence-Based Guideline. Chin J
Evid-based Med 2009; 9(2): 127-128.
59 Wang B, Zhan SY, Liu BY. Problems and strategies in
developing chinese medicine evidence-based clinical prac-
349 June 15, 2018 | Volume 38 | Issue 3 |
 Luo H et al. / Systematic Review

tice guidelines. Chin J Integr Med 2011; 31(11): egies 2014-2023. Available from URL: http://www.
1565-1569. who.int/medicines/publications/traditional/trm_strategy
60 World Health Organization. Traditional Medicine Strat- 14_23/en/.

JTCM | www. journaltcm. com 350 June 15, 2018 | Volume 38 | Issue 3 |