Systematic Literature

Review
Diana Setiyawati
PhD (Unimelb), MHSc.Psy (IIUM), Psi (UGM), S.Psi (UGM)
 Dunia kita saat ini
• Jutaan publikasi diproduksi
• Banjir informasi
• Tanpa tahu cara pengambilan keputusan, kita
seperti tersesat di hutan raya
 Traditional reviews
• Pencarian dan pemilihan artikel dasarnya tidk
jelas
• Personal interests
• Isyu Reliabilitas
 Levels of evidence & volume of grey
literature

Systematic reviews
and meta-analyses

RCT

Quasi
Levels of experimenta Volume of grey
evidence l literature
Controlled
observational
Observational studies
without controls

Expert opinion
 Apakah itu “grey literature”
“Grey literature is defined as any
literature that is not published in
academic peer reviewed journals and
available through indexed databases
for review”
 Search for grey literature
• Call for information

• Personal and institutional contact

• World-wide web search

 Challenges for a representative search
• Problems with call for information
– personal interest/limitations
• Problems with personal and institutional contact
– Staffs changes on NGOs
• World-wide web
– large number and difficulties in filter
 Different type of grey literature
• Annual reports
• Theses/dissertations
• Conference abstracts/proceedings
• Working papers
• Evaluation reports
• Facts sheets and others
 Systematic Literature Review
• Review sistematik terhadap penelitian-
penelitian bermutu yang relevan. Review ini
diarahkan dengan pertanyaan penelitian dalam
mengidentifikasi, menilai, menyeleksi dan
mensintesis.
 Tujuan Systematic Literature Review

• Meminimalkan ketidakpastian dengan

metodologi yang:
– komprehensif
– Transparant /eksplicit
– minimum bias
– dapat diproduksi kembali
 Untuk apakah systematic reviews?
• Support Evidence Based Practice
• Research
• Inform clinical policy
• Publication
• Personal professional development
How to do systematic reviews?
 Formulate clear clinical questions from our knowledge
needs
 Search the literature to identify relevant articles
 Critically appraise the evidence for its validity and
usefulness
 Synthesise evidence
 The systematic review process
Formulate Design Search
research search bibliographic
question strategy databases

Further selection of
primary studies Retrieve papers Identify possible
using inclusion papers from
criteria titles/abstracts

Quality
Extract data Synthesis
appraisal

Formulate research /
policy conclusions
 Key Characteristics of Systematic Reviews
• Clearly stated title and objectives
• Comprehensive strategy to search for relevant studies (unpublished and
published)
• Explicit and justified criteria for the inclusion or exclusion of any study
• Clear presentation of characteristics of each study included and an
analysis of methodological quality
• Comprehensive list of all studies excluded and justification for exclusion
Linda N. Meurer, MD, MPH Department of Family and Community Medicine. “Systematic Synthesis of the Literature:
Introduction to Meta-analysis”. Power Point Presentation.
Characteristics of Systematic Reviews (cont.)

• Clear analysis of the results of the eligible studies

– statistical synthesis of data (meta-analysis) if
appropriate and possible;
– or qualitative synthesis

• Structured report of the review clearly stating the aims,

describing the methods and materials and reporting the
results

Linda N. Meurer, MD, MPH Department of Family and Community Medicine. “Systematic Synthesis of the Literature:
Introduction to Meta-analysis”. Power Point Presentation.
 An author of a good Systematic Review…

• Formulates a Question
• Conducts a Literature Search
• Refines the search by applying predetermined inclusion and
exclusion criteria
• Extracts the appropriate data and assess their quality and
validity
• Synthesizes, interprets, and reports data
 Hypothesis
• “A systematic review should be based on
principles of hypothesis testing, and the
hypotheses must be conceived a priori.”
Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. PRS Journal. 120/7 (2007) p. 1836
 Focus of the Question
• The structured question will determine the
inclusion and exclusion criteria:
– What is the population of interest?
– What are the interventions?
– What are the outcomes of interest?
– What study designs are appropriate?
 Inclusion/Exclusion Criteria
• “Once the study question is formalized, the authors must
compose a comprehensive list of inclusion and exclusion
criteria.”
• “To avoid selection bias, inclusion and exclusion criteria
should be agreed upon and formalized before data extraction
and analysis.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1836
 Literature Search
• “A comprehensive and reproducible literature
search is the foundation of a systematic
review.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1837
 Literature Search Challenges
• Database Bias - “No single database is likely to contain all published studies on a
given subject.”
• Publication Bias - selective publication of articles that show positive treatment of
effects and statistical significance.
– Hence, it is important to search for unpublished studies through a manual
search of conference proceedings, correspondence with experts, and a search
of clinical trials registries.

Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1837
 Literature Review Challenges (cont.)

• English-language bias - occurs when reviewers

exclude papers published in languages other than English

• Citation bias - occurs when studies with significant or

positive results are referenced in other publications,
compared with studies with inconclusive or negative findings
 Quality Assessment
• “The validity of a systematic review ultimately
depends on the scientific method of the
retrieved studies and the reporting of data.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839
 Quality Assessment (cont.)
• Randomized Controlled Trials (RCT):
– RCT are considered to be more rigorous than
observational studies
– A review based on well-designed RCT will likely be
more valid and accurate than a review based on
observational studies or case reports
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839
 Quality Assessment (cont.)
• “The most common way to assess and report
study quality has been using a composite,
numerical scoring instrument.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839
 Quality Assessment (cont.)
• “More than 35 different quality assessment
instruments have been published in the
literature, and most are designed for
randomized clinical trials.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839
MATRIX METHODS : LITERATURE
REVIEW MADE EASY
 Matrix Methods
1. Paper Trail
2. Document Section
3. Review Matrix
4. Synthesis
 1. Paper TRAIL
• Paper Trail: Mendokumentasikan jejak
pencarian
• Didahului dengan merumuskan hipotesis dan
pertanyaan penelitian
• Menentukan search term (dengan bantuan
theasurus) dan boolean connector
 Boolean Connectors
• OR specifies that either word can appear in a record:
this will broaden your search.Example: "bear or bears"
• AND specifies that both words must appear
somewhere in a record: this will narrow your search.
Example: "violence and television"
• NOT specifies that a record contains one word but not
another.Example: "mining not alaska"
• XOR specifies that either word, but not both can
appear in the record.Example: "cats xor dogs"
 Additional positional features are available
for more complex searches. These include:

• SAME - (the default) finds records containing terms within the same field (title,
author, etc.).Example: "mary same tallmountain" finds: "mary tallmountain" or
"tallmountain mary" in the same field
• WITH - finds records that contain the terms within a sentence of the specified
field.
• NEAR - locates terms that are adjacent to each other, in any order.Example:
"red near rose " finds: "red rose", "Rose Red".
• ADJ - locates terms that are adjacent to each other, in the same order as
entered.Example: "global adj warming" finds: "global warming"
• NESTING - group terms together using parentheses. Terms within the
parentheses will be processed first.Example: "alaska and (denali or mckinley )"
• Precise phrase searching is also available using the BROWSE search, by
enclosing the phrase in single quotes. The expression will be searched as it is
typed.Example: "alaska national wildlife refuge"
 1. Paper TRAIL
• Menentukan search term
• Menentukan durasi publikasi (mengapa)
• Menentukan Database
 What is Evidence Based Public Health

• Evidence based public health as a public

health endeavour in which there is an
informed, explicit, and judicious use of
evidence that has been derived from any of a
variety of science and social science research
and evaluation methods.
» Sacket et al 1996 BMJ 312 71-72
 Evidence Based Public Health
• The definitions highlights two aspects of evidence
based public health
– The use of a multiple methodologies evidence to
inform public health decision; publication bias
– An emphasis on clear reasoning in the process of
appraising and interpreting evidence for policy
and implementation
 2. Document Sections
• Menyimpan dan mengorganize document
• Alat bantu manajemen bibliografi
3. Review Matrix
 4. Synthesis
• Quality study
• Qualitative
• Quantitative
Questions and comments

?
 References
• Garrard. J, (2007).
Health Sciences Literature Review Made Easy.
Sudbury, MA: Jones and Bartlett Publisher, Inc.
• www.blizard.qmul.ac.uk/images/mEsh/
Mesh_workshop_Intro.ppt
• www.greynet.org/images/GL6_Knight_et_al.ppt
• Conducting a Systematic Review - Alumni Medical
Library
• http://library.uaf.edu/goldmine-keyword-searching