Professional Documents
Culture Documents
Systemic Review of Research - Susila
Systemic Review of Research - Susila
Systemic Review of Research - Susila
Subjective Objective
Narrow question
Broad question
Methods clearly defined
Methods unclear Studies appraised
Who undertakes systematic reviews?
• Multidisciplinary teams
– Clinicians
– Health services researchers
– Information scientists
– Statisticians
– Health Economists
– Patient and public involvement
What are the steps in performing a systematic review?
Data extraction /checking
Define research/review question Develop data extraction from into which study information and outcome
In consultation/collaboration with the clinical community, data can be extracted, checked & verified
commissioners and patient/public representatives
Study assessment/appraisal
Develop review protocol Assess the quality and validity of the included studies using the pre-
Pre-specify the type of studies to be included, the methods of defined method.
collating, appraising and analysing data
Synthesis
Identify relevant studies Narratively and/or statistically summarise/describe the data, exploring
Develop a comprehensive search strategy and undertake similarities and differences between studies.
systematic searches of the literature
Assess eligibility
Select those studies which meet the pre-defined inclusion Knowledge translation
criteria Review details and results are disseminated to relevant target audiences
using appropriate formats
?
Delay or not delay?
• A clearly defined, focused systematic review begins
with a well formulated research question.
• Delayed surgeries are better than immediate surgeries?
• Recurrence of injury and return to former activity is
related to time of surgery?
• W question should be focused.
• How are these two questions different?
– Does homework improve student achievement?
– In high school students who are below average performers,
does daily homework lead to better performance on unit
tests, compared to no homework?
A research question should be:
◦ Feasible
◦ Interesting
◦ Novel
◦ Ethical
◦ Relevant
A well-established format for structuring research
questions is known by the acronym PICO.
◦ Patient or Population
◦ Intervention or Indicator
◦ Comparator or Control
◦ Outcome
Population
Amongst adults with acute ACL injuries, does
Intervention
early reconstructive surgery compared with
Control
delayed reconstructive surgery lead to
Outcome 1
favourable return to former activity and/or
Outcome 2
risk of recurrent knee injury?
The research question guides the author in working
through many stages of the systematic review process
◦ Defining inclusion and exclusion criteria
◦ Searching the literature
◦ Selecting studies
◦ Extracting data
◦ Analyzing and presenting results
Develop review protocol
Pre-specify the type of studies to be included, the
methods of collating, appraising and analysing
data
• Forces you to read and understand the background
• Makes you to formulate a focused question
• Makes you to plan your information retrieval strategy
• Makes you to think through and describe
inclusion/exclusion criteria clearly
• Makes you to think about the data you want to collect
and the methods you will use to analyze them
One of the features that distinguish a
systematic review from a traditional review is
the pre-specification of inclusion and
exclusion criteria.
Inclusion criteria are a combination of
• Aspects of the research question
Population
Intervention
Comparison
Outcome
• Study type
Randomized controlled trials
Observational studies
Develop a comprehensive search strategy and undertake systematic
searches of the literature
The goal of the literature search is to discover all studies that meet the
inclusion criteria
◦ Search comprehensively
Terminology
Databases
◦ Search for grey literature
Not commercially published
◦ Search for unpublished studies
Reduce risk of publication bias
The search strategy should be designed to identify the
maximum number of studies relevant to the research
question.
◦ The search strategy should be systematic, transparent and
reproducible.
◦ Database specific controlled vocabulary terms and all relevant
text words should be included in the search strategy.
The Cochrane Collaboration recommends searching the
following databases (at minimum):
◦ PubMed
◦ EMBASE
◦ Cochrane Central Register of Controlled Trials
Interdisciplinary databases
◦ Scopus
◦ Web of Science
Specialized databases
◦ CINAHL Plus
◦ PsycINFO
Search cited and citing references
◦ Scopus
◦ Web of Science
Hand search selected journals and conference
proceedings
Conduct author searches for recent articles
written by topic experts
The inclusion of unpublished and grey literature may
minimize the potential effects of publication bias.
◦ Publication bias
Occurs when the outcome of an experiment or research study
influences the decision about whether—or how quickly—the
manuscript may be published
Grey literature refers to academic, business, government
or industry print or electronic literature that is not
controlled by commercial publishers.
◦ Conference proceedings
◦ Research reports
◦ Government reports
◦ Dissertations, theses
◦ Research monographs
Conference proceedings
◦ EMBASE
◦ Scopus
◦ Web of Science
◦ Google
Clinical trials
◦ ClinicalTrials.gov
◦ Centerwatch.com
◦ EU Clinical Trials Register
◦ ISRCTN Registry
◦ OpenTrials
◦ WHO International Clinical Trials Registry Platform
Compile search results using reference
management software (EndNote or
Mendeley)
Remove duplicate records
Document the search process
◦ Databases
◦ Dates searched
◦ Search strategies
◦ Limits (date ranges, publication types, language restrictions)
Identification of studies meeting inclusion
criteria should be done independently by two
review authors.
Review titles and abstracts of retrieved citations.
Review full text of studies which are found to meet the
inclusion criteria.
Keep a record of reasons for inclusion or exclusion.
Tools for sorting
◦ Reference management software
EndNote
◦ New web application
Rayyan
◦ Excel
Extract reported findings from selected studies using a
data extraction form.
◦ Extraction forms and approaches should be determined by the
needs of the specific review.
◦ At least two review authors should independently extract data
from study reports.
Data Extraction
• Have one or a few studies reported data differently from the others?
• Will the data still be useful?
• Should you include it?
• Make sure the core information extracted is the same
• You may need to update the form, or have more than one form
• Any changes need to be agreed and made consistently
Stay on track……
• Be careful about collecting ‘extra’ data
• It is very tempting to collect data that are not directly relevant to the
review question
• The data needed to answer the review question should have already been
decided (REMEMBER YOUR PROTOCOL)
• Collect data for good reasons – stay focused and don’t get side-tracked
Cochrane
Handbook
Forest Plot
A graphical display designed to illustrate the relative strength
of treatment effects in multiple quantitative scientific studies
addressing the same question (meta-analysis)
Names of fictional studies on left; odds ratios and confidence intervals on right; odds ratios
(squares proportional to weights used in meta-analysis); summary measure (center line of
diamond); associated confidence intervals (lateral tips of diamond); solid vertical line of no
effect
Statement of findings, discussion and
conclusions
◦ Information on all important outcomes, including
adverse outcomes
◦ Quality of evidence for each outcome
◦ How values and preferences may bear on balance of
benefits, harms, and costs of interventions
Decisions about whether and when to update a
systematic review
◦ The currency of the question asked
◦ The need for updating to maintain credibility
◦ The availability of new evidence
◦ Whether new research or new methods will affect the
findings
Smith, Mary Lee, and Gene V. Glass. (1980) "Meta-Analysis of Research on Class Size and
Its Relationship to Attitudes and Instruction.“American Education Research Journal,
17,419-433
Whitehead, MF & Becker, BJ (2000). The Young Child’s Post-Divorce Adjustment: A Meta-
Analysis with implications for parenting arrangements. Journal of Family Therapy, 14, 5-26.
Thank you