Evaluating

Research Papers
David Erskine
David.erskine@gstt.nhs.uk
 Learning Outcomes
By the end of the session you should be able to:
◼ Describe the types of research paper

◼ Outline the hierarchy of evidence

◼ Describe the format of a research paper

◼ Identify the key issues to be reviewed in each section of

a research paper
 Learning Outcomes
By the end of the session you should be able to:
◼ Describe the types of research paper

◼ Outline the hierarchy of evidence

◼ Describe the format of a research paper

◼ Identify the key issues to be reviewed in each section of

a research paper
 Types of Research Papers

◼ Primary research: experiment, clinical trials, surveys

◼ Reviews: summaries of studies

◼ Systematic reviews: overviews of studies according to

criteria

◼ Meta-analysis: combining primary studies which were

done in the same way

◼ Other: guidelines, decisions, economics

 Learning Outcomes
By the end of the session you should be able to:
◼ Describe the types of research paper

◼ Outline the hierarchy of evidence

◼ Describe the format of a research paper

◼ Identify the key issues to be reviewed in each section of

a research paper
Hierarchy of Evidence

Systematic Review / Meta-Analysis

Randomised Controlled Trial

Cohort study

Case-controlled Study

Cross-sectional Study

Case Report

Expert opinion/ consensus

Cross-sectional study
◼ Things to consider
◼ Were subjects described in sufficient
detail?
◼ How are non-responders dealt with?
◼ How were outcomes assessed?
◼ How were potential confounding variables
addressed?
Cohort study

Exposed Outcome

Time
Sample
Population

Not exposed Outcome

Cohort study
◼ Things to consider
 No control for confounding variables (ie not
sure that exposed group and not exposed
group are the same at baseline)
 No ability to assess statistical power in
“negative studies”
 Publication bias?
 Closer to real life than RCTs?
Case control study
Study

Exposed
Cases
Not exposed

Population

Exposed
Controls
Not exposed

Time
Case control study
◼ Things to consider
 No control for confounding variables (ie not
sure that exposed group and not exposed
group are the same at baseline)
 No ability to assess statistical power in
“negative studies”
 Publication bias?
 Case report/ case series
No randomisation

Experimental
intervention
Outcome

Sample
Population
No
No control comparative
intervention outcome

Time
Case report/ case series
◼ Things to consider
 No control group – ie no ability to assess what
would have happened if you had not
intervened
 Publication bias?
Randomised, controlled trial

Randomisation

Experimental
intervention
Outcome

Sample
Population
Outcome
Control
intervention

Time
Randomised, controlled trial
◼ Things to consider – most robust form of
primary research
 How representative is the sample?
 How was randomisation conducted?
 Did anything happen after randomisation that could have affected it
◼ Blinding

◼ Loss to follow up

◼ Analysis of results

 Was there sufficient patients (negative studies)

What Type of Study?

Cross-
sectional

IJPP 2006; 14: 129-134

What Type of Study?

JAMA 1998; 279: 1200-1205

What Type of Study?

JAMA 1998; 279: 1200-1205

 What Type of Study?

Chronic valproic acid intoxication: reversal by naloxone

H K R Thanacoody
Scottish Poisons Information Bureau, Royal Infirmary Edinburgh, 51 Little
France Crescent, Edinburgh, UK
Correspondence to:
H Thanacoody, ruben.thanacoody@luht.scot.nhs.uk Case study

SUMMARY
A 76-year-old woman being treated with sodium valproate for bipolar
depression presented with a 4-day history of acute confusion and
tremulousness. She had apnoeic episodes, a reduced conscious level
and generalised myoclonic movements. Her plasma valproate
concentration was 848 µmol/litre (normal range 300–600 µmol/litre).
Administration of naloxone 0.8 mg led to rapid clinical improvement.
Naloxone may be useful in reversing the features of chronic valproate
toxicity.
Published 26 June 2009
BMJ 2009 [doi:10.1136/bcr.06.2009.1924]
Cohort
study

IJPP 2009;
17: 181-187
 Randomised
Controlled Trial

BMJ 2005; 330: 293-297

BJPsych 2009; 194: 165-167
BMC Nephrology 2008; 9: 2
 Learning Outcomes
By the end of the session you should be able to:
◼ Describe the types of research paper

◼ Outline the hierarchy of evidence

◼ Describe the format of a research paper

◼ Identify the key issues to be reviewed in each section of

a research paper
Hierarchy of Evidence

Systematic Review / Meta-Analysis

Randomised Controlled Trial

Cohort study

Case-controlled Study

Cross-sectional Study

Case Report

Expert opinion/ consensus

 Research Format - IMRAD
◼ Introduction

◼ Methods

◼ Results

◼ Discussion

◼ Abstract
 Learning Outcomes
By the end of the session you should be able to:
◼ Describe the types of research paper

◼ Outline the hierarchy of evidence

◼ Describe the format of a research paper

◼ Identify the key issues to be reviewed in each

section of a research paper
 Critical Review – Points to Consider

Introduction

 Why was the study done?

 Is the study original?

◼ Background
◼ Aims & objective/hypothesis
 Critical Review – Points to Consider

Methods
 Is the study design & research method sensible?
 Who is the study about?
 Is bias avoided/minimised?
 Sample large enough or data collection undertaken for
long enough
◼ Appropriate
◼ Reliable
◼ Accuracy/specificity
◼ Design
◼ Clarity
◼ Statistics appropriate?
 Critical Review – Points to Consider

Results
 How are the results expressed?

◼ Text supports tables/figures

◼ Findings ignored or misinterpreted
◼ Statistical analysis (sufficient and appropriate)
◼ Scales of graphs
◼ n values vs. percentages
 Critical Review – Points to Consider

Discussion

 Do results/references support conclusions?

◼ Relevant
◼ Logical
◼ Biased
◼ Referenced
◼ Limitations
◼ Further work
The importance of language
 BMJ support material
http://www.bmj.com/about-bmj/resources-readers/publications/how-read-paper