How to appraise Treatment Study

MUHAMMAD ALI FM USU

1

Principles in critical appraisal

Critical appraisal of primary research (clinical trial) involves three overall questions: Question 1: What is the PICO of the study, and is it close enough to your PICO? Question 2: How well was the study done?

Question 3: What do the results mean and could they have been due to chance?
2

MAN IN EBP

Sir Archie Cochrane

??
Dave Sackett
3

Steps in EBP
1. 2. Formulate clinical problems in answerable questions Search the best evidence: use internet or other online database for current evidence

VIA

3.

Critically appraise the evidence for Validity (was the study valid?) Importance (were the results clinically important?) Applicability (could we apply to our patient?)
Apply the evidence to patient
4

4.

Treatment

Is this evidence about treatment valid?

1. 2. 3. 4. 5. Was the assignment of patients to treatment randomized? Was the randomization conclealed? Were the groups similiar at the start of the trial? Was follow-up of patients sufficiently long and complete? Were all patients analyzed in the groups to which they were randomized?

Some finer points : 6. Were patients, clinicians, and study personnel kept blind to treatment? 7. Were groups treated equally, apart from the experimental therapy?

Is this valid evidence about treatment important?

1. What is the magnitude of the treatment effect? 2. How precise the estimate of the treatment effect?

Is this valid and important evidence applicable to our patient?

1. Is our patient so different from those in thr study that its results cannot apply? 2. Is the treatment feasible in our setting? What are our patients potential benefits and harms from the therapy? 3. What are our patients potential benefits and harms from the therapy? 4. What are our patients values and explectations for both the outcome we are trying to prevent and the treatment we are offering?

Internal Validity: RAMMbo

R Recruitment

: were the subjects representative?

(See Methods)

A Allocation M Maintenance

: was the allocation randomised and

concealed? (See Methods)

: did the groups have equal cointervention and adequate follow up? ( See Methods & Results)

Mbo Measurement, blinding, objectivity : were the subjects and

assessors kept blind to which treatment was being received and/or were the measures objective (See Methods)
10

 R Recruitment

: Were the subjects representative of

the target population?

* The best way to ensure that study groups are representative is to: @ Recruit potential subject sequentially and clearly describe the source of patients @ Only apply exclusion criteria that are relevant to study methods @ See the size of study groups

* Where do I find the information? Early in the Methods should tell you how patients were selected for the study

11

 A Allocation

: was the treatment allocation concealed before

randomisation and were the groups comparable at the start of trial?

@ Match groups as closely as possible in every way except for the intervention @ Allocation concealment used for randomisation : * centralised computer randomisation (the best one) * sealed envelopes: not as good because the allocation is not as well concealed *Where do I find the information? @ Methods: should tell you how patient were allocated and whether or not randomisation was concealed @ Results : see table of Baseline characteristics
12

 M Maintenance

: Was the comparable status of the

study groups maintained through equal management and adequate follow up?

Equal management: @ the only difference between the groups is factor being tested @ use an identical measurement strategy for everyone Adequate follow-up: @ subjects at the start = subjects at the end @ subject are analysed in the groups that they started * Where do I find the information? @ look in the Methods for the precise protocol @ look in Results for any further information and flowchart study
13

 Mbo Measurement, blinding, objectivity :

Were the outcomes measured with blinded subjects and assessors and/or objective measures?

Blinding
Best -- Double blind trial : subject and investigators (outcome assessors) both unware of group allocation Moderate -- Single blind trial: either the subject or investigators are unware of group allocation Worst -- Not blinded: subject and investigators both aware of group allocation * Where I do find the information? Methods section should describe how the outcome was assessed and whether the assessor/s were aware of the patients treatment
14

Question 3: What do the results mean and could they have been due to chance?

* Outcome measures
1. Binary outcomes:
Relative Risk/RR, Absolute Risk Reduction/ARR Relative Risk Reduction/RRR Number Needed to Treat/NNT

2. Continuous outcomes

* Are the results real and relevant?

P-values Confidence Interval/CI ( hypothesis testing) ( estimation)
15

16

THANK YOU

17