See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/256489554

The smallest worthwhile effect of nonsteroidal anti-inflammatory drugs and

physiotherapy for chronic low back pain: A benefit-harm trade-off study

Article  in  Journal of Clinical Epidemiology · September 2013

DOI: 10.1016/j.jclinepi.2013.02.018 · Source: PubMed

CITATIONS READS

54 320

7 authors, including:

Manuela Loureiro Ferreira Paulo H Ferreira

The University of Sydney The University of Sydney
219 PUBLICATIONS   13,470 CITATIONS    170 PUBLICATIONS   10,841 CITATIONS   

SEE PROFILE SEE PROFILE

Jane Latimer Raymond W Ostelo

The University of Sydney Vrije Universiteit Amsterdam
237 PUBLICATIONS   11,267 CITATIONS    312 PUBLICATIONS   30,909 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

An overview of low back pain in China View project

Infectious Disease Epidemiology View project

All content following this page was uploaded by Raymond W Ostelo on 16 January 2018.

The user has requested enhancement of the downloaded file.

 Journal of Clinical Epidemiology 66 (2013) 1397e1404

The smallest worthwhile effect of nonsteroidal anti-inflammatory

drugs and physiotherapy for chronic low back pain: a benefiteharm
trade-off study
Manuela L. Ferreiraa,*, Robert D. Herberta, Paulo H. Ferreirab, Jane Latimera,
Raymond W. Osteloc,d, Margreth Grotlee, Bruce Barrettf
a
The George Institute for Global Health, University of Sydney, Sydney, New South Wales, Australia
b
Clinical & Rehabilitation Sciences Research Group, University of Sydney, Sydney, New South Wales, Australia
c
Department of Health Sciences and the EMGO Institute for Health and Care Research, Faculty of Earth and Life Sciences, VU University,
Amsterdam, The Netherlands
d
Department of Epidemiology and Biostatistics and the EMGO Institute for Health and Care Research, VU University Medical Centre,
Amsterdam, The Netherlands
e
Section for Health Science, University of Oslo, Oslo, Norway
f
Department of Family Medicine, University of Wisconsin, Madison, WI, USA
Accepted 18 February 2013; Published online 8 September 2013

Abstract
Objective: The aim of this study was to determine the smallest worthwhile effects of two treatments for nonspecific low back pain
(LBP).
Study Design and Setting: The benefiteharm trade-off method was used to estimate the smallest worthwhile effect of nonsteroidal
anti-inflammatory drugs (NSAIDs) and physiotherapy for LBP. Patients seeking care for chronic LBP were interviewed by telephone before
treatment commenced and 4 weeks later.
Results: Patients need to see a median of 30% (interquartile range [IQR]: 10e40) more improvement in pain and 20% (IQR: 10e40)
more improvement in disability than would occur without intervention to perceive the effect of NSAIDs are worthwhile. They would need
to see 20% (IQR: 0e30) more improvement on pain and disability over natural recovery to perceive that the effect of physiotherapy was
worthwhile. There was no difference in estimates of the smallest worthwhile effect elicited at baseline and 4 weeks later.
Conclusions: People with chronic back pain need to see larger effects on pain of NSAIDS than physiotherapy to consider the effects of
these interventions worthwhile. These estimates of the smallest worthwhile effect can be used to interpret the findings of clinical trials and
to design adequately powered clinical trials. Ó 2013 Elsevier Inc. All rights reserved.
Keywords: Minimum clinically important difference; Sufficiently important difference; Low back pain; Clinical trials; Research design; Drug therapy

1. Introduction smallest worthwhile effects of a range of interventions.

Robust estimates of the smallest worthwhile effect of inter-
In 1989, Jaeschke et al. [1] defined the ‘‘minimum clin-
ventions can be used to inform sample size calculations
ically important difference’’ as ‘‘the smallest difference in in clinical trials and to interpret the findings of clinical
score in the domain of interest, which patients perceive as
trials [2].
beneficial and which would mandate, in the absence of
A recent systematic review located 31 studies and 129
troublesome side effects and excessive cost, a change in
estimates of the smallest worthwhile effect of interventions
the patient’s management.’’ Since Jaeschke’s seminal arti-
for nonspecific low back pain (LBP) [2]. Most of the stud-
cle, many studies have been conducted to ascertain the
ies identified in the review used anchor- or distribution-
based methods. These methods have important limitations
Part of this work was presented at the World Confederation of Physical that, we argue, mean they should not be used to inform
Therapy Congress; June 2011; Amsterdam, The Netherlands. sample size calculation for clinical trials or to interpret
* Corresponding author. The George Institute for Global Health, Uni-
versity of Sydney, PO Box M201, Missenden Road, New South Wales
treatment effects observed in clinical trials [2]. For in-
2050, Australia. Tel.: þ61-2-9657-0343; fax: þ61-2-9657-0301. stance, the review found that, of the 129 estimates elicited,
E-mail address: mferreira@georgeinstitute.org.au (M.L. Ferreira). only 5% were based directly on patients’ judgments, only
0895-4356/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved.
http://dx.doi.org/10.1016/j.jclinepi.2013.02.018
1398 M.L. Ferreira et al. / Journal of Clinical Epidemiology 66 (2013) 1397e1404
What is new?
 Patients with chronic low back pain (LBP) need
to see larger effects with nonsteroidal anti-
inflammatory drugs (NSAIDs) than with physiother-
apy to consider the intervention worthwhile.
 These estimates do not change over time and are in
general not associated with symptom severity, dura-
tion, or mood.
 The smallest worthwhile effects elicited in this study
reflect patients’ opinions; are based on between-
treatment differences; and consider the costs, risks,
and inconvenience of intervention.
 We advocate the use of these estimates in sample
size calculations and interpretation of trial findings
of NSAIDs or physiotherapy for chronic LBP.
4% were intervention specific (i.e., considered the costs,
risks, and inconveniences of intervention), and all were
based on changes in symptoms over time rather than on dif-
ferences in outcomes with and without intervention. The
latter is an important limitation of existing estimates be-
cause changes in outcomes that are measured over time
may partly reflect not only the effects of intervention but
can also be influenced by many other factors [3]. Effects
of intervention can only be understood in terms of differ-
ences in outcomes with and without intervention [4,5].
Thus, any attempts to identify the smallest worthwhile ef-
fects of intervention must define the smallest worthwhile
effect in terms of the difference in outcomes with and with-
out intervention [2].
In 2005, Barrett et al. [6e8] described the use of a
form of contingent analysis, the ‘‘benefiteharm trade-off
method,’’ to estimate the smallest worthwhile effect of
health interventions. This method has been previously used
to estimate the smallest worthwhile effect of interventions
for the common cold [7], cancer therapies [9e17], and larval
therapy [18]. It overcomes the limitations of anchor- and
distribution-based methods because it captures the judg-
ments of recipients of care; allows participants to weigh
the benefits of treatment against the risks, costs, and incon-
veniences of treatment; and potentially provides estimates
that are based on an interventionecontrol comparison.
In the present study, we use the benefiteharm trade-off
method to elicit estimates of smallest worthwhile effect
for two common treatments for nonspecific LBP, namely
nonsteroidal anti-inflammatory drugs (NSAIDs) and phys-
iotherapy (including manual therapy and exercise). In both
cases, the comparator was no intervention. The aims of the
study were to determine: (1) the distribution of the smallest
worthwhile effect for NSAIDs and physiotherapy, (2) if pa-
tients assign different smallest worthwhile effects after 4
weeks of intervention, and (3) if duration or severity of
symptoms (pain and disability) or mood (depression, stress
or anxiety) is associated with these estimates.
2. Methods
The study was approved by the University of Sydney
Human Research Ethics Committee (application 10859).
A sample of 102 patients with chronic nonspecific LBP
was recruited by inviting consecutive patients presenting
to two private physiotherapy practices in Sydney, Australia,
to participate between February 2009 and February 2010.
Patients with specific spinal pathology (e.g., nerve root
compromise, inflammatory disorders, fracture, or malig-
nancy) were excluded, as were those experiencing a back
pain episode of less than 3 months duration. The sample
size of 102 participants has a better than 80% power to de-
tect partial correlations among the six predictors and out-
comes of as small as 0.4, assuming a fully exchangeable
correlation among predictors of 0.2 [19]. The level of sig-
nificance was set at 0.05.
2.1. Estimation of the smallest worthwhile effect
All patients who gave consent were contacted by phone
before treatment commenced. During the interview, baseline
measures were obtained of pain intensity, disability, mood,
and duration of the current episode of back pain. Pain inten-
sity over the last 24 hours was measured on an 11-point pain
scale, anchored at ‘‘no pain’’ and ‘‘worst pain I have ever
experienced.’’ Disability was measured using the Roland
Morris Disability Questionnaire (a 0e24-point scale). Mood
was measured using the Depression, Anxiety, and Stress
Scale (DASS-21; a 0e42-point scale) [20]. These measures
were only used to describe the cohort and did not inform the
estimates of the smallest worthwhile effect. The benefite
harm trade-off method was used to obtain estimates, for
each participant, of the smallest worthwhile effect of
NSAIDs and of a course of physiotherapy for treatment of
LBP, when compared with no treatment. In addition, partic-
ipants were questioned about their age, gender, education
level, smoking, number of previous episodes of back pain,
presence of leg pain, country of birth, past experience with
NSAIDs and physiotherapy, and work status.
A trained interviewer interviewed each participant using
standardized scripts, which had been formally piloted on 10
patients with LBP (Appendix; available on the journal’s
website at www.jclinepi.com). First, the interviewer de-
scribed to the participant how much improvement could
be expected in the next 2 weeks without any treatment (nat-
ural recovery). This was the counterfactual against which
the outcome was compared. Specifically, the interviewer
indicated that, without treatment, the participant could
expect a 30% improvement in pain and disability, and re-
covery from the current episode of pain in 14 days. The
 M.L. Ferreira et al. / Journal of Clinical Epidemiology 66 (2013) 1397e1404 1399

interviewer then outlined in lay and general terms how the Table 1. Participant characteristics at baseline and after 4 weeks of
two interventions (NSAIDs and physiotherapy) were usu- treatment (N 5 102)
ally administered, as well as their costs and adverse events. Sample characteristics Mean (SD)
After the description, each participant was asked how much Age (yr) 45.2 (12.8)
additional improvement in symptom severity (in addition to Duration of symptoms (yr) 6.9 (9)
natural recovery) he or she would expect if given this treat- Number of previous episodes of low back pain 1 (2.6)
Pain severity at baseline (0e10) 3.5 (1.9)
ment. All participants indicated that they believed the ef- Pain severity at follow-up (0e10) 2.8 (2)a
fects of the magnitude they expected to experience were Disability level at baseline (0e24) 7.7 (4.9)
large enough to make the intervention worthwhile. The size Disability level at follow-up (0e24) 5.3 (5.2)a
of this hypothetical effect was then progressively reduced; Depression at baseline (0e42) 3.0 (6.3)
and with each iteration, the participants were asked again Anxiety at baseline (0e42) 2.6 (4.7)
Stress at baseline (0e42) 6.8 (7.9)
whether they would consider the effect large enough to Global perceived effect of treatment (11-point) 2.5 (1.7)
make the costs, risks, and inconveniences of intervention Gender (women), n (%) 78 (80)
worthwhile. The smallest expected benefit of intervention Educational level, n (%)
for which the participant would choose to have the inter- High school degree 14 (14)
vention was the smallest worthwhile effect for that patient. College degree 13 (13)
Bachelor/science degree 41 (42)
The procedure was repeated for each outcome (pain, dis- Postgraduate degree 32 (33)
ability, and time to recovery). The effects of each interven- Current smoker, n (%) 8 (8)
tion on pain and disability were expressed as percentages of Past experience with NSAIDs, n (%) 82 (83)
symptom reduction. The effects of each intervention on Past experience with physiotherapy, n (%) 86 (89)
time to recovery were measured in days. Presence of leg pain, n (%) 64 (65)
Work status, n (%)
Once the interview was finished, participants continued Not working 17 (17)
with the treatment initially planned for them. Treatment Working part-time 30 (31)
was chosen at the clinician’s discretion and did not neces- Working full-time 52 (53)
sarily include the interventions involved in the benefite Abbreviations: SD, standard deviation; NSAID, nonsteroidal anti-
harm trade-off interviews. Participants were reinterviewed inflammatory drug.
using the same procedures 4 weeks later. a
Indicates a statistically significant decrease from baseline; P !
0.001.

2.2. Analysis
Regression methods were used to quantify associations
between estimates of smallest worthwhile effect and base-
line pain intensity, disability, mood (depression, stress,
and anxiety), and duration of pain. These potential predic-
tors were chosen a priori. First, univariate analyses were
performed. The strongest predictors (P ! 0.2) were then
entered in a multivariate model, and a backward stepwise
approach was used to remove variables that did not contrib-
ute significantly to the model. The confidence intervals
(CIs) for regression coefficients were obtained using non-
parametric bootstrap methods (BCa CIs) [21]. Data were
analyzed using STATA 12 (StataCorp. 2011; Stata Statisti-
cal Software: Release 12. StataCorp LP, College Station,
TX). This study was funded by the Clive and Vera Rama-
ciotti Foundation, Australia.
3. Results
A total of 151 consecutive patients seeking physiother-
apy in private practices for back pain were invited to partic-
ipate in the study. Of these, 102 were eligible to participate,
consented, and were included in the study. Reasons for non-
inclusion were: (1) unwilling to participate in the inter-
views (n 5 30), (2) not presenting with nonspecific LBP
(n 5 11), (3) disconnected telephone (n 5 3), and (4)
participating in ongoing treatment (n 5 1). Participants’
characteristics are presented in Table 1.
3.1. Smallest worthwhile effects of interventions before
treatment commencement
Figures 1, 2 and 3 present the distribution of estimates of
the smallest worthwhile effects for pain, disability and days
to recover, respectively.
Approximately 25% of the participants stated they would
not take NSAIDs for back pain, irrespective of the additional
benefits provided, after considering the costs and risks in-
volved. The remaining participants (n 5 77) stated that they
would need to see a median of 30% (interquartile range
[IQR]: 10e40) more improvement in pain and a median
20% (IQR: 10e40) more improvement in disability than they
would with no treatment to make the costs, risks, and incon-
veniences of NSAIDs worthwhile. Participants also stated
they would need to recover a median of 7 (IQR: 4e9;
n 5 77) days sooner than they would with no treatment to
make the costs, risks, and inconveniences of NSAIDs worth-
while (Table 2).
Approximately 7% of the participants said that they
would not choose to have physiotherapy, irrespective of
its benefits. The remaining 95 participants nominated that
they would need to see a median of 20% or 20 points on
a 100-point scale (IQR: 0e30) improvement on pain and
1400 M.L. Ferreira et al. / Journal of Clinical Epidemiology 66 (2013) 1397e1404

A A
50th pc 90th pc
50th pc 90th pc

30
30

Frequency
Frequency

20
20

10
10

0
0 20 40 60 80
0

0 20 40 60 80 Smallest worthwhile effect (/100)

Smallest worthwhile effect (/100)

B B
50th pc 90th pc
50th pc 90th pc

30
30

Frequency
Frequency

20
20

10
10

0
0

0 20 40 60 80
0 20 40 60 80
Smallest worthwhile effect (/100)
Smallest worthwhile effect (/100)
Fig. 2. Distribution of estimates of the smallest worthwhile effects of
Fig. 1. Distribution of estimates of the smallest worthwhile effects of
disability at baseline. The corresponding 50th and 90th percentiles
pain at baseline. The corresponding 50th and 90th percentiles are in-
are indicated by arrows.
dicated by arrows.

disability compared with no treatment to perceive that the
effect of physiotherapy was worthwhile. Participants per-
ceived that they needed to recover a median of 5 (IQR;
0e6; n 5 96) days sooner than they would with natural re-
covery to make the effects of physiotherapy worthwhile.
Additionally, 90% of participants would consider physio-
therapy worthwhile if they experienced an additional 40%
improvement on pain and disability (Table 2).
Physiotherapy had significantly smaller smallest worth-
while effects than NSAIDs for pain (paired t-test,
P 5 0.009) and recovery (P ! 0.001), but not disability
(P 5 0.065, Table 2). No univariate associations were ob-
served among the smallest worthwhile effect for pain, dis-
ability, or recovery and the predictors of baseline pain
severity, disability severity, and mood status (Table 3). Du-
ration of symptoms was significantly associated with esti-
mates of smallest worthwhile effect of physiotherapy for
pain and disability, but not with days to recovery. A multi-
variate analysis showed that duration of symptoms was sig-
nificantly associated with smallest worthwhile effect on
pain and disability after adjusting for pain severity
(Table 4). The associations were negative, indicating that
the longer the duration of symptoms, the smaller the esti-
mates of smallest worthwhile effect. No other factors were
retained in the multivariate models (P O 0.20).
3.2. Smallest worthwhile effect of interventions
following a course of treatment
A total of 95 (93%) participants were followed-up 4
weeks after the first interview to assess whether estimates
of the smallest worthwhile effect change after a course of
treatment. Reasons for declining participation in follow-
ups were unwilling to participate in the interviews (n 5 5)
and disconnected telephone (n 5 2).
On average, at follow-up, participants had significantly
reduced levels of pain (mean change from baseline: 0.8;
95% CI: 0.3, 1.2 on an 11-point scale; P 5 0.001) and dis-
ability (mean change from baseline: 2.6; 95% CI: 1.9, 3.3;
P ! 0.0001). There were small, statistically nonsignificant
increases in estimates of smallest worthwhile effect of
NSAIDs on pain (mean change from baseline: 2.0; 95%
 M.L. Ferreira et al. / Journal of Clinical Epidemiology 66 (2013) 1397e1404 1401

A slightly, but not significantly, larger at follow-up compared

with before a course of treatment.
50th pc 90th pc
30

4. Discussion
These data demonstrate that typically, a patient with
Frequency

20

chronic LBP who are willing to take NSAIDS feel that

NSAIDs must reduce pain and disability by at least 20
points on a 100-point scale and hasten time to recovery
10

by at least a week to be worth the costs, risks, and inconve-

niences of NSAIDs. A patient willing to receive physiother-
apy feels that physiotherapy must reduce pain and disability
0

0 5 10 15
by at least 15 points on a 100-point scale and hasten time to
Smallest worthwhile effect (/100) recovery by at least 2 days to be worth the costs, risks, and
inconveniences of physiotherapy. These perceptions do not
B seem to be associated with severity of symptoms, mood, or
50

symptom intensity.
This is the first time the benefiteharm trade-off method
40

has been used to elicit estimates of the smallest worthwhile

50th pc 90th pc effect of interventions for LBP. The method overcomes
three major flaws of previously used methods [2,22]dit
Frequency

30

provides the first estimates of clinical significance of inter-

ventions for back pain which are based on the judgment of
20

recipients of care; incorporate consideration of the costs,

risks, and inconveniences of therapy; and are expressed in
10

terms of between-group differences (i.e., effect of interven-

tion). We argue that these characteristics of estimates of the
0

0 5 10
smallest worthwhile effect are needed if the estimates are to
Smallest worthwhile effect (/100) be used in the design and interpretation of randomized clin-
ical trials and meta-analyses [2]. We have recently demon-
Fig. 3. Distribution of estimates of the smallest worthwhile effects of
strated how estimates of the smallest worthwhile effect of
days to recover at baseline. The corresponding 50th and 90th percen-
tiles are indicated by arrows. interventions can be used to calculate sample sizes of future
trials using extended contour funnel plots [23]. We suggest
that a new trial should be designed to provide enough
CI: 3.4, 7.5), disability (mean change from baseline: 3.5; power to convert uncertainty regarding the clinical signifi-
95% CI: 1.3, 8.3), and days to recovery (mean change from cance of current evidence into certainty that treatment ef-
baseline: 0.1; 95% CI: 1, 1). Estimates of smallest worth- fects are worthwhile.
while effect of physiotherapy on pain (mean change from Typically, patients seeking care for chronic LBP per-
baseline: 3.5; 95% CI: 1, 8.1), disability (mean change ceive that the average effects of NSAIDs and exercise re-
from baseline: 3.9; 95% CI: 0.4, 8.1), and days to recovery ported in randomized trials and systematic reviews are
(mean change from baseline: 0.6; 95% CI: 0.2, 1.5) were too small to make the interventions worth their costs, risks,
Table 2. Estimates of the smallest worthwhile effect of interventions for back pain
NSAIDs Physiotherapy
10th 50th 90th 10th 50th 90th Mean effect
Parameters N Mean (SD) Percentile Percentile Percentile N Mean (SD) Percentile Percentile Percentile (95% CI); n
Baseline
Paina 77 29.0 (19.6) 10 30 60 95 19.0 (17.8) 0 20 40 7.4 (1.9, 13); 74
Disabilitya 74 26.3 (17.9) 10 20 50 95 18.2 (18.0) 0 15 40 4.8 (0.3, 10); 72
Recoveryb 77 6 (3.8) 0 7 11 96 3 (3.5) 0 2 8 2.6 (1.6, 3.6); 75
4 wk
Paina 55 31.6 (21.1) 3 30 65 82 21.2 (18.2) 0 30 50 5.7 (0.4, 11); 53
Disabilitya 55 31.5 (20.2) 3 30 65 82 20.9 (19.4) 0 30 50 4.7 (1.1, 10.5); 53
Recoveryb 53 6 (3.4) 0 6 11 82 4 (3.6) 0 4 9 1 (0.2, 1.9); 52
Abbreviations: NSAID, nonsteroidal anti-inflammatory drug; SD, standard deviation; CI, confidence interval.
a
Percent improvement.
b
Days.
1402 M.L. Ferreira et al. / Journal of Clinical Epidemiology 66 (2013) 1397e1404

and inconvenience. For instance, a recent meta-analysis

0.21 (0.45, 1.30; 0.62)

0.32 (0.38, 1.70; 0.50)
0.06 (0.07, 0.19; 0.39)

0.02 (0.58, 0.90; 0.96)

0.14 (0.46, 1.18; 0.72)
0.04 (0.07, 0.12; 0.43)
concluded that the effects of NSAIDs is to reduce pain

Depressiona (0e42)
by, on average, 12.4 points (95% CI: 15.5, 9.7) com-
pared with placebo in people with chronic LBP [24]. Our
findings suggest that a typical patient seeking care for
chronic LBP would not consider these effects large enough
to be worthwhile.
The benefiteharm trade-off method has been success-
fully used since 1978 to ascertain the smallest worthwhile
0.32 (0.95, 1.26; 0.51)
0.16 (0.73, 1.34; 0.76)
0.05 (0.10, 0.18; 0.49)

0.15 (0.84, 0.44; 0.64)

0.08 (0.68, 0.55; 0.82)
0.06 (0.06, 0.15; 0.24)
effect of interventions outside the musculoskeletal field
[7e13,15e18,25]. However, previous studies using the
Stressa

benefiteharm trade-off method have not always defined

the smallest worthwhile effect in terms of between-group
differences. In the present study, this was explicitly incor-
porated into the interviews. Participants were asked to con-
Abbreviations: CI, confidence interval; NSAID, nonsteroidal anti-inflammatory drug; DASS-21, Depression, Anxiety and Stress Scale-21.

sider the clinical importance of the additional benefits of

either physiotherapy or NSAIDs over and above natural re-
0.31 (0.81, 2.13; 0.66)
0.44 (0.95, 2.23; 0.60)
0.09 (0.05, 0.25; 0.26)

0.13 (0.88, 0.94; 0.77)

0.07 (0.85, 1.23; 0.90)
0.08 (0.08, 0.20; 0.27)

covery. Randomized clinical trials are designed to establish

Anxietya (0e42)

the causal effects of treatments, which are defined in terms

of the difference between outcomes with and without treat-
ment [4], so any estimate of the smallest worthwhile effect
that is to be used to design or interpret the findings of ran-
domized trials must also be defined in terms of between-
group differences.
Table 3. Univariate (95% CI; P-value) analyses of predictors of the smallest worthwhile effect for low back pain

A recently published consensus on what constitutes the

0.07 (0.123, 0.01; 0.05)
0.51b,c (0.81, 0.15; 0.002)
0.49b,c (0.77, 0.21; 0.001)

minimal important change, based on the opinions of re-

0.09 (0.47, 0.60; 0.74)
0.07 (0.59, 0.37; 0.77)
0.02 (0.09, 0.14; 0.69)

Predictors that reached a level of significance of 0.2 or less were entered in multivariate analyses.
Duration of symptoms (yr)

searchers, suggested that a 30% change in outcome from

baseline is a clinically meaningful improvement [26]. In
our study, participants nominated different worthwhile ef-
fects for different interventions. On average, the smallest
worthwhile effects of a course of NSAIDs were a third
larger than those for physiotherapy, and more participants
declared that they would not consider taking NSAIDS
(25%) than physiotherapy (7%). These findings demon-
0.12 (2.43, 2.45; 0.92)

1.5b (3.72, 0.28; 0.14)

1.60b (3.74, 0.19; 0.11)
0.94 (2.77, 1.40; 0.37)
0.15 (0.36, 0.60; 0.54)

0.10 (0.47, 0.27; 0.59)

strate that the smallest worthwhile effect of an intervention

should not be evaluated without reference to the costs and
Pain severity (0e10)

inconveniences of the intervention [1], and that the evalua-

tions of clinical importance must be intervention specific.
The smallest worthwhile effect of inexpensive and risk-
free interventions (such as provision of an information
booklet) would be expected to be much smaller than the
smallest worthwhile effect of major interventions such as
spinal surgery.
0.009 (0.98, 1.05; 0.99)
0.005 (0.88, 1.20; 0.99)

0.08 (0.07, 0.20; 0.28)

0.07 (0.11, 0.24; 0.43)

0.13 (1.02, 0.58; 0.56)

0.15 (1.03, 0.67; 0.35)

If estimates of the smallest worthwhile effect are to be

Disability level (0e24)

Statistically significant at P ! 0.05.

used in sample size calculations or for interpretation of

clinical trials and systematic reviews, the distribution of
the smallest worthwhile effect needs to be collapsed to
a single figure. For example, clinical trialists might base
Subscales of DASS-21.

sample size calculations on the mean smallest worthwhile

effect. In that case, the trial would be powered to detect
an effect, which, on average, participants feel would be just
Days to recovery
Days to recovery

worthwhile. Alternatively, the 10th or 90th percentiles of

the distribution of worthwhile effects (Table 2) could be
Physiotherapy
Disability

Disability
Parameters

used to design clinical trials, which would be powered to

NSAIDs
Pain

Pain

detect effect sizes that are large enough to meet the needs
b
a

of 10% or 90% of patients with chronic LBP, respectively.

 M.L. Ferreira et al. / Journal of Clinical Epidemiology 66 (2013) 1397e1404
Table 4. Multivariate regression coefficients (95% CI; P-value) for predictors of the smallest worthwhile effect for low back pain
Physiotherapy
Parameters Pain severity (0e10)
Pain 1.23 (3.17, 0.71; 0.21)
Disability 1.32 (3.10, 0.46; 0.15)
Abbreviation: CI, confidence interval.
The same applies when interpreting the clinical significance
of treatment effects provided in clinical trials: readers of re-
ports of clinical trials might consider that the effects of the
intervention examined in the randomized trial were big
enough to be worthwhile if they exceeded the smallest
worthwhile effect of the average patient, or alternatively,
of 10% or 90% of the patients.
In this study, we focus on the clinical importance of group
differencesdnot on individual patient improvementsd
and argue that the findings can be used by clinicians and
policy makers to select treatment programs with worthwhile
effects and researchers to design clinical trials powered to
detect worthwhile effects. It is usually not possible, after a
period of intervention, to determine if an individual patient
actually experienced an effect of intervention that was at least
as large as his or her smallest worthwhile effect because the
effects of interventions on an individual usually cannot be as-
certained. We cannot know what the effect of intervention is
on any individual patient because the effect of intervention is
the difference in outcomes with and without intervention, but
we can only observe outcomes with or without intervention at
any one time [4].
We acknowledge that the smallest worthwhile effect of
an intervention may depend on perspective. Thus, patients
may have different judgments to clinicians, health service
providers, or policy makers about how big the effect of an
intervention needs to be to justify costs, risks, and incon-
veniences. However, a minimum requirement for provi-
sion of an intervention is that the potential consumers of
health care services can expect that the intervention will
have a worthwhile effect. For that reason, we have focused
on the perspective of patients. We acknowledge that al-
though this perspective is necessary, it is not sufficient,
and that other criteria such as cost-effectiveness are neces-
sary for comprehensive justification of provision of a
health intervention.
A limitation of this study is that these estimates might
have been influenced by the characteristics of the study
sample, which may not be broadly representative of people
seeking physiotherapy for persistent LBP. It seems, for ex-
ample, that the sample consisted of a large proportion of
well-educated people, and the sample was restricted to peo-
ple who were seeking physiotherapy care. This might ex-
plain some or all of the observed difference in estimates
of the smallest worthwhile effect for NSAIDs and physio-
therapy. The estimates should be interpreted in the context
of the included population, as it is possible that different es-
timates would be elicited from different populations or
1403
Duration of symptoms (wk)
0.45 (0.82, 0.12; 0.01)
0.45 (0.81, 0.09; 0.02)
patients seeking other types of care for their LBP. Another
limitation is that, in the benefiteharm trade-off method,
subjects are presented with a series of scenarios that differ
in just one attributedthe hypothetical size of the effect of
intervention, whereas all other attributes are either explic-
itly held constant or left undefined. This might explain
why the smallest worthwhile effect of intervention can vary
greatly among participants [8,27]: it could be that the par-
ticipants assign different values to undefined attributes.
In the present study, we use a stated preference method
to estimate the smallest worthwhile effect. Stated pre-
ferences methods (for treatments) have been previously
compared [28] and criterion validated against revealed pref-
erences methods [29]. In the specific context of the present
study, cross-validation with revealed preferences is not pos-
sible because it is not possible to observe a treatment effect
on an individual [4]. A possible alternative is to validate
findings of the benefiteharm trade-off study by identifying
which attributes of intervention are most highly valued by
patients when nominating the smallest worthwhile effect.
This can be done using other stated preference methods
such as the discrete choice experiments. We are currently
conducting a discrete choice study to cross-validate esti-
mates of the smallest worthwhile effect in older people
who are offered to participate in physical activity programs
to prevent falls.
Acknowledgments
R.D.H. is supported by the National Health and Medical
Research Council of Australia and J.L. is supported by the
Australian Research Council. None of the authors have any
financial or other relationships that might lead to a conflict
of interest. All authors have contributed significantly to the
study.
Appendix
Supplementary data
Supplementary data related to this article can be found
online at http://dx.doi.org/10.1016/j.jclinepi.2013.02.018.
References
[1] Jaeschke R, Singer J, Guyatt GH. Ascertaining the minimal clinically
important difference. Control Clin Trials 1989;10:407e15.
1404 M.L. Ferreira et al. / Journal of Clinical Epidemiology 66 (2013) 1397e1404
[2] Ferreira ML, Herbert RD, Ferreira PH, Latimer J, Ostelo RW,
Nascimento DP, et al. A critical review of methods used to determine
the smallest worthwhile effect of interventions for low back pain.
J Clin Epidemiol 2012;65:253e61.
[3] Herbert RD, Jamtvedt G, Mead J, Hagen KB. Outcome measures
measure outcomes, not effects of intervention. Aust J Physiother
2005;51:3e4.
[4] Rubin DB. Formal modes of statistical inference for causal effects.
J Stat Plan Inference 1990;25:279e92.
[5] Heckman JJ. The scientific model of causality. Sociol Methodol
2005;35:1e97.
[6] Barrett B, Brown D, Mundt M, Brown R. Sufficiently important dif-
ference: expanding the framework of clinical significance. Med Decis
Making 2005;25:250e61.
[7] Barrett B, Brown R, Mundt M, Dye L, Alt J, Safdar N, et al. Us-
ing benefit harm tradeoffs to estimate sufficiently important differ-
ence: the case of the common cold. Med Decis Making 2005;25:
47e55.
[8] Barrett B, Haraban B, Brown D, Zhang Z, Brown R. Sufficiently im-
portant difference for common cold: severity reduction. Ann Fam
Med 2007;5:216e23.
[9] Duric VM, Butow PN, Sharpe L, Heritier S, Boyle F, Beith J, et al. Com-
paring patients’ and their partners’ preferences for adjuvant chemother-
apy in early breast cancer. Patient Educ Couns 2008;72(2):239e45.
[10] McNeil BJ, Weichselbaum R, Pauker SG. Fallacy of the five-year sur-
vival in lung cancer. N Engl J Med 1978;299:1397e401.
[11] Bremnes RM, Andersen K, Wist EA. Cancer patients, doctors and
nurses vary in their willingness to undertake cancer chemotherapy.
Eur J Cancer 1995;31A:1955e9.
[12] Duric V, Stockler M, Heritier S, Boyle F, Beith J, Sullivan A, et al.
Patients’ preferences for adjuvant chemotherapy in early breast can-
cer: what makes AC and CMF worthwhile now? Ann Oncol 2005;16:
1786e94.
[13] Hirose T, Horichi N, Ohmori T, Kusumoto S, Sugiyama T, Shirai T,
et al. Patients preferences in chemotherapy for advanced non-small-
cell lung cancer. Intern Med 2005;44:107e13.
[14] McNeil BJ, Weichselbaum R, Pauker SG. Speech and survival: trade-
offs between quality and quantity of life in laryngeal cancer. N Engl J
Med 1981;305:982e7.
[15] Silvestri G, Pritchard R, Welch HG. Preferences for chemotherapy in
patients with advanced non-small cell lung cancer: descriptive study
based on scripted interviews. BMJ 1998;317:771e5.
[16] Yellen SB, Cella DF, Leslie WT. Age and clinical decision making in
oncology patients. J Natl Cancer Inst 1994;86:1766e70.
View publication stats
[17] Jansen S, Lievit J, Nooij M, Haes J, Overpelt I, Slooten H, et al. Pa-
tients’ preferences for adjuvant chemotherapy in early-stage breast
cancer: is treatment worthwhile? Br J Cancer 2001;84:1577e85.
[18] Petherick E, O’Meara S, Spilsbury K, Iglesias C, Nelson E,
Torgerson D. Patient acceptability of larval therapy for leg ulcer treat-
ment: a randomised survey to inform the sample size calculation of
a randomised trial. BMC Med Res Methodol 2006;6:43e6.
[19] Maxwell S. Sample size and multiple regression analysis. Psychol
Methods 2000;5:434e58.
[20] Antony MM, Bieling PJ, Cox BJ, Enns MW, Swinson RP. Psycho-
metric properties of the 42-item and 21-item versions of the De-
pression Anxiety Stress Scales (DASS) in clinical groups and
a community sample. Psychol Assess 1998;10:176e81.
[21] Efron B, Tibshirani RJ. An introduction to the bootstrap. Monographs
on statistics and applied probability. New York, NY; London, UK:
Chapman & Hall; 1993:xvi, 436.
[22] Ferreira ML, Herbert RD. What does ’clinically important’ really
mean? Aust J Physiother 2008;54:229e30.
[23] Ferreira ML, Herbert RD, Crowther MJ, Verhagen A, Sutton AJ.
When is a further clinical trial justified? BMJ 2012;345:e5913.
[24] Kuijpers T, van Middelkoop M, Rubinstein S, Ostelo R, Verhagen A,
Koes B, et al. A systematic review on the effectiveness of pharmaco-
logical interventions for chronic non-specific low back pain. Eur
Spine J 2011;20:40e50.
[25] Blinman P, King M, Norman R, Viney R, Stockler M. Preferences for
cancer treatments: an overview of methods and applications in oncol-
ogy. Ann Oncol 2012;23:1104.
[26] Ostelo RW, Deyo RA, Stratford P, Waddell G, Croft P, Von Korff M,
et al. Interpreting change scores for pain and functional status in low
back pain: towards international consensus regarding minimal impor-
tant change. Spine 2008;33(1):90e4.
[27] Ferreira ML, Ferreira PH, Herbert RD, Latimer J. People with low
back pain typically need to feel ‘much better’ to consider intervention
worthwhile: an observational study. Aust J Physiother 2009;55:
123e7.
[28] Onwujekwe O, Dike N, Ojukwu J, Uzochukwu B, Ezumah N, Shu E,
et al. Consumers stated and revealed preferences for community
health workers and other strategies for the provision of timely and
appropriate treatment of malaria in southeast Nigeria. Malar J
2006;5:117.
[29] Onwujekwe O. Criterion and content validity of a novel structured
haggling contingent valuation question format versus the bidding
game and binary with follow-up format. Soc Sci Med 2004;58(3):
525e37.