Professional Documents
Culture Documents
Randomized Controlled Trial Quality in Pediatric Physical Therapy
Randomized Controlled Trial Quality in Pediatric Physical Therapy
ORIGINAL RESEARCH
Matteo Paci, PT, Msc1 , Niccolo` Landi, PT2 , Mariangela Marchettini, PT1 ,
& Marco Baccini, MSc3
1
Unit of Functional Rehabilitation, Prato Hospital, Prato, Italy, 2School of
Physiotherapy, University of Florence, Florence, Italy, 3Unit of Functional
Rehabilitation, Motion Analysis Laboratory, Azienda Sanitaria di Firenze,
Florence, Italy
The randomized controlled trial (RCT) is traditionally considered the gold stan-
dard for examining the efficacy of interventions. Recent articles have shown a
higher number and a faster increase of RCTs in studies involving adult
population, when compared to pediatric literature (Cohen et al., 2010; Cohen,
Uleryk, Jasuja, & Parkin, 2007; Martinez-Castaldi, Silverstein, & Bauchner,
2008; Thomson et al., 2010). In the field of physical therapy, RCTs represent a
small part of the totality of research articles, but their rate is comparable to
other disciplines (Kocak, Unver, & Karatosun, 2011; Paci, Briganti, &
Lombardi, 2011; Paci, Cigna, Baccini, & Rinaldi, 2009). Some authors suggest
that there are population-related specific reasons for the lack of RCTs in
pediatric literature, such as parental reluctance to agree to the participation of
their children in research studies involving unproven treatments,
Address correspondence to: Dr Matteo Paci, Prato Hospital, Unit of Functional Rehabilitation, Via Bottego,
4 50127 Firenze, Prato, Italy (E-mail: mpaci@usl4.toscana.it)
(Received 29 August 2012; accepted 8 July 2013)
260
RCT Quality in Pediatric PT 261
the difficulty of recruiting adequate sample sizes, in particular for uncommon con-
ditions, and ethical concerns (Caldwell, Murphy, Butow, & Craig, 2004; Thomson
et al., 2010).
In the pediatric physical therapy (PPT), specific barriers have been reported
for conducting clinical trials (Svien, Anderson, & Long, 2006), such as lack of
sensitive measurement tools, ambiguous description of interventions, natural
development of children over time, and the natural evolution and
pathophysiology of the health condition (Parette, Hendricks, & Rock, 1991).
These barriers may also account for the small number of RCTs. In fact, an
analysis of the type of articles published in the journal PPT indicated that only 4
RCTs (1.5% of total published articles) were published within the first 15 years
of publication (1989 to 2003) (Svien et al., 2006). Poor methodological quality
can affect the internal and external validity of an RCT and the application of
results in clinical practice. A number of validated tools have been used in
pediatric research to evaluate methodological quality of an RCT such as Jadad
scale (Cohen et al., 2007; Thomson et al., 2010) and the Consoli- dated
Standards of Reporting Trials (CONSORT) statement (Anttila, Malmivaara,
Kunz, Autti-Ra¨ mo¨ , & Ma¨ kela¨ , 2006; DeMauro, Giaccone, Kirpalani, &
Schmidt, 2011). Anttila et al., (2006) found that only a small number of RCTs
on the effec- tiveness of physical therapy for children with cerebral palsy
extensively reported criteria for methodological quality included on the
CONSORT checklist. In addi- tion, reported quality of RCTs published in
2005–2009 in six pediatric or general medical journals was found to be poor,
particularly in pediatric journals (DeMauro et al., 2011). On the other hand,
Thomson et al. (2010) showed a significant increase over time of
methodological quality, assessed with the Jadad score, in a sample of
578 trials extracted from the Trials Register of the Cochrane Child Health Field.
The most widely used tool to assess the quality of physical therapy
interventions clinical trials is the Physiotherapy Evidence Database (PEDro)
score (Olivo et al., 2008), which seems to be a more comprehensive measure of
methodological qual- ity when compared to Jadad score (Bhogal, Teasell,
Foley, & Speechley, 2005). A recent paper by Moseley, Herbert, Maher,
Sherrington, & Elkins (2011) reported an improvement with time of the total
PEDro score for physical therapy interven- tions clinical trials, which increased
by an average of about 0.6 points each decade between 1960 and 2009.
Despite previous studies to assess reported quality of RCTs in pediatric
(Anttila et al., 2006; DeMauro et al., 2011; Thomson et al., 2010) or physical
therapy lit- erature (Moseley et al., 2011), no investigation assessed
methodological quality specific to PPT. The aim of this study was to assess the
reported quality of RCTs published in the PPT using a standardized scale and
evolution of methodological quality over time.
METHODS
A search of the PEDro database was carried out for articles published after
1962 through May 9, 2012, limited to “pediatric” subdiscipline and “clinical tri-
als” method. PEDro is a free database of randomized trials, systematic reviews
and clinical practice guidelines in physiotherapy. Each record on PEDro is cat-
egorized using a series of PEDro codes for the subdiscipline of physiotherapy
(e.g., “pediatric”). Specific criteria are used to define which clinical trials, system-
atic reviews, and evidence-based clinical practice guidelines are archived in PEDro.
Once archived, all RCTs on PEDro are independently assessed for quality by two
raters. A third rater resolves any disagreements. When this has been done the rat-
ing process is compleded. (source: http://www.pedro.org.au/). Article title, journal
name, year of publication, ratings for each of the 11 items of the PEDro scale, and
total PEDro score of all RCTs that had complete PEDro scale ratings were down-
loaded from PEDro database.
Data Analysis
The quality of RCTs over time was investigated with four different analyses:
(1) the total score of each selected article and the year of publication were
correlated using the Pearson correlation coefficient (r), (2) after clustering
PEDro scores on the basis of decades of publication, the Jonckheere–Terpstra
nonparametric test (J-T) was used to verify whether the medians of the PEDro
scores increased decade after decade; (3) the Student t-test was used to compare
the PEDro mean scores of trials published before and after the publication of
the CONSORT checklist (1996) to test the hypothesis that this has improved the
methodological description of the RCT; (4) A stepwise multiple regression was
performed to test if years and the introduction of the CONSORT checklist were
significantly associated with the total
TABLE 1. PEDro Scale Items
1. Eligibility criteria
This criterion is satisfied if the report describes the source of subjects and a list of criteria used to
determine who was eligible to participate in the study.
2. Random allocation
A study is considered to have used random allocation if the report states that the allocation was
random. The precise method of randomization need not be specified.
3. Concealed allocation
Concealed allocation means that the person who determined if a subject was eligible for inclusion in
the trial was unaware, when this decision was made, of which group the subject would be
allocated to.
4. Baseline comparability
In studies of therapeutic interventions, the report must describe at least one measure of the severity
of the condition being treated and at least one (different) key outcome measure at baseline.
5. Blind subjects
6. Blind therapists
7. Blind assessors
Critera 5–7. Blinding means the person in question (subject, therapist or assessor) did not know
which group the subject had been allocated to. In trials in which key outcomes are self-reported, the
assessor is considered to be blind if the subject was blind.
8. Adequate follow-up
This criterion is only satisfied if the report explicitly states both the number of subjects initially
allocated to groups and the number of subjects from whom key outcome measures were obtained.
In trials in which outcomes are measured at several points in time, a key outcome must have been
measured in more than 85% of subjects at one of those points in time.
9. Intention-to-treat analysis
An intention to treat analysis means that, where subjects did not receive treatment (or the control
condition) as allocated, and where measures of outcomes were available, the analysis was
performed as if subjects received the treatment (or control condition) they were allocated to.
10. Between-group comparisons
A between-group statistical comparison involves statistical comparison of one group with another.
Depending on the design of the study, this may involve comparison of two or more treatments, or
comparison of treatment with a control condition.
11. Point estimates and variability
A point measure is a measure of the size of the treatment effect. The treatment effect may be
described as a difference in group outcomes, or as the outcome in (each of) all groups. Measures of
variability include standard deviations, standard errors, confidence intervals, interquartile ranges (or
other quantile ranges), and ranges.
Source (modified): http://www.pedro.org.au/english/downloads/pedro-scale/
PEDro score. The level of statistical significance was set at .05. Data were analyzed
using SPSS 13.0 software for Windows.
RESULTS
One thousand three hundred sixty-seven (1,367) RCTs related to PPT and pub-
lished after 1962 were indexed on PEDro and extracted for this study. The
number of RCTs published each year has increased over time (Figures 1 and 2).
The PE- Dro score is significantly correlated =with years (r 0.28; p < .001),
indicating higher PEDro scores are associated with more recent year of
publication (Figure 3). The
year published explained 7.7% of the variability in PEDro score (R2 = 0.077). With
100
90
80
70
No of articles
60
50
40
30
20
10
0
Years
the standard PEDro scale cut-off of≥6, only 29% of articles (n=391) were clas-
sified high-quality studies, while using the modified cut-off of≥5 the percentage
increased to 56% (n =757). When considering the trend in the average score by
decade, a steady increase can be observed. On average, the PEDro score has in-
= p < .001) (Figure 4).
creased each decade by 0.63 points (J-T 8.87,
The percentage of RCTs that met the criteria for each item on the PEDro
scale is listed in Figure 5. The items with the lowest percentages were: blind
therapists=(0.6%; n 8), intention-to-treat analysis (15.5%; n 209); concealed
allocation (19.9%;
= n 269), and blind assessors (31.1%; n 420). There was also a
slight effect of the introduction in 1996 of the CONSORT checklist: dividing
the sample into two groups, the first comprising the years 1962–1995 and the
second comprising the years 1996–2011, a significant increase in the average
score in the second group
800
700
600
No of articles
500
400
300
200
100
0
1962 - 1971 1972 - 1981 1982 - 1991 1992 - 2001 2002 - 2011
Decades
FIGURE 2. Number of RCTs in pediatric physical therapy indexed in the PEDro database
published each decade.
6
PEDro score
4
Year
FIGURE 3. Mean PEDro score per year for RCTs in pediatric physical therapy indexed in
the PEDro database.
was observed (4.2±1.3 compared with 5.0 1.5, p < .001). However, multiple re-
gressions did not show an independent effect of the introduction of the
CONSORT checklist on the total PEDro scores.
DISCUSSION
With the standard PEDro scale cut-off of 6-points (Maher et al., 2003), less than
30% of RCTs related to PPT and indexed in the PEDro database were classified
as high-quality studies. Since in most physical therapy clinical trials the blinding
of subjects and of therapists can hardly be achieved, Maher (2000) proposed to re-
duce the cut-off from the original 6 points to a less rigid 5 or even 4 points.
However, since the PEDro scale has been specifically designed and developed for
evaluation
FIGURE 4. Mean PEDro score per decade for RCTs in pediatric physical therapy indexed
in the PEDro database.
100% 94% 94%
87%
80% 75%
65%
60%
60%
40% 31%
20%
20% 15%
6%
1%
0%
Eligibility criteria
Adequate follow-up
Baseline comparability
Blind subjects
Blind therapists
Blind assessors
Between-group comparisons
Intention-to-treat analysis
Concealed allocation
CONCLUSIONS
The number of RCTs in PPT published each year and the methodological qual-
ity of RCTs has increased between 1962 and 2012. This finding has
implications for evidence-based practice. Despite improvements in
methodological quality of RCTs, only 29% of the studies met the criterion
≥ for
high quality (score 6);
≥ while 56% were considered high quality when the modified cut-off score was
used (score 5). Blinding assessors, concealed allocation, and intention-to-treat
analysis are three criteria that researchers should consider when designing
RCTs for PPT. Use of the CONSORT statement in designing and reporting
results is highly advisable.