ORIGINAL ARTICLE
Development of the Radiographic Union Score for Tibial Fractures
for the Assessment of Tibial Fracture Healing After Intramedullary
Fixation
Daniel B. Whelan, MD, FRCSC, Mohit Bhandari, MD, MSc, FRCSC, David Stephen, MD, FRCSC,
Hans Kreder, MD, FRCSC, Michael D. McKee, MD, FRCSC, Rad Zdero, PhD,
and Emil H. Schemitsch, MD, FRCSC
Downloaded from https://journals.lww.com/jtrauma by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3yRlXg5VZA8uSeVZO4Jyms/pLY0hDfylt4e15tYlc3xqBaS5BB7JNng== on 08/21/2018
Background: The objective was to evaluate the newly developed Radio-
graphic Union Score for Tibial fractures (RUST). Because there is no “gold
standard,” it was hypothesized that the RUST score would provide substan-
tial improvements compared with previous scores presented in the literature.
Methods: Forty-five sets of X-rays of tibial shaft fractures treated with
intramedullary fixation were selected. Seven orthopedic reviewers indepen-
dently scored bony union using RUST. Radiographs were reassessed at 9
weeks. Intraclass correlation coefficients (ICC) with 95% confidence inter-
vals (CI) measured agreement.
Results: Overall agreement was substantial (ICC, 0.86; 95% CI, 0.79 – 0.91).
There was improved reliability among traumatologists compared with others
(ICC ⫽ 0.86, 0.81, and 0.83, respectively). Overall intraobserver reliability
was also substantial (ICC, 0.88; 95% CI, 0.80 – 0.96).
Conclusions: The RUST score exhibits substantial improvements in reli-
ability from previously published scores and produces equally reproducible
results among a variety of orthopedic specialties and experience levels.
Because no “gold standards” currently exist against which RUST can be
compared, this study provides only the initial step in the score’s full
validation for use in a clinical context.
Key Words: Score, Intraobserver reliability, Interobserver reliability, Tibial
fractures, Radiographic union, Intramedullary fixation.
(J Trauma. 2010;68: 629 – 632)
D espite improvements in treatment strategies and empha-
sis on standardization of outcome measures in trauma,
such as using radiographic methods,1 delayed union and
nonunion of tibial fractures continue to be a significant
problem and a poorly defined entity. The incidence of non-
union after treatment of the fractured tibia with intramedul-
Submitted for publication October 30, 2008.
Accepted for publication February 6, 2009.
Copyright © 2010 by Lippincott Williams & Wilkins
From the Division of Orthopaedic Surgery (D.B.W., M.B., D.S., H.K., M.D.M.,
E.H.S.), Department of Surgery, University of Toronto, Toronto, Ontario,
Canada; and Martin Orthopaedic Biomechanics Laboratory (R.Z., E.H.S.), St.
Michael’s Hospital, Toronto, Department of Clinical Epidemiology and Bio-
statistics, McMaster Health Sciences Centre, Hamilton, Ontario, Canada.
Presented, in part, at the Orthopaedic Trauma Association, Toronto, Ontario,
Canada, October 11–13, 2002, and Orthopaedic Trauma Association, San
Antonio, Texas, October 12–14, 2000.
Address for reprints: Rad Zdero, PhD, Martin Orthopaedic Biomechanics Labo-
ratory, Shuter Wing (Room 5-066), St. Michael’s Hospital, 30 Bond Street,
Toronto, Ontario, Canada M5B-1W8; email: zderor@smh.toronto.on.ca.
DOI: 10.1097/TA.0b013e3181a7c16d
The Journal of TRAUMA® Injury, Infection, and Critical Care • Volume 68, Number 3, March 2010
lary nail fixation has been reported to range from 5% to 33%.2
Determination of union in tibial shaft fractures is routinely
based upon serial clinical and radiographic assessments. Al-
though there exist several accepted methods to evaluate tibial
fracture healing clinically, namely, pain on weight bearing or
palpation at the fracture site, there are no universally recog-
nized guidelines to evaluate radiographic union.
Clinical investigations have reported nonspecific crite-
ria, such as callus formation or absence of fracture gap, to
evaluate the progression of fracture healing using X-rays.3–9
Moreover, several authors have defined radiographic union as
the presence of bridging callus on at least two views.2,10,11
Elaborate methods, such as that of Hammer et al.,1 grade the
fracture healing based on presence of callus, bridging of
callus, and presence/absence of lucent fracture lines. Tower et
al.12 devised a scoring system based on periosteal callus,
bridging callus, and lucent lines. In a similar fashion, Lane et
al.13 assigned radiographic scores based on the presence of
bone formation, fracture lines, and remodeling.
In previous work, we have demonstrated that currently
available radiographic union scores, such as the one proposed
in 1985 by Hammer et al.,1 have considerable limitations and
that surgeons’ general impressions regarding fracture healing
are often discordant.14 Substantial agreement was seen with
quantifying the number of cortices bridged by callus and/or
with a definite fracture line. We would suggest that quanti-
fying the number of cortices bridged is appropriate in the
clinical setting for a number of reasons. First, the presence of
an intramedullary nail, as is commonly used in modern
practice, allows only the cortical bony detail to be assessed
accurately on X-ray. Further support comes from the work of
Panjabi et al.9 in 1985 who suggested that cortical continuity
on X-ray is the best single predictor of biomechanical
strength in an in vivo model. With this in mind, we developed
a radiographic scaling system that has the determination of
union from the appearance of the cortex as its basis.
Because there are no “gold standards” currently, the
purpose of this investigation was to present the initial devel-
opment stages of a novel system, namely, the Radiographic
Union Score for Tibial fractures (RUST) score, to assess
radiographic fracture healing in the presence of an intramed-
ullary nail and to evaluate its inter- and intraobserver reli-
ability. The investigators hypothesized that the RUST score
629
Whelan et al. The Journal of TRAUMA® Injury, Infection, and Critical Care • Volume 68, Number 3, March 2010
TABLE 1. Assessment Tool for the Radiographic Union Score for Tibial Fractures (RUST)
Radiographic Union Score for Tibial fractures (RUST)
Fracture Line, No Callus Fracture Line, Visible Callus
Cortex (Score ⴝ 1) (Score ⴝ 2)
Anterior
Posterior
Lateral
Medial
Figure 1. Anteroposterior and lateral radiographs of a tibial
shaft fracture after intramedullary nailing. Application of the
scoring system yielded medial cortex (RUST ⫽ 2), lateral cor-
tex (RUST ⫽ 3), anterior cortex (RUST ⫽ 2), and posterior
cortex (RUST ⫽ 1). Total RUST score possible is 12.
would provide substantial improvements compared with pre-
vious scores presented in the literature.
MATERIALS AND METHODS
Forty-five sets of anteroposterior and lateral radio-
graphs of tibial shaft fractures treated with intramedullary
fixation were selected from our institution’s trauma database
to represent fractures at various stages of healing. On the basis
of convenience sampling, a group of seven reviewers, including
two orthopedic residents, two community orthopedic surgeons,
and three orthopedic traumatologists, independently evaluated
the stage of fracture healing using the RUST.
The RUST system shown in Table 1 assigns a score to
a given set of anteroposterior and lateral radiographs based on
the assessment of healing at each of the four cortices visible
on these projections (i.e., medial and lateral cortices on the
anteroposterior X-ray, anterior and posterior cortices on the
lateral X-ray). Each cortex receives a score of 1 point, if it is
deemed to have a fracture line with no callus; 2 points, if
there is callus present but a fracture line is still visible; and 3
630
No Fracture Line, Bridging Callus Total score: Minimum ⴝ 4,
(Score ⴝ 3) Maximum ⴝ 12
points, if there is bridging callus with no evidence of a
fracture line. The individual cortical scores are added to give
a total for the set of films with 4 being the minimum score
indicating that the fracture is definitely not healed and 12
being the maximum score indicating that the fracture is
definitely healed (Fig. 1).
The examiners were blinded to patient history, age of
the fracture, and all clinical information for all sets of radio-
graphs. Interobserver reliability was evaluated by comparing
the scores of seven observers at the initial viewing of the
radiographs. To calculate intraobserver consistency, each
reviewer was asked to score the radiographs again at an
average of 9 weeks (range, 7–11 weeks) after the initial
assessment. The same orthopedic resident with 4 years expe-
rience selectively chose X-rays that were 6 weeks to 9 months
after follow-up. To avoid bias, X-rays were picked solely on
the basis of the above time interval and were only visually
screened to eliminate those which showed staples or had
other signs of acuity.
Intraclass correlation coefficients (ICC) with 95% con-
fidence intervals (CI) were used to measure agreement in the
observer’s RUST scores. The ICC, used to quantify agree-
ment for a continuous variable, is equivalent to the quadrati-
cally weighted kappa (␬) for categorical data. The weighted
kappa, as described by Fleiss,15 adjusts the observed propor-
tion of agreement by correction for the proportion of agree-
ment that could have occurred by chance alone. As they are
numerically equivalent, the same guidelines for interpretation
of kappa values can be applied to the ICC. Landis and Koch16
suggest kappa of 0 to 0.2 represents “slight agreement,” 0.21
to 0.40 “fair agreement,” 0.41 to 0.60 “moderate agreement,”
and 0.61 to 0.80 “substantial agreement.” A value above 0.80
is considered almost “perfect agreement.” The value of the
ICC ranges from ⫹1, in which case there is “perfect agree-
ment,” to ⫺1, which corresponds to “absolute disagreement.”
RESULTS
Among the 45 radiographs, the RUST score ranged
from 4 to 12 with a mean score of 8.3 ⫾ 2.2 (median score ⫽ 8)
(Fig. 2). The results for the assessment of fracture healing
using the RUST score are presented in Table 2. Overall
interobserver agreement among all reviewers collectively was
substantial, with ICC ⫽ 0.86 (95% CI, 0.79 – 0.91). Addition-
ally, although there was a limited sample size, there was a
trend toward improved reliability for traumatologists when
compared with either community surgeons or residents
(ICC ⫽ 0.86, 0.83, and 0.81, respectively).
© 2010 Lippincott Williams & Wilkins
The Journal of TRAUMA® Injury, Infection, and Critical Care • Volume 68, Number 3, March 2010
Figure 2. Distribution of the RUST scores among 45 radiographs.
TABLE 2. Inter- and Intraobserver ICC Values for the RUST
Score
Interobserver ICC Intraobserver ICC
Reviewers (95% CI) (95% CI)
Traumatologists 0.86 (0.76–0.92) 0.89 (0.80–0.95)
Community surgeons 0.83 (0.72–0.91) 0.85 (0.72–0.92)
Residents 0.81 (0.66–0.89) 0.83 (0.68–0.90)
Overall 0.86 (0.79–0.91) 0.88 (0.80–0.96)
To determine intraobserver agreement in RUST scores,
the reviewers evaluated the same radiographs a second time
at an average of 9 weeks (range, 7–11 weeks) from the
original session. The overall intraobserver ICC for the group
of reviewers was again substantial (ICC, 0.88; 95% CI,
0.80 – 0.96). The traumatologists achieved near perfect repro-
ducibility in their RUST scores between the two assessments
(ICC ⫽ 0.89) followed by community surgeons (ICC ⫽ 0.85)
and residents (ICC ⫽ 0.83).
DISCUSSION
The importance of a reliable, universal method for
grading the stages of bony healing on plain X-rays after
fracture or osteotomy cannot be overemphasized. Radio-
graphic union is used frequently as a study endpoint and can
be an invaluable index when findings on clinical examination
are confusing or unreliable. Numerous authors have high-
lighted the difficulty in evaluating tibial fracture healing from
plain radiographs.17–21 Techniques other than plain X-rays,
such as radionucleide imaging,18,22 resonant frequency anal-
ysis,12 computed tomography,23–26 and ultrasound,11 show
potential in their ability to quantify healing in the experimen-
tal setting but are significantly more costly and cumbersome
in their application to clinical practice.
Recently, we evaluated the inter- and intraobserver
reliability of several specific radiographic parameters cur-
rently used to determine tibial fracture healing.14 These were
© 2010 Lippincott Williams & Wilkins
RUST Score for Tibial Fracture
also compared with the agreement clinicians achieved using
the radiographic union score described by Hammer et al.,1
and their reliability using only their “general impression” of
healing. Results from this analysis suggested that cortical
continuity, as determined by quantifying the number of cor-
tices with bridging callus or with a visible fracture line, was
more reliably assessed than any of the other methods evalu-
ated (␬ ⫽ 0.70). Furthermore, the surgeon’s general impres-
sion, based only on the individual’s clinical experience,
yielded at least equally reliable results overall than the seem-
ingly more specific radiographic union score (␬ ⫽ 0.65 vs.
␬ ⫽ 0.60). These findings led us to develop the RUST score.
Although a formal statistical comparison is not possible, the
ICC values obtained in the current investigation are almost
similar with that of quadratically weighted kappas from the
previous study. Using the RUST score, we were able to
further improve upon the level of agreement achieved by
multiple observers in the assessment of fracture healing.
This study takes the next step beyond our original
work14 because we now provide a formal “score” that is
related to the number of cortices that are bridged. Although
there is no radiographic “gold standard” against which to
assess RUST, a feasible further step should be to compare
this measure with a clinical assessment of fracture union
before being recommended for clinical usage. Moreover,
although the RUST score would not necessarily be amenable
for measures of internal consistency or content validity,
aspects of construct validity could be explored. More specif-
ically, RUST should be evaluated to determine whether it can
discriminate between degrees of union and clinically proven
nonunion and predict fixation failure.
Tibial fracture healing of all types may be assessed
potentially using the RUST score, because the signs of union
and nonunion that present themselves upon clinical examina-
tion and in radiographs are not limited only to tibial fractures
repaired by nailing. Regardless of tibial fracture type, clinical
examination using palpation can detect callus formation (in-
dicating union) or no callus formation (indicating nonunion).
Weight bearing of the patient may yield pain (indicating
nonunion) or no pain (indicating union). These would show
themselves radiographically as union and nonunion at the
fracture site. Therefore, an optimal use of the RUST score
would be to use it in conjunction with follow-up radiographs
at preassigned intervals.
Several limitations to this study should be noted. First,
it is important to note, as Thomsen et al.27 point out in a
report on observer variation in the radiographic classification
of ankle fractures, that the determination of inter- and in-
traobserver reliability analyzes only the precision of a score
or classification system and not the accuracy. Thus, we have
studied the observers’ agreement with each other but not with
a “gold standard,” which does not currently exist. However,
the results here do support the use of the RUST to standardize
outcomes when comparing different investigations of tibial
fractures managed with intramedullary fixation. Second, there
were a limited number of observers used in the development
of the RUST score, which would need to be verified further
with a larger sample size in the future. Third, it is unknown
631
Whelan et al. The Journal of TRAUMA® Injury, Infection, and Critical Care • Volume 68, Number 3, March 2010
whether the RUST system can discriminate a healed fracture
from a nonunion, because no comparison was made to other
patient ratings, bending strength, or pain scores. Despite these
limitations and because there is currently no “gold standard,”
the RUST score at best may function as a supplemental tool
that clinicians can use to assess tibial fracture healing. It must
be emphasized that the value of this study is in its presenta-
tion of the initial steps of a full validation of the RUST
scoring system.
In conclusion, this work has presented the next step in
the development of the RUST score to grade fracture healing
in the setting of tibial fractures treated with intramedullary
fixation. The determination of a high degree of reliability and
reproducibility for any scaling system, whether it is a fracture
classification or radiographic grading scheme as proposed in
this study, suggests that it is a functional tool and is an
essential step toward its integration in clinical practice. Be-
cause there is currently no “gold standard” to grade fracture
union, further research will be directed at correlating prospec-
tively gathered RUST scores with data from serial physical
examinations and perhaps other imaging modalities in an
attempt to validate it fully for clinical use.
REFERENCES
1. Hammer RR, Hammerby S, Lindholm B. Accuracy of radiologic assess-
ment of tibial shaft fracture union in humans. Clin Orthop Relat Res.
1985;199:233–238.
2. Keating JF, O’Brien PJ, Blachut PA, Meek RN, Broekhuyse HM.
Locking intramedullary nailing with and without reaming for open
fractures of the tibial shaft. A prospective, randomized study. J Bone
Joint Surg Am. 1997;79: 334 –341.
3. Dickson K, Katzman S, Delgado E, Contreras D. Delayed unions and
nonunions of open tibial fractures: correlation with arteriography results.
Clin Orthop Relat Res. 1994;302:189 –193.
4. Boynton MD, Schmeling GJ. Nonreamed intramedullary nailing of open
tibial fractures. J Am Acad Orthop Surg. 1994;2:107–114.
5. Johner R, Wruhs O. Classification of tibial shaft fractures and correlation
with results after rigid internal fixation. Clin Orthop Relat Res. 1983;
178:7–25.
6. Sarmiento A, Sobol PA, Sew Hoy AL, Ross SD, Racette WL, Tarr RR.
Prefabricated functional braces for the treatment of fractures of the tibial
diaphysis. J Bone Joint Surg Am. 1984;66:1328 –1339.
7. Bone LB, Kassman S, Stegemann P, France J. Prospective study of
union rate of open tibial fractures treated with locked, unreamed in-
tramedullary nails. J Orthop Trauma. 1994;8:45– 49.
8. Nicholls PJ, Berg E, Bliven FE Jr, Kling JM. X-ray diagnosis of healing
fractures in rabbits. Clin Orthop Relat Res. 1979;142:234 –236.
632
9. Panjabi MM, Walter SD, Karuda M, White AA, Lawson JP. Correlations
of radiographic analysis of healing fractures with strength: a statistical
analysis of experimental osteotomies. J Orthop Res. 1985;3:212–218.
10. Duwelius PJ, Schmidt AH, Rubinstein RA, Green JM. Nonreamed
interlocked intramedullary tibial nailing. One community’s experience.
Clin Orthop Relat Res. 1995;315:104 –113.
11. Moed BR, Watson JT, Goldschmidt P, van Holsbeeck M. Ultrasound for
the early diagnosis of fracture healing after interlocking nailing of the
tibia without reaming. Clin Orthop Relat Res. 1995;310:137–144.
12. Tower SS, Beals RK, Duwelius PJ. Resonant frequency analysis of
the tibia as a measure of fracture healing. J Orthop Trauma. 1993;
7:552–557.
13. Lane JM, Sandhu HS. Current approaches to experimental bone grafting.
Orthop Clin North Am. 1987;18:213–225.
14. Whelan DB, Bhandari M, McKee MD, et al. Interobserver and Intraob-
server variation in the assessment of the healing of tibial fractures after
intramedullary fixation. J Bone Joint Surg Br. 2002;84:15–18.
15. Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. New
York: John Wiley & Sons; 1981.
16. Landis JR, Koch GG. The measurement of observer agreement for
categorical data. Biometrics. 1977;33:159 –174.
17. Ebraheim NA, Savolaine ER, Patel A, Skie M, Jackson WT. Assessment
of tibial fracture union by 35– 45 degrees internal oblique radiographs.
J Orthop Trauma. 1991;5:349 –350.
18. Esterhai J, Alavi A, Mandell GA, Brown J. Sequential technetium-99m/
gallium-67 scintigraphic evaluation of subclinical osteomyelitis compli-
cating fracture nonunion. J Orthop Res. 1985;3:219 –225.
19. McClelland D, Thomas PB, Bancroft G, Moorcraft CI. Fracture healing
assessment comparing stiffness measurements using radiographs. Clin
Orthop Relat Res. 2007;457:214 –219.
20. Davis BJ, Roberts PJ, Moorcroft CI, Brown MF, Thomas PB, Wade RH.
Reliability of radiographs in defining union of internally fixed fractures.
Injury. 2004;35:557–561.
21. Bhandari M, Guyatt GH, Swiontkowski MF, Tornetta P III, Sprague S,
Schemitsch EH. A lack of consensus in the assessment of fracture
healing among orthopaedic surgeons. J Orthop Trauma. 2002;16:
562–566.
22. Smith MA, Jones EA, Strachan RK, et al. Prediction of fracture healing
in the tibia by quantitative radionuclide imaging. J Bone Joint Surg Br.
1987;69:441– 447.
23. den Boer FC, Bramer JA, Patka P, et al. Quantification of fracture
healing with three-dimensional computed tomography. Arch Orthop
Trauma Surg. 1998;117:345–350.
24. Braunstein EM, Goldstein SA, Ku J, Smith P, Matthews LS. Computed
tomography and plain radiography in experimental fracture healing.
Skeletal Radiol. 1986;15:27–31.
25. Gershuni DH, Skyhar MJ, Thompson B, Resnick D, Donald G, Akeson
WH. A comparison of conventional radiography and computed tomog-
raphy in the evaluation of spiral fractures of the tibia. J Bone Joint Surg
Am. 1985; 67:1388 –1395.
26. Bhattacharyya T, Bouchard KA, Phadke A, Meigs JB, Kassarjian A,
Salamipour H. The accuracy of computed tomography for the diagnosis
of tibial nonunion. J Bone Joint Surg Am. 2006;88:692– 697.
27. Thomsen NO, Overgaard S, Olsen LH, Hansen H, Nielsen ST. Observer
variation in the radiographic classification of ankle fractures. J Bone
Joint Surg Br. 1991;73:676 – 678.
© 2010 Lippincott Williams & Wilkins