basic research www.kidney-international.

org

A multicenter study to develop a non-invasive

radiomic model to identify urinary infection stone
in vivo using machine-learning
Junjiong Zheng1,2,10, Hao Yu1,2,10, Jesur Batur3,10, Zhenfeng Shi4, Aierken Tuerxun3,
Abudukeyoumu Abulajiang5, Sihong Lu1,2, Jianqiu Kong1,2, Lifang Huang1,2, Shaoxu Wu1,2, Zhuo Wu6,
Ya Qiu7, Tianxin Lin1,2,8 and Xiaoguang Zou9
1
Department of Urology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, People’s Republic of China; 2Guangdong Provincial
Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou,
People’s Republic of China; 3Department of Urology, the First People’s Hospital of Kashi Prefecture, Affiliated Kashi Hospital of Sun Yat-Sen
University, Kashi, People’s Republic of China; 4Department of Urology, the People’s Hospital of Xinjiang Uyghur Autonomous Region, Xinjiang,
People’s Republic of China; 5Department of Information Technology, the First People’s Hospital of Kashi Prefecture, Affiliated Kashi Hospital of
Sun Yat-Sen University, Kashi, People’s Republic of China; 6Department of Radiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University,
Guangzhou, People’s Republic of China; 7Department of Radiology, the First People’s Hospital of Kashi Prefecture, Affiliated Kashi Hospital of
Sun Yat-Sen University, Kashi, People’s Republic of China; 8State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical
Research Center for Urological Diseases, Guangdong, People’s Republic of China; and 9Department of Pharmacy, the First People’s Hospital of
Kashi Prefecture, Affiliated Kashi Hospital of Sun Yat-Sen University, Kashi, People’s Republic of China

Urolithiasis is a common urological disease, and treatment
strategy options vary between different stone types.
However, accurate detection of stone composition can only
be performed in vitro. The management of infection stones is
particularly challenging with yet no effective approach to pre-
operatively identify infection stones from non-infection
stones. Therefore, we aimed to develop a radiomic model for
preoperatively identifying infection stones with multicenter
validation. In total, 1198 eligible patients with urolithiasis
from three centers were divided into a training set, an internal
validation set, and two external validation sets. Stone
composition was determined by Fourier transform infrared
spectroscopy. A total of 1316 radiomic features were
extracted from the pre-treatment Computer Tomography
images of each patient. Using the least absolute shrinkage
and selection operator algorithm, we identified a radiomic
signature that achieved favorable discrimination in the
training set, which was confirmed in the validation sets.
Moreover, we then developed a radiomic model
incorporating the radiomic signature, urease-producing
bacteria in urine, and urine pH based on multivariate logistic
regression analysis. The nomogram showed favorable
calibration and discrimination in the training and three
validation sets (area under the curve [95% confidence
interval], 0.898 [0.840-0.956], 0.832 [0.742-0.923], 0.825
Correspondence: Xiaoguang Zou, Department of Pharmacy, the First Peo-
ple’s Hospital of Kashi Prefecture, No. 66 Yingbin Avenue, Kashi, People’s
Republic of China. E-mail: zxgks@163.com; or Tianxin Lin, Department of
Urology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, 107 Yan
Jiang West Road, Guangzhou, People’s Republic of China. E-mail:
lintx@mail.sysu.edu.cn
10
Co-first authors.
Received 12 January 2021; revised 15 April 2021; accepted 14 May
2021; published online 12 June 2021
[0.783-0.866], and 0.812 [0.710-0.914], respectively). Decision
curve analysis demonstrated the clinical utility of the radiomic
model. Thus, our proposed radiomic model can serve as a
non-invasive tool to identify urinary infection stones in vivo,
which may optimize disease management in urolithiasis and
improve patient prognosis.
Kidney International (2021) 100, 870–880; https://doi.org/10.1016/
j.kint.2021.05.031
KEYWORDS: infection stone; machine learning; prediction model; radiomics;
urolithiasis
Copyright ª 2021, International Society of Nephrology. Published by
Elsevier Inc. All rights reserved.
Translational Statement
Urolithiasis is a common urological disease, and treat-
ment strategy options vary between different stone
types. However, accurate detection of stone composition
can only be performed in vitro. In this study, we
demonstrate that radiomics features extracted from
computed tomography images can be used to preop-
eratively identify infection stones from noninfection
stones in vivo. Our study provides a noninvasive tool to
detect stone composition in vivo, which may aid in
clinical decision-making. This will facilitate the applica-
tion of the radiomics techniques in urolithiasis research
and provide novel insights into solving other clinical is-
sues, such as the prediction of stone fragility before
treatment.
rolithiasis, a condition of urinary stone formation in
U the bladder or urinary tract, is a common urological
disease, which remains a major health problem
worldwide with increasing incidence and prevalence.1

870 Kidney International (2021) 100, 870–880

J Zheng et al.: A radiomics model for infection stone
Urolithiasis can afflict up to 10% to 15% of the population
over a lifetime,2 and the 5-year recurrence rate is estimated up
to 50%.3 The high prevalence and recurrence of urolithiasis
and its predominance in working age adults contribute to the
substantial impact on the individual and society.4,5
Urinary stones can be classified into those caused by in-
fectious or noninfectious causes (infection stones and non-
infection stones), genetic defects, or adverse drug effects
(drug stones).6 Infection stones are complex aggregates of
crystals amalgamated in an organic matrix that seem to be
strictly associated with urinary tract infections caused by
urease-producing gram-negative organisms.7,8 Infection
stones mainly comprise magnesium ammonium phosphate
(struvite) and/or carbonate apatite. They make up ~10% to
15% of urinary stone diseases.9,10 Because of the complex
structure of infection stones and the high rates of recurrence,
infection stone formers are one of the most challenging
populations among patients with urolithiasis.7
A therapeutic regimen of urolithiasis depends on the stone
size, number, location, and composition. Among them, stone
composition is the basis for further diagnostic and manage-
ment decisions.6,11 Knowledge of stone composition may aid
in directing the appropriate choice of urological procedures
and medical and lifestyle interventions to prevent stone
recurrence.6,11,12 Hence, current guidelines recommend that
stone composition analysis should be performed in all first-
time formers.6,13
Noncontrast computed tomography (CT) is the gold
standard imaging test to evaluate patients with urolithiasis,
which can provide information on the stone number, size,
and location. However, so far, the accurate detection of stone
composition can only be performed in vitro after surgery via
infrared spectroscopy or X-ray diffraction.7 As stone
composition is usually unknown before surgery, various
methods and quantitative metrics have been investigated for
preoperative prediction of the stone composition. Previous
studies have demonstrated that the CT stone attenuation
values in Hounsfield units could be used to predict stone
composition.14–16 As the nonlinear relationship of the atten-
uation value to the density of the material being imaged, the
prediction accuracy of these methods is less than satisfac-
tory.17 Dual-energy CT has also been investigated for stone
composition characterization. Dual-energy CT has proven to
distinguish uric acid and non–uric acid with high prediction
efficiency, while it failed to further separate non–uric acid
stones effectively, such as calcium stone, cystine stones, or
infection stones.18–21 Thus far, there is no effective approach
to preoperatively identify infection stones from noninfection
stones yet.
Radiomics is an approach that extracts high-throughput
quantitative image features from radiographic medical im-
ages using data characterization algorithms, which has the
potential to uncover disease (lesion) characteristics that fail to
be appreciated by the naked eye.22,23 Thus, radiomics method
is reported to improve disease diagnosis, prognostic evalua-
tion, and treatment response prediction, representing a
Kidney International (2021) 100, 870–880
basic research
promising approach to facilitating personalized and precise
therapy.22,24 Because the attenuation value of X-rays is
different for different substances, radiomics analysis can
provide more detailed information on urinary stones.
Whether the radiomics approach can be used to predict stone
composition preoperatively is an important issue that war-
rants investigation. Previously, a CT-based radiomics signa-
ture has been reported to differentiate infection stones from
noninfection stones.25 However, further studies are warranted
because of limitations, such as developing the model in a
relatively small data set (n ¼ 157) without external validation
and using a limited number of radiomics features.
Therefore, in this study, we sought to develop a radiomics
model that incorporated a radiomics signature on the basis of
noncontrast CT images and independent clinical predictors
for preoperatively identifying infection stones with multi-
center validation.
METHODS
Patients
The present retrospective analysis of anonymous data was approved
by the institutional review board at each participating institution,
and the requirement for informed consent was waived. In total, 1198
eligible patients were enrolled from the First People’s Hospital of
Kashi Prefecture (n ¼ 448), the Sun Yat-Sen Memorial Hospital (n ¼
594), and the People’s Hospital of Xinjiang Uyghur Autonomous
Region (n ¼ 156). Among them, 314 patients treated in the First
People’s Hospital of Kashi Prefecture between August 2014 and
December 2018 were assigned to the training set whereas 134 pa-
tients treated between January 2019 and December 2019 were allo-
cated to the internal validation set. Patients from the Sun Yat-Sen
Memorial Hospital and the People’s Hospital of Xinjiang Uyghur
Autonomous Region were used as external validation set I and
external validation set II, respectively. The patient recruitment
pathway along with the inclusion and exclusion criteria is presented
in Supplementary Figure S1.
Baseline clinical-pathological data, including age, sex, and uri-
nalysis results, were obtained from the medical records. Pretreat-
ment CT images were reviewed by 2 experienced radiologists, and
data derived from CT images were recorded, including the location
and number of stones. Any disagreement was resolved by
consultation.
Stone composition was analyzed by Fourier transform infrared
spectroscopy, and the predominant stone components were recor-
ded. The composition was considered as predominant one if it
exceeded 50% of the total composition of the stones, but stones
containing any struvite, brushite, or cystine were categorized as the
corresponding name groups.26 Those composed of magnesium
ammonium phosphate and/or carbonate apatite are categorized as
infection stones.9,10 The study flowchart is presented in Figure 1a.
Radiomics procedure
The radiomics workflow is shown in Figure 1b. All patients under-
went CT examination before surgery. The CT image acquisition
settings in each data set are presented in Supplementary Table S1. In
this study, CT Digital Imaging and Communications in Medicine
images were retrieved for radiomics analysis. Image segmentation
was performed using 3D Slicer software (Harvard Medical School,
version 4.10.0).27 The volume of interest of an entire urinary stone
871
basic research
Figure 1 | The study flowchart and the workflow of radiomics. (a) The flowchart depicts the design of our study. (b) The radiomics
workflow presents the procedure of radiomics analysis, consisting of 3 steps: image acquisition, volume of interest (VOI) segmentation, and
radiomics feature extraction. LASSO, least absolute shrinkage and selection operator.
having corresponding composition information determined in vitro
was semi-automatically segmented using the threshold segmentation
method in a blinded fashion. Then, a total of 1316 radiomics features
were extracted from each volume of interest using the PyRadiomics
platform (version 3.0) implanted in Python software (The Python
Software Foundation, version 3.7.4).27 And the volume of the vol-
ume of interest was calculated as the stone volume, which was used
as a candidate clinical variable in subsequent modeling. The
extracted radiomics features were normalized in a linear way in the
range of 0 to 1. Details about the radiomics procedure and radiomics
features are provided in Supplementary Method S1 and
Supplementary Table S2.
Radiomics signature building and performance evaluation
The least absolute shrinkage and selection operator (LASSO) algo-
rithm is a powerful machine learning method for regression with
high-dimensional data.28 Therefore, in the training set, we applied
the LASSO logistic regression algorithm to select the most useful
predictive features from the 1316 extracted radiomics features. Then,
a radiomics signature was built, with a radiomics score calculated as
a linear combination of the selected features weighted by their
respective regression coefficients to estimate the risk of infection
stone for each patient:
Radiomics score ¼ a1 F1 þ a2 F2 þ ::: þ ai Fi þ b
where Fi is the selected radiomics feature, ai is the LASSO coefficient
of Fi, and b is the intercept.
872
J Zheng et al.: A radiomics model for infection stone
In addition to the LASSO algorithm, several other feature selection
methods, including elastic net, random forest, and support vector
machine, were performed. And their performance was compared.
We used Mann-Whitney U tests to evaluate whether there were
statistical differences in radiomics scores between patients with
infection stones and those with noninfection stones. In addition, the
discrimination of the radiomics signature was quantified by the area
under the receiver operating characteristic curve (AUC) in the training
set, which was validated in the validation sets. We also performed
stratified analyses within various subgroups of all enrolled patients.
Radiomics model construction and performance evaluation
In the training set, the radiomics score and candidate clinical vari-
ables were subjected to a multivariate logistic regression algorithm.
Backward stepwise selection was applied by using the likelihood ratio
test with the stopping rule of the Akaike information criterion to
identify the independent predictors of infection stone. Meanwhile,
the variance inflation factor was calculated for the collinearity di-
agnostics of multivariate logistic regression. Then, a radiomics model
was constructed on the basis of the results of the multivariate logistic
analysis, and the logistic regression formula was used to calculate a
risk score for each patient to reflect the risk of infection stone.
The performance of the radiomics model was assessed with respect
to its discrimination and calibration in the training set. The AUC was
calculated to assess the discrimination of the model, while calibration
was evaluated using a calibration curve, along with the Hosmer-
Lemeshow test, to evaluate the goodness of fit of the model.29
Kidney International (2021) 100, 870–880
J Zheng et al.: A radiomics model for infection stone
Radiomics model validation
The internal validation set and 2 external validation sets were used to
test the performance of the radiomics model. The risk score was
calculated for each patient of the validation sets on the basis of the
radiomics model constructed in the training set. Then, the discrimi-
nation and calibration of the model were also assessed as stated above.
Clinical usefulness of the radiomics model
Decision curve analysis was performed to determine the clinical
usefulness of the radiomics model by calculating the net benefits at
different threshold probabilities.30 In addition, we applied receiver
operating characteristic analyses in all enrolled patients to compare the
discriminatory efficacy of the radiomics model to the urinary markers.
Statistical analysis
All statistical analyses were performed using R software (R Foun-
dation for Statistical Computing, version 3.5.1). All the R packages
used in our study are detailed in Supplementary Method S2. Detailed
information about the LASSO logistic regression algorithm and de-
cision curve analysis are provided in Supplementary Methods S3 and
S4, respectively. All statistical tests were 2-tailed, and P < 0.05 was
considered significant.
RESULTS
Patient clinical characteristics
The characteristics of the patients in the training and vali-
dation sets are summarized in Table 1. Infection stone was
present in 16.0% of patients (192 of 1198) overall. In the
training set, 13.1% patients suffered from infection stone.
And the prevalence of infection stone was 16.4%, 18.7%, and
11.5% in the internal validation set, external validation set I,
and external validation set II, respectively. The characteristics
of stone composition for all enrolled patients are presented in
Supplementary Table S3.
Radiomics signature building and performance evaluation
Totally, 1316 radiomics features were extracted from the
CT images of each patient. Because the margins of a uri-
nary stone in CT images is quite clear, satisfactory inter-
observer feature extraction reproducibility was achieved as
expected (Supplementary Method S1). Among 4 feature
selection methods, the LASSO algorithm showed the best
performance (Supplementary Table S4). In the training
set, 24 key radiomics features with nonzero coefficients
were selected on the basis of the LASSO logistic regression
algorithm (Figure 2a and b). These radiomics features and
their corresponding coefficients are presented in
Figure 2c.
There was a significant difference in radiomics scores be-
tween patients with infection stones and those with non-
infection stones in the training set (Mann-Whitney U test,
P < 0.001), which was confirmed in the internal validation set
and 2 external validation sets (Mann-Whitney U tests, P <
0.001). Moreover, significant differences were found in
stratified analyses (Supplementary Table S5).
The radiomics signature yielded an AUC of 0.864 (95%
confidence interval [CI], 0.802–0.926) in the training set,
which was validated in the internal validation set with an AUC
Kidney International (2021) 100, 870–880
basic research
of 0.813 (95% CI, 0.718–0.907) (Figure 3a). In addition, the
performance of the radiomics signature was confirmed in
external validation sets I and II, with AUCs of 0.803 (95% CI,
0.762–0.845) and 0.798 (95% CI, 0.683–0.912), respectively
(Figure 3a).
An optimal radiomics score cutoff value was chosen
as 1.878 according to the maximum Youden index in the
training set. The waterfall plots display the distribution of
radiomics scores and urinary stone types in 4 data sets,
with the dividing line drawn at the cutoff value
(Figure 3b).
Radiomics model construction and performance evaluation
Three independent predictors, including the radiomics score,
urease-producing bacteria in urine, and urine pH, were
identified by using the multivariate logistic regression algo-
rithm (Table 2). In the collinearity diagnosis, the variance
inflation factors ranged from 1.124 to 3.226, indicating that
there was no collinearity. The radiomics model that incor-
porated these independent predictors was constructed and the
regression formula for calculating the risk score is as follows:
Risk score ¼ 2:425  radiomics score
þ 1:722  urine pH þ 2:266
 I ðpresence of urease-producing
bacteria in urineÞ  8:002
The indicator function I is equal to 1 if the statement in the
parentheses is true and is equal to 0 otherwise.
The predicted probability of infection stone can be
calculated using 1/[1 þ exp (risk score)]. To provide a user-
friendly tool, we also developed a nomogram on the basis of
the radiomics model (Figure 4a).
In the training set, the radiomics model showed favorable
discrimination with an AUC of 0.898 (95% CI, 0.840–0.956)
(Figure 4b). The calibration curve showed good calibration of
the radiomics model (Figure 4c). In addition, the Hosmer-
Lemeshow test yielded a nonsignificant statistic (P ¼
0.115), indicating a good fit.
Radiomics model validation
Figure 4b shows that the favorable discrimination of the
radiomics model was validated in the internal validation set
(AUC [95% CI], 0.832 [0.742–0.923]). Furthermore, the
performance was confirmed in external validation sets I
and II, with AUCs of 0.825 (95% CI, 0.783–0.866) and
0.812 (95% CI, 0.710–0.914), respectively. Good consis-
tency was also observed between the actual infection stone
rate and the model prediction in the 3 validation data sets,
with nonsignificant P values (0.639, 0.365, and 0.824,
respectively) derived from the Hosmer-Lemeshow test
(Figure 4c).
Clinical usefulness of the radiomics model
The receiver operating characteristic curves in
Supplementary Figure S2 showed that the model had better
873
 874

basic research
Table 1 | Baseline characteristics of the patients
Training set (n [ 314) Internal validation set (n [ 134) External validation set I (n [ 594) External validation set II (n [ 156)
Noninfection Noninfection Infection Noninfection Infection Noninfection Infection
Characteristic stone Infection stone P stone stone P stone stone P stone stone P

Age, yr
Median (interquartile range) 26 (10–38) 26 (15–34) 0.809 30 (10–46) 32 (24–41) 0.822 53 (45–61) 54 (46–63) 0.189 46 (37–55) 54 (47–61) 0.049a
Sex
Male 190 (69.6) 26 (63.4) 0.426 76 (67.9) 10 (45.5) 0.045a 311 (64.4) 45 (40.5) <0.001a 104 (75.4) 10 (55.6) 0.134
Female 83 (30.4) 15 (36.6) 36 (32.1) 12 (54.5) 172 (35.6) 66 (59.5) 34 (24.6) 8 (44.4)
Number of stones
Single 88 (32.2) 7 (17.1) 0.049a 41 (36.6) 3 (13.6) 0.036a 107 (22.2) 18 (16.2) 0.166 67 (48.6) 3 (16.7) 0.011a
Multiple 185 (67.8) 34 (82.9) 71 (63.4) 19 (86.4) 376 (77.8) 93 (83.8) 71 (51.4) 15 (83.3)
Stone volume, mm3
Median (interquartile range) 1286 2492 (918–5672) 0.007a 1615 3233 0.005a 1763 4012 <0.001a 546 1524 <0.001a
(518–3624) (607–3663) (1583–10,076) (859–3746) (1707–11,817) (266–1289) (662–2897)
Stone location
Kidney 221 (80.9) 35 (85.4) 0.057 88 (78.6) 20 (91.0) 0.396 367 (76.0) 105 (94.6) <0.001a 72 (52.2) 14 (77.8) 0.082
Ureter 36 (13.2) 1 (2.4) 15 (13.4) 1 (4.5) 116 (24.0) 6 (5.4) 61 (44.2) 3 (16.7)
Bladder or urethra 16 (5.9) 5 (12.2) 9 (8.0) 1 (4.5) — — 5 (3.6) 1 (5.5)
Urine white blood cell count
Median (interquartile range) 117.6 904.7 <0.001a 96.9 1101.5 <0.001a 64.0 165.0 <0.001a 10.0 41.0 0.190
(38.6–464.7) (179.3–2981.8) (45.1–743.4) (493.5–1781.1) (17.5–210.0) (47.5–448.5) (3.0–137.3) (5.3–415.5)
<100 per ml 130 (47.6) 7 (17.1) <0.001a 58 (51.8) 2 (9.1) <0.001a 295 (61.1) 39 (35.1) <0.001a 99 (71.7) 11 (61.1) 0.352
$100 per ml 143 (52.4) 34 (82.9) 54 (48.2) 20 (90.9) 188 (38.9) 72 (64.9) 39 (28.3) 7 (38.9)
Urine nitrite
Negative 220 (80.6) 26 (63.4) 0.013a 85 (75.9) 11 (50.0) 0.014a 449 (93.0) 96 (86.5) 0.025a 129 (93.5) 16 (88.9) 0.821
Positive 53 (19.4) 15 (36.6) 27 (24.1) 11 (50.0) 34 (7.0) 15 (13.5) 9 (6.5) 2 (11.1)
Urine pH
Median (interquartile range) 5.5 (5.5–6.0) 6.0 (6.0–7.0) <0.001a 5.5 (5.5–6.0) 6.5 (5.5–6.5) 0.001a 6.0 (5.5–6.5) 6.5 <0.001a 6.0 6.25 (5.5–7.0) 0.068
(6.0–7.0) (5.5–6.5)
Urine culture
Negative 220 (80.6) 27 (65.9) 0.032a 85 (75.9) 11 (50.0) 0.014a 432 (89.4) 85 (76.6) <0.001a 102 (73.9) 11 (61.1) 0.388
Positive 53 (19.4) 14 (34.1) 27 (24.1) 11 (50.0) 51 (10.6) 26 (23.4) 36 (26.1) 7 (38.9)
Urease-producing bacteria
<0.001a 0.003a 0.029a

J Zheng et al.: A radiomics model for infection stone

Negative 265 (97.1) 34 (82.9) 104 (92.9) 20 (90.9) 0.900 471 (97.5) 101 (91.0) 131 (94.9) 14 (77.8)
Positive 8 (2.9) 7 (17.1) 8 (7.1) 2 (9.1) 12 (2.5) 10 (9.0) 7 (5.1) 4 (22.2)
Data are presented as n (%) unless indicated otherwise.
P values were derived from the univariate association analyses between each characteristic and stone types. Note that the differential diagnosis of infection stones vs. noninfection stones at baseline was made by an in vitro analysis
of stone composition using Fourier transform infrared spectroscopy.
a
P < 0.05.
Kidney International (2021) 100, 870–880
J Zheng et al.: A radiomics model for infection stone basic research

Figure 2 | Construction of the radiomics signature. (a,b) Radiomics feature selection using least absolute shrinkage and selection operator
(LASSO) logistic regression. (a) Selection of the tuning parameter (l). The LASSO logistic regression model was used with penalty parameter
tuning that was conducted by 10-fold cross-validation according to minimum criteria. The binomial deviance was plotted versus log(l). The
vertical dotted line is plotted at the optimal l value according to minimum criteria. The optimal l value of 0.018 with log(l) ¼ 4.010 was
selected. (b) LASSO coefficient profiles of the 1316 radiomics features. The vertical dotted line was plotted at the log(l) value of 4.010,
resulting in 24 nonzero coefficients. (c) Histogram showing the coefficients of the selected features in the radiomics signature. Each feature is
presented in the format of “filter_feature class_feature name.” For filter “LoG,” the value in parentheses indicates the filter width used for the
Gaussian kernel, and for filter “Wavelet,” the value in parentheses indicates the filters (H, high-pass filter; L, low-pass filter) applied in the x, y,
and z directions, respectively. GLCM, gray level co-occurence matrix; GLDM, gray level dependence matrix; GLSZM, gray level size zone matrix;
IMC, informal measure of correlation; LDHGLE, large dependence high gray level emphasis; LGLZE, low gray level zone emphasis; LoG,
Laplacian of Gaussian; MCC, maximal correlation coefficient; NGTDM, neighboring gray tone difference matrix; SALGLE, small area low gray
level emphasis; SZNUN, size zone nonuniformity normalized.

discriminatory efficiency than either the radiomics signature P < 0.001), urine nitrite (DeLong test, P < 0.001), or
(DeLong test, P ¼ 0.002), urine pH (DeLong test, urease-producing bacteria (DeLong test, P < 0.001). In
P < 0.001), urine white blood cell count (DeLong test, addition, we tested whether the radiomics signature adds to

Kidney International (2021) 100, 870–880 875

basic research J Zheng et al.: A radiomics model for infection stone

Figure 3 | Performance of the radiomics signature. (a) Receiver operating characteristic (ROC) curves of the radiomics signature. (b)
Waterfall plot for the distribution of radiomics scores and urinary stone types of individual patients in all enrolled patients. AUC, area under the
curve; CI, confidence interval.

the predictive value of a clinical model containing only
clinical predictors. Among the clinical candidate predictors,
urine white blood cell count, urine pH, and urease-
producing bacteria were identified as independent pre-
dictors of infection stone on the basis of the multivariate
logistic regression analysis. As a result, the radiomics model
had higher prediction efficacy than did the clinical model
(AUC [95% CI], 0.842 [0.812–0.873] vs. 0.746 [0.709–
0.783], respectively; P < 0.001) (Supplementary Figure S2),
indicating the incremental predictive value of the radiomics
signature. In all 4 data sets, the decision curve analysis
suggested that using the radiomics model to predict

Table 2 | Logistic regression analysis of the radiomics score and clinical candidate predictors in the training set
Univariate logistic regression Multivariate logistic regression
Variable OR (95% CI) P OR (95% CI) P
Radiomics score (per 0.1 increase) 1.272 (1.183–1.386) <0.001 a
1.274 (1.176–1.406) <0.001a
Sex (male vs. female) 1.321 (0.652–2.595) 0.427
Age (per 10-yr increase) 0.968 (0.805–1.154) 0.722
Number of stones (single vs. multiple) 2.310 (1.041–5.866) 0.054
Stone volume (cm3) 1.089 (1.044–1.148) <0.001a
Stone location
Bladder or urethra Reference —
Ureter 0.089 (0.004–0.609) 0.033a
Kidney 0.507 (0.185–1.627) 0.211
Urine white blood cell count (<100 per ml vs. $ 100 per ml) 4.416 (2.001–11.170) <0.001a
Urine nitrite (negative vs. positive) 2.395 (1.165–4.793) 0.015a
Urine pH 5.773 (3.346–10.497) <0.001a 5.597 (2.893–11.450) <0.001a
Urine culture (negative vs. positive) 2.152 (1.034–4.333) 0.035a
Urease-producing bacteria (negative vs. positive) 6.820 (2.263–20.189) <0.001a 9.645 (1.731–46.645) 0.006
CI, confidence interval; OR, odds ratio.
a
P < 0.05.

876 Kidney International (2021) 100, 870–880

J Zheng et al.: A radiomics model for infection stone basic research

Figure 4 | Performance of the radiomics model. (a) The nomogram developed on the basis of the radiomics model. (b) Receiver operating
characteristic (ROC) curves of the radiomics model. (c) Calibration curves of the radiomics model. The x axis and y axis show the predicted
probabilities and actual probabilities of having infection stone, respectively. The calibration curve depicts calibration of the model in terms of
agreement between the predicted probabilities and the observed probabilities. The diagonal gray dotted line represents perfect prediction,
and the solid line represents the performance of the model. The solid line has a closer fit to the dotted line, which represents a better
calibration.

infection stone adds more net benefit than either the “treat
all” or “treat none” strategies scheme for a wide range of
threshold probability, indicating the clinical usefulness of
the radiomics model (Figure 5).
DISCUSSION
In this study, we analyzed the noncontrast CT images using a
radiomics approach and then developed a radiomics
signature for preoperatively identifying infection stones
from noninfection stones. Moreover, by incorporating the
radiomics signature, urease-producing bacteria in urine,
and urine pH, a radiomics model was constructed. The
radiomics model showed favorable discrimination and
calibration with multicenter validation, providing a
noninvasive tool for individualized preoperative assessment
of the probability of having infection stones in patients
with urolithiasis.
Treatment strategy options vary between different stone
types. Indeed, infection stones are a special subset of urinary
stones that form as a result of urinary tract infections, and the
management of infection stones is particularly challenging.10
Infection stones were reported to have a higher risk of
postoperative infectious complications, which may potentially
result in life-threatening conditions, such as severe sepsis and
septic shock.31–33 The treatment of infection stones mainly
relies on the use of antibiotics (before and after stone
removal) to eliminate planktonic bacteria in the urinary tract
and surgical treatment to remove all stone fragments. The

Kidney International (2021) 100, 870–880 877

basic research J Zheng et al.: A radiomics model for infection stone

Figure 5 | Decision curve analyses for the radiomics model. The red line represents the radiomics model. The gray line represents the
hypothesis that all patients had infection stones. The black line represents the hypothesis that no patients had infection stone. (a) Training set.
(b) Internal validation set. (c) External validation set I. (d) External validation set II.

complete removal of the stones is crucial, because residual
stones after surgery are an independent risk factor for
infection stone recurrence.7,34 CT examination, the routinely
used imaging modality for urinary stone formers, can provide
important information about the stones. However, it does not
differentiate stone compositions.6 Whereupon, stone compo-
sition is usually unknown before treatment. If stone compo-
sition can be accurately predicted at the time of diagnosis,
disease management in urolithiasis can be optimized, which
will improve patient outcomes. Thus, in vivo determining
urinary stone composition has become a recent research focus.
Radiomics emerged from the medical field of oncology.
Previous studies have demonstrated that the radiomics fea-
tures can reflect the intrinsic characteristics, such as hetero-
geneity, of tumor lesions via converting medical images into
mineable high-dimensional data, thus being useful for dis-
ease, prognosis, and therapeutic response prediction.22
Moreover, this technique can be applied to any medical
research where a disease or a condition can be imaged.23
Indeed, radiomics studies on nontumor disease have been
reported, such as Alzheimer disease,35 liver fibrosis,36 chronic
pancreatitis,37 and cardiac diseases.38,39 As for urolithiasis,
because the composition varies from different stone types, it
is reasonable to hypothesize that radiomics features can reflect
the heterogeneity of the urinary stone. As expected, our study
confirmed this hypothesis. This will facilitate the application
of the radiomics technique in urolithiasis researches and
provide novel insights into solving other clinical issues, such
as prediction of stone fragility before treatment.
In this study, we demonstrated that the radiomics features
extracted from CT images can be used to accurately predict
the probability of infection stone, which enables the analysis
of the stone composition before removal. A radiomics
signature consisted of 24 radiomics features was developed to
identify infection stones from noninfection stones. In addi-
tion to the radiomics signature, urine pH was identified as an
independent predictor of infection stone with an odds ratio of
5.597 (Table 2). That is to say, the higher the urine pH value,
the higher the probability of infection stone. Indeed, alkaline
pH favors the crystallization of calcium- and phosphate-
containing stones. Previous researches have demonstrated
that carbonate apatite starts to crystallize at a urine pH level
of 6.8 while struvite precipitates at pH >7.2.10,40 Moreover,
the presence of urease-producing bacteria in urine was found

878 Kidney International (2021) 100, 870–880

J Zheng et al.: A radiomics model for infection stone
to be an independent predictor of infection stone. This is
consistent with the viewpoint that the formation of infection
stones is the result of a complex interaction between urease-
producing microorganisms and urine chemistry.7 Previously,
as mentioned in the introduction section, Cui et al. have also
developed a CT-based radiomics signature to differentiate
infection stones from noninfection stones.25 However, they
developed the radiomics signature in a relatively small data set
(n ¼ 157) without external validation. Furthermore, they used
only the most central 5 layers of the stone images for radiomics
analysis and used a limited number of radiomics features. In
the present study, on the contrary, 3-dimensional volume of
interests of the entire lesion were segmented from image slices
for radiomics analysis and high-dimensional radiomics features
were extracted, which more effectively reveals the intrinsic
characteristics of urinary stones. To our knowledge, this is the
largest radiomics study on urolithiasis with multicenter vali-
dation, providing a more reliable tool for preoperatively
discriminating infection stones from noninfection stones. In
addition, radiomics score and urine pH in the presented
radiomics model are available from routine CT examination
and urinalysis. Therefore, the prediction model can be used
easily without any extra burden or cost to the patient.
Our study has some limitations. First, potential selection
biases might occur because of the strict inclusion and
exclusion criteria used. This might affect the generalizability
of the radiomics model to the real-world population with
urolithiasis. Second, radiomics analysis for those small uri-
nary stones is still challenging. This may also limit the scope
of the application of the presented model. Third, although the
proposed model has been validated in multicenter data sets,
prospective validation is still needed to verify the robustness
of our model before implementation in clinical practice.
In conclusion, we developed and validated a radiomics
model to identify urinary infection stones in vivo. This tool
enables the analysis of the stone composition before removal,
which may optimize disease management in urolithiasis and
improve patient prognosis.
DISCLOSURE
All the authors declared no competing interests.
DATA STATEMENT
The data that support the findings of this study are available from the
corresponding author upon reasonable request.
ACKNOWLEDGMENTS
This study was supported by the National Key Research and
Development Program of China (2018YFA0902803), the National
Natural Science Foundation of China (81825016, 81961128027,
81772719, 81772728), the Key Areas Research and Development
Program of Guangdong (2018B010109006), the Science and
Technology Planning Project of Guangdong Province
(2017B020227007), Guangdong Special Support Program
(2017TX04R246), the Guangdong Provincial Clinical Research Center
for Urological Diseases (2020B1111170006), and Project Supported by
Guangdong Province Higher Vocational Colleges & Schools Pearl
River Scholar Funded Scheme (for TL).
Kidney International (2021) 100, 870–880
basic research
AUTHOR CONTRIBUTIONS
JZ, HY, XZ, and TL conceptualized and designed the study. JB, ZS, AT,
AA, JK, LH, and SW acquired the data. ZW and YQ processed the
images. JZ, HY, SL, and JB analyzed and interpreted the data. XZ and
TL supervised the study. All the authors wrote, reviewed, and/or
revised the manuscript.
SUPPLEMENTARY MATERIAL
Supplementary File (PDF)
Method S1. Radiomics Procedure.
Method S2. R packages used in this Study.
Method S3. Detailed description of the LASSO method.
Method S4. Decision curve analysis.
Table S1. The CT acquisition parameters in each cohort.
Table S2. Extracted radiomics features.
Table S3. The distribution of predominant stone composition for all
enrolled patients in the study.
Table S4. Predictive performances of different feature selection
methods.
Table S5. Stratified analysis of the association between the radiomics
signature and infection stone in all enrolled patients.
Figure S1. Inclusion and exclusion criteria and recruitment pathways
for patients in this study.
Figure S2. Model comparisons using ROC curve analyses. ROC curves
comparing the predictive performance of the radiomics model with
each selected predictor in all enrolled patients.
REFERENCES
1. De Coninck V, Antonelli J, Chew B, et al. Medical expulsive therapy for
urinary stones: future trends and knowledge gaps. Eur Urol. 2019;76:658–
666.
2. Spernat D, Kourambas J. Urolithiasis—medical therapies. BJU Int.
2011;108(suppl 2):9–13.
3. Fink HA, Wilt TJ, Eidman KE, et al. Medical management to prevent
recurrent nephrolithiasis in adults: a systematic review for an American
College of Physicians Clinical Guideline. Ann Intern Med. 2013;158:535–
543.
4. Morgan MS, Pearle MS. Medical management of renal stones. BMJ.
2016;352:i52.
5. Paranjpe I, Tsao N, Judy R, et al. Derivation and validation of genome-
wide polygenic score for urinary tract stone diagnosis. Kidney Int.
2020;98:1323–1330.
6. Turk C, Petrik A, Sarica K, et al. EAU Guidelines on Diagnosis and
Conservative Management of Urolithiasis. Eur Urol. 2016;69:468–474.
7. Espinosa-Ortiz EJ, Eisner BH, Lange D, et al. Current insights into the
mechanisms and management of infection stones. Nat Rev Urol. 2019;16:
35–53.
8. Miano R, Germani S, Vespasiani G. Stones and urinary tract infections.
Urol Int. 2007;79(suppl 1):32–36.
9. Khan SR, Pearle MS, Robertson WG, et al. Kidney stones. Nat Rev Dis
Primers. 2016;2:16008.
10. Bichler KH, Eipper E, Naber K, et al. Urinary infection stones. Int J
Antimicrob Agents. 2002;19:488–498.
11. Turk C, Petrik A, Sarica K, et al. EAU Guidelines on Interventional
Treatment for Urolithiasis. Eur Urol. 2016;69:475–482.
12. Miller AW, Choy D, Penniston KL, et al. Inhibition of urinary stone disease
by a multi-species bacterial network ensures healthy oxalate
homeostasis. Kidney Int. 2019;96:180–188.
13. Pearle MS, Goldfarb DS, Assimos DG, et al. Medical management of
kidney stones: AUA guideline. J Urol. 2014;192:316–324.
14. Deveci S, Coskun M, Tekin MI, et al. Spiral computed tomography: role in
determination of chemical compositions of pure and mixed urinary
stones—an in vitro study. Urology. 2004;64:237–240.
15. Ganesan V, De S, Shkumat N, et al. Accurately diagnosing uric acid stones
from conventional computerized tomography imaging: development
and preliminary assessment of a pixel mapping software. J Urol.
2018;199:487–494.
16. Motley G, Dalrymple N, Keesling C, et al. Hounsfield unit density in the
determination of urinary stone composition. Urology. 2001;58:170–173.
879
basic research
17. Zhang GM, Sun H, Xue HD, et al. Prospective prediction of the major
component of urinary stone composition with dual-source dual-energy
CT in vivo. Clin Radiol. 2016;71:1178–1183.
18. Thomas C, Heuschmid M, Schilling D, et al. Urinary calculi composed of
uric acid, cystine, and mineral salts: differentiation with dual-energy CT
at a radiation dose comparable to that of intravenous pyelography.
Radiology. 2010;257:402–409.
19. Stolzmann P, Scheffel H, Rentsch K, et al. Dual-energy computed
tomography for the differentiation of uric acid stones: ex vivo
performance evaluation. Urol Res. 2008;36:133–138.
20. Primak AN, Fletcher JG, Vrtiska TJ, et al. Noninvasive differentiation of
uric acid versus non-uric acid kidney stones using dual-energy CT. Acad
Radiol. 2007;14:1441–1447.
21. Qu M, Ramirez-Giraldo JC, Leng S, et al. Dual-energy dual-source CT with
additional spectral filtration can improve the differentiation of non-uric
acid renal stones: an ex vivo phantom study. AJR Am J Roentgenol.
2011;196:1279–1287.
22. Lambin P, Leijenaar RTH, Deist TM, et al. Radiomics: the bridge between
medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:
749–762.
23. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than
pictures, they are data. Radiology. 2016;278:563–577.
24. Zheng J, Kong J, Wu S, et al. Development of a noninvasive tool to
preoperatively evaluate the muscular invasiveness of bladder cancer
using a radiomics approach. Cancer. 2019;125:4388–4398.
25. Cui X, Che F, Wang N, et al. Preoperative prediction of infection stones
using radiomics features from computed tomography. IEEE Access.
2019;7:122675–122683.
26. Lieske JC, Rule AD, Krambeck AE, et al. Stone composition as a function
of age and sex. Clin J Am Soc Nephrol. 2014;9:2141–2146.
27. van Griethuysen JJM, Fedorov A, Parmar C, et al. Computational
radiomics system to decode the radiographic phenotype. Cancer Res.
2017;77:e104–e107.
28. Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat
Soc Ser B. 1996;58:267–288.
880
J Zheng et al.: A radiomics model for infection stone
29. Kramer AA, Zimmerman JE. Assessing the calibration of mortality
benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit
Care Med. 2007;35:2052–2056.
30. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for
evaluating prediction models. Med Decis Making. 2006;26:565–574.
31. Flannigan R, Choy WH, Chew B, et al. Renal struvite stones—pathogenesis,
microbiology, and management strategies. Nat Rev Urol. 2014;11:333–341.
32. Koras O, Bozkurt IH, Yonguc T, et al. Risk factors for postoperative
infectious complications following percutaneous nephrolithotomy: a
prospective clinical study. Urolithiasis. 2015;43:55–60.
33. Zisman AL, Evan AP, Coe FL, et al. Do kidney stone formers have a kidney
disease? Kidney Int. 2015;88:1240–1249.
34. Iqbal MW, Youssef RF, Neisius A, et al. Contemporary management of struvite
stones using combined endourologic and medical treatment: predictors of
unfavorable clinical outcome. J Endourol. 2016;30:771–777.
35. Feng Q, Ding Z. MRI radiomics classification and prediction in
Alzheimer’s disease and mild cognitive impairment: a review. Curr
Alzheimer Res. 2020;17:297–309.
36. Wang K, Lu X, Zhou H, et al. Deep learning radiomics of shear wave
elastography significantly improved diagnostic performance for
assessing liver fibrosis in chronic hepatitis B: a prospective multicentre
study. Gut. 2019;68:729–741.
37. Frokjaer JB, Lisitskaya MV, Jorgensen AS, et al. Pancreatic magnetic
resonance imaging texture analysis in chronic pancreatitis: a feasibility
and validation study. Abdom Radiol (NY). 2020;45:1497–1506.
38. Oikonomou EK, Williams MC, Kotanidis CP, et al. A novel machine
learning-derived radiotranscriptomic signature of perivascular fat
improves cardiac risk prediction using coronary CT angiography. Eur
Heart J. 2019;40:3529–3543.
39. Kagiyama N, Shrestha S, Cho JS, et al. A low-cost texture-based pipeline
for predicting myocardial tissue remodeling and fibrosis using cardiac
ultrasound. EBioMedicine. 2020;54:102726.
40. Carpentier X, Daudon M, Traxer O, et al. Relationships between
carbonation rate of carbapatite and morphologic characteristics of
calcium phosphate stones and etiology. Urology. 2009;73:968–975.
Kidney International (2021) 100, 870–880