La radiologia medica (2023) 128:509–519

https://doi.org/10.1007/s11547-023-01625-6

ABDOMINAL RADIOLOGY

Multi‑modal analysis for accurate prediction of preoperative stage

and indications of optimal treatment in gastric cancer
Shangqing Liu1 · Weiqi Liang2 · Pinyu Huang1 · Dianjie Chen2 · Qinglie He2 · Zhenyuan Ning1 · Yu Zhang1 ·
Wei Xiong3 · Jiang Yu2 · Tao Chen2,4

Received: 14 November 2022 / Accepted: 27 March 2023 / Published online: 28 April 2023
© Italian Society of Medical Radiology 2023

Abstract
Background Accurate preoperative clinical staging of gastric cancer helps determine therapeutic strategies. However, no
multi-category grading models for gastric cancer have been established. This study aimed to develop multi-modal (CT/EHRs)
artificial intelligence (AI) models for predicting tumor stages and optimal treatment indication based on preoperative CT
images and electronic health records (EHRs) in patients with gastric cancer.
Methods This retrospective study enrolled 602 patients with a pathological diagnosis of gastric cancer from Nanfang
hospital retrospectively and divided them into training (n = 452) and validation sets (n = 150). A total of 1326 features were
extracted of which 1316 radiomic features were extracted from the 3D CT images and 10 clinical parameters were obtained
from electronic health records (EHRs). Four multi-layer perceptrons (MLPs) whose input was the combination of radiomic
features and clinical parameters were automatically learned with the neural architecture search (NAS) strategy.
Results Two two-layer MLPs identified by NAS approach were employed to predict the stage of the tumor showed greater
discrimination with the average ACC value of 0.646 for five T stages, 0.838 for four N stages than traditional methods with
ACC of 0.543 (P value = 0.034) and 0.468 (P value = 0.021), respectively. Furthermore, our models reported high prediction
accuracy for the indication of endoscopic resection and the preoperative neoadjuvant chemotherapy with the AUC value of
0.771 and 0.661, respectively.
Conclusions Our multi-modal (CT/EHRs) artificial intelligence models generated with the NAS approach have high
accuracy for tumor stage prediction and optimal treatment regimen and timing, which could facilitate radiologists and
gastroenterologists to improve diagnosis and treatment efficiency.

Keywords Gastric cancer · Artificial intelligence · Multi-modal · Neural architecture search

Abbreviations
EGC Early gastric cancer
EHRs Electronic health records
Shangqing Liu and Weiqi Liang share co-first authorship.
LAGC​ Locally advanced gastric cancer
* Tao Chen AUC​ Area under the receiver operating characteristic
drchentao@163.com curve
1 CA125 Cancer antigen 125
School of Biomedical Engineering, Southern Medical
University, Guangzhou 510515, China CA19-9 Carbohydrate antigen 19-9
2 CEA Carcinoembryonic antigen
Department of General Surgery and Guangdong Provincial
Key Laboratory of Precision Medicine for Gastrointestinal CNN Convolutional Neural Network
Tumor, Nanfang Hospital, The First School of Clinical DL Deep learning
Medicine, Southern Medical University, Guangzhou 510515, MLP Multi-layer perceptron
Guangdong, China NAS Neural architecture search
3
Department of Radiology, Nanfang Hospital, Southern
Medical University, Guangzhou 510515, China
4
Department of Gastrointestinal and Hernia Surgery, Ganzhou
Hospital‑Nanfang Hospital, Southern Medical University,
Ganzhou 341000, Jiangxi, China

13
Vol.:(0123456789)
510
Background
Gastric cancer (GC) is one of the most common malignant
neoplasms worldwide and causes nearly 800,000
deaths per year [1]. TNM staging system is one of the
most important criteria for clinical decision-making
and prognostic evaluation of GC, which includes the
evaluation of the tumor invasion depth (T stage), the
number of regional lymph node metastasis (N stage), and
the appearance of distant metastasis (M stage) [2]. The
eighth edition of the TNM staging system firstly proposed
the clinical tumor staging method (cTNM) to standardize
the preoperative management of GC patients, and the
specific criteria for staging also refer to the pathological
diagnosis of the tumor. Early studies have shown that
T-stage and N-stage are independent prognostic factors
for survival in GC patients, and the risk of lymph node
metastasis is higher with the increase in T-stage [3].
Early gastric cancer (EGC) and locally advanced gastric
cancer (LAGC) are two key time points in the treatment
of gastric cancer. Anatomically, EGC is defined as the
tumor confined to the mucosa or submucosa, independent
of lymph node metastasis; LAGC is defined as the tumor
extends beyond the anatomical border of the stomach
without distant metastasis or invasion of the serosa layer.
According to statistics, the 5-year survival rate of early
gastric cancer can reach 90%, which is much higher than
the overall survival rate of gastric cancer (25–30%) [4].
At present, the technique of endoscopic gastrectomy for
early gastric cancer is rather well-established with smaller
trauma and quicker postoperative recovery than surgical
operation [5]. Despite the poor outcomes, several clinical
trials in recent years have demonstrated that neoadjuvant
chemotherapy significantly reduces the disease staging
and improves radical resection, disease-free survival, and
overall survival rate for LAGC patients [6]. Therefore, the
precise preoperative diagnosis of tumor staging is of great
significance to improve the survival rate and quality of life
for GC patients.
Computed tomography (CT) imaging is used widely in
preoperative assessment and clinically staging of gastric
cancer as the CT of the abdomen contains essential visual
data for the characterization of gastric tissue patterns
associated with gastric cancer [7]. Radiomics can
automatically provide many quantitative image features
from medical images, which tend to be hard for naked
eyes to recognize [8]. Radiomics-based analysis of GC
CT images has shown good performance on classification,
predicting tumor recurrence and overall survival [9–12].
Meanwhile, electronic health records (EHRs) systems
collect patient’s health information including diagnosis,
laboratory tests and medication. Therefore, EHRs
13
La radiologia medica (2023) 128:509–519
data can be used to identify patterns for some diseases
and provide useful information for clinicians in their
practice. These motivate us to apply machine learning
techniques into gastric cancer diagnosis and treatment
with the combination of CT visual data and EHRs clinical
information.
However, analyzing CT images by radiomics has some
technical limitations, including hand-crafted feature
engineering and multi-step pipelines [13]. Recently, DL
methods are becoming increasingly popular for medical
image analysis because of their impressive classification
performance [14]. Yet selecting a model from a wide
range of deep network architecture proposed or designed
is complex and often requires human prior, with a good
knowledge of the field and the data. Here we propose to
adopt the neural architecture search (NAS) approach for
deep learning model construction. NAS is a framework that
automates the process of designing and optimizing neural
networks for researchers and has recently been actively
studied in the field of 3D medical image segmentation [15,
16].
This study aimed to develop and optimize a multi-
modal (CT/EHRs) artificial intelligence model based on
both preoperative radiomics and clinical information that
automatically predict the tumor stage of GC patients and
help to identify optimal treatment regimen and timing. The
comparison between the diagnostic results obtained using
our NAS-refined model and those obtained by experienced
radiologists showed the superiority of rapid and precise
diagnosis.
Methods
Study population
Patients were enrolled based on the following inclusion
criteria: (1) pathologically diagnosed as GC; (2) CT
carried out less than 2 weeks before surgery; (3) complete
clinical and pathological data were available. Patients were
excluded based on the following criteria: (1) neoadjuvant
therapy (chemotherapy, radiotherapy, chemoradiotherapy,
immunotherapy, molecular targeted therapy, etc.); (2)
previous abdominal malignancies or inflammatory diseases;
(3) present with unresectable primary tumor lesion or
distant metastases (liver, peritoneal, etc.) during surgical
exploration (4) difficult to segment the tumor because of
unsatisfactory gastric distention; (5) artifacts on CT images
seriously deteriorating the observation of tumor infiltration
depth. A schematic flow chart for all enrolled patients is
shown in Fig. 1.
A cumulative total of 602 GC patients were enrolled in
our center between January 2005 and December 2015 for
La radiologia medica (2023) 128:509–519
Fig. 1  The flowchart of patients enrollment
the subsequent analysis (Fig. 2). EHRs were obtained from
the inpatient records system and the radiological data were
collected from the picture archiving and communication
system (PACS) of our institution as a single-center
dataset. The retrospective study was approved by the local
ethics committee of our hospital, and the requirement for
patient informed consent was waived (Approval Number:
NFEC-2016-132).
Reference standard for gastric cancer stage
The pathologic staging of tumor was based on the 8th edition
of AJCC/UICC tumor staging system. The clinical staging
of tumor T staging were described as follows: (1) cT1:
focal thickening (with or without internal enhancement)
of the gastric wall, and low-density striations in the basal
part of the lesions near the submucosa; (2) cT2 stage: the
wall of stomach was thickened obviously with the loss or
destruction of low-density striations, but the margin of
serosa on the outside of the focus was clear and the fat space
around the stomach was clearly visible; (3) cT3 stage: the
wall of stomach was thickened obviously at the focus, (4)
cT4: the lesion breaks through the serosa and encroaches
on the surrounding tissues or adjacent organs [17]. The
clinical N staging criteria of the lesions were mainly based
on the density, enhancement, and size of lymph nodes
on CT images [18]. When the diameter of lymph nodes
511
was ≥ 1 cm, it was considered metastatic lymph nodes. When
the two physicians disagreed on the diagnosis, a third senior
radiologist was consulted to resolve the issue.
Image acquisition and ROI segmentation
After a non-contrast CT scan (Scanner: SIEMENS
64-MDCT or GE Healthcare, Hino) with a thickness of
2·0 mm, a dynamic contrast-enhanced scan was performed,
with 90–100 ml iodine contrast medium (Ultravist 370,
Bayer Schering Pharma, Germany) injected intravenously
at a rate of 3.0–3.5 ml/s. The other acquisition parameters
are as follows: tube voltage, 120 kV; tube current, 160–300
mAs; tube rotation time, 0.5–0.8 s; detector collimation,
8 × 2.5 or 64 × 0.625 mm; field of view, 350 × 350 mm.
Arterial phase image of contrast t-enhanced abdominal
CT with manual region of interest (ROI) was selected for
analysis.
The ITK-SNAP Software (version 3.8.0, http://w ​ ww.i​ tksn​
ap.​org/) was utilized to segment regions of interest (ROIs)
of each tumor on CT images, which were delineated with
the whole data blindly by two radiologists with 12 (reader
1, W.X.) and 7 years (reader 2, Z.C.) of experience in the
interpretation of abdominal CT (ICC value = 0.93). Before
delineation, grey-level standardization was applied to reduce
the grey-level differences caused by the imaging procedure.
All voxels within the ROI were extracted for analysis.
Radiomic feature extraction
We extracted visual features in three categories: first-
order statistics, shape-based features, and texture features,
including grey level co-occurrence matrix (GLCM) features,
grey level size zone matrix (GLSZM) features, grey level
run length matrix (GLRLM) features, neighboring grey
tone difference matrix (NGTDM) features and grey level
dependence matrix (GLDM) features. Each feature was
extracted from the patient’s volume of interest, stacked
by the corresponding ROIs delineated slice-by-slice. An
open-source python package named Pyradiomics (https://​
github.​com/​AIM-​Harva​rd/​pyrad​iomics) was used to extract
radiomics features.
We also applied feature extraction on wavelet and
Laplacian of Gaussian (LoG) filtered images in addition
to the original images. The wavelet filtering yields
eight decompositions per level, including all possible
combinations of applying a high or a low pass filter in each
of the three dimensions. The LoG filter enhances the edge
by emphasizing areas of grey level change. Sigma value in
the LoG filter defines how coarse the emphasized texture
should be. We extracted features from LoG filtered images
with sigma value equals 1.0, 2.0, 3.0, 4.0, 5.0, respectively.
13
512
Fig. 2  The pipeline of model development and model application.
A Collection of clinical data. Digital data of patients were extracted
from electronic health records (EHRs) and three-dimensional (3D)
segmentation of tumors in CT images were performed. B Radiomic
feature extraction and clinical characteristics preprocessing.
Radiomics features, including intensity, shape, and texture features,
were extracted from ROIs. C Model construction. Radiomic
features and clinical characteristics fused. Finally, a two-layer fully
In total, we extracted 1316 features from each patient.
Features were divided into three groups: (1) First-order
statistics (n = 252); (2) shape (n = 14); (3) texture based
(n = 1050), including GLCM (n = 336), GLSZM (n = 224),
GLRLM (n = 224), GLDM (n = 196) and NGTDM (n = 70).
Clinical features extraction
Clinical information collected included age, gender, tumor
location, T stage, N stage, degree of differentiation, and
tumor markers such as CEA, CA19-9, CA72-4, CA12-5
from EHRs. In total, we extracted 10 clinical features from
each patient. All the values were categorized using integers.
Model algorithms and development
After extracting the clinical and visual features from each
patient, we concatenated these two features and got the
final multi-modal input which could make the model learn
effectively with more diverse features. Figure 2 presents
13
La radiologia medica (2023) 128:509–519
connected neural network was developed using the neural architecture
search strategy. D Model evaluation. The classification abilities
of our models were assessed by the ROC and classification metrics
such as accuracy, sensitivity and specificity. During the model
application process, the new patient information was processed by
the constructed model and got the patient-level tumor stage and the
indication of optimal treatment decisions
the pipeline of image processing, model development,
and model application. We use NAS approach to develop
four MLP (multi-layer perceptron) models for four tasks:
(1) prediction of T-stage; (2) prediction of N-stage; (3)
prediction of the indication for endoscopic procedure; (4)
prediction of the indication for neoadjuvant chemotherapy.
We applied NAS using the python package of Optuna, an
open-source hyperparameter optimization framework to
automate hyperparameter search [19]. Details of model
development and settings are described in Appendix E1.
Model evaluation
The area under the receiver operating characteristic curve
(AUC) was used as the performance measure. Accuracy,
sensitivity, and specificity were calculated too. We
calculated their macro average for evaluation. The true
positive definition of a specific category is accurately
predicted as the category, and the true negative definition is
accurately predicted as one of the remaining four categories.
La radiologia medica (2023) 128:509–519 513

Youden Index was calculated to determine the best cutoff Table 1  Patient characteristics in the training and validation cohorts
value to predict the outcomes. Training cohort Validation cohort P value
(N = 452) (N = 150)
Statistical analysis
Age (year) 56.0 ± 11.5 54.5 ± 10.8 0.155
Gender
Statistical analyses were performed using python software
Male 298 (65.9%) 105 (70.0%) 0.413
(version 3.8.3, https://w
​ ww.p​ ython.o​ rg/) with Scipy package
Female 154 (34.1%) 45 (30.0%)
(version 1.5.1, https://​www.​scipy.​org/) or SPSS software
Tumor location
(version 24.0; SPSS Inc., Chicago, IL). Differences between
Up 93 (20.6%) 30 (20.0%) 0.324
cohorts were assessed using t-tests or Mann–Whitney U
Medium 93 (20.6%) 23 (15.3%)
tests for continuous variables; chi-squared tests or Fisher’s
Low 266 (58.8%) 97 (64.7%)
exact test were applied for categorical variables. Receiver
Tumor grade
operating characteristic (ROC) curves were used to display
High 22 (4.9%) 7 (4.7%) 0.78
and evaluate model performance.
Medium 75 (16.6%) 30 (20.0%)
Low 350 (77.4%) 112 (74.7%)
Undifferentiated 5 (1.1%) 1 (0.7%)
Results
cT stage
T1a 16 (3.5%) 3 (2.0%) 0.776
Baseline information of participants
T1b 39 (8.6%) 13 (8.7%)
T2 57 (12.6%) 15 (10.0%)
The 602 patient cohort from the center was divided into a
T3 81 (17.9%) 26 (17.3%)
training cohort (N = 452) and a validation cohort (N = 150).
T4a 245 (54.2%) 86 (57.3%)
Detailed clinicopathological baseline data and comparison
T4b 14 (3.1%) 7 (4.7%)
results are shown in Table 1. The training cohort consisted of
cN stage
298 males and 154 females (mean age 56.0 ± 11.5 years, age
N0 233 (51.5%) 76 (50.7%) 0.656
range 22–83 years), and the validation cohort consisted of
N1 106 (23.5%) 33 (22.0%)
105 males and 45 females (mean age 54.5 ± 10.8 years, age
N2 82 (18.1%) 26 (17.3%)
range 22–79 years). There were no significant differences
N3 31 (6.9%) 15 (10.0%)
in tumor location (longitudinal), tumor differentiation,
cM stage
preoperative CA 19–9 level, and CA 125 level, and
M0 441 (97.6%) 148 (98.7%) 0.632
significant differences in preoperative CEA level and CA
M1 11 (2.4%) 2 (1.3%)
72–4 between these two cohorts.
cTNM
I 94 (20.8%) 23 (15.3%) 0.059
Prediction of tumor stage and feature importance
II A 10 (2.2%) 7 (4.7%)
analysis
II B 125 (27.7%) 34 (22.7%)
III 195 (43.1%) 72 (48.0%)
Based on the Yonden Index, we determined the model
IV A 17 (3.8%) 12 (8.0%)
threshold for each prediction task. The overall accuracy for
IV B 11 (2.4%) 2 (1.3%)
predicting T-stage in the training and validation cohort were
pT stage
0.803 and 0.646, respectively, and the macro-average AUC
T1a 33 (7.3%) 11 (7.3%) 0.912
values were 0.892 and 0.716 (Table E2-3, Fig. 3A, B). In
T1b 51 (11.3%) 13 (8.7%)
the prediction model of lymph node metastasis, the overall
T2 53 (11.7%) 15 (10.0%)
accuracy in the training and validation cohort were 0.669
T3 43 (9.5%) 17 (11.3%)
and 0.838. respectively, and the macro-average AUC values
T4a 234 (51.8%) 81 (54.0%)
were 0.712 and 0.698 (Table E4-5, Fig. 3C, D).
T4b 38 (8.4%) 13 (8.7%)
To better understand the landscape of clinical parameters
pN stage
and radiomic features in the prediction model, we took
N0 184 (40.7%) 49 (32.7%) 0.294
the CNN model as a dark box and got the weight of each
N1 68 (15.0%) 27 (18.0%)
input by permutation technique. In the prediction task of
N2 81 (17.9%) 27 (18.0%)
T-stage, original image derived dependence variance (DV)
N3a 67 (14.8%) 31 (20.7%)
of GLDM applying LoG filter with Gaussian kernel size
N3b 52 (11.5%) 16 (10.7%)
of 4 (log-4_gldm_DependenceVariance) weighted the most
important in the forecasting system (Fig. 4A). The top three

13
514 La radiologia medica (2023) 128:509–519

Table 1  (continued) higher log-4 _gldm_DependenceVariance values tend to

Training cohort Validation cohort P value be more advanced and aggressive (Fig. 5A). For N stage,
(N = 452) (N = 150) the correlation analysis revealed that Original_Shape_
maximum_2D_Diameter was positively correlative with
pM stage
lymph node metastasis (rs = 0.334, P < 0.001) (Fig. 5B).
M0 411 (90.9%) 137 (91.3%) 1.000
M1 41 (9.1%) 13 (8.7%)
Prediction model of optimal treatment regimen
pTNM
and timing based on tumor stage
IA 76 (16.8%) 20 (13.3%) 0.503
IB 34 (7.5%) 12 (8.0%)
Based on the above results, we intended to further
II A 30 (6.6%) 10 (6.7%)
construct predictive models for identifying potential benefit
II B 69 (15.3%) 18 (12.0%)
populations for preoperative neoadjuvant and endoscopic
III A 100 (22.1%) 36 (24.0%)
therapy according to the tumor stage prediction, thereby
III B 56 (12.4%) 29 (19.3%)
avoiding overtreatment by surgery. According to the
III C 46 (10.2%) 12 (8.0%)
literature, the preoperative staging of gastric cancer patients
IV 41 (9.1%) 13 (8.7%)
with indications for endoscopic resection (IER) should be
CEA (ng/mL)
T1aN0M0, we therefore screened patients with pT1aN0M0
≥  5 213 (47.1%) 56 (37.3%) 0.046*
staging (44/602, 7.3%) from the entire cohort and labeled
<5 239 (52.9%) 94 (62.7%)
them as predictors; we also manually selected patients with
CA 19–9 (U/mL)
indication for preoperative neoadjuvant chemotherapy (INC,
≥ 37 211 (46.7%) 56 (37.3%) 0.057
pT1-2N + M0 or pT3/4NxM0, 464/602, 77.1%) to further
< 37 241 (53.3%) 94 (62.7%)
our research. The results showed that the accuracy of IER in
CA 72-4 (U/mL)
the training and validation cohort were 1.000 and 0.718, and
≥6 162 (35.8%) 39 (26.0%) 0.034*
the AUC value were 1.000 and 0.771 (Table E4, Fig. 6). In
<6 290 (64.2%) 111 (74.0%)
the task of predicting INC, the accuracy of the training and
CA 125 (U/mL)
validation cohort were 0.720 and 0.713, and the AUC value
≥ 35 7 (1.5%) 4 (2.7%) 0.593
were 0.687 and 0.661, respectively. (Table E4, Fig. 6). In the
< 35 445 (98.5%) 146 (97.3%)
two models, the most important feature for predicting IER
*P < 0.05 and INC were wavelet-LHH_ngtdm_Strength and original_
glrlm_RunLengthNonUniformity, respectively (Fig. 6).

factors that played critical roles in the predicting process

were: log-4_gldm_DependenceVariance, original_glcm_
Inverse-Variance, log-5_gldm_SmallDependenceHigh-
GrayLevelEmphasis. In the prediction task of N-stage,
original image derived maximum 2D Diameter row ranked
highest above other features (Fig. 4B), and the following
two factors are log-3_glrlm_LongRunEmphasis, and
wavelet_LLLfirstorder_TotalEnergy. Notably, CA 19–9
exhibited great contribution to both prediction models as
the only clinical feature that appears in the top 1% of all
input variables.
Correlation between the important features
and tumor stages
From the top five important features mentioned above,
we managed to further discover the relationship between
input features and T-stage as well as N-stage. Using
Spearman’s correlation analysis, positive correlations
were found between log- 4 _gldm_DependenceVariance,
Original_glcm_InverseVarianceand and T-stage (rs = 0.277,
P < 0.001; rs = 0.157, P < 0.001). In general, tumors with
Discussion
The aim of this study was to develop and validate a multi-
modal (CT/EHRs) artificial intelligence model for predicting
the tumor stages and the optimal treatment regimen and
timing of GCs patients with NAS approach. To determine
its performance for grading tumor stages, we reported the
macro-average AUC of 0.716 for five T stages and 0.698 for
four N stages, respectively. Meanwhile, we also reported
the macro-average ACC of 0.646 for five T stages and
0.838 for four N stages, respectively, which showed greater
discrimination than traditional methods with ACC of 0.543
(P value = 0.034) and 0.468 (P value = 0.021), respectively.
Meanwhile, we reported the AUC of 0.771 for the indication
for endoscopic resection and 0.661 for the indication for
preoperative neoadjuvant chemotherapy, respectively.
During CT screening procedures, these models may assist
doctors in predicting the histology of ambiguous lesions and
determining diagnostic or therapeutic strategies.
Although guidelines for the treatment of GC vary from
country to country, overall treatment principles are based

13
La radiologia medica (2023) 128:509–519
Fig. 3  ROC curves of the prediction model. A infiltration depth
prediction model in the training cohort; B infiltration depth prediction
model in the validation cohort; C lymph node metastasis prediction
on the surgeon's accurate preoperative clinical staging
and risk stratification. Moreover, early identification of
LAGC patients before surgery can improve the response
rate of neoadjuvant chemotherapy and provide further
treatment for patients who lost the chance of surgery.
According to the literature, the accuracy rate of Radiologists
subjectively determining the depth of invasion and lymph
node metastasis of GC based on CT images are about 70%,
which is unsatisfactory [20, 21]. In previous studies, Wang
et al. and Meng et al. used traditional radiomics and clinical
515
model in the training cohort; D lymph node metastasis prediction
model in the validation cohort
information to classify T-stage in binary categories for GC
such as T2 vs T3/T4 and T1/T2/T3 vs T4 successfully with
the AUC values of 0.818–0.825 and 0.813–0.840 in the
testing cohort [11, 22]; Chen et al. and Wang et al. applied
the DL method to diagnostic analysis such as recurrence
prediction and metastasis identification [23, 24]. However,
most studies adopted a single examination method, and did
not integrate multi-modal examination methods.
In this study, we combined enhanced CT visual data with
the EHRs data to construct multi-modal models to focus on
13
516
Fig. 4  Top 10 important features for gastric cancer A infiltration depth prediction model; B lymph node metastasis prediction model
Fig. 5  Spearman’s correlation plots of tumor infiltration depth (A) and lymph node metastasis (B) with their top important feature, respectively
the prediction of specific stages of GC into five T categories
and four N categories as well as the identification to recive
endoscopic resection and neoadjuvant chemotherapy
preoperatively to simulate real-world clinical decision-
making. Given a limited number of training samples,
training multiple sophisticated deep learning models are
very challenging due to the choice of the hyperparameters
and construction of the networks architecture. Fortunately,
we can use the NAS approach adaptively to design the best
architecture and hyperparameters to tackle these challenges.
From the feature importance analysis, we can see that
the top 10 important features have both radiomic features
from CT images and clinical parameters from EHRs. This
means that the multi-modal (CT/EHRs) information is
more helpful for gastric cancer diagnosis and treatment
decisions compared with single modal CT images or
EHRs. In terms of visual information, radiomic features
extracted from the original image filtered by wavelet are
less critical than LoG and the original image. The most
significant feature in this study originated from the GLDM
method of measuring variance independence size in CT
images. We considered this finding to be relevant to the
interpretation of images. T-stage describes the expansion
of the lesion border within the gastric wall during tumor
13
La radiologia medica (2023) 128:509–519
growth, which from the view of graphic processing is
the relationship between a voxel and neighboring voxels.
Consequently, the histogram-like function of GLDM
were able to illustrate the color transitions between
adjacent pixels with better performance. As for clinical
parameters, CA19-9 gives a robust contribution to the
classification task. Carbohydrate antigen 19-9 has been
widely used for assisting diagnosis and treatment efficacy
evaluation in digestive tract malignancies like pancreatic
cancer or cholangiocarcinoma [25, 26]. The association
of elevated CA19-9 levels with gastric carcinoma has
been presented in a case report and other studies [27,
28]. Elevated CA19-9 levels have been significantly
correlated with lymph node metastasis, vascular invasion,
and liver metastasis. Our results indicated that CA19-9
aids in T-stage and N-stage differentiation, consistent with
previous researches [29, 30].
We found that the model performed poorly in
intermediate classes such as T3 or T4a (Table E2–E3). An
explanation for this phenomenon was that the category of
T-stage generally depends on the location and volume of
the primary tumor lesion. The difference between the sizes
of some intermediate tumors may be subtle. Secondly, there
existed a severe class imbalance in our data, consistent with
La radiologia medica (2023) 128:509–519
Fig. 6  ROC curves of the optimal clinical decision model and Top 10 important features of each model, respectively. A ROC curve of the IER
model; B ROC curve of the INC model; C important features in the IER model; D important features in the INC model
the real-world clinical practice in our country that the GC
patients graded T1/T2 is relatively fewer than T3/T4 [31].
The key strengths of this study include the use of multi-
modal (CT/EHRs) information with the combination of
visual features and clinical features, and the use of NAS
approach to develop p the artificial intelligence model which
alleviates the challenge for the choice of hyperparameters and
networks architecture during training. In detail, we built two
multi-category prediction model which is nearly consistent
with pathological findings using radiomics features from CT
images and clinical parameters. This will help doctors save a
certain amount of time spend on diagnosis, so as to gain more
treatment time for patients. To our knowledge, our method is
the first work combining the radiomics and NAS approach to
develop the AI models, which alleviates the challenge for the
choice of hyperparameters and network architecture during
training. However, there are also some potential limitations
517
need to be considered. Firstly, this was a retrospective study
conducted at a single center with inter-class imbalance.
Secondly, there is still room for the accuracy and overall
performance of our model to improve. Thirdly, the prediction
of the indications of neoadjuvant and endoscopic therapy were
only a stimulation of clinicians in clinical decision-making
based on clinical tumor stage prior to surgery, thus it can only
provide reference rather than medical instructions. Fourthly,
a multicenter study with a larger dataset is needed to further
validate our models’ reproducibility and generalization ability
in future studies.
13
518
Conclusions
To sum up, this AI-based staging model incorporating
high-dimensional radiomics features and clinical
information possess the potential to aid preoperative
clinical tumor stage classification, filter early or locally
advanced GC cases and make reasonable recommendations
in the clinical decision-making process.
Supplementary Information The online version contains
supplementary material available at https:// ​ d oi. ​ o rg/ ​ 1 0. ​ 1 007/​
s11547-​023-​01625-6.
Acknowledgements Not applicable.
Author contributions All authors have had access to the data and
all drafts of the manuscript. SL, WL contributed equally as co-first
authors. Specific contributions are as follows: SL: conceptualization,
methodology, software, investigation, writing - review & editing; WL:
conceptualization, resources, data curation, visualization, writing -
original draft; PH: formal analysis, writing - original draft; DC: data
curation, formal analysis; QH: validation, visualization; ZN: writing
- review & editing, project administration; YZ: project administration,
supervision; WX: writing - review & editing, supervision; JY: project
administration, supervision; TC: project administration, supervision,
funding acquisition. All authors read and approved the final manuscript.
Funding This project was supported by Guangdong Provincial Key
Laboratory of Precision Medicine for Gastrointestinal Cancer (No.
2020B121201004), Guangdong Natural Science Foundation General
Project (2019A1515011520) and Guangdong Natural Science
Foundation Outstanding Youth Project (2021B1515020055).
Data availability All data generated or analyzed during this study are
included either in this paper or in the additional information.
Declarations
Coonflict of interest The authors declare that they have no competing
interests.
Ethics approval and consent to participate The Institutional Review
Board of Nanfang Hospital of Southern Medical University approved
this study and waived the need for informed consent from patients
(Approval Number: NFEC-2016-132).
Consent for publication Not applicable.
References
1. Sung H, Ferlay J, Siegel RL (2021) Global cancer statistics
2020: GLOBOCAN estimates of incidence and mortality
worldwide for 36 cancers in 185 countries. CA Cancer J Clin
71(3):209–249
2. Amin M, Edge S, Greene F (2017) AJCC cancer staging manual.
Springer, Berlin
3. Xu YY, Huang BJ, Sun Z, Lu C, Liu YP (2007) Risk factors for
lymph node metastasis and evaluation of reasonable surgery for
early gastric cancer. World J Gastroenterol 13(38):5133–5138
13
La radiologia medica (2023) 128:509–519
4. Oliveira C, Pinheiro H, Figueiredo J, Seruca R, Carneiro F
(2015) Familial gastric cancer: genetic susceptibility, pathology,
and implications for management. Lancet Oncol 16(2):e60-70
5. Roy S, Kanda M, Nomura S et al (2022) Diagnostic efficacy of
circular RNAs as noninvasive, liquid biopsy biomarkers for early
detection of gastric cancer. Mol Cancer 21(1):42
6. Schernberg A, Rivin Del Campo E, Rousseau B et al (2018)
Adjuvant chemoradiation for gastric carcinoma: state of the art
and perspectives. Clin Transl Radiat Oncol 10:13–22
7. Barros RH, Penachim TJ, Martins DL, Andreollo NA, Caserta NM
(2015) Multidetector computed tomography in the preoperative
staging of gastric adenocarcinoma. Radiol Bras 48(2):74–80
8. Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are
more than pictures, they are data. Radiology 278(2):563–577
9. Giganti F, Antunes S, Salerno A et al (2017) Gastric cancer:
texture analysis from multidetector computed tomography
as a potential preoperative prognostic biomarker. Eur Radiol
27(5):1831–1839
10. Chen X, Yang Z, Yang J et al (2020) Radiomics analysis of
contrast-enhanced CT predicts lymphovascular invasion and
disease outcome in gastric cancer: a preliminary study. Cancer
Imaging 20:1–12
11. Meng L, Dong D, Chen X et al (2021) 2D and 3D CT radiomic
features performance comparison in characterization of gastric
cancer: a multi-center study. IEEE J Biomed Health Inform
25(3):755–763
12. Li J, Dong D, Fang M et al (2020) Dual-energy CT–based deep
learning radiomics can improve lymph node metastasis risk
prediction for gastric cancer. Eur Radiol 30(4):2324–2333
13. Koçak B, Durmaz E, Ateş E, Kılıçkesmez Ö (2019) Radiomics
with artificial intelligence: a practical guide for beginners. Diagn
Interv Radiol 25(6):485–495
14. Lee JH, Ha EJ (2020) Application of deep learning to the
diagnosis of cervical lymph node metastasis from thyroid cancer
with CT: external validation and clinical utility for resident
training. Eur Radiol 30(6):3066–3072
15. Zoph B, Le QV (2016) Neural architecture search with
reinforcement learning. arXiv preprint arXiv:​1611.​01578.
16. Bae W, Lee S, Lee Y, Park B, Chung M, Jung K-H (2019)
Resource optimized neural architecture search for 3D medical
image segmentation. In: International conference on medical
image computing and computer-assisted intervention. Springer,
pp 228–236
17. Kim JW, Shin SS, Heo SH et al (2012) Diagnostic performance
of 64-section CT using CT gastrography in preoperative T staging
of gastric cancer according to 7th edition of AJCC cancer staging
manual. Eur Radiol 22(3):654–662
18. Hallinan JT, Venkatesh SK, Peter L, Makmur A, Yong WP, So
JB (2014) CT volumetry for gastric carcinoma: association with
TNM stage. Eur Radiol 24(12):3105–3114
19. Akiba T, Sano S, Yanase T, Ohta T, Koyama M (2019) Optuna:
a next-generation hyperparameter optimization framework. In:
Proceedings of the 25th ACM SIGKDD international conference
on knowledge discovery & data mining, pp 2623–2631
20. Seevaratnam R, Cardoso R, McGregor C et al (2012) How useful
is preoperative imaging for tumor, node, metastasis (TNM)
staging of gastric cancer? A meta-analysis Gastric Cancer
15(Suppl 1):S3-18
21. Choi JI, Joo I, Lee JM (2014) State-of-the-art preoperative staging
of gastric cancer by MDCT and magnetic resonance imaging.
World J Gastroenterol 20(16):4546–4557
22. Wang Y, Liu W, Yu Y et al (2020) Prediction of the depth of tumor
invasion in gastric cancer: potential role of CT radiomics. Acad
Radiol 27(8):1077–1084
23. Chen T, Liu S, Li Y et al (2019) Developed and validated a
prognostic nomogram for recurrence-free survival after complete
La radiologia medica (2023) 128:509–519
surgical resection of local primary gastrointestinal stromal tumors
based on deep learning. EBioMedicine 39:272–279
24. Wang X, Chen Y, Gao Y et al (2021) Predicting gastric cancer
outcome from resected lymph node histopathology images using
deep learning. Nat Commun 12(1):1637
25. Carpelan-Holmström M, Louhimo J, Stenman UH, Alfthan H,
Haglund C (2002) CEA, CA 19-9 and CA 72-4 improve the
diagnostic accuracy in gastrointestinal cancers. Anticancer Res
22(4):2311–2316
26. Ballehaninna UK, Chamberlain RS (2012) The clinical utility
of serum CA 19-9 in the diagnosis, prognosis and management
of pancreatic adenocarcinoma: an evidence based appraisal. J
Gastrointest Oncol 3(2):105–119
27. Feng F, Tian Y, Xu G et al (2017) Diagnostic and prognostic value
of CEA, CA19-9, AFP and CA125 for early gastric cancer. BMC
Cancer 17(1):737
28. Jo JC, Ryu MH, Koo DH et al (2013) Serum CA 19–9 as a
prognostic factor in patients with metastatic gastric cancer. Asia
Pac J Clin Oncol 9(4):324–330
29. Guo X, Lv X, Ru Y et al (2020) Circulating exosomal gastric
cancer-associated long noncoding RNA1 as a biomarker for
519
early detection and monitoring progression of gastric cancer: a
multiphase study. JAMA Surg 155(7):572–579
30. Song XH, Liu K, Yang SJ et al (2020) Prognostic value of changes
in preoperative and postoperative serum CA19-9 levels in gastric
cancer. Front Oncol 10:1432
31. Tsendsuren T, Jun SM, Mian XH (2006) Usefulness of endoscopic
ultrasonography in preoperative TNM staging of gastric cancer.
World J Gastroenterol 12(1):43–47
Publisher's Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
Springer Nature or its licensor (e.g. a society or other partner) holds
exclusive rights to this article under a publishing agreement with the
author(s) or other rightsholder(s); author self-archiving of the accepted
manuscript version of this article is solely governed by the terms of
such publishing agreement and applicable law.
13