Artificial Intelligence Based Diagnosis For Cervical Lymph Node Malignancy Using The Point-Wise Gated Boltzmann Machine

This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2018.2873043, IEEE Access
Artificial intelligence based diagnosis for cervical lymph node

malignancy using the point-wise gated Boltzmann machine
Qi Zhang1,2, Yue Liu2, Hong Han3*, Jun Shi1, Wenping Wang3*

1. Shanghai Institute for Advanced Communication and Data Science, Shanghai University,
Shanghai, China.
2. The SMART (Smart Medicine and AI-based Radiology Technology) Lab, Institute of
Biomedical Engineering, Shanghai University, China.
3. Department of Ultrasound, Zhongshan Hospital, Fudan University, Shanghai, China
*Authors for Correspondence:

Hong Han, MD, Associate Professor
E-mail address: hanhongzs@163.com
Tel: +86-21-64041990-2474
Address: Department of Ultrasound, Zhongshan Hospital, Fudan University, No.
180, Fenglin Road, 200032, Shanghai, China
Wenping Wang, MD, Professor

E-mail address: puguang61@126.com
Tel: +86-21-64041990-2474
Address: Department of Ultrasound, Zhongshan Hospital, Fudan University, No.
180, Fenglin Road, 200032, Shanghai, China
2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2018.2873043, IEEE Access
ABSTRACT
Purpose: To build an artificial intelligence (AI) architecture for automated extraction of learned-
from-data image features from contrast-enhanced ultrasound (CEUS) videos, and to evaluate the
AI architecture for classification between benign and malignant cervical lymph nodes.
Methods: An AI architecture for CEUS feature extraction and classification was constructed by
using the point-wise gated Boltzmann machine (PGBM). The PGBM consisted of task-relevant
and task-irrelevant hidden units for both feature learning and feature selection, and the task-
relevant units were connected to the support vector machine (SVM) to yield the likelihood for
classification. The synthetic minority over-sampling technique was used to improve the
classification ability for an unbalanced dataset. Experimental evaluation was performed with the
five-fold cross validation on a database of 127 lymph nodes (39 benign and 88 malignant) from
88 patients.
Results: The SVM likelihood exhibited a significant difference between benign and malignant
cervical lymph nodes (0.74±0.21 vs. 0.33±0.28, p<0.001). On the test set, the accuracy,
precision, sensitivity, specificity and Youden’s index of the AI architecture were 82.55%,
89.58%, 84.75%, 77.56% and 62.32%, respectively.
Conclusions: The AI architecture using the PGBM shows promising classification results, and it
may be potentially used in clinical diagnosis for cervical lymph node malignancy.
Keywords: artificial intelligence; contrast-enhanced ultrasound; cervical lymph nodes; point-
wise gated Boltzmann machine
lymphadenopathy diagnosis. Therefore, it

Ⅰ. INTRODUCTION would be desirable to develop computer-
Cervical lymphadenopathy often aided diagnosis (CAD) approaches for more
presents with head and neck malignancies accurately, objectively and efficiently
such as thyroid cancer, lymphoma, and interpreting CEUS images and
esophageal cancer [1]. Differentiating distinguishing malignant from benign
malignant from benign cervical lymph nodes cervical lymph nodes [9, 10].
is important for clinical management Recent CAD systems for
including staging, prognosis, and lymphadenopathy on ultrasound uses the
optimization of treatment process [1]. statistical features (SFs), also called the
Contrast-enhanced ultrasound (CEUS) is an human-crafted features in research areas of
emerging technique for diagnosis of computer vision [11, 12]. The SFs include
lymphadenopathy [2-7]. CEUS involves shape features and intensity statistics of a
harmonic imaging and administration of lymph node that are calculated on B-mode,
ultrasound contrast agent intravenously to Doppler sonography and elastography [12-
enhance the backscattering signal and 15], as well as perfusion features that are
visualize the perfusion of intranodal blood derived from the time-intensity curve (TIC)
vessels [2, 4, 7]. analysis on CEUS [3]. The SFs are often
Lymph node examination using CEUS extracted by depending on professional
provides a video lasting for a few minutes knowledge or human labor, and the choice
that may include thousands of sequential of specific SFs largely affects the diagnosis
images. Visual interpretation of a CEUS performance [16]. Artificial neural networks
video is subjective, tedious and time- (ANNs) have gained a huge attention in
consuming for a radiologist or oncologist [8], various fields on account of recent advances
which also limits the accuracy of in artificial intelligence (AI) technology [17].
ANNs involve applying connected units relevant image features from CEUS videos,
called artificial neurons to raw data (such as and to evaluate the AI architecture in
the pixel values in an image) to learn classification between benign and malignant
features (representations) of the data. Thus cervical lymph nodes.
in an image classification scenario, ANNs Ⅱ. MATERIALS AND METHODS
allow a computer to be fed with raw pixel A. IMAGE ACQUISITION AND PRE-
values and to automatically explore the PROCESSING
learned-from-data features needed for end- This retrospective study was approved
to-end classification [17]. Recent by our institutional review board and
development of CAD for breast tumors with informed consent was obtained from all
ultrasound has suggested that the image patients. Cervical lymph nodes suspected for
features automatically induced from pixels malignancy were referred to CEUS
by ANNs could outperform the SFs [18, 19]. examination at the Department of
The performance of ANNs on the Ultrasound, Zhongshan Hospital, Fudan
classification of lymphadenopathy with University. The CEUS video images of 127
CEUS is yet to be determined. lymph nodes from 88 patients (33 male and
However, lymph node CEUS images 55 female; 49.1±15.4 years old; age range:
contain speckle noise, artifacts including 20–77 years) were collected. Malignant
acoustic shadows, and other irrelevant diagnosis was reached through pathology as
patterns including the interference from the gold standard, while the benignancy was
neighboring tissues or large vessels such as confirmed through pathology or negative
carotid arteries. These irrelevant patterns follow-up. There were 39 benign and 88
may all hinder precise classification of malignant lymph nodes in the cohort.
lymph nodes. To deal with the irrelevant The CEUS imaging was carried out
patterns is an extreme difficulty in building with the MyLab Twice ultrasound system
an ANN architecture that can robustly learn (Esaote SpA, Genoa, Italy) and a bolus
from complex CEUS video images. Hence injection of 1.5 mL of the contrast agent
the challenge is how to learn robust SonoVue (Bracco SpA, Milan, Italy). Each
representations that can differentiate useful CEUS video was in a size of 800 × 555 with
(i.e., task-relevant) patterns from large 256 gray scales, a resolution of 138.3 ± 24.3
amounts of distracting (i.e., task-irrelevant) pixel/cm, and a sampling frequency of 24.2
patterns [20]. ±3.4 frame/s.
Popular ANN methods including the After injection of contrast agent, the
convolutional neural network and average gray level (AGL) within a lymph
autoencoder are inappropriate for node on CEUS varies along time due to
overcoming the challenge, because they do blood perfusion. In a CEUS video sequence,
not focus on distinguishing task-relevant and an elliptical region of interest was specified
irrelevant patterns [11]. Instead, a newly along the nodal border, and the AGL within
developed ANN approach, the point-wise the region was calculated to generate a TIC.
gated Boltzmann machine (PGBM), appears From the TIC, the frame with the largest
to be promising through introduction of a AGL called the peak frame was detected. A
gating mechanism for estimating where sequence of successive frames, which lasted
task-relevant patterns occur [20]. Thus, the several seconds and was centered on the
aim of this study is to construct an AI peak frame, was automatically selected from
architecture by using PGBM for automated the video images with the Butterworth band-
extraction of learned-from-data, task- pass and low-pass filtering. These selected
frames were averaged to derive a temporal and our recent application of PGBM to
mean image (TMI). The TMI was breast tumor sonoelastogrpahy [11].
downsampled to a fixed size of 32 × 54 and C. COMBINATION OF PGBM AND
converted to a vector of 1728 dimensions as SMOTE FOR UNBALANCED DATA
the input of the AI architecture. For details The ratio of benign and malignant
of the CEUS image pre-processing, readers sample size was 1:2.3 in our study, which
can refer to our previous work [8, 21, 22]. means the dataset was unbalanced. The
B. PGBM-BASED AI ARCHITECTURE unbalance of data has a large influence on
FOR FEATURE EXTRACTION AND the classification performance of SVM [23,
CLASSIFICATION 24]. Therefore, we used the synthetic
A CEUS image of a cervical lymph minority over-sampling technique (SMOTE)
node contained a large amount of irrelevant to reduce the class unbalance by creating
sensory patterns such as speckle noise, synthetic minority class examples [23]. The
acoustic shadows, and interference from basic principle of the SMOTE is to
neighboring tissues or vessels, which were interpolate between two minority class
distracting to the diagnosis. Thus it was samples, resulting in a new minority class
crucial to explore an automated algorithm to sample.
distinguish between relevant and irrelevant The PGBM-based AI architecture for
patterns. Here we used the PGBM to diagnosis on an unbalanced dataset is
estimate where task-relevant patterns illustrated in Fig. 2, including the following
occurred on CEUS and thereby constructed steps:
an AI architecture for CEUS feature 1) The dataset was divided into a
extraction and classification based on the training set and a test set;
PGBM. 2) The SMOTE was used to increase
As shown in Fig. 1, first, the CEUS the number of minority class samples in the
video images were pre-processed to produce training set to reduce the unbalance of data;
a TMI image, and the TMI image was 3) The PGBM and SVM were used on
converted to a vector as an input to the the training set for training the feature
PGBM. Second, the PGBM introduced a extraction and classification model;
gating mechanism by relying on the so- 4) The PGBM and SVM were used on
called “switch units”, through which the the test set for validating the performance of
PGBM performed feature selection both on the trained model.
learned high-level features (i.e., hidden units) D. CROSS VALIDATION AND
and on raw features (i.e., visible units that PERFORMANCE EVALUATION
were TMI image pixels) [11, 20]. Finally, The proposed approach for AI based
based on the features selected by the PGBM, diagnosis, as well as the statistical analysis
the support vector machine (SVM) was used and validation, was implemented with
to yield the likelihood of classification and MATLAB R2014a (The MathWorks, Natick,
to identify the benign and malignant cervical MA, USA).
lymph nodes. Here, the SVM likelihood of 1) GENERAL EVALUATION
classification was a posterior probability First on the entire cohort, the
between 0 and 1 representing the possibility independent two-sample t-test was employed
of a lymph node being classified into a to examine the difference of SVM likelihood
benign one [11, 12]. For details of the between benign and malignant lymph nodes.
PGBM technique, please refer to the Then the five-fold cross validation was used
pioneering technical work by Sohn et al. [20] to verify the feature extraction and
classification model. The entire dataset was The TIC features consisted of the peak
randomly divided into five equal-sized intensity, enhanced intensity, mean transit
subsets, four of which were used for training time, time to peak, etc. [8, 21]. The gray
and the remaining one was used for testing. level statistical features included the mean,
This process was repeated five times, and median, standard deviation, entropy and
each time one different subset was used as a several percentiles of the gray levels within
test set. The five-fold cross validation was a lymph node on the TMI [21]. The gray
again randomly repeated five times, taking level co-occurrence matrix features included
the average values from 25 test sets as the contrast, energy, entropy and homogeneity
final results. [22, 25]. The binary image texture features
In order to quantitatively evaluate the included center deviation degree, dispersion
classification performance, the following degree, etc [21]. In total, 312 SFs were
indices were calculated: accuracy (ACC), calculated. Then we performed two schemes
precision (PRE), sensitivity (SEN), regarding feature selection of SFs: (a) the
specificity (SPE), and Youden’s index (YI). least absoulute shrinkage and selection
In addition, we summarized the numbers of operator (LASSO), and (b) t-test [11]. Thus,
cases correctly and wrongly classified, and two SF models for feature extraction and
we performed 2-test to compare two classification were derived and named SF-
classification models. LASSO and SF-T, respectively.
2)COMPARISON BETWEEN DIFFERENT Adding an error cost (EC) in the SVM
MODELS classifier may contribute to enhanced
Our final AI architecture (i.e., model) for performance for unbalanced data [24]. For
feature extraction and classification using comparison purposes in this study, we set a
both PGBM and SMOTE was named larger EC for the minority class in order to
PGBM-SMOTE. The PGBM-SMOTE reduce the influence of the unbalanced data.
model was compared with the model only The PGBM model modified with adding an
using PGBM but without SMOTE, namely EC term was named PGBM-EC.
the PGBM model. The PGBM-SMOTE and In total, there are 16 feature extraction
PGBM were both shallow ANNs which only and classification models for comparison,
had a single layer of networks. In our named as “XX-YY”. Here, “XX” stands for
previous work, we proposed using a deep PGBM, PGBM-RBM, SF-LASSO, or SF-T;
ANN model named the PGBM-RBM model, “YY” stands for SMOTE, EC, both SMOTE
which had two layers of networks, for and EC (“SMOTE-EC”), or none of
classification of breast tumors on SMOTE and EC (only “XX”).
sonoelastography [11]. Here, the PGBM-
SMOTE model was also compared with the Ⅲ. RESULTS
PGBM-RBM model and its variant A. CHARACTERISTICS OF CERVICAL
combining with SMOTE (PGBM-RBM- LYMPH NODES
SMOTE). Among 39 benign lymph nodes, 35 were
We further compared the features reactive hyperplasia and 4 were lymph node
learned and selected by using our AI tuberculosis. Among 88 malignant lymph
architecture with the SFs on classification of nodes, 13 were lymphoma, and 57 were
lymph nodes. The SFs included the TIC metastasis from thyroid cancer, 8 from lung
features, gray level statistical features, gray cancer, 4 from esophageal cancer, 3 from
level co-occurrence matrix features, and breast cancer, 1 from nasopharyngeal cancer,
binary image texture features [11, 21, 22].
1 from ovarian cancer, and 1 from shown in Fig. 3a. There were 16 cases
mesothelioma. correctly classified by both the PGBM-
The SVM likelihood in benign cervical SMOTE and PGBM but misclassified by the
lymph nodes was significantly different SF-LASSO-SMOTE (Fig. 3b). In addition,
from that in malignant nodes (0.74±0.21 vs. there were 7 cases correctly classified by the
0.33±0.28, p<0.001). PGBM-SMOTE but misclassified by both
B. CLASSIFICATION RESULTS OF the PGBM and SF-LASSO-SMOTE (Fig.
PGBM-BASED MODELS 3c). The PGBM-SMOTE correctly classified
Table 1 shows classification results of more cases than SF-LASSO-SMOTE (104
eight PGBM-based models on the test sets. correct: 23 wrong vs. 93:34, p = 0.098).
According to the YI-value, the PGBM- PGBM-SMOTE and PGBM had
SMOTE performed best among eight similar classification results (104:23 vs.
models (62.32%), and its ACC, PRE, SEN, 107:20, p=0.616). Table 4 further gives the
and SPE were 82.55%, 89.58%, 84.75%, numbers of cases correctly or wrongly
and 77.56%. From Table 1, we can also see diagnosed by the PGBM-SMOTE and the
that the four single-layer PGBM models (i.e., PGBM. In benignancy, there were 10 cases
the shallow networks) were all better than correctly classified by the PGBM-SMOTE
the corresponding two-layer PGBM-RBM but misclassified by PGBM. Meanwhile,
model (i.e., the deep networks) in terms of there was only 1 case correctly classified by
ACC and YI. the PGBM but misclassified by PGBM-
C. CLASSIFICATION RESULTS OF SMOTE. In malignancy, there were 12 cases
STATISTICAL FEATURES correctly classified by PGBM but
The classification results of SF models misclassified by PGBM-SMOTE.
on the test sets are enumerated in Table 2. Meanwhile the number of cases correctly
The SF-LASSO-SMOTE achieved the best classified by PGBM-SMOTE but
performance among eight models in terms of misclassified by PGBM was 0. The
YI (32.98%). Its ACC, PRE, SEN, and SPE classification results of the PGBM on the
were 73.53%, 78.73%, 84.77%, and 48.21%. benign and malignant subsets were very
The SF-LASSO-SMOTE was inferior to the different (22:17 vs. 85:3, p<0.001). In
PGBM-SMOTE. From the experimental contrast, the results of PGBM-SMOTE were
results, we can also see that the SF-LASSO relatively balanced between benignancy and
models were generally better than the SF-T malignancy (31:7 vs. 73:15, p=0.851). These
models. In addition, the SEN and SPC of the detailed results on benign and malignant
SF models had a large difference even they subsets showed that compared with the
used SMOTE or EC methods for PGBM, the PGBM-SMOTE increases the
compensation of the data unbalance. numbers of correctly classified in
D. COMPARISON BETWEEN benignancy but decreases them in
NUMBERS OF CASES CORRECTLY OR malignancy. It indicated that the PGBM-
WRONGLY DIAGNOSED SMOTE elevated the specificity while
Table 3 gives the numbers of cases, in sacrificed in the sensitivity, and thus made
one experiment of five-fold cross validation, the sensitivity and specificity balanced for
correctly or wrongly diagnosed by three such an unbalanced dataset consisting of
typical models, namely the PGBM-SMOTE, much more malignant nodes than benign.
PGBM and SF-LASSO-SMOTE. There Ⅳ. DISCUSSION
were 78 cases correctly classified by all of The main contribution of this study is
the three models; the typical examples are proposing an end-to-end AI architecture for
learning and selecting image features from results indicated that the task-relevant
CEUS videos and for automatic component in the PGBM could capture the
classification of cervical lymph nodes. The intrinsic differences in a CEUS video
AI architecture has elevated classification between benign and malignant nodes and
ability for the unbalanced dataset compared thus improve the classification performance,
with the traditional SF methods. In future while the SF models were disturbed by the
clinical diagnosis, our AI architecture could task-irrelevant patterns in the video. In
be used as a valuable tool for distinguishing addition, the SF generation required prior
malignant and benign cervical lymph nodes. expert knowledge of cervical lymph nodes,
Our AI architecture was constructed which complicated the diagnosis process.
based on the PGBM. The switch units in the Our AI architecture needed little prior
PGBM allowed the model to estimate for knowledge, and hence it could be more
each TMI of CEUS where the task-relevant proper and convenient for future clinical
patterns occurred and to make only those diagnosis.
relevant visible units (pixels) to contribute to The single-layer PGBM models were
the final classification. The architecture better than the two-layer PGBM-RBM
ignored the task-irrelevant portion of the models (Table 2), especially when SMOTE
visible units, and thus it performed dynamic was applied, implying that the shallow
feature selection, namely choosing a networks outperformed the deep networks
variable subset of the raw features (i.e., deep learning) for cervical lymph node
depending on adaptive interpretation of an classification on CEUS. Since the medical
individual image. image datasets are usually in a small size,
From Tables 1 and 2, we can see that deep networks on such small datasets may
the SMOTE and EC were both helpful for tend to overfit the data. A recent study on
improving the classification performance of music genre classification also demonstrated
the unbalanced data, but the combination of that shallow networks were better suited for
the two techniques did not perform well. In small datasets than deep networks [26]. In
addition, for the PGBM models, the SMOTE contrast, our previous work on classification
performed better than EC, while for the SF of breast tumors with sonoelastography
models, the EC seemed to yield better showed superior performance of deep
results than SMOTE. This phenomenon networks to shallow networks [11].
might indicate that the SMOTE and EC Therefore, whether deep or shallow models
were suitable for different feature spaces. are appropriate for a small medical dataset
The optimal solution for classification of still needs to be further investigated and the
unbalanced data still needs to be answer may depend on specific applications.
investigated in the future. In the PGBM-based models, the
It can be seen from Tables 1 and 2 that classification results with different numbers
three PGBM-based models, namely PGBM, of hidden units were similar. For instance,
PGBM-SMOTE and PGBM-SMOTE-EC, when we varied the unit number from 10 to
achieved better results than the 500 in an interval of 10, the classification
corresponding SF-based models in terms of accuracy only changed by 4.0%. It showed
all five indices. For instance, compared with the robustness of the proposed architecture
the SF-LASSO-SMOTE, the PGBM- to this parameter. Finally we set the number
SMOTE increased the ACC, PRE, SEN, as 80 to achieve best performance, where the
SPC, and YI by 9.02%, 10.85%, 0.98%, numbers of relevant and irrelevant units
29.35% and 30.34% respectively. These were both 40.
There are several limitations and

directions for future work. First, multiple
modalities, including conventional
ultrasound, sonoelastography, magnetic
REFERENCES
resonance imaging, computed tomography
[1] C. P. Giacomini, R. B. Jeffrey, and L. K. Shin,
and positron emission tomography, are "Ultrasonographic evaluation of malignant
expected to be incorporated in the AI and normal cervical lymph nodes," Semin
architecture for better differentiation of Ultrasound CT MR, vol. 34, pp. 236-47, Jun
lymphadenopathy. Second, this study 2013.
focuses on binary classification of benign [2] R. Slaisova, K. Benda, J. Jarkovsky, H.
and malignant lymph nodes. There are many Petrasova, P. Szturz, and V. Valek, "Contrast-
sub-classes in benign and malignant nodes, enhanced ultrasonography compared to
gray-scale and power doppler in the
and the detailed classification of nodal sub-
diagnosis of peripheral lymphadenopathy,"
classes is helpful for optimal therapeutic Eur J Radiol, vol. 82, pp. 693-8, Apr 2013.
planning and deserves future investigation. [3] L. Poanta, O. Serban, I. Pascu, S. Pop, M.
Third, in addition to the differential Cosgarea, and D. Fodor, "The place of CEUS
diagnosis of cervical lymph nodes, the in distinguishing benign from malignant
proposed AI architecture is expected to be cervical lymph nodes: a prospective study,"
easily applied to the diagnosis of other Med Ultrason, vol. 16, pp. 7-14, Mar 2014.
diseases with little modification but [4] X. W. Cui, C. Jenssen, A. Saftoiu, A. Ignee,
and C. F. Dietrich, "New ultrasound
retraining on samples of the target diseases, techniques for lymph node evaluation,"
and then a new network will be constructed World J Gastroenterol, vol. 19, pp. 4850-60,
for the new application [11]. Fourth, the AI Aug 14 2013.
architecture for diagnosis will be upgraded [5] L. Rubaltelli, V. Beltrame, A. Tregnaghi, E.
for evaluation of treatment response in a Scagliori, A. C. Frigo, and R. Stramare,
future study, for example, for timely "Contrast-enhanced ultrasound for
identification of head and neck cancer characterizing lymph nodes with focal
cortical thickening in patients with
patients at risk of residual malignant lymph
cutaneous melanoma," AJR Am J
nodes after chemo-radiotherapy [25]. Roentgenol, vol. 196, pp. W8-12, Jan 2011.
In conclusion, we construct an AI [6] J. B. Liu, D. A. Merton, A. C. Berger, F.
architecture by using the PGBM and Forsberg, A. Witkiewicz, H. Zhao, et al.,
SMOTE for differentiating malignant from "Contrast-enhanced sonography for
benign cervical lymph nodes on CEUS. The detection of secondary lymph nodes in a
experimental results show that the shallow melanoma tumor animal model," J
PGBM-SMOTE model has better Ultrasound Med, vol. 33, pp. 939-47, Jun
2014.
classification performance than the SF
[7] T. Sato, T. Takemura, T. Ouchi, S. Mori, M.
models and other PGBM-based models Sakamoto, Y. Arai, et al., "Monitoring of
including the deep models. Our AI Blood Vessel Density Using Contrast-
architecture could be potentially valuable for Enhanced High Frequency Ultrasound May
the future diagnosis and clinical Facilitate Early Diagnosis of Lymph Node
management of lymphadenopathy. Metastasis," Journal of Cancer, vol. 8, pp.
704-715, 2017.
ACKNOWLEDGEMENTS [8] Q. Zhang, C. Yuan, W. Dai, L. Tang, J. Shi, Z.
Li, et al., "Evaluating pathologic response of
This work was funded by the National breast cancer to neoadjuvant chemotherapy
Science Foundation of China (Nos. with computer-extracted features from
61671281; 81627804; 61471231).
contrast-enhanced ultrasound videos," Phys [18] Q. Zhang, Y. Xiao, W. Dai, J. Suo, C. Wang, J.
Med, vol. 39, pp. 156-163, Jul 2017. Shi, et al., "Deep learning based
[9] Q. Huang, X. Huang, L. Liu, Y. Lin, X. Long, classification of breast tumors with shear-
and X. Li, "A case-oriented web-based wave elastography," Ultrasonics, vol. 72, pp.
training system for breast cancer diagnosis," 150-7, Dec 2016.
Computer Methods & Programs in [19] S. Han, H. K. Kang, J. Y. Jeong, M. H. Park, W.
Biomedicine, vol. 156, p. 73, 2018. Kim, W. C. Bang, et al., "A deep learning
[10] X. Feng, X. Guo, and Q. Huang, "Systematic framework for supporting the classification
Evaluation on Speckle Suppression Methods of breast lesions in ultrasound images,"
in Examination of Ultrasound Breast Physics in Medicine & Biology, 2017.
Images," Applied Sciences, vol. 7, p. 37, [20] K. Sohn, G. Zhou, C. Lee, and H. Lee,
2016. "Learning and Selecting Features Jointly
[11] Q. Zhang, Y. Xiao, W. Dai, J. Suo, C. Wang, J. with Point-wise Gated Boltzmann
Shi, et al., "Deep learning based Machines," in Proceedings of The 30th
classification of breast tumors with shear- International Conference on Machine
wave elastography," Ultrasonics, vol. 72, pp. Learning, 2013, pp. 217-225.
150-157, 2016. [21] Q. Zhang, C. Li, H. Han, W. Dai, J. Shi, Y.
[12] Q. Zhang, J. Suo, W. Chang, J. Shi, and M. Wang, et al., "Spatio-temporal
Chen, "Dual-modal computer-assisted Quantification of Carotid Plaque
evaluation of axillary lymph node metastasis Neovascularization on Contrast Enhanced
in breast cancer patients on both real-time Ultrasound: Correlation with Visual Grading
elastography and B-mode ultrasound," Eur J and Histopathology," Eur J Vasc Endovasc
Radiol, vol. 95, pp. 66-74, Oct 2017. Surg, vol. 50, pp. 289-96, Sep 2015.
[13] J. Zhang, Y. Wang, B. Yu, X. Shi, and Y. Zhang, [22] Q. Zhang, C. L. Li, H. Han, L. J. Yang, Y. Y.
"Application of Computer-Aided Diagnosis Wang, and W. P. Wang, "Computer-aided
to the Sonographic Evaluation of Cervical quantification of contrast agent spatial
Lymph Nodes," Ultrason Imaging, vol. 38, distribution within atherosclerotic plaque in
pp. 159-71, Mar 2016. contrast-enhanced ultrasound image
[14] J. Lam, M. Ying, S. Y. Cheung, K. H. Yeung, P. sequences," Biomedical Signal Processing
H. Yu, H. C. Cheng, et al., "A Comparison of and Control, vol. 13, pp. 50-61, Sep 2014.
the Diagnostic Accuracy and Reliability of [23] N. V. Chawla, K. W. Bowyer, L. O. Hall, and
Subjective Grading and Computer-Aided W. P. Kegelmeyer, "SMOTE: Synthetic
Assessment of Intranodal Vascularity in minority over-sampling technique," Journal
Differentiating Metastatic and Reactive of Artificial Intelligence Research, vol. 16,
Cervical Lymphadenopathy," Ultraschall pp. 321-357, 2002.
Med, vol. 37, pp. 63-7, Feb 2016. [24] M. A. H. Farquad and I. Bose,
[15] A. Chmielewski, P. Dufort, and A. M. "Preprocessing unbalanced data using
Scaranelo, "A Computerized System to support vector machine," Decision Support
Assess Axillary Lymph Node Malignancy Systems, vol. 53, pp. 226-233, 2012/04/01/
from Sonographic Images," Ultrasound Med 2012.
Biol, vol. 41, pp. 2690-9, Oct 2015. [25] E. Scalco, S. Marzi, G. Sanguineti, A. Vidiri,
[16] M. A. Maraci, R. Napolitano, A. and G. Rizzo, "Characterization of cervical
Papageorghiou, and J. A. Noble, "Object lymph-nodes using a multi-parametric and
classification in an ultrasound video using multi-modal approach for an early
LP-SIFT features," in Medical Computer prediction of tumor response to chemo-
Vision: Algorithms for Big Data, ed: radiotherapy," Physica Medica, vol. 32, pp.
Springer, 2014, pp. 71-81. 1672-1680, 2016/12/01/ 2016.
[17] D. Silver, A. Huang, C. J. Maddison, A. Guez, [26] A. Schindler, T. Lidy, and A. Rauber,
L. Sifre, G. van den Driessche, et al., "Comparing Shallow versus Deep Neural
"Mastering the game of Go with deep Network Architectures for Automatic Music
neural networks and tree search," Nature, Genre Classification," in Proceedings of the
vol. 529, pp. 484-489, 2016. 9th Forum Media Technology, St. Poelten,
Austria, 2016, pp. 17-21.
8
TABLE 1 Results of eight models based on the point-wise gated Boltzmann machine (PGBM) for
classification of cervical lymph nodes (unit: %)
ACC PRE SEN SPC YI

PGBM 82.68±0.49 83.00±1.51 94.32±1.82 56.41±5.47 50.73±3.70
PGBM-SMOTE 82.55±1.40 89.58±1.88 84.75±2.76 77.56±4.92 62.32±3.61
PGBM-EC 72.07±1.94 74.48±0.38 88.18±0.91 31.79±2.05 19.98±1.14
PGBM-SMOTE-EC 77.80±0.32 82.72±0.15 85.91±0.91 59.49±1.03 45.40±0.12
PGBM-RBM 82.36±0.63 80.59±0.85 98.18±1.36 46.67±1.02 44.85±0.34

PGBM-RBM-SMOTE 74.49±0.39 93.03±2.59 68.41±1.67 88.21±5.02 56.61±3.35
PGBM-RBM-EC 65.04±2.20 72.31±0.83 80.23±3.18 30.77±0.00 10.99±3.18
PGBM-RBM-SMOTE-EC 62.99±6.22 79.86±3.07 62.05±8.99 65.13±4.76 27.17±9.63
TABLE 2 Results of eight models based on statistical features for classification of cervical
lymph nodes (unit: %)
ACC PRE SEN SPC YI
SF-T 68.50±3.73 73.33±1.96 85.68±4.47 29.74±5.28 15.42±7.47
SF-T-SMOTE 65.51±2.51 73.02±2.56 80.00±4.10 32.82±10.56 12.82±8.39
SF-T-EC 69.13±2.01 78.97±1.99 75.68±2.65 54.36±5.71 30.04±5.33
SF-T-SMOTE-EC 66.77±2.08 72.35±1.62 84.32±1.33 27.18±5.52 11.50±5.92
SF-LASSO 73.23±2.17 77.62±2.10 86.36±2.87 43.59±7.25 29.95±6.41

SF-LASSO-SMOTE 73.53±0.81 78.73±1.31 83.77±1.54 48.21±4.70 31.98±3.51
SF-LASSO-EC 68.03±3.36 78.06±2.85 75.00±2.24 52.31±6.99 27.31±8.56
SF-LASSO-SMOTE-EC 71.97±2.52 78.27±1.93 82.50±3.01 48.21±5.71 30.71±6.05
TABLE 3 Numbers of cases correctly or wrongly diagnosed with three feature extraction and
classification methods
PGBM-SMOTE PGBM SF-LASSO-SMOTE Case No.
Correct Correct Correct 78
Correct Correct Wrong 16
Correct Wrong Correct 3
Correct Wrong Wrong 7
Wrong Correct Correct 6
Wrong Correct Wrong 7
Wrong Wrong Correct 6
Wrong Wrong Wrong 4
TABLE 4 The numbers of cases correctly and wrongly classified in benign and malignant lymph
nodes with the PGBM-SMOTE and the PGBM models.
PGBM-SMOTE PGBM Case No.

Correct Correct 21
Correct Wrong 10
Benign Wrong Correct 1
Wrong Wrong 7
Correct Correct 73
Correct Wrong 0
Malignant Wrong Correct 12
Wrong Wrong 3
10
Fig. 1 Schematic diagram of our artifically intelligent diagnosis architecture for lymphanopathy
on contrast-enhanced ultrasound (CEUS) images based on the point-wise gated Boltzmann
machine (PGBM).
Fig. 2 The AI architecture for diagnosis on an unbalanced dataset with the point-wise gated
Boltzmann machine (PGBM) and the synthetic minority over-sampling technique (SMOTE).
Fig. 3 Typtical samples of benign (B, top) and maligant (M, bottom) cervical lymph nodes that
were correctly classified with our PGBM-SMOTE model. The lymph nodes shown in (a) were
also correctly classified with the PGBM and the SF-LASSO-SMOTE, the nodes shown in (b)
were correctly classified with the PGBM but misclassified with SF-LASSO-SMOTE, and the
nodes shown in (c) were misclassified with both PGBM and SF-LASSO-SMOTE.
11

Artificial Intelligence Based Diagnosis For Cervical Lymph Node Malignancy Using The Point-Wise Gated Boltzmann Machine

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Artificial Intelligence Based Diagnosis For Cervical Lymph Node Malignancy Using The Point-Wise Gated Boltzmann Machine

Uploaded by

Copyright:

Available Formats

This article has been accepted for publication in a future issue of this journal, but has not been

Artificial intelligence based diagnosis for cervical lymph node

Qi Zhang1,2, Yue Liu2, Hong Han3, Jun Shi1, Wenping Wang3

*Authors for Correspondence:

Wenping Wang, MD, Professor

lymphadenopathy diagnosis. Therefore, it

There are several limitations and

classification of cervical lymph nodes (unit: %)

ACC PRE SEN SPC YI

PGBM-RBM 82.36±0.63 80.59±0.85 98.18±1.36 46.67±1.02 44.85±0.34

SF-LASSO 73.23±2.17 77.62±2.10 86.36±2.87 43.59±7.25 29.95±6.41

nodes with the PGBM-SMOTE and the PGBM models.

PGBM-SMOTE PGBM Case No.

on contrast-enhanced ultrasound (CEUS) images based on the point-wise gated Boltzmann

You might also like