Professional Documents
Culture Documents
A New Framework For Multi Scale CNN Based Malignancy Classification of Pulmonary Lung Nodules
A New Framework For Multi Scale CNN Based Malignancy Classification of Pulmonary Lung Nodules
https://doi.org/10.1007/s12652-022-04368-w
ORIGINAL RESEARCH
Received: 2 November 2021 / Accepted: 30 July 2022 / Published online: 18 September 2022
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2022
Abstract
Lung cancer has the highest mortality rate among all types of cancers. Early detection of lung cancer may improve survival
rates. The two categories of pulmonary lung nodules have high visual similarities. So, distinguishing them is a challenging
task for radiologists. The main purpose of this work is to use convolutional neural network to perform binary classification
of pulmonary nodules in CT images. This paper proposes a new multi-scale (64 × 64, 32 × 32 and 16 × 16) convolutional
neural network architecture for benign and malignant nodules classification. In addition, transfer learning method is used
to initialize the weights of multi-scale architecture. Experimental results on the dataset LIDC-IDRI demonstrate that the
proposed method achieved accuracy of 93.88%, sensitivity of 93.36% and specificity of 93.26% on nodule malignancy
classification. The proposed method also outperforms the other state-of-the-art methods explicitly designed for malignancy
classification of pulmonary lung nodules.
13
Vol.:(0123456789)
4676 Sakshiwala, M. P. Singh
13
A new framework for multi‑scale CNN‑based malignancy classification of pulmonary lung nodules 4677
13
4678 Sakshiwala, M. P. Singh
maxpooling is that it pools out that feature from the feature LeakyReLU turns all negative input values to a value near
map which is most present. to zero. The idea is to let some gradient flow during training.
LeakyReLU operation: In order to get the desired While all positive values remain same.
results, activation functions is used to make decision on Flatten and the dense layer: Flatten layer translates the
whether to activate or deactivate a neuron. It helps model to multi-dimensional tensor into a single long feature vector.
learn non-linear transformations. Dense layers are the fully connected layers. Unlike convolu-
tion layers where a neuron is connected to local area of input
neurons, in the dense layer each neuron has a path to every
other neuron to next layer. Figure 5 illustrates the flatten and
dense layer.
13
A new framework for multi‑scale CNN‑based malignancy classification of pulmonary lung nodules 4679
respectively. The LeakyReLU activation function is used Such as rotation by 90°, 180°and 270° angle. The nodule
after every convolutional layer. It helps the model to con- images were cropped and flipped. The nodules are aug-
verge faster and is easy to implement. mented by translating the nodules images by x axis and
Then, this is followed by a max-pooling layer. This layer y axis ± 2 pixels. Data augmentation techniques helps to
reduces the spatial resolution and helps in learning the more generalize the model. Adam optimizer with learning rate of
complex features from the input. Further two dense layers 0.001 is used.
are used. The flattened output feature maps are then passed All experiments in this paper were performed on Python
to a dense layer of 512 neurons. After this, LeakyReLU programming language using Keras library. The net-
activation function is used. The last fully connected layer works were trained on 32 GB RAM with a single NVIDIA
has two neurons which classifies the nodules as benign and QUADRO M2000, Windows 10 operating system. Results
malignant. The outputs from the last fully-connected layer are given below with discussions.
is normalized using softmax. The fully-connected layers
often lead to overfitting. The problem of overfitting can be 4.2 Performance metrics
resolved using dropout. A dropout of 0.5 is used to avoid
the overfitting. A sparse categorical crossentropy loss (L) Different evaluation parameters, including accuracy, preci-
function is used. It is defined as: sion, recall (sensitivity or True positive rate), F1-score, spec-
2
ificity, false positive rate are used to evaluate the proposed
∑ framework. Some parameters depend on the outcome of the
L=- log(p(i))t(i), (1)
x=1
confusion matrix true positive (TP), false negative (FN), true
negative (TN) and false positive (FP). TP is the number of
where p(i) is the predicted probability, t(i) is the true prob- correctly predicted malignant nodules. TN is the number
ability, ‘x’ is the class label and ‘i’ is the sample training of correctly predicted benign nodules. FP is the number of
data. Smaller value of loss function implies that the pre- actual benign nodules that are predicted malignant. FN is the
dicted probability is closer to the true probability. actual malignant nodules that are predicted benign.
The statistical parameters are expressed as follows:
3.2.2 Multi‑scale feature extraction and transfer learning TP + TN
Accuracy = (2)
TP + FP + TN + FN
The purpose of multi-scale feature extraction is to learn dif-
ferent patterns in images using different filter sizes. Differ- TP
ent filter sizes can focus on the context information of the Precision = (3)
TP + FP
candidate nodules in different scale images and the features
extracted from the different streams can also complement TP
each other. Therefore, more effective features can be learnt Recall = (4)
TP + FN
through different scale images. The method uses transfer
learning concept to transfer the weights learned by convo- TN
lutional layer of 64 × 64 pixels input based model to the Specificity = (5)
TN + FP
respective layers of 32 × 32 pixels input based model. The
layers are then trained. The weights after training of 32 ×
FP
32 pixels input based model is transferred to the 16 × 16 FPR = . (6)
TN + FP
pixels input based model. Figure 6b shows the described
framework.
4.3 Tuning of hyperparameters
4 Results and discussion A series of experiments is carried out to justify the choices:
the number of convolutional layers, maxpooling layers, fully
4.1 Experimental environment connected layers, size and number of filters, increasing or
decreasing the batch size, initializing convolutional layer
This work uses dataset having 2091 images. To increase the weights. Table 1 shows accuracies obtained by different sets
randomness data augmentation techniques are employed. of hyperparameters.
13
4680 Sakshiwala, M. P. Singh
(a) Conv1
13
A new framework for multi‑scale CNN‑based malignancy classification of pulmonary lung nodules 4681
(a) 64*64 input model (b) 32*32 input model (c) 16*16 input model
13
4682 Sakshiwala, M. P. Singh
5 Conclusion
Fig. 11 Confusion matrix (label ‘0’: benign, label ‘1’: malignant)
In this paper, a new framework of multi-scale CNN-
based architecture is proposed for malignancy classifi-
cation of lung nodules on chest CT scans. The proposed
method adopts a transfer learning based multi-scale fea-
ture extraction method. In this method first a size of 64
× 64 pixels input images is fed as an input to the network
and trained. Then the learned weights are transferred to
the 32 × 32 pixels input based model of same architec-
ture and is then trained. The weights learned by convolu-
tional layers of 32 × 32 pixels input size based model is
initialized to the respective layers of 16 × 16 pixels input
based models and is then trained. Finally, the model
is evaluated on LIDC-IDRI dataset. The experimental
results showed that the proposed model is efficient for
Fig. 12 ROC curve malignancy classification of lung nodules in chest CT
scans.
In this work only 2D image of a single nodule is used
for cancer prediction. While in reality, the oncologists
Table 4 Performance of the proposed model
consider 3D view of CT scans. The current study does not
consider scores of attributes related to malignancy. For
Category Precision Recall F1-score the future work, in order to avoid human cropped nodule
Benign 0.96 0.93 0.94 regions, deep learning based malignancy classification
Malignant 0.90 0.93 0.92 method along with the method of nodule detection may
be added.
Xie et al. (2018b) Fuse-TSD LIDC-IDRI (1972) 89.53 84.19 92.02 96.65
Ren et al. (2020) Manifold regularized CNN LIDC-IDRI (1226) 90 81 95 -
Xie et al. (2019) Semi-supervised adversarial classification LIDC-IDRI (2557) 92.53 84.94 96.28 95.81
Proposed method Multi-scale CNN + transfer learning LIDC-IDRI (2091) 93.88 93.36 93.26 93.31
13
A new framework for multi‑scale CNN‑based malignancy classification of pulmonary lung nodules 4683
Data availibility statement The datasets used in the current study are deep learning era: novel supervised and unsupervised learning
available in the below https://wiki.cancerimagingarchive.net/display/ approaches. IEEE Trans Med Imaging 38(8):1777–1787
Public/LIDC-IDRI. Venugopal VK, Vaidhya K, Murugavel M, Chunduru A, Mahajan V,
Vaidya S, Mahra D, Rangasai A, Mahajan H (2020) Unboxing ai-
radiological insights into a deep neural network for lung nodule
Declarations characterization. Acad Radiol 27(1):88–95
Xie Y, Xia Y, Zhang J, Song Y, Feng D, Fulham M, Cai W (2018)
Conflict of interest The authors have no conflict of interest. Knowledge-based collaborative deep learning for benign-malig-
nant lung nodule classification on chest ct. IEEE Trans Med Imag-
ing 38(4):991–1004
Xie Y, Zhang J, Xia Y, Fulham M, Zhang Y (2018) Fusing texture,
shape and deep model-learned information at decision level for
References automated classification of lung nodules on chest ct. Inf Fusion
42(102–110):7. https://doi.org/10.1016/j.inffus.2017.10.005
Abid MMN, Zia T, Ghafoor M, Windridge D (2021) Multi-view con- Yan K, Tian L, Xueqiao P, Elvis SG, Qiao L, Junyi Z (2020) Unsuper-
volutional recurrent neural networks for lung cancer nodule iden- vised multi-discriminator generative adversarial network for lung
tification. Neurocomputing 453:299–311 nodule malignancy classification. IEEE Access 8:77725–77734.
Armato SG III, McLennan G, McNitt-Gray MF, Meyer CR, Yankel- https://doi.org/10.1109/ACCESS.2020.2987961
evitz D, Aberle DR, Henschke CI, Hoffman EA, Kazerooni EA, Yu G, Xiaoqi L, Yang L, Zhang B, Dahua Yu, Zhao Y, Gao L, Liang
MacMahon H et al (2004) Lung image database consortium: W, Zhou T (2018) Automatic lung nodule detection using a 3d
developing a resource for the medical imaging research commu- deep convolutional neural network combined with a multi-scale
nity. Radiology 232(3):739–748 prediction strategy in chest cts. Comput Biol Med 103:220–231
Attique Khan M, Sadia R, Asifa K, Imran SM, Nazeer M, Hussain SJ, Yutong X, Jianpeng Z, Yong X (2019) Semi-supervised adversarial
Yu-Dong Z, Chandra SS (2020) Lungs cancer classification from model for benign-malignant lung nodule classification on chest
ct images: An integrated design of contrast based classical features ct. Med Image Anal 57(10):237–248. https://doi.org/10.1016/j.
fusion and selection. Pattern Recogn Lett 129:77–85 media.2019.07.004
Baihua Z, Shouliang Q, Patrice M, Chen L, Fan Y, Yu-Dong Y, Wei Zhao W, Yang J, Sun Y, Li C, Wu W, Jin L, Yang Z, Ni B, Gao P,
Q (2019) Ensemble learners of multiple deep cnns for pulmonary Wang P, Hua Y, Li M (2018) 3d deep learning from ct scans pre-
nodules classification using ct images. IEEE Access 7(8):110358– dicts tumor invasiveness of subcentimeter pulmonary adenocar-
110371. https://doi.org/10.1109/access.2019.2933670 cinomas. Cancer Res 78(12):6881–6889. https://doi.org/10.1158/
Chen Y, Wang Y, Fei H, Wang D (2020) A lung dense deep convolution 0008-5472.CAN-18-0696
neural network for robust lung parenchyma segmentation. IEEE Zhao X, Liu L, Qi S, Teng Y, Li J, Qian W (2018) Agile convolu-
Access 8:93527–93547 tional neural network for pulmonary nodule classification using ct
Pritam Mukherjee M, Zhou EL, Schicht A, Balagurunathan Y, Napel images. Int J Comput Assist Radiol Surg 13(4):585–595
S, Gillies R, Wong S, Thieme A, Leung A et al (2020) A shallow
convolutional neural network predicts prognosis of lung cancer Publisher's Note Springer Nature remains neutral with regard to
patients in multi-institutional computed tomography image data- jurisdictional claims in published maps and institutional affiliations.
sets. Nat Mach Intell 2(5):274–282
Ren Y, Tsai M-Y, Chen L, Wang J, Li S, Liu Y, Jia X, Shen C (2020) A Springer Nature or its licensor holds exclusive rights to this article under
manifold learning regularization approach to enhance 3d ct image- a publishing agreement with the author(s) or other rightsholder(s);
based lung nodule classification. Int J Comput Assist Radiol Surg author self-archiving of the accepted manuscript version of this article
15(2):287–295 is solely governed by the terms of such publishing agreement and
Sarfaraz H, Pujan K, Bolan Candice W, Wallace Michael B, Ulas applicable law.
B (2019) Lung and pancreatic tumor characterization in the
13