Detection and Classification of Pulmonary Nodules

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/333771086
Detection and Classiﬁcation of Pulmonary Nodules Using Convolutional

Neural Networks: A Survey
Article in IEEE Access · June 2019

DOI: 10.1109/ACCESS.2019.2920980
CITATIONS READS
13 888
6 authors, including:
Patrice Monkam Shouliang Qi

Tsinghua University Northeastern University (Shenyang, China)
12 PUBLICATIONS 70 CITATIONS 114 PUBLICATIONS 704 CITATIONS
SEE PROFILE SEE PROFILE
He Ma Wei Qian
Northeastern University (Shenyang, China) University of Texas at El Paso
31 PUBLICATIONS 327 CITATIONS 225 PUBLICATIONS 3,398 CITATIONS
SEE PROFILE SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Image-Guided Lung Biopsy Surgery View project
All content following this page was uploaded by Patrice Monkam on 18 June 2019.
The user has requested enhancement of the downloaded file.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2019.2920980, IEEE Access
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2017.DOI
Detection and classification of

pulmonary nodules using convolutional
neural networks: A survey
PATRICE MONKAM1 , SHOULIANG QI1,2,3 , HE MA1 , WEIMING GAO4 , YUDONG YAO5 ,
(FELLOW, IEEE), AND WEI QIAN.6
1
Sino-Dutch Biomedical and Information Engineering School, Northeastern University, Shenyang, 110169, China
2
Key Laboratory of Medical Image Computing of Northeastern University (Ministry of Education), Shenyang, 110169, China
3
Neusoft Research of Intelligent Healthcare Technology, Co. Ltd., Shenyang, 110169, China
4
Shenzhen Qianhai AnyCheck Information Technologies, Co. Ltd., Shenzhen 518057, China
5
5Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ, 07030, USA
6
College of Engineering, University of Texas at El Paso, El Paso,79902, USA
Corresponding author: Shouliang Qi (e-mail: qisl@bmie.neu.edu.cn).
This study was supported by the National Natural Science Foundation of China under Grant (Grant number: 81671773, 61672146), the
Fundamental Research Funds for the Central Universities (N172008008), Open Program of Neusoft Research of Intelligent Healthcare
Technology, Co. Ltd. (No. NRIHTOP1803), and "Pearl River Talent Plan" of Guangdong Province (No. 2017ZT07X261).
ABSTRACT CT screening has been proven to be effective for diagnosing lung cancer at its early
manifestation in the form of pulmonary nodules, thus decreasing the mortality. However, the exponential
increase of image data makes their accurate assessment a very challenging task given that the number
of radiologists is limited and they have been overworked. Recently, numerous methods, especially ones
based on deep learning with convolutional neural network (CNN), have been developed to automatically
detect and classify pulmonary nodules in medical images. In this paper, we present a comprehensive
analysis of these methods and their performances. First, we briefly introduce the fundamental knowledge
of CNN as well as the reasons of their suitability to medical images analysis. Then, a brief description of
various medical images datasets as well as the environmental setup essential for facilitating lung nodule
investigations with CNNs is presented. Furthermore, comprehensive overviews of recent progresses in
pulmonary nodule analysis using CNNs are provided. Finally, existing challenges and promising directions
for further improving the application of CNN to medical images analysis and pulmonary nodule assessment
in particular are discussed. It is shown that CNNs have greatly transformed the early diagnosis and
management of lung cancer. We believe that this review will provide all the medical research communities
with necessary knowledge to master the concept of CNN so as to utilize it for improving the overall human
healthcare system.
INDEX TERMS Lung cancer, deep learning, convolutional neural networks, computed tomography (CT)
images, pulmonary nodules, image classification
I. INTRODUCTION stage I lung cancer had survived 10 years after the diagnosis
N 2018, lung cancer is ranked first for both incidence and [4]. Hence, early diagnosis of lung cancer is vital to improve
I mortality among all cancer types worldwide; accounting
for 11.6% and 18.4% of all cancer cases and all cancer
its therapeutic decisions and prognosis.
One of the effective approaches for reducing the mortality
deaths, respectively [1]. It presents very poor general progno- rate of lung cancer is screening as it helps the medical
sis due to the fact that the disease tends not to be diagnosed experts to diagnose the disease before it presents any signs or
until it is at a critical stage [2], [3]. For instance, the generally related symptoms. Specifically, the National Lung Screening
low (18%) five-year survival rate can be lifted to 56% if it is Trial (NLST) had demonstrated that there was a reduction
diagnosed at an early development; the study by Henschke of over 20% of the mortality rate in patients who under-
et al. demonstrated that 88% of 412 patients diagnosed with went low-dose computed tomography (LDCT) screening [5].
VOLUME 4, 2016 1
2169-3536 (c) 2019 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS
Lung screening enables the accurate depiction of pulmonary

nodules which constitute the critical indicators of the early
development of lung cancer.
In CT images, pulmonary nodules can be referred to as
round or oval tissue masses of lung with diameter less than
30 millimeters. They present large variations in sizes, density,
location and surrounding. As shown in Fig. 1(a) and 1(b),
pulmonary nodules usually have a diameter large than 3 mm
and those with diameter smaller than 3 mm are called micro-
nodules. Non-nodules including bronchi walls and blood
vessels present similar appearances as nodules and may cause
false positive during the process of detection (Fig. 1(c)). Ac-
cording to the density, pulmonary nodules can be categorized
into solid nodules, ground glass nodules and partly solid
nodules (Fig. 1(d, e, f)) [6], [7]. In term of location, nodules
can be isolated, juxta-pleural or juxta-vascular (Fig. 1(d, g, FIGURE 1. Illustration of the great diversity of pulmonary nodules in CT
h)). In many cases, the pleural effusion makes the delineation images. (a) Isolated nodule within a single CT slice (> 3.0 mm); (b)
Micro-nodule within a single CT slice (<3.0 mm); (c) Non-nodule within a
of nodule contour challenging (Fig. 1(i)). In summary, these single CT slice (>3.0 mm); (d) Isolated solid nodule; (e) Ground glass nodule;
diversities of pulmonary nodules substantially increase the (f)Partly solid nodule; (g) Juxta-pleural nodule; (h) Juxta-vascular nodules;(i)
Nodule immersed in pleural effusion.
difficulty of achieving accurate detection and diagnosis.
Furthermore, the speedy development of CT screening of
lung cancer has led to an exponential increase of images
•Demonstrate that CNNs stand out of other deep learning
data to be examined by doctors, which significantly increases
models and they have amazingly contributed to the early
their workload resulting in erroneous diagnosis causing un-
diagnosis and treatment of lung cancer.
necessary anxiety for the patients or decreasing the chances
• Provide researchers with available medical image
to be cured. Bechtold et al. demonstrated that the error rate
datasets as well as environmental (hardware and soft-
susceptible to occur in 20 CT images analysis conducted by a
ware) requirement needed for lung cancer studies using
radiologist per day ranges from 7 − 15% [8]. Therefore, with
deep learning.
the aim of reducing the radiologists workload and improving
• Point out the crucial obstacles to successfully applying
the early detection of lung cancer, numerous methods and
CNNs to medical image analyses as well as prospective
systems have been proposed for automatic medical images
solutions for overcoming them.
analysis.
Deep learning, especially convolutional neural network This investigation is structured in the following way. First,
(CNN), has been remarkably utilized in various medical fundamental knowledge of CNNs is described as well as the
imaging tasks due to the outstanding performances. For the advantages of their usage in pulmonary nodules analysis.
applications on analysis of lung, breast, prostate and brain Second, an overview of various medical images datasets as
cancers, several excellent reviews had been published re- well as the environmental setup necessary for conducting
cently [9]–[11]. On the other hand, there exist other survey lung nodules studies is presented. Third, a comprehensive
studies mainly dedicated to CNNs and their applications analysis of the recent studies on the detection and classifica-
to radiological tasks [12]–[14]. Although these papers have tion of pulmonary nodules is conducted. Finally, the existing
comprehensively described the applications of artificial in- challenges of deep learning based approaches for lung nod-
telligence (with special regard to deep learning) to medical ules analysis are pointed out followed by the identification of
imaging tasks, no specific review has been devoted to the some prospective directions.
detection and classification of pulmonary nodules by CNNs.
In this paper, we aim at illustrating some recent advanced II. CNN AND ITS DISTINCT CHARACTERISTICS
deep learning techniques applied to the analysis of pul- A. CNNS OVERVIEW
monary nodules. Specifically, a summary of various CNNs CNN algorithms, also known as deep learning models, are re-
based approaches developed in 2018 for the detection and ferred to as a class of machine learning techniques; which are
classification of pulmonary nodules is presented. We did not all subfields of artificial neural networks. CNN had known
consider studies whose methodology are based on other deep its great advancement in 2012 through the study conducted
learning models such as recurrent neural networks (RNNs) by Krizhevsky et al. [15]. They developed a CNN model
and auto-encoders (AEs) due to their higher computational namely AlexNet which won the ImageNet Large Scale Visual
complexity and lower recognition performance as compared Recognition Competition (ILSVRC) reducing the classifica-
with CNNs. Moreover, we limited the works reported here tion error record with a margin greater than 10%. Thereafter,
to those of 2018 to avoid overlapping with existing reviews. new CNN models with more layers have been put forward
With this review, we intend to: including VGG-Net [16], ResNet [17], [18], GoogLeNet
2 VOLUME 4, 2016
[19], SENet [20], etc. and 2.5D image classification frameworks achieving great
Generally, convolutional neural networks consist of a stack detection results [24]–[26].
of various convolutional layers which are learned accord- Segmentation CNN models refer to that group of CNNs
ingly with the aim of automatically extracting useful infor- destined to examine both medical and natural images with the
mation from the input data without involving any prepro- purpose of dividing them into multiple constituents according
cessing or features engineering procedures. Crucial compo- to the user’s need for further analysis. These networks can
nents of CNNs include convolutional layer, pooling layer, be fed with images larger than that on which they were
fully-connected and Softmax layers (Figure 2(a)). The main trained outputting a likelihood map for every single pixel
process for achieving the features discovery in CNNs is of the images. They are also known as fully convolutional
convolution; which is performed through the convolutional neural networks (FCNNs); and U-Net and its variants [22],
layers. It consists of applying a “dot product”mathematical [23] constitute some improved versions of this category as
operation of matrices of weights across the entire content of demonstrated by their remarkable performances (Fig. 2(d)).
every sample of the input data (images, videos, etc.) resulting
in the generation of the feature maps. Another essential C. DISTINCT CHARACTERISTICS OF CNNS
operation in CNNs is pooling. It is often applied after the
convolution process. Applying pooling helps in reducing the In recent years, the crucial role played by the automatic
dimensionality of the output which will consequently result and accurate detection of pulmonary nodules in the early
in the preservation of more important essential features. In diagnosis and precise management of lung cancer has led to
addition, it is worth mentioning that the size of the receptive the development of numerous methods. These methods can
field is to be chosen carefully as the recognition performance be categorized into conventional methods and deep learning
of a designed CNN architecture can be significantly influ- based methods. Conventional methods are those that are
enced by the amount of relevant information included in the mainly based on traditional image processing techniques and
network input data. For instance, if the surrounding environ- machine learning classifiers [27]–[32] and whose pipelines
ments of the objects of interest contain much unnecessary often include many sub-processes. Whereas, deep learning
details such as noises and artifacts or if they do not enclose methods specifically CNNs based methods are those whose
enough contextual information of the target objects, the CNN implementation does not involve any features engineering
model would be susceptible to yield very poor detection steps and they are referred to as end-to-end solutions. The
performance. Furthermore, according to the dimensions of workflow diagrams of these methods are illustrated in Fig. 3.
the convolutional filters, convolutional neural networks can Several differences between these two kinds of methods
be classified into 2D-CNN whose kernels are of dimensions are worth noting. First, the pipeline of conventional methods
of two and 3D-CNN whose kernels are of dimensions of includes many sub-processes; which increases the computa-
three. tional time as well as the error rate. Whereas, CNNs based
methods are straightforward systems not involving some
computationally costly processes such segmentation and fea-
B. CNN ARCHITECTURES COMMONLY USED IN
tures engineering; which allows them to yield more accurate
MEDICAL IMAGING
detection results. Another significant difference is that CNNs
Based on the structures of the CNN models utilized in based approaches can make full use of the essential infor-
existing clinical decision-support systems, CNNs can be cat- mation contained in three dimensional images data such as
egorized into classical classification structures, multi-stream computed tomography (CT) images and Positron Emission
structures and segmentation structures [21]. Tomography (PET) images. Doing so would definitely result
Classical CNN models are those networks built up by in better diagnostic results. Finally, the great successes of
stacking multiple layers in a linear way and which are gener- CNNs based methods are often subjective to the amount
ally aimed to perform classification tasks (Fig. 2(b)). AlexNet of samples that comprises the dataset; which limits their
[15] and VGG [16] are the most frequently used networks of application to the analysis of small dataset although they may
this category. achieve better results than conventional methods.
With regard to the fact that the amount of contextual in-
formation is of great significance for detecting abnormalities III. DATASETS AND EXPERIMENTAL SETUP
from images and given that the fusion of multiple sources of
Given that the implementation of CNNs requires a huge
image information may improve the detection performance,
amount of parameters to be estimated, some hardware and
multi-stream CNN models also known as multiple path net-
software requirements need to be specified. Some commonly
works have been proposed (Fig. 2(c)). This concept of multi-
used datasets as well as the environmental setup for auto-
ple paths originated from the idea of extracting the essential
matic detection and classification of pulmonary nodules are
features contained in adjacent images of the volumetric medi-
given below.
cal data without increasing the amount of network parameters
and computational cost. Investigating multiple path networks
has resulted in the development of multi-scale image analysis A. DATASETS OF LUNG CANCER CT IMAGES
VOLUME 4, 2016 3
FIGURE 2. CNNs architectures commonly used in medical imaging.(a) One CNN with 2 convolutional layers, 2 pooling layers,
and a fully connected layer; (b) Node graph of 1D representation of a classical CNN structure; (c) Node graph of 1D representation of a multi-stream
CNN structure; (d) Node graph of 1D representation of structure of a CNN for segmentation (U-NET with only one down-sampling stage).
FIGURE 3. The workflow diagrams of the two categories of methods for pulmonary nodule analysis.
(a) Conventional methods; (b) CNNs based methods.
4 VOLUME 4, 2016
1) LIDC/IDRI dataset 5) ELCAP dataset

LIDC/IDRI stands for Lung Image Database Consortium and ELCAP stands for Early Lung Cancer Action Program. It is
Image Database Resource Initiative [33]. It is a database a dataset made up of 50 low-dose CT cases [37]. The images
of thoracic CT scans owing its successful creation to three slice thickness is equal to 1.25 mm and the diameters of the
research organizations including the NCI (National Cancer majority of the identified nodules range from 2 to 5 mm. It is
Institute), the FNIH (Foundation for the National Institutes worthy to be mentioned that all the annotated lesions of this
of Health) and the FDA (Food and Drug Administration). dataset are nodules and no non-nodules were marked by the
LIDC/IDRI dataset consists of a total of 1, 018 cases. In medical experts.
effect, there are only 1, 010 different CT scans due to the fact
that there are eight cases that were inadvertently reproduced B. CNN SOFTWARE PLATFORMS AND HARDWARE
while gathering the CT scans. All the gathered image data are EQUIPMENT
stored in DICOM format and with uniform size of 512 × 512. The major programming language utilized for the implemen-
The images thickness ranges from 0.5 to 5 mm where the tation of deep learning models is python. Besides, there is
most recurrent image thicknesses include 1 mm, 1.25 mm, also Matlab which is a high level programming language
and 2.5 mm. It is worth noting that more than 50% of the and numerical analysis environment conceived for perform-
recent studies on lung cancer diagnosis have made use of the ing engineering and scientific works such as computational
LIDC/IDRI dataset. finance, image and signal processing, matrix calculations,
Every case of the LIDC/IDRI dataset consists of hundreds data analysis, system simulation, etc. [38]. Moreover, there
of images plus an XML file which contains the details of the exist many platforms based on python language which have
identified lung lesions. Making use of electronic calipers, the been proposed to facilitate the CNNs models implementa-
diameter of each of the detected lung lesions was assessed re- tion. These platforms include Keras, Caffe, Chainer, Then-
sulting in their classification into three main groups including sorFlow, Torch, etc. Caffe provides interfaces for both python
nodules (3-30 mm of diameter), non-nodules (diameter > or and C++; it was introduced by graduate students from the
equal to 3 mm) and micro-nodules (diameter < 3 mm). University of California Berkeley. Torch is a relatively easy
and efficient computing framework that provides its potential
2) LUNA 16 dataset users with an excellent C interface via LuaJIT.
This dataset is a subset of the publicly accessible LIDC/IDRI CNNs constitute a class of deep learning techniques;
dataset. It consists of a total of 888 thoracic CT scans which makes their implementation dependent on great
collected in the basis that the lesions contained in every case amount of experimental data as well as a huge number of
have been marked by at least three of the four medical experts parameters to be estimated. Thus, the great technological
having taken part in the annotation processes [34]. Consid- advancements have led to the creation of sophisticated com-
ering this inclusion agreement, only nodules with 3 mm < puters equipped with graphical processing unit (GPU) and
diameter were considered as positive samples meanwhile all compute unified device architecture (CUDA) supported by
the remaining lesions were referred to as negative samples. NVIDIA.
IV. PULMONARY NODULE DETECTION AND FALSE

3) NLST dataset
POSITIVE REDUCTION
It is a dataset collected in 2009 by the National Lung
Pulmonary nodule detection and false positive reduction
Screening Trial (NLST); which was a project that aimed
constitute the two most essential mechanisms for the early
at comparing the lung cancer diagnosis accuracy through
diagnosis and precise management of lung cancer. Thus,
low-dose CT screening with that of the chest radiography
extensive research works are being conducted with the aim
screening. The whole screening project was performed in
of improving the early and automatic detection of pulmonary
33 American medical institutions and comprised a total of
nodules. These research works have led to the development of
53, 454 participants. These patients were aged between 55
various methods and systems which are summarized in Table
and 74 years and with a smoking history of at least 30 packs
1 and whose details are presented below.
per year. This dataset is made up of both low-dose CT images
and chest radiographs. For more details, kindly refer to [35].
A. ADVANCED OFF-THE-SHELF CNNS
There exist many well-known CNN models for their re-
4) Kaggle Data Science Bowl (KDSB) dataset
markable performances in various recognition tasks; which
This dataset was generated from 2101 patients and every are also known as off-the-shelf CNNs. However, designing
patient’s file includes between 100 and 400 images [36]. All a great CNN model depends on the problem at hand and
the images have been annotated in the following manner: requires a certain level of computer science expertise; which
label 0 and label 1 were assigned to patient without cancer constitutes a very challenging task for researchers due to their
and patient with cancer, respectively. diversified background. Therefore, exploiting the advantages
of reinforcement learning, region proposal network (RPN),
VOLUME 4, 2016 5
TABLE 1. Recent studies on pulmonary nodule detection and false positive reduction using CNNs
Years Authors Models Datasets

Key Points
2018 Ali et al. 3D CNN LUNA16
First application of reinforcement learning (RL) to pulmonary
[39] nodules detection.
2018 Qin et al. 3D U-Net and LUNA16 3D U-Net based Regional Proposal Network (RPN) is employed
[40] 3D DenseNet for nodule candidates’generation and 3D DenseNet is designed
for false positives reduction.
2018 Tang et al. 3D Faster R- Tianchi AI Two-phase framework utilizing 3D U-Net-inspired Faster R-CNN
[41] CNN and 3D competition and 3D DCNN for false positives reduction.
DCNN
2018 Winkels et 3D G-CNN NLST and False positive reduction using 3D G-CNN (group convolutional
al. [42] LIDC/IDRI neural network).
2018 Silva et al. 2D CNN based LIDC/IDRI PSO (particle swarm optimization) algorithm is employed to
[43] PSO replace the manual search of network hyper-parameters.
2018 Monkam 2D CNN LIDC/IDRI Automated detection of pulmonary nodule is expanded to micro-
et al. [44] nodules (diameter < 3 mm). Effects of CNN depth and receptive
field size on the detection performance are clarified.
2018 Liu et al. 2D CNN LIDC/IDRI and 2D CNNs are trained with dataset including images with multiple
[45] ELCAP scales generated from multiple views to accurately identify the
nodule types.
2018 Jia et al. 2D U-Net, Tianchi AI An ensemble of RF and XGBoost is used to classify nodules
[46] GLCM, RF and competition features resulted from applying K-means clustering, median fil-
XGBoost tering, morphological operations and U-Net.
2018 Pezeshk et 3D FCN and 3D LUNA16 3D FCN is used for generating the nodule candidates and averag-
al. [47] CNN ing is performed to fuse the outputs of four identical 3D CNN for
reducing the false positive rate.
2018 Jung et al. Ensemble of 3D LIDC/IDRI 3D DCNN with shortcut connections and 3D DCNN with dense
[49] CNN connections are designed, checkpoint ensemble approach is pro-
posed to boost the networks performance.
2018 Monkam Ensemble of 3D LIDC/IDRI ELM is employed to integrate five 3D CNN with five different
et al. [50] CNN input sizes for distinguishing between micro-nodules and non-
nodules.
Faster Region based CNN, etc.; advanced CNN models have U-Net-inspired 3D Faster R-CNN: Tang et al. proposed
been designed for lung nodule detection. a two-phase framework for pulmonary nodules identification
and false positive reduction [41]. First, a U-Net-like 3D
Reinforcement learning: Ali et al. exploited CNN to Faster RCNN model is employed to generate the nodule
build up a reinforcement learning framework for detecting samples. Then, a 3D deep convolutional neural network is
pulmonary nodules in CT images [39]. This system con- built to identify the true nodule candidates. The developed
stitutes a pioneer work as regarding reinforcement learning system was trained based on hard negative mining approach
applied to medical image analysis tasks. Moreover, it pre- yielding a CPM score of 0.815.
sented excellent detection performance yielding a sensitivity,
specificity, accuracy, PPV and NPV above 99%, respectively. 3D G-CNN: Generally, successful implementation of
CNNs necessitates great amount of experimental data which
Region proposal network: Employing 3D U-Net, 3D are usually unavailable in medical area. Thus, Winkels et al.
DenseNet and RPN (region proposal network), Qin et al. proposed a system based on 3D G-CNNs which achieved
developed a system for automatically detecting pulmonary the same performance with that of a normal CNN model
nodule in CT images [40]. The overall system training pro- requiring ten times more data when applied to the reduction
cess was performed utilizing Multi-task residual learning of false positive in automated pulmonary identification [42].
and online hard negative example mining approaches. The The implementation of the proposed framework includes two
proposed methodology included two main modules which main processes which are filter transformation and spatial
are: nodule candidates’generation based on 3D U-Net and de- convolution. It yielded a CPM score of 0.856 on 3000 nodule
crease of false positives utilizing a 3D DenseNet. It achieved samples acquired from the NLST and LIDC/IDRI datasets;
a sensitivity of 96.7% and CPM score of 0.834 on LUNA16 which is almost equal to the performance (CPM = 0.869) of a
dataset. normal CNN model trained with ten times more data (30000
6 VOLUME 4, 2016
nodule samples). aimed at demonstrating the effectiveness of convolutional

neural networks to accurately identify the nodules type based
B. CNNS WITH ADVANCED IMPLEMENTATION on their internal characteristics. They employed linear inter-
TECHNIQUES polation, icosahedron divided sphere algorithm and thresh-
The detection performance of a designed CNN model is olding approach for raw CT images preprocessing, volume
strongly dependent on its implementation procedure as well of interest detection and generation and selection of views at
as the size of the images constituting the dataset. To this different scales, respectively. Then, the suitable views chosen
purpose, few solutions for improving the implementation of at all scales were used to train the 2D CNN model with
CNNs have been proposed. the aim of classifying the nodule candidates into G (ground
Particle swarm optimization: With the aim of facilitating glass optical), W (well- circumscribed), V (vascularized),
the parameters optimization which is regarded as a crucial J (juxta-pleural) and P (pleural-tail) and N (non-nodule),
step in the implementation of CNN models, Silva et al. respectively. This system achieved an overall accurate of
proposed a framework in which particle swarm optimization 92.1% and 90.3% on the nodules acquired from the LIDC-
(PSO) approach is utilized to automatically select the best IDRI dataset and ELCAP dataset, respectively.
network hyper-parameters [43]. The proposed framework 2D U-Net + (K-means, median filtering, Gabor trans-
was implemented on 12, 157 pulmonary nodule samples form, GLCM, XGBoost and RF): In an investigation by Jia
including 3, 415 nodules and 8, 742 non-nodules generated et al., K-means algorithm, median filtering, some morpho-
from the LIDC-IDRI dataset. It achieved an accuracy, sensi- logical operations, Gabor wavelet transform and gray level
tivity, specificity and AUC of 97.62%, 92.20%, 98.64% and co-occurrence matrix (GLCM) were integrated with convo-
0.955, respectively. lutional neural networks and XGBoost and Random Forest
CNN receptive field and identification of micro-nodule: (RF) classifiers to improve the automated identification of
Although great number of systems have been developed lung nodules in CT images [46]. They put forward a system
to accurately and precisely identify pulmonary nodules in whose pipeline comprises lung parenchyma segmentation,
medical images, very few of them can differentiate micro- nodule candidates’generation, features engineering and clas-
nodules from other lung lesions such non-nodules; which sification of the nodule candidates. First, K-means algorithm,
increases the amount of false positives. To this purpose, median filtering and morphological operations were applied
Monkam et al. extended lung nodules analysis to automatic on the raw CT scans to obtain the whole lung parenchyma.
identification of micro-nodules [44]. Through examination of Whereas, a 2D U-Net model was employed to locate the nod-
the whole LIDC-IDRI dataset, they built up an experimental ule candidates in the delineated lung parenchyma. Then, fea-
dataset consisting of 13, 179 micro-nodules and 21, 315 non- tures engineering was conducted based on various techniques
nodules. Considering the small sizes of both micro-nodules including Gabor wavelet transform, gray level co-occurrence
and non-nodules, they investigated the performance of three matrix (GLCM), etc. Finally, an ensemble learners consisting
CNN models with different depths as well as their recog- of XGBoost and Random Forest (RF) classifiers was trained
nition ability when fed with samples of different sizes. The to obtain the final prediction results. This system achieved
three CNN structures included one, two and four convo- an AUC of 0.93 on the 800 cases of the Tianchi medical AI
lutional layers, respectively. Whereas, the different image competition dataset.
patches sizes consisted of 16 × 16, 32 × 32 and 64 × 64, re-
spectively. The best classification performances were yielded D. ENSEMBLE LEARNERS OF MULTIPLE CNNS
by CNN model with two convolutional layers trained with Ensembling refers to an approach for integrating multiple
image patches of 32 × 32. learners into a single framework with the aim of increasing
the overall performance of the system. Generally, lung nod-
C. CNN+ ules present large variations in shapes, sizes, internal char-
Herein the term of CNN+ refers to those systems whose acteristics, etc. Moreover, different convolutional neural net-
framework includes both CNNs and some conventional im- work learners may yield different prediction results. Hence,
age processing and objects detection techniques. Combin- to the end of improving the accuracy of clinical decision-
ing CNNs with other techniques such as linear interpo- support systems, various CNNs based ensemble systems have
lation, icosahedron divided sphere algorithm, thresholding, been proposed. Their implementation process is shown in
K-means, median filtering, morphological operations, Ga- Fig. 4(b).
bor wavelet transform and gray level co-occurrence matrix To alleviate the poor performance of three-dimensional
(GLCM), new systems for classifying lung nodules have been CNNs in medical imaging data analysis due to limited
developed. A schematic representation of these systems is amount of dataset and data classes’imbalance, Pezeshk et al.
displayed in Fig. 4(a). proposed an automatic two-phase 3D CNNs based system for
2D CNN + (linear interpolation, icosahedron based pulmonary nodules detection named DeepMed [47]. First,
normalization and thresholding): Liu et al. developed a they employed a 3D FCN (fully convolutional network) to
novel framework that includes 2D CNNs with multiple views identify and generate the nodule samples. Then, averaging
and whose pipeline comprises four main phases [45].They was utilized to fuse the outputs of many CNN models
VOLUME 4, 2016 7
yielding the final detection results. To overcome the data A. CNNS WITH ADVANCED IMPLEMENTATION
imbalance and insufficiency of the training samples, they TECHNIQUES
put forward a new strategy for increasing the amount of NoduleX (CNN features + QIF features): In a study by
the training samples. This augmentation method; which can Causey et al., a hybrid features based system was developed
expand every sample 28 times, is derived from one of their [51]. The developed framework namely NoduleX, makes use
previous studies [48]. The proposed system outperformed of the features discovered by the CNN model and those of
various state-of-the-art methods implemented on the LUNA the radiological QIF (quantitative image features) to accu-
dataset. rately differentiate malignant nodules patterns from benign
Employing 3D deep CNN models with shortcut connec- nodules patterns. Its capability of predicting malignancy in
tions and with dense connections, respectively; a new en- pulmonary nodules is quite equal to that of expert physicians
semble system namely checkpoint ensemble method was (AUC = 0.99).
proposed by Jung et al. [49]. They investigated the perfor- DenseBTNet (center-crop operation): In another inves-
mance of ensemble models based on 3Dshortcut connection tigation, Liu et al. incorporated a center-crop process into
DCNNs and 3D dense connection DCNNs separately. On the conventional DenseNet to build up an improved network
the other hand, they also explored the dependence of the with a structure in form of binary tree namely DenseBTNet
performance of these methods on the receptive field sizes. It [52]. The introduction of the center-crop process helps in
was found that single deep CNN model with larger receptive discovering multi-scale features from the nodule candidates;
field size (64 × 64 × 64) yielded better classification results. which will consequently contribute to overcoming the influ-
Whereas, shortcut connection DCNN model outperformed ence of the great variations of their sizes, shapes and internal
dense connection DCNN regardless of the network input size. characteristics on the classification performance. The exper-
Furthermore, performing the checkpoint ensemble strategy imental results demonstrated that this new dense network
resulted in the overall system performance improvement model significantly outperforms the conventional DenseNet
yielding a CPM score of 0.910. as well as numerous existing systems validated on the same
Even though, micro-nodules are seen as earliest develop- dataset.
ment of pulmonary nodules, less attention has been paid to 3D dilated convolution: Zhang et al. attempted to differ-
improving their detection; which considerably contributes to entiate benign pulmonary nodules from malignant pulmonary
the high false positives rate in automated pulmonary nodules nodules employing a new CNN structure in which the pool-
detection systems. Moreover, the full use of information ing operation is replaced by a 3D dilated convolution opera-
contained in adjacent images of volumetric imaging data tion [53]. In effect, they made the following contributions to
such as MRI and CT may result in more accurate and precise the area of applications of CNN to medical images analysis.
detection performances. Thus, Monkam et al. proposed an First, instead of including pooling layers in the network
ensemble learning of multiple 3D CNNs for identifying pul- architecture, they included a 3D dilated convolution layer;
monary micro-nodules in CT images [50]. They designed 3D which keeps the image quality unchanged and helps retain
CNN models trained with image patches of sizes 20 × 20 × 3, much smaller image information. Second, they considered
16 × 16 × 3, 12 × 12 × 3, 8 × 8 × 3 and 4 × 4 × 3, respectively. different receptive field sizes for the dilated convolutions re-
They explored different approaches to integrate multiple deep sulting in the discovery of multi-scale features; which would
learning learners including, majority voting, ELM (extreme consequently increase the discrimination ability of the sys-
learning machine), logical operator AND, averaging and au- tem leading to more accurate and precise diagnostic results
toencoder (AE). The experimental results indicated that ELM regardless of the diversity of the nodules’characteristics.
is the most suitable approach for fusing multiple learners that PN-SAMP (multiple window widths and window cen-
aim to identify micro-nodules in volumetric CT images. ters): Wu et al. proposed a 3D CNN based framework that
takes advantage of multiple window widths and window cen-
V. CLASSIFICATION BETWEEN CANCEROUS AND ters and the multi-task learning concept to accurately identify
NON-CANCEROUS PULMONARY NODULES the nodules areas, to extract semantic information from the
Significant successes have been achieved in the automated detected nodules and to predict the nodules malignancy [54].
differentiation of pulmonary nodules from other lung lesions This method could be of significant help to the physicians for
such as non-nodules. However, it is still quite challenging not only differentiating malignant from benign nodules, but
to determine the status of the identified nodules. Moreover, also for assessing the malignancy level of the tumors; which
not all lung nodules turn out to be a cancer. Thus, given the will greatly improve the treatment planning of lung cancer.
great number of images to be analyzed and the criticalness Dual-path 3D CNN and multi-output network: Dey
of the task, many systems have been proposed to help the et al. conducted a comparative study aiming to find the
physicians in the process of distinguishing between benign most appropriate dual-path 3D CNN model for distinguish-
and malignant pulmonary nodules. The recently developed ing between malignant and benign pulmonary nodules [55].
systems for classifying pulmonary nodules into cancerous They investigated the discrimination capability of four 3D
and non-cancerous are briefly illustrated in Table 2 and deep learning models including an ordinary 3D CNN, a 3D
introduced in details in the following subsections. DenseNet, multi-output 3D DenseNet and an augmented 3D
8 VOLUME 4, 2016
FIGURE 4. Different categories of pulmonary nodule analysis systems.

(a) CNN+ system; (b) Ensemble learners system; (c) Hybrid system; (d) Transfer learning system.
DenseNet with multi-outputs. The performances of these of this system on the KDSB (Kaggle Data Science Bowl)
networks were assessed on nodule candidates of the publicly 2017 dataset yielded an accuracy, a recall and specificity of
accessible LIDC/IDRI dataset and the obtained results indi- 87.8%, 87.4% and 89.1%, respectively.
cated that the multi-output 3D DenseNet (MoDenseNet) is
the most suitable network model achieving an accuracy of B. CNN+
90.40%.
2D CNN + (Otsu algorithm, phylogenetic diversity index):
Local and global contextual features with triple-path Filho et al. built a framework employing image processing,
CNN: Considering both the local and global contextual pattern recognition and deep learning techniques [57]. In
features is of great significance for accurately assessing the their methodology, with the aim of extracting qualitative in-
malignancy of lung nodules. To this purpose, Soriet al. pro- formation from the internal regions of the identified nodules,
posed a multi-path CNN model that can simultaneously learn they utilized Otsu algorithm to segment the nodules into three
local and global contextual features from nodule candidates sections. Then, they designed a new indices of phylogenetic
to classify them into cancerous and non-cancerous [56]. To diversity based on topology to explore and select the essen-
the end of identifying the suspicious nodules, they designed a tial features of pulmonary nodules. Finally, a CNN model
network derived from the commonly used segmentation CNN was built and fed with the extracted nodules characteristics
model U-Net. Next, the collected nodules were fed into a to produce the final classification results. This system was
deep CNN model comprising three different paths wherein validated on a dataset consisting of 394 malignant candidates
the sizes of the receptive field are different. Later on, the and 1, 011 benign candidates. It achieved an accuracy value
essential features discovered by each path were concatenated of, a sensitivity value of and a specificity value of 92.63%,
to build up a much diversified vector features. The validation 90.7% and 93.47%, respectively.
VOLUME 4, 2016 9
TABLE 2. Recent studies on the prediction of nodule malignancy using CNNs
Years Authors Models Datasets Key Points

2018 Zhao et al. 2D LeNet + 2D LIDC/IDRI Layer settings of LeNet are combined with the parameter settings
[61] AlexNet of AlexNet to build up a CNN model for malignancy prediction.
2018 Filho et al. Otsu algorithm, LIDC/IDRI Otsu algorithm is employed for extracting and analyzing the
[57] 2D CNN ROI. New indexes of phylogenetic diversity based on topology
is proposed for features engineering and selection. 2D CNN is
fed with the obtained features.
2018 Causey et 3D CNN LIDC/IDRI and 3D CNN model is trained and the output features are collected.
al. [51] LUNA16 Then, these features are combined with the radiological QIF and
fed into another 3D CNN for malignancy classification.
2018 Xie et al. MV ResNet-50 LIDC/IDRI Nine KBC models based on ResNet-50are built corresponding to
[60] and KBC the nine views of every 3D nodule. An adaptive weighting scheme
is employed to fuse the outputs of the KBC models.
2018 Gupta et SRCNN, CNN, LUNA16 SRCNN is used for nodules detection, features are engineered
al. [58] SVM, SVR from the obtained nodules using a 2D CNN model. Then, the
collected features are fed into a SVM and a SVR for malignancy
prediction and malignancy score assessment, respectively.
2018 Liu et al. 3D CNN LIDC/IDRI Introduction of center-crop process into 3D DenseNet resulting in
[52] a DenseNet with the form of binary tree (DenseBTNet).
2018 Zhang et 3D CNN LIDC/IDRI 3D dilated convolution operation is put forward in replacement of
al. [53] the pooling operation.
2018 Wu et al. 3D CNN LIDC/IDRI Window widths, window centers and multi-task learning concept
[54] are exploited to build up a 3D CNN model.
2018 Dey et al. 3D CNN and LIDC/IDRI Performance comparison between basic 3D DCNN and some
[55] 3D DenseNet variants of 3D DenseNet.
2018 Zhu et al. 3D Faster R- LIDC/IDRI 3D Faster R-CNN is designed for nodules detection. GBM and
[59] CNN, 3D CNN, 3D dual path CNN are applied for malignancy prediction.
GBM
2018 Sori et al. 2D U-Net and KDSB 2017 2D U-Net is used to detect nodule candidates considering both
[56] 2D CNN local and global contextual features. Three paths 2D CNN with
three different input sizes is designed for the classification task.
2018 Nobrega et 2D CNNs, LIDC/IDRI Eleven popularly used 2D DCNN models are employed as fea-
al. [62] SVM, MLP, tures extractors. SVM, MLP, KNN, RF, Naive Bayes are trained
KNN, RF, separately with the collected features.
Naive Bayes
2018 Paula et al. 2D CNN, VGG- NLST Three ensemble CNN models based on averaging, majority voting
[63] s, RF and median probability, are built to classify the learned features.
2018 Nishio et VGG-16 LIDC/IDRI and Performance comparison between DCNN model, features engi-
al. [64] private dataset neering based model and transfer learning model.
2D CNN + (single image super-resolution, SVM, SVR): port Vector Regression) model. This system presented quite
Gupta et al. investigated the malignancy prediction as well as satisfactory classification capability, yielding an accuracy of
the level of malignancy of the tumor [58]. With the aim of 85.7% on nodule candidates acquired from the LUNA16
overcoming the influence of low resolution on the diagnostic dataset.
performance, they made use of the Single Image Super- DeepLung (dual-path 3D DCNN + (3D Faster R-CNN,
Resolution approach to preprocess the imaging data. Then, Gradient Boosting Machine)): To the end of automatically
the enhanced images were fed into a CNN model to identify detecting and classifying pulmonary nodules in volumetric
the positive nodule samples. Subsequently, essential features CT images, Zhu et al. employed 3D CNN to build up a dual
were extracted from the detected real nodules utilizing the path networks based system [59].First, a 3D Faster R-CNN
previously designed CNN model. Lastly, an SVM classifier model was designed to locate the nodule candidates. Then,
was designed to train the generated features vector producing they exploited a deep 3D dual path network, gradient boost-
the diagnostic results. Furthermore, the malignancy level was ing machine (GBM) algorithm and some detected nodules
predicted by training the features vector with a SVR (Sup- characteristics such as size and shape to achieve the final
10 VOLUME 4, 2016
classification results. The proposed system’s performances transfer learning has been adopted as a reliable alternative for
were assessed through tenfold cross-validation experiments analyzing pulmonary nodules in medical images using deep
conducted on the 888 CT scans of the LUNA16 dataset. It CNN models. The concept of transfer learning is illustrated
achieved an accuracy of 90.44% demonstrating the great ad- in Fig. 4(d).
vantages of considering deep 3-D dual path network features. Nobrega et al. employed transfer learning to learn quan-
titative features from nodule candidates of the LIDC/IDRI
C. ENSEMBLE LEARNERS OF MULTIPLE CNNS dataset; which were later utilized to classify them into
Xie et al. built a MV-KBC (multi-view knowledge-based col- benign and malignant [62]. The scenario of their frame-
laborative) framework for distinguishing malignant from be- work is as follows. First, they utilized eleven deep CNN
nign pulmonary nodules [60]. The proposed system achieves models previously trained on the ILSVRC (ImageNet
the malignancy prediction in the following manner. First, Large Scale Visual Recognition Challenge). These mod-
every nodule candidate was split into nine views. Then, each els include Xception,VGG16,Inception-ResNet-V2, VGG19,
view was utilized to build up a KBC model in which three DenseNet201, MobileNet, InceptionV3, DenseNet169,
pre-trained ResNet-50 with three different receptive field ResNet50, NASNetLarge and NASNetMobile. Then, each
sizes were fine-tuned. Finally, the outputs of the nine KBC set of obtained features was fed into five classifiers including
models were combined using an adaptive weighting based MLP (Multilayer Perceptron), Naive Bayes, (SVM) Support
strategy to generate the final prediction results. This study Vector Machine, KNN (K-Nearest Neighbor) and RF (Ran-
constitutes a pioneer work as it has expanded the concept dom Forest), to produce the diagnostic results. The experi-
of domain knowledge to convolutional neural networks with mental results demonstrated that the combination of ResNet-
application to pulmonary nodules malignancy prediction. 50 for features extraction and SVM-RBF for classification
is the most suitable approach for predicting malignancy in
D. HYBRID SYSTEMS nodule candidates based on pre-trained deep CNN models.
This group of clinical decision-support systems are those This combination yielded an ACC, an AUC, an F-score, a
whose frameworks comprise networks resulted from com- TPR and a PPV of 88.41%, 93.19%, 78.83%, 85.38% and
bining two or more layers or/and parameters of different 73.48%, respectively.
other network models. It could also be referred to as the A methodology employing radiomics features, transfer
class of systems whose methodology contains different types learning generated features and ensemble learning for nod-
of networks. Fig. 4(c) presents a brief overview of hybrid ules malignancy prediction was proposed by Paul et al.
model. [63]. First, they utilized the pre-trained VGG-s for features
Agile CNN (LeNet+AlexNet): A hybrid convolutional extraction. Second, they built up three different CNN models
neural network model resulted from combining the layers of including an ordinary CNN with two convolutional layers, a
LeNet and the network parameters of AlexNet for differenti- CNN-LSTM model and a CNN with a cascaded architecture.
ating benign pulmonary nodules from malignant pulmonary Then, 219 radiomics features were acquired and the useful
nodules in CT images was developed by Zhao et al [61].The ones were chosen using image segmentation and Random
implementation of the proposed model on only 743 nodule Forest algorithms, respectively. Finally, they designed three
candidates yielded an accuracy and an AUC of 82.2% and ensemble models using as fusion strategy averaging, majority
0.877, respectively. The obtained results demonstrated that voting and median probability, respectively. Utilizing averag-
this hybrid CNN model could be considered as a promising ing as the fusion strategy allows the third ensemble model to
alternative for applying deep learning to the analysis of small achieve the best AUC value of 0.96. Whereas, the highest
amount of medical image data. They also investigated the accuracy value of 89.45% was yielded by the same model
influence of some network hyper-parameters such as kernel utilizing majority voting as the fusion approach.
size, weight initialization, learning rate and training batch With the purpose of classifying nodule candidates into
size on the CNN classification performance. benign, primary cancer and metastatic cancer, Nishio et al.
investigated the following problems [64]: the recognition
E. TRANSFER LEARNING BASED SYSTEMS performance of deep CNN model as compared with that
Transfer learning refers to an approach wherein the stored of features engineering based methods, the importance of
knowledge resulted from learning a model while solving utilizing transfer learning and the effect of the receptive
a specific task can be applied to solving a different task field size on the deep CNN classification performance. They
of a related problem. Deep convolutional neural networks implemented the VGG-16 model and made some modifica-
have achieved impressive recognition performances in nat- tions to it to fit their datasets. Later on, they designed a
ural images analysis. However, such great performances are transfer learning model based on the pre-trained VGG-16
highly dependent on great amount of datasets. On the other model. Both deep CNN models (without and with transfer
hand, medical imaging data are very limited; which makes learning) were trained with image patches of sizes 56 × 56,
the application of deep CNN models to their analysis quite 112×112, and 224×224, respectively. The deep CNN model
challenging. Thus, with the aim of mitigating the poor perfor- trained from scratch yielded a validation accuracy of 60.2%,
mance of deep CNN due to small amount of medical images, 62.4% and 58.9% for image patches of sizes 56, 112, and
VOLUME 4, 2016 11
224, respectively. Whereas, the pre-trained model achieved methodologies are also the prior choices. The category of
a validation accuracy of 60.7% when fed with images of “CNNs with advanced implementation techniques”achieved
56 × 56; 64.7% when fed with images of 112 × 112 and better performance than the other categories. For instance,
68.0% when fed with images of 224 × 224. the sensitivity of four of the six studies referred in our survey
is higher than 95%. Such performances might originate from
VI. TRENDS, CHALLENGES, PROSPECTIVE their ability of learning greater amount and more diversified
DIRECTIONS AND SUGGESTIONS semantic and radiomics features from the nodule candidates
From the comprehensive analyses of existing clinical employing newly proposed convolution processes such as
decision-support systems presented above, it is observed that center-crop operation, 3D dilated convolution, etc.
recently, the applications of convolutional neural networks
to the early diagnosis of lung cancer have known astonish- B. EXISTING CHALLENGES
ing progresses. Specifically, great advancements have been 1) Insufficiency of well-labeled medical datasets with great
achieved in the automated nodules identification, false pos- number of cases
itive reduction, classification of nodules into cancerous and It is noted that most of the great successes of deep learning
non-cancerous, etc. However, there is still room for improve- techniques in general and convolutional neural networks in
ments due to existing challenges which are pointed out below particular, have been achieved on huge amounts of data.
as well as some proposals for future investigations. However, gathering such datasets in medical imaging is still
very challenging due to many factors such as the tediousness
A. METHODOLOGICAL TRENDS
of the annotation tasks for the physicians, privacy and ethical
Convolutional neural network has been the leading method requirements, etc. Thus, the lack of datasets with large num-
for the detection and classification of pulmonary nodules ber of samples constitutes a crucial obstacle to the application
using deep learning. Using “lung nodules”and “lung nod- of deep learning to the analysis of medical data [12].
ules and deep learning”as keywords, the statistics during
the period of 2015 to 2018 retrieved from the IEEEXplore
2) Generalizability capability issues
and PubMed databases show that the studies using CNNs
Numerous deep learning based models have been proposed
account for 93.8% and 60.5% of the overall publications,
to solve various diagnostic tasks in medicine achieving out-
respectively. In addition, it is found that the number of studies
standing performances. However, in most of the cases, a
using CNNs has remarkably increased by 153.3% and 50%
proposed model which performed very well on a specific task
from 2017 to 2018 for the two abovementioned datasets,
may not be valid for other tasks no matter how slight their
respectively. The study by Litjens et al. also supported this
difference maybe [65].
trend [21].
As illustrated in Sections IV and V, CNN methodolo-
gies for pulmonary nodule analysis can be categorized into 3) Poor interpretability and explication of the detection
six groups including (a) Advanced off-the-shelf CNNs; (b) results
CNNs with advanced implementation techniques; (c) CNNs Nowadays, priority and emphasis are given to performance
+; (d) Ensemble learners of CNNs; (e) Hybrid CNNs and improvements over understandability and interpretability.
(f) Transfer learning (Fig. 5). The methods of (a), (b), (c) However, better interpretability could be helpful in antici-
and (d) are utilized for the pulmonary nodule detection pating on the failures of computer-aided detection systems;
and false positive reduction. In our survey, the ratio of the which would result in the reduction of false positive or late
number of publications in 2018 using these four different diagnosis. Moreover, increasing the interpretability of deep
categories of methods is 4:2:2:3 (see Table 1); indicating that learning based detection systems will be of great significance
all these categories have been developed evenly and none of not only in figuring out how the predictions are generated, but
them is at dominant position. The category of “Advanced also in clearly understanding how the outcome of a specific
off-the-shelf CNNs”outperforms the other three categories patient is obtained [66]. This may lead to the definition of
and three of the four studies of this category achieved an more accurate and reliable clinical decision guidelines as
accuracy and sensitivity greater than 95%. This could be due well as to the formulation of new hypotheses.
to the fact that “Advanced off-the-shelf CNNs”make use of
more sophisticated and newer algorithms such reinforcement 4) Shortage of accurate clinical decision tools
learning, Faster R-CNN, group convolution, etc. Despite the great successes of deep learning in lung can-
Five categories of CNN methodologies consisting of (b), cer screening, predicting the nature of identified pulmonary
(c), (d), (e) and (f) have been employed for pulmonary nodules still constitutes a critical issue as very few of these
nodule classification. The ratio of the number of publications nodules turn out to be cancerous. For instance, in the National
in 2018 in the current survey using these five categories Lung Screening Trial (NLST), less than 5% of the detected
of methods is 6:3:1:1:3. It can be seen that more studies nodules were malignant [5]. Moreover, the study by Patz
prefer to design CNN frameworks with advanced implemen- et al. had demonstrated that over 18% of this very limited
tation techniques; meanwhile CNN+ and Transfer learning number of cancerous lesions seemed to be indolent [67].
12 VOLUME 4, 2016
FIGURE 5. Summary of the different categories of CNN methodologies for pulmonary nodule analysis.
Thus, the research community lacks robust and efficient generalizability capability of CNN models. Therefore, we
approaches and tools for determining whether identified pul- recommend the investigation of the influence of the scanners
monary nodules are malignant or benign and aggressive or settings such as reconstruction techniques and parameters
indolent. as several studies have demonstrated their impact on the
radiomics features [73]–[76]. In addition, the generalizability
C. PROSPECTIVE DIRECTIONS issues could also be alleviated through developing some
methods that can be validated on images of different types i.e.
To tackle the abovementioned challenges, the following ideas
Computer Tomography (CT), Magnetic Resonance Imaging
are put forward for future investigations.
(MRI), etc. Furthermore, to adapt other approaches such as
First, one of the crucial obstacles to the application of knowledge transfer can significantly mitigate the problems
CNNs to the analysis of pulmonary nodules is the lack of radiological heterogeneity of medical scanners and lung
of datasets with large number of samples. This issue can nodules diversity [77]; which will potentially enhance the
be mitigated by developing some new approaches for gen- generalizability of clinical-decision support systems.
erating synthetic medical images. For instance, Generative
Adversary Networks (GANs) could be further investigated Third, the training processes of CNN models often face the
as they have been proven to achieve remarkable results in problems of overfitting, convergence and high computational
generating both natural and medical synthetic images [68]– time. Thus, there are urgent needs of designing novel clinical
[71]. Moreover, the development of more deep learning based decision-support systems whose frameworks include two or
methods not involving great amount of data such as in the more of the newly developed CNN architectures to address
study by Zhao et al. [61] could also be considered as a these problems; which will consequently contribute to further
promising alternative to the insufficiency of labeled medical improving the early diagnosis of lung cancer. For instance,
data. Furthermore, an incredible improvement of the poor recently, Li et al. proposed a hybrid network namely H-
performance due to insufficient data may be achieved by DenseUNet resulted from combining DenseNet and U-Net
designing new CNNs frameworks in which the conventional [78]. Other CNN models with such architecture though Unet-
translational convolutions are substituted by group convolu- Vnet-Fast-R-CNN, Mask-R-CNN are worthy to be investi-
tions as proved in the studies by Winkels et al. [4], [72]. gated.
Second, given that different medical scanners operate Fourth, new approaches for both radiomics and semantic
under different settings and that there exist various imag- features analysis in screening data should be developed to
ing modalities, different datasets are often made up of im- reduce over-diagnosis and improve the early identification of
ages presenting heterogeneous characteristics. These het- lung cancer. For instance, one could integrate CNNs with
erogeneities constitute one of the major factors of the low other machine learning and image processing algorithms to
VOLUME 4, 2016 13
include the information of the lung parenchyma; which will and density has been proven to be associated with an increase
significantly enhance the extraction of the radiomics and in malignancy risk [85].
semantic features from the lesions as done in the study by
Xu et al. [79]. In addition, the crucial utility of semantic VII. CONCLUSION
and radiomics features in the lung cancer prognosis has been The precise and accurate detection and examination of pul-
demonstrated by many studies [80]–[82]. monary nodules is one of the best approaches to decrease the
Fifth, expert-in-the-loop or doctor-in-the-loop systems lung cancer-related deaths. To this purpose, numerous CNNs
should be developed to facilitate both the understanding and based methods and systems have been proposed for analyzing
explanation of how the predictions yielded by deep learning pulmonary nodules in medical images. A comprehensive
systems are generated. These systems can be referred to analysis of these methods has been provided in this paper.
as those resulted from getting both the clinical experts and We have focused mainly on the remarkable works published
artificial intelligence experts involved in their building and in 2018 and whose frameworks include convolutional neural
functioning processes [66]. There already exist some exam- networks. A brief overview of convolutional neural networks
ples of such system [83], [84]. Doing so would definitely as well as their advantages and rationale for applying them
increase the interpretability of the predictions; which will to pulmonary nodules analysis were reviewed. Then, we
lead to the identification of biomarkers and help improve both analyzed the recently developed CNNs based systems for
the diagnosis and prognosis. nodules classification with main focus on their methodolo-
gies, the datasets used for validation as well as their detection
D. POTENTIAL SUGGESTIONS results.
To improve the early diagnosis and management of lung It was observed that applying CNNs to the detection of
cancer, which will potentially reduce its critical death rate; pulmonary nodules as well as their classification into ma-
researchers can develop new methodologies for pulmonary lignant and benign have yielded remarkable performances;
nodule analysis based and centered on CNNs. The potential which makes them a promising approach to improving the
strategies can consist of: early diagnosis, treatment and management of lung cancer.
• Designing CNNs with completely new architectures However, there is still room for improvement as these exist-
specifically for nodule analysis. ing methods present some challenges whose overcoming is
• Adapting the proposed “advanced off-the-shelf CNNs of urgent need. We believe that the detailed description of
”to other tasks. CNNs, their advantages and limitations in medical imaging
• Adding the advanced techniques into the implementa- as well as prospective directions as presented in this paper,
tions of CNNs. will be of great help not only in the diagnosis and treatment
• Integrating CNNs with other machine learning algo- of severe diseases such as lung cancer, but also in various
rithms, hand-crafted features, imaging processing tech- areas of radiology.
niques.
• Generating ensemble learners of multiple CNNs. CONFLICT OF INTEREST
• Building hybrid CNNs inheriting great characteristics The authors declare no conflicts of interest.
from their parents.
• Using transfer learning from various levels (instance, REFERENCES
feature, and knowledge). [1] F. Bray, J. Ferlay, I. Soerjomataram, R. L. Siegel, L. A. Torre, A. Jemal,
"Global Cancer Statistics 2018: GLOBOCAN Estimates of Incidence and
From the comprehensive analyses conducted in this sur- Mortality Worldwide for 36 Cancers in 185 Countries," CA: Cancer J
vey, we suggest that to adapt the “advanced off-the-shelf Clin.vol. 68, pp. 394–424, 2018.
[2] Medicinet [Online]. Available:
CNNs”might yield competitive performance for pulmonary https://www.medicinenet.com/lung cancer/article.htm, Accessed on 23-
nodule detection, while “CNNs with advanced implementa- Feb-2019.
tion techniques ”are more suitable for nodule classification. [3] N. Howlader, A. M. Noone, M. Krapcho,et al. "SEER Cancer Statistics
Review, 1975âĂŞ2014, based on November 2016 SEER data submission,
To initiate multi-center clinical trials is suggested for it is posted to the SEER web site,", April 2017. Bethesda, MD: National Cancer
urgently required to verify the value of CNNs based auto- Institute,2017.
matic classification for improving the outcomes of lung can- [4] International Early Lung Cancer Action Program Investigators, C. I.
Henschke, D. F. Yankelevitz, D. M. Libby, M. W. Pasmantier, J. P. Smith,
cer patients. Beyond the detection and classification, CNNs O. S. Miettinen, "Survival of patients with stage I lung cancer detected on
analysis of pulmonary nodule will inevitably be expanded CT screening," N Engl J Medvol.355, pp.1763–1771, 2006.
to the patient stratification or subtyping, the prediction of [5] National Lung Screening Trial Research Team, D. R. Aberle, A. M.
outcomes (e.g., treatment response, survival), the analysis Adams, et al., "Reduced lung-cancer mortality with low-dose computed
tomographic screening," N Engl J Med, vol.365, pp.395–409.
of the multi-omics correlations (e.g., radiomics, pathomics, [6] M. D. Martin, J. P. Kanne, L. S. Broderick, Ella A. Kazerooni, C. A. Meyer,
genomics), and the understanding of the mechanisms un- "Lung-RADS: Pushing the Limits, " Radiographics: a Review Publication
derlying lung cancer. In addition, more attention should be of the Radiological Society of North America, vol. 37, no. 7, pp. 1975–
1993., 20 Oct 2017.
paid to micro-nodules given that apart from the expansion in [7] S. Diciotti, G. Picozzi, M. Falchini, M. Mascalchi, N. Villari, and G.
diameter of pulmonary nodules, their growth in attenuation Valli, "3-D Segmentation Algorithm of Small Lung Nodules in Spiral CT
14 VOLUME 4, 2016
Images," IEEE transactions on information technology in biomedicine., [31] A. Bhattacharjee, S. Majumder, "Automated Computer-Aided Lung Can-
vol. 12, no.1, pp. 7–19, 2008. cer Detection System, " In: Advances in Communication, Devices and
[8] R. E. Bechtold, M. Y. M. Chen, D. J. Ott, Ronald J. Zagoria, E. S. Networking. Lecture Notes in Electrical Engineering,vol. 537. Springer,
Scharling, N. T.Wolfman, and D. J. Vining, "Interpretation of Abdominal Singapore, pp. 425–433, 2019.
CT: Analysis of Errors and Their Causes," Journal of Computer Assisted [32] W. Marcin, P. Dawid, C. Giacomo et al., "Small lung nodules detection
Tomography, vol. 21,no.5, pp. 681–685, 1997. based on local variance analysis and probabilistic neural network, " Com-
[9] W. L. Bi; A. Hosny; M. B. Schabath; M. L. Giger; N. J. Birkbak et al., puter Methods and Programs in Biomedicine, vol. 161, pp. 173–180, 2018.
"Artificial Intelligence in Cancer Imaging: Clinical Challenges and Appli- .
cations, " CA Cancer J Clin, vol. 0, pp. 1–31, doi: 10.3322/caac.21552., [33] S. G. Armato III, G. McLennan, L. Bidaut et al., "The lung image database
2019. consortium (LIDC) and image database resource initiative (IDRI): a com-
[10] A. Hosny, C. Parmar, J. Quackenbush, L. H. Schwartz, and H. J. W. L. pleted reference database of lung nodules on CT scans, " Medical Physics,
Aerts, "Artificial intelligence in radiology," Nature Reviews Cancer,vol. vol. 38, no. 2, pp. 915–931, 2011.
18, no.8, pp. 500–510, doi:10.1038/s41568-018-0016-5, 2018. [34] A. A. A. Setio, A. Traverso, T. de Bel, et al., "Validation, comparison, and
[11] D. Shen, G. Wu and H. Suk, "Deep Learning in Medical Image Analysis, combination of algorithms for automatic detection of pulmonary nodules
" Annu Rev Biomed Eng., vol. 19, pp. 221–248, doi: 10.1146/annurev- in computed tomography images: the luna16 challenge, " Medical Image
bioeng-071516-044442, 2017. Analysis, vol.42, pp. 1–13, 2017.
[12] Y. Yang, X. Feng, W. Chi, Z. Li, W. Duan, H. Liu, W. Liang, et al., [35] The National Lung Screening Trial Research Team, "Reduced Lung-
"Deep learning aided decision support for pulmonary nodules diagnosing: Cancer Mortality with Low-Dose Computed Tomographic Screening, "
a review, " J Thorac Dis, vol. 10(Suppl 7), pp. S867–S875., 2018. New England Journal of Medicine, vol. 365, no. 5, pp. 395–409, PMID:
[13] R. Yamashita, M.o Nishio, R. K. Do and K. Togashi, "Convolutional 21714641, 2011.
neural networks: an overview and application in radiology, " Insights into [36] K. Kingsley, R. Mathieu, M. Gaurav, C. Huiling, L. Jie, N. Babar, et al.,
Imaging, vol. 9, pp. 611–629, 2018. "Deep learning for lung cancer detection: Tackling the Kaggle data science
[14] K. Yasaka, H. Akai, A. Kunimatsu, S. Kiryu, O. Abe, "Deep learning bowl 2017 challenge, " arXiv preprint arXiv: 1705.09435v1,2017.
with convolutional neural network in radiology, " Japanese Journal of [37] ELCAP dataset: http://www.via.cornell.edu/databases/lungdb.html.
Radiology, https://doi.org/10.1007/s11604-018-0726-3, 2018. [38] ] https://whatis.techtarget.com/definition/MATLAB [Online].
[15] A. Krizhevsky, I. Sutskever, G. E. Hinton, "ImageNet classification with [39] I. Ali, R. H. Gregory, G. Gowthaman, Y. Liang, W. Muhammad, B.
deep convolutional neural networks, " In: Advances in Neural Information Nartowt, M. Kane, X. Ma and J. Deng, "Lung Nodule Detection via
Processing Systems, vol. 25, pp.1106–1114, 2012. Deep Reinforcement Learning," Frontiers in Oncology, vol. 8, no. 108,
[16] K. Simonyan, A. Zisserman, "Very deep convolutional networks for large- https://doi.org/10.3389/fonc.2018.00108., 2018.
scale image recognition, " arXiv technical report), 2014. [40] Y. Qin, H. Zheng, Y. M. Zhu, J.Yang, "Simultaneous accurate detection
of pulmonary nodules and false positive reduction using 3D CNNs," 2018
[17] K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image
IEEE International Conference on Acoustic, Speech and Signal Processing
recognition, " arXiv :1512.03385, 2015.
(ICASSP), Calgary, AB, pp. pp.1005–1009, 2018.
[18] K. He, X. Zhang, S. Ren, J. Sun, "Identity mappings in deep residual
[41] H.Tang, D. R. Kim, X. Xie, "Automated pulmonary nodule detection using
networks," arXiv :1603.05027, 2016.
3D deep convolutional neural networks, " 2018 IEEE 15th International
[19] C. Szegedy, W. Liu, Y. Jia, et al.,âĂIJGoing deeper with convolutionsâĂİ,
Symposium on Biomedical Imaging (ISBI 2018, Washington, D.C., pp.
arXiv :1409.4842, 2014.
523–526, 2018.
[20] J. Hu, L. Shen, G. Sun, "Squeeze-and-Excitation Networks," arXiv [42] M. Winkels, T. S. Cohen, "3D G-CNNs for Pulmonary Nodule Detection,
preprint arXiv:1709.01507, 2017. " arXiv: 1804.04656v1, 2018.
[21] G. Litjens, T. Kooi , B. E. Bejnordi , Arnaud A. A. Setio , et al., "A survey [43] S. G. Lucca FranÃğa da, A. V. Thales Levi, C. S. Aristofanes et al.,
on deep learning in medical image analysis," Medical Image Analysis, vol. "Convolutional neural network-based PSO for lung nodule false posi-
42, pp. 60–88, 2017. tive reduction on CT images, " Computer Methods and Programs in
[22] O. Ronneberger, P. Fischer, T. Brox, "U-net: convolutional networks for Biomedicine,vol.162, pp.109–118, 2018.
biomedical image segmentation," In: Proceedings of the Medical Image [44] P. Monkam, S. Qi , M. Xu, F. Han, X. Zhao and W. Qian, "CNN models
Computing and Computer-Assisted Intervention. In: Lecture Notes in discriminating between pulmonary micro nodules and non nodules from
Computer Science, 9351 , 2015, pp. 234–241. CT images," Bio. Med.Eng. OnLine,vol. 17, no. 96, 2018.
[23] F. Milletari, S. A. Ahmadi, C. Kroll, A. Plate, V. Rozanski, J. Maiostre, et [45] X. Liu, F. Hou, H. Qin, A. Hao, "Multi-view multi-scale CNNs for lung
al., "Hough-CNN: deep learning for segmentation of deep brain regions in nodule type classification from CT images, " Pattern Recognition,vol. 77,
MRI and ultrasound," arxiv: 1601.07014, 2016. pp. 262–275, 2018.
[24] F. Ciompi, K. Chung, S.J. van Riel, " Towards automatic pulmonary [46] A. J. Jia, B. J. Liu, and C. Y. Gu, "Computer-aided diagnosis of pulmonary
nodule management in lung cancer screening with deep learning," arxiv: nodules on CT scan images," 10th International Conference on Modelling,
1610.09157, 2016. Identification and Control (ICMIC),Guiyang, pp.1–6, 2018.
[25] K. Kamnitsas, C. Ledig, V.F. Newcombe, J. P. Simpson, et al., "Ef- [47] A. Pezeshk, S. Hamidian, N. Petrick, and B. Sahiner, " 3D convolu-
ficient multi-scale 3D CNN with fully connected CRF for accurate tional neural networks for automatic detection of pulmonary nodules
brain lesion segmentation," Med. Image Anal, vol. 36, , pp 61–78, doi: in chest CT," IEEE Journal of biomedical and health informatics,Doi
10.1016/j.media.2016.10.004, 2017. 10.1109/JBHI.2018.2879449, 2018.
[26] H. R. Roth, L. Lu, J. Liu, J. Yao, A. Seff, et al., "Improving computer- [48] A. Pezeshk, N. Petrick, W. Chen, and B. Sahiner, "Seamless lesion
aided detection using convolutional neural networks and random view insertion for data augmentation in CAD training," IEEE Trans. Medical
aggregation, " IEEE Trans. Med. Imaging , vol. 35 , no. 5, pp. 1170–1181. Imaging, vol. 36, no.4, pp.1005–1015, 2017.
doi: 10. 1109/TMI.2015.2482920, 2016. [49] H. Jung, B. Kim, I. Lee, J. Lee and J. Kang, "Classification of lung nodules
[27] G. Jing, L. Jiyu, W. Lijia, Z. Bin, N. Shengdong, "Computer-aided detec- in CT scans using three-dimensional deep convolutional neural networks
tion of pulmonary nodules using dynamic self-adaptive template matching with a checkpoint ensemble method," BMC Medical Imaging, vol. 18, no.
and a FLDA classifier, " Physica Medica , vol. 32, pp. 1502–1509, 2016. 48, https://doi.org/10.1186/s12880-018-0286-0, 2018.
[28] J. Kassimu, H. Ma, Z. Yue, "Lung Cancer Detection and Analysis Using [50] P. Monkam, S. Qi, M. Xu, H. Li,F. Han, Y. Teng and W. Qian, "Ensemble
Data Mining Techniques, Principal Component Analysis and Artificial learning of multiple-view 3D-CNNs model for micro-nodules identifica-
Neural Network, " American Scientific Research Journal for Engineering, tion in CT images," IEEE Access, Doi:10.1109/ACCESS.2018.2889350,
Technology, and Sciences (ASRJETS), vol. 26, no.3, pp. 254–265, 2016. 2018.
[29] L. Jikui, J. Hongyang, G. Mengdi, H. Chenguang, W. Yu, W. Pu, M. He and [51] J. L. Causey, J. Zhang, S. Ma, B. Jiang, J. A. Qualls, D. G. Politte et al.,
L. Ye, "An Assisted Diagnosis System for Detection of Early Pulmonary "Highly accurate model for prediction of lung nodule malignancy with
Nodule in Computed Tomography Images, " J Med Syst, vol. 41, DOI: CT scans," Scientific Reports,vol. 8, no. 9286, Doi: 10.1038/s41598-018-
10.1007/s10916-016-0669-0, 2017. 27569-w, 2018.
[30] G. Luis, N. Jorge, C. Antonio, C. Aurelio, "Learning Lung Nodule Malig- [52] Y. Liu, P. Hao, P. Zhang, X. Xu, J. Wu, and W. Chen, "Dense Convolutional
nancy Likelihood from Radiologist Annotations or Diagnosis Data, " J. Binary-Tree Networks for Lung Nodule Classification," IEEE Access,vol.
Med. Biol. Eng., vol. 38, pp. 424–442, 2018. 6, pp.49080–49088, 2018.
VOLUME 4, 2016 15
[53] G. Zhang, Y. Luo, D. Zhu, Y. Xu, Y. Sun, J. Lu, "Spatial Pyramid [74] K. Yasaka, H. Akai, D. Mackin, L. Court, et al., "recision of quantitative
Dilated Network for Pulmonary Nodule Malignancy Classification," 4th computed tomography texture analysis using image filtering: a phantom
International Conference on Pattern Recognition (ICPR), pp. 3911–3916, study for scanner variability," Medicine (Baltimore),vol. 96, no. 21, Doi:
2018. 10.1097/MD.0000000000006993, 2018.
[54] B. Wu, Z. Zhou, J. Wang, Y. Wang, "Joint learning for pulmonary nodule [75] K. Hyungjin, P. C. Min, L. Myunghee, et al., "Impact of Reconstruction
segmentation, attributes and malignancy prediction," IEEE 15th Interna- Algorithms on CT Radiomic Features of Pulmonary Tumors: Analysis
tional Symposium on Biomedical Imaging (ISBI 2018), pp. 1109– 1113, of Intra- and Inter-Reader Variability and Inter-Reconstruction Algorithm
2018. Variability," PLos ONE,vol. 11, no. 10, e0164924, 2016.
[55] R. Dey, Z. Lu, Y. Hong, "Diagnostic classification of lung nodules using [76] M. Shafiq-UI-Hassan, G. G. Zhang, K. Latifi, G. Ullah, D. C. Hunt, et al.,
3D neural networks," 2018 IEEE 15th International Symposium on "Intrinsic dependencies of CT radiomics features on voxel size and number
Biomedical Imaging (ISBI 2018), pp. 774–778, 2018. of gray levels," Medical physics,vol. 44 , no. 3, pp. 1050–1062, 2017.
[56] W. Sori, J. Feng, S. Liu, "Multi-path convolutional neural network for lung [77] W. Zuo, F. Zhou, Z. Li and L. Wang, "Multi-Resolution CNN and
cancer detection," Scientific Reports, https://doi.org/10.1007/s11045-018- Knowledge Transfer for Candidate Classification in Lung Nodule Detec-
0626-9, 2018. tion," IEEE Access,vol. 7, pp. 32510–32521, 2019. doi: 10.1109/AC-
[57] A. O. de Carvalho Filho, A. C. Silva et al., "Classification of patterns of CESS.2019.2903587.
benignity and malignancy based on CT using topology-based phylogenetic [78] X. Li, H. Chen, X. Qi, Q. Dou, C. Fu and P. Heng, "H-DenseUNet: Hybrid
diversity index and convolutional neural network," SPattern Recognition, Densely Connected UNet for Liver and Tumor Segmentation From CT
vol. 81, pp. 200–212, 2018. Volumes," IEEE Transactions on Medical Imaging,vol. 37 , no. 12, pp.
[58] A. Gupta, S. Das, T. Khurana, K. Suri, "Prediction of Lung Cancer from 2663–2674, 2018.
Low-Resolution Nodules in CT-Scan Images by using Deep Features," [79] M. Xu, S. Qi, Y. Yue et al., "Segmentation of lung parenchyma in
2018 International Conference on Advances in Computing, Communica- CT images using CNN trained with the clustering algorithm generated
tions and Informatics (ICACCI, Bangalore, pp. 531-537, 2018. dataset," Bio. Med.Eng. OnLine, vol. 18, no. 2, 2019.
[59] W. Zhu, C. Liu, W. Fan, X. Xie, "DeepLung: Deep 3D Dual Path Nets [80] Y. Liu, H. Wang, Q. Li et al., "Radiologic features of small pulmonary
for Automated Pulmonary Nodule Detection and Classification," 2018 nodules and lung cancer risk in the National Lung Screening Trial: a nested
IEEE Winter Conference on Applications of Computer Vision (WACV), case-control study," Radiology,vol. 286, no. 1, pp.298–306, 2018.
pp. 673–681, 2018. [81] Y. Liu, Y. Balagurunathan, T. Atwater et al., "Radiological image traits
[60] Y. Xie, Y. Xia, J. Zhang, Y. Song, D. Feng, M. Fulham, and W. Cai, predictive of cancer status in pulmonary nodules," Clin. Cancer. Res.,vol.
"Knowledge-based Collaborative Deep Learning for Benign-Malignant 23, pp.1442–1449, 2017.
Lung Nodule Classification on Chest CT," IEEE Transactions on Medical [82] C. Parmar, P. Grossmann et al., "Machine learning methods for quanti-
Imaging, Doi: 10.1109/TMI.2018.2876510, 2018. tative radiomics biomarkers," Scientific Reports,vol. 5, no. 13087, 2015,
[61] X. Zhao, L. Liu, S. Qi, Y. Teng, J. Li, W. Qian, "Agile convolutional https://doi.org/10.1038/srep13087.
neural network for pulmonary nodule classification using CT images," [83] D. Girardi et al., "Interactive knowledge discovery with the doctor-in-
International Journal of Computer Assisted Radiology and Surgery,vol. the-loop: a practical example of cerebral aneurysms research," Brain
13, pp. 585–595, 2018. Inform.,vol.3, pp. 133–143, 2016.
[62] R. V. Medeiros da NÃşbrega, S. A. Peixoto et al., "Lung Nodule Classifi- [84] M. K. Yu et al., "Visible machine learning for biomedicine," Cell,vol. 173,
cation via Deep Transfer Learning in CT Lung Images," 2018 IEEE 31st pp. 1562–1565, 2018.
International Symposium on Computer-Based Medical Systems, pp. 244– [85] M. D. Martin, J. P. Kanne, L. S. Broderick, E. A. Kazerooni, and C. A.
249, 2018. Meyer, "Lung-RADS: pushing the limits," Radiographics , vol. 37, pp.
[63] R. Paula, L. Halla, D. Goldgofa, M. Schabathb, and R. Gillies, "Predicting 1975–1993, 2017.
Nodule Malignancy using a CNN Ensemble Approach," proc. In Interna-
tional Joint Conference on Neural Network, 2018.
[64] M. Nishio, O. Sugiyama, M. Yakami, S. Ueno, T. Kubo, T. Kuroda, K.
Togashi, "Computer-aided diagnosis of lung nodule classification between
benign nodule, primary lung cancer, and metastatic lung cancer at different
image size using deep convolutional neural network with transfer learn-
ing," PloS One ,vol. 13, no. 7, Doi: e0200721, 2018.
[65] C. Qin, D. Yao, Y. Shi and Z. Song, "Computer aided detection in chest
radiography based on artificial intelligence: a survey," Bio. Med.Eng.
OnLine,vol. 17, no. 113, 2018.
[66] A. Francisco, "Artificial intelligence for precision oncology: beyond pa-
tient stratification," npj Precision Oncology, doi: 10.1038/s41698-019-
0078-1, 2019.
[67] J. Patz EF, P. Pinsky, C. Gatsonis et al., "NLST Overdiagnosis Manuscript
Writing Team. Overdiagnosis in low-dose computed tomography screen-
ing for lung cancer," JAMA Intern Med.,vol. 174, no.2, pp. 269–274, 2014.
[68] R. Huang, S. Zhang, Li T, R. He, "HBeyond Face Rotation: Global and
Local Perception GAN for Photorealistic and Identity Preserving Frontal
View Synthesis," In: 2017 IEEE international conference on computer vi-
sion (ICCV),Venice, Italy, pp. 2458âĂŞ2467, https ://doi.ieeecomputersoc
iety.org/10.1109/iccv.2017.267, 2018.
[69] C. Han et al, "GAN-based synthetic brain MR image generation," 2018
IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018),
Washington, DC, pp. 734–738, 2018.
[70] M. Frid-Adar, I. Diamant, E. Klang, M. Amitai et al., "GAN-based
synthetic medical image augmentation for increased CNN performance in
liver lesion classification," Neurocomputing,vol. 321, pp. 321-331, 2018.
[71] C. F. Baumgartner, L. M. Koch, K. C. Tezcan et al., "Visual Feature
Attribution using Wasserstein GANs," arXiv:1711.08998v3, 2018.
[72] M. Winkels, T. S. Cohen, "Pulmonary nodule detection in CT scans with
equivariant CNNs," Medical Image Analysis,vol. 55, pp.15–26, 2019.
[73] Y. Tan, P. Guo, H. Mann, S. E. Marley, M. L. Juanita Scott, L. H. Schwartz,
D. C. Ghiorghiu, B. Zhao, "Assessing the effect of CT slice interval
on unidimensional, bidimensional and volumetric measurements of solid
tumors," Cancer Imaging,vol. 12, no. 3, pp. 497–505, 2012.
16 VOLUME 4, 2016
View publication stats http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Detection and Classification of Pulmonary Nodules

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Detection and Classification of Pulmonary Nodules

Uploaded by

Copyright:

Available Formats

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Detection and Classiﬁcation of Pulmonary Nodules Using Convolutional

Article in IEEE Access · June 2019

Patrice Monkam Shouliang Qi

SEE PROFILE SEE PROFILE

SEE PROFILE SEE PROFILE

Image-Guided Lung Biopsy Surgery View project

The user has requested enhancement of the downloaded file.

Detection and classification of

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Lung screening enables the accurate depiction of pulmonary

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

1) LIDC/IDRI dataset 5) ELCAP dataset

IV. PULMONARY NODULE DETECTION AND FALSE

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Years Authors Models Datasets

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

nodule samples). aimed at demonstrating the effectiveness of convolutional

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

FIGURE 4. Different categories of pulmonary nodule analysis systems.

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

TABLE 2. Recent studies on the prediction of nodule malignancy using CNNs

Years Authors Models Datasets Key Points

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS

You might also like