False Positive Reduction of Pulmonary Nodule on CT image using
Attention-based Multiple Instance Learning
Chi Cuong Nguyen Giang Son Tran
ICTLab, University of Science and ICTLab, University of Science and
Technology of Hanoi, Vietnam Technology of Hanoi, Vietnam
Academy of Science and Technology Academy of Science and Technology
of Hanoi of Hanoi
Hanoi, Vietnam Hanoi, Vietnam
ABSTRACT
Diagnosis and treatment of multiple pulmonary nodules are clin-
ically essential but challenging. Pulmonary nodule detection is
important in early lung cancer detection and diagnosis. False pos-
itive reduction (FPR) is a significant stage of pulmonary nodule
detection systems. Prior investigations on nodule candidate classi-
fication use solitary-nodule approaches, which ignore the relations
between nodules. In this study, we propose to use Attention-based
Deep Multiple Instance Learning, a variation of supervised learn-
ing to recognize true pulmonary nodules among a large group of
candidates proposed from the detection stage. By treating the mul-
tiple nodules from a different patient, critical relational information
between solitary-nodule is extracted and empirically proves the
benefit of learning the relations between multiple nodules. An at-
tention layer trained with CNN to replace typical pooling-based
aggregation in multiple instance learning (MIL). Experiments of
lung nodule FPR on the public LUNA16 dataset validate the effec-
tiveness of the proposed method. The proposed method achieved
an accuracy of 99.6%, specificity of 100%, recall of 99.92%, and F1
score of 99.6%. The experimental results reveal that our method can
achieve satisfactory performance in FPR.
KEYWORDS
Multiple Instance Learning, false positive reduction, pulmonary
nodules, CT images
ACM Reference Format:
Chi Cuong Nguyen, Giang Son Tran, and Vincent Nguyen. 2023. False Posi-
tive Reduction of Pulmonary Nodule on CT image using Attention-based
Multiple Instance Learning. In 20th International Conference on Content-
based Multimedia Indexing (CBMI 2023), September 20–22, 2023, Orléans,
France. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3617233.
3617272
1 INTRODUCTION
Lung cancer is the leading cause of cancer death in both males
and females worldwide. Early lung cancer inspection and diagnosis
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
CBMI 2023, September 20–22, 2023, Orléans, France
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 979-8-4007-0912-8/23/09. . . $15.00
https://doi.org/10.1145/3617233.3617272
Vincent Nguyen
Université d’Orléans
Orléans, France
are significant for treatment and survival. However, the beginning
symptoms of lung cancer are not obvious, causing easily mistakable
on optimal treatment period. Pulmonary nodules are abnormal
lesions, and they are also signs of suspected lung cancer. Low-dose
computed tomography (LDCT) imaging is a popular and effective
early lung cancer screening tool. Considering the massive amounts
of data required for diagnosis, manual pulmonary nodule screening
is laborious and prone to missed diagnosis, especially for small
nodules.
In general, pulmonary nodule detection is separated into two
problems: candidate detection and then FPR. The goal of FPR stage
is to recognize the true pulmonary nodule from the plentiful candi-
dates received from the first step of pulmonary nodule candidate
detection. In pulmonary candidates, there are some similarities
between nodules and normal tissues. Therefore, it is demanding
to predict accurate labels of an individual candidate. Current FPR
methods are mainly designed by effective classifiers combined with
discriminative attributes to reduce the number of candidates. Re-
cently, multiple instance learning (MIL) has been attracting atten-
tion as a weakly supervised learning method that can train the
network without making labels on a one-to-one basis.
MIL introduces the concept of "instances" representing individ-
ual data and "bags" containing all instances belonging to groups.
The network performs the classification task at bag level. In this
case, even if incorrectly predicted instances in the bag will not
contribute to performing the final classification. The influence is
low because the final result is performed comprehensively on a bag-
by-bag basis. Recently research has examined the instance-level
integration from a classifier [4, 7, 8]. Among them, [4] proposed
an attention-based deep MIL approach that incorporates two main
components: attention layer and classification layer. These two net-
works can concurrently be trained. The attention mechanism helps
the network evaluate information that should be paid attention to
when the training data contains a large amount of information. By
visualizing the internal parameters of the attention mechanism, it
is possible to determine the instances that have been paid attention
to and to visualize the basis for the decision. The classification
network optimizes learning parameters for the classification task.
This strategy improves the flexibility and interpretability of the
MIL method. By clarifying the information that should be focused
on, better performance can be obtained compared to conventional
methods.
This paper aims to address the problem of false detection as
similarities between nodules and normal tissues in CT images and
the problem of learning from imbalanced two-class dataset. First,
CBMI 2023, September 20–22, 2023, Orléans, France Chi Cuong et al.

we propose to use the attention-based deep MIL approach, which Nodule bags

performs classification at bag level and incorporates the interpreta- Instances

Non-nodule
bags
tion ability with attention mechanism. Second, we consider data
sampling and augmentation to solve the problem of imbalanced
data. We improve the current training algorithm to take into ac- Background
count the skewed distribution of the classes, by giving different Delete

weights to both the majority and minority classes.

The remainder of this paper is organized as follows. Section 2 Figure 2: Generation of instances for nodule and non-nodule
details of our proposed method. Section 3 presents our method’s bags.
experimental setup and results. Finally, we conclude our paper in
Section 4 and present potential directions for future works.
Nodule bags
Attention
2 METHODS CNN ... FC

...
...

...

...
... Mechanism Non-nodule
I1 I2 Ik bags
Output layer
Outline of proposed scheme: FPR in lung nodule detection has Instances Backbone layer h1 h2 hk
MIL Attention
Weighted
layer
Feature vectors
always been demanding in the computer-aided detection system. A feature vectors

high false positive rate reduces the precision of nodule detection.

Because of the objective correlation and largely varied size of nod-
ules from patients, attention-based Deep MIL is more suitable to
address this problem. MIL can learn features of multiple nodules
and non-nodules from different patients; the critical relationship
between multiple instances is extracted at bag level. An outline
of the proposed method is shown in Fig. 1. Lung CT scans were
divided into small image patches and stored in bags. Each bag was
then labeled as nodule or non-nodule. The final classification will
be performed by MIL classifier, as nodule bag or non-nodule bag.
Instances
Nodule
MIL Classifier
Non
Nodule
Bag
CT Scan
Figure 1: The overview of our method for False Positive Re-
duction.
Figure 3: Overall structure of the Attention-based deep MIL.
by providing the instance 𝐼𝑘 in the bag into the CNN, and the feature
vector ℎ𝑘 was obtained.
In the conventional MIL technique, the synthesized vector was
used for identification. This vector was obtained by calculating the
maximum or average value of all components from the feature vec-
tor group in each bag. However, the importance of every instance
in a bag could not be accurately identified with the simple math-
ematics of maximum or average calculation in the conventional
method. Hence, Ilse et al. [4] proposed a MIL technique using an
attention mechanism. The attention mechanism allows the CNN
to control the attention level on instances most likely labeled as
nodule and non-nodule. This method can learn critical information
from a large amount of data, consistent with the practical diagnosis
process.
This method uses attention-based CNN for image classification
to calculate the attention weight, representing instance importance
level. The weighted feature vector 𝑧 is calculated from a weighted
average of the instances using the following Equations:

Generation of instances: MIL has the ability to learn and clas- 𝐾

∑︁
sify data at bag level. Each CT slice, with a size of 512 × 512 pixels, 𝑧= 𝑎𝑘 h𝑘 (1)
was divided into small patches and used as instance candidates. For 𝑘=1
the input size, we used patches of 64 × 64 pixels as the baseline, where
as the largest nodule can be covered in the image. The CT image
captured the whole lung and therefore contained areas without re- 𝑒𝑥𝑝{w⊤𝑡𝑎𝑛ℎ(Vh⊤
gions of interest (nodules and non-nodule regions). Therefore, the k )}
𝑎𝑘 = (2)
𝐾
image was first divided into regions with nodules/non-nodules and ∑︁
⊤
𝑒𝑥𝑝{w 𝑡𝑎𝑛ℎ(Vh⊤
j )}
background regionsGenerally, background regions do not include
𝑗=1
any candidates, with the appearance of other components such as
lung air (black color) and other body tissues (blood vessels, airway where h𝑘 is the feature vector of the 𝑘 𝑡ℎ instance and 𝑎𝑘 is the
passages, ribs, etc.,.). A bag is labeled as non-nodule if all instances attention weight given for each instance in a bag so that the total
in the bag are non-nodule or labeled as nodule if there is at least value of 𝑎𝑘 is normalized to 1 for each bag. Two vectors of w and
one nodule instance. The whole process of instance generation is V are the parameters for calculating the attention weight, which is
described in Fig. 2. determined in training the network. The weighted features z were
Classification method: The structure of the attention-based then assigned to a fully connected (FC) layer with a single artificial
deep MIL [4] used for FPR is shown in Fig. 3. The network com- neuron, and bag classification (nodule and non-nodule) was then
prises a backbone network as CNN architecture, a MIL attention predicted. Here, a sigmoid [2] was used as the activation function.
module, and an output layer. The feature extraction was performed Therefore, attention mechanism weights all instances in the bag to
False Positive Reduction of Pulmonary Nodule on CT image using Attention-based Multiple Instance Learning
create a single vector to represent the bag. This vector was then
identified whether the bag as nodule or non-nodule.
Furthermore, class weighting should be considered to address
the imbalanced data classification problem between nodule and
non-nodule instances. This can be accomplished by giving different
weights to the majority and minority classes. In the training phase,
the weight difference will influence the final classification. However,
weight of each class is usually difficult to choose without prior
knowledge. A natural method of determining weight distribution
is using proportion of classes in training set. The whole purpose
of this method is to penalize the misclassification created by the
minority class by assigning a higher weight value while, at the
same time, reducing the weight of the majority class.
1 𝑛_𝑠𝑎𝑚𝑝𝑙𝑒𝑠
𝑤𝑗 = × (3)
𝑛_𝑠𝑎𝑚𝑝𝑙𝑒𝑠 𝑗 𝑛_𝑐𝑙𝑎𝑠𝑠𝑒𝑠
where 𝑤 𝑗 is weight of each class. 𝑛_𝑠𝑎𝑚𝑝𝑙𝑒𝑠 is total number of
samples in the dataset. 𝑛_𝑐𝑙𝑎𝑠𝑠𝑒𝑠 is total number of unique classes
in the target, here is 2 (nodule and non-nodule classes). 𝑛_𝑠𝑎𝑚𝑝𝑙𝑒𝑠 𝑗
is total number of sample of the respective class.
3 EXPERIMENTS
Image dataset: Our experiment used the public dataset LUng Nod-
ule Analysis 2016 (LUNA16) challenge [9], a widely utilized dataset
for the task of lung nodule detection and FPR. Four experienced
thoracic radiologists manually annotated the candidate nodules.
Each radiologist labeled the nodules as nodules ≥ 3 mm, nodules
< 3 mm, or non-nodules. LUNA16 dataset consists of CT scans from
888 patients with 1,186 nodules, confirmed by at least three of four
radiologists. LUNA16 dataset uses the candidate list provided by the
competition host, a total of 754,975 candidates as non-nodules. We
use 1,068 nodules (90%) from the total of 1,186 ground-truth nodules
for generating the training and validation set; the remaining 118
nodules (10%) are kept for testing data
The dataset provided by the organizers of LUNA16 has about 460
times more non-nodules than nodules. While applying more train-
ing samples can improve the training performance, but training
on an imbalanced dataset can lead to overfitting problems. There-
fore, we implement several sampling and augmentation methods
to handle the data skewness problem. We included all the nodule
samples in the training data. The training set is further balanced by
up-sampling the nodules by applying the following adjacent slice
and augmentation methods. In the adjacent slice strategy, most
pulmonary nodules are in spheroidal shapes while radiologists only
annotated CT slices with the largest cross-sectional area. Therefore,
we also measure two upper and two lower adjacent slices of the
ground-truth nodule image marked by the radiologists and add
them to the training set. This method results in 5,210 lung nodules
for the training set. All nodule samples are then augmented to
increase the total of training samples. Each sample is randomly ro-
tated by 20 degrees, horizontally and vertically flipped. In addition,
each nodule image is slightly shifted to a random position. The ran-
dom center shifting method prevents all objects from being located
in the center of the patch. These augmentation methods balance
the training set. All data sampling and augmentation techniques
were implemented before the training model.
CBMI 2023, September 20–22, 2023, Orléans, France
Figure 4: Example of attention weights for nodule bags.
Experiment Setting: We first build the attention CNN model
with a 64 × 64 input. A convolutional of 64 layers are added to
help the model increase attention on learning instances. Finally,
the network is followed by two fully-connected layers, with 512
and 256 neurons, respectively. We use an Adam optimizer to train
the CNN with an initial learning rate of 1 × 10 −8 . We exponentially
decay the learning rate with a ratio of 1 × 10 −2 after every epoch.
A lower learning rate can avoid a suboptimal set of weights that
are learned too quickly. The number of training epochs is 100, and
the batch size is 1. The training loss is a sparse cross-entropy loss
averaged by the number of candidates. In addition, we used early
stopping when performance on validation set was not improved.
To confirm the effectiveness of the proposed method, accuracy,
specificity, recall, and F1-score are used to evaluate the model pre-
diction. We consider the classification ability between nodule and
non-nodule of MIL model. We train our model from scratch without
pre-trained weights.
Results: Table 1 compares the nodule and non-nodule classifica-
tion performance of our proposal to other methods. The classifica-
tion by our method, which adopts an attention-deep MIL approach
structure, outperforms all other methods in classification metrics.
The proposed method is the most accurate and received the highest
recall. A higher recall is especially important in the medical domain
since false negatives could lead to severe consequences, includ-
ing patient fatality. We also notice that the attention mechanism
performs better than the plain CNN networks.
Finally, we present an exemplary result of the attention mecha-
nism of MIL in Fig. 4. In our experiments, each bag consists total of
three images. For each image, the corresponding attention score
is given by our trained network. The bag is accurately predicted
as nodules, and all nodule instances in a bag are correctly high-
lighted with higher scores. Hence, the attention mechanism works
as expected.
CBMI 2023, September 20–22, 2023, Orléans, France Chi Cuong et al.

Table 1: Classification performance of our method.

Accuracy Specificity Recall F1-score

Cao et al., 2018 [1] 96.9 98.6 95.1 91.1
El-Regaily et al., 2020 [3] 89.89 90.66 85.26 -
Yuan et al., 2021 [10] 95.6 99.8 81.2 -
Lai et al., 2021 [5] - 98.9 86.4 -
Nguyen et al., 2021 [6] 95.6 97.48 93.84 95.5
Our 99.6 100 99.2 99.6

4 CONCLUSION AND FUTURE WORK Maria Evelina Fantacci, Bram Geurts, et al. 2017. Validation, comparison, and
combination of algorithms for automatic detection of pulmonary nodules in
In this study, we developed a method for classifying nodule and computed tomography images: the LUNA16 challenge. Medical image analysis
non-nodule to reduce the false positive rate in CT images using 42 (2017), 1–13.
[10] Haiying Yuan, Zhongwei Fan, Yanrui Wu, and Junpeng Cheng. 2021. An efficient
attention-based MIL. Experimental show that our attention MIL multi-path 3D convolutional neural network for false-positive reduction of pul-
can outperform the learning ability with supervised learning meth- monary nodule detection. International Journal of Computer Assisted Radiology
ods and highlight the instance that contributed to the decision by and Surgery 16, 12 (2021), 2269–2277.
the attention mechanism. Our method automatically classifies nod-
ule candidates in CT images more efficiently and accurately than
traditional supervised learning methods. Our sampling and class
weighting can address the problem of imbalanced learning.
We used a binary classification problem to evaluate our proposed
approach. However, extending the current study to a multiple-class
classification task as nodule malignancy prediction through an
attention-based MIL network is an interesting direction for fur-
ther research. In addition, by taking advantage of three-dimension
spatial information of nodules in CT images, the three-dimension
attention-based MIL method might be one potential shortcoming
to be overcome in future research.

ACKNOWLEDGMENTS
This work was supported by the French Embassy in Vietnam under
the grant No. 16/2022/SCAC.

REFERENCES
[1] Guitao Cao, Tiantian Huang, Kai Hou, Wenming Cao, Peng Liu, and Jiawei Zhang.
2018. 3D convolutional neural networks fusion model for lung nodule detection
onclinical CT scans. In 2018 IEEE International Conference on Bioinformatics and
Biomedicine (BIBM). IEEE, 973–978.
[2] George Cybenko. 1989. Approximation by superpositions of a sigmoidal function.
Mathematics of control, signals and systems 2, 4 (1989), 303–314.
[3] Salsabil Amin El-Regaily, Mohammed Abdel Megeed Salem, Mohamed Has-
san Abdel Aziz, and Mohamed Ismail Roushdy. 2020. Multi-view Convolutional
Neural Network for lung nodule false positive reduction. Expert systems with
applications 162 (2020), 113017.
[4] Maximilian Ilse, Jakub Tomczak, and Max Welling. 2018. Attention-based deep
multiple instance learning. In International conference on machine learning. PMLR,
2127–2136.
[5] Khai Dinh Lai, Thuy Thanh Nguyen, and Thai Hoang Le. 2021. Detection of lung
nodules on ct images based on the convolutional neural network with attention
mechanism. Annals of Emerging Technologies in Computing (AETiC) 5, 2 (2021),
78–89.
[6] Chi Cuong Nguyen, Giang Son Tran, Jean-Christophe Burie, Thi Phuong Nghiem,
et al. 2021. Pulmonary nodule detection based on faster R-CNN with adaptive
anchor box. Ieee Access 9 (2021), 154740–154751.
[7] Jan Ramon and Luc De Raedt. 2000. Multi instance neural networks. In Proceedings
of the ICML-2000 workshop on attribute-value and relational learning. 53–60.
[8] Vikas C Raykar, Balaji Krishnapuram, Jinbo Bi, Murat Dundar, and R Bharat
Rao. 2008. Bayesian multiple instance learning: automatic feature selection and
inductive transfer. In Proceedings of the 25th international conference on Machine
learning. 808–815.
[9] Arnaud Arindra Adiyoso Setio, Alberto Traverso, Thomas De Bel, Moira SN
Berens, Cas Van Den Bogaard, Piergiorgio Cerello, Hao Chen, Qi Dou,