Indian Institute of Information Technology, Nagpur

Dept. of Computer Science and Engineering

Presentation-III on “Few Shot Learning for Chest X-rays”

G roup/Team
Members:
Dushyant Singh
BT19CSE108
Name of the
Supervisor:
BT19CSE07 Shivam Kumar
1 Dr. Tausif Diwan

BT19CSE122 Astitva

BT19CSE09 Yogesh Walecha

1
4
Introduction

• Machine Learning, Deep Learning works well if a large amount of labelled data is present.

• But deep learning models fail to work with less data as they will end up overfitting it.

• Few shot learning can be used to classify a new task just by looking at a few labeled
examples.
Dataset used
• Covid-19 Radiography dataset (Covid-19, Normal, Pneumonia)
• NIH Dataset 13 classes (Base classes)
Dataset
Features of Covid-19 X-ray

• Ground Glass Opacities (GGOs):

Hazy areas in the lungs.

• Consolidation:
Appears as white patches in the lungs.

• Peripheral Distribution:
Peripheral distribution of the lung abnormalities, meaning that they are more
commonly seen in the outer areas of the lungs.
Features of Covid-19 X-ray

Arrows depict Hazy areas and white patches on

the outer areas
Architecture of Our Network
• We have used Transfer Learning + Siamese Network
• Siamese network pairs naturally with Transfer Learning
• Pairs of images as input and it predicts if they are similar or not
• Each image is passed through a CNN feature extractor which gives
us the feature vector of the image.
• Then we combine these 2 feature vectors by taking Euclidian
distance and make the prediction.
Transfer Learning
• We are using VGG-16 classifier as our feature extractor.
• We tried some other models such as Xception , Resnets and
Mobilenet but VGG-16 worked out best for us.
• And instead of using random weights as initializer we are
‘imagenet’
weights.
• Because it will help us in capturing high level features and then we
can finetune the model according to our task.
• So, combining transfer learning with Siamese Network we can
increase performance and save a lot of time
Siamese Architecture
Our Hypothesis
• Hard pairs are the pairs which are from same class and have large
Euclidian distance between their feature vectors
• And the pairs belonging to different class but have similar Euclidian
distance.
• As there are N^2 pairs, randomly selecting we won’t be able to
capture the hard pairs.
• So, we find these hard pairs explicitly
• Training on hard pairs makes our model more model more robust
and it forces it to learn the true distinguishing feature
Continued…
• After every 10th epoch we use our trained model till now and
generate hard pairs equal to half our batch size
• For hard pairs we choose pairs from same class having distance
greater than threshold (> 0.7)
• And the pairs from different classes having less than threshold
distance (< 0.3)
Loss and Accuracy after training

A
C
C
L
U
O
R
S
A
S
C
Y

Epochs Epochs
 Base model vs ours

• Around 2-3 % improvement over base model.

• Query example Covid-19 image.
• Out of N images only 1 is Covid-19 image.
• N-way, form N- pairs and take least Euclidian distance pair.
• We do this to make our model task harder as now there are 20 pairs with only 1
possible correct answer.
Our Web Application
• Get diagnosis report from uploading chest Xray.

• Generate heat map of effected regions.

• Trained on CheXpert dataset which contains 14 different

classes of radiology conditions
Upload Xray
• The web application allows user to upload chest X-ray image in any
popular image format
After uploading the image user can see the
predicted classes by model
Heat Maps
• User can also generate heat map which highlights the features CNN
has used to predict the condition.
References
1Li, A.; Luo, T.; Lu, Z.; Xiang, T.; Wang, L. Large-scale few-shot learning: Knowledge transfer with class hierarchy. In Proceedings of the 2019 IEEE/CVF Conference on Computer
Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 7205–7213.
2Finn, C.; Abbeel, P.; Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning,
Sydney, NSW, Australia, 6–11 August 2017; pp. 1156–1168.
3 Liu, Y.; Zhang, H.; Zhang, W.; Lu, G.; Tian, Q.; Ling, N. Few-Shot Image Classification: Current Status and Research Trends. Electronics 2022, 11, 1752.
4Gui, L.; Wang, Y.-X.; Hebert, M. Few-shot hash learning for image retrieval. In Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops,
Venice, Italy, 22–29 October 2017; pp. 1228–1237
5 Chelsea Finn, Pieter Abbeel, and Sergey Levine. “Model-agnostic meta-learning for fast adaptation of deep networks." ICML 2017.
6Jamal, M.A.; Qi, G.-J. Task agnostic meta-learning for few-shot learning. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition,
Long Beach, CA, USA, 15–20 June 2019; pp. 11711–11719.
7Koch, G.; Zemel, R.; Salakhutdinov, R. Siamese neural networks for one-shot image recognition. In Proceedings of the Conference and Workshop on the 32nd International
Conference on Machine Learning, Lille, France, 6–11 July 2015.
8Vinyals, O.; Blundell, C.; Lillicrap, T.; Kavukcuoglu, K.; Wierstra, D. Matching networks for one shot learning. arXiv 2017, arXiv:1606.04080. Available online:
http://arxiv.org/abs/1606.04080 (accessed on 29 December 2017).
9 Shang, H.; Sun, Z.; Yang, W.; Fu, X.; Zheng, H.; Chang, J.; Huang, J. Leveraging other datasets for medical imaging classification: Evaluation of transfer, multi-task
and semi-supervised learning. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China, 13–17 October
2019; Springer: Cham, Switzerland, 2019; pp. 431–439.
10Cai, A.; Hu, W.; Zheng, J. Few-Shot Learning for Medical Image Classification. In Proceedings of the International Conference on Artificial Neural Networks, Bratislava,
Slovakia, 15–18 September 2020; Springer: Cham, Switzerland, 2020; pp. 441–452.
11Chen, X.; Yao, L.; Zhou, T.; Dong, J.; Zhang, Y. Momentum contrastive learning for few-shot covid-19 diagnosis from chest ct images. Pattern Recognit. 2021, 113, 107826
12 Jadon, S. COVID-19 detection from scarce chest X-ray image data using few-shot deep learning approach. In Medical Imaging 2021: Imaging Informatics for
Healthcare, Research, and Applications; International Society for Optics and Photonics: Bellingham, WA, USA, 2021.
13 Liu, L.; Zhou, T.; Long, G.; Jiang, J.; Yao, L.; Zhang, C. Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot
Learning on Category Graph. arXiv 2019, arXiv:1905.04042. Available online: http://arxiv.org/abs/1905.04042 (accessed on 2 June
2019).
14Fei-Fei, L.; Fergus, R.; Perona, P. One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell. 2006, 28, 594–611. [CrossRef]
15Fe-Fei, L.; Fergus; Perona. A Bayesian approach to unsupervised one-shot learning of object categories. In Proceedings of the Ninth IEEE
International Conference on Computer Vision, Nice, France, 13–16 October 2003; Volume 2, pp. 1134–1141.
Thank
You!