Karen Simonyan and Andrew Zisserman. 2014.

Very deep convolutional networks for large-

scale image recognition. In Proceedings of the
3rd International Conference on Learning Rep-
resentations (ICLR).
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre

rs
Sermanet, Scott Reed, Dragomir Anguelov, Du-

rs

ty
rro
rro

ali
mitru Erhan, Vincent Vanhoucke, and Andrew

%E

Qu
#E
Rabinovich. 2015. Going deeper with convo- Category
lutions. In Proceedings of the IEEE confer- 1 scientific 149 0.571 0.429
ence on computer vision and pattern recognition 2 report 101 0.381 0.619
(CVPR). 3 resume 37 0.308 0.692
Mingxing Tan and Quoc Le. 2019. EfficientNet: Re- 4 news 36 0.191 0.809
thinking model scaling for convolutional neural 5 memo 118 0.190 0.810
networks. In Proceedings of the 36th Interna- 6 note 35 0.174 0.826
tional Conference on Machine Learning (ICML). 7 form 71 0.165 0.835
8 letter 88 0.155 0.845
9 advertisement 14 0.0609 0.939
A. Implementation Details 10 email 25 0.0417 0.958
Overall 674 0.194 0.81
We adapt the evaluation scheme used by Afazal
et al. to evaluate the 3 CNN models on both To- Table 4: CleanLab reported label errors
bacco3482 and Clean Tobacco.

Data Split To create train-validation-test splits,

We randomly sampled 100 images from each
class of which 80 were used for training and the
other 20 were used for validation. The rest were
used for testing. Hence, we had 800 training im-
ages, 200 validation images and 2482 test images.
We created 5 such partitions of the dataset and re-
ported the median accuracy. We noted that the dis-
tribution of mislabeled images across splits were
similar among our partitions.

Preprocessing We resized the images to

p

)
+B
A

rla

224x224 and did a naive normalization using a

ss

ss

ve

mean value of 0.5 and standard deviation of 0.5

A
Cla

Cla

#O

%(

for each color channel.

1 letter memo 106 0.0893
Training We used ImageNet pretrained weights 2 letter report 93 0.117
on all models. We trained only the last linear layer 3 memo scientific 70 0.0795
and froze the rest of the network. We trained all 4 report scientific 66 0.125
networks using Stochastic Gradient Descent with 5 memo report 42 0.0475
a momentum of 0.9 and an initial learning rate of 6 form memo 42 0.0400
0.001. We used a learning rate that updated every 7 news scientific 36 0.0802
iteration to: 8 form scientific 29 0.0419
( )0.5 9 letter scientific 26 0.0314
epoch
lr = initial_lr ∗ 1 − 10 report resume 13 0.0339
num_epochs
We trained for 60 epochs using early stopping Table 5: CleanLab Top 10 Pairs of Class Overlap
with a patience of 5 epochs and evaluated the
epoch with the lowest validation loss on the test
set.

B. CleanLab