A Brief Analysis of U-Net and Mask R-CNN

for Skin Lesion Segmentation

E. Alfaro1 , X. Bolaños1 , E. Albornoz2 , C. Martínez2 and S. Calderón1

1 Computing Engineering School, Instituto Tecnológico de Costa Rica-Cartago, Costa Rica

2 sinc(i): Research institute for signals, systems and computational intelligence

Universidad Nacional del Litoral - CONICET, Santa Fe, Argentina

July 4, 2019
Introduction to the problem

Skin lesions: very common disease due to exoposition to ultraviolet

sun radiation.
A major problem: to isolate the lesion and diagnose it with a high
enough precision.
An automatic skin lesion assessment for a computer assisted diagnosis
would enable more accurate, faster, and even remote diagnosis of
patients.
It also would enable dermatologists to focus in patients with the most
dangerous skin lesions and treat them sooner.

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 2 / 14

Our approach

Segmentation: rst step in the analysis in most of computer based

skin lesion analysis systems.

System based on deep neural networks for the segmentation of skin

lesion in dermoscopic images.

We compare the performance of two previously not reported fully

convolutional architectures: U-Net and the Mask-R-CNN, for the task.

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 3 / 14

State of the art

International Skin Imaging Collaboration: challenges ISIC and ISBI

since 2016.
CNN: the melanoma test results got problems with image intensity,
uniformity and presence of imaging artifacts.
Multitask network based on GoogleNet to segment and get the
probability of the lesion being a melanoma or a seborroic keratosis.
FCN that learns the boundaries by joining partial segmentation results.
Resnet: conv-deconv layers, uses pre-trained Imagenet weights and
ne-tuning (3th place in 2017).
CDNN framework to handle dierent lightning conditions, Jaccard of
0.784 (1st place in 2017).
→ COMMON COARSE: lack of anatomically plausible masks.

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 4 / 14

Materials and methods

U-Net (Ronneberger et al, 2015): standard architecture

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 5 / 14

Materials and methods

Mask-R-CNN (He et al, 2017):

Image → backbone → RPN (Resnet-101) → output mask

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 6 / 14
Materials and methods

Network trained and evaluated using the ISIC 2017 challenge dataset:
2000 training images
150 validation images
600 test images

Implementation: python 3.5 using pytorch 0.3.

Experiments were conducted on a system with a 12 Gb Geforce Titan
GPU and a Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz.
360 epochs: 40 to train the network head (learning rate 0.01), 120 to
train the Resnet backbone, 200 to ne-tune the whole network
(learning rate 0.001).
5-fold cross validation scheme.
Post-procesing: union of the array of resulting mask.

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 7 / 14

Experiments and results

Figures of merit: similarity indexes for images segmentation

T
|A B|
Jaccard(A, B) = S
|A B|

2|A · B|
Dice(A, B) =
|A| + |B|

with A being the ground truth mask and B the prediction of the network.

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 8 / 14

Experiments and results

U-Net:
Evaluation over validation subset
Number of
Dice Jaccard
epochs

160 0.606299 0.515993

180 0.616017 0.539274

200 0.630315 0.521581

Evaluation over test subset
Number of
Dice Jaccard
epochs

160 0.675687 0.575864

180 0.676534 0.581769

200 0.681308 0.562630

180 epochs: best performance in terms of the Jaccard index.
75 epochs: reaches almost nal performance.
C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 9 / 14
Experiments and results

Mak-R-CNN:
Evaluation over validation subset
Number of
Dice Jaccard
epochs

160 0.785716 0.780492

180 0.790384 0.785926

200 0.776682 0.773677

Evaluation over test subset
Number of
Dice Jaccard
epochs

160 0.776743 0.739569

180 0.781558 0.743864

200 0.781415 0.740744

Best result: Jaccard=0.743 on the test set, close to best ISIC 2017
score (0.76).
C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 10 / 14
Experiments and results

Network loss during training

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 11 / 14

Experiments and results

Good segmentation example:

Bad segmentation example:

C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 12 / 14

Conclusions

First approach to tackle the skin lesion segmentation problem using

U-net and Mask-R-CNN deep learning architectures.
Results of this work-in-progress showed a signicant better
performance of Mask-R-CNN (also lower computation training times).

Future works:
The use of data augmentation techniques.
The addition of extra layers of information to the input image: texture,
color variability (inverse red channel) or other of medical signicance.
Modify loss functions to restrict the output to anatomically plausible
masks.
Integrate both tasks: the segmentation and classication of the lesion
into dierent skin diseases.
C. Martínez (sinc(i)) Librería para biometría facial July 4, 2019 13 / 14
Thank you!

More information:

http://sinc.unl.edu.ar

César Martínez
cmartinez@sinc.unl.edu.ar