Ultrasound Image Analysis

with Deep Learning

Sutanu Bera
Challenge 2
Background
The given task was to segment trans ventricular ultrasound image of fetal brain to six different
class.

Input Image Segmentation Mask

Dataset Details
The given dataset contained 50 labelled image of fetal brain using manual annotation.

Baseline Model

U-Net
Experimental Details
• Adam Optimizer.

• Pixel wise cross entropy as the loss function.

• Data was normalized to the range [0-1] using min-max normalization.

• The initial learning rate was initially set to 10^-4 and was set to reduce by a factor of 2
after every 2000 epochs.

• The batch size was 2 and I trained the model for 1000 epoch.
 Analysis and Proposal
• The performance of the network was limited as the network was only trained using 50
images.

• Given the high annotation cost for labelling the data, semi supervised learning
approach seems to be most apparent next step.

• Using semi-supervised approach, we can leverage the both labelled example in

supervised manner and the unlabelled data in self-supervised manner.

In the current challenge, the unlabelled ultrasound fetal brain image given for solving
the classification challenge is used as the unlabelled data for training the segmentation
model.
Semi Supervised Training Framework:
Perturbation Based Cross Consistency
Training
 Semi Supervised Training Framework:
Perturbation Based Cross Consistency Training

Experimental Details:

The labelled and both unlabelled image was normalized to the range (0,1). In case of the unlabelled image, we
centre crop the images, to make the size [300,300].

We used Adam optimizer with default hyper parameter setting. The learning rate was first set to 1e-4,
then was set to decrease by a factor of 2 after every 2000 iteration of supervised training. Pixel wise
cross entropy loss was used as supervised loss and MSE loss is used as the cross-consistency loss.
Result and Comparison with Baseline:

Input Image Naive UNet Semi Supervised Approach

Future Work

• Given sufficient time, the current framework can be improved significantly via experimenting with
different perturbation, and consistency training.

• Also, other semi supervised framework can be tried which will make the algorithm more suitable for
ultrasound image segmentation

• Active learning seems to be an option. Where we can use a trained model to label the unlabelled data and
then refine the prediction via manual supervision, and again train another model with the newly labelled
data.
Challenge 1
 Background
The task was to train a deep learning model which can be used to classify anatomical structure in 2d fetal ultrasound
images.
 Approach

ResNet-34

I used a pre-trained ResNet34 network ( trained using image net data) as the baseline classifier
network, and then fine tunned the model on the given dataset for 50 epochs.

First the input image was resized to 224,224, and then the image was renormalized to make the
mean and standard deviation [0.485, 0.456, 0.406], [0.229, 0.224, 0.225] along three channels.
Exploration
Given amount of training data available for fine tuning, it was evident the data augmentation
would help to make the model more robust and generalize. However, I found the simple data
augmentation-based training did not help in improving the performance of the classification
network by huge margin. At least in the validation dataset

So, instead of simple augmentation-based training I explored the applicability of using

consistency-based training. The main motivation comes from a recent work on test time domain
adaptation, a model was forced to make consistent prediction for different augmented version of
the same input image.
Exploration
 Experimental Details
• We used Adam optimizer with default hyper parameter setting.

• We divided the available training into two subgroup. First 80% of data was used to train the network and
remaining 20% for validation.

• A learning schedular was employed to modulate the learning rate where the learning rate was set to reduce
by a factor of 2 if there is no improvement in the validation loss for 10 consecutive epoch.

• For training we used cross entropy loss as the main supervised loss and MSE loss for consistency training.

• The model was finetuned for 50 epochs.

• Sequential but random augmentation of Horizontal and Vertical flip.

Result:
All the models expect the final model performed similarly and achieved a validation accuracy of 96.50 +. Only the
final model crossed the 97% mark and achieved 97.27 as the highest validation accuracy.

Key Findings:
All the model faced difficulties in discriminating between fetal femur and thorax. Manual inspection of the test data
showed the prediction of the model for fetal brain is most accurate.

Future Work:
Given sufficient time I would like to use the self-supervised training like SimCLR to finetune the classifier model on
real ultrasound image, for this any large scale publicly available dataset can be used. Again, more sophisticated
consistency training can be explored like KL divergence between the prediction augmented image vs prediction on
unaugmented image. Also, a student teacher knowledge distillation can be tried to explore its efficiency. We can
modify the backbone classifier model where we can embed a embedding trained using CLIP on real ultrasound
images to add some domain knowledge.
Thank
You