Inter and Intra Slice Reduction in Brain Tumour Segmentation

Abstract

Medical Image Segmentation is a process of segmenting abnormalities from normal

tissues. Due to the increasing growth of deep learning, there are various deep network models
used to segment 3D medical images. Recently, U-Net and V-Net are used to segment 3D
medical images. But these networks suffer from high computation burden. The objective of
this paper is to reduce the size of the input 3D data so as to reduce the computation time.
Initially, the 3D slices are reduced by taking the average of few slices (Inter-slice reduction).
Then, only the tumour area is segmented using detection window (Intra-slice reduction). The
reduced 3D Magnetic Resonance Imaging (MRI) data is given to UNet with Long Short Term
Memory (LSTM) layers for segmentation and classification. The proposed method is tested
on BRATS 2017 and BRATS 2018 dataset. It achieves 96.24% accuracy, 90.84% Dice Score
Coefficient (DSC) on BRATS 2017 dataset and 92% accuracy and 88.88% DSC on BRATS
2018 dataset in 12 and 10 seconds respectively. The proposed method is compared with some
recent methods. It achieved reasonable gain in computation time with negligible loss in other
metrics.

Keywords: tumor, Magnetic Resonance Imaging, convolutional neural network, dense block

I. Introduction

Brain tumor is one of the dreadful diseases since several decades. There are several
aids for curing brain tumor. The abnormal cells in brain are termed as Gliomas. It is divided
into Low Grade Glioma (LGG) and High Grade Glioma (HGG). Tumor starts with less area
(LGG) and grows severe (HGG) which leads to death. Though the disease is severe, it can be
diagnosed when it is treated earlier. Medical tools are growing day by day which increases
the life of person. One such tool is the Magnetic Resonance Imaging (MRI) which scans the
brain using magnetic field and computer-generated radio waves. This helps the physician in
identifying tumor region and its severity.

The MRI images are available in three dimensions similar to our brain. The BRATS
dataset consists of four different modalities for a single patient. Classifying tumor in the
multimodal 3D data has several challenges. The location, size and shape of the tumour vary
from brain to brain. Automatic segmentation of tumor should identify the tumour area very
accurately. The computer aided automatic segmentation depends on deep learning nowadays.
As deep learning is a supervised learning, the system has to be trained with more number of
data.

As already mentioned, the brain image of a single patient consists of four modalites
and each modality has three dimensions. Training such a huge data using simple CNN itself
suffers from high computation time. The deep learning model used for segmentation suffers
from high computation time due to two important reasons.

 It consists of encoder and decoder path.

 It is a fully connected network.
To improve the efficiency of deep learning models, this paper proposes to reduce the
dimension of MRI data. The inter-slice and intra-slice reduction are done to reduce the
dimension. The main contributions of this work are:
 Inter-slice Reduction – Takes average of few slices.
 Intra-slice Reduction - Crops only the tumour area
 UNet with LSTM Segmentation – Efficient volumetric segmentation network model
is used.

The remaining of the chapter is organized as follows: Section 2 gives very recent
algorithms in brain tumor segmentation and classification. Section 3 elaborates the proposed
methods with all its phases. Section 4 demonstrates the proposed method with some
experiments. It also analyses the results with some comparison. Section 5 concludes the
work.

II. Related Works

As brain tumour segmentation has many of its researches based on deep learning, this
section discusses some recent researches related to brain tumour segmentation and
classification. For the automated segmentation of prostate cancer magnetic resonance images,
a 3D AlexNet approach [1] has been implemented, and the generic network ResNet 50,
Inception -V4 compares network performance. 3D MRI pictures are first sliced, with roughly
10% of the top and bottom slices being normalized since they do not include any important
features [2]. Furthermore, tumor/glioma enhancement is carried out on normalized slices
employing high boost 'convolutional kernel processing in multiple trials with variable kernel
sizes of high boos convolution kernel. For glioma extraction from normalized slices, the
kernel-enhanced glioma areas are subjected to a local adaptive thresholding approach.
Based on encoder-decoder architecture, a semantic segmentation network for tumor
sub-region segmentation from 3D MRIs has been created [3]. In order to regularize the
shared decoder and apply extra restrictions on its layers, a variational auto-encoder branch is
introduced to rebuild the input picture itself due to a restricted training dataset size. Unlike
previous Neural Architecture Search (NAS) methods, which typically searched the optimal
operators in each network layer but missed a good strategy to search for feature aggregations,
UXNet [4] is a novel NAS method for 3D medical image segmentation that searches both
scale-wise feature aggregation strategies and blockwise operators in the encoder-decoder
network.
To fully exploit the potential of data augmentation, an algorithm [5] is designed to
automatically search for the optimal augmentation strategies. The 3D Context Residual
Network (ConResNet) [6] has been developed for the accurate segmentation of 3D medical
images.
Self-attention between nearby picture patches is used in the majority of deep neural
network designs. More accurate segmentations than CNNs may be produced without any
convolution procedures [7]. This network separates a 3D picture block into n3 3D patches,
where n = 3 or 5, then computes a 1D embedding for each patch. Based on the self-attention
between these patch embeddings, the network predicts the segmentation map for the block's
center patch. With hyperelastic regularization, a new 3D topology preserving registration-
based segmentation model has been developed that can handle both 2D and 3D pictures. The
existence of the suggested model's solution is established.
Within a highly flexible network topology search space, a novel differentiable search
framework has been devised to facilitate rapid gradient-based search. The discretization of
the sought-after optimal continuous model in a differentiable scheme may result in a less-
than-optimal final discrete model (discretization gap). As a result, a topology loss has been
implemented to address this issue. Furthermore, due to budget limits, the GPU memory use
for the searched 3D model is restricted. Machine-aided predictions were used to assess the
initial BraTS labels' compliance with radiologic criteria. Our classifier is able to detect
radiologically high grade patients amid the dataset's initial lower grade population, as
evidenced by prediction scores.
A deep learning model for glioma and stroke lesion identification is reported in [11].
Artificial Intelligence-based Medical Image Segmentation for 3D (AIMIS3D) printing and
naked eye 3D visualization [12] is a platform-independent, multi-modality image registration,
segmentation, and 3D visualization application. With sufficient training, the YOLOV3
algorithm has been utilized to detect the prostate organ from T2-weighted MRI images.
Based on U-net from MRI images, prostate cancer and bladder cancer were segregated. The
platform was loaded with CT scans of osteosarcoma for 3D printing segmentation of the
lumbar spine, osteosarcoma, arteries, and local nerves.
A Support Vector Machine (SVM) classifier was trained on the BraTS dataset using
World Health Organization (WHO)-based radiomic characteristics, and the prediction score
histogram was utilized to analyze the behavior of our classifier on the lower grade population
[13]. Five professional radiologists were asked to annotate BraTS pictures between low (as
opposed to lower) grade and high grade glioma classes to create a new groundtruth.
Key diagnostic indicators in the diagnosis of glioma, the most frequent primary brain
tumor, have been reviewed in clinical background information [14]. It provides an overview
of publicly available resources and datasets for the development of novel computational tools
and image biomarkers, with a focus on the BraTS Challenge. In the case of glioma picture
segmentation, an overview of the state-of-the-art methodologies was done using publicly
accessible tools and deep learning algorithms that evolved in the framework of the BraTS
challenge.
The AGSE-VNet [15] framework for automated brain tumor MRI data segmentation
has been created. The Squeeze and Excite (SE) module is added to each encoder, and the
Attention Guide (AG) filter module is added to each decoder, with the channel relationship
being used to automatically enhance the useful information in the channel while suppressing
the useless information, and the attention mechanism being used to guide the edge
information while removing the influence of irrelevant information like noise.
A unique technique for splitting the 3D segmentation process [16] among numerous
remote workstations has been developed. The principles underpinning distributed multimedia
network segmentation were used to reduce the training Hidden Markov Model (HMM)
segmentation computational time. Furthermore, in this distributed context, secure
transmission has been examined, and several bidirectional multimedia security techniques
have been employed.
III. Proposed Methodology

The multi modal 3D input images in BRATS 2018 consist of four modalities: T1,
T1c, T2 and FLAIR. The size of each input is 240 x 240 x 155. These four images are often
referred to as one modality of MRI, respectively, which can play different roles in the
segmentation of tumors. For example, the whole tumor segmentation is better performed with
Flair, and the segmentation of tumor core can be better segmented under T2. There are two
ways to fuse these modalities: Early Fusion and late fusion. In early fusion, the modalities on
low-level features are combined. In late fusion, each modality is merged with others in a deep
layer after an independent Convolutional Neural Network (CNN). Processing these images
takes very long time. There are several pre-processing methods to reduce the size of the
input. This paper identifies the Region of Interest from the MRI and then fed to the UNet
with LSTM model for segmentation. The architecture is shown in Fig. 1. It consists of Slice
Trimming (Inter-slice reduction) and Region Of Interest (ROI) Segmentation (Inter-slice
reduction). The average processing time of LSTM is 5x times faster than CNN model [17].
Hence, LSTM layer is to UNet model in this work.

3D Input Image Slice Trimming ROI Segmentation LSTM

Fig. 1 System Architecture

Slice Trimming (Inter-slice Reduction)

The correlation between nearer slices paves the way to this section. The number of
slices in the BRATS dataset is 155. The slices are reduced by taking the average of nearby 5
slices. Let the number of slices in the 3D input image be

I ={S 1 , S 2 ,. . . , S n } where n=155 (1)

Now the trimmed slice will look like

I t={S a 1 , S a 2 , . .. , S m } (2)

where

S1 + S2 +S 3 + S4 + S 5 S +S +S +S +S S +S + S +S +S
Sa 1 = , Sa 2= 6 7 8 9 10 , . . . , Sm = 151 152 153 154 155
5 5 5
 155
The value of m= =31. The number of slices is reduced from 155 to 31. The value
5
of 5 is taken, because 155 is divisible by 5 and 31.

ROI Segmentation (Intra-slice Reduction)

The next step is to find the ROI. In Brain tumor segmentation, the ROI denotes the
tumourous area in the brain. The two important challenges in identifying ROI are:

1. The location and

2. The size of the tumour

The first challenge is resolved by identifying the center pixel location of the tumour
area from the FlAIR image. The second challenge is resolved by having common detection
window for all images. The algorithm for identifying ROI is given in Algorithm 1.

Identifying Center Pixel Location: To find the center pixel location of the tumour
area, the histogram equalized image I hist of Flair is calculated. From I hist , a binary image D
eq eq

is created using

D= {0 1 i>T
Otherwise
(3)

In the above equation, i is the pixel value and T is a threshold. By looking the values
in I hist , it is observed that the pixel values of tumour area are above 250. Hence the
eq

threshold value is set to 250. The center pixel location is identified by calculating the
minimum and maximum location of white pixels in D. The average of these locations is the
center location.

Selection of detection window: By observing the minimum and maximum location of

white pixels, the size of the tumour is approximately calculated. This study is carried out for
few images. The analysis leads to 150 x 150 sizes for detection window.

Algorithm: 1 ROI Segmentation

Input: Trimmed 3D MRI Data, Detection Window
Output: ROI
Steps:
1. For each Flair in {T 1, T 1 c, T 2, Flair }
 1.1 Find the histogram equalized image I hist . eq

1.2 For each pixel I in I hist eq

D= {0 1 Otherwise
i> 250

1.3 End
1.4 Find the minimum ( x min, y min) coordinate whose pixel value is 1 in D.
1.5 Find the maximum ( x max , y max ) coordinate whose pixel value is 1 in D.

1.6 Center c= { x min + x max y min + y max

2
,
2 }
1.7 Crop T 1, T 1 c, T 2, Flair to the size of detection window around the center.
2. End

Hence the ROI size is made common for all images. But the location depends on the
input image.

3D Multimodal
T1c MRI InputT2Data Flair
T1

Trimmed Slices

Mask

ROI
 LSTM

Fig. 2 Flow of the Proposed Method

After finding the ROI in each modal of 3D MRI, it is given LSTM network model. It
segments and classifies the labels of tumors.

Feedback UNet with Convolutional LSTM Layer

The most effective network for brain tumor segmentation is UNet. Our approach uses
convolutional LSTM instead of recurrent convolutional layer in UNet model. Convolutional
LSTM is convolutional version of LSTM [18], and it deals sequential data. The Feedback
Unet with Convolutional LSTM layer architecture is shown in Fig. 3. The features extracted
in the first round are maintained in the cell, and the features in the second round are extracted
based on the maintained features. Three major changes are made in U-Net. The first change is
to do feedback the output of U-Net to input layer. The second change is concatenation of the
outputs at the first round to the outputs at the second round. The third change is the usage of
convolutional

Segmentation
Results

Fig. 3 Feedback UNet with Convolutional LSTM Architecture

The input for the U-Net are the ROIs extracted in intra-slice reduction. The
probability map of each class is obtained from the softmax function at the final layer.
Segmentation results are obtained from those probability maps. In this network model, image
and probability maps are the input of network, and probability maps of each class are
obtained at the final layer. After that, the input image and probability maps obtained at the
first round are fed into the network again, and the probability maps obtained at the second
round are used as the final segmentation result.

IV. Experimental Results

This section analyses the proposed method with some experiments. Initially, the
datasets used for the proposed method is briefly discussed, followed by performance
measures and the results. Finally, the proposed method is compared with recent methods and
various analyses are done with ablation study.

Datasets and Performance Metrics

The proposed method is tested on two publicly available BRATS 2017 and 2018
datasets [19]. There are 431 cases (both HGG and LGG) in 2017 dataset and 476 cases in
2018 dataset. There are 146 training samples in both the datasets. The testing samples of
2017 and 2018 dataset are 146 and 191 respectively. Some examples of BRATS 2017 and
2018 are shown in Fig. 3. As already mentioned, the BRATS dataset consists of 4 MRI
modules: T1, T1c, T2 and Flair. Annotations include 3 tumor sub-regions: the enhancing
tumor, the peritumoral edema, and the necrotic and non-enhancing tumor core. The
annotations were combined into 3 nested sub-regions: Whole Tumor (WT), Tumor Core (TC)
and Enhancing Tumor (ET). Fig. 4 shows some sample MRI images from BRATS dataset.

Fig. 4 Examples of BRATS Dataset

 The most common metrics used for measuring the performance of brain tumor
classification are sensitivity and DSC. In this paper, in addition to these measures, accuracy (
Acr ) and Specificity are also used. The formulas for the above-mentioned metrics are defined
in Table 1.

Table 1 Performance Measures

Measure Formula
TP+TN
Accuracy Acr = × 100
TP+TN + FP+ FN
TP
Sensitivity Sensitivity=
TP+ FN
TN
Specificity Specificity=
(FP+TN )
2 ×TP
DSC DSC=
FP+ ( 2 ×TP ) + FN
*TP- True Positive, TN - True Negative, FP - False Positive, FN – False Negative

Results Analysis

The whole implementation is done in Matlab 2020a. The results of the proposed
method are given in Table 2 for both datasets. For training UNet, we adopted the Adaptive
Moment Estimation (Adam) optimizer, with a batch size 2 and an initial learning rate 10 -4.

Table 2 Results Obtained by the Proposed Method on Both Datasets

Dataset/
BRATS 2017 BRATS 2018
Measure
Accuracy (%) 97.24 92.8
Sensitivity (%) 97.45 91.25
Specificity (%) 96.87 90.4
DSC (%) 97.84 88.88

From the above table, it is observed that the proposed method achieves high accuracy,
sensitivity, specificity and DSC (which are above 90%) on both datasets.
 The efficiency of the proposed method is evaluated by comparing the results of the
proposed method with recent methods [2, 11, 20 – 25]. All these methods use deep learning
for segmentation and classification.

Table 3 Performance Comparison of Proposed Method with Recent Methods

Accuracy Sensitivity Specificity DSC

Method/ Measure Dataset
(%) (%) (%) (%)
Ranjbarzadeh et
97.78 97.89 97.7 97.54
al. [11]
RescueNet [20] - 91.05 - 91.26
Khan et al. [21] 96.9 - - -
2017
Zhou et al. [22] - - - 81.33
BrainSeg-net [23] - - - 87.47
AResU-Net [24] - - - 79.33
Proposed Method 97.24 97.45 96.87 97.84
Rani et al. [2] - - - 88.46
Zhou et al. [21] 92.5 - - -
2018
BrainSeg-net [23] - - - 81.17
Proposed Method 92.8 91.25 90.4 88.88

From the Table 3, it is observed that the results of the proposed method is little higher
than other methods on both the datasets.

The objective of this paper is to reduce the computation burden of deep learning. In
this section, the computation time is analysed with and without inter-intra slice reduction.
The proposed method is tested on Intel Core i7 2.8 GHz machine with a GPU NVIDIA
GeForce GTX 1050. Table 4 displays the computation time of proposed method.

Table 4 Computation Time Comparison of Proposed Method with Other Methods

Computation Time (s)

Dataset/Method
BRATS 2017 BRATS 2018
Ranjbarzadeh et al. [11] 38 -
U-Net 35 30
Proposed method (Inter- 12 10
 intra-slice reduction)

The computation time of the proposed method in the machine with the above-
mentioned configuration is 12s for BRATS 2017 dataset and 10s for BRATS 2018 dataset.
This speed is 3 times faster than simple U-Net model. Thus the proposed method is efficient
in computation time with satisfiable performance metrics values.

V. Conclusion

Knowledge of researchers should be useful to human. One major problem that

humans face is the brain tumor. There are several researches in this field that uses deep
learning. In this paper, a research is made in brain tumor segmentation. Though the deep
learning suffers from high computation time, this paper tries to reduce the time by reducing
the three dimension MRI data. The reduced slice is given to UNet for segmentation and
classification. The proposed method is tested on BRATS 2017 and 2018 dataset which have 5
labels including non-tumor area. It is evaluated using accuracy, sensitivity, specificity and
DSC. The proposed method is compared with recent methods in terms of these metrics and
proved that there is a negligible loss in performance. But the computation time is very much
reduced when compared to using only UNet. In future, a new network model can be designed
according to the size of the tumor.