Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1
Image Segmentation using Mask R-CNN
for Tumor Detection from Medical Images
T. Padma , Ch Usha Kumari, Dommeti Yamini, Kapilavai Pravallika,
2022 International Conference on Electronics and Renewable Systems (ICEARS) | 978-1-6654-8425-1/22/$31.00 ©2022 IEEE | DOI: 10.1109/ICEARS53579.2022.9751891
Konduru Bhargavi, Mula Nithya
Department of Electronics and Communication engineering ,
GRIET, Hyderabad, Telangana
Abstract- One of the dreadful diseases that the
world encounters today is brain tumor. When
abnormal cells form in the brain, it is called a
brain tumor. There are lot of variations in sizes
and location of tumor, and hence this makes it
really hard for a complete understanding of
tumor. Radiologists can easily diagnose the
disease with the help of medical image
techniques, but making this process automatic
is obviously useful. Magnetic Resonance
Imaging (MRI) is the most effective method for
detecting brain tumors where, MRI images are
trained and tested in order to detect the tumor.
The automated system would be able to detect
and pin-point the exact location of the tumor in
an MRI image. In this project, the automated
system is built using Mask Regional-based
Convolution Neural Network (Mask R-CNN)
which segments the abnormal tissues in the
brain and mask is applied over the segmented
region. Mask R-CNN has the best performance
compared to other methods such as R-CNN
(Regional-based CNN), Fast R-CNN and Faster
R-CNN.
Keywords: Tumor Detection, Image
Segmentation, MRI Images, Mask R-CNN.
I. INTRODUCTION
The brain tumor [1] is an abnormal grouping of
cells related to the brain. Malignant (cancerous) or
benign (non-cancerous) are types of brain tumors.
Malignant brain tumors originate in the brain,
grow rapidly and quickly infect the surrounding
tissues. It also has the tendency to expand to other
978-1-6654-8425-1/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: MANIPAL INSTITUTE OF TECHNOLOGY. Downloaded on April 22,2024 at 05:23:57 UTC from IEEE Xplore. Restrictions apply.
parts of the brain and cause dementia and
damage the central nervous system [2]. Cancerous
tumors (Malignant) can be divided into primary
tumors and secondary tumors. Primary
brain tumors start in the brain and spread from
there, whereas the origination of secondary
brain tumors are elsewhere. In contrary, a benign
brain tumor is a cluster of cells that develops
slowly in the brain. When a tumor grows in the
brain, the pressure inside it rises, causing brain
damage and death. As a result, early accurate
diagnosis of brain tumors can help to improve
options for treatment and increase the probability
of survival. Medical Imaging Techniques such as
MRI, CT scan, Angiography, Skull-Xray, and
Biopsy are utilised to diagnosis the brain tumor.
The best tool for detecting tumors is Magnetic
Resonance Imaging (MRI). However, because a
large number of MRI images are generated in
medical practise, manual segmentation of
tumors is a time consuming, difficult, and
burdensome task. As a result, obtaining accurate
tumor segmentation from the human brain is a
very difficult task. Therefore, an automated
system would be able to detect the tumor in an
MRI and determine its exact location.
A. Image segmentation
Image segmentation [3] belongs to the branch of
image processing which focuses on dividing an
image into separate parts usually based on their
characteristics and properties. The primary goal of
image segmentation is to simplify image such
that images can be analyzed easily. Digital image
1015
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1
segmentation in processing divides the image into
various segments having similar characteristics.
Therefore, image segmentation may involve
clustering of pixel regions based on similarities in
shape or color, or separating foreground from
background. Image segmentation is commonly
used in medical imaging to detect tumors in
organs like brain, etc.
B. CNN
Convolution Neural Network is an Artificial
Neural Network model used in image processing,
recognition and for the classification of images.
CNN is a fundamental building block in
computer vision and also in image segmentation.
CNN consist mainly of three layers that are as
follows:
Convolutional Layer: In this layer, the input
image is abstracted as feature map with the help
of kernels and filters.
Pooling Layer: It helps to down sample the
feature maps and thus, decreases the
computational power required to process the data.
Fully Connected Layer: Every neuron in each
layer is connected to every neuron in other layer
with its fully connected layer architecture.
By combining these layers in CNN, this Artificial
Neural Network can learn how to identify objects
of interest in any image.
A. RCNN
RCNN or Regional Based-Convolutional Neural-
Network is a family of ML (Machine Learning)
models applied in computer vision and image
processing tasks, mainly to detect objects.
Bounding boxes are used across object regions,
and CNNs are evaluated on all Regions of Interest
(ROI) to classify multiple regions into proposed
classes.
B. Faster R-CNN
978-1-6654-8425-1/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: MANIPAL INSTITUTE OF TECHNOLOGY. Downloaded on April 22,2024 at 05:23:57 UTC from IEEE Xplore. Restrictions apply.
Faster R-CNN consists of two stage architecture
as shown below:
RPN: RPN stands for Regional Proposal Network,
and it is a fully CNN. It generates the proposals
for the objects in an image.
Fast R-CNN: ROI Pooling is used to extract
features from each candidate box and performs
bounding-box regression and classification.
Fast R-CNN works faster than R-CNN because
it is not required to connect 2,000 proposed
regions to CNN every time instead feature map is
generated by using convolutional operation only
once per image. Compared to R-CNN, Faster R-
CNN has improved in computational speed.
The key difference in Fast and Faster R-CNN is
Fast R-CNN generates Regions of Interest through
selective search, whereas Faster R-CNN generates
ROIs using RPN.
This project shows how a machine learning-based
approach, specifically a Mask Region-based
Convolutional Neural Network is used to segment
the tumor in brain from medical images.
II. LITERATURE REVIEW
H.N.T.K.Kaldera, et al., [4] developed a model
which classified and segmented the brain tumor
from MRI images. After data preprocessing,
Convolutional Neural Network (CNN) was used
for tumor classification and feature extraction.
Faster R-CNN model was used for tumor
extraction. The model had an average accuracy of
94% for all the classifiers and 94.6% for the tumor
region extraction. However, contours of the skull
were visible in the final result.
The research paper [5] represents a method to
extract brain tumor from 2D Magnetic Resonance
brain Images by Fuzzy C-Means clustering
algorithm which was followed by traditional
classifiers and Convolutional Neural Network
(CNN). Based on texture and statistical features,
normal and abnormal pixels were distinguished.
The work used a five-layer CNN model, which
1016
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1
was not very significant. Computation time, the
complexity of the method batch size and steps
were immensely high, when there is an increase in
the layers of CNN model.
In paper [6], the proposed algorithm performed
feature extraction, classification, and
segmentation. Brain tumor segmentation was
performed using Berkeley Wavelet
Transformation (BWT), and feature extraction for
accuracy and quality rate was performed by
Support Vector Machine (SVM).The experimental
results of proposed technique were 96.51%
accuracy, 94.2% specificity, and 97.72%
sensitivity. They used only a single classifier and
feature extraction technique.
Aye Min and Dr. Zin Mar Kyu [7] proposed a
model for image enhancement and tumor
segmentation. Fusion method was proposed for
enhancement, combination of Adaptive K-means
clustering and Morphological operations
(AKMM) for tumor segmentation.
The research [8] presented an efficient Neural
Network (NN) based technique for tumor
detection in brain MRI images. Brain tissues like
CerebroSpinal Fluid (CSF), White Matter (WM),
Gray Matter (GM) along with the tumor were
segmented. The feature vectors like mean,
variance, entropy and wavelet based energy sub
bands of the segmented regions were extracted
and trained using the feed forward NN. The
accuracy of the research was 80% with a smaller
amount of data trained.
III. METHODOLOGY
Proposed Mask R-CNN Model
Mask R-CNN [9][10] is very popular in areas of
deep learning network in the computer vision field
for instance, in segmentation tasks. Faster R-CNN
is for bounding-box objects detection. The fully
convolutional network added to Faster R-CNN
978-1-6654-8425-1/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: MANIPAL INSTITUTE OF TECHNOLOGY. Downloaded on April 22,2024 at 05:23:57 UTC from IEEE Xplore. Restrictions apply.
will generate mask to each object. Backbone,
Region Proposal Network (RPN), Region of
Interest alignment layer (ROI Align), object
detection branch and mask generation branch are
building blocks of Mask R-CNN. The Faster R-
CNN model is made up of the first four. As a
result, the overall structure can be seen in fig.1.
Fig. 1: Mask RCNN Flowchart
Backbone:
Mask R-CNN's main feature extractor is a
backbone. The Residual Networks (ResNets) with
or without the Feature Pyramid Network (FPN)
can be used as popular choices for this part. As a
backbone, ResNet without FPN is used for
simplicity. Whenever raw images are fed through
ResNet backbone, the data are transformed into a
feature map after passing through multiple
residual bottleneck blocks.
The feature map contains abstract information
about images, such as different object instances,
classes, and also spatial properties. The feature
map is fed to the RPN.
RPN:
RPN’s task is to scan the feature map and
propose regions (ROI) where objects might be
found.
A 3X3 convolutional network layers are used
for scanning image with sliding windows to
generate its relevant anchors which represent
bounding boxes in various sizes, and distributed
1017
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1

across the image. To cover the image, an Class: Class of these objects in ROI.
approximately 20k anchors with different scales
Bounding Box : Its purpose is to refine its location
and sizes corresponding to one another are used.
& size of bounding box which encapsulates the
To check whether anchor contains any object or
background, binary classification is used. object.

ROI Align: Mask Generation Branch:

Mask generation branch is a fully convolutional
The RPN-derived regions could have a variety
neural network. In this, ROI feature map is fed to
of shapes. As a result, ROI Align uses pooling
convolutional layers and mask is generated
layer to transform all of regions into same shape.
accordingly. For each class, a segmentation mask
For the proposed regions, Intersection over is generated.
Union (IoU) is computed based on ground truth
The mask RCNN not only generates bounding
boxes. If IoU is found greater than 0.85, it is
considered as Region of Interest. The results are box and class of the tumor, but also masks the
area of the tumor which helps in easier analysis.
processed by object detection and mask
The mask R-CNN is simple to train. This method
generation blocks to generate class, bounding box,
and mask on objects. is efficient and accurate compared to CNN and
other two stage detectors. CNN models have an
Object Detection Branch: accuracy of 95% whereas mask R-CNN model
has an accuracy of 96.79%.
This stage processes the ROIs proposed by
RPN. For each ROI, two different outputs are
generated:

Fig. 2 : Mask R-CNN Architecture

A. Dataset Fig.3). In which, 200 images are used to train the

model, 60 MRI images are used to validate the
For the current proposed method, only one model, and 40 MRI images are used to test the
dataset is used. The dataset [11] contains 300 MRI model.
images of brain tumors (a sample is shown in

978-1-6654-8425-1/22/$31.00 ©2022 IEEE 1018

Authorized licensed use limited to: MANIPAL INSTITUTE OF TECHNOLOGY. Downloaded on April 22,2024 at 05:23:57 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1
Fig 3 : Dataset Sample Brain MRI Images
The dataset images are already pre-processed
and are in grayscale format. Each test train and
validate folders contains a JSON (JavaScript
Object Notation) file. JSON is a file containing
data in the form of x-y coordinates of polygon
which represents the custom masked tumors. This
file is used in training the Mask R-CNN model to
locate the exact location of the tumor. These
polygons of tumor in JSON file help to validate
and test the model by comparing our results to
actual ground truth of tumor.
● Setting the training configuration of the
brain tumor dataset.
The configurations are set up, which include
properties such as the number of Graphics
Processing Unit(s) or GPUs to use, as well as the
number of images per GPU (here one image per
GPU), and the number of classes, which is set to
two, one for tumor and other to background. The
number of training steps per epoch is set to100,
the learning rate to 0.001, and skip detections
with less than 85 percent confidence.
Therefore, these are the model training
configurations for the brain tumor dataset.
978-1-6654-8425-1/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: MANIPAL INSTITUTE OF TECHNOLOGY. Downloaded on April 22,2024 at 05:23:57 UTC from IEEE Xplore. Restrictions apply.
B. Training And Testing
● Download the Mask R-CNN repository and
Brain MRI scan dataset.
The Mask R-CNN repository is downloaded from
GitHub. The repository contains some building
code blocks to implement Mask R-CNN [12].
https://github.com/matterport/Mask_RCNN.git
Brain MRI scan dataset is imported from GitHub
[11].
https://github.com/rastislavkopal/brain-tumor-
segmentation.git
● Selecting right versions of libraries and
importing dependencies.
It is important to select the right versions to
implement Mask R-CNN architecture without any
errors. In this project, Keras version 2.2.4 and
Tensor Flow version 1.14.0 are used.
Import dependencies are os, sys, matplotlib,
json, numpy, and cv2, and the libraries needed for
Mask R-CNN are config, utils, modellib,
visualize, and coco.
● Setting up the Custom Dataset class.
The brain tumors dataset and json annotations are
generated [13].
Custom dataset iterates through all the files in
the image and annotations datasets, to add a new
category, image, and annotation, resulting in a
dataset that connects the annotations data to the
image data with the class tumor.
1019
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1
● Initializing the Mask R-CNN model for
training.
An instance of the config that was created
earlier is used to initialize the Mask R-CNN. Then
the weights of previously trained Mask-R-CNN
model is downloaded from the coco dataset
excluding the last couple of layers.
For this training data, the Mask R-CNN head
layers should probably be sufficient. Now the
dataset is loaded into the model and trained for 15
epochs with the assigned learning rate of 0.001.
IV. RESULTS
The function “display differences” in Mask R-
CNN implementation, plots the output Mask
Polygon on the given picture. The Mask R-CNN’s
results are then given into the function, and a
visual representation of the mask’s position,
where the tumor is placed, can be viewed on top
of the brain MRI picture. After 15 epochs, there is
a loss of 0.4597 and a greater accuracy.
The obtained results are shown in figure 4 and the
graphical representation of epoch vs loss is
depicted in figure 5.
Fig 4 : Results
978-1-6654-8425-1/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: MANIPAL INSTITUTE OF TECHNOLOGY. Downloaded on April 22,2024 at 05:23:57 UTC from IEEE Xplore. Restrictions apply.
Fig 5 : Epoch VS Loss Graph
V. CONCLUSION
This project concludes that the best way to
predict brain tumors is to use MRI scanned
images and then use Mask R-CNN to detect the
tumor within them. To implement this proposed
method, the Mask R-CNN model, as well as
various machine learning techniques and some
necessary pre-processing steps are used. The
method has shown to be quite accurate in
predicting the tumor’s location and size from a
brain MRI scan. This method is highly dependent
on the quality of the brain’s MRI image;
therefore, if the image is incorrect, blurred,
blacked, overexposed to light, low brightness, or
incomplete, it may not yield the precise result. It
may consume some time to train, however it
provides accurate segmentation. This method can
be used to create reports and automate processes.
VI. References
[1]“Brain_Tumor-Types-Risk-factors-Symptoms-Diagnosis-T reatment-
Outlook”-Medically reviewed by Seunggu Han, M.D. — Written
by Verneda Lights — Updated on June 6, 2017
[2] “Brain tumor” From Wikipedia, the free encyclopedia.
[3] Sharma, Pulkit. "Computer Vision T utorial: A Step -by-Step
Introduction to Image Segmentation T echniques." (2019).
[4] Kaldera, H. N. T . K., Shanaka Ramesh Gunasekara, and Maheshi B.
Dissanayake. "Brain tumor classification and segmentation using faster R-
CNN." 2019 Advances in Science and Engineering T echnology
International Conferences (ASET ). IEEE, 2019.
[5] Hossain, Tonmoy, et al. "Brain tumor detection using convolutional
neural network." 2019 1st international conference on advances in science,
engineering and robotics technology (ICASERT ). IEEE, 2019.
1020
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1

[6] Bahadure, Nilesh Bhaskarrao, Arun Kumar Ray, and Har Pal T hethi.
"Image analysis for MRI based brain tumor detection and feature extraction
using biologically inspired BWT and SVM." International journal of
biomedical imaging 2017 (2017).
[7] Min, Aye, and Zin Mar Kyu. "MRI images enhancement and tumor
segmentation for brain." 2017 18th International Conference on Parallel
and Distributed Computing, Applications and T echnologies (PDCAT ).
IEEE, 2017.
[10] Sharma, Pulkit. "Computer Vision T utorial: Implementing Mask R-
CNN for Image Segmentation (with Python Code).(2019)."
[11] “Brain Tumor Dataset” By Rastislav Kopal. Available at
https://github.com/rastislavkopal/brain-tumor-segmentation.git
[12] “Mask R-CNN for Object Detection and Segmentation” (Github).
Available at

[8] Damodharan, Selvaraj, and Dhanasekaran Raghavan. "Combining https://github.com/matterport/Mask_RCNN#mask-r-cnn-for-object-

tissue segmentation and neural network for brain tumor detection." detection-and-segmentation
International Arab Journal of Information Technology (IAJIT) 12.1 (2015).
[13] “Mask RCNN implementation on a custom dataset!” All incorporated
[9] “ Mask R-CNN: A Beginner’s Guide”-Elisha Odemakinde, March 19,
in a single python notebook!-Dhruvil Shah, Dec 26, 2020.
2021

978-1-6654-8425-1/22/$31.00 ©2022 IEEE 1021

Authorized licensed use limited to: MANIPAL INSTITUTE OF TECHNOLOGY. Downloaded on April 22,2024 at 05:23:57 UTC from IEEE Xplore. Restrictions apply.