Professional Documents
Culture Documents
A Comprehensive Review of Image Segmentation Techn
A Comprehensive Review of Image Segmentation Techn
Open Access
Correspondence
* Salwa Khalid Abdulateef
Computer Science and Mathematics College,
Tikrit University, Iraq
Email: khalid.salwa@tu.edu.iq
Abstract
Image segmentation is a wide research topic; a huge amount of research has been performed in this context. Image
segmentation is a crucial procedure for most object detection, image recognition, feature extraction, and classification tasks
depend on the quality of the segmentation process. Image segmentation is the dividing of a specific image into a numeral of
homogeneous segments; therefore, the representation of an image into simple and easy forms increases the effectiveness of
pattern recognition. The effectiveness of approaches varies according to the conditions of objects arrangement, lighting, shadow
and other factors. However, there is no generic approach for successfully segmenting all images, where some approaches have
been proven to be more effective than others. The major goal of this study is to provide summarize of the disadvantages and the
advantages of each of the reviewed approaches of image segmentation.
KEYWORDS: Clustering, edge detection, graph-cut, image segmentation, region, thresholding.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and
reproduction in any medium, provided the original work is properly cited.
© 2021 The Authors. Iraqi Journal for Electrical and Electronic Engineering by College of Engineering, University of Basrah.
https://doi.org/10.37917/ijeee.17.2.18 https://www.ijeee.edu.iq 166
Abdulateef & Salman | 167
II. IMAGE SEGMENTATION TECHNIQUES The existing segmentation methods usually separate
objects from background only, pre-knowledge about data,
The majority of image segmentation algorithms may be
User-specified input parameters are required, which might
divided into three techniques: boundary-based segmentation,
lead to under-segmentation or over-segmentation, control on
region-based segmentation, and hybrid-based segmentation
environment [14-17]. Therefore, improvement of the current
[13].
techniques, thus enable better classification of images.
The first technique relies on discontinuity to split an
Despite the huge amount of the literature in the topic of
image by recognizing lines, edges, and isolated points based
image segmentation and the variety of methods that were
on abrupt changes in local attributes. The boundaries of the
used, Segmentation can be accomplished by different
regions are subsequently inferred. To create the
categories such as thresholding [18,19], edge-based
segmentation result, the second technique uses the
segmentation [20, 21], region based segmentation [22- 25]
homogeneity of spatially dense information such as texture,
and energy based segmentation [26-28]. Therefore, the
intensity, color. The third technique is combining between
following sub sections begin by reviewing thresholding
boundary-based segmentation and region-based
based segmentation followed by the other three categories.
segmentation as in Fig. 1. Besides, Table 1 present
summarizes these techniques.
TABLE 1.
COMPARISON BETWEEN IMAGE SEGMENTATION TECHNIQUES
A. THRESHODING- BASED SEGMENT and background. For most of the cases, the edges that were
This category of approaches depends on the changes in produced by this technique were disjoint. The second factor
the grey scale values of the image to divide it into sub- is its noise sensitivity. As a result, for noisy images, the edge
regions. It can also be used for extracting foreground objects frequently failed to achieve correct segmentation. The third
from the background by selected threshold value .A gray factor is its sensitivity to expanding of edges between
level image can be converted to a binary image. All of the important regions. These difficulties occur in images with
relevant information regarding the position and shape of the high spatial resolution and complex geometric shapes.
items of interest should be contained in the binary image. The Consequently, the edges between regions are hardly defined,
advantage of obtaining a binary image is that it reduces data thus producing over or under segmentation. An example of
complexity and simplifies the recognition operation. edge-based segment is shown in Fig. 3.
Significant peaks and dips in the image are difficult to
identification. However, the limitations are it does not take
into account the spatial detail so the segmented regions may
not be contiguous, it is sensitive to noise and difficult to set
threshold [30]. Another disadvantage is the concern about
the computational complexity, which increases
proportionally when the size of the image increases [19].
Fig.2. shows example of threshold-based segment.
is partition where pixels are divided into k cluster. K-means a principled energy minimization framework and permit
algorithm is used for classifying image pixels into k number consolidation of different earlier knowledge. Second, they
of clusters where k is a whole positive number. The can give smooth and closed contours as a segmentation
algorithm of K-means proposed by MacQueen [46] which output, which are fundamental and may be quickly used to
has been used for segmenting images in several areas such as further applications, such as analysis and recognition of
calories estimation [47, 48], and medical [39, 49]. This shape [52]. Also, active contour method were extensively
classification is performed based on some similarity features used for image segmentation due to find boundary of objects,
such as intensity of pixels, color and distance [43]. It is fast and efficient method.
important to notice that the advantage of using clustering is For active contour, there are two types involved edge-
that the non-need of prior knowledge about distribution of based, and region-based. For edge-based, an edge detector is
the data. Efficiency, straightforwardness, easy often used to come up with the boundaries of image regions
implementation and ability to cluster huge data points very and to attract the contours to these boundaries. For region-
quickly have attracted researchers to do k-means for image based, statistical information of the image intensities are
segmentation. This algorithm suffers from several used to evolve the contour [53]. Region-based active
drawbacks as lacking right criteria to define the number k contours often have no restrictions on the placement of initial
that is required; results are not the same when execution is contours due to global energy minimization. That is,
repeated and high dependency on the initial conditions. regardless of the position of initial contours, region-based
However, the k-mean technique has several limitations. active contour can detect internal borders. A method of
Firstly, is sensitive to noise. Secondly, it has limited choice autonomous segmentation is provided by the usage of pre-
numbers of the cluster. Thirdly, various initial centroids defined start contours. They are also less susceptible to local
produce different outcome and computationally more minima or noise than active contours based on edges. Thus,
complex, thus increasing the computational time. Fig.4. initializing active contour is a fundamental segmentation
illustrates a region-based segment. problem.
TABLE 2
SUMMARIES THE METHODS FOR IMAGE SEGMENTATION
Category Method Advantages Disadvantages
Thresholding 1-It does not require any prior knowledge 1- Significant peaks and dips in the image
Thresholding -Based
Gradient operator. 1-Easy, quick, and simple to compute. 1- Noise sensitivity is greater in these.
2-Edges and their orientation are 2-Edge detection can be shaky at times.
recognized.[32] 3- Inconsistent.[21]
Second derivative 1- Due to the approximation of gradient 1- As noise increases, the magnitude of
Edge -Based Segment
operator. magnitude, the cross operation detection of edges declines due to edge detection and
edges and their orientation is orientation.
straightforward. 2- Faults in the gray level intensity function
2-In all directions, the properties are fixed. in corners, curves, and where the gray level
3-A wide area around the pixel can be intensity function fluctuates.
tested.[33]
Optimal edge 1-The signal-to-noise ratio has been 1- Complex computations that take a long
detector improved. time to complete.
2-More accurate detection in noisy 2- Generic thresholds that operate on all
environments.[34] images are challenging to come up with.[34]
1- Can correctly distinguish between 1- Both in terms of processing time and
regions with the same attributes. memory, this is a costly option.
2- Make sure the borders are clean. 2- The seed region selected and the order in
Region growing 3-The principle is straightforward. which pixels and regions are evaluated are
4-Its results have a good shape matching. intricately tied to region growth.
5-Can select many criteria at the same 3- Seed point can obtain by manual
time. [35] interactive.
Region -Based Segment
TABLE 3.
SUMMARIES OF SOME APPLICATIONS
Seq. Real-world Applications Purpose
1- Medical applications 1- To detect MRI using region growing [35].
2- Measuring Tissue volumes
Image-guided therapy is one of the most important methods for accurate and rapid
diagnosis of diseases, as image segmentation is useful that allows for easy and
accurate analysis [61, 2].
2- Pattern Recognition 1- Face [62] and Iris [63].
3- Remote sensing 1- To segment satellite images.
The segment satellite images for instance in urban planning or precision agriculture
to treatment major environmental issues that may affect a changing climate [55]
4- Earth observation (EO) 1- Road network.
2- Local climate zones.
3- Traffic control systems.
The dynamic analysis of Earth's surface through traditional approaches gives an
increase in data, to extract high-resolution data that helps to better understand the
dynamics of the Earth's surface have used several methods, including a deep learning
model or convolutional neural networks (CNNs) [64].
5- Calories estimation Calorie estimation utilizing a mobile phone to separate the image's background from
the items that make it up [65].
6- Shape Detection It is an important part of image processing to detecting boundary from image parts
which differ in color, brightness, or texture [66].
7- Security 1- Object recognition and matching.
2- Scene understanding.
3- Fingerprint [67].
4- Video surveillance
Security is a critical and difficult issue. Image segmentation is used as a major pre-
processing in many applications such as security monitoring by separating a given
image into many salient parts [68].
8- Industrial Automation 1- Drones
applications [55] 2- Self- driving cars
3- Robotics
9- Image Processing [69] 1- image-sharpening.
2- Color and video processing
10- Agricultural imaging - Detection of crops' disease [70].
[9] I. R. I. Haque, & J. Neubert, "Deep learning approaches system based on vision: A review," Proceedings of the 5th
to biomedical image segmentation," Informatics in International Conference on Computing and Informatics,
Medicine Unlocked, vol.18, 100297, 2020. ICOCI 2015, Turkey. pp. 651-658, 2015.
[10] R. E. Armya, & A. M. Abdulazeez, " Medical images [13] N. M. Zaitoun, & M. J. Aqel, "Survey on image
segmentation based on unsupervised algorithms: A segmentation techniques". Procedia Computer Science,
review," Qubahan Academic Journal, vol. 1, no.2, pp.71- vol.65, pp.797-806, 2015.
80, 2021. [14] W. Min, S. Jiang, L. Liu, Y. Rui, & R. Jain, "A survey
[11] S. K. Abdulateef, M. Mahmuddin, & N. H. Harun, on food computing," ACM Computing Surveys (CSUR),
"Developing a new features approach for colour food vol. 52, no.5, pp. 1-36, 2019.
image segmentation," ARPN Journal of Engineering and [15] I. Chen, H. Shen, & F. Wang, "Quantifying the thickness
Applied Sciences, vol.12, no. 23, pp. 6904-6910, 2017. of each color material in multilayer transparent specimen
[12] S. K. Abdulateef, M. Mahmuddin, N. H. Harun, & Y. based on transmission image," Textile Research Journal,
Aljeroudi, "Dietary assessment and obesity aviodance vol. 90, no.21-22, pp.2522-2532, 2020.
174 | Abdulateef & Salman
[16] P. Pouladzadeh, S. Shirmohammadi, & A. Yassine, [29] X. Zheng, Q. Lei, R. Yao, Y. Gong, & Q. Yin. "Image
"Using graph cut segmentation for food calorie segmentation based on adaptive K-means algorithm,"
measurement," In Proceedings of MeMeA: The IEEE EURASIP Journal on Image and Video Processing,
International Symposium on Medical Measurements and vol.2018, no. 1, pp.1-10, 2018.
Applications, pp.1–6,2014. [30] F. M. Abubakar, "Study of image segmentation using
[17] C. J. Boushey, M. Spoden, F. M. Zhu, E. J. Delp, & D. thresholding technique on a noisy image," International
A. Kerr, "New mobile methods for dietary assessment: Journal of Science and Research (IJSR), vol.2, no.1,
review of image-assisted and image-based dietary pp.49–51, 2013.
assessment methods," Proceedings of the Nutrition [31] Thresholding(imageprocessing).
Society, vol. 76, no. 3, pp. 283-294, 2017. http://en.wikipedia.org/w/index.php?title=Thresholding
[18] C. Tulasigeri, & M. Irulappan, "An advanced (image_processing)&oldid=606970852, 2014. [Online;
thresholding algorithm for diagnosis of glaucoma in Accessed August -2014].
fundus images," In 2016 IEEE International Conference [32] S. Chakraborty, "An advanced approach to detect edges
on Recent Trends in Electronics, Information & of digital images for image segmentation," In Applications
Communication Technology (RTEICT), pp. 1676-1680, of Advanced Machine Intelligence in Computer Vision and
2016. Object Recognition: Emerging Research and
[19] A. Garg, "A Review on Image Segmentation Opportunities, pp. 90-118, 2020.
Techniques," International Journal of Recent Research [33] S. E. R. T. Eser, & A. V. C. I. Derya, "A new edge
Aspects (IJRRA), pp. 53-55, 2016. detection approach via neutrosophy based on maximum
[20] R. Nirgude, & S. Jain, "Color image segmentation with norm entropy," Expert Systems with Applications, vol.115,
k- means clustering and dynamic region merging," Journal pp.499-511, 2019.
of Science, Engineering & Technology: A Peer Reviewed [34] D. Sangeetha, & P. Deepa, " FPGA implementation of
National Journal, vol.1, no.5, pp.1–10, 2014. cost-effective robust Canny edge detection algorithm,"
[21] B. K. Shah, V. Kedia, R. Raut, S. Ansari, & A. Shroff, Journal of Real-Time Image Processing, vol.16, no.4,
"Evaluation and Comparative Study of Edge Detection pp.957-970, 2019.
Techniques," IOSR Journal of Computer Engineering, vol. [35] N. S. M. Raja, S. L. Fernandes, N. Dey, S. C. Satapathy,
22, no. 5, pp. 6-15, 2020. & V. Rajinikanth, "Contrast enhanced medical MRI
[22] S. Fan, Y. Sun, & P. Shui, "Region-merging method evaluation using Tsallis entropy and region growing
with texture pattern attention for SAR image segmentation," Journal of Ambient Intelligence and
segmentation," IEEE Geoscience and Remote Sensing Humanized Computing, 1-12, 2018.
Letters,vol.18, no.1, pp.112-116. 2020. [36] A. Khwairakpam, R. A. Hazarika, & D. Kandar, "Image
[23] J. Bhattacharjee, S. Santra, & A. Deyasi, "A segmentation by fuzzy edge detection and region growing
Metaheuristic Approach for Image Segmentation Using technique," In Proceedings of the Third International
Genetic Algorithm," In Advances in Smart Conference on Microelectronics, Computing and
Communication Technology and Information Processing: Communication Systems, pp. 51-64, 2019.
OPTRONIX, pp. 125-134, 2021. [37] J. Acharya, S. Gadhiya, & K. Raviya, "Segmentation
[24] D. Kumar, R. K. Agrawal, & P. Kumar, "Bias-corrected techniques for image analysis: A review," International
intuitionistic fuzzy c-means with spatial neighborhood Journal of Computer Science and Management Research,
information approach for human brain MRI image vol.2, no.1, pp.1218–1221, 2013.
segmentation," IEEE Transactions on Fuzzy Systems, [38] T. Wang, L. Yin, & X. Wang, "A community detection
2020. method based on local similarity and degree clustering
[25] P. Pouladzadeh, S. Shirmohammadi, & T. Arici, information," Physica A: Statistical Mechanics and its
"Intelligent SVM based food intake measurement system," Applications, vol.490, pp.1344-1354, 2018.
In Proceedings of CIVEMSA: The IEEE International [39] K. Shrivastava, N. Gupta, & N. Sharma, "Medical image
Conference on Computational Intelligence and Virtual segmentation using modified k means clustering,"
Environments for Measurement Systems and Applications, International Journal of Computer Applications, vol.103,
pp. 87–92, 2013. no.16, pp.12–16, 2014.
[26] L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, & [40] M. Satokangas, S. Lumme, M. Arffman, & I.
A. L. Yuille, "Deeplab: Semantic image segmentation with Keskimäki,"Trajectory modelling of ambulatory care
deep convolutional nets, atrous convolution, and fully sensitive conditions in Finland in 1996–2013: assessing
connected crfs," IEEE transactions on pattern analysis the development of equity in primary health care through
and machine intelligence, vol. 40, no. 4, pp. 834-848, clustering of geographic areas–an observational
2017. retrospective study," BMC health services research,
[27] K. Ding, L. Xiao, & G. Weng, "Active contours driven vol.19, no.1, pp.1-12, 2019.
by local pre-fitting energy for fast image segmentation," [41] D. P. Mashinini, K. J. Fogarty, R. C. Potter, & J. D.
Pattern Recognition Letters, vol.104, pp.29-36, 2018. Berles, "Geographic hot spot analysis of vaccine
[28] W. Wang, Z. Li, J. Yue, & D. Li, "Image segmentation exemption clustering patterns in Michigan from 2008 to
incorporating double-mask via graph cuts," Computers & 2017," Vaccine, vol.38, no.51, pp.8116-8120, 2020.
Electrical Engineering, vol. 54, pp. 246-254, 2016. [42] A. Aher, J. Kasar, P. Ahuja, & V. Jadhav, "Smart
agriculture using clustering and IOT". International
Abdulateef & Salman | 175
Research Journal of Engineering and Technology [57] P. F. Felzenszwalb, & D. P. Huttenlocher, "Efficient
(IRJET), vol.5, no. 3, pp.2395-0056, 2018. graph-based image segmentation," International Journal
[43] H. Xiao, L. He, X. Li, Q. Zhang, & W. Li, " Texture of Computer Vision, vol.59, no.2, pp.167–181, 2014.
synthesis: a novel method for generating digital models [58] P. Subudhi, & S. Mukhopadhyay, "A statistical active
with heterogeneous diversity of rock materials and its contour model for interactive clutter image segmentation
CGM verification," Computers and Geotechnics, vol.130, using graph cut optimization," Signal Processing, vol.184,
103895, 2021. 108056, 2021.
[44] V. Cohen-Addad, V. Kanade, F. Mallmann-Trenn, & C. [59] S. Zafari, T. Eerola, J. Sampo, H. Kälviäinen, & H.
Mathieu, "Hierarchical clustering: Objective functions and Haario, " Segmentation of partially overlapping convex
algorithms," Journal of the ACM (JACM), vol. 66, no.4, objects using branch and bound algorithm," In Asian
pp.1-42, 2019. Conference on Computer Vision, pp. 76-90. 2016.
[45] Q. Xu, Q. Zhang, J. Liu, & B. Luo, "Efficient synthetical [60] K. Jeevitha, A. Iyswariya, V. RamKumar, S. M. Basha,
clustering validity indexes for hierarchical clustering," & V. P. Kumar, "A review on various segmentation
Expert Systems with Applications, vol.151, 113367, 2020. techniques in image processing," European Journal of
[46] J. Xu, & K. Lange, "Power k-means clustering," In Molecular & Clinical Medicine, vol.7, no.4, pp.1342-
International Conference on Machine Learning, pp. 6921- 1348, 2020.
6931, 2019. [61] A. Abdulrahman, & S. Varol, "A Review of Image
[47] T. Ege, Y. Ando, R. Tanno, W. Shimoda, &K. Yanai, Segmentation Using MATLAB Environment," In 2020
"Image-based estimation of real food size for accurate 8th International Symposium on Digital Forensics and
food calorie estimation," In 2019 IEEE Conference on Security (ISDFS) pp. 1-5, 2020.
Multimedia Information Processing and Retrieval (MIPR) [62] Y. Kortli, M. Jridi, A. Al Falou, & M. Atri,"Face
pp. 274-279, 2019. recognition systems: A Survey," Sensors, vol.20, no.2,
[48] S. Turmchokkasam, & K. Chamnongthai, "The design pp.1-36, 2020.
and implementation of an ingredient-based food calorie [63] M. Trokielewicz, A. Czajka, & P. Maciejewicz, "Post-
estimation system using nutrition knowledge and fusion of mortem iris recognition with deep-learning-based image
brightness and heat information," IEEE Access,vol. 6, segmentation," Image and Vision Computing, vol.94,
pp.46863-46876, 2018. 103866, 2020.
[49] C. Zhu, C. U. Idemudia, & W. Feng, "Improved logistic [64] T. Hoeser, F. Bachofer, & C. Kuenzer, "Object detection
regression model for diabetes prediction by integrating and image segmentation with deep learning on Earth
PCA and K-means techniques," Informatics in Medicine observation data: A review—Part II: Applications,"
Unlocked, vol. 17, 100179, 2019. Remote Sensing, vo. 12, no. 18, 3053, 2020.
[50] B. R. Lee, "An image segmentation approach for fruit [65] T. Ege, Y. Ando, R. Tanno, W. Shimoda, & K. Yanai,
defect detection using k-means clustering and graph-based "Image-based estimation of real food size for accurate
algorithm," Vietnam Journal of Computer Science, vol. 2, food calorie estimation," In 2019 IEEE Conference on
no. 1, pp. 25-33, 2015. Multimedia Information Processing and Retrieval
[51] A. González-López, J. de Moura, J. Novo, M. Ortega, (MIPR), pp. 274-279, 2019.
&M. G. Penedo, "Robust segmentation of retinal layers in [66] Y. Yuan, J. Xie, X. Chen, & J. Wang, "Segfix: Model-
optical coherence tomography images based on a agnostic boundary refinement for segmentation," In
multistage active contour model," Heliyon, vol. 5, no. 2, European Conference on Computer Vision, pp. 489-506,
e01271, 2019. 2020.
[52] L. Yang, D. Xin, L. Zhai, F. Yuan, &X. Li, " Active [67] E. M. Cherrat, R. Alaoui, & H. Bouzahir, "Improving of
contours driven by visual saliency fitting energy for image fingerprint segmentation images based on K-means and
segmentation in SAR images," In 2019 IEEE 4th DBSCAN clustering," International Journal of Electrical
International Conference on Cloud Computing and Big & Computer Engineering (2088-8708), vol.9, no.4, pp.
Data Analysis (ICCCBDA) pp. 393-397, 2019. 2425-2432, 2019.
[53] H. Yu, F. He, & Y. Pan, "A survey of level set method [68] J. Ren, G. Chen, X. Li, & K. Mao, "Striped-texture
for image segmentation with intensity inhomogeneity," image segmentation with application to multimedia
Multimedia Tools and Applications, vol.79, no.39, pp. security," Multimedia Tools and Applications, vol.78,
28525-28549, 2020. no.19, pp.26965-26978, 2019.
[54] Z. Zhang, C.Duan, T.Lin, S. Zhou, Y. Wang, & X. Gao, [69] A. P. Agrawal, & N. Tyagi, "Review on digital image
" GVFOM: a novel external force for active contour based segmentation techniques," Journal of Critical Reviews,
image segmentation," Information Sciences, vol.506, pp.1- vol.7, no. 3, pp. 779-784, 2020.
18, 2020. [70] S. Kalaivani, S. P. Shantharajah, & T. Padma,
[55] S. Minaee, Y. Y. Boykov, F. Porikli, A. J.Plaza, N. "Agricultural leaf blight disease segmentation using
Kehtarnavaz, & D. Terzopoulos, "Image segmentation indices based histogram intensity segmentation approach,"
using deep learning: A survey," IEEE Transactions on Multimedia Tools and Applications, vol. 79,
Pattern Analysis and Machine Intelligence, 2021. no.13,pp.9145-9159,2020.
[56] G. Dimauro, & L. Simone, "Novel biased normalized
cuts approach for the automatic segmentation of the
conjunctiva," Electronics, vol.9, no.6, 997, 2020.