Professional Documents
Culture Documents
Automated Cell Nuclei Segmentation in Overlapping Cervical Images Using Deep Learning Model
Automated Cell Nuclei Segmentation in Overlapping Cervical Images Using Deep Learning Model
segmentation from clamps. The development of next- accuracy, precision, recall, sensitivity, specificity and the
generation computer-aided diagnosis depends on the coefficient metrics for benchmark data. Finally, section 5
segmentation of individual cells from the overlapping clumps. presents the conclusion and future work of proposed work.
An accurate detection and segmentation of both the nucleus
and the cytoplasm with the partial occlusion is the difficult 2 Related Work
task. Due to the extensive shape, size and contrast variations,
the boundary detection faced the issues. This section discusses major issues behind the
The prevention of cervical cancer requires the periodical existing cell / nuclei segmentation in overlapping cervical
screening tests in which the swipe of cervical regions using the cells. Prior identification and quantification of shape
cotton swab or the tissue sample handling. The problems irregularities and size variations are necessary to discriminate
observed in the periodical screening tests are time consuming, the normal and abnormal cells in the input images. Plissiti et
error prone and the inabilities of doctors. The features such as al [11] presented the automated method for cell nuclei
the shape, texture and color are used to extract lot of details detection with the prior knowledge regarding the locations of
from the image. The Radiating Gradient Vector Flow (RGVF) nuclei centroids and their refining capabilities. The utilization
snake model[5], the combination of low-intensity contrast with of distance- dependent rule on the centroids provided better
the nuclei loss result[6], clustering[7], contour detector are classification performance. Segmentation of large numbers of
applied to segment the nuclei, cytoplasm effectively. The nuclei in isolated cells suffered from the problem under the
extraction of shape, texture and the image intensity efficient different acquisition conditions in high-resolution scenarios.
efficiently reduces the false positive edges. An automatic Bergmeir et al [12] proposed the new segmentation algorithm
segmentation of nucleus and cytoplasm from the cervical to segment the nuclei with the adequate control of the expert
images requires the proper distinguish between the user. The utilization of voting scheme with the prior
background and the target region. With the diverse standard knowledge regarding the cell nuclei shape localized the cell
deviations or the classes the traditional threshold-based nuclei effectively. The removal of edges adjacent to the
methods did not provide any proper threshold value. There are background was achieved by using the randomized Hough
two phases in the Nucleus Cytoplasm Contour (NCC) detector transform and the noise-free edges were processed by using
detector[8]such as detection of nucleus and contour. On the the level set algorithm. An accurate boundary approximation
basis of the gray-level differences between the nucleus and suffered from the computational cost and the numerical
cytoplasm, the Adaptive Threshold Decision (ATD)-based instability. Lou et al [13] studied the problem of segmenting
methods are employed without proper initial contour. But, the the multiple-cell nuclei from the stained microscope images
major issues observed from these methods [9, 10] are with a shape prior. They proposed the novel extension to the
inaccurate boundary detection, presence of noisy pixels, graph cut approach which incorporated the blob-like shape
unclear image analysis, more false positive edges and prior. The structured learning was performed to identify the
misclassification. The novel technical contributions of energy terms and parameterized them clearly. The lack of
proposed work are listed as follows: correct interpretation of the Pap smear images highly
x The integration of cell nuclei region of interest locating contributes to the accurate detection of nucleus shape. Plissiti
method with the segmentation methods helpful to extract the et al [14] presented the overlapping cell segmentation in
exact cell nuclei region in low-resolution cervical images which local characteristics of boundary and the expected
x The improvement in noise removal method through the shape were combined together to form the deformable model.
connected components improves the splitting performance With the utilization of necessary weight parameters, the
effectively. energy of the deformable model was controlled. The
x The employment of N-ary ternary-texture pattern-based maximization of focus measure added the noise to the final
methods extracted more accurate features which are helpful result in energy maximization approaches and hence the
to improve the performance and accuracy attenuation of the effect of noises was the major requirement
x The utilization of fisher model selects the relevant features to remove the noise present in the images. Pertuz et al [15]
which reduces the computational complexity and time proposed the approach that fused the different frames
complexity. adaptively in order to reduce the noise effectively. The
x The composition of multiple hidden units through the simple operating stages of the fused approach were focus measure,
learning modules in deep learning enhances the selectivity measure and the image fusion.
classification performance effectively. Poor contrast directly affects the quality of
This paper is organized as follows: Section 2 describes the segmentation of overlapping cells. Happy et al [16] presented
related works on cervical cancer image analysis with the the unsupervised cell segmentation on the basis of the
overlapping cells. Section 3 discusses the implementation Extended Depth of Field (EDF) created under the different
process of proposed Boundary Refinement-N-ary ternary focal planes. They proposed the Otsu method and the level set
Texture Pattern-based Deep Learning (BR-N-DL) to segment method on the basis of the prior class weights to segment the
the nuclei, cytoplasm and the cell clump. Section 4 presents nuclei and cytoplasm respectively. The optimization of energy
the performance analysis of BR-ATP-DL regarding the function dependent on the various terms such as
regularization, shape prior, intensity ratio and overlap ratio. With these factors, this paper proposes the new boundary
Due to the inefficient elliptical shape, the segmentation refinement model through the Deep Learning (DL) framework
problem was not declared accurately. Nosrati and Hamarneh to improve the accuracy and performance.
[17] proposed the new continuous variational segmentation
according to the prior star-shape. The segmentation of 3 Boundary Refinement-based Deep
overlapping cervical cells in the Pap smear images was
achieved by using the directional derivatives. The threshold- Learning Model
based, morphological-based and the front propagation-based An efficient cell nuclei segmentation and the
methods were available for cervical cell segmentation. boundary discrimination through the DL framework is
Ushizima et al [18] proposed the computer vision algorithm to discussed in detail in this section. The proposed work
detect and distinguish the subcellular components from the comprises various successive processes such as Neighborhood
cervical images by using three steps such as cellular mass Concentric Filtering (NCF), Multi-level set segmentation for
estimation, nuclei through the super-pixel representation and nuclei, cytoplasm, N-ary ternary-based texture feature
cytoplasm detection through the nuclear narrow-band seeding extraction, feature selection and the Neural Network (NN)
and Voronoi diagrams. An automated segmentation of based DL as shown in Fig. 1. Initially, Neighborhood
overlapping cells and the reduction of cell overlap were the Concentric Filtering (NCF) removes the noise present in the
major issues in traditional approaches. Lu et al [19] performed images through the connected component analysis. The
the cell segmentation based on the integration of level set extraction of information present in the images is the major
methods with the optimization methods. They presented the requirement for segmentation. The enhanced image is the pre-
quantitative results related to the nuclei detection and requisite for the segmentation process. Hence, the NCF is
segmentation. The reasonable relationship management of integrated with the Gaussian model to enhance the image
irregular super-pixels was the major requirement to reduce the quality effectively. Then, the identification of cell nuclei
computational complexity effectively. Zhao et al [20] boundary and the isolation of nuclei with the cytoplasm
proposed the novel superpixel-based methods such as Markov through the level set formulation are performed. An N-ary
Random Field (MRF) and the composition of superpixels with ternary-based texture pattern is applied to extract the features
the connecting edges. The gap search algorithm was designed for clear image analysis. With the increase in dimensionality
to enhance the model efficiency in fast manner. The occlusion, of features, the time required for classification is more. Hence,
staining variation and the cell diversity were the major the fisher model-based feature selection process is applied to
challenges in automated cervical cytology. select the relevant features corresponding to classification.
Ramalho et al [21] tackled the various issues through the Finally, the DL model is used to classify the normal and
rough segmentation of sub-cellular compartments, structural abnormal samples in the images.
refinement of cytoplasm boundary and the morphological
reconstruction. However, the multiple-touching cell splitting
was the challenging issue under dust impurities, uneven
illumination and the irregular leucocytes. Hence, Song et al
[22] proposed the Multi-Scale Convolution Network (MSCN)
and graph-partitioning-based methods to extract the scale
invariant features which refined the coarse segmentation.
With this framework, the computational complexity was
reduced effectively. An approximate representation of
overlapping cell contour points as either strong points or weak
points was modelled as the contour locating problem in
Gradient Vector Flow Snake model [23]. The presence of
inflammatory cells, poor contrast of the cell cytoplasm and the
degree of overlap made the segmentation process as the
difficult issue. Lu et al [24] presented the improved
Fig. 1. Workflow of proposed work
segmentation algorithm by using the joint optimization of
3.1 Neighborhood Concentric Filtering
multiple level set formulations. Sharma and Mangat [25]
presented the brief review of existing methods to detect, Prior to the segmentation of cell nuclei regions, the
segment and discriminate the boundaries between the cell quality of the images is to be improved with the suitable noise
nuclei and cytoplasm segmentation. They highlighted that removal techniques. Here, the NCF and the Gaussian
some of the research studies did not consider the presence of enhancement are integrated to remove the noise and enhance
unwanted components and the inconsistent staining in cervical the quality of images effectively. The matrix window with the
cancer image analysis. The fixed number of clusters led to the size 3 X 3 is projected over the input image. From the matrix,
accuracy and performance degradation. The clustering the maximum Connected Components (CC) is extracted and it
uncertain data was the major issue in traditional methods. replaces the original pixels of the matrix. Then, the center
pixel is extracted from the matrix and it is compared with the Where, – Potential energy. The steady state solution to the
neighboring pixels. If the difference between the center pixel regularization term through the gradient flow equations as
with the neighboring pixel is greater than the center pixel, then follows:
the cell is replaced with the average value otherwise it is (4)
replaced with the median value effectively. Once the noisy
Where, – Energy functional. This equation is regarded as
pixels are removed, the quality of the image is enhanced with
the evolution equation of the time-dependent function of
the Gaussian model of standard deviation ( ) which is
( ).
calculated in equation (1). The enhanced image ( ) is
The overall derivative is called as Gateaux derivative which is
expressed as follows:
applied to the regularization as follows:
(1) (5)
(2) From the gradient flow of energy, the Distance Regularization
Level Set Evolution (DRLSE)[26] is expressed as follows:
Where, Row and column size of the image. Fig. 2 (a), (6)
(b) and (c) show the input, filtered and enhanced images. With these formulations, the level set formulations performed
to detect the nucleus effectively. The DRLSE framework does
not requires any reinitialization set-up. The binary step
function to initialize the level set formula to identify the
nucleus a shown in Fig. 3 is expressed as
(7)
nucleus part in the cervical images. A contour of interest ( ) Step 2: For iteration = 1 to N
is identified using the level set formulation with the desirable Step 3:
properties. Here, the energy function is integrated with the Step 4:
distance regularization framework which is expressed as Step 5: Energy Update,
follows:
(3) , Internal Energy
, External Energy.
Step 8: Contour Weight Update 3.3 N-ary Ternary Pattern (NTP)-based feature extraction
The – ary pattern discussed in [27] is regarded as the
base for the proposed feature extraction process. The pixel
Step 9: End for value selection for comparison is the major difference between
the proposed and traditional work. The median value is
The Region of Interest (ROI) masking is the prior operational selected value for the comparison of pixel value with the
stage in the segmentation process. On the basis of the neighboring pixels which is the basic difference in proposed
difference of pixel values in various directions with the work. Then, the row (R) and column (C) values
spacing of 0°, 30°, 45°, 60°, 90°, 120°, 135°, 180° and the corresponding to the pixel values of (3, 4), (2, 4), (2, 3), (2, 2),
opposite values, the curvature is computed as follows (3,2), (4,2), (4,3) and (4, 4) are updated by subtracting the
(8) median value (med) in proposed N-ary Ternary-based feature
The internal and external energy values from the comparison extraction as shown in Fig. 5.
of magnitude values of with the value 0. If the magnitude
value is less than or equal to zero, then the energy is regarded
as internal energy. Otherwise it is termed as external energy as
follows:
(9)
Step 6: Extract the maximum value of the temporary list added which is multiplied together with the value stored in the
Step 7: Compute the mean and standard deviation of the left part of the unit.
values in the list During the learning process, the error function depends on the
Step 8: Update the feature matrix with the following formula weight parameter is minimum in order to deal with all weights
to the neurons. The back propagation computes the error
function and brings all the neuron’s attention to one of the
weight parameter called with the edge points ( ). The
Once the features from the N-ary ternary patterns are extracted information necessary to the classification is fed to the sub-
are more, the dimensionality of the feature set will increase the network during the feed forward step as . Then the gradient
computational time. Hence, the relevant feature that describe of error function with respect to the given input and the
the better classification performance are selected to reduce the weight as . The back propagated error to the node
dimensionality. is represented as . Then, the partial derivative of the error
is expressed as
3.4 Fisher Model-based feature selection (15)
The utilization of statistical and discrimination methods With the addition of each weight to the increment, the gradient
select the best features to describe the normal and abnormal value of descent is formulated as
levels clearly. Here, the fisher score-based feature selection is (16)
used to identify the relevant features from the extracted
This correction step is required to make the back propagation
feature-set. The operating principle to select the relevant
as the learning model for neural network. Finally, the deep
features is the distance between the data points is small if they
learning model through the back propagation algorithm
lies in same class and it is large if they lie in different class.
classifies the normal and abnormal effectively.
Let us consider the set of features
is regarded as the vector of
all ones and it is zero for all zero values. The mathematical 4 Performance Analysis
formulation of fisher score is expressed as follows: Table 1 presents the comparative analysis proposed
BR-N-DL with the SVM on the basic performance parameters
(12) for cervical cancer image analysis.
Where, - positive regularization parameter
– Mean variation between the class scatter matrix Table 1. Performance analysis of proposed BR-N-DL with
= (13) SVM
Parameters BR-N-DL SVM
– Total scatter matrix TP 11 7
(14) TN 0 3
FP 0 1
Where, – mean vector of – th class
FN 1 1
– Subset of features
The features with the top score are regarded as the Sensitivity (%) 98.6 95.4
relevant features to classify the normal and abnormal levels in Specificity (%) 100 98.1
the cervical images with the overlapping cells. Precision (%) 96.1 94.1
Recall (%) 98.6 95.4
3.5 Deep Learning for classification Jaccard Coeff 91.6 90.3
The classification of normal and abnormal cells in the Dice Overlap 95.65 90.3
cervical images is modelled as the deep learning problem with Kappa Coeff. 67 50
the back propagation algorithm in this paper. The formulation Accuracy (%) 91.6 83.3
of back propagation algorithm with the single input and output
is expressed as follows:
Feed forward: Input feature set is feed forward to the network.
The primitive computations of functions and associated Fig. 6 shows the comparative analysis of sensitivity,
derivatives are evaluated and stored on the node in the specificity measures for BT-N-DL and SVM. The optimal
weight update by the multi-level set formulation and the NTP
neurons.
improves both the sensitivity and specificity values. Compared
Back propagation: The total network is run backwards with
to SVM, the BT-N-DL offers 3.24 and 1.9 % better in
the constant value-1. The incoming information to a node is
sensitivity and specificity values.
6 References
[1] I. M. Orfanoudaki, D. Kappou, and S. Sifakis, "Recent
advances in optical imaging for cervical cancer detection,"
Arch Gynecol Obstet, vol. 284, pp. 1197-208, Nov 2011.
[2] S. Tasoglu, H. C. Tekin, F. Inci, S. Knowlton, S. Wang, F.
Wang-Johanning, et al., "Advances in nanotechnology and
microfluidics for human papillomavirus diagnostics,"
Proceedings of the IEEE, vol. 103, pp. 161-178, 2015.
Fig. 7. Accuracy, precision and recall analysis [3] Z. Lu, G. Carneiro, A. Bradley, D. Ushizima, M. S.
Nosrati, A. Bianchi, et al., "Evaluation of three algorithms
for the segmentation of overlapping cervical cells," IEEE
journal of biomedical and health informatics, 2016.
[4] H. A. Phoulady, D. B. Goldgof, L. O. Hall, and P. R.
Mouton, "An approach for overlapping cell segmentation
in multi-layer cervical cell volumes," The Second
Overlapping Cervical Cytology Image Segmentation
Challenge-IEEE ISBI, 2015.
[5] K. Li, Z. Lu, W. Liu, and J. Yin, "Cytoplasm and nucleus
segmentation in cervical smear images using Radiating
GVF Snake," Pattern Recognition, vol. 45, pp. 1255-1264,
2012.
[6] S. Allwin, S. P. K. Kenny, and M. E. Jeyasingh, "Feature
set for extraction of cervical cyto images," in
Computational Intelligence and Computing Research
(ICCIC), 2010 IEEE International Conference on, 2010,
pp. 1-4.
[7] F. Wang, B. Xu, and M. Lu, "A Cluster Based Method for [20] L. Zhao, K. Li, M. Wang, J. Yin, E. Zhu, C. Wu, et al.,
Cell Segmentation," in International Conference in Swarm "Automatic cytoplasm and nuclei segmentation for color
Intelligence, 2014, pp. 253-258. cervical smear image using an efficient gap-search MRF,"
[8] P.-Y. Pai, C.-C. Chang, and Y.-K. Chan, "Nucleus and Comput Biol Med, vol. 71, pp. 46-56, Apr 01 2016.
cytoplast contour detector from a cervical smear image," [21] G. L. Ramalho, D. S. Ferreira, A. G. Bianchi, C. M.
Expert Systems with Applications, vol. 39, pp. 154-161, Carneiro, F. N. Medeiros, and D. M. Ushizima, "Cell
2012. reconstruction under voronoi and enclosing ellipses from
[9] T. Guan, D. Zhou, W. Fan, K. Peng, C. Xu, and X. Cai, 3d microscopy," Overlapping Cervical Cytology Image
"Nuclei enhancement and segmentation in color cervical Segmentation Challenge-IEEE ISBI, pp. 3-4, 2015.
smear images," in Robotics and Biomimetics (ROBIO), [22] Y. Song, L. Zhang, S. Chen, D. Ni, B. Lei, and T. Wang,
2014 IEEE International Conference on, 2014, pp. 107- "Accurate segmentation of cervical cytoplasm and nuclei
112. based on multiscale convolutional network and graph
[10] M. E. Plissiti, C. Nikou, and A. Charchanti, "Combining partitioning," IEEE Transactions on Biomedical
shape, texture and intensity features for cell nuclei Engineering, vol. 62, pp. 2421-2433, 2015.
extraction in Pap smear images," Pattern Recognition [23] T. Guan, D. Zhou, and Y. Liu, "Accurate segmentation of
Letters, vol. 32, pp. 838-853, 2011. partially overlapping cervical cells based on dynamic
[11] M. E. Plissiti, C. Nikou, and A. Charchanti, "Automated sparse contour searching and GVF snake model," IEEE
detection of cell nuclei in Pap smear images using journal of biomedical and health informatics, vol. 19, pp.
morphological reconstruction and clustering," IEEE 1494-1504, 2015.
Transactions on information technology in biomedicine, [24] Z. Lu, G. Carneiro, and A. P. Bradley, "An improved joint
vol. 15, pp. 233-241, 2011. optimization of multiple level set functions for the
[12] C. Bergmeir, M. Garcia Silvente, and J. M. Benitez, segmentation of overlapping cervical cells," IEEE
"Segmentation of cervical cell nuclei in high-resolution Transactions on Image Processing, vol. 24, pp. 1261-
microscopic images: A new algorithm and a web-based 1272, 2015.
software framework," Comput Methods Programs Biomed, [25] B. Sharma and K. K. Mangat, "Various Techniques for
vol. 107, pp. 497-512, Sep 2012. Classification and Segmentation of Cervical Cell Images-A
[13] X. Lou, U. Koethe, J. Wittbrodt, and F. A. Hamprecht, Review," International Journal of Computer Applications,
"Learning to segment dense cell nuclei with shape prior," vol. 147, 2016.
in Computer Vision and Pattern Recognition (CVPR), [26] C. Li, C. Xu, C. Gui, and M. D. Fox, "Distance regularized
2012 IEEE Conference on, 2012, pp. 1012-1018. level set evolution and its application to image
[14] M. E. Plissiti and C. Nikou, "Overlapping cell nuclei segmentation," IEEE Transactions on image processing,
segmentation using a spatially adaptive active physical vol. 19, pp. 3243-3254, 2010.
model," IEEE Transactions on Image Processing, vol. 21, [27] S. Wang, Q. Wu, X. He, J. Yang, and Y. Wang, "Local N-
pp. 4568-4580, 2012. Ary Pattern and Its Extension for Texture Classification,"
[15] S. Pertuz, D. Puig, M. A. Garcia, and A. Fusiello, IEEE Transactions on Circuits and Systems for Video
"Generation of all-in-focus images by noise-robust Technology, vol. 25, pp. 1495-1506, 2015.
selective fusion of limited depth-of-field images," IEEE
Transactions on Image Processing, vol. 22, pp. 1242-
1251, 2013.
[16] S. Happy, S. Chatterjee, and D. Sheet, "Unsupervised
Segmentation of Overlapping Cervical Cell Cytoplasm,"
arXiv preprint arXiv:1505.05601, 2015.
[17] M. S. Nosrati and G. Hamarneh, "Segmentation of
overlapping cervical cells: A variational method with star-
shape prior," in Biomedical Imaging (ISBI), 2015 IEEE
12th International Symposium on, 2015, pp. 186-189.
[18] D. M. Ushizima, A. G. Bianchi, and C. M. Carneiro,
"Segmentation of subcellular compartments combining
superpixel representation with voronoi diagrams,"
Overlapping Cervical Cytology Image Segmentation
Challenge-IEEE ISBI, pp. 1-2, 2014.
[19] Z. Lu, G. Carneiro, and A. P. Bradley, "Automated nucleus
and cytoplasm segmentation of overlapping cervical cells,"
in International Conference on Medical Image Computing
and Computer-Assisted Intervention, 2013, pp. 452-460.