Professional Documents
Culture Documents
Content-Adaptive Region-Based Color Texture Descriptors For Medical Images
Content-Adaptive Region-Based Color Texture Descriptors For Medical Images
1, JANUARY 2017
2168-2194 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
RIAZ et al.: CONTENT-ADAPTIVE REGION-BASED COLOR TEXTURE DESCRIPTORS FOR MEDICAL IMAGES 163
scenarios such as gastroenterology [3], [7], dermoscopy [32], 1) First, 2-D Gabor filters are very similar with a simple cell
[33] just to name a few. Though good results have been obtained in the visual cortex of mammals [36]. These cells have
in the literature, one important challenge is to incorporate the two important properties: their directional selectivity and
intraimage illumination variations in the image descriptor that bandpass nature. Therefore, they respond to specific spa-
can lead to more suitable color features for the description of tial frequencies in some specific directions. Mathematical
medical images. modeling of these cells lead to bandpass filter bank struc-
tures that are tuned for various orientations and scales
C. Contributions just like Gabor filter banks.
2) Another important characteristic of Gabor filters is that
Our main focus in this paper is to design the generic image they have the capability to achieve maximum localiza-
descriptors, which encapsulate the texture and color information tion in both the space and frequency (this characteristic
in the images. The texture and color descriptors should be able of Gabor filters is distinct and not shared by any other
to handle the rotation, scale, and illumination changes in the method [37]).
images. Additionally, the color descriptor should adapt itself 1) Gabor Filters: Gabor filters are obtained as a result of
to represent the most typical colors that occur in the images. multiplication of a Gaussian function with a complex Sinusoid.
Correspondingly, the main contributions of this paper are as A 2-D Gabor function g(x, y) and its Fourier transform G(u, v)
follows. can be written as
1) A novel rotation, scale, and illumination invariant tex-
ture descriptor is proposed as an extension of a dis- 1 1 x2 y2
g(x, y) = exp − + + 2πjW x
crete Fourier transform (DFT)-based Gabor descriptor 2πσx σy 2 σx2 σy2
in [34]. The proposed methodology introduces features (1)
that are robust to complex illumination changes, un-
like [34] where the descriptor design does not effectively 1 (u − W )2 v2
G(u, v) = exp + (2)
handle nonuniform illumination changes in the images. 2 σu2 σv2
These changes are handled by the use of analytic Gabor where σu = 2π1σ x , σv = 2π1σ y , and W is a constant representing
filters [35] with a significantly improve low frequency
the center frequency of the filter having the highest frequency.
response that encapsulates the illumination variations in
Equation (1) is the product of a Gaussian function with a com-
textures.
plex sinusoid. In the frequency domain, we obtain a bandpass
2) CIELUV color space separates image color and illumi-
filter where the characteristics of the filter are controlled by the
nation components. This fact is exploited to extract illu-
standard deviation of the Gaussian function and the frequency
mination invariant color features by prefiltering the “L”
of the sinusoid. A Gabor filter bank is formed by obtaining
channel using “homomorphic filtering.” The illumination
several filters with varying parameters. The Gabor filter banks
corrected color image is subjected to vector quantization
are nonorthogonal due to the nature of the Gaussian function.
(VQ) to identify the most typical colors (“words”) that
Therefore, the redundancy incured by the Gabor filters has to be
appear in the images. The pixels in the novel images are
reduced to ensure that the information represented by various
quantized to these “words” followed by the generation of
filter banks is as distinct as possible. We have used Manjunath’s
adapted color histograms constructed using bag-of-words
[38] strategy to achieve this: design filter parameters such that
approach.
the half peak amplitudes of the filters touch each other. This
3) The proposed features are used in a classification frame-
strategy ensures that the filters capture maximum information
work to identify the lesions in images from two distinct
with minimum redundancy. An input image, ξ(x, y) when fil-
imaging modalities, namely, gastroenterology and der-
tered by the set of Gabor filters g(x, y) is given as
moscopy.
The rest of this paper is organized as follows: We discuss the
R(x, y) = ξ(x, y)g(x − x1 , y − y1 )dx1 dy1 . (3)
texture and color features used in this paper for feature extraction
(see Section II) followed by a description of the datasets used Gabor filters have been widely used for the extraction of
in this paper with our results (see Section III) and conclude the texture features from images [39], [40].
paper (see Section IV). 2) Analytic Gabor Filters: The foundation of using the an-
alytic Gabor filters for feature extraction was laid by G. Haley
II. METHODOLOGIES et al. [35]. Their use is motivated by the fact that the standard
Texture and color are the two fundamental visual features that Gabor filters exhibit a potentially significant response at very
are of primary importance in the analysis of visual content. In low frequencies. This response is undesirable as it exists due to
this section, we will discuss the texture and color features that the inter and intraimage variations in contrast and image inten-
we have used for feature extraction from the images. sities. These variations are attributed to the imaging conditions
and not to the texture of the images resulting in misclassification.
There could be several reasons including image sampling with
A. Texture Features
different average intensity (variation in homogeneous illumina-
For texture feature extraction, we chose Gabor filters due to tion), illumination gradients, etc. There are two different ap-
various reasons. proaches to avoid these problems: preprocessing the images, or
RIAZ et al.: CONTENT-ADAPTIVE REGION-BASED COLOR TEXTURE DESCRIPTORS FOR MEDICAL IMAGES 165
(S )
modifying the Gabor filter banks. Using the former methodol- represented by Rμ (S ) that is a shifted representation of Rμ . It
ogy, the interimage variations can be corrected but the intraim- should be noted that the application of a shift invariant transform
age variations cannot be handled. For these variations, other (S )
on Rμ can give us rotation invariant features for the image. It
image processing methods such as modeling the illumination in is well known that the magnitude of the DFT is shift invariant
the logarithmic domain are needed. [42], and thus, the effect of rotation on the feature vector can be
An effective and more straightforward approach to handle curtailed using DFT of Rμ
(S )
this issue is to modify the Gabor filter to be analytic. This
is accomplished by replacing the real part of g(x, y) with the N
k
inverse Hilbert transform of the imaginary part −ĝim (x, y). FμS R = Rμ(S ) . exp(−j2π n) (9)
n =1
N
gA (x, y) = −ĝim (x, y) + jgim (x, y). (4)
where FμS R is invariant to rotation and scale changes in the
The Fourier transforms of real and imaginary parts of g(x, y) images. Let us now define
are, respectively, conjugate symmetric and conjugate antisym- N
metric resulting in the cancellation for ω ≤ 0.
Rμ(R ) = Rμ k n (10)
G(ω) − G∗ (ω), for ω > 0 n =1
GA (ω) = (5) (R )
0, for ω ≤ 0. where Rμ represents the summation of the aggregated filter
(R )
responses at each scale. Note that Rμ is a vector that is rota-
GA (ω) is the analytic Gabor filter impulse response and has
tion invariant as the responses across all orientations have been
several advantages over the standard Gabor filters for various
summed up. However, if an image is scaled by a certain amount,
applications.
(R )
1) Improved low frequency response: G(x, y) < GA (ω) at this will be represented by Rμ , which is a shifted representa-
(R )
low frequencies and GA (ω) = 0 at ω = 0. tion of Rμ . The magnitude of the DFT is shift invariant, and
2) Reduced computations as the negative frequency spec- thus, the effect of scale changes on the feature vector can be
(R )
trum does not exist for analytic form of Gabor filters. curtailed using the DFT of Rμ .
All these advantages are acquired without any additional pro- M
cessing. For feature extraction, the images are filtered using the k
FμR S = Rμ(R ) . exp(−j2π n) (11)
analytic form of Gabor filters. M
k =1
r(x, y) = ξ(x, y) ∗ gA (x, y) (6) where FμR S is invariant to rotation and scale changes in the
where r is the response at (x, y), ξ(x, y) is the image, and images. Similarly, the assumption of homogeneousness can be
gA (x, y) is the analytic Gabor filter. extended to calculate the variances of the Gabor filter responses
3) DFT-Based Invariant Features: Assuming that the lo- 2 2 2 2 2
Rσ k n = σ11 σ12 · · · σ1N σ21 · · · σM N (12)
cal texture in an image is spatially homogeneous [41], the Gabor
filter responses for an image region can aggregated by taking which represents a collection of variances of Gabor filter re-
the mean of Gabor filter responses across various scales and sponses in the form of a vector. M is the total number of scales
orientations giving us and N is the total number of orientations for which Gabor filter
responses have been calculated. As in the case of Rμ , the vec-
Rμ k n = μ11 μ12 · · · μ1N μ21 · · · μM N (7) tor representing the variances for a rotated or scale image will
which represents a collection of means of Gabor filter responses be a shifted representation of Rσ k n . Scale and rotation invari-
in the form of a vector. M is the total number of scales and N is ant texture features based on variances (FσR S and FσS R ) similar
the total number of orientations for which Gabor filter responses to those based on means can be constructed. Concatenation of
have been calculated. For an image that is transformed (rotated rotation and scale invariant features gives us
or scaled), its responses will be represented by a filter that is
IGabor = FμS R FμR S FσS R FσR S (13)
transformed by the same amount as that of the image. These
transformations are thus indicated by shifts within the feature where IGabor are the novel features that are invariant to rotation,
vector (for rotation or scale changes) indicated in (7). Rotation scale, and illumination changes in the images. The proposed de-
and scale invariant features can be constructed if these shifts are scriptor is an extension of the texture descriptor that has been
effectively catered for in the feature vectors. Let us now define proposed in [34], where only first-order moments of standard
M
Gabor filter responses are used for the extraction of invariant
Rμ(S ) = Rμ k n (8) features. In contrast, this paper proposed the use of analytic
k =1
Gabor filters for the extraction of features that are invariant to
illumination variations, in addition to scale and rotation invari-
where Rμ (S ) represents the summation of the aggregated filter ance. Additionally, the first- and second-order moments are both
(S )
responses at the each orientation. Rμ is a vector, which is scale used for image description. The analytic Gabor filters exhibit
invariant as the responses across all scales have been summed improved low frequency response making the resulting features
up. However, if an image is rotated by a certain angle, this will be very robust to illumination variations in the images.
166 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 21, NO. 1, JANUARY 2017
B. Color Features
Algorithm 1 Visual Dictionary
For the extraction of color features in the images, we have 1: procedure CODEBOOK
used the CIELUV color space. This selection provides two ad- 2: Input: training images
vantages. 3: for each training image do
1) Distance between two colors in the CIELUV color space 4: Convert image to LUV space
are perceptually uniform. This can help in the construc- 5: Perform homomorphic filtering on ‘L’
tion of decision boundaries by the pattern recognition component
methodologies, which are more consistent with the hu- 6: Initialize codebook C to data mean, Th and
man visual system. 7: p ← 1, where p is number of partitions
2) In the CIELUV color space, the “L” component repre- 8: while E{||W − C(W )||2 > Th do
sents the luminosity and the “UV” components represent 9: Replace μj as {μj }pj=1 + and {μj }pj=1 −
the color (chrominance) of a pixel. Due to this decompo- 10: Set p ← 2p
sition, it is possible to cope with illumination gradients 11: end while
in the images more easily. 12: M ← p2 words
For the medical imaging scenarios, there are two distinct chal- 13: end for
lenges: 1) a tissue analyzed under varying illumination condi- 14: end procedure
tions can create very different visualizations that can potentially
lead to misclassification, and 2) the color features are highly
redundant as most of the organs have very typical colors and code words is obtained giving us a spatial visual vocabulary.
using the full color spaces for image description is not useful. To construct C, the LBG algorithm starts with one code vector
A better color description of the images can be obtained if the (average of the training data) and iteratively bisects the code
color features are indicative of the specific color characteristics vector in power of 2. The distortion monotonically decreases in
of the images. To address these issues, we perform the prepro- the LBG algorithm at each iteration. An important consideration
cessing of illumination component of the images followed by is the use of an appropriate distortion measure and we use the
the extraction of specific color features. mean squared error between the sample and the initial clusters
1) Preprocessing: The luminosity component in the as our measure of distortion, i.e.,
CIELUV space can be used to remove illumination changes
in the images. The illumination gradient which is present in the d = arg minE{||W − C||2 } (16)
C(W )
images is typically composed of low-frequency components.
These components are removed from the images using homo- where W are the data points and C is the distance of each point
morphic filtering [43]. It is based on modeling the image in- from each word in the codebook.
tensity l(x, y) as a combination of the illumination α(x, y) and 3) Histogram of Codebook: Once the codebook is gener-
reflectance r(x, y) components such that ated, every image for which the features have to be calculated is
first preprocessed using conversion to the CIELUV color space
l(x, y) ∝ α(x, y).r(x, y). (14) followed by homomorphic filtering of the luminance compo-
This multiplicative model can be converted to additive model nent. Later, every pixel in the image is compared with all the
by taking the logarithmic transformation on both sides words available in the codebook as follows:
D = [d1 , d2 , . . . dN ] (17)
log(l(x, y)) = log(α(x, y)) + log(r(x, y)). (15)
where each di is an M -dimensional vector indicating the dis-
These additive components are considered separable in the
tance of the ith pixel from the mth word in the codebook. A
frequency domain. The log(α(x, y)) component is composed
histogram of these code words is later generated that is the un-
of low-frequency components (illumination gradient), whereas
derlying color feature that is used for the extraction of adapted
the log(r(x, y)) is composed of high-frequency components.
color features from the images.
An illumination invariant luminance component can thus be
obtained by high pass filtering of log(l(x, y)) followed by an
C. Classification
inverse logarithmic transformation of log(r(x, y)).
2) Vecor Quantization: As discussed previously, uniform In our experiments, we have used supervised classification
quantization of the color space is not expected to work well methods for the characterization of tissues. Supervised methods
given the very specific colors available in the medical images. involve two steps: a training phase in which some part of the data
Therefore, we have resorted to the extraction of adapted color are used to characterize the underlying model that it follows and
histograms for color feature extraction from the images using a testing phase in which the model is applied on unseen data to
a bag-of-words approach. This approach involves the gener- see how well does the model fit the novel data. Many approaches
ation of codebook using the Linde-Buzo Gray (LBG) vector to supervised classification exist and their choice depends on
quantization algorithm. Given that we have a set of input vec- the underlying applications. In this study, we have evaluated
tors W = {w1 , w2 . . . wN } ∈ Rk of N distinct points in a k- the performance of four different classifiers for our application,
dimensional space Rk , a codebook C = {c1 , c2 . . . cM } of M namely, the one nearest neighbor (1NN), naive Bayes (NB),
RIAZ et al.: CONTENT-ADAPTIVE REGION-BASED COLOR TEXTURE DESCRIPTORS FOR MEDICAL IMAGES 167
III. EXPERIMENTS
In this paper, we have used two datasets from very differ-
ent imaging scenarios: vital stained magnification endoscopy
for the identification of gastric cancer and dermoscopy for the
classification of melanoma. Among texture features, we have Fig. 5. Vital stained magnification endoscopy images along with their
used 2DDFT [26], AHT [44], Gabsum [34], LBP [45], and bag- manual annotations. The images suffer severe illumination problems
of-words-based SIFT [46] descriptors for comparison purposes. that are partially solved by the manual annotations. Even within the
annotations, there is uneven illumination in the images.
It is a complementary set of region-based and local visual de-
scriptors. For color feature extraction, we have used standard
LUV (LUV), adapted LUV (ALUV), and adapted illumination the gastric mucosa. The CH images are clinically classified as
invariant (ALUVill) histograms. Due to the limited data that we normal (Group I), precancerous (Group II) or cancerous (Group
have from the two imaging modalities (specifically the cases III) based on the color and texture of the images according to
of cancer), we have used cross validation for our experiments. Dinis–Ribeiro classification proposal for CH images [47] (see
The statistical relevance of our experiments was calculated by Fig. 4). The CH images were acquired by the physicians at the
repeating the cross validation experiments 50 times and per- Portuguese Institute of Oncology (IPO) Porto, Portugal, during
forming a T-test for statistical significance (threshold to 0.05). routine clinical work (Please refer to [3] for more details on
Classification results are obtained using four difference clas- the data). The CH dataset consists of 130 images, which are
sifiers to ensure that obtained results are not classifier model distributed as: 31% (40) Group I, 57% (75) Group II, and 12%
specific. (15) Group III images.
For our classification experiments, we have used only the
manually annotated image regions (see Fig. 5). Feature extrac-
A. Vital Stained Magnification Endoscopy
tion was performed on full images, followed by the selection
One of the potential applications of the proposed descriptor of features from those pixels that lie in the regions representing
is its usage for feature extraction from gastroenterology images. the clinical annotations for the respective images. The objective
This is a complicated problem given that various organs hav- of the classification task is to identify the cancerous tissues,
ing distinct structures result in various perspectives of viewing i.e., classify the images as either normal (Group A) or poten-
the same tissue at various scales from various angles. Addition- tially cancerous (Group B). Consequently, the images in the DR
ally, various imaging modalities focus on complementary visual proposal (see Fig. 4) belonging to Group I should be classi-
characteristics of the images that are useful for diagnosis. Given fied as Normal (Group A), whereas those belonging to Group
the dynamics in the GE scenario due to the organs and imaging II or III should be classified as potentially cancerous (Group
modalities, we have used images from vital stained magnifi- B). The weka data mining tool was used for the classification
cation endoscopy (Chromoendoscopy—CH) for diagnosis of results presented in this paper. All the results were obtained
stomach cancer. In CH, the full visible spectrum of the light is using tenfold cross validation. We have used the overall classi-
used for tissue illumination and tissue enhancement is achieved fication accuracy and area under the receiver operating charac-
using dyes (e.g., methylene blue) for improving the visibility of teristics (ROC) curves for evaluation of the performance of the
168 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 21, NO. 1, JANUARY 2017
TABLE I
OVERALL CLASSIFICATION RESULTS OBTAINED USING THE PROPOSED DESCRIPTOR FOR GASTROENTEROLOGY IMAGES
1NN NB SVM DT
Features TP Rate ROC Area TP Rate ROC area TP Rate ROC Area TP Rate ROC Area
The statistical significance of all texture and color features are calculated against IGabor+ALUVill. All statistically significant
results are marked with “* .”
classifier. However, it should be noted that a fair comparison can are not specific to any classification model. Given that the IGabor
be done using the area under ROC curve given that the dataset and ALUVill show the best performance as texture and color
has unequal instance of normal and cancer/precancer lesions. descriptors, respectively, we use them in combination to assess
The statistical significance of the results is also calculated using if they represent complementary image characteristics for the
the T-test. classification. Our experiments show that the best performance
When only texture features are used for feature extraction, is obtained when a combination of texture and color features
the IGabor features outperform all the other methods that have are used for feature extraction. The results are always better and
been considered in this paper (Table I). We owe this perfor- statistically significant than using only texture or color features.
mance to the illumination invariant nature of IGabor features An important observation is a relatively inferior performance
that is done by using analytic Gabor filters that improve the low of local descriptors (LBP and SIFT) indicating toward the lack
frequency response of the filters. Even when using the simple of discriminative information between the normal and cancer
classifier such as 1NN and NB, the IGabor features show good tissues at higher frequencies (microtexture) owing to the texture
performance elucidating on the fact that the features are linearly noise/distortion in the images [48]–[50]. Our observation was
separable and not model specific. The best performance is ob- complemented by conducting experiments in which the features
tained when IGabor features are used in combination with an obtained only from lower and medium frequency bands (macro-
SVM classifier. However, it should be noted that the difference texture) in the Gabor filter were used as image descriptors (for
in the performance of IGabor from other texture features is not Gabsum, AHT, and 2DDFT) giving similar classification re-
very large. This is because we use only the manually annotated sults. This validated our suspicion that most of the descriptive
image regions for feature extraction and this partly solves the content for cancer lies in lower frequency bands (macrotexture).
problem of complex illumination changes in the images (see
Fig. 5). Additionally, the 2DDFT, AHT, and Gabsum features B. Dermoscopy
are invariant to homogeneous illumination variations in the im- Another medical imaging modality that shares the imaging
ages already but illumination variations within the images are dynamics of gastroenterology is Dermoscopy. Dermoscopy is a
not effectively handled by these methods in contrast to that of clinical procedure that is usually used to identify the presence
IGabor. It should be noted that the IGabor that is an extension of of melanoma in patients. Melanoma is considered as one of the
Gabsum (use of analytic filters improving low frequency re- deadliest forms of skin cancer as an increase in its incidence
sponses and using standard deviation of filter responses in ad- and the consequent mortality has become one of the major con-
dition to means) demonstrates significantly good performance cerns for public health. Researchers have shown that the two
using any classifier showing the superiority of IGabor over visual descriptors, which are imperative in the identification of
Gabsum. melanoma are texture and color (details in [51]). The most vital
When color features are used for classification, LUV features imaging dynamics to be catered for by the visual descriptors
show poor performance in the classification of lesions (Table I). for this imaging scenario are the rotation, scale, and illumi-
When ALUV features are used a significant performance im- nation invariance [32]. The need for rotation invariance arises
provement is obtained, whereas when illumination variations are from the fact that an imaging site can be visualized from arbi-
handled, a further performance improvement is obtained. Our trary angles and this variation should not manifest itself in the
results are consistent with the nature of the vital stained magni- resulting features. Additionally, different images of the same le-
fication endoscopy images where the images have very specific sion acquired from the same equipment and even with the same
color features that essentially encompass a very small range of zooming factor can represent considerably “different” lesions to
colors in the color space. The observation is consistent across all the viewer as it is difficult to guarantee consistent illumination as
the classifiers elucidating on the fact that the proposed features even a mild pressure from the dermoscope can cause significant
RIAZ et al.: CONTENT-ADAPTIVE REGION-BASED COLOR TEXTURE DESCRIPTORS FOR MEDICAL IMAGES 169
TABLE II
OVERALL CLASSIFICATION RESULTS OBTAINED USING THE PROPOSED DESCRIPTOR FOR DERMOSCOPY IMAGES
1NN NB SVM DT
Features TP Rate ROC Area TP Rate ROC Area TP Rate ROC Area TP Rate ROC Area
The statistical significance of all texture and color features are calculated against IGabor+ALUVill. All statistically significant
results are marked with “* .”
illumination, scale, and rotation changes in the images. For color [9] M.-C. Yang, W. K. Moon, Y.-C. Wang, M. S. Bae, C.-S. Huang, J.-H. Chen,
feature extraction, the images are transformed to the LUV color and R.-F. Chang, “Robust texture analysis using multi-resolution gray-
scale invariant features for breast sonographic tumor diagnosis,” IEEE
space to decouple luminance and color. The illumination varia- Trans. Med. Imag., vol. 32, no. 12, pp. 2262–2273, Dec. 2013.
tions are compensated using homomorphic filtering on the “L” [10] H. Kong, M. Gurcan, and K. Belkacem-Boussaid, “Partitioning
channel followed by the calculation of adapted color features. histopathological images: An integrated framework for supervised color-
texture segmentation and cell splitting,” IEEE Trans. Med. Imag., vol. 30,
The features are calculated by using some “representative” im- no. 9, pp. 1661–1677, Sep. 2011.
ages to calculate their most typical color characteristics using [11] M. Sadeghi, T. Lee, D. Mclean, H. Lui, and M. Atkins, “Detection and
LBG algorithm to generate a codebook. Pixels in the remaining analysis of irregular streaks in dermoscopic images of skin lesions,” IEEE
Trans. Med. Imag., vol. 32, no. 5, pp. 849–861, May 2013.
images are quantized to the words available in the codebook [12] Z. HongQing, “Segmentation of blood vessels in retinal images using 2D
followed by the generation of a histogram of codebooks that is entropies of gray level-gradient cooccurrence matrix,” in Proc. IEEE Int.
later used for classification of the images. Conf. Acoust. Speech Signal Process., vol. 3, May 2004, pp. iii-509–iii-
512.
The proposed feature set is used for classifying images from [13] S. Park, B. Kim, J. Lee, J. M. Goo, and Y.-G. Shin, “Ggo nodule volume-
two distinct imaging modalities, carrying very different objec- preserving nonrigid lung registration using GLCM texture analysis,” IEEE
tives but sharing the imaging dynamics: chromoendoscopy and Trans. Biomed. Eng., vol. 58, no. 10, pp. 2885–2894, Oct. 2011.
[14] F. van der Lijn, M. de Bruijne, S. Klein, T. den Heijer, Y. Hoogendam,
dermoscopy. Our experiments have demonstrated the superiority A. van der Lugt, M. M. B. Breteler, and W. Niessen, “Automated brain
of both texture and color features over other methods considered structure segmentation based on atlas registration and appearance models,”
in this paper. We have also used a combination of both texture IEEE Trans. Med. Imag., vol. 31, no. 2, pp. 276–286, Feb. 2012.
[15] R. Bansal, L. Staib, D. Xu, H. Zhu, and B. Peterson, “Statistical analyses
and color features and shown that the proposed features outper- of brain surfaces using Gaussian random fields on 2-D manifolds,” IEEE
form the other methods that have been considered in this paper, Trans. Med. Imag., vol. 26, no. 1, pp. 46–57, Jan. 2007.
elucidating on the generic nature of the proposed descriptors. [16] S. Dua, U. Acharya, P. Chowriappa, and S. Sree, “Wavelet-based en-
ergy features for glaucomatous image classification,” IEEE Trans. Inform.
In addition to rotation and scale invariance, the descriptors that Technol. Biomed., vol. 16, no. 1, pp. 80–87, Jan. 2012.
compensate for illumination variations in the images tend to [17] D. Ai, X. Han, X. Ruan, and Y. wei Chen, “Adaptive color independent
work better. Last but not the least, our experiments demonstrate components based sift descriptors for image classification,” in Proc. 20th
Int. Conf. Pattern Recog., Aug. 2010, pp. 2436–2439.
that the proposed features are not specific to a classification [18] T. Takahashi, T. Kawano, K. Ito, T. Aoki, and S. Kondo, “Performance
model as they almost always perform better than other feature evaluation of a geometric correction method for multi-projector display
extraction methods irrespective of the classifier used. using sift and phase-only correlation,” in Proc. 17th IEEE Int. Conf. Image
Process., Sep. 2010, pp. 1189–1192.
[19] W. Lu and M. Wu, “Multimedia forensic hash based on visual words,” in
ACKNOWLEDGMENT Proc. 17th IEEE Int. Conf. Image Process., Sep. 2010, pp. 989–992.
[20] A. Sousa, “Analysis of color and texture features of vital stained magnifi-
The authors would like to thank Portuguese Institute of On- cation endoscopy images for computer assisted diagnosis of precancerous
cology, Porto, Portugal, for their support during the manual and cancer lesions,” Ph.D. Thesis, Faculdade de Engenharia da Universi-
dade do Porto, Porto, Portugal, 2008.
annotation of the dataset. [21] T. Ojala, M. Pietikainen, and D. Harwood, “A comparative study of tex-
ture measures with classification based on feature distributions,” Pattern
REFERENCES Recog., vol. 29, no. 1, pp. 51–59, Jan. 1996.
[22] F. Riaz, M.-D. Ribeiro, P. Pimentel-Nunes, and M. Tavares Coimbra,
[1] J. Y. Choi, Y. M. Ro, and K. N. Plataniotis, “Color local texture features “Integral scale histogram local binary patterns for classification of narrow-
for color face recognition,” IEEE Trans. Image Process., vol. 21, no. 3, band gastroenterology images,” in Proc. IEEE 35th Annu. Int. Conf. Eng.
pp. 1366–1380, Mar. 2012. Med. Biol. Soc., Jul. 2013, pp. 3714–3717.
[2] Y. Shen, P. Guturu, and B. P. Buckles, “Wireless capsule endoscopy video [23] W.-H. Liao, “Region description using extended local ternary patterns,”
segmentation using an unsupervised learning approach based on proba- in Proc. IEEE 20th Int. Conf., 2010, pp. 1003–1006.
bilistic latent semantic analysis with scale invariant features,” IEEE Trans. [24] L. Nanni, A. Lumini, and S. Brahnam, “Local binary patterns variants
Inform. Technol. Biomed., vol. 16, no. 1, pp. 98–105, Jan. 2012. as texture descriptors for medical image analysis,” Artif. Intell. Med.,
[3] F. Riaz, F. Silva, M. Ribeiro, and M. Coimbra, “Invariant Gabor texture vol. 49, no. 2, pp. 117–125, 2010.
descriptors for classification of gastroenterology images,” IEEE Trans. [25] N. Dalal and B. Triggs, “Histograms of oriented gradients for human
Biomed. Eng., vol. 59, no. 10, pp. 2893–2904, Oct. 2012. detection,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog.,
[4] S. Atasoy, D. Mateus, A. Meining, G.-Z. Yang, and N. Navab, “Endoscopic vol. 1, Jun. 2005, pp. 886–893.
video manifolds for targeted optical biopsy,” IEEE Trans. Med. Imag., [26] F. Riaz, A. Hassan, S. Rehman, and U. Qamar, “Texture classification
vol. 31, no. 3, pp. 637–653, Mar. 2012. using rotation- and scale-invariant Gabor texture features,” IEEE Signal
[5] S. H. Raza, R. M. Parry, R. A. Moffitt, A. N. Young, and M. D. Wang, Process. Lett., vol. 20, no. 6, pp. 607–610, 2013.
“An analysis of scale and rotation invariance in the bag-of-features method [27] S. Jayaram, S. Schmugge, M. Shin, and L. V. Tsap, “Effect of colorspace
for histopathological image classification,” in Medical Image Computing transformation, the illuminance component, and color modeling on skin
and Computer-Assisted Intervention. Berlin, Germany: Springer, 2011, detection,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog.,
pp. 66–74. vol. 2, Jun. 2004, pp. II-813–II-818.
[6] J. C. Caicedo, A. Cruz, and F. A. Gonzalez, “Histopathology image clas- [28] A. Ferman, A. Tekalp, and R. Mehrotra, “Robust color histogram descrip-
sification using bag of features and kernel functions,” in Artificial Intelli- tors for video segment retrieval and identification,” IEEE Trans. Image
gence in Medicine. New York, NY, USA: Springer, 2009, pp. 126–135. Process., vol. 11, no. 5, pp. 497–508, May 2002.
[7] A. Sousa, M. Dinis-Ribeiro, M. Areia, and M. Coimbra, “Identifying can- [29] W. Wang and M. Hua, “Extracting dominant textures in real time with
cer regions in vital-stained magnification endoscopy images using adapted multi-scale hue-saturation-intensity histograms,” Image IEEE Trans. Im-
color histograms,” in Proc. IEEE 16th Int. Conf. Image Process., Nov. age Process., vol. 22, no. 11, pp. 4237–4248, Nov. 2013.
2009, pp. 681–684. [30] B. Manjunath, J.-R. Ohm, V. Vasudevan, and A. Yamada, “Color and
[8] B. Zhang, B. Kumar, and D. Zhang, “Detecting diabetes mellitus and texture descriptors,” IEEE Trans. Circuits Syst. Video Technol., vol. 11,
nonproliferative diabetic retinopathy using tongue color, texture, and no. 6, pp. 703–715, Jun. 2001.
geometry features,” IEEE Trans. Biomed. Eng., vol. 61, no. 2, pp. 491–501, [31] C.-H. Yao and S.-Y. Chen, “Retrieval of translated, rotated and scaled
Feb. 2014. color textures,” Pattern Recog., vol. 36, no. 4, pp. 913–929, 2003.
RIAZ et al.: CONTENT-ADAPTIVE REGION-BASED COLOR TEXTURE DESCRIPTORS FOR MEDICAL IMAGES 171
[32] H. Zhou, M. Chen, and J. Rehg, “Dermoscopic interest point detector and [44] F. Riaz, “Assisted analysis of gastroenterology images using computer
descriptor,” in Proc. IEEE Int. Symp. Biomed. Imag., Nano Macro, 2009, vision methodologies,” Ph.D. dissertation, Faculdade de Ciencias da Uni-
pp. 1318–1321. versidade do Porto, Porto, Portugal, 2012.
[33] F. Peruch, F. Bogo, M. Bonazza, V.-M. Cappelleri, and E. Peserico, [45] T. Ojala, M. Pietikäinen, and T. Mäenpää, “Multiresolution gray-scale and
“Simpler, faster, more accurate melanocytic lesion segmentation through rotation invariant texture classification with local binary patterns,” IEEE
MEDS,” IEEE Trans. Biomed. Eng., vol. 61, no. 2, pp. 557–565, Feb. Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971–987, Jul. 2002.
2014. [46] M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi, “Describ-
[34] F. Riaz, M. Dinis-Ribeiro, P. Pimentel-Nunes, and M. T. Coimbra, “A ing textures in the wild,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog.,
DFT based rotation and scale invariant Gabor texture descriptor and its 2014, pp. 3606–3613.
application to gastroenterology,” in Proc. Int. Conf. Image Process., 2013, [47] M. D. Ribeiro, “Clinical, endoscopic and laboratorial assessment of pa-
pp. 1443–1446. tients with associated lesions to gastric adenocarcinoma,” Ph.D. disserta-
[35] G. Haley and B. Manjunath, “Rotation-invariant texture classification us- tion, Faculdade de Medicina da Universidade do Porto, Porto, Portugal,
ing a complete space-frequency model,” IEEE Trans. Image Process., vol. 2005.
8, no. 2, pp. 255–269, Feb. 1999. [48] G. Bao, Y. Ye, U. Khan, X. Zheng, and K. Pahlavan, “Modeling of the
[36] D. J. Field, “Relations between the statistics of natural images and the movement of the endoscopy capsule inside GI tract based on the captured
response properties of cortical cells,” IEE J. Radio Commun. Eng., vol. 4, endoscopic images,” presented at the Int. Conf. Modeling, Simulation,
no. 12, 1987. Visualization Methods, Las Vegas, Las Vegas, NV, USA, 2012.
[37] J. G. Daugman, “High confidence visual recognition of persons by a test [49] D. Koppel, Y.-F. Wang, and H. Lee, “Image-based rendering and model-
of statistical independence,” IEEE Trans. Pattern Anal. Mach. Intell., vol. ing in video-endoscopy,” in Proc. IEEE Int. Symp. Biomed. Imag, Nano
15, no. 11, pp. 1148–1161, Nov. 1993. Macro, 2004, pp. 269–272.
[38] P. Wu, B. S. Manjunath, S. Newsam, and H. D. Shin, “A texture descriptor [50] A. Vécsei, G. Amann, S. Hegenbart, M. Liedlgruber, and A. Uhl, “Auto-
for browsing and similarity retrieval,” J. Signal Process.: Image Commun., mated marsh-like classification of celiac disease in children using local
vol. 16, no. 1, pp. 33–43, 2000. texture operators,” Comput. Biol. Med., vol. 41, no. 6, pp. 313–325, 2011.
[39] R. Porter and N. Canagarajah, “Robust rotation-invariant texture classifi- [51] I. Maglogiannis and C. N. Doukas, “Overview of advanced computer
cation: Wavelet, Gabor filter and GMRF based schemes,” IEE Proc. Vis. vision systems for skin lesions characterization,” IEEE Trans. Inf. Technol.
Image Signal Process., vol. 144, no. 3, pp. 180–188, Jun. 1997. Biomed., vol. 13, no. 5, pp. 721–733, Sep. 2009.
[40] J. K. Kmrinen, “Feature extraction using Gabor filters,” Ph.D. disser- [52] C. Barata, M. Ruela, M. Francisco, T. Mendonca, and J. Marques, “Two
tation, Lappeenranta University of Technology, Lappeenranta, Finland, systems for the detection of melanomas in dermoscopy images using
2003. texture and color features,” IEEE Syst. J., vol. 8, no. 3, pp. 965–979, Sep.
[41] B. S. Manjunath and W. Y. Ma, “Texture features for browsing and retrieval 2013.
of image data,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 8, [53] C. Barata, J. Marques, and J. Rozeira, “A system for the detection of
pp. 837–842, Aug. 1996. pigment network in dermoscopy images using directional filters,” IEEE
[42] A. V. Oppenheim, R. W. Schafer, and J. R. Buck, Discrete-Time Signal Trans. Biomed. Eng., vol. 59, no. 10, pp. 2744–2754, Oct. 2012.
Processing. Englewood Cliffs, NJ, USA: Prentice-Hall, 1999.
[43] T. Schuchert, T. Aach, and H. Scharr, “Range flow in varying illumination:
Algorithms and comparisons,” IEEE Trans. Pattern Anal. Mach. Intell., Authors’ photographs and biographies not available at the time of
vol. 32, no. 9, pp. 1646–1658, Sep. 2010. publication.