CLASSIFICATION OF ADRENAL LESIONS THROUGH SPATIAL BAYESIAN MODELING
OF GLCM
X. Li1,4 , M. Guindani2 , C. S. Ng3 , B. P. Hobbs1
1,4
Department of Biostatistics, The University of Texas MD Anderson Cancer Center
2
Department of Statistics, University of California, Irvine
3
Department of Diagnostic Radiology,
The University of Texas MD Anderson Cancer Center
4
Department of Biostatistics, The University of Texas Health Science Center at Houston
ABSTRACT
Radiomics, an emerging field of quantitative imaging, en-
compasses a broad class of analytical techniques. Recent lit-
erature have interrogated associations between quantitatively
derived GLCM-based texture features and clinical/pathology
information using machine learning algorithms in many can-
cer settings, but often fail to elucidate the predictive power
of these features. Moreover, for many cancers characterized
by complex histopathological profiles, such as adrenocorti-
cal carcinoma, reducing the multivariate functional structure
of GLCM to a set of summary statistics is potentially reduc-
tive, masking the patterns that distinguish malignancy from
benignity. We develop a Bayesian probabilistic framework
for predictive classification of lesion types, based on the entire
GLCM. Our method, which uses a spatial Gaussian random
field to model dependencies among neighboring cells of the
GLCMs, was applied in a cancer detection context to discrim-
inant malignant from benign adrenal lesions using GLCMs
arising from non-contrast CT scans. Our method is shown to
yield improved predictive power both in simulations as well
as the adrenal CT application when compared to state-of-the-
art diagnostic algorithms that use GLCM derived features.
Index Terms— Bayesian modeling, discriminant anal-
ysis, Gaussian Markov Random Field, Gray Level Co-
Occurrence Matrix, Prediction accuracy, lesion classification.
1. INTRODUCTION
Despite recent advances in available scanning instruments
and reconstruction techniques, tissue characterization re-
mains a challenge in many cancer diagnostic settings as
patients present heterogeneity in the complex internal micro-
environment of targeted lesions. Diagnosing adrenocortical
carcinoma poses a particular challenge as benign and malig-
nant adrenal lesions are difficult to distinguish on the basis
of CT imaging at early stages [1, 2]. Early identification of
adrenal masses is considered pivotal for selecting an appro-
priate treatment regimen as well as determining whether a
978-1-5090-1172-8/17/$31.00 ©2017 IEEE 147
patient’s disease prognosis entails distant migration. As one
of the earliest techniques used for image texture analysis,
Gray Level Co-occurrence Matrix (GLCM), also known as
the gray level spatial dependence matrix, is a matrix defined
over an image to be the distribution of co-occurring gray-
th
scale pixel values at a given offset and angle [3]. The (i, j)
entry of GLCM represents how often a pixel with gray level
i occurs either horizontally, vertically or diagonally to adja-
cent pixels with gray level j. Figure 1 depicts enhancement
patterns on the Hounsfield unit domain of ROIs comprised
of adrenal lesions as well as their corresponding gray level
co-occurrence matrices. The top image is fairly representa-
tive of cases of malignancies within study which tended to
reflect a high degree of co-occurrence at higher gray levels.
Conversely, the bottom image tends to be indicative of benign
lesions. The central image, reflecting high co-occurrence at
mid gray levels, depicts an intermediate case.
There has been substantial work aiming to establish meth-
ods for differentiating among tissue types on the basis of such
radiomics features. For example, Harshvardhan et al.,2016
recently proposed a logistic regression classifier that uses
GLCM features as predictors for early detection and recog-
nition of glaucoma by ocular thermal images [4]. Zulpe et
al.,2012 used artificial neural network techniques on GLCM
texture features to classify brain tumor MR images, achieving
a success rate of 97.5% [5]. Hitam et al.,2004 have also used
artificial neural networks for classifying Brodatzs album im-
ages [6]. More recently, Singh et al.,2016 found that random
forests yield the best classification performance compared to
other classifiers based on the GLCM feature extraction in a
breast cancer study [7].
The aforementioned approaches should be considered
limited, however, as reducing the multivariate functional
structure inherent to GLCM to a set of summary statistics (or
GLCM-derived features) results in information loss. More-
over, many of textural features are sensitive to rotations of
the image and/or GLCM, limiting robustness and potentially
masking patterns that may be important to consider when

Fig. 1: Illustrative patterns of GLCM in adrenal lesions from
CT imaging for 3 representative subjects, pixel-level ROIs on the
left, GLCMs on the right.
discerning the extent and pathogenesis of a particular disease.
To overcome this limitation, we developed a model-based
Bayesian probabilistic framework for classification of GLCM
objects. By introducing a Gaussian process to account for
spatial dependencies among the GLCM entries, we derive a
probability model to characterize distributions of the space
of the entire normalized GLCM as a multivariate response
surface. The method, which we refer to as a Bayesian Spa-
tial Gaussian Process Classifier (BSGC), was applied in a
cancer detection context to discriminate malignant and be-
nign adrenal lesions from enhancement patterns observed
from CT. When compared to currently employed analytic
approaches which use radiomics-based texture features, the
proposed method is shown to effectuate substantial improve-
ments in classification accuracy when compared to prediction
based on the methods of support vector Machine, logistic
regression, artificial neural network, and random forest. The
methodology offers insights into the manner in which imag-
ing data may be better utilized to identify complex patterns
that characterize the intrinsic heterogeneity observed in tumor
pathology.
2. HIERARCHICAL STATISTICAL FRAMEWORK
This section describes how spatial modeling techniques,
namely Gaussian Markov Random Field (GMRF) priors,
can be employed in a hierarchical statistical model to capture
patterns of dependence within the GLCM structure, and yield
probability-based measures for prediction and classification.
148
2.1. Model formulation
Let i denote a patient index, (s1 , s2 , ..., sn ) the finite collec-
tion of GLCM location indices spanning the unique cells of
0
the symmetric structure; yi = (yi (s1 ) , yi (s2 ) , ..., yi (sn )) ,
values of normalized GLCM counts for the ith patient; β =
(β1 , β2 , ..., βp ), the p- dimensional coefficient vector corre-
sponding to unique gray-levels of the GLCM; xi the n × p
dimensional patient-level design matrix. A linear model for
the GLCM of patient i can be expressed as:
yi = x i β + η i + i ,
0
where ηi = (ηi (s1 ) , ηi (s2 ) , ..., ηi (sn )) defines a collec-
tion of spatial random effects capturing spatial dependencies
among the GLCM cells. The vector i denotes random noise
(arising from unknown sources of variation), which is as-
sumed an n-dimensional normal distribution with zero mean
and diagonal covariance matrix with common variance τ .
2.2. Gaussian Markov Random Field (GMRF)
GMRF models have been studied extensively as tools for
characterizing spatial variation of observable lattice-type data
as well as unobservable latent variables in hierarchical mod-
els. We define a GMRF by specifying a set of full conditional
distributions for ηi :
ηi (sk )|ηi (s1 ), .., ηi (sk−1 ), ηi (sk+1 ), ..., ηi (sn )
 
X Wkd ηi (sd ) 1
∼N , ,
Bk τη Bk
d6=k
where Bk is a normalizing factor; τη is a precision parameter
and Wkd represents a “weight” characterizing the strength
of dependence between lattice elements k and d, k 6= d.
Through the Brook’s Lemma [8], the joint distribution is
0
f (ηi ) ∝ exp(− τ2 ηi (B − W )ηi ). Specification of W as
the adjacency matrix with Wkd indicating whether lattice el-
ements k and d are adjacent neighbors and B as a diagonal
matrix with kth entry Bk = Dk + q, where Dk denotes the
number of neighbors for lattice k, and q > 0 a diagonal offset
term, ensures a proper joint multivariate normal distribution
with (B −W ) symmetric, positive definite with non-negative
off-diagonal entries [9].
2.3. Bayesian Hierarchical model and Posterior inference
To complete the conditional sampling model, given specifica-
tion of the model parameters and latent, spatially correlated
random effects, the observed empirical rates for patient i can
assume the following Gaussian distribution:
yi | β, ηi , τ ∼ M V N yi | xi β + ηi , τ−1 I


Prior specification for ηi follows from the GMRF formula-
tion,
ηi | τη , d ∼ M V N ηi | 0, τη−1 (D + diag(d) − W ) .


Hierarchical model specification is complete with prior distri- 3. SIMULATION STUDY
butions for the parameters:
β ∼ M V N (β | m0 , Σ0 ) , Performance for the proposed BSGC classifier was evaluated
through simulation and compared to four popular approaches
τ ∼ Gamma (τ | a, d) , in the GLCM texture analysis literature. Specifically, classifi-
τη ∼ Gamma (τη | b, g) , and cation performance based on GLCM derived texture features
d ∼ Gamma (d | e, f ) . (such as contrast, homogeneity, energy and entropy) using
the methods of logistic regression, support vector machine
A vague prior for β e.g. Σ0 = 1000 ∗ I and m0 = (SVM), artificial neural network (ANN) and random forest
0
(0, 0, ..., 0) , was used in our study to promote maximal were compared to the BSGC approach [4, 7, 10–14]. To for-
data learning. Following recommendations in [8], we set mulate an appropriate sampling model capable of effectuating
a = d = 0.001, b = g = 0.1, and chose e = 1, f = 0.0001. characterizations of spatial patterns of GLCMs that were ev-
Bayesian posterior inference was conducted through Markov ident in our diagnostic cancer study of adrenal lesions, we
chain Monte Carlo (MCMC) sampling and post-MCMC considered GLCMs constructed from 8 gray-levels with nor-
computation of posterior summary statistics. malized element-wise probability densities arising from a bi-
0
variate normal distribution
 with µ = (2 + c, 6 − c) and
2.4. Predictive Discriminant Analysis 10 −0.5
Σ = . A final smoothed event rate surface
−0.5 10
Unlike regression-based classifiers, which rely on linear was obtained for GLCM simulation by smoothing over the
predictors, in the presence of col-linearity the Bayesian Gaussian-derived empirical rate surface, calculated in propor-
paradigm facilitates class prediction through probabilistic- tion to the number of generated points in each grid with re-
based measures that characterize the distributions of inter- spect to the total number of generated points of the entire grid
dependent observable predictors under candidate classes. surface. In addition, to effectuate the discrete GLCM space,
Bayesian discriminant-type predictive classifiers are fully we scaled the rate surface by a random integer that is sampled
specified through the predictive density of an observable pre- from 50 to 5000, characterizing image size and rounded the
dictor and the prior probability of each class. Denote the values to nearest integers, with a distribution obtained from
observed GLCM of a new, heretofore unclassified object the empirical distribution of image sizes observed in our case
by yN +1 with potential covariates xN +1 . Additionally, let study. Simulation scenarios were constructed by considering
c = {c1 , c2 , ..., ch } denote the set of all possible classes distributions of GLCMs generated under different choices of
to which object yN +1 could be assigned. The classifica- mean shift parameter c.
tion probability for any class configuration follows from our Figure 2 depicts the mean rate surface used to simulate
model specification as proportional to the product of the GLCM-derived patterns with c = 0, 1, 2, 3.5, respectively.
prior probability of class ck and the value of the conditional Simulation scenarios were formulated to shift the peak count
predictive distribution for yN +1 under class ck , cells from upper left toward lower right along the diagonal,
P (c = ck | yN +1 , y, xN +1 , x) = reflecting varying patterns pertaining to the spatial clustering
of dense versus non-dense tissues observeable in our study of
P (yN +1 | y, x, xN +1 , c = ck ) P (c = ck ) (1) adrenal lesion using GLCMs based on CT. Class c = 0 is con-
Pk ,
l=1 P (yN +1 | y, x, xN +1 , c = cl ) P (c = cl ) sidered the reference group to which every class is compared.
where Simulated results of classification accuracy under LOOCV
are displayed in Figure 3, where BSGC is shown to outper-
P (yN +1 | y, x, xN +1 , c = ck ) = form competing methods which attempt to use information
Z
P yN +1 | ηN ck ck ck ck ck ck ck

contained in GLCMs by reducing the multivariate functional
+1 , y , x , xN +1 , β , τ , τη , ρ
structure to a set of summary statistics (i.e., GLCM-derived
ck
P (β ck , ηN ck ck
+1 , τ , τη , ρ
ck
| y ck , xck ) features). In fact, all feature-based classification approaches
ck failed to yield monotonic trends as c increased in our simu-
d β ck , ηN ck ck ck


+1 , τ , τη , ρ ,
lation study; on the other hand, when c becomes larger, the
is computed by averaging over posterior samples obtained patterns in the GLCM space are more separable from the ref-
from MCMC. The prior values for P (c = ck ) can be specified erence group, resulting in enhanced discrimination accuracy
based on the frequency of class ck observed in a training sam- for the BSGC method, resulting in monotonic behavior. Thus,
ple or through other available information. Class labels can be when functional structure is presented in the GLCM, our pro-
assigned in accordance with the highest class probability (1), posed BSGC method was more robust than the feature-based
which is used as the basis for evaluation in our simulation and classification approaches. Non-monotonic trends in per-
case studies using a leave-one-out cross-validation (LOOCV) formance where apparent for feature-based classifiers, which
strategy. tended to achieve best performance near c = 2, corresponding

149
Fig. 2: Simulation scenarios for comparing GLCM-based
classifiers. Each figure depicts the assumed mean rate sur-
face used to simulate GLCM-derived patterns with different
choices of c. The upper-left is generated with c = 0, the
upper-right is generated with c = 1, the lower-left is gener-
ated with c = 2, the lower-right is generated with c = 3.5
to GLCMs with mean peak counts of occurrence at moderate
gray levels. Considering the symmetry inherent to GLCMs
that characterize all directions, these methods resulted in di-
minished predictive performance for increasing values of c
due to the fact that 180◦ counterclockwise rotations produce
similar GLCM-derived summary statistics thereby attenuat-
ing the true extent of data-signal between classes of c = 0 and
c = 4. By way of contrast, the BSGC classifier characterizes
the positional information of peak counts via the adjacency
matrix which results in robustness to shifts in the mean count
surface, enhancing separability with increasing c.
4. CASE STUDY
This section describes the application of the proposed Bayesian
spatial Gaussian classifier to the diagnostic study of adrenal
lesions. The study cohort consisted of 230 adrenal lesions,
in 223 patients. Of the 230 lesions, 121 were benign and
109 were malignant. For each lesion, the index CT image
that contained the maximal axial cross-sectional area of the
adrenal mass was used for analysis based GLCM. A region
of interest (ROI) was carefully drawn free-hand, with an
electronic cursor and mouse, around the periphery of the
adrenal lesion on the CT image which contained the maximal
cross-sectional area. The classification accuracy of compet-
ing methods, reported in Table 1, was assessed by LOOCV.
150
Fig. 3: Simulation results. Classification accuracy obtained
from prediction under LOOCV.
Specifically, to obtain the summary statistics reported in Ta-
ble 1, LOOCV was implemented as follows: at each step,
the observables (either GLCM or GLCM-derived features)
from a single lesion were omitted from the training set, while
posterior inference was implemented using data from the re-
maining lesions. Thereafter, a class (benign or malignant)
is predicted for ROIs contributed by the omitted patient for
each method and compared with each region’s true known
status. Effectuating an improvement in accuracy of >70%,
the BSGC method outperformed the existing textural feature-
based classifiers.
Table 1: Case study results. Classification accuracy
obtained from prediction under LOOCV
Logistic Random
BSGC SVM ANN
Regression Forest
Accuracy 0.80 0.46 0.47 0.47 0.45
5. REFERENCES
[1] E. Altinmakas, B. P. Hobbs, H. Ye, E. G. Grubbs, N. D.
Perrier, V. G. Prieto, J. E. Lee, and C. S. Ng, “Diagnostic
performance of 18-F-FDG-PET–CT in adrenal lesions
using histopathology as reference standard,” Abdominal
Radiology, pp. 1–8, 2016.
[2] K. N. Wanis and R. Kanthan, “Diagnostic and prognos-
tic features in adrenocortical carcinoma: a single insti-
tution case series and review of the literature,” World
 Journal of Surgical Oncology, vol. 13, no. 1, pp. 117, [14] M. Pawar, D. K. Sharma, and R. Giri, “Multiclass skin
2015. disease classification using Neural Network,” Inter-
national Journal of Computer Science and Information
[3] R. M. Haralick, K. Shanmugam, et al., “Textural fea- Technology Research, vol. 2, no. 4, pp. 189–193, 2014.
tures for image classification,” IEEE Transactions on
systems, man, and cybernetics, , no. 6, pp. 610–621,
1973.

[4] G. Harshvardhan, N. Venkateswaran, and

N. Padmapriya, “Assessment of Glaucoma with
ocular thermal images using GLCM techniques and
Logistic Regression classifier,” in Wireless Com-
munications, Signal Processing and Networking
(WiSPNET), International Conference on. IEEE, 2016,
pp. 1534–1537.

[5] N. Zulpe and V. Pawar, “GLCM textural features for

brain tumor classification,” 2012.

[6] M. S. Hitam, M. Y. H. Muslan, M. M. Deris, and

M. Y. M. Saman, “Image texture classification using
gray level co-occurrence matrix and neural network,”
matrix, vol. 1, no. 2, pp. 3–4, 2004.

[7] V. P. Singh, A. Srivastava, D. Kulshreshtha, A. Chaud-

hary, and R. Srivastava, “Mammogram Classification
Using Selected GLCM Features and Random Forest
Classifier,” International Journal of Computer Science
and Information Security, vol. 14, no. 6, pp. 82, 2016.

[8] S. Banerjee, B. P. Carlin, and A. E. Gelfand, Hierarchi-

cal modeling and analysis for spatial data, Crc Press,
2014.

[9] R. W. Cottle, J.-S. Pang, and R. E. Stone, The linear

complementarity problem, vol. 60, Siam, 2009.

[10] I. Hassan, A. Kotrotsou, A. S. Bakhtiari, G. A. Thomas,

J. S. Weinberg, A. J. Kumar, R. Sawaya, M. M. Luedi,
P. O. Zinn, and R. R. Colen, “Radiomic Texture Anal-
ysis Mapping Predicts Areas of True Functional MRI
Activity,” Scientific reports, vol. 6, 2016.

[11] G. Preethi and V. Sornagopal, “MRI image classifica-

tion using GLCM texture features,” in Green Comput-
ing Communication and Electrical Engineering (ICGC-
CEE), 2014 International Conference on. IEEE, 2014,
pp. 1–6.

[12] S. Jafarpour, Z. Sedghi, and M. C. Amirani, “A robust

brain MRI classification with GLCM features,” Int. J.
Comput. Appl, vol. 37, no. 12, pp. 1–5, 2012.

[13] R. Kumari, “SVM Classification an Approach on De-

tecting Abnormality in Brain MRI Images,” Interna-
tional Journal of Engineering Research and Applica-
tions, vol. 3, no. 4, pp. 1686–1690, 2013.

151