POLARIMETRIC SAR IMAGES CLASSIFICATION USING DEEP BELIEF

NETWORKS WITH LEARNING FEATURES

1 1 1 1 1
Biao Hou Xiaohuan Luo Shuang Wang Licheng Jiao Xiangrong Zhang
1
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China
Xidian University, Xi'an 710071, P. R. China
(e-mail: avcodec@hotmail.com)

ABSTRACT machine (SVM) with multiple-component scattering model

A novel polarimetric synthetic aperture radar (PolSAR)
image classification method based on Deep Belief Networks
(DBNs) is proposed in this paper. First, the coherency
matrix data are converted to a 9-dimentional data. Second,
many patches are randomly selected from each dimension in
the 9-dimentional data, and many filters can be obtained
from a Restricted Boltzmann Machine (RBM) trained by
using these patches. Thus we can get the features for each
pixel from each dimension in the 9-dimentional space.
Finally, the learned features and the elements of coherent
matrix are combined to train a 3-layers DBNs for PolSAR
image classification. Experimental results show that the
proposed method is efficient and effective for PolSAR
image classification.
Index Terms—Deep Belief Networks, RBM, PolSAR,
Image classification
1. INTRODUCTION
Synthetic aperture radar (SAR) plays an important role in
geoscience and remote sensing image classification.
Compared with single-polarization SAR, polarimetric
synthetic aperture radar (PolSAR) can obtain richer
information with polarimetric measurements.
There are mainly two types of methods for PolSAR image
classification: unsupervised and supervised methods. In the
past few decades, many supervised methods for PolSAR
image classification have been proposed. As single-look
PolSAR data represented by a Sinclair matrix [1] is a
Gaussian distribution, J. S. Lee et al. [2] proposed the
Wishart maximum likelihood (WML) method which used
the Cloude and Pottier decomposition for multilook PolSAR
data. More recently, many machine learning methods have
been used for PolSAR image classification. Y. D. Zhang et
al. [3] used a neural network to classify PolSAR images, in
which the feature sets consist of texture features and Cloude
and Pottier decomposition parameters. M. Zhang et al. [4]
proposed a classification method based on support vector
(MCSM) and texture features.
Although these aforementioned methods have achieved
good results, these methods still have some drawbacks. Most
of methods only deal with each pixel independently and
ignore the influence of the information in the local
neighborhood. In addition, these methods can not extract
feature directly from raw data and need to design many
complicated feature extraction algorithms. To solve the
problems mentioned above, a new classification method
based on deep belief networks (DBNs) [5] is proposed in
this paper.
2. PROPOSED METHOD
A new method for PolSAR image classification based on
DBNs is proposed in this paper. First, we convert the
original coherency matrix data into a 9-dimensional data,
Second, many patches are randomly selected from each
dimension in the data, and many filters are obtained from a
RBM trained by these patches. Features of each pixel can be
automatically learned by these filters from the 9-dimensional
data, and these learned features are combined with the
original coherent matrix elements to form the final feature
sets. Finally, the feature sets are used to train a 3-layer
DBNs for PolSAR image classification.
2.1. Forming a 9-dimensional data
According to the method in [1], a multilook PolSAR data
can be expressed in the form of a covariance or coherency
matrix. Particularly, the coherency matrix can be defined as
follows:
T11 T12 T13 
(1)
T 
T21 T22 T23 


T31 T 32 T33 

As the coherency matrix T is a symmetric complex matrix
and the diagonal elements are real-valued, we define a new
vector to represent the coherency matrix T as follows,
V  [T11 , T22 , T33 , real (T12 ), imag (T12 ), real (T13 ),  (2)
imag (T13 ), real (T23 ), imag (T23 )]

978-1-4799-7929-5/15/$31.00 ©2015 IEEE 2366 IGARSS 2015

where real  and imag  represents the real part of and
the imaginary part of the complex respectively.
As the values of the elements in the original coherency
matrix has not been changed in the vector V , the inherent
characteristics of physical scattering mechanisms can be
effectively been remained. When the coherency matrixes of
each pixel in the PolSAR image are converted into the
vector V , we can obtain a 9-dimentional data.
2.2. Learning features using RBM
Each dimension in the 9-dimentional data mentioned above
can be regarded as a gray image and this gray image in each
dimension contains rich structural information for a pixel in
a region. To capture this structure, many image patches are
randomly sampled from each dimension in the 9-dimentional
data, and then these image patches are converted in column-
major order to a vector for training a RBM. Therefore, the
parameters of the RBM are obtained and we can get many
filters from the RBM.
For each pixel i in a PolSAR image, we can get a patch
 i , j centered in pixel i for each dimension in the 9-
dimentional data, and then the region is converted in
column-major order to a 9-dimensional vector. The feature
in the jth dimension can be learned as follows,
fi , j   (Vi , j  W  b) j  1, 29 (3)
where f i , j represents the feature for the pixel i in dimension
j .  () is the sigmoid function. W and b are the parameters
of the RBM.
Considering the scattering properties, the raw coherency
matrix elements represented by the vector V are combined
with learned features fi , j ( j  1, 29) to form the final
feature sets.
2.3. Training a DBN for PolSAR image classification
Once the final feature sets are obtained for each pixel, a 3-
layer DBNs is trained by the L-BFGS algorithm [6]. From
the DBNs, high-level feature representations can be learned
and these representations are more abstract and can be
efficiently used for classification. The procedure is
summarized as follows:.
Step 1: Using the feature sets as input to train the first
RBM and freezing the parameters;
Step 2: Regarding the feature obtained from the first
RBM as the input to train the second RBM and freezing the
parameters;
Step 3: Similarly, the feature obtained from the second
RBM as the input to train a softmax classifier as follow,
2367
e1 x 
T (i )
 T (i ) 
1 e2 x  (4)
h ( x ) 
(i )
 
 j 1 e
k j x
T ( i )
  
ekT x( i ) 
 
where h ( x (i ) ) denotes the predicted probability
distributions of the i-th pixel, x (i ) is the input, j  1,2,, k
( j  1,2,, k ) denote the
T
is the value of the class label. j
parameters in the softmax classifier.
Step 4: Some labeled samples are used to fine-tune the
DBNs to perform PolSAR image classification, and the
parameters of the whole network can be obtained.
3. EXPERIMENTS
In order to evaluate the performance of the proposed method,
experiments on three real PolSAR data sets are performed.
The first two data sets are obtained from Flevoland four-
look polarimetric L-band scene, an agricultural area,
acquired by the AIRSAR airborne platform on August 16,
1989. The scene covers Flevoland, Netherlands. The third
experimental data set is a General Areas. This data set is
obtained from an L-band, multilook PolSAR image,
acquired by the ESAR airborne platform.
For all experiments, we use the DBNs with two hidden
layers to classify the PolSAR image. The visible units of the
first RBM are real-valued and the visible units of the second
RBM are binary. According to the ground truth map of the
Flevoland, 10% of the samples are randomly selected as the
training set and the remainder is regarded as the testing set.
First, we analyze the effect of variable parameters on the
subimage of Flevoland data set. The parameter m that
represents the size of patches varies from 1 to 40 with an
interval 2. The parameter hid that represents the number of
hidden units in the RBM varies from 1 to 100 with an
interval 1. The parameters hid1 and hid2 that respectively
represent the numbers of the first and second hidden units in
the DBNs vary from 1 to 200 with an interval 1. The overall
accuracy (OA) and Kappa coefficient (Kappa) are used to
evaluate the performance.
(a) (b) (c) (d)
Fig.1. Variation of OA and Kappa with different parameters for Flevoland
data set. (a) With the size of patches. (b) With the number of hidden units
in RBM. (c) With the number of units of the first hidden layer in DBNs. (d)
With the number of units of the second hidden layer in DBNs.
The variation of OA and Kappa with different parameters
for Flevoland data set is shown in Fig. 1. Fig. 1(a) indicates
that the large size of patches can improve the classification
performance. To make a tradeoff between the computational can represent the feature sets in a more abstract form, so our
complexity and the classification accuracy, we set the method obtains the better result.
parameter which represents the size of patches as 11. Fig.
1(b) shows that we can get the better classification accuracy
when the number of hidden units in the RBM is greater than
25. Fig. 1(c) and Fig. 1(d) show that the parameters should
be set smaller. In this paper, the parameters are set as
follows, (a) (b) (c) (d)
m  11, hid  25, hid1  100, hid 2  25
For comparison, the method based on neural net (NN) [7]
and Radial Basis Function network (RBF) [8] are also
presented. To validate the effectiveness of the DBNs for
classification, the SVM with the feature set learned by the (e) (f) (g) (h)
proposed method is also introduced. Fig.2. Classification results of different methods on the first data set. (a) is
The first experiment is carried out on a subimage of Pauli RGB composite. (b) are the 204 image patches. (c) are 25 filters from
the RBM. (d) is the ground truth map. (e) is the final classification results
Flevoland, as shown in Fig. 2(a), an RGB image formed by proposed algorithm. (f) is the result of NN-based method. (g) is the
with the intensities from Pauli decomposition. The size of result of RBF-based method. (h) is the result of SVM-based method.
image is 380  420 . According to Section 2, we get 10000 Table I gives the classification accuracy of Flevoland for
image patches of size 1111 from each dimension in the 9- different methods. From Table I, we can see that our method
dimentional image to train a RBM with 25 hidden units. A obtains a better performance compared with the NN-based,
set of randomly-selected 204 image patches are shown in Fig. RBF-based and SVM-based methods. The total
2(b) and the 25 filters from the RBM are shown in Fig. 2(c). classification accuracy of our approach is 99.21%, which is
The ground truth map is shown in Fig. 2(d). There are nine obviously higher than the other three. Especially for peas
identified crop classes including rape seed, beet, peas, and Barley that cannot be effectively classified, our method
potatoes, barley, grass, lucerne, wheat, and bare soil. Fig. can still achieves the classification accuracy of 97.04% and
2(e) is the final classification results by the proposed 97.35%, respectively. The total classification accuracy of the
algorithm. Fig. 2(f) and Fig. 2(g) are the results obtained by SVM-based method is 97.63%, which shows that the
the NN-based and RBF-based methods, respectively. Fig. features learned by our approach are very effective.
2(h) is the result of SVM with the feature set obtained by the TABLE I
proposed method. COMPARISON OF CLASSIFICATION ACCURACY (%) OF
As shown in Fig. 2(f), the NN-based method used a back SUBIMAGE OF FLEVOLAND FOR DIFFERENT METHODS
propagation algorithm to train the parameters of the Method Proposed NN- RBF- SVM-
multilayer neural network. Although it can obtain a better Class Method Based Based Based
result, it is difficult to control the final trainable parameters Rape Seed 99.77 94.61 92.84 99.63
in traditional neural network such that a good local optimum Bare Soil 99.15 96.49 96.39 99.30
is obtained. As shown in Fig. 2(g), RBF-based method can Potatoes 99.74 89.73 96.65 99.64
get a better result than the NN-based method because the Beet 99.59 85.85 95.01 99.51
Lucerne 99.29 97.81 91.63 98.86
RBF network can be trained very fast to get a global optimal
Grass 99.54 94.46 93.42 94.96
solution. In this paper, we use a RBM to learn features from
Wheat 99.53 98.16 97.67 99.99
each dimension in the 9-dimentional data. From Fig. 2(b), Peas 97.04 89.59 92.19 88.96
the 25 filters from the RBM are Gabor-like bases. When Barley 97.35 50.89 76.21 90.12
these filters are used to obtain features, the structural Total 99.21 89.77 93.52 97.63
information can be captured. Considering the scattering
properties, the elements in the coherency matrix are The second experiment is carried out on Flevoland data,
combined with the learned features to form the final feature as shown in Fig. 3(a), an RGB image formed with the
set. To validate the feature set learned by the proposed intensities from Pauli decomposition. Fig. 3(b) is the ground
method, SVM is used for classification. From the Fig. 2(h), truth map. The size of image is 7501024 . There are fifteen
we can see that it is easy to obtain a better result compared identified crop classes including peas, stembeans, beet,
with the NN-based and RBF-based methods. But the SVM is forest, bare soil, rapeseed, grasses, lucerne, potatoes, wheat,
a shallow learning method, there still have some wheat2, wheat3, barely, water, and buildings. Fig. 3(c) is the
shortcomings. In this paper, our proposed approach uses final classification result with the proposed algorithm. Fig.
DBNs to construct the deep network and train the 3(d) and Fig. 3(e) are the results obtained by the NN-based
parameters with a greedy layer-wise training method in order and RBF-based methods, respectively. Fig. 3(f) is the result
to obtain a better local optimum. The feature set can be obtained by the SVM-based method. Table II shows that the
learned again by the DBNs and the units in the hidden layers

2368
proposed method is highly successful for classification with based method.
TABLE III
the overall accuracy of 94.28%, and the classification
COMPARISON OF CLASSIFICATION ACCURACY (%) OF
accuracy is about 9% higher than that of NN-based and FLEVOLAND FOR DIFFERENT METHODS
RBF-based methods. Compared with the SVM-based Method Proposed NN RBF SVM
method, our method still has 4% higher classification Accuracy Method Based Based Based
accuracy.
Total 90.59 84.68 88.86 87.29

4. CONCLUSION

This paper proposed a effective PolSAR image classification

(a) (b) (c) method using Deep Belief networks. Many randomly-
selected patches are used to train a RBM. The features for
each pixel can be learned by the RBM from each dimension.
Considering the scattering properties, the raw coherency
matrix elements are combined with the learned features to
form the final feature set. Finally, a DBNs with two hidden
(d) (e) (f)
Fig.3. Classification results of different methods on the second data set. (a) layers is trained to classify the PolSAR image. Experiments
is Pauli RGB composite. (b) is the ground truth map. (c) is the final of three real PolSAR data show that the proposed method
classification results by proposed algorithm. (d) is the result of NN-based provides higher accuracies compared with the traditional
method. (e) is the result of RBF-based method. (f) is the result of SVM- method.
based method.
TABLE II
COMPARISON OF CLASSIFICATION ACCURACY (%) OF 5. REFERENCES
FLEVOLAND FOR DIFFERENT METHODS
Method Proposed NN- RBF- SVM- [1] J. S. Lee and E. Pottier, “Polarimetric, Radar Imaging: From
Accuracy Method Based Based Based Basics toApplication,” New York: CRC Press, 2009.
Total 94.28 83.90 84.55 90.36
[2] J. S. Lee, M. R. Grunes, and R. Kwok, “classification of multi-
look Polarimetric SAR imagery based on complex Wishart
The third experiment is carried out on General Areas as distribution,” International,” Journal of Remote Sensing, vol. 15,
shown in Fig. 4(a), an RGB image formed with the pp. 2299-2311, 1994.
intensities from Pauli decomposition. The size of the image
is 1300 1200 . Fig. 4(b) is the ground true map. There are [3] Y. D. Zhang, L. Wu, and G. Wei, “A new classifier for
polarimetric SAR images,” Progr. Electromagn. Res., vol. 94, pp.
mainly three classes including built-up areas, wood land, and
83-104, 2009
open areas. Fig. 4(c) is the classification result by the
proposed method. Figs. 4(d) and 4(e) are the results [4] L. Zhang, B. Zou, J. Zhang, and Y. Zhang, “Classification of
obtained by the NN-based and RBF-based methods, polarimetric SAR image based on support vector machine using
respectively. Fig. 4(f) is the result obtained by the SVM- multiple-component scattering model and texture features,”
based method. Fig. 4 and Table II shows that the proposed EURASIP Journal on Advances in Signal Processing. 2010
method reaches a better classification result, and obtains the
overall accuracy of 90.59%. [5] G. E. Hinton, S. Osindero, and Y. W. Teh, “A fast learning
algorithm for deep belief nets,” Neural Comput., vol. 18, no. 7, pp.
1527-1554, 2006.

[6] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-

based learning applied to document recognition,” Proc. IEEE, vol.
86, pp. 2278-2324, 1998
(a) (b) (c)
[7] O. Antropov, Y. Rauste, H. Astola, and J. Praks, “Land cover
and soil type mapping from spaceborne PolSAR data at L-Band
with probabilistic neural network,” IEEE Trans. Geosci. Remote
Sens., vol. 52, no. 9, pp. 5256-5270, 2014

(d) (e) (f)
Fig.4. Classification results of different methods on the third data set. (a) is
Pauli RGB composite. (b) is the ground truth map. (c) is the final
classification results by proposed algorithm. (d) is the result of NN-based
method. (e) is the result of RBF-based method. (f) is the result of SVM-
[8] T. Ince, S. Kiranyaz, and M. Gabbouj, “Evolutionary RBF
classifier for polarimetric SAR images,” Expert Syst. Appl., vol.39,
no. 5, pp. 4710-4717, 2012.

2369