FEATURE SELECTION USING KERNEL BASED LOCAL FISHER DISCRIMINANT ANALYSIS FOR HYPERSPECTRAL IMAGE CLASSIFICATION

Guangyun Zhang and Xiuping Jia School of Engineering and Information Technology University College, Australian Defence Force Academy, University of New South Wales Canberra, Australia ABSTRACT Feature extraction is an important research aspect for hyperspectral remote sensing image classification to reduce the complexity and improve the classification accuracy. In this paper, a new feature extraction method, Kernel based Local Fisher Discriminative Analysis (KLFDA), is applied to hyperspectral remote sensing processing. This method integrates the advantages of conventional supervised Fisher Discriminative Analysis and unsupervised Locality Preserving Projection methods. Several experiments using the real images have been conducted, which indicate a high efficiency of this algorithm for hyperspectral image classification. Index Terms hyperspectral extraction, KLFDA, Gabor texture images, feature Discriminative Analysis (FDA) [2]. Fukunaga [3] developed a Non-parametric Discriminative Analysis using local statistical properties. For hyperspectral remote sensing data, Lee and Landgrebe [4] extended the method of Fukunaga to Decision Boundary Feature Extraction. Kuo and Landgrebe [5] developed a nonparametric weighted feature extraction method weighting the training samples in calculation of the local means. While PCA is easy to implement as an unsupervised approach, the first few PCs may not provide the best separation between the classes of interest. FDA mainly works for linear separable unimodal classes and the number of extracted features from FDA is limited to one less than the number of classes. This technique has been improved to Local Fisher Discriminative Analysis (LFDA) by Sugiyama (2007) [6], which integrated conventional supervised Fisher Discriminative Analysis and unsupervised locality preserving projection (LPP) [7] methods for dimensionality reduction. In this paper, this new technique LFDA and its kernel style of LFDA (KLFDA) are imported to remote sensing data analysis. They are investigated as a dimensionality reduction method for hyperspectral image classification. 2. METHODOLOGY 2.1. Local fisher Discriminative analysis (LFDA) be -dimensional samples Let and be associated class labels, where is the number of the samples and is the number of be low-dimensional classes. Let representation of , where is the reduced dimension. Let be the transform matrix, which leads to the dimensional reduction of the input data. Define as

1. INTRODUCTION The hyperspectral remotely sensed images are more and more popular because of its abundant spectral information for applications ranging from land cover mapping to target detection. However, this superiority also brings great challenges to construct models for image classification. From this point, reducing the dimensionality of data without losing any important information for classification is very critical. Band selection (selecting a good subset spectral bands from the original data) and feature extraction (generating a new subset features from the original data) have been widely investigated since hyperspectral data became available in late 1980s [1]. Feature extraction is often preferred since the new subsets of features are formed from the complete set of original measurements. The most popular feature extraction methods include principal component analysis (PCA) and Fisher

978-1-4577-1005-6/11/$26.00 2011 IEEE

1728

IGARSS 2011

the local mixture scatter matrix, as the local as the local within-class scatter matrix and between-class scatter matrix as follows [6]. (1) (2) (3) Where

The LPP methods can handle the multi-modal class data, just like different reflectance from various types of house roofs in spectral space. However LFDA is a linear transformation. When the classes of interest are not separable by linear feature extraction, Kernel trick as explained in the next section is introduced in this paper. 2.2. Kernel LFDA The kernel trick for LFDA can be described as follows. be the sample matrix. Let From Eq. (2), can be expressed as:

is the sample number of the -th class. Affinity -th matrix, , is the -dimension matrix with the being the affinity between and . element is large if and are close to each other in the feature space, otherwise it is small. For more details about calculation of affinity matrix, please refer to [6]. We use the affinity matrix to weigh sample pairs from the same class in the scatter matrix but not influence sample pairs from different classes. The LFDA transform matrix can be calculated as: in (4) The solution of above maximization problem is given by matrix with -th this form This can be expressed in matrix form as (6) where . Similarly, as , and diagonal is , and is -

dimensional matrix with -th diagonal element being can be expressed , element where being -dimensional

. So Eq. (5) can be expressed as (7)

represent the eigenvectors associated with where largest eigenvalues of the next equation: or The low dimensional representation of input can be expressed by: (5)

. Since belongs to where the range of , it can be expressed using some vector as . Substituting it into Eq. (7) and multiplying Eq. (7) by from left side, we can obtain: (8)

1729

We use

, with the

-th element being , into Eq. (8). We can get the

In this paper, is set to 1. The regularization element in Eq. (10) is set to 0.01. Based on the parameters above, 6 new features were extracted from the original spectral features. Fig.1(b) shows the first feature extracted using KLFDA. Using the Maximum likelihood classifier again, the overall testing data accuracy increased to 92.03%. To incorporate spatial property, twelve texture features were generated using Gabor filters (2 radial frequencies and 6 directions). KLPDA was applied to the texture features and one new spatial feature was extracted. Fig.1(c) shows this extracted texture feature. When this new feature is added to the 6 features extracted from the spectral data, the overall testing data accuracy is 95.28%. The experiments show that KLPDA is effective for hyperspectral data processing and classification.

linear kernel form for LFDA. Other kernel functions can be used to yield kernel form for LFDA: (9) be the transform matrix for Let dimensional reduction in kernel space, where represent the eigenvectors associated with largest eigenvalues of Eq. (9). However, the above setting is ill-posed because (and therefore ) is degenerated so we can not solve generalized eigenvalue problem. Instead this needs to . Eq. (9) becomes be solved by regularizing (10) We can get low dimension result of the input . Feature extraction from hyperspectral data is challenging, especially when class data has a multimodal distribution. It is valuable to introduce LFDA and KLFDA to overcome the difficult issue. They are implemented and tested on a real hyperspectral data in this study to demonstrate its effectiveness. 3. EXPERIMENTS AND CONCLUSIONS DC Mall image with 191 spectral bands from Hyperspectral Digital Image Collection Experiment was used in this study. Fig.1(a) shows the false colour image with bands 63, 52 and 36. Seven ground classes (Roof, Street, Path, Grass, Trees, Water and Shadow) are defined for classification. A small set of samples was chosen for these seven classes from the ground truth. When Maximum likelihood classifier is performed on the first 6 PCs from PCA (Principal Component Analysis), the overall testing data accuracy is 87.17%. To improve the performance, we applied KLFDA method to the spectral features. We chose Gaussian kernel, which is defined as by

(a) (b) (c) Fig.1 (a) The original false colour with channels (63, 52, 36). (b) The first spectral feature extracted using KLFDA. (c) The textural feature extracted from Gabor textures using KLFDA.

1730

This algorithm integrates the characteristics of supervised FDA and unsupervised LPP methods for feature extraction. The kernel style provides a nonlinear transformation, which is suitable for the complex distribution of hyperspectral images. From the experimental results, we can see it is effective for remote sensing image processing and improve the overall accuracy of classifications. 5. REFERENCES [1] D.A. Landgrebe, Signal Theory Methods in Mulltispectral Remote Sensing. New Jersy: John Wiley & Sons, 2003. [2] J.A. Richards and X. Jia, Remote Sensing Digital Image Analysis, 4 ed.: Springer-Verlag, 2006. [3] K. Fukunaga, Introduction to Statistical Pattern Recognition, 2 ed. boston: Academic Press, 1990. [4] C. Lee and D.A. Landgrebe, "Feature extraction based on decision boundaries," IEEE Trans. Pattern Analysis and Machine Intelligence, pp. 388-400, 1993. [5] B.C. Kuo and D.A. Landgrebe, "Nonparametric weighted feature extraction for classification," IEEE Trans. Geoscience and Remote Sensing, vol. 42, pp. 1096-1105, 2004. [6] M. Sugiyama, "Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis," Journal of Machine Learning Research, vol. 8, pp. 1027-1061, 2007. [7] X. He and P. Niyogi, "Locality preserving projections," in Advances in Neural Information Processing Systems, S. Thrun, et al., Eds., Cambridge: MIT Press, 2004.

(a)

(b)

(c)

Fig.2 The Maximum Likelihood classification results (a) using reduced features based on PCA (b) using reduced spectral features based on kernel LFDA, and (c) using reduced spectral features plus colour texture feature based on kernel LFDA. 4. CONCLUSIONS In this paper, the Kernel version of LFDA feature extraction method is developed and tested for hyperspectral remote sensing image classification.

1731