See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/308836417

Spectral unmixing of urban land cover using a generic library approach

Conference Paper · September 2016

DOI: 10.1117/12.2241189

CITATION READS

1 486

6 authors, including:

Jeroen Degerickx Marian-Daniel Iordache

Flemish Institute for Technological Research Flemish Institute for Technological Research
18 PUBLICATIONS   191 CITATIONS    50 PUBLICATIONS   2,186 CITATIONS   

SEE PROFILE SEE PROFILE

Akpona Okujeni Martin Hermy

Humboldt-Universität zu Berlin KU Leuven
51 PUBLICATIONS   891 CITATIONS    463 PUBLICATIONS   22,859 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

FLEUR network View project

ERA-NET BiodivERsA URBANMYCOSERVE: Understanding and Managing Urban Ectomycorrhizal Fungus Communities to Increase the Health and Ecosystem Service
Provisioning of Urban Trees View project

All content following this page was uploaded by Jeroen Degerickx on 22 March 2018.

The user has requested enhancement of the downloaded file.

 Spectral unmixing of urban land cover using a generic library
approach
Jeroen Degerickx 1, Marian-Daniel Iordache 2, Akpona Okujeni 3, Martin Hermy 1, Sebastian van der
Linden 3, Ben Somers 1
1
Division of Forest, Nature and Landscape, KU Leuven, Celestijnenlaan 200E box 2411, 3001
Leuven, Belgium
2
VITO TAP, Boeretang, 2400 Mol, Belgium
3
Geography Department, Humboldt-Universität zu Berlin, Unter den Linden 6, 10099 Berlin,
Germany

ABSTRACT

Remote sensing based land cover classification in urban areas generally requires the use of subpixel classification
algorithms to take into account the high spatial heterogeneity. These spectral unmixing techniques often rely on spectral
libraries, i.e. collections of pure material spectra (endmembers, EM), which ideally cover the large EM variability typically
present in urban scenes. Despite the advent of several (semi-) automated EM detection algorithms, the collection of such
image-specific libraries remains a tedious and time-consuming task. As an alternative, we suggest the use of a generic
urban EM library, containing material spectra under varying conditions, acquired from different locations and sensors.
This approach requires an efficient EM selection technique, capable of only selecting those spectra relevant for a specific
image. In this paper, we evaluate and compare the potential of different existing library pruning algorithms (Iterative
Endmember Selection and MUSIC) using simulated hyperspectral (APEX) data of the Brussels metropolitan area. In
addition, we develop a new hybrid EM selection method which is shown to be highly efficient in dealing with both image-
specific and generic libraries, subsequently yielding more robust land cover classification results compared to existing
methods. Future research will include further optimization of the proposed algorithm and additional tests on both simulated
and real hyperspectral data.

Keywords: spectral library pruning, endmember selection, MUSIC, Iterative Endmember Selection (IES), MESMA,
urban land use classification, hyperspectral remote sensing

1 INTRODUCTION

Urban areas pose a real challenge to the remote sensing community due to their large spatial [1] and spectral [2]
heterogeneity, making it hard to derive detailed information on land cover or other surface properties using satellite or
even airborne sensors. The high spatial heterogeneity typically gives rise to mixed pixels containing multiple materials.
This problem can be approached using Spectral Mixture Analysis [3] (SMA), which aims to identify subpixel fractions by
modelling a mixed spectrum as a (non-)linear combination of pure material spectra (called endmembers; EM). However,
urban EMs are typically characterized by a high intra-class variability and a high inter-class similarity, leading to spectral
confusion. Various alternatives or adaptations of SMA have been proposed, of which Multiple Endmember Spectral
Mixture Analysis [4] (MESMA) is the most widely used, including in urban areas [5, 6, 7]. MESMA allows the set of pixel
EMs to vary on a per-pixel basis and evaluates for each pixel all possible combinations of available EM spectra. As such,
the classification results provided by MESMA (and other approaches) highly rely on the quality of the input spectral
library. An ideal library for unmixing should consist of pure spectra being highly relevant with regard to the image to be
classified, should be sufficiently large to capture all EM variability present, but at the same time small enough to reduce
the computational efforts and the risk of spectral confusion between classes. As a consequence, the remote sensing
community has recently invested a lot of effort in developing algorithms that detect image EMs in a (semi-)automatic way
[8, 9, 10, 11]
. However, given the spatial and spectral nature of urban areas, the collection of a fully representative set of pure
pixels from a given scene may be problematic, even when working with high-resolution hyperspectral imagery. In addition,
these techniques often require manual verification of selected EMs using a reference spectral library, making it a time-
consuming task.

As an alternative approach, we introduce the concept of generic EM libraries for urban areas. A generic library is defined
as a large collection of spectra covering a wide variety of materials under varying conditions, recorded by various sensors
in different locations worldwide. In theory, such a generic library (if extended enough) should cover most existing EM
variability and would allow using the same library for classification over different study sites, sensors and timings. From
an operational perspective, once the generic library is established one would only require a highly efficient EM selection
algorithm (also termed spectral library pruning) that automatically selects the smallest set of relevant EMs to be used for
classifying a certain image. Many different library pruning techniques have already been developed in the context of image-
specific libraries, often to increase the computational efficiency of MESMA. They can be subdivided into three functional
categories: (1) pruning with respect to the library itself, (2) pruning with respect to the image to be classified and (3)
pruning using spatial constraints. Techniques in the first category are designed to create the smallest possible subset that
still represents the total variability of the original library, typically by removing redundant spectra, i.e. spectra that can be
modelled by (a combination of) other library spectra. Examples include IES [12], EAR, MASA and CoB [13]. In the context
of generic libraries however, these methods are not suitable because they operate independently from the image to be
classified. In response to this, a new set of library pruning techniques have been introduced which selects EMs based on
their similarity with (part of) the image. The SASD-MESMA algorithm [14] uses spectral distance (SD) and spectral angle
(SA) metrics calculated between image and library spectra as EM selection criteria. As an alternative, the MUSIC algorithm
[15]
ranks all library spectra according to their distance relative to the image subspace and retains only the closest spectra.
The technique has been successfully applied on both simulated and real hyperspectral data of citrus orchards [16], but has
never been tested in a more complex situation like the urban environment. The third category of library pruning techniques
includes approaches often using external data sources to create a first rough classification of the image. This step is then
followed by MESMA in which the selection of EMs for a given pixel is restricted by the classification label assigned
during the earlier classification stage, leading to a small but locally relevant EM library [5, 17, 18, 19].

The goal of this paper is to develop a new version of the MUSIC pruning algorithm in which we combine the merits of all
three categories of pruning methods listed above. In particular, we will apply MUSIC in a moving window approach,
allowing the spectral library to vary spatially across the image, depending on the local content of the image. In addition,
we iteratively combine this method with IES to reduce the load of redundant spectra in the final EM selection. This hybrid
pruning method is tested in a complex urban setting using simulated hyperspectral data and is compared to both IES and
MUSIC to show its added value both in case of image-specific (i.e. the more traditional setting) and generic EM libraries.

2 DATA AND METHODS

2.1 Data and study area

2.1.1 Hyperspectral imagery

This study has been conducted using a hyperspectral image of the Brussels Capital Region (Belgium) acquired using the
APEX sensor on June 30, 2015 around 12:00 local time. The image consists of six flightlines, five of which are centered
over the Eastern part of the city containing a mixture of different urban structure types (dense residential, sparse residential,
parks, commercial, industrial) and part of the Sonian forest complex in the South. The last flightline is situated in the
Western part of the city in a highly industrialized region. The imagery was acquired at an altitude of about 3600 m above
sea level and has a spatial resolution of 2 m. It consists of 285 spectral bands, covering the spectral range of 412 - 2431
nm, of which 218 were retained for analysis after removal of the atmospheric absorption bands (1-4, 147-163, 193-225,
273-285). All image pre-processing was performed at the Flemish Institute for Technological Research (VITO). Geometric
correction was performed by VITO’s C++ module based on direct georeferencing and using a detailed digital terrain model.
The image was then projected in the Belgian Lambert 72 reference system and atmospherically corrected using the
MODTRAN4 radiative transfer model incorporated in VITO’s processing chain [20].
2.1.2 Spectral libraries

Throughout the entire Brussels hyperspectral image, groups of pure pixels (min. 3 and max. 12 pixels per group) were
manually digitized using ArcGIS software and based on a combination of 7.5 cm resolution RGB orthophotos, Google
Street View, oblique aerial RGB imagery and an occasional field visit. Land cover identification was performed as detailed
as possible, i.e. on material level (Table 1). After digitization, the spectra of the selected pixels were extracted using a beta
version of the VIPER Tools 2 plugin to ENVI 5.2, imported in MATLAB R2012a (Mathworks, USA) and visually
evaluated after which obviously mixed/wrong pixels were removed. In total, 38 material classes were digitized with on
average 226 pixels per material class (Table 1). After identification, all spectra were labeled according to the classification
scheme used throughout this study (1st column in Table 1). During one of our experiments, we have combined our Brussels
spectral library with an urban spectral library from Berlin to generate a generic spectral library (see section 2.5.2). The
Berlin library consists of HyMap spectra and was adopted from Okujeni et al. [21]. This library was extended using the
same procedure as explained above, yielding a total of 12,199 spectra.

2.1.3 Validation data

Classification results were compared with manually digitized land cover data. Twenty squares of 100x100 m were
delineated across the entire hyperspectral image in which land cover was manually digitized in the same way as explained
above for the spectral library. The sample locations were chosen in a stratified random way, resulting in seven
industrial/commercial, four green, four dense residential and five sparse residential blocks. Land cover identification was
initially performed at material level and afterwards converted to the classes mentioned in Table 1. Finally, the twenty
blocks were combined into one image (Figure 1).

Figure 1: Illustration of validation data consisting of 20 individual image blocks. The colors represent the land cover classes, as
mentioned in the classification scheme (Table 1). NW vegetation = Non-woody vegetation, W vegetation = woody vegetation.

2.1.4 Simulated hyperspectral data

The experiments conducted in this study were performed on simulated APEX data. This enabled us to fully control the
EMs present in each part of the image, in turn allowing us to better assess the performance of the different library pruning
methods (see section 2.4). Twenty hyperspectral images (one for each validation block) were created and then merged
together to yield a realistic urban scene. Table 1 shows which materials were included in the simulated imagery. Creation
of the artificial image started with selecting a semi-random sample for each material class from the spectral library. For
each material class, the average spectrum and the distance between the maximum and minimum spectrum were calculated
(= total variability). Then a random sample of 50 spectra was drawn from the spectra in the range +30% and -30% of the
total variability from the average spectrum (to reduce the within-class variability). Together, these samples make up the
base library for image simulation. In a next stage, 5 spectra per material class were randomly drawn from this base library
to create 20 block-specific EM libraries. For each pixel, the dominant material class was identified based on the validation
data, after which a random fraction between 0.4 and 1 was assigned to the pixel for that material. Subsequently, a random
fraction of the material present in each pixel was added to its neighboring pixels (unless that neighboring pixel contained
the same dominant material and ensuring the total fraction of each pixel did not exceed one). Based on the resulting fraction
map and a random selection of EMs from the block-specific EM library, the simulated spectrum for each pixel was
generated using the linear mixing equation. Afterwards, Gaussian noise was added to all pixels with medium signal-to-
noise ratio (35). The actual set of EMs used per block was stored for comparison with the different pruned libraries.

Table 1: Classification scheme used in this study, including an indication of Brussels spectral library size for each material class and
whether or not the material has been used in the creation of the simulated image.

Land cover classes Materials Number of spectra Simulation

Roof Bitumen 315 x
Bright roof material 245 x
Dark ceramic tile 269 x
Dark shingle 259
Fiber cement 243
Glass 129 x
Gray metal 278 x
Gravel roofing 291 x
Green metal 130 x
Hydrocarbon roofing 286
Paved terrace 131
Red ceramic tile 266 x
Solar panel 269
Pavement Artificial turf 265
Asphalt 282 x
Bright gravel 122
Cobblestone 168
Concrete 412 x
Dark gravel 9
Green surface 293 x
Railroad track 299
Red concrete pavers 176
Red gravel 228
Tartan 212
Woody vegetation Deciduous broadleaf shrub 64 x
Deciduous broadleaf tree 486 x
Evergreen broadleaf shrub 88
Evergreen coniferous shrub 57 x
Evergreen coniferous tree 340 x
Non-woody vegetation Cereals 233
Extensive green roof 237 x
Horticulture 247
Lawn 255 x
Meadow 343 x
Other herbaceous 21
Soil Bare soil 207 x
Sand 140 x
Water Water 321 x

2.2 Library pruning

2.2.1 Iterative Endmember Selection (IES)

IES [12] reduces the size of a given spectral library by removing redundant spectra, i.e. spectra that can be modelled using
one or more spectra from the same library. The algorithm starts by classifying the original spectral library using MESMA
with every possible combination of two spectra and finding the combination that yields the highest kappa value. Those
spectra yielding the highest increase in kappa value are then added and removed in an iterative manner until the maximum
kappa value is reached. The IES algorithm was executed in MATLAB R2012a based on the IDL source code of the same
algorithm in VIPER Tools 2 (beta). IES was run in partially constrained mode with default parameter settings, i.e. minimum
and maximum allowable fractions of -0.05 and 1.05, maximum allowable RMSE of 0.025.

2.2.2 MUSIC

The MUSIC pruning algorithm for hyperspectral libraries [15] is based on the Multiple Signal Classification algorithm [22,
23]
. This technique aims to identify the spectra from a given spectral library with the largest similarity to the real EMs of
an (optionally independent) hyperspectral image. MUSIC starts by estimating the signal subspace of the image under
consideration using the HYperspectral Subspace Identification by Minimum Error (HySime) algorithm [22], resulting in a
subset of image eigenvectors and an estimated number of real image EMs (kf). Next, the algorithm calculates the
normalized Euclidean distance from each library spectrum to the identified image subspace. All library spectra are ranked
according to this distance and the user can decide (unlike in IES) how many spectra are retained in the final library. In our
experiments, we opted to define the final MUSIC library size as a multiple of kf, with factors 1, 2 and 4 respectively. Factor
4 turned out to more or less match the final IES library size. Larger library sizes were considered impractical because of
the long computation times. MUSIC was implemented in a moving window approach on each individual image block,
resulting in 20 different, locally relevant libraries for the entire image. MUSIC was executed in MATLAB R2012a.

2.2.3 MUSIC+IES

From an operational point of view, the MUSIC pruning algorithm has one major disadvantage, i.e. it does not take the
similarity into account between selected spectra. In case the original spectral library contains many replications of the same
material (e.g. in order to capture the large EM variability present in urban areas), the algorithm might result in a collection
of highly similar spectra of the few dominant materials present in the image, which does not contain the more rare image
EMs. Increasing the size of the resulting spectral library is likely to resolve this issue, but then the identification of the
optimal library size will require a lot of trial and error. In addition, a larger EM library implies longer unmixing processing
times and potentially more confusion between land cover classes. Therefore, we introduce the idea of combining MUSIC
with the IES algorithm in an iterative way to ensure similar (redundant) spectra are removed from the final library selection.
In theory, this approach should result in smaller, but equally relevant spectral libraries yielding a similar or even better
classification result. MUSIC+IES starts by applying MUSIC on the full spectral library, after which the first x spectra (x
is termed the step size parameter of the algorithm and needs to be defined by the user; default value = 50) are fed into the
IES algorithm resulting in a small subset of relevant but dissimilar spectra. Subsequently, IES is run on the next x (50)
spectra of the MUSIC library and the result is added to the final library. This iterative procedure stops as soon as the
minimum size of the final library (to be defined by the user) is exceeded or when all spectra have been processed by IES.
By applying IES repeatedly on small subsets of the library, we prevent the IES algorithm to remove most or all spectra
that were identified by MUSIC as the most relevant with regard to the image under consideration. In our experiments, the
minimum library size was again related to kf in the same way as for MUSIC. MUSIC+IES was implemented in MATLAB
R2012a using the same moving window approach as MUSIC.

2.3 Spectral unmixing using MESMA

MESMA [4] is specifically designed to account for EM variability and is hence frequently used in urban areas [5, 6, 7]. The
algorithm solves the linear mixing equation per pixel by minimizing the root mean square error (RMSE) using all possible
EM combinations and hence allows for different EM spectra to be selected for each individual pixel. In this study, we used
2 and 3 EM models. A 2 EM model consists of one material and shade, implying we maximally allowed 2 material mixtures
(which is considered realistic on the 2m scale). MESMA favors solutions with the least number of EMs involved by using
a relative RMSE threshold between the best 2 and 3 EM models of 0.007 which should be exceeded before the 3 EM model
is chosen. Throughout this study, the default MESMA constraints were applied, i.e. minimum and maximum allowable
fractions of -0.05 and 1.05, minimum and maximum shade fractions of 0 and 0.8 and a maximum allowable RMSE of
0.025. In case none of the models matched these criteria for a certain pixel, it was labeled “unmodelled”. MESMA was
implemented and executed in MATLAB R2012a based on the IDL source code originating from VIPER Tools 2 (beta).
2.4 Validation

In unmixing studies of urban areas, the results are often validated based on block fractions for each land cover class [14, 21].
However, multiple classification results may lead to similar estimated block fractions, which is why we chose for a more
detailed, pixel-based validation strategy. For each pixel, we determined the dominant land cover class based on the
estimated pixel fractions by MESMA. These are then compared to the validation data using the confusion matrix and kappa
statistic. Pixel-based approaches are sensitive to geometric shifts between the spectral imagery and validation data, but this
factor does not apply in our study since we only use simulated imagery, which is directly derived from the validation data.
For each experiment, we calculated one kappa value per image block and one value for the entire image by merging all
blocks together. The latter is used to compare the different pruning methods in the results section.

Additionally, the library pruning methods were validated by comparing the pruned spectral library with the actual set of
image EMs used to simulate each block. We calculated a similarity index (SI) between two libraries, using the inverse of
the spectral separability index defined in Somers et al. [25] and summing this index over all wavelengths. As a second
measure of similarity, the total number of times (1) a certain land cover class was represented in the pruned library while
not being present in the image block and (2) a land cover class was mistakenly removed during the pruning process, were
counted. Finally, for generic library experiments, the fraction of relevant spectra (i.e. spectra present in the simulated
image) was determined as well.

2.5 Experiments

2.5.1 Image-specific library

In a first set of experiments, we start from an image-specific library, i.e. a random subset of the actual set of image EMs.
Our aim was to show the added value of applying the MUSIC pruning algorithm in a moving window approach, allowing
the pruned library to vary across the image depending on the local context. First, we conducted a “conservative” baseline
simulation in which we set all disturbing factors, such as spectral library size, EM variability and number of EMs to a
relatively low level to keep things simple. Parameter values for this simulation match the ones mentioned in previous
sections. After creation of the simulated data, the starting spectral library was pruned using seven methods (IES, 3 ×
MUSIC and 3 × MUSIC+IES) for each of the twenty blocks. Each block was then classified using MESMA and the seven
associated pruned libraries and validated based on the kappa statistic. Starting from the same simulated image, this process
was replicated 3 times because of the random factor involved, i.e. the starting spectral library being a random sample of
image EMs.

In a first experiment, we tested the robustness of the different pruning methods against increasing complexity of the spectral
library. We started from the same simulated image, but increased the spectral library size from 209 to 474. In urban
environments, the number and variability of image EMs are typically high compared to other land use types. Therefore,
we increased each of these factors individually and ran the simulations again to check the performance of the different
methods in a more realistic scene. In experiment 2, the maximum number of EMs per block and per material class was
increased from 5 to 20, introducing more EMs in the simulated image. In the baseline simulation, the EMs used to construct
the simulated imagery were randomly drawn from the original Brussels spectral library, but limited to ± 30% of the mean
spectrum per material. In the final experiment, this limitation was changed to 80%, meaning that only extreme spectra
could not be selected to be used in the simulations, in turn resulting in more EM variability in the simulated imagery.

2.5.2 Generic library

All generic library experiments started from the simulated image created in the baseline simulation. To create a generic
spectral library, the initial spectral library used in previous experiments was contaminated with irrelevant spectra with
regard to the image to be classified. In experiment 1, randomly selected spectra from materials not present in the simulated
image (see Table 1) were added to the library. The number of spectra added amounted to half of the size of the original
library. Experiment 2 was similar to the first, but this time random spectra from the Berlin spectral library were added.
Prior to merging these spectra from different sensors, the Berlin (HyMap) spectra were interpolated to match the properties
of APEX spectra using an inter-sensor calibration tool implemented in the EnMap-Box software [26]. In experiments 3 and
4, we lowered the fraction of relevant spectra in the generic library even further by replacing half of the original spectral
library with irrelevant material spectra (Brussels only). Based on observations in experiment 3, the step size parameter of
MUSIC+IES was lowered from 50 (default) to 25 in experiment 4. This parameter controls the relative importance of
MUSIC in the combined algorithm. By lowering this value, it is forcing the algorithm to select more EMs being ranked
higher by MUSIC, or put in another way, with a higher resemblance to the image.

3 RESULTS

3.1 Image-specific library

Based on the classification results for the baseline simulation (Figure 2,upper left panel), IES seems to be the most suitable
library pruning method in case of image-specific libraries. On average, MU4 (MUSIC with comparable pruned library
sizes as IES) performs equally well but its results depend more on the input spectral library (larger variance in results).
Important to note is the dependency of the classification result on the size of the final (pruned) library for unmixing, which
is significantly higher for MUSIC compared to MUSIC+IES. The latter technique enables to yield similar classification
results with half (MI2) or even one fourth (MI1) of the amount of library spectra. When increasing the complexity of either
the simulated image or the spectral library, the added value of MUSIC and especially the combination of both pruning
methods becomes apparent (other panels in Figure 2). The strength of MUSIC, but especially MUSIC+IES in combination
with a moving window approach is illustrated by the first experiment in which more image EMs are available in the initial
spectral library. Whereas IES will use the same set of EM spectra to unmix each separate block, MUSIC and MUSIC+IES
will adapt the selection based on what is actually present in the specific block to be classified, thereby making better use
of the wealth of spectral information in the initial spectral library. In addition, the remaining two experiments show an
increased robustness of MUSIC and especially MUSIC+IES compared to IES towards increasing image complexity. IES
results become particularly unstable in case of increased EM variability in the image to be classified.

Figure 2: Comparison of classification accuracies (expressed as Kappa values) in function of spectral library pruning method for three
experiments in which resp. the spectral library size, number of image endmembers and endmember variability were increased with
respect to the baseline simulation. ‘MUx’ and ‘MIx' stand for MUSIC and MUSIC+IES in which the pruned library size equals x
multiplied by the estimated number of image endmembers by the HySime algorithm.

In order to better understand the results mentioned above, the pruned spectral libraries were compared with the actual EMs
present in each image block (i.e. the EMs used to simulate the image block spectra) using a spectral Similarity Index (SI;
see section 2.4). When comparing the libraries of more or less the same sizes (IES, MU4 and MI4) we see that both MUSIC
and MUSIC+IES result in libraries significantly more similar to the real set of image EMs compared to IES (Figure 3). In
addition, we again notice an increased robustness of MUSIC+IES against decreasing library size compared to MUSIC. In
Figure 4Figure 4, the total frequencies of (1) land cover classes being mistakenly removed from and (2) classes being
unnecessarily included in the pruned libraries are compared between the different library pruning methods. The lower the
sum of the two data series, the more capable the pruning method is of identifying the land cover classes in the image to be
classified and subsequently selecting those from a mixed spectral library. Although not all land cover classes are present
in each image block (Figure 1), IES uses the same library in which all classes are represented to classify all blocks, leading
to a high number of wrongly represented classes. MUSIC on the contrary is well capable of identifying only the relevant
land cover classes with the error increasing with increasing library size. Since MUSIC+IES is a combination of MUSIC
and IES, it performs in between the two other pruning methods. Due to the small library size of MU1 and the fact that a
lot of highly similar replications are present within the original spectral library, the number of classes not represented in
the MU1 library is large. This number gets smaller with increased library size, but is decreased even further by
MUSIC+IES. This phenomenon is in fact the main reason why we introduced MUSIC+IES (see section 2.2.3), i.e. to get
rid of highly similar (redundant) spectra in the pruned spectral library. We illustrate this with an example of one image
block in Figure 5. Based on Figures 4 and 5 we can conclude that MUSIC only performs well if the library size remains
substantially large, whereas MUSIC+IES is specifically designed to yield a representative selection with as little spectra
as possible.

Figure 3: Average similarity index of pruned spectral library
versus actual set of image EMs for different library pruning
methods (experiment 3). ‘MUx’ and ‘MIx' stand for MUSIC
and MUSIC+IES in which the pruned library size equals x
multiplied by the estimated number of image endmembers by
the HySime algorithm.
Figure 4: Total number of land cover classes wrongly
represented and not represented in the pruned spectral libraries
for the different library pruning methods (experiment 3).
“Total” signifies the sum of the two data series. ‘MUx’ and
‘MIx' stand for MUSIC and MUSIC+IES in which the pruned
library size equals x multiplied by the estimated number of
image endmembers by the HySime algorithm.
Figure 5: Comparison of actual set of EMs versus final spectral libraries obtained with the different library pruning methods. Results
are shown for image block nr. 18 (central block in the lowest row in Figure 1) and experiment 3. The number of spectra in each library
(n) is indicated in the subplot titles. ‘MUx’ and ‘MIx' stand for MUSIC and MUSIC+IES in which the pruned library size equals x
multiplied by the estimated number of image endmembers by the HySime algorithm. NW vegetation = Non-woody vegetation, W
vegetation = woody vegetation.

3.2 Generic library

The results of the four experiments conducted with generic spectral libraries (Figure 6) show that IES is less suited as a
library pruning technique when dealing with spectral libraries that are not specifically designed for the image to be
processed. When simply adding irrelevant spectra to an image-specific library (experiment 1), the MUSIC-based
approaches are capable of selecting the relevant spectra from the library and show little decrease in classification accuracy,
whereas IES yields less stable results (large variance). In case we merge our spectral library with a library from a different
sensor and region (experiment 2), the classification results for MUSIC do not change, meaning the algorithm can perfectly
separate spectra from the two different sources (illustrated in Figure 7-a). IES-based approaches wrongly include Berlin
spectra in their resulting libraries, leading to a decrease in classification accuracy. When part of the image-specific library
is replaced by irrelevant spectra from the same sensor/area (experiment 3), classification accuracies decrease for all
methods. As can be seen from Figure 7-b, the magnitude of this decrease is correlated with the fraction of relevant spectra
selected by the pruning method and is lowest for MUSIC and highest for IES. Based on this observation, we lowered the
step size parameter in the MUSIC+IES algorithm in experiment 4 (see section 2.5.2). As can be seen from Figures 6, 7-b
and 8, this resulted in spectral libraries being more closely related with the image to be classified and eventually in higher
classification accuracies.
Figure 6: Comparison of classification accuracies (expressed as Kappa values) in function of spectral library pruning method for four
generic library experiments. Exp 1: irrelevant material classes added to library; Exp 2: Berlin spectra added to library; Exp 3: library
spectra replaced by irrelevant material spectra; Exp 4: same as exp 3, but step size parameter of MUSIC+IES lowered from 50 to 25.
‘MUx’ and ‘MIx' stand for MUSIC and MUSIC+IES in which the pruned library size equals x multiplied by the estimated number of
image endmembers by the HySime algorithm.

(a) (b)

Figure 7: Comparison of fractions of relevant spectra in the pruned library between the seven library pruning methods for (a) experiment
2 and (b) experiments 3 and 4. Relevant spectra are defined as spectra actually present as EMs in the image to be classified. ‘MUx’ and
‘MIx' stand for MUSIC and MUSIC+IES in which the pruned library size equals x multiplied by the estimated number of image
endmembers by the HySime algorithm.
Figure 8: Total number of land cover classes wrongly represented and not represented in the pruned spectral libraries for the different
library pruning methods. “Total” signifies the sum of the two data series. Experiments 3 and 4 are represented in this plot, the latter only
by dotted black lines. ‘MUx’ and ‘MIx' stand for MUSIC and MUSIC+IES in which the pruned library size equals x multiplied by the
estimated number of image endmembers by the HySime algorithm.

4 DISCUSSION

Our experiments involving image-specific libraries have shown the added value of having locally adapted (MUSIC and
MUSIC+IES) versus static (IES) EM libraries, confirming previous results [5, 19]. Although MUSIC and MUSIC+IES
clearly improve the EM selection (Figures 3, 4 and 5), the resulting classification accuracies (Figure 2) are not as different
as expected. A first reason for this phenomenon could be our “conservative” approach in the baseline simulation, in which
we kept the complexity of both simulated image and spectral library relatively low. Moreover, in subsequent experiments
we chose to increase each complexity factor (number and variability of EMs) individually, whereas a combined increase
would be both more realistic and would likely result in a more clear distinction in performance between pruning methods.
The latter will be verified in future research. From comparing results of individual image blocks (not shown), we noticed
MUSIC approaches outperform IES more with decreasing number of materials present in the block. Based on this, one
could decide to lower the block window size in future applications to reduce the complexity within an individual block.
However, this would dramatically increase the total computation time and is therefore not considered to be a viable option.

The MUSIC algorithm has shown to be capable of identifying image EMs within an extensive spectral library quite well
(Figures 3 and 4). Errors in this process mainly occurred due to the large spectral similarity between different urban
materials (e.g. asphalt and bitumen, concrete and soil), a well-known problem in urban areas [2]. Additionally, MUSIC
showed a small bias towards brighter materials, causing dark (especially water) being rarely selected by the algorithm (not
shown). The combination with IES proved to largely solve this problem, since water did get selected when necessary. The
most pronounced added value of incorporating IES in the MUSIC algorithm however involves its reduced dependency on
final library size. This parameter is to be defined by the user and will be a function of EM complexity in both the image
and the initial spectral library. Classification accuracies for MUSIC+IES libraries proved to be more robust against this
parameter and, maybe more importantly, were often comparable to results for MUSIC libraries being two or four times as
large as the former.

Our first generic library experiments have shown the potential of MUSIC and MUSIC+IES in this respect as well, but
clearly more work remains to be done to boost the performance of these methods. Especially experiment 4 indicates there
is indeed still room for improvement. An idea which will be further explored is the possibility to systematically decrease
the step size parameter of MUSIC+IES in between individual iterations of the algorithm, thereby further increasing the
weight of the MUSIC result. Finally, future work will also focus on applying the proposed methodology on real
hyperspectral imagery.
 5 CONCLUSION

In this paper we introduced the concept of generic EM libraries for urban areas which, in combination with an efficient
library pruning method, could be used to classify multiple images from different regions and sensors. As such, this concept
represents an alternative to the time-consuming process of defining image-specific libraries prior to spectral unmixing of
urban land cover. We combined two existing pruning methods (MUSIC and IES) and applied them in a moving window
approach to simulated hyperspectral imagery of Brussels. Our experiments reveal that the hybrid pruning approach has
clear potential both in case of image-specific and generic spectral libraries. Future work will focus on further testing the
proposed algorithm in a generic library context, including tests on real hyperspectral data.

6 ACKNOWLEDGEMENTS

The research presented in this paper is funded by the Belgian Science Policy Office in the framework of the STEREOIII
program (projects UrbanEARS (SR/00/307) and VIPER2 (SR/XX/171)). The authors would additionally like to
acknowledge Frederik Priem, Marie-Leen Verdonck and Charlotte Wirion for their help in gathering land cover validation
data in the Brussels region. Special thanks also goes to Frederik Priem for providing an extensive spectral library of the
Brussels urban region.

7 REFERENCES

[1] Jensen, J.R. and Cowen, D.C., “Remote sensing of urban/suburban infrastructure and socio-economic attributes”,
Photogrammetric Engineering & Remote Sensing, 65(5), 611-622 (1999).
[2] Herold, M., Gardner, M.E. and Roberts, D.A., “Spectral resolution requirements for mapping urban areas”, IEEE
Transactions on Geoscience and Remote Sensing, 41(9), 1907-1919 (2003).
[3] Keshava, N. and Mustard, J.F., “Spectral unmixing”, IEEE Signal Processing, 19, 44-57 (2002).
[4] Roberts, D.A., Gardner, M., Church, R., Ustin, S., Scheer, G. and Green, R.O., “Mapping Chaparral in the Santa
Monica mountains using Multiple Endmember Spectral Mixture models”, Remote Sensing of Environment, 65,
267-279 (1998).
[5] Franke, J., Roberts, D.A., Halligan, K. and Menz, G., “Hierarchical multiple endmember spectral mixture analysis
(MESMA) of hyperspectral imagery for urban environments”, Remote Sensing of Environment, 113, 1712-1723
(2009).
[6] Powell, R.L. and Roberts, D.A., “Characterizing variability of the urban physical environment for a suite of cities
in Rondônia, Brazil”, Earth Interactions, 12(13), 1-32 (2008).
[7] Demarchi, L., Canters, F., Chan, J.C. and Van de Voorde, T., “Multiple endmember unmixing of CHRIS/Proba
imagery for mapping impervious surfaces in urban and suburban environments”, IEEE Transactions on
Geoscience and Remote Sensing, 50(9), 3409-3424 (2012).
[8] Boardman, J.W., Kruse, F.A. and Green, R.O., “Mapping target signatures via partial unmixing of AVIRIS data”,
Summaries, Fifth JPL Airborne Earth Science Workshop, JPL Publication, 95, 23-26 (1995).
[9] Winter, M.E., “ N-FINDR: An algorithm for fast autonomous spectral end-member determination in hyperspectral
data”, SPIE Proceedings Imaging Spectrometry, V, 266-275 (1999).
[10] Plaza, M., Martinez, P., Peres, R. and Plaza, J.,”Spatial/spectral endmember extraction by multidimensional
morphological operations”, IEEE Transactions on Geoscience and Remote Sensing, 40(9), 2025-2041 (2002).
[11] Segl, K., Bochow, M., Roessner, S., Kaufmann, H. and Heiden, U. “Feature-based identification of urban
endmember spectra using hyperspectral hymap data”, EARSeL Conference paper, (2006).
[12] Schaaf, A.N., Dennison, P.E., Fyer, G.K., Roth, K.L. and Roberts, D.A., “Mapping plant functional types at
multiple spatial resolutions using imaging spectrometer data”, GIScience & Remote Sensing, 48(3), 324-344
(2011).
[13] Roth, K.L., Dennison, P.E. and Roberts, D.A., “Comparing endmember selection techniques for accurate mapping
of plant species and land cover using imaging spectrometer data”, Remote Sensing of Environment, 127, 139-152
(2012).
[14] Fan, F. and Deng, Y., “Enhancing endmember selection in multiple endmember spectral mixture analysis
(MESMA) for urban impervious surface area mapping using spectral angle and spectral distance parameters”,
International Journal of Applied Earth Observation and Geoinformation, 33, 290-301 (2014).
 [15] Iordache, M.D., Bioucas-Dias, J.M., Plaza, A. and Somers, B., “MUSIC-CSR: Hyperspectral unmixing via
multiple signal classification and collaborative sparse regression”, IEEE Transactions on Geoscience and Remote
Sensing, 52(7), 4364-4382 (2014).
[16] Iordache, M.D., Tits, L., Bioucas-Dias, J.M., Plaza, A. and Somers, B. “A dynamic unmixing framework for plant
production system monitoring”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote
Sensing, 7(6), 2016-2034 (2014).
[17] Garcia-Haro, F.J., Sommer, S. and Kemper, T., “A new tool for variable multiple endmember spectral mixture
analysis (VMESMA)”, International Journal of Remote Sensing, 26(10), 2135-2162 (2007).
[18] Liu, T. and Yang, X., “Mapping vegetation in an urban area with stratified classification and multiple endmember
spectral mixture analysis”, Remote Sensing of Environment, 133, 251-264 (2013).
[19] Deng, Y. and Wu, C., “Development of a class-based multiple endmember spectral mixture analysis (C-MESMA)
approach for analyzing urban environments”, Remote Sensing, 8, 349 (2016).
[20] Biesemans, J., Sterckx, S., Knaeps, E., Vreys, K., Adriaensen, S., Hooyberghs, J., Meuleman, K., Kempeneers,
P., Deronde, B., Everaerts, J., Schläpfer, D. and Nieke, J., “Image processing workflows for airborne remote
sensing”, Proceedings 5the EARSeL Workshop on Imaging Spectroscopy (2007).
[21] Okujeni, A., van der Linden, S. and Hostert, P., “Extending the vegetation-impervious-soil model using simulated
EnMAP data and machine learning”, Remote Sensing of Environment, 158, 69-80 (2015).
[22] Bienvenu, G. and Kopp, L., “Adaptivity to background noise spatial coherence for high resolution passive
methods”, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 5, 307-
310 (1980).
[23] Schmidt, R., “Multiple emitter location and signal parameter estimation”, IEEE Transactions on Antennas and
Propagation, AP-34(3), 276-280 (1986).
[24] Bioucas-Dias, J. and Nascimento, J., “Hyperspectral subspace identification”, IEEE Transactions on Geoscience
and Remote Sensing, 46(8), 2435-2445 (2005).
[25] Somers, B. and Asner, G.P., “Tree species mapping in tropical forests using multi-temporal imaging spectroscopy:
Wavelength adaptive spectral mixture analysis”, International Journal of Applied Earth Observation and
Geoinformation, 31, 57-66 (2014).
[26] van der Linden, S., Rabe, A., Held, M., Jakimow, B., Leitão, P.J., Okujeni, A., Schwieder, M., Suess, S. and
Hostert, P., “The EnMAP-Box – A toolbox and application programming interface for EnMAP Data Processing”,
Remote Sensing, 7, 11249-11266 (2015).

View publication stats