Validating Assessments of Seismic

Damage Made from Remote Sensing

Edmund Booth,a) M.EERI, Keiko Saito,b) Robin Spence,c) M.EERI,
Gopal Madabhushi,d) and Ronald T. Eguchi,e) M.EERI

Assessments of damage following the 2010 Haitian earthquake were vali-

dated by comparing three datasets. The first, for 107,000 buildings, used vertical
aerial images with a 15–25 cm spatial resolution. The second, for 1,241 build-
ings, used Pictometry images (oblique angle shots with a resolution of about 10
cm taken in four directions by aircraft). The third dataset, for 142 buildings, used
ground observations. The ground observations confirmed the tendency of remote
sensing to underestimate the proportion of heavily damaged and collapsed build-
ings, and the difficulty of making remote assessments of moderate or low dam-
age. Bayesian statistics and sample surveys made from Pictometry images and
ground observations were used to improve remote damage assessments from ver-
tical images. The possibility of developing standard factors to correct remote
assessments is discussed. The field exercise pointed to the need to produce an
internationally agreed-upon set of damage definitions, suitable for postdisaster
needs assessments as well as for other uses. [DOI: 10.1193/1.3632109]

INTRODUCTION
Innovative and unprecedented use was made of remotely sensed images to provide
assessments of damage caused by the January 2010 Haitian earthquake. Remotely sensed
images proved to be of immense value in planning and coordinating the massive interna-
tional relief effort required after the disaster. This use is discussed elsewhere (Ghosh et al.
2011); this paper describes the validation of damage assessments made using three distinct
sets of observations using increasingly detailed data. The first dataset, comprising assess-
ments of 107,000 buildings in the Port-au-Prince urban area, was based on images with a
15–25 cm spatial resolution, taken in a vertical direction by high-altitude aircraft within a
few days after the earthquake and assessed in the Global Earth Observation-Catastrophe
Assessment Network II (GEO-CAN II) exercise. This exercise was carried out in the days
following the earthquake by around 600 engineers and scientists around the world, linked
by the Internet.
The second dataset, comprising a subset of 1,241 buildings from the GEO-CAN II
data, used Pictometry images. These are multiangle (both vertical and oblique) images
with a resolution of better than 25 cm. The images were taken in February 2010 and the

a)
Edmund Booth Consulting Engineer, 2 Miswell Cottages, Tring HP23 4JU, UK
b)
Willis Research Fellow, Cambridge Architectural Research Ltd, 25 Gwydir St No. 6, Cambridge CB1 2LG, UK
c)
Cambridge Architectural Research Ltd, 25 Gwydir St No. 6, Cambridge CB1 2LG, UK
d)
Department of Engineering Soils Group, Trumpington Street, Cambridge CB2 1PZ, UK
e)
ImageCat, Inc., 400 Oceangate, Suite 1050, Long Beach, CA 90802, USA

S157
Earthquake Spectra, Volume 27, No. S1, pages S157–S177, October 2011; V
C 2011, Earthquake Engineering Research Institute
S158 BOOTH ET AL.

assessments were made in March 2010 by Cambridge Architectural Research Ltd.

(Spence and Saito 2010), as part of an exercise funded by the World Bank’s Global Facil-
ity for Disaster Reduction and Recovery (GFDRR). The main objective was to inform and
refine the damage assessments, which were based in part on GEO-CAN-II, made jointly
by the United Nations Operational Satellite Applications Programme (UNOSAT), the Eu-
ropean Commission’s Joint Research Council (JRC), and the World Bank for the Haitian
Government’s postdisaster needs assessment (PDNA) exercise (GoH 2010). In contrast to
the views taken vertically, which were available from satellite and high-resolution aerial
imagery, Pictometry offers high-resolution views of the four facades of each building
(Figure 3).
The third dataset, comprising a subset of 142 buildings from the Pictometry data, was
based on ground observations carried out over a week in April 2010 by a team from the
United Kingdom’s post-earthquake field mission team, the Earthquake Engineering Field
Investigation Team (EEFIT; Booth et al. 2011). The team comprised Edmund Booth (struc-
tural engineer), Keiko Saito (remote sensing specialist), and Gopal Madabhushi (geotechni-
cal engineer). Fifteen percent of the buildings (other than collapsed buildings) were
inspected internally as well as externally. The field survey used damage definitions based
on the European Macroseismic Scale, EMS-98 (Grünthal et al. 1998). Detailed photographs
of all the buildings and their locations will be freely available for download from EEFIT’s
website (www.eefit.org.uk), hosted by EEFIT’s sponsors, the Institution of Structural Engi-
neers in London.
The remainder of the paper presents the following:
• A description of the three datasets, and the methodology each used to make dam-
age assessments,
• The results of the comparisons between the assessments using the three datasets,
• A discussion of the causes identified for the wide differences found between the
three sets of results and possible strategies for improving remote assessments,
• A proposed methodology for improving the accuracy of assessments from
vertical aerial imagery, based on the results of a smaller sample using more
accurate methods (Pictometry and ground observations) and employing Bayesian
statistics, and
• A discussion of the need to improve and standardize damage state descriptions, for
the purposes of postdisaster needs assessments from remote imagery and ground
observations, as well as for other uses.

DESCRIPTION OF THE THREE DATASETS

The GEO-CAN II assessment covered an area of 650 km2 centered on Port-au-Prince,
as shown in Figure 1. Aerial images with a resolution of better than 25 cm taken on 21 Janu-
ary 2010 were made available by the World Bank (World Bank-ImageCat-RIT Remote
Sensing mission) and assessed by the GEO-CAN team (a volunteer team over 600 engineers
and scientists brought together by ImageCat), who were each assigned one or more 0.5 km
by 0.5 km areas. Every visible building within the square was assigned one of the three
damage levels, D5 (collapse), D4 (very heavy damage) or no visible damage, as shown in
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S159

Figure 1. Part of the GEO-CAN II survey area in Port-au-Prince, with the EEFIT ground obser-
vation locations and the ImageCat land use classes shown.

Table 1a. A total of 107,000 buildings were assessed in the Port-au-Prince area. Figure 2
shows a typical image used in the assessment.
The Pictometry assessment was made of 1,241 buildings, consisting of about 20 build-
ings at each of 60 locations within the GEO-CAN II area, all of which were identified as
having also been assessed by the GEO-CAN II exercise. Images with a spatial resolution
of better than 15 cm were used, taken during February 2010 from aircraft flying at a height
of around 1 km with cameras taking orthorectified (i.e., corrected to give a true horizontal
plane) oblique angle images in four orthogonal directions and vertical images. Damage
assessments were made by the staff of Cambridge Architectural Research Ltd. (CAR),
using the five-point damage scale shown in Table 1b. The 60 locations were planned to be
reasonably distributed between four land-use classes (commercial=downtown, residential
high-density, residential low-density, and slums or shantytowns) as identified on a pre-
event land-use map produced by ImageCat using pre- and post-earthquake imagery of
Haiti (Spence and Saito 2010, see Figure 1). Commercial and downtown, identified sepa-
rately in the PDNA document, were here amalgamated as the building types and collapse
rates seemed similar. Figure 3 shows four orthogonal images at one of the locations
assessed.
The third dataset, comprising a subset of 142 buildings from the Pictometry data at the
nine locations shown in Figure 1, was collected in and around Port-au-Prince by a three-
S160 BOOTH ET AL.

Table 1. Damage descriptions: (a) GEO-CAN; (b) Pictometry and EEFIT ground observation

(a) Damage level Damage description

nvd No severe damage observable

D4 Very heavy damage
D5 Destroyed or collapsed
(b) Damage level Damage description

nvd (Pictometry) No visible damage.

D1 (ground Negligible to slight damage (no structural damage, slight nonstruc-
observations) tural damage): Fine cracks in plaster over frame members or in walls
at the base. Fine cracks in partitions and infills.
D2 (Pictometry and Moderate damage (slight structural damage, moderate nonstruc-
ground observations) tural damage): Cracks in columns and beams of frames and in struc-
tural walls. Cracks in partition and infill walls; fallen brittle cladding
and plaster.
Fallen mortar from the joints of wall panels.
D3 (Pictometry and Substantial to heavy damage (moderate structural damage, heavy
ground observations) nonstructural damage): Cracks in columns and beam column joints of
frames at the base and at joints of coupled walls. Spalling of concrete
cover, buckling of reinforced rods. Large cracks in partition and infill
walls, failure of individual infill panels.
D4 (Pictometry and Very heavy damage (heavy structural damage, very heavy non-
ground observations) structural damage): Large cracks in structural elements with compres-
sion failure of concrete and fracture of rebars; bond failure of beam
reinforced bars; tilting of columns. Collapse of a few columns or of a
single upper floor.
D5 (Pictometry and Destruction (very heavy structural damage): Collapse of ground
ground observations) floor or parts (e.g., wings) of buildings.

person team from EEFIT from 7–11 April 2010. Their full report and photos of the survey
buildings, which is freely downloadable, has been published on the EEFIT website
(www.EEFIT.org.uk). The surveys were conducted on foot and 15% of the buildings (other
than those which had been destroyed) were inspected internally as well as externally. Basic
survey methods were used, comprising a detailed photographic record, handwritten notes,
basic dimensional measurements, and checks with a spirit (carpenter’s) level. Interviews
with a number of building occupants and owners were carried out.

COMPARISONS BETWEEN DAMAGE ASSESSMENTS

Detailed comparisons between the full Pictometry dataset with the GEO-CAN II dataset
are presented by Spence and Saito (2010). Results presented here concentrate on those
obtained for the 142 buildings common to all three surveys; the survey locations are
described in Table 2 and shown in Figure 1.
Figure 4 compares the damage assessments made using the three observation meth-
ods, aggregated for all 142 buildings common to the three surveys. Table 3a compares
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S161

Figure 2. Typical detail of a vertical angle aerial image used by GEO-CAN II.

Figure 3. A set of Pictometry oblique aerial images for a Port-au-Prince location.

Compare the four views of the building that has its roof marked with a white spot.
S162 BOOTH ET AL.

Table 2. Locations of buildings common to all three surveys

Location No. of Typical building

IDi) buildings use Topography Superficial soils
L1 12 Retail (small Flat Reclaimed land, loose,
shops) sandy soil with shallow
water table
L3 17 Mainly retail Moderate slopes, Dense, sandy=silty soil
but close to steeper
slopes
L9 18 Mixed commercial Steep Rocky
and residential
L17 15 Mixed residential Moderate-to-steep Rocky, weathered
and retail slopes
L22 8 Residential Steep Rocky, Lateritic
L33 13 Mainly retail Steep Alluvial deposits, dense,
sandy soil
L41 41 Mainly residential Moderate slopes Dense, sandy soil
L55 9 Hospital complex Flat Alluvial deposits, moder-
ately dense, sandy soil
L61 9 Low-density Moderate-to-steep Rocky, weathered
residential slopes
TOTAL 142
i)
Location IDs are given in Figure 1.

the damage allocation made using the Pictometry data and ground observations by EEFIT
for all buildings in the common dataset of 142 buildings, while Tables 3b and 3c present
similar comparisons of GEO-CAN assessments with the ground observation and Pictom-
etry data, respectively. Table 4 presents a comparison between the Pictometry data and
ground observations, disaggregating the information of Table 3a into the nine survey
locations.

DISCUSSION OF DIFFERENCES BETWEEN THE THREE ASSESSMENTS

Figure 4 shows that the proportion of buildings assessed as being in the high damage
states D4 (severe structural) and D5 (partial or total collapse) increased very significantly
with increasing detail. For (D4 þ D5) combined, the percentage found by ground observa-
tion was 50% greater than that found using the Pictometry data, and over twice that
recorded by the GEO-CAN assessments, although it can be seen from Table 4 that these
proportions varied widely between locations. Of the 142 buildings in the dataset common
to all three surveys, 58 (data in crosshatched boxes in Table 3a) had damage underesti-
mated by the Pictometry data, and a further 14 (data in horizontally hatched boxes) had
damage overestimated.
Commission errors (number wrongly assigned to a damage state by the less accurate
method, as a percentage of the total assigned to that state by the less accurate method), and
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S163

Figure 4. Comparison between the assessments by the three methods aggregated over 142
buildings. (NB: GEO-CAN assessments gave only nvd, D4, or D5—see Table 1a—so no GEO-
CAN II values are shown for D2 or D3.).

omission errors (number omitted from a damage state by the less accurate method, as a per-
centage of the total assigned to that state by the more accurate method) are shown in Table
3. It is striking that both commission and omission errors are particularly high for D4 in all
three parts of the table, suggesting that the assessment of D4 is most in need of improve-
ment for remote damage assessments. This may be partly associated with the difficulty of
distinguishing between D4 and D5, discussed in the section below on damage state defini-
tions, and suggests that D4 is very difficult to assess from vertical images alone. The Kappa
index, which measures the statistical correlation between the two assessments being com-
pared, is very low for the comparison of ground observations with GEO-CAN II assess-
ments, increasing to 0.32 for ground observations with Pictometry data; this value is still
low, however. The higher Kappa value of 0.42 for the comparison of the Pictometry data
with the GEO-CAN II assessments reflects the fact that both assessments have wrongly
attributed many damaged buildings to the lowest damage state.
A detailed examination was made of the buildings where discrepancies arose. It identified
the following main reasons for the divergence in buildings assigned a damage grade of D5 by
ground observation but a lower damage grade by the Pictometry data (Spence and Saito 2010):
1) The difficulty of spotting soft-story collapses, even using Pictometry data (Figure
5a and 5b).
2) The problem of damage being obscured by buildings and trees (Figure 5c).
Similar reasons are considered to apply in some of the cases of the buildings classified
as D4 or D3 by ground observation, but classified as at least two damage states lower by
Pictometry data. However, Pictometry cannot be expected to be a reliable method to
observe damage such as major cracking of walls, failure of beam-column joints, etc, which
are properly classified as damage level D3.
S164 BOOTH ET AL.

Table 3. Contingency tables showing the agreement between the damage assessment and
interpretation results from ground observation by EEFIT and Pictometry data, aggregated over
142 buildings. (a) Pictometry data compared to ground observations; (b) GEO-CAN II assess-
ments compared to ground observations; (c) GEO-CAN II assessments compared to Pictome-
try data
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S165

Table 3. Continued

Notes to Table 3:
1) Matrices show numbers of buildings assigned to the damage levels of two surveys.
2) When a building was assigned a damage level that was between two states, it appears under both headings as
0.5 (e.g., a building with damage between states D2 and D3 appears as 0.5 at D2 and 0.5 at D3).
3) Damage states are defined in Table 1.
4) Using the symbols shown as a, b, and c for damage ratio D2 in Table 3a as an example:
commission error is derived as: (1 – a=b),
omission error is derived as: (1 – a=c).
5) The Kappa index is a well-established discrete multivariate technique used for the accuracy assessment of clas-
sification results (Congalton 1991). The Kappa index has a range of values between [0,1] with 1 meaning perfect
agreement and 0 indicating a pattern resulting from chance. Kappa index Khat is derived using the following
formula:
P
r P
r
N xii  ðxiþ  xþi Þ
i¼1 i¼1
Khat ¼ P
r
N2  ðxiþ  xþi Þ
i¼1

where r is the number of rows in the matrix, xii is the number of observations in row i and column i, and xiþ and
xþi are the marginal totals for row i and column i (the total rows and columns), respectively. N is the total number
of observations (Jensen 1996).

Remarkably, Table 3a shows that 14 buildings were assigned a higher damage state
using Pictometry data than ground observation, including five assigned as having severe
damage (D4) found in the field to be only moderately or lightly damaged (D2 or D1). The
reasons were found to be misattributing damage from a neighboring collapsed building and
mistaking construction materials stored on roofs as earthquake damage.
S166 BOOTH ET AL.

Table 4. Comparison between the assessments using ground observation and Pictometry data
classified by the nine locations shown in Figure 1: (a) L1, (b) L3, (c) L9, (d) L17, (e) L22, (f)
L33, (g) L41, (h) L55, and (i) L61 (see Table 3 for notes)
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S167

Table 4. Continued

The data collected by the EEFIT ground observations reported here indicate that dam-
age assessments from remote observations, made not only from vertical images but also
from more detailed oblique angle photographs (Pictometry), can have significant errors,
when compared with ground observations. However, in the aftermath of a very large catas-
trophe such as that which struck Haiti in January 2010, it is unlikely that comprehensive
damage assessments can be made using ground observations, or even using Pictometry, suf-
ficiently rapidly to suit the urgent needs of a postdisaster needs assessment, and remote
sensing is likely to continue to play a vital role, as it did in Haiti. Therefore, ways of
improving assessments made by remote sensing are discussed below.
Three possibilities suggest themselves. First, it may be that the underestimate of high
damage using remote sensing using a particular method (e.g., vertical aerial images or Pic-
tometry) may be corrected, using standard correction factors which are a function of varia-
bles such as instrumentally determined intensity, proportion of buildings assessed as having
damage class D5, type of construction, intensity of ground motion and local soil, topograph-
ical conditions, and so on. If such standard correction factors could be established, they
could immediately be applied to the remotely assessed damage proportions without any
fieldwork. However, much more data would be needed than that collected by EEFIT to
S168 BOOTH ET AL.

Figure 5. Examples of underestimated damage by Pictometry. (a) Soft-story collapse not evi-
dent from Pictometry: ground photos; (b) soft-story collapse not evident from Pictometry: Pic-
tometry image; (c) damage obscured by trees: ground photos.

establish the values of such factors with sufficient reliability to use after an earthquake. The
widespread availability of high-resolution satellite images, and the possibility of pooling
damage descriptions and photographs after earthquakes via the Internet make this a feasible
task for the future, and one which would be very valuable.
The second possibility for improving assessments is to subject a small sample of the
total number of assessed buildings to an assessment using more detailed methods,
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S169

Figure 6. Prior distributions of heavy damage or collapse (D4 þ D5) probability for the four
land-use classes from GEO-CAN II.

preferably with ground observations, although it is possible to improve assessments made

using vertical images by also considering oblique images (Pictometry). The more
detailed—and hence reliable—results are used to improve and adjust the more comprehen-
sive assessment. Bayesian principles are appropriate for this situation. These principles
were used for the Haiti data, as described in the next section.
A third possibility concerns estimating the proportion of buildings with light or moder-
ate damage states (D2 or D3), which cannot be established with any certainty from remotely
sensed images. The extensive damage at these lower damage states can form a significant
part of the total damage. If it can be shown that the distribution across damage states typi-
cally follows a particular mathematical form (e.g., a binomial distribution), then knowledge
of the proportion in the higher damage states can be used to infer the probable proportions
of buildings in the lower damage states. This approach has been investigated in a prelimi-
nary way (Spence and Saito 2010) assuming that the damage distribution does follow a bi-
nomial distribution (following Braga et al. 1982 and Dolce et al. 1999). Unfortunately, the
relative ratios of buildings with damage states D4 and D5 in the Haiti data are quite unlike
what would be expected in the binomial distribution. This may relate in part to the difficul-
ties in interpretation of D5 relative to D4 discussed in the Definition of Damage States sec-
tion below, and further investigation is needed before firm proposals on such extrapolation
methods can be made.
S170 BOOTH ET AL.

Table 5. Distributional parameters for commercial=downtown, residential areas, and

shantytowns

Pictometry

GEO-CAN prior Likelihood Posterior

(a) Commercial=downtown
mean 51% 31% 40%
qþr 6.00 7.86 13.86
q 3.06 2.47 5.53
r 2.94 5.39 8.33
r2 0.036 0.024 0.016
(b) Residential: high-density
mean 19% 19% 19%
qþr 6.00 6.38 12.38
q 1.14 1.19 2.33
r 4.86 5.19 10.05
r2 0.022 0.021 0.011
(c) Residential: low-density
mean 14% 19% 16%
qþr 6.00 6.38 12.38
q 0.84 1.19 2.03
r 5.16 5.19 10.35
r2 0.017 0.021 0.010
(d) Shantytown
mean 25% 22% 24%
qþr 6.00 3.94 9.94
q 1.50 0.87 2.37
r 4.50 3.07 7.57
r2 0.027 0.035 0.017

UPDATING DAMAGE STATE PROBABILITY USING A BAYESIAN APPROACH

The sample damage surveys using Pictometry and ground investigation can, in principle,
be used to improve an initial estimate of the damage probability distribution, using a Bayesian
approach. This section describes how this was done, and discusses its implications. The con-
cepts used are those set out in general by Ang and Tang (1975), and recently adapted for
earthquake vulnerability estimation by Jaiswal et al. (2011). In Bayesian analysis, a set of
prior assumptions about the probability distribution of a given variable is updated on the basis
of a set of observations. If a probability distribution for both the prior estimate and the set of
observations can be established, then according to Bayes’ theorem a more informative poste-
rior probability distribution can be defined. Each probability distribution is associated with an
expected value, therefore the initial expected value can also be modified using the data.
Here we will initially look at the probability of heavy damage or collapse of individual
structures, i.e., the probability that a given structure has a damage state of D4 or D5
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S171

following the earthquake. For this analysis the prior assumed damage distribution is based
on all 107,000 buildings in the GEO-CAN II survey, which assessed rapidly and remotely
the proportion of collapsed structures across the entire area of Port-au-Prince. The results
were published online on 11 March 2010 (European Commission et al. 2010), and gave
mean proportions of collapsed and heavily damaged (D4 þ D5) buildings for each of four
separate land-use classes. Following earlier work on earthquake vulnerability distributions
(Spence et al. 2008), it can be assumed that the prior distribution follows a beta distribution
of the form f(phd) ¼ b(phd, q, r, a, b),1) where phd is the probability of heavy damage or col-
lapse, and a and b are the limits of the distribution, here taken as 0 and 1. Spence et al.
(2008) propose that where data quality is poor, a beta distribution for which q þ r ¼ 6 is
appropriate, and this is considered suitable for a prior distribution. The resulting prior distri-
butions are shown in Figure 6 for the four land-use classes, for all 107,000 buildings in the
GEO-CAN II survey. The distribution shows the probability density function for the aver-
age ratio of buildings with damage at D4 and D5 levels combined in an affected zone. Dif-
ferent distributions would apply for other damage levels.
Updating these distributions using the observed data from the Pictometry survey is now
possible. The data for all 1,241 buildings from the Pictometry assessment was also divided
into the same land-use classes, as identified in Table 5, although for the Pictometry analysis,
the high- and low-density residential land-use data were amalgamated. For each land-use class
there were a number of separate survey locations, and the mean and variance of the proportion
of collapsed buildings in each location was used to find the likelihood function for the distri-
bution of the observed data (also shown in Table 5), assuming this is also a beta distribution.
A v2 test was used to check the goodness of fit of the data points to the beta distribution, and
the fit was found to be acceptable with an average significance of about 90% at a 10% level.
With this prior and likelihood function defined, it is possible to define the posterior col-
lapse probability distribution. Where both prior and likelihood functions have the form of a
beta distribution, the posterior function also has a beta distribution, whose parameters, q
and r, turn out to be the arithmetic sum of those for the prior and likelihood function
(Jaiswal et al. 2011). Table 5 shows the detail of the posterior functions for the four land-
use classes, and the corresponding mean and variance; these are presented graphically in
Figure 7 for the commercial=downtown land-use class.
In principle, a further update is possible using the posterior determined above as a new
prior and the ground observation data as a new set of observations. However, the small
1)
The density function of the beta distribution bðx; q; r; a; bÞ is
1 ðx  aÞq1 ðb  xÞr1
fxðxÞ ¼ ; a < x < b;
Bðq; rÞ ðb  aÞqþr1

in which q and r are parameters of the distribution and Bðq; rÞ is the beta function:
CðqÞCðrÞ
Bðq; rÞ ¼ and C is the gamma function ðAng and Tang 1975Þ:
Cðq þ rÞ

The parameters q and r of the beta distribution given depend on the mean lx , and variance r2x of the variable,
such that
lx ¼ q=ðq þ rÞ; r2x ¼ qr=ððq þ rÞ2 ðq þ r þ 1ÞÞ
S172 BOOTH ET AL.

Figure 7. Prior, likelihood, and posterior distributions for probability of D4 þ D5 in commer-

cial=downtown area.

amount of EEFIT ground observation data available was not well enough distributed across
the four land-use classes and damage states for a separate likelihood function to be deter-
mined for each land-use class. Instead, as a demonstration of the method, a single likelihood
function for collapse probability aggregated across all land-use classes was estimated with
parameters, as shown in Table 6; this is compared (Figure 8) with the equivalent prior and
posterior data. Table 6 and Figure 8 show that the spatial variability of the (D4 þ D5) dam-
age ratio between the nine survey locations of the ground observation data is much greater
than that between the 60 survey locations of the Pictometry data, so in this case the further
update yields results of limited value. However, the further update would be potentially use-
ful if more ground observation data were available. This is because the likelihood values of
variance shown in Table 6 for the Pictometry and ground observation data are based on spa-
tial variability alone. While that is reasonable for the ground observation data, it neglects
the epistemological (knowledge) uncertainty attached to the Pictometry data, which Figure
4 suggests is large; hence the variance is underestimated. A Bayesian updating process
using ground observation data has the potential to remove this difficulty because the dam-
age values are established with near certainty. The GEO-CAN prior estimate of uncertainty
in Table 6 (derived by assuming q þ r ¼ 6) is associated solely with epistemological

Table 6. Distributional parameters: All land-use areas aggregated

GEO-CAN prior Likelihood Pictometry Posterior Likelihood ground obs. Posterior 2

mean 23% 26% 24% 38% 27%

qþr 6.00 5.64 11.64 2.68 14.32
q 1.41 1.44 2.85 1.01 3.86
r 4.59 4.20 8.80 1.67 10.46
r2 0.026 0.029 0.015 0.064 0.013
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S173

Figure 8. All land use areas aggregated: Prior distribution from GEO-CAN-II, likelihood func-
tion based on Pictometry data, first posterior, likelihood function based on ground observation,
and posterior 2 distribution for probability of D4 þ D5.

uncertainty, since the sampling uncertainty in damage estimates for Port-au-Prince as a

whole made by the Pictometry and ground observation surveys does not apply to the GEO-
CAN estimate, which is based on observations of all the buildings in the city, rather than on
spatially separated samples.
The preliminary observations from this analysis are:
• Using the plausible assumption that both prior and likelihood functions are beta
distributions, the analysis is simple and straightforward, and provides a useful set
of data to update prior assumptions about collapse rates. The significance of this
analysis for estimating the cost estimate for recovery, and the uncertainty of this
sum, is considerable.
• In the case of the commercial=downtown zone, adding the Pictometry analysis
results in a posterior distribution with a reduced spread or dispersion (i.e., the pos-
terior contains more information, combining prior assumptions with new observa-
tions on collapse rates) and hence also reduced uncertainties, compared with either
prior or likelihood functions (Figure 7). Also, the expected value of collapse proba-
bility estimates is reduced due to fewer instances of commercial property collapses
in the downtown area than initially expected based on prior distributions.
• Adding the data from the ground survey (aggregated across all areas) changed the
resulting probability distribution of the collapse rate somewhat, modifying its mean
and reducing its uncertainty, although the low number of buildings involved in this
case limited the usefulness of this step. However, a Bayesian updating process
using ground observations could be potentially valuable, because the epistemologi-
cal variability associated with uncertainties in assessment is effectively removed,
leaving only the spatial variability associated with assessing sample groups of
buildings at a variety of locations.
S174 BOOTH ET AL.

DEFINITION OF DAMAGE STATES

An earthquake-damaged building is a highly complex system, comprising many parts
affected in various ways and to many degrees by the ground shaking. To describe the damage
using a single parameter—damage grade D2, for instance—is therefore a gross simplification,
but nevertheless a useful and necessary one for the purposes of postdisaster needs assessments
and other uses.
One surprising observation to emerge from the EEFIT fieldwork was that there were no
universally agreed definitions of structural and non-structural damage. The structural and non-
structural damage definitions in intensity scales such as the modified Mercalli intensity (MMI)
scale are unsuitable because they are not given separately from the intensity definitions, and
are not sufficiently detailed. Two sources, however, do provide detailed and separate damage
descriptions. Applied Technology Council’s field manual, ATC-20 (ATC 1989), uses a three-
point scale of severity (minor=none, moderate, severe) that is applied to 21 categories of gen-
eral, structural, nonstructural, and geotechnical hazards that are potentially life-threatening. The
severity is judged on an essentially qualitative basis. The matrix of 21 hazard types in three se-
verity classes is used to assign a building to one of three categories (inspected, restricted use,
and unsafe) and hence to control the use and occupancy of buildings immediately after an
earthquake. Wood-frame, masonry, tilt-up, concrete, and steel-frame structures are described,
and particular points for inspection are identified for each structural type. Three levels of evalu-
ation are described; the most rapid is expected to take 10–20 minutes per building, but even
this rapid level would require ground observation. The EEFIT team decided that the ATC-20
procedures, despite their comprehensiveness, were not readily adaptable to their particular task
of “ground truthing” damage assessments. This was because ATC-20 utilizes a three-point
scale based on occupancy hazard rather than a six-point scale based on degree of damage, as
used by the Pictometry assessments, and because the level of detail needed for each inspection
exceeded the limited time available to the EEFIT team.
The other source that provides detailed damage descriptions is the European Macroseis-
mic Scale, EMS-98 (Grünthal et al. 1998). Uniquely among intensity scales, it provides
detailed damages descriptions on a six-point scale (Table 1b). These formed the basis of the
GEO-CAN, Pictometry, and EEFIT ground assessments made in Port-au-Prince. EMS-98
gives extensive descriptions of damage states for concrete-frame buildings with masonry
infill, and also for unreinforced masonry buildings. There is a very useful appendix of pho-
tographs of building facades with damage states and commentary added, and (in the
authors’ opinion) less useful icon-like sketches of the damage states for both building types.
However, the EEFIT team found the EMS-98 damage descriptions were not entirely satis-
factory for use in Port-au-Prince. The difficulties the team found in making damage assign-
ments were as follows:

1) Most of the buildings inspected utilized neither unreinforced masonry nor concrete-
frames with masonry infill, but were a form of lightly confined masonry in which
vertical and (sometimes) horizontal reinforced concrete strips were added after con-
struction of the masonry walls. In such structures, the distinction made in EMS-98
between “structural” damage to beams and columns and “nonstructural” damage
(assumed greater) to infill masonry was therefore hard to apply, since the masonry
and reinforced elements worked intimately together.
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S175

2) It was sometimes difficult to make distinctions between damage states D4 and D5.
The short definition of D5 found in EMS-98 is, “Destruction - total or near total
collapse,” and the icon-like sketches show ruined buildings that are little more than a
heap of rubble. D4 is described as “…heavy structural damage…partial collapse…”
However, Figure A-18 in Grünthal et al. (1998) suggests that the collapse of part of
the wing of a building may constitute level D5, even though the rest of the building
shown appears much less damaged, and Figure A-19 shows the loss of an upper
floor defined as D5, even though much lower damage is apparent on lower floors,.
3) In EMS-98 descriptions are only given for unreinforced masonry buildings or
concrete-frame buildings with masonry infill. There is a need for definitions appli-
cable to construction in confined or reinforced masonry, timber, steel, and precast
concrete. No distinction is made between concrete-frame and concrete shear wall
construction, let alone the various intermediate possibilities.
The goal of producing an internationally agreed-upon set of definitions, suitable for
postdisaster needs assessments, as well as for other uses, is proposed as an important
task which could form part of the development of the Global Earthquake Model (www.
globalquakemodel.org), perhaps in association the World Housing Encyclopedia
(www.world-housing.net) by the Earthquake Engineering Research Institute (EERI) and the
International Association for Earthquake Engineering (IAEE), particularly since it would
need to reflect regional differences. The task would include:
1) A review of the needs for the damage definitions for various applications, including
those in postdisaster needs assessments, the immediate civic response to an earth-
quake (habitability, repairability, danger to adjacent areas from aftershocks), and
loss estimates in future earthquakes for civic planning or insurance purposes.
2) A detailed set of damage descriptions for a comprehensive range of common build-
ing construction types. The special needs of assessments made by remote sensing
as well as by ground observation should be taken into account. For example, since
aerial surveys seldom give reliable information on construction types, it would also
be necessary to propose damage rates for the mixed building stocks found in partic-
ular areas for the particular needs of remote assessments.
3) A Web-based set of photos of damaged buildings with damage assignments, anno-
tations, and classifications. Remote vertical and oblique photos would be needed, as
well as those taken from the ground internally and externally.
CONCLUSIONS
1) Building damage assessments made from ground observations carried out for 142
buildings in Port-au-Prince following the 2010 Haiti earthquake show significant
discrepancies from those made remotely using high-quality vertical and oblique aer-
ial images. High damage was missed in the remote assessments, mainly because
damage was obscured by vegetation and other buildings, or because soft-story col-
lapses were not evident when the upper stories were largely intact. More rarely,
remote sensing overestimated damage for reasons including cases where remote
assessments grouped undamaged buildings with adjacent damaged ones, and where
materials stored on roofs were mistaken for damage to structures.
S176 BOOTH ET AL.

2) While there is no doubt that there is a general tendency in remote sensing to underes-
timate damage, there is a need to carry out more detailed “ground truthing” exercises
to establish how typical the degree of error found in Port-au-Prince was. The under-
estimation of high damage states (D4 þ D5) between estimates from oblique aerial
images (Pictometry) and ground observation was found to be a factor of 1.5, rising
to over 2 for vertical aerial images (GEO-CAN II), but this was based on a sample of
only 142 buildings and more data is urgently needed. It is important that the ratios
found in this limited exercise are not taken as typical and adopted for use elsewhere.
3) Rapid damage assessments made using remote sensing played a vital role in the
relief effort in Haiti, and will undoubtedly continue to do so in future disasters. It is
therefore important to develop methods by which such assessments can be
improved. Some possibilities are as follows.
4) Where damage assessments are made solely on the basis of remote sensing, it would
be very useful to have standard correction factors, which might depend on factors
such as instrumentally determined intensity, proportion of buildings assessed as
having collapsed, type of construction, intensity of ground motion, local soil and to-
pographical conditions, and so on. To establish such correction factors, an extensive
comparison between assessments made from ground observations and remote sens-
ing would be needed for a wide range of future earthquakes. Given the possibility
of pooling damage data via the Internet, this should be a feasible task, but would
need leadership from an organization such as the IAEE.
5) A method is described by which damage assessments made using vertical imagery are
improved using a much smaller sample of assessment from oblique images (Pictome-
try) or ground observations, using Bayesian methods. The method is demonstrated for
the Port-au-Prince data, and appears promising enough to test for other cases.
6) While heavy damage can be observed in vertical images, they do not reveal the pro-
portion of buildings that are only moderately or lightly damaged. A preliminary
attempt was made to determine whether extrapolation from proportions of buildings
suffering heavy damage to proportions suffering lower damage levels could be
made using an assumed binomial distribution. The investigation showed that, for
the data available, this particular distribution was not appropriate. Investigating
other possible distributions would be worthwhile.
7) The goal of producing an internationally agreed-upon set of damage definitions is
proposed as an important task. The set would need to be suitable for postdisaster
needs assessments as well as for other uses, including estimates of future losses,
and for damage assessments made using both ground observation and remote sens-
ing. This task could be undertaken as a contribution to the Global Earthquake
Model (www.globalquakemodel.org).

ACKNOWLEDGEMENTS
The authors acknowledge the funding provided by the UK government’s Engineering
and Physical Sciences Research Council (EPSRC) which met the travel expenses of all three
members of the EEFIT team. Thorough and thought-provoking reviews of the manuscript
were provided by three anonymous reviewers, to whom the authors are most grateful. The
VALIDATING ASSESSMENTS OF SEISMIC DAMAGE MADE FROM REMOTE SENSING S177

authors are grateful to many colleagues for constructive comments on earlier versions, espe-
cially Dr. Kishor Jaiswal and Fiona McRobie.

REFERENCES
Ang, A. H-S, and Tang, W. H., 1975. Probability Concepts in Engineering Planning and
Design, Vol 1, John Wiley & Sons, Hoboken, NJ, pp. 420.
Applied Technology Council (ATC), 1989. Procedures for the Postearthquake Safety Evalua-
tion of Buildings, Report No. ATC-20, Redwood City, CA.
Booth, E., Wilkinson, S., Spence, R., Free, M., and Rossetto, T., 2011. Proceedings of the Insti-
tution of Civil Engineers - Forensic Engineering, vol. 164, issue 3, U.K. Earthquake Engi-
neering Field Investigation Team (EEFIT), London, England, in press.
Braga, F., Dolce, M., and Liberatore, D., 1982. A statistical study on damaged buildings and an
ensuing review of the MSK 1976 scale, in The Southern Italy 23.11.1980 Earthquake,
National Research Council (CNR) Publication No. 503, Rome, Italy.
Congalton, R., 1991. A review of assessing the accuracy of classifications of remotely sensed
data, Remote Sensing of Environment 37, 35–46.
Dolce, M., Masi, A., and Goretti, A., 1999. Damage to buildings due to the 1997 Umbria-
Marche Earthquake, in Seismic Damage to Masonry Buildings (A. Bernardini, ed.) Balkema,
Rotterdam, Holland, pp. 243.
European Commission Joint Research Centre (JRC), United Nations Institute for Training and
Research Operational Satellite Applications Programme (UNOSAT), World Bank Global Fa-
cility for Disaster Reduction and Recovery (GFDRR), and Centre National d’Information
Géo-Spatial (CNIGS), 2010. Building Damage Assessment Report, Haiti Earthquake, 12
January 2010, Report to the Haitian Government, http://www.eqclearinghouse.org/
20100112-haiti/wp-content/uploads/2010/02/PDNA_damage_assessment_report_v03-1.pdf.
Ghosh, S., Huyck, C. K., Greene, M., Gill, S. P., Bevington, J., Svekla, W., DesRoches, R., and
Eguchi, R. T., 2011. Crowdsourcing for rapid damage assessment: The Global Earth Obser-
vation Catastrophe Assessment Network (GEO-CAN), Earthquake Spectra 27.
Government of Haiti (GoH), 2010. Haiti Earthquake PDNA: Assessment of damage, losses,
general and sectoral needs, Annex to the Action Plan for National Recovery and Develop-
ment of Haiti, http://www.refondation.ht/resources/PDNA_Working_Document.pdf.
Grünthal, G. (ed.), 1998. European Macroseismic Scale, Cahiers du Centre Européen de Géody-
namique et de Séismolgie, Vol. 7, European Seismological Commission, Luxembourg.
Jaiswal, K., Wald, D., and D’Ayala, D., 2011. Developing empirical collapse functions for
global building types, Earthquake Spectra, this issue.
Jensen, J. R., 1996. Introductory Digital Image Processing: A Remote Sensing Perspective,
Prentice-Hall, Englewood Cliffs, NJ, pp. 316.
Spence, R., So, E., Jenny, S., Castella, H., Ewald, M., and Booth, E., 2008. The global earth-
quake vulnerability estimations system: An approach for earthquake risk assessment for in-
surance applications, Bulletin of Earthquake Engineering 6, 463–483.
Spence, R., and Saito, K., 2010. Port-au-Prince Earthquake Damage Assessment using Pictom-
etry, A Report for ImageCat, Inc., June 2010, Cambridge Architectural Research Ltd., Cam-
bridge, England, www.carltd.com/downloads/Haiti Pictometry.pdf.
(Received 9 November 2010; accepted 1 June 2011)