Professional Documents
Culture Documents
An Evaluation Framework For Assessing Resilience of Post-Disaster Housing
An Evaluation Framework For Assessing Resilience of Post-Disaster Housing
An Evaluation Framework For Assessing Resilience of Post-Disaster Housing
www.emeraldinsight.com/1759-5908.htm
IJDRBE
6,3
An evaluation framework for
assessing resilience of
post-disaster housing
300 Iftekhar Ahmed
School of Architecture and Design, RMIT University, Craigieburn,
Received 12 November 2013
Revised 3 October 2014
Australia, and
20 October 2014
Accepted 21 October 2014 Esther Ruth Charlesworth
Department of Architecture, RMIT University, Melbourne, Australia
Abstract
Purpose – The purpose of this paper is to discuss the utility of a tool for assessing resilience of
housing. After disasters, maximum resources are often allocated for housing reconstruction, and most
initiatives on disaster resilient housing have arisen after disasters. With widespread claims by agencies
of having “built back better”, it is important to establish an evaluation framework that allows
understanding to what extent resilience has been successfully achieved in such housing projects. This
paper discusses such a tool developed by the authors.
Design/methodology/approach – In a study commissioned by the Australian Shelter Reference
Group, the authors have developed an evaluation tool for assessing resilience in housing and tested it in
several housing reconstruction projects in the Asia-Pacific region. Various evaluation frameworks were
reviewed to develop the tool. An approach derived from the log frame was adapted in alignment with
other key approaches. The tool is practical and targeted for agency staff involved in housing projects,
evaluators of housing reconstruction projects and communities to assess their housing in terms of
resilience. It comprises three main stages of an assessment process with guided activities at each stage.
Findings – The tool was tested in the Cook Islands and Sri Lanka, and the key findings of the test
assessments are presented to demonstrate the prospects of the tool. While the case study projects all
indicated achievement of a level of resilience, problems were evident in terms of designs issues and
external factors.
Originality/value – Such a tool has the potential to be used more widely through advocacy to
prioritise resilience in post-disaster housing reconstruction.
Keywords Evaluation, NGO, Housing, Asia-Pacific, Resilience, Reconstruction
Paper type Research paper
Introduction
Housing is usually the most valuable asset for people and is often extensively
devastated in disasters, resulting in significant economic loss at the national level
The authors acknowledge Caritas-Australia for allocating funding for the above study; AusAID
International Journal of Disaster
Resilience in the Built (ex) for providing the funds to Caritas-Australia under the Humanitarian Partnership Agreement,
Environment a component of which related specifically to disaster risk reduction and management; and Shelter
Vol. 6 No. 3, 2015
pp. 300-312 Reference Group (SRG), Australia for commissioning and facilitating the research project entitled
© Emerald Group Publishing Limited
1759-5908
Scoping Study: Shelter and Disaster Risk Reduction in the Asia-Pacific Region, from which this
DOI 10.1108/IJDRBE-11-2013-0042 paper is derived.
(Lyons, 2009; Marti, 2005). The impact of disasters on the built environment is Resilience of
particularly high in developing countries, estimated at more than 20 times than in post-disaster
developed countries (Barakat, 2003), because of the often widespread weak construction
and, consequently, extensive devastation. Hence, in post-disaster recovery programmes,
housing
maximum resources and priority are often allocated to housing and infrastructure
reconstruction (Lang, 2008), and it is also common to find the development of guidelines
and initiatives for safer buildings mainly after major disasters (ERRA, 2006, after the 301
2005 Pakistan Earthquake; NHDA, 2005, after the 2004 Indian Ocean Tsunami in Sri
Lanka). It is unfortunate that unless a disaster strikes, there is little concern for building
resilient housing. It is, however, understandably difficult for housing providers to
anticipate disaster impacts in advance, unless there is a history of disasters in the area.
Additionally, it proves difficult to mobilise extra funds for resilience from donors, unless
a disaster impact is evident. Thus, resilient housing initiatives often tend to be part of
post-disaster reconstruction programmes.
During reconstruction, there is the opportunity to understand and thereby address
and overcome the underlying vulnerabilities that had previously led to weak housing,
and also the future risks that threaten durability and sustainability of housing. As
suggested by Lyons and Schilderman (2010), building housing back to a better standard,
based on local knowledge and participation, has the potential to contribute to long-term
disaster risk reduction. Reconstructed or rehabilitated housing with future risk in mind
can thus prove more sustainable.
There is a complex set of challenges for agencies to incorporate disaster resilience
into housing because of the dynamic nature of disasters. Because of the long-term nature
of housing, often with an expected service life of at least 50 years, agencies building
housing are confronted with a wide range of factors that may emerge in the future, for
example rapid urbanisation and climate change, which may undermine resilience.
Additionally, housing is not only about constructing dwelling units, but encompasses
various social, economic and environmental factors, demanding a sustainable and
holistic approach that can lead to disaster resilient housing. Thus, understanding
whether resilience has been achieved in housing can be a complex process, as it has to
take into consideration this wide array of factors.
Background
Asia is the continent most affected by natural disasters (EM-DAT, 2012), and the Pacific
region is highly vulnerable to changes in sea level and exposed to coastal hazards such
as cyclones and tsunamis (World Bank and SOPAC, 2009). A number of member
agencies of the Shelter Reference Group (SRG), Australia, such as Caritas,
Emergency Architects, Habitat for Humanity, Red Cross and World Vision, are active in
the Asia-Pacific region in implementing post-disaster housing reconstruction
programmes. In response to the interest of the SRG to obtain a better understanding of
the provision of resilient housing in the disaster-prone Asia-Pacific region, a scoping
study was carried out by the authors (Charlesworth and Ahmed, 2012), from which this
paper has been derived.
The key aim of the study was to develop a resource for the SRG and other agencies
which can support the assessment and thereby implementation of disaster-resilient
housing in the Asia-Pacific region. To address this aim, an evaluation tool was
developed through a literature review and extensive consultation with SRG member
IJDRBE agencies. It was then tested in the context of housing projects of some of these agencies
6,3 in two countries in the Asia-Pacific region – the Cook Islands and Sri Lanka. This paper
discusses the development of the evaluation tool, key findings of the assessment tests in
the case study countries and future prospects of the tool.
Inputs Human, material and financial Efficiency Were the local and external
resources required to resources optimised (cost-
incorporate resilience in effectiveness)?
housing Was the community specifically
engaged in design/construction?
Was there a dedicated skills
transfer/training component?
Output Articulation of resilience Results Were the resilience options realised?
options before applying it Timing Were they available at the right
time?
Quality Are the resilience options “good” in
the local context?
Result Direct consequence for the Pertinence Were the resilience options
beneficiary of applying the available to the most vulnerable
resilience options people?
Acceptability Did the local community use the
resilience options?
Were they pre-determined/required,
or optional?
Were they replicated outside the
project?
Are they easy to maintain?
Impacts Indirect or later consequences Strategy Did the resilience options
and for the beneficiary of using correspond to the needs of the
effects resilience options (or the community?
situation originating from the Scope What proportion of vulnerable
project) people was covered?
Ultimate objective Did the project reduce the disaster
risks of the community?
Do the community/households feel a
greater sense of security?
External Factors beyond the control of External aspects How did the context and
factors the implementing agency environment affect the results of the Table I.
project? Analytical
framework for the
Source: Lizarralde (2002) SRG adaptation tool
IJDRBE factors. When developing the SRG evaluation tool, these factors and aspects were
6,3 incorporated into the adapted framework shown above in Table I, with a corresponding
set of questions that guided the investigation.
The extra factor added, sustainability, relates importantly to livelihood, which
although not strictly a design or technical aspect, yet nonetheless of great significance in
the Asia-Pacific context because of its linkage with housing. Beside the house being a
304 workplace and having strong implications for health and well-being contributing to
economic productivity, production of housing after a disaster can create local jobs and
regenerate the local economy through production, procurement and transport of
building materials (Cosgrave, 2008; Feinstein International Center, 2011). A study
showed that households whose homes were rebuilt after a disaster were able to resume
income-generating activities, which allowed economic recovery from the disaster’s
impacts (HFHA, n.d., b), thus contributing to resilience. Importantly, local capacity can
be developed for building resilient housing and in this sense allows the disaster risk
reduction initiative to be sustained over the long term.
The tool comprises three main stages of the assessment process with a corresponding
set of activities (Figure 1):
(1) Pre-assessment stage: Firstly, the geographical boundaries are set and the project
case studies to be assessed are selected. Once this is done, relevant project
documents of agencies and other secondary literature on the project context and
environment, particularly on disasters, are collected and reviewed. Then,
preparation is taken for fieldwork by establishing local contacts.
(2) Assessment stage: This is the central stage of the evaluation where assessments are
carried out at two levels: community level, including project beneficiaries and/or
residents of the housing built in the project, and agency level, mainly staff of agencies
Factors Aspects
who were involved with and/or familiar with the project. A series of semi-structured
activities guide the assessments.
(3) Consolidation stage: After conducting fieldwork, the data collected are analysed and
a draft report produced. The findings of the assessment are shared with stakeholders
for validation, and thereby a final evaluation report is produced.
As mentioned, the assessment stage was central and within it, the Key Informant Interviews
were found to be the most important activity for gaining insights on the performance of
housing vis-à-vis resilience. A checklist of investigation issues based on the analytical
framework shown above in Tables I and II was the basis of the interviews. The same
checklist was used with key informants at the agency and community levels with the
objective of triangulating and thereby validating the information collected. The Key
Informant Interviews were strongly supplemented by on-site Direct Observation and
Documentation. The hazard mapping and ranking activities were not conducted during the
test assessments in the scoping study, as the case study housing projects were not
sufficiently multi-hazard to warrant these activities. However, these activities would be
important elements of the tool if used elsewhere with more complex conditions.
The tool provides guidance and templates for most of the activities at each stage of the
assessment process. The scope of this paper does not allow sharing all those materials;
complete details are available from the scoping report (Charlesworth and Ahmed, 2012).
No. of
Country Location Lead agency houses Key resilience features
Plate 1.
One-bedroom type
(left) and
two-bedroom type
(right) houses of the
reconstruction
project in Aitutaki
after a cyclone in
2010
Plate 2.
House in Mangaia
with the roof
anchoring system to
resist cyclones
IJDRBE
6,3
308
Plate 3.
Habitat for Humanity
house in Galle
Plate 4.
Caritas house in
Galle
Plate 5.
World Vision house
in Kirinda
commonalities and congruence within the responses, as well as drawing out unique Resilience of
aspects. The analysis led to two key insights as below at the broader level across all the post-disaster
projects:
housing
(1) All the five case study housing projects were found to have reduced disaster risk
to varying levels, and had contributed to beneficiaries’ resilience.
(2) All the projects represented a marked improvement to previous vulnerable
living conditions. 309
However, exploring the uniqueness of the projects, it was found that despite the overall
positive findings, various problematic issues were revealed in the test assessments,
especially with regard to the Result factor concerning housing design issues. For
example, the one-size-fits-all approach followed in all the projects (except in Mangaia,
where the scope was different) resulted in the obvious lack of space for large households
and too much space for small ones, affecting the local acceptability of the housing.
External Factors also played a significant role in undermining the sustainability of
beneficiary communities at a general level in all the projects, manifested in different
ways in the two countries. In the Cook Islands, due to lack of local production, heavy
reliance on outside commodities acted as a serious constraint to resilience and
sustainability. In Sri Lanka, varying levels of service provision by NGOs and the
government affected resilience; while communities were sheltered in well-built disaster
resilient houses built by NGOs, lack of public service provision (drainage, roads, etc.)
contributed to vulnerability, undoing the efforts of NGOs.
The case study findings also allowed identifying positive elements that have broader
strategic implications for effective post-disaster housing reconstruction, which include
two key insights as below:
(1) Projects that provided a wider set of Inputs integrated with housing contributed
to better community resilience; to some extent, the World Vision project in
Kirinda demonstrated this.
(2) Agencies that engaged with communities over a long term and did not fold up
their operations in the beneficiary community soon after project completion were
able to assist contributing to the typically gradual process of consolidation of
community resilience, as the Caritas project in Galle demonstrated.
In addition to gaining insights into the outcomes of the housing projects, the field tests
allowed reviewing the evaluation tool itself to understand the practicalities of utilising it
in the future. In this respect, the following aspects became evident:
• Need for experience to decide which issues are more relevant and not follow the
tool mechanically and thereby avoid information overload.
• Hazard Mapping and Ranking should be done in multi-hazard areas where there
are varying levels of exposure and sensitivity to hazards. These activities require
a larger team (at least two or more evaluators) and a strong involvement of
agencies commissioning the evaluation.
• Key Informant Interviews (complemented by Direct Observation and
Documentation) were found to be the most pertinent and readily applicable
activity. However, interviews of agency staff and beneficiaries should be done
independently to avoid influencing the responses.
IJDRBE • Importance of the Pre-Assessment Stage, particularly to plan the fieldwork, needs
6,3 to be recognised to avoid frustrations during the assessment stage due to varying
cultural and organisational perceptions and practices. Also, key agency
informants who worked on the project might not be available in an extended
period after project completion, and therefore adequately knowledgeable agency
representatives need to be identified.
310
• Extensive Documentation during the assessment stage is most important for
recall after completion of fieldwork. Digital photography and audio recording
complements the evaluation tool, which may require additional skills.
The evaluation tool was designed to capture a wide range of issues relating to housing
and disaster resilience. Because it is comprehensive, it allows examining different types
of projects. Not all the issues included in the tool would be relevant for all projects, and
some issues might be more important according to specific projects. To prove relevant to
agencies, it would need to be adapted to the particular context and project while
adhering to its structure and processes.
The SRG is considering appropriate ways of disseminating the evaluation tool to its
counterpart and partner agencies in different Asia-Pacific countries, and modalities of
training provision. It is also anticipated, that the tool would be used for advocacy to
bilateral and multi-lateral donor agencies to promote prioritisation of resilience in
post-disaster housing reconstruction and to provide a solid backing to the “building
back better” slogan.
References Resilience of
Barakat, S. (2003), Housing Reconstruction After Conflict and Disaster, Humanitarian Practice post-disaster
Network, London.
housing
Charlesworth, E. and Ahmed, I. (2012), Scoping Study: Shelter and Disaster Risk Reduction in the
Asia-Pacific Region (report), Shelter Reference Group, Sydney.
Cosgrave, J. (2008), Responding to Earthquakes: Learning From Earthquake Relief and Recovery
Operations, Active Learning Network for Accountability and Performance in 311
Humanitarian Action (ALNAP), London.
Djik, S.V. and Leersum, A.V. (2009), Measuring Socio-Economic Impact of Post-Disaster Shelter
(PowerPoint Presentation), Red Cross, The Hague.
EAA (n.d.), Post-Occupancy Assessment/Performance Evaluation (template), Emergency
Architects Australia (EAA), Sydney.
EM-DAT (2012), “The international disaster database”, available at: www.emdat.be/natural-
disasters-trends (accessed 5 September, 2012).
ERRA (2006), Guidelines for Earthquake Resistant Construction of Non-Engineered Rural and
Suburban Masonry Houses in Cement Sand Mortar in Earthquake Affected Areas,
Earthquake Reconstruction & Rehabilitation Authority (ERRA), Pakistan.
Feinstein International Center (2011), Examining Linkages Between Disaster Risk Reduction and
Livelihoods, Tufts University, Medford, MA.
HFHA (n.d., a), Minimum Housing Standards, Habitat for Humanity Australia (HFHA), Sydney.
HFHA (n.d., b), Housing & Livelihoods (factsheet), Habitat for Humanity Australia (HFHA),
Sydney.
HFHI-SL (2009), Defining Indicators to Measure Adequate Housing in Sri Lanka, Habitat for
Humanity International – Sri Lanka (HFHI-SL), Colombo.
Hunnarshaala Foundation (2006), Mid-Term Socio-Technical Assessment of Shelter
Reconstruction, UNDP (United Nations Development Programme), New Delhi.
IFRC (2008), Annual Report 2008, International Federation of Red Cross and Red Crescent
Societies (IFRC), Geneva.
Lang, H. (2008), “Community housing in post disaster area on Nias islands, Indonesia: responding
to community needs”, Proceedings of the 4th International I-Rec Conference, University of
Canterbury, Christchurch.
Lizarralde, G. (2002), “Organizational design, performance and evaluation of post-disaster
reconstruction projects”, Improving Post-disaster Reconstruction in Developing Countries
(Conference Proceedings), Université de Montréal, Montreal.
Lyons, M. (2009), “Building back better: the large-scale impact of small-scale approaches to
reconstruction”, World Development, Vol. 37 No. 2, pp. 385-398.
Lyons, M. and Schilderman, T. (Eds) (2010), Building Back Better: Delivering People-Centred
Housing Reconstruction at Scale, Practical Action Publishing, Rugby.
Marti, R.Z. (2005), The 2004 Hurricanes in the Caribbean and the Tsunami in the Indian Ocean,
United Nations Publications, New York, NY.
MDF (2009), External (global) Evaluation Report of the Project “SOA 3/2005” Tsunami Relief,
Rehabilitation & Reconstruction Program (report), Management for Development (MDF)
South Asia, Newala.
NHDA (National Housing Development Authority) (2005), Guidelines for Housing Development in
Coastal Sri Lanka, Ministry of Housing and Construction, Colombo.
IJDRBE Pearce, I. and Batchelor, V. (2010), Evaluating the Long-Term Impact of Shelter Programmes
(PowerPoint Presentation), Habitat for Humanity Great Britain, & Arup International
6,3 Development, London.
Villanueva, P.S. (2010), Learning to ADAPT: Monitoring and Evaluation Approaches in Climate
Change Adaptation and Disaster Risk Reduction, Strengthening Climate Resilience
Consortium, UK.
312 World Bank and SOPAC (2009), Preparedness, Planning, and Prevention: Assessment of National
and Regional Efforts to Reduce Natural Disaster and Climate Change Risks in the Pacific,
World Bank, Washington, DC.
For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com