Number 11 · April 2016

Evaluation Insights are informal working

papers issued by the Network on
Development Evaluation of the OECD
Forests and
Development Assistance Committee (DAC).
The views and opinions expressed here are
those of the author and do not necessarily
reflect the official policy or position of the
Sustainable Forest
OECD DAC or its member countries. This
paper is published under the responsibility
of the Director of the Development Co-
operation Directorate.
Management
Learn more about the DAC Evaluation
Evaluation evidence on addressing
Network at: deforestation to reduce CO2 emissions
Susanna Morrison-Métois, Hans Lundgren
www.oecd.org/dac/evaluation
OECD DAC Network on Development Evaluation Secretariat

Comments and feedback on a previous draft of this paper were received from
Lauren Kelly (IEG), Jeneen Rayes Garcia (GEF-IEO), and Balbir Singh (Norad).
Why forests?
Consensus on the need for international co-operation to combat climate change has resulted in increased attention to the role
of forests in storing carbon and the large quantity of C02 emissions that could be avoided if deforestation was halted.
Deforestation and forest degradation are the second leading human cause of CO2 emissions contributing to global warming
1
according to the Intergovernmental Panel on Climate Change. It is estimated that deforestation and forest degradation account
2
for approximately 17 percent of global greenhouse gas (GHG) emissions. Furthermore, tropical forests capture and store carbon
– since the turn of the century tropical forests are estimated to have removed 22-26% of all human caused carbon emissions. 3
Forests are also important storehouses of biodiversity and provide livelihoods for over a billion people worldwide including
many living in extreme poverty.

Forests at the climate change development nexus

Developing countries have an excellent opportunity to pursue low-carbon development strategies going forward. Many low and
middle income countries are seeking to pursue the twin goals of development (poverty reduction and economic growth) and
combatting climate change. Deforestation and degradation represent over one third of total emissions in developing countries,
where many large tropical forests are found. The important role that forest-rich developing countries can play in combatting
climate change by reducing emissions from deforestation and forest degradation has become central to international dialogues
on preventing global temperature increases as a global public good. There are currently many new initiatives and programmes
working at the forefront of the development/climate change nexus. As the number of policy and programme evaluations in this
area rises, there is an opportunity to learn from existing evidence and emerging findings. Given the importance of the sector, it
is surprising that there have been relatively few attempts to synthesise evidence from evaluations to learn lessons about the use
of development assistance to combat deforestation.
The Sustainable Development Goals
The Sustainable Development Goals underline the need to balance objectives and potential trade-offs between poverty
reduction, growth and sustainability. Goal 15: “Sustainably manage forests, combat desertification, halt and reverse land
degradation, halt biodiversity loss” and Goal 13: “Take urgent action to combat climate change and its impact” place forest
management and sustainability into the international development framework and underscore the importance of these
objectives in both developing and developed countries.

1
The UNFCCC & COP21

At the 2015 United Nations Climate Change Conference in Paris, 195 nations reached a climate agreement with the ambitious
goal of pursuing efforts to limit global temperature increases to 1.5° C above pre-industrial levels. At this meeting the Parties to
the United Nations Framework Convention on Climate Change (UNFCCC) negotiated the text of the agreement which explicitly
acknowledges the key role of forests in combating climate change. At the same time, the governments of Germany, Norway and
the United Kingdom pledged to provide US $ 5 billion (by 2020) in financial support for countries implementing Reducing
Emissions from Deforestation and Degradation (REDD+) programmes and to scale up support for technical assistance and
capacity building. To maintain the momentum from COP21, it is appropriate to take a closer look at the existing evidence on
forest sector programmes in developing countries.

EMERGING EVALUATION EVIDENCE

There is a growing body of evidence from recent evaluations

conducted by the World Bank Group, the United Nations
and OECD DAC countries’ development ministries and
agencies on using ODA to incentivise reform. This paper
aims to give insights into forest management and
deforestation programming – it highlights findings from a
number of recent evaluations and discusses some of the
4
various approaches and programmes. It aims to attract
attention to the existing evidence base and to highlight
areas that merit further analysis. The paper concludes with
brief policy implications based on emerging evaluative
evidence.
Source: Susanna Morrison-Métois

COMMON EVALUATION FINDINGS

This section highlights some common findings from recent evaluations of interventions in the forest sector, with respect to: 1)
synergies and trade-offs between different goals; 2) co-ordination, alignment and leadership from partners and donors; 3)
inclusive engagement of stakeholders and local ownership; and 4) specific findings on common programmatic approaches.

1. Trade-offs between climate change objectives and other goals

International efforts to help developing countries decrease deforestation rates must balance carbon reduction and development
goals and strive to formulate clear, coherent models of change. Evaluations of UN, World Bank Group and bilateral projects on
deforestation and sustainable forest management often highlight the need to clearly articulate a vision of long-term progress
(or theory of change) and to better define and measure the delicate balance between environmental, poverty reduction and
5
other social goals or objectives. Furthermore, programme managers, policy makers and evaluators must do more to capture
and take into consideration ‘co-benefits’, which should be more explicitly defined in programme planning and policy. Several
recent evaluations have underscored the need to better understand the potential trade-offs between climate objectives and
6
broader development benefits.

A few evaluations have gone a step further and have questioned whether programmes which may have proven successful in
preventing deforestation include and provide positive benefits for the poorest households, and whether these goals are fully
7
compatible. Several mention the growing agreement on the need to further encourage and promote sustainable livelihoods
for those living in or near forest areas and to address land-use issues as a necessary step in reducing deforestation rates and CO2
8
emissions. Additionally, the drivers of deforestation are often factors outside of the forest sector and therefore programmes
designed to halt or reduce deforestation must address a broader range of related issues, including: land tenure, agricultural
policies, the potential for climate smart agricultural practices, alternative livelihoods, livestock and gazing practices, urban
9
expansion, mining policies, and other social and economic drivers of deforestation and change in land use.

Encouragingly, there is evidence that programme managers and policy makers have been responsive to recommendations
concerning the need to better clarify programme objectives and rationales. For example, the UN REDD Programme made
10
changes to its official strategy following the 2014 programme evaluation. The new REDD Programme strategic framework
includes a clearly articulated theory of change, demonstrating that international programmes have been able to adapt and
11
incorporate learning from evaluative findings. The World Bank Group’s strategy in the forest sector has also evolved over
12
time, in part as a result of the Independent Evaluation Group’s (IEG) findings. Additionally, the Forest Carbon Partnership

2
Facility (FCPF) Readiness Fund first established an M&E framework following a recommendation made in the Facility‘s first
13
programme evaluation.

Overall, policy and strategy has moved in the direction of more explicitly recognising the need to balance environmental, social
and economic objectives and more clearly articulating theories of change. Greater attention is being paid to measuring
intermediate outcomes and objectives that are expected to lead to longer term impacts. Policies and programmes in the sector
have become more holistic in addressing a broader range of issues rather than maintaining a narrow focus on forest carbon.
Notwithstanding, there have been some concerns raised in evaluations that despite increased recognition of various tensions
and trade-offs, more still needs to be done to increase synergies between development and environmental goals.

2. Need for co-ordination, alignment and leadership from partners and donors

The complexity of multilateral frameworks working to halt deforestation and the intricate international architecture of aid
14
delivery in the forest sector has featured in a number of evaluations and reviews. While some question the necessity of the
complicated aid architecture in this sector, the more general finding is that the multiplicity of institutions and financing
mechanisms requires greater levels of co-ordination among donors and partners. Several evaluations underscore the need for
15
co-operation among donors working at the country level and concerning decisions on financing for multilaterals. Overall,
recent evaluations suggest that multilateral aid in the forest sector has shown mitigation effectiveness and is particularly
important in helping to ensure that donors respect their environmental commitments. Pooled funding mechanisms are
considered essential to gather momentum and ensure harmonisation of approaches between donors. There is also a need for
collaboration between multilateral initiatives. The 2014 evaluation of the UN REDD programme, for example, recommended
that UN agencies further their collaboration with the World Bank’s FCPF in order to harmonise approaches and to reduce the
16
duplication of effort.

There have been some challenges with multilateral programmes and pooled funding, such as those seen with the UN REDD
programme, which has had a relatively slow rate of implementation. These challenges seem to stem from unanticipated
obstacles and capacity gaps that need to be addressed prior to full
programme implementation. Recent evaluations highlight the complexity of
initiatives in the sector, the slow pace of progress and the need for
programmes to adopt a cross-sectorial approach. A global programme
review of the FCPF conducted by IEG in 2012 summarised this view, stating
that “the REDD+ readiness process is a more expensive, complex, and time-
consuming process than originally envisaged” and suggested that a cross-
17
sectorial approach would help increase effectiveness. Similarly the 2014 UN
REDD Programme evaluation found that “UN partner agencies and
participating countries should place greater emphasis on integrated cross-
18
sectorial approaches to REDD+…”.

Not only is a cross-sectorial approach needed, but bilateral development

providers also need to focus on environmental policy integration and the
overall coherence of their international development co-operation
portfolios. A recent evaluation by Belgium found that while many of their
programmes had positive results in stabilising or increasing the land area
covered by forests, greater attention could be paid to negative
environmental externalities, such as the impact of other programmes on
19
forest coverage. Furthermore, a 2014 SDC evaluation, ‘Swiss International
Co-operation in Climate Change 2000-2012’, noted that climate change
mitigation requires long-term commitment and suggested that it be
“mainstreamed as an aspect of all development projects… This approach Source: Susanna Morrison-Métois
would ensure that good progress in climate change mitigation or adaptation
20 21
is not undermined or ‘undone’ as a result of other interventions.”

3
 How Brazil reduced deforestation – A case study
Brazil is often considered the success story in efforts to halt deforestation. From 2000 to 2012 GHG emissions declined in Brazil by
over 40%. This impressive reduction in GHG emissions happened at a time of economic growth [per capita income in Brazil
increased by approximately 30% between 2000 and 2013 (OECD 2015)].

How did this happen?

In 2004 Brazil’s government launched the Action Plan for Prevention and Control of Deforestation in the Amazonia Legal and has
passed legislation such as the Forest Code, a key legal document, which requires landholders to set aside a share of their land for
forest and soil conservation and restoration. In addition, the business sector has been brought on board and a number of large
companies have made voluntary agreements to stop using soya beans that have been grown on illegally cleared land.

What practices can other countries seek to replicate? What is unique about Brazil?
Brazil’s protected areas system is one of the largest in the world and is organised under the National System of Protected Areas
(SNUC). The SNUC requires public consultations with local communities, stakeholders and relevant sectorial institutions prior to
the establishment of new protected areas (OECD, 2015).

Brazil has also implemented PES programmes and income support schemes designed to compensate communities for
environmental conservation. Furthermore, Brazil is considered the world’s leader in monitoring deforestation via satellite imaging.
New technology and almost real time information monitoring has helped improve targeting and enforcement. Civil society
organisations in Brazil have also been actively engaged and influential in this sector.

Finally, Brazil has benefited from international development co-operation funds from the World Bank Group, Norway, Germany,
the UK, and other providers. The Amazon Fund, for example, was launched in 2008 and as of early 2015 had accumulated more
than 970 million USD and supported over 70 projects (OECD, 2015). Significantly, Brazil has received around half of the total
approved international finance from REDD+ (through the Amazon Fund) and Brazil was the first country to submit its forest
reference emissions levels for payments under REDD+. A 2014 real-time evaluation of Norway’s International Climate and Forest
Initiative emphasised how:

“In Brazil, Norway’s US $ 1 billion provided validation of national reforms already underway to tackle deforestation, and
probably gave strength to continue this route… the funds were a motivator which had a much deeper and potentially
more sustainable change rather than an economic incentive, which was likely to have only a short-term impact and be
potentially less sustainable.” (Norad, 2014, P. XXV)

Despite Brazil’s impressive reduction over the last 10 years, there are concerns that these achievements may not be sustained,
21
suggesting that efforts to maintain low rates of deforestation must continue.

[Information in this section partially adapted from OECD Environmental Performance Reviews: Brazil 2015]

A few evaluations have expressed concern regarding the large number of partner countries now engaging in multilateral
22
programmes. In particular, there is concern that some partners interested in REDD+ readiness programmes may lack capacity
or have unrealistic expectations; some evaluations question the appropriateness of engaging such a large number of new
partner countries when many have faced challenges moving successfully into the results based payment phase. For instance, as
of early 2016 there are 64 partner countries engaged in the UN-REDD Programme. Hence there are apprehensions, such as
those highlighted in a 2014 ‘Real-time evaluation of Norway’s International Climate and Forest Initiative (NICFI)’, about raising
the expectations of partners unrealistically when an international co-operation agreement on financing for REDD+ has so far not
been put in place. The 2012 Global Review of the FCPF mentions the large number of interested client countries as a case of
23
‘over-demand’. Serious discussions among donors, multilaterals and partner countries need to be had to assess if the current

4
approach of multilateral engagement with a large number of partner countries is advisable and realistic, including how these
24
programmes can best align with country generated strategies.

Norway’s 2014 real-time evaluation of its forest sector programme found that strong bilateral relationships, such as between
Norway and Brazil, can lead to successful outcomes; however there are concerns that this success may not be replicable with
other partners. While evaluations have repeatedly highlighted Brazil as a key success story, it has often been noted that this
25
success has been possible due largely to the strong leadership and political will in Brazil to halt deforestation. Norway played
a key role in supporting Brazil and the Amazon Fund with financing and strong partnership; although the most recent real-time
evaluation of Norway’s large-scale commitments in the sector suggest that Norway’s bilateral relationships with other key
partners have advanced less quickly. The uneven progress has been attributed to differences in local governance and varying
levels of commitment and political decision-making within partner countries.

Donors such as Norway have been successful in using large financing commitments to gather support for and to influence the
international dialogue on approaches in the sector. Evaluations of Norway’s and Switzerland’s development co-operation in the
forest sector both conclude that their countries’ aid has been crucial in advancing the dialogue and has influenced decisions and
contributed to progress made on REDD+ and in the UNFCCC climate change negotiations. Generally, there is strong recognition
of the need for co-ordination among donors, particularly in regards to financing for multilaterals and on thinking about ways to
improve their transaction costs and to address the challenges caused by the slow implementation process, which has often led
26
to a lower level of disbursements than anticipated.

3. Local ownership and inclusive engagement of national and local stakeholders

Recent evaluations suggest that more attention should

be given to local ownership and ensuring inclusive
Challenges for putting safeguards ‘in practice’:
dialogue with a wider range of stakeholders. Some
Lessons from the UN-REDD Programme?
evaluations, such as the 2014 UN REDD Programme
“The UN-REDD Programme has been a strong voice in the
evaluation, have recommended “a more bottom-up
27 prioritisation of free, prior and informed consent (FPIC)
approach”. Within this evaluation were some
principles and the recognition of Indigenous Peoples’ rights in
concrete recommendations on how to ensure greater
the implementation of UNFCCC safeguards, especially in
partner country ownership including: 1) prioritising the
decision-making fora. Programme contributions to social and
use of national and regional experts 2) using existing
environmental safeguards are helping to raise awareness at
structures and co-ordination mechanisms 3)
national and global levels, but it remains a challenge to put such
strengthening national capacity in regards to financial
principles into practice, notably due to the high number of
management and accountability mechanisms, and 4)
safeguards and the lack of clear guidance on how to implement,
delegating decision making authority to the level
monitor, and enforce these. “ – (UN-REDD Programme, 2014c)
closest to the field “while applying basic principles for
28
robust resource governance…”.

Evaluations also conclude that local stakeholders, particularly forest-dwelling communities and those that depend on forest
resources for their livelihoods could be better incorporated into local forest governance structures. While there has been
growing recognition of the rights of indigenous groups, recent evaluations have highlighted the need to consider the
29
participation of non-indigenous local groups and the specific role of women as forest users and stakeholders. Programmes
must seek to understand the ways that forest dwellers and local people traditionally use and manage forest resources and
better understand the existing governance structures, local access issues, land tenure and tribal or local authority systems.
These findings have led to a greater acknowledgment of the fact that success in halting deforestation may depend on the extent
to which programmes incorporate understanding of these systems and practices into forest management programming. This
requires more careful consideration and analysis of local contexts, including more extensive political economy analysis and the
deliberate inclusion or design of mechanisms to ensure that local peoples’ needs and voices are fully incorporated into
programming.

While programmes in the sector have developed official policies of inclusiveness and increasingly work with more local
stakeholders, some evaluations such as the 2014 UN REDD Programme evaluation have noted that substantial ownership and
decision making powers regarding programming may be lagging. There is a need to move from the ‘recognition of local peoples
rights’ to ensuring that in practice, local communities are fully involved in all stages of programming, starting from conception.
Well-organised civil society groups have been relatively effective in influencing the dialogue and practice in many countries,
however, it is likely that the needs and views of groups lacking more structured representation will need continued attention.
At the same time, there is acknowledgement that the process of involving more stakeholders and the move towards greater
inclusiveness comes with its own set of trade-offs and potential short-term efficiency losses.

Repeatedly, success in reducing deforestation rates and progress within the REDD+ framework has been attributed to strong
local political will in partner countries. Many stakeholders have clearly understood that future payments and distributions of
benefits must be equitable and that there must be assurances that the benefits will accrue to local people, to avoid the risk of
elite capture, if programmes are not well-managed. Overall, the trend has been towards recognition of the need to focus more
5
extensively on capacity building, readiness and ensuring inclusive local governance. Significant attention and funding must
continue in these areas as potential achievements in halting deforestation and cutting CO2 emissions appear, unsurprisingly, to
rely to a large degree on strong local political will and good governance.

4. Emerging specific findings on programme approaches

This section looks briefly at what some recent evaluations have to tell us about the effectiveness of various approaches to
programming in the forest sector, specifically looking at the available evidence on: 1) payments for environmental services; 2)
protected areas; and 3) community / participatory forest management.

Payments for Environmental Services

A common approach in forest sector programming has been the use of Payments for Environmental Services (PES) which have
frequently been implemented under the REDD+ umbrella. A 2014 systematic review of PES schemes drew evidence from 11
30
quantitative evaluations of PES programmes in four countries (Costa Rica, China, Mexico and Mozambique). It concluded that
31
PES schemes likely have a modest, positive effect on deforestation. However, there appears to be a lack of evidence that PES
has beneficial effects on poverty, with evidence suggesting that PES schemes have been “less effective in poor areas and are less
32
likely to attract the participation of poor households than wealthier ones.” A 2013 IEG evaluation, for example, found that
33
“…running PES in the most cost-effective manner may conflict with other objectives such as poverty alleviation”. The
evaluation found that there are “possible trade-offs between the effectiveness of particular payment arrangements and the
34
transaction costs of implementing them”. Furthermore, an IEG evaluation entitled ‘Climate Change Phase II: The Challenge of
Low Carbon Development, Climate Change and the World Bank Group’ found that while PES schemes have been popular, “a
substantial proportion of payments has gone to areas that are not at high risk for deforestation, diluting carbon and
35
environmental benefits and prompting attention to targeting.” While attention to targeting and poverty has increased in PES
schemes, to date there does not appear to be a strong evidence base demonstrating that conservation and poverty-reduction
36
goals can effectively be complementary in PES programmes. That said, the existing evidence base in this area is weak and more
37
efforts should be made to assess the effects of PES programmes in order to potentially refute these initial findings.

Protected Areas
An approach which has been widely considered successful for the conservation
38
of forest resources is the creation of protected areas (PAs). Evaluations of
protected areas agree that they are at least modestly effective in reducing
deforestation and suggest that when local people and forest dwellers are part
of the forest management process, protected areas are even more successful.
For example, a global analysis conducted in 2010 by IEG found that
“…[protected areas] are on average effective in reducing deforestation. Areas
that allow sustainable use are more effective than strictly protected areas and
39
indigenous areas are most effective of all.” This finding was based on a study
by Nelson and Chomitz (2009) that used spatial data on protected areas and
40
the location of forest fires, as a proxy indicator of deforestation. A 2013 IEG
evaluation, ‘Managing Forest Resources for Sustainable Development’
confirmed that “the level of community participation in the management of
41
protected areas mattered both in terms of conservation and sustainability”.
The evaluation warned against ‘’the siloed nature of many protected areas’’ as
too often livelihoods are considered in the context of alternative income
42
schemes without addressing the root causes of deforestation. There have
been some concerns that there is not enough evidence to date that protected
areas specifically designed to address biodiversity are, in fact, effective at
43
Source: Susanna Morrison-Métois achieving this objective. IEG has also pointed out that few projects integrate
44
sensitivity to climate change into the monitoring and design of PA projects. A
major recent (2015) impact evaluation by the Global Environment Facility which included the use of remote sensing to measure
forest coverage, found that GEF supported protected areas have had positive impacts on biodiversity, yet recommended that
more should be done to improve targeting, to address socio-economic conditions in local communities, to improve results
45
monitoring and to address large-scale drivers and governance issues. The evaluation recommended more investment in
understanding what works and why, including looking further at how to address local livelihood needs.

Participatory Forest Management / Community Forest Management

Participatory or Community forest management (PFM or CFM), as it is often called, has been a frequent approach in the sector.
The FAO defines participatory forestry as “the processes and mechanisms which enable people with a direct stake in forest
46
resources to be part of decision-making in all aspects of forest management, including policy formulation processes”. While
PFM/CFM is a common approach, the evidence base on its results and impacts is mixed and highly context specific. A 2015
evaluative study conducted
Curt Carnemark, by theBank
1993 / World Natural Environment Research Council entitled ‘Effectiveness of Community Forest
Management at reducing deforestation in Madagascar’ found that Community Forest Management, as broadly defined,

6
 47
encompasses various practices and approaches and “is not a guarantee of forest conservation”. Significantly the study found
that community forest management which allowed commercial use did not reduce deforestation, while the areas that
prohibited commercial forest use did manage to reduce deforestation. It also concluded that “differentiating among types of
48
community forestry is important in impact evaluation”. A 2010 review of the evidence base for community forest management
by The Scientific and Technical Advisory Panel (GEF) found that “there is insufficient evidence to form any conclusions about the
49
effect of CFM on local livelihoods, which is in part due to the absence of common indicators of CFM effects on livelihoods”.
Furthermore the study was unable to clearly determine the impacts of CFM as there was often no clear pattern. This study
highlighted the challenge of comparing and synthesising findings between programmes due to differences in indicators and
methodologies used, they noted that “of great concern for the studies reviewed is the inability of most studies to credibly
50
eliminate rival explanations of any changes, or absence of changes, in the outcome variables.”

Comparably, IEG’s 2013 evaluation found examples of success and highlighted best practices in countries such as Mexico. The
World Bank supported Mexico’s Community Forestry Program; the initial pilot programme has been institutionalized and is now
implemented nationwide. The evaluation also contains examples from World Bank supported PFM programmes in India, Lao
51
PDR and Liberia. Moreover, the evaluation outlined some of the successes of Participatory Forest Management, finding
evidence of positive outcomes:

“Participatory Forestry Management projects exhibit the most balanced goals as compared to other interventions in the
sector. World Bank support for participatory forestry management has yielded positive livelihood benefits, such as the
generation of employment, increased incomes, and diversification of revenue streams. Where specifically targeted,
these projects have also achieved positive environmental outcomes such as reduced deforestation rates, regeneration of
degraded forests, reduced incidence of fires, and protection of biodiversity.” (IEG, 2013b p. XIII)

This evaluation highlighted that while the World Bank successfully worked with the formal sector through programmes such as
concession regimes and the creation of formal conservation areas, it ought to do more to work with actors in the informal sector
52
upon which many forest-dwelling communities are dependent.

Hence, while there is still limited evaluation evidence on the overall effectiveness of community forest management (likely due
to the lack of common indicators and the context specific nature of CFM/PFM), there is evidence from the World Bank Group’s
portfolio that participatory forest management can be successful in achieving the triple goals of poverty reduction, growth, and
sustainability.

WHAT WE NEED TO KNOW MORE ABOUT

While there is a growing body of evidence on what approaches work best in the forest sector, there are a large number of
remaining unknowns and some obvious outstanding evidence gaps. Four areas where strong evaluative evidence and/or
international consensus is clearly lacking are: 1) the appropriateness and effectiveness of results based payments; 2) challenges
and issues around measuring progress, intermediate outcomes, and longer term results; 3) the cost effectiveness of multilateral
initiatives and country specific cost effectiveness for engagement; and 4) the role and potential of the private sector and forest
concessions in conservation and sustainable forest management.

The most obvious evidence gap is related to the lack of consensus around the use of results-based payments. There are a
number of questions that have not been answered related to results-based payments, questions such as: How well do they
work? Do they provide the right incentives to the right people? What quantity of CO2 emissions is actually avoided? There have
been a number of debates about how to ensure accurate baseline data and whether measurement and verification systems can
be robust and precise enough to ensure that payments are made based on accurately measured results. Furthermore, there are
still many who question the payment for results approach, who argue that forests and other natural resources have intrinsic
value, hence paying local people or developing countries to protect forests risks not only undermining the intrinsic value of
these resources, but also shifts the responsibility for protection from a local responsibility to a potentially less sustainable
programme approach dependent on financing from developed nations.

Another area that emerges as a continuous challenge is issues relating to the measurement of progress. Climate change and
environmental degradation are long-term processes that often follow non-linear trajectories which pose a number of problems
for monitoring and evaluation. For one, it has often proven challenging to define significant outcome indicators, as change in
this sector is often measured by proxy indicators and over short time horizons. Secondly, programmes and approaches such as
REDD+ readiness programmes have invested considerable time and money on intermediate steps (capacity building,
partnerships, local governance issues, etc.). At times, the theories of change relating these intermediate steps to long-term
outcomes have not been clearly articulated and appropriate indicators to measure intermediate outcomes have hence been
neglected. As more time has been spent on preparedness and intermediate steps than originally anticipated at the onset, it has
proven both more important and more challenging to evaluate the effectiveness of these intermediate steps. While there are
many sources of hard data or scientific evidence that allow us to measure CO2 emissions or to visualise the area of forest
coverage, there are fewer internationally agreed upon indicators and standards for measuring the extent of relevant

7
 53
international partnerships and capacity building processes. The challenges of monitoring progress underscore the importance
of considering evaluability in the initial stages of programme design. Likewise, more attention ought to be given to incorporating
54
rigorous impact evaluation methods and/or quasi experimental design approaches into programming. This would help better
ensure that programmes are achieving their stated objectives and are fully accountable.

The need to pay more attention to measuring and evaluating the cost effectiveness of programmes is related in many ways to
55
the slower than anticipated implementation of multilateral initiatives, such as REDD+. In order to ensure the cost effectiveness
of REDD+, it is important to continue to improve cost analyses and models both for multilaterals and for partner countries.
Country-driven strategies and increased political economy analysis may help to better anticipate and address countries' specific
advantages and challenges earlier in the process. As the drivers of deforestation are often country specific, donors and
multilateral will need to work with partners to support context specific approaches, while also working to ensure cost
effectiveness and achievability.

Another large evidence gap is related to the role and impact of

the private sector in sustainable forest management. More
research and evaluation work ought to be done to better
determine the scale, positive and negative
consequences/outcomes, and potential future benefits of
increased private sector engagement. There are concerns,
expressed in some evaluations, that some forest or biodiversity
conservation approaches may unintentionally curtail private
sector development. Better country level political economy
analysis of forest use should be undertaken to gain a more
complete understanding of the utilisation of forest resources in
the informal sector, the extent of illegal forest activities, and the
current level of private sector interest and opportunities. While
illegal logging has rightly been recognised as a crucial problem
and is frequently addressed through programmes focused on Source: Yosef Hadar/ World Bank
improving governance, the full scale and potential of forest resources deserves further attention from evaluators. Initiatives led
by the private sector and driven by consumer demand (such as commitments to deforestation-free supply chains and
commodities) have also shown high potential. Increased involvement and engagement with the private sector is inevitable for
those seeking to achieve long-term forest conservation goals, hence more evaluation studies should be commissioned in this
area. In particular, the potential to leverage additional financing from private sector actors and the repartition of benefits at the
local level deserve further attention from evaluators. That said, efforts by development actors to better understand and
promote private sector approaches are likely to be contingent upon the development of appropriate legal frameworks (and
institutions for their enforcement) which are capable of adequately protecting local peoples’ rights and interests.

IMPLICATIONS FOR POLICY MAKERS

Evaluation findings in the forest sector to date suggest that:

1) Deforestation and forest management programmes must work from the conception stage with a wide range of local
stakeholders to design programmes and approaches ‘from the bottom up’. Evidence suggests that inclusive processes are
essential to address and overcome local obstacles and barriers to behaviour change and to bridge capacity gaps while working to
improve local governance. Inclusive processes must be ensured from the onset in order to better anticipate challenges and to
build consensus at an early stage. While funding and co-ordination between donors needs to take place at a global level, there is
growing evidence to suggest that inclusive, country-driven approaches are needed.

 Programmes need to ensure the inclusive participation of civil society groups, local peoples, local government
authorities, women, the rural poor and traditionally marginalised groups in decision making processes.
 Financial partners should focus on designing flexible programmes, promoting decentralised decision making and
supporting greater country ownership so that programmes are fit to context.
 Programme managers and policy makers should ensure that M&E systems are designed from the onset that include
indicators on inclusiveness, local ownership and substantive stakeholder involvement.

2) Bilateral development co-operation providers should continue to invest in pooled financing mechanisms and multilateral
programmes (such as the UN REDD Programme, GEF and World Bank initiatives), but should remain engaged in order to
ensure that these mechanisms are inclusive, pro-poor and efficient. Financial partners have a role to play in influencing the
future development of these initiatives. Focus should be given to leveraging the comparative advantages of each
partner/programme while pursuing a harmonised policy approach in relevant sectors. Multilateral programmes working on
REDD+ readiness programmes have proven to be adaptable and have led to successes in assisting countries achieve emission

8
reductions through the provision of technical services and knowledge sharing. However, to date these programmes have not
56
made as much progress as originally expected in piloting performance-based systems of payments.

 Policy makers should ensure that reporting on the application of safeguards is thorough and consistent.
 As evidence seems to suggest that political will is a necessary precondition for successful programming, policy makers
and bilateral development providers may need to consider that countries which currently lack strong political will to
halt deforestation may require different modalities of engagement over longer timeframes in order to gradually
increase awareness and local political buy-in, to ensure local ownership of programmes and to foster the emergence of
country specific approaches. At the same time, financial partners need to work with multilaterals to ensure that
programmes with partner countries remain focused and based on realistic expectations.

3) Bilateral providers and multilaterals have room to improve programme monitoring and results frameworks and can also
work to improve communication on programme achievements. These frameworks must be designed to measure both climate
change (CO2, biodiversity) and socio-economic goals. More efforts and resources need to be dedicated to monitoring and
communicating results, including communications targeted towards general audiences. There is also room for increased learning
which could come from more systematic reviews and more evaluations looking at the coherence of bi-lateral providers’ overall
portfolios for achieving climate change goals. Policy makers should increase the level of focus on evidence and on the
communication of evaluation findings.

4) There is a strong opportunity for innovation and pilot programmes. Policy makers should continue or increase financing for
innovative pilot programmes and projects and should accept high risk tolerance levels, generating evidence on effective
programmes. Innovative pilot programmes such as those using new technologies and real-time monitoring will likely need
increased public sector funding, to develop the tools and methods needed to help developing countries implement their
Nationally Determined Contributions.

CONCLUSION

Deforestation and climate change are global challenges and the international community must develop more inclusive
partnerships, institutions and financing mechanisms to work with partner countries, local communities and local peoples to
address them. There has been growing pressure for visible results and rapid successes, while progress on programmes such as
REDD+ and readiness initiatives have proven to take time and persistence to build strong coalitions, change attitudes and
behaviours and gain political and local buy-in. Evidence from a range of evaluations in the forest sector suggest that success can
only be achieved when local ownership is at the heart of programme design and implementation, when countries and local
communities are empowered and motivated to find workable solutions and alternatives to deforestation. Therefore, the
international community, including bilateral donors, multilateral funds and institutions, and partners in developing countries,
must work together where there is political will to find country level solutions. There is reason to be optimistic as current
programmes are well positioned and capable of adapting while moving forward on climate change goals and commitments.

Source: Susanna Morrison-Métois

9
SOURCES
Baastel and ECO Consult (on behalf of FCPF) (2013), Readiness Fund: Monitoring and Evaluation Framework for the FCPF, Forest
Carbon Partnership Facility.

Baastel and Nordic Agency for Development and Ecology (commissioned by FCPF) (2011), First Program Evaluation for the Forest
Carbon Partnership Facility.

Nelson, A., Chomitz, K. (2009), Protected Area Effectiveness in Reducing Tropical Deforestation A Global Analysis of the Impact of
Protection Status, Evaluation Brief 7, Independent Evaluation Group, Washington, D.C.

FAO (2015a), Community Based Forestry, www.fao.org/forestry/participatory/en/ (accessed 8 November 2015).

FAO (2015b), Strengthening Forest Resources Management and Enhancing its Contribution to Sustainable Development, Land
use and Livelihoods Project-GCP/GLO/194/MUL: Final Evaluation Report, Food and Agriculture Organization of the United
Nations, Rome.

FAO and JRC (2012), Global forest land-use change 1990–2005, by E.J. Lindquist, R. D’Annunzio, A. Gerrand, K. MacDicken, F.
Achard, R. Beuchle, A. Brink, H.D. Eva, P. Mayaux, J. San-Miguel-Ayanz & H-J. Stibig, FAO Forestry Paper No. 169, Food and
Agriculture Organization of the United Nations and European Commission Joint Research Centre, Rome.

GEF (2015), Impact Evaluation of GEF Support to Protected Areas and Protected Area Systems, The Global Environmental Facility,
Washington, D.C.

GEF (2012), Evaluation of the GEF Focal Area Strategies, Technical Paper 6: SFM/REDD+, Global Environmental Facility,
Washington, D.C.

GEF (2006), The Role of Local Benefits in Global Enviornmental Programs, Global Envionrnmental Facility, Washington, D.C.

Gelo D., Koch S. and Muchapondwa E. (2013), “Do the poor benefit from devolution policies? Evidence from quantile treatment
effect evaluation of joint forest management,” Economic Research Southern Africa (ESRA) Working paper 400.

Goodman Rosa C. and Herold Martin (2014), "Why Maintaining Tropical Forests Is Essential and Urgent for a Stable Climate,"
CGD Working Paper 385, Washington, DC: Center for Global Development. http://www.cgdev.org/publication/why-maintaining-
tropical-forests-essential-and-urgent-stable-climate-working-paper-385

Hardcastle P, Davenport D, Cowling P, Watson C (2011), Discussion of Effectiveness of Multilateral REDD+ Initiatives, IDL Group
for REDD+ Partnership. http://www.fao.org/partnerships/redd-plus-partnership/29149-
0db58e819221ae34c06b8182087d74669.pdf

Hardcastle P, Forbes A, Karani I, Tuominen K, Sandom J, Murtland R, Müller-Plantenberg V & Davenport D (2010), Evaluation of
Finnish Support to Forestry and Biological Resources, Evaluation report 2010:5/I Main report, Ministry for Foreign Affairs of
Finland, Kopijyvä Oy, Jyväskylä, 174 p. ISBN 978-951-724-876-1 (printed).

ICF International (2014), Independent Evaluation of the Climate Investment Funds, World Bank, Washington, D.C.

IEG (2015), Project Performance Assessment Report, Brazil: First Programmatic Reform Loan for Environmental Sustainability
(LOAN NO. 7256-BR), Independent Evaluation Group, Washington, DC.

IEG (2014), Project Performance Assessment Report, Brazil: Ecosystem Restoration of Riparian Forests in Sao Paulo, Independent
Evaluation Group, Washington, D.C.

IEG (2013a), Brazil Country Program Evaluation FY 2004–11, Independent Evaluation Group, Washington, DC: World Bank.
License: Creative Commons Attribution CC BY 3.0.

IEG (2013b), Managing Forest Resources for Sustainable Development: An Evaluation of World Bank Group Experience,
Independent Evaluation Group, Washington, D.C.

IEG (2012), The Forest Carbon Partnership Facility, Global Program Review Vol. 6, Issue 3, Independent Evaluation Group,
Washington, D.C.

10
IEG (2010), Phase II: The Challenge of low-carbon development, Climate Change and the World Bank Group, Independent
Evaluation Group, Washington, D.C.

IPCC (2007), Climate Change 2007: Synthesis Report. Contribution of Working Groups I, II and III to the Fourth Assessment
Report of the Intergovernmental Panel on Climate Change [Core Writing Team, Pachauri, R.K and Reisinger, A. (eds.)]. IPCC,
Geneva, Switzerland, 104 pp.

JICA (2015), JICA Annual Evaluation Report 2014, Evaluation Department Japan International Cooperation Agency, Tokyo.

Joppa, L. Pfaff, A. (2011) Global Protected area impacts, Proceeding of the Royal Society of London: Biological Sciences, vol.
278(1712), pp 1633-1638.

Karousakis, K. (2009), "Promoting Biodiversity Co-Benefits in REDD", OECD Environment Working Papers, No. 11, OECD
Publishing, © OECD. doi:10.1787/220188577008

Kissinger, G., M. Herold, V. De Sy (2012), Drivers of Deforestation and Forest Degradation: A Synthesis Report for REDD
Policymakers, Lexeme Consulting, Vancouver.

Kronenberg, J., and K. Hubacek. (2013), Could payments for ecosystem services create an "ecosystem service curse"?, Ecology
and Society 18(1): 10. http://dx.doi.org/10.5751/ES-05240-180110

Lele, Uma, Zazueta, Aaron and Singer, Benjamin (2011), "The Environment and Global Governance: Can the Global Community
Rise to the Challenge?," Climate Change and Land Policies, eds. Ingram, Gregory K. and Yu-Hung Hong., Lincoln Institute of Land
Policy, Cambridge, MA.

Lund JF, Balooni K, Casse T, (2009), Change We can Believe in? Reviewing Studies on the Conservation Impact of Popular
Participation in Forest Management, Conservat Soc [serial online] vol. 7 pp. 71-82.

Miteva, D. A., et al. (2012), "Evaluation of biodiversity policy instruments: what works and what doesn’t?," Oxford Review of
Economic Policy 28(1): 69-92.

Norad (2014), Real-Time Evaluation of Norway’s International Climate and Forest Initiative Synthesising Report 2007-2013,
Norwegian Agency for Development Cooperation, Oslo.

Norad (2012), Real-Time Evaluation of Norway’s International Climate and Forest Initiative: Lessons Learned from Support to Civil
Society Organisations, Norwegian Agency for Development Cooperation, Olso.

Norad (2011), Real-Time Evaluation of Norway’s International Climate and Forest Initiative Contributions to National REDD+
Processes 2007-2010 Country Report: Brazil, Norwegian Agency for Development Cooperation, Olso.

Norad (2010), Real-Time Evaluation of Norway’s International Climate Change and Forest Initiative, Norwegian Agency for
Development Cooperation, Oslo.

Nelson A. and K. Chomitz (2009), “Protected Area Effectiveness in Reducing Tropical Deforestation: A Global Analysis of the
Impact of Protection Status,” Evaluation Brief 7, Independent Evaluation Group, Washington, D.C.

OECD (2015), OECD Environmental Performance Reviews: Brazil 2015, OECD Publishing, Paris.
http://dx.doi.org/10.1787/9789264240094-en

OECD (2013), Scaling-up Finance Mechanisms for Biodiversity, OECD Publising, Paris DOI:10.1787/9789264193833-en

OECD (2010), Paying for Biodoversoty: Enhanching the cost-effectiveness of payments for ecosystem services, OECD Publishing,
Paris. DOI:10.1787/9789264090279-en

Office of the Special Evaluator for Development Cooperation/SEO (2014), How green is our Development Aid? Thematic
evaluation of the Belgian Cooperation in the Field of Environment, Federal Public Service Foreign Affairs, Foreign Trade and
Development Cooperation, Brussels.

Pattanayak S. K., Wunder S & Ferraro P. J. (2010), Show me the Money: Do Payments Supply Enviornmental Services in
Developping Countries, Review of Environmental Economics and Policy, Vol. 4 (2), pp 254-274.

Porter-Bolland, L., et al, (2011) Community managed forests and forest protected areas: An assessment of their conservation
effectiveness across the tropics, Forest Ecology and Management, doi:10.1016/j.foreco.2011.05.034
11
Rasolofosan R. A., P.J. Ferraro, C.N. Jenkins & Julia P.G. Jones (2014), Effectiveness of Community Forest Management at
reducing deforestation in Madagascar, Biological Conservation, Vol. 184, pp. 477-458.

Samii C, Lisiecki M, Kulkarni P, Paler L, Chavis L (2014), Effects of Payment for Environmental Services (PES) on Deforestation and
Poverty in Low and Middle Income Countries: A Systematic Review, Campell Systematic Reviews.

Scientific and Technical Advisory Panel of GEF (2010), The Evidence Base for Community Forest Management as a Mechanism for
Supplying Global Environmental Benefits and Improving Local Welfare, Scientific and Technical Advisory Panel of the Global
Environmental Facility, Washington, D.C.

SDC (2014), Reports of Effectiveness 2014: Swiss International Cooperation in Climate Change 2000-2012, Swiss Agency for
Development and Cooperation, Bern.

Spracklen, D. V., and L. Garcia-Carreras (2015), The impact of Amazonian deforestation on Amazon basin rainfall, Geophys. Res.
Lett., 42, 9546–9552.

UN-REDD Programme/ FAO, UNDP, UNEP (2015a), UN-REDD Programme Strategic Framework 2016-20 (Revised Draft-7 May
2015), UN-REDD Programme Fouthteenth Board Meeting, Washington, D.C.
http://www.unredd.net/index.php?view=document&alias=14096-un-redd-pb14-2015-strategic-
framework&category_slug=session-3-strategic-and-policy-issues&layout=default&option=com_docman&Itemid=134 (accessed
12 October 2015).

UN-REDD Programme/ FAO, UNDP, UNEP (2014b), Consolidated Management Response to the Programme Evaluation, UN-REDD
Programme.http://www.unredd.net/index.php?view=list&slug=un-redd-programme-evaluation-
3266&option=com_docman&Itemid=134 (accessed 12 October 2015).

UN-REDD Programme/ FAO, UNDP, UNEP (2014c), External Evaluation of the United Nations Collaborative Programme on
Reducing Emissions from Deforestation and Forest Degradation in Developing Countries (the UN-REDD Programme), Volume I –
Final Report, UN-REDD Programme.

USAID (2014), Measuring Impact: Lessons Learned from the Forest, Climate and Communities Alliance, United States Agency for
International Devleopment, Washington, D.C.

World Bank Group (2004), Sustaining Forests: A Development Strategy, The World Bank Group, Washington, D.C. http://www-
wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2004/07/28/000009486_20040728090355/Rendered/PDF/29
7040v.1.pdf

CITATIONS AND RESEARCH NOTES

1
(IPCC, 2007)
2
(IPCC, 2007)
3
(Goodman and Herold, 2014)
4
This Insights draws mainly from recent evaluations of programmes by major actors in the sector.
5
[(Baastel and ECO Consult (on behalf of FCPF 2013), FAO 2015b, GEF 2012, ICF 2014, IEG 2013b, Miteva, D. A., et al. 2012,
Norad 2014, UN REDD programme 2014c]
6
[(Baastel and Nordic Agency for Development and Ecology (commissioned by FCPF) 2011, GEF 2015, ICF 2014, IEG 2015b, IEG
2013b]
7
(FAO 2015b, GEF 2006, Gelo et al. 2013, Hardcastle 2010, IEG 2013a, IEG, 2013b, Samii 2014)
8
[(Baastel and Nordic Agency for Development and Ecology (commissioned by FCPF) 2011, GEF 2006, ICF 2014, IEG 2013b, IEG
2011, Norad 2014, Norad 2012]
9
(GEF 2015, Hardcastle 2011, ICF 2014, IEG 2013b, IEG 2011, Norad 2014, Norad 2012, UN REDD Programe 2014c)
10
(UN-REDD Programme/ FAO, UNDP, UNEP, 2014b)
11
(UN-REDD Programme/ FAO, UNDP, UNEP, 2015a)
12
(IEG, 2013b)
13
[(Baastel and ECO Consult (on behalf of FCPF),2013]
14
(CIF 2014, IEG 2013b, IEG 2011, Lele 2011, Norad 2014, Norad 2012, UN REDD Programme 2014c)
15
[(Baastel and Nordic Agency for Development and Ecology (commissioned by FCPF) 2011, GEF 2015, Hardcastle 2010, ICF
2014, IEG 2013b, IEG 2011, Norad 2014, Norad 2012, UN REDD programme 2014c]
16
(IEG 2012, UN-REDD Programme/ FAO, UNDP, UNEP, 2014b)

12
17
(IEG, 2011)
18
(Norad 2014, UN-REDD Programme/ FAO, UNDP, UNEP, 2014b)
19
(Office of the Special Evaluator for Development Cooperation/SEO, 2014)
20
(SDC, 2014)
21
See for instance: (Spracklen, D. V., and L. Garcia-Carreras, 2015)
22
(Norad 2014, Norad 2012, IEG 2011)
23
(IEG, 2011)
24
(Norad, 2014)
25
(Norad 2014, Norad 2012, IEG 2011)
26
See for example (Norad, 2014)
27
(UN-REDD Programme/ FAO, UNDP, UNEP, 2014b)
28
(UN-REDD Programme/ FAO, UNDP, UNEP, 2014b)
29
(Norad, 2014)
30
(Samii et al., 2014)
31
(Samii et al., 2014)
32
(Samii et al., 2014)
33
(IEG, 2013b)
34
(IEG, 2013b)
35
(IEG, 2010)
36
(Kronenberg and Hubacek 2013)
37
(Pattanayak, 2010)
38
(Miteva, D. A., et al, 2012)
39
(IEG, 2010)
40
The spatial data on the location of forest fires and protected areas was used as a proxy indicator of deforestation, while
controlling for confounding influences.
41
(IEG, 2013b)
42
The 2013 evaluation included a portfolio review of protected area projects implemented by the World Bank. (IEG, 2013b)
43
For instance, the 2013 IEG evaluation found that the indicators being used to measure biodiversity were process indicators
related to the management of PAs (as a proxy for biodiversity outcomes) with a lack of data related to the protection of flora
and fauna and that more needed to be done to understand the impact of PAs at the species level. (IEG, 2013b)
44
(IEG, 2013b)
45
(GEF, 2015) Notably, this evaluation also analysed changes in submitted ‘Management Effectiveness Tracking Tool (MEET)’
scores over time, which “ are considered to be a proxy for a PA’s potential to deliver desired conservation outcomes”.
46
(FAO, 2015a)
47
(Rasolofosan et al., 2014)
48
(Rasolofosan et al., 2014)
49
(Scientific and Technical Advisory Panel (STAP) GEF, 2010)
50
(Scientific and Technical Advisory Panel (STAP) GEF, 2010)
51
(IEG, 2013b)
52
(IEG, 2013b)
53
One recommendation made in the GEF evaluation (2015) to allow for results to be monitored consistently within a country is
to establish long-term partnerships for biodiversity and socio-economic monitoring with country institutions that already
have this as their mandate.
54
(Miteva, D. A., et al, 2012)
55
(Norad, 2014)
56
(FCPF, 2011)

13
Number 11 · April 2016 Evaluation Insights: Forests and Sustainable Forest Management

Evaluation Insights are informal working papers issued by the Network on Development Evaluation of the OECD
Development Assistance Committee (DAC). These notes highlight emerging findings and policy messages from
evaluations and share insights into the policy and practice of development evaluation.

Further reading

Real-Time Evaluation of Norway’s International Climate and Forest Initiative.

Synthesising Report 2007-2013
Norway’s International Climate and Forest Initiative (NICFI) was launched in 2007
with the aim of reducing greenhouse gas emissions resulting from deforestation and
forest degradation in developing countries (REDD+). More than NOK 10 billion has
been disbursed since 2008 to support such efforts both internationally and at the
country level.

Managing Forest Resources for Sustainable Development: An Evaluation of World

Bank Group Experience

The World Bank has played a major role in shaping the dialogue on the role of
forests and climate change and the development of forest carbon instruments. IEG’s
review of the Bank managed Forest Carbon Partnership Facility found that it is
providing an effective and highly inclusive platform for defining the modalities of
REDD+. But it has also raised client expectations that will only be met if financial
flows follow. IEG suggested the need for a high level internal dialogue about how
the Bank will integrate countries’ REDD strategies into overall development
planning.

These and many other development evaluation reports are available from
DEReC: the Development Evaluation Resource Centre: www.oecd.org/derec

Further OECD publications

14