COUNTRY PORTFOLIO EVALUATION

DEFINITION
A country portfolio evaluation is an assessment of the portfolio of development
interventions supported by UNDP and the funds in association with UNDP in a
particular country in any given five-year period. The portfolio represents a broad
spectrum of initiatives that include programs, projects, advocacy and other
development-related activities. The five-year period may or may not correspond to the
programming cycle.

MAIN PURPOSE
This type of evaluation aims to draw lessons from experience that can be the basis for
instituting improvements in the development initiatives supported by UNDP and the
funds in association with it. Such lessons include those that pertain to the formulation,
implementation and management of development programs and projects and the
conduct of other development-oriented activities such as advocacy and networking
among development partners.

STAKEHOLDERS
The main stakeholders of the evaluation are the Government, UNDP and the funds in
association with UNDP. They are interested in knowing the results of their
cooperation as a basis for instituting necessary improvements. The cross-fertilization
of lessons drawn from country portfolio evaluations in various programme countries
can improve the way UNDP and the funds conduct their business - both programming
and non-programming work.

Other stakeholders include the private sector, CSOs and donors. The private sector
and CSOs are directly involved in, and/or affected by, interventions supported by
UNDP and the funds in association with it. The donors want to ensure that resources
are used as efficiently as possible to reach target groups.

SCOPE
The evaluation sample should include programmes, projects and activities that have
been carried out over a period long enough to offer lessons from experience. It will
vary depending on the timing of the evaluation. For example, if the country portfolio
evaluation covers a five-year period that corresponds to the CCF period, then the
evaluation sample should include those programmes, projects and activities that: (a)
were carried over from an earlier period and completed during the CCF period, and
(b) were started and completed (or almost completed) during the CCF period.

PLANNING AND MANAGEMENT

A country portfolio evaluation may be under-taken on the initiative of a UNDP
country office, a regional bureau, OESP and/or funds in association with UNDP. The
Programme Management Oversight Committee (PMOC) may also call for the conduct
of the evaluation. Since the evaluation also covers the interventions and activities of
funds in association with UNDP, it may be conducted as a joint evaluation by UNDP
and the funds concerned.

Regardless of who initiated the evaluation, the UNDP country office must be involved
in its planning and organization and must consult the Government during the process.

The evaluation itself must be conducted by a team of independent consultants.

A country portfolio evaluation may be undertaken in any year since it is not confined
or linked to a specific programming period.

The time required for the evaluation will vary depending on the size and complexity
of the country portfolio being evaluated

MONITORING AND EVALUATION AND THE UNDP

AREAS OF THEMATIC FOCUS
Monitoring and evaluation must provide assessments of three dimensions of all
programmes and projects: relevance, performance and success.

It is also useful to identify the key elements that are specific to programmes and
projects with a particular thematic focus in order to monitor and evaluate those
programmes and projects effectively. Some of these key elements are described
below. Their assessment must be undertaken within the overall framework for
monitoring and evaluation described in the preceding chapters, e.g., use of specific
criteria of relevance, performance and success and use of appropriate indicators.
Moreover, since the areas of thematic focus are themselves interrelated, the key
elements should be assessed in an interrelated, holistic manner.

POVERTY ERADICATION AND SUSTAINABLE LIVELIHOODS

Poverty eradication and the promotion of sustainable livelihoods constitute the top
priority of UNDP. UNDP follows a two-pronged strategy to achieve the sustainable
eradication of poverty and the shift from poverty to equity:8

 creation of an enabling environment, which involves attacking the root causes

of poverty and removing obstacles that limit people's choices and opportunities;
and
  empowerment of people, emphasizing people's participation in decision-
making, mobilization of social energy for SHD and capacity development.

Although poverty eradication interventions span a wide range of different types

of programmes and projects, monitoring and evaluation of those interventions
should always provide assessments of the following elements:

Enabling Environment

 Formulation or modification of policies, plans or legal frameworks to bend the

incentive structures in favour of the poor;
 Sensitization of key institutions to issues of poverty eradication (e.g., through
advocacy);
 Removal of obstacles hindering access by the poor to assets and public
services.

Empowerment

 Opportunities provided to the target groups to participate in programme or

project formulation, implementation, monitoring and evaluation;
 Extent to which programmes and projects enable the target groups to overcome
constraints by themselves and on their own terms;
 Programme or project contribution to intra-household, intra-community
networking and self-reliance;
 Programme or project contribution to making local government institutions
accountable to the poor in their communities.

GENDER CONSIDERATIONS IN DEVELOPMENT

Gender-mainstreaming is a strategy for promoting gender equality. It is defined as
taking account of the social, cultural, economic and political inequality between
women and men in all policy, programming, administrative, human resource and
financial activities and all organizational procedures for the empowerment of women.
In practice, this means:

 identifying gender issues with regard to a given situation through gender

situation analysis;
 advocating the incorporation of this information in all aspects of decision-
making and implementation through the involvement of local gender experts
and consultation with women as well as men at every stage; and
 ensuring monitoring and gender impact assessments using appropriate
indicators.9
The Panel on Monitoring and Evaluation of the ACC Sub-Committee on Rural
Development has initiated the development of core evaluation guidelines on gender
issues. Such guidelines, which may be used for both monitoring and evaluation,
emphasize the following:

 assessment of the extent to which gender differences have been considered in

the programme or project design and imple-mentation stages, particularly in
targeting beneficiaries and determining the resources, delivery systems and
participatory mechanisms required;
 collection of gender-differentiated data regarding the delivery of all programme
or project services and resources;
 use of gender-differentiated impact indicators to measure, in quantitative and
qualitative terms, the extent to which:
o poor women and men have been reached by and benefited from the
programmes or projects, especially in terms of empowering them to
overcome poverty,
o local capacities have been strengthened to address gender issues, and
o necessary safeguards and institutional linkages have been put in place to
ensure the sustainability of benefits to women and men.

SUSTAINABLE ENVIRONMENT AND SOUND MANAGEMENT OF

NATURAL RESOURCES
Some of the key elements that should be assessed when monitoring and evaluating
environmental programmes and projects are listed below.

Environmental Context of the Programme or Project

 Natural environment
 Main environmental issues
 Economics and the environmental context
 Environmental management, including gender-differentiated roles

Environmental Management Strategies

 Environmental objectives and targets

 Expected environmental and socio-economic effects
 Conflicts of interest or consensus among stakeholders
 Stakeholder participation
 Implementation responsibilities
 Financing
Integration of an Environmental Programme or Project into the Overall Development
Programme of the Area (i.e., local, national, regional or global)

SOUND GOVERNANCE
The following key elements of the framework for UNDP partnerships in good
governance must be examined:

Characteristics of Sound Governance

 Participation - all women and men have a voice in decision-making, directly or

through institutions that represent them;
 Rule of law - legal frameworks are fair and impartially enforced, particularly
the laws on human rights;
 Transparency - processes, institutions and information are directly accessible to
those concerned with them;
 Responsiveness - institutions and processes try to serve all stakeholders;
 Consensus orientation - good governance mediates differing interests to reach a
broad consensus among the parties concerned;
 Equity - all women and men have opportunities to improve or maintain their
well-being;
 Effectiveness and efficiency - institutions and processes produce results that
meet needs while making the best use of resources;
 Accountability - decision-makers in the government, the private sector and
CSOs are accountable to the public and to institutional stakeholders;
 Strategic vision - leaders and the public have a long-term perspective on human
development.

Economic, Political and Administrative Governance

 Structures and processes that guide political and socio-economic relationships.

Constructive Interaction among the State, the Private Sector and CSOs

 State - political and public-sector institutions;

 Private sector - private enterprises and the informal sector in the marketplace;
 CSOs - associations around which society voluntarily organizes, e.g., NGOs,
trade unions, academic institutions.

INDICATORS
Indicators of performance for corporate planning within UNDP are outside the scope
of the present document.

Indicators constitute a critical component of a results-oriented monitoring and

evaluation framework. Generally speaking, they are signs that show changes in certain
conditions or results from specific interventions. They provide evidence of the
progress of programme or project activities in the attainment of development
objectives.

Within the context of UNDP programmes and projects, indicators are pre-established
signs that the people carrying out monitoring and evaluation actions look for in
determining the extent to which a programme or project remains relevant, is
performing well and is achieving its objectives. In this context, they may be classified
as relevance, performance and success indicators.13

SELECTION OF INDICATORS
Indicators for use in monitoring and evaluation should be selected during the
formulation stage of a programme or project when the objectives are being
established. The following questions should be answered as part of the process of
establishing the indicators.

 What are the objectives of the programme or project?

 Who are the target groups and what are their needs and expectations?
 What changes are anticipated as a result of the programme or project?
 To what extent and how efficiently is the programme or project achieving its
objectives?
 What are the criteria for judging the success of the programme or project?

Guiding Principles for the Selection of Indicators

Management Responsibility and Involvement of Stakeholders
At the programme or project formulation stage, the institutions that will be directly
responsible for the programme or project and other stakeholders should be involved in
selecting a preliminary list of indicators. During implementation, the indicators should
be revised in accordance with changes in the programme or project context and design
through consensus of the various stakeholders. The participatory process is intended
to promote ownership of, and responsibility for, the planned results of the programme
or project.

Quantitative, Qualitative and Proxy Indicators

Both quantitative and qualitative indicators should be selected based on the nature of
the particular aspect of the programme or project that is being assessed. Efficiency,
for instance, lends itself easily to quantitative indicators. On the other hand,
measuring dynamic sustainability, which involves people's adaptability to a changing
environment, necessitates some qualitative assessment of attitudes and behaviours.
However, there are some methodologies, such as beneficiary assessment, rapid rural
appraisal (RRA) and structured interviews, that can be used to convert qualitative
indicators into quantitative indicators.

In cases where cost, complexity and/or timeliness of data collection prevents a result
from being measured directly, proxy indicators may be used to reveal performance
trends and make managers aware of potential problems or areas of success. For
example, in an environmental protection programme where a target result is the
improve-ment in the health of certain lakes, the level of toxins in duck eggs may serve
as a proxy indicator of that improvement.14

Limiting the Number of Indicators

A good balance should be achieved between theory and practice, i.e., between what
should be and what can be measured. An ideal set of indicators includes indicators of
relevance, performance and success. However, for practical purposes, it is absolutely
necessary that a thorough selection process be undertaken, through negotiation among
the various stakeholders, to arrive at a realistic number of meaningful indicators. The
major considerations in selecting the indicators are:

 appropriateness of the indicator vis-à-vis the immediate and development

(long-term) objectives of the programme or project;
 ownership by beneficiaries and other stakeholders;
 cost-effectiveness of data collection.

CHARACTERISTICS OF GOOD INDICATORS

A popular code for remembering the characteristics of good indicators is

SMART.

S Specific
M Measurable
A Attainable
R Relevant
T Trackable

Collection and Analysis of Data for Indicators

Based on selected indicators, time-series data must be collected and analysed during
and after programme or project implementation to support monitoring and evaluation.
At the programme or project formulation stage, the following elements must be
defined:

 types and sources of data needed;

 methods and frequency of data collection;
 methods of data analysis;
 who will be responsible for data collection and analysis; and
 who will use the resulting information.

The time-series data must be compared with the baseline data (constructed at the
formulation stage) describing the problems to be addressed by the programme or
project. The data comparison will enable programme or project managers and other
key stakeholders to assess whether the programme or project is achieving its
objectives.

USES OF INDICATORS
In general, indicators must be incorporated into various stages of the programme or
project cycle. At the design or formulation stage, indicators (even those that are
tentative) must be established to help to clarify the logical framework of the
programme or project. The indicators selected should then be used during programme
or project implementation as part of the monitoring process to measure progress,
including the identification of potential problems or success. Finally, they should be
part of evaluations to assess results, including beneficiary satisfaction with results.

Specifically, indicators must be integrated into the rating and reporting system for
monitoring and evaluation purposes