PROJECT

EVALUATION
 INTRODUCTION

• Evaluation has been defined as a “process which attempts to

determine as systematically and objectively as possible the
relevance, effectiveness and impact of activities in the light of
the objectives”
• The purpose of Evaluation is to transmit judgmental
information to decision-makers
• Evaluation is an action-oriented management tool
• Evaluation attempts to assess objectively:
i). the relevance and validity of objectives and design of the
project
ii). the efficiency and adequacy of the implementation
process of the project
iii). the identification of reasons for the satisfactory or
unsatisfactory performance of the results of the project.
Key Elements Of Evaluation Function

• The key elements in the evaluation function are that it:

i). takes long-range view in the light of project life
ii). questions their pertinence and validity in the context of
medium-term. Plan objectives priorities and strategies
iii). measures progress, financial, physical and policies in terms of
the objectives and explains reasons for the results
iv). Emphasizes achievement of objectives and the anticipated
impact on the beneficiaries
v). Assesses planned elements and looks for unplanned change,
searches for causes, challenges, the assumptions made; and
vi). Provides an in-depth assessment of performance and
advancement, and reports lessons learnt to the relevant agencies for
necessary information and guidance for future planning
 Evaluation Indicators

• The evaluation indicators are specific variables and factors, the

management of which would facilitate verification of changes or the
occurrence of results expected from a project. Indicators provide a
scale against which a change can be measured. An indicator may be
a direct measurement of a change or an indirect measurement
(proxy indicator). Indicators may be quantitative (percent growth in
the net income of a target group) or qualititative. Indicators should
possess the following qualities:
i). be specific, i.e., respond to particular needs for the analysis
ii). be valid, i.e., measure what is intended
iii). be verifiable, i.e., contain a reasonable degree of objectivity and
be empirically ascertained; and
iv). be assessable, i.e., possible to develop them with relative ease
and with readily available data.
 Types of Evaluation Indicators
• The types of evaluation indicators are:

i). Input Indicators: financial, physical,

managerial and manpower inputs, etc.
ii). Effect Indicators: effect in terms of
quantities, e.g., hectares planted with seedlings,
mortality rate and the timber yield
iii). Impact Indicators: impact in terms of
employment, increase in income, reduction in
imports depending on the development objectives
of the project
 Types of Evaluation

Evaluation is a distinct process which can be applied for

different purposes (in terms of feed-back) as well as to a specific
activity or project. Various types of project evaluations are as under:
a). Built-in/ Self Evaluation: Evaluation can be conducted
internally (by those directly involved in the execution of the project
or by individuals outside the implementing organization, i.e., the
staff of the line ministry, central planning authority or financing
institution). The former is often called built-in or self-evaluation.
The clients of built-in or self-evaluation are the managers (all
levels) who are directly responsible for project operations. This
involves continuous analysis of information/ data obtained through
exploring ways and means of increasing the efficiency and
effectiveness of the project. Built-in evaluation is a mechanism by
which the implementing agency can take preventive or corrective
actions in maintaining the project.
 Types of Evaluation (Contd.)

To be most effective, built-in evaluation must possess following key features:

1. It must be a regular activity of project implementation
2. There should be an integrated design for built-in evaluation which is clear,
simple and easy implement and which is output-oriented
3. It needs to be a participatory and not a command process
4. Support from management is essential, both in the application of a built-
in evaluation as well as in the use of its results
5. Quality control needs to be vigorous and that will necessitate the training
of staff in the design and application of the evaluation system; and
6. The involvement of the central evaluation agency in the design of the built-
in evaluation needs to be encouraged.

b. On-going evaluation: This is an essential part of the management

information system and takes place at intervals during the implementation
of the project. This type of evaluation draws mainly on the data and
information gathered through monitoring (contd. to next page).
 Types of Evaluation (Contd.)

The primary focus of on-going evaluation is usually on the process of

implementation and is generally concerned with the following issues:

i). to ascertain the continuing validity of project objectives and design

ii). to review the efficiency of the implementation process, timeliness of
activities, adequacy of inputs, including institutional arrangements, and
their utilization
iii). adequacy, nature and timeliness of output
iv). The likelihood of project success in achieving the expected immediate
objectives (results)
v). the extent to which the immediate objectives are being realized
vi). any corrective measures needed to improve project efficiency and
effectiveness.
The findings and recommendations of an on-going evaluation are meant to
assist decision-makers in resolving immediate problems by taking necessary
actions to improve the implementation process.
 Types of Evaluation (Contd.)

c. Terminal evaluation: Terminal evaluation is the analysis of the

project at or about the end of its life in order to determine its relevance,
effectiveness and likely impact. It will normally be undertaken by an external
evaluation team. Terminal evaluations often have to consider not only the
results of of the project but also to provide the basis for decision about future
actions. The findings and recommendations of terminal evaluation are
frequently used when the prolongation of the project is considered, including
any alterations. They can also be useful in deriving lessons for similar future
projects.

d. Ex-post evaluation: Ex-post evaluation takes place after project

completion and is often more in-depth than on-going or terminal evaluation.
The focus of this type of evaluation is on the analysis of impact. A thorough
analysis of outputs produced and the effects generated is required because
without in-depth analysis of these two elements it is not feasible to trace the
actual or potential impact of the project. There are frequently so many
factors external to a project that only an informed empirical judgment of its
impact can be made (contd. on next page).
 Types of Evaluation (Contd.)

Ex-post evaluation provides broad lessons for future in terms of policies,

design, implementation strategy and the role played by complementary
factors (assumptions) in generating effects and impact:
- relevance of objectives in relation to sector needs and priorities;
- validity of project design in relation to objectives;
- management and institutional aspects of project implementation and
monitoring; the soundness of the approach followed during implementation;
- strong points which need to be observed in the design of future projects
and pitfalls which need to be avoided.
Ex-post evaluation should make use of the data produced through
monitoring as well as the findings of on-going and terminal evaluation.
However, the need may arise for the creation of new data and information
especially regarding impact on beneficiaries. Ex-post evaluation is time-
consuming costly and calls for persons with special skills. Hence, it can only
be applied selectively to projects where there are particular lessons to be
learnt for wider application.
 Types of Evaluation (Contd.)

e. Program Evaluation: There is no clear dividing line between large

projects and programs designed to achieve specific objectives. The conceptual
framework and analytical tools of project and program evaluation are the
same, the distinction between the two lies in the scope and focus of analysis.
While the thrust of project evaluation is usually on the outputs, effects and
possible impact of the project itself. Program evaluation focuses mainly on
broader policy and organizational issues using the aggregation of detailed
findings. It puts stress on the relevance and feasibility of priority objectives,
the validity of the strategy for implementation and its cost-effectiveness,
including the examination of alternative ways by which the same objectives
can be achieved.
In particular, program evaluation concentrates on four main aspects:
• Program analysis
• Evaluation of efficiency and effectiveness
• Organizational analysis
• Synthesis of program achievements and feed-back of findings
 Feed-back from Evaluation

Evaluation is undertaken not only to assess the performance and

achievements of projects in terms of their relevancy, efficiency and
effectiveness but also to enable managers, planners and policy-makers to
learn lessons for the improvement of both existing and new projects. This is
known as feed-back and evaluation without feedback apart from being a
discouragement to evaluators is a waste of efforts and resources. To be
effective, “feedback” must satisfy several levels in the decision-making
hierarchy.
• Feed-back for whom? The feed-back from evaluation is strong when
evaluation is in response to active demands. The most important source of
demand is management which not only influences the nature, scope,
direction and timeliness of evaluation but also looks forwards to valuable
lessons to be gained. Feed-back from evaluation is a basic requirement of
management and evaluation without any direction and support from
management can hardly be effective. It has to be treated as an integral part
of the normal management decision-making process.
 Feed-back from Evaluation (contd.)

• Purpose of Feed-back: Feed-back from evaluation is used for different

purposes, e.g., for operational, analytical and policy purposes. These are:
i). For operational purposes:
- by verifying the extent to which a project is achieving or has achieved
the intended outputs and effects
- by exploring and determining the reasons, and identifying the critical
factors in the achievement of outputs and effects
- by providing justification for the continuation, modification or
suspension of the project works and
- by providing objective and analytical information for accountability
report
ii). For use as an important analytical tool:
- to improve project design
- to improve and sharpen project objectives
- to verify project assumptions and render them more explicit and
- to review the entire process/ framework of means and ends to achieve
the desired objectives
 Feed-back from Evaluation (contd.)

iii). For Policy Purposes:

- by ascertaining the validity of a given development strategy, approach,
assumption or hypothesis
- by exploring and/or gaining more knowledge about the inter
relationships of several actions or policy decisions which have a bearing on
the efficiency and effectiveness of the project and
- by reviewing and ascertaining the effectiveness of a group of projects,
approaches, strategies on a comparative basis.
Essentials For Timely and Effective Use Of Feed-Back
To ensure that feed-back is timely and effectively used, three conditions are
essential:
• There must be an adequate institutional mechanism for directing and
channeling the findings and recommendations to appropriate decision-
makers who can mobilize follow-up action;
• Managers and policy-makers must understand the purpose, findings and
limitations of evaluation and they should know how to guide and use
evaluation for their needs; and
• Evaluation staff must establish themselves by performing useful and credible
work and should maintain good rapport with managers as well as project
staff.
 Relevance, efficiency and effectiveness

The evaluation of a project/ program is incomplete unless it addresses

the three basic issues of relevance, efficiency and effectiveness. Although
each of them covers a different aspect of the analysis, in many ways, they
complement one another.
Relevance: When assessing the relevance of a project the intention is to
determine whether the particular solution proposed in in the project:
- is pertinent to well identified development needs;
- has a rationale and development objectives are in harmony with the
priorities and development strategies of the sector/ sub-sector or region/
nation;
- is clearly and adequately formulated in terms of objectives and design;
- can be implemented within the time and resources permitted; and
- is likely to produce outputs of immediate use to solve the identified
problems.
Thus, the question of relevance entails an analysis of the objectives and
design of a project in terms of broader issues of development policy, sector
priorities and strategies as well as other problems of a wider nature, e.g.,
institutional capability for implementation, receptivity of beneficiaries to the
project and prospects for follow-up action.
 Relevance, efficiency and effectiveness (contd.)

• Efficiency: Efficiency is primarily concerned with the optimal use of

resources under the control of project/ program management in producing
the targeted outputs. The focus is mainly on managerial performance and
productivity in the use of resources. Two levels of analysis should be
attempted – operational and financial. For operational efficiency, it is
necessary to analyze the merits and demerits of the work plan used during
implementation. This is done from the standpoint of adequacy and timeliness
of input delivery, work schedules and the sequence of activities, operational
methods and procedures applied, the generation of outputs in relation to
established targets and their quality control. In a financial sense, efficiency
has to be judged on some notion of cost-effectiveness (i.e., cost per unit of
outcome achieved). The issue to be tackled is whether the targeted output
could have been obtained from lesser inputs (output maximization). The
concept of efficiency can also be applied to appraisal of effects and impact,
but such an analysis would be not only complicated but also of questionable
validity as effects and impact are seldom entirely the results of a project.
 Relevance, efficiency and effectiveness (contd.)

Effectiveness: The assessment of effectiveness is a major part of an

evaluation exercise, it entails an in-depth analysis of the intended results of
a project from a variety of angles and is often a complicated task to
accomplish. The analysis of effectiveness should draw conclusions about the
following aspects of the project:
• the relevance, timeliness and adequacy of the outputs in terms of the needs
of the beneficiaries;
• use of outputs by the beneficiaries in producing effects;
• the nature and extent of the effects generated (quantified through effect
indicators);
• benefits derived from the effects generated; and
• The nature and extent of the ultimate benefits realized (quantified through
impact indicators).
 Difference Between Monitoring and Evaluation

Evaluation is a learning management tool but differs materially from

monitoring. Project monitoring is undertaken at the implementation stage
while evaluation is generally preferred when a project is complete. The
monitoring reports provide the database for the evaluation but evaluation
can not contribute directly to monitoring. The evaluation studies are more
comprehensive in nature, covering all aspects of the projects, whereas
monitoring provides information mainly to assess and help maintain or
accelerate the progress of implementation. Key differences between M&E are:
Monitoring (M) Evaluation (E)
- keeps track of daily activities a - takes long-range view through in-depth
continuous function study a one-time function
- accepts objectives, targets and - questions pertinence and validity of
norms stipulated in the project document project objectives/ targets
- Checks progress towards output - measures performance in terms of objectives
- Reports on current progress at short - provides an in-depth assessment of
Intervals for immediate corrective actions performance for future feedback
- Stresses conversion of inputs to - emphasizes achievement of overall objectives
outputs