Monitoring and Evaluation

in the Road Sector

Seminar-workshop on the updating of the Local Road Network
Development Plan (LRNDP) – Batch 3

Brenda B. Furagganan
1 June 2017
Why do you want to build roads?
 Development Hypothesis

Build Roads LRNDP Goals

…to promote general safety and well being of the

people through better and equal access to economic
opportunities and improved delivery of services
(Surigao del Norte)

….to increase economic activities and improve public

access to infrastructure and services (Bohol)

….to uplift the populations’ general well-being

(Bukidnon)

Monitoring & Evaluation

4
Session objectives

At the end of the session, the participants

are expected to:
OUTPUT
1. Recognize the importance of M&E in
the successful execution of the u Increased knowledge on RBME in the
LRNDP; and context of road development

2. Formulate a draft Results-Based

Monitoring and Evaluation Plan for the u LRNDP Results chain
LRNDP. u LRNDP Performance indicators
u RBME TEMPLATE
Results-Based Management (RBM)
…in a nutshell
Traditional public sector management

u Preoccupied with rules and

procedures, with emphasis on
delivering services in compliance
with specific guidelines and
business processes (emphasis on
doing things right)
u Provided limited attention to
clients’ needs and customizing
services for specific circumstances
u More supply than demand driven
Results-Based Management (RBM)

a management
strategy focusing on
performance and
achievement of
outputs, outcomes and
impacts (OECD, 2002)
Results-Based Management (RBM)

Emphasis is not only on whether

government departments have
undertaken their mandated activities,
accountable spending of the budgets
associated with those activities, and
whether all applicable laws have
been complied with, but on the
results achieved by those activities
in relation to government objectives
like safety, employment and
development (SO WHAT?)
Results-Based Management (RBM)

u Focuses on external results

(outcomes) rather than on internal
processes to drive change
u Integrates strategy resources,
performance measurement, learning
and adjusting to improve decision
making
u Results information is used for
learning and decision making as well
as reporting and accountability
Results-Based Management
(RBM)
 Immediate Intermediate Final
Input
Activities Outputs Outcomes Outcomes Outcome
(Resources)
(Short-term) (Medium-term) (Impact)

Traditional Public Sector Results-Based

Management Management
Results-Based Monitoring & Evaluation
(RBME) OF RURAL ROAD DEVELOPMENT
PROJECTS
an overview
Monitoring

A continuing function that uses

systematic collection of data on
specified indicators to provide
management and the main
stakeholders of an ongoing
development intervention with
indications of the extent of
progress and achievement of
objectives and progress in the use
of allocated funds
Monitoring
Evaluation

u The systematic and objective

assessment of an on-going or
completed project, programme or
policy, its design, implementation and
results. The aim is to determine the
relevance and fulfillment of objectives,
development efficiency, effectiveness,
impact and sustainability (OECD, 2002)
u The process of determining the worth
or significance of an activity, policy or
program
evaluation
monitoring
2 leaves
and one track
flower, so
far progress
Theory of Change

u a description of how an intervention is supposed to deliver

the desired results
u it describes the causal logic of how and why a particular
project, program, or policy will reach its intended
outcomes
u it depicts a sequence of events leading to outcomes; it
explores the conditions and assumptions needed for the
change to take place, make explicit the causal logic behind
the program, and map the program interventions along
logical causal pathways
 Theory of Change
(Development Hypothesis)

Build Roads LRNDP Goals

…to promote general safety and well being of the

people through better and equal access to economic
opportunities and improved delivery of services
(Surigao del Norte)

….to increase economic activities and improve public

access to infrastructure and services (Bohol)

….to uplift the populations’ general well-being

(Bukidnon)

19
Development hypothesis

A predictive statement about a causal

relationship (Cause-Effect) between two
elements involving uncertainty.
IF THEN

… activity is … output will be

conducted produced …

… problem will be
reduced…
Sample development Hypothesis

IF THEN

21
 Cause and Effect Relationship
IF THEN

… if roads in … then poverty among

Municipality of X farmers in Municipality
is paved …… of X will be reduced

… then farmers in … then the farmers in

Municipality X will have Municipality X will be
better access to markets able to sell more
for their agricultural products at better
products prices…
 Immediate Intermediate Final
Input
Activities Outputs Outcomes Outcomes Outcome
(Resources)
(Short-term) (Medium-term) (Impact)
 Results-Based Monitoring & Evaluation
(RBME) cycle

Collecting & Using analysis to

Identifying Analyzing
Defining Formulating managing inform
Performance performance
Results Chain M&E Plan performance management
Indicators data
data decisions
Results Chain (Program logic)
Step 1
Results are the output, effect Impact

outcome or impact(intended

Results chain
or unintended, positive cause Intermediate
and/or negative) of a effect Outcome

development intervention
(OECD, 2002) Immediate
cause Outcome
effect
Results are measurable or
describable changes cause Output

arising from a cause and

effect relationship
Activities

Input
 Goal Changes at the societal or sectoral level affecting people;
(Impact) Program/Project outcomes contribute to the attainment of impact

Intermediate Changes at the sector or community level; Medium-term results

Outcome for specific beneficiaries that are attributed to achieving specific
outputs.

Immediate Changes at the individual or organizational level; Short-term

results for specific beneficiaries that are attributed to achieving
Outcome
specific outputs.

Products, capital goods and services which result from a

Outputs development intervention; a collection of outputs lead to a desired
outcome

The processes or actions that are conducted to convert a wide

Activities range of inputs to produce the desired outputs; describes ‘what
are things to be done’

All the resources that contribute to production and delivery of outputs;

Input or what are used to do the work or tasks such as money, personnel,
equipment and buildings
Inputs

u All the resources that

contribute to production and
delivery of outputs.
u What are used to do the work
or tasks
u include finances, personnel,
equipment and buildings
Activities

u The processes or actions that

conducted to convert a wide
range of inputs to produce
the desired outputs
u describes ‘what are done’.
Outputs

u Products, capital goods and

services which result from a
development intervention;
u Include changes resulting from
the intervention which are
relevant to the achievement of
outcomes
u A collection of outputs lead to a
desired outcome
Outcome

u Changes at the individual or

community or organizational
levels
u Short- or medium-term results
for specific beneficiaries that
are attributed to achieving
specific outputs
Impact

These are changes at the

societal or sectoral level
affecting people;
These are attributed to the
attainment of a series of
outcomes
 3
Impact

1. Decreased travel time

2. Public sector trained to carry out road works (A1) Intermediate
3. Improved quality and prolonged life of road network Outcome
4. Reduced poverty through economic growth
5. Axle Load Control Law implemented
6. Road maintenance management systems created with assets inventoried
7. Road maintenance funded Immediate
Outcome
8. Decreased vehicle operating costs
9. Increased income
10. Pilot road maintenance centers constructed (A2)
11. Systematic and predictable asset management system
12. Roads maintained according to plan Output
13. Matching funds for maintenance provided
 Axle Load
Matching Control Law
implemente

Reduced poverty through economic growth

funds for
maintenance d
Decreased
provided
vehicle
Road operating
maintenance Road costs
mgnt systems maintenance
created with funded (A4)
assets
inventoried Improved
quality and Increased
Public sector prolonged
Systematic &
trained to
predictable life of road income
carry out network
asset mgnt
road works
system
(A1)

Pilot road
maintenance Roads
centers maintained Decreased
constructed according to travel time
(A2) plan

Source: Monitoring and Evaluation Plan of the Liberia Compact between the United States of
America, acting through the Millennium Challenge Corporation and the Republic of Liberia , July
Performance indicators
Step 2
Indicators

u signposts of change along the

path to a destination
u describes the way to track
intended results
u a way of measuring progress
toward targets
u Informs whether the changes
occurring because of
program/project interventions
are getting better or worse
Quantitative Indicators

u Quantitative indicators are statistical

measures that measure results in terms of:
u Number
u Percentage
u Rate (example: birth rate—births per
1,000 population)
u Ratio (example: sex ratio—number of
males per number of females)
u Source of information: reports, inventory,
formal surveys or questionnaires
Qualitative Indicators

u Reflect people’s judgements, u Qualitative indicators measure results

opinions, perceptions and in terms of:
attitudes towards a given u Compliance with...
situation or subject.
u Quality of...
u Can include changes in
sensitivity, satisfaction, u Extent of...
influence, awareness, u Level of...
understanding, attitudes, u Degree of…
quality, perceptions, dialogue
or sense of well-being. u Source of information: testimonials,
focus group discussions, attitude
surveys, and participant observations
Levels of Indicators

Impact

Intermediate
Outcome

Immediate
Outcome

Outputs
Goal / Impact

IMPACT 1.1. By 2020, poverty index decrease

u Economic poverty among from ___ to ____
people of Province X reduced 1.2 Household income of residents of
Province X increased
Outcome Indicators

Sector Outcome
u Reliability of road system u % reduction in travel time
enhanced u % reduction in road-related accidents

u Access of farmers to markets u % reduction in transportation/freight

increased costs
u % decrease in cost of agricultural
products
Outcome Indicator

Organizational Outcome
u Effectiveness of local u % reduction in unserved
government service providers communities
to deliver basic services to
hard-to-reach areas increased
Output Indicators

u Core provincial roads paved u Length of paved core provincial

roads increased from 3,000kms
in 2016 to 4,000kms in 2018
u Length of unpaved core
provincial roads reduced from
8,000kms in 2016 to 5,000kms
in 2018
 Indicators Baselines Targets
2011 2017

Length of paved road (kms)

• Concrete 39.326 49.326
• Asphalt 8.714 18.714
Length of unpaved roads 149.518 129.518
(kms)
 Writing good indicators
Step 5. Indicate
timeframe
By 2020,
Step 1. State what is
being measured length of paved core provincial roads
in the entire country
Step 2. Indicate the
change
increased from 2,198kms (2016) to 10,000kms,
with at least a rating of Fair
Step 3. Indicate for
whom the change will
occur

Step 4. Indicate
quality aspects
 4

1. Costs of transport reduced significantly

2. Availability of transport and travel service (at a given price)
increased
3. Markets that depend on transportation expanded
4. Higher farm gate prices
5. Improved security along the roads
6. Access to inputs for rural production easier
7. Improved safety regulations
Results-based Monitoring and
Evaluation (RBME) Plan
What is a Monitoring & evaluation Plan?

u A plan to manage the process of assessing and reporting progress towards

achieving output and outcomes and to identify potential evaluation questions
u A roadmap to the successful implementation of monitoring and evaluation
activities
u It identifies:
Ø How evaluation questions directly link to programmatic goals
Ø Variables you need to measure in order to provide answers to your monitoring
and evaluation questions
Ø Data collection tools for measuring variables
Ø Processes for data collection and data management
Ø How resulting data and information will be used
Ø Staffing and other resources needed to implement M&E activities
Monitoring and Evaluation Plan

u Should clarify:
o What is to be monitored and evaluated
o The activities needed to monitor and evaluate
o Who is responsible for monitoring and evaluation activities
o When monitoring and evaluation activities are planned (timing)
o How monitoring and evaluation are carried out (methods)
o What resources are required and where they are committed
u Relevant risks and assumptions in carrying out planned monitoring and
evaluation activities should be seriously considered, anticipated and
included in the M&E
RBME Organization
Thank you for
your active
participation