Office of Social Sector Development:

Performance Management Plan Workshop

May 7, 2014
Muncy Center, U.S. Embassy
Kabul, Afghanistan
 Workshop Agenda
• Arrival: 8:00-9:00
• Welcome Message: 9:00-9:05
• Introductions: 9:05-9:10
• Workshop Objectives: 9:10-
9:15
• USAID/Mission Results
Framework, PMP, Strategies
& Reporting: 9:15-9:25
• Results Frameworks & PMP
Components: 9:25-10:15
• Refresher, Afghan Info
Database: 10:15-10:30
• BREAK: 10:30-10:45
• Refresher, DOC: 10:45-11:00
• Components of a PMP con’t:
11:00-11:20
• PPR Indicators for Health &
Education: 11:20-11:25
• Developing Performance
Indicators: 11:25-12:00
• PMP as a Management Tool:
12:00-12:10
• Re-cap: 12:20-12:30
• Closure & Departure: 12:30
2
 Workshop Objectives
 By the end of the workshop participants’ knowledge of
USG strategy and reporting processes will be
strengthened.

 By the end of the workshop participants’ use of

Performance Management Plans as management tools
will be increased.

 By the end of the workshop participants’ ability to

develop performance indicators will be increased.

3
Cross Cutting Issues:
• Gender Equality and
Female Empowerment
• Youth Participation
• Capacity Development
 RESULTS FRAMEWORKS &
Performance Management Plans (PMPs)

5
Development Hypothesis – describes the theory of
change, logic and causal relationships between the
building blocks needed to achieve a long-term result.
It is based on development theory, practice, literature,
and experience, is country-specific, and explains why
and how the proposed investments lead to achieving
the project Goal. Can be expressed in a short narrative
that explains each layer of results in the RF, upwards
from the sub-IRs to IRs to the Goal using If-Then
statements.
6
 Goal:
Highest Level Result effected by the Project.
If Objectives 1-3 are reached then the project can achieve the goal

Objective 1: Objective 2: Objective 3:

If Intermediate Result 1.1 is reached then If IRs 2.1 & 2.2 are reached then the If IRs 3.1 – 3.3 are reached then the project
the project can achieve Objective 1 project can achieve Objective 2 can achieve Objective 3

IR 1.1: IR 2.1: IR 3.1: IR 3.2: IR 3.3:

IR 2.2:
If all Sub-IRs are If all Sub-IRs are If all Sub-IRs are If all Sub-IRs are If all Sub-IRs are
reached then If all Sub-IRs are
reached then the reached then reached then reached then
the project can reached then
project can achieve the project can the project can the project can
achieve IR 2.1 the project can achieve IR 3.3
IR 1.1 achieve IR 3.1 achieve IR 3.2
achieve IR 2.2

Sub-IR 1.1.1 Sub-IR 1.1.2 Sub IR 2.1.1 Sub IR 2.2.1 Sub IR 3.1.1 Sub IR 3.2.1 Sub IR 3.3.1

Sub-IR 1.1.3 Sub IR 1.1.4 Sub IR 2.1.2 Sub IR 3.1.2 Sub IR 3.3.2
Results Statements

• Are found at each level of the RF –

– Goal (highest level)
– Objective (next-highest after Goal),
– Intermediate Result (IR),
– Sub-Intermediate Result (sub-IR)

A Results Statement reads like a

“newspaper headline”
8
Group Exercise – Newspaper Headlines

• With the people at your table write down two

headlines you would expect to read in the local paper
if the goal of your current project is achieved (using
past tense).
• Write the headlines on flipchart paper and hang on the
wall
• Nominate one person from the group to read the
headlines aloud at the end of the exercise

9
Results Frameworks – Things to Avoid
• Results statement combines more than one
activity/component
• Results statement explains how the result will be
achieved
• Results statement refers to actors or responsible parties
• Results statement is at the wrong level – i.e. its an
objective-level result but in the RF its listed as an IR
• The RF includes more results than could realistically be
achieved during the LOP
10
 Example: Education
Goal: Increased # of school aged children through education
services across Afghanistan by providing standard, quality-driven Project, Version 1
education services under the leadership of the MoE

Objective 1: Objective 3: To improve the

To increase access to schools Objective 4: To monitor school
Objective 2: To improve quality of capacity of MoE to deliver and
especially for girls, in marginalized classes in order to ensure the
education particularly teaching manage education activities and
areas high quality of classes
services

IR 1.1: Increase student

IR 2.1: IR 2.2: Provide
enrollment, as access to
Improved learning and
community schools is increased, IR 3.1: Improve the capacity
teaching material
for children in remote areas quality of of MoE through
and provide new
instruction by materials for management trainings
teachers reading
Sub-IR 1.1.1:
Access to catch-up
learning programs
Sub-IR 1.1.2:
for over-aged Sub IR 4.1: Quality
Increase number Sub-IR 2.??:
learners Sub IR 3.1.1: of education
of qualified Improve
(particular Improve capacity of delivered in
teachers, reading and
emphasis on girls), MoE through program through
particularly numeracy
by expanding finance trainings continued
women
catch programs skills monitoring levels
throughout
Afghanistan

Sub IR 4.2: MoE

Sub IR 3.1.2: strategies, policies
Sub-IR 1.1.3: Improved capacity and systems with
Greater Sub IR 1.1.4: of MoE through regard to education
community Mobilization of monitoring trainings are properly
involvement and community implemented
decision-making education
on local education advisory groups
issues
 Example:
Goal: Equitable access to quality education services in Education Project,
Afghanistan increased
Revised Version

Objective 1: Equitable access to schools for

children, especially girls, in marginalized areas
Objective 2: Quality of community Objective 3: Education service delivery improved
increased
education is improved

IR 1.1: Student enrollment increased

IR 2.2: Monitoring IR 3.1: MoE capacity to deliver

IR 2.1: Quality
data used for education services improved
of instruction
decision-making
Sub-IR 1.1.2: Catch-up improved within the MoE
Sub-IR 1.1.1: Student learning programs for
enrollment in remote learners whose age
areas increased exceeds their education
level expanded Sub IR 3.1.1:
Sub IR 2.1.1: Sub IR 2.2.1: Sub IR 3.1.4:
Operational
Performance Coordination
Educators’ management
monitoring capacity of MoE
teaching skills capacity of MoE
capacity of the strengthened
Sub-IR 1.1.3: Sub IR 1.1.4: improved strengthened
MoE strengthened
Employment of qualified Community
teachers, particularly involvement in
female teachers is education decision-
increased Sub IR 3.1.2:
making increased Sub IR 2.1.2: Performance
Availability of monitoring capacity
teaching and of the MoE
learning strengthened
materials
increased

Sub IR 3.1.3:
Financial
management
capacity of the MoE
strengthened
 Where do Results Statements Fit in the PMP???
Indicator # Performance Indicator Data Sources Methods of
Indicator Definition and Data Collection
Unit of
Measure
Goal: Equitable access to schools for children, especially girls, in marginalized areas increased
Indicator #1
Objective 1: Equitable access to schools for children, especially girls, in marginalized areas increased
Indicator #2
Indicator #3
Intermediate Result 1.1 (IR): Student enrollment increased
Indicator #4
Sub-Intermediate Result 1.1.1 (Sub-IR): Student enrollment in remote areas increased
Indicator #5

13
Results Frameworks – Key Takeaways
• Every project should have one
• Remember logical progression – IF-THEN
linkages
• Results Statements should read like a
newspaper headline and describe the end result
achieved
• The RF should be drafted before work begins on
the PMP
14
Components of a Performance
Management Plan

15
Performance Management Plan (PMP) – is a tool
to plan and manage the process of monitoring,
evaluating and analyzing progress towards
achieving results identified in the RF in order to
inform decision-making, resource allocation,
learning and adapting project activities.

16
Performance Management Plan Components
• Results Framework
• Performance Indicator Tracking Table
– Performance Indicators
– Definitions of Key Indicator Terms
– Baselines
– Targets
– Target Rationales
– Actuals (Results achieved)
– Data Collection Methods and Staff Responsible

17
PMP Components, Cont’d
• Performance Indicator Reference Sheets (PIRS)
• Data quality assessment plan
• Evaluation Plan
– Type of Evaluation
– Timeframe
– Draft evaluation questions

18
Performance Indicators – measure a particular
characteristic or dimension of a project’s results based
on the project’s RF. Performance indicators are the
basis for observing progress and measuring actual
results compared to expected results. Performance
indicators help answer the extent to which the project
is progressing towards its objectives.

19
 Things to consider when developing and selecting
Performance Indicators:
• Indicators are needed for each level of the RF and for each results
statement
• The number of indicators selected should be sufficient (not too
few not too many) to measure progress towards the expected
result
– Rule of thumb: 1-3 indicators per results statement
• Cost and utility
– The greater the number of indicators the greater the cost to collect the
data
– Only collect data that will be useful to have
• Select indicators that have already been developed and defined

20
PMP Components Cont’d
• Definition of Key Terms – if not done can create
problem of definition bias.
• Baseline – measurements taken prior to or at the
outset of an intervention
• Actuals – Actual results for each indicator.
– Reflect indicator data that has been collected, verified and
reported at a specific time interval after the baseline has
been established.
– As compared to data that has been planned or projected,
such as a target.
21
PMP Components Cont’d - Targets
• Targets – Specific planned level of change from
the baseline to what is expected to be achieved
within an explicit time frame.
• Target Setting – based on projections using data
available from a variety of sources known to the
project.
• Also see Handout – USAID Works! Preparing a
Performance Monitoring Plan
22
Example
• The result you seek to achieve is, “employment of targeted youth
increased.”
• Your baseline is how many in the targeted group of youth are
unemployed
– “30,000 youth in the target group are unemployed at the start of the project.”
• Your target is based on research of employment statistics in targeted
geographic regions and findings from past youth employment
interventions
– “15,000 unemployed youth from the target group the end of the 3-year project”.

Indicator - Number of youth within the target group who are

unemployed
23
PMP Components Cont’d – Data Collection
• Data can be qualitative or quantitative from
primary or secondary data sources
• Example Data Collection Sources:
– Interviews
– Observations
– File and document reviews

Data can be collected from all of the above sources on

a regular site visit, using an Activity Site Report
24
Things to consider when planning for Data Collection:
• How often do we want the data?
• Are we able to collect the data in a timely manner?
• Who will be responsible for collecting the data and
who will be responsible for analyzing and reporting
it?
• In order to yield consistent results the data collection
methodology for the baseline and the actuals must be
the same over the life of the performance indicator.
• Above all the data collected should be useful.
25
PMP Components - Performance Indicator Tracking Table
• Essential part of the PMP
• Should include at the very least: indicators,
baseline, targets, actuals, data collection sources
and methods for every year the project is active.
• Better to use Excel than Word
• If making a PITT for a follow-on project its ok to
use the last actuals for the old projects as the
baseline values for the follow-on
26
Performance Indicator Tracking Tables – Things to Avoid
• Indicators that combine more than one characteristic to be
measured
• Indicators are at the wrong level of the RF – i.e. its an objective-
level indicator but in the PITT its listed as an IR-level indicator
• The indicator is written as a results statement
• The indicator is written as a target
• Data sources would not lend sufficient or reliable information
• The PITT includes more indicators then it would be possible to
collect and use data for during the LOP
• No baseline data entered
• Key terms are ambiguous
27
 Example: Health
Project, Version 1

Indicator Definition Data Methods Data Base Target Target Target

Sources of Data Collection line FY 2014 FY FY
Collection Frequency 2015 2016

Objective 2: Access to critical health services improved

Improved Reports Quarterly Annual 530
effectiveness of Increase
health system in
providing services

% increase in Quarterly 85% 100% 100%

number of
pharmacies
provided with
assistance

Sub-IR 2.3: Community health workers’ (CHWs) skills improved

Number of CHWs Training Quarterly 10800 11000 11200
and midwives that database
receive training in
vitamin A,
deworming and
nutrition
28
 Example: Health
Project, Revised
Version
Indicator Definition Data Methods Data Base Target Target Target
Sources of Data Collection line FY 2014 FY FY
Collection Frequency 2015 2016

Sub-IR 2.2: Quality of critical health services improved

Number of health Health HMIS, PCH- Health facility Quarterly 400 470 500 530
facilities facilities in supported managers
implementing PCH- health complete
supported facility data
quality assurance provinces; technical collection
standards MoPH QA reports templates
standards and send to
HMIS Kabul…

Percent of health Essential HMIS, health Health facility Quarterly 78% 85% 100% 100%
facilities stocked drugs as facility managers
with essential drugs defined by inventory compile
MoPH reports quarterly
during reporting Essential pharmacy
period Drugs List inventories…

Sub-IR 2.3: Community health workers’ (CHWs) skills improved

Number of CHWs Refresher HMIS, CHWs: Quarterly 10000 10800 11000 11200
who receive the first training at training sign disaggregate
round of refresher least 3 days; in sheets male/female;
cumulative review sign-in
training per year sheets

29
PMP Components, Continued
• Performance Indicator Reference Sheet (PIRS)
– Project must develop a PIRS for each new or custom indicator
in the PMP
– USAID Standard F Indicators already have a PIRS with very
specific definitions and information on how to record the data
If you are using a Standard F indicator in your PMP you cannot
change the indicator at all!

Take a moment to look at the PIRS template and example

handout

30
Sample Performance Indicator Reference Sheet (PIRS)

31
Evaluation – the systematic collection and analysis
of information about the characteristics and
outcomes of projects as a basis for judgments to
improve effectiveness and/or to inform decisions
about current and future programming. Evaluation
is distinct from assessment, which may be designed
to examine country or sector context.

32
PMP Components, Continued

• Evaluation Plan
– Evaluation is part of the Program Cycle
– Should be part of every project PMP
– Should be timed so that findings can be used for
decision-making, i.e., at the mid-term of the project
or if there will be a follow-on

33
 An Evaluation Plan should contain the following information:
Evaluation POC Project to Project Evaluation Internal Est. Evaluation
Questions be Start & Type or Budget Start and
Evaluated End External End Dates
Dates
1) To what extent M&E Family Start: Final, External $180,000 Start:
did the family Officer Performance
planning project Planning Oct. 2012 Jan. 2014
increase capacity of Project
local family
planning centers? End: End: Mar.
Mar.2015 2014
2) Did use of
modern family
planning methods
increase in target
areas?

34
Performance Plan Indicators for Health & Education
1. Standard Indicators – Bureau of Foreign Assistance (F) Master
Indicator List (MIL)
USG Standard or “F” indicators are organized in accordance with the Foreign
Assistance Standardized Program Structure (SPS) which is a hierarchy of
Objective, Program Areas, Program Elements and Program Sub-Elements.
Indicators are mainly associated with measuring performance at the Program
Area and Element levels within each of the five objectives.
2. Custom Indicators
Custom Indicators are measures that USAID Missions use in combination with
standard indicators to monitor progress towards intended results. Many
measures that USAID can select as custom indicators are widely used across the
development community (example, GDP growth per capita). Custom indicators
are also those developed for to measure a specific project or activity when no
equivalent standard indicator is available.
35
OSSD- Health Standard Indicators (3)
• 3.1.2.9-1: Number of individuals trained in any
component of the WHO Stop TB Strategy with
USG funding
• 3.1.6.8-4: Number of liters of drinking water
disinfected with point-of-use treatment
products as a result of USG assistance
• 3.1.7.1-1: Couple years protection in USG
supported programs
36
OSSD- Health Custom Indicators (8)
• Percent of estimated number of new smear-positive pulmonary TB cases that
were detected under DOTS (i.e. Case Detection Rate).
• Number of deliveries with a skilled birth attendant (SBA) in USG-assisted
programs
• Number of antenatal care (ANC) visits by skilled providers at USG-assisted
facilities
• Number of new antenatal care (ANC) visits during pregnancy at USG-assisted
facilities
• Number of postnatal care (PNC) visits to USG-assisted facilities
• Number of children less than 12 months of age who received PENTA 2 vaccines
• Rate of non-polio acute flaccid paralysis (AFP) cases occurring per 100,000
children less than 15 years of age (non-polio AFP rate)
• Number of USG-assisted service delivery points providing FP counseling or
services 37
OSSD- Education Standard Indicators (5)
• 3.2.1-3: Number of administrators and officials successfully
trained with USG support
• 3.2.1-14: Number of learners enrolled in primary schools
and/or equivalent non-school based settings with USG support
• 3.2.1-33: Number of textbooks and other teaching and
learning materials (TLM) provided with USG assistance
• 3.2.1-35: Number of learners receiving reading interventions
at the primary level
• 4.6.3-2: Number of persons receiving new or better
employment (including better employment) as a result of
participation in USG-funded workforce development project 38
OSSD-Education Custom Indicators (3)
• Number of teachers/educators trained with
USG support
• Number of higher education partnerships
between US and host country higher education
institutions that address regional, national
and/or local development needs
• Number of USG-funded scholarship and
exchange programs conducted through higher
education institutions 39
Developing Performance Indicators
A GROUP EXERCISE

40
Group Exercise Instructions

• “Sticky-note Brainstorming”
• Goal – In a small group identify and build
consensus around a “best” set of performance
indicators to measure results in the RF.
• Materials – Flip charts, package of sticky notes,
pens or markers
• Also refer to handout – PMP Indicator Criteria
Worksheet
41
Group Exercise Instructions
• Step 1: Write a single results statement on flip chart paper. Can use Newspaper Headline
from earlier.
• Step 2: Review the result statement in your group and ensure everyone has the same
understanding, then pass out materials.
• Step 3: Each participant thinks about indicators for the group’s results statement. Each
person writes their ideas down on sticky-notes – one idea per one sticky note.
• Step 4: Participants group indicators together on a blank piece of flip chart paper, the wall
or the table. As participants group the “like” indicators together the grouping can evolve
as the indicators are discussed.
• Step 5: For each grouping of similar indicators, the participants should collectively select
the best from each group.
• Step 6: The best indicators are posted on the results statement flip chart. Participants
then discuss when they think particular indicators are good or not good measure of the
results statement.
• Step 7: The group narrows down the top 1-3 indicators to measure the result.
42
Using a PMP to Track Performance

• High quality performance management helps to

build the evidence base for management decisions
• Performance management requires access to
useful and timely performance information over
the LOP
• By systematically tracking performance info the
PMP is an important tool for decision-making,
resource allocation, and project adaptation.
43
Using a PMP to Track Performance

Questions the PMP can help the project answer:

1) Is the project on track to achieve the results
detailed in its RF within the LOP? Why or why
not?
2) Do the hypothesized causal linkages in the RF
move in the direction we would expect?
3) Do the causal linkages need to be revised?

44
 Example: PMP as a
Management Tool

Indicator Data Baseline Target FY Actual Target Actual Target Actual FY

Collection 2014 2014 FY FY FY 2016
Frequency 2015 2015 2016

Sub-IR 2.2: Quality of critical health services improved

Number of health Annually 400 470 490 530 480 600
facilities
implementing
quality assurance
standards

Percent of health Annually 78% 85% 87% 100% 95% 100%

facilities stocked
with essential
drugs during
reporting period

Sub-IR 2.3: Community health workers’ (CHWs) skills improved

Number of CHWs Annually 10,000 10,800 10,900 11000 12,245 11200
who receive the
first round of
refresher training
per year

45
QUESTIONS???

46