Introduction to Monitoring & Evaluation

(M&E) for Economic Recovery and

Development (ERD) programs

From Harm to Home | Rescue.org

Purpose
The purpose of this session is to familiarize
new staff with the key technical resources and tools
for monitoring and evaluation of IRC Economic
Recovery and Development programs

From Harm to Home | Rescue.org

Outline
Introduction to the ERD M&E team
Our support approach and M&E resources
Basic M&E concepts and tools
Roles in country team
Support structure at region, HQ levels

From Harm to Home | Rescue.org

Who’s involved in M&E work at IRC?

From Harm to Home | Rescue.org

The ERD M&E support team
In ERD technical unit:
• M&E Advisor - Simon Fuchs
• M&E Officer – Dominique
Ndomya
• Cash Officer – Samantha Allen
• EDE TA – Zeina Shuhaibar
• Rural Livelihoods TA – Benson
Adoko
• Sr. Research Coordinator –
Clare Clingain
From Harm to Home | Rescue.org
Other key support staff:
• ERD TAs
• Country-level M&E
Coordinators
• Regional Measurement Advisor
(RMAC)
• Regional Snr Specialist – M&E
Tech
• Measurement Unit (MU)
The ERD M&E support team, cont.

Simon Fuchs Dominique

Ndomya

Zeina Shuhaibar Samantha Allen

From Harm to Home | Rescue.org

Rough outline of typical M&E roles in country team
Position M&E role
ERD coordinator Accountable for ERD program delivery, ensuring adherence
with donor requirements and sector/program specific best
practices where applicable, supervising M&E resourcing and
planning
Program Responsible for ensuring that M&E activities are adequately
manager and planned, resourced and executed, and are aligned with the
project officers project’s key activities and objectives. Sometimes handle
data collection, mgmt, and analysis.
M&E specialist in Responsible for ensuring technical quality and accuracy of
ERD team M&E data. Often handles data collection, mgmt, analysis.
M&E coordinator Supports cross-sectoral programs, M&E planning and
(across organization, ensuring that M&E tools and standards (i.e.
programs) MFA) are consistent with good practices, general
troubleshooting

From Harm to Home | Rescue.org

Example Job Descriptions for M&E
related positions
• M&E Coordinator
• M&E Manager
• Senior M&E Officer
• Data Officer

From Harm to Home | Rescue.org

Where can I get support beyond my country team?
Role Contact
Finding data on ERD in my context TA, relevant cluster/WG
Reviewing or developing economic/livelihoods TA, ERD M&E
indicators for a proposal
Hiring M&E staff for ERD project M&E co, RMAC, TA
General M&E planning/team structure questions TA, RMAC
Livelihoods/economic needs assessment TA, field coordinator
ERD program M&E technical questions: ERD M&E, TA, RMAC
- Questionnaires/data collection tools,
- Mobile data collection
- Databases & data analysis
Impact evaluations, RCTs, etc. TA, ERD Research Advisor,
REL Dept. (HQ)

From Harm to Home | Rescue.org

Key IRC M&E initiatives
• IRC MEAL Handbook
• Theories of Change + Core Indicators for Economic
Wellbeing
• Annual Statistics
• M&E Toolkits
• Cash Toolkit
• Livelihoods Center Toolkit
• Program Indicator Dashboard (PID)
• Measurement Community of Practice
• ICT for Programs Community of Practice

10

From Harm to Home | Rescue.org

Our guiding principles
1. Be responsive to needs of IRC programs
2. Build skills in data and M&E skills at all levels
3. Standardize tools and approaches where possible
4. Measure outcomes using Core Indicators
5. Incorporate gender lens into M&E approaches
6. Integrate program and M&E activities
7. Use electronic data collection and mgmt. tools
8. Handle sensitive data securely and responsibly
9. Collect only data that will be used
10. Promote critical analysis and use of data for program
decision-making

11

From Harm to Home | Rescue.org

How we prioritize support to projects
• ERD programs, or multi-sectoral with substantial ERD
• M&E/data needs are flagged by the TA and/or country program
• Program is of minimum scale: ~300+ benfs, $250k+ budget, 6+ months
• Strategic project/donor: new funder for IRC, or high profile project
• Program activities/modalities are widely implemented by IRC at present
• Program activities/modalities are "new" and being developed for broader
use by country programs (NB: we avoid supporting “one-off” projects!)
• Staff bandwidth/capacity both in ERD TU team, and country team needs:
• - Programmatic champion
• - In country super users / admin / technical focal point
• Budget for support: programs do not need to budget for our support, but
should budget adequately for M&E activities and key costs like staff,
technology, logistics.

12

From Harm to Home | Rescue.org

How we prioritize cont.
• Region/location: we are not region-matrixed, and try to support all
regions/countries who request support equitably
• Language: we can provide direct support in English, French or
Spanish. If support in Arabic and other languages is required please
let us know.
• Types of programming - aside from the programs prioritized in ERD's
overall strategy, we do not limit our support to certain project types.
• Simon Fuchs focuses on Cash and Urban LLHs

13

From Harm to Home | Rescue.org

When and how to request M&E support

• Initial requests for M&E support on new projects

should be made to the ERD M&E Advisor, by
email through the ERD Coordinator and ERD TA
supporting your country:
simon.fuchs@rescue.org
Or
dominique.ndomya@rescue.org

14

From Harm to Home | Rescue.org

General M&E Resources
• IRC MEAL Handbook
• Overview of Economic Wellbeing Core Indicators
• ERD basic programs and sectors
Describes ERD’s strategy and common program types
Describes IRC’s organizational best practices/standards for M&E
• IRC Outcome and Evidence Framework
IRC’s standard Theories of Change and Core Indicators
• Better Data Better Decisions
Data quality checks, analysis and use

15

From Harm to Home | Rescue.org

ERD-specific M&E Library  Box link
1. M&E Basics
2. ERD donor indicator lists and guidance
3. ERD data collection tools/surveys (ODK etc)
4. ERD databases (coming soon)
5. Project assessments/evaluations (coming soon)
6. Other program-specific tools

16

From Harm to Home | Rescue.org

Basic M&E Concepts

From Harm to Home | Rescue.org

Logframe: how change occurs in theory!

Sustainability
Outcomes
Activities

Outputs

Impact
Inputs

18

From Harm to Home | Rescue.org

TOC: how change occurs in reality!

19

From Harm to Home | Rescue.org

Monitoring
The process of regularly and systematically collecting and
analyzing information about a project and, when
appropriate, using it to make adjustments to the
project. Project monitoring data may be used to adjust
project implementation, enable internal and external
reporting, inform project design and advocacy, and promote
accountability to beneficiaries.
• Input, output, activity and process

20

From Harm to Home | Rescue.org

Evaluation
The systematic collection of information about the
activities, characteristics, and outcomes of
programs to make judgments about the program,
improve program effectiveness, and/or inform
decisions about future program development.
• Relevance, effectiveness, efficiency,
impact, sustainability

21

From Harm to Home | Rescue.org

What questions does M&E seek to
answer?
• M: Are we doing things right?

• E: Are we doing the right things?

22

From Harm to Home | Rescue.org

 Why do we do M&E?
• Donor reporting
• Accountability
• Performance improvement
• Getting feedback from clients and stakeholders
• Assessment of needs, gaps and program
coverage
• Demonstrating value, scale and reach

23

From Harm to Home | Rescue.org

Practical steps in project M&E
• Define logic: How should we define success? How are we going to
get there?
• Define indicators: Which indicators should we use to measure and
understand success and challenges?
• M&E planning: When and how will activities be done? Who will be
responsible for doing M&E work?
• Design of tools: How will we measure the indicators?
• Data collection: What is the most efficient way to collect and
compile the data we need?
• Data analysis: How can we analyze the information?
• Data reporting and dissemination: How do we present and share
information and with whom?
• Using data for performance improvement: How does our data
inform changes to programming?

24

From Harm to Home | Rescue.org

Some M&E basics that every IRC
project should have! (from MFA)
• Theory of change
• Logframe/results matrix
• Indicator matrix
• M&E plan/calendar w/ key reporting deadlines
• Structured data collection and analysis tools
• Electronic data collection strongly encouraged

25

From Harm to Home | Rescue.org

At the Project Design & Proposal Development Stage:
A project strategy is developed using a theory of change, relevant
indicators, and all available evidence and learning

Develop a logframe using SMART IRC Core

Indicators and donor indicators that measure
Align the project to IRC’s high-level OEF Outcomes project progress, quality, and outcomes

Start here
and work
your way
backward

26

From Harm to Home | Rescue.org

At the Start Up /Grant Start Stage:
A plan for monitoring project activities and outcomes is adopted prior to
project launch
Develop an indicator matrix that outlines (at least):
• Indicators and data elements
• Data collection frequency
• Indicator calculations
• How data will be disaggregated
• Data collection tools for each data element

27

From Harm to Home | Rescue.org

… and …
Include monitoring activities in the project work plan or develop a stand-alone work plan for monitoring
activities that includes:
• Routine data collection activities
• Data Quality Audits
• Critical data collection events
• Staff M&E training
• End of Project Learning
• Donor report deadlines

28

From Harm to Home | Rescue.org

… and …
Data collection tools are developed and a data
management systems is established

The ERD TU supports CommCare and Kobo

for data collection and Power BI and Excel
for data visualization and analysis

29

From Harm to Home | Rescue.org

Common sources of M&E data
• At output, activity and process level, most data can be
pulled from project records
• At outcome level, we often depend on:
• - baseline-endline survey data,
• - focus groups,
• - key informant interviews
• - other assessments
• Data is generally self-reported and can be
costly/challenging to collect.

30

From Harm to Home | Rescue.org

The Data Pyramid!

Global • Ex: Aggregate output numbers

data
Project • Ex: Project logframe + indicator
tracking table
data
• Ex: Beneficiary /
facility / transaction
Field data data

31

From Harm to Home | Rescue.org

M&E Support structures

From Harm to Home | Rescue.org

M&E should not be a silo
M&E is a critical component of IRC’s overall project
management and implementation approach. There
is no single “correct” structure for M&E roles and
responsibilities, but the whole project team must
take ownership of M&E.

It is our collective responsibility to ensure that

projects are working for our clients!

33

From Harm to Home | Rescue.org

Budgeting for M&E
Rule of thumb is to budget 3-5% of program costs
for M&E. Typical cost drivers for M&E:
• Staff
• Logistics/transport for field data collection
• Technology: Hardware (tablets!) + software
(Commcare where applicable)

34

From Harm to Home | Rescue.org

Using technology for M&E
Why do we advocate for electronic data collection?

• Standardization: Consistent, structured tools and

processes (can integrate Core Indicators)
• Efficiency: Faster data entry/collation at lower cost
• Data Quality: Much lower rate of errors
Using technology for M&E
• Accountability: You know who is doing what, when
and how (mitigates risk of fraud)
• Analysis: clean data is more useful for programs

35

From Harm to Home | Rescue.org

What are the keys to successful use of tech by programs?

• Scope: Defining what we want from the tech (business analysis)

• People: finding skillsets + program/tech champions in team
• Cost: budgeting properly for tech
• Time: 3-6 months needed for development, testing, feedback (less time
needed for adaptation)
• Process: dataflow mapping (“business analysis”), work-planning and
executing the tech setup and maintenance
• Technology: finding a technology platform that fits what we want to do
• Adaptation: flexibility of tools, working with third-party software vendors
• Infrastructure: power, internet, hardware

36

From Harm to Home | Rescue.org

 Is mobile data collection appropriate for my situation?

37

From Harm to Home | Rescue.org

Economic Well-being Core Indicators

From Harm to Home | Rescue.org

OEF: IRC’s Theories of Change
• IRC2020 strategy sets forward 32 outcomes,
including four high-level outcomes to achieve
“Economic Wellbeing” for IRC clients:
1. People meet basic needs and avoid negative
coping strategies
2. People are food secure
3. People generate income and assets
4. Women use and control resources

39

From Harm to Home | Rescue.org

TOC example:

• Conceptual framework describing the key causal pathways (sub-

outcomes) needed to reach each outcome. Based on evidence +
experience.

40

From Harm to Home | Rescue.org

What are the Core Indicators?
• Core Indicators are used to measure IRC's progress in
achieving outcomes of the OEF
• Core indicators provide a menu of high quality, standard
indicators for proposals
• Core indicators include outcome, output and client
satisfaction-level measures
• Core indicators are based on standard donor indicators
• Each TOC has a core indicator list
• Using IRC’s core indicators = G&G standard for
Measurement!

41

From Harm to Home | Rescue.org

Core Indicator Resources
1. Indicator Selection Tool (IST) website:
indicators.rescue.org
2. Introductory Presentation
3. Core Indicator Guidance Box folder

42

From Harm to Home | Rescue.org

Economic Well-being Core Indicator toolkit
Indicator Selection
 IST Website: https://indicators.rescue.org
 Core Indicator Selection Tool (IST)
 Core Indicator Guidance Notes
Data Collection
 Outcome Survey
 Annual Statistics (Outputs) template
 Client Satisfaction Survey
Data Analysis + Use
 Outcome Survey Dashboard
 Client Satisfaction Survey Dashboard
 Annual Statistics Dashboard

43

From Harm to Home | Rescue.org

The project cycle

Design
Core Indicator Selection Tool (IST)
Core Indicator Guidance Notes

Close-out Start-up

Outcome Survey Dashboard Outcome Survey

Client Satisfaction Dashboard Annual Statistics template
Annual Statistics Dashboard Client Satisfaction Survey
Implementa
tion

44

From Harm to Home | Rescue.org

Other Key resources
 IRC Monitoring Fundamentals Guide
 ERD M&E Induction Training
 EWB Theories of Change (OEF) Folder
 OEF Website
 Donor Indicator and M&E Guidance folder
 USAID (OFDA/PRM), DFID, ECHO, SEEP, SPHERE, etc.
 FEWSNet website
 Introduction to KoboCollect
 Introduction to Commcare
 Gender-sensitive M&E guidance note
 Survey sampling guidance note

45

From Harm to Home | Rescue.org

Selecting core indicators
Consult with your TA!
1. Select indicators relevant the project
2. Select only indicators where you can reasonably expect to see a
change during project
3. Choose indicators that can be measured in the timeframe of the
project
4. Select indicators that are feasible to collect (consider time, budget,
staff, etc.)
5. Select indicators for which you can reasonably expect a minimum
level of data quality - defined as consistent, complete, and correct.

46

From Harm to Home | Rescue.org

Core indicator guidance notes

47

From Harm to Home | Rescue.org

Data collection tools
Data collection tools folder

• Outcome Survey
• Annual Statistics template
• Satisfaction Survey
• Cash PDM survey
• Etc.

48

From Harm to Home | Rescue.org

Data Analysis tools

Work in progress!

49

From Harm to Home | Rescue.org