MONITORING AND EVALUATION TRAINING MANUAL FOR

CBOs AND NGOs

Overall Goal:
To give concepts, skills and knowledge on monitoring and evaluation for
development

Developed by:
Institutional Capacity Building Team
SHIELD Project
International Rescue Committee - Thailand
1

Table of Contents
TIME-TABLE ................................................................................................................ 3
monitoring and evaluation Training, Pre Test ........................................................... 4
Session I: Orientation ............................................................................................... 5
Session II:
Warm-up .............................................................................................. 7
WORKSHEET 2 1, Key steps of project management ...................................... 10
WORKSHEET 2 2, CAUSAL HYPOTHESIS ..................................................... 11
Session- III: what is monitoring? .............................................................................. 12
WORKSHEET 3 1, MONITORING .................................................................... 13
Session- IV: what is evaluation? ............................................................................... 14
WORKSHEET 4 1, EVALUATION .................................................................... 16
Session V:
Developing indicators ....................................................................... 18
WORKSHEET 5 1, INDICATORS ...................................................................... 22
WORKSHEET 5 2, DEVELOPING OUTPUT/EFFECT/IMPACT IMPACT
INDICATORS .23
WOKSHEET-5-3DEVELOPING EFFICIENCY, EFFECTIVENESS AND
IMPACT INDICATORS INDICATORS ..................................................... 24
Session VI:
Planning for monitoring and evaluation ......................................... 26
WORKSHEET 6 1 ,Develop monitoring plan 1 .............................................. 31
WORKSHEET 6 2, DEVELOP MONITORING PLAN 2................................ 32
WORKSHEET 6 3, DIFFERENT TYPES OF EVALUATION ........................... 34
WORKSHEET 6 4, Case Study............................................................................ 36
WORKSHEET 6 5, EVALUATION BASICS ...................................................... 39
WORKSHEET 6 6, TERMS OF REFERENCE .................................................... 40
Session VII: Gathering information for monitoring and evaluation .................... 43
WORKSHEET 7 1, Different social research tools............................................. 45
Session VIII: Data analysis ........................................................................................ 48
WORKSHEET 8 1, DATA ANALYSIS ............................................................... 51
.................................................................................................................................. 51
Session IX: Reporting ................................................................................................ 53
WORKSHEET 9 1, EVALUATION REPORT FORMAT................................... 55
Monitoring and evaluation Training, Post-Test....................................................... 56
References ................................................................................................................... 57

TIME-TABLE
DAY

DAY 1

9:00 10:30

PLANNING FOR
MONITORING
AND EVALUATION

DAY 3

DATA
COLLECTION
METHODS

PLANNING FOR
MONITORING AND
EVALUATION

BASIC DATA
ANALYSIS
METHODS

WHAT IS EVALUATION? ( 30
MIN)
DEVELOPING INDICATORS

PLANNING FOR
MONITORING AND
EVALUATION

REPORTING

3:15 TO 4:30 PM

COFFEE BREAK (15 MINUTES)

WARM UP (30 45
MIN)
WHAT IS
MONITORING? ( 30
MIN)

1:00 TO 3:00 PM

LUNCH BREAK (ONE HOUR)

DAY 2

COFFEE BREAK (15 MINUTES)

REGISTRATION (15
MIN)
PRE-TEST (15 MIN)
ORIENTATION ( 1
HOUR)

10:45 TO 12:00 AM

DEVELOPING
INDICATORS

PLANNING FOR
MONITORING AND
EVALUATION

WRAP UP SESSION
POST TEST
TRAINING EVALUATION

MONITORING AND EVALUATION TRAINING

PRE TEST

Please answer the following questions to know your existing knowledge and
skills.
1) What is Monitoring?

2) Why do we do Monitoring?

3) Who does Monitoring?

4) When do we do Monitoring?

5) What are the important things to look at when you monitor a project?

6) What are indicators? Please give an example of indicator.

7) What is evaluation?

8) For what purposes do you normally do evaluation?

9) When do you normally evaluate?

10) Write down the monitoring and evaluation tools you know.

SESSION I:

ORIENTATION

The Facilitator/s and participants will get an opportunity to introduce

themselves, to exchange information about past experiences and to share
workshop expectations
Time:

1 hour

Materials:

A4 size paper (cut into four pieces), flipcharts, markers

Introduction
Step 1:

Ask the participants to divide into two groups equally: Group A

and Group B

Step 2:

Then cut A4 size paper into four pieces and distribute one piece
to each participant. Ask them to write down
their name
Mother organization
M&E related experience they have before
When everyone finishes writing, ask each group to stand in line
facing other group.

Step 3:
Step 4:

Step 5:

Ask people from Group A to give their card to the one who is in
front of him/her in Group B introducing themselves. Then the
one from Group B also introduces about himself/herself
exchanging their cards.
When all have finished introducing each other, ask each pair to
introduce their friends to the whole class.

Forming groups
Step 6:

Ask all the participants to stand up in line again based on the

birth months starting from January. Based on the number of
participants, ask them to count 1 to 5 (if you want to form 5
groups). Then ask all people with 1, 2, 3, 4, 5 number can sit
together. Ask them to give a name for their group. Tell them
that they have to be together for group work for three days.

Expectations
Step 7:

Step 8:

Step 9:

Tell the participants that this workshop will focus on

Monitoring and Evaluation of a project included in project
cycle management. Ask them to write down their expectations
and concerns related to this workshop on a flipchart. (5 10
minutes)
When all the groups finish, ask them to read other groups
expectations and concerns and if they find the same ones, they
can write down group number in the bracket.
Then share the workshop objectives and if what they expect
cannot be included in the workshop objectives, discuss with
them. Related to the concerns, if the facilitators can deal with
them, please tell them how and if the participants need to deal
with, tell them to set ground rules and how they want to take
action if somebody breaks the rule/s. e.g. phone calls,
punctuality, dominate the class by one or two participants.

SESSION II:

WARM-UP

This session revisits key steps of project management and terminology used in
M&E to refresh the memory.
Time:

About 30 to 45 minutes

Material:

flipcharts, markers, worksheets

Step 1:

The facilitator starts the session by asking the key steps of project
management. Then show the key steps of project management
distributing Worksheet 2.1: project management.

Key steps of project management

Assessing needs

Learning from the

project and
evolving

Evaluating the
project

Planning the project

Implementing and
monitoring the
project

When a project is started to implement, it is necessary to monitor the project activities

while implementing the project and at the end of the project, normally, the project is
evaluated.

Step 2:

The facilitator ask the participants to fill the blanks included in the
worksheet.

The causal hypothesis is a concise statement of the logic behind the project.
Normally, causal hypothesis is written as follows:
8

This set of --------- and ---------- will result in these products and services (--------).
These will in turn lead to these changes in peoples behavior (------------), which will
contribute to the desired ---------------. (activities, IMPACT, outputs, inputs, effects)
Step 3:

Then discuss with the participants about the terms used in causal
hypothesis and show example.
Below is the meaning of the terms used in causal hypothesis.

IMPACT:
A change in population's health, economic or social status.
EFFECT:
A change in the knowledge, attitudes, skills, intentions, or and
behaviors needed of the population that contribute to the desired IMPACT.
OUTPUTS: Products and services that must be in place before the EFFECTS can
occur.
ACTIVITIES:
The technical and support tasks required to produce the
OUTPUTS.
INPUTS:
Resources required to support your ACTIVITIES.
Inputs refer to the set of resources (i.e. financial, policies, personnel, facilities,
space, equipment and supplies, etc.) that are the basic materials of the
project/program.
Causal Pathway Framework
Design Direction

Activities

Inputs

Outputs

Effect

Implementation, Monitoring and Evaluation

Inputs

Trainers
Training materials
Training equipments
Training venue
Training participants
(staff from
CBOs/NGOs)

Activities

M&E training
Training
follow-up

Outputs

# of participants
who attended
M&E training
# of participants
who gets M&E
skills

Effects

# of staff who
apply
monitoring and
evaluation skills
in their
organizations
# of staff who
can monitor
effectively and
evaluate the
project

Impact

Improved
project quality
with effective
monitoring
and evaluation
system

WORKSHEET 2 1
Key steps of project management

Assessing needs

Learning from the

project and
evolving

Evaluating the
project

Planning the project

Implementing and
monitoring the
project

When a project is started to implement, it is necessary to monitor the project activities

while implementing the project and at the end of the project, normally, the project is
evaluated.

10

WORKSHEET 2 2
CAUSAL HYPOTHESIS
The causal hypothesis is a concise statement of the logic behind the project.
Normally, causal hypothesis is written as follows:
This set of --------- and ---------- will result in these products and services (--------).
These will in turn lead to these changes in peoples behavior (------------), which will
contribute to the desired ---------------. (activities, IMPACT, outputs, inputs, effects)
The terms used in causal hypothesis.
IMPACT:
A change in population's health, economic or social status.
EFFECT:
A change in the knowledge, attitudes, skills, intentions, or and
behaviors needed of the population that contribute to the desired IMPACT.
OUTPUTS: Products and services that must be in place before the EFFECTS can
occur.
ACTIVITIES:
The technical and support tasks required to produce the
OUTPUTS.
INPUTS:
Resources required to support your ACTIVITIES.
Inputs refer to the set of resources (i.e. financial, policies, personnel, facilities,
space, equipment and supplies, etc.) that are the basic materials of the
project/program.
Causal Pathway Framework
Design Direction

Activities

Inputs

Outputs

Effect

Implementation, Monitoring and Evaluation

Inputs

Trainers
Training materials
Training equipments
Training venue
Training participants
(staff from
CBOs/NGOs)

Activities

M&E training
Training
follow-up

Outputs

# of participants
who attended
M&E training
# of participants
who gets M&E
skills

Effects

# of staff who
apply
monitoring and
evaluation skills
in their
organizations
# of staff who
can monitor
effectively and
evaluate the
project

Impact

Improved
project quality
with effective
monitoring
and evaluation
system
11

SESSION- III:

WHAT IS MONITORING?

After this session, the participants will understand the concept of monitoring and
how to develop good indicators.
Time:

30 45 minutes

Material:

flip chart, markers, worksheets

Step1:

Eliciting (15 - 20 minutes)

Ask the following questions to the participants:
- What is Monitoring?
- Why do we do Monitoring?
- Who does monitoring?
- When do we do Monitoring?

Write down what the participants discuss on flipchart.

Worksheet 3 1 : Monitoring and discuss for a while.

Then distribute the

12

WORKSHEET 3 1
MONITORING
What is Monitoring?
Monitoring is the systematic collection of information on all aspects of the project
while it is being implemented.
It can be divided into internal monitoring (staff performance, planned expenditure
for each activity versus actual expenses, procurement procedures etc) and
external monitoring (planned versus actual activities, timely implementation of
activities, targeted beneficiaries versus true beneficiaries, unintended effects on
the community and unexpected problems etc). Both are important and both need
to be monitored.
Why Do We Do Monitoring?
We do monitoring to analyze the current situation, identify problems and find
solutions, discover trends and patterns, keep project activities on schedule,
measure progress towards objectives, formulate/revise future goals and
objectives, make decisions about human, financial, and material resources.
Actually, it is a very useful tool for management and provides necessary
information for evaluation.
In other words, monitoring means checking how things are going on and
comparing actual progress to what is planned.
Who Does Monitoring?
Monitoring is concerned both with project staff, implementation organization and
donors. As mentioned above, it is useful for management and project manager or
program coordinator conducts monitoring on all aspects of the projectbudget/finance, materials, staff, activities, outputs/results etc. Respected project
staff are also responsible for monitoring staff and tasks under them-for example,
finance manager has to monitor the accountant and cashier as well as budget
allocations. At the same time, representatives of donors also conduct monitoring
to measure the progress towards objectives and goal/impact.
When Do We Do Monitoring?
According to the desired schedule of implementation of the projects.
e.g. monthly, bi-monthly, quarterly

13

SESSION- IV:

WHAT IS EVALUATION?

After this session, the participants will understand the

concept of evaluation
Time:

30 45 minutes

Material:

flipcharts, markers, worksheet

Step 1:

In plenary, ask the following questions to the participants: (15 - 20

minutes)
What is evaluation?
Why Do We Do Evaluation?
When Do We Evaluate?
Who Does Evaluation?

The facilitator can read through the worksheet 4 1: Evaluation together with the
participants to understand more about evaluation.
Step 2:

Show the below project activity cycle to the participants.

14

Step 4:

The facilitator can end the session with below explanation.

M&E is an ongoing process in a project since the project is started.
It is important to recognize that monitoring and evaluation are not
magic wands that can be waved to make problems disappear, or to
cure them, or to miraculously make changes without a lot of hard
work being put in by the project or organization. In themselves,
they are not a solution, but they are valuable tools.

Monitoring and evaluation can:

Help you identify problems and their causes;
Suggest possible solutions to problems;
Raise questions about assumptions and strategy;
Push you to reflect on where you are going and how you are getting there;
Provide you with information and insight;
Encourage you to act on the information and insight;
Increase the likelihood that you will make a positive development difference.
15

WORKSHEET 4 1
EVALUATION
What is evaluation?
Evaluation is a systematic and objective assessment of ongoing or completed
project. It makes comparison of the outcomes of the project with planned ones.
Why Do We Do Evaluation?
The primary objective of evaluation is to ascertain whether the project has
achieved its intended objectives. By drawing conclusions, evaluation intends to
provide recommendations for the improvement on the future course of the
project as well as lessons learned for other projects. Some big organizations use
specific criteria when they do evaluation.
When Do We Evaluate?
Periodically, mid-term, at the end of the project (final evaluation) and years after
the completion of the project (post-evaluation).
Who Does Evaluation?
Project manager or assigned project staff can conduct internal evaluation and
donor/s or consultant/s can conduct external evaluation.
Project activity cycle

16

M&E is an ongoing process in a project since the project is started. It is important

to recognize that monitoring and evaluation are not magic wands that can be
waved to make problems disappear, or to cure them, or to miraculously make
changes without a lot of hard work being put in by the project or organization.
In themselves, they are not a solution, but they are valuable tools.
Monitoring and evaluation can:
Help you identify problems and their causes;
Suggest possible solutions to problems;
Raise questions about assumptions and strategy;
Push you to reflect on where you are going and how you are getting there;
Provide you with information and insight;
Encourage you to act on the information and insight;
Increase the likelihood that you will make a positive development difference.

17

SESSION V:

DEVELOPING INDICATORS

At the end of this session, the participants will be able to develop indicators
Time:

About 2:30 hour

Material:

flipcharts, markers, worksheets

Step 1:

The session can be started with the question, What is indicator?

After discussion for a while, distribute Worksheet 5 1: Indicators
Indicators are Units of measurement that tell us about IMPACT,
EFFECT and OUTPUT to judge the effectiveness of the Project
Indicators should be:
Ethical
Useful
Scientifically Robust
Reliable
Accessible
Good indicators should be 1) practical, 2) independent, 3)
measurable and 4) targeted.
18

Give the below examples and ask them if they are good indicators or not.
Yes No
Mortality rate

Health education materials

# of clinics offering STD treatment

% of clients satisfied with quality of STD treatment

# of community health workers trained

% of student who score at least 90% on final exam

Sexual violence counseling

Step 2:

e.g.

Ask the groups to read Worksheet 5 2: the causal pathway

hypothesis of Ah Lin Yaung. And develop indicators to measure
outputs, effects and impact(s). (Group work 15 30 minutes and 5
minutes presentation for each group)
Impact indicators
Reduced HIV transmission rate among young people in targeted
areas (compare with a baseline data)
Effect indicators
Increased use of disposable needles among drug users (survey and
compare with a baseline data)
Increased number of young people who understand how they can
contract HIV/AIDS
Output indicators
# of health education sessions organized by peer educators
# of trained peer educators in targeted community

Developing efficiency, effectiveness and impact indicators 1

Step 4:

Identify the problem situation you are trying to address.

following might be problems for Ah Lin Yaung:

The

Young people dont have HIV/AIDS health education.

Needle sharing among young people
People living with HIV/AIDS (PLHA) do not get access to
health care services.

Taken from CIVICUS Monitoring and Evaluation

19


Step 5:

HIV transmission rate among young people is high.

Develop the vision (impact) of the project you want to achieve. This
will give you impact indicators. For Ah Lin Yaung, the end result
(vision) can be: Healthy young people in targeted areas.
What will tell you that the vision has been achieved? What signs
will you see that you can measure that will prove that the vision
has been achieved?
For example: if your vision was that young people in the
community would be healthy, then you can use health indicators to
measure how well you are doing. Do fewer young people die of
HIV/AIDS? Has the HIV/AIDS infection rate been reduced? If you
can answer yes to these questions then progress is being made.

Step 6:

Develop a process vision for how you want things to be achieved.

This will give you process indicators.
If, for example, you want success to be achieved through
community efforts and participation, then your process vision
might include things like peer educators from the community
trained and providing health education, counseling session and
other necessary healthcare services used by all; targeted
community gets involved in all planned activities of the project,
and so on.

Step 7:

Develop indicators for effectiveness.

For example, if you believe that you can reduce HIV/AIDS
transmission rate by using disposable needles and condoms among
young people, then you need indicators that show you have been
effective in promoting use of disposable needles and condoms e.g.
evidence from a survey among targeted communities compared
with a baseline survey.

Step 8:

Develop indicators for your efficiency targets

Here you can set indicators such as: planned health education
sessions are run within the stated timeframe, costs for needles,
condoms and health sessions are kept to a maximum of US$ ___
per participant, no more than ____ hours in total of staff time to be
spent on organizing a health education session, no complaints
about health education session etc.
20

With this framework in place, you are in a position to monitor and evaluate
efficiency, effectiveness and impact

21

WORKSHEET 5 1
INDICATORS
Indicators are Units of measurement that tell us about IMPACT, EFFECT and
OUTPUT to judge the effectiveness of the Project
Indicators should be:
Ethical
Useful
Scientifically Robust
Reliable
Accessible
Good indicators should be 1) practical, 2) independent, 3) measurable and 4)
targeted.
Please give the below examples and ask them if they are good indicators or not.
Yes No
Mortality rate
Health education materials
# of clinics offering STD treatment
% of clients satisfied with quality of STD treatment
# of community health workers trained
% of student who score at least 90% on final exam
Sexual violence counseling

22

WORKSHEET 5 2
DEVELOPING OUTPUT/EFFECT/IMPACT INDICATORS
Causal hypothesis
Ah Lin Yaung (ALY) will identify vulnerable groups among young population in
targeted areas and train peer educators to give health education among
vulnerable groups. At the same time, ALY will provide needles for drug-users
and distribute condoms among high-risked groups. ALY will open counseling
center for People Living with HIV/AIDS (PLHA) and mobile clinics to give
treatments for opportunistic infections. These activities can change knowledge
and attitudes of young people and PLHA to prevent getting HIV or spreading
HIV to other people. Accessibility of needles and condoms can also change the
practice and behaviors of young people for safer sex. These changes will then
contribute to reduce HIV transmission rate among young people in targeted
areas.
e.g.

Impact indicators
Reduced HIV transmission rate among young people in targeted
areas (compare with a baseline data)
Effect indicators
Increased use of disposable needles among drug users (survey and
compare with a baseline data)
Increased number of young people who understand how they can
contract HIV/AIDS
Output indicators
# of health education sessions organized by peer educators
# of trained peer educators in targeted community

23

WORKSHEET 5 3
DEVELOPING EFFICIENCY, EFFECTIVENESS AND IMPACT
INDICATORS 2
1) Identify the problem situation you are trying to address. The following
might be problems for Ah Lin Yaung:

Young people dont have HIV/AIDS health education.

Needle sharing among young people
People living with HIV/AIDS (PLHA) do not get access to
health care services.
HIV transmission rate among young people is high.

2) Develop the vision (impact) of the project you want to achieve. This will
give you impact indicators. For Ah Lin Yaung, the end result (vision) can
be: Healthy young people in targeted areas.
What will tell you that the vision has been achieved? What signs
will you see that you can measure that will prove that the vision
has been achieved?
For example: if your vision was that young people in the
community would be healthy, then you can use health indicators to
measure how well you are doing. Do fewer young people die of
HIV/AIDS? Has the HIV/AIDS infection rate been reduced? If you
can answer yes to these questions then progress is being made.
3) Develop a process vision for how you want things to be achieved. This
will give you process indicators.
If, for example, you want success to be achieved through
community efforts and participation, then your process vision
might include things like peer educators from the community
trained and providing health education, counseling session and
other necessary healthcare services used by all; targeted
community gets involved in all planned activities of the project,
and so on.
4) Develop indicators for effectiveness.

Taken from CIVICUS Monitoring and Evaluation

24

For example, if you believe that you can reduce HIV/AIDS

transmission rate by using disposable needles and condoms among
young people, then you need indicators that show you have been
effective in promoting use of disposable needles and condoms e.g.
evidence from a survey among targeted communities compared
with a baseline survey.
5) Develop indicators for your efficiency targets
Here you can set indicators such as: planned health education
sessions are run within the stated timeframe, costs for needles,
condoms and health sessions are kept to a maximum of US$ ___
per participant, no more than ____ hours in total of staff time to be
spent on organizing a health education session, no complaints
about health education session etc.
With this framework in place, you are in a position to monitor and evaluate
efficiency, effectiveness and impact

25

SESSION VI: PLANNING FOR MONITORING AND

EVALUATION

At the end of this session, the participants will know monitoring and evaluation
process and how to plan monitoring and evaluation for a project.
Time:

About 4 - 5 hours

Material:

flipcharts, markers, Worksheets

Designing a monitoring system

Step 1:

The facilitator can start the session as follows:

Monitoring and evaluation should be part of your planning process.
It is very difficult to go back and set up monitoring and evaluation
systems once things have begun to happen. You need to begin
gathering information about performance and in relation to targets
from the word go. The first information gathering should, in fact,
take place when you do your needs assessment. This will give you
the information you need against which to assess improvements
over time.
Indicators developed in previous session can also provide the
framework for your monitoring and evaluation system. They tell you
what you want to know and the kinds of information it will be useful
to collect.
26

Step 2:

Revisit causal hypothesis, Ah Lin Yaung.

Ah Lin Yaung will identify vulnerable groups among young population in

targeted areas and train peer educators to give health education among
vulnerable groups. At the same time, Ah Lin Yaung will provide needles for
drug-users and distribute condoms among high-risked groups. Ah Lin Yaung
will open counseling center for PLHA and mobile clinics to give treatments for
opportunistic infections. These activities can change knowledge and attitudes of
young people and PLHA to prevent getting HIV or spreading HIV to other
people. Accessibility of needles and condoms can also change the practice and
behaviors of young people for safer sex. These changes will then contribute to
reduce HIV transmission rate among young people in targeted areas.
Develop monitoring plan - 1

Step 3:

Introduce the concepts of efficiency, effectiveness and impact

Efficiency: Whether the resources used for the activities are
appropriate in terms of output or they are cost-effective.
For example, training program that train 50 peer educators costs
$10000. Is it efficient or cost-effective?
Effectiveness: It is the measure of the extent of the achievement of
the project against the target objectives.
For example, we plan to improve the qualifications of all high
school teachers in particular area, did we succeed after project
completion?
Impact: It measures whether or not project implementation has
contributed to the change in the situation of the problem it was
trying to address.
For example, high HIV transmission rate among young people in
the community was the main problem before and our project really
made a difference to this situation that is reduction in HIV
transmission rate among young people in the community.
Explain to the participants that a monitoring system needs to cover
all three. In the previous session, the groups have already
developed a list of indicators for each of the three aspects.

Step 4:

Ask the participants to develop a monitoring system of HIV/AIDs

project of Ah Lin Yaung covering all these three. ( 30 minutes
discussion and 5 minutes presentation)
Efficiency:
e.g.
i. How much does it cost to organize a training workshop?
ii. How many participants join?
27

iii. How much does it cost per participant?

iv. Is it cost effective?
v. Whether the resources used for the activities are appropriate
in terms of output or they are cost-effective.
No.

Date

All these information can be tracked easily in a tracking sheet.

Training
topic

No.
participant
M
F

Duration

Total training cost

Remark

Effectiveness:
For example, we plan to improve the qualifications of all high
school teachers in particular area, did we succeed after project
completion?
You can develop indicators to monitor the progress to achieve this
target.
Impact:
For example, you might have an indicator of impact which is that
safer sex options are chosen as an indicator that young people
are now making informed and mature lifestyle choices. The
variables that might affect the indicator include:

Age
Gender
Religion
Urban/rural
Economic category
Family environment
Length of exposure to your projects initiative
Number of workshops attended.

By keeping the right information you will be able to answer many

questions such as:

Does age make a difference to the way our message is received?

Does economic category i.e. do young people in richer areas
respond better or worse to the message or does it make no
difference?
Does the number of workshops attended make a difference to
the impact?
28

Answers to these kinds of questions enable a project or

organization to make decisions about what they do and how they
do it, to make informed changes to programmes, and to measure
their impact and effectiveness. Answers to questions such as:

Do more people attend sessions that are organized well in

advance?
Do more schools participate when there is no charge?
Do more young people attend when sessions are over weekends
or in the evenings?
Does it cost less to run a workshop in the community, or to
bring people to our training centre to run the workshop?

Step 5:

Decide how you will collect the information you need and where it
will be kept (on computer, in manual files).

Step 6:

Decide how often you will analyze the information this means
putting it together and trying to answer the questions you think are
important.

Develop monitoring plan - 2

Step 7:

Or simply, distribute worksheet 6-2: Develop monitoring plan 2,

you can look at the table information needed and design monitoring
plan.

Planning for Evaluation

Step 8:
The facilitator explains to the participants:
Similar to monitoring plan, we should start evaluation plan right from the
beginning. When we do needs assessment, we will collect data about economic,
health, education and social situation of the community. They will become
baseline data for us and to measure the impact or achievement of objectives, we
can compare these data with those that we collect after or during the
implementation of the project.
Evaluation involves:

Looking at what the project or organization intended to achieve what

difference did it want to make? What impact did it want to make?
Assessing its progress towards what it wanted to achieve, its impact
targets.
Looking at the strategy of the project or organization. Did it have a
strategy? Was it effective in following its strategy? Did the strategy
work? If not, why not?
29

Looking at how it worked. Was there an efficient use of resources? What

were the opportunity costs of the way it chose to work? How sustainable
is the way in which the project or organization works? What are the
implications for the various stakeholders in the way the organization
works.

In an evaluation, we also look at efficiency, effectiveness and impact.

Step 9:

Ask the participants to share with the class the evaluation types they
know. ( 5 10 minutes) Then distribute the names of different types of
evaluation and definitions to match each other. Distribute Worksheet
6 3: different evaluation types to the participants. The facilitator can
go through the table together with the participants and discuss about
different types of evaluation.

Step 10:

Distribute worksheet 6 4: Case study and ask the groups to discuss

among themselves first. ( 15 20 minutes)

Step 11:

Ask the groups to fill the form included in worksheet 6 5: Evaluation

Basics. ( 15 20 minutes)

Step 12:

Explain to the participants that normally, the first step in evaluation is

to develop Terms of Reference for evaluation. Share worksheet 6 6:
format of ToR included and go through the format together
explaining the points they want to clarify more.
Group work: Each group needs to develop Terms of Reference for
evaluation of Youths Energys projects. (45 to 60 minutes)

30

WORKSHEET 6 1
DEVELOP MONITORING PLAN 1
Efficiency: Whether the resources used for the activities are appropriate in terms
of output or they are cost-effective.
For example, training program that train 50 peer educators costs $10000. Is it
efficient or cost-effective?
e.g.
i. How much does it cost to organize a training workshop?
ii. How many participants join?
iii. How much does it cost per participant?
iv. Is it cost effective?
v.
Whether the resources used for the activities are appropriate in terms
of output or they are cost-effective.
All these information can be tracked easily in a tracking sheet.

No.

Date

Training
topic

No.
participant
M
F

Duration

Total training cost

Remark

Effectiveness: It is the measure of the extent of the achievement of the

development project against the target objectives.
For example, we plan to improve the qualifications of all high school teachers in
particular area, did we succeed after project completion?
We can monitor the quality of high school teachers regularly using different indicators.
Impact: It measures whether or not project implementation has contributed to
the change in the situation of the problem it was trying to address.
For example, high HIV transmission rate among young people in the community
was the main problem before and our project really made a difference to this
situation that is reduction in HIV transmission rate among young people in the
community.
We can monitor the progress to achieve targeted impact comparing baseline data with
current situation.

31

WORKSHEET 6 2
DEVELOP MONITORING PLAN 2
What do we want to know?
This includes looking at indicators for both internal issues and external issues.
There are three basic types of monitoring commonly used:
Management/administration
staff/personnel
vehicles
supplies
Finance
project budget and expenditure
staff salaries
cash flow analysis
Project activities
project inputs budget, equipment, key staff needed
results of activities project outputs/outcomes/impact (using indicators)
the way the project is managed
situation or context policy environment, political situation
The table below shows the information needed to collect for monitoring programs and
project.
INFORMATION FOR MONITORING PROGRAM OPERATIONS 3
CATEGORIES
OF
INFORMATION

WHAT TO
MONITOR

WHAT
RECORDS
TO KEEP

WHO
COLLECTS
DATA

WHO USES
DATA

HOW TO USE
INFORMATIO
N

WHAT
DECISIONS CAN
BE MADE

1.

Timing of
activities
Availability
of
personnel,
resources

Monthly
/quarterly
work plans
Work
schedules

Project
Manager
Supervisors

Project
Manager
Donor
agency

Ensure staff
and
other
resources are
available

Reschedule
activities and
deployment of
resources as
needed

Budgeted
amounts,

Ledger of
expenditure

Financial
officer/

Project
Manager

Ensure
funds are

Authorize
expenditures

Work
plan
Activities

2.Costs
and

Source: The Family Planning Managers Handbook. Editors: James A. Wolff, Linda J. Suttenfield,
Susanna C. Binzen. Management Sciences for Health, Boston.
3

32

Expenditure

funds on
hand and
expenditures
Balance in
budget
by approved
cost
categories

s by
budget
category
Receipts
Bank
transactions
Reports to
donor

accountant

Auditor
Donor
agency

3.Staff
and
Supervision

Knowledge,
attitudes and
skills of staff
Educational
level of staff
Salaries and
benefits
Job
performance

Supervisors
Personnel
director
Trainers

Superviso
r
Project
Manager

Personnel
Director

2.

Commo
dities

Stock
Ordering
and
shipment
status

Procurement
regulations

Performanc
e
reviews
Job
descriptions
Resumes
of staff
Feedback
from
training
attended
Stock
registers
Invoices
Field
worker
reports

Logistics
manager

Project
Manager
Donor
agency

3.

Results

No. and
type of
services
provided/
commodities
dispensed

Characteristic
s of
persons
served/
educated

Client
cards/forms
Clinic
registers
Field
worker
reports

CBD
Workers
Clinic
nurse
Field
supervisors

Field
supervisor
Project
Manager
Donor
agency

available to
execute
activities
Ensure
compliance
w/funding
regulations
If fee for
service,
determine fee
structure
Motivate
staff and
resolve
employment
problems
Advise staff
on
career

Make budget
and
project revisions
Determine need
for
other funding
sources

Ensure
availability of
commodities
in stock
and
distribution to
field
Ensure
good
condition
Commodities
Ensure
objectives are
realistic
Assess
quality of
services
provided
Assess
appropriatene
ss of services

Quantity to
order
When to order
Amount to
keep in
reserve for
emergency

Placement
Training needs
Promotions
Disciplinary
action

Revise
objectives
Retrain staff
Revise IEC
strategy
Revise project
strategy and
approach

33

WORKSHEET 6 3
DIFFERENT TYPES OF EVALUATION
This involves an organization or project holding up a
mirror to itself and assessing how it is doing, as a way of
learning and improving practice. It takes a very selfreflective and honest organization to do this effectively,
but it can be an important learning experience.
Participatory
This is a form of internal evaluation. The intention is to
evaluation:
involve as many people with a direct stake in the work as
possible. This may mean project staff and beneficiaries
working together on the evaluation. If an outsider is called
in, it is to act as a facilitator of the process, not an
evaluator.
Rapid Participatory Originally used in rural areas, the same methodology can,
Appraisal:
in fact, be applied in most communities. This is a
qualitative way of doing evaluations. It is semi-structured
and carried out by an interdisciplinary team over a short
time. It is used as a starting point for understanding a
local situation and is a quick, cheap, useful way to gather
information. It involves the use of secondary data review,
direct observation, semi-structured interviews, key
informants, group interviews, games, diagrams, maps and
Self-evaluation:

34

External
evaluation:
Interactive
evaluation:

calendars. In an evaluation context, it allows one to get

valuable input from those who are supposed to be
benefiting from the project. It is flexible and interactive.
This is an evaluation done by a carefully chosen outsider
or outsider team.
This involves a very active interaction between an outside
evaluator or evaluation team and the organization or
project being evaluated. Sometimes an insider may be
included in the evaluation team.

35

WORKSHEET 6 - 4
CASE STUDY

Introduction to Youths Energy4

The CBO Youths Energy (Lu Nget Swan In) is an organization formed with
young volunteers that has operated in Mawlamyaing, Mon State since 2000. They
started the organization with five young volunteers who were newly graduates
at that time. They started giving free tuition to the tenth standard students in
their ward. Most of the volunteers are the 10th standard passed students waiting
to join universities. They receive all of their funding from donor grants, and since
their budgets reached a peak in 2007, have begun to see a gradual decline in
donor aid. They have a small staff of 6 full-time employees, and 2 part-time
employees who work 15-20 hours per week. They have more than 20 volunteers
from different areas of Mawlamyaing.
Youths Energy provides recreational and educational activities to youth from
the ages of approx. 10 18 years old. They make an effort to attract and welcome
youth from all backgrounds and experiences. They currently have 4 main
projects:

Annual organization of a summer youth camp (sessions held over a

period of 8 weeks each summer)
Management of a Youth Center in the center of town, which includes an
old gymnasium, 3 classroom-style all-purpose rooms, and a small room
dedicated to computer use training, with 9 donated computers.
Sponsorship of an adolescent reproductive health educational course
module and peer education certification program that they coordinate and
teach in cooperation with 4 local schools
Organization of after-school sports and recreation activities (held at the
youth center)

2009 has been a difficult year for the NGO. They have recently learned that their
primary donor is cutting funding by 50% for 2010. While they were devastated to
receive this news, they also recognized that they were fortunate that the donor
did not withdraw completely, as has happened to other NGOs in the region.
While they have begun to investigate various fundraising options, they realize
that they are going to have to make some decisions about their program
priorities and overall organization.
Youths Energy director,Ko Thiha , has been speaking to the deputy director Ma
Nwe about his concerns, and they decided to hold a meeting one evening after
4

Adapted from IRC Bosnia Evaluation training manual

36

work with all of the staff to clearly and openly explain their current and
projected future financial status. They also wanted to use that time to let the
other employees express their concerns, as they recognized that everyone was
feeling very vulnerable and concerned about the future of the organization.
The meeting was difficult, but worthwhile. Everyone knew that times were
tough, but it was good to pull the whole team together to discuss it. A general
consensus emerged that in order to survive, the organization would have to do
two things. First, it would have to more actively work on alternative fundraising
efforts in order to seek contributions to replace the lost donor funding. Second,
they realized that they needed to look at their current projects, and determine
which projects should be continued in the future, and which should be radically
refocused or possibly even dropped entirely.
Everyone knew that one project in particular needed attention: the Youth Center.
The Youth Center had been a focal point of the organization for over 3 years, and
has been a showcase of the Youths Energy initiative. Its prime location in the
center of town made it accessible to many people, and also served as a great way
to increase the organizations visibility and reputation as a CBO that was active
and involved.
However, over the past 12 months several of the circumstances surrounding the
center had changed, making it a drain on the CBOs time and financial resources.
For the first 2 years they were able to use the facility virtually free of charge, as
the township authorities provided the space rent-free. Youths Energy only had
to pay for the utilities and general maintenance and upkeep. However, one year
ago the authorities asked the organization to begin paying a nominal rent for the
facility, with a rent payment schedule that was tapered over a 4-year period so
that it would gradually be rented out at standard market rates. The rent for the
first year was a shock to their budget, but was reasonable and management.
However, the increased rent for the next year will be a serious strain.
Additionally, the facility is in need of some repairs that can not be put off much
longer. Some are simple, such as fresh paint and a good cleaning, and could
probably be done with help from volunteers in the community. However, others
will necessitate professional expert support, such as a re-wiring of the electrical
system, repair of a leaky roof, and purchase of new desks, chairs and tables that
are more suitable to the needs of people using computers and other specialized
equipment.
Everyone recognized that, while long a favorite program of the staff, the Youth
Center was beginning to drain resources form other initiatives, in particular, the
adolescent reproductive health (ARH) training module, and related peer
education certification program. Over the past few years it was easy to
demonstrate action to donors through the Youth Center projects: they could
37

easily report on how many activities were held, how many kids participated, etc.
It was tangible activity. Since the ARH training was not the same kind of tangible
concrete activity, and was a lower profile project, it was often easily lost in the
shuffle during budget development.
When confronted with the new budget and resource situation, a few of the team
members reflexively suggested that the Youth Center be maintained and the
ARH module dropped, in order to shift and re-focus resources and energy.
However, when Ma Nwe asked them to explain why they thought this was a
good idea, no one in the room could provide an explanation based on facts or
specific data. They suddenly realized that they did not know what their users or
people in the community thought about either of these efforts, and that they
didnt know which project was more successful or had had a greater impact on
the community. These were the questions that needed to be asked and answered
before decision on the future direction of the organization could be made.
Ko Thiha pointed out that it was time to begin an evaluation of the NGOs
projects, starting with the Youth Center and the ARH module. The organization
has done basic evaluation work in the past, but always with guidance or staff
support from the donor organization requesting the evaluation. They provided
annual reports, but these were more of a summary of activities and events rather
than a rigorous evaluation of these projects. Outside consultants had come in
twice over the past 6 years to meet with the staff and assess their work, but while
the organization had welcomed the consultants and provided them with
information, they had not been closely involved in the evaluation process, and
had been limited to simply providing information and answering questions.
They realized that they needed to learn the basic of evaluation themselves, in
order to conduct the project evaluations themselves. They began to gather
evaluation training materials from some NGO resource centers, and began a dual
effort: training of the staff in basic evaluation techniques, and applying these
skills to the evaluation of the Youth Center and ARH projects.

38

WORKSHEET 6 5
EVALUATION BASICS
1. Read the case study carefully and break it down into its component parts. Fill
in the sample table, adding more space if needed.
Program

Project

Activity

2. What are some of the factors motivating Youths Energy to undertake an

evaluation?

3. Think about a case in which the organization had to make some difficult
decisions. Did you use data gathered from evaluations to make that decision?
Why or why not?

4. Is there an internal process within the organization that you think could
benefit from evaluation? Why?

39

WORKSHEET 6 6
TERMS OF REFERENCE

Terms of Reference usually include:

Background: This is background to the project or organization, something about
the problem identified, what you do, how long you have existed, why you have
decided to do an evaluation.
Purpose: Here you would say what it is the organization or project wants the
evaluation to achieve.
The purpose of an evaluation is the reason why you are doing it. It goes beyond
what you want to know to why you want to know it. It is usually a sentence or,
at most, a paragraph. It has two parts:

What you want evaluated;

To what end you want it done.

Examples of an evaluation purpose:

To provide the organization with information needed to make decisions about the future
of the project.
To assess whether the organization/project is having the planned impact in order to
decide whether or not to replicate the model elsewhere.

40

To assess the programme in terms of effectiveness, impact on the target group, efficiency
and sustainability in order to improve its functioning.
The purpose gives some focus to the broad evaluation process.
Key evaluation questions: What the central questions are that the evaluation
must address.
The key evaluation questions are the central questions you want the evaluation
process to answer. They are not simple questions. You can seldom answer yes
or no them. A useful evaluation question is:

Thought provoking
Challenges assumptions.
Focuses inquiry and reflection.
Raises many additional questions.

Some examples of key evaluation questions related to a project purpose:

The purpose of the evaluation is to assess how efficient the project is in delivering
benefits to the identified community in order to inform Board decisions about continuity
and replicability.
Key evaluation questions:
Who is currently benefiting from the project and in what ways?
Do the inputs (in money and time) justify the outputs and, if so/if not, on what
basis is this claim justified?
What would improve the efficiency, effectiveness and impact of the current
project?
What are the lessons that can be learned from this project in terms of
replicability?
Specific objectives: What specific areas, internal and/or external, you want the
evaluation to address. So, for example, you might want the evaluation to include
a review of finances, or to include certain specific programme sites.
Methodology: should provide a broad framework for how the project or
organization wants the work of the evaluation done. You might also suggest the
kinds of techniques you would like the evaluation team to use. Most terms of
reference will ask for some combination of these but they may also specify how
they want the evaluation team to get input from stakeholder groups.
Here too one would expect to find some indication of reporting formats: Will all
reporting be written? Will the team report to management, or to all staff, or to
staff and Board and beneficiaries? Will there be interim reports or only a final
41

report? What sort of evidence does the organization or project require to back up
evaluator opinions? Who will be involved in analysis?
Logistical issues: These would include timing, costing, requirements of team
composition and so on.

42

SESSION VII: GATHERING INFORMATION

FOR MONITORING

AND EVALUATION

At the end of this session, the participants will understand how different
information gathering methods can be used to get the information they want to
get for M&E purposes
Time:
Material:

About 1:15 minutes

flipcharts, markers, Worksheets

Step 1: The facilitator can start the session by explaining to the participants as
follows:
Data collection is possibly the most important part of monitoring and evaluation,
as you must have reliable and concrete information available to you in order to
assess needs, measure progress, or determine cause and effect. Detailed data
collection and information-gathering techniques are included in Social research
tools training manual. So the ones who have joined social research skills
training will be familiar with many of these techniques.
Step 2: Distribute worksheet 7 1: Different social research tools and discuss
together. (The facilitator needs to read social research tools training
manual in advance if they are not so familiar with social research skills.)
Step 3: Ask the participants if anyone knows the words, primary and
secondary sources. Then explain to all the participants that
43

Primary sources are first-hand, original sources that provide raw, un-analyzed or
un-interpreted facts and information.
Examples of primary sources include:
Training evaluation forms, Beneficiary satisfaction forms, Organizational assessment
forms
Secondary sources are exactly what they sound like- data from sources that refer
to or interpret information gathered by another researcher.
Examples of secondary sources include:
A newspaper article on the problem of AIDS in Burma/Myanmar
EIU report on Burma/Myanmar
Step 4: Ask each group to pick up one topic OR the facilitator can assign one
topic to each group and ask them how they will gather data on the
following topics identifying primary and secondary sources. (30 minutes
discussion and 5 minutes for each presentation)

How the role of women in the community has changed in the past five
years
The process through which drug addicts successfully kick the habit
The most effective media to give health education among young people
The role of monasteries in providing education services to young people
during Burmese kingdoms
Reading habit among young people in urban areas

Step 5: After presentation, the facilitator can end the session by the below
statement.
They will probably find that they want or need to use more that one data collection
technique. This is a standard practice, and is referred to as adopting a mixed method
approach. A mixed method approach combines data gathered through different
techniques, bringing all of the information together for the final review and analysis. The
data is then analyzed and interpreted collectively.
Bringing information together from a variety of sources is a good way to confirm or
validate your data. This process is called data triangulation, and it can help to ensure
the validity of the conclusions. If data gathered through interview process yields similar
feedback as data gathered through a questionnaire distribution or through basic research
and document review, then they can have more confidence in the results. If the data
gathered through one method is very different than data gathered through another, then
they should analyze your results with caution, possibly even obtaining additional data to
try to determine the reasons for the discrepancy.
44

WORKSHEET 7 1
DIFFERENT SOCIAL RESEARCH TOOLS
Tool
Interviews

Key informant
interviews

Description

These can be structured,

semi-structured or
unstructured. They
involve asking specific
questions aimed at
getting information that
will enable indicators to
be measured. Questions
can be open-ended or
closed (yes/no answers).
Can be a source of
qualitative and
quantitative information.
These are interviews that
are carried out with
specialists in a topic or
someone who may be
able to shed a particular
light on the process.

Questionnaires

These are written

questions that are used to
get written responses
which, when analyzed,
will enable indicators to
be measured.

Focus groups

In a focus group, a group

of about six to 12 people
are interviewed together
by a skilled
interviewer/facilitator
with a carefully

Usefulness

Disadvantages

As these key informants

often have little to do
with the project or
organization, they can be
quite objective and offer
useful insights. They can
provide something of the
big picture where
people more involved
may focus at the micro
(small) level.
This tool can save lots of
time if it is selfcompleting, enabling you
to get to many people.
Done in this way it gives
people a feeling of
anonymity and they may
say things they would not
say to an interviewer.

Needs a skilled
interviewer with a good
understanding of the
topic. Be careful not to
turn something into an
absolute truth (cannot be
challenged) because it has
been said by a key
informant.

Can be used with almost

anyone who has some
involvement with the
project. Can be done in
person or on the
telephone or even by email. Very flexible.

This can be a useful way

of getting opinions from
quite a large sample of
people.

Requires some skill in the

interviewer.

With people who do not

read and write, someone
has to go through the
questionnaire with them
which means no time is
saved and the numbers
one can reach are limited.
With questionnaires, it is
not possible to explore
what people are saying
any further.
Questionnaires are also
over-used and people get
tired of completing them.
Questionnaires must be
piloted to ensure that
questions can be
understood and cannot be
misunderstood. If the
questionnaire is complex
and will need
computerized analysis,
you need expert help in
designing it.
It is quite difficult to do
random sampling for
focus groups and this
means findings may not
be generalised.
Sometimes people
45

structured interview
schedule. Questions are
usually focused around a
specific topic or issue.

Community meetings

This involves a gathering

of a fairly large group of
beneficiaries to whom
questions, problems,
situations are put for
input to help in
measuring indicators.

Visual/audio stimuli

These include pictures,

movies, tapes, stories, role
plays, photographs, used
to illustrate problems or
issues or past events or
even future events.
This technique makes use
of a continuum, along
which people are
expected to place their
own feelings,
observations etc. People
are usually asked to say
whether they agree
strongly, agree, dont
know, disagree, disagree
strongly with a statement.
You can use pictures and
symbols in this technique
if people cannot read and
write.
This method is a way of
focusing interviews with
individuals or groups on
particular events or
incidents. The purpose of
doing this is to get a very
full picture of what
actually happened.

Rating scales

Critical event/incident
Analysis

Participant observation

This involves direct

observation of events,
processes, relationships
and behaviors.
Participant here implies

Community meetings are

useful for getting a broad
response from many
people on specific issues.
It is also a way of
involving beneficiaries
directly in an evaluation
process, giving them a
sense of ownership of the
process. They are useful
to have at critical points
in community projects.
Very useful to use
together with other tools,
particularly with people
who cannot read or write.

influence one another

either to say something or
to keep quiet about
something. If possible,
focus groups interviews
should be recorded and
then transcribed. This
requires special
equipment and can be
very time-consuming.
Difficult to facilitate
requires a very
experienced facilitator.
May require breaking into
small groups followed by
plenary sessions when
everyone comes together
again.

You have to have

appropriate stimuli and
the facilitator needs to be
skilled in using such
stimuli.

It is useful to measure
attitudes, opinions,
perceptions.

You need to test the

statements very carefully
to make sure that there is
no possibility of
misunderstanding. A
common problem is when
two concepts are included
in the statement and you
cannot be sure whether
an opinion is being given
on one or the other or
both.

Very useful when

something problematic
has occurred and people
feel strongly about it. If
all those involved are
included, it should help
the evaluation team to get
a picture that is
reasonably close to what
actually happened and to
be able to diagnose what
went wrong.
It can be a useful way of
confirming, or otherwise,
information provided in
other ways.

The evaluation team can

end up submerged in a
vast amount of
contradictory detail and
lots of he said/she said.
It can be difficult not to
take sides and to remain
objective.

It is difficult to observe
and participate. The
process is very timeconsuming.
46

Self-drawings

that the observer gets

involved in activities
rather than maintaining a
distance.
This involves getting
participants to draw
pictures, usually of how
they feel or think about
something.

Can be very useful,

particularly with younger
children.

Can be difficult to explain

and interpret.

Primary sources are first-hand, original sources that provide raw, un-analyzed or
un-interpreted facts and information.
Examples of primary sources include:
Training evaluation forms, Beneficiary satisfaction forms, Organizational assessment
forms
Secondary sources are exactly what they sound like- data from sources that refer
to or interpret information gathered by another researcher.
Examples of secondary sources include:
A newspaper article on the problem of AIDS in Burma/Myanmar
EIU report on Burma/Myanmar
Mixed method approach
They will probably find that they want or need to use more that one data collection
technique. This is a standard practice, and is referred to as adopting a mixed method
approach. A mixed method approach combines data gathered through different
techniques, bringing all of the information together for the final review and analysis. The
data is then analyzed and interpreted collectively.
Bringing information together from a variety of sources is a good way to confirm or
validate your data. This process is called data triangulation, and it can help to ensure
the validity of the conclusions. If data gathered through interview process yields similar
feedback as data gathered through a questionnaire distribution or through basic research
and document review, then they can have more confidence in the results. If the data
gathered through one method is very different than data gathered through another, then
they should analyze your results with caution, possibly even obtaining additional data to
try to determine the reasons for the discrepancy.

47

SESSION VIII: DATA ANALYSIS

At the end of this session, the participants will understand some basic methods
for data analysis, breaking analysis down into qualitative and quantitative data
analysis methods
Time:

About 1:30 hours

Material:

flipcharts, markers, Worksheets

Step 1:

Ask the participants to go around the class and ask them to

interview at least 10 friends the following questions. (15 20
minutes)
1) How old are you? (dont need to ask exact age)
(Under 20), (within 20 30 years old), (30 40 years old), (40 years
above)
2) Do you like reading? If yes, continue the following questions. If
no, skip the questions 4 and 5.
3) How many hours a week do you spend time for reading?
4) What kind of books do you like to read?
5) What do you do if you have free time?
6) How many hours a day do you spend to do what you like?

Step 2:

Ask the participants to review and analyze the responses like the
examples.
1) Age range
48

Among (10) respondents, most of them are within the age of 20

to 30. ( ) are above 30 and ( ) is under 20.
OR
Among ( ) respondents, ( ) are within the age of 20 30 years
old and ( ) are in mid-30. ( ) are above 40.
2) Among ( ) people interviewed, ( ) like reading. They spend
( ) to ( ) hours per week for reading. Out of ( ), ( ) like
reading newspapers and journals and ( ) read biographies and
( ) like to read variety of books. Most of the people who like
reading are within the age of
years old.
3) Among ( ) respondents, ( ) dont like reading. Among them, (
) like listening music and spend ( ) hours a week for listening
music. ( ) like watching movie and ( ) like -----------------------.
Step 3:

Explain to the participants that above review can identify some

trends among targeted population.

Whether working with qualitative or quantitative data, the most basic part of
data analysis will consist in identifying trends and patterns. Consider the
following examples of potential trends that they might identify through its data
analysis:
A questionnaire revealed that people within the age of 30 to 40 liked reading and they
liked to read magazines and journals most. Respondents with the age of 20 to 30 preferred
music or shopping to reading.
A focus group indicated that while young people liked the new brand of candy, older
people thought it was too sweet.
The study showed that there was no difference in music tastes among children from poor
and wealthy families.
They may be able to clearly and immediately identify some trends, though others
may be more difficult to identify, and may necessitate detailed review and
reflection.
Step 4:

The facilitator tells the participants if they have some basic

understanding of trends and response frequencies, they can begin
to interpret and analyze this information.
What does the trend mean?
What does the data suggest?

e.g.
The data indicates that 80% of the children who completed the World Religions course
module had a favorable opinion of people from different ethnic groups, while only 23% of

49

the child respondents who did not complete the course noted a similar favorable opinion.
As the children were all from the same school, the results suggest that the course
increases childrens respect for diversity.
91% of the respondents noted that they would not feel personally affected by the
elimination of ARH program, suggesting that the program could be cut without any
serious repercussions in the community.
When asked about their satisfaction with the training program of IRC ICB team, 14% of
respondents rated interpreters translation skills as Excellent, 56% as Satisfactory,
and 25% as Poor. (5% responded No Opinion.) This suggests that ICB team might
need to consider providing training on translation skills of interpreters.
However, tell them to be careful to consider other explanations for cause and
effect. Later, these trends will be turned into a summary of general findings and
recommendations.

50

WORKSHEET 8 1
DATA ANALYSIS
1) Go around the class and ask the following questions to your friends.
i. How old are you?
(Under 20), (within 20 30 years old), (30 40 years old), (40 years
above)
ii. Do you like reading? If yes, continue the following questions. If no,
skip the questions 4 and 5.
iii. How many hours a week do you spend time for reading?
iv. What kind of books do you like to read?
v.
What do you do if you have free time?
vi. How many hours a day do you spend to do what you like?
2) Review and analyze the responses like the following examples.
1) Age range
Among (10) respondents, most of them are within the age of 20 to 30. ( )
are above 30 and ( ) is under 20.
OR
Among ( ) respondents, ( ) are within the age of 20 30 years old and
( ) are in mid-30. ( ) are above 40.
2) Among ( ) people interviewed, ( ) like reading. They spend
( ) to ( ) hours per week for reading. Out of ( ), ( ) like reading
newspapers and journals and ( ) read biographies and ( ) like to read
variety of books. Most of the people who like reading are within the age of
years old.
3) Among ( ) respondents, ( ) dont like reading. Among them, (
listening music and spend ( ) hours a week for listening music. (
watching movie and ( ) like -----------------------.

) like
) like

1) Whether working with qualitative or quantitative data, the most basic part of
data analysis will consist in identifying trends and patterns. Consider the
following examples of potential trends that you might identify through its
data analysis:
A questionnaire revealed that people within the age of 30 to 40 liked reading and they
liked to read magazines and journals most. Respondents with the age of 20 to 30 preferred
music or shopping to reading.
A focus group indicated that while young people liked the new brand of candy, older
people thought it was too sweet.
51

The study showed that there was no difference in music tastes among children from poor
and wealthy families.
You may be able to clearly and immediately identify some trends, though others
may be more difficult to identify, and you may necessitate detailed review and
reflection.
2) They can begin to interpret and analyze this information.
What does the trend mean?
What does the data suggest?
e.g.
The data indicates that 80% of the children who completed the World Religions course
module had a favorable opinion of people from different ethnic groups, while only 23% of
the child respondents who did not complete the course noted a similar favorable opinion.
As the children were all from the same school, the results suggest that the course
increases childrens respect for diversity.
91% of the respondents noted that they would not feel personally affected by the
elimination of ARH program, suggesting that the program could be cut without any
serious repercussions in the community.
When asked about their satisfaction with the training program of IRC ICB team, 14% of
respondents rated interpreters translation skills as Excellent, 56% as Satisfactory,
and 25% as Poor. (5% responded No Opinion.) This suggests that ICB team might
need to consider providing training on translation skills of interpreters.
However, be careful to consider other explanations for cause and effect. Later,
these trends will be turned into a summary of general findings and
recommendations.

52

SESSION IX: REPORTING

At the end of this session, the participants will understand how to present all the
information and findings in the report.
Time:

About 1 hour

Material:

flipcharts, markers, Worksheets

53

Step 1:
Distribute Evaluation report format to the participants. Like
other reporting, evaluation report needs to do SPR analysis first.

Step 2:

Subject (What do you want to focus?)

Purpose (What is the purpose?)
Reader (Who will read?)

The facilitator can explain to the participants that in general, your

evaluation should include the following elements.
1. Executive Summary: Provide a short (2-3 paragraph) summary
of the main issues evaluated, the main findings of the
evaluation, and recommendations
2. Introduction: Explain the issue being evaluated and the reason
for and goals of the evaluation (needs assessment, impact
analysis, process evaluation, or a combination of the 3)
3. Methodology: Explain how you developed your questions,
your data sources (respondents), and how you gathered your
data
4. Findings: Explain the results of your data analysis
5. Recommendations: Explain your recommendations based on
the results of the evaluation
6. Appendices:
You would include Terms of Reference, list of
people interviewed, questionnaires used, possibly a map of the area
and so on.)

The facilitator can conclude the training by sharing the following

information.

If the report is for public consumption, they may want to (or be required to)
prepare a short presentation for the donors in which they orally summarize the
main results and recommendations, possibly using a PowerPoint slide
presentation to draw attention to key results.

54

WORKSHEET 9 1
EVALUATION REPORT FORMAT
SPR analysis

Subject (What do you want to focus?)

Purpose (What is the purpose?)
Reader (Who will read?)
EVALUATION REPORT FORMAT

1. Executive Summary: Provide a short (2-3 paragraph) summary of the main

issues evaluated, the main findings of the evaluation, and recommendations
2. Introduction: Explain the issue being evaluated and the reason for and goals
of the evaluation (needs assessment, impact analysis, process evaluation, or a
combination of the 3)
3. Methodology: Explain how you developed your questions, your data sources
(respondents), and how you gathered your data
4. Findings: Explain the results of your data analysis
5. Recommendations: Explain your recommendations based on the results of
the evaluation
6. Appendices:

You would include Terms of Reference, list of people interviewed,

questionnaires used, possibly a map of the area and so on.)

If the report is for public consumption, they may want to (or be required to)
prepare a short presentation for the donors in which they orally summarize the
main results and recommendations, possibly using a PowerPoint slide
presentation to draw attention to key results.

55

MONITORING AND EVALUATION TRAINING

POST-TEST

Please answer the following questions to know your existing knowledge and
skills.
1)What is Monitoring?

2) Why do we do Monitoring?

3) Who does Monitoring?

4) When do we do Monitoring?

5) What are the important things to look at when you monitor a project?

6) What are indicators? Please give an example of indicator.

7) What is evaluation?

8) For what purposes do you normally do evaluation?

9) When do you normally evaluate?

10) Write down the monitoring and evaluation tools you know.
56

REFERENCES
1. Perry, Valery,Sarajevo, Bosnia and Herzegovina, Did it Work? The Basics of
Evaluation and Assessment for NGOs, Sponsored by: International Rescue
Committee (IRC): NGO Development Program
2. CIVICUS toolkits: Monitoring and Evaluation
http://www.civicus.org/toolkits/managing-csos-toolkits
3. Gosling, Louisa with Mike Edwards,Toolkits: A practical guide to planning,
monitoring, evalution and impact assessment, Save the Children UK 2003
Pulbications

57