MONITORING AND EVALUATION TRAINING MANUAL FOR

CBOs AND NGOs

Overall Goal:

To give concepts, skills and knowledge on monitoring and evaluation for

development

Developed by:
Institutional Capacity Building Team
SHIELD Project
International Rescue Committee - Thailand

1
 Table of Contents

TIME-TABLE ................................................................................................................ 3
monitoring and evaluation Training, Pre Test ........................................................... 4
Session I: Orientation ............................................................................................... 5
Session II: Warm-up .............................................................................................. 7
WORKSHEET 2 – 1, Key steps of project management ...................................... 10
WORKSHEET 2 – 2, CAUSAL HYPOTHESIS ..................................................... 11
Session- III: what is monitoring? .............................................................................. 12
WORKSHEET 3 – 1, MONITORING .................................................................... 13
Session- IV: what is evaluation? ............................................................................... 14
WORKSHEET 4 – 1, EVALUATION .................................................................... 16
Session V: Developing indicators ....................................................................... 18
WORKSHEET 5 – 1, INDICATORS ...................................................................... 22
WORKSHEET 5 – 2, DEVELOPING OUTPUT/EFFECT/IMPACT IMPACT
INDICATORS ……………………………………………………………………….23
WOKSHEET-5-3DEVELOPING EFFICIENCY, EFFECTIVENESS AND
IMPACT INDICATORS INDICATORS ..................................................... …… 24
Session VI: Planning for monitoring and evaluation ......................................... 26
WORKSHEET 6 – 1 ,Develop monitoring plan – 1 .............................................. 31
WORKSHEET 6 – 2, DEVELOP MONITORING PLAN – 2................................ 32
WORKSHEET 6 – 3, DIFFERENT TYPES OF EVALUATION ........................... 34
WORKSHEET 6 – 4, Case Study............................................................................ 36
WORKSHEET 6 – 5, EVALUATION BASICS ...................................................... 39
WORKSHEET 6 – 6, TERMS OF REFERENCE .................................................... 40
Session VII: Gathering information for monitoring and evaluation .................... 43
WORKSHEET 7 – 1, Different social research tools............................................. 45
Session VIII: Data analysis ........................................................................................ 48
WORKSHEET 8 – 1, DATA ANALYSIS ............................................................... 51
.................................................................................................................................. 51
Session IX: Reporting ................................................................................................ 53
WORKSHEET 9 – 1, EVALUATION REPORT FORMAT................................... 55
Monitoring and evaluation Training, Post-Test....................................................... 56
References ................................................................................................................... 57

2
TIME-TABLE

DAY 9:00 – 10:30 10:45 TO 12:00 AM 1:00 TO 3:00 PM 3:15 TO 4:30 PM

DAY – 1 • REGISTRATION (15 • WARM UP (30 – 45 • WHAT IS EVALUATION? ( 30 • DEVELOPING
MIN) MIN) MIN) INDICATORS
• PRE-TEST (15 MIN) • WHAT IS • DEVELOPING INDICATORS

COFFEE BREAK (15 MINUTES)

COFFEE BREAK (15 MINUTES)

LUNCH BREAK (ONE HOUR)
• ORIENTATION ( 1 MONITORING? ( 30
HOUR) MIN)

DAY – 2 • PLANNING FOR • PLANNING FOR • PLANNING FOR • PLANNING FOR

MONITORING MONITORING AND MONITORING AND MONITORING AND
AND EVALUATION EVALUATION EVALUATION EVALUATION

DAY – 3 • DATA • BASIC DATA • REPORTING • WRAP UP SESSION

COLLECTION ANALYSIS • POST TEST
METHODS METHODS • TRAINING EVALUATION

3
 MONITORING AND EVALUATION TRAINING
PRE TEST
Please answer the following questions to know your existing knowledge and
skills.

1) What is Monitoring?

2) Why do we do Monitoring?

3) Who does Monitoring?

4) When do we do Monitoring?

5) What are the important things to look at when you monitor a project?

6) What are indicators? Please give an example of indicator.

7) What is evaluation?

8) For what purposes do you normally do evaluation?

9) When do you normally evaluate?

10) Write down the monitoring and evaluation tools you know.

4
 SESSION I: ORIENTATION

The Facilitator/s and participants will get an opportunity to introduce

themselves, to exchange information about past experiences and to share
workshop expectations

Time: 1 hour

Materials: A4 size paper (cut into four pieces), flipcharts, markers

Introduction

Step 1: Ask the participants to divide into two groups equally: Group A
and Group B

Step 2: Then cut A4 size paper into four pieces and distribute one piece
to each participant. Ask them to write down
• their name
• Mother organization
• M&E related experience they have before
Step 3: When everyone finishes writing, ask each group to stand in line
facing other group.

Step 4: Ask people from Group A to give their card to the one who is in
front of him/her in Group B introducing themselves. Then the
one from Group B also introduces about himself/herself
exchanging their cards.
Step 5: When all have finished introducing each other, ask each pair to
introduce their friends to the whole class.

5
Forming groups

Step 6: Ask all the participants to stand up in line again based on the
birth months starting from January. Based on the number of
participants, ask them to count 1 to 5 (if you want to form 5
groups). Then ask all people with 1, 2, 3, 4, 5 number can sit
together. Ask them to give a name for their group. Tell them
that they have to be together for group work for three days.

Expectations

Step 7: Tell the participants that this workshop will focus on

“Monitoring and Evaluation” of a project included in project
cycle management. Ask them to write down their expectations
and concerns related to this workshop on a flipchart. (5 – 10
minutes)
Step 8: When all the groups finish, ask them to read other groups’
expectations and concerns and if they find the same ones, they
can write down group number in the bracket.

Step 9: Then share the workshop objectives and if what they expect
cannot be included in the workshop objectives, discuss with
them. Related to the concerns, if the facilitators can deal with
them, please tell them how and if the participants need to deal
with, tell them to set ground rules and how they want to take
action if somebody breaks the rule/s. e.g. phone calls,
punctuality, dominate the class by one or two participants.

6
 SESSION II: WARM-UP

This session revisits key steps of project management and terminology used in
M&E to refresh the memory.

Time: About 30 to 45 minutes

Material: flipcharts, markers, worksheets

Step 1: The facilitator starts the session by asking the key steps of project
management. Then show the key steps of project management
distributing Worksheet 2.1: project management.

7
Key steps of project management

Assessing needs

Planning the project

Learning from the
project and
evolving

Implementing and
Evaluating the monitoring the
project project

When a project is started to implement, it is necessary to monitor the project activities

while implementing the project and at the end of the project, normally, the project is
evaluated.

Step 2: The facilitator ask the participants to fill the blanks included in the
worksheet.

The causal hypothesis is a concise statement of the logic behind the project.
Normally, causal hypothesis is written as follows:

8
 This set of --------- and ---------- will result in these products and services (--------).
These will in turn lead to these changes in people’s behavior (------------), which will
contribute to the desired ---------------. (activities, IMPACT, outputs, inputs, effects)

Step 3: Then discuss with the participants about the terms used in causal
hypothesis and show example.

Below is the meaning of the terms used in causal hypothesis.

IMPACT: A change in population's health, economic or social status.

EFFECT: A change in the knowledge, attitudes, skills, intentions, or and
behaviors needed of the population that contribute to the desired IMPACT.
OUTPUTS: Products and services that must be in place before the EFFECTS can
occur.
ACTIVITIES: The technical and support tasks required to produce the
OUTPUTS.
INPUTS: Resources required to support your ACTIVITIES.
Inputs refer to the set of resources (i.e. financial, policies, personnel, facilities,
space, equipment and supplies, etc.) that are the basic materials of the
project/program.

Causal Pathway Framework

Design Direction

Inputs Activities Outputs Effect

Implementation, Monitoring and Evaluation

Inputs Activities Outputs Effects Impact

# of staff who
apply
Trainers M&E training # of participants Improved
monitoring and
Training materials Training who attended project quality
evaluation skills
Training equipments follow-up M&E training with effective
in their
Training venue # of participants monitoring
organizations
Training participants who gets M&E and evaluation
# of staff who
(staff from skills system
can monitor
CBOs/NGOs)
effectively and
evaluate the
project 9
 WORKSHEET 2 – 1

Key steps of project management

Assessing needs

Planning the project

Learning from the
project and
evolving

Implementing and
Evaluating the monitoring the
project project

When a project is started to implement, it is necessary to monitor the project activities

while implementing the project and at the end of the project, normally, the project is
evaluated.

10
 WORKSHEET 2 – 2

CAUSAL HYPOTHESIS
The causal hypothesis is a concise statement of the logic behind the project.
Normally, causal hypothesis is written as follows:

This set of --------- and ---------- will result in these products and services (--------).
These will in turn lead to these changes in people’s behavior (------------), which will
contribute to the desired ---------------. (activities, IMPACT, outputs, inputs, effects)
The terms used in causal hypothesis.
IMPACT: A change in population's health, economic or social status.
EFFECT: A change in the knowledge, attitudes, skills, intentions, or and
behaviors needed of the population that contribute to the desired IMPACT.
OUTPUTS: Products and services that must be in place before the EFFECTS can
occur.
ACTIVITIES: The technical and support tasks required to produce the
OUTPUTS.
INPUTS: Resources required to support your ACTIVITIES.
Inputs refer to the set of resources (i.e. financial, policies, personnel, facilities,
space, equipment and supplies, etc.) that are the basic materials of the
project/program.

Causal Pathway Framework

Design Direction

Inputs Activities Outputs Effect

Implementation, Monitoring and Evaluation

Inputs Activities Outputs Effects Impact

# of staff who
apply
Trainers M&E training # of participants Improved
monitoring and
Training materials Training who attended project quality
evaluation skills
Training equipments follow-up M&E training with effective
in their
Training venue # of participants monitoring
organizations
Training participants who gets M&E and evaluation
# of staff who
(staff from skills system
can monitor
CBOs/NGOs)
effectively and 11
evaluate the
project
 SESSION- III: WHAT IS MONITORING?

After this session, the participants will understand the concept of monitoring and
how to develop good indicators.

Time: 30 – 45 minutes

Material: flip chart, markers, worksheets

Step1: Eliciting (15 - 20 minutes)

Ask the following questions to the participants:
- What is Monitoring?
- Why do we do Monitoring?
- Who does monitoring?
- When do we do Monitoring?

Write down what the participants discuss on flipchart. Then distribute the
Worksheet 3 – 1 : Monitoring and discuss for a while.

12
 WORKSHEET 3 – 1

MONITORING
What is Monitoring?
Monitoring is the systematic collection of information on all aspects of the project
while it is being implemented.

It can be divided into internal monitoring (staff performance, planned expenditure

for each activity versus actual expenses, procurement procedures etc) and
external monitoring (planned versus actual activities, timely implementation of
activities, targeted beneficiaries versus true beneficiaries, unintended effects on
the community and unexpected problems etc). Both are important and both need
to be monitored.

Why Do We Do Monitoring?
We do monitoring to analyze the current situation, identify problems and find
solutions, discover trends and patterns, keep project activities on schedule,
measure progress towards objectives, formulate/revise future goals and
objectives, make decisions about human, financial, and material resources.
Actually, it is a very useful tool for management and provides necessary
information for evaluation.

In other words, monitoring means checking how things are going on and
comparing actual progress to what is planned.

Who Does Monitoring?

Monitoring is concerned both with project staff, implementation organization and
donors. As mentioned above, it is useful for management and project manager or
program coordinator conducts monitoring on all aspects of the project-
budget/finance, materials, staff, activities, outputs/results etc. Respected project
staff are also responsible for monitoring staff and tasks under them-for example,
finance manager has to monitor the accountant and cashier as well as budget
allocations. At the same time, representatives of donors also conduct monitoring
to measure the progress towards objectives and goal/impact.

When Do We Do Monitoring?
According to the desired schedule of implementation of the projects….
e.g. monthly, bi-monthly, quarterly

13
 SESSION- IV: WHAT IS EVALUATION?

After this session, the participants will understand the

concept of evaluation

Time: 30 – 45 minutes

Material: flipcharts, markers, worksheet

Step 1: In plenary, ask the following questions to the participants: (15 - 20

minutes)
• What is evaluation?
• Why Do We Do Evaluation?
• When Do We Evaluate?
• Who Does Evaluation?

The facilitator can read through the worksheet 4 – 1: Evaluation together with the
participants to understand more about evaluation.

Step 2: Show the below project activity cycle to the participants.

14
Step 4: The facilitator can end the session with below explanation.

M&E is an ongoing process in a project since the project is started.

It is important to recognize that monitoring and evaluation are not
magic wands that can be waved to make problems disappear, or to
cure them, or to miraculously make changes without a lot of hard
work being put in by the project or organization. In themselves,
they are not a solution, but they are valuable tools.

Monitoring and evaluation can:

Help you identify problems and their causes;
Suggest possible solutions to problems;
Raise questions about assumptions and strategy;
Push you to reflect on where you are going and how you are getting there;
Provide you with information and insight;
Encourage you to act on the information and insight;
Increase the likelihood that you will make a positive development difference.
15
 WORKSHEET 4 – 1

EVALUATION
What is evaluation?
Evaluation is a systematic and objective assessment of ongoing or completed
project. It makes comparison of the outcomes of the project with planned ones.

Why Do We Do Evaluation?
The primary objective of evaluation is to ascertain whether the project has
achieved its intended objectives. By drawing conclusions, evaluation intends to
provide recommendations for the improvement on the future course of the
project as well as lessons learned for other projects. Some big organizations use
specific criteria when they do evaluation.

When Do We Evaluate?
Periodically, mid-term, at the end of the project (final evaluation) and years after
the completion of the project (post-evaluation).

Who Does Evaluation?

Project manager or assigned project staff can conduct internal evaluation and
donor/s or consultant/s can conduct external evaluation.

Project activity cycle

16
M&E is an ongoing process in a project since the project is started. It is important
to recognize that monitoring and evaluation are not magic wands that can be
waved to make problems disappear, or to cure them, or to miraculously make
changes without a lot of hard work being put in by the project or organization.
In themselves, they are not a solution, but they are valuable tools.

Monitoring and evaluation can:

Help you identify problems and their causes;
Suggest possible solutions to problems;
Raise questions about assumptions and strategy;
Push you to reflect on where you are going and how you are getting there;
Provide you with information and insight;
Encourage you to act on the information and insight;
Increase the likelihood that you will make a positive development difference.

17
 SESSION V: DEVELOPING INDICATORS

At the end of this session, the participants will be able to develop indicators

Time: About 2:30 hour

Material: flipcharts, markers, worksheets

Step 1: The session can be started with the question, “What is indicator?”
After discussion for a while, distribute Worksheet 5 – 1: Indicators

“Indicators are Units of measurement that tell us about IMPACT,

EFFECT and OUTPUT to judge the effectiveness of the Project

Indicators should be:

Ethical
Useful
Scientifically Robust
Reliable
Accessible

Good indicators should be 1) practical, 2) independent, 3)

measurable and 4) targeted.

18
Give the below examples and ask them if they are good indicators or not.
Yes No
Mortality rate x

Health education materials x

# of clinics offering STD treatment x

% of clients satisfied with quality of STD treatment x

# of community health workers trained x

% of student who score at least 90% on final exam x
Sexual violence counseling x

Step 2: Ask the groups to read Worksheet 5 – 2: the causal pathway

hypothesis of Ah Lin Yaung. And develop indicators to measure
outputs, effects and impact(s). (Group work 15 – 30 minutes and 5
minutes’ presentation for each group)
e.g.
Impact indicators
 Reduced HIV transmission rate among young people in targeted
areas (compare with a baseline data)

Effect indicators
 Increased use of disposable needles among drug users (survey and
compare with a baseline data)
 Increased number of young people who understand how they can
contract HIV/AIDS

Output indicators
 # of health education sessions organized by peer educators
 # of trained peer educators in targeted community

Developing efficiency, effectiveness and impact indicators 1

Step 4: Identify the problem situation you are trying to address. The
following might be problems for Ah Lin Yaung:

• Young people don’t have HIV/AIDS health education.

• Needle sharing among young people
• People living with HIV/AIDS (PLHA) do not get access to
health care services.

1 Taken from CIVICUS Monitoring and Evaluation

19
 • HIV transmission rate among young people is high.

Step 5: Develop the vision (impact) of the project you want to achieve. This
will give you impact indicators. For Ah Lin Yaung, the end result
(vision) can be: “Healthy young people in targeted areas”.

What will tell you that the vision has been achieved? What signs
will you see that you can measure that will “prove” that the vision
has been achieved?

For example: if your vision was that young people in the

community would be healthy, then you can use health indicators to
measure how well you are doing. Do fewer young people die of
HIV/AIDS? Has the HIV/AIDS infection rate been reduced? If you
can answer “yes” to these questions then progress is being made.

Step 6: Develop a process vision for how you want things to be achieved.
This will give you process indicators.

If, for example, you want success to be achieved through

community efforts and participation, then your process vision
might include things like peer educators from the community
trained and providing health education, counseling session and
other necessary healthcare services used by all; targeted
community gets involved in all planned activities of the project,
and so on.

Step 7: Develop indicators for effectiveness.

For example, if you believe that you can reduce HIV/AIDS

transmission rate by using disposable needles and condoms among
young people, then you need indicators that show you have been
effective in promoting use of disposable needles and condoms e.g.
evidence from a survey among targeted communities compared
with a baseline survey.

Step 8: Develop indicators for your efficiency targets

Here you can set indicators such as: planned health education
sessions are run within the stated timeframe, costs for needles,
condoms and health sessions are kept to a maximum of US$ ___
per participant, no more than ____ hours in total of staff time to be
spent on organizing a health education session, no complaints
about health education session etc.

20
With this framework in place, you are in a position to monitor and evaluate
efficiency, effectiveness and impact

21
 WORKSHEET 5 – 1

INDICATORS

“Indicators are Units of measurement that tell us about IMPACT, EFFECT and
OUTPUT to judge the effectiveness of the Project

Indicators should be:

Ethical
Useful
Scientifically Robust
Reliable
Accessible

Good indicators should be 1) practical, 2) independent, 3) measurable and 4)

targeted.

Please give the below examples and ask them if they are good indicators or not.
Yes No
Mortality rate

Health education materials

# of clinics offering STD treatment

% of clients satisfied with quality of STD treatment

# of community health workers trained

% of student who score at least 90% on final exam
Sexual violence counseling

22
 WORKSHEET 5 – 2

DEVELOPING OUTPUT/EFFECT/IMPACT INDICATORS

Causal hypothesis
Ah Lin Yaung (ALY) will identify vulnerable groups among young population in
targeted areas and train peer educators to give health education among
vulnerable groups. At the same time, ALY will provide needles for drug-users
and distribute condoms among high-risked groups. ALY will open counseling
center for People Living with HIV/AIDS (PLHA) and mobile clinics to give
treatments for opportunistic infections. These activities can change knowledge
and attitudes of young people and PLHA to prevent getting HIV or spreading
HIV to other people. Accessibility of needles and condoms can also change the
practice and behaviors of young people for safer sex. These changes will then
contribute to reduce HIV transmission rate among young people in targeted
areas.

e.g.
Impact indicators
 Reduced HIV transmission rate among young people in targeted
areas (compare with a baseline data)

Effect indicators
 Increased use of disposable needles among drug users (survey and
compare with a baseline data)
 Increased number of young people who understand how they can
contract HIV/AIDS

Output indicators
 # of health education sessions organized by peer educators
 # of trained peer educators in targeted community

23
 WORKSHEET 5 – 3

DEVELOPING EFFICIENCY, EFFECTIVENESS AND IMPACT

INDICATORS 2

1) Identify the problem situation you are trying to address. The following
might be problems for Ah Lin Yaung:

• Young people don’t have HIV/AIDS health education.

• Needle sharing among young people
• People living with HIV/AIDS (PLHA) do not get access to
health care services.
• HIV transmission rate among young people is high.

2) Develop the vision (impact) of the project you want to achieve. This will
give you impact indicators. For Ah Lin Yaung, the end result (vision) can
be: “Healthy young people in targeted areas”.

What will tell you that the vision has been achieved? What signs
will you see that you can measure that will “prove” that the vision
has been achieved?

For example: if your vision was that young people in the

community would be healthy, then you can use health indicators to
measure how well you are doing. Do fewer young people die of
HIV/AIDS? Has the HIV/AIDS infection rate been reduced? If you
can answer “yes” to these questions then progress is being made.

3) Develop a process vision for how you want things to be achieved. This
will give you process indicators.

If, for example, you want success to be achieved through

community efforts and participation, then your process vision
might include things like peer educators from the community
trained and providing health education, counseling session and
other necessary healthcare services used by all; targeted
community gets involved in all planned activities of the project,
and so on.

4) Develop indicators for effectiveness.

2 Taken from CIVICUS Monitoring and Evaluation

24
 For example, if you believe that you can reduce HIV/AIDS
transmission rate by using disposable needles and condoms among
young people, then you need indicators that show you have been
effective in promoting use of disposable needles and condoms e.g.
evidence from a survey among targeted communities compared
with a baseline survey.

5) Develop indicators for your efficiency targets

Here you can set indicators such as: planned health education
sessions are run within the stated timeframe, costs for needles,
condoms and health sessions are kept to a maximum of US$ ___
per participant, no more than ____ hours in total of staff time to be
spent on organizing a health education session, no complaints
about health education session etc.

With this framework in place, you are in a position to monitor and evaluate
efficiency, effectiveness and impact

25
 SESSION VI: PLANNING FOR MONITORING AND
EVALUATION

At the end of this session, the participants will know monitoring and evaluation
process and how to plan monitoring and evaluation for a project.

Time: About 4 - 5 hours

Material: flipcharts, markers, Worksheets

Designing a monitoring system

Step 1: The facilitator can start the session as follows:

Monitoring and evaluation should be part of your planning process.

It is very difficult to go back and set up monitoring and evaluation
systems once things have begun to happen. You need to begin
gathering information about performance and in relation to targets
from the word go. The first information gathering should, in fact,
take place when you do your needs assessment. This will give you
the information you need against which to assess improvements
over time.

Indicators developed in previous session can also provide the

framework for your monitoring and evaluation system. They tell you
what you want to know and the kinds of information it will be useful
to collect.

26
Step 2: Revisit causal hypothesis, “Ah Lin Yaung”.

Ah Lin Yaung will identify vulnerable groups among young population in

targeted areas and train peer educators to give health education among
vulnerable groups. At the same time, Ah Lin Yaung will provide needles for
drug-users and distribute condoms among high-risked groups. Ah Lin Yaung
will open counseling center for PLHA and mobile clinics to give treatments for
opportunistic infections. These activities can change knowledge and attitudes of
young people and PLHA to prevent getting HIV or spreading HIV to other
people. Accessibility of needles and condoms can also change the practice and
behaviors of young people for safer sex. These changes will then contribute to
reduce HIV transmission rate among young people in targeted areas.

Develop monitoring plan - 1

Step 3: Introduce the concepts of efficiency, effectiveness and impact

Efficiency: Whether the resources used for the activities are

appropriate in terms of output or they are cost-effective.
For example, training program that train 50 peer educators costs
$10000. Is it efficient or cost-effective?
Effectiveness: It is the measure of the extent of the achievement of
the project against the target objectives.
For example, we plan to improve the qualifications of all high
school teachers in particular area, did we succeed after project
completion?
Impact: It measures whether or not project implementation has
contributed to the change in the situation of the problem it was
trying to address.
For example, high HIV transmission rate among young people in
the community was the main problem before and our project really
made a difference to this situation that is reduction in HIV
transmission rate among young people in the community.

Explain to the participants that a monitoring system needs to cover

all three. In the previous session, the groups have already
developed a list of indicators for each of the three aspects.

Step 4: Ask the participants to develop a monitoring system of HIV/AIDs

project of “Ah Lin Yaung” covering all these three. ( 30 minutes’
discussion and 5 minutes’ presentation)

Efficiency:
e.g.
i. How much does it cost to organize a training workshop?
ii. How many participants join?
27
 iii. How much does it cost per participant?
iv. Is it cost effective?
v. Whether the resources used for the activities are appropriate
in terms of output or they are cost-effective.

All these information can be tracked easily in a tracking sheet.

No. Date Training No. Duration Total training cost Remark
topic participant
M F

Effectiveness:
For example, we plan to improve the qualifications of all high
school teachers in particular area, did we succeed after project
completion?

You can develop indicators to monitor the progress to achieve this

target.

Impact:
For example, you might have an indicator of impact which is that
“safer sex options are chosen” as an indicator that “young people
are now making informed and mature lifestyle choices”. The
variables that might affect the indicator include:

 Age
 Gender
 Religion
 Urban/rural
 Economic category
 Family environment
 Length of exposure to your project’s initiative
 Number of workshops attended.

By keeping the right information you will be able to answer many

questions such as:

 Does age make a difference to the way our message is received?

 Does economic category i.e. do young people in richer areas
respond better or worse to the message or does it make no
difference?
 Does the number of workshops attended make a difference to
the impact?

28
 Answers to these kinds of questions enable a project or
organization to make decisions about what they do and how they
do it, to make informed changes to programmes, and to measure
their impact and effectiveness. Answers to questions such as:

 Do more people attend sessions that are organized well in

advance?
 Do more schools participate when there is no charge?
 Do more young people attend when sessions are over weekends
or in the evenings?
 Does it cost less to run a workshop in the community, or to
bring people to our training centre to run the workshop?

Step 5: Decide how you will collect the information you need and where it
will be kept (on computer, in manual files).

Step 6: Decide how often you will analyze the information – this means
putting it together and trying to answer the questions you think are
important.

Develop monitoring plan - 2

Step 7: Or simply, distribute worksheet 6-2: Develop monitoring plan 2,

you can look at the table information needed and design monitoring
plan.

Planning for Evaluation

Step 8: The facilitator explains to the participants:

Similar to monitoring plan, we should start evaluation plan right from the
beginning. When we do needs assessment, we will collect data about economic,
health, education and social situation of the community. They will become
baseline data for us and to measure the impact or achievement of objectives, we
can compare these data with those that we collect after or during the
implementation of the project.

Evaluation involves:

 Looking at what the project or organization intended to achieve – what

difference did it want to make? What impact did it want to make?
 Assessing its progress towards what it wanted to achieve, its impact
targets.
 Looking at the strategy of the project or organization. Did it have a
strategy? Was it effective in following its strategy? Did the strategy
work? If not, why not?
29
  Looking at how it worked. Was there an efficient use of resources? What
were the opportunity costs of the way it chose to work? How sustainable
is the way in which the project or organization works? What are the
implications for the various stakeholders in the way the organization
works.

In an evaluation, we also look at efficiency, effectiveness and impact.

Step 9: Ask the participants to share with the class the evaluation types they
know. ( 5 – 10 minutes) Then distribute the names of different types of
evaluation and definitions to match each other. Distribute Worksheet
6 – 3: different evaluation types to the participants. The facilitator can
go through the table together with the participants and discuss about
different types of evaluation.

Step 10: Distribute worksheet 6 – 4: Case study and ask the groups to discuss
among themselves first. ( 15 – 20 minutes)

Step 11: Ask the groups to fill the form included in worksheet 6 – 5: Evaluation
Basics. ( 15 – 20 minutes)

Step 12: Explain to the participants that normally, the first step in evaluation is
to develop Terms of Reference for evaluation. Share worksheet 6 – 6:
format of ToR included and go through the format together
explaining the points they want to clarify more.

Group work: Each group needs to develop Terms of Reference for

evaluation of Youth’s Energy’s projects. (45 to 60 minutes)

30
 WORKSHEET 6 – 1

DEVELOP MONITORING PLAN – 1

Efficiency: Whether the resources used for the activities are appropriate in terms
of output or they are cost-effective.
For example, training program that train 50 peer educators costs $10000. Is it
efficient or cost-effective?
e.g.
i. How much does it cost to organize a training workshop?
ii. How many participants join?
iii. How much does it cost per participant?
iv. Is it cost effective?
v. Whether the resources used for the activities are appropriate in terms
of output or they are cost-effective.

All these information can be tracked easily in a tracking sheet.

No. Date Training No. Duration Total training cost Remark
topic participant
M F

Effectiveness: It is the measure of the extent of the achievement of the

development project against the target objectives.
For example, we plan to improve the qualifications of all high school teachers in
particular area, did we succeed after project completion?

We can monitor the quality of high school teachers regularly using different indicators.

Impact: It measures whether or not project implementation has contributed to

the change in the situation of the problem it was trying to address.
For example, high HIV transmission rate among young people in the community
was the main problem before and our project really made a difference to this
situation that is reduction in HIV transmission rate among young people in the
community.

We can monitor the progress to achieve targeted impact comparing baseline data with
current situation.

31
 WORKSHEET 6 – 2

DEVELOP MONITORING PLAN – 2

What do we want to know?
This includes looking at indicators for both internal issues and external issues.

There are three basic types of monitoring commonly used:

Management/administration
staff/personnel
vehicles
supplies

Finance
project budget and expenditure
staff salaries
cash flow analysis

Project activities
project inputs – budget, equipment, key staff needed
results of activities – project outputs/outcomes/impact (using indicators)
the way the project is managed
situation or context – policy environment, political situation

The table below shows the information needed to collect for monitoring programs and
project.

INFORMATION FOR MONITORING PROGRAM OPERATIONS 3

CATEGORIES WHAT TO WHAT WHO WHO USES HOW TO USE WHAT

OF MONITOR RECORDS COLLECTS DATA INFORMATIO DECISIONS CAN
INFORMATION TO KEEP DATA N BE MADE

1. Work • Timing of •Monthly • Project • Project • Ensure staff • Reschedule

plan activities /quarterly Manager Manager and activities and
Activities • Availability work plans •Supervisors • Donor other deployment of
of • Work agency resources are resources as
personnel, schedules available needed
resources

2.Costs Budgeted • Ledger of • Financial • Project • Ensure • Authorize

and amounts, expenditure officer/ Manager funds are expenditures

3Source: The Family Planning Manager’s Handbook. Editors: James A. Wolff, Linda J. Suttenfield,
Susanna C. Binzen. Management Sciences for Health, Boston.

32
Expenditure funds on s by accountant • Auditor available to • Make budget
hand and budget • Donor execute and
expenditures category agency activities project revisions
• Balance in • Receipts • Ensure • Determine need
budget • Bank compliance for
by approved transactions w/funding other funding
cost • Reports to regulations sources
categories donor • If fee for
service,
determine fee
structure
3.Staff • Knowledge, • • •Superviso • Motivate • Placement
and attitudes and Performanc Supervisors r staff and • Training needs
Supervision skills of staff e • Personnel • Project resolve • Promotions
• Educational reviews director Manager employment • Disciplinary
level of staff • Job • Trainers • problems action
• Salaries and descriptions Personnel • Advise staff
benefits • Resumes Director on
• Job of staff career
performance • Feedback
from
training
attended
2. Commo • Stock • Stock • Logistics • Project • Ensure • Quantity to
dities • Ordering registers manager Manager availability of order
and • Invoices • Donor commodities • When to order
shipment • Field agency in stock • Amount to
status worker and keep in
• reports distribution to reserve for
Procurement field emergency
regulations • Ensure
good
condition
Commodities
3. Results • No. and • Client • CBD • Field • Ensure • Revise
type of cards/forms Workers supervisor objectives are objectives
services • Clinic • Clinic • Project realistic • Retrain staff
provided/ registers nurse Manager • Assess • Revise IEC
commodities • Field • Field • Donor quality of strategy
dispensed worker supervisors agency services • Revise project
• reports provided strategy and
Characteristic • Assess approach
s of appropriatene
persons ss of services
served/
educated

33
 WORKSHEET 6 – 3

DIFFERENT TYPES OF EVALUATION

Self-evaluation: This involves an organization or project holding up a
mirror to itself and assessing how it is doing, as a way of
learning and improving practice. It takes a very self-
reflective and honest organization to do this effectively,
but it can be an important learning experience.
Participatory This is a form of internal evaluation. The intention is to
evaluation: involve as many people with a direct stake in the work as
possible. This may mean project staff and beneficiaries
working together on the evaluation. If an outsider is called
in, it is to act as a facilitator of the process, not an
evaluator.
Rapid Participatory Originally used in rural areas, the same methodology can,
Appraisal: in fact, be applied in most communities. This is a
qualitative way of doing evaluations. It is semi-structured
and carried out by an interdisciplinary team over a short
time. It is used as a starting point for understanding a
local situation and is a quick, cheap, useful way to gather
information. It involves the use of secondary data review,
direct observation, semi-structured interviews, key
informants, group interviews, games, diagrams, maps and

34
 calendars. In an evaluation context, it allows one to get
valuable input from those who are supposed to be
benefiting from the project. It is flexible and interactive.
External This is an evaluation done by a carefully chosen outsider
evaluation: or outsider team.

Interactive This involves a very active interaction between an outside

evaluation: evaluator or evaluation team and the organization or
project being evaluated. Sometimes an insider may be
included in the evaluation team.

35
 WORKSHEET 6 - 4
CASE STUDY

Introduction to “Youth’s Energy4”

The CBO “Youth’s Energy (Lu Nget Swan In)” is an organization formed with
young volunteers that has operated in Mawlamyaing, Mon State since 2000. They
started the organization with five young volunteers who were newly graduates
at that time. They started giving free tuition to the tenth standard students in
their ward. Most of the volunteers are the 10th standard passed students waiting
to join universities. They receive all of their funding from donor grants, and since
their budgets reached a peak in 2007, have begun to see a gradual decline in
donor aid. They have a small staff of 6 full-time employees, and 2 part-time
employees who work 15-20 hours per week. They have more than 20 volunteers
from different areas of Mawlamyaing.

Youth’s Energy provides recreational and educational activities to youth from

the ages of approx. 10 – 18 years old. They make an effort to attract and welcome
youth from all backgrounds and experiences. They currently have 4 main
projects:

• Annual organization of a summer youth camp (sessions held over a

period of 8 weeks each summer)
• Management of a Youth Center in the center of town, which includes an
old gymnasium, 3 classroom-style all-purpose rooms, and a small room
dedicated to computer use training, with 9 donated computers.
• Sponsorship of an adolescent reproductive health educational course
module and peer education certification program that they coordinate and
teach in cooperation with 4 local schools
• Organization of after-school sports and recreation activities (held at the
youth center)

2009 has been a difficult year for the NGO. They have recently learned that their
primary donor is cutting funding by 50% for 2010. While they were devastated to
receive this news, they also recognized that they were fortunate that the donor
did not withdraw completely, as has happened to other NGOs in the region.
While they have begun to investigate various fundraising options, they realize
that they are going to have to make some decisions about their program
priorities and overall organization.

Youth’s Energy director,Ko Thiha , has been speaking to the deputy director Ma
Nwe about his concerns, and they decided to hold a meeting one evening after

4 Adapted from IRC Bosnia Evaluation training manual

36
work with all of the staff to clearly and openly explain their current and
projected future financial status. They also wanted to use that time to let the
other employees express their concerns, as they recognized that everyone was
feeling very vulnerable and concerned about the future of the organization.

The meeting was difficult, but worthwhile. Everyone knew that times were
tough, but it was good to pull the whole team together to discuss it. A general
consensus emerged that in order to survive, the organization would have to do
two things. First, it would have to more actively work on alternative fundraising
efforts in order to seek contributions to replace the lost donor funding. Second,
they realized that they needed to look at their current projects, and determine
which projects should be continued in the future, and which should be radically
refocused or possibly even dropped entirely.

Everyone knew that one project in particular needed attention: the Youth Center.
The Youth Center had been a focal point of the organization for over 3 years, and
has been a showcase of the Youth’s Energy initiative. Its prime location in the
center of town made it accessible to many people, and also served as a great way
to increase the organization’s visibility and reputation as a CBO that was active
and involved.

However, over the past 12 months several of the circumstances surrounding the
center had changed, making it a drain on the CBO’s time and financial resources.
For the first 2 years they were able to use the facility virtually free of charge, as
the township authorities provided the space rent-free. Youth’s Energy only had
to pay for the utilities and general maintenance and upkeep. However, one year
ago the authorities asked the organization to begin paying a nominal rent for the
facility, with a rent payment schedule that was tapered over a 4-year period so
that it would gradually be rented out at standard market rates. The rent for the
first year was a shock to their budget, but was reasonable and management.
However, the increased rent for the next year will be a serious strain.

Additionally, the facility is in need of some repairs that can not be put off much
longer. Some are simple, such as fresh paint and a good cleaning, and could
probably be done with help from volunteers in the community. However, others
will necessitate professional expert support, such as a re-wiring of the electrical
system, repair of a leaky roof, and purchase of new desks, chairs and tables that
are more suitable to the needs of people using computers and other specialized
equipment.

Everyone recognized that, while long a favorite program of the staff, the Youth
Center was beginning to drain resources form other initiatives, in particular, the
adolescent reproductive health (ARH) training module, and related peer
education certification program. Over the past few years it was easy to
demonstrate action to donors through the Youth Center projects: they could

37
easily report on how many activities were held, how many kids participated, etc.
It was tangible activity. Since the ARH training was not the same kind of tangible
concrete activity, and was a lower profile project, it was often easily “lost in the
shuffle” during budget development.

When confronted with the new budget and resource situation, a few of the team
members reflexively suggested that the Youth Center be maintained and the
ARH module dropped, in order to shift and re-focus resources and energy.
However, when Ma Nwe asked them to explain why they thought this was a
good idea, no one in the room could provide an explanation based on facts or
specific data. They suddenly realized that they did not know what their users or
people in the community thought about either of these efforts, and that they
didn’t know which project was more successful or had had a greater impact on
the community. These were the questions that needed to be asked and answered
before decision on the future direction of the organization could be made.

Ko Thiha pointed out that it was time to begin an evaluation of the NGO’s
projects, starting with the Youth Center and the ARH module. The organization
has done basic evaluation work in the past, but always with guidance or staff
support from the donor organization requesting the evaluation. They provided
annual reports, but these were more of a summary of activities and events rather
than a rigorous evaluation of these projects. Outside consultants had come in
twice over the past 6 years to meet with the staff and assess their work, but while
the organization had welcomed the consultants and provided them with
information, they had not been closely involved in the evaluation process, and
had been limited to simply providing information and answering questions.

They realized that they needed to learn the basic of evaluation themselves, in
order to conduct the project evaluations themselves. They began to gather
evaluation training materials from some NGO resource centers, and began a dual
effort: training of the staff in basic evaluation techniques, and applying these
skills to the evaluation of the Youth Center and ARH projects.

38
 WORKSHEET 6 – 5

EVALUATION BASICS
1. Read the case study carefully and break it down into its component parts. Fill
in the sample table, adding more space if needed.

Program Project Activity

2. What are some of the factors motivating “Youth’s Energy” to undertake an

evaluation?

3. Think about a case in which the organization had to make some difficult
decisions. Did you use data gathered from evaluations to make that decision?
Why or why not?

4. Is there an internal process within the organization that you think could
benefit from evaluation? Why?

39
 WORKSHEET 6 – 6

TERMS OF REFERENCE

Terms of Reference usually include:

Background: This is background to the project or organization, something about

the problem identified, what you do, how long you have existed, why you have
decided to do an evaluation.

Purpose: Here you would say what it is the organization or project wants the
evaluation to achieve.

The purpose of an evaluation is the reason why you are doing it. It goes beyond
what you want to know to why you want to know it. It is usually a sentence or,
at most, a paragraph. It has two parts:

• What you want evaluated;

• To what end you want it done.

Examples of an evaluation purpose:

To provide the organization with information needed to make decisions about the future
of the project.

To assess whether the organization/project is having the planned impact in order to

decide whether or not to replicate the model elsewhere.

40
To assess the programme in terms of effectiveness, impact on the target group, efficiency
and sustainability in order to improve its functioning.

The purpose gives some focus to the broad evaluation process.

Key evaluation questions: What the central questions are that the evaluation
must address.

The key evaluation questions are the central questions you want the evaluation
process to answer. They are not simple questions. You can seldom answer “yes”
or “no” them. A useful evaluation question is:

• Thought provoking
• Challenges assumptions.
• Focuses inquiry and reflection.
• Raises many additional questions.

Some examples of key evaluation questions related to a project purpose:

The purpose of the evaluation is to assess how efficient the project is in delivering
benefits to the identified community in order to inform Board decisions about continuity
and replicability.

Key evaluation questions:

Who is currently benefiting from the project and in what ways?
Do the inputs (in money and time) justify the outputs and, if so/if not, on what
basis is this claim justified?
What would improve the efficiency, effectiveness and impact of the current
project?
What are the lessons that can be learned from this project in terms of
replicability?

Specific objectives: What specific areas, internal and/or external, you want the
evaluation to address. So, for example, you might want the evaluation to include
a review of finances, or to include certain specific programme sites.

Methodology: should provide a broad framework for how the project or

organization wants the work of the evaluation done. You might also suggest the
kinds of techniques you would like the evaluation team to use. Most terms of
reference will ask for some combination of these but they may also specify how
they want the evaluation team to get input from stakeholder groups.

Here too one would expect to find some indication of reporting formats: Will all
reporting be written? Will the team report to management, or to all staff, or to
staff and Board and beneficiaries? Will there be interim reports or only a final

41
report? What sort of evidence does the organization or project require to back up
evaluator opinions? Who will be involved in analysis?

Logistical issues: These would include timing, costing, requirements of team

composition and so on.

42
 SESSION VII: GATHERING INFORMATION FOR MONITORING
AND EVALUATION

At the end of this session, the participants will understand how different
information gathering methods can be used to get the information they want to
get for M&E purposes

Time: About 1:15 minutes

Material: flipcharts, markers, Worksheets

Step 1: The facilitator can start the session by explaining to the participants as
follows:

Data collection is possibly the most important part of monitoring and evaluation,
as you must have reliable and concrete information available to you in order to
assess needs, measure progress, or determine cause and effect. Detailed data
collection and information-gathering techniques are included in “Social research
tools training manual”. So the ones who have joined social research skills
training will be familiar with many of these techniques.

Step 2: Distribute worksheet 7 – 1: “Different social research tools” and discuss

together. (The facilitator needs to read social research tools training
manual in advance if they are not so familiar with social research skills.)

Step 3: Ask the participants if anyone knows the words, “primary and
secondary sources”. Then explain to all the participants that

43
Primary sources are first-hand, original sources that provide raw, un-analyzed or
un-interpreted facts and information.

Examples of primary sources include:

Training evaluation forms, Beneficiary satisfaction forms, Organizational assessment

forms

Secondary sources are exactly what they sound like- data from sources that refer
to or interpret information gathered by another researcher.
Examples of secondary sources include:

A newspaper article on the problem of AIDS in Burma/Myanmar

EIU report on Burma/Myanmar

Step 4: Ask each group to pick up one topic OR the facilitator can assign one
topic to each group and ask them how they will gather data on the
following topics identifying primary and secondary sources. (30 minutes’
discussion and 5 minutes for each presentation)

• How the role of women in the community has changed in the past five
years
• The process through which drug addicts successfully kick the habit
• The most effective media to give health education among young people
• The role of monasteries in providing education services to young people
during Burmese kingdoms
• Reading habit among young people in urban areas

Step 5: After presentation, the facilitator can end the session by the below
statement.

They will probably find that they want or need to use more that one data collection
technique. This is a standard practice, and is referred to as adopting a mixed method
approach. A mixed method approach combines data gathered through different
techniques, bringing all of the information together for the final review and analysis. The
data is then analyzed and interpreted collectively.

Bringing information together from a variety of sources is a good way to confirm or

validate your data. This process is called data triangulation, and it can help to ensure
the validity of the conclusions. If data gathered through interview process yields similar
feedback as data gathered through a questionnaire distribution or through basic research
and document review, then they can have more confidence in the results. If the data
gathered through one method is very different than data gathered through another, then
they should analyze your results with caution, possibly even obtaining additional data to
try to determine the reasons for the discrepancy.

44
 WORKSHEET 7 – 1

DIFFERENT SOCIAL RESEARCH TOOLS

Tool Description Usefulness Disadvantages
Interviews These can be structured, Can be used with almost Requires some skill in the
semi-structured or anyone who has some interviewer.
unstructured. They involvement with the
involve asking specific project. Can be done in
questions aimed at person or on the
getting information that telephone or even by e-
will enable indicators to mail. Very flexible.
be measured. Questions
can be open-ended or
closed (yes/no answers).
Can be a source of
qualitative and
quantitative information.
Key informant These are interviews that As these key informants Needs a skilled
interviews are carried out with often have little to do interviewer with a good
specialists in a topic or with the project or understanding of the
someone who may be organization, they can be topic. Be careful not to
able to shed a particular quite objective and offer turn something into an
light on the process. useful insights. They can absolute truth (cannot be
provide something of the challenged) because it has
“big picture” where been said by a key
people more involved informant.
may focus at the micro
(small) level.
Questionnaires These are written This tool can save lots of With people who do not
questions that are used to time if it is self- read and write, someone
get written responses completing, enabling you has to go through the
which, when analyzed, to get to many people. questionnaire with them
will enable indicators to Done in this way it gives which means no time is
be measured. people a feeling of saved and the numbers
anonymity and they may one can reach are limited.
say things they would not With questionnaires, it is
say to an interviewer. not possible to explore
what people are saying
any further.
Questionnaires are also
over-used and people get
tired of completing them.
Questionnaires must be
piloted to ensure that
questions can be
understood and cannot be
misunderstood. If the
questionnaire is complex
and will need
computerized analysis,
you need expert help in
designing it.
Focus groups In a focus group, a group This can be a useful way It is quite difficult to do
of about six to 12 people of getting opinions from random sampling for
are interviewed together quite a large sample of focus groups and this
by a skilled people. means findings may not
interviewer/facilitator be generalised.
with a carefully Sometimes people

45
 structured interview
schedule. Questions are
usually focused around a
specific topic or issue.
Community meetings This involves a gathering
of a fairly large group of
beneficiaries to whom
questions, problems,
situations are put for
input to help in
measuring indicators.
Visual/audio stimuli These include pictures,
movies, tapes, stories, role
plays, photographs, used
to illustrate problems or
issues or past events or
even future events.
Rating scales This technique makes use
of a continuum, along
which people are
expected to place their
own feelings,
observations etc. People
are usually asked to say
whether they agree
strongly, agree, don’t
know, disagree, disagree
strongly with a statement.
You can use pictures and
symbols in this technique
if people cannot read and
write.
Critical event/incident This method is a way of
Analysis focusing interviews with
individuals or groups on
particular events or
incidents. The purpose of
doing this is to get a very
full picture of what
actually happened.
Participant observation This involves direct
observation of events,
processes, relationships
and behaviors.
“Participant” here implies
influence one another
either to say something or
to keep quiet about
something. If possible,
focus groups interviews
should be recorded and
then transcribed. This
requires special
equipment and can be
very time-consuming.
Community meetings are Difficult to facilitate –
useful for getting a broad requires a very
response from many experienced facilitator.
people on specific issues. May require breaking into
It is also a way of small groups followed by
involving beneficiaries plenary sessions when
directly in an evaluation everyone comes together
process, giving them a again.
sense of ownership of the
process. They are useful
to have at critical points
in community projects.
Very useful to use You have to have
together with other tools, appropriate stimuli and
particularly with people the facilitator needs to be
who cannot read or write. skilled in using such
stimuli.
It is useful to measure You need to test the
attitudes, opinions, statements very carefully
perceptions. to make sure that there is
no possibility of
misunderstanding. A
common problem is when
two concepts are included
in the statement and you
cannot be sure whether
an opinion is being given
on one or the other or
both.
Very useful when The evaluation team can
something problematic end up submerged in a
has occurred and people vast amount of
feel strongly about it. If contradictory detail and
all those involved are lots of “he said/she said”.
included, it should help It can be difficult not to
the evaluation team to get take sides and to remain
a picture that is objective.
reasonably close to what
actually happened and to
be able to diagnose what
went wrong.
It can be a useful way of It is difficult to observe
confirming, or otherwise, and participate. The
information provided in process is very time-
other ways. consuming.
46
 that the observer gets
involved in activities
rather than maintaining a
distance.
Self-drawings This involves getting Can be very useful, Can be difficult to explain
participants to draw particularly with younger and interpret.
pictures, usually of how children.
they feel or think about
something.

Primary sources are first-hand, original sources that provide raw, un-analyzed or
un-interpreted facts and information.

Examples of primary sources include:

Training evaluation forms, Beneficiary satisfaction forms, Organizational assessment

forms

Secondary sources are exactly what they sound like- data from sources that refer
to or interpret information gathered by another researcher.
Examples of secondary sources include:

A newspaper article on the problem of AIDS in Burma/Myanmar

EIU report on Burma/Myanmar

Mixed method approach

They will probably find that they want or need to use more that one data collection
technique. This is a standard practice, and is referred to as adopting a mixed method
approach. A mixed method approach combines data gathered through different
techniques, bringing all of the information together for the final review and analysis. The
data is then analyzed and interpreted collectively.

Bringing information together from a variety of sources is a good way to confirm or

validate your data. This process is called data triangulation, and it can help to ensure
the validity of the conclusions. If data gathered through interview process yields similar
feedback as data gathered through a questionnaire distribution or through basic research
and document review, then they can have more confidence in the results. If the data
gathered through one method is very different than data gathered through another, then
they should analyze your results with caution, possibly even obtaining additional data to
try to determine the reasons for the discrepancy.

47
 SESSION VIII: DATA ANALYSIS

At the end of this session, the participants will understand some basic methods
for data analysis, breaking analysis down into qualitative and quantitative data
analysis methods

Time: About 1:30 hours

Material: flipcharts, markers, Worksheets

Step 1: Ask the participants to go around the class and ask them to
interview at least 10 friends the following questions. (15 – 20
minutes)

1) How old are you? (don’t need to ask exact age)

(Under 20), (within 20 – 30 years old), (30 – 40 years old), (40 years
above)
2) Do you like reading? If yes, continue the following questions. If
no, skip the questions 4 and 5.
3) How many hours a week do you spend time for reading?
4) What kind of books do you like to read?
5) What do you do if you have free time?
6) How many hours a day do you spend to do what you like?

Step 2: Ask the participants to review and analyze the responses like the
examples.

1) Age range
48
 Among (10) respondents, most of them are within the age of 20
to 30. ( ) are above 30 and ( ) is under 20.
OR
Among ( ) respondents, ( ) are within the age of 20 – 30 years
old and ( ) are in mid-30. ( ) are above 40.

2) Among ( ) people interviewed, ( ) like reading. They spend

( ) to ( ) hours per week for reading. Out of ( ), ( ) like
reading newspapers and journals and ( ) read biographies and
( ) like to read variety of books. Most of the people who like
reading are within the age of - years old.

3) Among ( ) respondents, ( ) don’t like reading. Among them, (

) like listening music and spend ( ) hours a week for listening
music. ( ) like watching movie and ( ) like -----------------------.

Step 3: Explain to the participants that above review can identify some
trends among targeted population.

Whether working with qualitative or quantitative data, the most basic part of
data analysis will consist in identifying trends and patterns. Consider the
following examples of potential trends that they might identify through its data
analysis:

A questionnaire revealed that people within the age of 30 to 40 liked reading and they
liked to read magazines and journals most. Respondents with the age of 20 to 30 preferred
music or shopping to reading.

A focus group indicated that while young people liked the new brand of candy, older
people thought it was too sweet.

The study showed that there was no difference in music tastes among children from poor
and wealthy families.

They may be able to clearly and immediately identify some trends, though others
may be more difficult to identify, and may necessitate detailed review and
reflection.

Step 4: The facilitator tells the participants if they have some basic
understanding of trends and response frequencies, they can begin
to interpret and analyze this information.
• What does the trend mean?
• What does the data suggest?
e.g.
The data indicates that 80% of the children who completed the World Religions course
module had a favorable opinion of people from different ethnic groups, while only 23% of
49
the child respondents who did not complete the course noted a similar favorable opinion.
As the children were all from the same school, the results suggest that the course
increases children’s respect for diversity.

91% of the respondents noted that they would not feel personally affected by the
elimination of ARH program, suggesting that the program could be cut without any
serious repercussions in the community.

When asked about their satisfaction with the training program of IRC ICB team, 14% of
respondents rated interpreter’s translation skills as “Excellent,” 56% as “Satisfactory,”
and 25% as “Poor.” (5% responded “No Opinion”.) This suggests that ICB team might
need to consider providing training on translation skills of interpreters.

However, tell them to be careful to consider other explanations for cause and
effect. Later, these trends will be turned into a summary of general findings and
recommendations.

50
 WORKSHEET 8 – 1

DATA ANALYSIS

1) Go around the class and ask the following questions to your friends.
i. How old are you?
(Under 20), (within 20 – 30 years old), (30 – 40 years old), (40 years
above)
ii. Do you like reading? If yes, continue the following questions. If no,
skip the questions 4 and 5.
iii. How many hours a week do you spend time for reading?
iv. What kind of books do you like to read?
v. What do you do if you have free time?
vi. How many hours a day do you spend to do what you like?

2) Review and analyze the responses like the following examples.

1) Age range
Among (10) respondents, most of them are within the age of 20 to 30. ( )
are above 30 and ( ) is under 20.
OR
Among ( ) respondents, ( ) are within the age of 20 – 30 years old and
( ) are in mid-30. ( ) are above 40.

2) Among ( ) people interviewed, ( ) like reading. They spend

( ) to ( ) hours per week for reading. Out of ( ), ( ) like reading
newspapers and journals and ( ) read biographies and ( ) like to read
variety of books. Most of the people who like reading are within the age of
- years old.

3) Among ( ) respondents, ( ) don’t like reading. Among them, ( ) like

listening music and spend ( ) hours a week for listening music. ( ) like
watching movie and ( ) like -----------------------.

1) Whether working with qualitative or quantitative data, the most basic part of
data analysis will consist in identifying trends and patterns. Consider the
following examples of potential trends that you might identify through its
data analysis:

A questionnaire revealed that people within the age of 30 to 40 liked reading and they
liked to read magazines and journals most. Respondents with the age of 20 to 30 preferred
music or shopping to reading.

A focus group indicated that while young people liked the new brand of candy, older
people thought it was too sweet.
51
The study showed that there was no difference in music tastes among children from poor
and wealthy families.

You may be able to clearly and immediately identify some trends, though others
may be more difficult to identify, and you may necessitate detailed review and
reflection.

2) They can begin to interpret and analyze this information.

• What does the trend mean?
• What does the data suggest?
e.g.
The data indicates that 80% of the children who completed the World Religions course
module had a favorable opinion of people from different ethnic groups, while only 23% of
the child respondents who did not complete the course noted a similar favorable opinion.
As the children were all from the same school, the results suggest that the course
increases children’s respect for diversity.

91% of the respondents noted that they would not feel personally affected by the
elimination of ARH program, suggesting that the program could be cut without any
serious repercussions in the community.

When asked about their satisfaction with the training program of IRC ICB team, 14% of
respondents rated interpreter’s translation skills as “Excellent,” 56% as “Satisfactory,”
and 25% as “Poor.” (5% responded “No Opinion”.) This suggests that ICB team might
need to consider providing training on translation skills of interpreters.

However, be careful to consider other explanations for cause and effect. Later,
these trends will be turned into a summary of general findings and
recommendations.

52
 SESSION IX: REPORTING

At the end of this session, the participants will understand how to present all the
information and findings in the report.

Time: About 1 hour

Material: flipcharts, markers, Worksheets

53
Step 1: Distribute “Evaluation report format” to the participants. Like
other reporting, “evaluation report” needs to do SPR analysis first.

• Subject (What do you want to focus?)

• Purpose (What is the purpose?)
• Reader (Who will read?)

Step 2: The facilitator can explain to the participants that in general, your
evaluation should include the following elements.

1. Executive Summary: Provide a short (2-3 paragraph) summary

of the main issues evaluated, the main findings of the
evaluation, and recommendations
2. Introduction: Explain the issue being evaluated and the reason
for and goals of the evaluation (needs assessment, impact
analysis, process evaluation, or a combination of the 3)
3. Methodology: Explain how you developed your questions,
your data sources (respondents), and how you gathered your
data
4. Findings: Explain the results of your data analysis
5. Recommendations: Explain your recommendations based on
the results of the evaluation
6. Appendices: You would include Terms of Reference, list of
people interviewed, questionnaires used, possibly a map of the area
and so on.)

The facilitator can conclude the training by sharing the following

information.

If the report is for public consumption, they may want to (or be required to)
prepare a short presentation for the donors in which they orally summarize the
main results and recommendations, possibly using a PowerPoint slide
presentation to draw attention to key results.

54
 WORKSHEET 9 – 1

EVALUATION REPORT FORMAT

SPR analysis
• Subject (What do you want to focus?)
• Purpose (What is the purpose?)
• Reader (Who will read?)

EVALUATION REPORT FORMAT

1. Executive Summary: Provide a short (2-3 paragraph) summary of the main

issues evaluated, the main findings of the evaluation, and recommendations

2. Introduction: Explain the issue being evaluated and the reason for and goals
of the evaluation (needs assessment, impact analysis, process evaluation, or a
combination of the 3)

3. Methodology: Explain how you developed your questions, your data sources
(respondents), and how you gathered your data

4. Findings: Explain the results of your data analysis

5. Recommendations: Explain your recommendations based on the results of

the evaluation

6. Appendices: You would include Terms of Reference, list of people interviewed,

questionnaires used, possibly a map of the area and so on.)

If the report is for public consumption, they may want to (or be required to)
prepare a short presentation for the donors in which they orally summarize the
main results and recommendations, possibly using a PowerPoint slide
presentation to draw attention to key results.

55
 MONITORING AND EVALUATION TRAINING
POST-TEST
Please answer the following questions to know your existing knowledge and
skills.

1)What is Monitoring?

2) Why do we do Monitoring?

3) Who does Monitoring?

4) When do we do Monitoring?

5) What are the important things to look at when you monitor a project?

6) What are indicators? Please give an example of indicator.

7) What is evaluation?

8) For what purposes do you normally do evaluation?

9) When do you normally evaluate?

10) Write down the monitoring and evaluation tools you know.

56
REFERENCES

1. Perry, Valery,Sarajevo, Bosnia and Herzegovina, “Did it Work? The Basics of

Evaluation and Assessment for NGOs”, Sponsored by: International Rescue
Committee (IRC): NGO Development Program

2. CIVICUS toolkits: Monitoring and Evaluation

http://www.civicus.org/toolkits/managing-csos-toolkits

3. Gosling, Louisa with Mike Edwards,Toolkits: A practical guide to planning,

monitoring, evalution and impact assessment, Save the Children – UK 2003
Pulbications

57