Professional Documents
Culture Documents
Technical Guidelines N 2 Conducting An Annual Outcome Survey
Technical Guidelines N 2 Conducting An Annual Outcome Survey
Village 1 Village 20
1 Key
2 Key 10 HH
10 HH informa
informan interview
interview 1 Focus 2 Focus nt
t s
s group groups intervie
interview
Overview:
Estimated
Activity/step Comments Responsibility
Duration
1 Development of the 1-3 days Can take longer if M&E officer, with inputs from
questionnaire and questions are added or component coordinators and project
interview guides for modified from the coordinator.
qualitative standard template.
assessments (see TG
3 and 4)
4 Training of 1-2 days The training should M&E officer and external consultant if
enumerators and include a session on any.
field testing of sampling.
questionnaire and
qualitative
assessment guides
6 Data collection 1-3 weeks Depends on availability Project coordinator for supervision,
of cars/motorbikes. extension workers for data collection.
7 Data entry 1-2 weeks If Excel file is used, M&E officer for supervision, assistant
data entry can only be for data entry.
conducted by one
person at a time.
• The sampling frame (ie. population from where the sample is selected) is the list of
villages targeted by the project. This list should be available at project level.
• The sample will be composed of 200 households (20 villages, 10 households per
village).
• The selection of the sample is done in two stages: (1) Selection of 20 villages; and (2)
Selection of 10 households in each village.
• The same sampling frame will be used every year, but the sample selection will be re-
done every year. This means that different villages and different households will be
surveyed each year
Numbe Village
r
TECHNICAL GUIDELINES No2 - Conducting an annual outcome survey
1 Banora
2 Boubere
3 Boukaria
4 Diarendli
5 Kolla
6 Loppe Soun
…
…
…
…
209 Vienza
210 Vouk
211 Watagan
212 Winso
Round up if the decimal is 0.5 or greater. Round down if the decimal is less than
0.5.
Example:
SI = 212/20 = 10.6 SI = 11
Select a Random Number (R) between 1 and SI. To find a random number, you can
use the Excel function Rand (type “=rand()” in an Excel worksheet) which gives a
random number comprised between 0 and 1, and multiply this number by SI to get
a number between 1 and SI.
Round up if the decimal is 0.5 or greater. Round down if the decimal is less than
0.5.
Example:
Typing “rand()” on an Excel spreadsheet we get the number 0.54. We multiply this
number by SI.
Select the Rth village from the list. This is the first village selected for your sample.
Add the sampling interval (SI) and select the next village. This is the second village
selected for your sample. Add again the sampling interval and go to the next
village, which will be your third village. Continue like this until you reach 20
villages. If you get to the end of the list, just continue from the beginning.
Example:
R=6 The 6th village (Dsik) is selected
th
6 + SI = 6 + 11 =17 The 17 village (Loufta) is selected
Etc.
TECHNICAL GUIDELINES No2 - Conducting an annual outcome survey
The exercise is over and your list of 20 villages randomly selected is ready!
Example:
Example:
From there on, apply the same methodology as explained in Option A above.
Calculate the Sampling Interval (SI) according to the formula below:
For the next steps, you will random select the villages using two separate lists of
villages (corresponding to your two categories of villages).
Etc.
• Once you have the list of 20 villages where you will go to collect data, you will need to
select randomly 10 households in each village.
• Two options exist for selecting households in the villages, described below.
• Option 1 is similar to the method for selection of the villages. It is preferred over
option 2. However, it requires having a reliable list of all households living in the
selected villages. If this information is not available, option 2 can be applied. Option 2
is cheaper and faster than option 1.
• Choosing the appropriate method for household selection will depend on the
information available and the logistic, time and resources constraints.
• More detailed information on sampling procedures is given in Annex III.
Option 1
• In each village, each household is given a unique number (no two households can
have the same number).
Example: In the village there are 35 households, such that households are numbered 1
to 35.
• A sampling interval (SI) is calculated, by dividing the total number of households by
10. Round up if the decimal is 0.5 or greater. Round down if the decimal is less
than 0.5.
Example: 35/10=3.5, SI=4.
TECHNICAL GUIDELINES No2 - Conducting an annual outcome survey
• As in the selection of the villages, a random number between 1 and SI is selected (for
example, you can write numbers from 1 to 4 on small pieces of paper, fold them and
ask someone to pick one paper).
Example: The number 2 was picked. The household number 2 of the list is selected as the
first household. Then we add 4 (the sampling interval) and select the next household
(household number 6). Add again 4 and so on until you reach 10 households.
Option 2
• Once the data collection team arrives in the village, the approximate center of
the village is identified. A pencil or bottle is spun to select a random walking
direction (also called a transect line).
• The data collection team then counts the number of households encountered
along the transect line between the centre and the perimeter of the cluster
(end of the village). This number is divided by 10 (total number of household
needed in our case) to determine the interval at which households will be
selected in the transect line.
• When the transect line contains less than the number of households required,
all households in the line are included in the sample and the data collection
team returns to the center of the cluster to pick a second random walking
direction and the process is repeated.
• This may require returning to the center and repeating the process as for
transects with fewer than the number of required households.
• The survey team will consist of enumerators and their supervisors. It is recommended
that each team should consist of three enumerators and one supervisor (see below).
• Training and field testing must be organized with all enumerators and supervisors
before going to the field.
• The training should include sessions on: (i) questionnaire comprehension and
interviewing skills; (ii) sampling procedure for selection of the households in the
villages; (iii) how to conduct key informant interviews.
• The duration of the training can vary from one to two days, and the training can be
conducted in the project office.
• The training should include field testing of the data collection tools (questionnaire and
interview guidelines), the purpose being to verify that all questions are clear and easy
to understand by the respondent. If necessary, the questionnaire should be revised
after training and field testing. The revision must involve the trainees who did the field
testing.
Day 4: 1 supervisor
(project staff)
1 supervisor
(project staff) 1 enumerator
1 supervisor (field officer)
(project staff)
Day 5 - 15 1 enumerator
(field officer)
1 enumerator
(field officer)
6 villages
6 villages
6 villages
TECHNICAL GUIDELINES No2 - Conducting an annual outcome survey
Data collection
Household surveys
• The respondent should be the head of household or any person designated by
him/her.
• If there is no respondent in the household (e.g. head of household is absent and the
other people do not want or cannot respond), the team should proceed to the next
selected household.
• See do & don’ts for interviewers in Annex IV.
ANNEXES
Example:
One expected outcome is “Improved quality of the crops and sustainable operating
conditions for small farmers”.
In order to know whether this outcome was achieved, the performance questions might be
as follows:
1 - “To what extent small farmers have improved the quality of their crops?”
2 - “Do small farmers have a more secured and stable access to market?”
What kind of information do you need in order to answer these two questions? Information
will be related to the quality of the crops and on how farmers are organized to sell their
products. You may need information both at farmer (household) level and at community
level (ie. talking to the marketing groups or community market institutions). Let’s focus on
household level indicators (ie. that we can inform through household survey).
Now that we have the indicators, we need to translate them into questions to insert in the
questionnaire. One or more questions can be needed to inform one single indicator.
1.1 – “Do you grow high value crops?” Y/N
1.2 – “This year, did you improve the quality of your crops compared to last year?” Y/N
1.3 - “Do you see any relation between your participation in project activities and the quality
of your crops?” Y/N
2.1 - “Please explain your answer”
2.2 – “Are you part of a marketing group?” Y/N
2.3 – “Do you think that the fact that you are in this group has improved your operating
conditions?” Y/N
TECHNICAL GUIDELINES No2 - Conducting an annual outcome survey
II - Sampling
Probability sampling will be used for the survey, since it minimizes the risk of biased selection.
This means that all target villages will have the same probability of being selected. In other
words, villages located far away from the project office in a remote area will have as much
probability to be selected as villages closer to the main urban centre and more easily
accessible. This will increase the cost and duration of the survey (enumerators are likely to
travel more) but will considerably increase the credibility of the results.
Probability sampling is opposed to convenience sampling, in which the prime consideration is
accessibility: only those easily reached by interviewers are included in the sample.
Convenience sampling does not permit generalization of the data.
To select the sample, the first thing you need is a sampling frame. The sampling frame is the
list of villages that will be considered for inclusion in the sample. Since the survey will cover
only villages who are participating in project activities, in our case the sampling frame will be
composed exclusively of all the villages targeted by the project. A list of these villages should
be available at project level.
For the selection of households in the village, sampling option 1 is recommended since it is
more statistically rigorous, but it is feasible only if an accurate list of all households in the
selected villages is available (or if a map can be drawn with the village leader indicating all the
households). If a list can be obtained from a recent census or from local authorities, and if the
villages are not too large in size, the survey team should use this option to select the
households.
The second option for selecting households is the most rapid, but also the least preferred
method. This method is commonly used in anthropometric surveys.
• DO test the interview schedule beforehand for clarity, and to make sure questions
cannot be misunderstood.
• DO state clearly what the purpose of the interview is.
• DO assure the interviewee that what is said will be treated in confidence.
• DO ask if the interviewee minds if you take notes or tape record the interview.
TECHNICAL GUIDELINES No2 - Conducting an annual outcome survey
IV – Questionnaire
The following standard questionnaire may be amended to capture the essence of your
project interventions.
Changing the questionnaire will however have implications on the Excel database that
IFAD has developed to facilitate data entry and analysis.
In case you will need support to amend this database to reflect the changes introduced
in the standard questionnnaire, please contact the IFAD Asia and Pacific Division
(m.donnat@ifad.org).
[Insert logo of
QUESTIONNAIRE ID: /___/___/___/ implementing agency]
Note for enumerator : Before starting the interview, introduce yourself, explain what are the objectives of
the survey and ask the person if he/she consents to respond to the questions. If not, go to the next
household.
Circle codes corresponding to the answers in the right column.
A – HOUSEHOLD IDENTIFICATION
A.4 What is the gender (sex) of the household head? A. Male B. Female
A. Well off
In terms of income group, in which of the following categories would you B. Average
A.5
consider that your households belongs?: C. Poor
D. Very poor
Over the last 12 months, were you (or any household member) involved A. Yes
B.2
in any activity of the [insert name of the project] B. No (->go to B4)
In which of the following project activities were you (or any member of your household) involved during the
B.3
past 12 months? [For each main set of activity below (e.g. “Forestry training”, circle the corresponding code]
B.3.1 - [Insert name of activity a] A. Yes B. No
A. Yes B. No
B.3.4 - [Insert name of activity d]
A. Very satisfied
In general, regarding how your participation in project activities has
B.4 B. Moderately satisfied
impacted on your living conditions, are you:
C. Not satisfied at all
A. Frequently
How often do you have contact with project staff (extension workers;
B.5 B. Occasionally
facilitators, etc.)?
C. Rarely
C - LIVELIHOODS
A. Yes
C.1 Do you have a source of cash income? B. No (no cash income, only
subsistence agriculture)
A. Higher
C.1.1 – Compared to last year (i.e. 12 months ago), would you say
B. Equal
that, this year, your income is:
C. Lower
/_ __/
What is the main source of income of your household?
C.2
[Use codes below and write corresponding code on the right] [One answer only]
[Adapt the response options below to the
context of your project]
G. Salaries, wages (employees)
A. Agriculture and sales of crops
H. Handicraft
B. Fishing and sales of fish
I. Remittances
C. Livestock and sales of animals
J. Begging, assistance
D. Natural resources
K. Other (specify:
E. Petty trading
F. Unskilled labour
C.3.1 - What are these other income sources for your household?
/_ _ _/ ; /_ __/ ; /__ _/
[Use same codes as above ]
[Max 3 answers]
D – FOOD SECURITY
Over the last 12 months, was there any period(s) during which you were
A. Yes
D.1 not able to provide 3 meals per day for your household (food shortage
B. No (-> go to D.2)
period)?
D.1.1 - For how many weeks in total, over the last 12 months? /___/___/
A. Some improvement
E – LAND TENURE
E.1 Do you own productive land (including fish ponds)? A. Yes B. No (-> go to E.2)
E.1.1- What is the size of your land (in acres)? /_ __/__ _/ acres
A. Very secure
B. Moderately secure
E.2.1 - How secure do you consider your property rights?
C. Insecure
D. Very insecure
During the past 12 months, did you adopt any new agricultural A. Yes B. No (-> go to
F.2
production technology promoted by the project? F.3)
/____/ ; /____/
F.2.1 - Which one? [response options to be adapted by project]
[Two answers maximum]
Compared to last year (ie. 12 months ago), this year, did your crop A. Yes B. No (-> go to
F.3
F.4)
production area increase:
A. Small
A. Yes B. No (->go to
F.4 Do you use irrigation system(s)? F.5)
F.4.1 - Did you manage to increase your irrigated area this year A. Yes B. No (->go to
F.5)
compared to last year (i.e. 12 months ago)?
A. Small
A. Yes B. No (->go to
F.5 Do you have access to a fish pond? F.6)
F.5.1 - Compared to last year (i.e. 12 months ago) did the A. Yes B. No (-> go to
productivity of this fish pond increase this year? F.6)
A. Small
A. Yes B. No (->go to
F.6 Do you own livestock? G.1)
F.6.1- Compared to last year (i.e. 12 months ago), is your herd A. Yes B. No (->go to
G.1)
size larger this year?
A. Small
F.6.1.1 - How would you quantify this increase? B. Medium
C. Large
G – ACCESS TO MARKETS
A. Yes B. No (->go to
G.1 Do you get an income from sales of agricultural production? G.2)
A. Increased
G.1.1 - Compared to last year (i.e. 12 months ago), has your
B. Did not change
income from sales of agricultural production increased?
C. Decreased
A. Yes B. No (->go to
G.1.2 - Do you have a contract for selling your production? G.2)
*******************************************************************************
The following sections (H, I and J) are optional: they should be included in the questionnaire only if
the project is involved in activities related to the particular topic. For example, if the project has a
natural resources management component, include the section on NRM (section J). Remove the
sections that are not relevant to the project
H – ACCESS TO RURAL FINANCIAL SERVICES
Did you (or any household member) borrow money over the last 12
H.1 A. Yes B. No (-> go to H.2)
months?
A. Informal (friends, NGOs,
Cooperatives/credit group)
H.1.1 – What was the source of credit?
B. Formal (bank/micro-finance
institution)
/___/___/___/___/___/___/___/
H.1.2 - How much did you borrow (total last 12 months)?
[Write the amount in the local currency]
[Currency]: _______________
A. Yes
H.1.3 – Have you repaid the amount borrowed? B. Not yet, but soon
C. No, cannot repay
Overall, do you consider that, compared to last year (ie. 12 months ago),
H.2 A. Yes B. No
your household has now a better access to financial services?
H.2.1 - Would you say this is related to any project activity? A. Yes B. No(-> go to I.1)
A. Yes, establish
I.1.2 - Did the project help your household establish or expand
B. Yes , expand
your enterprise/business?
C. No (-> go to I.2)
[Section to adapt. Select only the parts that are relevant for your project activities.].
J.1 Do you have access to community fish ponds? A. Yes B. No (-> go to J.2)
J.1.2 – Has your access to the fish ponds improved since last year? A. Yes B. No
J.1.3 – Do you consider that the productivity of the fish ponds has A. Yes B. No (-> go to J.2)
improved as compared to last year?
J.1.3.1 - If yes, was it thanks to project activities? A. Yes B. No
J.3.2 – Do you consider that your access to the forest has improved A. Yes B. No (-> go to J.4)
as compared to last year?
J.3.3 – Do you consider that the productivity of the forest has A. Yes B. No (-> go to J.4)
improved as compared to last year?
J.3.3.1 – If yes, was it thanks to project activities? A. Yes B. No
J.4 Do you have access to community pasture land? A. Yes B. No (-> end)
A. Yes B. No
J.4.1 – Is this access regulated?
A. Yes B. No
J.4.2 – Do you consider that your access to pasture land has
improved as compared to last year?
J.4.3 – Do you consider that the productivity of the forest has A. Yes B. No (-> go to J.4)
improved as compared to last year?
A. Yes B. No
J.4.3.1 – If yes, was it thanks to project activities?
**********************************************************************************
***THIS IS THE END OF THE INTERVIEW, THANK YOU
QUESTIONNAIRE B
FOR NON BENEFICIARIES
This questionnaire is the same as questionnaire A except that it does not contain the questions related to
project activities. The numbering should not be changed, so that same questions have the same number in
both questionnaires.
C - LIVELIHOODS
A. Yes
C.1 Do you have a source of cash income? B. No (no cash income, only
subsistence agriculture)
A. Higher
C.1.1 – Compared to last year (i.e. 12 months ago), would you say
B. Equal
that, this year, your income is:
C. Lower
/_ __/
What is the main source of income of your household?
C.2
[Use codes below and write corresponding code on the right] [One answer only]
[Adapt the response options below to the
context of your project]
G. Salaries, wages (employees)
A. Agriculture and sales of crops
H. Handicraft
B. Fishing and sales of fish
I. Remittances
C. Livestock and sales of animals
J. Begging, assistance
D. Natural resources
K. Other (specify:
E. Petty trading
F. Unskilled labour
/_ _ _/ ; /_ __/ ; /__ _/
C.3.1 - What are these other income sources for your household?
[Use same codes as above ]
[Max 3 answers]
D – FOOD SECURITY
Over the last 12 months, was there any period(s) during which you were
A. Yes
D.1 not able to provide 3 meals per day for your household (food shortage
B. No (-> go to D.2)
period)?
D.1.1 - For how many weeks in total, over the last 12 months? /___/___/
A. Some improvement
E.1 Do you own productive land (including fish ponds)? A. Yes B. No (-> go to E.2)
E.1.1- What is the size of your land (in acres)? /_ __/__ _/ acres
A. Very secure
B. Moderately secure
E.2.1 - How secure do you consider your property rights?
C. Insecure
D. Very insecure
During the past 12 months, did you adopt any new agricultural
F.2 production technology (e.g. promoted by government extension A. Yes B. No (-> go to F.3)
services or by a donor-funded project)?
A. Through government
extension services
B. Through an NGO
F.2.2 – How did you learn about this new technology?
C. Through donor-funded project
D. Through a neighbour/friend
E. On my own initiative
Compared to last year (ie. 12 months ago), this year, did your crop
F.3 A. Yes B. No (-> go to F.4)
production area increase:
A. Small
F.6.1- Compared to last year (i.e. 12 months ago), is your herd size
A. Yes B. No (->go to G.1)
larger this year?
A Small
F.6.1.1 - How would you quantify this increase? B. Medium
C. Large
G – ACCESS TO MARKETS
G.1 Do you get an income from sales of agricultural production? A. Yes B. No (->go to G.2)
A. Increased
G.1.1 - Compared to last year (i.e. 12 months ago), has your
B. Did not change
income from sales of agricultural production increased?
C. Decreased
G.1.2 - Do you have a contract for selling your production? A. Yes B. No (->go to G.2)
*******************************************************************************
The following sections (H, I and J) are optional: they should be included in the questionnaire only if
the project is involved in activities related to the particular topic. For example, if the project has a
natural resources management component, include the section on NRM (section J). Remove the
sections that are not relevant to the project
/___/___/___/___/___/___/___/
H.1.2 - How much did you borrow (total last 12 months)?
[Write the amount in the local currency]
[Currency]: _______________
A. Yes
H.1.3 – Have you repaid the amount borrowed? B. Not yet, but soon
C. No, cannot repay
Overall, do you consider that, compared to last year (i.e. 12 months ago),
H.2 A. Yes B. No
your household has now a better access to financial services?
A. Yes (Establish)
I.1.2 - Did anyone help you establish or expand your enterprise or
B. Yes (Expand)
business?
C. No
A. Government extension services
[Section to adapt. Select only the parts that are relevant for your project activities.].
J.1 Do you have access to community fish ponds? A. Yes B. No (-> go to J.2)
J.4 Do you have access to community pasture land? A. Yes B. No (-> end)
J.4.2 – Has your access to fish ponds improved compare to last year?
J.4.3– Do you consider that your access to pasture land has improved A. Yes B. No
as compared to last year (i.e. 12 months ago)?
**********************************************************************************
***THIS IS THE END OF THE INTERVIEW, THANK YOU