ID 5011

A Hybrid Course on Water Quality

– An Approach to People’s Water Data

Lecture 20: Research Survey

1
Collecting Survey Data:
General Principles
Prof. Ram Fishman
Tel Aviv University

2
Why?
• Collecting data through surveys continues to
provide us with most information about the human
condition

• Especially important in low and middle income

contexts

3
• Especially important in low and middle income
contexts

4
Why?
• Several large-scale survey efforts provide us with
crucial information about important aspects of life
in under-developed areas.

5
Example: DHS
Demographic and Health Surveys

6
7
Example:
Joint
Monitoring
Program

8
Challenges
• People do not always report accurate, reliable
data…

• Sampling challenges
• Limited accuracy
• Subjective biases
•…

9
How?
• Traditionally, paper and pen…
• Increasingly, use of ICT:
• Phone surveys
• Using mobile phones to conduct in-person
interviews

10
General Guidelines
for Survey Design
• Garbage in, garbage out…
• Put yourself in the place of the respondent
• Be efficient and precise
• Use existing, tested surveys
• Avoid ambiguity…

11
Designing a survey

12
Types of Questions
• Objective / Subjective

• Qualitative / Quantitative

• Open ended / Multiple choice / Numerical

• Observational data

13
Formulating Effective Questions

Each question should be SMART:

• Specific: Asks one precise question

• Measurable: Quantifies accurate and unbiased information

• Accessible (understandable): Easy to comprehend and clearly defined

• Relevant: Measures a key or intermediate outcome for your project

• Time-bound: Has a clear specification of the relevant time period

14
I. Respondent’s time is costly
• Treat it as a very scarce resource.
• Only ask what you really need and know how to
use.
• Include exactly what you need to answer your
research questions, or potential future research
questions, but do not include questions if you do
not have a clear idea of how they would be used.
Follow the checklist for proofing each question.

15
II. Leave no room for ambiguity
• Questions should be phrased precisely and
concretely, leaving no room for subjective
interpretation.
• Put yourself in the place of the respondent

16
III. Respect sensitivities
• Be sensitive to the cultural and social contexts
• Work with someone intimately familiar with community
• Be mindful of sensitivities that can cause biases
• Use “deliberate loading,” i.e., set the question in a context that the
behavior is acceptable. For example: “Disciplining children can be
hard and parents do it in several ways. Have you ever hit your child
to discipline them?”
• Be aware of “social desirability”
• Do not “lead the witness”
• Do not place sensitive questions too early the survey
• Enumerators should not know the respondent at all
• Pilot!
17
IV. Data Validation
• Minimise room for human error on both the
respondent and surveyor sides.
• Ask important questions in more than one way,
cross-validate
• Catch inconsistencies dynamically.
• Limit ranges for numerical variables.
• Use Digital tools such as geo-location and images to
check data.

18
V. Observation
• Use very standard and precise procedures
• Worry about enumerator bias
• Avoid “hawthorne effects”:

19
Additional Reading

• J-PAL/IPA Datahub for Field Experiments in Economics and Public Policy

• World Bank microdata catalog

• The World Bank’s Enterprise Surveys database

• IFPRI microdata catalog

• International Household Survey Network survey catalog

• Demographic and Health Surveys Model Questionnaires

20