Capture reliable data

in the international
development sector

A step-by-step guide to
successful project implementation
Design, Capture, Understand, Act
#withAkvo
Contents

Introduction 04
1 | What is a data journey and why do you need it for your programme? 06
2 | How to conduct data research for your programme in four steps 09
3 | How to instill data standardisation within your programme 11
4 | How to choose your sampling strategy to guarantee relevant results 14
5 | How to design your survey in five steps 19
6 | How to choose the best tool for data collection 23
7 | How to prepare for smooth project implementation 27
8 | How to ensure success in the field in five steps 35
Conclusion 40
About Akvo 41
Credits 41

03
 Introduction
Why this eBook?
This eBook guides international development
professionals through every step in the capture
phase of the data journey1, resulting in reliable and
effective data collection. This eBook will help you
to:
• Successfully implement your projects at local
level
• Effectively manage and align project data
collectors
• Consistently collect quality data in time for
reporting
• Achieve a good relationship with local
stakeholders
• Improve your programmes based on accurate
and reliable data
The importance of a design phase
In the data journey philosophy, a thorough design
phase is central to the success of any programme.
A design phase is conducted at the beginning of a
1  https://akvo.org/our-approach/
Introduction
programme but revisited and revised throughout
to ensure accuracy. Before you enter into the
capture phase of your data journey, you need to
gain a comprehensive overview of what data is
needed as well as the opportunities, challenges and
objectives or your programme. What problem do
you aim to address and which factors will influence
your goals? What work has already been done and
who was involved? What do you need to monitor
to be accountable, steer and learn? How can you
effectively track implementation across different
locations? What data do you need to monitor
results? How can you ensure that the data collected
is used for decision making?
These questions are at the heart of the design
phase, and answering them will ensure that you
don’t end up with unnecessary data, inaccurate
data due to inconsistent collection methods, and a
lack of progress to show to managers and donors.
There are three steps in programme design: conduct
a context analysis, create a Theory of Change, and
design the monitoring framework. For a deep dive
04
 into each of these steps, download our design eBook:
Design data-driven programmes that deliver results
Design effectively2.

Gain clarity on the context of your Capture data that matters

programme, the problem you are trying The data collection process can be cumbersome and
to solve, the data you need, and the roles frustrating. Gaining an accurate overview of the data
and responsibilities of each partner. collection progress in the field can be difficult, leading to
Share insights with the missed deadlines and delays in reporting. When data does
relevant people, generate come in, it can be inaccurate and incomplete. Was the
dialogue, encourage decision
survey not designed and tested properly? Have the data
making and continuously
improve your work.
collectors been properly trained? With inaccurate and
incomplete datasets, it’s impossible to provide the right
Act information to programme managers, monitoring and
evaluation (M&E) managers, and donors. What’s more,
Capture it’s sometimes hard to square project requirements with
the field reality, which can have an adverse effect on data
Capture reliable and high collection and slow the impact of your programme.
quality data from the start.
Monitor your data collection Following the data journey methodology will help you
to ensure accuracy and track to avoid these common pitfalls. This eBook will take you
Extract the insights that matter. Clean,
progress. through every step of the capture phase, from choosing
analyse and visualise your data and turn it
your sampling strategy to training data collectors and
into valuable information and knowledge.
heading out into the field. This way, you can capture
reliable and high quality data from the start and monitor
Understand your data collection to ensure accuracy and track
progress.

2  https://cdn2.hubspot.net/hubfs/1891949/Akvo%20eBook%20-%20
Design%20data-driven%20development%20programmes%20that%20
deliver%20results%20effectively.pdf

Introduction
05
1 | What is a data journey
and why do you need it for The data journey methodology consists of four phases: design, capture,
understand and act. They form the starting point for organisations to

your programme? ensure data is used to contribute to lasting and inclusive impact. These
phases aren’t always consecutive or prescriptive, there may be some
overlap, and it may be necessary to go back to a previous phase due to
findings at a later stage.
Design
Context analysis What does each data journey phase consist of?
Theory of Change
Website / reporting Monitoring framework Design
Open data sharing Data research Gain clarity on the context of your programme, the problem you are
System integration Survey design trying to solve, the results you are trying to achieve, the partners you’ll
Data informed decision making Data collection plan work with, the data you need to monitor progress, and the roles and
Sampling plan responsibilities of each partner. The design phase should enable you to
Act
define your data needs and prepare for a smooth data capture process.

Capture Questions to consider include:

Prepare data collection • What is the context you are operating in? Who is involved and what
Data collection is their role?
Verify data collection • Which impact do you want to achieve and which outcomes will
Data cleaning Monitoring cycles contribute to it?
Data analysis • Which data will you need for which purpose, which data already
Data visualisation exists and which do you still need to collect?
Understand • What does the optimal survey design look like to ensure success?
• Which sampling plan fits best and is most cost-effective?

1 | What is a data journey and why do you need it for your programme?
06
Capture
Collect relevant, high quality data from the start.
Implement your data collection plan and track
progress. Questions to consider include:
• Are the tools and skills and logistical plan in
place to commence data collection?
• How can you verify and ensure the quality of
your data on the go?
• How best do you organise monitoring cycles of
repeated data collection?
1 | What is a data journey and why do you need it for your programme?
Understand
Clean, analyse and visualise your data and turn it
into valuable information. Extract the insights that
you can act upon. In the understand phase of your
programme, you can generate information which
can be interpreted to extract insights.
• What data sources are you planning to combine?
Is your data clean and ready for analysis?
• How can you extract insights from your data?
• How will you visualise the data and ensure
effective data storytelling1?
1  https://akvo.org/blog/five-tips-for-effective-data-storytelling/
Act
Share insights with the relevant people, generate
dialogue, encourage decision making and
continuously improve your work. In the act phase,
you’ll share your data to influence change.
• How will you share with the key audiences?
• Which systems does the data need to be stored
in?
• How can you amplify your insights and create
lasting impact?
For international development professionals,
following this data journey will ensure smooth and
successful implementation of your programme,
allowing you to focus on capturing data that
matters. In this eBook, we’re focusing on the
capture phase of the data journey.
07
Round out your
data journey knowledge
Out later this year

Download the eBook now Visit our knowledge library

08
 2 | How to conduct
data research for your
programme in four steps
Data research is a method that helps you to
systematically assess existing data and data
sources, allowing you to identify where there are
gaps in the data you need and where you can add
value with your programme. At the same time, data
research gives you the tools to think about your
stakeholders and audience, and save resources by
avoiding duplicate data collection. Strictly speaking,
data research belongs in the design phase of the
data journey. We’ve included it here as well as in
our design eBook 1 because it provides a really
good starting point for your data collection. In this
1  https://cdn2.hubspot.net/hubfs/1891949/Akvo%20eBook%20
-%20Design%20data-driven%20development%20programmes%20
that%20deliver%20results%20effectively.pdf
2 | How to conduct data research for your programme in four steps
chapter, you’ll find an overview of four consecutive
steps that will help you in conducting data research:
• Make an inventory of existing data
• Evaluate existing data
• Perform a gap analysis
• Understand who will use your data
Make an inventory of existing data
Once you’ve identified what data needs you have
within your programme in the design phase, you will
need to start gathering it. Some data may be readily
available, while other data may still need to be
captured. You can start off by making an inventory
of existing data.
First of all, look into the data resources of your own
organisation, including what is gathered in reports
and stored in databases. Consider both quantitative
data, expressing a certain quantity, amount
or range, and qualitative data, which is more
descriptive, resulting from small scale surveys, focus
group discussions, observations and interviews. You
can then think about what data may be available
and easily accessible outside of your organisation.
Are there any data sharing platforms or other
organisations that deal with the same problem or
try to answer the same question? What data do they
have on this problem? Is it open access? Even if the
data is not openly accessible, it might be possible to
persuade this organisation to share its data.
Evaluate existing data
Once you’ve created an inventory of existing data
sources, it is important to evaluate the existing
data on its accessibility, granularity, credibility and
relevance. The following questions can help you
understand whether the existing data is available
for usage, detailed enough and has the right
scale, and reliable enough for you to use in your
programme:
• Is the data openly available, or does it require
special permission to access? (Accessibility)
• Is the data structured in a way that is useful for
your programme? (Relevance)
09
• How often is the data collected? (Granularity)
• How granular or detailed is the data geographically? (Granularity)
• How granular or detailed is the data demographically? (Granularity)
• When was the data collected? How long has it been retained? (Relevance and
Granularity)
• Do the current problem solvers use it for decision making, evaluation, or
something else? (Credibility)
• Who collected the data? What was the purpose of their data collection? Has
the data been cleaned and/or analysed? And if so, in what way? (Credibility)
Perform a gap analysis
Now that you’ve identified the data sources that are available to you and
what data you can use for your programme, you need to think about what
data you still need to collect to answer your questions. To do so, it helps to
ask the following: what data do I need to answer my questions or describe
my indicators? It’s important that, in the first instance, you don’t think about
restrictions that might be apparent in collecting this data. Only after identifying
the data you need should you start considering potential restrictions, such as
time, resources and feasibility. It might turn out that data you initially deemed
infeasible to collect isn’t as difficult to gather after all.
Once you have identified all the data gaps, take a critical look at the data you’ve
identified as necessary. Do you really need to collect all that data? And what
are you going to use all the different elements for? Although it’s tempting to
collect data that you may think will be useful in future, a general rule of thumb
is that less data is more. It’s better to focus on the things that really matter and
minimise complexity. It’s less expensive, less time consuming, and you don’t run
the risk of collecting the wrong data.
2 | How to conduct data research for your programme in four steps
Understand who will use your data
If you are collecting data to contribute to solving a problem, or to underline the
importance of addressing a certain problem, keep in mind that it is crucial to
involve all relevant stakeholders from the start of the data research process.
This will create ownership of the data, ensure relevance and usefulness of the
data, result in communities feeling represented by the data, and avoid decision
makers questioning the credibility of the data. Start your data collection
exercise with an inventory of what the different stakeholders want to know
and how you are going to reach them. Sharing the data with the people directly
involved in the problem empowers them to take action. However, this involves
thinking about how to share the data in an understandable and accessible way.
In remote communities, accessing the data online may prove to be difficult,
and radio stations or distribution of offline materials may be a better mode of
dissemination. You might want to consider making a data dissemination plan,
in which you identify your stakeholders and their respective communication
channels.
Data research is an approach that will help you to create focus in your
programme. Thinking in this structured way about data gathering will avoid
collection of duplicate data and encourage everyone involved to determine
the quality and usefulness of available data. This method also allows you to
assess whether the data you are collecting is truly relevant to your programme,
and the different stakeholders involved, and forces you to think about how to
disseminate the data to them before the data collection has actually started.
10
 3 | How to instill data standardisation
within your programme

Data standardisation is an essential but often

overlooked part of the data journey. By instilling
data standardisation into your data processes right
from the start, you can make sure that your data
provides valuable and comparable insights both
now and in the future. Data standardisation can be
interpreted a number of ways. Here, we mean both
data standards, which are standardised approaches
developed with a sector or thematic focus, and the
standardisation of the data process, which focuses
more on how you process your data to ensure
consistency across the board.

Incorporating these two aspects into your work

will not only mean that the data will be comparable
within your own programme, but also across time
and space. Data standardisation is a continuation of
the work you’ve done in the data research chapter.
In this chapter, we’ll show you how to instill data
standardisation into your programme step by step.

3 | How to instill data standardisation within your programme

11
 Here are some key things to keep in mind for data standardisation:
• Ensure that your standardisation is instilled in every part of your data
process, beginning with design.
• Review and evaluate whether specific data or survey standards are relevant
to your data collection. If so, look at how you can incorporate them into your
work.
• Test run and pilot the full process before implementing - you may find you
have overlooked a key question or piece of information.
• Ensure that all the actors in the process, from data collectors to analysts,
understand the standards you have set in place and their importance.
• Document your logic, and any variations to the standards you have made.
Step one: Research and review
pre-existing data and survey standards
Increasingly, organisations and programmes are seeing the value in having
standardised data definitions, tools, and processes. Regardless of which
sector you’re working in, there’s likely to already be a standardised survey or
certification standard available. For instance, in water, and sanitation, hygiene
(WASH) the Joint Monitoring Programme (JMP)1 between UNICEF and WHO has
produced several standard definitions and questionnaires for measuring the
access level to WASH in households, schools, and medical centres. Likewise in
agriculture, certification systems exist for just about every commodity on the
market. Some, such as the Sustainable Rice Platform (SRP)2, are set to have a
standardised series of questions that are asked over the course of several years
to demonstrate a farmer’s progress towards sustainably produced rice. There
are even standardised question series and full questionnaires for screening for
1  https://washdata.org/
2  http://www.sustainablerice.org/
3 | How to instill data standardisation within your programme
trauma3 and progress towards empowerment4.
There are two key areas to consider when reviewing data standards:
• Definitions - these are the defined terms and parameters within a data
standard. This covers both what is meant by a specific term, for instance
“pit latrine,” and also the defined parameters that make up a definition, for
instance “safe water source.”
• Standardised questionnaires - Often, a data standard will come with either a
full questionnaire or a series of standardised questions which incorporates
set definitions.
Start by researching the available standards and tools in your area of work,
and compare these to your own needs. It may be that the data standards and
definitions only partially match your needs. In step two, we will look at what you
can do if this is the case.
Step two: Incorporate existing standards into your data process
Once you have identified the relevant standard (or standards, depending on
your needs and topic), you will need to decide whether the standard matches
your needs as it is or whether you only need to incorporate part of it. Both of
these options have their pros and cons. If you take the whole standard, you’ll
need to double check that you don’t need any additional information to what the
standard asks for. Alternatively, if you incorporate only part of a standardised
questionnaire, you’ll need to make sure that it fits well into the flow of your
overall survey. It’s important to make sure you aren’t asking for the same
information multiple times in different ways.
3  https://www.ptsd.va.gov/professional/assessment/te-measures/index.asp
4  https://oxfamilibrary.openrepository.com/bitstream/handle/10546/620271/gt-measuring-womens-
empowerment-250517-en.pdf?sequence=4
12
If you are using an international standard, or one that has been used in a
different country of context, you should be very careful when incorporating it to
ensure that it is relevant to your situation.
Step three: Map out the data process according to your standard
According to the data standard, you can map out your data processes. This
involves defining and documenting what you plan to do at each step of your
data process to ensure it is consistent and replicable. Often, a data standard
will have a calculation system in place in order to measure compliance with, or
adherence to, the practices the data standard is promoting. You will need to
be aware of these calculations when incorporating the data standard into your
questionnaire.
Step four: Train data collectors on the methods, tools and
standards
This step is covered in-depth in chapter seven, however it is important to
emphasise the importance of data collectors having a common understanding of
the data collection tools, and in particular the standard (or partial standard) you
have incorporated. Without building this into your training process, you could
encounter issues with data quality.
Step five: Field test the data collection tools and validate
During the data collector training, field test the survey to ensure that the
questions and standards are relevant/appropriate to the local context of your
work. This is particularly important when using an international standard.
Some data standards allow adjustments to the questions. You will also need
to continue to monitor the incoming data to ensure there are no outliers or
inconsistencies. For any changes to be made to the standardised questions,
these should be documented for the data analysts and for future use.
3 | How to instill data standardisation within your programme
Step six: Set up a standard approach for data cleaning,
transformations and visualisations
As mentioned in step three, you will have already noted the scoring pattern
and built it into your data collection. The data standard is likely to have an in-
built scoring system as a way of calculating compliance to the practices they
are promoting. By using a data standard, you will be able to set up standard
data processes such as standard transformations/calculations in analysis and
visualisations. Ideally, this will reduce the amount of data cleaning you need to
do as the answers will already be standardised. However, there are always a few
outliers that need to be cleaned, for instance a qualitative answer that needs to
be thematically coded.
Data standards allow a structured approach to data cleaning and recoding.
This should include backing up the raw data, any cleaned/recoded data, and
an overview of how (if any) answers have been coded. By having these in place,
you can create standardised visualisations for comparison across projects and
locations.
Step seven: Reuse the data process and your data
Following a data standard and/or a standardised data process will allow you
to replicate the process easily. It also gives you a place to better evaluate your
pre-existing systems and procedures in order to improve them. Actively use your
data to do cross comparisons, and ensure that you use your data for decision
making. Once you’ve standardised your data process, you can make sharing,
merging, comparing and ultimately using data easier.
13
 4 | How to choose your sampling
strategy to guarantee relevant results
The sampling strategy is the plan you put forward
to ensure that the sample you use is characteristic
of the population you are researching. If you
don’t have a sampling strategy in place, you may
collect data which is biased or not representative,
rendering your data invalid.
Following your data research, you’ll know which
data already exists (secondary data) and which
data you still need to capture (primary data). This
chapter will outline ways in which primary data
is collected using an example in which a survey
on characteristics (tax, education levels, etc) is
collected on residents in five towns. The towns
are of different sizes and have a total of 3,200
households. These 3,200 households make up the
target population for your survey.
4 | How to choose your sampling strategy to guarantee relevant results
Step one: Define your sample and target
population
At times, your survey may require you to cover the
entire target population, as is the case in mapping
or population studies. That’s usually referred to
as a census survey. However, target populations
are generally large and expensive to survey. In
our example, it may not be feasible to visit all
3,200 households of the five towns. Instead, you’d
want to choose a smaller sample that would be
representative of the population and reflect its
characteristics.
A survey that is done on a smaller number of
the target population is referred to as a sample
survey. You can infer your findings for the
entire population based on this representative
sample. In the following sections, we’ll describe
the different terminologies that are associated
with sample surveys, such as sample size and
sampling technique. These concepts will enable
14
you to determine the number of surveys needed
to accurately reflect the true characteristics of
a population and to choose the best method of
selecting a sample from that population.
Step two: Define your sample size
The first step in your sampling exercise will be to
decide on an appropriate sample size. There are no
strict rules for selecting a sample size. You can make
a decision based on the objectives of the project,
time available, budget, and the necessary degree of
precision.
In order to select the appropriate sample size,
you will need to determine the degree of accuracy
that you want to achieve. For this, you’ll need to
establish the confidence interval and confidence
level of your sample.
The confidence interval, also called the margin of
error, is a plus or minus figure. It is the range within
which the likelihood of a response occurs. The most
commonly used confidence interval is +/- 5. If you
wish to increase the precision level of your data, you
would further reduce the error margin or confidence
interval to a +/- 2. For example, if your survey
question is “does the household pay tax?” and 65%
of your sampled households say “yes,” then using
4 | How to choose your sampling strategy to guarantee relevant results
a confidence interval of +/- 5, you can state with
confidence that if you had asked the question to all
3,200 households, between 60% (i.e. 65-5) and 70%
(i.e. 65+5) would have also responded “yes.”
The confidence level tells you how sure you want to
be and is expressed as a percentage. It represents
how often the responses from your selected sample
reflect the responses of the total population. Thus,
a 95% confidence level means you can be 95%
certain. The lower the confidence level, the less
certain you will be.
Most surveys use the 95% confidence level and a +/-
5 confidence interval. When you put the confidence
level and the confidence interval together, you can
say that you are 95% sure that, if you had surveyed
all (3,200) households, between 60% and 70% of
the households of the target population would
have answered “yes,” to the question “does the
household pay tax?”.
The size of your sample may be determined using
any standard sample size calculator such as Survey
Monkey or Raosoft. Using a standard sample size
calculator (as can be seen in table one below) for
our example of 3,200 households in five towns, we
can examine the difference in sample sizes based on
different confidence levels and intervals.
Option A
If you decide on a 5% confidence interval and want
to achieve a 95% confidence level, the sample size
will be 345 households.
Option B
If you wish to have higher accuracy and increase the
confidence level to 99%, the recommended sample
size would be 551.
Option C
For an even higher accuracy you could choose a 2%
confidence interval and 99% confidence level and
arrive at a sample size of 1807.
If time and resources permit, you could opt for
larger samples and choose option C, to survey
1807 households. However, the quality of your
findings are likely to only be marginally better than
with option A or B, as the rate of improvement of
accuracy gradually diminishes with the increase
in sample size. The size of your sample should
therefore be decided by the objectives of the study
and resources available.
15
Table 1: Calculate your sample size Step three: Define your sampling technique
Once you’ve chosen the sample size for your survey, you’ll need to define which
Factors Factors description Option A Option B Option C sampling technique you’ll use to select your sample from the target population.
Population The total population that your 3,200 3,200 3,200 The sampling technique that’s right for you depends on the nature and
(no.) sample will present objectives of your project. Sampling techniques can be broadly divided into two
Confidence The probability that your sample types: random sampling and non-random sampling.
level (%) accurately represents the 95% 99% 99%
characteristics of your population Random sampling:
Confidence The range that your population’s • Simple random sampling with or without replacement
interval (%) responses may deviate from 5 5 2 • Stratified random sampling
your samples • Systematic random sampling
Sample size calculated 345 551 1807
Non-random sampling:
• Purposive or subjective sampling
• Convenience sampling
• Judgment sampling
• Quota sampling
• Snowball sampling

Random sampling
As the name suggests, random sampling literally means selection of the sample
randomly from a population, without any specific conditions. This may be done
by selecting the sample from a list, such as a directory, or physically at the
location of the survey. If you want to ensure that a particular household does
not get selected more than once, you can remove it from the list. This type of
sampling is called simple random sampling without replacement. If you choose
not to remove duplicate households from the list, you would do a simple random
sampling with replacement.

4 | How to choose your sampling strategy to guarantee relevant results

16
Systematic sampling is the most commonly used method of random sampling, Table 2: Calculate stratified random sample
whereby you divide the total population by the sample size and arrive at a figure
which becomes the sampling interval for selection. For example, if you need to Location Population size Proportion (%) of Stratified
choose 20 samples from a total population of 100, your sampling interval would population sample size
be five. Systematic sampling works best when the population is homogeneous, Town 1 1200 38% 129
i.e. most people share the same characteristics. In our example, the sampling Town 2 900 28% 97
interval would be nine (3200/345 = 9 for a 95% confidence level and 5% Town 3 800 25% 86
confidence interval). You would thus select every ninth household in a town. Town 4 180 6% 19
Town 5 120 4% 13
However, populations are generally mixed and heterogeneous. To ensure Total 3200 345
sufficient inclusion of all categories of the population, you will need to identify
the different strata or characteristics and their actual representation (i.e.
proportion) in the population. In such cases, you can use the stratified random
sampling technique, whereby you first calculate the proportion of each strata
within the population and then select the sample in the same proportion,
randomly or systematically, from all the strata. Non-random sampling
In non-random sampling, the sample selection follows a particular set of
If we take our earlier example of five towns, to calculate a stratified random conditions and is generally used in studies where the sample needs to be
sample, you will need to calculate the proportion of each town within the sample collected based on a specific characteristic of the population. For example, you
size of 345 as shown in table two opposite. Column three gives the proportion of may need to select only households which own a car, or have children under
each town of the total population (3,200). In column four, the sample size (345) is six years of age. For this, you would consciously select only the 345 or 551
proportionately divided across the five towns. For example, town three, which is households that have those characteristics. Also termed purposive or subjective
25% of the total population, will select 86 households with a sampling interval of sampling, non-random sampling methods include convenience, judgment,
nine (i.e. 800/86) in the same manner as was done for systematic sampling. quota and snowball sampling.

4 | How to choose your sampling strategy to guarantee relevant results

17
Step four: Minimise sampling error
It’s normal to make mistakes during sample selection. Your
efforts should therefore always be to reduce the sampling
error and make your chosen sample as representative of
the population as possible. The robustness of your sample
depends on how you minimise the sampling error. The extent
of errors during sampling vary according to the technique or
method you choose for sample selection.

For samples selected randomly from a target population,

the results are generally prefixed with the +/- sampling error,
which is the degree to which the sample differs from the
population. If your study requires you to know the extent
of sampling error that is acceptable for the survey, you can
select a random sampling technique. In random sampling,
you will be able to regulate the survey design to arrive
at an acceptable level of error. In a non-random sample
selection, the sampling error remains unknown. Thus,
when your sample survey needs to infer the proportion of
a certain characteristics of the target population, you can
select a random sampling method. But if you want to know
the perceptions of residents regarding taxation laws or the
school curriculum, you would want to capture as many
perceptions as possible, and therefore select a non-random
method in situations where sampling errors or sampling for
proportionality are not of concern. Non-random sampling
techniques can be very useful in situations when you need to
reach a targeted sample with specified characteristics very
quickly.

4 | How to choose your sampling strategy to guarantee relevant results

18
 5 | How to design your survey in five steps
Before you start designing your survey, you
should’ve covered the following steps in your data
journey:
• Clearly defined your research questions
• Identified the characteristics / parameters you
want to collect information about
• Completed the secondary data review and
identified data gaps
• Identified the target sample and the
geographical location/s for your survey
Now, you can design your survey by following this
step by step guide.
Step one: Outline the design of your
survey
After you’ve defined your sample, you can decide
how you’ll collect the data from the respondents.
Your survey design will include a survey format
with a list of questions which correspond to your
data needs and the frequency at which the data
will be collected. The frequency of data collection
5 | How to design your survey in five steps
depends on the type of data being collected.
Baseline or mapping studies are one time surveys
while tracking/monitoring surveys are conducted at
time intervals. The periodicity of data collection is
determined by the project goals and objectives and
the related set of indicators listed in your Theory
of Change (download our design eBook 1 for more
information).
Step two: Adopt good practices while
designing your questionnaire
A questionnaire is likely to be most effective if
you KISS: Keep it short and simple. If you don’t
have a satisfactory answer to what you will do
with the answer to a question, leave it out. Avoid
the temptation to add a few more questions just
because you are doing a questionnaire anyway. If
necessary, place your questions into three groups:
1  https://datajourney.akvo.org/ebook-design-data-driven-
development-programmes-that-deliver-results-effectively
must know, useful to know and nice to know.
Discard the last group, unless the previous two
groups are very short.
Start your questionnaire with an introduction or
welcome message clearly stating who you are and
why you want the information in the survey. A good
introduction or welcome message will encourage
respondents to cooperate and participate. In
case of sensitive or private information, reassure
your respondent that their responses will not
be revealed. In some cases e.g. child/underage
surveys, it may be mandatory to seek the consent
of the respondent or guardian. In any case, it’s
important that you gain the informed consent of
the participants interviewed. This may require a
signature or verbal consent, so think about how you
can incorporate this into your survey.
19
While designing a questionnaire, it’s important to reflect on how the order of the
questions can impact the results of your survey. Ideally, you should:
• Place the easiest and most pleasant to answer questions at the beginning of
your survey.
• Group together questions on the same topic.
• Leave difficult or sensitive questions until near the end of your survey.
• Address the data collector observations, validation issues, GPS readings,
photographs, testing (e.g. water quality testing) at the end of the survey, and
avoid breaks while interviewing the respondent.
• Use a logical or natural order to answer choices, presenting positive to
negative or excellent to poor scales and agree-disagree choices in that order.
Step three: Design your questionnaire according to the data type
Broadly speaking, there are two types of data: quantitative data and qualitative
data. Quantitative data is collected with a structured questionnaire which may
have closed ended questions (i.e. with a list of options to choose from) and/or
open ended questions, depending on the type of information you need.
Qualitative data is often essential to understanding the context and explaining
the quantitative data. It is generally collected as free text, which may be
translated into numbers by classifying the information or assigning codes. It is
recommended that qualitative information be used sparingly, where the possible
responses are not known in advance and will add value to your survey. This is
because qualitative answers tend to take longer to check, clean and process.
5 | How to design your survey in five steps
Most questionnaires will gain value using a combination of both types of data.
Your questionnaire will depend on the objectives of your project. If you need
data to monitor the status of water points across a city, you are likely to ask the
following questions:
• What type of waterpoint is it? Respondents can select from a list of options,
e.g. hand pump, well or tap.
• Where is the waterpoint located? Respondents can provide the name of the
city/village and a GPS reading.
• Is the waterpoint functioning? You’ll need to clearly define functionality to
ensure a common understanding for all data collectors.
• Is the water safe to drink? For this, you may need to test the water for certain
parameters, document perceptions on water safety from respondents, or
collect healthcare information from existing records.
• Who owns or is responsible for this water point and is it maintained? You can
ask whether it is publicly owned (government) or privately owned, what type
of repair (major or minor) has been done in the last few years, how much
it cost and who paid for it. Again, define what constitutes major and minor
repairs.
In the above example of the waterpoint monitoring, questions (a) and (b) are
examples of (structured) questions to collect quantitative data. Questions (c), (d)
and (e) could be framed to collect both quantitative and qualitative information.
Question (c), where you ask about perceptions of safe water, is an example of
qualitative information, where you will record the responses verbatim and enter
the data as free text.
20
Step four: Choose the question type to match your data needs
There are three basic ways in which questions are designed in surveys:

1. Multiple choice Rating Scales and Agreement Scales

e.g. Have you watched this movie? Rating Scales and Agreement Scales are two common types of questions also
Yes No used to qualify multiple choice questions.

2. Numeric open ended 1. How would you rate the movie?

e.g. How many times have you watched the movie? Excellent Good Fair Poor

2. On a scale from 1 to 10, where ‘10’ means you have enjoyed this aspect the
3.Text open ended most, how would you rate the movie?
e.g. What did you like about this movie? Acting Editing Casting

Direction Production Cinematography

3. How much do you agree/disagree with the following statements?

Strongly Agree Disagree Strongly

Agree Disagree
a) The movie has a
strong social message
b) Children should not
watch the movie
c) There is unnecessary
violence in the movie
d) The storyline of the
movie is weak

5 | How to design your survey in five steps

21
While designing a closed ended questionnaire, you ‘yes’ in question one.”
should try to include the maximum possible list of
relevant alternatives as answer choices. This helps Step five: Build in steps for quality control
to systemise and categorise respondent’s answers Pretesting your questionnaire is recommended as it
and saves time on text entries. However, this also will help you to ensure that your survey design is in
reduces the scope for capturing detail and you will sync with your data needs. During a pretest, you get
need to decide how flexible you want to be and to to know whether the questions have been worded
what extent the additional detail will improve the properly and are soliciting the expected responses.
findings of the survey. Choosing a question type is You may need to edit/remove/add questions and
largely based on how you want the data to come explanations after a pretest. It also gives you a
out, which depends on how you want to use the chance to check the quality of the data collectors.
data. Digital technology allows survey designs in which
you can factor in quality checks. Questions can be
You can pretest the questionnaire before the survey marked as mandatory or optional and submission of
if you want to generate a list of alternative question forms can be made dependent on completion. Also,
types. When you’re unsure about the possible data quality can be periodically checked for timely
answer choices, use an open ended format by feedback and course correction.
adding “other (specify)” as one of the alternatives.
Also allow a “don’t know” or “not applicable”
response to all questions, except to those in which
you are certain that all respondents will have a clear
answer.
In your survey, some questions may be dependent
on responses to other questions. For example,
in our case of the moviegoer, if the respondent’s
answer to question one is “no,” i.e. they have not
watched the movie, the rest of the questions would
be redundant. In this case, you would add an
instruction “continue survey only if the response is

5 | How to design your survey in five steps

22
 6 | How to choose the best
tool for data collection
Now that you’ve decided what data you need and how you’ll collect it, it’s time to
think about what you will use to get the job done. It’s easy to reach for the pen and
paper, but there’s an abundance of tools out there which will save you time, ensure
higher quality data, and streamline your processes. In this chapter, we’ll focus on
mobile-based survey applications, since they allow you to collect a large variety
of data and development projects usually do their baseline and data monitoring
exercises based on questionnaires.

Choosing the right tool for your project and organisation may seem like a daunting
task. To make it easier, we’ve put together three main questions to ask yourself before
you pick your new data product:
• How does the tool fit my data needs?
• How does the tool fit my organisation?
• Who is behind the product?

How does the tool fit my data needs?

In the design phase and the beginning of the capture phase, we’ve looked at all of the
aspects of the data you need to generate insights for decision making. The product
you choose should tick all the boxes to match your data needs.

6 | How to choose the best tool for data collection

23
What type of data do you need?
Most mobile-based data collection tools support capturing data as text,
numbers, dates, GPS, photos, video and audio. But some projects might need
to be able to capture other types of data, such as geographical shapes, which
are commonly used in agricultural projects to map farm plots, in conservation
projects to track deforestation, or in WASH projects to map water resources.
You may want to scan barcodes or QR codes, or input biometric details of your
respondents. You might want to enrich your data with water or soil quality
measurements, air pollution data, drill logs, or other sensor-based data.
When selecting your tool, check what other data types the tool provides or if
integrations with other data sources are possible.
Furthermore, when analysing data you might want to combine your field survey
data with other datasets to enrich your analysis. Does the product support
importing data from other sources? If so, in what format?
What scale are you aiming for?
Let’s start with the overall data scale. Do you need to collect data from 100
respondents or 100,000? Will you use the tool at an organisational or national
level? The larger the scale, the better the product needs to be to support traffic
without limiting the data collection efforts or slowing down the analysis.
How many questions will one survey hold? Not only does your tool of preference
need to be able to support long surveys and work flawlessly in the field, but the
more questions you ask, the more variables you will have in your dataset. Does
the tool support large datasets, not only in terms of rows (submissions) but also
columns (questions)? Does it have a functionality in place to make it easy for you
to clean, analyse and work with such large datasets?
6 | How to choose the best tool for data collection
Do you foresee using the tool only for one project, meaning you’ll fill out only a
handful of different questionnaires, or will you use it across different projects?
If so, you’ll need to find a tool which supports not only more data, but multiple
surveys and users. Some products have built-in data limitations to ensure the
performance is steady. When deciding on your product, it is worthwhile asking
about any scale limitations that are in place.
How frequently will you capture and work with data?
Are you searching for a tool just to conduct a one-time data collection and
analysis exercise or are you planning to work with data on a regular basis? The
more frequently you capture data, the greater volume of data the product
will need to support. Moreover, you will need to be able to easily handle and
analyse data coming in at a higher frequency to avoid bottlenecks with cleaning,
reporting and analysis.
Frequent data collection often happens when you’re monitoring your projects.
So with frequency, you also need to look at how you monitor. Do you gather data
from a representative sample? Does the product support that? Or do you want
to go back to the source, a specific household, water pump, farmer or school, to
collect another round of data? Does the tool allow you to link data about your
source to the following data rounds so that you can easily track the changes over
time?
How do you aim to use the data?
One thing is having a mobile-based survey tool that makes it simple to collect
data, but you also need to be able to use the data, analyse it, learn from it
and act upon the insights you’ve gained. How does the product support you
in cleaning and analysing the data? Does the product also make it easy for
you to create the reports you need? Does the product provide integrations to
24
share your data externally or to push the data to
other tools you use? Are you going to export your
data to a dedicated analytics platform? Or do you
need an easily accessible dashboard for multiple
stakeholders to review your data in real time? These
types of considerations help determine whether
you need a simple export to Excel, more complex
data visualisation features, or even full platform
integrations with specialised reporting software.
Where will you work with data?
Many mobile-based data collection tools support
offline data collection. If you are working in areas
with low connectivity, or none at all, it is important
to check not only if you can capture data offline,
but also how will you get the surveys onto the
devices and the data from the devices for analysis.
And what about survey creation, data cleaning and
data analysis? If your team works in an area where
Internet connection is not stable, find out whether
the tool provider offers offline support for these
tasks too.
6 | How to choose the best tool for data collection
How does the tool fit your organisation?
We have covered your data needs, but it is also
important to look at the needs of your organisation.
Who will be using the product?
The first thing to look at is the team in your
organisation that will be working with the product.
Who will be creating the questionnaires, collecting
data, analysing it, etc? How many people will be
working with the product? Some products come
with restrictions on the number of users.
The more people working with data, the more
you need to think about user access. Will all the
users have access to everything or do you want
to organise your data to ensure people only have
access to data they work with? Does the product
support different levels of user access and does it
have ways to keep things organised?
Last but not least, do not forget to look at language
support. First of all, you need to make sure that the
product comes in the languages your teams speak.
Many of the current products are multilingual to
accommodate international needs. If the product
does not support your language of preference, is the
tool provider willing to add the language? Secondly,
language support for your surveys can be quite a
deal breaker. Often, data is analysed centrally, in
one language, but the questionnaires are asked in
remote areas where the respondents speak their
local language. Does the product provide options to
translate the surveys?
How much effort is required to get the
product going?
As with any other product or new system, you need
to learn how to use it. You might need to adapt your
workflow and roll it out to other team members.
How much effort is required to get started? Does
the platform require custom development by an
engineer in order to fit your needs or are all required
features available in a self-service mode for you
to start right away? Does the product supplier
provide training on the product? Do they have
documentation easily available and accessible?
Is there a channel in place where you can contact
them to ask questions or report issues?
With mobile-based data collection products, there
are two main aspects to keep in mind. First is the
actual data collection part. Who will collect the data
in the field? Are the data collectors familiar with
smartphones and other tech tools? Is the mobile
app simple to use? Will they need to be trained?
How easy is it to get a data collector started? How
quickly can you get the survey to the data collector?
25
Secondly, you need to think about the team
creating the surveys and analysing the incoming
data. Will you create the surveys from scratch and
is it possible to independently create and modify
surveys within the product? What about data
analysis? Do you have an in-house analyst who will
use the product or do you expect the product to
provide basic analytics for you?
What is your budget?
You can choose from free products to higher end
ones, but the main thing to focus on when it comes
to price is what it is that you get. With free products,
you might not get the support you need or you may
be limited in scale. With more expensive ones, you
might get features and services that you won’t use.
Many organisations first look at the price when
choosing a product, but we advise understanding
your data and organisational needs before you look
at costs. Introducing a new product will always
require investment from your end, so it is better to
invest in something you can fully utilise in the long
term.
Secondly, the cost of the product also depends
on the devices you need to run it. Will you need to
set up a server internally to run the product? Does
the mobile data collection app support any device
or are their certain requirements that may limit
6 | How to choose the best tool for data collection
your purchase choice? When purchasing devices, such as respondents’ names. It is important to
it is important to also think about overall security. have answers to these questions, not only for the
What are the risks and costs of a device getting lost, duration of your partnership with the tool provider,
damaged or stolen? but also for once your contract ends. How will the
data be handled, archived, or removed?
Who is behind the product?
When you choose a data tool, you are also going Picking the right data product is a very important
for the organisation behind the product. Where are task, and it’s well worth putting the time and
they based? Will they be on hand to help you get thought into picking the right one for you so that
started? What about if you need help later on in you can simplify the way you work with data.
your journey? Is the product regularly maintained
and further developed? When it comes to choosing
a product you aim to use for a longer period or on a
large scale, you want to make sure the product will
be there, working flawlessly, for as long as you need
it.
Secondly, you should make sure that the tool
provider handles your data following the legislation
of the country you work in and according to the
principles of responsible data management. How
is the data secured? Where does it flow once it’s
collected and where is it stored and hosted?
Thirdly, we recommend understanding who
owns the data, who has rights over the data, and
who decides if and how it can be shared. This
is important if your data is confidential, holds
sensitive information, or contains private data,
26
7 | How to prepare for smooth
project implementation

Once you’ve identified the data you need, defined your sample strategy,
designed your survey, and chosen your tool, it’s time to prepare for data
collection. Ticking all these boxes before going out into the field is key to
ensuring smooth and efficient data collection.

First off, here are some practical tips:

• Test the survey thoroughly before you head out into the field.
• Ensure data collectors have a consistent understanding of terminology,
survey logic, and the purpose of each question.
• Define clear roles, responsibilities, and timelines, and communicate these to
the team members.
• Build contingencies into the data collection plan - things rarely go as planned
and it is important to have a plan B (and C, D, E).

7 | How to prepare for smooth project implementation

27
Step one: Map processes and logistics
Mapping out the data collection process from start to finish will assist in
determining the logistics that need to be prepared. This includes creating a list
of all necessary tasks and roles throughout the data collection process, and
considering the necessary skill sets for successful completion. Doing this early on
helps to identify potential gaps or issues that may arise, and roles that need to
be filled. The following steps should be taken when mapping out the process.
Seek approval and inform the right people
In many locations, it is important to inform local authorities and the local
population (interviewees) before proceeding with data collection. In some cases,
you may need a certificate or formal approval from local authorities in order to
conduct the surveys. You may need to organise a local information session, or
assign a community liaison who will be a point of contact and responsible for
handling incoming questions. Besides the formal procedures, it’s also advisable
to get informed about the attitude of the local population towards the data
collection itself as well as any tools you might use to collect the data. If you have
to do water quality tests for example, make sure the people know what you are
doing to avoid misconceptions about the materials you are using.
Finally, it’s important that you gain the informed consent of the participants
interviewed. This may require a signature or verbal consent, so be sure to keep a
copy of the consent form and information about the project at the ready.
7 | How to prepare for smooth project implementation
Clarify logistics and budget constraints
Logistics and budget will take up a large part of this planning stage. During field
work, data collectors can be exposed to different risks. Knowing how to deal
with those risks and create a safe environment is part of everyone’s role, and
having responsibilities clearly defined is essential. It is necessary to define what
equipment will be used, who owns it, and what happens if something is broken
or lost.
For field work logistics and budgeting, the following points need to be
considered:
Equipment
• What equipment will data collectors need for their field work and travelling?
• What technical features does the material require, especially phones for
digital collection systems?
• Does one of the parties have administrative constraints in ordering
equipment? Government bodies often have protocols to follow which can
delay the procurement process for several months.
• Is there any equipment that needs to be shipped from another country and,
if yes, what are the procedures and timeframes for clearing the shipment?
To whom will the shipment be addressed? Is the help from custom brokers
needed and, if yes, who will be responsible for starting the procedure?
• For paper based surveys, you will need to arrange printing of the final survey
form. Who will be responsible? How many forms will need to be printed?
28
Local context
• What is the safety situation in that area and are
special precautions necessary?
• At what time of the year are the data collectors
travelling, and are there any constraints related
to climate?
• What means of transportation is most suitable
for the local context?
• Are the data collectors and their equipment
insured and, if not, how will any accidents be
handled?
• What official documents do the data collectors
need to carry with them?
• Are there travel schedules available? Take holy
days and holidays into account during which
data collectors or interviewees might not be
available. Seasonality may also have a major
impact on the availability of them both. For
instance, try to plan the data collection outside
planting and harvest seasons.
7 | How to prepare for smooth project implementation
Accommodation and infrastructure
• Take care to arrange accommodation and
means of communication in advance, especially
in rural areas.
• Is there network reception available and do you
need an Internet connection?
• Is electricity readily available at the destination,
will power packs and battery charging packs
need to be purchased?
Budgeting
• Who will cover the costs of the aforementioned
elements?
• How will the payment of the data collectors be
calculated and what will the rate be?
• Will meal costs be covered?
• Is the whole sum given in advance or will it be
paid in installments?
Check communication systems
Communications will need to work in two directions
- from the data collectors to the team leads and
coordinators and back from the coordinators to
the data collectors. In case any answers need to
be cross checked, ensure that a clear identification
system is in place that links data collectors with
each survey that they collected. If using a mobile-
based data collection system, ensure that each
device used for mobile data collection is linked to a
data collector.
Also ensure support materials and a support system
are in place in case data collectors and other users
require assistance while in the field.
29
 Role Purpose Tips
Step two: Recruit team and set responsibilities Data collector The data collector coordinator Should be a centralised person, and
Now that roles and responsibilities have been set for practical matters, the same coordinator will ensure that the logistics of data collectors should know who
has to be done for the processes within the data collection chain. This will allow the training sessions and the they are and how to contact them.
the whole team to work well together and focus on evaluating and improving the data collection are prepared in If using a digital system, this person
quality of the data. For example, who will be responsible if there are problems advance. doesn’t have to be the same as a
with the survey, who will check and clean the data, and who will oversee the project coordinator.
performance of data collectors? Data processor/ Data processors will digitise the Depending on the size of your
digitiser surveys if they are paper based sample, this role may be filled
Defining roles so that the data is aggregated by multiple people. Therefore,
Below is an example of the general roles that appear during the data collection and ready for analysis. This can have a simple system set up and
process. These will differ depending on the size and scope of your programme. be the scanning/typing up of communicated before processing
When assigning roles and responsibilities consider the skill sets already available responses, or the coding of any begins.
– who may already have experience that would make them suitable for a specific qualitative answers.
role? Who will be responsible for survey creation and data checking, cleaning, Data quality The data quality auditor will If using a digital system this can
and analysis? auditor check incoming data for quality, be done daily. If using paper based
and ensure the sampling is system, it may need to be done in
Role Purpose Tips evenly spread. Read more batches as data arrives. Only when
Data collector Data collectors collect the data The field work can be very about this role in step six of this doing this from the start and on a
and interview participants. demanding and it is needed to make chapter. regular basis can data quality be
data collectors aware of what it ensured as issues are detected at an
entails during training. early stage.
Team lead The team lead will supervise Project The project coordinator will
small groups of data collector, coordinator oversee the data collection
and act as contact points (digital system) process and ensure that all runs
between the data collectors smoothly.
and the coordinator/data team. Data analyst To perform analysis and It would help for this person to be
necessary visualisations. familiar with the survey content,
programme objectives, the sample
design being used.

7 | How to prepare for smooth project implementation

30
Step three: Prepare survey
Once the questions and order have been designed,
it’s important to ensure that the survey has been
translated into the relevant languages during
preparation for data collection, and prior to the
training.
Digital systems
If you are using a digital system, such as a
smartphone, ensure that the survey has been
digitised and tested on an appropriate device to
confirm that the in-built logic of questions work in
practice.
A final and vital step in preparing for digital data
collection is to ensure that appropriate devices
and relevant accessories (power banks, screen
protectors, adaptors, sensors and/or test strips)
have been procured well in advance of any training
or pilot exercises, to minimise confusion and user
error. When selecting hardware for smartphone-
based data collection, it is important to be clear
about the necessary functionality and specifications
required for your data collection platform.
7 | How to prepare for smooth project implementation
Hardware:
• Internal memory of the phone and possibility
for adding external memory.
• Camera, battery and screen requirements.
• USB On-The-Go connection (OTG) and GPS.
Software:
• Which operating system is the phone running
on?
• What software version is required as a minimum
for the app to run?
Step four: Select and prepare data
collectors
The next step will be to define who will collect your
data. Data collectors should be selected with care
and should be sufficiently briefed/trained on the
questionnaire before data collection.
Consider creating a FAQ document for the data
collectors if you haven’t already. Having a simple
two pager summarising important and relevant
information for the survey and data collection
process, and relevant project team contact details,
can be extremely useful. This two pager can also
include survey protocols, definition lists, sampling
guides, a day-to-day timetable of activities, and a
checklist for equipment.
In the FAQ document, include a guide with a set of
instructions and an explanation to each question
in the survey/questionnaire. The guide should also
provide definitions for each alternative to ensure
a common understanding across data collectors.
For example, if the questionnaire requires the
data collector to list the type of a water source as
improved or unimproved, the guide should clearly
mention what constitutes an improved source (i.e.
hand pump, tap) and an unimproved source (open
well, pond).
Consistency in the data collected and in sampling
is essential for good quality data. In the field, data
collectors need to be able to know what to do in
case of an issue with sampling, for example if the
household head is not at home or if a water point
is inaccessible, and who to notify. As mentioned,
data collectors will need to have a common
understanding of the questions in their survey so
that the data collection can be consistent across the
team.
31
 Guidelines for working with data collectors
Guidelines
Identify and train more data collectors than you
need.
Select data collectors who have an understanding/
familiarity with the local culture and sociology.
Select data collectors who are conversant in the
local language/dialect.
Brief all data collectors well before the survey.
Advise data collectors to be sensitive and value the
time being given by the respondent.
Advise data collectors to be courteous and
respectful to the respondents.
Advise data collectors to select time slots when the
respondent is free to talk.
Pretest the questionnaire through sample visits or
simulation exercises.
7 | How to prepare for smooth project implementation
Importance
This will enable you to replace data collectors at
short notice, without having to train them again.
This will help to build rapport quickly with the
respondents and demonstrate sensitivity to local
cultures.
This will help communication during the survey. By
minimising the use of interpreters/translators, you’ll
also save time and resources.
You need to ensure that the data collectors have
understood the questionnaire properly. During field
tests or simulations you will be able to judge their
capacities and train/guide them further, if needed.
Demonstrating sensitivity is the best way of
thanking a respondent for the time they are
voluntarily giving you.
Always remember that a respondent is not obliged
to answer your questions. You will need to reassure
them that their responses matter.
The timing of your survey is important because
if respondents are busy or preoccupied, they are
more likely to give a wrong response.
You need to ensure that the data collectors have
understood the questionnaire properly. During field
tests or simulations you will be able to judge their
capacities and train them if needed.
32
Step five: Train participants
Nothing can replace thorough training sessions,
ensuring all members of the team understand their
roles and responsibilities as well as the tools and
survey being used. These factors will significantly
affect the efficiency of the data collection process
and the quality of data that is collected.
In order to get the most out of the training sessions,
it’s a good idea to consider the following:
Venue and logistics
The venue will need to be appropriately equipped,
including (but not limited to) presentation material,
air conditioning, Internet connection, projector,
flip boards, microphones, pens, markers, paper
and multiple plug sockets. Also think about size,
accessibility and location (within or outside of the
city). Will transport need to be organised?
Who will cover the renting costs, costs of materials,
costs of meals and costs of participants (both for
travelling and participation of data collectors and
trainers)? Also consider the administration needed
to justify and keep track of expenses, for example
keeping a list of participants for receipt of their daily
allowance, their equipment, etc.
7 | How to prepare for smooth project implementation
Training materials
Develop presentation materials, guides and
tutorials and prepare all necessary data collection
tools for use during the training session (for
example, configure smartphones, equip them with
the right software and content, print the survey if
using paper based forms).
Training methodology and agenda
Training objectives for data collectors are often
threefold:
1. Master data collection tools.
2. Master the content of the questions on paper
and their translation in the field.
3. Have an understanding of the concept behind
the monitoring and evaluation (M&E) system in
which the survey is integrated.
This means training always involves teamwork. A
clear methodology should be agreed upon and all
decisions relating to the agenda should be defined
once all parties have been consulted.
To ensure that data collectors have a clear
understanding of the logic which underpins the
survey, it’s advisable to go through each question
as a group, as well as providing them with the FAQ
document to ensure consistent understanding and
then test the process as a group. Data collectors
often have field knowledge and can add valuable
contributions.
Lastly, the training should inform the data collectors
sufficiently regarding workload, their roles and
responsibilities (for example regarding materials
used in the field) and the risks of doing fieldwork.
This will enable you to filter out participants who do
not want or are not able to meet these expectations.
It is highly advisable to clearly describe this in the
data collector’s contract.
Step six: Define the data monitoring
process
Depending on the size of the project, different
people will be responsible for monitoring the data
as it comes in. In a smaller project, the data coming
in from the field will be synchronised to the online
workspace where, for example, an M&E manager is
responsible for the monitoring. On a larger scale,
let’s say a national water point mapping, this won’t
suffice, as the burden on the M&E manager will be
too big.
• Reflect on the size of your project. How many
layers of validation will work?
• Are the people identified skilled enough
to perform the job? Do they have the right
material?
33
When you talk about data monitoring, it is important to define clear rules. These Step seven: Pilot data collection process
rules are the framework guiding how we rate our data. If it does not comply with It is best practice and strongly recommended to pilot a survey before
this framework, an action needs to be taken. The action to be taking also needs commencing the full data collection round. This will help to resolve any practical
to be defined. A good way to do this is to define a rule per variable or indicator, as issues before data collection has begun, which is far easier than making changes
you can see in the example below. during data collection.
A short pilot serves a number of purposes, including:
• Testing the survey (and any tools/software) for kinks, faults or user errors.
Indicator / variable Rules Action Responsible • Providing a chance to run the entire ‘workflow’ in a controlled environment,
1.1 Full name Needs to be Check for duplicates. When National ensuring everybody knows what they are meant to be doing, when and how.
unique duplicate, liaise with data coordinator • Checking if the data collectors understand the survey questions, and test
collector which entry needs to Data collector them in a realistic context.
be removed
1.2 Age Currently in Check for errors. If error, verify National While the pilot of the survey is occurring, project coordinators should use this
school - age with data collector and correct coordinator as an opportunity to practice monitoring incoming data for quality, and to
needs to be <18 Data collector cross check that the questions asked in the survey are generating the right
information. If there are any questions with strange answers, make sure to clarify
with the data collectors. This is best done as a group activity, in order to discuss
A second thing to monitor during data collection is the progress. Are we on track questions that may need revision and to get feedback from the data collectors
according to the targets we have set? This is crucial to knowing whether the data on any problems they encountered, either with the survey content or the
collection will finish in time or whether we need to allocate resources for an technology being used.
extension. In the design phase, you have already worked out a data collection
plan, with the targets data points per data collector per day. On either a weekly Step eight: Finalise and launch data collection
or monthly basis, you can compare the targets with the actual results. This is After incorporating feedback from the pilot and making final adjustments, the
a task for the overall coordinator, who shares and discusses progress with the survey can be finalised. You will need to ensure that all devices to be used for
team leads. To make sure your overview is up to date, agree with your data data collection have the correct survey version present, or that the correct
collection teams on what interval they should synchronise the data. survey version has been printed and distributed.

Finally, to avoid confusion, ensure that all pilot data collected during the training
sessions has been removed from your database, or at least clearly marked.

7 | How to prepare for smooth project implementation

34
8 | How to ensure success in the field in five steps
The quality of the data is established from the
moment the data collectors introduce themselves
to the interviewees. This chapter consists of useful
tips and tricks for each stage of data collection.
Step one: Conduct final checks before
entering the field
If you’ve followed the previous steps, you should be
close to sending the data collectors into the field.
Here we will touch on a few final considerations to
check off beforehand.
8 | How to ensure success in the field in five steps
Communication
Crosscheck the communication system that you’ve
already put in place.
• How are field issues communicated within
the project team, from data collector to data
analyst?
• Is it clear which surveys are assigned to which
data collectors?
• Is the communication flow between data
collectors and team leads clear?
• Have the data collectors been provided a FAQ
35
 document guiding them through the survey,
protocols, etc?
Logistics
Have transportation, agreements on daily service
allowance, and accommodation been finalised?
Make sure data collectors and supervisors have:
• Clear ID, document or badge showing the
organisation and its mission, preferably with
local government endorsement.
• The survey available on their smartphone, or a
generous number of surveys printed and stored
in an envelope.
• A letter from the relevant authorities permitting
them to administer the survey.
• The FAQ two pager outlined above.
• Appropriate mobile phone credits and/or data
packages.
• A list of phone numbers to call in case of an
emergency or when support is needed.
• Information to give the interviewee upon
request, or any other party concerned by the
data collection, and to ensure informed consent
has been attained.
• A clear understanding of the survey and sample
design, and the purpose of both.
8 | How to ensure success in the field in five steps
Step two: Collect data
To ensure that your data collection runs smoothly,
there are a number of things to consider.
Here is a list of practical tips to keep in mind during
collection:
• Wear culturally appropriate clothing. i.e. some
places it may be inappropriate for women to
show their legs, or wear trousers.
• Be respectful of an interviewee’s time and
answers, as well as possible restrictions on their
answers.
• Prompt without leading the responses. Try
rephrasing questions to get the information
needed.
• Record the answers that the interviewee gives,
not the answers that the data collector feels
they should have given.
• Ensure that global positioning system (GPS)
points are taken when mapping out a site for
the first time. Try to get as close to the site, such
as a water point, as possible.
• When taking a photo of a site, try to capture
it in such a way that another person would be
able to find it without you in future, for example
when photographing a water pump, include any
familiar landmark in the background.
Interviewing the right person
Who will you need to interview to gather all the
required data? For instance, your survey has a
question about how long it takes to collect water for
the household. You are interviewing the household
head, however they are not responsible for the
collection of water. Instead, you’ll need to find and
ask the person who does collect the water, if it is
culturally appropriate to do so.
Restrictions on the interviewer
When collecting data, there may be a number of
cultural limitations on when and who the data
collector will be able to interview. Gender can come
up as a potential restriction. For instance, it may be
inappropriate for a male data collector to interview
a female interviewee, and vice versa. A potential
practical solution to this is to have mixed genders in
the data collection team.
Securing the interview location
Be conscious of the location and length of the
interview. For instance, if you are recruiting in
a market place, but need to ask some personal
questions, it might be wise to find a secluded
location to conduct the survey. This will again be
determined by cultural norms.
36
Timing
Choose the timing of your survey that would be
most convenient for the respondent. Be sensitive to
the cultural and practical elements that may affect
this. For instance, there may be times where the
interviewees are attending religious ceremonies. If
you want to interview farmers, they are unlikely to
have time to fill in a long survey during harvest. If
a participant is willing to participate in the survey,
but unavailable when the data collector arrives, try
to find out when they will be available. What are
potential times or days that your interviewees may
be unavailable? What other factors could affect their
availability or participation in the data collection?
Step three: Submit, process and maintain
data
It is important to set definitive timelines for the
collection, uploading and checking of data. A
common setup is for each data collector in a team
to hand in their completed surveys to the team lead,
who checks that they are complete at the end of the
day before approving them. This also gives the team
lead an opportunity to check the sampling progress
and discuss any issues with the data collector
during collection.
8 | How to ensure success in the field in five steps
Database entry procedures
For the entry of data from the paper based surveys,
it is good to make sure that those entering the
data have a clear format to follow. This commonly
takes the format of a spreadsheet with variables
(questions) as column headings, and each survey
respondent as a separate row. This is a stage where
many data errors can occur, so it’s essential to
have a way of linking a specific paper based survey
respondent with the spreadsheet entry. This can
take the place of an identifier (i.e gender, country,
age could look like F-NL-30).
In the case of mobile data collection, setting a time
limit for the data collectors to upload data will be
determined by a number of factors.
• Is this a monitoring survey? If yes, they may
need to sync regularly throughout the day to
ensure the data available in the device is up to
date.
• Will the data collectors have access to Internet
via mobile data packs, or only WiFi? If using
mobile data, then uploading will be more or less
continuous. If not, then uploading periodically
at hotspots will be necessary.
• When should they do the data syncing and
uploading? Making sure that the timing of
uploading the data is clear and consistent that
will make it easier to coordinate data quality
checks.
Data management protocol
Be mindful of data privacy and security
considerations when collecting data. Consider the
following questions:
• How will you secure the completed surveys?
• Will the data collectors carry the surveys with
them, or hand them into the supervisor?
• How does the data get from the field to the
processing location?
How will the original data be stored (i.e.
paper storage, digitally or both)?
For paper based surveys, you’ll need to consider
how to store the paper so that it is secure and won’t
be affected by environmental factors, such as bugs
or damp. It may be preferable to scan the paper
version to store digitally.
When storing data digitally you can consider
whether to use Cloud-based systems, as copies on
computers or in external hard drives. All of these are
options, but ensuring their security and encryption
is also essential. If multiple agencies are involved
in the data collection project, you’ll need to decide
who will host the original data, and which other
organisations will need copies.
37
Backing up data
It’s important to ensure that the original data is
still available in its unedited form as a backup. This
can be because someone may need to check the
data and code questions differently. Accidents and
errors do occur, and ensuring that there are regular
back ups that are clearly named and stored means
that you’ll always be able to go back. It’s generally
recommended to back up at least weekly during the
initial data entry and cleaning phase, and reduce
the frequency later on.
Track changes
Setting up a process to record any errors or changes
made to the data allows for the process to be
transparent. This is commonly referred to as a Log
Book. This can also include creating field reports on
difficulties encountered during data collection, for
example concerning the survey, data entry or field
experiences. During cleaning it’s also vital to ensure
there is a consistent system for the naming and
controlling of different versions.
Step four: Check data quality as it is
collected in the field
Ensuring the quality of your data is an essential
part of the data collection process, and includes
checking that everything is running smoothly
logistically, tracking the collected data, and
8 | How to ensure success in the field in five steps
checking field work activities. Checking data
quality should occur simultaneously with your data
collection in the field.
When should a team lead intervene?
The team lead should ideally check the surveys at
the end of each day and initiate course correction
immediately. Once a team leaves a survey
location it may be difficult to resurvey. Team
leads should review the sample/survey answers
where possible to ensure that all answers have
been filled in. Regularly checking in with the data
collectors themselves to ensure that they are
keeping to quotas is crucial. If issues occur either
with the survey answers, or the data collector’s
performance, the team lead should look for why
this might have occurred and provide guidance
accordingly. The following table shows a list of
important things to consider to ensure the quality of
the data.
38
Regular data checks Data/project coordinators need to check the incoming data at regular and defined Step five: Finalising data collection
intervals (daily, weekly, etc). In order to finalise the fieldwork, you should
Data quality assessment A skilled data quality auditor is required to isolate inconsistencies and ensure that the perform some final checks on the activities and
skills sampling breakdown is correct. create reports to document field operations.
Data tracking system A data tracking system is used to check what comes in per data collector. For digital data
collection, the project coordinator can easily track the last time a data collector was • Check that all data collectors have handed in
connected to the online workspace. By setting up these systems during data collection, or uploaded their survey forms and that quotas
you will be able to track sample size and distribution and infer data quality during the have been met.
data collection process. • Check the sampling and numbers of data
Data triangulation and Data triangulation is a powerful technique that uses a combination of questions and points collected per locality. Has anything been
quality checks question types to ensure data quality and validity. For example, by having a series of missed and will there need to be further data
questions with region, district and village, coupled with a GPS location question, you collection?
can ensure validity and minimise the risk of data entry error. • For paper-based data collection, archive all
Consider using a tool such as Akvo Lumen, Carto, or another mapping tool to check the survey forms consistently and digitise them.
points are in the correct locations. For digital data collection, download raw data
You can also use tools such as Microsoft Excel to filter and sort answers to find outliers reports and store them in your system.
and inconsistencies in the data. Checking data quality during data collection is a skill set • Create reports on difficulties encountered
on its own, to find out more check this FAO online article .
1 during data collection, initial results of the
Feedback and cross Set up a process for feedback and the cross checking of data collected with the data data collection, considerations regarding data
checking process collector on any errors so that they can notify the data managers ahead of time. quality, and the actual sampling in the field
Important: avoid changing the survey during data collection. It can be tempting to against the original sampling method.
make small adjustments, especially with mobile data collection. This will endanger
consistency.
Documentation For reporting purposes, keep track of errors found with data entry, difficulties
encountered during field implementation, and steps taken to resolve them.
Additionally, it is best practice to maintain a codebook to keep track of the editing and/
or coding of data.

1  http://www.fao.org/docrep/005/AC665E/ac665e07.htm

8 | How to ensure success in the field in five steps

39
 Conclusion

Regardless of your field of study, the goal of data collection is to capture

convincing and credible evidence. There are multiple data collection techniques
and methods available. What’s important is finding the method that best fits
your objectives and your available resources. Conducting a thorough design
phase is invaluable. By taking the time to achieve clarity and oversight on
the project or programme from the beginning, you’ll be equipped with the
knowledge and alignment within your team to make informed decisions in the
capture phase.

By following the steps in this eBook - from conducting data research to choosing
your sampling strategy and designing your survey - you’ll ensure accuracy and
save resources in the long run. A clean and representative data set is essential
in analysing data and helps to reduce time spent on preparing the data before
analysis. The better you’ve prepared your survey, planned the data collection
process, trained the data collectors, and resolved impurities in the data on
the go, the easier it will be to clean your data and gain valuable insights from
the analysis. This will allow you to collect, combine, analyse and visualise high
quality digital data from the field and boost the impact of your development
work.

Conclusion
40
 About Akvo
We believe in equal access to public services, reliable infrastructure
and a safer environment for everyone. We are convinced that this will
happen faster if governments and non-governmental organisations
become more effective, accountable and collaborative.
Since 2008, we’ve worked with over 20 governments and 200
organisations in more than 70 countries to improve the way they
implement development projects and make decisions using data. We
call them partners.
With our combination of tools, services, local expertise and sector
knowledge, our partners improve the management of water,
sanitation and agriculture, with a strong commitment to accelerating
the progress of the sustainable development goals.
With our unique approach to development, we help our partners
design their projects so that they can capture and understand
reliable data which they can act upon.
Visit us at www.akvo.org to learn more.
Credits
Project manager
Georgia Walker
Authors
Jana Gombitova, Geert Soet, Annabelle Poelert, Karolina Sarna, Rajashi
Mukherjee, Camille Clerx, Nikki Sloan, Stefan Kraus
Editor
Georgia Walker
Art direction / graphic design
Linda Leunissen
Photographers
P04 SmartSeeds by Stefan Kraus (RGB Collective). P05 (from left) Stefan Kraus,
Tiipaalga, Greencoffee by Lissy van Noort, Oxfam. P07 Kopernik by Stefan Kraus
(RGB Collective). P10 UNICEF Sierra Leone by Geert Soet. P14 ICCO by Joy Ghosh.
P17 Pani Programme by Sanaj Man Shrestha. P21 Kopernik by Stefan Kraus (RGB
Collective). P22 Kehijan Berbak - MCA by Aulia Rahman. P26 Kopernik by Ima
Puspita Sari. P28 UNICEF Sierra Leone by Abdoulaye Rabdo. P31 WWF by Stefan
Kraus (RGB Collective). P34 SNV Indonesia by Aulia Rahman. P37 Kopernik by
Stefan Kraus (RGB Collective). P39 Helvetas Guatamala by Ethel Mendez.
The contents of this eBook are partly based on the contents of the
AfriAlliance Handbook (see www.afrialliance.org). The AfriAlliance
project has received funding from the European Union’s Horizon
2020 research and innovation programme under grant agreement
No 689162.
41
Looking to unlock the power
of data for sustainable change?
Embark on a data journey
#withAkvo and amplify the impact
of your development work

Get in touch now