Primary Data Collection

Questionnaire
Survey Research Processes
1. Define the research aims/objectives

2. Identify the population and sample

3. Decide how to collect replies

4. Design the questionnaire

5. Run a pilot survey

6. Carry out the main survey

7. Analyse the data

 Interview Guide

• A list of topics to cover and the order in

which to cover them that can be used to
guide less structured interviews.

• The questions must be judged its

appropriateness for each interview, and
may be re-ordered, and re-worded if
necessary.
 Focus Groups

• A type of group interview.

• Focuses on group interaction on a topic
selected by the researcher.
• Ideally 4- 12 participants.
• The interaction is directed by a moderator
who asks questions and keeps the discussion
on the topic.
What Is A Questionnaire?

“A questionnaire is a document used as a data

capture instrument and does two things:
•it lists all the questions a researcher wishes to
address to each respondent and
•it provides a space or some mechanism for
recording the responses”
Questionnaire Types

Questionnaires can be:

• Unstructured/semi-structured – qualitative

• Structured - quantitative
• list all questions
• put in logical sequence
• specify precise wording to be used
• provide pre defined categories for recording the
replies

• Main aim is that questions are standardised so

individual responses can be counted up and compared
Questionnaire Design

Designing an effective series of questions is never easy:

• respondents must be able to understand the

questions and understand them in the same way as
other respondents
• respondents must be able to provide the answers
• respondents must be willing to provide the
information
Questionnaire Design

• The crucial part of good research design is making sure that

the questionnaire addresses the objectives of the research
• Fulfilling these aims should drive the design of the
questionnaire and help with the selection of questions that
are relevant, concise and efficient
• Researchers can make the mistake of asking too many
questions
• Clear and concise questionnaires can help get the best
response rate
Data Collection

Decide how to collect replies:

1) Self administered (online, postal etc) – i.e. completed by

the respondent
Questionnaires can be delivered to the respondent by
post, e-mail attachments or publishing on a web site for
interactive completion.

2) Interviewer administered (face to face)

Questionnaire introduction

It is important to explain to the potential respondents

why you believe they should answer the questions – this
can be in the introduction or an accompanying message
(email) or letter (postal)

Introductory letters, reply paid envelopes and follow-up

telephone calls also help to raise the response rate for
self-administered questionnaires.
Self-Administered Questionnaires
• Respondents are more likely to commit to answer a
questionnaire when they see it as interesting, of value, short,
clearly thought through, and well presented.
• Inducements to complete surveys can be used,
e.g. offering respondents a report of the survey findings or
entry in a prize draw.
• It is good practice to send questionnaires to an identified
individual which should lead to a higher response rate.
• Questionnaires can be delivered to the respondent by various
means including post, e-mail attachments or via publishing
on a web site for interactive completion
Questionnaire Design
Design of the questionnaire can be split into three
elements:
• Determine the questions to be asked
• Select the question type for each question and
specify the wording
• Design the question sequence and overall
questionnaire layout
Software is available to support type, wording and
design
 Questionnaires
• Much of the data in social research is gathered using
questionnaires or interviews.
• The validity of the results depends on the quality of
these instruments.
– Good questionnaires are difficult to construct; bad
questionnaires are difficult to analyze.
• Difficult to design for several reasons:
– Each question must provide a valid and reliable measure.
– The questions must clearly communicate the research
intention to the survey respondent.
– The questions must be assembled into a logical, clear
instrument that flows naturally and will keep the
respondent sufficiently interested to continue to cooperate.
Quality aims in survey research
Goal is to collect information that is:
• Valid: measures the quantity or concept that is
supposed to be measured
• Reliable: measures the quantity or concept in a
consistent or reproducible manner
• Unbiased: measures the quantity or concept in a
way that does not systematically under- or
overestimate the true value
• Discriminating: can distinguish adequately
between respondents for whom the underlying
level of the quantity or concept is different
Steps to design a questionnaire:
1. Write out the primary and secondary aims
of your study.
2. Write out concepts/information to be
collected that relates to these aims.
3. Review the current literature to identify
already validated questionnaires that
measure your specific area of interest.
4. Compose a draft of your questionnaire.
5. Revise the draft.
6. Assemble the final questionnaire.
 Step 1: Define the aims of the
study
• Write out the problem, primary and
secondary aims using one sentence per
aim. Formulate a plan for the statistical
analysis of each aim.
• Make sure to define the target population
in your aim(s).
 Step 2: Define the variables to be
collected
• Write a detailed list of the information to be collected and the
concepts to be measured in the study. Are you trying to
identify:
– Attitudes
– Needs
– Behavior
– Demographics
– Some combination of these concepts
• Translate these concepts into variables that can be measured.
• Define the role of each variable in the statistical analysis:
– Predictor
– Confounder
– Outcome
Question Types
Different types of questions can be used:
• open vs. closed
• single vs. multiple responses
• ‘tick one box only’ vs. “tick all boxes that
apply”.
• Ranking
E.g. Place in order of importance to you the
following features of a camping holiday
(Indicate by numbering from 1-4 in order where 1 is
the most important)
• Scaling e.g. Likert scale
• Level of agreement with a statement
• Can also include the catch-all option
‘Other’
• Include N/A for those who may not have
had a particular experience
 Implications of question types
Open questions Closed questions
Elicit “rich” qualitative
Elicit quantitative data
data
Encourage thought and Can encourage ‘mindless’
freedom of expression replies
May discourage responses
Are easy for all literacy levels
from less literate
to respond to
respondents
Take longer to answer and Are quick to answer and may
may put some people off improve your response rate
Are more difficult to
analyse – responses can be Are easy to ‘code’ and analyse
misinterpreted.
 General principles when writing
questions
• Avoid leading questions: “Wouldn’t you say that...”, “Isn’t it fair to
say...”
• Be specific. Avoid words like “regularly”, “often”, or “locally” – as
everyone’s idea of what is regular, often or local will be different.
• Avoid jargon and colloquialisms – ensure your language caters for
all levels of literacy. Remember for some respondents English may
be their second language.
• Avoid double-barrelled questions: “Do you enjoy playing badminton
and tennis? “ or “Do you agree with the recommendations of the Stern
Review on the economics of climate change?” Ask for one piece of
information at a time.
• Avoid double negatives – e.g. instead of asking respondents
whether they agree with the negative statement, “Smoking in public
places should not be abolished”, use the positive “Smoking in public
places should be abolished”.
 General principles
• Minimise bias. People sometimes answer
questions in a way they perceive to be
socially acceptable. Make it easy for
respondents to admit social lapses by
wording questions carefully. For example,
“How many times have you broken the speed
limit because you were late?” could be
rephrased, “Have you ever felt under pressure to
drive over the speed limit in order to keep an
appointment?”. Then you could ask, “How
many times have you prioritised the appointment
over keeping the speed limit?”
• Handling difficult or embarrassing
questions. To encourage a greater response
to difficult questions, explain why you need
to know that information. E.g., “It would be
very helpful if you could give us some information
about yourself to help us put your answers in
context:”
• Ensure options are mutually exclusive.– e.g.
“How many years have you worked in academia:
0-5, 6-10, 11-15, over 15.” Not, “0-5, 5-10, 10-
15...”
Layout and Sequence
It is good practice for the questionnaire to:
• Be of reasonable length
• Flow logically and very simply, i.e. avoiding complex routing
• Begin with questions that will raise interest (especially for self-completion
questionnaires)
• Have a title
• Have a brief introductory statement
• Include contact and return information
• Have numbered questions for reference purposes
• Have questions and answer choices attractively and neatly laid out
• Be consistent in wording
• Be standardised by using as few question types as possible
• Be written in a legible font
• Make good use of italics and bold types e.g. use italics consistently to give
instructions such as ‘please tick the box’.
 Step 3: Review the literature
• Review current literature to identify related
surveys and data collection instruments that have
measured concepts similar to those related to your
study s aims.
• Saves development time and allows for
comparison with other studies if used
appropriately.
• Proceed with caution if using only a subset of an
existing questionnaire as this may change the
meaning of the scores. Contact the authors of the
questionnaire to determine if a smaller version of
the instrument exists that has also been validated.

25
 Step 4: Compose a draft [1]:
• Determine the mode of survey administration: face-
to-face interviews, telephone interviews, self-
completed questionnaires, computer-assisted
approaches.
• Write more questions than will be included in the final
draft.
• Format the draft as if it were the final version with
appropriate white space to get an accurate estimate as
to its length – longer questionnaires reduce the
response rate.
• Place the most important items in the first half of the
questionnaire to increase response on the important
measures even in partially completed surveys.
• Make sure questions flow naturally from one to
another.

26
 Compose a draft [2]:
• Question: How many cups of coffee or tea
do you drink in a day?
• Principle: Ask for an answer in only one
dimension.
• Solution: Separate the question into two –
– (1) How many cups of coffee do you drink
during a typical day?
– (2) How many cups of tea do you drink
during a typical day?

27
 Compose a draft [3]:
• Question: What brand of computer do you own?
– (A) IBM PC
– (B) Apple
• Principle: Avoid hidden assumptions. Make sure to
accommodate all possible answers.
• Solution:
– (1) Make each response a separate dichotomous item
• Do you own an IBM PC? (Circle: Yes or No)
• Do you own an Apple computer? (Circle: Yes or No)
– (2) Add necessary response categories and allow for multiple
responses.
• What brand of computer do you own? (Circle all that apply)
– Do not own computer
– IBM PC
– Apple
– Other

28
 Compose a draft [4]:
• Question: Have you had pain in the last
week?
[ ] Never [ ] Seldom [ ] Often [ ] Very often
• Principle: Make sure question and answer
options match.
• Solution: Reword either question or answer
to match.
– How often have you had pain in the last week?
[ ] Never [ ] Seldom [ ] Often [ ] Very Often

29
 Compose a draft [5]:
• Question: Where did you grow up?
– Rural area
– In a Farm
– In a City
• Principle: Avoid questions having non-
mutually exclusive answers.
• Solution: Design the question with mutually
exclusive options.
– Where did you grow up?
• House in the rural area
• Farm in the rural area
• In the City

30
 Compose a draft [6]:
• Question: Are you against drug abuse?
(Circle: Yes or No)
• Principle: Write questions that will
produce variability in the responses.
• Solution: Eliminate the question.

31
 Compose a draft [7]:
• Question: Which one of the following do you think increases
a person s chance of having a heart attack the most? (Check
one.)
[ ] Smoking [ ] Being overweight [ ] Stress
• Principle: Encourage the respondent to consider each
possible response to avoid the uncertainty of whether a
missing item may represent either an answer that does not
apply or an overlooked item.
• Solution: Which of the following increases the chance of
having a heart attack?
– Smoking: [ ] Yes [ ] No [ ] Don t know
– Being overweight: [ ] Yes [ ] No [ ] Don t know
– Stress: [ ] Yes [ ] No [ ] Don t know

32
 Compose a draft [8]:
• Question:
– (1) Do you currently have a life insurance policy?
(Circle: Yes or No)
– If no, go to question 3.
– (2) How much is your annual life insurance
premium?
• Principle: Avoid branching as much as
possible to avoid confusing respondents.
• Solution: If possible, write as one question.
– How much did you spend last year for life
insurance? (Write 0 if none).

33
 Step 5: Revise
• Shorten the set of questions for the study. If
a question does not address one of your aims,
discard it.
• Refine the questions included and their
wording by testing them with a variety of
respondents.
– Ensure the flow is natural.
– Verify that terms and concepts are familiar and
easy to understand for your target audience.
– Keep recall to a minimum and focus on the recent
past.

34
 Step 6: Assemble the final questionnaire
[1]:
• Decide whether you will format the questionnaire yourself or
use computer-based programs for assistance:
– SurveyMonkey.com
• At the top, clearly state:
– The purpose of the study
– How the data will be used
– Instructions on how to fill out the questionnaire
– Your policy on confidentiality
• Include identifying data on each page of a multi-page, paper-
based questionnaire such as a respondent ID number in case
the pages separate.

35
 Assemble the final questionnaire
[2]:
• Group questions concerning major subject
areas together and introduce them by
heading or short descriptive statements.
• Order questions in order to stimulate
recall.
• Order and format questions to ensure
unbiased and balanced results.

36
 Assemble the final questionnaire
[3]:
• Include white space to make answers clear and to
help increase response rate.
• Space response scales widely enough so that it is
easy to circle or check the correct answer without
the mark accidentally including the answer above
or below.
– Open-ended questions: the space for the response
should be big enough to allow respondents with large
handwriting to write comfortably in the space.
– Closed-ended questions: line up answers vertically
and precede them with boxes or brackets to check, or
by numbers to circle, rather than open blanks.
• Use larger font size (e.g., 14) and high contrast
(black on white).

37
Confidentiality and Ethics
• Allow for privacy and do not ask questions which may
offend, or ask for data that is not essential. Apart from
anything else, the response rate will suffer.

• In surveys promises of confidentiality are often made to

the respondents to reassure and encourage replies.

• Data Protection and Human Rights legislation has

implications for privacy and confidentiality of survey
data.

• Aside from the legal issues, it is unethical to obtain data

from respondents by misleading them about the survey
purpose and the method of analysis.
Pilot Survey
The aim of a pilot survey is to detect any flaws in the
questionnaire and correct these prior to the main survey.

• The questionnaire should be tested on a small sample of

subjects first.

• If this is not possible, it should at least be tested on some

colleagues or friends.

Having undertaken a pilot survey, amendments can be

made to the questionnaire which will help to maximise the
response rate and minimise the error rate on answers.
Main Survey
• Identify Respondents and Keep Track of Status

A simple database is useful to keep track of when questionnaires are sent

out and to whom. This can be updated with details of follow-up contact and
dates of reminders.
• Number Each Questionnaire

Each individual questionnaire must be uniquely identified by you, either

before it is distributed or, to preserve anonymity, afterwards. This
identification could be via a number, or letters and numbers.
This code should be transferred to all the computer records that will be
produced from the questionnaire.
• Deliver the Questionnaire

Follow through on the decisions made earlier about how the questionnaires
should be administered and who the recipients should be. Deadlines for
distribution should have been decided earlier with allowances for sending
out reminders or carrying out follow-up calls.
Logical Flow
Introduction

Simple related questions

Main body

Classification details

Thank and close

Questionnaire Development
• Requires:
•Consideration
•Preparation
•Time
•Effort

•Can be a long and frustrating process, but is worth it

to achieve a questionnaire that will deliver the
required results
•Software to support questionnaire design is
available in house
Questionnaire Terms
Attitude Questions
Questions that ask respondents about their likes/dislikes, preferences, evaluation of
objects, persons, organisations, events or situations

•Behaviour Questions
Essentially factual and relate to what respondents did in the past, are currently doing
or may do in the future

•Classifications Questions
Usually factual and relate to demographic characteristics

•Pre-coding
Numbering all response categories on a questionnaire so all responses receive a
number as they are given

•Routing
Guiding the respondent or interviewer in a questionnaire to the next relevant question
depending on the answer given
 Enhance response rate
• When writing questions and assembling the
final questionnaire, edit with a view towards
saliency: apparent relevance, importance,
and interest of the survey to the respondent
• Consider either pre-notifying those in your
sample or sending reminders to those who
received the survey (if self-administered).
Studies have shown that making contact with
the sampled individuals increases the
response rate.
• If possible, offer an incentive.

44
 Non-responders
• Understanding the characteristics of those
who did not respond to the survey is
important to quantify what, if any, bias exists
in the results.
• To quantify the characteristics of the non-
responders to postal surveys, it is suggested
that tracking the length of time it takes for
surveys to be returned is important. Those
who take the longest to return the survey are
most like the non-responders. This result
may be situation-dependent.

45
 Conclusions
• You need plenty of time!
– Design your questionnaire from research
hypotheses that have been carefully studied and
thought out.
– Discuss the research problem with colleagues and
subject matter experts is critical to developing
good questions.
– Review, revise and test the questions on an
iterative basis.
– Examine the questionnaire as a whole for flow
and presentation.

46
Thank you