MIT 203: Lesson 3

Designing a Questionnaire
Introduction
Introduction
• There are many ideologies when coming up with good
questionnaires.
• Many books have focused on market research, however, a lot of
research in Africa consider opinions etc.
• In this section we will consider the guidance through a book by Paul
Hague (A practical Guide to Market Research)
Outcome
• By the end of this session you will be able to:
• State and explain key principles of designing effective questionnaires.
• Formulate meaningful questions.
• Distinguish between use of structured, semi-structured and unstructured
questionnaires in different types of research design
• Design the three most important types of questions for asking about
behaviour, attitudes or classifying respondents
• Explain key terms used in questionnaire design
• Conceptualize the link between the interviewer, the respondent and the
questionnaire.
Questionnaire Definition
A research instrument that consists of a set of questions or other types
of prompts that aims to collect information from a respondent.
A research questionnaire is typically a mix of close-ended questions
and open-ended questions. Open-ended, long-form questions offer the
respondent the ability to elaborate on their thoughts. Research
questionnaires were developed in 1838 by the Statistical Society of
London. (cited from QuestionPro)
Survey Versus Questionnaire
Survey Questionnaire
• Process of collecting and analyzing • The instrument of data collection
that data
• Consists of questionnaire and survey • Subset of survey
design, logic and data collection
• A questionnaire can is a research
• A survey is a research method used instrument that consists of a set of
for collecting data from a pre-defined questions to collect information from
group of respondents to gain a respondent
information and insights on various
topics of interest.
Advantages of having a well-designed
questionnaire
• One can gather a lot of data in less time.
• There is less chance of any bias creeping if you have a standard set of questions
to be used to your target audience.
• One can apply logic to questions based on the respondents’ answers, but the questionnaire
will remain standard for a group of respondents that fall in the same segment.
• The questionnaire can be easily converted to online survey software which are is
quick and cost-effective.
• It offers you a rich set of features to design, distribute, and analyze the response data.
• Responses can be compared with the historical data and understand the shift in
respondents’ choices and experiences.
• Respondents can answer the questionnaire without needing an interviewer
depending on mode of administration
Types of Questionnaires
Types of Questionnaires
• Generally can be categorized into two types
1. Structured Questionnaires: that collect quantitative data.
• Are planned and designed to gather precise information.
• They also initiates a formal inquiry, supplements data, checks previously
accumulated data, and helps validate any prior hypothesis.
2. Unstructured Questionnaires: that collect qualitative data.
• Questions are more open-ended to collect specific data from participants.
• They use a basic structure and some branching questions but nothing that
limits the responses of a respondent.
Types of questions in a questionnaire
• The structured questionnaires have multiple types of questions. The type
of question is categorized by the eventual feedback from the responded.
They can be:
1. Dichotomous: that is generally a “YES/NO”.
• This question is usually used in case of the need for necessary validation. It is the
most natural form of a questionnaire
2. Multiple-Choice: that one selects response(s) from a given list of options.
• They are further classified as:
• Single-select multiple-choice question: in which a respondent has to select one, or
• Multi-select multiple choice question (Multiple response): in which a respondent
can select more than one among the list.
Types of questions in a questionnaire
• The structured questionnaires have multiple types of questions. The type
of question is categorized by the eventual feedback from the responded.
They can be:
3. Scaling: that are based on the principles of the four measurement scales
- nominal, ordinal, interval, and ratio.
• Question types that utilize these scales’ fundamental properties include:
• Rank order: that allow respondents to rearrange and rank multiple-choice options in a
specific order (read more)
• Likert scale: which is a psychometric scale where questions choices are based on 'agreeing' or
'disagreeing' on a particular survey question. (read more)
• Semantic differential scale: where products etc are rated within the frames of a multi-point
rating option that are grammatically on opposite adjectives at each end, e.g: love-hate,
satisfied-unsatisfied (read more)
• Stapel scale: where rating scale is with a single adjective (unipolar), developed to gather
respondent insights about a particular subject or event. E.g scales like +2 to -2 or +5 to -5
(read more)
Types of questions in a questionnaire
• The structured questionnaires have multiple types of questions. The
type of question is categorized by the eventual feedback from the
responded. They can be:
4. Pictorial: questions where the answer choices are images.
• Helps respondents choose an answer quickly without over-thinking their
answers, giving you more accurate data.
5. Open-Ended: where the respondent can answer in a free form with
little to no restrictions

Note: The types of questions 1 – 4 are applied in structured

questionnaires while 5 only is used in both
Open-ended questions
Merits
• Freedom and spontaneity of
answers
• Opportunity to probe
• Useful for testing hypotheses
about ideas or awareness
Demerits
• Time Consuming
• In interviews: Costly of
interviewers time
• Coding: very costly and slow to
process, and maybe unreliable
• Demands more effort from
respondent
Close-ended questions
Merits Demerits
• Require little time • Loss of spontaneous responses
• No extended writing • Bias in answer categories
• Low costs • Sometimes too crude
• Easy to process • May irritate respondents
• Makes group comparisons easy
• Useful for testing specific
hypotheses
• Less interviewer training
Principles of a good
questionnaire design
Characteristics of a good questionnaire
• Uniformity

• Exploratory

• Question Sequence
 Characteristics of a good questionnaire
(Uniformity)
• The essence of Uniformity is where every respondent sees the same
questions.
• In most cases, questionnaires collect demographic information, personal opinions,
facts, or attitudes from respondents.
• Uniform design and standardization is among most significant attributes .
• This helps in data collection and statistical analysis of the data.
• In addition, it can help facilitate the replication of the same study (and data analysis)
• Example:
• A retail store evaluation questionnaire template contains questions for evaluating
retail store experiences. Questions relate to purchase value, range of options for
product selections, and quality of merchandise. These questions are uniform for all
customers.
 Characteristics of a good questionnaire
(Exploratory)
• This is allowing probing of insightful information that are naturally not
collected through quantitative data
• Exploration (probing) is usually done using open ended questions.
• There is no restriction on questions that can be in your questionnaire.
• Example
• Use a data collection questionnaire and send it to the female of the
household to understand her spending and saving habits relative to the
household income. Open-ended questions give you more insight and allow
the respondents to explain their practices. A very structured question list
could limit the data collection.
 Characteristics of a good questionnaire
(Exploratory)
Close-ended question Open-ended question
• Do you like working with us? • Tell us about your experience
[Yes/No] with our organization so far.
• Have you been stressed lately? • Share with us what has been
[Yes/No/Unsure] troubling you.
• How satisfied are you with your • What do you expect from this
current job role? appraisal?
• [Very satisfied/ Somewhat
satisfied/ Somewhat unsatisfied/
Very unsatisfied]
 Characteristics of a good questionnaire
(Question Sequence)
• This is the structured flow of questions to increase the number of
responses.
• This sequence of questions is:
• screening questions
• warm-up questions
• transition questions
• skip questions
• challenging questions, and
• classification questions.
• Example,
• In a motivation and buying experience questionnaire template covers (1) initial
demographic questions, (2) asks for time spent in sections of the store, and (3) the
rationale behind purchases.
 Characteristics of a good questionnaire
(Question Sequence)
• Within a section in a questionnaire, two methods for ordering can be
applied.
• Funnel Approach:
• One starts with very broad questions the progressively narrows down the scope of the
question until finally it focusses on very specific points
• Filter:
• Used for excluding some respondents from a particular set of questions in case they
don’t apply to them
Principles of designing effective
questionnaires
• This is considers on a chronological order on when what should be
done when designing a questionnaire. They include:
• Step 1: Decide what information is required
• Step 2: Make a rough listing of the questions
• Step 3: Refine the question phrasing
• Step 4: Develop the response format
• Step 5: Put the questionnaires into an appropriate sequence
• Step 6: Finalize the layout of the questionnaire
• Step 7: Pretest and revise
Principles of designing effective
questionnaires
• Step 1: Decide what information is required
• One should fully comprehend the context.
• First, the researcher to goes through the proposal/brief
• Second, list all the objectives and what information is required in order that
they are achieved.
• Step 2: Make a rough listing of the questions
• List is all the questions that could go into the questionnaire.
• Aim to be as comprehensive as possible.
• Don’t focus a lot on phrasing of the questions.
Principles of designing effective
questionnaires
• Step 3: Refine the question phrasing
• Consider the tone and grammar used.
• The questions must now be developed close to the point where they make
sense and will generate the right answers. by Paul Hague.
• Step 4: Develop the response format
• Responses are what the interviewee/respondent will answer and how it will
be recorded.
• Responses could be a pre-coded list of answers (Close-ended) or it could be
open-ended to collect verbatim comments.
Principles of designing effective
questionnaires
• Step 5: Put the questionnaires into an appropriate sequence
• Consider the logic flow for the interview and order the questions
• Normally the respondent is eased into the task with relatively straightforward
questions while the more difficult or sensitive ones are left until they are warmed up.
• Questions on brand awareness are asked first unprompted and then they are
prompted.
• Step 6: Finalize the layout of the questionnaire
• Format the questionnaire with clear instructions to the interviewer. Include an
appropriate introduction, routings and probes.
• Ensure there is enough space to write in answers and the response codes need to be
well separated from each other so there is no danger of circling the wrong one.
Principles of designing effective
questionnaires
• Step 7: Pretest and revise
• The questionnaire should be piloted using the interviewing method that will
be used in the field (over the phone if telephone interviews are to be used;
self completed if it will be a self completion questionnaire).
• Carry out between 10 and 20 interviews in a pilot. The aim is to make sure
that it works, and not to obtain pilot results.
• Time and money may hinder a proper pilot so at the very least it should be tested on one
or two colleagues for sense, flow and clarity of instructions.
• The whole purpose of the test is to find out if changes are needed so that
final revisions can be made.
• When carrying out the pilot it is best to run through the questionnaire with the guinea
pig respondent and then go back over the questions and ask for each one, what went
through your mind when you were asked this question?.
Sectioning the questionnaire
Sectioning the questionnaire
• Sections help to avoid mixing the information being collected.
• It is like point (6 and 7) in the principles of a good questionnaire.
• The sections can be prepared through themes etc
• Sections can be many, but some most important include:
Sectioning the questionnaire
• An introduction: this is the interviewer’s request for help.
• The interview should not proceed without informed consent from the
respondent
• It is normally scripted and lays out the credentials of the market research
company, the purpose of the study and any aspects of confidentiality
• The respondent’s identification data: such as their name, address,
date of the interview, name of the interviewer.
• The questionnaire would also have a unique identifier for purposes of
entering the data into the computer. Why is this important?
Sectioning the questionnaire
• Instructions: the interviewer and the respondent need to know how
to move through the questionnaire such as which questions to skip
and where to move to if certain answers are given.
• Information: this is the main body of the document and is made up
of the many questions and response codes.
• The
• Classification data: these questions, sometimes at the front of the
questionnaire, sometimes at the end, establish the important
characteristics of the respondent, particularly related to their
demographics
The persons to consider when
designing a questionnaire
Designing effective questionnaires
For questionnaires to be effective, one needs to consider the context of
the persons whom they will work with. They include:
1. Client:
• He wants answers to their particular problem and even, on occasion, to have
their worst fears shown up to be unlikely or improbable. (Funder)
2. Researcher:
• Hired to uncover information and balance the needs of three groups of
people.
• He needs to ensure that the interviewer can manage the questionnaire easily,
that the questions are interesting for the respondent and that the
questionnaire matches the client’s needs
Designing effective questionnaires
For questionnaires to be effective, one needs to consider the context of the
persons whom they will work with. They include:
3. Interviewer:
• Wants a questionnaire which is easy to follow and which can be completed in the
time specified by the researcher.
4. Respondent:
• They respond to the questions the researcher formulated.
• Hence, they want to enjoy the interview experience.
• They need to feel that the questions are phrased so that they can be answered
truthfully, and so that they allow the respondent to actually say what he or she
thinks.
• They may also want to know if they will receive anything in return for giving their
opinion.
Designing effective questionnaires
For questionnaires to be effective, one needs to consider the context of
the persons whom they will work with. They include:
5. Data-processor (Analyst/Manager):
• Looks forward to a questionnaire which will result in data which can be
processed efficiently and with minimum error.
• Usually, its recommended to have more clo
Wording the quesitions
Questions wording
• Consider the following dichotomous type questions:
Do you approve or disapprove of people that go for football matches?
[YES/NO]
• This is an unanswerable question
• The answer categories are not appropriate for the form of the question –
use [Approve/Disapprove]
• It is too broad. There are many complexities in people’s attitudes about
going for football matches
• It is leading [Approve is before Disapprove]
• Has not considered a case of [Not Applicable]
Questions wording [Some rules]
• Avoid double barreled questions: