Questionnaire Design

What is a Questionnaire?
 A questionnaire consists of formalized and pre-specified set of
questions designed to obtain responses from potential
respondents.
 Questions in the questionnaire reflect the research objective
under investigation.
 Questionnaires are generally situation and culture specific.
 Questionnaire design process requires a careful attention to each
step as the questionnaire or research instrument should be
adapted to the specific cultural environment and should not be
biased in terms of any one culture (Malhotra et al., 1996).
 Questionnaire Design Process

 Designing of the questionnaire is a systematic process.

This section explores the systematic process of
questionnaire design in three phases: pre-construction
phase, construction phase, and post-construction
phase.
Figure 4.1: Steps in questionnaire design process
 Phase I: Pre-Construction Phase
 Phase I is the pre-construction phase of the questionnaire design
process. It consists of three steps: specific required information in
the light of research objective, an overview of respondent’s
characteristics, and decision regarding selecting an appropriate
survey technique.
Step 1: Specific Required Information
in the Light of Research Objective

 The first and the foremost important decision relates to finding

the required information through a questionnaire.
 To generate specific information, the researcher should clearly
define the objective and the other research components such as
theoretical model, research questions, and hypotheses.
 Clarity in all these components is very important as these only will
provide the base of specific information to be collected through
survey.
 Step 2: An Overview of
Respondent’s Characteristics
 While collecting the information, an overview of the respondent’s
characteristics is a vital consideration.
 A researcher must construct the questionnaire in the light of the
respondent’s profile.
 Even while constructing a questionnaire, a researcher has to keep
the profile of a probable respondent in mind.
 Various factors such as the respondent’s qualification, age,
experience, income, marital status, occupation, and so on have a
decisive and impactful role in conducting a research.
Step 3: Decision Regarding Selecting an
Appropriate Survey Technique
 While constructing a questionnaire, a deep thinking process is
required to select an appropriate survey technique.

 The questions in the questionnaire must be constructed,

sequenced, and placed according to the mode of the survey.
 Phase II: Construction Phase
 Phase II is the real construction phase of the questionnaire design
process.
 It consists of six steps:
 Decision regarding question format :structured questions versus
unstructured questions
 Decision regarding question relevance and wording
 Decision regarding question sequencing
 Decision regarding question response choice
 Decision regarding the questionnaire layout
 Producing first draft of the questionnaire.
Step 1: Decision Regarding Question
Format: Structured Questions Versus
Unstructured Questions
 Questionnaires use two types of question formats. These are
open-ended questions and closed-ended questions.
 In the case of a “closed-ended” question, the respondent has to
format the judgment to fit the response categories and when
“open-ended” questions are used, the judgment has to be
verbalized into a preliminary answer (DeLeeuw, 2001).
 The closed ended question format can be further divided into
dichotomous, multiple-choice questions, and scales. The
following sections focus on open-ended questions and closed-
ended questions.
 Open-ended Questions

One of the major limitations is to handle the interviewer and the

interpretation bias.
 Closed-ended Questions
 Closed-ended questions are structured questions. The question
structure does matter (Connolly et al., 2005).
 The closed-ended questions provide response alternative to the
respondents instead of giving them a free-to-express response
option.
 The choice offered to the respondents can be either in the form of
a rating system or a set of response alternatives.
 These response alternatives are presented to the respondents and
they select the most appropriate one.
 The closed-ended questionnaires are generally cheaper, more
reliable, and faster to code, and analyse the collected data
(Swamy, 2007).
 The closed-ended question format can be further divided into
dichotomous, multiple-choice questions, and scales.
 Dichotomous Questions
Dichotomous questions have only two response alternatives usually
presenting the two extremes “yes” or “no.” To make the alternatives
balanced, the researchers often present a third neutral alternative
“don’t know.”
Multiple-Choice Questions
Step 2: Decision Regarding Question
Wording

 The negative wordings impact the process of interpreting the

questions, leading at least some respondents to misinterpret how
to respond and, thus, reducing or destroying the usefulness of the
questions or the series of questions (Johnson et al., 2004).
 In some cases of conducting personal interviews, the researchers
feel that they will be able to overcome the wrong wording during
the discussion, but they suggest a different understanding.
 The effects of question wording on response accuracy did not
seem to be moderated by the respondent’s knowledge gained via
discussion (Bickart et al., 2006).
Figure 4.2: Decision parameters regarding question wording
Question Wordings Must Be Simple and
Easy to Understand
 Vague or Ambiguous Words
Must Be Avoided

Some words such as “often,” “occasionally” and “usually,” “how

long,” “how much,” and “reasonably well” may be confusing for a
respondent because these words specify a specific time frame.
 Double-Barrelled Questions
Must Be Avoided
Double-barrelled questions are those with wordings such as “and” or
“or.” In a double-barrelled question, a respondent may agree to one
part of the question but not to the other part.

√
Avoid Leading and Loaded Questions
A leading question is the one which clearly reveals the researcher’s
opinion about the answer to the question.

Identifying the loaded question bias in a question requires more

judgment because the wording elements in a loaded question allude
to the universal belief or rules of behaviour (Burns & Bush, 1999).
 Split-Ballot Technique
 A leading question generally emphasizes either the positive or
negative aspect of the question.
 Sometimes, it becomes necessary to ask a question with either the
positive or negative aspect.
 In this situation, a split-ballot technique is used to avoid bias due
to positive or negative aspect of the question.
 This technique involves the construction of a single question in
two alternative phrases, and the question based on one phrase is
administered to half of the respondents and the question based
on the other phrase is administered to the other half of the
respondents.
Split-ballot technique
 Avoid Using Overstated Words

The answer will always be overblown due to the first part of the
question, which generates a worry in the mind of the respondent and
results in a positive answer, which is not possible otherwise. A more
poised way of asking the same question is shown below.

√
Implied Assumptions Must Be Avoided

Above question has an implicit assumption that the discount policy

on bulk purchase offered by Company “A” is working excellent and by
answering “yes,” the company will continue its policy.

√
 Respondent’s Memory Should
Not Be Overtaxed

√
 Generalization and Estimation
Must Be Avoided

Generalization means respondent’s belief, “what must happen” or

“what should happen.”

√
 Respondent’s Ability to Answer
Must Be Considered
A question targeted to officers older than 55 years to assess the
importance of Internet banking is as follows:

Targeting following question to young respondents may not be an

appropriate choice.

×
Step 3:Decision Regarding Question
Sequencing
 Question sequence also plays a key role in generating the
respondent’s interest and motivation to answer the question.
Questions should have a logical sequencing in the questionnaire
and should not be placed abruptly.
 To facilitate the responses, a researcher has to follow some logical
steps in sequencing the questions in the questionnaire. This
arrangement usually requires the considerations as shown in
Figure 4.3.
 Figure 4.3: Decision parameters regarding question sequence

Decision regarding question sequencing

Identification Logical
Difficult to and order
Screening Opening Transition
answer categorizatio of
questions questions statements n questionin
questions
questions g
 Screening Questions
 Researchers generally begin with some screening questions to
make sure that the target respondent is qualified for the
interview.
 In some cases, when the researcher is very sure about the
qualification status of the respondent, he or she does not
incorporate the screening question and starts from some “opening
questions.”
 Opening Questions
 The opening questions should be simple, encouraging, and trust
building. From the research objective point of view, these
questions may sometimes be little irrelevant but should be good
initiators.
 These questions should not seek in-depth information and
should be as general as possible.
 For example, a microwave company, trying to assess “shift in
consumer attitude” from traditional way of cooking, should ask a
first opening question as follows:
 Transition Statements
 The movement from one set of questions to another set requires transition statements.
 For example, a mineral water bottle manufacturing company is encouraged with the
expanding market.
 The company wants to assess the potential future market and hence conducted a survey
on non-users.
 Its researchers have identified various variables to get the potential use, of which
“awareness” and “taste” are important.
 It has prepared the first 11 questions with the first 5 questions based on the
“awareness” and the next 6 questions on “taste.”
 After asking the first set of 5 questions, a researcher moves to the second set of 6
questions to get the potential consumer feeling for mineral water taste.
 Thus, before asking the next set of 6 questions, a transition statement is required to
make the respondent familiar with the coming questions.
 Hence, a transition statement “Now, I would like to understand your opinion about the
mineral water taste” will develop respondent’s connectivity for the next set of 6
questions related to “taste,” and he or she will be in a comfortable state of mind to
answer these questions.
 Difficult to Answer Questions
 Difficult to answer, sensitive, or complicated questions should be
placed later in the questionnaire.
 Placing it first will confuse the respondent and he or she will tend
to terminate the interview process.
 Asking difficult questions first in a telephone interview reduces a
respondent’s interest in the interview process and he or she tends
to terminate the interview. Under telephone interview conditions,
substantively related questions affect the responses to the target
question only when asked first (Schwarz & Hippler, 1995).
Identification and Categorization Questions

 Identification questions are used to generate some basic

identification information such as name, mailing address, office
phone number, personal phone number, or cell phone number.
 Categorization questions are mainly used to generate
demographic information.
 For example, researchers generally want to generate the
information related to age, experience, gender, and occupation of
the respondents.
 Logical Order of Questioning
 In a questionnaire, the questions must flow in a logical sequence.
There are at least three approaches to suggest the roadmap to
place the questions in a logical sequence; they are funnel
technique, work technique, and sections technique.
 Funnel technique suggests asking general questions first and then
the specific questions.
 Work technique suggests that difficult-to-answer, sensitive, or
complicated questions should be placed later in the
questionnaire.
 The third technique is the section technique in which questions
are placed in different sections with respect to some common
base.
Step 4: Decision Regarding Question
Response Choice

 It is important to understand that too many response choices will

burden the respondent and he or she will be perplexed while
answering.
 Few response choices will not be able to cover all ranges of
possible alternatives.
 As a general rule, the researchers present a question with five to
seven response alternatives.
 Step 5: Decision Regarding
Questionnaire Layout

 Questionnaire layout is important to enhance the response rate. A

recent study (Lagarace & Washburn, 1995) revealed that a user-
friendly format, and to some extent colour, is valuable to increase
mail survey response rate.
 The appearance of a questionnaire is particularly important in
mail surveys because the instrument, along with the preliminary
letter and/ or cover letter, must sell itself and convince the
recipient to complete and return it (Boser, 1990).
 It has been observed that the respondent emphasizes the
questions that are placed at the top of the questionnaire
compared with that at the bottom.
Step 6: Producing First Draft of the
Questionnaire

 Printing on a poor, quality paper or an unprofessional look of the

questionnaire may generate a non-serious feeling among the
respondents.
 So, the questionnaire may be printed on a good, quality paper
and must have a professional look.
 The appearance of the front cover on a mail questionnaire and the
nature of first questions have been purported to have an
important influence on the respondent’s decision to complete the
questionnaire (Frey, 1991).
Phase III: Post-Construction Phase
 Phase III is the post-construction phase of the questionnaire
design process. It consists of four steps:
 Pre-testing of the questionnaire
 Revisiting the questionnaire based on the inputs obtained from
the pre-testing
 Revising final draft of the questionnaire
 Administering the questionnaire and obtaining responses.
Step 1: Pre-Testing of the Questionnaire
 Pre-testing of the questionnaire involves administering the
questionnaire to a small sample of the population to identify and
eliminate the potential problems of the questionnaire, if any.
 Testing a questionnaire can be a time-consuming process, and this
stage of questionnaire has often been overlooked in the past in
researcher’s eagerness to start collecting data (Williams, 2003).
 Researchers generally use two common procedures to pre-test:
protocol analysis and debriefing.
Step 1: Pre-Testing of the Questionnaire
(Cont.)
 Using protocol analysis, a researcher asks the respondent to “think aloud”
while answering the question.
 Debriefing is an interview conducted when a respondent has filled the
questionnaire.
 After completing, the respondents are informed that the questionnaire
they have just filled was a pre-test questionnaire and are requested to
share their views about various dimensions of the questionnaire.
 They are also requested to find out the problems with the questionnaire,
which they realized while filling it.
 While taking a decision about the sample size for pre-testing, as a thumb
rule, a researcher should keep the nature of population diverse in mind.
 More heterogeneity in population requires relatively large size of the
sample compared with the situation when population is relatively
homogeneous.
Step 2: Revisiting the Questionnaire Based
on the Inputs Obtained from Pre-Testing

 To enhance the accuracy, after incorporating suggestions from the

pre-testing, a researcher can go for second pre-testing.
 It is always possible that the second pre-testing may also reveal
some of the problems of the questionnaire.
 Pre-testing of the questionnaire might have provided many inputs
in a subjective manner.
 At this stage, the researcher must objectively incorporate all the
inputs obtained from the pre-testing exercise.
 All the parameters related to the question wording (as discussed
in the previous sections) must be carefully considered.
 Step 3: Revised Final Draft of the
Questionnaire

 At this stage, the researcher once again carefully examines the

questionnaire.
 Insertions and deletions of the previous stage must be re-checked
to provide the desired accuracy.
 The researcher makes the questionnaire “ready to administer” by
eliminating all the minute mistakes and tries to make it error free.
 After careful examination of all the incorporations obtained from
pre-testing, the researcher is now ready to have the final draft of
the questionnaire and administers to the sample taken from a
target population.
Step 4: Administration of the Questionnaire
and Obtaining Responses

 At this stage, the researchers administer the questionnaire to the

respondents and obtain the responses.
 These responses are coded, data are tabulated, and appropriate
statistical techniques are applied to analyse the data.