Composing Questions

BUILDING QUESTIONNAIRES
(CH. 9)
Agenda and objectives
2
 Three modules on questionnaire design:

 1. How do you build a questionnaire
(overview)?
 2. How do you determine level of
measurement and scale types?
 3. What are problems respondents have in
responding to questions?
Where we are:
Stage 3: Design Data-Collection Method
and Forms
3
Formulate Problem
Determine Research Design
Design Data-Collection Method and

Forms
Sampling and Data Collection
Analyze and Interpret Data
Prepare and Present Research Report

Questionnaire
4
 Conversation between researchers and

respondents
 Impossible to get good information from bad

questions
 To minimize response errors, the
questionnaire must be crafted in accordance
with best practices
5 Objective
1. How do you build a

questionnaire (overview)?
Procedure for Developing a
Questionnaire
Step 1 Specify what information will be sought
Step 2 Determine type of questionnaire and method of administra
Step 3 Determine content of individual questions
Step 4 Determine form of response to each question
Step 5 Determine wording of each question
Determine question sequence

Step 6
Determine physical characteristics of questionnaire

Step 7
Reexamine steps 1-7 and revise if necessary

Step 8
Pretest questionnaire and revise if necessary

Step 9
Questionnaire Design
7
 Questionnaire development is a process

with a sequence of steps;
 Starts with linking research problems to
information to be sought in questionnaire
Research
Problems Variables Hypotheses
Information to
Guided by theory be collected in
questionnaire
example
8
 Decision problem:
 Sell a home food in the dormitory or not?
 If you can get enough profit, you can do it!
 Research problem:
 What’s the target market?
 Demographic information
 Demand of the product?
 Purchase intention
 What’s the price?
 Cost
 What kind of the food they would like to buy?
 Detail about the product. E.g. packet, size, etc.
 Time
 Promotion?
 …..
Questionnaire

Step 6

Step 7

Step 8

Step 9
Step 1 – Information Sought
10
 Clearly define your research problems

 Make a list of key research questions
 Search for existing questions
 Identify your target population (respondents)
 Specify variables/constructs to study
 Hypothesize relationships between key

variables
Example: Coors Distributorship
case
11
 Research problem:
 Whether it’s profitable in the first year?
 Target population:
 Beer consumers in two countries of Delaware
 Variables/construct to be studied:
 Demand for Coors beer
 Price of Coors beer
 Cost
 Relationship between variables:
 If Demand * Price > Cost, then it’s profitable.
Questionnaire
Step 2 Determine type of questionnaire and method of adm

Step 6

Step 7

Step 8

Step 9
Step 2 -Determine Type of
Questionnaire and Method of
13 Administration
 Determine Type of Questionnaire and
Method of Administration
 Type of Questionnaire: the structure and disguise
 Method of Administration: mail, telephone,
personal interviews, fax, email, or Web
 Considerations
 The research topic
 The type of data to be collected
 Flexibility, cost, speed, response rate, and more…
Determine Survey
14
Method
 Mail
 Telephone
 Personal
 In-Home
 Mall Intercept
 Electronic
 Email
 Web Survey
Example: Effect of Method on
15
Mail/E-mail Questionnaire
 Please rank order the following department stores in order of
your preference to shop at these stores. Begin by picking out the
one store that you like most and assign it a number
1. Then find the second most preferred department store and
assign it a number
2. Continue this procedure until you have ranked all the stores in
order of preference. The least preferred store should be assigned a
rank of 10. No two stores should receive the same rank number.
Store Rank Order

1. Lord & Taylor ____________
2. Macy's ____________
.
.
10. Wal-Mart ____________
16
Telephone Questionnaire
 I will read to you the names of some department stores. Please rate them in
terms of your preference to shop at these stores. Use a ten point scale, where
1 denotes not so preferred and 10 denotes greatly preferred. Numbers
between 1 and 10 reflect intermediate degrees of preference. Again, please
remember that the higher the number, the greater the degree of preference.
Now, please tell me your preference to shop at .......(READ ONE STORE AT A
TIME)

Store Not So Greatly
Preferred Preferred
1. Lord & Taylor 1 2 3 4 5 6 7 8 9 10
2. Macy's 1 2 3 4 5 6 7 8 9 10
.
.
10. Wal-Mart 1 2 3 4 5 6 7 8 9 10
17
Personal Interview Questionnaire

 (HAND DEPARTMENT STORE CARDS TO THE RESPONDENT). Here is a set of
department store names, each written on a separate card. Please examine these
cards carefully. (GIVE RESPONDENT TIME). Now, please examine these cards again
and pull out that card which has the name of the store you like the most, i.e., your
most preferred store for shopping. (RECORD THE STORE NAME AND KEEP THIS CARD
WITH YOU). Now, please examine the remaining nine cards. Of these remaining nine
stores, what is your most preferred store for shopping? (REPEAT THIS PROCEDURE
SEQUENTIALLY UNTIL THE RESPONDENT HAS ONLY ONE CARD LEFT)

Store Rank Name of the Store
1. 1 __________________
2. 2 __________________
.
10. 10 __________________
18
 Can you see the difference between

different methods?
 What would be the appropriate method?
 Considerations
 The research topic
 The type of data to be collected
 Flexibility, cost, speed, response rate, and
more…
Questionnaire

Step 6

Step 7

Step 8

Step 9
Step 3 – Content of Questions
20
 Questions to ask yourself

 Is the question necessary?
 Are several questions needed for the

variable/construct?
 Are respondents able to answer the question?
 Are respondents willing to answer the

question?
21

 If there is no satisfactory use for the data resulting
from a question, that question should be eliminated.

variable/construct?

question?
22


variable/construct?

question?
Individual Question Content
Are Several Questions Needed Instead of One?
23
 Sometimes, several questions are needed to obtain the

required information in an unambiguous manner. Consider
the question,
(Incorrect)
“Do you think Coca-Cola is a tasty and refreshing soft drink?”
 Such a question is called a double-barreled question, because

two or more questions are combined into one. To obtain the
required information, two distinct questions should be asked:
“Do you think Coca-Cola is a tasty soft drink?” and

“Do you think Coca-Cola is a refreshing soft drink?”
(Correct)
24


variable/construct?
 Are respondents able to answer the

question?

question?
Overcoming Inability To Answer
Is the Respondent Informed?
25
 In situations where not all respondents are likely to be

informed about the topic of interest, filter questions
that measure familiarity and past experience should
be asked before questions about the topics
themselves.
 Respondents can’t know/remember everything.

 How much do you support pluripotent stem cells research?
 How much electricity (kWh) did you use last year?
 A “don't know” option appears to reduce uninformed

responses without reducing the response rate.
Can the Respondent Remember?
26
 How many gallons of soft drinks did you

consume during the last four weeks?
(Incorrect)
 How often do you consume soft drinks in a

typical week?
a. ___ Less than once a week
b. ___ 1 to 3 times per week
c. ___ 4 to 6 times per week
d. ___ 7 or more times per week
(Correct)
Can the Respondent Articulate?
27
 Respondents may be unable to articulate certain

types of responses
 (e.g., describe the atmosphere of a department store)
 Respondents should be given aids, such as

pictures, maps, and descriptions to help them
articulate their responses.
28


variable/construct?

question?
Overcoming Unwillingness To Answer
Effort Required of the Respondents
29
 Effort
 Most respondents are unwilling to devote a lot
of effort to provide information.
Example: Overcoming Unwillingness To
Answer
30
 Please list all the departments from which you purchased

merchandise on your most recent shopping trip to a
department store.
(Incorrect)
In the list that follows, please check all the departments from
which you purchased merchandise on your most recent
shopping trip to a department store.
1. Women's dresses ____

2. Men's apparel ____
3. Children's apparel ____
4. Cosmetics ____
.
.
.
16. Jewelry ____
31
 Legitimate Purpose
 Explaining why the data are needed can make
the request for the information seem legitimate
and increase the respondents' willingness to
answer.
 Sensitive Information
 Respondents are unwilling to disclose, at least
accurately, sensitive information because this
may cause embarrassment or threaten the
respondent's prestige or self-image.
Increasing the Willingness of Respondents
32
 Some tips:
 Place sensitive topics at the end of the
questionnaire.
 Preface the question with a statement that the
behavior of interest is common. e.g. HBV
 Ask the question using the third-person technique:
phrase the question as if it referred to other
people. e.g. overdraft
 Provide response categories rather than asking for
specific figures.
 Use less threatening words in questions.
Example:
Please Answer the Question
33
Have you ever committed the crime of shoplifting?

yes no
Example:
34
 “Have you ever committed the crime of

shoplifting?” (Incorrect)
 Use less threatening terminology

 Have you ever taken something from a
store without paying for it? (Correct)
 As you may know, every person’s

circumstances are different. Some people
shoplift from stores all the time, some
people occasionally shoplift, some people
have tried it once or twice, and others (Correct)
have
Example:
35
 Ask from a range of responses

 I have shoplifted...
(Correct)
 Never
 Only once or twice
 Three to five times
 Six to twenty times
 More times than I can count
Effects:
people know that some people may not be in the never cate
nge is less precise so less threatening to answer.
36


variable/construct?

question?
Questionnaire

Step 6

Step 7

Step 8

Step 9
Step 4 – Form of Response
38
 Format of the Question/Type of Instrument

 Open-ended
 Close-ended
 Categorical
 Dichotomous
 Scales
 Format of the Data/Level of Measure
(Ch.10)
 Nominal
 Ordinal
 Interval
 Ratio
39
Open-Ended Questions Close-Ended Questions

 Respondents are free  Standardized questions
to express views and answers
 Time-consuming
 Mentally less
demanding
 Mentally more  Bias from response
demanding
order, choice list
 Coding is subjective  Easy and fast to analyze
and difficult  Objective (no bias from
coder)
example
40
Have you ever taken something from a store

without paying for it?
Open-ended
 Have you ever committed the crime of shoplifting?
yes no Dichotomous
 I have shoplifted...
 Never
 Only once or twice
 Three to five times
 Six to twenty times
 More times than I can count
Scale
41
 Format of the question: Open-ended

versus Closed
 Use closed response whenever possible
 Ask open questions whenever you cannot be
sure of the universe of possible answers to a
categorical question.
 The only way to be sure you know the universe of
possible answers is to pretest.
 If open-ended questions are necessary, give
respondents a frame of reference
42
 Measurement Scales Selection

 Use established measurement scales if
possible
 Develop your own measurement scales
 Define conceptually what the concept
(construct) is (is not).
 Define operationally what will be the
indicator of the concept.
 Test this operational measure’s reliability
and validity.
Questionnaire

Step 6

Step 7

Step 8

Step 9
Step 5 - Wording
44
 (a) Use simple words & short sentences

 (b) Avoid ambiguous words and questions
(occasionally, sometimes, often, etc. Try to
anchor each point with a specific description)
 (c) Avoid leading questions
 (d) Avoid implicit alternatives
 (e) Avoid implicit assumptions
 (f) Avoid generalizations and estimates
 (g) Avoid double-barreled questions
 (h) Avoid response order (i.e., position) bias
Example:
Flame-broiled or Fried?
45
 Original Question: “Do you prefer your hamburgers flame-broiled

or fried?”
 Result: flame-broiling (Burger King) beat frying (McDonald’s) by
a 3-1 margin
 Revised Question #1: “Do you prefer a hamburger that is grilled

on a hot stainless-steel grill or cooked by passing the raw meat
through an open gas flame?”
 Result: 53% preferred the grill (McDonald’s)
 Revised Question #2: “The chain that grills on a hot stainless-

steel griddle serves its cooked hamburgers at the proper
temperature without having to use a microwave oven. And the
chain that uses the gas flame puts the hamburgers after they are
cooked into a microwave oven before serving them. Just knowing
this, from which of these two chains would you prefer to buy a
hamburger?”
Choosing Question Wording
Define the Issue
46
 Define the issue in terms of who, what,

when, where, and how (the five Ws). Who,
what, when, and where are particularly
important.
 Example:
Which brand of shampoo do you use?
(Incorrect)
Shampoo Example: What is wrong?
47
The W's Defining the

Question
Who The Respondent
It is not clear whether this
question relates to the
individual respondent or the
respondent's total household.
What The Brand of Shampoo
It is unclear how the
respondent is to answer this
question if more than one
brand is used.
When Unclear
The time frame is not specified
in this question. The
respondent could interpret it
as meaning the shampoo used
this morning, this week, or
over the past year.
Where At home, at the gym, on the
road?
Shampoo Example
48
 Which brand of shampoo do you use?

(Incorrect)
 Which brand or brands of shampoo have you

personally used at home during the last one month?
In case of more than one brand, please list all the
brands that apply.
(correct)
Use Ordinary Words
49
“Do you think the distribution of soft drinks is

adequate?”
(Incorrect)
“Do you think soft drinks are readily available

when you want to buy them?”
(Correct)
Use Unambiguous Words
50
In a typical month, how often do you shop in department

stores?
_____ Never
_____ Occasionally
_____ Sometimes
_____ Often
_____ Regularly
(Incorrect)
In a typical month, how often do you shop in department

stores?
_____ Less than once
_____ 1 or 2 times
_____ 3 or 4 times
_____ More than 4 times
(Correct)
Avoid Leading or Biasing Questions
51
 A leading question is one that clues the

respondent to what the answer should be, as
in the following:

Do you think that patriotic Chinese should buy imported
automobiles when that would put Chinese labor out of work?
_____ Yes
_____ No
_____ Don't know
(Incorrect)
Do you think that Chinese should buy imported automobiles?

_____ Yes
_____ No
_____ Don't know (Correct)
Avoid Implicit Alternatives
52
 An alternative that is not explicitly expressed in

the options is an implicit alternative.

1. Do you like to fly when traveling short
distances?
(Incorrect)
2. Do you like to fly when traveling short

distances, or would you rather drive?
(Correct)
Avoid Implicit Assumptions
53
 Questions should not be worded so that the

answer is dependent upon implicit assumptions
about what will happen as a consequence.

1. Are you in favor of the university providing
every student with a personal computer?
(Incorrect)
2. Are you in favor of the university providing

every student with a personal computer even
though it will mean a substantial increase in
tuition?
(Correct)
Overgeneralizations
 Don’t ask respondents to generalize too
much.
 When you buy “fast food”, what

percentage of the times do you order each
of the following type of food? (Incorrect)
 Of the last 10 times you bought “fast 54
food”, how many times did you order each

(Correct)
type of food?
Double-barreled Questions
 Do you regularly take vitamins to avoid

getting sick? (Incorrect)
 Do you regularly take vitamins? Why or

why not?
(Correct)
55
Response Bias (next week)
 Response Bias:
 The type of bias introduced because of the
mentality or predispositions of respondents
 Social Desirability
 Yea- and Nay-saying
 Prestige
 Threat
 Mental Set
 Order 56
Questionnaire

Step 6

Step 7

Step 8

Step 9
Step 6- Question Sequence
58
 Question sequencing
 Use simple interesting opening questions
 Use the funnel approach, ask broad questions
first
 Carefully design branching questions
 Ask classification information last
 Place difficult or sensitive questions near the
end
Determine Question Sequence
Location Type Examples Rationale
Screeners Qualifying “Have you been snow To identify target respondents

questions boarding in the past twelve (snowboard owners who have
months?” snowboarded in the past year).
First few Warm-ups “What brand of snowboards Easy to answer. Shows respondents questions do you own?”
survey is simple.
First third of Transitions “What feature do you Relate to research objectives, questions
/focus questions like best about snowboards?” slightly more effort needed to answer.
Middle Difficult and Following are ten Respondent has committed to

complicated characteristics of snow completing survey and can see
boards. Please rate your that just a few questions are left.
snowboards on each
characteristic
using the scale below.
Last section Demographic “What is the highest level Some questions may be considered
informationof education you have” “personal” and respondent may leave
attained?’ them blank, but they are at the end of the survey.
Determining the Order of
60
Questions
Effect on Subsequent Questions
 Funnel sequence
 General questions should precede the specific questions.

Q1: “What considerations are important to you in
selecting a department store?”
Q2: “In selecting a department store, how

important is convenience of location?”
Questionnaire

Step 6

Step 7

Step 8

Step 9
Step 7- Physical
62
Characteristics
 Physical appearance can be important for
participation and completion
Step 7- Physical
63
Characteristics
 Three Parts: Introduction, Main Body, Conclusion
 Introduction
 Thank the respondent
 Who you are and why you are doing research
 Confidentiality
 Length and topic
 Main Body
 Sensitive Questions and demographics at the end
 Conclusion:
 Thank the respondent again
Step 7- Physical
64
Characteristics
 Professional Quality
 Three Parts: Introduction, Main Body,
Conclusion
 Number questions and pages
 State how the responses are to be reported,
e.g., check mark, number, circle, etc.
 Check for typos
Questionnaire

Step 6

Step 7

Step 8

Step 9
Step 8 – Re-examine and
66
revise
Re-examine Steps 1-7 and Revise if
Necessary
 Read the questionnaire word by word
 Get peer evaluations of the draft

Questionnaire

Step 6

Step 7

Step 8

Step 9
Step 9 - Pretest
68
 Pretest first using personal interviews among targeted

respondents
 interviewers can observe respondents' reactions and
attitudes.
 Obtain comments from interviewers and respondents
 Pretest the questionnaire to uncover problems unique to

the mode of administration
 Code the pretest responses to determine if more/less

information is needed
Questionnaire

Step 6

Step 7

Step 8

Step 9
Review
70
 Three modules on questionnaire design:

 1. How do you build a questionnaire
(overview)?
 2. How do you determine level of
measurement and scale types? -> Next
class
 3. What are problems respondents have in
responding to questions? -> Next class

Composing Questions

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Composing Questions

Uploaded by

Copyright:

Available Formats

BUILDING QUESTIONNAIRES

 Three modules on questionnaire design:

Determine Research Design

Design Data-Collection Method and

Sampling and Data Collection

Analyze and Interpret Data

Prepare and Present Research Report

 Conversation between researchers and

 Impossible to get good information from bad

1. How do you build a

Step 2 Determine type of questionnaire and method of administra

Step 3 Determine content of individual questions

Step 4 Determine form of response to each question

Step 5 Determine wording of each question

Determine question sequence

Determine physical characteristics of questionnaire

Reexamine steps 1-7 and revise if necessary

Pretest questionnaire and revise if necessary

 Questionnaire development is a process

Step 2 Determine type of questionnaire and method of administra

Step 3 Determine content of individual questions

Step 4 Determine form of response to each question

Step 5 Determine wording of each question

Determine question sequence

Determine physical characteristics of questionnaire

Reexamine steps 1-7 and revise if necessary

Pretest questionnaire and revise if necessary

 Clearly define your research problems

 Identify your target population (respondents)

 Specify variables/constructs to study

 Hypothesize relationships between key

Step 2 Determine type of questionnaire and method of adm

Step 3 Determine content of individual questions

Step 4 Determine form of response to each question

Step 5 Determine wording of each question

Determine question sequence

Determine physical characteristics of questionnaire

Reexamine steps 1-7 and revise if necessary

Pretest questionnaire and revise if necessary

Personal Interview Questionnaire

 Can you see the difference between

Step 2 Determine type of questionnaire and method of administra

Step 3 Determine content of individual questions

Step 4 Determine form of response to each question

Step 5 Determine wording of each question

Determine question sequence

Determine physical characteristics of questionnaire

Reexamine steps 1-7 and revise if necessary

Pretest questionnaire and revise if necessary

 Questions to ask yourself

 Are several questions needed for the

 Are respondents able to answer the question?

 Are respondents willing to answer the

 Questions to ask yourself

 Are several questions needed for the

 Are respondents able to answer the question?

 Are respondents willing to answer the

 Questions to ask yourself

 Are several questions needed for the

 Are respondents able to answer the question?

 Are respondents willing to answer the

 Sometimes, several questions are needed to obtain the

 Such a question is called a double-barreled question, because