QUESTIONNAIRE

CONSTRUCTION
FIND THE MISTAKE IN QUESTION

 Vendors that offer free upgrades always win higher satisfaction scores
than those that don't. How much more satisfied are you when a vendor
offers free upgrades?
1. Slightly more satisfied
2. Moderately more satisfied
3. More satisfied
4. Much more satisfied
5. Definitively more satisfied

 It’s a leading questions stating free upgrades win higher

satisfaction.
 All the options are positive with no negative option.
FIND THE MISTAKE IN QUESTION
 What is the primary reason your company will add new [type here]
partners? (Select all that apply.)

1. Company reputation for excellent staff

2. Product complements existing areas of focus
3. Offers high margins
4. Offers marketing services with highest co-op funds
5. Provides alternative product choices
6. Superior service in the form of sales support
7. Best technology in the market
8. Other (specify) ___________________________
 The text says "primary," but the question is structured as a multiple-
response question with instructions to "Select all that apply." The
word "primary" means "main"—and so, in the example, the one
reason or primary reason. If you want more than one answer, don't
use the word "primary."
FIND THE MISTAKE IN QUESTION

 According to you, what are the key factors that contribute to

the success of a company in the present-day poor economic
conditions?
(open ended question)

 Correct
What, in your opinion, are the key factors that contribute to the
success of a [type of] company in the present recessionary
economy? (Please briefly describe up to three.)
1. _____________________________
2. _____________________________
3. _____________________________
 FIND THE MISTAKE IN QUESTION

 Rank the following value-added services in terms or order of importance or

level of desirability beginning with the most important 1 meaning "most
important" through 5 meaning "least important."
Rank (1 to 5)
1. Access to demonstration equipment ____
2. Co-op marketing funds ____
3. Lead generation ____
4. Access to Web tools ____
5. Sales support ____

 How would you answer that question? It includes a typo (the first "or" should be
an "of") and two different ways in which to rank the services listed (by "order of
importance or level of desirability"), and, as if those problems weren't enough,
its instructions are awkwardly structured.
QUESTION REWORKED

 Rank the following value-added services in terms of

their importance to your organization's success, where
the most important is ranked 1, the next most important
is ranked 2, and the least important is ranked 5.
Rank from 1 to 5:
____ Access to demonstration equipment
____ Co-op marketing funds
____ Lead generation
____ Access to Web tools
____ Sales support
 FIND THE MISTAKE IN QUESTION
OUT OF THESE THREE QUESTIONS, ONE IS A LEADING QUESTION. CAN YOU SPOT
IT?

 1. How would you rate your experience?  This leading question is an

- Very good example of unbalanced scales,
- Good which is an easy mistake to
- Neutral
make. Question 3 only gives
- Poor
- Very poor
the respondent positive options
to choose from – there is no
 2. To what level do you agree with Statement
room for a neutral or negative
A?
- Strongly agree response. While anyone would
- Somewhat agree like to only get positive
- Neutral feedback, there is nothing
- Somewhat disagree beneficial in false data.
- Strongly disagree
 3. How would you rate our idea?
- Extremely good
- Very good
- Pretty good
- Good
- Okay
FIND THE MISTAKE IN QUESTION

 Torquay is one of the sunniest seaside resorts in the

United Kingdom. Would you consider going to Torquay
for a holiday?

 The first sentence in this question creates a positive

impression of Torquay and may influence responses to
the question. Leaving out the first sentence and just
asking “Would you consider going to Torquay for a
holiday” would remove this bias.
FIND THE MISTAKE IN QUESTION

What is occasionally to one person is going to be different to the next

survey respondent and what is regularly to another respondent could be
often to another. Instead, replace this ordinal scale with one with more
measurable answers: Never, 1-2 times, 3-5 times, 6-10 times, More than
10 times, Don't know.
QUESTION SEQUENCE
 Example: people may say "yes" to donate money if you ask the
questions this way

Do you love nature?

Will you donate money to help the river?

 But probably will say "no" if you ask the questions this way:

Is lack of money a problem for you?

Will you donate money to help the river?
FIND THE MISTAKE IN QUESTION

 "Do you love nature?“

 This is a bad question because it is almost forcing the person

to say "Yes, of course."

 Correct question:
"How important is the natural environment to you?"
Not Important
Some Importance
Very Important
FIND THE MISTAKE IN QUESTION

 Should concerned parents use infant car seats?

 The term “concerned parents” leads the respondent away

from the topic at hand. Instead, stay focused by only
including what is needed in the question.

 Do you think special car seats should be required for infant

passengers?
FIND THE MISTAKE IN QUESTION

 People working at ABC Corp love their jobs. How do

you feel about your job here at ABC Corp?

 It’s a leading question where first part suggests that

employees here love their jobs.

 On a scale from 1 to 5, with 1 being very dissatisfied and

5 being very satisfied, how satisfied are you with your
position at ABC Corp?
FIND THE MISTAKE IN QUESTION

 Most Americans prefer to purchase products manufactured

in the United States. Do you prefer to purchase products
manufactured in the United States?

 Better Question:
Where do you prefer products you purchase to be
manufactured?
FIND THE MISTAKE IN QUESTION

 How many times do you visit DMart?

Never Not often Fairly Often All the time

 During a typical quarter, how many times do you visit DMart?

1. Never
2. 1-3 times
3. About every other week
4. About once per week
5. 2 or more times
 FIND THE MISTAKE IN QUESTION
 The TAs in this course were helpful and responsive.
Strongly Disagree Neutral Agree Strongly
Disagree Agree

 Correct question:
Rate the performance of the TAs on the following dimensions.
1. Helping you understand the material

Poor 2 3 4 Excellent
1 5
2. Responding to inquires about your grade

Poor 2 3 4 Excellent
1 5
FEW OTHER EXAMPLES

 Watch out for loaded words that have a history of being

attached to extreme situations. For example, avoid
questions like "What should be done about murderous
terrorists who threaten the freedom of good citizens and
the safety of our children?“

 Avoid loading questions with extra adjectives and

adverbs, like "Should the mayor spend even more tax
money trying to keep the streets in top shape?"
FEW OTHER EXAMPLES

 Avoid leading questions like "You don't smoke, do you?"

or "I assume you would agree that the teachers do a heroic
job for our children".

 Watch for prestige markers that cue the respondent to give

the "right" answer. For example, the question "Most
doctors say that cigarette smoke causes lung disease for
those near a smoker. Do you agree", tends to provoke
"yes" answers because people trust doctors. Similarly, "Do
you support the president's policy on Zimbobutu?"
provokes "yes" answers from people who have never even
heard of Zimbobutu (and, in fact, I made the name up --
yet lot's of people will say "yes" to this question).