4. Questionnaire design.

Errors in questionnaire design with example

Most experienced researchers and consultants can put together an effective

survey instrument, but even then, small issues can creep into the
questionnaire that can compromise quality, introduce bias or lead to
misleading results. While many of the common mistakes listed here may seem
obvious, it is always good practice to review a questionnaire with these
potential issues in mind.

Here are some of the common questionnaire mistakes market researchers

should avoid:

Mistake #1: Leading the Respondent

This issue results when an adjective is used that can cause bias. While often
unintentional, it is sometimes seen in political polling when trying to provoke a
certain reaction to an issue. When it occurs in market research surveys, it
tends to inflate ratings in aspects like customer satisfaction or product
research.

Examples:
#1: Our dedicated, top-rated customer service representatives do their best to
serve your needs. How satisfied are you with your experience today?
[SATISFACTION SCALE]

Issue: The preamble to the actual question "signals" to the respondent that

the representatives are highly rated and may lead to a respondent to discount
a less than satisfactory experience. Eliminate any language that may add to a
respondent's knowledge (good or bad) about a product or service.

#2: How likely are you to purchase the newly designed, state-of-the-art cell
phone? [PURCHASE LIKELIHOOD SCALE]
Issue: The adjectives "newly designed" and "state-of-the-art" adds and/or
highlights information about the product; use this type of language in
product/concept descriptions, not in the question itself.

Mistake #2: Overlapping, Incomplete or Unclear

Response Choices
Response options should almost always be mutually exclusive and definitive.
Any ambiguity will confuse and/or frustrate respondents.

Examples:
#1: How long have you been in your current occupation?

 1 - 2 years
 2 - 5 years
  5 - 10 years
Issue: The options do not take into account under 1 year or over 10 years.
And the overlap of time would lead to those in the occupation for exactly 2 or 5
years to potentially answer differently, resulting in incorrect results. Add more
granularity to choices or use number fill-ins when appropriate.

#2: What type of vehicle do you own?

 2-door
 4-door
 Van
 Hatchback
 Truck
 SUV
 Sedan
Issue: A respondent's vehicle would likely fall across multiple categories from
the options provided, and some vehicle types are not included. Analyzing the
results from this question would not provide a clear or accurate picture of the
marketplace. Be sure lists are exhaustive and cover all possibilities.

 
5

Qualities & characteristics of a good questionnaire

  Need to know what needs to be measured – Having a clear

picture and understanding of what data needs to be
collected contributes to the quality of data collection
 Should know how to word/frame question and words
should be neutral and should not be leading – Whatever
your opinion may be, own opinion should never be
reflected in the questions. This is done both intentionally
and unintentionally but should be taken care of.
 Emphasis on right word/phrase should be kept in mind –
The language should be clear so that the required data can
be received. This also makes the question and the
requirement of survey easy to understand and thus help in
getting a better response and answer.
 Define and qualify terms – This is most essential when a
technical survey or a field specific survey is being done. If
you think that the audience being surveyed might not know
about some terms than they must be defined to get any
proper response. This will increase the quality and decrease
the bounce rate or number of questions which were left
unanswered.
 Avoid double negatives or more than 1 negative word in
question – Use of negative word has a psychological effect
and can influence the answer.
 Sufficient or adequate alternatives should be provided –
Available options should have the most expected answers
 Multiple questions in question should be avoided – One
question should have one answer. If more than one
question needs to be asked than it should be made a
separate question to improve clarity of questions.
 Word requiring emphasis should be emphasized – It helps
in making a point and question clear
 options like good/bad/fair/average should be quantified
through photographs or other mean – These are very vague
 terms and interpretations of these varies from person to
person
 Unwanted assumptions should be avoided – A survey is
about getting factual data and assumptions should be
avoided.