The influence of the direction of Likert type scales in web The influence of the direction of Likert-type scales in web

The influence of the direction of Likert type scales in web

surveys on response behavior in different respondent groups surveys on response behavior in different respondent groups surveys on response behavior in different respondent groups
Florian Keusch Florian Keusch
I tit t f Ad ti i d M k ti R h Institute for Advertising and Marketing Research, Institute for Advertising and Marketing Research,
WU Vienna University of Economics and Business WU Vienna University of Economics and Business y
Introduction Study 3 Introduction Study 3
Attitude measurement in web surveys mostly relies on requiring respondents In a web survey 491 Austrian online panel members had to rate five brands on Attitude measurement in web surveys mostly relies on requiring respondents
i di h i di i h h f l i d
In a web survey 491 Austrian online panel members had to rate five brands on
i i t Lik t l ith l it h R d t d l to indicate their agreement or disagreement with each of several items under a six-point Likert scale with eleven items each. Respondents were randomly g g
the same Likert type scale in a grid format As it is known that respondents do
p p y
assigned to one of four treatment groups The direction of the scales as well as the same Likert-type scale in a grid format. As it is known that respondents do assigned to one of four treatment groups. The direction of the scales as well as
h f i l l b li f l i i ll i d i not only attend to the words that convey the questions but also to the visual the use of numerical labeling of scale points was experimentally varied in a y y q
language of a questionnaire (i e format and shape of response scales verbal
g p p y
full factorial 2 (agree to disagree vs disagree to agree) x 2 (with numerical language of a questionnaire (i.e., format and shape of response scales, verbal full-factorial 2 (agree-to-disagree vs. disagree-to-agree) x 2 (with numerical
and numerical labels, spacing, positioning, and order of response options) it is labels vs. without numerical labels) design. The influence of scale format was and numerical labels, spacing, positioning, and order of response options) it is
ti l t d t d h thi ff t th Alth h th i
labels vs. without numerical labels) design. The influence of scale format was
d diff t i di t f d t lit ( bi essential to understand how this effects the response process. Although there is measured on different indicators of data quality (response bias, response
no conclusive evidence about the influence of the direction of extreme point latency non-differentiation) no conclusive evidence about the influence of the direction of extreme point
l b li ( l 1966 i d i d & Gl k 1993 S l b
latency, non differentiation).
All 55 i l d i ifi diff f l di i i h labeling (e.g., Belson, 1966; Friedman, Friedman, & Gluck, 1993; Salzberger All 55 comparisons revealed a significant difference for scale direction with g ( g , , ; , , , ; g
& Koller 2010; Weng & Cheng 2000) applying the near means related
p g
respondents showing more agreement if confronted with the agree to disagree & Koller, 2010; Weng & Cheng, 2000), applying the near means related respondents showing more agreement if confronted with the agree-to-disagree
heuristic (Tourangeau, Couper, & Conrad, 2004; 2007) to horizontal Likert- format (see Brand A in Figure 2). Only one statistically significant difference heuristic (Tourangeau, Couper, & Conrad, 2004; 2007) to horizontal Likert
t l ld t th t th i it b t th it d th iti
o at (see a d gu e ). O y o e stat st ca y s g ca t d e e ce
was found for numerical labeling Scale format did not influence response type scales would suggest that the proximity between the item and the positive was found for numerical labeling. Scale format did not influence response
anchor of the scale in a agree-to-disagree format would lead to different time and non-differentiation (i e proportion of straightliners) anchor of the scale in a agree to disagree format would lead to different
l h d l (di )
time and non differentiation (i.e., proportion of straightliners).
results than a reversed scale (disagree-to-agree). ( g g )
Study 1 Study 1
During a period of four months (Nov 10 Feb 11) the author participated in During a period of four months (Nov. 10 Feb. 11) the author participated in
36 l b (31 i li l ) 23 d h 36 german-language web surveys (31 in online panels). 23 used the agree-to- g g g y ( p ) g
disagree format (64%) 13 the disagree to agree format (36%) disagree format (64%), 13 the disagree-to-agree format (36%).
Fig. 2: Higher agreement with items if scale runs from agree- g g g g
to-disagree than from disagree-to-agree (Brand A) to-disagree than from disagree-to-agree (Brand A)
Conclusion and further research Conclusion and further research
h d di h h i ll li l b The presented studies show that especially online panel members are very p p y p y
prone to changes in the direction of Likert type scales as they show a strong prone to changes in the direction of Likert-type scales as they show a strong
Fig 1: Two thirds of german-language surveys use the agree-
tendency to the left-hand side of the scale regardless of its actual direction.
Fig. 1: Two thirds of german-language surveys use the agree-
t di f t
tendency to the left hand side of the scale regardless of its actual direction.
Thi i ht b th lt f l i i li l th t to-disagree format This might be the result of a learning process in online panels as the agree-to-
disagree format is more often used Numerical labeling of scale points had no disagree format is more often used. Numerical labeling of scale points had no
i fl b h i influence on response behavior. p
Currently a MTMM study is planned that should help to determine whether
Study 2
Currently, a MTMM-study is planned that should help to determine whether
Study 2
the near means related heuristic can be applied to horizontal Likert-type
In a web survey among Austrian small business owners (n=29) respondents
pp yp
scales and what format produces more valid results
In a web survey among Austrian small business owners (n 29) respondents
k d h i f 16 h i i f h b i
scales and what format produces more valid results.
were asked to rate the importance of 16 characteristics of cars when buying a p y g
company car on a 6 point rating scale 13 respondents received the scale in a company car on a 6-point rating scale. 13 respondents received the scale in a
positive-to-negative format (1=very important to 6=not at all important), p g ( y p p ),
16 in a negative to positive format (1=not at all important to 6=very
Literature
16 in a negative-to-positive format (1= not at all important to 6= very
Literature
B l W A (1966) Th ff t f i th t ti d f b l
important) Only two items (tailored financing options and low CO
2
Belson, W. A. (1966). The effects of reversing the presentation order of verbal
important ). Only two items ( tailored financing options and low CO
2
i i ) h d t ti ti ll i ifi t diff b t th t l
( ) g p
rating scales Journal of Advertising Research 6(4) 30-37
emission) showed a statistically significant difference between the two scale
rating scales. Journal of Advertising Research, 6(4), 30 37.
F i d H H F i d L W & Gl k B (1988) Th ff t f l
) y g
formats (Mann-Whitney-U; p<0 05)
Friedman, H. H., Friedman, L. W., & Gluck, B. (1988). The effects of scale-
formats (Mann-Whitney-U; p<0.05).
checking styles on response to a semantic differential scale Journal of the Market checking styles on response to a semantic differential scale. Journal of the Market
R h S i t 30(4) 477 481 Research Society, 30(4), 477-481.
Salzberger T & Koller M (2010) Examining the functioning of different response Salzberger, T. & Koller, M. (2010). Examining the functioning of different response
l b d i ti t d l I B k S C Ri b T
Contact information
scales based on innovative measurement models. In Beckmann, S. C., Ringberg, T.,
Contact information
& Ritter T (Eds ) Proceedings of the 39th European Marketing Academy
For further informationen please contact florian.keusch@wu.ac.at.
& Ritter, T. (Eds.). Proceedings of the 39th European Marketing Academy
C f (EMAC) 2010 Th i Th E ti l f M k ti
p @
Conference (EMAC) 2010, The six senses - The Essentials of Marketing,
Copenhagen, Denmark. Copenhagen, Denmark.
T R C M P & C d F (2004) S i iti i d Tourangeau, R., Couper, M. P., & Conrad, F. (2004). Spacing, positioning, and
order. Interpretive heuristics for visual features of survey questions. Public Opinion order. Interpretive heuristics for visual features of survey questions. Public Opinion
Q t l 68(3) 368 393 Quarterly, 68(3), 368-393.
Tourangeau, R., Couper, M. P., & Conrad, F. (2007). Color, labels, and interpretive Tourangeau, R., Couper, M. P., & Conrad, F. (2007). Color, labels, and interpretive
heuristics for response scales Public Opinion Quarterly 71(1) 91 112 heuristics for response scales. Public Opinion Quarterly, 71(1), 91-112.
Weng, L.-J. & Cheng, C.-P. (2007). Effects of response order on Likert-type scales. Weng, L. J. & Cheng, C. P. (2007). Effects of response order on Likert type scales.
Educational and Psychological Measurement 60(6) 908 924 Educational and Psychological Measurement, 60(6), 908-924.