Assesing Attribute Importance

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 6

Assessing Attribute Importance:

A Comparison of Six Methods


JAMES JACCARD
DAVID BRINBERG
LEE J. ACKERMAN*

Six methods of measuring attribute importance were evaluated for convergent va-
lidity. The methods were (1) an open-ended elicitation approach, (2) an information-
search approach based on Jacoby's behavioral process technology, (3) direct ratings
of importance, (4) conjoint measurement, (5) indices based on Jaccard's subjective
probability approach, and (6) a paired comparison approach. The convergent validity
of importance methods was investigated for two product classes: birth control and
cars. Results indicated relatively low levels of convergence among measures.

A major focus of consumer research has been the


development of methods for identifying product
uate their psychometric properties in terms of validity
and reliability.
attributes that are important in influencing product Importance is conceptualized in the present research
preferences and choice. Several approaches have in terms of change; i.e., an attribute is said to be im-
evolved for this purpose within the disciplines of con- portant if a change in the individual'S perception of
sumer behavior and social psychology, such as conjoint that product attribute leads to a change in the attitude
measurement techniques (e.g., Green and Wind 1973), toward the product. This conceptualization is generally
direct ratings of importance (e.g., Myers and Gutman consistent with Myers and Alpert's (1977) definition of
1974), methods based on subjective probability models attribute importance. I There exist only a few studies
(e.g., Jaccard, Knox, and Brinberg 1979), information that have empirically compared methods for assessing
display boards (lOB; e.g., Jacoby 1975, 1977), open- attribute importance in the consumer literature (e.g.,
ended elicitation techniques (e.g., Fishbein 1972), and Heeler, Okechuku, and Reid 1979; Myers and Alpert
Thurstonian methods (Fishbein and Ajzen 1975), to 1977; Myers and Chay 1979; Neslin 1981; Ryan and
name a few. Etzel 1976; Wiley, MacLachlan, and Moinpur 1977).
There have been relatively few evaluations of the de- The design used by Heeler et al. (1979) is typical of
gree of agreement between importance measures. If most research contrasting importance measures. It
there is relatively high convergence between methods, compared attribute importance as assessed by the lOB
then use of a given technique should be dictated pri- approach, a direct rating of attribute importance, and
marily by practical concerns. Such convergence would conjoint measurement. Subjects in this experiment
favor the direct rating approach relative to the more completed only one of the three tasks. Mean scores were
cumbersome conjoint and lOB methods. If the methods obtained for each measure for each of 10 attributes,
are relatively uncorrelated, however, then this implies and the within-attribute rank ordering of means was
that either one or more of the methods is suspect, or compared for the three methods. The rank-order cor-
that they are measuring different constructs. Of interest relations were of moderate magnitude (approximately
would then be the determination of which approach is 0.50) but of marginal statistical significance due to the
most appropriate for identifying product attributes that
can influence product choice. The purpose of the pres-
ent research was to assess the convergence among com- 'Myers and Alpert imply that certain methods of assessment are
monly used indices of attribute importance and to eval- directly linked to the concepts ofattribute salience, importance, and
determinance. For example, they associate open-ended elicitation
approaches with salience, rating scale approaches with importance
and correlational approaches with determinance. In fact, there is no
*James Jaccard is Associate Professor, Department of Psychology, one-to-one correspondence between methods and concepts. Corre-
David Brinberg is Associate Professor, Department of Marketing, lations can be spurious, and hence may reflect neither determinance
and Lee J. Ackerman is a graduate student in the Department of nor importance. Elicitation questions can be phrased so that they
Psychology, all at the State University of New York, Albany, NY reflect (in principle) importance or determinance (e.g., "name the
12222. attributes that are important to you in making a choice," as opposed
to "name the attributes that characterize this product").
463
JOURNAL OF CONSUMER RESEARCH. Vol. 12. March 1986
464 THE JOURNAL OF CONSUMER RESEARCH

small number of observations (n = 10). The vast ma- experimenter was present to answer questions and to
jority of research comparing measures of attribute im- give a brief oral introduction to the study. Subjects were
portance has used a similar methodology, wherein sub- asked to answer all questions carefully and honestly
jects complete only one measurement task and where and were warned of the redundancy of the questions.
comparisons of methods are based on aggregate-level Confidentiality of responses was assured. Each task was
analyses. This approach has limitations because incon- introduced with a cover sheet describing the nature of
sistency can exist between measures at the individual the task. Practice items concerning the choice of a cam-
level that are consistent at the aggregate level. A more era preceded the actual test questions in order to fa-
sensitive index of convergence would be to obtain mea- miliarize subjects with the scales and question formats
sures on each of the methods for each individual and and to eliminate warm-up effects from the data.
then correlate measures across subjects (or alternatively,
within subjects, idiographically). In addition, research Materials
has examined the relationship between only three or
fewer measures. Comprehensive comparisons of the Measures will be described in terms of the car prod-
major methods in the consumer literature for assessing ucts. Comparable measures were obtained for the birth
attribute importance do not exist. The present research control products.
explored the convergence of six major approaches at Elicitation Measure. Subjects were asked to consider
both the aggregate and individual levels. the characteristics that are important to them in eval-
uating a car to buy and to list the factors on a sheet of
METHOD paper. Responses were content-analyzed by two coders
Subjects to determine if any of the nine attributes were men-
Subjects were 110 female undergraduate students in tioned and in what order. High levels of agreement were
introductory psychology classes at a large midwestern observed between coders (e.g., inter-coder agreement
university. for whether a given attribute was mentioned averaged
95 percent). In cases of disagreement, one ofthe authors
resolved the discrepancy. Several researchers have sug-
Products and Attributes gested that the order of elicitation will also be sensitive
Two product decisions were studied: (I) the decision to attribute importance. Indirect support for this is pro-
of what type of car to purchase, and (2) the decision of vided by Fishbein's (e.g., Kaplan and Fishbein 1969)
what type of birth control method to use. Based on pre- research on belief salience and belief strength, Tversky
vious research, nine attributes of each product were and Kahneman's (1973) research on the availability
identified for the focus of investigation. For the choice heuristic, Szalay and Deese's (1978) associational re-
of a car, these were: cost, warranty, 2 versus 4 door, gas search, and Hullian learning theory. An importance in-
mileage, roominess, manufacturer, repair record, boy- dex for a given attribute was defined for each individual
friend's approval/disapproval, and automatic versus to incorporate order of elicitation. This involved divid-
standard transmission. For the choice of birth control ing the rank order of elicitation of the attribute (where
method, the attributes were: the chances of major health I = last elicited attribute through n = first elicited at-
consequences, the chances of minor side effects, whether tribute) by the number of attributes elicited by the sub..
the method requires a doctor's attention, convenience ject. If a given attribute was not mentioned, the index
of use, convenience in obtaining the method, boy- for that attribute was set to zero. Approximately 3 per-
friend's approval/disapproval, potential effects on sex- cent of the elicited responses did not correspond to any
ual pleasure, cost, and effectiveness in preventing preg- of the nine-attribute categories for the two product
nancy. classes.
Information-Search Measures. The information-
Procedure search tasks were based on Jacoby (1977), and were
All subjects completed tasks relevant to both products presented on a micro-computer. Subjects were shown
using all six approaches. The order of the two products a list of five hypothetical options (e.g., car A, car B, etc.)
was counterbalanced across subjects. Within a given and the list of nine attributes. Subjects indicated which
product, the free elicitation task was always adminis- information they wanted to see and were asked to
tered first in order to avoid biasing subjects with ex- choose that car from among the five that they would
perimenter-generated attributes. The order of presen- be most likely to purchase. They were permitted to select
tation of the remaining five tasks was varied via a latin as much or as little information as they wanted, and
square design. Analyses revealed no impact of order of they controlled what information they saw along with
presentation on mean ratings of attributes or on the the order, duration, and number of times a piece of
relative ranking of means. information was accessed. However, only one piece of
Each subject participated in an hour and one-half information was presented at a time. Two indices of
session in which all tasks were self-administered. The importance were computed for each subject (Jacoby et
ASSESSING ATTRIBUTE IMPORTANCE 465

al. 1976). The first was based on the frequency of search logic of this measure). Technically, the response format
and was defined as the number oftimes that an attribute used does not represent a subjective probability. How-
dimension was searched divided by the total number ever, it captures the spirit of the approach in terms of
of values acquired by the individual. The second was differential response to the presence or absence of an
based on the order of search and was defined as 1-(S/ attribute.
p)/T, where S = the sum of the rank orders of infor-
mation search on the attribute dimension in question, Thurstone Measure. Subjects were presented with
p = the total number of values searched on the dimen-
pairs of attributes and asked to indicate which of the
sion, and T = the total number of values searched by two would be more important to them in evaluating a
the subject. As an example, if an individual acquired car. All possible pairs were presented and scale values
20 pieces ofinformation (T), the first and fifth of which of importance were derived using traditional paired
were on the price dimension, then the order index would comparison analysis (Edwards 1957, pp. 19-52). This
be 1-1 + 5)/2)/20 = 0.S5. The closer the number to approach is only applicable at the aggregate level, be-
1, the more important the dimension. If a dimension cause individual estimates are not possible.
was not searched, the indices were set to zero. Ranks
for re-acquisition of values already searched were ex- RESULTS
cluded from the calculations.
Importance Ratings. Subjects were asked to rate how
Analysis of Individual Attributes
important each attribute would be in evaluating a car The six individual measures of importance were used
on an II-point scale anchored by 0 (not at all important) to generate a multitrait-multimethod matrix for each
and 10 (very important). topic area. This was, in principle, a nine trait (attribute)-
Conjoint Measures. Subjects were presented with six method matrix of 54 measures. However, the con-
cards, each of which described a car using information joint measures were only obtained on four attributes.
about four ofthe attributes: cost, warranty, boyfriend's Thus, two different matrices could be specified: a four
approval/disapproval, and gas mileage. The attributes attribute-six method matrix (in which the conjoint
were manipulated with two levels each, yielding 16 dif- measures were included) and a nine attribute-five
ferent stimuli via a 2 X 2 X 2 X 2 factorial design. Only method matrix (in which the conjoint measures were
four attributes were manipulated to keep the number excluded). A separate path model, designed to estimate
of stimuli to a manageable number and to permit tests reliabilities as well as contributions of true score vari-
of additivity (which require a full factorial design). Val- ance and method variance (see Schmidt 1975) was fitted
ues were described as "good" or "bad" for purposes of to each matrix using LISREL (Joreskog 1970). In both
manipulating a factor (see Jaccard and Wood 1986 for cases, a four attribute-six method/nine attribute-five
the rationale of this), except for the disapproval/ap- method model was rejected due to a poor fit (i.e., sta-
proval factor. Subjects were asked to rank order the tistically significant and sizeable discrepancies between
cards from most preferred to least preferred. Tests of reproduced correlations based on the model and ob-
additivity were performed (in no case was additivity served correlations). Several alternative models were
rejected), and importance estimates were derived from investigated using this approach, none of which proved
utility estimates provided by MONANOV A. satisfactory. Inspection of the correlations revealed that
the measures were, for the most part, uncorrelated, sug-
Subjective Probability Measures. A popular subjec- gesting that the best fitting models would be those that
tive probability model that can be used to define at- are statistically underidentified. As such, the confir-
tribute importance is one based on Jaccard's extension matory factor analytic approach was abandoned and
ofWyer's (1975) probabilistic model of the relationship more traditional analyses were performed.
between beliefs (Danes et al. 19S0; Jaccard et al. 1979). For a given attribute, the six measures of importance
The importance index involves calculating the absolute were correlated (in cases where the conjoint measure
difference between two subjective conditional proba- was not obtained, only five measures were correlated).
bilities. These were obtained for each attribute, using Each correlation matrix was analyzed by means of a
an It-point scale with endpoints 0 (not at all willing to principal components factor analysis and rotated to an
consider it) to 11 (very willing to consider it). The P(lIA) oblique (oblimax) criterion. The number of factors se-
statements were of the format "Suppose that a particular lected for purposes of rotation was determined by a scree
car was inexpensive. Would you be willing to consider test. Eighteen analyses were performed (nine for the
this as a car you might buy?" The P(lIA') conditional birth control attributes and nine for the car attributes).
had the following format: "Suppose that a particular Analyses tended to yield either three-factor or four-fac-
car was expensive. Would you still be willing to consider tor solutions. Factor correlations in each of the eighteen
this as a car you might buy?" The absolute difference analyses were generally low, ranging from -0.07 to 0.24
between these two conditional measures served as the with an absolute average ofO.OS. In two of the analyses,
index of importance (see Jaccard et al. 1979 for the the conjoint measure tended to load on the same factor
466 THE JOURNAL OF CONSUMER RESEARCH

as the probability measure and the direct rating mea- TABLE 1


sure. (A measure was said to "load" on a factor if the AVERAGE CORRELATION BETWEEN IMPORTANCE
structure coefficient was greater than 0.50.) In all other INDICES: BIRTH CONTROL
analyses, the conjoint measure loaded on a separate
factor (i.e., a factor with no other loadings greater than Fre- Paired
Indices E SP quency Order DR comparison
0.50). The elicitation index loaded on a separate factor
in all analyses. The probability index and the direct Elicitation (E) .30 .15 .19 .33 .23
rating index tended to load on the same factor in all Subjective
but three of the analyses. Frequency of search and order probability (SP) .08 .15 .15 .68 .33
of search from the behavioral process method always Frequency: lOB .12 .11 .82 .19 .19
Order: lOB .10 .13 .49 .58 .15
loaded on the same factor. 2 .11 .09 .22
Direct rating (DR) .07 .38
The average intercorrelation matrices between each Conjoint .11 .10 .12 .09 .10
of the six measures are presented in the lower off-di-
agonals of Tables 1 and 2 for the birth control and car NOTE: A_age correlations with conjoint measures are based on four attributes; others are

decisions, respectively. The data are generally in agree- based on nine attributes. Lower ofl-diagonal elements are from across-individual analysis.
Upper ofl-<liagonal elements are from aggregate-level analysis.
ment with the factor analysis, although the size of the
correlations between "common" indices (as identified
by the factor analysis) is not very impressive. The av-
erage correlation of the two lOB indices was 0.49, while correlation matrices across subjects, but we observed
that for the probability measure and the direct rating few instances of strong, consistent convergence between
measure was 0.38. These represent rather low coeffi- the multiple measures.
cients in the context of convergent validity.
DISCUSSION
Analyses of Aggregate Indices
The present research evaluated the convergence of
Aggregate-level indices were also evaluated for con- six approaches for assessing attribute importance. When
vergence in terms of the rank-order importance of dif- analyzed in terms of correlations across individuals,
ferent attributes. The aggregate-level measure for a given there was relatively little convergence between methods.
index was defined as the mean importance score of the The two behavioral process measures were moderately
attribute on that index (with the exception of the paired correlated, as were the probability and direct rating
comparison method, which is defined only on the ag- methods. The lOB measures use the same measurement
gregate level). The convergence between aggregate level method (information acquisition in an lOB task), and
indices was assessed in the following way. For a given both the probability measure and the direct rating
index, the attributes were rank ordered in terms of the method use rating scales. The correlations between these
aggregate score. Kendall rank-order correlations were indices thus may be inflated due to shared method vari-
computed for the ranks derived from one index with ance. Overall, strong evidence for convergence was not
the ranks derived from another index. The results are apparent in the data. This result raises questions about
reported in the upper off-diagonal elements of Tables consumer research that relies on a single measure for
1 and 2. Although the correlations generally are larger inferring attribute importance. Each of the methods
than the across-individual analyses, their magnitude is considered either has a substantial theoretical base or
not very impressive, given the small n. is widely used in applied research. The present results
suggest that conclusions made about attribute impor-
Within-Subjects Analyses tance may be quite different depending on the method
used to assess importance. Clearly, further research is
Convergence between measures for each individual needed on what the different indices are in fact mea-
was also assessed using procedures similar to the aggre- suring.
gate level analysis, but where the rank orders were de- In terms of attribute importance, it is possible that
termined on the basis of a given individual's scores. one of the measures is valid relative to the other mea-
Thus, a total of 110 5 X 5 rank-order correlation ma- sures. Change studies can be designed in which two or
trices were computed, one for each subject. The cor- more of the approaches make different predictions
relations were not impressive in magnitude and tended about the efficacy of changing a perception about a given
to follow the same pattern as the aggregate-level anal- attribute relative to product evaluation. The "better"
ysis. There was a moderate degree of variability in the measure would be that which best predicts the effects
of a perception change. Alternatively, one could con-
ceptualize importance as a multidimensional concept
'For brevity, tabular summaries of the factor analyses (and cor- in which each of the measures taps into different aspects
relations discussed later) are not presented. They can be obtained on of importance. The relative lack of correlation between
request from the first author. indices results from the different measures tapping into
ASSESSING ATTRIBUTE IMPORTANCE 467

TABLE 2 boredom may have led subjects to cease taking the task
AVERAGE CORRELATION BETWEEN IMPORTANCE seriously toward the end of each topic, hence decreasing
INDICES: CARS convergence. Two points can be made in this regard.
First, it can just as easily be argued that subjects tried
Fre- Paired to be consistent across tasks and, if anything, that the
Indices E SP quency Order DR comparison current data overestimate convergence. Second, we
found little evidence for order effects when these were
Elicitation (E) .19 .15 .22 .19 .28
Subjective
specifically analyzed in the context of the counterbal-
probability (SP) .08 .28 .25 .62 .68 anced design .
Frequency: lOB .12 .12 .78 .22 .19 The present results are consistent with those reported
Order: lOB .14 .11 .49 .25 .15 by Jaccard and Sheng (1983), who compared the con-
Direct rating (DR) .13 .37 .09 .13 .58
Conjoint .09 .11 .10 .11 .11
vergence of six importance indices. Jaccard and Sheng
included neither the conjoint task nor the pairwise
NOTE: Average correlations with conjoint measures are based on four attributes; others are comparison task. Their focus was on attribute impor-
based on nine attributes. Lower off-<liagonal elements are from aa-oss-indivldual analysis. tance at the brand level, whereas the present research
Upper off-<liagonal elements are from aggregate-leval analysis.
focused on attribute importance at the product level.
Taken together, the results of both studies make it ev-
ident that the conceptual foundations of measures of
different aspects of importance. For example, the elic- attribute importance require greater attention from
itation index appears to measure, albeit imperfectly, consumer researchers and that the measures must
factors that people consciously consider in their eval- emerge from a strong theoretical base.
uations. The other indices may, in contrast, indicate
whether a given consequence could be important in in-
fluencing an evaluation, were it to be made salient and [Received February 1985. Revised September 1985.]
manipulated. This does not, however, account for the
lack of correlation between the latter indices. A mul-
tidimensional approach would require an elaboration REFERENCES
of the conceptual foundations of the dimensions un-
derlying attribute importance and how these map onto Danes, Jeffrey E., James G. Johnston, Phillipe Cattin, and
product evaluation. Measures would then be evaluated John E. Hunter (1980), "Evaluating Marketing Com-
relative to the conceptual framework. munication Strategies: Attribute Importance versus
Several methodological points should be considered Transfer Discrepancy," Advances in Consumer Research,
Vol. 7, ed. Jerry C. Olson, Ann Arbor: MI: Association
in the context of the present data. First, the low levels for Consumer Research, 561-565.
of convergence may primarily be the result of unreli- Fishbein, Martin (1972), "Towards an Understanding of
ability of measures. We originally intended to assess Family Planning Behaviors," Journal of Applied Social
reliability using confirmatory factor analysis. However, Psychology, 2,219-227.
because all viable models were statistically underiden- - - - and Icek Ajzen (1975), Beliefs, Attitudes, Intentions,
tified, this was not possible. We doubt that reliability and Behavior: An Introduction to Theory and Research,
is a major problem for several reasons. First, the study Reading, MA: Addison-Wesley.
was conducted with college students who are relatively Green Paul E. and V. Srinivasan (1978), "Conjoint Analysis
bright and very capable of understanding the scales in Consumer Research: Issues and Outlook," Journal of
used. If anything, this population would probably yield Consumer Research, 5, 317-328.
- - - and Yoram Wind (1973), Multiattribute Decisions in
more reliable measures than populations typically used Marketing: A Measurement Approach, Hinsdale, IL:
in consumer research. Second, we took considerable Dryden.
care to motivate subjects and to eliminate factors such Heeler, Roger M., Chike Okechuku, and Stan Reid (1979),
as warm-up effects and failure to understand scale for- "Attribute Importance: Contrasting Measurements,"
mats. The items were pretested on a small sample of Journal of Marketing Research, 16,60-63.
subjects, respondents were given thorough instructions Jaccard, James and Diana Sheng (1983), "Assessing the Im-
about the tasks, the order of tasks was counterbalanced, portance of Information in Influencing Decisions: Im-
and each task was preceded by practice items. Finally, plications for Nonprofit Marketing Research," in Non-
every attempt was made to use the measures as they profit Marketing: Conceptual and Empirical Research,
eds. F.K. Shuptrine and P.H. Reingen, American Mar-
have been used in traditional consumer research studies.
keting Association.
If lack of reliability is a problem, then it is a problem - - - and Gregory N. Wood (1986), "An Idiothetic Analysis
that is probably characteristic of typical applications of Behavioral Decision Making," in Perspectives on
that assess attribute importance. Methodology in Consumer Research, eds. David Brinberg
Another methodological concern is that the present and Richard J. Lutz, New York: Springer-Verlag.
research obtained multiple indices of importance from - - - , Richard Knox, and David Brinberg (1979), "Predic-
the same subjects. It could be argued that fatigue and tion of Behavior from Beliefs: An Extension and Test of
468 THE JOURNAL OF CONSUMER RESEARCH

a Subjective Probability Model," Journal of Personality Vol. 6, ed. William L. Wilkie, Ann Arbor, MI: Associ-
and Social Psychology, 37, 1239-1248. ation for Consumer Research, 259-262.
Jacoby, Jacob (1975), "Perspectives on a Consumer Infor- - - - and J. Gutman (1974), "Validating Multi-attribute
mation Processing Research Program," Communications Attitude Models," American Marketing Association
Research, 2, 203-215. Proceedings, 95-99.
- - - (1977), "The Emerging Behavioral Process Technol- Neslin, Scott A. (1981), "Linking Product Features to Per-
ogy in Consumer Decision Making," in Advances in ceptions: Self-stated Versus Statistically Revealed Im-
Consumer Research, Vol. 4, ed. William D. Perreault, portance Weights," Journal of Marketing Research, 18,
Jr., Atlanta, GA: Association for Consumer Research, 80-86.
263-265. Ryan, Michael and M.J. Etzel (1976), "The Nature of Salient
- - - , Robert W. Chestnut, Keith C. Weigel, and William Outcomes and Referents in the Extended Model," in
A. Fisher (1976), "Pre-purchase Information Acquisition: Advances in Consumer Research, Vol. 3, ed. Beverlee B.
Description of a Process Methodology, Research Para- Anderson, Ann Arbor, MI: Association for Consumer
digm, and Pilot Investigation," in Advances in Consumer Research, 485-490.
Research, Vol. 3, ed. Beverlee B. Anderson, Ann Arbor, Schmidt, Neal (1978), "Path Analysis of Multitrait-Multi-
MI: Association for Consumer Research, 306-314. method Matrices," Applied Psychological Measurements,
Joreskog, Karl G. (1970), "A General Method for Analysis 2,157-173.
of Covariance Structures," Biometrika, 57, 239-251. Szalay, Lorand B. and James Deese (1978), Subjective Mean-
Kaplan, Kalman J. and Martin Fishbein (1969), "The Source ing and Culture: An Assessment Through Word Associ-
of Beliefs, Their Saliency, and Prediction of Attitude," ations, Hillsdale, NJ: Lawrence Erlbaum.
Journal of Social Psychology, 78, 63-74. Tversky, Amos and Daniel Kahneman (1973), " Availability:
Myers, James H. and Mark I. Alpert (1977), "Semantic Con- A Heuristic for Judging Frequency and Probability,"
fusion in Attitude Research: Salience vs. Importance vs. Cognitive Psychology, 5, 207-232.
Determinance," in Advances in Consumer Research, Vol. Wiley, James B., Douglas L. MacLachlan, and Reza Moinpur
4, ed. William D. Perreault, Jr., Altanta, GA: Association (1977), "Comparison of Stated and Inferred Parameter
for Consumer Research, 106-110. Values in Additive Models: An Illustration of a Para-
- - - and Richard F. Chay (1979), "A Comparison of Two digm," in Advances in Consumer Research, Vol. 4, ed.
Methods for Determining Optimum Level of Producer William D. Perreault, Jr., Atlanta, GA: Association for
Characteristics," in Advances in Consumer Research, Consumer Research, 106-110.

You might also like