JBR-08739; No of Pages 10

Journal of Business Research xxx (2015) xxx–xxx

Contents lists available at ScienceDirect

Journal of Business Research

Debunking legendary beliefs about student samples in marketing research

Jennifer A. Espinosa ⁎, David J. Ortinau 1
Department of Marketing, College of Business, University of South Florida, 4202 East Fowler Avenue, Tampa, FL 33620-5500, United States

a r t i c l e i n f o a b s t r a c t

Article history: The current research investigates three legendary beliefs related to sample definition and the selection of contex-
Received 1 October 2014 tual target populations: the representativeness of samples, the willingness of research participants, and the ho-
Received in revised form 1 February 2015 mogeneity of samples. After identifying several current trends related to the use of college-students as data
Accepted 1 September 2015
sources, data from three randomly drawn samples of students and consumers empirically debunks the legendary
Available online xxxx
beliefs. Findings indicate the contextual setting of the research directly shapes the representativeness of drawn
Keywords:
samples, the willingness of college students equals the willingness of non-college consumers, and drawing sam-
Defined target population ples from the same institution artificially increases sample homogeneity. Debunking these legendary beliefs re-
Sampling frame veals the need for multiple‐sample research, probabilistic sample selection procedures, clearer discussions of
Academic legendary beliefs qualifying criteria, and the consideration of new confounding factors (e.g., research subject incentivization).
Generalizability The article discusses how the truths of the legendary beliefs influence the generalizability of research results to
Motivation contextual populations of interest.
Homogeneity © 2015 Elsevier Inc. All rights reserved.

1. Introduction
The demands for analytical and scientific rigor in research continue
to rise, with quality journals asking for stronger theoretical justifications
and improved managerial contributions (Lehmann, McAlister, & Staelin,
2011). Identifying the correct sample frame for empirical studies
remains a critically important decision that can influence the quality
of data a researcher obtains. When conducting empirical research in-
volving human beings, one persistent data collection issue that con-
tinues to create controversy and debate centers on if college students
are a reasonable sampling frame.
Inherent within this sampling frame issue is the concept of defined
target population or target population (Dillon, Madden, & Firtle,
1990). No universal definition of target population exists. For example,
Burns and Bush (2003) define target population as “the entire group
under study as specified by the objectives of the research project”
(p. 334). Dillon et al. (1990) define target population as a “set of people,
products, firms, markets, etc. that contains the information that is of
interest to the research” (p. 265). Hair, Bush, and Ortinau (2009) define
this concept as a “specified group of people or objects for which
questions can be asked or observations made to obtain the desired
information” (p. 52).
Prior to handling sample frame issues, researchers need to have a
clear understanding of the characteristics of the individuals who
⁎ Corresponding author. Tel.: +1 813 974 9510; fax: +1 813 974 6175.
E-mail addresses: jaespinosa@usf.edu (J.A. Espinosa), dortinau@usf.edu (D.J. Ortinau).
1
Tel.: +1 813 974 6236; fax: +1 813 974 6175.
represent the defined target population, to effectively assess the inter-
nal and/or external validity of a study's results and inferences. The de-
scription of the target population needs to be consistent with the
research objectives, including a clear description of all the specific qual-
ifying criteria that subjects must meet for inclusion in the sample frame
and study.
The existing literature focuses heavily on debates of the appropriate-
ness of using college student sampling frames in research, instead of
informing researchers about the reasonableness of college student sam-
pling frames. The debates center on the convenience, accessibility, rep-
resentativeness, and costs of obtaining research subjects (Bello, Leung,
Radebaugh, Tung, & Van Witteloostuijn, 2009; Creswell, 2009; Lynch,
1982; Wells, 1993). For example, pro arguments include the preserva-
tion of “internal validity” for theory development or causal hypothesis
testing (Peterson, 2001; Thomas, 2011; Webster & Sell, 2007). Con
arguments include limitations of the generalizability of the results
and implications to non-student target populations (Beltramini, 1983;
Campbell & Stanley, 1963; Cunningham, Anderson, & Murphy, 1974;
Steinfatt, 1991; Stevens, 2011). Additional debate topics include the
homogeneity of samples (e.g., Hair, Black, Babin, & Anderson, 2010;
Oakes, 1972; Peterson, 2001; Thomas, 2011), the willingness of stu-
dents to participate in research (e.g., Bello et al., 2009; Dobbins, Lane,
& Steiner, 1988; Malaviya & John, 2001) and the costs of student re-
search (e.g., Huang, Gattiker, & Schwarz, 2008; Lamb & Stem, 1979;
Knemeyer & Naylor, 2011; Malaviya & John, 2001).
Many of these debates surrounding the use of college student sam-
ple frames stem from established beliefs that, over time, have become
deep-rooted legendary beliefs and act as a justification for using or not
using college students as data sources. Among the different debates,

http://dx.doi.org/10.1016/j.jbusres.2015.12.003
0148-2963/© 2015 Elsevier Inc. All rights reserved.

Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
2 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx

the debate over establishing internal/external validity with student-
based data continues to be most important and relevant to academic
scholars conducting empirical research. Given the interest and impor-
tance of the long standing beliefs about college students as data sources,
the two fundamental questions that drive the current study are:
(1) when are college students an appropriate sampling frame and
(2) how should college students be sampled?
The current research differs from previous research in several ways.
First, the main research objective is not to validate existing pros and
cons for using college student data, but rather to empirically investigate
some of the deep-rooted legendary beliefs (i.e., representativeness,
incentivization, and sample homogeneity) that underlie college student
based-data. The empirical findings and inferences of these legendary
beliefs can further enhance the reader's understanding of the appropri-
ateness of college students as data sources. Second, the collection of data
from three sub-segments of restaurant consumers (non-student con-
sumers, incentivized college students, and non-incentivized college stu-
dents) represents a replication-study approach, meaning that the
generalizability of the results can be directly tested across the three
sub-segments. Third, unlike most previous studies that include manipu-
lated independent variables and measured dependent behavioral con-
structs of interest, the current study limits the focus to one attitudinal
construct termed the consumers' overall restaurant image (ORI).
The remainder of this article proceeds as follows. Section 2 high-
lights the results of a partial review of several high-ranking marketing
journals demonstrating the prevalence of college student data in aca-
demic research. Section 3 presents the deep-rooted legendary beliefs
relating to college student data sources and the main construct (ORI)
using casual-dining (CD) and fine-dining (FD) restaurant contextual
settings. The following sections present the multi-replication research
design, followed by the main results, discussion, and implications of
the findings, with the closing section presenting limitations and sugges-
tions for future research.
2. Current trends on the usage of college students and non-students
in high-ranking marketing journals
The main objective of the partial review of selected high-ranking
marketing journals is to gain a better perspective on the reliance on col-
lege students in academic research. The review covers 1090 published
articles in four high-ranking marketing journals (148 from Journal of
the Academy of Marketing Sciences [JAMS], 567 from Journal of Business
Research [JBR], 224 from Journal of Consumer Research [JCR], and 151
from Journal of Marketing [JM]) for the years 2009–2011. Using the
five specific evaluative criteria reported in Table 1, preliminary analysis
reveals that 228 articles employ only student-based data, 399 articles
use only non-student data, and 109 articles use a combination of stu-
dent and non-student samples. Given the current study's focus on inter-
nal/external validity issues, those 109 articles that include both student
and non-student samples with different contextual target populations
are excluded, leaving 627 articles that contain either college student
or non-student sampling frames.
Content analysis shows that college student subjects appear most
frequently in experimental research (77%), while non-student sample
frames appear most in traditional survey research (57%). College stu-
dents are the subject of choice in experimental research most frequently
in JCR (96%), and less frequently in JM (68%), JAMS (50%) or JBR (46%).
Despite the high usage of college student samples in research, potential
limitations with using college students often go unnoticed. Among the
articles employing student sample frames, only 21% recognize that
college students create any potential limitations to the study's findings,
inferences, and implications. Disclosure of potential limitations occurs

Table 1
Summary of current trends regarding the use of student and non-student sample frames in high-ranked marketing journals.a

Criteria of evaluation Total (N = 736) JM (N = 88) JCR (N = 206) JAMS (N = 98) JBR (N = 344)

Student Non-St. Both Student Non-St. Both Student Non-St. Both Student Non-St. Both Student Non-St. Both

Sub-totals 228 399 109 19 57 12 131 30 45 10 77 11 68 235 41

Research design
Experiment 175 37 55 13 5 3 126 9 39 5 5 3 31 8 10
(77) (9) (50) (68) (9) (25) (96) (30) (87) (50) (6) (27) (46) (3) (24)
Surveys 37 227 26 2 34 4 4 1 3 4 40 6 27 152 13
(16) (57) (24) (11) (60) (33) (3) (3) (7) (40) (52) (55) (40) (65) (32)
Recognized limitation
Recognized 49 – 10 2 – – 8 – 1 3 – – 36 – 9
(21) (9) (11) (6) (2) (30) (53) (22)
Not recognized 170 – 86 16 – 10 123 – 43 6 – 10 25 – 23
(75) (79) (84) (83) (94) (96) (60) (91) (37) (56)
Not applicable 9 – 13 1 – 2 – – 1 1 – 1 7 – 9
(4) (12) (5) (17) (2) (10) (9) (10) (22)
Results of Hypotheses tests
Supported 94 101 37 15 33 9 57 5 11 7 10 4 15 53 13
(41) (25) (34) (79) (58) (75) (44) (17) (24) (70) (13) (36) (22) (23) (32)
Mixed 47 181 28 1 11 1 8 3 5 3 50 6 35 118 16
(21) (45) (26) (5) (19) (8) (6) (10) (11) (30) (65) (55) (52) (50) (39)
Not applicable 87 117 44 3 13 2 66 22 29 – 17 1 18 64 12
(38) (30) (40) (16) (23) (17) (50) (73) (65) (22) (9) (26) (27) (29)
Incentives offered
Offered 147 68 56 12 36 12 113 3 30 4 11 4 18 18 10
(65) (17) (51) (63) (63) (100) (86) (10) (67) (40) (14) (36) (26) (8) (24)
Not offered 81 331 53 7 21 – 18 27 15 6 66 7 50 217 31
(35) (83) (49) (37) (37) (14) (90) (33) (60) (86) (64) (74) (92) (76)
Underlying theories
Used 61 136 23 12 28 8 15 3 3 5 44 3 29 61 9
(27) (34) (21) (63) (49) (67) (11) (10) (7) (50) (57) (27) (43) (26) (22)
Not used 167 263 86 7 29 4 116 27 42 5 33 8 39 174 32
(73) (66) (79) (37) (51) (33) (89) (90) (93) (50) (43) (73) (57) (74) (78)
a
Cell numbers represent: frequency (percentage).

Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx
most often in JBR (53%), and less frequently in JAMS (30%), JM (11%)
and JCR (6%). The articles most often cite the lack of generalizability of
the sample results to broader contextual populations as a limitation
(98%), followed by limitations of homogeneity of the sample units (8%).
Incentivizing college students to encourage participation is a fre-
quent practice in published academic research. In the articles containing
only college student participants, 65% report the use of incentives
such as extra credit, small gifts, or money to increase participation. Con-
versely, of the articles containing only non-student participants, only
17% offer incentives to complete the research (p b .01). In terms of hy-
potheses testing outcomes, researchers report support for all hypothe-
ses in 41% of the articles employing only college students, and another
21% report mixed support for some hypotheses. In contrast, only 25%
of the articles consisting of only non-student consumers report support
for all hypotheses and 45% mixed support for some hypotheses. There-
fore, obtaining data results and findings that provide support for all hy-
potheses may be easier with college students, rather than non-student
consumers. Among the 627 articles, only 36 articles (5.7%) identify
and describe the contextual target population as college students and
actually survey college students. The other 591 articles (94.3%) either
explicitly or implicitly suggest a broader contextual target population
than college students. Surprisingly, 192 articles (32.5%) with non-
student target populations restrict the sampling frames to only college
students. This trend suggests that around 30% of the time, academic
scholars do not do a good job accurately matching drawn samples of re-
search participants to the study's target population.
3. Deep-rooted legendary beliefs
The current research objective aims to assess empirically three deep-
rooted legendary beliefs about the representativeness, incentivization,
and sample homogeneity of college-student samples to determine
the reasonableness of college student sample frames. The current inves-
tigation defines a deep-rooted legendary belief as a long standing,
unverified cognitive thought or emotional expression of a specific opin-
ion or phenomenon.
3.1. Legendary belief I: generalizability of college student samples to broad
contextual target populations
The first legendary belief under investigation states that college stu-
dents, as research participants, are representative of broader contextual
target populations. This belief allows researchers to make statistical and
real-world inferences about other members of a broader contextual
target population from college student generated results. The heart of
this legendary belief centers on how generalizable sample-based data,
results, and inferences are to a broader target population (Bello et al.,
2009; Huang et al., 2008; Stevens, 2011). Compeau, Marcolin, Kelley,
and Higgins (2012) describe two methods for determining the general-
izability of sample data to a target population. The first method consists
of conducting replication studies with different samples and/or contex-
tual settings, requiring (1) a clear target population representative of
the contextual setting, (2) an accurate sampling frame representative
of the target population, and (3) sample units randomly selected from
the sample frame using a probabilistic selection procedure (Compeau
et al., 2012). When researchers meet these conditions, replication
studies provide an objective assessment of the generalizability of
sample-based results to a broader contextual target population.
When researchers face difficulties in meeting these conditions, the
use of non-probabilistic selection procedures can result in the creation
of a convenience or quota type sample (e.g. student or Mechanical
Turk crowdsourcing samples). With non-probabilistic sample results,
researchers cannot argue that the sample represents a target popula-
tion. Instead, non-probabilistic samples require the researcher to work
backwards and try to identify the group of individuals (e.g., a relevant
population) the results and findings may apply to. Given the inherent
Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
3
limitations of convenience or quota sample results, some researchers
use a secondary approach in an attempt to establish generalizability of
the data results. This secondary approach focuses on generalizing a
study's results to a relevant population through a discussion of the in-
herent limitations of the sample frame, including what general implica-
tions are derived from the sample results and how the limitations of the
sample frame temper those general implications (see Lynch, 1999;
Compeau et al., 2012 for in-depth arguments and discussions).
Although this alternative discussion approach to generalizability ap-
pears common in academic research, numerous researchers provide
an incomplete assessment or neglect an assessment altogether. The re-
view of the four high-ranking marketing journals reveals that only 21%
of the articles using college student samples acknowledge any potential
limitations of student samples. More troublesome, 79% of the articles fail
to provide a discussion of how the limitations of student samples might
temper the results and insights of the research.
Within the context of restaurant consumers, the current research
specifically focuses on the generalizability of different kinds of student
and non-student participants to the target population of adult US
consumers who are familiar with and have dining experience with
two specific national franchised CD restaurants (Applebee's and Chili's)
and two specific FD restaurants (i.e., Morton's and Ruth Chris
Steakhouse) restaurants. The contextual target population of interest
consists of experienced restaurant consumers and contains diverse sub-
groups of people who differ in age, income, life experiences, personalities,
attitudes, liking/disliking, etc. Realistically, the heterogeneous nature of
the study's defined target population consists of different pluralistic
subgroupings, from which college students represent only one possible
sub-segment of all consumers who eat out at CD and FD restaurants.
Thus, how well do the results and inferences from college students repre-
sent the broader target population of CD and FD restaurant consumers?
To test the reality of this representativeness legendary belief, the
current investigation empirically examines how well a sub-sample of
students' overall restaurant image (ORI) ratings represents broader
populations of CD and FD restaurant consumers. The current study
limits the empirical assessment to one specific dependent outcome
construct – ORI –, defined as a person's overall attitudinal-oriented
impression of a restaurant. Using the theoretical underpinnings of
Ajzen and Fishbein's (1980) multiplicative–additive attitude theory, a
person's ORI represents a composite of the importance and cognitive
judgments of performance of a set of common restaurant features
(e.g., variety of food, cleanliness of restaurant, speed of service, food
quality, and atmospherics) for a specific restaurant.
College students tend to be more similar to other consumers within
the target populations of CD restaurant consumers than FD restaurant
consumers. Although college students can be members of both types
of restaurant populations, one expects college students to display ORI
ratings similar to the ORI rating of non-student casual restaurant con-
sumers. The underlying rationale for this prediction is because the attri-
butes of CD restaurants (e.g., food variety, speed of service, and price)
are more consistent with the wants and desires of college students
than for FD restaurant attributes. Students have less first-hand experi-
ence with FD restaurants, which will impede their ability to judge ORI
for FD restaurants. Therefore, expectations are students will rate their
FD ORI rating more similar to their CD ORI, than non-student consumers
due to a lack of first-hand consumption experience with FD restaurants.
The above rationale supports the following hypotheses:
H1. No differences occur in the overall CD restaurant image ratings for
college students and non-student consumers.
H2a. Significant differences exist in the overall FD restaurant image
ratings for college students and non-student consumers.
H2b. College students' FD restaurant image ratings are more similar to
their overall CD restaurant image ratings than non-student consumers'
FD restaurant images to their CD restaurant image ratings.
4 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx
3.2. Legendary belief II: college students are more motivated to participate
in research than non-student consumers
The second legendary belief states that college students demon-
strate an eagerness and willingness to participate in research (Bello
et al., 2009). Some researchers claim that college students are
eager and willing to serve as research participants. However, this
belief is more likely the result of academic researchers using some
form of an incentive to motivate students to participate in research
(Cunningham et al., 1974; Lamb & Stem, 1979; Ok, Shanklin, and
Back (2008). For example, college students can typically be persuad-
ed to participate in a lab experiment or complete some type of survey
through offerings of extra credit, small gifts, or sums of money
(Malaviya & John, 2001; Tangpong & Ro, 2008) because such induce-
ments are more meaningful to students than non-students. Since the
typical incentive matches the desires of college students more so
than non-college consumers, students appear more willing to partic-
ipate in experiments, complex research projects, longitudinal stud-
ies, and surveys than non-college student participants (Bello et al.,
2009; Lamb & Stem, 1979; Malaviya & John, 2001; Tangpong & Ro,
2008). Part of the rationale is that collecting data from managers re-
quires managers to step away from their normal responsibilities,
meaning a larger incentive than $3 would be necessary (Bello et al.,
2009).
While some researchers portray college students as willing research
participants, in reality, most students do not seek out research opportu-
nities. In some situations, students must become part of a subject pool as
a requirement of their degree program. More frequently, researchers
bring research opportunities to students, and justify such practices
through rationalizations of affordability and accessibility (Bello et al.,
2009; Malaviya & John, 2001). When costs and time-frames are prohib-
itive, researchers tend to rely on incentivized college students instead
of more representative subjects from the target population. The review
of the four high-ranking marketing journals illustrates the dispropor-
tionate trend of incentivizing students, with 65% of the articles using
student samples offering some form of incentive to motivate participa-
tion, compared to 17% of the articles using non-student samples offering
incentives.
The inclusion of sample units from a sample frame depends pri-
marily on how well the sample frame members represent the target
population, not on how accessible or inexpensive sample frame
members are (even though these factors cannot be completely
ignored). Inducements to increase a person's willingness to partici-
pate in research have the ability to alter response patterns, motiva-
tions, and ultimately the external validity of research (Burnett &
Dunne, 1986). For example, college students have more free time
available, and therefore, are more willing to donate their time
(Pentecost & Andrews, 2007). Carlson (1971) and Croson (2007)
caution that the use of incentives can lead to uncertainty or bias in
the conclusions drawn from incentivized student subjects and re-
searchers must use care in determining if a student's motivation to
participate is driven by the incentive, rather than the qualifying
characteristics for participation.
To empirically investigate the ability of incentives to alter the
motivation of research subjects, respondent's ORI ratings of CD
restaurants are compared across three different samples: incentivized
college students (those receiving extra credit for participation), non-
incentivized college students (those required to participate as a class
requirement), and non-student restaurant consumers. Since legendary
belief I argues that college students are more representative of the
CD restaurant target population, the remainder of the article focuses
only on the CD restaurant population. Given the above rationale, incen-
tivizing students is predicted to alter their motivation for participating
in the research study (i.e., students complete a survey to get extra
credit without much concern about their responses) resulting in
significant differences between the CD ORI ratings of incentivized
Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
and non-incentivized college students, and non-college consumers.
Therefore:
H3. Overall CD restaurant image ratings differ between incentivized
college student respondents and non-incentivized college student
respondents.
H4. Overall CD restaurant image ratings differ between incentivized
college student respondents and non-student restaurant consumers.
3.3. Legendary belief III: use college students because they are more
homogeneous
The third legendary belief states that the level of similarity (or
homogeneity) among college students' demographic characteristics
(e.g., age, life experiences, and income resources) makes college
students an appropriate sample frame. From a research perspective,
homogeneity within the sample frame can be desirable because some
research designs require a sample frame with minimal noise and vari-
ability, to provide tight manipulations of causality or tests of hypotheses
(Peterson, 2001; Thomas, 2011). Furthermore, homogeneity is a
requirement of many statistical processes, such as ANOVA, MANOVA,
and discriminant analysis (Hair et al., 2010).
While homogeneity can enhance internal validity in laboratory
experiments, homogeneity cannot be determined prior to running an
experiment. Too much homogeneity can inhibit research, or artificially
boost internal validity and provide conservative effect size estimates
(Malaviya & John, 2001; Oakes, 1972; Peterson, 2001; Thomas, 2011).
For example, Churchill and Peter (1984) point out that scale measures
developed for investigating marketing relationships in one highly ho-
mogeneous population may require redesigning for different defined
target populations. While some researchers claim that college students
represent one of the most homogeneous sample frames possible
(Peterson, 2001; Thomas, 2011), researchers often overlook personality,
value trait, and information processing differences of students
(Creswell, 2009). For example, college students are more educated
than the general US population (Pham, 2013), more conservative in
their opinions than older adults (Ro & Kubickova, 2013), and may differ
in job experience, age, cognitive abilities, test scores and geographic lo-
cation characteristics (Thomas, 2011). Researchers need to describe and
evaluate the participation criteria to determine when student subjects
can serve as qualified respondents and the conditions that preclude
their participation (Huang et al., 2008; Peterson & Merunka, 2014).
Researchers might compromise the homogeneity of a sample frame
by confining the selection of students to one institution. Such confine-
ments place geographical limitations on the sampled student units,
which can further decrease heterogeneity of a contextual target popula-
tion. The current investigation predicts that a highly homogeneous stu-
dent sample drawn from the same institution will show less variation in
ORI ratings for CD restaurants than a more geographic heterogeneous
sample of students drawn from different institutions across the United
States. In addition, the expectation is that a randomly drawn sample
of non-student restaurant consumers from across the US will show
the most variation in ORI ratings for CD restaurants. Thus,
H5. Overall CD restaurant image ratings from college students at the
same institution show less variation than CD image ratings from college
students at different institutions.
H6. Overall CD restaurant image ratings from college students show
less variation than CD image ratings from non-student consumers
4. Method
The data for testing the hypotheses come from a multiple-method
restaurant image study. First, the qualitative phase consists of a focus
 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx
group and in-depth interviews among randomly selected college stu-
dents from a major Southeastern university and non-student individ-
uals from a metropolitan market of over 3.5 million people. To ensure
participants represent restaurant consumers, researchers screened all
participants on their dining experiences at CD and FD restaurants. The
main objective was to gather preliminary insight on consumers' restau-
rant patronage dining patterns, familiarity of types of franchised CD and
FD restaurants, as well as identifiable restaurant features that con-
sumers use in creating restaurant images. Based on qualitative insights,
a restaurant image survey was created and administered to different
randomly selected samples of college students and non-student restau-
rant consumers.
4.1. Contextual restaurant setting and attributes
The contextual setting of the restaurant image study involves the use
of two distinctive types of restaurants: casual-sit-down (CD) and fine
dining (FD) restaurants. The preliminary results and insights from the
qualitative pretest suggest that both college students and non-student
consumers have experience and familiarity with CD restaurants, and
non-student consumers have more experience and familiarity with FD
restaurants. Using a second pretest study, 40 randomly selected college
students and 60 non-student restaurant consumers were asked ques-
tions about their own familiarity and actual consumption experiences
with a list of sixteen CD and FD restaurants identified through the
focus group and in-depth interview dialogues, to learn which specific
restaurants to include in the main study. The results indicate that partic-
ipants were most familiar and experienced with Chili's Grill and Bar and
Applebee's Neighborhood Grill and Bar as franchised CD restaurants,
and Ruth's Chris Steak House and Morton's Steakhouse as franchised
FD restaurants. The results support the use of CD and FD restaurants
as the contextual setting because both college students and non-
student individuals represent consumers who eat at these particular
restaurants.
Although numerous attributes contribute to a person's restaurant
and dining experience, the comprehensive restaurant image investiga-
tion limits the scope to fourteen restaurant features serving as attitudi-
nal image indicators, based on evidence from the marketing literature
and the qualitative pretest. The literature provides empirical evidence
for relationships between restaurant image and restaurant attributes,
such as assortment of merchandise (Hoch, Bradlow, & Wansink,
1999), convenience of location (Chebat, Sirgy, & St-James, 2006), friend-
ly/courteous employees, quality of merchandise/service offerings, mer-
chandise/service prices (Arnould, 2005; Haytko & Baker, 2004; Ortinau
& Babin, 2007), service elements (Brennan & Lunsten, 2000; Sirohl &
McLaughlin, 1998), store atmospherics — music, lighting, air condition-
ing, layout (Babin & Attaway, 2000; Baker, Parsuraman, Grewal, & Voss,
2002), cleanliness of store and adequate parking (Finn & Louviere,
1996). In addition, the pretest results suggest that food taste, offering
of price discounts, speed of service, past experiences with restaurants,
and restaurant familiarity may also influence consumers' overall restau-
rant images.
4.2. Defined target population and sample frames
Identification and description of the contextual target population
play significant roles in drawing representative samples. Prior to
conducting the replication studies, the contextual restaurant population
is described as adult male and female (18 to 70 + years of age) con-
sumers living in the US, who are familiar with and have dining experi-
ence at Chili's Grill and Bar and Applebee's Neighborhood Grill and Bar
restaurants, as well as Ruth's Chris and Morton's Steakhouse
restaurants.
Each of the three separate replication studies comes from different
sampling frames. In the first study, the sample frame and units include
college students who randomly enrolled in marketing research and
Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
5
buyer behavior courses at one Southeastern University. Participation
in the study was mandatory as one of the overall requirements of the
courses and offered no extra incentives for participation. This approach
resulted in a usable sample size of 138 participants.
In the second study, the sample frame and units include college stu-
dents who randomly enrolled in marketing courses at a variety of urban
universities throughout the United States. Using partial randomization
procedures, this sample of college students emphasizes student volun-
teerism with incentives for participation. More specifically, twenty-
one universities located in metropolitan consumer markets across the
US were randomly selected and one marketing faculty at each institu-
tion was randomly selected and contacted asking for help in obtaining
student responses to the restaurant image study. Researchers under-
took extra efforts to ensure the metropolitan areas of the 21 selected
universities contained store locations of each of the CD and FD restau-
rants included in the main study. Confirmation of willingness to help
came from 15 marketing faculty, each indicating that they would ask
their current students to participate on a volunteer basis with the
opportunity to receive some extra credit points as an incentive. This
approach resulted in a usable sample size of 438 participants.
In the third study, the sample frame and units were restricted to
non-student restaurant consumers living in metropolitan markets
across the US that contain the specific CD and FD restaurants. Therefore,
the sample units consisted of adult males and females between the ages
of 31 and 70+ living in large metropolitan markets across the US, famil-
iar with and having dining experience with the selected CD and FD
restaurants and were randomly recruited for participation through
Qualtrics Corporation, Inc. panel pools. This third replication study con-
tains a usable sample size of 327 non-student restaurant consumers.
While efforts were made to draw participants from markets containing
restaurant locations of the focal restaurants, a regression was run on
ORI, including the fourteen attributes and location, to rule out location
as an alternative explanation (location p N .10).
4.3. Construct and measures
Given the current study's unique research objectives, the following
discussion highlights the focal construct of the current research, ORI,
as well as the indicators underlying ORI. Although the main emphasis
of this article is not the identification and explanation of causal relation-
ships between multiple constructs or methodological measurement
issues, Table 2 provides the characteristics, reliability and validity infor-
mation for the ORI, familiarity and experience constructs across the
three different samples, for interested readers.
4.3.1. Importance of restaurant attributes
The restaurant image study includes fourteen attitudinal indicators
representing the restaurant features that customers identify and use
to describe the tangible (e.g., competitive prices, restaurant atmospher-
ics, friendly employees) and intangible (e.g., food quality, food taste, ac-
ceptable service) features making up a restaurant (Baker et al., 2002;
Finn & Louviere, 1996). From an attitudinal perspective, customers
form overall attitudes toward these features by combining their impor-
tance feelings and separate performance judgments for each feature.
These features directly influence the formation of a customer's overall
impression of the image they attach to a restaurant (Ajzen & Fishbein,
1980). To capture the intensity of the importance attached to each
restaurant feature, all three studies measure the importance of the
same fourteen-items on five-point continuous rating scales, where
each scale point is narratively described as ranging from 1 = not at all
important to 5 = extremely important.
4.3.2. Performance of restaurant attributes
The performance construct represents the customer's evaluative
judgment of how well each selected restaurant performs on each
of the fourteen-attitudinal indicators. The performance data for each
6 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx

Table 2
Summary of constructs' characteristics, reliability, and validity assessments.

Construct # of items μ σ Inter-correlation range CFA factor loading range Cronbach's alpha AVE

CD familiaritya 2 3.8 1.03 .55 .73–.76 .71 .77

CD experienceb 2 2.6 3.15 .52 .57–.66 .68 .76
CD imagec 14 4.0 .92 .29–.83 .65–.94 .93 .63
FD familiaritya 2 1.8 1.04 .59 .76–.78 .74 .79
FD experienceb 2 .5 1.59 .79 .71–.72 .88 .84
FD imagec 14 3.7 1.38 .11–.91 .65–.92 .94 .69
a
Five-point standardized scale ranging from 1 = not at all familiar with CD (FD) restaurants to 5 = extremely familiar with CD (FD) restaurants, summed across the two CD
(FD) restaurants.
b
Standardized actual frequency scale ranging from 0 = 0 times with CD (FD) restaurants to 14 times = quite often with CD (FD) restaurants, summed across the two CD (FD)
restaurants.
c
Created according to the Ajzen and Fishbein multiplicative-additive model by using the importance and performance measures of fourteen restaurant-items, and
transposing into a six-point aggregate restaurant image scale ranging from 1 = very negative image to 6 = very positive image.

selected CD and FD restaurants are captured using unique six-point, characteristics allowed us to investigate the homogeneity of members
continuous assumed interval letter grade scales, where each scale of a contextual target population.
point is narratively described as ranging from 1 = terrible (F) to 6 =
Outstanding (A+). 5. Analysis and results

4.3.3. Overall restaurant image
Consumers' expressed overall attitudinal scores toward each CD and
FD restaurant were initially derived using Ajzen and Fishbein's (1980)
multiplicative–additive attitude model, where each attribute's perfor-
mance is multiplied by its corresponding importance weight factor,
and then the fourteen multiplicative values are summed to derive an
overall attitudinal score. The overall attitudinal score is then transposed
into an image intensity using a six-point image scale, where each
scale point is narratively described as ranging from 1 = very negative
restaurant image to 6 = very positive restaurant image (alphas for CD
image = .93, FD image = .88).
4.3.4. Characteristics of restaurant consumers
The current investigation captured eight basic characteristics includ-
ing gender, age, marital status, employment status, income, education,
number of dependent children, and likelihood of dining out. These
With the primary research objective to empirically investigate three
deep-rooted legendary beliefs concerning college students data sources,
the focus of this research is to determine the existence of ORI rating
differences between known, but limited sample segments by using
three replication studies. Planned contrast differences and Z-testing
procedures test the legendary beliefs. Table 3 displays the summary of
the hypotheses testing results.
5.1. Defined target restaurant population and characteristic differences
between college students and non-student restaurant consumers
Researchers need to have some idea of the characteristics of the peo-
ple who make up the contextual target population of interest to assess
the representativeness of their research findings, inferences, and impli-
cations from the drawn sample to the broader target population.
Although none of the three-replication studies, by themselves, employ

Table 3
Summary of hypothesis testing result.

Hypothesis μ SE σ σ2 Z-value Results

H1: No differences exist in the overall CD restaurant image ratings for college students and
non-college consumers.
College students [n = 576] 3.9 (.04) .80 .64 4.32 Reject (p b .01)
Non-college consumers [n = 327] 4.2 (.05) .98 .97
H2A: Significant differences exist in the overall FD restaurant image ratings for college
students and non-college consumers.
College students [n = 576] 3.7 (.07) 1.60 2.55 .23 Reject (p N .10)
Non-college consumers [n = 327] 3.7 (.05) .93 .87
H2B: College students' FD image ratings will be more similar to their overall CD restaurant
image ratings than non-college consumers' FD image ratings are to their CD image ratings.
College students' FD image [n = 576] 3.7 (.07) 1.60 2.55 2.44 Support
College students' CD image [n = 576] 3.9 (.04) .80 .64
Non-college consumers' FD image [n = 327] 3.7 (.05) .93 .87 6.53
Non-college consumers' CD image [n = 327] 4.2 (.05) .98 .97
H3: Overall CD restaurant image ratings differ between incentivized college student respondents
and non-incentivized college student respondents.
Incentivized students CD image [n = 438] 3.8 (.04) .87 .75 1.71 Accept (p b .10)
Non-incentivized students CD image [n = 138] 4.0 (.07) .82 .68
H4: Overall CD restaurant image ratings differ between incentivized college student respondents
and non-student consumers.
Incentivized students CD image [n = 438] 3.8 (.04) .87 .75 4.64 Accept (p b .01)
Non-college consumers' CD image [n = 327] 4.2 (.05) .98 .97
H5: Overall CD restaurant image ratings from college students at the same institution will show
less variation than CD image ratings from college students at different institutions.
Same school students' CD image [n = 237] 4.0 (.05) .80 .64 2.65 Accept (p b .01)
Nation-wide students' CD image [n = 339] 3.8 (.05) .91 .82
H6: Overall CD restaurant image ratings from college students will show less variation than CD
image ratings from non-college consumers.
College students' CD image [n = 576] 3.9 (.04) .86 .74 4.31 Accept (p b .01)
Non-college consumers' CD image [n = 327] 4.2 (.05) .95 .90

Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx
a representative sample of the overall defined target population, com-
bining the studies together as one large comprehensive study of 903
restaurant consumers (non-student consumers = 327 and college
students = 576) comes closer to offering an overall sample of restau-
rant consumers representative of the restaurant target population
than any sample alone.
The current study argues that characteristic differences between col-
lege students (CS) and non-student restaurant consumers (NonC) con-
tribute to the observed differences of overall CD and FD restaurant
images. Non-college student consumers, on average, are approximately
30 years older than college students (NonC — 50.4, CS — 22.2: p b .01)
and more likely to be married (NonC —54%, CS — 4%) or divorced
(NonC — 15%, CS — 1%) than college students. Significantly more non-
student consumers are employed full-time, but less have part-time
jobs compared to college students (NonCfull-time — 50%, CS — 18%;
NonCpart-time — 14%, CSpart-time — 54%: p b .01). College students
earn a mean annual income of $31,887, while non-student consumers'
earn a significantly higher mean annual income of approximately
$54,925 (p b .01). The vast majority (93%) of college students have com-
pleted some college; whereas, 33% of non-student consumers hold a
four-year or graduate degree with another 24% holding a two-year
AA/AS degree, and another 16% indicating they have some college expe-
rience. Non-student consumers have more dependent children living at
home than college students (NonCdependent — 1.9, CS dependent
— 1.4: p b .01). Non-student consumers dine out less often in a 30-day
period than college students (NonC — 5.8, CS — 8.2: p b .01). Gender
did not vary between non-college and college students (NonCfemale
— 57.2%, CSfemale — 51.7%; NonCmale — 42.8%, CSmale — 48.3%).
These characteristic differences provide evidence that college stu-
dents differ from non-student consumers, who are also included in
the overall contextual target population of CD and FD restaurant con-
sumers. The characteristic information also provides evidence that
despite potential unique living arrangements (e.g., living in dorms, on
campus, and with roommates) and lower income, college students actu-
ally eat out at restaurants more frequently than non-student con-
sumers. When characteristic differences do exist, researchers must
assess for possible moderating (or confounding) effects on the research
results, findings, and implications.
5.2. Legendary belief I: generalizability of college student samples to broad
contextual target populations
The first legendary belief investigates realities of truth concerning
the representativeness of college students to a defined target population
and the researchers' ability to generalize research findings, inferences,
and implications to that broader target population. H1 postulates invari-
ance between the CD ORI ratings held by college students and non-
student restaurant consumers. The test results indicate that, on average,
non-student consumers' CD ORI ratings (NonCCDImage = 4.2) are sig-
nificantly more positive than those ORI ratings held by college students
(CSCDImage = 3.9: p b .01); thus, H1 is rejected. H2a predicts that a
significant difference exists between the FD ORI ratings held by college
students and non-student restaurant consumers. The test results fails to
support FD ORI rating differences between college students and non-
college consumers (CSFDImage = 3.7, NonCFDImage = 3.7: p N .10),
resulting in rejection of H2a. H2b postulates that college students will
assimilate their FD ORI ratings closer to their CD ORI ratings than non-
student consumers. To test H2b, the differences between the FD and
CD images were calculated for both college students and non-student
consumers, and then those differences (Δ) tested. The results suggest
that college students assimilate their FD ORI ratings closer to their CD
ORI ratings (|CSFDImage 3.7 − CSCDImage 3.9| = | Δ | = | .20 |) more
so than non-student consumers (|NonCFDImage 3.7 − NonCCDImage
4.2| = | Δ | = | .50 |: p b .01), providing empirical evidence that supports
H2b.
Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
7
5.3. Legendary belief II: college students are more motivated to participate
in research than non-student consumers
The second legendary belief investigates realities of truth concerning
the eagerness of research subjects to participate in research due to being
qualified subjects, versus being encouraged through incentives. H3 pre-
dicts that CD ORI rating differences exist between incentivized college
student (students receiving extra credit points for participating) and
non-incentivized college student participants. Utilizing a traditional
Z-testing procedure, the results provide marginal empirical evidence
that incentivized college students produce, on average, significantly
less positive CD ORI ratings than do non-incentivized college students
(CSIncentivized CDImage = 3.8, CSnon-incentivized CDImage = 4.0:
p b .10). The results marginally support H3.
To understand further the implications of incentivizing college stu-
dents in an effort to gain their willingness to participate in research,
H4 postulates that incentivized college students provide different CD
ORI ratings than non-student restaurant consumers. The test results
provide empirical evidence that non-student consumers, on average,
hold a significantly stronger positive CD ORI than do incentivized col-
lege students (NonC-CDImage = 4.2, CSIncentivized CDImage = 3.8:
p b .01), providing support for H4.
5.4. Legendary belief III: use college student because they are more
homogeneous
The third legendary belief focuses on the degree to which research
participants need to be similar as a data source. To investigate the real-
ities of truth within this belief, the current study explores the influence
that the homogeneity traits of college students have on capturing the
amount of variation in their CD ORI ratings. H5 predicts that college
students from the same higher-education institution will produce less
variation in their CD ORI ratings than do college students from different
higher education institutions. The test results provide empirical evi-
dence that, on average, college students from the same higher educa-
tional institution produce CD ORI ratings (CSsame school CDImage =
4.0, σ2 = .64) having significantly less variation than the CD ORI ratings
of students from different higher education institutions (CSdifferent
schools CDImage = 3.8, σ2 = .82: p b .01), providing support for H5.
Finally, H6 postulates that college students, in general, produce CD ORI
ratings that have less variation than do non-student CD restaurant con-
sumers. Test results indicate significantly less variation in the CD ORI
ratings of college students (CSCDImage = 3.9, σ2 = .86) than non-
student CD consumers (NonCCDImage = 4.2, σ2 = .95: p b .01), giving
support to H6.
6. Discussion and implications
The legendary beliefs regarding the use of college students as a data
source fuel some of the earlier reported current trends in academic re-
search from the marketing literature review (see Table 1), including
the types of research opportunities offered to students, recognition of
sample limitations, and offerings of incentives. An important objective
of the current research is to uncover the hidden truths regarding these
trends in an effort to enhance the reader's understanding of the reason-
ableness of college student data sources.
6.1. Legendary belief I
The first legendary belief addresses the ability of college students to
represent a broader target population. Although the literature suggests
that representation depends primarily on whether respondents are
asked to perform fundamental (a task any person should be able to
do; Bello et al., 2009) or proximate processing activities (a task that
depends upon the contextual problem; Knemeyer & Naylor, 2011), the
current study finds that demographic, motivation, familiarity, and
8 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx
experience factors determine the ability of samples of college students
to represent a broader population.
The results of H1, H2a, and H2b show that college students are not
equivalent to non-student consumers on a number of dimensions.
First, college student CD ORI ratings are significantly less positive than
non-student consumers' ratings. Furthermore, researchers must under-
stand that when the constructs of interest are attitudinal or affective in
nature (i.e., ORI) the respondents' level of familiarity and experience
with the contextual setting of interest needs to be captured. Similarity
in the demographic characteristics of respondents does not assure re-
searchers that other differences do not exist along attitudinal, affective
or psychographic dimensions.
When using college students or any other type of sampling frame
(e.g., M-Turk crowdsourcing pool or online consumer source panels),
researchers must use some type of randomization procedure to
select sample units, and avoid easily obtained convenience or quota
samples. Generalizing results and findings from convenience samples
(e.g., online consumer source panels and M-Turk crowdsourcing pool)
apply only to those individuals in the sample or sample frame, who par-
ticipated in the study. Similarly, recruiting students from one location is
not representative of all students. The contextual setting of a research
project plays a role in establishing internal and external validity, and
the researchers' ability to generalize results, findings, inferences, and
implications beyond just the people who were included in the research.
Although not part of the current study, another challenge
concerning sample representativeness requiring further investigation
is the prevalence of convenience or quota samples of students and
non-students to generate empirical and experimental data structures.
For example, the appropriateness of using sample units from Amazon's
M-Turk crowdsourcing pool (c.f., Buhrmester, Kwang, & Gosling, 2011;
Golden, Larson, & Smith, 2011) or online consumer panels (c.f., Baker
et al., 2010; Brick, 2011) represents a growing research concern. These
types of nonprobability samples defy a researcher's ability to provide
any meaningful statistical or managerial inferences about members of
a broader contextual population. For an in-depth discussion of problem-
atic concerns with using M-Turk crowdsourcing and online consumer
panel sample frames, see Smith, Roster, Golden, and Albaum (2015).
6.2. Legendary belief II
The second legendary belief addresses data collection affordability,
accessibility, and the motivation of research participants through incen-
tives. The results of H3 and H4 indicate that incentivized college
students hold significant, but only slightly lower (p b .10) overall CD
restaurant image ratings than non-incentivized college students. In con-
trast, incentivized college students hold significantly lower overall CD
restaurant image ratings than non-student restaurant consumers. The
current study does not try to explain why incentivized students' CD
ORI ratings are, on average, lower than non-incentivized students'
image ratings, yet intuitively two factors may be causing this.
First, ORI represents an attitude and variations in the expressed
image occur naturally and independently of any type of mechanism
(incentives) used to increase the likelihood of participation. Second,
the non-incentivized students, who completed the survey as part of
their course, may be motivated differently than those students who re-
ceived some extra credit points. Furthermore, not only do these results
show that incentivization can alter the ratings provided by research
subjects, but these results also show that incentivization does not
come with a 100% guarantee that participation takes place. Illustrating
this point, the current study secured the help of fifteen marketing facul-
ty members across the country that offered students an incentive of
earning extra credit points to participate. On average, each faculty mem-
ber had access to about 150 students, meaning a potential so sample
size of 2250 students. The current study only netted a sample size of
438 (or 19.5% response rate) incentivized students.
Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
Although not part of the current study's scope, concerns are also
emerging about students, who participate in a lot of research surveys
or lab experiments just to earn the incentive. These professional
research respondents create a new potential confounding factor by
changing the response patterns, motivations, and ultimately the inter-
nal and external validity of the research. Researchers need to somehow
assess and account for when analyzing and testing their empirical data
structures. Crowdsourcing samples, like those obtained through
Amazon's M-Turk, or online consumer panel data samples may also be
prone to professional confounds, and this remains a critical topic that
is only recently receiving some attention; see Smith et al. (2015) for a
detailed discussion of these issues.
6.3. Legendary belief III
The third legendary belief addresses the degree of similarity
amongst members of a sampled group of respondents. Experimental re-
searchers present convincing arguments to use college student samples
as research subjects based on their homogeneity. However, homogene-
ity of a sample group should not be based primarily on a few selected
demographic characteristics of the group's members, but should also
use pertinent non-demographic factors including attitudinal, affective,
psychographic, knowledge, experiential, and motivational factors. H5
test results indicate significant differences in the CD ORI ratings of col-
lege student respondents from the same university compared to ratings
of college students from different universities, even though both groups
have similar demographic factors. The test results of H6 challenge the
importance of group homogeneity. College students hold significantly
lower CD restaurant image ratings than do non-student, restaurant
consumers. The results indicate that college students appear similar
on demographic characteristics, while non-student consumers appear
different on most demographic characteristics. While college students
and non-student consumers are both part of the target population of
restaurant consumers, neither subgroup of consumers, by themselves,
represents the contextual population of interest.
7. Limitations, future research, and final comments
7.1. Limitations and future research
Due to space concerns, the current study investigates only three leg-
endary beliefs (representativeness, incentivization, and homogeneity)
regarding the reasonableness of college student samples. Additional rel-
evant legendary beliefs exist concerning sample memberships and gen-
eralizability of results that need to be empirically explored. For example,
the representativeness of data obtained through private research com-
panies like Qualtrics Corporation, Inc. and Amazon's Mechanical Turk
online survey panels represents a growing legendary belief requiring
debunking. While many other empirical research endeavors quickly
gloss over generalizability limitations and call for more research, the
current investigation focuses on identifying the strengths and weak-
nesses of college student samples.
In addition, the current study limits the investigation of
incentivization to only three sub-samples of restaurant consumers
(non-student consumers, incentivized college students, and non-
incentivized college students) within a restaurant image context.
While some aspects of incentivization of students are addressed, at an
aggregate level, in legendary belief II, additional research is needed on
this topic. The finding that incentivization alters ORI ratings leads to
some unanswered questions about the makeup of subject pools, the fre-
quency of usage of subject pools, and the potential creation of profes-
sional survey takers or experimental subjects. In addition, there is a
need for more research to investigate the influence of different types
of incentives (e.g. extra credit vs. money) on attitudes.
A greater limitation is that most of the students involved in this re-
search are marketing or business students. Although building elements
 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx
of randomness into the selection procedures for obtaining both college
students and non-student participants, the different student samples
of the current study do not fairly represent all students; therefore, cau-
tion is necessary about making certain generalizations across academic
disciplines (e.g., generalizing from business students to medical or engi-
neering student-restaurant consumers). Investigating the representa-
tiveness of additional samples from different university departments
would further enhance the reader's understanding of how the nature
of the sample influences the generalizability of responses to the defined
target population. In addition to looking at different types of college stu-
dent samples, the departmental and institutional culture may also influ-
ence the usage of college student samples. An interesting research
question not covered in this article, but relevant for future research
is — how does a departmental culture shape the researcher's choice of
sample subjects?
Although the restaurants in this study were pretested and selected
based on subjects' familiarity, many college students and some non-
student participants were unfamiliar with the two FD restaurants.
While the lack of familiarity poses a potential limitation to this study,
the unfamiliarity actually falls in line with the current study's underly-
ing arguments, and is another indication that researchers must take
care in identifying and describing the target population. Finally, the cur-
rent research examines only academic legendary beliefs pertinent to
drawing student samples and using college students as data sources,
but managerial legendary beliefs likely exist as well. Managers may
have intuitive rules of thumb they employ when selecting participants
to employ in a study. Investigating managerial legendary beliefs
presents another manner in which to extend insights on the topic and
facilitate a more representative selection of samples.
7.2. Closing comments
In conclusion, when conducting research projects, researchers need
to identify and carefully describe the target population. No matter how
hard a researcher tries, or how elegant the justification, if the contextual
setting does not suggest that college students represent a broader target
population, college students should not be the only source of data.
Assessing the ability of college student samples to represent a broader
population needs to be through multiple samples and/or a thorough
discussion of limitations created by the chosen sample. Such assess-
ments are necessary because generalizability and heterogeneity depend
on many demographic, attitudinal, affective and psychological charac-
teristics. Therefore, determination of sample homogeneity cannot be
simply by comparing a few demographic characteristics such as gender,
age, and income.
Researchers need to use probabilistic randomization procedures in
drawing sampling units from the respective sample frames and contex-
tual target population, and critically assess any potential confounds
created by the use of incentives. If generalizability of research results
is to avoid becoming a lost cause in published journal articles, then re-
searchers should refrain from: 1) using college-student data sources
as the sole sample unless the research objectives and questions focus
on a contextual setting specific to a particular college-student target
population and 2) using convenience and quota sampling frames.
Furthermore, to enhance the generalizability of research results and in-
ferences, researchers must randomly select sampling units from sample
frames that more closely represent the members of the broader contex-
tual target population.
Acknowledgments
The authors appreciate the helpful comments and guidance from the
anonymous reviewers, special issue editors, Barry J. Babin (Louisiana
Tech University), Joe F. Hair (Kennesaw State University) and Mitch
Griffin (Bradley University), and JBR editor, Arch Woodside (Boston
College), during the review process. The authors thank Carlin A. Nguyen
Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
9
for his assistance with the data collection, and Nazuk Sharma for her
help with the original concept.
References
Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behaviour.
New Jersey: Prentice-Hall.
Arnould, E. (2005). Animating the big middle. Journal of Retailing, 81(2), 89–96. http://dx.
doi.org/10.1016/j.jretai.2005.03.001.
Babin, B.J., & Attaway, J.S. (2000). Atmospheric affect as a tool for creating value and
gaining share of customer. Journal of Business Research, 49(August), 91–99.
Baker, J., Parsuraman, A., Grewal, D., & Voss, G.B. (2002). The influence of multiple store
environment cues on perceived merchandise value and patronage intentions.
Journal of Marketing, 66(April), 120–141.
Baker, R., Blumberg, S.J., Brick, M.J., Couper, C.P., Courtright, M., & Dennis, J.M. (2010). Re-
search synthesis: AAPOR report on online panels. AAPRO standards committee by
task force. AAPRO standards committee. Public Opinion Quarterly, 74(4), 711–781.
Bello, D., Leung, K., Radebaugh, L., Tung, R.L., & Van Witteloostuijn, A. (2009). From the ed-
itors: Student samples in international business research. Journal of International
Business Studies, 40(3), 361–364. http://dx.doi.org/10.1057/jibs.2008.101.
Beltramini, R.F. (1983). Student surrogates in consumer research. Journal of the Academy
of Marketing Science, 11(4), 438–443. http://dx.doi.org/10.1007/BF02721926.
Brennan, D.P., & Lunsten, L. (2000). Impacts of large discount stores on small U.S. towns:
Reasons for shopping and retail strategies. International Journal of Retail & Distribution
Management, 28(4–5), 155–161. http://dx.doi.org/10.1108/09590550010319904.
Brick, M.J. (2011). The future of survey sampling. Public Opinion Quarterly, 75(5),
872–888.
Buhrmester, M., Kwang, T., & Gosling, S. (2011). Amazon's mechanical Turk: A new source
of inexpensive, yet high-quality data? Perspectives on Psychological Science. http://dx.
doi.org/10.1177/1745691610393980.
Burnett, J.J., & Dunne, P.M. (1986). An appraisal of the use of student subjects in market-
ing research. Journal of Business Research, 14(4), 329–343.
Burns, A.C., & Bush, R.F. (2003). Marketing research: Online research applications (4th ed.).
Upper Saddle River, NJ: Prentice Hall, 265.
Campbell, B.L., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for
research. Boston, MA: Houghton Mifflin.
Carlson, R. (1971). Where is the person in personality research? Psychological Bulletin,
75(3), 203–219. http://dx.doi.org/10.1037/h0030469.
Chebat, J., Sirgy, J.M., & St-James, V. (2006). Upscale image transfer from malls to stores: A
self-image congruence explanation. Journal of Business Research, 59(November),
1492–1502. http://dx.doi.org/10.1016/j.jbusres.2006.09.007.
Churchill, G.A., Jr., & Peter, J.R. (1984). Research design effects on the reliability of rating
scales: A meta-analysis. Journal of Marketing Research, 21(4), 360–375.
Compeau, D., Marcolin, B., Kelley, H., & Higgins, C. (2012). Generalizability of information
systems research using student subjects— A reflection of our practices and recom-
mendations for future research. Information Systems Research, 23(4), 1093–1109.
Creswell, J.W. (2009). Research design: Qualitative, quantitative, and mixed methods
approaches (3rd ed.). Los Angeles, CA: Sage.
Croson, R. (2007). The use of students as participants in experimental research.
Unpublished manuscript, University of Texas, Dallas.
Cunningham, W.H., Anderson, W.T., Jr., & Murphy, J.H. (1974). Are students real people?
The Journal of Business, 47(3), 399. http://dx.doi.org/10.1086/295654.
Dillon, W.R., Madden, T.J., & Firtle, N.H. (1990). Marketing research in a marketing
environment (2nd ed.). Boston, MA: Irwin, 334.
Dobbins, G.H., Lane, I.M., & Steiner, D.D. (1988). A note on the role of laboratory
methodologies in applied behavioural research: Don't throw out the baby with the
bath water. Journal of Organizational Behavior, 9(3), 281–286.
Finn, A., & Louviere, J.J. (1996). Shopping center image, consideration, and choice: Anchor
store contribution. Journal of Business Research, 35(March), 241–251. http://dx.doi.
org/10.1016/0148-2963(95)00129-8.
Golden, L.L., Larson, J., & Smith, S. (2011). Data quality and sample integrity in Internet re-
search. Presented at the 15th biennial world marketing conference. Academy of
Marketing Science, Reims, France: July.
Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R.E. (2010). Multivariate data analysis (7th
ed.). Upper Saddle River, NJ: Prentice Hall.
Hair, J.F., Bush, R.P., & Ortinau, D.J. (2009). Marketing research in a digital information
environment (4th ed.). New York, NY: McGraw-Hill/Irwin, 52.
Haytko, D.L., & Baker, J. (2004). It's all at the mall: Exploring adolescent girls' experiences.
Journal of Retailing, 80(January), 67–83. http://dx.doi.org/10.1016/j.jretai.2004.01.
005.
Hoch, S.J., Bradlow, E., & Wansink, B. (1999). The variety of an assortment. Marketing
Science, 18(4), 527–546.
Huang, X., Gattiker, T.F., & Schwarz, J.L. (2008). Can students pass the test: Should we use
student subjects in operations management research. Proceedings of the 38th annual
meeting of the Decision Sciences Institute (pp. 22–25).
Knemeyer, A.M., & Naylor, R.W. (2011). Using behavioral experiments to expand our
horizons and deepen our understanding of logistics and supply chain decision
making. Journal of Business Logistics, 32(4), 296–302.
Lamb, C.W., Jr., & Stem, D.E., Jr. (1979?). An evaluation of students as surrogates in
marketing studies. (Retrieved from) http://www.acrwebsite.org/search/view-
conference-proceedings.aspx?Id=9246
Lehmann, D.R., McAlister, L., & Staelin, R. (2011). Sophistication in research in marketing.
Journal of Marketing, 75(4), 155–165.
Lynch, J.G., Jr. (1982). On the external validity of experiments in consumer research.
Journal of Consumer Research, 9(3), 225. http://dx.doi.org/10.1086/208919.
10 J.A. Espinosa, D.J. Ortinau / Journal of Business Research xxx (2015) xxx–xxx
Lynch, J.G., Jr. (1999). Theory and external validity. Journal of the Academy of Marketing
Science, 27(3), 367–376. http://dx.doi.org/10.1177/0092070399273007.
Malaviya, P., & John, D.R. (2001). Recruiting experimental participants. Journal of
Consumer Psychology, 10(1/2), 115–117.
Oakes, W. (1972). External validity and the use of real people as subjects. American
Psychologist, 27(10), 959–962. http://dx.doi.org/10.1037/h0033454.
Ok, C., Shanklin, C.W., & Back, K. (2008). Generalizing survey results from student
samples: Implications from service recovery research. Journal of Quality Assurance in
Hospitality and Tourism, 8(4), 1–23.
Ortinau, D.J., & Babin, B.J. (2007, November). Restaurant image: A driving force behind
customers' overall satisfaction and perceived restaurant loyalty. Tenth SMA-retail
strategy and patronage behavior research symposium (pp. 34–46).
Pentecost, R., & Andrews, L.M. (2007). Differences between student and non students' will-
ingness to donate to a charitable organization. In proceedings of the 2007 Australia
and New Zealand Marketing Academy Conference (ANZMAC) (pp. 1060–1066). New
Zealand: Dunedin.
Peterson, R. (2001). On the use of college students in social science research: Insights
from a second order meta-analysis. Journal of Consumer Research, 28(3), 450–461.
http://dx.doi.org/10.1086/323732.
Peterson, R.A., & Merunka, D.R. (2014). Convenience samples of college students and
research reproducibility. Journal of Business Research, 67(5), 1035–1041. http://dx.
doi.org/10.1016/j.jbusres.2013.08.010.
Pham, M.T. (2013). The seven sins of consumer psychology. Journal of Consumer
Psychology, 23(4), 411–423.
Please cite this article as: Espinosa, J.A., & Ortinau, D.J., Debunking legendary beliefs about student samples in marketing research, Journal of Busi-
ness Research (2015), http://dx.doi.org/10.1016/j.jbusres.2015.12.003
Ro, H., & Kubickova, M. (2013). The use of student subjects in hospitality research:
Insights from subjective knowledge. Journal of Quality Assurance in Hospitality &
Tourism, 14(4), 295–320. http://dx.doi.org/10.1080/1528008X.2013.802624.
Sirohl, N., & McLaughlin, E.W. (1998). A model of consumer perceptions and store loyalty
intentions for a supermarket retailer. Journal of Retailing, 74(2), 223–245. http://dx.
doi.org/10.1016/S0022-4359(99)80094-3.
Smith, S.M., Roster, C.A., Golden, L.L., & Albaum, G.S. (2015). Do bad respondents equal
poor data quality? Comparing a regular online United States consumer panel with
a M-Turk panel. Journal of Business Research: Special Issue (forthcoming).
Steinfatt, T.M. (1991). External validity, internal validity, and organizational reality: A re-
sponse to Robert L. Cardy. Management Communication Quarterly, 5(1), 120–125.
Stevens, K.S. (2011). Question to consider when selecting student samples. Journal of
Supply Chain Management, 47(3), 19–21.
Tangpong, C., & Ro, Y.K. (2008). Student-manager surrogacy in supply chain decision
making. Journal of Business and Management, 14(1), 77–91.
Thomas, R.W. (2011). When student samples make sense in logistics research. Journal of
Business Logistics, 32(3), 287–290.
Webster, M., & Sell, J. (2007). Laboratory experiments in the social sciences. Burlington, MA:
Elsevier Inc.
Wells, W.D. (1993). Discovery-oriented consumer research. Journal of Consumer Research,
19(4), 489. http://dx.doi.org/10.1086/209318.