See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/220014626

Personality Testing and Industrial-Organizational Psychology: A

Productive Exchange and Some Future Directions

Article  in  Industrial and Organizational Psychology · September 2008

DOI: 10.1111/j.1754-9434.2008.00057.x

CITATIONS READS

10 556

2 authors, including:

Frederick L Oswald
Rice University
185 PUBLICATIONS   10,648 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Relative Importance View project

Impact of team configuration and team stability on primary care quality View project

All content following this page was uploaded by Frederick L Oswald on 24 June 2020.

The user has requested enhancement of the downloaded file.

Industrial and Organizational Psychology, 1 (2008), 323–332.
Copyright ª 2008 Society for Industrial and Organizational Psychology. 1754-9426/08

RESPONSE

Personality Testing and Industrial–

Organizational Psychology: A Productive
Exchange and Some Future Directions

FREDERICK L. OSWALD
Michigan State University
LEAETTA M. HOUGH
The Dunnette Group, Ltd.

Abstract
The goal of our focal article was to provide a current perspective on personality testing and its use in organizational
research and to elicit constructive discussion and suggestions for future research and practice. The present article caps off
the discussion by integrating the main ideas presented in the commentaries within our original framework of questions
and topics, with the immodest hope of advancing our understanding of personality and its measurement in the context of
industrial–organizational psychology. In short, we recommend continuing to take advantage of the organizing frame-
work of the Big Five while also pursuing more ‘‘bottom-up’’ approaches that examine facet-level relationships with
multidimensional performance outcomes, in addition to developing process models that include more proximal moti-
vational and situational variables. Work along these lines is valuable to both organizational science and practice.

In our focal article (Hough & Oswald, 2008),
our goal was to provide a current perspective
on personality testing and its use in organi-
zational research and to elicit constructive
discussion and suggestions for future
research and practice. We are thrilled with
the wide range of insightful commentary that
our article inspired, and more generally that
this new Society for Industrial and Organi-
zational Psychology’s journal is creating
a useful and exciting intellectual exchange.
Below, we have integrated the ideas pre-
sented in the commentaries into our original
framework of questions and topics, compar-
Correspondence concerning this article should be
addressed to Fred Oswald, E-mail: foswald@rice.edu
Address: Rice University, Department of Psychol-
ogy, 6100 Main Street, MS 205, Houston, TX 77007
Frederick L. Oswald, Department of Psychology,
Michigan State University; Leaetta M. Hough, The Dun-
nette Group, Ltd.
ing and contrasting the differing and some-
times opposing points of view.
Multidimensional Models of Job
Performance, Personality
Measurement, and Taxonomic Issues
We noted in our focal article that (a) multi-
dimensional models of job performance had
become more refined over the past 2 de-
cades, in particular incorporating more
personality-relevant performance constructs;
(b) such refinement should enable us to focus
more precisely on those personality con-
structs that enhance our understanding of
the relationships between personality varia-
bles and performance; (c) as a taxonomic
structure, each factor of the five-factor model
of personality is usually too heterogeneous,
a facet-level approach to validity is a
‘‘bottom-up’’ approach that can tell us em-
pirically whether it is more useful than the

323
324
five-factor model in building our science
and practice; (d) additional theory-based
and process-based variables need to be incor-
porated into our models relating personality
to performance; and (e) we should build
detailed cumulative databases that contain
information about personality and criterion
relationships, systematically organized by
measurement method, situation, and other
known moderator variables.
Stewart (2008) suggests that to focus on
narrower personality constructs, as we have
encouraged, is to sacrifice or at least turn
a blind eye to the validity that has been found
for broader personality constructs predicting
broad performance behaviors. We agree that
from a practical perspective, broad personal-
ity traits can be predictive of similarly broad
work behaviors (Hough & Ones, 2001). In
fact, the more complex the performance cri-
terion is, the more multifaceted the person-
ality predictor may have to be (Hogan &
Roberts, 1996; Hough & Ones, 2001). The
prediction afforded by broad personality
constructs, however, does not preclude the
need for additional research on relationships
between personality facets and performance
dimensions. Do all facets of Conscientious-
ness contribute to the prediction of absentee-
ism or are only some of them responsible? Do
different facets of Conscientiousness predict
attention to detail for detail-oriented jobs?
and Do facets of Conscientiousness help
us better understand the mediating and
moderating variables that qualify or explain
Conscientiousness–performance relation-
ships? Research involving facet-level per-
sonality measures and multidimensional
performance outcomes helps address ques-
tions like these, and it also helps to make very
general statements about the Big Five factors
predicting job performance more useful
and precise. In other words, both practice
and theory can improve when we understand
better exactly how and when broad traits are
predictive. Personality-based job analysis
has led to developing performance criteria
that are predicted better by facets than by
broader constructs (Jenkins & Griffith, 2004).
But neither Stewart nor you, gentle
reader, should mistake us: We are not sug-
F.L. Oswald and L.M. Hough
gesting, for instance, that traits be conceptu-
alized and measured on the independent
variable side so narrowly that personality
items essentially repeat the same question
just to raise the alpha level or that job perfor-
mance on the dependent variable side be
broken down into constituent parts that are
microscopic or mechanistic. Furthermore,
we agree with Stewart that personality traits
are especially predictive in the sort of open
social and teamwork environments that he
describes; however, we argue that an empir-
ical understanding that facets are driving
prediction by the Big Five can be illuminat-
ing. We are not suggesting an infinite regress
of ‘‘going narrow.’’ Conceptually and prac-
tically we, like other researchers (e.g.,
Paunonen, Rothstein, & Jackson, 1999;
Schneider, Hough, & Dunnette, 1999; van
Iddekinge, Taylor, & Eidson, 2005), think
that a bottom-up approach to understanding
personality at the facet-level complements
the Big Five top-down approach and may
help us understand how and when personal-
ity is predictive of performance outcomes,
such as for the traits of Conscientiousness
(Roberts, Bogg, Walton, Chernyshenko, &
Stark, 2004; Roberts, Chernyshenko, Stark,
& Goldberg, 2005) and Openness to Experi-
ence (Chernyshenko, Stark, Woo, & Conz,
2008). As Barrett (2008) notes, facets can
have low correlations between one another
within the same Big Five construct and
hence are empirically distinguishable, leav-
ing the door open to finding useful patterns
of differential validity (see also Hough,
1992). We clearly agree with Barrett that
Conscientiousness is too broad and hetero-
geneous of a factor to consider it as a unitary
and consistently valid predictor across all
occupations and criteria. The broad con-
struct of Conscientiousness is not highly pre-
dictive of creative outcomes, for example
(Feist, 1998; Hough, 1992; Hough &
Dilchert, 2007; Hough & Furnham, 2003),
and the Conscientiousness facets of depend-
ability and achievement have demonstrated
differential prediction for performance out-
comes at both the individual and team
levels (LePine, 2003; LePine, Colquitt, &
Erez, 2000). Other differential relationships
Personality testing and industrial–organizational psychology: Response
between personality facets and performance
were discussed in our focal article.
Although arguing for more bottom-up re-
search on personality, we also acknowledge
that for practical and/or theoretical reasons,
it is likely that some personality constructs
are more profitably explored at a facet level
than are others. For instance, a meta-analysis
has indicated that some facets of Conscien-
tiousness (e.g., achievement, dependability
and order) are found to provide incremental
validity for some combinations of job types
and performance criterion but not for others
(Dudley, Orvis, Liebecki, & Cortina, 2006).
Similarly, the facets of Extraversion (e.g.,
dominance, sociability, and energy level)
have shown differential relationships with
criteria (Hough, 1992; Hough, Ones, &
Viswesvaran, 1998). Facets of Agreeable-
ness, on the other hand, do not appear to
exhibit such differential relationships.
Hence, broad Agreeableness may be useful
for predicting the social aspects of work, but
facets of Extraversion may be more useful to
this end. It is also worth considering that
methods other than self-report for collecting
personality data may lead to different pat-
terns of empirical distinctiveness and incre-
mental validity at the facet level.
Barrett is skeptical of meta-analytic
results on personality–performance relation-
ships, for one because by contrast, practi-
tioners want to obtain the best validity
coefficients for particular measures and in
a particular occupational setting. It is true
that different personality scales do comprise
different sets of items, and therefore some
organizations may be especially interested
in the specific measures they use. But from
a theoretical standpoint, these items and
scales are indicators of constructs, and
meta-analyses that have organized personal-
ity scales at the construct and facet levels of
the Big Five have provided theoretically use-
ful and important information concerning
criterion-related validity. A measure-specific
approach can also be useful, but if followed
exclusively, it could severely hinder our field
as a science. We do not want to return to the
‘‘good old daze’’ (Hough, 1997) of the past
with as many personality constructs as we
325
have scales. Current progress has instead
been made through several major meta-
analyses of different personality scales,
applied in organizational contexts, that have
provided our field with knowledge that gen-
eralizes. There is a fair middle ground
between broad meta-analyses and scale-
specific analyses that is consistent with Bar-
rett’s concerns: developing databases and
meta-analyses of more refined personality
and performance constructs, such as facets
of the Big Five and types of counterproduc-
tive work behavior (see Hough & Ones,
2001, for a nomological-web clustering
approach).
Johnson (2008) takes this more refined
approach when he reviews and integrates
theoretical models that involve substantive
moderators and mediators that explain vari-
ance in and across personality–performance
relationships. The Johnson and Hezlett
(2008) model that he describes is a demon-
stration of the complexity of factors and var-
iables that determine performance. Clearly,
the world of work is not bivariate, and their
model confirms our original comment that
the ‘‘complexity of the nomological nets of
personality constructs is enormous.’’ They
are, thankfully, much more specific than
we were: Their model contains variables
related to motivation that, are relatively dis-
tal (e.g., work attitudes) and proximal (e.g.,
self-regulation) with respect to job perfor-
mance outcomes. Models like these are
important because they suggest that moti-
vational variables more proximal to perfor-
mance are meaningful (see also Humphreys
& Revelle, 1984; Kanfer, 1990) and also
because they make explicit the fact that not
all relevant variables in a performance
model may be available in the practice of
personnel selection. For instance, Johnson’s
model indicates that on-the-job stress will
show negative correlations with job perfor-
mance, but in a selection setting, a more dis-
tal measure of general stress tolerance may
have to suffice for understanding that partic-
ular relationship in a job applicant sample.
We concur with Stewart that we also need
good models of the determinants of team
performance, and models of individual
326
performance are not enough in many cases.
We reiterate our point that if we understand
the nomological net of narrower (e.g., Big
Five facet-level) variables, we are more
likely to combine these facets more strategi-
cally and efficiently into broader variables
that are useful for both individual- and
team-level performance models, thereby
advancing both our science and practice.
Composition models suggest that such vari-
ables may differ for individual- and team-
level performance models (Chan, 1998),
but at each level, they are composed of
well-understood facet-level personality var-
iables. We are suggesting that measures of
facet-level constructs remain intact; they
are simply configured differently at the
broader level, depending upon factors such
as the level of analysis and the dependent
variables figuring into the performance
model of interest.
Regarding measurement in personality,
multiple items (or other measures) that sam-
ple content representatively from the rele-
vant construct domain are required for item
covariances to build up geometrically and
overwhelm the idiosyncrasies of item-
specific variance. The heterogeneity of con-
tent sampled by items tends toward lowering
alpha reliability, but sampling a large
number of such items tends to raise it (Little,
Lindenberger, & Nesselroade, 1999). Under-
standing the nature of content heterogeneity
is critical when sampling items to form
a measure of a broad construct, such as
a Big Five personality construct, and to us
that often means understanding the reliabil-
ity and validity of its facets. This same argu-
ment applies to defining a facet carefully
such that content sampling is appropriate
at the facet level as well (see Comrey,
1988; Hogan, 1983).
We believe a consensus is emerging: As
we refine our conceptualization and mea-
surement of the dependent variables of criti-
cal importance to organizations, the science
and practice of industrial–organizational
psychology will benefit from conceptualizing
and measuring most personality variables at
a commensurate level not too far off from the
level of specificity of the facets of the five-
F.L. Oswald and L.M. Hough
factor model of personality (e.g., Paunonen,
1998; Paunonen & Ashton, 2001; Roberts
et al., 2005). Personality constructs mea-
sured at this level will become increasingly
important in models, measures, and meta-
analyses of individual- and team-level
job performance.
Importance of Situations
In our focal article, we asserted that (a) sit-
uations often moderate or mediate relation-
ships between personality constructs and
work performance at both the individual
and the team levels, (b) more attention to
the situation is needed in the measurement
of personality constructs, (c) more attention
to the situation is needed in our models of
performance, and (d) our science and prac-
tice would be aided by the development of
a taxonomy of situations.
Christiansen and Tett (2008) cogently
detail the need to examine the effects of
situational characteristics on the expression
of personality-relevant behavior because
research to date has found moderate and het-
erogeneous levels of validity in personality–
performance relationships. Situational
characteristics, if they are investigated,
often lack the detail that the authors
would recommend. From a psychological
point of view, it may be profitable to mea-
sure situations more directly in terms of
personality, in terms of how much choice
behavior is allowable, how much respon-
sibility is provided, and so on. Personality-
relevant performance data are often too
coarse (e.g., a cursory supervisory perfor-
mance appraisal); better data may come
from multiple-perspective feedback instru-
ments that provide different types of data
(e.g., data on teamwork/helping behavior
from peer ratings; data on customer inter-
actions from the customer and employee
ratings themselves). Also the O*NET may
provide a good start, as the authors note.
Hierarchical linear models (HLM) in-
corporate and test relationships at the
individual level, situational level, and cross-
level interactions: Personality-relevant situ-
ational characteristics—at the job, social,
Personality testing and industrial–organizational psychology: Response
and organizational levels—can be mode-
led to predict differences in personality–
performance relationships across jobs.
Strong and weak situations (of various types)
exert range-restricting and range-enhancing
effects (see Ackerman, Kanfer, & Goff, 1995;
Beaty, Cleveland, & Murphy, 2001; Klehe &
Anderson, 2007; Ployhart, Lim, & Chan,
2001), and HLM can inform how those effects
lead to differences in mean performance and
criterion-validity across jobs or departments
and over time (see Voelkle, Wittmann, &
Ackerman, 2006, for a longitudinal example).
To argue for the importance of the situa-
tion, Christiansen and Tett focus on the SDq
estimates from meta-analysis of personality–
performance relationships as evidence of
heterogeneity that mitigates overall meta-
analytic findings. We did not focus on these
estimates by intent, not by accident: Simula-
tion research has found SDq estimates to be
highly inaccurate in terms of generating both
Type I and Type II errors, leading to false
conclusions about validity generalization
or a lack thereof (see Cornwell & Ladd,
1993; Kemery, Mossholder, & Dunlap,
1989; Oswald & Johnson, 1998). This re-
search is relevant for all meta-analyses not
just those in the personality domain. A brief
story provides a second reason not to focus
on SDq: when the first author was teething
on articles in graduate school, he read the
now-famous Barrick and Mount (1991)
meta-analysis of the Big Five. Being curi-
ous, he requested and received from
Barrick a list of the personality measures
assigned to Big Five constructs, and what
was discovered supports the point of
Barrett’s: The personality measures were
quite diverse in nature, and several of the
measures were narrower than the broad Big
Five constructs to which they were assigned
(for instance, validity involving a measure
of dependability may have been assigned
to the umbrella construct of Conscienti-
ousness). Thus, although meta-analytically
averaged validities organized into Big Five
categories are eminently useful for summa-
rizing research, the aforementioned con-
struct deficiencies can lead to systematic
increases, decreases, or heterogeneity in
327
validity that cannot be appropriately
accounted for statistically by SDq or any
correction factor; they must be examined
substantively by conducting research that
examines situations and the narrower facet
level at which the personality measures
were derived. Barrett also cites differences
in job types, concurrent versus predictive
validities, and faking effects across studies
as other sources of heterogeneity underly-
ing meta-analysis; the nature of these differ-
ences and other important ones (e.g.,
leadership style; type of performance crite-
rion) also must be researched directly and
not be guided by the statistical estimate of
SDq associated with an overall meta-ana-
lytic personality–performance correlation.
In general, we strongly encourage studies
and meta-analyses of important moderators
of personality–performance relationships
no matter what the value of SDq is estimated
to be. Heggestad and Gordon (2008) cite
several recent articles that report increased
validity for personality measures that are
contextualized to focus on the workplace.
White, Young, Hunter, and Rumsey (2008),
however, come to a different conclusion
with regard to research adopting a greater
focus on context. They point out that in their
large sample military studies, items that are
less content relevant (less contextualized)
have greater criterion-related validity in
longitudinal studies and less validity in con-
current-related validity studies. Their find-
ings serve as an important caveat in the use
of personality measures in personnel selec-
tion, and they suggest that items that are less
obvious about what characteristic is being
measured retain their validity in real-life
applicant settings. A similar argument has
been made for conditional reasoning mea-
sures of personality (James, 1998).
The Stewart discussion of team tasks
involving team-member interdependency
also addresses the situation as a moderator
of the relationships between personality
variables and criteria. We would add that
a team task requiring a creative solution
versus a routine solution is another exam-
ple of a ‘‘situational’’ moderator where
personality variables will show different
328
patterns of prediction (Hough, 1992; Hough
& Dilchert, 2007).
Personality Variables and
Incremental Validity
Barrett appears to question whether person-
ality variables will increase the accuracy of
prediction of work performance when com-
bined with measures of other characteristics
such as cognitive ability. Empirical research
from Project A (see White et al.) involving
thousands of soldiers found that personality
variables often provide a modest but impor-
tant increment to validity when combined
with measures of cognitive ability, with no
reason to suggest that such findings would
not exist in the civilian population as well.
In fact, dozens of other empirical studies
have found similar results in nonmilitary set-
tings. Second, what Barrett refers to as sim-
ulation studies of the increment in validity
resulting from the combination of per-
sonality and cognitive variables are, in fact,
mathematical calculations based on meta-
analytic estimates, which in turn are not sim-
ulated but instead are based on cumulative
research evidence. We do not pretend that
these are perfect or universal estimates, as
we have previously noted, but they are infor-
mative. Similarly, all the other commenta-
tors to our focal article are clearly
persuaded by the accumulated evidence
that personality variables offer the possibility
of increasing the overall accuracy of pre-
diction of work performance—in general
and when combined with measures of cog-
nitive ability. To be fair with regard to this
issue of incremental validity, we note that
empirical comparisons of broad versus nar-
row personality constructs in regression
models need to account for the fact that
models with facets will contain a greater
number of variables and thus have greater
chance of capitalizing on chance. Therefore,
model results comparing broad versus nar-
row personality variables should be cross
validated or made comparable in other
appropriate ways (Paunonen & Ashton,
2001; Tett, Steele, & Beauregard, 2003).
F.L. Oswald and L.M. Hough
Measurement Methods and Faking
We are heartened that researchers are cur-
rently developing and investigating alterna-
tive strategies to typical self-report measures
using a Likert-type or yes/no format. The
work of White et al. has provided significant
insights and advances in our knowledge
about the measurement of personality con-
structs and ways to address problems with
self-report measures. Their commentary is
a must-read for academics and practitioners
alike.
White et al. reflect on the contributions of
Project A and related military research that
has followed. The Army’s Assessment of
Individual Motivation (AIM) led the resur-
gence of examining forced-choice personal-
ity measures in the attempt to reduce faking
in high-stakes personality testing. In a similar
vein, the U.S. Navy has recently developed
a set of unidimensional forced-choice mea-
sures called the Navy Computer Adaptive
Personality System (NCAPS; see Houston,
Borman, Farmer, & Bearden, 2006).
Although forced-choice measures induce
negative intercorrelations induced by the
dependence between item responses
(Hicks, 1970), the computerized nature of
forced-choice measures provides several
potential advantages. First, randomized item
pairings help increase test security. Second,
the adaptive nature of the measure presents
items to the test-taker based on previous
responses, thereby allowing fewer items to
be administered while maintaining high
reliability. Third, statistical modeling and
scoring tools that account for the aforemen-
tioned dependencies in these types of data
are available, though large sample sizes (at
least 450) are required for stable estimation
(Stark & Drasgow, 2002; also see Cheung,
2004; Maydeu-Olivares & Böckenholt,
2005). As White et al. note, forced-choice
measures appear to show promise by retain-
ing higher levels of validity in high-stakes
situations that tend to reduce criterion-
related validity for Likert-scale personality
measures (see also Jackson, Wrobleski, &
Ashton, 2000).
Personality testing and industrial–organizational psychology: Response
Faking on Likert-scale measures contin-
ues to receive research attention. White
et al. found that items conceptually linked
to job performance, but low in job content
(low context), result in predictive validity
coefficients in high-stakes testing situations
that are similar to concurrent validity coeffi-
cients, perhaps because the low job content
is less prone to faking. Kuncel and Borneman
(2007) used idiosyncratic item responses to
detect deliberately faked personality inven-
tories. Their method successfully classified
about one-fifth to one-third of intentionally
distorted responses to personality items with
only a 1% false-positive rate in a sample of
approximately 50% honest respondents.
Griffith and Peterson (2008) provide a
thoughtful and detailed analysis comparing
social desirability measures with the faking
behavior on personality tests that they are
intended to predict. As they note, more pro-
active approaches to personality testing
would appear to be more effective in reduc-
ing faking than the reactive approach of
attempting to detect faking after it occurred.
Regarding the latter, we are not alone in
believing that social desirability scales to
detect faking are problematic, for reasons
described in our focal article. The Paulhus
Balanced Inventory of Desirable Respond-
ing (Paulhus, 1998) is often used as a measure
self-deception and impression management
dimensions of social desirability. After re-
viewing the research since his influential
article on social desirability (Paulhus,
1984), Paulhus himself concluded (in Paul-
hus & John, 1998) that self-deception and
impression management are more usefully
considered within a broader framework of
self-perception than instantiated as mea-
sures to control for faking personality mea-
sures. Specifically, self-deception is related
to an egoistic tendency to exaggerate one’s
ability or status and is correlated with
Extraversion, Openness, and Neuroticism;
impression management is related to a mor-
alistic tendency to exaggerate one’s good-
ness as a member of society and is
correlated with Agreeableness and Con-
scientiousness. These so-called biasing
mechanisms may be adaptive, in general
329
and specifically, in predicting work
performance.
In taking a more proactive approach to
controlling for faking, three critical areas
for research come to mind: having test-takers
understand the personality test within the
larger context of person–job fit, other tests
in the test battery, and an organization’s per-
sonnel selection process; providing warn-
ings not to fake (or ‘‘positive warnings’’ to
respond honestly); and examining test for-
mats such as the forced-choice formats pre-
viously mentioned. Converse et al. (2008),
have examined the latter two factors, noting
that despite the potential advantages of these
testing interventions, it may come at the cost
of an increase in negative test-taker reactions.
Certainly, more research in all three of these
areas would be viewed as more productive
than continuing individual-differences work
on social desirability measurement or exper-
imental work in instructed faking. We add
that future research would benefit from a crit-
ical review and framework to classify the key
characteristics that distinguish high-stakes
testing situations that are likely to induce or
reduce test faking.
Legal Issues
Jones and Arnold (2008) remind us about the
possibility that, in practice, personality test-
ing might be significantly restricted—and
even prohibited—under state and federal
law. They provide important cautions and
information about how to avoid organiza-
tional exposure to risk when incorporating
personality tests into selection systems,
including the need to remain vigilant about
marketing claims.
Barrett also addresses legal issues in his
commentary, more specifically arguments
made by plaintiffs’ and defendants’ experts.
According to Barrett, plaintiff’s experts often
argue, without providing empirical evi-
dence, that a self-report personality test in
combination with a cognitive ability test will
reduce adverse impact against protected
classes. A significant amount of such evi-
dence already exists in the literature to allow
for some safe bets about the likely adverse
330
impact of a scale when used operationally,
and White et al. provide empirical evidence
with sample sizes in the thousands. Hough,
Oswald, and Ployhart (2001) summarized
mean score differences between Whites
and protected classes and men and women
at broadly and more narrowly defined con-
struct levels. They concluded:
Research clearly indicates that the set-
ting, the sample, the construct and the
level of construct specificity can all,
either individually or in combination,
moderate the magnitude of differences
between groups. Employers using tests
in employment settings need to assess
accurately the requirements of work.
When the exact nature of work is speci-
fied, the appropriate predictors may or
may not have adverse impact against
some groups. (p. 152)
Adverse impact concerns are yet another
reason for the importance of accumulat-
ing information about mean scores and
criterion-related validities at a narrower con-
struct level than the factors in the five factor
model of personality. Subgroup mean differ-
ences across facets have implications for the
facets one might choose to combine when
creating a predictor composite.
Barrett commented that the U.S. Depart-
ment of Justice (DOJ) has a history of advo-
cating for the sole use of personality tests as
an ‘‘alternative’’ to cognitive ability tests for
the selection of safety forces. DOJ’s actual
history is to argue that personality in combi-
nation with cognitive ability is a preferable
‘‘alternative’’ to the use of cognitive ability
tests alone, although the court never ruled
on the ‘‘alternatives’’ part of the City of
Garland, TX, case to which Barrett refers.
Conclusions
This exchange on the role of personality test-
ing in organizational research has stimulated
our own thinking and hopefully yours as
well. We have taken the position that,
although meta-analyses of the Big Five have
yielded useful empirical results, and the Big
F.L. Oswald and L.M. Hough
Five remains a useful organizing framework,
that facet-level personality research should
become very productive these days, given
that the job performance domain on the
dependent variable side of the equation has
become more refined and multidimensional
than ever before. Moreover, process models
that incorporate motivational mediators and
situational moderators will also benefit
greatly from empirical and theoretical work
taking a facet-level approach to personality.
Situational characteristics in such models
deserve to be measured in their own right
rather than by proxy, so that situational mod-
erating effects are modeled appropriately.
Furthermore, process models will be en-
hanced by incorporating the dimension of
time, and continued use of longitudinal
analytic methods will inform how and
when personality prediction unfolds over
the course of time (see Chan & Schmitt,
2000; Schmitt, Oswald, Friede, Imus, &
Merritt, 2008).
Our maturing development of personality
theory and measurement in organizational
research cannot be blind to the real-world
context in which measurement occurs. The
high-stakes context of personnel selection
means that test faking will continue to be
a topic worthy of pursuit, and we noted par-
ticularly productive and unproductive direc-
tions for future research. The legal context
also must not be ignored, and the pressure
to ensure the job relevance of personality
measures administered as part of a selection
battery of tests will continue to mount.
We want to close with our appreciation
for the opportunity to participate with our
commentators in the scholarly interchange
of this new journal format, and we hope to
have done no less than to inform and inspire
those involved in organizational research
and practice in personality testing, at least
by some small amount.
References
Ackerman, P. L., Kanfer, R., & Goff, M. (1995). Cognitive
and noncognitive determinants and consequences
of complex skill acquisition. Journal of Experimental
Psychology: Applied, 1, 270–304.
Personality testing and industrial–organizational psychology: Response
Barrett, G. V. (2008). Practitioner’s view of personality
testing and industrial–organizational psychology:
Practical and legal issues. Industrial and Organiza-
tional Psychology: Perspectives on Science and
Practice, 1, 299–302.
Barrick, M. R., & Mount, M. K. (1991). The big five per-
sonality dimensions and job performance: A meta-
analysis. Personnel Psychology, 44, 1–26.
Beaty, J. C., Cleveland, J. N., & Murphy, K. R. (2001). The
relation between personality and contextual perfor-
mance in ‘‘strong’’ versus ‘‘weak’’ situations. Human
Performance, 14, 125–148.
Chan, D. (1998). Functional relations among constructs
in the same content domain at different levels of
analysis: A typology of composition models. Journal
of Applied Psychology, 83, 234–246.
Chan, D., & Schmitt, N. (2000). Interindividual differ-
ences in intraindividual changes in proactivity dur-
ing organizational entry: A latent growth modeling
approach to understanding newcomer adaptation.
Journal of Applied Psychology, 85, 190–210.
Chernyshenko, O. S., Stark, S., Woo, S. E., & Conz, G.
(2008, April). Openness to experience: Its facet
structure, measurement, and validity. Paper pre-
sented at the 23rd Annual Conference of the Society
for Industrial and Organizational Psychology (SIOP),
San Francisco, CA.
Cheung, M. W. L. (2004). A direct estimation method on
analyzing ipsative data with Chan and Bentler’s
(1993) method. Structural Equation Modeling, 11,
217–243.
Christiansen, N. D., & Tett, R. P. (2008). Toward a better
understanding of the role of situations in linking per-
sonality, work behavior, and job performance.
Industrial and Organizational Psychology: Perspec-
tives on Science and Practice, 1, 312–316.
Comrey, A. L. (1988). Factor-analytic methods of scale
development in personality and clinical psychology.
Journal of Consulting and Clinical Psychology, 56,
754–761.
Converse, P. D., Oswald, F. L., Imus, A., Hedricks, C.,
Roy, R., & Butera, H. (2008). Comparing personality
test formats and warnings: Effects on criterion-
related validity and test-taker reactions. Interna-
tional Journal of Selection and Assessment, 16,
155–169.
Cornwell, J. M., & Ladd, R. T. (1993). Power and accuracy
of the Schmidt and Hunter meta-analytic procedures.
Educational and Psychological Measurement, 53,
877–895.
Dudley, N. M., Orvis, K. A., Liebecki, J. E., & Cortina, J.
M. (2006). A meta-analytic investigation of consci-
entiousness in the prediction of job performance:
Examining the intercorrelations and incremental
validity of narrow traits. Journal of Applied Psychol-
ogy, 91, 40–57.
Feist, G. J. (1998). A meta-analysis of personality in sci-
entific and artistic creativity. Personality and Social
Psychology Review, 2, 290–309.
Griffith, R. L., & Peterson, M. H. (2008). The failure of
social desirability measures to capture applicant faking
behavior. Industrial and Organizational Psychology:
Perspectives on Science and Practice, 1, 308–311.
Heggestad, E. D., & Gordon, H. L. (2008). An argument
for context-specific personality assessments. Indus-
trial and Organizational Psychology: Perspectives
on Science and Practice, 1, 320–322.
331
Hicks, L. E. (1970). Some properties of ipsative, norma-
tive, and forced-choice normative measures. Psy-
chological Bulletin, 74, 167–184.
Hogan, J., & Roberts, B. W. (1996). Issues and non-issues
in the fidelity-bandwidth trade-off. Journal of Orga-
nizational Behavior, 17, 627–637.
Hogan, R. T. (1983). A socioanalytic theory of person-
ality. In M. Page (Ed.), Nebraska symposium on
motivation (pp. 55–89). Lincoln: University of
Nebraska Press.
Hough, L. M. (1992). The ‘‘Big Five’’ personality
variables—construct confusion: Description versus
prediction. Human Performance, 5, 139–155.
Hough, L. M. (1997). The millennium for personality
psychology: New horizons or good old daze.
Applied Psychology: An International Review, 47,
233–261.
Hough, L. M., & Dilchert, S. (2007, October). Inventors,
innovators, and their leaders: Selecting for consci-
entiousness will keep you ‘‘inside the box.’’ Invited
presentation, SIOP Leading Edge Consortium:
Enabling Innovation in Organizations. Kansas City,
MO.
Hough, L. M., & Furnham, A. (2003). Importance and
use of personality variables in work settings. In I. B.
Weiner (Ed.-in-Chief) & W. Borman, D. Ilgen, &
R. Klimoski (Vol. Eds.), Handbook of Psychology:
Industrial and Organizational Psychology (Vol. 12,
pp. 131–169). New York: Wiley.
Hough, L. M., & Ones, D. S. (2001). The structure, mea-
surement, validity, and use of personality variables
in industrial, work, and organizational psychology.
In N. R. Anderson, D. S. Ones, H. K. Sinangil, & C.
Viswesvaran (Eds.), Handbook of work psychology
(pp. 233–377). New York: Sage.
Hough, L. M., Ones, D. S., & Viswesvaran, C. (1998,
April). Personality correlates of managerial per-
formance constructs. In R. C. Page (Chair), Person-
ality determinants of managerial potential
performance, progression and ascendancy. Sym-
posium conducted at 13th Annual Conference of
the Society of Industrial & Organizational Psychol-
ogy, Dallas, TX.
Hough, L. M., & Oswald, F. L. (2008). Personality testing
and industrial–organizational psychology: Reflec-
tions, progress, and prospects. Industrial and Orga-
nizational Psychology: Perspectives on Science and
Practice, 1, 272–290.
Hough, L. M., Oswald, F. L., & Ployhart, R. E. (2001).
Determinants, detection, and amelioration of
adverse impact in personnel selection procedures:
Issues, evidence, and lessons learned. International
Journal of Selection and Assessment, 9, 152–194.
Houston, J. A., Borman, W. C., Farmer, W. F., & Bearden,
R. M. (2006). Development of the Navy Computer
Adaptive Personality Scales (NCAPS) Technical
Report (NPRST-TR-06-2). Millington, TN: Navy Per-
sonnel Studies, Research and Technology.
Humphreys, M. S., & Revelle, W. (1984). Personality,
motivation, and performance: A theory of the rela-
tionship between individual differences and infor-
mation processing. Psychological Review, 91,
153–184.
Jackson, D. N., Wrobleski, V. R., & Ashton, M. C. (2000).
The impact of faking on employment tests: Does
forced-choice offer a solution? Human Perfor-
mance, 13, 371–388.
332
James, L. R. (1998). Measurement of personality via con-
ditional reasoning. Organizational Research Meth-
ods, 1, 131–163.
Jenkins, M., & Griffith, R. (2004). Using personality con-
structs to predict performance: Narrow or broad
bandwidth. Journal of Business and Psychology,
19, 255–269.
Johnson, J. W. (2008). Process models of personality and
work behavior. Industrial and Organizational Psy-
chology: Perspectives on Science and Practice, 1,
303–307.
Johnson, J. W., & Hezlett, S. A. (2008). Modeling
the influence of personality on individuals at work:
A review and research agenda. In S. Cartwright &
C. L. Cooper (Eds.), The Oxford handbook of person-
nel psychology. Oxford, UK: Oxford University
Press.
Jones, J. W., & Arnold, D. W. (2008). Protecting the legal
and appropriate use of personality testing: A practi-
tioner perspective. Industrial and Organizational
Psychology: Perspectives on Science and Practice,
1, 296–298.
Kanfer, R.. (1990). Motivation theory and Industrial/
Organizational psychology. In M. D. Dunnette & L.
M. Hough (Eds.), Handbook of industrial and orga-
nizational psychology. Volume 1. Theory in indus-
trial and organizational psychology (pp.75–170).
Palo Alto, CA: Consulting Psychologists Press.
Kemery, E. R., Mossholder, K. W., & Dunlap, W. P.
(1989). Meta-analysis and moderator variables: A
cautionary note on transportability. Journal of
Applied Psychology, 74, 168–170.
Klehe, U.-C., & Anderson, N. (2007). Working hard and
working smart: Motivation and ability during typical
and maximum performance. Journal of Applied
Psychology, 92, 978–992.
Kuncel, N. R., & Borneman, M. J. (2007). Toward a new
method of detecting deliberately faked personality
tests: The use of idiosyncratic item responses. Inter-
national Journal of Selection and Assessment, 15,
220–231.
LePine, J. A. (2003). Team adaptation and post change
performance: Effects of team composition in terms of
members’ cognitive ability and personality. Journal
of Applied Psychology, 88, 27–39.
LePine, J. A., Colquitt, J. A., & Erez, A. (2000). Adapt-
ability to changing task contexts: Effects of general
cognitive ability, conscientiousness and openness to
experience. Personnel Psychology, 53, 563–593.
Little, T. D., Lindenberger, U., & Nesselroade, J. R.
(1999). On selecting indicators for multivariate mea-
surement and modeling with latent variables: When
‘‘good’’ indicators are bad and ‘‘bad’’ indicators are
good. Psychological Methods, 4, 192–211.
Maydeu-Olivares, A., & Böckenholt, U. (2005). Struc-
tural equation modeling of paired comparison and
ranking data. Psychological Methods, 10, 285–304.
Oswald, F. L., & Johnson, J. W. (1998). On the robustness,
bias, and stability of statistics from meta-analysis of
correlation coefficients: Some initial Monte Carlo fin-
dings. Journal of Applied Psychology, 83, 164–178.
Paulhus, D. L. (1984). Two-component models of
socially desirable responding. Journal of Personality
and Social Psychology, 46, 598–609.
View publication stats
F.L. Oswald and L.M. Hough
Paulhus, D. L. (1998). Manual for the Balanced Inven-
tory of Desirable Responding (BIDR-7). Toronto,
ON: Multi-Health Systems.
Paulhus, D. L., & John, O. P. (1998). Egoistic and moral-
istic biases in self-perception: The interplay of
self-deceptive styles with basic traits and motives.
Journal of Personality, 66, 1025–1060.
Paunonen, S. V. (1998). Hierarchical organization of
personality and prediction of behavior. Journal of
Personality and Social Psychology, 74, 538–556.
Paunonen, S. V., & Ashton, M. C. (2001). Big Five
factors and facets and the prediction of behavior.
Journal of Personality and Social Psychology, 81,
524–539.
Paunonen, S. V., Rothstein, M. G., & Jackson, D. N.
(1999). Narrow reasoning about the use of broad
personality measures for personnel selection. Jour-
nal of Organizational Behavior, 20, 389–405.
Ployhart, R. E., Lim, B.-C., & Chan, K.-Y. (2001).
Exploring relations between typical and maxi-
mum performance ratings and the five factor
model of personality. Personnel Psychology, 54,
809–843.
Roberts, B. W., Bogg, T., Walton, K., Chernyshenko, O., &
Stark, S. (2004). A lexical approach to identifying the
lower-order structure of conscientiousness. Journal of
Research in Personality, 38, 164–178.
Roberts, B. W., Chernyshenko, O., Stark, S., & Goldberg,
L. (2005). The structure of conscientiousness: An
empirical investigation based on seven major per-
sonality questionnaires. Personnel Psychology, 58,
103–139.
Schmitt, N., Oswald, F. L., Friede, A., Imus, A., & Merritt,
S. (2008). Perceived fit with an academic environ-
ment: Attitudinal and behavioral outcomes. Journal
of Vocational Behavior, 72, 317–355.
Schneider, R. J., Hough, L. M., & Dunnette, M. D.
(1999). Broadsided by broad traits: How to sink sci-
ence in five dimensions or less. Journal of Organiza-
tional Behavior, 17, 639–655.
Stark, S., & Drasgow, F. (2002). An EM approach to
parameter estimation for the Zinnes and Griggs
paired comparison IRT model. Applied Psychologi-
cal Measurement, 26, 207–227.
Stewart, G. L. (2008). Let us not become too narrow.
Industrial and Organizational Psychology: Perspec-
tives on Science and Practice, 1, 317–319.
Tett, R. P., Steele, J. R., & Beauregard, R. S. (2003). Broad
and narrow measures on both sides of the personality-
job performance relationship. Journal of Organiza-
tional Behavior, 24, 335–356.
van Iddekinge, C. H., Taylor, M. A., & Eidson, Jr., C. E.
(2005). Broad versus narrow facets of integrity: Pre-
dictive validity and subgroup differences. Human
Performance, 18, 151–177.
Voelkle, M. C., Wittmann, W. W., & Ackerman, P. L.
(2006). Abilities and skill acquisition: A latent
growth curve approach. Learning and Individual
Differences, 16, 303–319.
White, L. A., Young, M. C., Hunter, A. E., & Rumsey, M.
G. (2008). Lessons learned in transitioning person-
ality measures from research to operational settings.
Industrial and Organizational Psychology: Perspec-
tives on Science and Practice, 1, 291–295.