.

Module 4
 J:-ITEfl\'IEWS 'l;J,7

If pre-intel"\iew information is not equally available on all candi<lates, reliabil-

ity suffers. If interviewers review applicants' materials, we cannot be sure that
the estimated interview validity is actually due to the interview. Rather, it inay
be because the reviewed materials are valid. As I mentioned above, if interview-
ers are effective in assessing attributes, such as intelligence, it may not be be-
cause they are particularly good at this, but because they have looked at other,
more valid assessments of the attribute (e.g., intelligence test scores). Inter-
viewer use of ancillary material can disrupt interview structure in another way.
In most unstructured interviews, interviewers use ancillary material to frame
. the questions they will ask. This practice generally results in different questions I
i
for different candidates, a loss of standardization and structure, and damage to 1:
I
the reliability and validity of the interview. Thus, it is not a good idea for an in-
terviewer to review ancillary materials preparatory to an interview.

Preparing _Interview Questions

Job-Relevant Questions A key difference between structured and Un':"
structured interviews is the use of a job analysis to develop questions. A job
analysis provides the basis for writing questions that are·job relevant and that
can be asked equally well of all can~dates for a job. Any of the job analysis
methods can be used in developing questions. The most frequently used is the
critical incide1,1ts technique. 2 As you will see in the discussion of performance
appraisal in Chapter 14, a critical incidents job analysis is particularly helpful in
identifying important behavioral di~ensions of a job and describing different .
levels of employee performance quality on these dimensions. The descriptions
of the dimensions ~ily can be turned into i~terview questions, and _the levels
of performance can be 1!5ed as rating.scale anchors. This makes the rating of
applicants' responses to questions an easier task for interviewers to accomplish.
Research on interviews has long stressed the importance of job analysis in .
improving interview validity. Certainly, job analrsis can be expected to contrib-
ute to validity by increasing the job :relatedness of questions. With better ques-
tions, interviewers can obtain relatively more job-relevant information and
relatively less extraneous information from candidates. Studies have d~cu- .
mented that using a job analysis can produc~ a structured interview with high
validity. For example, from their meta-analysis, McDaniel et al. (1994) re-
ported average validity coefficients_of .50 for highly structured, situational in-
terviews that had been designed using a job analysis. By comparison, validity
coefficients for interviews that were not based on job analysis averaged .29.
A lingering question, for which the ~nswer is not entirely clear, brings up
· the possibility that job analysis might prime the interview to perform like a
test. That is, given that job analysis places stronger emphasis on cognitively

~For a quick re,iew of critical incidents job analvsis. see Chapters 3 and 14.
 ...
c-
298 CHAPTER 11 c-
related KSAs than on other attributes, perhaps it produces an oral examination
of cognitive ability. If so, this might explain why structured interviews, based EXAMPLE'
oil a job analysis, are so much more valid for predicting job performance. Cam-
pion et al. (1997) were unable to clearly determine whether this might be hap-
c-
pening. Some studies they reviewed showed little relationship between struc-
tured interviews and ability tests (e.g., Pulakos & ~chmitt, 1995), whereas
Type of Question

Past-oriented work
e-
other ~tudies showed that they were highly correlated (e.g., Campion, Cam-
~-
experience
pion, & Hudson, 1994). So, again; we are face-to-face with the question of
whether interviews-even_structured interviews-are valid for assessing any-
thing beyond what the less expensive tests do. Some stu"dies have not been able
to show that they are (e.g., Campion.et al., 1988). Others, hpwever, have re-
e-
ported incremental validity of structured interviews beyond cognitive ability
tests (Campion, Campion, et al., 1994; Pulakos & Schmitt, 1995). Probably, we .
e-
will ~ee more study of this question in. the future.
Job knowledge
C
Experience-Based and Situational Questions Much of what deter-
mines whether an interview is considered structured or unstructured is the type ·c -
of questions.used. In the traditional job interview, the questions often are gen-
i
eral in nature and have little to do with the job. An interviewer may ask an ap- '·
i C=
plicant to identify his or her strengths and weaknesses, or simply to "tell me . iI
something about yourself." These questions give.the candidate license to deter-
.
p
.--'l~
.c; -:
mine the interview content. In contrast, questions for a structured interview fo- ., Situational

cus on candidates' preparation for the job. In their review, Campion et al.
(1997) found four .types of structured interview questions: background, past .
1•J
'I
~-
behavior, job knowledge, and situational questions. These questions contribute 'i
varying levels of interview structure. ·or the four types, background questions 1. --:'~-=-
. . .
contribute the lowest level. They focus on a candidate's p~t work experiences
·~
;

in particular job-related areas. For example; if a job involves close interaction

!i
'.I
with coworkers, a candidate might be asked; 'What experience have you had in 1
working·on team projects?" Structure is low in such a question because it does '!
-~
not address specific teamwork issues and probablywill yield a variety of answers j
from candidates that will be difficult to compare. Also, such questions p robably i
]
will use a general rating scale, which itself d~s not contribute much structqre l
to an interview. · s effective in gettir
. rl"".._
The other three types of questions produce more highly structured inter-
type of question, ~ '
views. Each type calls for a specific sample of job-related behavior from candi-
"What actions dr
dates. A question may focus on past work activity, present job-related hi.owl~
·tea.Ji'. \nembers ·a
ation·a1 interview,
edge, or hypothetical work situati,ons . (See Table 11.3 for some example
questions.) First, past-oriented questions ask candidates to describe what they situations._The it(',
did in previous work or other experience that shows how they actually handled didate what he t
a situation similar to what they might face on the job in question. For example, Campion, 1980),~
a candidate might be asked, 'Think about a time when you had to make sure
are t _h e leader of
that a group of workers met strict deadlines. What ·did you do that was most is not d9ing his f;(\
The work. How v
-~
.~ -,
1
~xamination
INTER\llEWS 299

~ ~ s , .based
~ 1ance. Cam- EXAMPLES OF EXPERIENCE-BASED AND SITIIATIONAL 0UESTIONS
.. be hap-
"'tween :Struc- Type of Question Examples
~hereas
Past-oriented work
pion, Cam- experience
Think about a time when you had to evaluate an employee who was
not performing well on the job. How did you handle that situation?
...~ ~stion of
Think about a time in your previous work experience when equip-
~st!!ingany-
1111~ \een able
mel]t !hat you maintained apparently Vl'.as being _misused, causing it
to require extra maintenance and repair. What d1d you do to deal
with this?
- v~~ave re-
_,.~ '(e ability Can you give an example from your previous experierce in which
you had to make sure that a group of workers met strict production
111111 P1Wably, we . deadlines? What did you do that was most effective in getting the
group to do its work on time?

Job knowledge What kinds of documentation do you think would be needed if a

- ~ vhat deter- company wanted to discharge a poorly performing employee?
. .~ e type ·What kinds of Information should an office equipment installer
~en are geu- provide for users of the new machinery?
What actions do you think would be best for a team leader to take

-~•pdeter-
~ ::: :fe when the team members are engaging in conflict among them-
selves?

Situational
-~t~ewfo- Suppose one of the employees ·you supervise and evaluate is not
performing as well as the company expects. His next performance
~ net al. review is due in a week and you have scheduled a meeting with him
IP'c.11'1.d, past to talk about this. How would you handle the meeting?
~ ftribute Imagine you work on a Job that requires you to set up and maintain

f [ld-..estions ·
-~eriences
· ~ raction
machines for office workers to use. In the course of doing your job,
· suppose· you find that some of the users are particularly hard on the
machinery, which results in extra repair and maintenance.. What '
would you do in suet:, a sitµation? ·

feyouhadin Imagine that you are the leader of a WQr~ team and you find out that
one of the team members is not doing his share of the work but,

r:.~s~swers
~ it does instead, Is relying on the others to pick up the slack. How would you
handle this situation?
l
.<
~~bably
r-.
. ":· uctqre effective in getting the group to do its work on time?" Second, with a related
i
ttuid inter-
.
type of question, a candidate's joo knowledge can be sampled. For example, i
rfrom candi-
1~ owl-
"Wh11t actions do you think would be helpful for !l team leader t9 take when
·teaJ{members are engaging in conflict among themselves?" Finally, _in a situ-
he example ationa:! interview, the questions ask for reactions to hypothetical or future work
situations, The interviewer describes a hypothetical incident and asks the can-
~ t they
. didate what he or she would do if this happened (Latham, Saari, Pursell, &
Jlyhandled
• · Campion, 1980). For exaJ!lple, a candidate might be asked, "Imagine that you
~ple
make sur~ are the leader of a work team and you discover that .one of the team members
is n~t d<:>ing his full share of the work, but instead is rel)'ing on the others to do
most the work. How would you handle this situatioQ?" As you can see, these three
 ........

30.IJJ CHAl'TEH II

~es of questions call for ~~wers that ure highly specific to the job which ap-
plicants are seeldng. The questions are meant to elicit samples of behavior and
average • · · s,·.,
appraisal have ·
scales are anc\ v
;J'' h'
as a re~ult they lend a great deal of structure to the interview. ' haviorally anchr ~1;.
It 1s reasonable to expect that questions which contribute higher versus mensions, and : '[
lower
d' levels
'd of structure
f also will increase interview validity. ·Th ere1s
· some -·m. supervisors mak gt
irect evi ence o this. Biodata studies have shown that questions calling The same I'' ~ !
for samples of past work behavior can raise validity levels of biodata instru- chored rating s, _ · ·0
ments (cf. Mumford & Stokes,, 1992). However, because interview research supervisors o;:- " .
often ?oes not distinguish effects due to question 1YPe from effects due to job behavior that v· e..! v
analysis o_r other sources of structure, it is difficult to decipher the impact of shows four qu,- -
the questions used. A few studies.have attempted to determine whether work rating scale v.ri., ~ :
experience-based or situational questions are better. However when consid-
paor answer rr, -~ "
ered together, the studies do not present a clear distinction. In s~me, questions sible ·a b9ut tht
about _past work behavior are associated with higher validity (e.g., Campion, _interview, an e.-
C~~1on, ~t al., 1994; Pulakos & Schmitt, 1995). In other studies, situational whether it is a' g,
questions yield the more ·valid interviews (e.g., McDaniel et al., 1994). Actu- Behavioral t:";._~~
~y, we ~>Uld expect ~t situational and past experience-based questions con- of an interview
trt~ute diffe~ntly to mterviews because they are inherently different. For one ti.on againsfwhi• ·
thing, questions about past experience are more verifiable. Also, situational anchors are ex:c.-,
questions may be tapping into different constructs, such as motivation. Situ- task is simply to '
ational questions appear to require a certain amount of self-insight and aware:.
ness,of intentions. In practice, it is probably a good idea to include a variety of
a candidate's ai, - I
chored rating sc~
job-related questions because each type contrtbutes validity, albeit for differ-
ent reasons. l'he best interview probably includes some job.knowledge ques- -
on job-relevant - [i
interviewer's at\-.f' H
tions, some past-be4avior questions, and some situational questions.
evant informati· R
Ancho~ Rating.Scales An essential part of astructured interview ques- .Q I<
tion is a rating scale that uses descriptions of work behavior to anchor scale Structuring t. .,,
points. Anchored rating scales are designe4 using the same job analysis infor-
Certain proces~
mation, typically from a critical incidents study. Because the rating scale points whe~er it inc:< - I
are anchored with performance descriptions that are specific to each question,
anchored rating scales allow interviewers to more easily score candidates' an-
during an inte -~ '.I
students vary
swers. By quantifying.interviewers' e_valuations, use of the rating scales also
· Some simply
makes it easier to determine interview validity. study in the sr
' The development of anchored rating scales for interview purposes can 1
~d then revi, ('._
benefit from the rese~ch on performance appraisal. As I discuss _in Chapter This. can.imfi
14; rating scales used to evaluate dimensions of employee performance on the · ams and ~o nr A\.
job vary as to how scale paints are anchored.3 Some provide minimal informa- Similarly, fute,
tion. For example, performance rating scale points sometimes are anchored terviewers, lit _
simply by placing numbers at the ends and at the midpoint of the scale, such as and exte~iv~ .
"1 .... 3 ... 5," or by adding evaluative terms_, such as "unsatisfactory. _.. initial reactior.
jdb interview~~
3 See Figures 14.3, 14.4, and 14.5 in Chapter 14 for_examples of rating scale anchors.
 "-~
\.)
\) INTERVIEWS 301

::;
1
~ ch ap- average . . . superior." However, more effective scales used for performance
"11V>r, and
appraisal have been ~eveloped using critical incidents job analysis. These
scales are anchored with detailed behavioral descriptions of performance. Be-
~ versus haviorally anchored rating scales are meant explicitly to assess jo~-relevant di-
some in- mensions, and they are used in performance appraisal to control mistakes that
10 ~alling supervisors make in rating employee performance.
datPtnstru- The same procedure can be followed in 'developing questions and an-
'\Search chored rating scales for interview purposes. In a critical incidents job analysis,
s ci~to job supervisors or other subject-matter experts are consulted regarding employee
~ :-Pact of behavior that varies in its effectiveness on the job. For example, Table 11.4
1etMwork shows four questions using performance descriptions to anchor a.l~ point
l).~ ~nsid- rating scale with a goo~ answer at scale point 5, a marginal answer at 3, and a
e,~ stions poor ~er at 1. Interviewers need to be given as much infonnatio:Q as pos-
""' · .
.,~amp1on, sible lll>9Ut the meaning of the scale points. Certainly, for a highly structured
l si tional · .interview, an example answer at each scale point and some indication as to
whether it is a good or apoor answer will be helpful to interviewers. · .
I
~). Actu-
~ \ scon- Behaviorally anchored-rating scales can improve the accuracy and validity
1
-;_l'orone of an interview because the performance anchors provide objective informa-
r:..._ i tional tion against which to compare candidates' answers . .This is particularly so if the
l_ctJ'- Situ- anchors are examples of what a candidate might say. Then, the interviewer's

~ :;:£
l~ fiffer-
task is simply to determine the anchot point description that is most similar fo
a candidate's answer and assign the score of that description. Effectively an-
cho~ rating scales increase the likelihood that interview ratings will be based
1ed.~ques- on job-relevant information from candidates. They also can help to focus ·an ·
interviewers attention· on job requirements .and prevent the distraction of irrel- ·
evant information. ·
rv(ewques-
scale Structuring the Interview Process
tlysis infor-
--..:_""J)Oints Certain processes will be valuable in conducting any interview, regardless of
h qYestion, whether it incorporates other features of a structured interview. Notetaking
~,1,~s· an~ during an interview is one such process. As you undoubtedly 'have noticed,
~ salso students vaiy as to whether they are personally inclined to take .notes in class.
· Some simply do riot. Others take more or less detailed notes and use them•to ·
study in the ·same way that they study their textbooks. Taking notes in class
rp~s can
n Chapter _and then reviewing the~ helps students remember the material covered.
=-:;,n the This can. improve their ability to use the material to answer questions on ex-
1 informa-
.ams and to apply the knowledge they gain for other personal and work uses.
Similarly, interviewers vary in terms of their propensity to tiµce note~. Some in-
~ ored
terviewers like some studeats, simply do not take notes. Others take detailed
1
e, such as
and exte~ive notes, summarize each of a candidate's answers, and write down
initial reactions or evaluations of the candidate's performance. Notetaking in .
job interviews contributes to an interviewer's 11.hility to recall a ·candidate's
 302 CHAl'TEH 11
:.~i·
during or im n-
the intervic \.",• , . , ~ /
ANCHORED RATING SCALES FOR SmucruRED INTERVIEW OuEsTioNs
assessments. '-l
Question Rating Score and Anchors
Controllin" ~ -
1. After repairing a piece of 5 Good Dust end dirt can result in wear and tear on chored ratini; : ~,

J~ -~
machinery, why would it be
r:noving parts. Also, to Inspect for wear and fail to use th<-·
necessary to clean all the . •
damage, parts need to be clean. 91
parts before putting it back
together? 3 Marginal Parts will go back together easier, and ~he the question:
machine will run more smoothly. tively un~tmc' I~\)
Poor · g remains tl:· b~ d
I [?_
So it gets a cleaning, I guess.
2. Can you provide an 5
:s have ide~~j
Good
. example from your previous
Reviewed and analyzed financial records
of a business my employer was structure (C,•; tC'
work experience showing candidate in .- - pial re
considering for purchase, reported on the
that you can analyze
value of the business, and wrote a . IS
1
complex technical data?
. proposal that served as the basis of
this reqwrem -
my employer's offer. obtained frmi
3 Marginal Analyzed bills being considered in . lowered if in·
Congress as to their potential impact on the .. asked. ff owp·; '· , in
operation of the agency where I worked,
. and reported my assessments to my tain standard , o
supervisor. for each dime. .I
,,
1 Poor Collected operating budget estimaies from different can<l
department supervisors, and relaye? them
to the general manager. • of questionin~· "er,
temate ques1 ' 3la -'
. 3. On some jobs, it is 5 Good Working at heights doesn't both me. I have interview is I:
necessary to climb a ladder done similar work before [gives example].
to -the height of a four- or five-
$!Ory building and do the 3 Marginal I don't think it would bother' me to work at A secon< · (t ~~~·-:...- .
work. How would you feel .heights. I know that this ls part of the
job. questions; Al _
·about doing that? tive behavior · ~~-:,
Poor ·Heights make me nervous. I would do it
only if necessary. · · '
actually corn
tively to heh
fl'C ,"°·
4. If you had an idea on how 5 Good I would explain the idea in en open
to improve the procedures ployers allov- ~~,~ ~ --~~

(Rc,::1 &1
meeting and try to show the benefits.
your work group was using
3 Marginal I would ask them why they are against the
.--., standard foll · n
but you knew some of the
group would not want to change, and I would try to persuade them to add?" 1
39
to give it a try.
make the change, what are meant to · ~ e tt
would y6u do? 1 Poor - I would tell the supervisor.
the extent tb •
Adapted from "Structured lnl8Niewlng: Raising Ille Psycharretrk: Pl0l)8fliea of lhe Employment Interview." by M. A. Campion,
interview's "- -
E. 0. Pursell, and B. K. Brown, 1988, Personnel Psychology. 41, pp. 30-31. Copyright 1988 by Per1011nel Psychology, Inc. .
Bowling Green, Ohio (questia\s 1 and 3) Adapled by permission; "Struclured lntervl~lng: A Note on Incremental Validity '
views could i M!P, .

and Altarnalive Question Types." by M .. A. C ~. J. E. Campion, and J. P. Hudson, Jr., 1994, Journal ol Applied However all __ :holey ·
' ! Va11
Ps)'Chology, '19, table 1, p. 999. Copyright 1994 by Amelican PaychOloglcal Assoc:iatlon, Washlhgton, OC (question~)
Adapted by permission; and The Behavioral Consistency ~fhod of Unassembled Examining (Technical memorandum 79-
:Mainly, an< -
·21 ), by F. L. Sctmidl,J. R. Caplan, S. E. Bems, R. Oecw, L.Oum, and L. Antone, 1979, pp. 47-50, Washington, DC : U.S. . ·1eges tO' .
pnVl -odu,
Office ot Personnel Mariagement (question 2).
a n swers·.. - ~
answers to questions. It improves his or her ability to evaluate the person's in-
terview performance. In addition, taking and using notes helps interviewbrs
A third,,~
until' after tl )
~
,r~r ·
, .

---------------------.;.._~ :
make decisions about the relative suitability of applicants for hire. Notetaking are in~erest::~ r . ' i '
allowing a JO •otP

~ --
 INTETI\·lf.WS 303

during or immediately after a job interview contributes strongly to structuring

the interview process, and it improves the iuterrater reliability of candidate
assessments.

-
- ;tear on
t~"'1tarand
Controlling Questioning Although highly strnc:tured questions and an-
chored rating scales may have hee11 prepart-d for an interview, if interviewers .
""\
fail to use them, or if they use them imprope11:,·, the structme contributed by
~
Iv. dthe
.
the questions and rating scale anchors is compromised. What results is a rela-
tively unstructured inteJView with reduced Vltlidity. Thus, candidate question-
. ing remains the single most important aspect of a structured interview. Review-
'records ers have identified three aspects of interview question control that contribute to
structure (Campion et al., 1997). First,' each question must be asked of each
1

1s '
Prtc_ n the candidate in exactly the same order and in exactly the same manner. Meeting

r~
. Ii
this requirement increases the likelihood that comparable information will be
obtained from each candidate. Interview strµcture and standardization will be ·
lowered if interviewers change the questions or the order in which they are
·· on the .·
l ~'\(ed, asked. However, a certain amount of flexibility might be allowed and still main-
tain standardization. For example, if equivalent questions can be constructed ·
for each dimension, then interviewers may select·slightly different questions for
--..:nates from different candidates: Remember, however, that maintaining a standardized line
~'' -''{them
of questioning is an essential aspect of a structured interview. Therefore, the· al-
I •· ternate questions must truly. be equivalent. To the extent that they are not, the
1'':!i.... i have . interview is less highly structured (Conway, Jako, & Goodman, 1995) . .
; e;Wple] .
A second way to 'control questioning is to limit prompting and follow-up . 1
o~
•.., work at
ob. questions; Although ·some interviewers might believe that this type of interac- l
>uld~it tive behavior with an interviewee is the '.'art" of the interview, these behaviors !
actually constitute sources of erro~. Still, because interviewers may react nega- tI
tively to being told they cannot ask probe or follow-up questions, some em- t
pen
. players allow interviewers to use planned prompts or probe questions, and a l
""!fits.
standard follow-up for all candidates, such as "Is there anything else you want
~ the
uade them to add?" (Robertson, Gratton, & Rout, 1990). Follow-up and probe questions
are meant to clarify what a candidate has said and to gain full information. To
the extent that such questions accomplish this, they might contribute to an
interview's validity. Huffcutt and Arthur (1994) found that structured inter-:
- -~ mpiof'!,
ooiJ!r1r-c. views could include a certain amount of follow-up without damage to validity.
ntal Validity
However, an employer needs to limit the use of probe and follow-up questions.
SU\. ,. . , Mainly, an employer must try to· guard against extending these interviewer
)l"andum 79-
n. OC: U.S. privileg~s to such an extent that candid~tes are coached into giving the right
answers ..
A third way to control questioning is to disallow questions from candidates
rs~'s in- until after the structured interview has been completed. Many job applicants
rv V/ers are interest~d in getting information about the job and organization. However,
~ng allowing a. job candidate to use time in the structured interview to ask his or
 304 CHAPTEll ll
The use i .
commonly use•
her own questions disrupts the c011cluct of the interview, degrades its standard.:
ization, and effectively turns the meeting into an unstructured or recruitment tions. Multipl t
interview. This can mean that the interviewer obtains little or no job-relevant wise unstroct,;,
.!
information from the candidate. Further, somP- job applicants are awa!'e that sistent sample 1

"taking over" the interview with their own questions can both distract t1'!e i~-
terviewer from gathering information and distort the interview outcome in the ~~-·-'
candidates favor.
Evaluatiom;, ·
Multiple Interviewers Job interviews sometimes involve two or more in~ There are t:\\'(..
terviewers. There are two ways to arrange this. In serial interviewing, individual views. The fir
interviewers meet separately with each applicant. The serial interview can be questions or '
conducted the same as an ordinary, single int~rview with each interviewer aslc- j:J This sometim,:~
ing the same questions. Or the separate in_terviews might be specifically tive manner I ··
planned so as to divide the questions among interviewers. In panel interview- questions or (
ing, a group of two to five interviewers mee~ with each candidate. Procedures The second is;
for how a panel will operate also vaiy. One me~ber ·might be responsible for viewers, sud,
asking all questions, or the responsibility for asking questions ,might be shared sus. It also en,
among members. If probes are allowed, one panel member might ask the main
questions, with c;lifferent members asking the follow-up questions. At the end of
averaging t1 te
dures should a~
questioning, all members.independently rate a candidate's responses . .' .
when ratings a .--- I"" r
Using·multiple int.erviewers can increase the accuracy of candidate assess-
ments; One reason .is that the personal biases of a single inten;iewer carry l~ss dates in trying : : } ' ;~
of
weight when th~ are consolidated with the evaluations multiple interview-
-~ erallR_ati;1W
. · ~
n-......
_,
. ers. Another possibility is that a broader range of relevant information is g~th-
eted by maltiple inte:rviewers than by a· single interviewer. For exampl~. when interview that 7kl
serial ilftenriewers ask different planned questions, vali~ty might be increased interviewers n •'-'_,
.]"- ;,a
heoouse a broader range of information is obtained (Campion et al., 1997). tip le aspects o;

:i
-~_(
:there is evidence that reliability is higher when multiple interviewers examine ~g because e::~--
candidates. In a meta-analysis of interratei reliabilities, Conway and his associ- havior that ha,
ates (1995) found that the-overall average correlation between raters was .70. ii:iterview~r tr'.
When this value·was considered according to the type of interview, the re- However. soi . ' , ·~ -
searchers found that the average reliability coefficient for panel intervj.ews was date's perfoni· :~ - C
· ,. 77, whereas the average for interviews conducted separately was .53. There subjective pr, · - :......_
could be two related reasons for the difference in these values. Panel inter:- , · likely will iu, , C
viewers hear the same candidate responses, and it could .be expected tha~ their different q, ,.~-
......
ratings of the responses would he similar. However, ·s~rial interviewers are <;>f the candid , -
likely to hear somewhat different information, either because a candidate gives mote genera: ~ -
different answers to the same question or because the interviewers ask differ- . ·
I viewers are rt, ,
ent questions. Conway et al. (1995) found that standardizing ~e questi,ons af-
fected the .reliability of both types of interviews, but it had a stronger effect on
serial interviews. This sugges~ that the reliability of serial interviews is lower ·
l
i
tiple question ,{'
clarity on how
the rating the:l.'
~r:i.
becaus~ different questions are asked and samples of applicant infonnation are
not comparable.
rating-may c
cdmhined a ct~ q-
~ ;\-

iegrades lits standard.
tured or recruitment
:1e or no j iob-rele~t
icants are, aware that
both distract the in-
N rview outcome fu th~
! Evaluations in the Structured lnteroiew
I~ olve two or more in-
erviewmg.; individual
I~ rial intertiew can be
,ach inteMewer aslc- I views. The first is the method with which to combine ratings o·n individual
N / This sometimes is done subjectively. However, it can be done in a more objec-
Il
ight be specifically
In panel interview-
N ndidate. Procedures
t be responsib1e for
>ns _might be shared
N
· might ask the main
di.. stions. At die end of
-v responses. . .j,
of candidate assess-
!~
' terviewer cany less
multiple interview-
1
.j
nformatio:n is girth-
. For example, when
\ might be increased
Y 1pion et al .• 1997).
:erviewers eiamine
way and his associ-
- een raters was .70.
interview,. the re-
mel interviews was
:Jly was .53. There .
tlues. Panel inter~
~xpected that their
- .., l interviewers are
_ a candidate gives .
"1 .riewers ask differ-
' the questions af- ·
stronger effect on
1terviews is lower .
- , 1t information ar~
(
I1
!, INTERVIEWS 305
i
' . The use of multiple interviewers in either a panel or a serial interview is
J.
commonly used.with other means of structuring, especially standardized ques-
! ti~ns. Multiple interviewing also can be used to good advantage in an other-
':1se unstructured interview because it has potential for providing a more con-
if sistent sample of applicant information.
There are two main issues relating to evaluation and decision making in _inte~~
a
k? questions or dimensions in order to fonn total score for each job candidate.
tive manner by defµung weights that give higher values to the more important
questions or dimensions, as is done for items on weighted application blariks.
The second issue·concerns how. to combine or consolidate ratings across inter-
viewers, such as when a panel is being used. Sometimes this is done by consen-.
sus. It also can be done using some actuarial procedure, such as summing or
averaging the interviewers' ratings. In structured interviews, objective proce-
dures should be used for both purposes. because errors can be introduced
when ratings_are subjectively combined and when interviewers discuss candi-
dates in trying to reach consensus: ·
~erall Ratings and Total Scores 'To maintain.the high structure of an
interview that uses detailed job-related questions and anchored rating scales,
intervii;:wers must rate a candidate's answers on all questions. Ratings-of mul-
tiple aspects of a candidate's performance are preferable to a single, overall ~at-
ing ~cause each question has been des_igned to elicit a relevant sample of be-
havior that has been weighted in terms of importance. This•makes it easy for an
interviewer to evaluate' each sample with minimal subjectivity in the process.
However, some interviewers prefer to give a single, overall rating to a candi-:: •
date's performance. The·disadvantage of this is that the interviewer will use a
subjective process to decide the overall:assessment. The subjective process
~ly will involve the interviewer's own sense of the relative importance of the
different questions answered, as well as the formation of a general impression-
of the candicl,ate. Whenever rating requirements are less question-specific and
· more general in nature, subjectivity is introduced. For example, when inter~
viewers are required to rate broad dimensions that might be implied from mul- ·
tiple questions; instead of rating.answers to single questions, the rater has less
clarity on how the individual answers are relevant and the amount of weight in
the rating they should carry. Thus, dimension ratings-like the single, overall
rating-may contain error because the rater has subjectively weighted and
combined a candidate's answers (Conway et al., 1995). · ·
 308 CHAPTER 11 ·

" \ .

· they were not standardized (Arvey & Campion; 1982; Harris, 1989; ·
Erro~
Schmitt, 1976). Research evaluating the validity of job interviews has been Iary .r,.
more 'positive in recent years, and the change appears l~ely to have re- review \,
sulted from increased study of structured interviews. Clearly, inte~ews rem •·
can be relial;>le and valid, but this depends on the extent to which they use .
but
stan9ardized procedures and incorporate high levels of structure (Cam- : ·
pion, C~pion, et al., 1994; McDaniel et al;, 1994). · /

descriptl,.
Int_erviews can be structured in many different ways. In this section, f
discussed several .that appear to be n~cessary for reliability and validity. oth~
be eva1u,f1
_ Probably, any of these ·elements of structure could·help to improve an
employer's interview. Some, undoubtedly, would contribute more struc- . Ho~ I

:~ i
L• '
ture than others. Certain fea~res of structuring are essential. Job analysis
is required to develop job-related interview questions and performance-
anchored rating scales. Interviewers must control the conduct.of the inter-
view by asking the same questions of each candidate -and rating each an-
,;:ill
__rors&:!
swer. The· employer must provide in-depth trai~ing fo~ interviewers.
Certain other features of the_structure~ interview appear to_be less.critical,
such as disallowing probe questions and prohibiting panel interview di~
. cussion of candidates. Some of these features mighfbe made flexible, if
:~iJ
in __ ,. , [

the interview is otherwise highly structured, in order to make it seem less

mechanical and more acceptable to interviewers~ I~
'
H
as~[
well ,
llmRVIEWER AITRIBUTES AND BEHAYIO~
-F~~ [-
p_non~f,
.. . Although I en~ed the above ~cussion by opservirig that researchers now.are VIe~ i::.s
' more positive about the effectiveness of interviewing than they: were in the '
' past, I would not like to give the impression that the·problem of unreliable and
mo~g 1•"'=
te~
invalid inte~ews has gone away. The improyement of the job interview clearly als~ ~
depe~ds on the extent to which measures are taken for standardizing and · · onl · ha.:IJ
.) .. strµcturing. Job interviews are conducted daily that-have no such.improve- a •-!(n,o•-
ments. In these interviews especially, the accuracy·of applicant evaluations de-- mation 1

pends on the individual abilities and behavior of interviewers. In this· final sec- jo _--...,.~
tion of tile chapter, I discuss some of the variables that -might contribute to pli~ts

I
differences in interviewers and the accuracy of their evaluations. · a~
!ion . ir
Preparing for Interviewing . - allow~
Interviewers differ as to whether they prepare for conducting job· interviews. no~ cc.
Some give very little attention to preparation. This usually results in an erratic

e,
line of questioning or takeover of questt'oning by the job candidate. Other hi-
terviewers prepare by reviewing candidates' job applications, resumes, and at'. ~ ~'
other ancillary materials. As I mentioned earlier, this can result in an intel'view
which has little validity beyond that of the pre-interview application materials.

12: lfons, 1989;
,. tervieWs has been
~-wargely to bave re- ·
~ -" :!early, inteJV:iews
" _to which they we
- ~ : structure (Cam_- ,
:a; .In this section, I
1111111~ >ility and validity.
;. Ip to improve an
'11111111~ bute more struc-
ntial. Job analysis
. . ... nd performance-
1duct_ of th.e inter-
d rating each an-
~r interviewers;
111111 to be less critical,
.. 1ei interview dis-
_ made flexible, if
~ --w
-~- -._ '\
:nake it seem less
~ c
. --
_ - --
I
earchers now are
· "W th~J.'
were in the
of µnreliable and
I int~ rview clearly
andardizing and
o such improve-
I - 1t evaluations de-
. In this final sec-
I ~ht contribute to
~ ms.
-
1- .,
g job ·interviews.
--:, mlts in an erratic
:lidat~. Other in-
IS, resumes, and
It in an interview
cation materials.
INTERVIEWS 309
Errors will result because different information likely is obtained from ancil-
lary matertals on different applicants. For exam_p le, pre-interview application
review will contribute to a generai impression of the job applicant, which will
remain. with the interviewer and affect not only how the interview is conducted
.but also the evaluation of the person. _
Better preparation for interviewing would be to review job analyses, job
descriptions, or other job-related materials describing the required KSAs and
other attributes. It would be helpful to an interviewer if the job dimensions to
be evaluated had already ·been identified and-if questions had been framed,
Howe\ler, if this has not been done, then the interviewer personally should de-
fine the dimensio~s and <;levelop questions. Even better preparation would be
to provide formitl training for the inteJV:iewer that includes instruction and
practice _on conducting an interview and on how to avoid making common er-
rors in rating. As· I discussed previously, the research on structured interviews
has demonstrated that interviewers can conduct a valid interview and provide
ratings that are predictive of future-job performance. However, this is not an
intuitive process. Interviewers must learn how to do it.
Interviewing Behavior
How an interviewer behaves depends on his or her own attributes and goals,
as well as the interpersonal dynamics that develop within an interview,
For example, a study of campus interviewing showed that an interviewer's
priorities-to recruit Qr to screen applicants-influenced the campus inter.,.
viewer's behavior (Stevens, 1998). Those who were focused primarily on pro-
moting their company and recruiting applicants tended to monopolize the in-
teJV:iew. They talked 50 percent more than interviewers who were concerned
as
also .with screening. They offered twice much information, and they asked
only half as many questions as their more selection-oriented counterparts .. .l\s
a result. of course, the r~cruiting interviewers· collected relatively little infor-
mation froi:n interviewees that could be used to evaluate their suitability for a
job with the company. I suspect that many workplace interviews with job ap·
plicants are conducted in the same way with the same result, especially when
an interviewer has _n ot prepared. Sue~ behavior, when the purpose is selec~
Jion, indicates that the inteJV:iew~r has lost control of the interview.
To obtain enough information to make an evaluation, an interviewe~ must
allow and encourage the candidate to do most of the talking~ When this does
not occur, the interviewer will base evaluations on a general impression of the
candidate or on information gained from pre-interview examination of ancil-
lary materials. Interviewers also should be aware that some candidates might
attempt to distract the interviewer by asking questions of their own, in order to ·-
a
avoid being as_k ed interview questions that might be difficult. If qmdidate
knows he or she looks good "on paper," but does not wish to answer job-related
questions, the individual might maintain the paper image by consuming
 310 CHAPTER 11 j C

'
I
].
r
interview time in this way. An interviewer who is not adequately prepared to I
regarj-:c
control the .interview is susceptible to this type of manipulation.
11 '
;; have~
In addition to asking the questions and then listening, interviewers should A._ ,
take notes on what a candidate says and use them to decide on the individual~ ·fects ~
suitability for hire. Notetaking is beneficial because it increases the listener's ·
,of the c
attention to and understanding of the candidate's information. Notes also help
· beca~
in recalling what a,candidate said. Some research demonstrates that notetaking
can contribute to more accu,rate assessments of interviewees (Burnett, Fan,
_lidity in
Motowidlo; & Degroot, 199~; Macan & Dipboye, 1994). Both the act of writ- valicC:
intervie•
e:
ing notes and the content of notes _make a difference. Interviewers who volun-
' ,
tarily take notes have ·been found to make more .valid evaluations than those _
.

who do not take notes. Behavioral notes are p:uticularly useful. Interviewers· · Person
who write down"behavioral infonnation about a candidate, such as what activi- - ethn· '-+
ties the person performed on ajob, make more valid evaluations than" inter:- asses~
viewers who take other types of notes, such as notes on mannerisms (e.g.; over;Jt:_
"good eye contact") (Burnett et at; 1998) .. · - · havep i
beco,!Z!:_
Impression Con&mung An interviewer should not review a candidate's . cow~
application materials in preparing for an interview, because doing so can lead to sions. ,
· bias. The problem occurs because the interviewer fonns a pre-inte.rview iin- - ent~
pression of an applicant from the ancillary materials and then.conducts the in.. , Mc
terview in a manner so as to conBnn the imp~ession. Impression-confirming the ~
behavior also can ·be initiated when a general impression is formed at the start views or
of.an interview, such as mighti>e based on an applicant's.personal appe~ce.• . 198~
Impression-'COnfirming behavior can inc_lude' cognitive distortion, s_uch as ·_ than.W tj
when an interviewer p!lys attention to ce~ain infonnat;ion provided.by the can- . cei\.l~ '
didate and ignores or discounts other information. Impre~sion-confinning is ,
commonly seen in an interviewer's overt behavior, -such as in the kirids of ques:- -
inco 1.fo _'
in ~
tions asked. Intervtewers who develop their own questions for individual appli.,. r _
cants-and thus conduct a largely w;istructur~d interview-may design ques-
tion~ that confirm their assessments .of application _Illaterials. Research shows
.;:iii
. :~f
of i _
that is especially likely when the application revi~w suggests that the ap\. that inc~:
plicant is relatively unsuitable for the job. For such c~didates, the questions. stil~
- are more likely to be framed so as t~ elicit neg11tive inforri:iation ( Binning,
Goldstein, Gar~ia, & Scattaregia, 1988). Inte.rviewers also tend to ask these can-
didates more difficult questions li!ld fewer positive questions than they ask can~.
~:;e~t
~
examii
didates whose ·applications ·show moderate or high qualifications (Macan & . ing~
Dipboye, 1988 ). An interviewer'.s interpersonal interaction with a candidate also plex°(e.: Ii
differs depending on whether a negative or a positive impression has been re~lk:i~
formed. Further, candidates react accordingly, so a job candidate can become a mis~
contributor to a self-fulfilling prophesy by responding in a way that is consistent hiii,1rnl;

: ~.a.•
with an interviewer's behavior, whether that behavior is positive or negative. tiy. ~ ~
Dougherty, Turban, and Callender (1994) ·found that interviewers who had
a positive pre-interview impression of a candidate exhibited greater positive
views
~ -
('~
 INTERVIEWS 311

=' ly prepared to
-> I,

viewers sllould
regard and personal warmth .in their interactions. In response, the candidate be-
haved positively and attempted to establish rapport with the interviewer.
As you might conclude, impression-confirming is a bias that- seriously af-
W the indivi <lual's fects the interview. It reduces its value as a selection device, not only because
~s the listener's
of the curtailed use of the interview as an information-gathering tool, but also
otes also help
because this kind of behavior places !lJl additional burden on the Iieed for va-
that notet aking
lidity in the pre-interview application materials. If application materials are not
(Burnett, Fan,
valid, .impression-confirming behavior carries. the invalidity forwar~into the
the act of writ-
interyiew. If they are valid, such an i~terview is probably a waste of time.
·ers who volun-
W ons than those
Personal Biases D~m~graphic variables,. such as sex, age, .a nd race or
I. Interviewers
ethnicity, potentially are at the source of some impression-confirming or other
as what adivi-
assessment biases. In .the past, as you lmow, such applicant characteristics were
)ns than inter-
·overtly considered in selection decisions. Fair employment laws, of course,
" merisms ( e.g., have prohibited selection discrimination ..As a result, job interviewers may have
become sensitized to the unfairness of taking demographic variables into ac-
v a candiclate's ·
count; and they may now.be more carefuito avoid making discriminatory deci-
sions. Another possibility is that the bias remains, but is now more covert and
U 5 socanleadto ·
enters only indirectly into interview evaluations through some other variable. ·
• -interview im- .
. Most of the research investigating race and ethnicity effects has _compared
mducts the in-
the differences fn interview scores. between Black and White applicants. Re-.
W on-confinning
views of the earlier studies found the results to be largely in~onsistent (Harris,
1ed at the start
. 1989)."In·some studies, Black candidates received lower interview evaluations
LI appe~ce.
than White candidates (e.g., Parsons & Llden; 1984). I.n other studies, they re~
Jrtion, such as
ceived higher evaluations (e,g., Campion et al., .1988). Although there s9,ll is
led by the can-
'inconsi~tency in research-results, it .now ap_pears that raci~ group differences
·cohfinning is
in interview assessments are rela~vely mJnor. In ·1998, Huffcutt and Roth
;) ~ds of ques-
meta-analyzed the ·results of 31 published· and unpublished studies that had
div'idual appli-
been conducted between 1970 and 1996. These studies included comparisons
""\ y design ques-
of interview scores of Black and White .intel"\l'iewees, as well as a few studies
rl esearch shows
that included Hispanic applicants. Overall, the studies showed that there was
ts that the ap-
still variability in the direction and extent of group differences. On average;
the questions
however, White interviewees received ratings that w~re less than one standard
:ion (Binning,
deviation (.25) higher than those of Black or Hispanio intervie"'.ees. Further
ask these can-
1 they ask can-
examination of the data showed that certain moderator variables .were operat-
ing. One moderator was the complexity of the job. When a job was highly com-
ms (Macan &
candidate also plex (e.g._. technical management or nursing), Black and Hispanic candidates
,ion has been received higher overall evaluations than White candidates. The reviewers sur- .
1 can become a
mised that because experience and educational requirements for such jobs are
high, minority candidates may represent a select group that is sought in indus-
tt is consistent
try. A second moderator was interview design. Group differences were lower
e or negative .
when an interview was highly structured than when it was relatively unstruc~
.vers who had
1 tured. That is, candidate rac_e had less impact on ratings in structured inter-
eater positive
views. Huffcutt and Roth (1998) thought structuring might be suppressing the
 312 -
. .,,,~ l
CHAPrER 11
\l manage~ ·a1 , I
use of stereotypes 10· interview
· · decision making. This is yet another reason
· when ~
to
recommend structured inteIViews. . that such a h~
whethe~
On~y.a few shldies have assessed the effect of applicant age and, according
to Harnss. (1989) revie_w, th e resUlts
in
__,. of these are inconsistent.
. Age appears· to been oc'"upH ~
i such joh~ a
. t~ract Wl~ other vanables, and some studies consider both age and gender ,I
·o f JOb applicants. . · I ''viewed~ ~
l
Harri~ fowid that gender studies conducted in the early 1980s more often
'·
\ assertiv~nes,
·I
viewh '_ _
rep~rted that females received lower interview ratings than males, whereas i · Reacti:;_~
studies con~ucted later in th~ decade showed little difference -in these ratings.
Perhaps, as m all fonns .of illegal discrimination, interviewers were becoming
-.~
course.!. . . , _I
more aware and were trying to avoid sex discrimination. Raza and Carpeiltei\ agers, the _')'r·:::.
- ( l~7) suggested another possibility-that demographic variables, such_ as a~ · study i~ _ II
~licant sex and age, do not directly affect interview evaluations and hiring deci-. _ '!hey thoui [ ~
sions. Rather, they suggested, the effect on interview outcomes is i n ~ oc-· · mgcan~ -1
curring mainly through intervening variables, especially attractiveness.' They - ness~bi~ r --..,
~xpec_ted t~at the greatest direct impa~t c;>f s_ex and •age would be on an experi~ -~
~temewer s evaluation of an applicant's physical attractiveness and that attrac.; matenai. -~
tiveness ~ould indirectly affect interview decisions. Raza and Carpenter (1987).
~ee::"~ ~
th~1~ I?.
conducted a field study that generally supported the hypothesized relationships~
at least as far as applicant sex was concerned. The strongest direct impact of sex _ both
Wl:lS on assessments of attractiveness1 with relatively minor effects on hiring de- - evalua , .
cisions. In the c_a se of applicant age,·howev~r, the predictions were not upheld.· :i as more suit l
Interviewers evaluated older applicants negatively on multiple measures. Fe- menas~
male interviewers considered them less attractive: Both male and female inter" . gene~~
were ~
l
.
l
· viewers rated them less intelligent -Male raters ~o rated them as less hirableo
· ,-,.- More recent studies have focused ;on;attractiveness l;>ias_, particularly-as it low le_veis c•
' might affect female job candidates, For example~in one ~tudy, re~earchers expene~ -~
wanted to know whether overweight applfoants--especially females--:-are .dis- ~tudy ai't:: co
· attracti .ue4
a.
i
advantaged in eniployment'interviews (Pingitore/Dugoni, Tmdale, & Spring,
1994). They developed videotapes of simulated job interviews using·profes..; · tivenes$ ,1,; a
j
sional actors as job applicants. The male and female actors appeared as their · 1
normal weight in half of the videotaped interviews; and in the other hal£,.they
were made up with prostheses to appear 30 pounds heavier than normal. Stui ·
dent raters viewed the videotapes and rated applicants' suitability for hire. The · ~ ·
results of the_study showed hiring bias against both overweight applicants and- What C:C-r
female applic~nts in general, and particularly ag~st overweight femal~s. . eS{le., ·' I
· Being physically unattractive can be a disadvantage in job interviews.- planati, ....
· em·-.,,
However, some researchers have suggested that physical appearance might be
as recent "•
somewhat double-edged in its effect, as far women are concerned. That is,
although attractiveness is an advantage to some applicants in some situations, e_m!S!
it might be a disadvantage in other cases. Heilman and Saruwlttari (1979)
found that physical attractiveness had different effects for rt1en and women
depending on the type of job they sought. For male applicants, attractiveness
;;:~~~:.1~-.Lil
ClSIC.,,

was an advantage regardless of whether they appli,e d for .manag~rial or non- .

a
 INTERVIEWS 313

managerial jobs. For women, however; attractiveness was an advantage only

nother reason to
when they were seeking nonmanagerial jobs. Heilman ( 1983) hypothesized
that such a bias occurs because interviewers evaluate personal attributes as to
, · · ~e and, according
whether they flt the perceived job requirements. Managerial jobs have long
t. Age ap?ears to
been occupied by men, and these jobs have become male gender-typed. In
1 age and gender
such jobs, a female applicant might be seen as a "poor fit" because she is
'viewed as having fewer of the attributes of a successful manager; for example,
.980s more often
assertiveness ancl competitiveness. If she is attractive, the interviewer might
, 1 males, whereas
view her as even more unlikely to have these attributes.
-W , in these ratings;
. Reactions to women applying for managerial jobs might have changed, of
,.. , were becoming
course. Because of the increased numbers of women actually working as man-
W a and Carpenter agers, the use of biasing stereotypes in interviews might have c,ieclined. In a
~, "' .hies, .as a~- · study in 1996, Marlowe, Schneider, and Nelson considered this possibility.
--V , and hiring deci-
They thought that managers who have more experience in hiring and promot-
:!S is indirect, oc- .
ing candi :iates for managerial positions would show less gender- and attractive- · ·
:1ctiveness. They
ness-bias because of their longer exposure to female managers. In their field .
.vould be on an
experiment, managers with high or low experience reviewed simulated resume
~ sand that attrac-
. ~arpenter (1987) materials.that included a.photograph of the "candidate." The manage.rs rated
the candidates in terms of their suitability for initial hire and promotability to
,. ;ed relationships,
--., impact of sex an executive vice presidency: Results.of the study showed significant effects for
both the physical attractiveness and gender of applicants, indicating that the
.. :cts on hiring de-
· . evaluations were biased. As a group, the managers rated attractive candidates ··
W vere not upheld.
as more suitable for hire and more likely to be· promoted. Similarly, they rated
e measures. Fe-
men as more hirable and.promotable than W?men. Although the effects were
llld f~male inter- . .
generally smaller for the more experi~nced managers, less athqctive females
1asless~e. ·
were disadvantaged regardle~s of the level of rater experience: Managers with
particularly as it
low levels of experience conside~ them to be iess hirable; whereas the more .
1dy.~ searchers ·
experienced man~gers considere~ them less.promotable. The results of this .
""'imaf~s--are dis-
study are consistent with other studies showing that a job candidate's physical
1dale, & Spring.
attractiveness is' a compelling variable in interviewers' judgments. U nattrac-
_ ..._, vs u~ng profes-
Jpeared as their ·
-.. other ha!£.. they
-,, ,an normal. Stu-
. ' - -
tiveness is a disadvan~age for female candidates in.managerial as in other j 9 ~

.ity for hire. The

t applicants and•
What can we conclude about inte~ewer biases in job candidate ratings,
ht females.
especially considering that the research seems so inconsistent? Several ex-
job interviews.
planations for the inconsistencies have been offen~d. For example, fair
irance might be
employment-laws might have caused a change in interviewer behavior in
"?cerned. That is,
.recent years. Interviewers might now be more sensitive to the issues offair
,ome situations,
employment, and maybe they try to avoid making discriminatory deci-
~-uw~tari (1979)
sions. Interviewers in. industry may have become more sensitive to the po-
...,,en and women
-tential for EEO complaints and la':Vsuits, and they may try to avoid any de-
_...s, attractiveness
cision that might result in one. (I think employment interviewers do tend
-__;,agtrial or non- .
314 CHAYl'ER 11

to be alert to this.) Also, perhaps .there are more women and mino~ty
members who, having personally experienced discrimination, are now m a
position tq change things. That is, perhaps there are more who will evalu- .
. ate positively and hire "one of their own." This might not be entirely off ·
l>ase because some racl;'-similarity research on employee performan,ce ap-
praisal has shown that supervisors rate subordinates of the same race
higher than _they rate subordinates-of a different race (e.g .• Kraiger &
Ford, 1985). The effect also.has been ·observed in interviewing studies in
which interviewers give higher ratings to applicants who·are of the same
race as theY-(Lln, Dobbins, &·Farh, 1992). · · .· ·
Another explanation of the inconsistency in studies of interviewer bi-
ases is that it is due to research methodology that clouds the issue. One
point in the methodology explanation is that lab research •done in univer-
sity. settings using students as raters has serious shortcomings in that it is
not a good simulation of the workplace. Student raters ·may ha~e little ex• ·
perience and motivation and expect no consequences for a poor perfor-
as
mance a rese~h subject. Critics ask.us to compare·this to what takes ·
place in industry when interview(lrs are experienced and motivated and ·
must take responsibility for the decisions they.make.·If i could be sure that
real interviewers greatly contrast with students in this respect; I would ·
agree that lab research does not tell us much about the workplace. How,.
ever, I am not entirely convinced.. . . .
There is one aspect of the research methodology explanation that
should get more attention. This ·has to do with the overinterpretation. of
failed research. results. What does it mean: when an expected effect is not
shown in a study? We do not know. We Cll;fl only guess. If a researcher ex.:
.pected to observe race,- sex; or age effocts in interview ratings; but. the
·· study failed to show. these effects; there are any numqer of explanations of
why the expected result" was not shown. One that. ~ight especially appeal
· to a researcher is that intervie~ers rio longer consider-these demographic ·
characteristics. Of course; it is possible that it is the best explanation. How..
ever, the researc~er·cannot know whether it is the best, .unless the study
design included a special feature capable of ~upporting such an interpreta-
. tion. I think researchers sometimes need to remind themselves to be espe- .
cially cautious in interpreting the meaning of "no effect" in their research
analyses. Failing to find interview differences may not mean that there
weren't any.

SUMMING UP
'
In this chapter, the purpose has been to examine and evaluate job interviews
and explore the possibilities for improving this popular selection method. We
can sum up as follows: · ·
.N~ ASSESSMENT CENTERS AND OTHER SIMULATIONS . 317

,~N
I~ Job simulations, which are the focus of this chapter, are nearest to the job
tryout in terms of realism. Although they show less fidelity to the job than the ·
job tryout, simulations mimic the work contents, and. the responses evaluiifod
are similar to those required on the job. Simply-put; a job simulation uses tasks
like those in the job as the contents of the selection measure, and it calls for
test performance that is more or less comparable to job performance. The ra-

I~ tionale is that if a representative sample of the job is obtained, it can be consid-

ered to be a miniature version of the job, and any applicant who can success-
fully perform it can be assumed capable of performing the whole job. Because
I~ the job simulation can be evaluated as to its measurement quaiity, the user can
be assured of its outcomes. In addition, because the simulation method is con-
I~. let them try it. A
.. ducted prior to placing an applicant on the job, it carries less risk and expense ·

I~ 1ethods. The job •

mance is the be-
for U}e employer than the job tryout. . . . .
In this chapter, I discuss job simulation methods that are used in employee

~
1~ . .red temporarily
J for a predeter-
selection. These include situational judgment tests, work sample perfonn~ce
tests, and assessment centers.

.~
:mance is moni- .
1
m
-,. , :essfully perform
se, the person is
Joe SIMULATION TEsrs
.· Job simul~tions are distinguished in te~s of the fidelity of the measure .to the
_ an applicant can job. High-fidelity measl!fes simulate the job most realistically. A work sample -
, ~ >ably can do .the test, in which applicants perform a sample of the job, is an example of a high-..

I~ ( employers who
1ey Clµ} use it to
ods -~ in place,
fid~lity tneasure. . . · ·. .
. . _Low-fidelity simulations, although clearly job-relevant in content, often
use hypothetical mat~rial and are less directly reflective of the actual job: In ·
. ·

e:... ~ re is.needed im- low-fidelity simulations, a. work situation is presented to an applicant-either

as
l~ --., > m~ e other
acq;,ptable per-
se:. orally or in writing-and the applicant responds to how he.or she might deal ·
with it (Motowidlo, Dunnette, & Carter, 1990). For example, the ~ituational ·
interview -~ considered a low-fidelity job simulation. The interviewer describes
1-.;) ,t, there ~an be
full capacity to
I equipment can ·
a work situation or problem that might be encountered on the job, and the in-
terviewee says how he or she would deal with it. (Refer back t-0 Table 11.3 for

I~ ?rvices and cus-

.ch problems re-
some examples of such questions.) Like other low-fidelity job simulations, both
the situation described in the situational interyiew and the applicant's response ·

1.
_
onetary cos.ts. If
1e employer will
are hypothetical rather than factu~.

Written and Video-Based Situational Tests

I ing process over
___ ve procedure to
olves ""Pensive
Situational judgment tests are low-fidelity simulations. To construct such a test,
realistic descriptions of work situations are drawn from a job analysis and u~ed
as items on the test. Either a written or a video-based fonnat can be used m a
situational judgment test to present work situat~ons a~d to collect appl~cant re-
I ,talion ls a fonn of
__., fer to anotlier Job. sponses. After each situation is presented, the applicant responds either by

,/
 318 CHAPTER12 ' ''.J-=-:""Ill
ltn
appli~~
Imagine that part of your job involves selling up and maintaining machines for office workers
to use. In the course of doing your job, you have discovered that some of the office staff
test. For ·UlllLi
t o r ~lt:Ja
mishandle the machinery, causing it to require more maintenance and repair. What woulc1 you
do about this7 · . · I
• I activity ::.: IUJJII
bee:{r=
Most likely, I would _ _. ·,
It is least likely that I would _ _ ofrespor. :Q
"tj
be a(rr" .
a. Tell my boss the reason for the increased ma_
interiance and repair costs. showed c Br
me81::=::W.
Ull ,I
b. Alert the office manager that maintenance and repair costs are due to some staff
. misusing the machinery. ·. . . . . · • . · · · ·
were
· c. Post signs alerting us.ers asto the need for careful use of the machii:,ery. .
ing i_,,,Y).C:

:v!:-:~ 1 :-
d. Suggest to the office manager that no one be allowed to operate the eq1:1ipment
unless they have received trainin1r . , .. . .. ,. . . .. · · · · • .. ab~:.
·e. Warn the office staff that after a point you will b~ unable to repair the equi15ment encl

:;e~
-l ' . there may be no money for replacements. . · .. . ·. ·
I •'
.·.p FIGURE 12.1 l , l f ; , • ' I ; • :; ~. • •.· ' • ,l • .

1 r.
Example of a written situational judgmeAt test item and response alternati_
ves. Applicants
D~ -
· I

would select one response to represent the most likely action taken and another to represent . I
the least likely action taken. . · · .I sihl~ti. -J :_
I In ~ '
' H~sor ,--
writing out a statement of what he or she would do in: the·situation or, •more
typically, by choosing one of several I11ultj.ple-choice resporises, For_an ex- mut;;:~ 1
ample, see Figure 12.1. Sometimes, applicants are as~ea to choose both their
ture.·or ,-
,
most- and least-likely response from an array of respons~·alternatives. Iri situ- re~~~:.:,_ !

j
ational judgment tests, as in situational interviews1.hypothetical rather.than iml_:,_,h 7--
. real job behavior is elicited. ~ven though very. realistic job situations might. be cessful -
depicted. For this reason, situational judgment tests show.lower fidelity'than
l.~-----
Tn.;. , :,.. -
J
1

worst I n
I.

~imulations calling for actual ·performance 0£,real job tasks.- such as work . \·!
dec:~
'; I

sample perfonnance-.tests, which I discuss later. '·: •· · l. , ,, _

1
Over the past two:decades, as convenient.,idep equipment has become cal incJ,
available, several video-based situational judgme'nt tests have been developed. .
For e~ple; video-based tests have been.devised for selecting metropolitan'
~ ' - . ., ,
. .,
cussic ·
transit operators. (Smiderle, Perry, & Cronshaw, 1994);·insurance agents ti.r..--.. ,. .
'' . (Dalessio; 1994); employees in the hotel and hospitality in~ustry (Jones & was e ,-
DeCotiis, 1986); hourly retail store employees; and nursing home caregivers I.. inF~~-,
(Weekley & Jones, .1997). Typically, in these tests,._the video depicts the process I ..i\ .., '
of a problem arising in a work situation. At the point when the problem be- I thP~ U'.iI'
. comes apparent, the scene·freezes and the applicant is instructed to select ( _.I
Cl..Jl-.
on-screen response alternative that best describes· how he or she would handle \
their l.
the problem.
The video format can have certain advantages over a written test. More
.. I
\
C ..
( ... ,:,,,

detailed and realistic work situations that have richer meaning can be pr~-
sented by using the video format (Weekley & Jones; _1997). Because of this,
,~
,~I~ inesfor ffi
of th . o. ce workers
• ..
ASSESSMENT CENTERS 'AND arHER SIMULATIONS 319

re .e office staff
applicants may react more positively to a video-based situational judgment

I · pair. What woulrl ycu test. For many job applicants who are mar~ accustomed to watching 'IV than
to reading, responding to a video-based test may seem to be a more natural
activity and more acceptable than.taking a written test. A video-based test may
, I be especially appealing if the video includes voice-over narration and a reading
l

1, ~pair costs.
e due lo some staff ·
-. , of response alternatives. For jobs that do not involve reading, the video might
be a more valid presentation mode. A comparison of situational judgment tests
showed that test takers' scores on a written test were correlated positively with
measures of the individuals' reading ability, whereas scores on a .v ideo test
, ~ machinery. were unrelated to reading ability (Chan & Schmitt, 1997). Therefore, if read-
I~ ate _u,e eqaipment ing is not required on the job, or if the test is not needed to measure reading

, •air the equipment and

ability, then a video-based test might be preferable. However, despite these
advantages, there is a serious disadvantage. Video-based situational tests can
I ~ ' " be very expensive to ·produce, especially if professional actors are used.
Weekley and Jones .(1997) reported that .the cost of developing the video test
~.
II~ .
lives. Applicants
for their study was about $1,500 per finished minute.
' '
another to represent Developing a Situational Judgment Test The- basic construction of a
situational judgment. test is done similarly for both written and video formats.
In both cases, the process is lengthy, as it involves multiple steps (Motowidlo;
,, situation or, more Hanson, &.Crafts, 1997) .. In the first stage of development, the test developer
I~ • onses. Fo~ an ex- must identify situations that actually occur on the job. This is the distinctive fea-
I • choose both their ture. of all job simulation methods. The situations selected should be ones that
ematives. In situ- represent a challenge. for the employee. Each one should be something that is
=--w
I etical. rather than
tuatio.ns might be
important to handle appropriately and.in which employees vary as to how sue.:.
cessful they are. Response alternatives need to be developed at this same time,
>wej fidelity than · · They should.describe behaviors that vary in effectiveness-from.the best to the _
worst thing an -employee c?uld do. Obviously, a job analysis is necessary in or.:. ·
ks. rJ.•·:s uch as work .
. l_ der to define and describe work situa~ons and response alternatives. The criti-
neni has become cal inciqents technique is pafticularly helpful because it identifies the critical
aspects qf work, which not all employees can perform. Recall froni previous dis-
. been developed.
cussion.of this job analysi~ method that each critical incident includes a descrip--
ing metropolitan'
.tion of the background situation, details on exactly what an employee did that
nsurance agents
was effective or ineffective, and an outline of the consequences. Thus, a critical
1dustry (Jones &
incidents analysis provides the basic material for developing a job simulation.
home caregivers
As an example, Motowidlo and his associates (1997) outlined the steps
:picts the process
they used to develop a situational test for a skilled electrician's job. First, they
the problem be-
.collected a large number of critical incidents and organized these according to
- ; , cted to select an
their behavioral conte!}t. They grouped. the incidents into three categories. in-
he would handle
cluding planning and organizing the work, troubleshooting, and supervising.
These categories suggested the performance dimensions on which effective
itten test. More
and ineffective electricians differed. Second, because the researchers needed .
.ing can be pre- more detailed information on the situations surrounding 'these performance
Because of this,
 320 CHAFI'ER 12
41
dimensions, another group of SMEs-exp~rienced electricians-was as-
sembled. They were asked to develop full descriptions of a problem, which · ratin5-- ·1
they had encountered and handled, that exemplified one of the performance
dimensions. To accomplish this, the researchers instructed the SME to de--
scribe the problem i~ detail, state how it was resolved, and ·identify ·the rel;_
~vant perfor:mance dimension. Third, they then sorted these problem descrip- ·
.tton_s according to the type of work activity addressed. This resulted in another
.set of ~erforman~ cate~ories. For example, some of the ptoblems·described
=~,
: shown

rangi;(~.
·r~search h1:1I
1 •'

dealt WI~ blueprutt readi~g, completing jobs on time, !Uld working with overly· ·ies,aC=-
demanding.customers. Fmally, a selection of the p~blems were drawn from scores on ;:I
each catego:ry to make up the items in ~'e situational judgment test-. Problems . ratin~-.:...::.:.
were selected so as to represent the important kinds ·o f situations that occurred· .
on_ the job, The proportio.n of iteins fro~ each-category depended on how im.; · ·
portant the catego:ry was to successful job performance. . . · · ,. , ; i·
tS:
rion-relat;-1
tion
2

, ·: As last step, in developing a situational:judgment·test; response altenia': •

tives for the indiViduat items need to be created. Motowidlo et al. (1997) de- ·
~!r~J
video .....,..,
scribed their procedure for doing this. They prese0:ted test items, open-ended . Weekky
to a sample of relatively inexperienced jo.b incu·m bents, and·asked them· to situ~_: -
writ~ a description of how they would handle each problem: The- researchers ·· home ca:
then selected a set of five response alternatives from the·incumbents' answers S1,1pe1~-~
for each item. They recommended that the resp~nse. alternatives selected. in the reta
should represent a ·range of strategies that applicants might use ·~o hllndle· a,.·
problem. Next, a panel of SMEs rated each resporise·al~ernative as to its effee.;.
prob" ;:-
.30s. Thes.
tiveness for solving the problem. The researchers used these ratings to identify •. tests ~ '-
the best and worst alternative for each question .and to develop ascoring key: ;,. ·
• • • • •• : •, p; • •' :~:•' :. , ::, k• 1 ' • : _( ..... • . ~::.} :•.'. ;'•~•;:--J, ,.

Validity Sbidies-i ijeviews ·o_f research conclude thatWiitte~·:r situiitiotihl judj ·,· ·
ment tests that are well-developed and-content-valid also-.have criterion-related ·, . Awo~s~
validity. Various criterion m~asures have been·used in the validity studies, iri.:. · sim£oi
eluding promotion rate, turnover; and supetvisors'. ,ratings of perfol'IT\arice; For:· · ·iature ·tS
example, Rosen (19(?1,reviewed the eVidence.on· the "How Supervise?". test .. Ashe~.JJ
(File,, 1945), which is a standardized situational·test used to evaluate supe_rviso:ry,. .. 1970on '
and managerial job candidates. Although the validation studies reported a range that tf; :-
of validity coefficients, Rosen concluded 'that the research evipence was strong .·· involving.
enough to support the measure as valid for predicting supervisory performance: guag-.:;:---
More recently,, re\l'iewers have evaluated validity evidence for various written requirin~
situational judgment tests. In general, the scores correlate significantly with rat-
ings of job perform~ce, producing validity coefficients .ranging from .20 to .50 · . I
"-"
(Motowidlo et al., 1997). For example, Motowidlo and his associates (1990) de-
veloped a situational judgment inventory for ·selectingjunior-level managers for . l. - 2Crou'( "'!, i
crciss-vai td:.
in
a consortium of companies the telecommunications industry. Using a sample l
I . orimn<>I val.
6"'"'; ....
of relatively new managers to validate the instrument,-the researchers found • sampl~ ,''
3Because o
that the scores on the inventory correlated from .28 to .37 with supervisors'
::r~·
 ASSESSMENT CENTERS AND OTHER SIMULATIONS 321
-vas as-
. which ratings of performance on the job. Criterion-related validity 'also has been
mance shown for ~tten situational tests of other types of abilities. Using performance
to de- · ratings as·criteria in .validating her tests, Phillips (1992, 1993) reported signifi-
he rel- cant validity coefficients of .18 and .24 for a test of sales skills and coefficients
escrip- ranging from .41 to .45 for a test of negotiation skills. .
nother Because video-based simulations are a more recent innovation, not much
cribed research has been published on their validity. Still, there have been some stud-
overly I
1 ies, and,some of the-results are positive. Jones arid DeCotiis(l986) compared
1from j scores on a video-based situational test for hotel employees with supervisors'
blems ratings and found a significant correlation of .38. Clearly, however, the crite-
:urred rion-related validity ofvideo~based tests varies. Dalessio's (1994) -cross-valida-
,wim-- · tion2 study, using insurance agents, showed a significant hut low correlation of
.
I
.13 with a criterion measure of turnover. Smiderle et al. (1994) reported that
they were unable to find criterion-related validity in their attempt to validate a
:erna-
r) de-
:nded
I !
video test of interpersonal skills required of transit operators. Mor~ recently,
Weekley and Jones (1997) conducted validation studies on two video-based
im·to situational judgment tests, using sampl~s of retail store employees and nursing
~hers . II' home caregivers: Test scores of cross-validation samples were correlated with
!
.wers ! 51,1pervisory ratings of performance. Validity· coefficients of .22 ·were obtained _
:cted in the retail store study and .24 in the nursing honie study. When corrected for
die a . probable unreliability in the criterion measure, the validities rose to the middle
ffec- .30s. rbese results provide at least some indication that video-based situational
ntify tes~ can be valid for selecting employees. ·
cey. i
I
I
j Wo,:k Sample Performance Tests. ·'. ·:· . _
1dg: i
: . . . .

1ted l A work sample used as a perfo~ance test< is considered to he a high-fidelity

,,in- simulation.3 The work sample test is a oo.mplex measure and constitutes a min-
For . ! .iature· replica of the job. As such, it is near a job tryout in terms of realisrt1.:- ·
test . ' Asher and Sciarrino (1974) reviewed research conducted between 1940 and
o.ry.. . 1970 on various types of work sample-tests that were used for selection during .
rige that time. They found that the tests could he categorized into two groups, one
mg involving physical manipulation or 11UJtor activities, and the other involving.Jan.,. _
ce. I guage- and people-oriented prohiems or verbal activities. For example, a te~t
:en requiring an applicant to type a sample _of material drawn from a secret~ s
at-
50
le- 2C~-validation is an empirical method for confirming a prevlousiy obtained validity coefficient. In a
cross-validation study, a second sample is Independently drawn from the same populatjon as the ·
or original validation sample. The te51 is administered and criterion data are collected on the new
,le sample. Analysis yields a new validity coefficient. . _ ·
1d lBecawe of their high fidelity to the job, work sample tests are sometimes used as cntenon m~a- .
, instead of supervisors' ratings of performance-to evaluate the validity of another predictor
:s :In.I., they can be used as criterion measures to assess the validity of training programs.
measure. s·1m.......
sures- .,,
 • c'I

. ~
,;,
-, ~i
..,,I. ..' I
322 CHAFI"ER 12,.
. ' i;-:
~sed in~ ill.!:..
;
,1 ,
'j 1ng app~a.
. ExAMPLES OF MOTOR AND VERBAL WORK SAMPLE TESTS establis_b..tL.
ments.'1!,
1
l
·' ' Typeot
: Test category Performance Exampln otTesta firm that t l•
f ' itywas ~
.,. ..
I
I Motor '· .· Physical object Drill press and l~the operation tests.for wood .
manipul_
ation _: ; , . _ shop workers . ·
Repafr of a-vehicle. engine for mecha,nlcs -
Validi( :,c,
· . have been :
· - .• ·t Stitching test for sewing machine operators ' · ·
' , Typing test'for. office clerks · .- · , ··•
1982). t:-i~
paredJo m1
Complexdrc~it ·sltnulation' festtcir;electricians"· - ·
' <. . .. , :: . I
I •
Drivingtestforvehfcle'ci~erafors ' :, ·.1:-\ ·, ,, ; : ,
sarnpl~ -~t
criteria. 10
Verbaj . ' Communication o'r _: . Leaderless group discussion for supervisors '- '
' 1 ; .~~. · '. ) -: ·, l·'. ·-, lnterpersonal :•; ~;--; t::·; ··:: . . .. · ;- ..... .;,. .. " .- ~··,· f. ,: · f -:- ~: :. t , 4 :- . • ~ 'I d ~.,.·r •;.---,. ~-· ' t·: r eritl, a~ir-
interactions
.
•· · In-basket s_
· -) •, · · -
imulation for managers · · · ~- · -
.., ·. ' -·.. . . - ,• -· -· criteriol:: , 1
Customer servfce _role-playing test ined work
.·: .~:J ;;~~~ .r:!' ;-:...r~_-.,•~rL._ ;~\ :l ~-~-{. . ;. . ; .._ ': .:'i ~.
. ' ' . . '. t . ! .. . .. . ;· ". { . '~ . .•...
.; n: . =~'_-.- ::. ·:·,;:~ - lated ~
··J
· Simulation pf testimony:takingJor go:,,ernrnen~ . ·-~- sample im
' _; -
•ins·i:,ectors_· :: -- , .. _ ·, . : ' . •., · - , _,., i ,
.- rion·w~
. . .
B~slrie'ss' lett~r'...wrfring test for administrators ..
,., ·, , .. . -· . beene~all
· critericf:

~-1, ,, !._'•: ,~~:t :•.·q >·-__., _:- c·;<_:;'.::,.- ,•.i-.:} 'f 1

) ; , • : , ; :· · ; :·: ~
'\ tests sho~
of the ,::-
work is.a work ~ample-test because of the ·use of typewriter or.coin- ·.. the ''_.~~i1· . were less
-~ uter _k e~~ --~ !~lep~o~e int~~ctio~~~ ~ a f H~~~IT\tr~--~~cp. _ ~,m!&?,t.~W' _. _ . ... :
mo~*-
·
m customer seIVIce. wor'k is an example of a _verbal work·sample test. Table ·
1.2.1 lists:otlier examples.ofwork sample tests::.'. · :,, -, , '. ·.:-•: "1 •· -: · :-: · :,_: ·,n q -:::, ' ,....., _ ; ,:
-

; J ~- ' (· ~j ;~
• • •

·•.; 1-_-.:
•

_; ,!.
I

: ;_;.; .··. H ~J ·. ·_1.-;, ... _. -

; • • I

-~.·.. · .· .. < ·; ;_,_-.:.,

• •

.i !. '. . ·· ·· t.:·3<··.., · : )f ~1--;~-~-·- t ·,~·- .

~l·
. _,..
verb"al'tc~
found,t~
least·.;j'.?',
DeVelopmeDt ·1Work satnple te~are,co~tructed fromthejob;:tisinga·d~ -- sample ~
-tailed job analysis,.-:11iecoiitents-of theJOb' are·carefully .analyzed:to obtain~<iet-i ' quite_/u
tain sample of required work actb,ities. ·The best sample includes,the m,ost im~ p~rfonrni
portant job activities; but•it_omits those that are learned eii the.'joh;·,To the· :- relate~
e~ent that w~rk sample tests arl:r'developed this ~y..they are.considered-to -be ··· worl
-content-,~d. Cascio and Phillips (1S79) -described the process ofdev.eloping 1,, verse•:;:-
.' :l e
and content-validating work sample performance tests that were used.to hire Kand ola.
city government employees.' The process began .with a comprehensive job and Phi~
analysis, which formed the basis for constructing the tes'ts. The job analysis was . no signs
conducte.d using a task inventory analysis procedure. (Recall from the previO:wi . were-~
discussion of job analysis that this technique produces a full description of a job \ . fromai-1
\Q.

and its requirements, including the relative importance of tasks and KSAs.-) -1 ingtheJ
· From the task analysis, the developers selected tasks that \\'.ould constitute the
work sanipl~ tests: Also, performance standards were determine4 that de-:
I may ha~

scribed levels of acceptable· performance on the tasks. These standards were I I

I
nence _
' · . "'
a realisti
 . I

ASSESSMENT CENTERS AND OTHER SIMUl,.ATIONS . 323

'I
l
used in developing instructions for test administrators on evaluating and scor-
ing applicant performance. Finally, SMEs evaluated the work sample tests to
establish that they were realistic and that they covered the es_sential job require-
ments. Through this process of thorough job analysis and use ofSMEs to con-
firm that the tests constituted representative samples of the job, conten.t valid-
m tests for wood , ity was established.

,r mechanics
Validity Studies In addition· ~o content validity, work sampl~ tests often
chine operators · have been found to have criterion-related validity (cf. Robertson & Kandola,
1982). This is not surprising; considering the high .realism-in such test~. Com-
est for electricians · pared to more abstract and less factual predictor measures; content-valid work
ators : sam·pie tests ~ight be expected to have_high validity for predicting performance
i
n for supervisors · i criteria. In a review of studies, Asher and Sciarrino (1974) argued that, in gen-
,agers
·.i; eral, a point-to-point correspondence between test content.and the contents of
ng test
'
criterion measures would lead to the highest levels of validity. When they exam-
e~r~ters
ined work sample.test validities, they did find·strong evidence of criterion-re- ·
ng for government,'.'. ··
lated validity. However, the level of validity depended on-whether the work 0 ·

sample involved motor or verbal performance. It also depended on what crite..; ·

for administrators rion was being predicted. The reviewers found that two types of criteria had
been e'131uated in ~ese studies: job proficiency and training success; When the
10, 1974, Prnonne/ criterion was job proficiency, ·a majority of the studies of motor work sample ·
tests showed significant validity coefficien~s of .30 or better. In fact, almost half
of the motor test validities reached at least .50. Verbal work sample measures .
>ewriter or com- ·. f were less effective than the motor tests in predicting job proficiency;. although
as might occur "i more than half of ~e studies did show significant validiti~of .30 or higher. The-
m~le test. Table I .verbal tests actually were be~er at'predicting training success; The reviewers·· ··
ti · II found that a· majority of the studies_ using :verb.al tests reported -validities of at · ·
t
11. .

1
1 least .30 and _that almost 40 percent had validities of .50 or higher. Motor work ·
.joh, using a de- '
1'
:l to obtain a cer-
les the_most im-
the job. To the ·
~onsidered to be
ss of developing
ere used to hire
tprehensive job ·
job analysis was
om the previous .
:cription of a job
LSks and KSAs.)
d constitute the
mined that de-
standards were
I
a
sample tests·also were able to predict training criterion·measure, although not ·
quite _as well as the verbal tests. Overall; these values indicate that work sample ·
performance tests, whether they are motor or verbal tests, show high criterion- ·
related validity (Asher&: Sciarrlno, 1974). · ·.
Work sample tests have other advantages. They show little indication of ad- .
verse impact, and applicants react po~!tively toward them (Robertson &
Kandola, 1982; Schmidt, Greenthal, Hunter, Bemer, & Seaton, 1977). Cascio
and Phillips (1979) reported that the.work sample tests they developed showed
no signs of adverse impact against minorities or women. In addition, the tests
were CO$t effective and resulted .in decreased turnover rates. Turnover droppf ·
. from average of 40 perc~nt to less than 3 percent during the period follow-·
ing the introduction of work sample testing. The researchers surmised that this
may have resulted because perfonnance of the work sample actually behaved as
a realistic job preview. That is, the work sample test allowed aP.plicants to expe-
rience wh,at the job was like before they accepted it.
 ti

3u cffAl'l'ER 12
cal p
t
:!
·'

:J
·I
·I

Joh simulations-especially work sample tests-are.versatile assessment · ·-1!.

methods. They may play either of two role~ in assessment. Historically, job·
simulations have been used in selection, as predictors of job perfonnance.
However, when other selection instruments are being validated, job simu-
.,

>l :J
mea:
lations are sometimes used as criterion measures. Their versatility demon~ co~
strates the close relationship between the concepts of predictor an~ cn1:e- thee
. rion. To determine whether a measure is a predictor or a criterion, we
. must look at how it is being used. Some measures, such as appllcations ~d ityo
biodata instruments, are used only as predictor measures in validation
. studies.·However, .work samples and certain.other measures, such as situ-., 4
ational interviews, can be used as either predictor: or criterion- measure~ m; AssES~
validation studies ..We- are most likely to see- that measures. involving rat,
ings play this role; Obviously, ratings are used as criterion me~ures; super- • Anasse1
- . visors routinely rate the job ·performance of employees and; fhµs·, provi~,· simulatl
-t he criterion measure for criterion-relatedvalidation..However~ ratings. in a sim1
. also are collected with certain other measures, such situational inter- serve 4
views and job simulation evaluations: Thes·e ratings ~an be used either to Because
,predict perfoimailce or as a-criterion me~ure of performance; ,, . .,, , _, ·andbe4
··A requirement of measures that .are..used as either.prec;lictors or crite-: categori:
ria concerns the extent to which the measure replicates .ilie jQb. A work . ·:·:.•·, As•
sample or other job simulation does not ha"e w.'be exact replica of a jqb ~uring '
·. -in-order.t o be a-.~ d·predicto.r or sel~tiQiimeasure.. In previous ~aptets"
·. .Idiscussed selectjon methods that ar~ ,quite.abstnict,. such as, cogajtiv~ S~teg.
viceag~
ability tests, andyet that; have been shownJo.haye:,high -validity. He>wev~i;: str~~ sjj
• when used ~ -a criterion measure, a job simulation does need to ass~ss _ all· · stress. ,i
· critical.taslcs. -Felker and Rose (1897) argue~ th~t.works~ples are _eff~-:-
search -'
•. '. , tive~ not only.as p~ctor measures b11t .~s.Q...as ctjteriop me~ure~. Th~-rel
cisely because they m_e et this requiremeQ.t_. Work sample performance ·c linical
tests that are well-constructed µsing a thorougli job analysis are conunonly .the~ ~
recognized as ·having the highest fidelity to. the job ~d as being tlie most develop
valid indicators of actual job. perf~rmance~ .. . . . . ·. ! .
&Ma~
Predictor and criterion· measures do not produce identical results, theirpe
There are at least two important aspects of the measurement process that. ·1 ·ties ~
· contribute.to the differences. First, ·interviewers and obseivers of_appli,- · l mentso
cants on job simulations have a different f~us than supervisors who ob- i behavi~
seive and rate the perfQrmance of their employees. Obseivers of appli-
cants focus on evaluating potential for job prpficiency rather than actual
t
·.I
i
part or
presen1
proficiency. Second, an applicant behaves differently while being observed
overco?
for selection purposes, than an employee who is performing the job
signme
(Sackett, Zedeck, & Fogli, 1988). An applicant probably exhibits his or her
..helpet
TMXimum performance, whereas an employee exhibits a lower, more typi-