This article was downloaded by: [b-on: Biblioteca do conhecimento online UC]

On: 12 February 2013, At: 16:01

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Applied Neuropsychology: Adult

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/hapn20

WAIS-IV Digit Span Variables: Are They Valuable for Use

in Predicting TOMM and MSVT Failure?
a b b c d
Kriscinda A. Whitney , Polly H. Shepard & Jeremy J. Davis
a
Psychiatry Department, Richard L. Roudebush Veterans Administration Medical Center,
Indianapolis, Indiana
b
Department of Psychiatry, Indiana University School of Medicine, Indianapolis, Indiana
c
Professional Psychological Services, Indianapolis, Indiana
d
Division of Physical Medicine and Rehabilitation, University of Utah School of Medicine, Salt
Lake City, Utah
Version of record first published: 08 Nov 2012.

To cite this article: Kriscinda A. Whitney , Polly H. Shepard & Jeremy J. Davis (2013): WAIS-IV Digit Span Variables: Are They
Valuable for Use in Predicting TOMM and MSVT Failure?, Applied Neuropsychology: Adult, 20:2, 83-94

To link to this article: http://dx.doi.org/10.1080/09084282.2012.670167

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to
anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should
be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims,
proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in
connection with or arising out of the use of this material.
 APPLIED NEUROPSYCHOLOGY: ADULT, 20: 83–94, 2013
Copyright # Taylor & Francis Group, LLC
ISSN: 0908-4282 print=1532-4826 online
DOI: 10.1080/09084282.2012.670167

WAIS-IV Digit Span Variables: Are They Valuable for

Use in Predicting TOMM and MSVT Failure?
Kriscinda A. Whitney
Psychiatry Department, Richard L. Roudebush Veterans Administration Medical Center and
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013

Department of Psychiatry, Indiana University School of Medicine, Indianapolis, Indiana

Polly H. Shepard
Department of Psychiatry, Indiana University School of Medicine and Professional
Psychological Services, Indianapolis, Indiana

Jeremy J. Davis
Division of Physical Medicine and Rehabilitation, University of Utah School of Medicine,
Salt Lake City, Utah

The Digit Span (DS) task in the Wechsler Adult Intelligence Scale-Fourth Edition differs
substantially from earlier versions of the measure, with one of the major changes being the
addition of a sequencing component. In the present investigation, the usefulness of the new
sequencing task and other DS variables (i.e., DS Age-Scaled Score, DS Forward Total, DS
Backward Total, and Reliable DS) was investigated with regard to the ability of these
variables to predict negative response bias. Negative response bias was first defined and
examined using below-cutoff performance on the Test of Memory Malingering (TOMM)
(N ¼ 99). Then, for comparison purposes, negative response bias was examined using
below-cutoff performance on the Medical Symptom Validity Test (MSVT; N ¼ 95). Study
participants included primarily middle-aged outpatients at a Veterans Affairs medical
center. Findings from this retrospective analysis showed that, regardless of whether the
TOMM or the MSVT was used as the negative response bias criterion, of all the DS
variables examined, DS Sequencing Total showed the best classification accuracy. Yet,
due to its relatively low positive and negative predictive power, DS Sequencing Total is
not recommended for use in isolation to identify negative response bias.

Key words: military, motivation, neuropsychology, tests

INTRODUCTION Wechsler, 2008) differs in several ways from the DS task

The Digit Span (DS) task appearing in the Wechsler
Adult Intelligence Scale-Fourth Edition (WAIS-IV;
The contents of this article do not necessarily represent the views of
the Department of Veterans affairs or the U.S. government.
Address correspondence to Kriscinda A. Whitney, Psychiatry
Department, Richard L. Roudebush Veterans Administration Medical
Center, 1481 W. 10th Street, Indianapolis, IN 46202. E-mail: kriscinda.
whitney@va.gov
that appeared both in the Wechsler Adult Intelligence
Scale-Third Edition (WAIS-III; Wechsler, 1997a) and,
identically, in the Wechsler Memory Scale-Third Edition
(WMS-III; Wechsler, 1997b). With regard to minor
differences, in many cases, the trials of the WAIS-IV
DS task contain different digits than those contained in
the WAIS-III and WMS-III version of the trials. In
addition, the WAIS-IV introduced a sample trial to the
DS Backward task, whereas the WAIS-III and WMS-III
 84 WHITNEY, SHEPARD, & DAVIS
did not include this feature. Certainly, however, the most
striking difference between the old and new versions of
the DS tasks is the addition of a sequencing task to
WAIS-IV DS. The sequencing task appears much like
the forward and backwards tasks, but in it, instead of
being asked to repeat digits forward or backward, the
examinee is asked to repeat the digits back in order, start-
ing with the lowest number. Given the modifications to
DS, the possibility arises that the new DS task may be
measuring something different than in previous versions,
which is concerning given the substantial research
conducted using earlier versions of DS.
Beyond changes in test format, differences in the stan-
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013
dardization sample between the WAIS-III and the
WAIS-IV also raise concern for the comparability of
the old and new DS Age-Scaled Scores (DS Age SS).
Specifically, it has been reported that, ‘‘unlike the
WAIS-III, the WAIS-IV standardization sample care-
fully excluded older normative participants to insure that
cases with mild dementia were not accidentally included’’
(Loring & Bauer, 2010, p. 687). Also according to Loring
and Bauer, the WAIS-IV explicitly excluded subjects
from the normative data who demonstrated poor effort
or inadequate task engagement. Given the aforemen-
tioned discrepancies between the new and old versions
of the DS task, concern about the applicability of past
study findings regarding the use of DS variables in
detecting inadequate effort on testing appears justified.
Although a variety of DS variables have been
researched as potential indicators of negative response
bias on testing, Reliable DS and DS Age SS appear to
have the most research support (see Whitney, Davis,
Shepard, Bertram, & Adams, 2009, for a review).
Reliable DS was introduced by Greiffenstein, Baker,
and Gola (1994) and is calculated by summing the long-
est string of digits repeated without error over two trials
under both forward and backward conditions of the DS
task. A few recently published studies compared the
relative usefulness of Reliable DS and DS Age SS in
predicting negative response bias (see Table 1). Results
of these studies have generally shown that, when using
the traditional cutoffs on these variables, Reliable DS
appears to be more sensitive (sensitivities ranging from
.54 to .65) to negative response bias than does DS Age
SS (sensitivities ranging from .35 to .42); however, there
is a negative trade-off in terms of specificity. That is,
Reliable DS tends to produce more false-positive errors
(specificity ranging from .77 to .89) than does DS Age
SS (specificities ranging from .92 to .96; Babikian,
Boone, Lu, & Arnold, 2006; Greve et al., 2007; Heinley,
Greve, Bianchini, Love, & Brennan, 2005; Whitney,
Davis et al., 2009). As previously noted by Whitney,
Davis and colleagues (2009), the tendency of Reliable DS
to make more false-positive errors is a concern because
mislabeling patients=forensic clients as displaying
negative response bias can be associated with negative
emotional, and sometimes financial, consequences.
Given the importance of accuracy in assessing negative
response on neuropsychological testing, the aim of the
present study was to examine the clinical utility of
WAIS-IV DS variables (i.e., Reliable DS, DS Age SS,
DS Forward Total, DS Backward Total, and DS Sequen-
cing) in predicting negative response bias. The latter was
accomplished first by comparing performance on the DS
variables to performance on one of the most commonly
used symptom validity tests, the Test of Memory Malin-
gering (TOMM; Tombaugh, 1996). Although the TOMM
is well validated in numerous populations, including per-
sons with fibromyalgia and chronic pain and=or
depression, neurological patients, community-dwelling
geriatric individuals, college student normal controls and
simulators, persons simulating traumatic brain injury,
actual brain injury litigants, and persons with depression
(Ashendorf, Constantinou, & McCaffrey, 2004; Iverson,
Le Page, Koehler, Shojania, & Badii, 2007; Rees, Tom-
baugh, Gansler, & Moczynski, 1998; Tombaugh, 1997),
research findings suggest that its rate of false-positive diag-
noses of negative response bias is unacceptably high in per-
sons with dementia, falling at
75% in a study conducted
by Teichner and Wagner (2004).
Research also suggests that the sensitivity of the
TOMM is low. In a recent study including 345 indivi-
duals referred for disability evaluations primarily related
to anxiety, depression, and chronic pain, more than
twice as many patients failed a separate free-standing
symptom validity test, namely the Nonverbal Medical
Symptom Validity Test (NV-MSVT; Green, 2008), than
failed the TOMM (Armistead-Jehle & Gervais, 2011).
According to the study authors, among those who
passed the TOMM but failed the NV-MSVT, 96%
demonstrated profiles marked by inconsistency and were
thus strongly suggestive of suboptimal effort. A separate
study examining the TOMM and NV-MSVT showed
nearly identical results (Green, 2011). Specifically,
among 244 disability claimants, twice as many patients
failed the NV-MSVT than failed the TOMM. In 100%
of the cases where the TOMM was passed but the
NV-MSVT was failed, on the NV-MSVT, patients
displayed a paradoxical superiority on one of the hard
subtests compared with the mean of the easier subtests,
again suggesting suboptimal effort. Due to the concerns
regarding sensitivity and specificity of the TOMM, and
because there is no gold standard for symptom validity
assessment, with researchers noting concern about the
lack of concordance among existing measures (Axelrod
& Schutte, 2011), the decision was made to also compare
performances on the DS variables to a separate symp-
tom validity test, the Medical Symptom Validity Test
(MSVT; Green, 2004). The MSVT is an easy memory
test, which research has shown can easily be passed by
 WAIS-IV DIGIT SPAN VARIABLES 85

TABLE 1
Summary of Published Sensitivity and Specificity Values for Studies Comparing DS Age SS and Reliable DS

Age of
Description of Description of Groups
Sensitivity Specificity Adequate Questionable (Mean
z y
Na Cutoff (%) (%) PPV NPV Effort Group Effort Group Years) Authors

DS Age SS
88=66 5 42 93 .81 .70 Neuropsychology clinic Probable: nearly all litigation= 42–75 Babikian
referrals, normal compensation seeking for et al.
controls cognitive symptoms (2006)
146=71a 5 36 93 .78 .68 Traumatic brain injury Probable=definite: traumatic 40 Heinly et al.
brain injury (2005)
38=46 5 46 92 .80 .71 Toxic exposure Probable=definite: toxic 40–43 Greve et al.
exposure (2007)
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013

26=20 6 35 96 .86 .68 27% traumatic brain injury, Probable: 30% traumatic brain 51–47 Whitney,
73% younger than age injury, 70% younger than Davis
65 with no obvious age 65 with no obvious et al.
neurological problems neurological problems (2009)
Reliable DS
88=66 7 62 77 .65 .74 Neuropsychology clinic Probable: nearly all litigation= 42–75 Babikian
referrals, normal compensation seeking for et al.
controls cognitive symptoms (2006)
146=71y 7 65 87 .78 .78 Traumatic brain injury Probable=definite: traumatic 40 Heinly et al.
brain injury (2005)
38=46 7 54 89 .77 .74 Toxic exposure Probable=definite: toxic 40–43 Greve et al.
exposure (2007)
26=20 7 40 85 .65 .67 27% traumatic brain injury, Probable: 30% traumatic brain 51–47 Whitney,
73% younger than age injury, 70% younger than age Davis
65 with no obvious 65 with no obvious et al.
neurological problems neurological problems (2009)

PPV ¼ positive predictive value; NPV ¼ negative predictive value.

a
Reported Ns are for adequate=questionable effort participants.
z
Values calculated using a base rate of .41.
y
For clarity, only the traumatic brain injury statistics are presented from this study.

at least 95% children with moderate-to-severe brain
injury=dysfunction (e.g., traumatic brain injury, stroke)
and=or developmental disabilities (N ¼ 38; Carone,
2008). In fact, as stated by Howe (2007), ‘‘The only
clinical group shown to not consistently score in the
normal range on MSVT validity measures due to neuro-
pathology instead of diminished symptom validity is
individuals diagnosed with dementia who have accom-
panying clinical correlates of significant disability’’
(p. 755). However, the latter is not necessarily a down-
fall of the MSVT, because the MSVT actually considers
the role of severe cognitive impairment by allowing the
examiner to analyze each patient’s profile of scores to
determine whether that patient’s profile is more similar
to individuals with genuine memory impairment or to
that of individuals simulating cognitive impairment. A
recent study showed that 85% of patients with dementia
who failed the MSVT met criteria for genuine memory
impairment, as defined by a set of rules called the
Dementia Profile (DP; Howe & Loring, 2008). In the
present study, the data from patients whose profiles of
scores met criteria for the DP, including having had
clinical correlates of disability as required in the MSVT
DP rules, were excluded from all analyses involving
the MSVT. The aim of excluding data from these
individuals was to lessen the confounding effect of cogni-
tive impairment on symptom validity test performance.
However, because only a modest number of patients in
this primarily middle-aged sample was anticipated to fail
the MSVT based on the DP, similar hypotheses were
made with regard to the relationships between the DS
variables and the TOMM and MSVT, respectively.
Specifically, it was hypothesized that among all the DS
variables, Reliable DS and DS Age SS would be the best
predictors of both TOMM and MSVT failure. Based on
previous research findings, it was predicted that Reliable
DS would likely demonstrate higher sensitivity but lower
specificity compared with DS Age SS in predicting
TOMM and MSVT failure, respectively.
METHOD
Participants
Data were collected from the files of 99 outpatients
who were referred to the lead author (K.W.) for
 86 WHITNEY, SHEPARD, & DAVIS
neuropsychological testing during approximately a
9-month period at a Department of Veterans Affairs
Medical Center. Although the author and her colleagues
have published the results of other investigations using
the TOMM and=or the MSVT, the patients included in
the present study do not overlap with any of those refer-
enced in previously published studies with the exception
of the in press study authored by Davis, Wall, and
Whitney (in press). Referral sources included the Psy-
chiatry Ambulatory Care Clinic, primary medical clinics,
and the neurology clinic. No patients were diagnosed with
mental retardation. Consecutive referrals were reviewed
for cases that were administered both the TOMM and
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013
DS of the WAIS-IV as part of the clinical neuropsycholo-
gical evaluation. All but one patient included in the study
were also administered the MSVT.
As a rule, all patients referred to the lead author were
administered both the TOMM and the MSVT. The only
exceptions were those patients who were referred for a
retest. If a patient was referred for a retest, they were
given an alternative symptom validity test; thus, their
data are not included in this study. It should be noted
that in the present study, all patients were primarily
referred for neuropsychological evaluation to assess
the potential presence of cognitive dysfunction, not pri-
marily to assess for the presence of psychiatric disorder.
At the time of this study, participants’ medical records
were retrospectively reviewed and they were categorized
into groups according to whether or not they passed
(n ¼ 74, 75% of total sample) or failed (n ¼ 25, 25% of
total sample) the TOMM.
They were also categorized into groups according to
whether or not they passed or failed the MSVT. As noted
above, data from patients who demonstrated a DP on
the MSVT were excluded from analyses involving the
MSVT. Specifically, 3 patients who failed the MSVT
met criteria for the DP and demonstrated advanced cog-
nitive difficulties with functional evidence of decline (i.e.,
1 experienced stroke with hemiplegia, 1 resided in a nurs-
ing home, and 1 required 24-hour supervision to ensure
his safety). Data from their files were excluded from all
statistical analyses of the relationship between DS vari-
ables and the MSVT. Incidentally, 2 of these patients
passed the TOMM, while 1 narrowly failed the TOMM
with a score 3 points below expectation on the retention
trial. In addition, 5 more patients who failed the MSVT
met the technical criteria for the DP but did not evidence
clinical correlates of disability as required in the DP
rules. Thus, they were retained in the Fail MSVT group
because there was no clinical evidence that they had
functional impairments consistent with dementia.
Incidentally, all of these patients failed the TOMM.
Given aforementioned procedures, the final MSVT
sample included 63 patients (66% of the total sample)
who passed the MSVT (Pass MSVT group) and 32
patients (34% of the total sample) who failed the MSVT
and did not meet full criteria for the DP.
Patient demographic and diagnostic information is
listed in Table 2. In all four symptom validity groups
(i.e., Pass TOMM, Fail TOMM, Pass MSVT, and Fail
MSVT), 32% to 41% of patients were referred with
symptoms of traumatic brain injury; judging by their
self-reports, most of these patients reported brain injuries
would have fallen within the ‘‘mild traumatic brain
injury’’ or lesser ‘‘possible traumatic brain injury’’ range
(n ¼ 26=38, 68% overall; Malec, Brown, Leibson, Flaada,
& Mandrekar, 2007). Approximately another 30% to 40%
of all four symptom validity groups (i.e., Pass TOMM,
Fail TOMM, Pass MSVT, and Fail MSVT) consisted of
individuals who were referred with memory complaints,
and even though they may have had a neurological
and=or psychiatric diagnosis associated with potential
brain dysfunction, they had no signs of neurologic dys-
function that were obvious to the observer. Approxi-
mately 70% of these individuals with no obvious major
neurologic dysfunction in both the Fail TOMM and Fail
MSVT groups had both a neurologic and psychiatric
diagnosis, 20% had a psychiatric diagnosis only, and 0%
to 10% of patients had a neurological diagnosis only. In
both the Pass TOMM and Pass MSVT groups, 42% to
45% of these individuals with no obvious major neurolo-
gic dysfunction had both a neurologic and psychiatric
diagnosis, 25% to 27% had a psychiatric diagnosis only,
and 25% to 27% had neurologic diagnoses only.
When pass and fail symptom validity groups were
considered as a whole, among individuals with no obvi-
ous major neurologic dysfunction (n ¼ 38 TOMM, 37
MSVT) who had only a psychiatric diagnosis (n ¼ 9=38
TOMM, 9=37 MSVT), the most common problem was
anxiety=depression (n ¼ 4=9, 44.4%). In patients with
no obvious major neurologic dysfunction and only a
neurologic diagnosis (n ¼ 8=38 TOMM, 9=37 MSVT),
the most common problem was hypertension (n ¼ 3=8,
38%). Finally, in patients with no major neurologic dys-
function who had both psychiatric and neurologic
diagnoses (n ¼ 20 TOMM, 19 MSVT), the most common
comorbid diagnosis was depression=anxiety and hyper-
tension (n ¼ 5=20 TOMM, 5=19 MSVT; 25% to 26%).
Measures and Procedures
Patients were administered all study measures as part of
a larger neuropsychological test battery given as part of
routine clinical care. The TOMM (Tombaugh, 1996)
was administered to all patients. It is a recognition
memory task in which 50 line drawings of common
objects are presented during two learning trials that
are each followed by forced-choice recognition trials.
An optional forced-choice recognition trial can be admi-
nistered following a 15-minute delay. In the present
 WAIS-IV DIGIT SPAN VARIABLES 87

TABLE 2
Patient Demographic and Diagnostic Information

Pass TOMM (N ¼ 74) Fail TOMM (N ¼ 25) Pass MSVT (N ¼ 63) Fail MSVT (N ¼ 32)
Demographic and M (SD) n (%) M (SD) n (%) M (SD) n (%) M (SD) n (%)
Diagnostic Information of Group of Group of Group of Group

Age 49.4 (13.2) 51.6 (17.3) 48.8 (13.3) 50.6 (15.7)

Range ¼ 22–74 Range ¼ 21–87 Range ¼ 22–72 Range ¼ 21–87
Highest Year of Education 12.7 (0.3) 11.5 (2.0) 13.0 (2.7) 11.6 (2.0)
Range ¼ 7–20 Range ¼ 7–16 Range ¼ 7–20 Range ¼ 7–16
Gender
Female 5 (6.8%) 3 (12.0%) 3 (4.8%) 5 (15.6%)
Male 69 (93.2%) 22 (88.0%) 63 (95.2%) 27 (84.4%)
Ethnicity
Caucasian 64 (86.5%) 17 (68.0%) 54 (85.7%) 24 (75.0%)
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013

African American 10 (13.5%) 6 (24.0%) 9 (14.3%) 7 (21.9%)

Other 0 (0.0%) 2 (8.0%) 0 (0.0%) 1 (3.1%)
Traumatic Brain Injury 30 (40.5%) 8 (32.0%) 25 (39.7%) 13 (40.6%)
Mild Traumatic Brain Injury 19 (25.7%) 7 (28.0%) 17 (27.0%) 9 (28.1%)
Moderate-to-Severe Traumatic Brain Injury 11 (14.9%) 1 (4.0%) 8 (12.7%) 4 (12.5%)
Major Neurological Diagnosis 16 (21.6%) 7 (28.0%) 12 (19.0%) 8 (25.0%)
Dementia=Capacity Issues 4 (5.4%) 3 (12.0%) 3 (4.8%) 2 (6.3%)
Stroke 4 (5.4%) 3 (12.0%) 3 (4.8%) 3 (9.4%)
Electrical Injury 1 (1.4%) 1 (4.0%) 1 (1.6%) 1 (3.1%)
Encephalopathy 2 (2.7%) 0 (0.0%) 1 (1.6%) 1 (3.1%)
Removal of Brain Abscess 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Normal Pressure Hydrocephalus 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Arnold-Chiari Malformation & Repair 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Aortic Dissection with Possible Hypoxia 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Primary Cerebellar Degeneration 1 (1.4%) 0 (0.0%) 0 (0.0%) 1 (0.0%)
No Obvious Major Neurologic Dysfunction, 28 (37.8%) 10 (40.0%) 26 (41.3%) 11 (34.4%)
Younger than 69 Years Old
Psychiatric Diagnosis Only 7 (9.5%) 2 (8.0%) 7 (11.1%) 2 (6.3%)
Anxiety=Depression 3 (4.1%) 1 (4.0%) 3 (4.8%) 1 (3.1%)
Depression and PTSD 0 (0.0%) 1 (4.0%) 0 (0.0%) 1 (3.1%)
Depression, PTSD, ADHD 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Depression & Psychosis 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
PTSD, Bipolar Disorder, ADHD 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Learning Disorder 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Neurologic Diagnosis Only 7 (9.5%) 1 (4.0%) 7 (11.1%) 1 (3.1%)
Hypertension 3 (4.1%) 0 (0.0%) 3 (4.8%) 0 (0.0%)
CAD 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
TIA=Minor Stroke=Small Vessel Ischemic 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Disease
Coronary Atherosclerosis 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Anemia 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Type 2 Diabetes 0 (0.0%) 1 (4.0%) 0 (0.0%) 1 (3.1%)
Psychiatric & Neurologic Diagnoses 13 (17.6%) 7 (28.0%) 11 (17.5%) 8 (25.0%)
Schizoaffective Disorder & Sleep Apnea 1 (1.4%) 2 (8.0%) 1 (1.6%) 1 (3.1%)
Depression=Anxiety & Hypertension 3 (4.1%) 2 (8.0%) 2 (3.2%) 3 (9.4%)
Depression and CAD 1 (1.4%) 1 (4.0%) 1 (1.6%) 1 (3.1%)
Depression, PTSD, & Hypertension 2 (2.7%) 1 (4.0%) 2 (3.2%) 1 (3.1%)
Depression, Conversion Disorder & CAD 0 (0.0%) 1 (4.0%) 0 (0.0%) 1 (3.1%)
Depression, PTSD, & Sleep Apnea 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Depression=Anxiety & Hepatitis C 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Depression=Anxiety & Remote Stroke 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Depression=Anxiety & Anemia 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Depression=Anxiety & Sleep-Related 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)
Hypoxemia
Bereavement and Hepatitis C 1 (1.4%) 0 (0.0%) 0 (0.0%) 1 (3.1%)
No Diagnosis 1 (1.4%) 0 (0.0%) 1 (1.6%) 0 (0.0%)

t(97) ¼ 2.07, p ¼ .04.

t(93) ¼ 2.56, p ¼ .01.
Traumatic brain injury categorization was based on patient self-report and is of questionable validity.
ADHD ¼ attention-deficit hyperactivity disorder; CAD ¼ coronary artery disease; PTSD ¼ posttraumatic stress disorder; TIA ¼ transient
ischemic attack.
 88 WHITNEY, SHEPARD, & DAVIS

study, all three trials were administered, and patients Statistical Analyses
were considered to have failed the TOMM if they
Except where otherwise indicated, statistical analyses
performed below cutoffs specified in the manual on
were calculated using the Statistical Package for the
Trial 1, Trial 2, or the retention trial, the latter of which
Social Sciences, Version 10.1. Initial analyses consisted
was always administered.
of conducting independent t-tests to compare group
The MSVT (Green, 2004) is a computer-administered
differences in age, education, DS Forward Total, DS
measure of effort and memory that requires 5 minutes of
Backward Total, DS Sequencing Total, DS Age SS,
patient on-task time and a 10-minute delay. Specific
and Reliable DS among persons either failing or passing
administration procedures are detailed in the test
the TOMM. These analyses were repeated for those
manual. To briefly summarize, after being presented
either failing or passing the MSVT. Alpha was set at
twice with a series of 10 word pairs, the test taker is
.05 for this and all other analyses. Cohen’s d effect sizes,
asked to complete four trials, which result in five test
along with 95% confidence intervals (CIs) for these
scores. These trials include an immediate forced-choice
group differences were calculated using a computer pro-
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013

recognition trial (IR), a delayed forced-choice recog-

gram (Devilly, 2004). The aforementioned analyses all
nition trial (DR), a paired-associate recall trial (PA),
focused on a dichotomous distinction between groups
and a free recall trial (FR). The aforementioned trials
(i.e., Pass TOMM or Fail TOMM and Pass MSVT or
represent four test scores. The fifth test score is com-
Fail MSVT). Given the additional variability inherent
puted by examining the consistency in the test takers’
in continuous scores, an alternative view of the relation-
responses to two of the trials (CNS). Some of these five
ship between the DS variables and the TOMM and
scores are considered ‘‘easy’’ to do well on (i.e., IR, DR,
MSVT, respectively, was created by computing two-
and CNS), whereas others are considered ‘‘hard’’ to do
tailed Spearman’s rho correlations between DS variables
well on (i.e., PA and FR). Participants were considered
and all trials on the TOMM and MSVT, respectively.
to have failed the MSVT if their score on IR, DR, or
Spearman’s rho correlations were used in place of
CNS was below the specified cutoffs. Participants were
Pearson correlations because neither TOMM scores
considered to have met the criteria for the DP if: (i)
nor MSVT scores were normally distributed. Two-tailed
either IR, DR, or CNS were below the specified cutoffs;
Pearson correlations were computed to examine the
(ii) there were no scores below chance; (iii) there was a
relationship between age, education, and the various
specified point difference between the mean of the easy
DS variables.
items (IR þDR þCNS=3) and the mean of the hard
Receiver-operating characteristic (ROC) curve
items (–PA þFR=2); (iv) IR and DR were greater than
analyses were used to evaluate the classification accu-
FR; and (v) PA was greater than FR. The DP rules also
racy of DS variables in identifying TOMM and MSVT
require that clinical correlates of significant disability
failure. In addition to examining the sensitivity and
exist. The specific MSVT cutoffs and specified point dif-
specificity of different cut scores on DS variables, posi-
ference mentioned earlier are detailed in the test manual
tive and negative predictive values (PPVs, NPVs) were
(Green, 2004).
calculated for a range of base rates using the formulas
The DS subtest of the WAIS-IV (Wechsler, 2008) was
presented in O’Bryant and Lucas (2006).
administered to all study participants as part of their
routine clinical care. Administration followed instruc-
tions specified in the test manual. From each parti-
RESULTS
cipants’ performance on DS, five scores were derived:
DS Forward Total, DS Backward Total, DS Sequencing
Group Differences and Pearson Correlations
Total, DS Age SS, and Reliable DS. Procedures for
obtaining the first four of the latter scores are explained In terms of age, results of the t-tests showed that there were
in the test manual. Notably, unlike in previous versions, no significant differences between the group who passed
scores from the DS Sequencing task are now summed the TOMM (N ¼ 74) and the group who failed the TOMM
with scores from DS Forward and DS Backward to (N ¼ 25) nor were there differences between the group who
determine the DS Age SS. Reliable DS represents the passed the MSVT (N ¼ 63) and the group who failed the
sum of the longest string of digits repeated without error MSVT (N ¼ 32; see Table 2). Results of separate t-tests
over two trials under both forward and backward showed that education level was significantly higher
conditions (Greiffenstein et al., 1994). For example, a among persons passing versus failing the TOMM
participant who passed both trials of three digits (Table 2). Education level was also slightly higher among
forward, passed both trials of two digits backward, persons passing versus failing the MSVT (Table 2).
but failed one trial each of four digits forward and In terms of performances on the various DS variables,
three digits backward would receive an Reliable DS the group who failed the TOMM performed significantly
score of 5. worse than did the group who passed the TOMM on DS
 WAIS-IV DIGIT SPAN VARIABLES 89

TABLE 3
Group Differences in WAIS-IV Digit Span Scores Among Groups Passing and Failing the TOMM

Pass TOMM (n ¼ 74) Fail TOMM (n ¼ 25) t(97) Cohen’s d

M SD M SD t p d 95% CI

DS Forward Total 9.34 2.54 8.24 2.31 1.91 .06 .45 –0.00–0.91
DS Backward Total 7.77 2.10 7.20 1.87 1.21 .23 .29 –0.17–0.74
DS Sequencing Total 7.58 2.29 6.40 2.58 2.16 .03 .48 0.03–0.94
DS Age SS 8.74 2.87 7.44 2.79 1.98 .05 .46 0.00–0.92
Reliable DS 7.58 2.29 6.40 2.58 1.69 .10 .48 0.03–0.94

TOMM ¼ Test of Memory Malingering; DS ¼ Digit Span; DS Age SS ¼ DS Age-Scaled Score.

Sequencing Total (Table 3). There were also trends for Level of Education was positively and significantly
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013

the group who failed the TOMM to perform significantly
lower compared with the group who passed the TOMM
on DS Age SS and DS Forward Total (p < .10). The
effect sizes for these differences were moderate, ranging
from .45 to .48 (Lipsey, 1990). Similarly, the group
who failed the MSVT performed significantly worse than
did the group who passed the MSVT on DS Sequencing
Total, and again, there were trends for the group who
failed the MSVT to perform significantly lower on DS
Age SS and DS Forward Total (p < .10; Table 4). The
effect sizes for these group differences are considered
moderate, ranging from .41 to .53 (Lipsey).
Results of Spearman’s rho correlations showed that,
with the exception of DS Backward Total, all DS vari-
ables were positively and significantly correlated with
at least one trial of the TOMM. DS Sequencing Total
showed the highest and most consistent correlations
(Table 5). Similarly, with the exception of DS Backward
Total, patient scores on the MSVT were positively cor-
related with all DS variables. Of all the DS variables,
DS Sequencing Total was most highly correlated with
MSVT Easy subtests, while DS Forward Total was most
highly correlated with MSVT Hard subtests (Table 5).
With regard to the relationship between age and DS
variables, age was negatively and significantly correlated
with DS Sequencing Total (r ¼  .35, p < .001) and
Reliable DS (r ¼  .24, p < .05) but not with DS
Forward Total (r ¼  .20, p ¼ .05), DS Backward Total
(r ¼  .17, p ¼ .10), or DS Age SS (r ¼  .11, p ¼ .29).
correlated with DS Sequencing Total (r ¼  .34,
p < .01) but not with DS Forward Total (r ¼ .11,
p ¼ .27), DS Backward Total (r ¼ .14, p ¼ .16), DS Age
SS (r ¼ .14, p ¼ .16), or Reliable DS (r ¼ .14, p ¼ .16).
ROC, Sensitivity, Specificity, and Predictive Power
Analyses
As shown by the ROC analysis, in predicting TOMM
performance, DS Sequencing Total, DS Age SS, and
DS Forward Total all demonstrated an area under the
curve of .62, which is acceptable (Table 6). Of the afore-
mentioned DS variables, DS Sequencing Total (3),
appeared to best maximize sensitivity (12%) while main-
taining a high level of specificity (95%). Even so, while
DS Sequencing Total (3) produced acceptable NPVs
across base rate conditions (.81 to .62), its PPVs (.38
to .62) were unimpressive.
As shown by separate ROC analysis, in predicting
MSVT performance, DS Sequencing Total produced a
higher area under the curve (.67, 95% CI ¼ 0.55–0.78)
compared with either DS Age SS or DS Forward Total.
Here, using a cutoff of  4 appeared to maximize sensi-
tivity (19%), while maintaining specificity (94%). Yet,
again, across base rates, DS Sequencing Total (4) pro-
duced acceptable NPVs (.82–.63) but unimpressive
PPVs (.44–.68). See Table 7 for failure rates on the
TOMM, MSVT, and DS variables among the various
study groups.

TABLE 4
Group Differences in WAIS-IV Digit Span Scores Among Groups Passing and Failing the MSVT

Pass MSVT (n ¼ 63) Fail MSVT (n ¼ 32) t(94) Cohen’s d

M SD M SD t p d 95% CI

DS Forward Total 9.56 2.39 8.50 2.45 2.02 .05 .44 0.01–0.87
DS Backward Total 7.81 1.97 7.56 2.12 0.56 .58 .12 –0.30–0.55
DS Sequencing Total 7.87 2.10 6.66 2.42 2.54 .02 .53 0.10–0.97
DS Age SS 8.98 2.62 7.84 2.94 1.92 .06 .41 –0.02–0.84
Reliable DS 8.89 1.97 8.31 2.21 1.30 .20 .28 –0.15–0.70

DS ¼ Digit Span; SS ¼ Scaled Score; TOMM ¼ Test of Memory Malingering; MSVT ¼ Medical Symptom Validity Test.
 90 WHITNEY, SHEPARD, & DAVIS

TABLE 5
Spearman’s Rho Correlations Between the Digit Span Scores and Independent Symptom Validity Test Scores (TOMM and MSVT)

DS Forward Total DS Backward Total DS Sequencing Total DS Age SS Reliable DS

TOMM Trial 1 (N ¼ 99) .20 .07 .22 .20 .10

TOMM Trial 2 (N ¼ 99) .15 .15 .29 .24 .11
TOMM Retention Trial (N ¼ 99) .28 .17 .34 .32 .23
MSVT Easy (N ¼ 95) .29 .14 .33 .26 .25
MSVT Hard (N ¼ 95) .38 .11 .34 .31 .25

TOMM ¼ Test of Memory Malingering; DS ¼ Digit Span; SS ¼ Scaled Score; MSVT ¼ Medical Symptom Validity Test.

Correlation is significant at the .01 level, two-tailed.

Correlation is significant at the .05 level, two-tailed.
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013

Of the individuals who failed DS Sequencing Total
(4), 46% (n ¼ 6=13) failed the TOMM and 60%
(n ¼ 6=10) failed the MSVT. There were 3 fewer patients
among the MSVT group than among the TOMM
group, as 1 individual was not administered the MSVT
and 2 patients who failed DS Sequencing Total (4)
were among the 3 patients who were excluded from
the study for satisfying full criteria for the DP on the
MSVT. Among the 10 patients who were included in
the study who failed DS Sequencing Total (4) and also
did not meet criteria for the DP on the MSVT, 40%
(n ¼ 4=10) failed both the TOMM and the MSVT,

TABLE 6
Sensitivity and Specificity Values for WAIS-IV Digit Span Variables When Using the TOMM and MSVT, Respectively

ROC Analysis Sensitivity Specificity PPV at Base Rate of NPV at Base Rate of
(95% CI) (%) (%)
20% 30% 40% 20% 30% 40%

Predicting TOMM Failure

DS Sequencing Total .62 (0.49–0.75)
2 8 97 .40 .53 .64 .81 .71 .61
3 12 95 .38 .51 .62 .81 .72 .62
4 24 90 .38 .51 .62 .83 .73 .64
5 36 86 .39 .52 .63 .84 .76 .67
DS Age SS .62 (0.49–0.75)
2 4 96 .20 .30 .40 .80 .70 .60
3 8 93 .22 .33 .43 .80 .70 .60
4 16 93 .36 .49 .60 .82 .72 .62
5 24 88 .33 .46 .57 .82 .73 .63
DS Forward Total .62 (0.50–0.75)
4 4 97 .25 .36 .47 .80 .70 .60
5 4 96 .20 .30 .40 .80 .70 .60
6 17 89 .28 .40 .51 .81 .71 .62
7 35 82 .33 .45 .56 .83 .75 .65
Predicting MSVT Failure
DS Sequencing Total .67 (0.55–0.78)
2 3 97 .20 .30 .40 .80 .70 .60
3 6 97 .33 .46 .57 .80 .71 .61
4 19 94 .44 .58 .68 .82 .73 .63
5 31 92 .49 .62 .72 .84 .76 .67
DS Age SS .62 (0.49–0.75)
2 0 97 .00 .00 .00 .80 .70 .60
3 6 95 .23 .34 .44 .80 .70 .60
4 13 95 .39 .53 .63 .81 .72 .62
5 25 92 .44 .57 .68 .83 .74 .65
DS Forward Total .62 (0.51–0.75)
4 3 97 .20 .30 .40 .80 .70 .60
5 6 97 .33 .46 .57 .80 .71 .61
6 25 94 .51 .64 .74 .83 .75 .65
7 38 86 .40 .54 .64 .85 .76 .68

Note. Prevalence of TOMM failure was .25, prevalence rate of MSVT failure was .34. PPV ¼ positive predictive value; NPV ¼ negative predictive
value; DS ¼ Digit Span; SS ¼ Scaled Score; TOMM ¼ Test of Memory Malingering; MSVT ¼ Medical Symptom Validity Test.
 WAIS-IV DIGIT SPAN VARIABLES 91

TABLE 7
Failure Rates on the TOMM, MSVT, and Digit Span Variables Among Study Groups

% Failing DS % Failing DS
% Failing % Failing Sequencing % Failing DS Forward
Diagnostic Info=Referral Reason TOMM MSVT Total (3) Age SS (4) Total (5)

Traumatic Brain Injury (n ¼ 38) 21.1% 34.2% 2.6% 2.6% 2.6%

Mild Traumatic Brain Injury (n ¼ 26) 26.9% 34.6% 3.8% 0.0% 0.0%
Moderate-to-Severe Traumatic Brain Injury (n ¼ 12) 8.3% 33.3% 0.0% 8.3% 8.3%
Major Neurologic Diagnosis (n ¼ 20 MSVT, n ¼ 23 Other Variables) 30.4% 34.8% 13.0% 13.0% 4.3%
No Obvious Major Neurologic Dysfunction, younger than 69 years 26.3% 28.9% 7.9% 13.2% 10.5%
old (n ¼ 37 MSVT, n ¼ 38 Other variables)
Psychiatric Diagnosis Only (n ¼ 9) 22.2% 22.2% 0.0% 0.0% 0.0%
Neurologic Diagnosis Only (n ¼ 8) 12.5% 12.5% 0.0% 12.5% 0.0%
Psychiatric & Neurologic Diagnosis 35.0% 40.0% 15.0% 20.0% 20.0%
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013

(n ¼ 19 MSVT, n ¼ 20 Other Variables)


Traumatic brain injury categorization was based on patient self-report and is of questionable validity.

20% (n ¼ 2=10) failed only the MSVT, and 40% passed
both the TOMM and the MSVT.
DISCUSSION
The primary aim of the present investigation was to
examine the clinical utility of the WAIS-IV DS variables
(i.e., DS Forward Total, DS Backward Total, DS
Sequencing Total, DS Age SS, and Reliable DS) in
predicting negative response bias on symptom validity
testing (i.e., the TOMM and the MSVT). Previous
research suggests that DS Age SS and, to a lesser extent,
Reliable DS are useful indicators of potential negative
response bias. However, modifications to the DS subtest
in the WAIS-IV, including the introduction of a new
sequencing task and associated changes to the deri-
vation of DS Age SS, necessitate the reexamination of
embedded effort measures based on the DS subtest.
Current study findings provide evidence supporting
the use of DS Sequencing Total in predicting symptom
validity test failure among a group of primarily middle-
aged outpatient military veterans. When DS variables
were compared between the groups who failed versus
passed the TOMM and groups who passed versus failed
the MSVT, only DS Sequencing Total scores were sig-
nificantly different between groups. There were trends
for significant differences in group means on DS Age
SS and DS Forward Total. There were no significant
differences for Reliable DS or DS Backward Total. As
expected, the groups who failed the TOMM and MSVT,
respectively, scored lower than the groups who passed
the TOMM and MSVT, respectively, on DS Sequencing
Total, DS Age SS, and DS Forward Total. Of DS vari-
ables that were either significantly different or showed
trends for significant differences between groups passing
and failing the TOMM and MSVT, DS Sequencing
Total demonstrated the largest effect size (d ¼ .48 and
.53, respectively). Of all the DS variables, DS Sequen-
cing Total showed the highest and most consistent
correlations with scores on the TOMM and the easy
subtests (primary effort subtests) of the MSVT.
Although the aforementioned findings may appear to
lend credibility to the use of DS Sequencing as an effort
measure, further analyses suggest that caution be
employed when using DS variables to predict negative
response bias. ROC analysis for the TOMM suggested
that a DS Sequencing Total cutoff of 3 minimized false
positives (specificity ¼.95) while retaining a reasonable
sensitivity of .12, both of which are fairly typical in
the practice of embedded symptom validity testing.
However, these data suggest that in terms of PPV, using
a DS Sequencing Total of 3 and a negative response
bias base rate of .30, a clinician would have only a
51% probability of being correct in suspecting a patient
of TOMM failure given a below-cutoff performance on
DS Sequencing Total. In terms of NPV, the same
clinician would have a better, 72%, probability of being
correct in not suspecting a patient of possible TOMM
failure given an above-cutoff performance on DS
Sequencing Total. The situation is similar when predict-
ing MSVT failure. ROC analysis for the MSVT sug-
gested that a DS Sequencing Total cutoff of 4
minimized false positives (specificity of .94) while
retaining a reasonable sensitivity of .19. Using this DS
Sequencing Total cutoff (4) and a negative response
bias base rate of .30 in predicting MSVT performance,
a clinician would have only a 58% probability of being
correct in suspecting a patient of MSVT failure and a
73% probability of being correct in not suspecting a
patient of possible MSVT failure.
Alarmingly, when used in settings where the base rate
of negative response bias is lower (i.e., 20%), the PPVs
for the aforementioned cutoffs on the TOMM and
MSVT decrease to <.50, indicating that a clinician would
be more likely to be incorrect than correct in suspecting
 92 WHITNEY, SHEPARD, & DAVIS
a patient of MSVT failure. This situation is not
improved by switching to an alternate cutoff for DS
Sequencing Total or by switching to an alternate DS
variable for use in predicting symptom validity test
failure. As shown in Table 6, using DS Age SS and
DS Forward Total (both of which showed trends for sig-
nificant differences between groups who passed vs. failed
the TOMM and MSVT, respectively) at a base rate of
20%, all cutoff for both of these measures and DS
Sequencing Total had PPVs at or below .51. PPVs at
or below this level indicate that the use of these variables
is either not considerably better or is, more often, worse
than randomly guessing the outcome of performance on
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013
the TOMM and MSVT.
The inaccuracy of using any DS variable in isolation
to predict negative response bias is, perhaps, not surpris-
ing if taken in the context of previous research into the
interpretation of multiple embedded symptom validity
test failures among clinical groups. For example, Victor,
Boone, Serpa, Buehler, and Zielger (2009) examined the
relative usefulness of using four embedded symptom
validity tests, including the WAIS-III Reliable DS, to
predict whether or not 105 outpatients referred for
neuropsychological testing belonged to a ‘‘credible’’ or
‘‘noncredible’’ group. The credible group was composed
of patients who had a financial incentive to underper-
form (e.g., in litigation or were attempting to secure
disability) and failed two or more of five freestanding
symptom validity tests, while the noncredible group did
not have a clear financial incentive to underperform
and failed one or fewer of five freestanding symptom val-
idity tests. Results of the study showed that the rate of
failure on any one embedded symptom validity test was
fairly common in their clinical sample, with 41% of the
credible group failing at least one measure. The accuracy
of predicting group membership was reported to be
superior when using any pairwise (i.e., two or more) fail-
ure combination of the embedded symptom validity tests
(sensitivity of 83.8%, specificity of 93.9%) as compared
with using one test by itself (sensitivity of 94.6%, speci-
ficity of 53.0%) or compared with using any three-test
failure combination (sensitivity of 51.5%, specificity of
98.5%), where sensitivity was substantially lowered.
Thus, it is likely that using one of the DS variables in
combination with another embedded symptom validity
test would increase the accuracy of identifying negative
response bias. With regard to which DS variable to
include in this combination, the results of the current
study are consistent with previous research suggesting
that DS Age SS is likely preferable for use over Reliable
DS as a measure of negative response bias on cognitive
testing. Study findings also go a step further and provide
evidence that the newly introduced DS Sequencing Total
score is preferable for use over both DS Age SS and
Reliable DS in predicting TOMM and MSVT failure.
Perhaps even more importantly, however, study data
suggest that clinicians should use caution and carefully
consider the predictive value of such measures in the spe-
cific population in which they serve. The latter is strongly
encouraged because, even using a fairly high base rate of
negative response bias (30%), the PPVs of the DS vari-
ables studied are either not too much higher than chance
or, more often, are worse than chance in predicting
symptom validity test failure. Even more concerning is
the finding that the already low PPVs of all the DS
variables are substantially reduced among populations
at low risk for negative response bias on cognitive test-
ing. Although there is little research on the topic, it is
believed that the base rate of negative response bias
among veterans referred for neuropsychological testing
is relatively high. It is important to note, however, that
veterans referred for neuropsychological testing are
likely not representative of the general veteran popu-
lation, as clinicians often refer patients for neuropsycho-
logical testing in the face of diagnostic uncertainty and,
often, potential concerns about dissimulation. That
being said, in the current study, the rate of negative
response bias was found to be 25% when using the
TOMM and 34% when using the MSVT. The latter rates
are higher than the 17% rate of MSVT failure we found
among a polytrauma sample (N ¼ 23; Whitney, Shepard,
Williams, Davis, & Adams, 2009). However, they are
substantially below the 58% MSVT failure found by
Armistead-Jehle (2010) in his examination of 45 veterans
consecutively referred for neuropsychological assessment
after screening positive on the Second-Level Veterans
Health Administration Traumatic Brain Injury Screen.
The high 58% failure rate in the latter study may have been
associated with increased financial incentive, as all patients
in that study self-reported brain injury symptoms that were
linked to combat, and thus were potentially compensable.
In the latter study, there were no significant differences
in failure rates on the MSVT as a function of any
demographic variable or clinical diagnoses of PTSD or
substance abuse disorder. It is notable that the 58% rate
of failure in the latter study is quite similar to that reported
among other groups with clear financial incentive to
underperform. For example, when Chafetz (2008)
examined the rate of probable malingering among
low-functioning social security disability claimants, it
was found that the rates were 45.8% among the TOMM
sample and 59.7% in the MSVT sample. Future research
should continue to examine the rate of negative response
bias among veteran populations and other populations
to better understand its occurrence and predictors.
The current investigation is certainly not without limi-
tation. Specifically, it should be noted that evaluating the
clinical utility of one symptom validity test by comparing
it to another is limited by the validity of the criterion
measure used in the comparison—in this case the

TOMM and the MSVT. However, there was not com-
plete concordance even between these measures, both
of which have substantial research support. As shown
in Table 7, failure rates on the MSVT were actually
higher than were failure rates on the TOMM across all
diagnostic groups. This finding was apparently not just
due to increased sensitivity of the MSVT because, of the
25 patients who failed the TOMM, 3 actually passed
the MSVT. The majority of the other patients who failed
the TOMM also failed the MSVT and did not satisfy
criteria for the DP (n ¼ 20=25), while another patient
failed the MSVT but met the DP (n ¼ 1=25), and another
did not take the MSVT (n ¼ 1=25).
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013
Thus, although we agree with Slick, Sherman, and
Iverson (1999) that the use of at least two well-validated
symptom validity tests, along with behavioral and
collateral data, should be used as a core element of
neuropsychological assessment of individuals for whom
secondary gain may be a factor, it remains unclear as to
which symptom validity test results should be given pri-
ority when results differ among them. Additionally, as in
all similar clinical studies, when compared with simulated
malingering studies, an inherent weakness of the present
study is not having a-priori knowledge of the validity of
the patient’s test-taking approach. Perhaps future research
can address this issue with a mixed design including both
simulators and clinical patients with the goal of comparing
the pattern of study results across study groups. Future
research should continue to investigate the relationships
between independent cognitive symptom validity tests,
such as the TOMM and MSVT, and embedded indicators
of negative response bias, such as DS scores.
REFERENCES
Armistead-Jehle, P. (2010). Symptom validity test performance in U.S.
veterans referred for evaluation of mild TBI. Applied Neuropsychol-
ogy, 17, 52–59.
Armistead-Jehle, P., & Gervais, R. O. (2011). Sensitivity of the Test
of Memory Malingering and the Nonverbal Medical Symptom
Validity Test: A replication study. Applied Neuropsychology, 18,
284–290.
Ashendorf, L., Constantinou, M., & McCaffrey, R. J. (2004). The effects
of depression and anxiety on the TOMM in community-dwelling
older adults. Archives of Clinical Neuropsychology, 19, 125–130.
Axelrod, B. N., Lu, P., & Schutte, C. (2011). Concurrent validity of
three forced-choice measures of symptom validity. Applied Neurop-
sychology, 18, 27–33.
Babikian, T., Boone, K. B., Lu, P., & Arnold, G. (2006). Sensitivity
and specificity of various Digit Span scores in the detection of
suspect effort. The Clinical Neuropsychologist, 20, 145–159.
Carone, D. (2008). Children with moderate=severe brain damage=
dysfunction outperform adults with mild-to-no brain damage on
the Medical Symptom Validity Test. Brain Injury, 22, 960–971.
Chafetz, M. D. (2008). Malingering on the social security disability
consultative exam: Predictors and base rates. The Clinical Neuropsy-
chologist, 22, 529–546.
WAIS-IV DIGIT SPAN VARIABLES 93
Davis, J. J., Wall, J. R., & Whitney, K. A. (in press). Derivation and
clinical validation of consistency indices on the Test of Memory
Malingering. Archives of Clinical Neuropsychology.
Devilly, G. J. (2004). Effect Size Generator for Windows: Version 2.3
[Computer software]. Hawthorn, Australia: Centre for Neuropsy-
chology, Swinburne University.
Green, P. (2004). Green’s Medical Symptom Validity Test (MSVT) for
Windows: User’s manual. Edmonton, AB, Canada: Green’s Publishing.
Green, P. (2008). Manual for the Nonverbal Medical Symptom Validity
Test. Edmonton, AB, Canada: Green’s Publishing.
Green, P. (2011). Comparison between the Test of Memory Malingering
(TOMM) and the Nonverbal Medical Symptom Validity Test in
adults with disability claims. Applied Neuropsychology, 18, 18–26.
Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of
malingered amnesia measures with a large clinical sample. Psycho-
logical Assessment, 6(3), 218–224.
Greve, K. W., Springer, S., Bianchini, K. J., Black, F. W., Heinly, M.
T., Love, J. M., . . . Ciota, M. A. (2007). Malingering in toxic
exposure: Classification accuracy of Reliable Digit Span and
WAIS-III Digit Span scaled score. Assessment, 14, 12–21.
Heinly, M. T., Greve, K. W., Bianchini, K. J., Love, J. L., & Brennan,
A. (2005). WAIS Digit Span-based indicators of malingered neuro-
cognitive dysfunction: Classification accuracy in traumatic brain
injury. Assessment, 12, 429–444.
Howe, L. L. S., Anderson, A. M., Kaufman, D. A. S., Sachs, B. C., &
Loring, D. W. (2007). Characterization of the Medical Symptom
Validity Test in evaluation of clinically referred memory disorders
clinic patients. Archives of Clinical Neuropsychology, 22, 753–761.
Howe, L. S., & Loring, D. W. (2008). Classification accuracy and
predictive ability of the Medical Symptom Validity Test’s dementia
profile and general memory impairment profile. The Clinical
Neuropsychologist, 23, 329–342.
Iverson, G. L., Le Page, J., Koehler, B. E., Shojania, K., & Badii, M.
(2007). Test of Memory Malingering (TOMM) scores are not
affected by chronic pain or depression in patients with fibromyalgia.
The Clinical Neuropsychologist, 21, 532–546.
Lipsey, M. W. (1990). Design sensitivity. Newberry Park, CA: Sage.
Loring, D. W., & Bauer, R. M. (2010). Testing the limits: Cautions
and concerns regarding the new Wechsler IQ and memory scales.
Neurology, 74, 685–690.
Malec, J. F., Brown, A. W., Leibson, C. L., Flaada, J. T., &
Mandrekar, J. N. (2007). The Mayo classification system for trau-
matic brain injury severity. Journal of Neurotrauma, 24, 1417–1424.
O’Bryant, S. E., & Lucas, J. A. (2006). Estimating the predictive value
of the Test of Memory Malingering: An illustrative example for
clinicians. The Clinical Neuropsychologist, 20, 533–540.
Rees, L. M., Tombaugh, T. N., Gansler, D. A., & Moczynski, N. P.
(1998). Five validation experiments of the Test of Memory Malin-
gering (TOMM). Psychological Assessment, 10, 10–20.
Slick, D. J., Sherman, E. M. S., & Iverson, G. L. (1999). Diagnostic
criteria for malingered neurocognitive dysfunction: Proposed
standards for clinical practice and research. The Clinical Neuropsy-
chologist, 13, 545–561.
Teichner, G., & Wagner, M. T. (2004). The Test of Memory Malinger-
ing (TOMM): Normative data from cognitively intact, cognitively
impaired, and elderly patients with dementia. Archives of Clinical
Neuropsychology, 19, 455–464.
Tombaugh, T. N. (1996). TOMM: Test of Memory Malingering. North
Tonawanda, NY: Multi-Health Systems, Inc.
Tombaugh, T. N. (1997). The Test of Memory Malingering (TOMM):
Normative data from cognitively intact and cognitively impaired
individuals. Psychological Assessment, 9, 260–268.
Victor, T. L., Boone, K. B., Serpa, G., Buehler, J., & Ziegler, E. A.
(2009). Interpreting the meaning of multiple symptom validity test
failure. The Clinical Neuropsychologist, 23, 297–313.
 94 WHITNEY, SHEPARD, & DAVIS
Wechsler, D. A. (1997a). Wechsler Adult Intelligence Scale (3rd ed.).
New York, NY: Psychological Corporation.
Wechsler, D. A. (1997b). Wechsler Memory Scale (3rd ed.). New York,
NY: Psychological Corporation.
Wechsler, D. A. (2008). Wechsler Adult Intelligence Scale (4th ed.). San
Antonio, TX: Psychological Corporation.
Whitney, K. A., Davis, J. J., Shepard, P. H., Bertram, D., & Adams,
K. M. (2009). Digit Span Age-Scaled Score in middle-aged military
Downloaded by [b-on: Biblioteca do conhecimento online UC] at 16:01 12 February 2013
veterans: Is it a better predictor of negative response bias than
Reliable Digit Span? Archives of Clinical Neuropsychology, 24,
263–272.
Whitney, K. A., Shepard, P. H., Williams, A. L., Davis, J. J., &
Adams, K. M. (2009). The Medical Symptom Validity Test in the
evaluation of Operation Iraqi Freedom=Operation Enduring
Freedom soldiers: A preliminary study. Archives of Clinical
Neuropsychology, 24, 145–152.