See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/9887159

A Validation of the Test of Memory Malingering in a Forensic Psychiatric

Setting

Article in Journal of Clinical and Experimental Neuropsychology · November 2003

DOI: 10.1076/jcen.25.7.979.16481 · Source: PubMed

CITATIONS READS

77 2,299

5 authors, including:

Michael Weinborn Steven Paul Woods

University of Western Australia University of Houston
124 PUBLICATIONS 2,406 CITATIONS 443 PUBLICATIONS 18,038 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Prospective memory partially mediates the association between aging and everyday functioning View project

Substance use and risk taking View project

All content following this page was uploaded by Michael Weinborn on 27 May 2014.

The user has requested enhancement of the downloaded file.

 Journal of Clinical and Experimental Neuropsychology 1380-3395/03/2507-979$16.00
2003, Vol. 25, No. 7, pp. 979–990 # Swets & Zeitlinger

A Validation of the Test of Memory Malingering

in a Forensic Psychiatric Setting
Michael Weinborn1, Tamara Orr2, Steven Paul Woods3, Emily Conover3, and Jeffrey Feix4
1
Martinsburg VA Medical Center, Mental Health Service Line, Martinsburg, WV, USA, 2Central State Hospital,
Virginia Commonwealth University, Petersburg, VA, USA, 3University of California, San Diego, CA, USA,
and 4Central State Hospital, Medical College of Virginia, Richmond, VA, USA

ABSTRACT

The Test of Memory Malingering (TOMM) has not been adequately validated in a forensic psychiatric setting.
Dissimulation of cognitive impairment, as assessed by the TOMM, was evaluated in a group of 25 forensic
inpatients admitted for evaluation of Competency to Stand Trial (CST/MSO group), and hypothesized to be at
higher risk for feigning cognitive impairment. A comparison group of 36 patients, who were either civilly
committed or adjudicated Not Guilty by Reason of Insanity (CIVIL/NGRI group), were hypothesized to be less
likely to feign cognitive impairment. Groups were comparable in age, education, premorbid intelligence, and
psychiatric symptom severity. Significantly more CST/MSO patients (36%) scored below a recommended
TOMM cutoff score relative to CIVIL/NGRI patients (6%). Findings indicate excellent specificity and modest
sensitivity, and generally support the validity of the TOMM in a forensic psychiatric population. The utility of
different cutoff scores and need for multiple indicators of effort are discussed.

The detection of invalid or suboptimal neuropsy-
chological test performance is an area of increas-
ing clinical and empirical interest. A variety of
procedures and approaches have been employed
to detect invalid neuropsychological test per-
formance, including analyses of level and pat-
terns (i.e., inconsistency or infrequent/unusual
responses) of performance for existing neuropsy-
chological tests and test batteries (e.g., Baker,
Donders, & Thompson, 2000; Milanovich,
Axelrod, & Millis, 1996; Trueblood & Schmidt,
1993). Of increasing popularity, however, is the
use of tests specifically designed to identify in-
complete effort or blatant feigning of cognitive
impairment, commonly described as symptom
validity testing (SVT; see Bianchini, Mathias, &
Greve, 2001; Vickery, Berry, Inman, Harris, &
Orey, 2001). Among the available procedures, the
forced choice recognition recall paradigm has
become perhaps the most common approach to
SVT. For example, SVTs based on forced choice
recognition recall of digits (Portland Digit Rec-
ognition Test, PDRT; Binder, 1990; Computer-
ized Assessment of Response Bias, CARB; Allen,
Condor, Green, & Cox, 1997), and words (Word
Memory Test, WMT; Green, Allen, & Astner,
1996) have been developed.
Although forced choice recognition tests were
initially used to identify below chance-level
responding (commonly cited as compelling evi-
dence for deliberate feigning of impairment), it
became clear that below-chance performance
was not common among malingerers (e.g.,
Griffenstein, Baker, & Gola, 1994; Martin, Hayes,
& Gouvier, 1996), and therefore resulted in
unacceptably low sensitivity. An alternative to
the criterion of below-chance responding is the
use of cutoff scores derived from the performance
of genuinely impaired patients. Cutoff scores are
chosen to maximize classification accuracy of

Address correspondence to: Michael Weinborn, Ph.D., Martinsburg VA Medical Center, Mental Health Service
Line, Martinsburg, WV 25401, USA. E-mail: mweinbo@hotmail.com
Accepted for publication: January 3, 2003.
980 MICHAEL WEINBORN ET AL.
valid and invalid test performances (e.g., specifi-
city and sensitivity, respectively). A central tenet
of the cutoff score approach is that forced choice
recognition recall tasks are very easy for all but
the most severely impaired patients, provided
adequate effort is put forward. That is, these tasks
are meant to be sensitive to malingering or
incomplete effort, while relatively insensitive to
genuine memory impairment.
THE TEST OF MEMORY MALINGERING
The Test of Memory Malingering (TOMM;
Tombaugh, 1996) is a visual forced choice rec-
ognition SVT, in which patients are shown 50 line
drawings of common objects for 3 s each during
two learning trials. Each learning trial is followed
by a two-choice recognition recall task, in which
the individual attempts to identify the target ob-
ject from a distracter item. The examiner provides
feedback regarding correctness for each item. An
optional retention trial can be administered fol-
lowing an approximately 15-min delay. The man-
ual recommends a cutoff raw score of <45 on
either Trial 2 or the Retention Trial as indicative
of suboptimal effort.
Prior research indicates that the TOMM dem-
onstrates adequate classification accuracy in dis-
criminating individuals with genuine cognitive
impairment from simulated malingerers (Rees,
Tombaugh, Gansler, & Moczynski, 1998;
Tombaugh, 1997; Vallabhajosula & van Gorp,
2001), while not being adversely affected by age
or education (Rees et al., 1998), depression (Rees,
Tombaugh, & Boulay, 2001), and many types of
neurological conditions that are not accompanied
by at least moderate dementia (Tombaugh, 1997).
It has been argued that the TOMM possesses
sufficient reliability and validity to meet the
Daubert criteria for admissibility of scientific
evidence in the courtroom (Vallabhajosula & van
Gorp, 2001).
RATIONALE FOR THE PRESENT STUDY
In a recent review, Vallabhajosula and van Gorp
(2001) argue that the TOMM displays great prom-
ise as a tool in evaluating test-taking effort; how-
ever, they also detail important methodological
limitations in the current research literature. Most
importantly, they highlight the limitations of re-
search designs utilizing ‘‘simulated,’’ rather than
‘‘real-life’’ malingerers. Specifically, the simula-
tor design has been widely criticized as having
uncertain ecological validity for ‘‘real world’’
clinical experience. For example, it has been
suggested that many ‘‘real’’ malingerers have a
great deal of practice and skill with deception,
whereas some ‘‘normals’’ asked to feign cognitive
deficits may have difficulty allowing themselves
to ‘‘pretend’’ to be dishonest (Pankratz & Binder,
1997). Studies evaluating the TOMM have at-
tempted to address the aforementioned ecolog-
ical validity concerns by various means, including
differential payment of subjects based on test
performance, education of simulators regarding
the effects of brain injury, and assessment of
participants’ self-reported compliance with the
simulation instructions (e.g., Rees et al., 1998;
Tombaugh, 1997).
Studying ‘‘real’’ malingerers also poses prob-
lems. Given the logistical difficulties of identifying
and recruiting a group of individuals definitively
known to be malingering neurocognitive deficits
based on irrefutable evidence (e.g., confession or
videotaped evidence), it is not surprising that very
few such studies have been published. Subsequent-
ly, the study of ‘‘suspected’’ or ‘‘probable’’ malin-
gerers has emerged as an alternative methodology.
Such individuals are usually identified based on
inclusion criteria such as external incentive,
‘‘improbable’’ performances on formal testing,
failure on SVTs, and inconsistencies between
reported and observed symptoms (e.g., Griffenstein
et al., 1994; Slick, Sherman, & Iverson, 1999). The
‘‘known group’’ approaches have nevertheless
been criticized as having a significant risk of
misclassification due to both circularity in defini-
tion, as well as low diagnostic sensitivity and inter-
clinician agreement (Pankratz & Binder, 1997).
Perhaps a more practical approach to evaluate
the ecological validity of SVTs is to compare
groups with differential motivations to malinger;
that is, groups with known or expected differences
in base rates of malingering (i.e., ‘‘differential
prevalence’’ designs). For example, Grote and
 TOMM VALIDATION
colleagues found that 100% of a sample of
noncompensation-seeking patients scored in the
valid range on the Victoria Symptom Validity
Test, whereas only 58.5% of compensation-
seeking patients produced data in the valid range
(Grote et al., 2000). Other studies using the
differential prevalence design have found differ-
ences on SVT performance between groups with
discrepant motivation to feign cognitive impair-
ment (e.g., Binder, 1993; Doss, Chelune, &
Naugle, 1999; Meyers & Diep, 2000; Meyers &
Volbrecht, 1998). The authors are aware of only
one such study that has been completed with the
TOMM, which found that TBI patients in litiga-
tion produced significantly lower scores than TBI
patients not in litigation, or cognitively intact
controls (Rees et al., 1998). A more recent study
found that the TOMM was sensitive to malinger-
ing in a group of 57 men suspected of feigning
incompetence to stand trial (Heinze & Purisch,
2001). However, the authors did not include a
comparison group, which significantly limits the
conclusions to be drawn from this study.
While differential prevalence designs usually
involve an ‘‘all-or-none’’ approach to motivation
to malinger – that is, groups are presumed to
either be motivated to malinger or without such
motivation – an interesting extension of the
differential prevalence design might include
multiple groups with varying levels of motivation.
For example, TBI litigants in current financial
crisis may be more motivated to exaggerate to
meet immediate needs than litigants who have
greater financial stability. Similarly, one might
hypothesize that individuals facing murder
charges may be more motivated to feign incom-
petence to stand trial than defendants facing lesser
charges. The authors are unaware of any such
published studies using the TOMM to evaluate
the potential impact of differing levels of moti-
vation to feign cognitive impairment.
Vallabhajosula and van Gorp (2001) highlight-
ed additional methodological concerns regarding
the present TOMM literature. They note that most
studies using the TOMM were limited by small
sample sizes and the failure to evaluate the
performance of psychiatric and other clinical
groups. It is also important to note that those
studies that have addressed the impact of educa-
981
tion have not included individuals with the lowest
levels of education. For example, in the sample
used in the Rees et al. (1998) experiment most
relevant to the effects of education, the lowest
level of education represented was 8 years, and
the mean education for all subjects was 13.1
years. Similarly, the performance of individuals
with mental retardation or borderline intellectual
functioning on the TOMM has not been evaluat-
ed. Of note, research with other SVT measures
has shown false positive rates between 13 and
27% for individuals with mental retardation
(Hayes, Hale, & Gouvier, 1998) or schizophrenia
(Back et al., 1996).
Finally, Colby (2001) raised an additional
criticism regarding the sensitivity and specificity
obtained with the presently used cut-points for the
TOMM as recommended by the test manual. His
analysis of alternate cut-points indicated that false
positive errors are reduced (without a substantial
decline in sensitivity) in individuals with and
without dementia when a cutoff of more than 14
errors on Trial 2 and Retention combined, or more
than 13 errors if only Trial 2 is given and
dementia cannot be ruled out. However, these
alternative cutoff criteria have not, as of yet, been
evaluated elsewhere in the literature.
Competency to Stand Trial Defendants
In their review of the research literature,
Wynkoop and Denney (1999) found only two
peer reviewed studies concerning malingering
among individuals being evaluated for competen-
cy to stand trial (CST; Daniel & Resnick, 1987;
Gothard, Viglione, Meloy, & Sherman, 1995),
neither of which specifically addressed feigning
of cognitive impairment. Subsequent research has
found a modest prevalence of feigned psychiatric
symptoms in CST samples; for example, Lewis
and colleagues found that of 55 individuals un-
dergoing CST evaluation in the federal justice
system, 24 (44%) were classified as feigning
based on the Structured Interview of Reported
Symptoms (Lewis, Simcox, & Berry, 2002). With
regard to feigning of cognitive impairment among
the CST population, Heinze and Purisch (2001)
found 57 of 438 (13%) defendants displayed
‘‘compelling clinical evidence’’ of malingering
incompetence to stand trial, but felt that their
982 MICHAEL WEINBORN ET AL.
finding likely represented an underestimate of the
true base rate of malingering due to characteristics
of their sample. Using personal and normative
‘‘floor effect’’ strategies to identify validity of test
performance, Frederick (2000a) found that ap-
proximately 15–17% of subjects displayed suspi-
cious performances. In another study, Frederick
(2000b) attempted to estimate base rates of ma-
lingered cognitive impairments based on clinical
ratings, scores on the Word Recognition Test
(WRT; Rey, 1941), and Bayesian statistical pro-
cedures. Estimates based on the WRT were very
low (0.5%) after correction for false positive rates.
However, the other clinical (15.5%) and statistical
estimates (13.5%) produced base rates more sim-
ilar to those found by Heinze and Purisch (2001),
and indicate that the base rate for malingering
cognitive impairment in CST samples is at least
13–17%, and perhaps may be higher.
OBJECTIVES OF THE PRESENT STUDY
The primary objective of the present study was to
evaluate the predictive validity and clinical use-
fulness of the TOMM using a differential preva-
lence design with patients in a state psychiatric
hospital, with a mixed forensic and civil popula-
tion. The first group of interest (CST/MSO group)
was comprised of inpatients being evaluated for,
or treated for restoration of, competency to stand
trial (CST). Treatment in an effort to restore
competency to stand trial is focused on the factors
that are believed to be interfering with competen-
cy for that individual, typically uncontrolled psy-
chiatric symptoms (addressed through medication
and psychosocial interventions) and/or lack of
basic and necessary knowledge of the judicial
system (addressed through educational interven-
tions). Seven of the CST patients were also eval-
uated for sanity at the time of the offense (MSO,
Mental State at the Time of Offense). Individuals
in the CST/MSO group were hypothesized to
have greater motivation to feign or exaggerate
impairment in an attempt to avoid prosecution or
incarceration, or at least to serve what is com-
monly perceived as ‘‘easy time’’ in a psychiatric
hospital instead of prison if found Not Guilty by
Reason of Insanity (NGRI; Hayes et al., 1998).
The second group consisted of inpatients who
were either civilly committed to the hospital, or
who had already been adjudicated NGRI (CIVIL/
NGRI group). These individuals were hypothe-
sized to be more likely to minimize or deny
cognitive impairment in order to increase the
likelihood of discharge. As an extension of the
previous analysis, the performance of individuals
within the CST/MSO group with the most serious
charge (i.e., murder) were compared with indi-
viduals facing less serious charges (e.g., misde-
meanor and lesser felony charges). Given the
greater consequences associated with conviction,
individuals charged with murder were hypothe-
sized to be more likely to feign impairment than
individuals facing lesser charges.
An additional goal of the present study was to
evaluate the usefulness of the TOMM in a setting
where most individuals have one or more factors
that may influence performance for SVT mea-
sures, including chronic psychotic illness, mental
retardation, borderline intellectual functioning,
dementia, and very limited education (i.e., less
than 8 years). Finally, the usefulness of two
proposed cutoff scores used to classify individuals
putting forth adequate effort from those with
suboptimal effort will be evaluated; namely those
recommended by the TOMM manual, and those
suggested by Colby (2001).
METHOD
Participants
Participants were 61 of 68 inpatients consecutively
referred to the first author for evaluation in the
Neuropsychology Service of a large state psychiatric
hospital. The admitting status of all patients included in
the present study was CIVIL commitment (n ¼ 21),
NGRI (n ¼ 15), or evaluation/treatment for CST/MSO
(n ¼ 25). Four patients (all CIVIL/NGRI) with severe
dementia or very poorly controlled psychosis docu-
mented by their treatment team were unable to
meaningfully complete more than a minimal cognitive
screening evaluation (e.g., Mini Mental Status Exam-
ination; Folstein, Folstein, & McHugh, 1975) and were
not included. TOMM data were also unavailable for an
additional three patients: one CST/MSO patient was
discharged before completing the full evaluation; one
CIVIL/NGRI patient did not complete the TOMM due
to examiner error; and one CIVIL/NGRI patient did not
 TOMM VALIDATION 983

complete the TOMM due to time limitations imposed
by the referral source.
The 61 participants with complete TOMM data
included 49 (80%) men and 12 (20%) women, and were
mostly African-American (n ¼ 31, 51%) or Caucasian
(n ¼ 26, 43%). Two subjects were biracial (3%) and
one was Asian-American (2%). The mean age for all
participants was 39.6 (SD ¼ 13.8). Participants reported
a relatively low level of education (M ¼ 9.8, SD ¼ 3.1,
range ¼ 2–18 years). Estimated premorbid intelligence
was in the low average to average range based on
demographic variables (Wechsler Test of Adult Reading
demographics estimated Full Scale IQ score M ¼ 90.8,
SD ¼ 10.3; Psychological Corporation, 2001).
Primary Diagnostic and statistical manual of
mental disorders (DSM-IV; American Psychiatric
Association, 1994) Axis I diagnoses for all subjects
included Schizophrenia-spectrum and other psychotic
disorders (56%), affective disorders (15%), dementia
and other neurological disorders (10%), and other
diagnoses (3%). Thirteen percent of subjects carried a
primary or secondary Axis II personality disorder
diagnosis. Seventy-two percent of all subjects had a
reported history of alcohol or substance abuse. Eight-
een percent of the subjects had a history of mental
retardation (CST/MSO, n ¼ 5; CIVIL/NGRI, n ¼ 6),
and 15% were diagnosed with borderline intellectual
functioning (CST/MSO, n ¼ 5; CIVIL/NGRI, n ¼ 4).
As shown in Table 1, the CST/MSO group had
significantly fewer patients (36%) with psychotic
disorders than the CIVIL/NGRI group (69%), but
groups did not differ for other diagnoses. The groups

Table 1. Demographic Characteristics and TOMM Scores in the CST/MSO and CIVIL/NGRI Groups.

Variable CST/MSO (n ¼ 25) CIVIL/NGRI (n ¼ 36)

Demographic and diagnostic characteristics

Age 36.44 (15.05) 41.83 (12.68)
Education 10.48 (2.93) 9.28 (3.22)
WTAR Demographic Full Scale IQ 93.00 (10.15) 90.26 (10.00)
Gender
Male 76% 83%
Female 24% 17%
Race
Caucasian 44% 44%
African-American 52% 50%
Asian 3% 0%
Biracial 4% 3%
Diagnoses
Psychotic disordersa 36% 69%
Affective disorders 16% 14%
Personality disorders 28% 22%
Dementia 8% 3%
Substance abuse/dependence 72% 72%
Mental retardation 20% 17%
Borderline intelligence 20% 11%
Other diagnoses 4% 3%
BPRS total 38.25 (8.74) 38.97 (11.03)

Test of Memory Malingering

TOMM Trial 1 37.68 (11.46) 41.19 (8.32)
TOMM Trial 2a 42.52 (9.19) 47.56 (4.30)
TOMM retention triala 43.64 (9.09)b 48.21 (3.79)c
Note. Data are presented either as the mean (with standard deviation in parentheses), or as valid population
percentages. BPRS ¼ Brief Psychiatric Rating Scale; CST/MSO ¼ Competency to Stand Trial/Mental Status
at the Time of Offense group; CIVIL/NGRI ¼ civilly committed/Not Guilty by Reason of Insanity group;
TOMM ¼ Test of Memory Malingering; WTAR ¼ Wechsler Test of Adult Reading.
a
p < .05.
b
CST/MSO, n ¼ 23.
c
CIVIL/NGRI, n ¼ 26.
984 MICHAEL WEINBORN ET AL.

did not differ significantly for age, education, race,
estimated premorbid intellectual functioning, or gross
level of psychopathology, as measured by the Brief
Psychiatric Rating Scale, Expanded Version (BPRS;
Lukoff, Liberman, & Nuechterlein, 1986).
Procedure
All subjects completed the TOMM as part of a larger
neuropsychological evaluation. The TOMM was admin-
istered and scored according to the procedures
outlined in the test manual (Tombaugh, 1996) by the
first author, or doctoral level clinical psychology
students under supervision by the first author. Subjects
were classified as passing or failing the TOMM as
indicated by: (1) the cut scores recommended by the
manual (i.e., a score of <45 on either Trial 2 or
Retention; Tombaugh, 1996); and (2) the cut scores
recommended by Colby (i.e., greater than 14 total
errors on Trial 2 and Retention, or greater than 13 errors
on Trial 2 if Retention Trial is not administered and
dementia cannot be ruled out; 2001).
operating characteristic (ROC) curves and descriptive
classification accuracy statistics were then generated to
determine whether the TOMM was significantly better
than chance in correctly classifying participants in the
CST/MSO and CIVIL/NGRI groups. McNemar’s chi-
square test was used to assess whether there was a
significant difference in the classification accuracy rates
obtained using the cutoff scores recommended by the
Colby (2001) and the TOMM manual (Tombaugh,
1996). Finally, a chi-square test was used to determine
if there was a significant difference in the proportion of
TOMM failure rates among CST/MSO participants
with (n ¼ 5) and without (n ¼ 20) pending murder
charges. Of note, the sensitivity, specificity, and other
classification accuracy statistics associated with these
data do not reflect ‘‘true’’ values associated with the
TOMM given the differential prevalence design of the
present study. That is, the reported classification
accuracy values likely underestimate the ability of the
TOMM to correctly classify subjects as there was no
‘‘gold standard’’ for suboptimal effort.

Data Analyses
A power analysis indicated that a total sample size of 61
provided adequate power to detect large effect sizes
(power ¼ 0.95; Erdfelder, Faul, & Buchner, 1996).
Given that the TOMM data were not normally distrib-
uted (all Kolmogrov–Smirnov p values <.001), a series
of Mann–Whitney U tests were conducted to evaluate
potential group differences between the CST/MSO and
CIVIL/NGRI participants. Next, the chi-square statistic
was used to test the hypothesis that a significantly
greater proportion of CST/MSO than CIVIL/NGRI
patients would fail the TOMM as indicated by perfor-
mance below the cutoff scores provided by the test
manual (Tombaugh, 1996) or Colby (2001). Receiver-
RESULTS
As shown in Table 1, the CST/MSO group dem-
onstrated significantly lower performance than
the CIVIL/NGRI group on Trial 2 (z ¼ 2.16,
p < .05, Cohen’s d ¼ 0.75), and the Retention
Trial (z ¼ 2.04, p < .05, d ¼ 0.71) of the
TOMM. The groups did not differ on Trial 1
(z ¼ 1.07, p > .05, d ¼ 0.35.). The frequency
of specific scores on all three trials of the TOMM
in each of the two groups is provided in Table 2.

Table 2. Frequency of TOMM Scores in the CIVIL/NGRI and CST/MSO Groups.

TOMM score CIVIL/NGRI (n ¼ 36) CST/MSO (n ¼ 25)

a
Trial 1 Trial 2 Retention Trial 1 Trial 2 Retentionb

17c 0 (0) 0 (0) 0 (0) 1 (4.0) 0 (0) 1 (4.5)

18–25 2 (5.6) 0 (0) 0 (0) 4 (20.0) 1 (4.0) 0 (4.5)
26–30 5 (19.4) 0 (0) 0 (0) 2 (28.0) 3 (16.0) 1 (9.1)
31–35 2 (25.0) 2 (5.6) 1 (3.8) 2 (36.0) 3 (28.0) 2 (18.2)
36–40 5 (38.9) 1 (8.3) 0 (3.8) 3 (48.0) 1 (32.0) 1 (22.7)
41–45 6 (55.6) 3 (16.7) 2 (11.5) 3 (60.0) 2 (40.0) 4 (40.9)
45–50 16 (100.0) 30 (100.0) 23 (100.0) 10 (100.0) 15 (100.0) 13 (100.0)
Note. Values in parentheses represent cumulative population percentages.
a
n ¼ 26.
b
n ¼ 23.
c
Suggestive of below chance responding.
 TOMM VALIDATION 985

Fig. 1. TOMM failure rates among psychiatric inpatients using cutoff criteria from the test manual (Tombaugh,
1996) and Colby (2001). Two groups of psychiatric inpatients are compared; inpatients being examined for
competency to stand trial and/or sanity at the time of offense (CST/MSO), and inpatients either civilly
committed to the hospital, or who had already been found NGRI (CIVIL/NGRI).

Figure 1 displays the proportion of CST/MSO Table 3. Accuracy of Two TOMM Cutoff Methods in
and CIVIL/NGRI participants who failed the Classifying the CST/MSO and CIVIL/NGRI
TOMM according to the cutoff scores recom- Groups.
mended by the test manual (Tombaugh, 1996) and Statistic TOMM Colby (2001)
Colby (2001). Relative to the CIVIL/NGRI group, manual cutoffs
a significantly larger proportion of participants in cutoffs
the CST/MSO group scored below the cutoff points
Sensitivity 0.44 0.36
indicative of poor effort, whether using those
Specificity 0.83 0.94
recommended by the test manual (44% vs. 17%; Hit rate 0.67 0.71
2[1, N ¼ 61] ¼ 5.48, p < .05, Cramér’s V ¼ 0.30), Positive predictive power 0.65 0.82
or by Colby (36% vs. 6%; 2[1, N ¼ 61] ¼ 9.52, Negative predictive power 0.68 0.68
p < .005, Cramér’s V ¼ 0.39). Likelihood ratio (binary) 2.59 6.00
The TOMM manual cutoff scores performed Note. CST/MSO ¼ Competency to Stand Trial/Mental
better than chance in correctly classifying parti- Status at the time of Offense; NGRI ¼ Not
cipants in the CST/MSO and CIVIL/NGRI Guilty by Reason of Insanity; TOMM ¼ Test of
groups, but only at a trend level (Area Under Memory Malingering.
the ROC Curve ¼ 0.64, SE ¼ 0.07, p ¼ .07).
Colby’s recommended cutoff scores, on the other
hand, significantly classified participants in the and Colby were significantly different from one
two groups (Area Under the ROC Curve ¼ 0.65, another (p < .05), with the Colby cut-points
SE ¼ 0.08, p < .05). In fact, the classification resulting in notably fewer false positive errors in
accuracy rates provided by the TOMM manual the CIVIL/NGRI groups (see Table 3).
986 MICHAEL WEINBORN ET AL.
Fig. 2. TOMM failure rates among individuals with pending murder charges and persons with other pending
criminal charges.
Finally, Figure 2 shows that, using Colby’s
recommended cut-points, a significantly larger
proportion of participants in the CST/MSO group
with pending murder charges (80%) failed the
TOMM as compared to participants without such
charges (25%; 2[1, N ¼ 25] ¼ 5.25, p < .05,
Cramér’s V ¼ 0.46).
DISCUSSION
The present study demonstrates that the TOMM
possesses adequate predictive validity in distin-
guishing a group with higher motivation to feign
cognitive deficits from a demographically similar
group without such motivation. Specifically, sub-
jects being evaluated for, or treated for restoration
to, competency to stand trial produced TOMM
scores indicative of dissimulation at rates as much
as six times higher than a comparison group of
similar patients who were civilly committed pa-
tients, or had already been adjudicated NGRI.
These results are consistent with previous findings
suggesting that individuals being evaluated for
competency to stand trial have an increased like-
lihood of feigning cognitive impairment (e.g.,
Heinze & Purisch, 2001; Frederick, 2000a,
2000b). Moreover, those facing the most serious
charge (i.e., murder) had a much higher propor-
tion of TOMM failures (80%) than those CST/
MSO subjects facing lesser charges (25%). This
finding suggests that even within a group hypoth-
esized to have greater motivation to feign cogni-
tive impairment, levels of motivation may vary
with levels of external incentive. However, the
limited sample size of this secondary analysis
limits the conclusions that can be drawn, and
warrants replication with a larger sample.
Group differences on variables other than
motivation to feign cognitive impairment are
unlikely to account for the present findings. The
CST/MSO and CIVIL/NGRI groups did not differ
significantly on any demographic (e.g., age and
education) or other potentially confounding var-
iables (e.g., premorbid intelligence and psychiat-
ric status), with the exception of psychiatric
 TOMM VALIDATION
diagnosis, where the CIVIL/NGRI group con-
tained a greater proportion of individuals diag-
nosed with schizophrenia-spectrum disorders.
However, it could be argued that psychosis may
adversely impact performance on SVT measures,
and therefore the higher rate of schizophrenic
disorders among the CIVIL/NGRI group would,
if it had any effect at all, increase our risk of Type
II error (i.e., not finding that the CST/MSO group
would perform more poorly on the TOMM).
The TOMM cutoff scores recommended by the
test manual (Tombaugh, 1996) and Colby (2001)
both performed better than chance in accurately
classifying patients into their respective groups.
However the results produced by the cutoff scores
differed significantly from each other, with those
recommended by Colby producing a larger effect
size, as well as fewer false positive errors in
comparison with the cutoff recommended by the
test manual. False positives using Colby’s cutoffs
(n ¼ 2) were both functioning in the moderate
range of mental retardation as documented by
extensive medical or school records. False posi-
tives using the manual cutoffs (n ¼ 6) included
the two moderately mentally retarded individuals,
but also a mildly mentally retarded patient and
two treatment-resistant schizophrenics with
prominent thought disorder. Indeed, in the present
study, the specificity levels associated with the
manual cutoffs (83%) were much lower in
comparison than those produced by the Colby
cutoffs (94%), while yielding only a modest
increase in sensitivity (44% vs. 36%). These
findings support the use of Colby’s recommended
TOMM cutoffs for this population, in comparison
with those suggested by the manual.
While the sensitivity estimates found for the
TOMM in this sample were quite low, it is
important to note that, as the present study
employed a differential prevalence design, not all
individuals in the CST/MSO group were hypoth-
esized to be malingerers; rather it was hypothesized
only that the base rates of dissimulation would be
higher in the CST/MSO group than in the compar-
ison group. Indeed, it is also possible that not all
individuals in the CIVIL/NGRI group were putting
forth optimal effort, and that some of the ‘‘false
positives’’ may actually have been exaggerating
impairment to remain hospitalized for a variety of
987
reasons (e.g., to receive meals and a place to
sleep, fear of leaving the secure, known environ-
ment of the hospital, etc.). Therefore, the presently
obtained values are not a true reflection of the
sensitivity of the TOMM, and likely underestimate
its ability to accurately detect dissimulation.
Indeed, the lack of a gold standard in reporting
classification accuracy statistics for malingering
research (as well as in other behavioral disorders)
remains problematic.
In order to obtain a more accurate estimate of
the sensitivity of the TOMM in this sample,
individuals in the CST/MSO group whose medical
records included reports by treating staff or eval-
uators indicative of suspected malingering (e.g.,
displaying symptoms that were inconsistent or
unlikely), or who had produced at or below chance
performance on one or more forced choice or
recognition memory test (i.e., Millis & Volinsky,
2001) were identified post hoc as possible malin-
gerers. Fifteen of the 25 CST/MSO subjects were
identified as ‘‘possible malingerers’’ according to
these criteria. Of these 15 individuals, nine were
correctly classified by the TOMM using Colby’s
cutoffs (60% sensitivity), whereas none of the
remaining 10 CST/MSO subjects were identified as
malingering (100% specificity). Thus, TOMM
scores below Colby’s cutoffs were highly accurate
in classifying suspected malingerers (100% posi-
tive predictive power), whereas scores above cutoff
were only modestly useful for classifying individ-
uals not suspected of malingering (63% negative
predictive power). Overall, a person scoring below
Colby’s TOMM cutoff was over 30 times more
likely to be a suspected malingerer as compared to
individuals scoring above the cutoff (odds ratio ¼
30.69; 95% confidence interval 0.67–626.41).
Using the test manual cutoffs, 10 of 15
possible malingerers were correctly identified
(67% sensitivity), but there was one false positive
(90% specificity), which resulted in a slightly
lower positive predictive power (91%). Relative
to persons scoring above the TOMM manual
cutoff, an individual scoring below the cutoff was
approximately 12 times more likely to be a
suspected malingerer (odds ratio ¼ 12.09; 95%
confidence interval 1.60–90.92). Consistent with
the above findings, normal TOMM scores were
only a modest predictor of participants not
988 MICHAEL WEINBORN ET AL.
identified as possible malingerers (64% negative
predictive power).
Importantly, one must consider the influence of
base rates of malingering in interpreting predictive
values (e.g., Baldessarini, Finklestein, & Arana,
1983). The prevalence of possible malingering in
the CST/MSO sample was approximately 60%,
which is somewhat higher than previously reported
(see Heinze & Purisch, 2001). Therefore, the rates
of positive and negative predictive power described
above may be viewed as over- and underestimates,
respectively. Clinicians and other research con-
sumers are encouraged to tailor the predictive
values derived from the current analyses to more
accurately reflect the estimated prevalence rates of
suspected malingering in their clinical or research
settings (see Baldessarini, Finklestein, & Arana,
1983; Heaton, Grant, & Matthews, 1991; Woods,
Weinborn, & Lovejoy, in press).
While these post hoc analyses indicate that the
sensitivity of the TOMM to malingering remains
modest (60–67%), our findings are quite similar
to those of Heinze and Purisch (2001), who
reported approximately 65% sensitivity using the
manual cutoffs, in a similar group of 57 men
suspected of feigning incompetence to stand trial.
In fact, reports of low sensitivity are not uncom-
mon among SVT studies using probable or
suspected malingerers (e.g., Bianchini et al.,
2001), as opposed to simulated malingerers, and
argue for the need to use multiple SVT measures,
as well as other indicators of response bias, such
as floor effects, multivariable composites, or
performance curve analysis (Heinze & Purisch,
2001; Millis & Volinsky, 2001).
Limitations of the Present Study
The present study was limited by a relatively
small sample size, particularly for the group fac-
ing murder charges. However, given the inherent
difficulties in collecting data with these relatively
uncommon subjects, it is interesting to note that
effect sizes were large enough that differences can
be detected with small samples. It remains im-
portant, however, that these findings are replicat-
ed with larger sample sizes and in individuals
with different cultural backgrounds.
The present study also has limitations associ-
ated with its use of a differential prevalence
design. Rogers (1997) has criticized the differen-
tial prevalence approach as being limited by lack
of empirical data supporting the assumptions that
groups such as litigating and nonlitigating TBI
patients will have different rates of malingering.
However, it could also be argued that differential
prevalence studies provide such empirical data,
and that knowledge regarding base rates of
malingering in different populations is inherently
valuable. Rogers also contends that base rates of
malingering are low, even in forensic samples.
However, the low base rates of malingering may
more accurately reflect situations that ‘‘real
world’’ clinicians are faced with on a daily basis,
and provide a better test of SVT performance. For
example, in their description of a differential
prevalence study using the Word Memory Test
(WMT) in litigating and nonlitigating groups,
Millis and Volinsky (2001) state that this ap-
proach ‘‘may have been an especially rigorous
test of the WMT because of the possibly low
prevalence of malingering in the sample.’’
Millis and Volinsky (2001) also point out that
that many ‘‘known group’’ studies use inclusion
criteria that increases the likelihood of only
evaluating the more extreme forms of malinger-
ing, potentially artificially inflating the sensitivity
of the SVT being evaluated. Frederick (2000b)
similarly criticizes an over-reliance on known
group designs (which he describes as Criterion
Group Validation), and suggested using complex
statistical approaches to differential prevalence
designs (described as Mixed Group Validation),
including the use of Bayesian equations to
estimate group base rates. However, these hy-
pothesized statistical adjustments have not yet
been validated elsewhere in the literature.
CONCLUSIONS AND FUTURE
DIRECTIONS
The findings from the present study support the
utility and predictive validity of the TOMM with-
in a forensic psychiatric population, which
includes individuals with severe mental illness,
very limited education, and lower premorbid in-
telligence. The findings also support the use of
Colby’s (2001) suggested cutoffs in assessing
 TOMM VALIDATION
test-taking effort among patients with a known
history of severe psychotic mental illness and/or
mild mental retardation. Indeed, three of the four
false positive errors found using the TOMM man-
ual cutoffs, but not with the Colby cutoffs, dis-
played significant residual psychosis or mild
mental retardation. While the use of Colby’s cut-
offs produces excellent specificity, multiple mea-
sures and approaches to detect feigning cognitive
impairment are needed, given the limited sensi-
tivity that appears to be common to most SVTs. In
fact, future studies might seek to assess the incre-
mental validity of the TOMM and other promis-
ing SVT measures and approaches to detecting
dissimulation. In addition, further evaluation of
the potential impact of demographic and diagnos-
tic variables on SVT performance (e.g., education
and psychiatric diagnosis) is needed. For exam-
ple, the present study finding suggests that the
TOMM may not be appropriate for use with the
moderately mentally retarded; however, there
were too few such subjects to adequately evaluate
that hypothesis. Similarly, while prior studies
have indicated that educational levels and some
psychiatric diagnoses do not impact SVT perfor-
mance, it is important to note that the general-
izability of these findings is unknown given that
much of the previous SVT research is directed
towards detection of malingering in a mild TBI
population.
REFERENCES
Allen, L., Conder, R.L., Green, P., & Cox, D.R.
(1997). CARB 1997 Manual for the computerized
assessment of response bias. Durham, NC:
CogniSyst.
American Psychiatric Association. (1994). Diag-
nostic and statistical manual of mental disorders.
Washington, DC: Author.
Back, C., Boone, K.B., Edwards, C., Parks, C.,
Burgoyne, B., & Silver, B. (1996). The performance
of schizophrenics on three cognitive tests of
malingering: Rey 15-Item Memory Test, Rey Dot
Counting, and Hiscock Forced-Choice Method.
Assessment, 3, 449–457.
Baker, R., Donders, J., & Thompson, E. (2000).
Assessment of incomplete effort with the California
Verbal Learning Test. Applied Neuropsychology, 7,
111–114.
989
Baldessarini, R.J., Finklestein, S., & Arana, G.W.
(1983). The predictive power of diagnostic tests
and the effect of prevalence of illness. Archives of
General Psychiatry, 40, 569–573.
Bianchini, K.J., Mathias, C.W., & Greve, K.W. (2001).
Symptom validity testing: A critical review. Clinical
Neuropsychologist, 15, 19–45.
Binder, L.M. (1990). Malingering following minor
head trauma. The Clinical Neuropsychologist, 4,
25–36.
Binder, L.M. (1993). Assessment of malingering after
mild head trauma with the Portland Digit Recogni-
tion Test. Journal of Clinical and Experimental
Neuropsychology, 15, 170–182.
Colby, F. (2001). Using the binomial distribution to
assess effort: Forced-choice testing in neuropsycho-
logical settings. Neurorehabilitation, 16, 253–265.
Daniel, A.E., & Resnick, P.J. (1987). Mutism, mal-
ingering, and competency to stand trial. Bulletin of
the American Academy of Psychiatry and the Law,
15, 301–308.
Doss, R.C., Chelune, G.J., & Naugle, R.I. (1999).
Victoria Symptom Validity Test: Compensation-
seeking vs. non-compensation-seeking patients in a
general clinical setting. Journal of Forensic Neu-
ropsychology, 1, 5–20.
Erdfelder, E., Faul, F., & Buchner, A. (1996).
GPOWER: A general power analysis program.
Behavior Research Methods, Instruments, and
Computers, 28, 1–11.
Folstein, M.F., Folstein, S.E., & McHugh, P.R. (1975).
Mini mental state: A practical method for grading
the cognitive state of patients for the clinician.
Journal of Psychiatric Research, 12, 189–198.
Frederick, R.I. (2000a). Mixed group validation: A
method to address the limitations of criterion group
validation in research on malingering detection.
Behavioral Sciences and the Law, 18, 693–718.
Frederick, R.I. (2000b). A personal floor effect strategy
to evaluate the validity of performance on memory
tests. Journal of Clinical and Experimental Neu-
ropsychology, 22, 720–730.
Gothard, S., Viglione, D.J., Meloy, J.R., & Sherman, M.
(1995). Detection of malingering in competency to
stand trial evaluations. Law and Human Behavior,
19, 493–505.
Green, P., Allen, K., & Astner, K. (1996). The Word
Memory Test: A user’s guide to the oral and
computer-administered forms. Durham, NC: Cog-
niSyst.
Griffenstein, M.F., Baker, W.J., & Gola, T. (1994).
Validation of malingered amnesic measures with a
large clinical sample. Psychological Assessment, 6,
218–224.
Grote, C.L., Kooker, E.K., Garron, D.C., Nyenhuis,
D.L., Smith, C.A., & Mattingly, M.L. (2000).
 990 MICHAEL WEINBORN ET AL.
Performance of compensation seeking and non-
compensation seeking samples on the Victoria
Symptom Validity Test: Cross-validation and ex-
tension of a standardization study. Journal of
Clinical and Experimental Neuropsychology, 22,
709–719.
Hayes, J.S., Hale, D.B., & Gouvier, W.D. (1998).
Malingering detection in a mentally retarded foren-
sic population. Applied Neuropsychology, 5, 33–36.
Heaton, R.K., Grant, I., & Matthews, C.G. (1991).
Comprehensive norms for an expanded Halstead-
Reitan Battery. Demographic corrections, research
findings, and clinical applications. Odessa, FL:
Psychological Assessment Resources, Inc.
Heinze, M.C., & Purisch, A.D. (2001). Beneath the
mask: Use of psychological tests to detect and
subtype malingering in criminal defendants. Journal
of Forensic Psychology Practice, 1, 23–52.
Lewis, J.L., Simcox, A.M., & Berry, D.T.R. (2002).
Screening for feigned psychiatric symptoms in a
forensic sample by using the MMPI-2 and the
structured inventory of malingered symptomatolo-
gy. Psychological Assessment, 14, 170–176.
Lukoff, D., Liberman, R.P., & Nuechterlein, K.H.
(1986). Symptom monitoring in the rehabilitation of
schizophrenic patients. Schizophrenia Bulletin, 12,
578–601.
Martin, R.C., Hayes, J.S., & Gouvier, W.D. (1996).
Differential vulnerability between postconcussion
self-report and objective malingering tests in iden-
tifying simulated mild head injury. Journal of
Clinical and Experimental Neuropsychology, 18,
265–275.
Meyers, J.E., & Diep, A. (2000). Assessment of
malingering in chronic pain patients using neuro-
psychological tests. Applied Neuropsychology, 7,
133–139.
Meyers, J.E., & Volbrecht, M. (1998). Validation of
reliable digits for detection of malingering. Assess-
ment, 5, 303–307.
Milanovich, J.R., Axelrod, B.N., & Millis, S.R. (1996).
Validation of the simulation index-revised with a
mixed clinical population. Archives of Clinical
Neuropsychology, 11, 53–59.
Millis, S.R., & Volinsky, C.T. (2001). Assessment of
response bias in mild head injury: Beyond mal-
ingering tests. Journal of Clinical and Experimental
Neuropsychology, 23, 809–828.
Pankratz, L., & Binder, L.M. (1997). Malingering on
intellectual and neuropsychological measures. In R.
Rogers (Ed.), Clinical assessment of malingering
View publication stats
and deception (2nd ed., pp. 223–236). New York:
Guilford Press.
Psychological Corporation. (2001). Wechsler Test of
Adult Reading. San Antonio: Author.
Rey, A. (1941). L’examen psychologie dans les cas
d’encephalopathie tramatique. Archives de Psycho-
logie, 28, 286–340.
Rees, L.M., Tombaugh, T.N., & Boulay, L. (2001).
Depression and the Test of Memory Malingering.
Archives of Clinical Neuropsychology, 16, 501–506.
Rees, L.M., Tombaugh, T.N., Gansler, D.A., &
Moczynski, N.P. (1998). Five validation experi-
ments of the Test of Memory Malingering (TOMM).
Psychological Assessment, 10, 10–20.
Rogers, R. (Ed.). (1997). Clinical assessment of
malingering and deception (2nd ed.). New York:
Guilford.
Slick, D.J., Sherman, E.M.S., & Iverson, G.L. (1999).
Diagnostic criteria for malingered neurocognitive
dysfunction: Proposed standards for clinical prac-
tice and research. The Clinical Neuropsychologist,
13, 545–561.
Tombaugh, T.N. (1996). Test of Memory Malingering
(TOMM). New York: Multi Health Systems.
Tombaugh, T.N. (1997). The Test of Memory Mal-
ingering (TOMM): Normative data from cognitively
intact and cognitively impaired individuals. Psy-
chological Assessment, 9, 260–268.
Trueblood, W., & Schmidt, M. (1993). Malingering and
other validity considerations in the neuropsycholo-
gical evaluation of mild head injury. Journal of
Clinical and Experimental Neuropsychology, 15,
578–590.
Vallabhajosula, B., & van Gorp, W.G. (2001). Post-
Daubert admissibility of scientific evidence on
malingering of cognitive deficits. Journal of the
American Academy of Psychiatry and the Law, 29,
207–215.
Vickery, C.D., Berry, D.T.R., Inman, T.H., Harris, M.J.,
& Orey, S.A. (2001). Detection of inadequate effort
on neuropsychological testing: A meta-analytic
review of selected procedures. Archives of Clinical
Neuropsychology, 16, 45–73.
Woods, S.P., Weinborn, M., & Lovejoy, D.W. (2003).
Are classification accuracy statistics underused in
neuropsychological research? Journal of Clinical
and Experimental Neuropsychology, 25, 431–439.
Wynkoop, T.F., & Denney, R.L. (1999). Exaggeration
of neuropsychological deficit in competency to
stand trial. Journal of Forensic Neuropsychology,
1, 29–53.