Journal of Clinical and Experimental Neuropsychology

ISSN: 1380-3395 (Print) 1744-411X (Online) Journal homepage: http://www.tandfonline.com/loi/ncen20

Demographically Corrected Norms for the

California Verbal Learning Test

Marc A. Norman , Jovier D. Evans , Walden S. Miller & Robert K. Heaton

To cite this article: Marc A. Norman , Jovier D. Evans , Walden S. Miller & Robert K. Heaton
(2000) Demographically Corrected Norms for the California Verbal Learning Test, Journal of
Clinical and Experimental Neuropsychology, 22:1, 80-94

To link to this article: http://dx.doi.org/10.1076/1380-3395(200002)22:1;1-8;FT080

Published online: 09 Aug 2010.

Submit your article to this journal

Article views: 424

View related articles

Citing articles: 61 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=ncen20

Download by: [Deakin University Library] Date: 30 October 2015, At: 16:08
 Journal of Clinical and Experimental Neuropsychology 1380-3395/00/2201-080$15.00
2000, Vol. 22, No. 1, pp. 80-94 © Swets & Zeitlinger

Demographically Corrected Norms for the California Verbal

Learning Test*
Marc A. Norman1, Jovier D. Evans2,3, S. Walden Miller2, and Robert K. Heaton2
Departments of 1Neurosciences and 2 Psychiatry , University of California, San Diego,
and 3 Indiana University Purdue University, Indianapolis

ABSTRACT
Downloaded by [Deakin University Library] at 16:08 30 October 2015

The California Verbal Learning Test (CVLT) is designed to quantify components of verbal learning, reten-
tion and retrieval. The present study used multiple regression analyses to correct for demographic charac-
teristics on CVLT performance measures. There were 906 subjects, of whom 549 were Caucasians (61%)
and 357 were African Americans (39%). Age, education, ethnicity, and gender were found to be significant
predictors of performance on several CVLT indices, including Total Words Recalled, Trial 1, Trial 5, List
B, Short Delay Free Recall (SDFR), and Long Delay Free Recall (LDFR). Demographically corrected T-
scores were calculated for a base sample of 672 subjects and cross-validated on 234 separate subjects.
Tables and regression equations are offered to convert raw scores into T-scores corrected for age, gender,
education, and ethnicity. Demographically corrected Recognition Discriminability cutoff scores were
calculated for age and education levels. In order to provide some indices of important memory processes,
we also computed indices of retrieval, Short-Delay forgetting and Long-Delay forgetting and present
normative information for them.

Memory impairment is a common neurobeha-
vioral abnormality associated with brain dys-
function, so memory testing is an especially im-
portant aspect of most neuropsychological eval-
uations (Delis, Cullum, Butters, Cairns, & Pri-
fitera, 1988; Wiens, Tindall, & Crossen, 1994).
The California Verbal Learning Test (CVLT;
Delis, Kramer, Kaplan, & Ober, 1987) is an ex-
amination of verbal list acquisition and memory,
composed of 16 shopping items from four cate-
gories. The 16 shopping items on List A are read
to the subject over five sequential trials, with the
subject recalling as many items as possible after
each trial. After Trial 5, the subject is presented
with List B, an interference trial using a differ-
ent list of 16 items. Following a recall trial of
List B, the subject is again asked to recall the
words from List A; this is the Short Delay Free
Recall Trial (SDFR). The subject is then cued
with each of the four categories of words in List
A; this is the Short Delay Cued Recall Trial
(SDCR). After a 20–min delay, Long Delay
Free– (LDFR) and Long Delay Cued Recall
(LDCR) Trials are administered. Finally, the
subject is read a list of 40 shopping items and is
asked to indicate whether or not each item was
on List A, providing a measure of Recognition
memory.
The CVLT was developed not only to quan-
tify learning and memory abilities, but also to
qualitatively characterize components of learn-
ing and memory. The scoring system allows for
analysis of overall learning, as measured by the
total words recalled across Trials 1–5, as well as
enabling potentially important comparisons in-
volving Trial 1 versus Trial 5 and List B, SDFR
versus Trial 5 and SDCR, Trial 5 and LDFR,
and SDFR and LDFR. The subject’s Learning

*
This work was funded in part by National Institute of Mental Health Grant MH49550–01A1.
Address correspondence to: Marc A. Norman, Ph.D., Psychology service (116B), VA Medical Center, 3350 La
Jolla Village Drive, San Diego, CA 921226, USA. E-mail: mnorman@ucsd.edu
Accepted for publication: August 11, 1999.
 CVLT NORMS
Characteristics are further clarified in indices of
Semantic Clustering, Serial Clustering, Percent
Primacy Recall, Percent Middle Recall, Percent
Recency Recall, Learning Slope, and Recall
Consistency. Also scored are error types includ-
ing the number of Perseverations and Intrusions
on the Free Recall and Cued Recall trials. Fi-
nally, the number of Recognition Hits, Dis-
criminability, False Positive responses, and Re-
sponse Bias characterize Recognition memory.
Comparisons of these measures with LDFR re-
flect the presence/absence and degree of diffi-
culty with retrieval processes.
Downloaded by [Deakin University Library] at 16:08 30 October 2015
The commonly used norms presented in the
CVLT manual (Delis et al., 1987) consisted of a
combination of several independently collected
subject samples. It included 273 (104 male, 169
female) neurologically intact subjects. These
norms have been described as limited because
they are based upon a relatively small sample
size with less than desirable representativeness,
and thus caution is advised in their clinical use
(Delis et al., 1987; Elwood, 1995). Information
regarding ethnicity was not analyzed in the
CVLT norming process, but ethnic minorities
are believed to be poorly represented in this
sample (Kramer, personal communication, No-
vember, 1998). The lack of information regard-
ing the contribution of ethnicity may lead to
misclassification of memory impairment, espe-
cially within ethnic minority populations.
Most existing studies utilizing the CVLT
compare cerebrally compromised populations
and normal controls, reporting means and stan-
dard deviations; however, they fail to analyze
demographic contributors to CVLT performan-
ce. Without knowledge of possible demographic
contributors, subjects may appear ‘‘impaired’’
when they are not. In an attempt to expand the
normative data, Wiens, Tindall, and Crossen
(1994) analyzed CVLT performance of 700 (568
male, 132 female) successive job applicants for
the Civil Service. This sample, including 596
Caucasians (85%), 39 African Americans
(5.6%), 30 Asians (4.3%), 24 Hispanics (3.4%),
and 11 Native Americans (1.6%), performed
slightly lower on the number of words they re-
called on the learning trials when compared to
the Delis et al. (1987) sample. The authors did
81
not provide data regarding performances of the
separate ethnic groups, although analyses of
possible ethnicity effects reportedly did not
yield significant results. Limitations of this
study are that the subjects were relatively young
(M = 29.1), well educated (M = 14.5 years), and
mostly male (81%), and that quite small num-
bers of subjects were included within individual
ethnic minority groups. Only 6% of the popula-
tion were over the age of 40. This study did not
offer comprehensively presented corrections for
age and gender, which were found to signifi-
cantly contribute to CVLT performance.
To extend the CVLT norms for older individ-
uals, Paolo, Troster, and Ryan (1997) used over-
lapping age ranges to provide data on the 26
CVLT indices for persons between 53 and 83. In
contrast to the Wiens et al. (1994) study, Paolo
et al. presented clinically useful norms tables.
However, this normative sample is even more
limited regarding inclusion of ethnic minorities
as only 4 of the 212 subjects were minorities (2
African Americans and 2 Hispanics).
In general, previous CVLT normative studies
have yielded consistent findings with regard to
gender. Gender has been reported to signifi-
cantly contribute to CVLT performance, with
females generally performing better on immedi-
ate and delayed free recall trials, as well as on
semantic clustering (Delis et al., 1987; Kramer,
Delis, & Daniel, 1988). Delis et al. (1987) and
Kramer et al. (1988) found that males and fe-
males did not differ significantly on Recognition
Hits. In contrast, Paolo (1997) and Wiens (1994)
found gender differences favoring females on
Recognition Hits, False Positive errors, and
Discriminability, as well as on free recall trials.
In addition to gender, age consistently has
been found to significantly influence CVLT per-
formance. Delis et al. (1987) and Wiens et al.
(1994) found that subjects’ CVLT performances
generally worsened with advancing age. How-
ever, not all measures of performance may be
influenced by advancing age. For example, Kra-
mer, Blusewicz and Preston (1989) found that
performances on Trial 1 and recognition were
essentially unrelated to age.
Reports on the effects of education on CVLT
performance have been less consistent. Paolo et
 82 MARC A. NORMAN ET AL.
al. (1997) found that education affected only
14% of the CVLT indices and always contrib-
uted the least in a multiple regression paradigm.
Delis et al (1987) and Wiens et al. (1994) did
not address the effects of education on CVLT
performance.
Although ethnicity may influence neuropsy-
chological test performance, little effort has
been put forth to characterize neuropsychologi-
cal test performance for minority groups (Manly
et al., 1998). Roberts and Hamsher (1984), using
a standard cutoff, found that 22% of African
Americans would have been misclassified as
Downloaded by [Deakin University Library] at 16:08 30 October 2015
impaired on their naming performance. Ethnic
group differences have been demonstrated on
the WAIS-R (Kaufman, McLean, and Reynolds,
1988), Category Test (Bernard, 1989), and tasks
of simple and choice reaction time (Miller,
Bing, Selnes, Wesch, and Becker, 1993). Pre-
liminary data (Norman, Evans, Miller, Delis,
and Heaton, 1997; Evans et al., 1999) showed
that African American subjects performed out-
side of the standards set by Caucasian normative
subjects on many of the CVLT indices. The
presence and severity of neurocognitive deficits
may be overestimated within minority popula-
tions due to nonrepresentative normative data.
The present large scale normative study was de-
signed to assess the contribution of influences of
ethnicity as well as other demographic factors
on CVLT performance. Additionally, predicted
discriminability and predicted Short-Delay re-
call and Long-Delay recall were calculated and
used to generate a Retrieval Index as well as
indices of Short-Delay Forgetting and Long-De-
lay Forgetting.
METHODS
Subjects
Subjects included in the present investigation were
normal, community dwelling volunteers recruited
from five separate research studies. One of these
studies was specifically designed to develop neu-
ropsychological norms for African Americans
(Miller et al. in press); another was the original
CVLT normative study (Delis et al., 1987), while
the remaining three studies collected normal con-
trol data for comparison with that of various clini-
cal populations. All participants were screened to
exclude those with any history of significant
trauma or disease involving the brain. Specifically,
in all contributing studies exclusion criteria in-
cluded (a) meeting DSM-III-R criteria for sub-
stance dependence within the last year; (b) current
therapeutic use of psychoactive drugs; (c) history
of serious mental illness (i.e., schizophrenia, bipo-
lar disorder, or major depression); (d) history of
head trauma with loss of consciousness greater
than 5-minutes and/or any head injury which re-
sulted in persisting neurological signs; and (e) his-
tory of diagnosed brain disease or brain surgery.
Two hundred and eighty six subjects from the
original normative sample for the CVLT were used
in the present analysis (32% of the current study’s
total sample). Some of these subjects were ex-
cluded from the original normative analysis be-
cause they were especially young (<17 years) or
old (>80). 339 African American volunteers who
were seen specifically to generate normative data
on an expanded Halstead-Reitan neuropsychologi-
cal test battery were also included in the sample
(37%). Subjects in this normative study were se-
lected from the San Diego, California community
using a census based survey method designed to
select a representative sample of neurologically
normal subjects from the area. In addition, normal
comparison subjects were used from ongoing re-
search studies of various neurological and psychi-
atric disorders at the University of California, San
Diego School of Medicine Geriatric Psychiatry
Clinical Research Center (n = 141; 16%), Alzhei-
mer’s Disease Research Center (n = 59; 7%), and
the HIV Neurobehavioral Research Center (n = 81;
9%). All subjects were able to speak and under-
stand both written and spoken English, and were
paid for their participation in the component stud-
ies. The combined sample of participants for the
current study totaled 906.
The subjects in the combined group ranged in
age from 17 to 101 (M 51.1, SD = 20.6). About
half (55.7%) were female. 61% (n = 549) were
non-Hispanic Caucasian, and 39% (n = 357) were
African American. Subjects in other ethnic minor-
ity groups were excluded because they were not
available in sufficient numbers to provide valid
estimates of performances of the minority popula-
tions they represent. In all, 362 (40%) of our sub-
jects were age 60 or older. Education ranged from
1 to 20 years (M = 13.72, SD = 2.62). 29.8% of
subjects had a college education or better (16+
years), 29.5% had some college (13-15 years),
25.3% had a high school education (12 years), and
15.5% had fewer than 12 years of formal educa-
tion.
 CVLT NORMS
Education and age distributions were different
in our African American and Caucasian samples,
due to sampling differences between the various
studies from which these subjects were drawn.
Specifically, Caucasian subjects were older (M =
60.3, SD = 18.9, range = 17-101) than African
American subjects (M = 36.9, SD =14.1, range =
17-65, p < .001), and reported a higher mean level
of education (M = 14.1 years, SD = 2.7, range = 1-
24) than the African American sample (M = 13.2
years, SD = 2.4, range = 9-20; p < .001). Although
most participants had at least 12 years of educa-
tion, 24.9% of the African American sample had
not graduated from high school, compared to only
Downloaded by [Deakin University Library] at 16:08 30 October 2015
9.3% of the Caucasian sample.
Measures
California Verbal Learning Test (CVLT)
Each subject completed the California Verbal
Learning Test (CVLT), administered in the stan-
dard manner by trained psychometrists as part of a
larger neuropsychological examination. The fol-
lowing raw score measures were used to develop
demographically corrected T-scores: Monday List
Trial 1 recall, Monday List Trial 5 recall, Monday
List Total Trials 1-5, Tuesday List (List B) recall,
and Monday List Short Delay Free Recall and
Long Delay Free Recall. The distributional proper-
ties of the Recognition Discriminability measures
were significantly skewed; however, impairment
cut-off scores were calculated with age and educa-
tion corrections.
Analyses
Base and Validation subject samples
Generation of normative equations generally fol-
lowed procedures described in Heaton, Grant, and
Matthews (1991). First, the subject sample was
split into Base (n = 672) and Validation (n = 234)
subsamples, using a random selection procedure.
Due to the fact that relatively few African Ameri-
cans in the entire sample were over the age of 60
(n = 24), all of these older subjects were placed in
the Base sample to improve the resulting predic-
tion equations’ generalizability with regard to age
effects on CVLT performance. To verify that the
resulting groups were comparable on demographic
characteristics, simple t tests and Chi-square anal-
yses compared the groups on demographic charac-
teristics (age, education, and gender).
Generation and Validation of Prediction Equa-
tions
Next, we generated prediction equations using the
83
data from the Base sample (see Appendix). The
dependent variables used were the measures de-
scribed above, transformed into normally-distrib-
uted scaled scores (M = 10, SD = 3; see Table 2).
Stepwise multiple regressions, predicting the indi-
vidual scaled score from demographic variables
(age, education, gender, and ethnicity) as well as
non-linear components (age-squared, age-cubed,
education-squared, education-cubed) and interac-
tion terms (age by education, age by gender, age
by ethnicity for example) were performed to screen
for variables that did not significantly and
uniquely contribute to the prediction equation;
these were then removed. To be considered signifi-
cant, these variables had to contribute at least one
percent of the variance of the scaled scores in the
base sample. Multiple regression analyses with all
remaining demographic variables were then re-en-
tered in a stepwise fashion to predict the various
CVLT scaled scores. This prediction equation was
then used to generate predicted CVLT scores for
each subject. These predicted scores were then
subtracted from each subject’s actual score to cal-
culate a residual score. Finally, these residual
scores were transformed to T-scores (M = 50, SD =
10) according to the following formula: T-score =
{[(residual score/standard error of estimate for the
regression equation) x 10] + 50} (Heaton et al.,
1991).
Following the procedures outlined in Heaton et
al. (1991) ages between 17 and 34 were coded as
34 in the present analysis. This was due to the fact
that there were no apparent age effects seen in neu-
ropsychological performance in early adulthood,
and using actual age in the younger group might
attenuate the linear relationships seen between age
and test scores in the remaining (older) sample.
The current study supported the hypothesis that no
linear or curvilinear age effects were noted prior to
age thirty-four.
We then applied the T-score conversions to the
data from the Validation sample. To determine
whether the demographic influences in the Valida-
tion sample CVLT-scores had been removed, we
computed bivariate correlations between the com-
puted CVLT T-scores and demographic variables
(age, education, gender, and ethnicity) in both the
Base and Validation samples. In addition, to deter-
mine whether these T-score conversions were
functioning equivalently at various levels of the
demographic variables, we split the subjects into
three age groups (<40, 40 – 60, 60 +) and three
education groups (<12, 12-15, 16+), and computed
ANOVAs to look for significant main effects of
age, education, gender, or ethnicity, as well as sig-
nificant interaction effects on CVLT T-scores
 84 MARC A. NORMAN ET AL.
which would indicate that the demographic correc-
tions were not sufficient or were not equivalent
across various levels of the demographic variables.
In addition, the Kolmogorov-Smirnov procedure
was used to determine if the resulting T-score dis-
tributions differed significantly from normality.
Measures of predicted recognition discrimina-
bility performance and rates of predicted SDFR
and LDFR were then calculated. Predicted Recog-
nition Discriminability performance was calcu-
lated in a regression formula with LDFR as the
predictor variable. The predicted rate of short de-
lay recall was calculated using the Trial 5 raw
score as the predictor variable and SDFR raw score
Downloaded by [Deakin University Library] at 16:08 30 October 2015
as the dependent variable in a regression formula.
The same calculation was then applied with Trial 5
to predict LDFR. Next, discrepancy scores were
obtained by subtracting predicted scores from the
obtained scores to estimate retrieval, Short-Delay
forgetting, and Long-Delay forgetting. These dis-
crepancy scores were then converted to percentile
ranges.
RESULTS
Base and Validation Subject Samples and
CVLT Raw Scores
Table 1 summarizes the demographic variables
and CVLT raw scores for the Base and Valida-
tion samples. The two samples did not differ on
age, education, or gender. However, there was a
slightly greater percentage of African American
subjects in the Base sample due to our sampling
adjustment. In addition, there were no differ-
ences noted in CVLT performance across the
two samples.
In the combined group, there were significant
demographic influences on the raw score CVLT
performances. A 3 (age group) × 3 (education
group) × 2 (ethnicity) × 2 (gender) analysis of
variance (ANOVA) revealed significant main
effects of age, education, gender, and ethnicity
on the total words recalled for Trials 1-5, Trial
1, Trial 5, List B, Short Delay Free Recall, and
Long Delay Free Recall, such that younger per-
sons, those with more advanced education, Cau-
casians, and females tended to perform better. In
addition, a significant education by ethnicity
interaction was found only on Long Delay Free
Recall (p = .03).
Development of the Prediction Equation and
Cross-Validation
The raw to scaled score conversions are pre-
sented in Table 2. Stepwise multiple regressions
in the Base sample were used to predict the vari-
ous CVLT scaled scores from demographic vari-
ables (age, education, gender, and ethnicity).
Nonlinear components (e.g., age squared, age
cubed, education squared, education cubed, and
the interaction terms) were analyzed, but did not
significantly contribute to the prediction equa-
tions. The basic demographic variables all con-
tributed significantly to prediction of CVLT per-
formance; together they accounted for 18.8% of
the variance on Trial 1 scaled scores (F(4, 667)
= 39.84, p <.0001); 29.3% for Trial 5 (F(4, 667)
= 70.6, p <.0001); 31.9% for Total Words Trials
1-5 (F(4, 667) = 79.4, p <.0001); 17.2% for List
B (F(4, 667) = 35.8, p <.0001); 26.4% for Short
Delay Free Recall (F(4,667) = 61.3, p <.0001);
and 27.4% Long Delay Free Recall (F(4, 667) =
64.2, p <.0001). The prediction equations are
presented in Appendix.
Application of the T-score formulae to the
Validation sample demonstrated the generali-
zability of the derived equations. As expected,
the mean T-scores did not differ significantly
from 50 (see Table 3). Table 4 shows the fre-
quency distributions of the T-scores on Total
Words Recalled on Trials 1 to 5, for the Base,
Validation, and Total Samples, as compared to
what would be expected from a normal distribu-
tion. Neither sample differed significantly from
predictions based on assumptions of a normal
distribution (Kolmogorov-Smirnov p > .05).
Similar results were obtained for all other CVLT
T-score distributions; that is, none differed sig-
nificantly form normal for the base, validation,
or total samples.
Table 5 presents the proportion of all subjects
within clinically relevant T-score ranges for To-
tal Words Recalled on Trials 1-5, as suggested
by Heaton et al. (1991). In that study, a one-
standard deviation cutoff was used to define
‘‘impairment’’ which results in an expected
84.4% specificity rate in neurologically normal
subjects. The total sample had an ‘‘impairment’’
rate (T-score less than 40) of 16.7%, which is
very close to what would be predicted on the
 Downloaded by [Deakin University Library] at 16:08 30 October 2015

Table 1. CVLT Raw Scores.

Age n Education Total Words Trial 1 Trial 5 List B SDFR LDFR

(years) (years)

M (SD) M (SD) M (SD) M (SD) M (SD) M (SD) M (SD)

Base Sample
< 40 222 13.38 (2.45) 56.13 1(9.45) 7.76 (2.06) 13.08 (2.16) 7.26 (2.16) 11.69 (2.69) 12.05 (2.78)
40-50 184 13.90 (2.94) 52.42 (10.20) 7.26 (2.05) 12.28 (2.37) 6.60 (2.15) 10.77 (3.08) 11.02 (3.09)
60+ 266 13.79 (2.56) 46.37 (11.11) 6.29 (2.13) 11.06 (2.69) 6.07 (2.07) 19.00 (3.12) 19.35 (3.25)
Total 672 13.69 (2.64) 51.25 (11.14) 7.04 (2.18) 12.06 (2.58) 6.61 (2.18) 10.38 (3.19) 10.70 (3.27)
Validation Sample
<40 190 13.74 (2.30) 55.47 1(8.75) 7.41 (1.94) 13.33 (1.85) 7.01 (2.01) 11.83 (2.76) 12.29 (2.80)
40-50 148 14.27 (2.20) 54.60 (10.72) 8.17 (2.18) 12.19 (2.55) 7.23 (2.24) 10.63 (3.27) 11.19 (3.24)
60+ 196 13.60 (2.96) 46.31 (12.14) 6.43 (2.39) 11.04 (2.67) 5.85 (2.22) 18.90 (3.64) 19.31 (3.89)
Total 234 13.79 (2.58) 51.53 (11.47) 7.16 (2.27) 12.16 (2.56) 6.58 (2.26) 10.38 (3.49) 10.84 (3.61)

Note. SDFR = Short Delay Free Recall; LDFR = Long Delay Free Recall.
CVLT NORMS
85
 86 MARC A. NORMAN ET AL.

Table 2. Raw Score Conversion to Scale Score Conversions for CVLT Variables.

Scaled Score Total Words Trial 1 Trial 5 List B SDFR LDFR

19
18 >13
17 >71 12 >12
16 69 – 70 11 11 16 16
15 67 – 68 10 16 10 15 15
14 63 – 66 15 9 14
13 60 – 62 9 14 13 14
12 57 – 59 8 8 12 13
11 53 – 56 13 7 11 12
10 49 – 52 7 12 6 10 10 – 11
9 45 – 48 6 11 5 9 9
Downloaded by [Deakin University Library] at 16:08 30 October 2015

8 41 – 44 10 8 8
7 37 – 40 5 9 4 6–7 6–7
6 32 – 36 4 8 5 5
5 28 – 31 3 7 3 3–4 3–4
4 24 – 27 1–2 6 2
3 17 – 23 5 1–2 2
2 14 – 16 0 4 1 0–1
1 <13 0–3 0 0

Note. SDFR = Short Delay Free Recall; LDFR = Long Delay Free Recall.

basis of the theoretical normal distribution (i.e.,
15.6%). The proportions of subjects within clini-
cally relevant T-score ranges for Trial 1, Trial 5,
List B, SDFR, and LDFR are presented in Table
5.
Validation of the Prediction Equation
In order to determine if the influence of demo-
graphic factors had been removed, and to assess
whether the corrections were functioning equiv-
alently at different levels of the demographic
variables, a series of 3 (age group) × 3 (educa-
tion group) × 2 (gender) × 2 (ethnicity)

Table 3. T Scores and Standard Deviations for Base-, Validation-, and Total Samples.

Base Validation Total

(n = 672) (n = 234) (n = 906)

M (SD) M (SD) M (SD)

Age (years) 51.21 (20.31) 53.32 (18.28) 51.06 (20.63)
Education (years) 13.69 1(2.64) 13.79 1(2.58) 13.72 1(2.62)
Total Words 50.00 1(9.97) 49.66 (10.12) 50.07 (10.00)
Trial 1 50.00 1(9.97) 50.27 (10.12) 49.92 1(9.94)
Trial 5 50.00 1(9.97) 49.67 1(9.85) 49.91 (10.01)
List B 50.00 1(9.97) 49.36 (10.14) 49.84 (10.01)
SDFR 50.00 1(9.97) 49.48 (10.77) 49.86 (10.18)
LDFR 50.00 1(9.97) 50.14 (11.46) 50.04 (10.37)

% Female 55.5 56.4 55.7

% Caucasian 58.6 66.2 60.6
 CVLT NORMS
Table 4. Predicted And Actual Percentages Of T Scores For Total Words Recalled For Trials 1-5 Within Seven
T Score Ranges.
T score range Predicted % Base
0-24 10.62 10.3
25-34 16.05 16.5
34-44 24.15 23.8
45-54 38.29 39.6
55-64 24.20 23.4
65-74 16.07 15.5
75+ 10.62 10.9
Downloaded by [Deakin University Library] at 16:08 30 October 2015
ANOVAs were conducted on the T-scores in
both the Validation and total samples. There
were no significant main or interaction effects,
indicating that the T-score conversions were
similarly effective at all levels of age, education,
gender, and ethnicity.
Finally, in order to more fully examine the
diagnostic accuracy of the demographically cor-
rected T-score formulae, we compared the spec-
ificity rates in the African American subsample,
with and without ethnicity adjustments (i.e.,
without adjustments means treating the African
American subjects as if they were Caucasian).
First, when we applied a standard cutoff of less
than a T-score of 40 to the Delis et al. (1987)
norms, 46% of the African American subjects
were classified as impaired based on the total
number of words they recalled on Trials 1-5.
This number is significantly greater than what
would be expected (e.g. 15.6%) within the nor-
mal distribution (P2 (1, N =357) = 249.57, p <
.001). In comparison, when we applied the Delis
et al. (1987) norms to our Caucasian sample that
was not part of the original normative group,
only 16.2% were classified as impaired (P2 (1, N
= 281) = 1.39, p = .24).
When our own corrections for age, gender,
and education, but not ethnicity were applied,
36% of the African American subjects were
classified as impaired. Again, this is signifi-
cantly greater than would be expected under the
normal curve (P2 (1, N = 357) = 108.18, p <
.001). These results clearly demonstrate the un-
acceptable false positive error rates for African
Americans on CVLT without correcting for age,
87
Validation Total
10.9 10.4
17.7 16.8
22.2 23.4
42.3 40.3
19.7 22.4
16.8 15.8
10.4 10.8
gender, education, and ethnicity. After the eth-
nicity correction was applied to the equation,
17.78% of the African American subjects were
classified as impaired, approximating what
would be expected within a neurologically nor-
mal population; that is, these results do not dif-
fer significantly from the expected false positive
error rate of 15.6% (P2 (1, N = 357) = 1.14 , p =
.29).
Table 6 shows the results of these compari-
sons for all six CVLT measures. In each case,
the percent of African Americans classified as
impaired differs significantly from expectations
and is unacceptably high, when ethnicity correc-
tions are not applied. After ethnicity corrections,
however, the false positive error rates do not
differ significantly from expectations based, on
a one standard deviation cutoff with a normal
distribution, on five of the six measures. The
only exception was on Short Delay Free recall.
Although the false positive rate for this measure
exceeded normal expectations by 5.7 percent,
this still represents a substantial improvement
over the 15.7 percent with the uncorrected score.
Recognition discriminability was signifi-
cantly skewed; however demographic variables
(e.g. age, education, gender, and ethnicity) were
examined to assess their relative contribution to
discriminability performance. Gender (r =
–.094) and ethnicity (r = .033) accounted for
less than one percent of the variance, but age
and education accounted for 10% and 2 % of the
variance, respectively. Table 7 presents Recog-
nition Discriminability impairment cutoff
scores, using one standard deviation, were cal-
 88 MARC A. NORMAN ET AL.

Table 5. Predicted and actual percentages of T scores within clinically relevant ranges for Trials 1 – 5, Trial 1, Trial 5, List
B, SDFR, and LDFR.

T Score Range Classification Predicted Base Validation Total

Trials 1 – 5 0 – 19 Severe 0.13 0.1 0.0 0.1

20 – 24 Moderate to Severe 0.49 0.1 0.9 0.3
25 – 29 Moderate 1.60 1.8 0.9 1.5
30 – 34 Mild to Moderate 4.45 4.8 6.8 5.3
35 – 39 Mild 8.95 10.0 8.1 9.5
40 – 44 Below Average 15.20 13.8 14.1 13.9
45 – 54 Average 38.29 39.6 42.3 40.3
55 + Above Average 30.89 29.8 26.9 29.0
0 – 39 Total Impaired 15.60 16.8 16.7 16.7

Trial 1 0 – 19 Severe 0.13 0.0 0.0 0.0

20 – 24 Moderate to Severe 0.49 0.1 0.4 0.2
Downloaded by [Deakin University Library] at 16:08 30 October 2015

25 – 29 Moderate 1.60 0.7 0.4 0.7

30 – 34 Mild to Moderate 4.45 4.8 5.6 5.0
35 – 39 Mild 8.95 11.8 9.0 11.0
40 – 44 Below Average 15.20 16.2 17.1 16.4
45 – 54 Average 38.29 36.3 33.3 35.5
55 + Above Average 30.89 30.1 34.2 31.4
0 – 39 Total Impaired 15.60 15.6 17.4 16.9

Trial 5 0 – 19 Severe 0.13 0.1 0.0 0.1

20 – 24 Moderate to Severe 0.49 0.7 0.0 0.6
25 – 29 Moderate 1.60 1.3 2.6 1.7
30 – 34 Mild to Moderate 4.45 4.3 4.3 4.3
35 – 39 Mild 8.95 9.7 12.4 10.4
40 – 44 Below Average 15.20 14.0 12.0 13.5
45 – 54 Average 38.29 40.0 39.3 39.8
55 + Above Average 30.89 29.8 26.5 29.7
0 – 39 Total Impaired 15.60 15.6 19.3 17.1

List B 0 – 19 Severe 0.13 0.3 0.9 0.4

20 – 24 Moderate to Severe 0.49 0.3 0.4 0.3
25 – 29 Moderate 1.60 1.9 0.4 1.5
30 – 34 Mild to Moderate 4.45 4.6 7.3 5.3
35 – 39 Mild 8.95 7.6 6.8 7.4
40 – 44 Below Average 15.20 13.4 15.8 14.0
45 – 54 Average 38.29 42.1 41.5 41.9
55 + Above Average 30.89 29.8 26.9 29.0
0 – 39 Total Impaired 15.60 15.6 14.7 14.9

SDFR 0 – 19 Severe 0.13 0.1 0.4 0.2

20 – 24 Moderate to Severe 0.49 0.3 0.9 0.4
25 – 29 Moderate 1.60 1.6 1.7 1.7
30 – 34 Mild to Moderate 4.45 4.6 6.8 5.2
35 – 39 Mild 8.95 10.7 12.0 11.0
40 – 44 Below Average 15.20 14.7 10.7 13.7
45 – 54 Average 38.29 35.0 33.8 34.7
55 + Above Average 30.89 32.9 33.8 33.1
0 – 39 Total Impaired 15.60 17.3 21.8 18.5

LDFR 0 – 19 Severe 0.13 0.3 0.0 0.2

20 – 24 Moderate to Severe 0.49 0.3 0.9 0.4
25 – 29 Moderate 1.60 1.2 2.1 1.4
30 – 34 Mild to Moderate 4.45 4.0 7.7 5.0
35 – 39 Mild 8.95 10.9 9.8 10.6
40 – 44 Below Average 15.20 14.4 12.8 14.0
45 – 54 Average 38.29 39.1 33.8 37.7
55 + Above Average 30.89 29.8 32.9 30.6
0 – 39 Total Impaired 15.60 16.7 20.5 17.6
 CVLT NORMS 89

culated correcting for age (<40, 40 –60, and
60+) and education (<12, 12 – 15, and 16+) (Ta-
ble 7).
Retrieval Index, Short-Delay Forgetting In-
dex, and Long-Delay Forgetting Index
Formulae for calculating predicted Recognition
Discriminability, predicted SDFR, and predicted
LDFR are presented in the Appendix. These pre-
dicted values were then compared to the sub-
jects’ obtained Discriminability, SDFR, and
LDFR values, to yield indexes of Retrieval,
Short-Delay Forgetting, and Long-Delay Forget-
Downloaded by [Deakin University Library] at 16:08 30 October 2015
formance. Our large subject sample (n = 906)
not only included approximately equal numbers
of males and females at various levels of age
and education, but also a large number of Afri-
can Americans. Multiple regression analyses
using a Base sample of 672 subjects revealed
significant contributions of age, gender, educa-
tion, and ethnicity on the total words recalled on
the CVLT learning trials, as well as on Trial 1,
Trial 5, List B, Short Delay Free Recall, and
Long Delay Free Recall. Predication equations
were then calculated to generate demographi-
cally corrected T-scores. These T-scores were

ting. For the three index scores, age accounted
for 2% of the variance for Short-Delay Forget-
ting (r = –.141) and Long-Delay Forgetting (r =
–.129). All other demographics accounted for
less than one percent of the variance. Raw
scores were categorized into five percentile dis-
tributions (e.g., > 16, 16th though 11th, 10th
through 6th, 5th through 2nd, and less than or
equal to 1st ) (Tables 8 and 9). Although the
three discrepancy distributions were normally
distributed, it was decided that providing per-
centile ranges would accurately reflect ranges of
impaired performance.
DISCUSSION
This study examined the contributions of age,
gender, education, and ethnicity on CVLT per-
successfully applied to a Validation sample of
234 separate subjects. The successful validation
of these T-scores supports the generalizability of
these corrected CVLT performances at the vari-
ous levels of all demographic variables. In addi-
tion to the demographically corrected T-scores,
predicted performances of discriminability,
SDFR, and LDFR were calculated. These pre-
dicted values were then compared to obtained
values to yield measures of verbal retrieval,
Short-Delay forgetting, and Long-Delay forget-
ting.
Two findings within this sample population
were consistent with previous research reported
in the CVLT manual (Delis et al., 1987), as well
as in Wiens et al. (1994), and Paolo et al.
(1997): females generally performed better than
males, and performance decreased with age. In
addition, however, in the present study educa-

Table 6. Percentage of the total African American Sample Classified as Impaired Based Upon Six CVLT Mea-
sures, With and Without Correction for Ethnicity.

Without ethnicity correction With ethnicity correction

% ‘Impaired’ P2 % ‘Impaired’ P2
Expecteda 15.6 15.6
Trials 1-5 35.6 108.18** 17.7 1.14
Trial 1 39.5 154.83** 14.0 .69
Trial 5 36.1 114.33** 15.1 .06
List B 31.9 72.33** 14.4 .47
SDFR 31.3 67.45** 21.3 8.77*
LDFR 31.7 69.87** 18.8 1.47

a
Chi square compares obtained percent with the 15.6 percent expected using a one standard deviation cutoff and
assuming a normal distribution.
* p = .003; ** p < .001.
 90 MARC A. NORMAN ET AL.

Table 7. Impairment Cutoffsa for Recognition Discriminability Corrected for Age and Education.

Education

<12 years 12 – 15 years 16+ years

Age
< 40 89% 91% 93%
(n = 64) (n = 166) (n = 82)
40 – 59 84% 89% 91%
(n = 35) (n = 122) (n = 74)
60 – 69 73% 89% 89%
(n = 16) (n = 97) (n = 44)
70 – 79 84% 84% 84%
(n = 13) (n = 76) (n = 37)
Downloaded by [Deakin University Library] at 16:08 30 October 2015

80+ 84% 77% 84%

(n = 11) (n = 33) (n = 30)

a
Impairment cutoffs are based on the 15.6 percent expected using one standard deviation cutoff.

tion was found to significantly contribute to
CVLT performance.
Overall, the contribution of education to
CVLT performance remains unclear. Education
effects were not reported in the CVLT manual
(Delis et al., 1987). Weins et al. (1994) did not
find that education significantly contributed to
CVLT performance and Paolo et al. (1997) did
not find consistent education effects in their
sample of elderly persons. In the latter study,
education was significantly related to only three
of the twenty-nine CVLT variables investigated.
Three of these variables (e.g. Total Words, Trial
5, and SDFR) were analyzed in the current
study, and in all cases education was found to
significantly contribute to CVLT performance.
It is possible that the more robust education ef-
fects in the present study are due to a broader
range of our subjects’ education levels (e.g., 140
subjects with less than high school and 267 with
at least a Bachelor’s degree).
An important factor in the present investiga-
tion was the ability of these norms to contribute
to the correct classification of normal memory
performance within the African American popu-
lation. The contribution of ethnicity to neuro-
psychological performance in this study was
consistent with previous studies of other neuro-
psychological measures (Kaufman et al., 1988;
Roberts & Hamsher, 1984). However, no prior
study has adequately addressed possible ethnic-
ity effects on the CVLT. Where previous studies
have consistently found age and gender to sig-
nificantly contribute to CVLT performance (De-

Table 8. Raw Score Conversion to Percentile Range for Retrieval Index.

Retrieval Indexa Percentile Rangeb

< 3.48 > 16
3.48 to 4.97 11 – 16
4.98 to 6.61 6 – 10
6.62 to 12.85 2–5
> 16.62 <1

a
Retrieval Index = obtained Discriminability (in Discriminability percent units) – predicted Discriminability
(Formula in Appendix).
b
Percent of people in the normative sample that obtained this discrepancy.
 CVLT NORMS 91

Table 9. Raw Score Conversion to Percentile Ranges for Short-Delay Forgetting and Long-Delay Forgetting

Short-Delay Forgettinga Long-Delay Forgettingb Percentile Rangec

> –1.71 > –1.50 >16
–1.86 to –1.71 –2.31 to –1.50 11 – 16
–2.76 to –1.87 –2.55 to –2.32 6 – 10
–3.85 to –2.77 –3.55 to –2.56 2–5
< –3.86 < –4.40 <1

a
Short-Delay Forgetting = obtained SDFR (in raw score units) – predicted SDFR (Formula in Appendix)
b
Long-Delay Forgetting = obtained LDFR (in raw score units) – predicted LDFR (Formula in Appendix)
c
Percent of people in the normative sample that obtained this discrepancy.
Downloaded by [Deakin University Library] at 16:08 30 October 2015

lis et al., 1987; Paolo et al., 1997; Wiens et al.,
1994), this study demonstrated the importance
of ethnicity on CVLT performance. Consistent
with the results of Roberts and Hamsher (1984),
our findings indicate that correcting for ethnicity
lowers the rate of normal African Americans
classified as impaired. When removing the cor-
rection on the learning trials, African Americans
were categorized as impaired over twice as fre-
quently as were Caucasians, and this discrep-
ancy was noted on the delayed memory trials as
well. False positive error rates of this magnitude
are unacceptable in any clinical work, and repre-
sent a major disservice to African Americans.
Present findings also suggest that Afri-
can–American subjects may frequently be inac-
curately characterized as impaired when using
the published CVLT norms, which reportedly
are based upon a primarily Caucasian sample. It
is important to note that these norms, as well as
our own, can only be confidently applied to the
ethnic groups on which they were derived. Clini-
cians and researchers should generally not use
norms based on one ethnic group with other eth-
nic populations without crossvalidation; alterna-
tively, if appropriate norms do not exist, avail-
able norms based on other groups should be
used with caution, and the limitations of the
norms should be acknowledged whenever inter-
pretations are offered. This is not only true for
just the CVLT, but for neuropsychological tests
generally (Heaton, Ryan, Grant, & Matthews,
1996). Further research is needed to clarify the
nature and causes of ethnic differences on this
and other neuropsychological tests, and to pro-
vide appropriate normative standards for each
group with which the tests will be used. Re-
searchers should make this a priority.
To expand the CVLT’s clinical and research
utility, three new indices were explored. The
Retrieval Index compared obtained Recognition
Discriminability with performance predicted
from results on contemporaneous assessment of
free recall. By comparing the obtained and pre-
dicted Recognition Discriminability values, it is
believed that researchers and clinicians will be
better able to quantify verbal retrieval deficits.
Higher positive discrepancies indicate that an
individual performed better on Recognition
Discriminability than would be expected given
their LDFR performance. The disproportionate
ability to correctly identify the words on Recog-
nition testing may reflect a deficit in verbal re-
trieval. By comparing predicted performance to
obtained values it is believed that clinicians and
researchers will be able to better assess free-re-
call relative to recognition performance.
Patterns of rapid forgetting have proven to be
important in differentiating subcortical and cor-
tical cognitive and memory profiles (e.g., Par-
kinson’s Disease and Alzheimer’s disease; But-
ters, Delis,& Lucas, 1995). To date, the litera-
ture has lacked normative data measuring CVLT
rates of forgetting. The current study attempted
to estimate both rates of short-delay and long-
delay forgetting by comparing actual perfor-
mance on short-delay and long-delay free recall
with performance that was predicted on the basis
on scores on the last learning trial (Trial 5). It is
expected that the difference between the ob-
 92 MARC A. NORMAN ET AL.
tained delayed recall and predicted delayed re-
call should not deviate substantially from zero.
When subjects’ actual performance is substan-
tially below predicted values, an atypically rapid
rate of forgetting may be present. Thus, these
normative data may assist clinicians and re-
searchers in measuring rates of forgetting.
Special consideration needs to be given when
interpreting the Retrieval Index and measures of
Short-Delay and Long-Delay Forgetting. These
discrepancy scores in particular need to be inter-
preted in light of the overall pattern of perfor-
mance. For example, if both the obtained and
Downloaded by [Deakin University Library] at 16:08 30 October 2015
predicted scores fall within the normal range,
but the retrieval index falls within the ‘‘im-
paired’’ range, it is questionable whether or not
even a subtle retrieval deficit exists. Similar
caution should be used in interpreting the forget-
ting indices. Finally, caution should be used in
using the age and education corrected discri-
minability cutoff scores because of the small
subgroup sizes for subjects with less than 12
years of education and subjects with greater than
59 years of age.
In conclusion, we again emphasize that the
lack of normative data on minority populations
has broad implications in making clinical as well
as research interpretations. Individuals may be
misdiagnosed, scientific conclusions may be
erroneously drawn, and improper social policies
may be implemented. Practitioners and research-
ers are urged to carefully consider addressing
these issues in working with minority individu-
als, especially given that normative data is lack-
ing for these populations.
REFERENCES
Bernard, L. C. (1989). Halstead–Reitan neuropsycho-
logical test performance of black, Hispanic, and
white young adult males from poor academic back-
grounds. Archives of Clinical Neuropsychology, 4,
267–274.
Butters, N., Delis, D.C., & Lucas, J.A. (1995). Clini-
cal assessment of memory disorders in amnesia
and dementia. Annual Review of Psychology, 46,
493 – 523.
Delis, D. C., Cullum, C. M., Butters, N., Cairns, P., &
Prifitera, A. (1988). Wechsler Memory Scale-Re-
vised and California Verbal Learning Test: Con-
vergence and divergence. The Clinical Neuropsy-
chologist, 2, 188-196.
Delis, D. C., Kramer, J., Kaplan, E., & Ober, B.
(1987). The California Verbal Learning Test. New
York: Psychological Corporation.
Elwood. R. (1995). The California Verbal Learning
test: Psychometric characteristics and clinical ap-
plication. Neuropsychology Review, 5, 173-201.
Evans, J.D., Norman, M.A., Miller, S.W., Kramer,
J.H., Delis, D.C., & Heaton, R.K. (1999). Age-,
education-, gender-, and race-corrected norms for
the California Verbal Learning Test. Journal of the
International Neuropsychological Society, 5, 110.
Heaton, R. K., Grant, I., & Matthews, C. G. (1991).
Comprehensive Norms for an Expanded Halstead-
Reitan Battery: Demographic Corrections, Re-
search Findings, and Clinical Applications.
Odessa, FL: Psychological Assessment Resources.
Heaton, R. K., Ryan, L., Grant, I., & Matthews, C. G.
(1996). Demographic influences on neuropsycho-
logical test performance. In I. Grant & K Adams
(Eds.) Neuropsychological assessment of neuro-
psychiatric disorders, 2nd Ed. (pp 141-163). New
York: Oxford University Press.
Kaufman, A. S., McLean, J. E., & Reynolds, C. R.
(1988). Sex, race, residence, region, and education
differences on the 11 WAIS–R subtests. Journal of
Clinical Psychology, 44, 231–248.
Kramer, J. H., Blusewicz, M., & Preston, K. (1989).
The premature aging hypothesis: Old before its
time? Journal of Consulting and Clinical Psychol-
ogy, 57, 257-262.
Kramer, J. H., Delis, D. C., & Daniel, M. (1988). Sex
differences in verbal learning. Journal of Clinical
Psychology, 44, 907-915.
Manly, J. J., Miller, S. W., Heaton, R. K., et al.
(1998). The effect of African-American accultura-
tion of neuropsychological test performance in nor-
mal and HIV-positive individuals. Journal of the
International Neuropsychological Society, 4, 291-
302.
Miller, E., Bing, E., Selnes, O., Wesch, J., & Becker,
J. (1993). The effects of sociodemographic factors
on reaction time and speed of information process-
ing. Journal of Clinical and Experimental Neuro-
psychology, 15, 66.
Miller, S. W., Heaton, R. K., Grant, I., & White, (in
press). African American Norms Supplement to
Heaton Comprehensive Halstead-Reitan Battery
Norms. Miami, FL: Psychological Assessment Re-
sources.
Norman, M. A., Evans, J. D., Miller, S. W., Delis, D.
C., & Heaton, R. K. (1997). Normative data on an
African American population on the California
Verbal Learning Test. Journal of the International
Neuropsychological Society, 31.
 CVLT NORMS
Paolo, A. M., Troster, A. I., & Ryan, J. J. (1997). Cal-
ifornia Verbal Learning Test: Normative Data for
the Elderly. Journal of Clinical and Experimental
Neuropsychology, 19, 220-234.
Roberts, R. J., & Hamsher, K. D. (1984). Effects of
minority status on facial recognition and naming
performance. Journal of Clinical Psychology, 40,
539-545.
Sattler, J. (1970). Racial experimenter effects in ex-
perimentation, testing, interviewing, and psycho-
therapy. Psychological Bulletin, 73, 137-160.
Downloaded by [Deakin University Library] at 16:08 30 October 2015
93
Sundberg, N., & Gonzales, L. (1981). Cross-cultural
and cross-ethnic assessment: Overview and issues.
In P. McReynolds (ed.) Advances in psychological
assessment. ( Vol. 5) San Francisco, CA: Jossey-
Bass Publishers.
Wiens, A. N., Tindall, A. A. G., & Crossen, J. R.
(1994). California Verbal Learning Test: A norma-
tive study. The Clinical Neuropsychologist, 8, 75-
90.
 94 MARC A. NORMAN ET AL.

APPENDIX

CVLT Norms Formulae

Demographically corrected T-scores for CVLT indices can be calculated as follows:

Trial 1 = 3.58(Trial 1 SS) – 3.12 + .323(age) – .369(education) + 3.35(gender) + 5.65(ethnicity).

Trial 5 = 4.04(Trial 5 SS) – 4.24 + .396(age) –.998(education) + 4.90(gender) + 5.99(ethnicity).

Trials 1-5 = 4.07(Trials 1-5 SS) – 10.8 + .443(age) –.748(education) + 5.08(gender) + 6.77 (ethnic-
ity).
Downloaded by [Deakin University Library] at 16:08 30 October 2015

List B = 3.82(List B SS) – 5.03 + .294(age) – .386(education) + 4.17(gender) + 5.799(ethnicity).

Short Delay Free Recall = 3.955(SDFR SS) – 4.13 + .372(age) – .815(education) + 4.48(gender) +
4.11(ethnicity).

Long Delay Free Recall = 4.17(LDFR SS) – 4.16 + .375(age) – .959(education) + 4.50(gender) +
4.42(ethnicity).

Predicted Discriminability = 1.492(LDFR Raw Score) + 77.268.

Predicted SDFR = 1.048(Trial 5 Raw Score) – 2.288.

Predicted LDFR = 1.048(Trial 5 Raw Score) – 1.930.

Note: SS = Scaled Score (see Table 2).

Discriminability = Obtained Recognition Discriminability

Age = true age (except that if age is 20-34, age is coded as 34).

Education = years of education successfully completed.

Gender: Female = 0, Male = 1

Ethnicity: Caucasian = 0, African American = 1.