This article was downloaded by: [University of South Florida]

On: 20 April 2013, At: 00:15

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Clinical and Experimental

Neuropsychology
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/ncen20

Construct validity of various verbal and

visual memory tests
a

Glenn J. Larrabee & Glenn Curtiss

a

Independent Practice

James A. Haley V. A. Medical Center

Version of record first published: 04 Jan 2008.

To cite this article: Glenn J. Larrabee & Glenn Curtiss (1995): Construct validity of various verbal and
visual memory tests, Journal of Clinical and Experimental Neuropsychology, 17:4, 536-547
To link to this article: http://dx.doi.org/10.1080/01688639508405144

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.

Journal of Clinical and Experimental Neuropsychology

1995, Vol. 17, NO.4, pp. 536-547

I380-3395/95/1704-536$6.00

0 Swets & Zeitlinger

Construct Validity of Various Verbal and

Visual Memory Tests*
Glenn J. Larrabee' and Glenn C u r t i s 2
'Independent Practice, and 'James A. Haley V.A. Medical Center

Downloaded by [University of South Florida] at 00:15 20 April 2013

ABSTRACT
Factor analysis was conducted on attention, information processing, verbal and visual memory scores of
I12 patients. Factor structure did not vary as a function of age. The Expanded Paired Associates Test,
Verbal Selective Reminding Test, Continuous Recognition Memory Test, and Continuous Visual Memory
Test defined a general memory factor. The PASAT, WMS Mental Control, and WAIS-R Digit Span defined an attentionhiformation processing factor. Immediate Visual Reproduction (VR) loaded primarily
on visuahonverbal intelligence, whereas delayed VR loaded primarily with the memory factor. The Trail
Making Test, Part B was more closely associated with visual/nonverbal intelligence than with attentionhnformation processing. Serial Digit Learning was more closely associated with attentionhnformation
processing than with general memory.

Comprehensive evaluation of learning and memory requires examination of multiple dimensions

including attention/concentration, information
processing rate, immediate memory, verbal and
visual learning and retention, and remote recall
(Cunningham, 1986; Erickson & Scott, 1977;
Loring & Papanicolaou, 1987; Mayes, 1986).
Memory test development has generally followed two strategies: (a) development of omnibus batteries such as the Wechsler Memory
Scale-Revised (Wechsler, I987), Memory Assessment Scales (Williams, 1991), and Denman
Neuropsychology Memory Scale (Denman,
1984); and (b) development of specialized individual tests of verbal memory, visual memory,
attention, and information processing rate, such
as the California Verbal Learning Test (Delis,
Kramer, Kaplan, & Ober, 1987), Verbal Selective Reminding Test (Buschke, 1973), Continuous Visual Memory Test (Trahan & Larrabee,
1988), Visual Spatial Learning Test (Malec,
Ivnik, & Hinkeldy, 1991), and Paced Auditory

Serial Addition Test (Gronwall, 1977).

Validation of memory batteries and specialized individual memory tests has involved administering these procedures to groups of patients known to have a high base-rate of memory
impairment (e.g., severe closed-head trauma) as
well as factor analysis, which is used to evaluate
the factor structure and construct validity of
the$e measures (Delis et al., 1987; Larrabee &
Crook, 1988; Trahan & Larrabee, 1988; Wechsler, 1987). To demonstrate construct validity,
measures of learning and memory should load
on factors which are distinct and separate from
measures of verbal and visuospatial reasoning
skills and separate from measures of attention,
concentration, and information processing speed
(Cunningham, 1986; Erickson & Scott, 1977;
Larrabee, 1987; Larrabee, Kane, Schuck, &
Francis, 1985). Cunningham (1986) has recommended programmatic, linked factor analyses
for improved identification of key memory variables and factors. Larrabee et al. (1985) have

We gratefully acknowledge the assistance of Lisa Meyer, Todd Snyder, and Christine Todd in data management, and Susan Towers in manuscript preparation. Portions of this paper were presented at the annual meeting
of the American Psychological Association, San Francisco, August, 1991. Requests for reprints should be sent
to Glenn J. Larrabee, Ph.D., 630 South Orange, Suite 202, Sarasota, FL 34236, USA.

Accepted for publication: November 28, 1994.

MEMORY TEST CONSTRUCT VALlDlTY

537

Downloaded by [University of South Florida] at 00:15 20 April 2013

suggested a framework for factor analytic evaluation of memory test construct validity employing marker variables for verbal intelligence,
visuospatial intelligence, attention, and concentration, as well as conducting separate factor
analyses for immediate versus delayed recall
scores. This second recommendation is particularly important, since including immediate and
delayed memory scores in the same factor analysis results in factors defined by method variance.
Hence, rather than representing a true construct,
these factors would be spurious, due to the high
intercorrelation between immediate and delayed
trials secondary to similar testing procedures for
identical stimulus materials (Smith, Malec, &
Ivnik, 1992).
Recent factor analyses of the WMS-R and
other collections of specialized memory assessment procedures which have factor analyzed
acquisition and delayed recall scores separately,
demonstrate two recurrent patterns of factor
loadings. First, verbal memory tests such as the
WMS or WMS-R Paired Associate Learning and
Logical Memory load on a general memory or
verbal memory factor that is independent of attention and verbal reasoning, irrespective of
whether the score is based on learning trials versus delayed recall (Larrabee et al., 1985;
Leonberger, Nicks, Goldfader & Larrabee,
1990; Leonberger, Nicks, Larrabee, & Goldfader, 1992; Smith et al., 1992). By contrast,
purported measures of visual memory such as
the WMS Visual Reproduction show a greater
association with visuospatial intelligence than
memory during the acquisition phase (i.e., immediate reproduction of a design following 10-s
exposure) and a stronger association with a
memory factor for delayed recall trials (Larrabee et al., 1985; Larrabee, Trahan, & Curtiss,
1992; Leonberger et al., 1990, 1992; Smith et
al., 1992). The above investigations have all
found a general memory factor rather than separate verbal and visual modality specific memory
factors, with the exception of Larrabee et al.,
(1992) who found separate verbal and visual
memory factors independent of factors for verbal and visuospatial reasoning and intelligence,
when delayed trials of WMS Visual Reproduction and the CVMT were factored.

Of additional interest is whether factor structure varies as a function of age. Age effects in
level of performance on memory tests are well
established. The effects of age on structure of
memory abilities have not been studied as extensively. Bornstein and Chelune ( 1989) evaluated
the factor structure of the WMS-R in three different age groups: 39 years or less, 40-55 years,
and 56 years or greater. The structure was quite
similar when IQ scores were not included, but
changed when IQ was included with extraction
of additional factors i n the younger age group.
Other investigators have used a different approach in which age is considered as a continuous variable that is regressed on memory test
scores, with factor analysis of the residuals.
These studies have found no effect of age on
factor structure (Crook & Larrabee, 1988; Delis,
Freeland, Kramer, & Kaplan, 1988) even when
intelligence scores were included (Larrabee et
al., 1992).
In the present study, we evaluate the factor
structure and construct validity of several specialized tests of memory and information processing including the following: Serial Digit
Learning (Serial Digits; Benton, Hamsher,
Varney, & Spreen, 1983), the Expanded Paired
Associate Test (EPAT; Trahan, Larrabee, Quintana, Goethe, & Willingham, 1989), Verbal Selective Reminding Test (VSRT; Buschke, 1973;
Hannay & Levin, 1985),Immediate and Delayed
Visual Reproduction from the WMS (Russell,
1975; Trahan, Quintana, Willingham, & Goethe,
1988),Continuous Recognition Memory (CRM;
Hannay, Levin, & Grossman, 1979), the CVMT
(Trahan & Larrabee, 1988), t h e PASAT
(Gronwall, 1977; Levin et al., 1987), and the
Trail Making Test (Reitan & Wolfson, 1985).
Marker variables were selected for verbal reasoning and intelligence (WAIS-R Vocabulary
and Information subtests), visuospatial reasoning and intelligence (WAIS-R Block Design and
Object Assembly subtests), and attentionhmmediate memory span (WMS Mental
Control and WAIS-R Digit Span), following the
design of Larrabee et al. (1 985) and providing a
programmatic link to this previous study (viz.
Cunningham, 1986). Stability of factor structure
relative to age was analyzed by factoring raw

538

GLENN J. LARRABEE A N D GLENN CURTISS

scores and scores residualized for age t o replicate previous research, which has yielded similar factor structure f o r age-adjusted an d raw
memory test scores (Crook & Larrabee, 1988;
Delis e t al., 1988; Larrabee e t a ] . , 1992).

METHOD

Downloaded by [University of South Florida] at 00:15 20 April 2013

Subjects
Fifty-six male and 56 female outpatients, ages 16 to
70 ( M = 39.34, SD = 13.19), and with 7 to 18 years of
education ( M = 12.65, SD = 3.09) were examined with
a comprehensive neuropsychological battery. Average
WAIS-R VIQ was 99.08 ( S D = 13.67), average PIQ
was 100.58 (SD = 14.00), and average FSIQ was
99.79 (SD = 13.27). Primary diagnoses included
Closed-HeadTrauma ( 3 3 ,Somatoform Disorder(21),
Depression ( 1 7), Seizure Disorder (6), Unspecified
Central Nervous System Disease (positive neurologic
findings with unclear etiology) ( 5 ) , Alcohol Abuse
( 3 ) ,Dementia ( 3 ) ,Pseudodementia (2), Posttraumatic
Stress Disorder (2), Adjustment Disorder (2), Hydrocephalus (2), Bipolar Disorder (2), Multiple Sclerosis
(2), Personality Disorder (2), Penetrating Head Injury
( I ) , Schizophrenia ( I ) , Bulimia ( I ) , Learning Disability ( I ) , Polysubstance Abuse ( I ) , Anterior Communicating Artery Aneursym ( 1 ), Narcolepsy ( I ) , and CVA
(1). All patients were seen in the senior authors private practice. Subjects were not included if they were
aphasic, had neglect, or demonstrated evidence of
motivational impairment (i.e., worse-than-chance performance on the CRM or CVMT, c.f. Larrabee, 1992;
invalid MMPI).

Procedures
The Information, Vocabulary, Digit Span, Block Design, and Object Assembly subtests were administered
and scored following procedures detailed in the
WAIS-R manual (Wechsler, 1981). The WMS Mental
Control subtest was administered and scored following the procedures outlined by Wechsler ( I 945).
The Serial Digit procedure requires the subject to
learn an 8- or 9-digit supraspan sequence over a maximum of 12 trials (Benton et al., 1983). The 9-digit
sequence is administered to subjects under the age of
65 who have 12 or more years of formal education,
and the 8-digit series is administered to subjects over
64 years of age or those who have less than 12 years
of education. Testing is terminated if two correct sequences are recalled prior to completion of 12 trials.
Scoring is based on full credit (2 points) for each correct sequence, and partial credit ( 1 point) for near
correct responses (ix., one digit is either omitted,
added, or substituted, or there is a simple reversal of
two adjacent digits). In the present analysis, the total
raw score points were utilized. This test is reported to

be particularly sensitive to bilateral brain disease

(Benton et al., 1983).
The EPAT is a modification of the original WMS
Paired Associate Learning subtest, wherein four additional low associational value word-pairs were added
(lampshade-sidewalk, crossroad-pillow, lawnmowerenvelope, automobile-scissors), as well as a 30-min
delayed recall trial (Trahan et al., 1989). The test
yields an acquisition or learning trial score (sum of
easy divided by 2 plus the sum of the hard pairs
recalled), and a delayed recall score (sum of easy
divided by 2 plus the sum of the hard word pairs
recalled). The EPAT i s normed on 306 adults ages I8
to 91, and is sensitive to severe closed-head injury,
CVA (in particular, left CVA), and presumptive Alzheimer-type dementia (Trahan et al., 1989).
The VSRT employed in the present analysis is
Form I of the version developed by Levin and colleagues (Hannay & Levin, 1985; Larrabee, Trahan,
Curtiss, & Levin, 1988). This form of the VSRT requires the subject to learn a list of 12 unrelated words
over a maximum of 12 trials. Unlike other verbal
learning and memory measures, the subject is not presented with the entire list (other than on the initial
trial) prior to each recall attempt. Rather, the subject
only hears those words repeated that he/she did not
recall on the preceding trial, yet the subject is still expected to give all of the words on the list. Administration is continued until three perfect recall trials are
achieved, without reminding, or for 12 trials. A variety of scores can be computed including Long-Term
Storage (LTS; a word is assumed to have entered LTS
if it is recalled at least once without reminding), Consistent Long-Term Retrieval (CLTR; a word is in
CLTR if it is recalled consistently, without reminding,
to criterion), and 30-min delayed free recall. In the
present analysis, the cumulative sum of words in
CLTR was employed as the acquisition variable, with
delayed recall based on free recall at 30 min. The version of the VSRT used in the present study has been
found to be sensitive to residual verbal memory impairment following closed-head injury (Levin,
Benton, & Grossman, 1982), and early stage Alzheimer-type dementia (Larrabee, Largen, & Levin,
1985).
The WMS Visual Reproduction subtest is from the
original WMS (Wechsler, 1945), with the Russell
( 1 975) procedure of immediate and 30-min delayed
reproduction (Trahan et al., 1988). Drawings were
scored following the criteria specified by Wechsler
( 1 945) and total points correct for the immediate and
30-min trials were used in the present analysis. Data
presented by Trahan et al. (1988) demonstrate an association of performance with age in 255 normal subjects, ages 18 to 91, with sensitivity of the procedure
to left and right CVA, severe closed-head injury, and
presumptive Alzheimer-type dementia.
The CRM was originally developed for evaluation
of the effects of closed-head injury on visual recogni-

Downloaded by [University of South Florida] at 00:15 20 April 2013

MEMORY TEST CONSTRUCT VALIDITY

tion memory (Hannay et at., 1979). The test employs

120 line drawings of living things and objects (e.g.,
flowers, sea shells, birds) as stimuli. There are eight
different categories of stimuli, each containing six
perceptually similar yet different exemplars, one of
which recurs while the others occur only once in the
120 cards. The subjects task is to identify the recurring stimuli as old and the nonrecurring stimuli as
new. For the present analysis, the total correct
score (correct identification of both old and
new stimuli) was employed for the acquisition
score. Although the original CRM did not include a
delayed recognition score, the procedure was modified for the present analysis by having the subject indicate, at 30-min delay, which of the six perceptually
similar exemplars had repeated throughout the deck of
cards. Hannay et al., (1979) and Hannay and Levin
(1989) report data demonstrating the sensitivity of the
CRM to the effects of varying degrees of head trauma
severity in adults and adolescents.
The CVMT was developed to incorporate five basic features: (a) a visual recognition memory format,
(b) use of complex ambiguous designs not easily susceptible to verbal labeling, (c) a large number of stimuli, including classes of perceptually similar stimuli,
(d) limited exposure time to each stimulus, and (e) a
delayed recognition test (Trahan & Larrabee, 1988;
Trahan, Larrabee, & Quintana, 1990). The CVMT has
I12 designs, seven of which are repeated six times.
Each of the repeated stimuli are from categories containing six perceptually similar stimuli that do not recur. The subject must identify the recurring stimuli as
old and the nonrecurring stimuli as new in a
manner similar to the CRM. In the present study, the
total correct score (correct identification of recurring
stimuli as old and nonrecurring as new) was
used for the acquisition measure, with the number of
stimuli correctly identified at 30-min delayed recognition representing the delayed score. The CVMT has
been demonstrated to be sensitive to the effects of amnestic disorder, severe closed- head injury, presumptive Alzheimer-type dementia, and right hemisphere
CVA (Trahan & Larrabee, 1988; Trahan et al., 1990).
The Trail Making Test was administered according
to instructions provided by Reitan and Wolfson
(1985). Part A was scored for time taken to complete
the connection of a scattered array of numbers, in increasing order of value. Part B was scored for the total
time taken to connect a scattered array of numbers and
letters in increasing, alternating order of value. This
procedure, in particular, Part B, i s well-established as
a sensitive measure of brain function (Reitan &
Wolfson, 1985). In the present study, the score for the
more complex Part B was employed.
The PASAT was developed as a measure of information processing rate (Gronwall, 1977). The task
employed in the present study utilized a tape recording of computer-synthesized speech requiring the subject to perform rapid serial addition across four blocks

539

of 50 numbers, with the time between numbers decreasing across the trial blocks from 2.4 s to 2.0, I .6,
and I.2 s. (Brittain, La Marche, Reeder, Roth, & Boll,
1991; Levin et al., 1987; Roman, Edwall, Buchanan,
& Patton, 1991). The score is based on the number of
correct additions for each trial block. In the present
analysis, the number correct for the most rapid presentation rate (numbers every 1.2 s Trial Block 4) was
utilized. The PASAT is quite sensitive to the residual
effects of concussion and mild closed- head injury
(Gronwall, 1977; Levin et al., 1987).
Scores from the above procedures were submitted
to two sets of factor analyses. Raw test scores were
factored in the first set of analyses and test scores
residualized for the effects of age were factored in the
second set. Within each set of factor analyses, separate analyses were conducted for the immediate and
delayed verbal and visual memory test scores. Variables remaining constant in the two sets of factor analyses included Information, Vocabulary, Digit Span,
Block Design, Object Assembly, Serial Digits, Mental
Control, Trail Making Test, Part B, and PASAT trial
4. As noted earlier, only one score was selected from
the PASAT and Trail Making Test. This was done in
order to avoid factors determined by method variance
(cf. Larrabee et al., 1985; Smith et al., 1992).

RESULTS
Test scores were submitted to principal axis factor analysis, utilizing squared multiple correlations as initial communality estimates. T h e Kaiser Guttman criterion and scree plot were consistent in suggesting a four-factor solution
(Gorsuch, 1983).
For the immediate recall raw score analysis,
the eigenvalues for the first four components in
the unrotated solution were 4.93, I .87, I .32, and
1.24. The final, varimax-rotated principal axis
solution is displayed in Table 1. The eigenvalues
for the varimax rotation are 2.22 for factor I ,
2.10 for factor 2 , I .56 for factor 3, and 1.48 for
factor 4,accounting for 52.6% of the variance.
T h e first factor, with loadings from the EPAT,
VSRT, CRM, Serial Digits, CVMT, and Visual
Reproduction, defines a dimension of general
(verbal and visual) memory, accounting for
15.9% of the variance. The second factor, with
loadings from Block Design, Object Assembly,
Trail Making Test, Part B, and Visual Reproduction, defines a factor of visuospatial intelligence/ability accounting for 15% of the vari-

5 40

GLENN J. LARRABEE AND GLENN CURTISS

Downloaded by [University of South Florida] at 00:15 20 April 2013

ance. The third factor, defined by loadings from

Digit Span, Serial Digits, Mental Control, and
the PASAT, defines a dimension of attentionhmmediate memory and information processing, accounting for 11.1% of the variance.
The fourth factor, defined by loadings from Information and Vocabulary, comprises a factor of
verbal intelligence/ability, accounting for 10.6%
of the variance. Noteworthy findings include a
greater loading of Serial Digits on attentionhmmediate memory and information processing than on general memory, a greater loading of Visual Reproduction on spatial intelligence/ability than on general memory, and a
greater loading of Trail Making Test, Part B on
spatial intelligencefability than on attentionhmmediate memory and information processing.
For the delayed recallhecognition raw score
analysis, the eigenvalues for the first four components in the unrotated solution were 4.60,
I .93, I .29, and 1.26. The final, varimax-rotated
principal axis solution is displayed in Table 2.

The eigenvalues for the varimax rotation are

2.25 for factor 1, 1.68 for factor 2, 1.58 for factor 3, and 1.50 for factor 4, accounting for
50.1% of the variance. The first factor, with
loadings from VSRT, Visual Reproduction,
CRM, CVMT, EPAT, and Serial Digits defines
a general (verbal and visual) memory factor,
accounting for 16.1% of the variance. The second factor, with loadings from Block Design,
Object Assembly, and Trail Making Test, Part B
defines a factor of visuospatial intelligence/ability, accounting for 12% of the variance. The third factor, defined by loadings from
Digit Span, Serial Digits, Mental Control, and
PASAT, represents a dimension of attentiodimmediate memory and information processing, accounting for 11.3% of the variance.
The fourth factor, defined by Information and
Vocabulary, defines a dimension of verbal intelligence/ability, accounting for 10.7% of the
variance. Again, Serial Digits reflected a stronger association with attentionhmmediate memory and information processing than with gen-

Table I . Factor Analysis Using Immediate RecalllRecognition Memory Raw Scores.

Factors
1

Serial Digits

.46

.09

.57

-.02

EPAT

.69

.15

.I8

.I3

VSRT

.68

.I5

.27

-.02

Visual Reproduction

.42

.59

.I8

.12

CRM

.67

.27

-.03

-.07

CVMT

.49

.29

.I2

.I6

Mental Control

.07

.Ol

.49

.I2

Digit Span

.I6

.22

.66

.30

PASAT

.10

.35

.53

.08

Trail Making Test, Part B

Test"

-. 16

-. 62

-.20

.06

Block Design

.34

.72

.I2

.I8

Object Assembly

.I2

.67

.I4

.08

Information

.06

.05

.11

.80

Vocabulary

.02

.11

.22

.79

Note. N = 1 12.
aEPAT=Expanded Paired Associate Test; VSRT = Verbal Selective Reminding Test; CRM = Continuous Recognition Memory; CVMT = Continuous Visual Memory Test.

54 I

Downloaded by [University of South Florida] at 00:15 20 April 2013

MEMORY TEST CONSTRUCT VALlDITY

era1 memory, and Trail Making Test, Part B reflected a stronger association with spatial intelligencefability than with attentiodimmediate
memory and information processing. By contrast, Visual Reproduction demonstrated a
stronger loading with general memory than with
spatial intelligence/ability, reversing the pattern
obtained in Table 1 when the immediate recall
scores were factored.
The second set of factor analyses were performed on immediate and delayed rec a l l l r e c o g n i t i o n s c o r e s which h a d b e e n
residualized for age. Age was regressed on each
memory, attention, and intellectual ability measure, with subsequent analyses performed on the
residual variance of each variable that was not
accounted for by age.
For the immediate recall/recognition ageresidualized score analysis, the eigenvalues for
the first four components in the unrotated solution were 4.75, 1.66, 1.40, and 1.29. The final
varimax-rotated principal axis solution is displayed in Table 3. The eigenvalues for the vari-

max solution are 2.01 for factor I , 1.92 for factor 2, 1.57 for factor 3, and 1.54 for factor 4,
accounting for 50.3% of the variance. The first
factor, defined by loadings from EPAT, VSRT,
CRM, CVMT, and Serial Digits defines a dimension of general (verbal and visual) memory,
accounting for 14.4% of the variance. The second factor, defined by loadings from Block Design, Object Assembly, Trail Making Test, Part
B, and Visual Reproduction, represents a dimension of visuospatial intelligence/ability, accounting for 13.7% of the variance. The third factor,
defined by loadings from Information and Vocabulary, represents a dimension of verbal intelligencelability, accounting for 1 1.2% of the
variance. The fourth factor, defined by loadings
from Digit Span, Serial Digits, the PASAT, and
Mental Control, defines a dimension of attentionhmmediate memory and information processing, accounting for 1 1 .O% of the variance.
It is noteworthy that the loadings remain quite
similar when comparing the age-residualized
solution in Table 3 to the raw score solution in

Table 2. Factor Analysis Using Delayed RecalURecognition Memory Test Raw Scores.
Factors
1

Serial Digits

.41

.06

.61

-.03

EPAT

.47

.I2

.05

.08

VSRT

.73

.I0

.21

-.03

Testa

Visual Reproduction

.68

.33

.20

.I3

CRM

.56

.I9

.I8

-.20

CVMT

.55

.2 1

.02

.16

Mental Control

.04

.06

.50

. I2

Digit Span

.I3

.23

.66

.30

PASAT

.16

.34

.51

.09

-.2 1

-.61

-.17

.04

Trail Making Test, Part B

Block Design

3.5

.66

.I7

.I8

Object Assembly

.I1

.69

.I3

.08

Information

.06

.04

.12

.78

Vocabularv

.03

.09

.21

.80

Note. N = 112.
a EPAT = Expanded Paired Associate Test; VSRT = Verbal Selective Reminding Test; CRM = Continuous
Recognition Memory; CVMT = Continuous Visual Memory Test.

Downloaded by [University of South Florida] at 00:15 20 April 2013

542

GLENN J. LARRABEE AND GLENN CURTISS

-

Table I , with the exception of the reversal of

factors 3 and 4. Also noteworthy are patterns
similar to those observed in the raw score analysis, with Serial Digits reflecting a stronger
loading with attentionhmmediate memory and
information processing than with general memory, Immediate Visual Reproduction reflecting
a stronger association with visuospatial intelligencelability than with general memory, and
Trail Making Test, Part B demonstrating an association with visuospatial intelligence/problem
solving rather than attentiodimmediate memory
and information processing.
These apparent similarities in factor structure
were evaluated with Cattell's salient similarity
index (Cattell, Balcar, Horn, & Nesselroade,
1969; Tabachnick & Fidell, 1989). The salient
similarity index or s, which can range between 0
and I , evaluates similarities in the pattern of
loadings between two separate factor solutions,
and is derived by completing a contingency table of positively salient, negatively salient, and
hyperplane (nonsalient) loadings. Saliency of

loadings must be specified prior to completion

of the table. In this case, saliency was specified
to be .40. Positively salient loadings are positive
loadings of .40 or greater; the converse holds for
negatively salient loadings. Hence, comparing
factor 1 in Table 1 to factor 1 in Table 3, five
variables are positively salient (Serial Digits,
EPAT, VSRT, CRM, and CVMT), none are negatively salient, eight variables are nonsalient and
fall in the hyperplane (Mental Control, Digit
Span, PASAT, Trail Making Test, Part B, Block
Design, Object Assembly, Information, and Vocabulary), and one variable falls outside the
hyperplane (Visual Reproduction, which is positively salient in Table I , but nonsalient in Table
3). Utilizing formulae provided by Cattell et al.,
(1969) and Tabachnick and Fidell (l989), an s
of .9 1 is obtained, p < .OO I . For the comparison
of the raw score and age residualized factor 2, s
= 1 .OO, p < .002. Comparison of raw factor 3 and
age residualized factor 4 yielded s = I .OO, p <
,002, and comparison of raw factor 4 with ageresidualized factor 3 yielded s = 1 .OO, 17 < .O 12.

Table 3. Factor Analysis Using Immediate RecaWRecognition Memory Test Scores Residualized for Age
Factors
Test"

Serial Digits

.45

.09

-.o I

.57

EPAT

.h7

.o9

.I7

.I8

VSRT

.h6

.I2

.o I

.26

Visual Reproduction

.36

.55

.24

.I6

CRM

.65

.24

-.02

-.05

CVMT

.45

.25

.2 I

.I0

Mental Control

.05

.07

.I3

.40

Digit Span

.I4

.2 1

32

.66

PASAT

.09

36

.08

.52

-.I0

-. 58

.oo

-.I8

BlockDesign

.29

.69

.2s

.I

Object Assembly

.I 1

.67

.09

.I3

Information

.I6

.I5

.80

.I3

Vocabulary

.03

.I3

.7u

.23

Trail Making Test, Part B

Note. N = 1 12.
'' EPAT = Expanded Paired Associate Test; VSRT = Verbal Selective Reminding Test; CRM = Continuous
Recognition Memory; CVMT = Continuous Visual Memory Test.

543

Downloaded by [University of South Florida] at 00:15 20 April 2013

MEMORY TEST CONSTRUCT VALIDITY

Altogether these analyses provided strong statistical support for the similarity of the immediate raw score and immediate age-residualized
factor structures.
For the delayed recall/recognition age-residualized score analysis, the eigenvalues for the
first four components in the unrotated solution
were 4.45, 1.75, 1.35, and 1.33. The final varimax-rotated principal axis solution is displayed
in Table 4. The eigenvalues for this varimax solution are 2.04 for factor l , l .58 for factor 2,
1.55 for factor 3, and 1.55 for factor 4, accounting for 48% of the variance. The first factor,
defined by loadings from the VSRT, Visual Reproduction, CRM, CVMT, EPAT, and Serial
Digits, defines a general (verbal and visual)
memory factor, accounting for 14.6% of the
variance. The second factor, defined by loadings
from Block Design, Object Assembly, and Trail
Making Test, Part B, represents a dimension of
visuospatial intelligence/ability, accounting for
I I .3% of the variance. The third factor, defined
by loadings from Information and Vocabulary,

defines a d i m e n s i o n of verbal i n t e l l i gencelability, accounting for I I . I % of the variance. The fourth factor, defined by loadings
from Digit Span, Serial Digits, Mental Control,
and the PASAT, defines a dimension of attentiodimmediate memory and information processing, accounting for 1 I . 1% of the variance.
The results obtained in the age residualized
analysis are quite similar to those obtained in the
raw score analysis with the exception of the reversal of factors 3 and 4. Other noteworthy findings include the stronger association of Serial
Digits with attentionhmmediate memory and
information processing than with general memory and the association of Trail Making Test,
Part B with visuospatial intelligencelability
rather than attentionlimmediate memory and
information processing. Also, the primary association of Delayed Visual Reproduction is with
general memory rather than with spatial intelligencelability, the opposite of the pattern found
for Immediate Visual Reproduction.
Similarity of the delayed raw score and de-

Table 4. Factor Analysis Using Delayed RecalllRecognition Memory Test Scores Residualized for Age.
Factors
Test"

Serial Digits

.40

.06

-.02

.6/

EPAT

.44

.09

.I0

.04

VSRT

.72

.07

.00

.20

Visual Reproduction

.65

.29

.20

.I9

CRM

.55

.I9

-.I9

.I6

CVMT

.5I

.I5

.22

.oo

Mental Control

.02

.06

.I2

.51

Digit Span

.I I

.22

.3 I

.66

PASAT

.15

34

.09

.50

-.I4

-. 58

-.O 1

-.I6

Trail Making Test, Part B

Block Design

.28

.63

.26

.IS

Object Assembly

.I5

.70

.09

.I 1

Information

.I6

.I3

.77

.I5

Vocabu 1ary

.0s

.I I

.79

.22

Note. N = 112
a EPAT = Expanded Paired Associate Test; VSRT = Verbal Selective Reminding Test; CRM = Continuous
Recognition Memory; CVMT = Continuous Visual Memory Test.

544

GLENN J . LARRABEE AND GLENN CURTISS

Downloaded by [University of South Florida] at 00:15 20 April 2013

layed age-residualized factor structure was analyzed with Cattells s, specifying saliency as .40
or greater. The comparison of delayed raw factor 1 with delayed age-residualized factor 1
yielded s = 1 .00, p < .00 1 . The comparison of
delayed raw factor 2 with delayed age-residualized factor 2 yielded s = 1.00, p < ,012. Comparison of delayed raw factor 3 and delayed ageresidualized factor 4 resulted in s = 1.00, p <
,002, and comparison of delayed raw factor 4
with delayed age-residualized factor 3 yielded s
= 1 .00, p < ,012. These results provide strong
statistical support for the similarity of the delayed raw and delayed age-residualized factor
structures.

DISCUSSION
The present data support the construct validity
of the EPAT, VSRT, Delayed Visual Reproduction, CRM, and CVMT as measures of learning
and memory, and support the construct validity
of the PASAT as a measure of attentiodimmediate memory and information processing. Consistent with previous factor analyses of verbal learning and memory procedures,
the loadings of the EPAT and VSRT on the
memory factor were consistent, irrespective of
whether the immediate or delayed recall scores
were factored (Larrabee et al., 1985; Leonberger
et al., 1990, 1992; Smith et a]., 1992).
Results for the purported measures of visual
memory were different than in previous factor
analytic investigations, in that the CRM and
CVMT showed loadings on the general memory
factor, irrespective of whether they were based
on acquisition versus delayed recognition. This
differs from the results for the WMS Visual Reproduction subtest, which showed a stronger
association with spatial intelligencelability than
memory for immediate recalllacquisition, and a
stronger association with memory than spatial
ability for the delayed recall score, a pattern typically reported for this measure (Larrabee et al.,
1985, 1992). These results suggest that the CRM
and CVMT are purer measures of memory; that
is, the acquisition trials are not confounded with
spatial ability, contrary to the WMS Visual Re-

production. The failure to find a separate, modality specific visual memory factor is similar to
several other factor analyses (Larrabee et al.,
1985; Leonberger et al., 1990, 1992; Smith et
al., 1992), but does differ from the Larrabee et
a]., (1992) study, which did report a modality
specific visual memory factor defined by the
CVMT and Visual Reproduction delayed scores.
This may be a function of differences in test
variables and subjects. The Larrabee et al.,
( 1 992) analyses were based on normal controls,
and substituted Shipley Vocabulary for WAIS-R
Vocabulary, Picture Completion for Object Assembly, and an experimental vigilence test
(which did not load on any factor) for WMS
Mental Control.
The results for the WMS Visual Reproduction
bear further comment. The loading pattern of .42
on memory and .59 on spatial intelligence for
immediate raw score recall, and .68 on memory
and .33 on spatial intelligence for delayed recall
is quite similar to the results reported by
Larrabee et al., 1985 (.41 on memory, .66 on
spatial for immediate recall; .59 on memory, .5 1
on spatial for delayed recall). This can be contrasted with the factor loadings reported by
Leonberger et al., for the WMS-R Visual Reproduction I and I1 (1990, 1992). In the Leonberger
et al., ( 1990) investigation employing the WMSR, WAIS, and Halstead Reitan Neuropsychology
Battery (HRNB), WMS-R Visual Reproduction
I loaded .72 on spatial ability and .21 on memory, while WMS-R Visual Reproduction I1
loaded .66 on spatial ability and .44 on memory.
In the Leonberger et al., (1992) study, employing the WMS-R, WAIS-R, and HRNB, WMS-R
Visual Reproduction I loaded .64 on spatial ability, and .17 on memory, with Visual Reproduction I1 loading .55 on spatial ability and .49 on
memory. Altogether, the current analyses, and
previous data of Larrabee et al., (1985) and
Leonberger et al., (1990, 1992) suggest that on a
factorial basis, the original WMS Visual Reproduction subtest in delayed reproduction format
is a better measure of memory than the WMS-R
Visual Reproduction 11, which is more strongly
associated with spatial ability. This may be secondary to the greater spatial demands inherent in
the WMS-R Visual Reproduction stimuli.

MEMORY TEST CONSTRUCT VALIDITY

545

Downloaded by [University of South Florida] at 00:15 20 April 2013

The present data provide only partial support

for the construct validity of Serial Digit Learning as a measure of verbal learning and memory.
Although this test loads on the general memory
factor, its major association is with the attentionhmmediate memory and information processing factor. The findings of shared loadings
between these two factors do support the previous report of greater sensitivity of Serial Digit
Learning to brain disease than the traditional
Digit Span procedure (Hamsher, Benton, &
Digre, 1980).
The association of the PASAT, Digit Span.
and Mental Control with the same factor is similar to previous factor analytic investigations
demonstrating the close interrelationship of attention, speed of processing, and immediate
memory (Larrabee, Kane, & Schuck, 1983;
Larrabee et al., 1985; Leonberger et al., 1990,
1992; Wechsler, 1987). The association of Trail
Making Test, Part B with spatial intelligence/
ability rather than the attentionhmmediate memory and information processing factor is noteworthy, given prior characterizations of this test
as a measure of attention and information processing (Lezak. 1983; Mirsky, 1989). Mirsky
( 1989) reported a perceptual-motor speed factor,
which had loadings from the Trail Making Test,
letter cancellation, WAIS-R Digit Symbol, and
the Stroop Test, on a factor separate from Digit
Span and Arithmetic. Hence, the current results
may be due to the absence of perceptual-motor
speed variables i n the present study. However, a
prior factor analysis including the WMS-R.
WAIS-R, and HRNB demonstrated a complex
loading pattern for Trail Making Test, Part B,
which loaded on a spatial ability as well as a
perceptual-motor speed factor (Leonberger el
al., 1992). Additionally, Leonberger, Nicks,
Goldfader, and Munz ( I99 I ) reported a primary
loading for Trail Making Test, Part B on a factor
defined by the Seashore Rhythm and Finger
Tapping tests, with secondary loadings on a factor defined by Digit Span and Mental Control.
and on a factor defined by WAIS Performance
IQ and the HRNB Tactual Performance Test.
Finally, Swiercinsky and colleagues reported
multiple loadings for Trail Making Test, Part B.
including associations with spatial functioning.

motor speed and fine motor coordination, and

concentration (Swiercinsky, 1979; Swiercinsky
& Howard, 1982). These data suggest that Trail
Making, Part B is a complex, multi-factorial
cognitive task.
The stability of the present factor solution,
with similar findings for the raw score and the
age-residualized factor structure, is consistent
with prior factor analytic investigations comparing raw scores and age-adjusted scores (Crook
& Larrabee, 1988; Delis et al., 1988; Larrabee et
al., 1992). Hence, although memory tests show
definite age-associated level of performance
effects, the factor structure appears to be invariant across the age range. Consequently, the clinician can be assured that the same memory
constructs are being assessed across the age
range, even though there are age-related level of
performance changes in these constructs.
I n closing, Fame cautions are appropriate concerning the generalizability of the current results, given the relatively small sample size and
subject-to-variable ratio. The similarity of loading patterns for the EPAT, VSRT, Immediate
and Delayed Visual Reproduction, WAIS-R
subtests (Information, Vocabulary, Digit Span,
Block Design, Object Assembly) and WMS
Mental Control to previously published solutions, supports the generalizability of results for
these procedures. Further replication of the loading patterns for Serial Digits, CRM, CVMT,
PASAT, and Trail Making Test, Part B with a
larger sample size is recommended to support
the generalizability of the current findings for
these procedures.

REFERENCES
Benton, A.L., Hamsher, K. deS., Varney, N.R., &
Spreen, 0. (1983). Contributions to iieuropsychological as.se.s,snzent. New York: Oxford University

Press.
Bornstein, R.A., & Chelune, G.J. (1989). Factor structure of the Wechsler Memory Scale-Revised in
relation to age and education level. Archives o j
Clinical Neuropsycholugy,4 , 15-24.

Brittain, J.L., LaMarche, J.A., Reeder, K.P., Roth,

D.L., & Boll, T.J. ( 1 99 1 ). Effects of age and IQ on
Paced Auditory Serial Addition Task (PASAT)
performance. The Clinical Neuropsychologist, 5 ,

546

GLENN J . LARRABEE AND GLENN CURTISS

Downloaded by [University of South Florida] at 00:15 20 April 2013

163- 175.
Buschke, H. ( I 973). Selective reminding for analysis
of memory and learning. Journul of Verhal Lenrnirig and Verbal Behavior, 12, 543-550.
Cattell, R.B., Balcar, K.R., Horn, J.L., & Nesselroade,
J.R. (1969). Factor matching procedures: An improvement of the s index; with tables. Educrrtional
cind Psycliologicnl Mensurenient, 29, 78 1-792.
Crook, T.H., & Larrabee, G.J. (1988). Interrelationships among everyday memory tests: Stability of
factor structure with age. Nriiropsyc.holofiy, 2, I 12.
Cunningham, W.R. ( 1986). Psychometric perspectives: Validity and reliability. In L.W. Poon, T.
Crook, K.L. Davis, C. Eisdorfer, B.J. Gurland,
A.W. Kaszniak, & L.W. Thompson (Eds.), H m d hook ,for clinical nieinory asse.ssnient o f older
cidu1t.s (pp. 27-3 1 ). Washington, DC: American
Psychological Association.
Delis, D.C., Freeland, J., Kramer, J.H., & Kaplan, E.
(1988). Integrating clinical asse
tive neuroscience: Construct va
variate verbal learning test. Journal qf Consulting
hology, 56, 123- 130.
Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A.
( 1987). California Verhril Leurnirig Test. Reseurch
edition. Manutrl. San Antonio: The Psychological
Corporation.
Denman, S.B. ( 1984). Denmuri Neuropsychology
Memory Scale. Charleston, SC: Author.
Erickson, R.C., & Scott, M.L. ( I 977). Clinical memory testing: A review. Psychological Bulletin, 84,
I 130- I 149.
Gorsuch, R.L. (1983). Factor analysis (2nd ed.).
Hillsdale, NJ.: Lawrence Erlbaum Associates.
Gronwall, D.M.A. (1977). Paced auditory serial addition task: A measure of recovery from concussion.
Perceptuul and Motor Skills, 44, 361-373.
Hamsher, K., Benton,A.L., &Digre, K. (1980). Serial
digit learning: Normative and clinical aspects.
Journul of Clinical Neirropsyc~icrlog~,
2, 39-50.
Hannay, H.J., & Levin, H.S. (1985). Selective Reminding Test: An examination of the equivalence
of four forms. Journal of Clinical and Exprrimenttil Neuropsycliology, 7, 25 1-263.
Hannay, H.J., & Levin, H.S. (1989). Visual continuous recognition memory in normal and closedhead-injured adolescents. Journal of Cliniccil and
Experimental Neuropsychology, I I , 444-460.
Hannay, H.J., Levin, H.S., &Grossman, R.G. (1979).
Impaired recognition memory after head injury.
Cortex, 15, 269-283.
x r a b e e , G.J. (1987). Further cautions in interpretation of comparisons between the WAIS-R and the
Wechsler Memory Scale. Journal of Clinical and
Experimental Neuropsychology, 9,456-460.
x r a b e e , G.J. (1992). On modifying recognition
ry tests for detection of malingering. Neuro)lOgv, 6, 23-27.

Larrabee, G.J., & Crook, T.H. ( I 988). Assessment of

drug effects in age-related memory disorders: Clinical, theoretical and psychometric considerations.
hoph~rrnicicologyBulletin, 24, 5 15-522.
e, G.J., Kane, R.L., & Schuck, J.R. (1983).
Factor analysis of the WAIS and Wechsler Memory Scale: An analysis of the construct validity of
the Wechsler Memory Scale. Journal o f Clinicul
Neuropsychology, 5, 159- 168.
Larrabee, G.J., Kane, R.L., Schuck, J.R., & Francis,
D.E. (1985). The construct validity of various
memory testing procedures. Journrrl of Clinicrrl
iiiitl Exprrinzentcil Neur~i~is?.c.hology,
7, 239-250.
Larrabee, G.J., Largen, J.W., & Levin, H.S. ( 1985).
Sensitivity of age-decline resistant (Hold)
WAIS subtests to Alzheimers disease. Journal of
Clinicul trnd E.xperirneritu1 Neurol,sychology, 7,
497-504.
Larrabee, G.J., Trahan, D.E., & Curtiss, G. (1992).
Construct validity of the Continuous
ory Test. Archives ofClinicid Neiirop
395-405.
Larrabee, G.J., Trahan, D.E., Curtiss, G., & Levin,
H.S. (1988). Normative data for the Verbal Selective Reminding Test. Neuro~,sycholo~y,
2 , 173182.
Leonberger, F.T., Nicks, S.D., Goldfader, P.R., &
Larrabee, G.J. (1990). Construct validity of the
WMS-R indices (Abstract). The C1inic.d Neuropsychologist, 4 , 284.
Leonberger, F.T., Nicks, S.D., Goldfader, P.R., &
Munz, D.C. (1991). Factor analysis of the Wechsler Memory Scale-Revised and the Halstead-Reitan
Neuropsychological Battery. The Clinical Neuropsvchologist, 5, 83- 88.
Leonberger, F.T., Nicks, S.D., Larrabee, G.J., &
Goldfader, P.R. (1992). Factor structure of the
Wechsler Memory Scale-Revised within a comprehensive neuropsychological battery. Nruro-psychology, 6 , 239-249.
Levin, H.S., Benton, A.L., & Grossman, R.G. (1982).
Neurohehaviorul consequences qf closed head injury. New York: Oxford University Press.
Levin, H.S., Mattis, S., Ruff, R.M., Eisenberg, H.M.,
Marshall, L.F., Tabaddor, K., High, W.M., &
Frankowski, R.F. ( I 987). Neurobehavioral outcome following minor head injury: A three-center
study. Journal of Neurosu
Lezak, M.D. (1983). Neurop
(2nd ed.). New York: Oxford University Press.
Luring, D.W., & Papanicolaou, A.C. (1987). Memory
assessment in neuropsychology: Theoretical considerations and practical utility. Journal of Clinical
und Experimental Neuropsychology, 9, 340-358.
Malec, J.F., Ivnik, R.J., & Hinkeldy, N.S. (1991). Visual spatial learning test. Psychological Assessment: A Journal of Consulting crnd Clinical Psychology, 3, 82-88.
Mayes, A.R. (1986). Learning and memory disorders

MEMORY TEST CONSTRUCT VALIDITY

Downloaded by [University of South Florida] at 00:15 20 April 2013

and their assessment. N r i i r ~ ~ ~ ~ . s y c / i o24,

l o ~25i~i,
39.
Mirsky, A.F. ( 19x9). The neuropsychology of attention: Elements of a complex behavior. In E. Perecman (Ed.), iritegrutirig theory a r ~ d p r r i c t i ciri~cliniReitan, R.M., & Wolfson, D. (1985). The Hrrlstecidhologicul Test Buttery. Theory
t i r i d r~lirric~cil
iriterprettrtion. Tucson, AZ: Neuropsychoiogy Press.
Roman, D.D., Edwall, G.E., Buchanan, R.J., & Patton, J.H. (1991). Extended norms for the Paced
Auditory Serial Addition Task. The Clinicul Nrrir~i~~,sy~~l7ologi.s1,
5. 33-40.
Russell, E.W. (1975). A multiple scoring method for
the assessment of complex memory functions.
Jou niul of Cons~iltirigtind Cliriical Psychology, 43,
800-809.
Smith, G.E., Malec, J.F., & Ivnik, R.J. (1992). Validity of the construct of nonverbal memory: A factor
analytic study in a normal elderly sample. Journd
oj'Clinicu1 uric1 Experimental Neiirops?lchology, 14,
21 1-221.
Swiercinsky, D.P. (1979). Factorial pattern description and comparison of functional abilities in neuropsychological assessment. Perceptiral and Motor
Skills, 4K, 23 1-24 I .
Swiercinsky, D.P., & Howard, M.E. (1982). Programmatic series of factor analyses for evaluating the
structure of neuropsychological test batteries. Clinical Neuropsychology, 4 , 147- 152.

547

Tabachnick, B.G., & Fidell, L.S. (1989). Ilsirig rruil?ivrrriute .stufistics (2nd ed.). Philadelphia: Harper &
Row.
Trahan, D.E., & Larrabee, G.J. (1988). Proje.ssioriu1
riiuriuul: Contiri~iousVisual Menlory Test. Odessa,
FL: Psychological Assessment Resources.
Trahan, D.E., Larrabee. G.J., & Quintana, J.W.
(1990). Visual recognition memory in normal
adults and patients with unilateral vascular lesions.
Joiirnnl of' Cliniccil trtrd Experirnerztul NeuropsyCIIOIORJ, 12, 85 7-872.
Trahan, D.E., Larrabee, G.J., Quintana, J.W., Goethe,
K.E., & Willingham, A.C. (1989). Development
and clinical validation of an Expanded Paired Associate Test with delayed recall. The Cliriicd Nruologi.st, 3 , 169- I 83.
E., Quintana, J., Willingham, A.C., & Goethe, K.E. ( 1988). The Visual Reproduction Subtest:
Standardization and clinical validation of a delayed
recall procedure. Neurop
Wechsler, D. ( 1945). A standardized memory scale
for clinical use. The Joumu/ of'fsychologv,19, 8795.
Wechsler, D. ( I 981). W A I S - R mnrirrul. New York: The
Psychological Corporation.
Wechsler D. ( 1 987). Wechsler Mernory Scule-Revised
Manuul. San Antonio, TX: The Psychological Corporation.
Williams, J.M. (199 I).Mernory A.s.se.ssnzent Scales.
Odessa, FL: Psychological Assessment Resources,
Inc.