APPLIED NEUROPSYCHOLOGY, 18: 136142, 2011 Copyright # Taylor & Francis Group, LLC ISSN: 0908-4282 print=1532-4826 online

DOI: 10.1080/09084282.2010.547785

The Flynn Effect in Neuropsychological Assessment

Mercedes D. Dickinson
Southern Oregon Neuropsychological Clinic, Medford, Oregon

Merrill Hiscock
Department of Psychology, University of Houston, Houston, Texas and Center for Neuro-Engineering and Cognitive Science, University of Houston, Houston, Texas

The Flynn effect refers to the rise in IQ throughout the 20th century. This study examined whether the Flynn effect has also elevated performance on neuropsychological tests. A search of published norms revealed ve tests with appropriate normative data available for comparison. These tests were the Trail-Making Test (TMT), Symbol Digit Modalities Test (SDMT), Boston Naming Test, Finger Tapping, and Grooved Pegboard. Results indicated a strong Flynn effect for Parts A and B of the TMT and a probable Flynn effect for the oral SDMT. No Flynn effect was evident for the other tests. Implications for clinical assessment are discussed.

Key words:

aging, Cohort effect, Flynn effect, gerontology, neuropsychological assessment

FLYNN EFFECT During periodic revision and restandardization of the Wechsler intelligence scales, individuals in the validation samples who were administered both the older and newer versions of the same test consistently obtained higher IQ scores on the older version (Wechsler, 1974, 1981, 1997). This demonstrated that norms for the newer tests were more stringent: A person must perform better to obtain a specied IQ on the newer test compared with the older test. James R. Flynn (1984) attributed the differences in test norms to a pervasive intergenerational rise in IQ. Individuals who are born more recently must earn higher raw scores to achieve an IQ that is average for their respective cohorts, as the average IQ scores continue to increase from one cohort to the next younger cohort. The rising IQ curve, or Flynn effect, has been studied extensively, and there is abundant evidence that scores on intelligence tests have been rising in various parts
Address correspondence to Mercedes D. Dickinson, Southern Oregon Neuropsychological Clinic, 837 Alder Creek Drive, Medford, OR 97504. E-mail: mdickinson78@mac.com

of the world since the beginning of the 20th century (Flynn, 1984, 1987, 1999, 2007; Neisser, 1998b). The rate of increase ranges from about 3 IQ points per decade for the Wechsler and Stanford-Binet IQ tests, to as many as 6 IQ points per decade for culture-reduced tests such as Ravens Progressive Matrices (Neisser, 1998b). Despite numerous speculations regarding factors that may contribute to the Flynn effect, its cause remains uncertain (Flynn, 2007; Hiscock, 2007; Neisser, 1998b). Whereas the Flynn effect inuences Full Scale IQ from the various versions of the Wechsler IQ Test, it elevates Verbal IQ less dramatically than Performance IQ. Verbal IQ has been increasing about 2 IQ points per decade, while Performance IQ has been rising at twice that rate (Flynn, 1999, 2006; Flynn & Weiss, 2007). Moreover, there is considerable variability within verbal and performance categories in sensitivity to the Flynn effect. Dickinson and Hiscock (2010) found that the rate of change for each of the 11 Wechsler subtests ranges from 0.18 to 0.90 scaled-score units per decade (equivalent to 0.9 to 4.5 IQ points). Although Verbal subtests exhibit relatively small changes over time, Similarities is a notable exception. Some studies have

FLYNN EFFECT IN NEUROPSYCHOLOGICAL ASSESSMENT

137

estimated the rate of the increase in Similarities to be as great as 6 IQ points per decade (Flynn, 1999, 2007). The impact of the Flynn effect may extend beyond the measurement of IQ. As noted by Strauss, Sherman, and Spreen (2006), the Flynn effect comprises a number of potential effects of major consequence for neuropsychological assessment (p. 45). There are at least three reasons for suspecting that the Flynn effect might inuence scores on neuropsychological instruments. One reason is that many neuropsychological tests t the prole of the intelligence tests most sensitive to the Flynn effect, such as Ravens Matrices. These sensitive tests are nonverbal and relatively culture free, and they emphasize novel problem solving (i.e., uid intelligence; Neisser, 1998a). Moreover, scores on many neuropsychological tests are correlated with scores on IQ tests, at least in normal populations (Diaz-Asper, Schretlen, & Pearlson, 2004). Thirdly, many neuropsychological tests are old and are scored using norms from studies that were conducted several decades ago. Therefore, contamination by the Flynn effect in certain neuropsychological tests is highly probable and it may be widespread. If present and sufciently robust, the Flynn effect would create signicant problems in the interpretation of neuropsychological test data. The objective of the present study is to determine the extent to which the Flynn effect has altered norms for several well-known neuropsychological tests. This was accomplished by nding norms for tests that were published at different times and then tracking changes in mean performance over time. By adjusting scores for the ages of participants in the various studies, and by ascertaining the linearity of the changes over time, it was possible to differentiate a probable Flynn effect from uctuations attributable to other sources.

METHOD Compilations of normative neuropsychological data were examined as an initial means of selecting sets of norms. The compilations included published works by Lezak (1995), Lezak, Howieson, and Loring (2004), Mitrushina, Boone, Razani, and DElia (2005), Spreen and Strauss (1998), and Strauss et al. (2006). In addition, an extensive Internet literature search was conducted using the search terms, normative, neuropsychological, and assessment. Studies were selected according to the following criteria: (1) Samples were drawn from nonclinical populations, (2) samples included participants of various ages, (3) separate mean scores were provided for different age groups, (4) samples included at least 100 individuals, and (5) publication dates spanned an interval of approximately one decade or more. IQ test norms were excluded.

Tests representing any domain of cognitive functioning were considered (e.g., memory, visuospatial, processing speed, and language). However, a paucity of older studies (prior to 1965) with adequate normative data constrained the choices of tests to be examined. Studies meeting the rst four of the aforementioned criteria provided norms for the Symbol Digit Modalities Test (SDMT), Trail-Making Test (TMT), Grooved Pegboard (GP), Finger Tapping (FT), and Boston Naming Test (BNT). Data for the SDMT and GP were analyzed even though they did not quite meet the last criterion (regarding the interval between publication dates of different studies). The only studies of nonclinical samples with adequate sample sizes for the SDMT were based on data published only 5 years apart, but the data were collected approximately 12 years apart (Smith, 1981; Yeudall, Fromm, Reddon, & Stefanyk, 1986). In the case of the GP, the three studies of nonclinical samples of adequate size spanned an interval of approximately 8 years (Bornstein, 1985; Ryan, Morrow, Bromet, & Parkinson, 1987; Ruff & Parker, 1993). Norms were tabulated according to age group for all selected studies, and the normative values were compared across studies. The different studies represent the different years in which norms were established, and each age-group mean within a study is treated as the score of one subject within a study. To adjust for the effect of having different age categories in different studies, we computed analyses of covariance (ANCOVAs) in which the midpoint of the age range for each age group served as the covariate. When there were more than two studies under consideration, we calculated linear and quadratic trends to provide more specic information about the nature of the change in performance over time. Trend analyses were adjusted to account for unequal intervals between the years in which different studies were published. For tests showing susceptibility to the Flynn effect, median values for various age groups from the oldest set of norms were mapped onto the distribution of the most recent norms. These calculations illustrate the clinical signicance of the change in norms across the time span for which norms were available.

RESULTS Trail-Making Test We identied 8 sets of norms for the TMT, with publication dates that spanned a range of 36 years (Bornstein, 1985; Davies, 1968; Elias, Robbins, Walter, & Schultz, 1993; Fromm-Auch & Yeudall, 1983; Kennedy, 1981; Stuss, Stethem, & Poirier, 1987; Tombaugh, 2004;

138

DICKINSON & HISCOCK

Yeudall et al., 1986). The eight studies comprised a total of 45 age-group norms. A preliminary analysis of variance (ANOVA) indicated that age varied signicantly across studies, F(7, 37) 2.27, p .050. Consequently, ANCOVA, with age as a covariable, was used to analyze change over time. The ve age groups with age midpoints above 70 years were eliminated from these analyses to satisfy the ANCOVA equality-of-slope assumption. One data point was eliminated from the Davies (1968) study, and 4 points were eliminated from the Tombaugh (2004) study. A separate one-way ANCOVA was computed for mean scores from each part of the TMT. The independent variable was the study from which the data were obtained (which coincided with the year in which each study was published). The dependent variable was the mean score for each study, which was calculated from the means for each age group represented in that study. Raw-score means and covariance-adjusted means for Parts A and B are shown in Figure 1. ANCOVA for Part A scores yielded a signicant difference across studies, F(7, 31) 3.87, p .004. Trend analysis, after adjustment for the unequal spacing of data points, indicated that the decline in completion time from the earliest to the latest studies was largely linear, F(1, 31) 24.80, p < .0001. The quadratic trend was nonsignicant, p > .15. The corresponding ANCOVA for Part B

scores also yielded a signicant difference across time, F(7, 31) 11.40, p < .001. Trend analyses again yielded a signicant linear trend, F(1, 31) 64.07, p < .0001, and a nonsignicant quadratic trend, p > .35. Additional ANCOVAs were computed to compare the earliest (Davies, 1968) and most recent (Tombaugh, 2004) norms after restoration of data from groups with age midpoints above 70 years. Sixteen age groups were included in this analysis, and the assumption of equal slopes was not violated. The ANCOVA yielded a signicant difference between studies for both Part A, F(1, 13) 10.23, p .007, and Part B, F(1, 13) 11.08, p .005. Mean completion times for Part A were 10.7 seconds longer in the Davies data than in the Tombaugh data, and the difference for Part B was 28.5 seconds. Analyses of differences in normative scores at distinct age levels between the earliest study (Davies, 1968) and latest study (Tombaugh, 2004) revealed clinically signicant differences for both Part A and Part B. This is illustrated in the instance of a hypothetical 65-year-old person who obtains a score of 48 seconds on Part A. This score falls exactly at the median for his or her age group according to Daviess norms. The same score in Tombaughs sample would lie at the 16th percentile for 65-year-olds. During the 36-year interval between Daviess publication of norms for Part A and Tombaughs publication of norms for Part A, mean performance evidently had increased by 1.0 standard deviation. On TMT Part B, the median for a 65-year-old in Daviess normative sample would be 119 seconds. The same score, earned by a 65-year-old in Tombaughs sample, would fall at the 11th percentile. In this instance, the mean performance level had increased by 1.23 standard deviations between 1968 and 2004. Other examples for individuals between the ages of 30 and 70 years are summarized in Table 1. The magnitude of the changes appears to be smaller for younger groups

TABLE 1 Comparison of Davies (1968) and Tombaugh (2004) Norms for the Trail-Making Test Age (Years) Raw Score (Seconds) Percentile Score (Davies) Part A 50 50 50 50 50 Part B 50 50 50 50 50 Percentile Score (Tombaugh)

30 40 50 60 70 30 40 50 60 70

32 34 38 48 80 69 78 98 119 196

22 25 20 4 2 7 15 2 4 2

FIGURE 1 Trail-Making Test (TMT).

FLYNN EFFECT IN NEUROPSYCHOLOGICAL ASSESSMENT

139

than for older groups and smaller for Part A than for Part B. Nonetheless, 7 of the 10 changes are between 1.0 and 2.1 standard deviations in magnitude, which renders them sufciently large as to affect clinical interpretation. The average magnitude of the change is 1.4 standard deviations across 36 years, or 0.4 standard deviations per decade. This is comparable to 6 IQ points per decade, the size of the Flynn effect on Ravens Matrices scores (Neisser, 1998b). Symbol Digit Modalities Test Only two normative SDMT studies fullled the inclusion criteria originally set forth: the original test development data published by Smith (1981), which used data from 1973 and 1975, and a study by Yeudall et al. (1986). The interval between studies was about 12 years, and the age range was restricted in the Yeudall et al. (1986) norms. The Smith study yielded norms for six groups with age midpoints between 21 and 71.5 years, and the Yeudall et al. (1986) study yielded norms for four groups with age midpoints between 17.5 and 35.5 years. As the studies differed signicantly in the mean age of participants, t(8) 9.25, p < .0001, ANCOVA was used to compare the respective norms. The analysis for the oral administration of the SDMT yielded a signicant difference, F(1, 7) 18.13, p .004. The covariance-adjusted mean for the Yeudall et al. (1986) norms (66.0) was higher than that for the norms collected in 1973 to 1975 (60.2; Smith, 1981). The corresponding difference for the written administration of the test was not signicant, p > .10. Raw-score means and covariance-adjusted means are shown in Figure 2. The effect of using the older and newer norms is illustrated in Table 2 Because there are no Yeudall et al. (1986) norms for people older than 40 years, our illustrative examples are based on hypothetical raw scores that fall at the medians for 20-, 30-, and 40-year-olds according to the Smith (1981) norms. In all instances, the raw scores yield lower z-scores and percentiles when the Yeudall et al. (1986) norms are applied. Depending on age group and test modality (written vs. oral), the magnitude of the decrease ranges from 0.12 to 0.78 standard deviations. The overall average change (z 0.52) across a 12-year interval represents a change of 0.4 standard deviations (6 IQ points) per decade. Boston Naming Test Four studies yielded norms for the BNT (Farmer, 1990; Kaplan, Goodglass, & Weintraub, 1983; Tombaugh & Hubley, 1997; Van Gorp, Satz, Kiersch, & Henry, 1986). These studies comprised a total of 29 age groups and spanned a period of 14 years. Because the midpoints

FIGURE 2 Symbol Digit Modalities Test (SDMT).

of the age ranges differed signicantly across studies, F(3, 25) 5.25, p .006, the normative values were analyzed in an ANCOVA in which age was used as a covariable. Even though scores on the BNT were not strongly correlated with age, the equality-of-slopes assumption for ANCOVA was violated when all four groups were included in the analysis. Elimination of either of the intermediate studies (i.e., Farmer, 1990 or Van Gorp et al., 1986) failed to yield homogeneous relationships between age and BNT performance. Consequently, we resorted to comparison of the earliest and most recent studies with a combined total of 16 age groups. Slopes were equal for these two studies. ANCOVA yielded no

TABLE 2 Comparison of Smith (1981) and Yeudall et al. (1986) Norms for the Symbol Digit Modalities Test Age (Years) Raw Score (No. Correct) Percentile Score (Smith) Percentile Score (Yeudall et al., 1986)

20 30 40 20 30 40

65.6 63.1 60.4 58.2 55.5 52.9

Oral Responses 50 50 50 Written Responses 50 50 50

32 22 26 45 23 36

140

DICKINSON & HISCOCK

signicant difference between age-adjusted means (55.2 vs. 55.7, p > .30). Finger Tapping Ten studies provided 36 sets of normative data for FT (Bornstein, 1985; Dodrill, 1978a, 1978b; Elias et al., 1993; Fromm-Auch & Yeudall, 1983; Goldstein & Braun, 1974; Ruff & Parker, 1993; Trahan, Patterson, Quintana, & Biron, 1986; Yeudall, Reddon, Gill, & Stefanyk, 1987; Wiens & Matarrazo, 1977). Data from studies with the same publication dates (Dodrill, 1978a, 1978b; Elias et al., 1993; Ruff & Parker, 1993) were combined and treated as if they came from a single study. Thus, in effect, there were 8 studies (i.e., clusters of norms) representing a span of 19 years from 1974 to 1993. Norms for the preferred and nonpreferred hands were averaged to obtain a single measure of tapping speed. As the studies did not differ signicantly with respect to participant age, F < 1, there was no need to use age as a covariable. However, males tend to tap faster than females, and not all of the norms were segregated according to gender. Consequently, we averaged tapping scores across males and females and calculated the proportion of males represented in each set of norms. This gender index (proportion of males) differed signicantly amongst studies, F(7, 28) 38.53, p < .0001. Therefore, ANCOVA was computed with gender (proportion of males) as a covariable. The analysis yielded a signicant difference among the adjusted means, F(7, 27) 6.05, p < .001. This difference is attributable primarily to the cubic component, F(1, 27) 14.61, p < .001, although there is also a signicant quadratic trend F(1, 27) 5.30, p .029. The linear trend was nonsignicant, F < 1. Figure 3 shows the nonlinearity of mean scores over time. Specic comparisons between the earliest and latest studies and between the earliest three and latest three studies yielded nonsignicant differences, t < 1. Grooved Pegboard Test Norms for the GP test were obtained from three studies (Bornstein, 1985; Ruff & Parker, 1993; Ryan et al., 1987). These studies comprised 10 age groups and represented a time span of 8 years. We analyzed the average of scores for the preferred and nonpreferred hands. There was no difference among studies in the midpoint age of groups, F < 1. Consequently ANOVA was computed instead of ANCOVA. The analysis yielded a nonsignicant difference among groups in mean score, F < 1. Neither the linear nor the quadratic contrast indicated any change in completion times across the 8-year interval.

FIGURE 3 Finger Tapping Test.

DISCUSSION Of the various sets of norms evaluated, only those for the TMT yielded strong evidence of the Flynn effect. Evidence from eight normative TMT studies comprising 40 age groups showed a highly signicant decrease in completion time during the period from 1968 to 2004. Performance on both Part A and Part B changed over time, and the changes were almost entirely linear. Age differences among samples were controlled through ANCOVA. Although other confounding variables cannot be ruled out, no particular difference among samples is salient. Gender has little or no effect on TMT performance (Strauss et al., 2006), and Tombaughs (2004) norms indicate that education accounts for virtually none of the variance in the 25-to-54 age range (Strauss et al., 2006, p. 658). Only 12 of our 40 age groups were older than 54 years of age. Using Tombaughs norms, we calculate that eliminating older participants who had more than 12 years of education would add less than 1 second to Part A completion times and less than 5 seconds to Part B times. Accordingly, even in the most extreme case poorly educated participants in the earliest studies and better educated participants in the more recent studiesthe magnitude of the Flynn effect would be inated only minimally. The SDMT results, though consistent with the Flynn effect, are more ambiguous than results from

FLYNN EFFECT IN NEUROPSYCHOLOGICAL ASSESSMENT

141

the TMT. Only two sets of SDMT norms were found, and those sets are separated by only 12 years. Without additional norms that would allow linearity to be assessed, we cannot be sure that the increase in performance on the oral administration of the SDMT is due to the Flynn effect rather than other sources of uctuation. It is noteworthy, however, that the rate of the increase in SDMT means, which is equivalent to 6 IQ points per decade, is identical to the rate at which TMT performance improved. This rate of change is characteristic of a strong Flynn effect, such as that observed in scores from Ravens Matrices (Neisser, 1998b). For practical reasons, it is reassuring that some test scores do not appear to be rising over time. Negative results for BNT, FT, and GP indicate that the Flynn effect may be absent or greatly attenuated with respect to certain tasks in the linguistic and motor domains. Given the consistent rise of 3 IQ points per decade in Verbal IQ (Flynn, 2006), however, it would be imprudent to generalize the negative ndings for BNT to all language tests that are used in neuropsychological assessment. The apparent absence of the Flynn effect in scores from motor tests suggests that the rise in performance does not extend to motor skills. This is consistent with a previously reported failure to nd a signicant change in FT scores during a 20-year interval (Bengtson, Mittenberg, Schneider, & Sellers, 1996). Our analysis of FT scores exemplies the advantage of having norms available from several points on the time scale. Even though tapping speed changed quite dramatically over time, the function is markedly nonlinear and unlike the Flynn effect patterns that have been observed in IQ test scores. Along with recent evidence that tests of episodic and semantic memory are inuenced by the Flynn effect (Ro nnlund & Nilsson, 2009), our results indicate that rising scores are not restricted to the realm of intelligence testing. The Flynn effect seems to be present in neuropsychological assessment instruments, although it is not ubiquitous across tests. Use of older norms for an affected test (e.g., TMT) will inate an individuals perceived performance (relative to his or her actual cohort) because that individual is being compared with an older cohort having lower average performance. If one uses the most outdated norms for Part A of the TMT (those reported by Davies, 1968) to evaluate a patients performance, the obtained standard score will be inated by 1 standard deviation for younger individuals and by more than 1 standard deviation for older patients. The error would be somewhat greater for Part B. Use of the 1968 norms would cause the persons performance to be misclassied as normal when he or she was actually performing in the impaired range. The classication problem is compounded when interpreting

the performance of older adults. Not only does the Flynn effect inuence some tests more than others, but its magnitude is increased in the normative data for older cohorts. Thus, the potential for misclassication is greatest when (1) the test is highly susceptible to the Flynn effect, (2) the norms are severely out of date, and (3) the patient is elderly. For the clinical neuropsychologist, test norms are both essential and problematic. The problematic aspects are summarized by Lezak et al. (2004): Different norms, derived on different samples in different places, and sometimes for different reasons, can produce quite different evaluations for some subjects resulting in false positives or false negatives, depending on the subjects score, condition, and the norm against which the score is compared (p. 147). We now have reason to suspect that these problems might be compounded by the Flynn effect, which causes certain norms to become progressively less valid with each passing year and which affects some tests more than others but may have no impact at all on a particular test. Flynn effect problems can be mitigated, of course, by using the most current norms available. However, the most recent norms might be decient on other grounds (Strauss et al., 2006). The most recent norms for some tests were collected several decades ago. The SDMT is one example. Scores from the oral administration not only appear to be vulnerable to the Flynn effect, but the most recent norms available were published more than 20 years ago. The apparent linearity of the Flynn effect can be used to justify extrapolation of norms into the future (Flynn, 2006, 2009). If scores for a particular test are rising at a known rate, the clinician can adjust the norms accordingly (see Hiscock, Inch, & Gleason, 2002). Such extrapolation must be done cautiously, however, as the slope of the rise in scores is always subject to change. Although comparisons of test norms are unlikely to elucidate causal factors underlying the Flynn effect, the present ndings may be contributory. First, our ndings constitute evidence that the Flynn effect can be generalized beyond IQ and memory. This is a step forward in mapping cognitive domains with respect to Flynn effect vulnerability. More specically, the negative ndings for FT and GP suggest that the rise in performance does not extend to motor skill. This is important to know, because it decreases the likelihood that there is a gradual improvement in nervous system efciency that extends to motor speed, dexterity, and other sensorimotor aspects of neural functioning. The absence of a Flynn effect for motor performance is consistent with cultural explanations for the effect and is inconsistent with physiological explanations such as improved nutrition (cf. Neisser, 1998b).

142

DICKINSON & HISCOCK

REFERENCES
Bengtson, M. L., Mittenberg, W., Schneider, B., & Sellers, A. (1996). An assessment of Halstead-Reitan test score changes over 20 years. Archives of Clinical Neuropsychology, 11, 268. Bornstein, R. A. (1985). Normative data on selected neuropsychological measures from a nonclinical sample. Journal of Clinical Psychology, 41, 651659. Davies, A. D. (1968). The inuence of age on Trail-Making Test performance. Journal of Clinical Psychology, 24, 9698. Diaz-Asper, C. M., Schretlen, D. J., & Pearlson, G. D. (2004). How well does IQ predict neuropsychological test performance in normal adults? Journal of the International Neuropsychological Society, 10, 8290. Dickinson, M. D., & Hiscock, M. (2010). Age-related IQ decline is markedly reduced after adjustment for the Flynn effect. Journal of Clinical and Experimental Neuropsychology, 32, 865870. Dodrill, C. B. (1978a). The hand dynamometer as a neuropsychological measure. Journal of Consulting and Clinical Psychology, 35, 236241. Dodrill, C. B. (1978b). A neuropsychological battery for epilepsy. Epilepsia, 19, 611623. Elias, M. F., Robbins, M. A., Walter, L. J., & Schultz, N. R. (1993). The inuence of gender and age on Halstead-Reitan neuropsychological test performance. Journal of Gerontology, 48, 278281. Farmer, A. (1990). Performance of normal males on the Boston Naming Test and the word test. Aphasiology, 4, 293296. Flynn, J. R. (1984). The mean IQ of Americans: Massive gains 1932 to 1978. Psychological Bulletin, 95, 2951. Flynn, J. R. (1987). Massive IQ gains in 14 nations: What IQ tests really measure. Psychological Bulletin, 101, 171191. Flynn, J. R. (1999). Searching for justice: The discovery of IQ gains over time. American Psychologist, 54, 520. Flynn, J. R. (2006). Tethering the elephant: Capital cases, IQ, and the Flynn effect. Psychology, Public Policy, and Law, 12, 170178. Flynn, J. R. (2007). What is intelligence? Beyond the Flynn effect. Cambridge, England: Cambridge University Press. Flynn, J. R. (2009). The WAIS-III and WAIS-IV: Daubert motions favor the certainly false over the approximately true. Applied Neuropsychology, 16, 98104. Flynn, J. R., & Weiss, L. G. (2007). American IQ gains from 1932 to 2002: The signicance of the WISC subtests. Journal of International Testing, 7, 209224. Fromm-Auch, D., & Yeudall, L. T. (1983). Normative data for the Halstead-Reitan neuropsychological tests. Journal of Clinical Neuropsychology, 5, 221238. Goldstein, S. G., & Braun, L. S. (1974). Reversal of expected transfer as a function of increased age. Perceptual and Motor Skills, 38, 11391145. Hiscock, M. (2007). The Flynn effect and its relevance to neuropsychology. Journal of Clinical and Experimental Neuropsychology, 29, 514529. Hiscock, M., Inch, R., & Gleason, A. (2002). Ravens progressive matrices performance in adults with traumatic brain injury. Applied Neuropsychology, 9, 129138. Kaplan, E., Goodglass, H., & Weintraub, S. (1983). The Boston Naming Test. Philadelphia, PA: Lea and Febiger. Kennedy, K. J. (1981). Age effects on Trail-Making Test performance. Perceptual and Motor Skills, 52, 671675. Lezak, M. D. (1995). Neuropsychological assessment (3rd ed.). New York, NY: Oxford University Press. Lezak, M. D., Howieson, D. B., & Loring, D. W. (2004). Neuropsychological assessment (4th ed.). New York, NY: Oxford University Press.

Mitrushina, M., Boone, K. B., Razani, J., & DElia, L. F. (2005). Handbook of normative data for neuropsychological assessment (2nd ed.). New York, NY: Oxford University Press. Neisser, U. (1998a). Introduction: Rising test scores and what they mean. In U. Neisser (Ed.), The rising curve: Long-term gains in IQ and related measures (pp. 322). Washington, DC: American Psychological Association. Neisser, U. (Ed.). (1998b). The rising curve: Long-term gains in IQ and related measures. Washington, DC: American Psychological Association. Ro nnlund, M., & Nilsson, L.-G. (2009). Flynn effects on subfactors of episodic and semantic memory: Parallel gains over time and the same set of determining factors. Neuropsychologia, 47, 21742180. Ruff, R. M., & Parker, S. B. (1993). Gender- and age-specic changes in motor speed and eyehand coordination in adults: Normative values for the Finger Tapping and Grooved Pegboard tests. Perceptual and Motor Skills, 76, 12191230. Ryan, C. M., Morrow, L. A., Bromet, E. J., & Parkinson, D. K. (1987). Assessment of neuropsychological dysfunction in the workplace: Normative data from the Pittsburgh occupational exposures test battery. Journal of Clinical and Experimental Neuropsychology, 9, 665679. Smith, A. (1981). Symbol Digit Modalities Test. Los Angeles, CA: Western Psychological Services. Spreen, O., & Strauss, E. (1998). A compendium of neuropsychological tests: Administration, norms, and commentary (2nd ed.). New York, NY: Oxford University Press. Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). A compendium of neuropsychological tests: Administration, norms, and commentary (3rd ed.). New York, NY: Oxford University Press. Stuss, D. T., Stethem, L. L., & Poirier, C. A. (1987). Comparison of three tests of attention and rapid information processing across six age groups. The Clinical Neuropsychologist, 1, 139152. Tombaugh, T. N. (2004). Trail-Making Test A and B: Normative data stratied by age and education. Archives of Clinical Neuropsychology, 19, 203214. Tombaugh, T. N., & Hubley, A. M. (1997). The 60-item Boston Naming Test: Norms for cognitively intact adults aged 25 to 88 years. Journal of Clinical and Experimental Neuropsychology, 14, 167177. Trahan, D., Patterson, J., Quintana, J., & Biron, R. (1986, February). The Finger Tapping Test: A reexamination of traditional hypotheses regarding normal adult performance. Paper presented at the annual meeting of the International Neuropsychological Society, Washington, DC. Van Gorp, W. G., Satz, P., Kiersch, M. E., & Henry, R. (1986). Normative data on the Boston Naming Test for a group of normal older adults. Journal of Clinical and Experimental Neuropsychology, 8, 702705. Wechsler, D. (1974). Wechsler Intelligence Scale for Children-Revised. New York, NY: Psychological Corporation. Wechsler, D. (1981). Wechsler Adult Intelligence Scale-Revised. New York, NY: Psychological Corporation. Wechsler, D. (1997). Wechsler Adult Intelligence Scale-III. New York, NY: Psychological Corporation. Wiens, A. M., & Matarrazo, J. D. (1977). WAIS and MMPI correlates of the Halstead-Reitan Neuropsychology Battery in normal male subjects. The Journal of Nervous and Mental Disease, 164, 112121. Yeudall, L. T., Fromm, D., Reddon, J. R., & Stefanyk, W. O. (1986). Normative data stratied by age and sex for 12 neuropsychological tests. Journal of Clinical Psychology, 42, 918946. Yeudall, L. T., Reddon, J. R., Gill, D. M., & Stefanyk, W. O. (1987). Normative data for the Halstead-Reitan neuropsychological tests stratied by age and sex. Journal of Clinical Psychology, 43, 346367.

Copyright of Applied Neuropsychology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.