Available online at www.sciencedirect.

com

Schizophrenia Research 102 (2008) 108 – 115

www.elsevier.com/locate/schres

Norms and standardization of the Brief Assessment of

Cognition in Schizophrenia (BACS)
Richard S.E. Keefe a,⁎, Philip D. Harvey b , Terry E. Goldberg c , James M. Gold d ,
Trina M. Walker a , Courtney Kennel a , Kirsten Hawkins a
a
Department of Psychiatry, Box 3207, Duke University Medical Center, Durham, North Carolina, 27710 USA
b
Department of Psychiatry, Emory University School of Medicine, Woodruff Memorial Building, 101 Woodruff Circle, Suite 4000,
Atlanta, Georgia, 30032, USA
c
Division of Psychiatry Research, Zucker Hillside Hospital, 7559 263rd St., Glen Oaks, New York, 11004, USA
d
Department of Psychiatry, Box 21247, Maryland Psychiatric Research Hospital, Baltimore, Maryland, 21228 USA
Received 2 November 2007; received in revised form 25 March 2008; accepted 31 March 2008
Available online 20 May 2008

Abstract

According to the recommendations of the Measurement and Treatment Research to Improve Cognition in Schizophrenia
(MATRICS) Neurocognition Committee, one of the desired characteristics of a cognitive battery for assessing cognition in
schizophrenia studies and clinical trials is the availability of normative data. This report describes normative data collected on the
Brief Assessment of Cognition in Schizophrenia (BACS) from 404 healthy controls with demographic characteristics matching the
2005 United States Census of English-speakers. The six test measures demonstrated the expected pattern of correlations with age,
gender, and education. Individual test scores were converted into standardized (T and z) scores and composite scores that were
corrected for age and gender. An education-correction factor was calculated and recommended only for non-schizophrenia patients.
Eight different verbal memory tests were found to have equivalent levels of difficulty.
© 2008 Elsevier B.V. All rights reserved.

Keywords: Cognition; Neuropsychology; Normative data; Psychological testing

1. Introduction Research to Improve Cognition in Schizophrenia

The availability of population norms on perfor-
mance-based measures of cognition was described as a
desired attribute of cognitive tests by the Neurocogni-
tion Committee for the Measurement and Treatment
⁎ Corresponding author. Box 3270, Duke University Medical
Center, Durham, NC 27710 USA. Tel.: +1 919 684 4306; fax: +1
919 684 2632.
E-mail address: richard.keefe@duke.edu (R.S.E. Keefe).
0920-9964/$ - see front matter © 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.schres.2008.03.024
(MATRICS) Project (Green et al., 2004). Such popula-
tion norms provide useful data for a test or test battery
such that the performance of any group of individuals
(as in a clinical trial) or the performance of any
individual (as in a clinic) can be immediately compared
to demographically similar members of the healthy
population and to their own performance across
cognitive domains. The population norms must be
from a sample that is of sufficient size and diversity so
that individual scores can be reliably compared to the
 R.S.E. Keefe et al. / Schizophrenia Research 102 (2008) 108–115
Table 1
BACS tests in the MATRICS neurocognitive domains
MATRICS neurocognitive BACS tests
domain
Processing ○ Verbal Fluency: category instances
speed (supermarket items) and letter fluency
(F and S words)
○ Token Motor Task
○ Symbol coding
Reasoning and ○ Tower of London Test
problem
solving
Verbal ○ List learning
memory
Working memory ○ Digit sequencing
norms. In addition, since test batteries used in clinical
trials require multiple assessments, it is important that
tests with large expected learning effects in the healthy
population, such as list learning tasks, have suitable
equivalent forms to avoid repetition of the same list
during the trial.
The Brief Assessment of Cognition in Schizophrenia
(BACS) takes approximately 30 min to administer in
healthy controls, and includes brief assessments of four of
the seven neurocognitive domains designated as important
by the MATRICS process: reasoning and problem solving,
processing speed, verbal memory, and working memory
(Table 1). The BACS is devised for easy administration
and scoring, and has been used in over 30 clinical trials in
schizophrenia. The battery was specifically designed to
measure treatment-related changes in cognition, and has
alternate forms. Its reliability, validity and comparability
of original forms have been established empirically (Keefe
et al., 2004). The composite score has high test–retest
reliability in patients with schizophrenia and healthy
controls (ICCs N .80), has been shown to be as sensitive to
the cognitive deficits of schizophrenia as a standard 2.5-
hour battery (Keefe et al., 2004) and is highly correlated
(r = 0.84, P b.001) with the composite score derived from
the CATIE Neurocognitive Test Battery (Keefe et al.,
2007). The BACS also has clear functional relevance, as
the composite score is strongly related to functional
measures such as independent living skills (r = .45),
performance-based assessments of performance of every-
day living skills (r = .56), and interview-based assessments
of cognition in patients with schizophrenia (r = .48) (Keefe
et al., 2006).
This report describes a study of 404 healthy controls,
representative of the general population, performing the
BACS. We performed a variety of statistical procedures
in order to develop normative scores, including exam-
ination of the effects of sex, age, ethnicity, and education,
109
presenting the results of these analyses in service of
development of a normative understanding of perfor-
mance on this assessment battery.
2. Methods
2.1. Study design
2.1.1. Subject identification and recruitment
Community controls were ascertained randomly using a
random digit dialing procedure to enhance the degree to
which this sample is representative of individuals in the
community as determined by the 2005 Census data. To
help avoid sampling bias, we used a scientific survey
sampling method. To obtain the sample, zip codes sur-
rounding our site were selected based on the diversity of
demographic characteristics needed. We purchased lists of
randomly sampled residential telephone numbers from
Survey Sampling, Inc, Fairfield, CT. Research staff called
these numbers, no voicemails were left but multiple call
attempts were made to those households with no answer.
The research staff followed a standardized, IRB approved,
recruitment script that involved minimal pressure by
offering the respondent a number of occasions to end the
call. No written letters were sent in advance and the
respondents were not offered any financial incentives to
listen to the call. Because this recruitment attempt took
place prior to the written informed consent process, the
respondent was not asked to reveal any sensitive or
personally identifying information. After a general intro-
duction to the study, and an opportunity to discontinue the
call, the respondent was read a list of exclusionary criteria
and asked to rule themselves out of participation in the
study if they met any exclusion criteria. If the respondent
met basic study entry criteria, an initial eligibility interview
was scheduled. Potential subjects who met screening
criteria based on the telephone screen, and who were
willing to participate received a call at a later time from the
project coordinator to schedule an appointment for partic-
ipation in the study.
2.1.1.1. Subjects. Four hundred four subjects were
included in the study. Subjects were screened with the
Structured Clinical Interview for DSM-IV Axis I
Disorders the (SCID-I), and were asked about the use
of drugs and alcohol for lifetime and past month.
Subjects were excluded for a past history of heavy
sustained abuse for either alcohol (defined as more than
4 drinks per day for a sustained period of N10 years or
regular binging persisting longer than a 10 year period)
or drugs (defined as intoxicated most of the time (50+%
of the waking day) or regular binging that persisted over
110 R.S.E. Keefe et al. / Schizophrenia Research 102 (2008) 108–115

Table 2
Demographics of normative sample
Demographics 20–29 30–39 40–49 50–59 60–69 70–79
Expected Actual Expected Actual Expected Actual Expected Actual Expected Actual Expected Actual
Sex Male 25 23 25 25 50 48 50 51 25 23 25 23
Female 25 26 25 25 50 52 50 53 25 28 25 27
Race African- 7 8 6 10 12 14 11 14 5 6 5 10
American
White/other 43 41 44 40 88 86 89 90 45 45 45 40
Education* b=HS 22 20 21 20 44 43 42 46 26 24 32 31
NHS 28 29 29 30 56 57 58 58 24 27 18 19
Total 50 49 50 50 100 100 100 104 50 51 50 50
*Education level was estimated from Wide Range Achievement Test Reading Score.

longer than a 10-year period). Subjects who were
regularly engaging in the abuse of illicit substances
currently were not included in the study. These criteria
for inclusion were liberal to allow a realistic comparison
to clinical samples, especially patients with schizophre-
nia, many of whom have histories of substance abuse. If
drug use occurred every day in the 3 days prior to
testing, or if alcohol use exceeded 4 drinks every day in
the 3 days prior to testing, subjects were excluded. No
subjects who would be considered in withdrawal from
substances were included.
We aimed for the subjects in this sample to have a
distribution of sex, age, education, and ethnicity similar
to the U.S. population, as indexed by the 2005 United
States Census data (U.S. Census Bureau, 2005). Age
was the primary demographic variable of interest since it
has the greatest impact on cognition. Sex, education
level, and race were also considered. Subjects were
required to have English as their primary language.
2.1.1.2. Age. The mean age for the sample was 49.60
with a standard deviation of 14.93. The 400 participants
were divided up into 6 age groups with the 40s and 50s
decades having the most number of participants. The
projected number of participants in each of the age range
was as follows:
• 20–29 years old—50 participants
• 30–39 years old—50 participants
• 40–49 years old—100 participants
• 50–59 years old—100 participants
• 60–69 years old—50 participants
• 70–79 years old—50 participants.
2.1.1.3. Sex. Within each age group, an equal number of
men and women were sought. The final sample includes
52.2% women and 47.8% men (women: N = 211; men:
N = 193).
2.1.1.4. Education level. Educational attainment was
examined in two ways. First, we collected information
from the subjects on the highest grade of school they had
completed. There were very few subjects who did not
obtain a high school degree (N = 8) or obtained a high
school Graduate Equivalency Diploma (GED; N = 17).
These subjects were thus combined with high school
graduates into a single group of subjects with “less than or
equal to high school.” Those with some college but no
degree through doctoral degree were included in the group
“greater than high school.” However, since we wanted the
results of this study to have greater generalizability
beyond the immediate geographical area, we estimated
academic achievement using the WRAT Reading score
(Jastak and Wilkinson, 1993). We thus stratified subjects
according to their WRAT Reading-estimated educational
attainment along with their age, sex, and race. According
to the 2005 Census data there was little variation in the
percentages of educational level between the 20–59 age
groups. Approximately 43% had a less than or equal to
high school education and approximately 57% had a
greater than high school degree. According to the Census,
people in the 60–69 age group had approximately 52%
less than or equal to high school education and the 70–79
age group had 64% of the people with a less than or equal
to high school education. Our final sample included
45.2% (N = 183) with a WRAT Reading-estimated educa-
tion less than or equal to high school and 54.8% (N = 220)
with greater than high school. Table 2 below shows the
distribution of the sample by age group and estimated
educational level. The mean WRAT-Reading standardized
score for the sample was 100.7 and the standard deviation
was 12.1, highly consistent with the reference group. One
subject did not complete the WRAT Reading Test and two
subjects did not report their highest education obtained.
2.1.1.5. Race. The two major race groups included
White/Other and African-American, as a breakdown by
 R.S.E. Keefe et al. / Schizophrenia Research 102 (2008) 108–115
any additional racial groups would create cells that were
so small that they would yield unstable data. Since we
did not include subjects for whom English was not the
primary language, the representation of Hispanic or
Latino subjects was smaller than in the general U.S.
population. In the 2005 U.S. Census data, the racial
distribution is approximately 88% White/Other and
approximately 12% African-American. These numbers
were similarly represented in the current sample. The
racial distribution of this sample was as follows: an
average of 15.3% of participants was African-American
and 84.7% of the participants were White/Other.
Considering all of the demographic variables together,
the targeted numbers for each demographic group and
the actual numbers are included in the normative sample
are listed in Table 2.
2.1.2. Assessment procedures
Subjects were tested in person by trained and
experienced research assistants on the Brief Assessment
of Cognition in Schizophrenia (BACS) and the WRAT-
Reading (III) subtest to estimate premorbid IQ.
Following is a description of the six subtests of the
BACS:
List learning (Verbal Memory): Subjects are read a
list of 15 words and then asked to recall as many as
possible. This procedure is repeated 5 times. There are
eight alternate forms designed to minimize practice
effects. This test is designed to measure episodic
memory functions. Measure: number of words recalled
per trial. Time: 7 min.
Digit Sequencing Task (Working Memory): Subjects
are presented with randomly ordered clusters of
numbers with steadily increasing trial length. They are
asked to report the numbers in order, from lowest to
highest. This test is designed to measure working
memory. Measure: number of trials with all items in the
correct order. Time: 5 min.
Token Motor Task (Motor Speed): Subjects are given
100 plastic tokens and asked to pick up one token with
each hand simultaneously as rapidly as possible for 60 s
and place them into a container. This test is designed to
measure motor speed. Measure: the number of tokens
placed into the container. Time: 3 min.
Verbal Fluency (Processing Speed). Category
Instances (Semantic Fluency): Subjects are given 60 s
to produce as many different words as possible within
the animals category. Controlled Oral Word Association
Test (Letter Fluency): In two separate trials, subjects are
given 60 s to generate as many words as possible that
begin with a given letter, here F and S. These two tests
are designed to measure processing speed and the
111
Table 3
Mean raw score performance for all healthy control subjects on all
BACS subtests and composite score
BACS subtest N Mean S.D.
Verbal memory 403 45.7 9.6
Token motor 392 67.8 15.1
Digit sequencing 404 21.2 3.9
Verbal fluency 403 51.3 12.2
Symbol coding 401 55.7 12.6
Tower of London 401 16.7 3.6
Composite Score (T) 404 50 10.0
intactness of the semantic system. Measure: number of
words generated per trial. Time: 5 min.
Tower Test (Executive Functions/Reasoning and
Problem Solving) Subjects look at two pictures
simultaneously. Each shows 3 different-colored balls
arranged on 3 pegs, with the balls in a unique
arrangement in each picture. The patient is required to
accurately estimate the total number of times the balls in
one picture would have to be moved in order to make the
arrangement of balls identical to that of the other,
opposing picture, while employing the standard rules
employed in tower tests (balls are moved one at a time
and balls on top of other balls must be moved first).
There are two alternate forms. The test is designed to
measure executive and problem solving abilities. Mea-
sure: number of trials where the correct number of
moves is the response. Time: 7 min.
Symbol Coding (Attention and Processing Speed) In
this test, subjects write numerals 1–9 as matches to non-
meaningful symbols on a response sheet for 90 s, as
based on a key provided to them. This test is designed to
measure attention and processing speed. Measure:
number of correct items. Time: 3 min.
3. Results
Each of the measures were found to be normally
distributed, which allowed subsequent analyses to
proceed without transformation. The mean performances
for all healthy control subjects on all of the tests including
the composite score, are presented in Table 3. Raw scores
for each test are presented by sex and age group in Table 4.
z-scores for each test and the composite score are also
presented by sex and age group in Table 4.
The relationship between performance on the various
BACS subscales and the demographic factors of age,
sex, race and education was examined with linear
regression analyses:
Age. Greater age was significantly correlated with
reduced performance on all tests: Verbal Memory (r =
− .38), Token Motor Task (r = − .44), Digit Sequencing
112 R.S.E. Keefe et al. / Schizophrenia Research 102 (2008) 108–115

Table 4
Mean performance for each measure across age groups
Age group: 20–29 Age group: 30–39 Age group: 40–49 Age group: 50–59 Age group: 60–69 Age Group: 70–79
Mean S.D. Z Mean S.D. Z Mean S.D. Z Mean S.D. Z Mean S.D. Z Mean S.D. Z
Verbal Memory
Male 46.83 8.56 .12 47.92 9.17 .23 45.27 9.20 −.04 40.71 8.76 − .51 41.96 10.28 − .38 35.43 8.15 − 1.06
Female 53.92 6.41 .86 50.00 7.25 .45 50.71 8.27 .52 46.70 8.16 .11 45.61 10.79 − .01 41.11 7.23 − .47
Token Motor
Male 78.43 8.82 .70 72.42 12.13 .31 65.87 14.38 −.13 60.78 14.59 − .46 60.78 13.88 − .46 52.10 15.17 − 1.04
Female 79.13 8.78 .75 74.40 12.52 .44 75.98 13.99 .54 68.29 14.07 .03 67.79 11.76 .00 57.04 12.81 − .71
Digit Sequencing
Male 22.09 3.66 .23 22.80 3.11 .41 22.21 3.99 .26 21.69 3.86 .12 21.61 4.41 .10 19.22 3.77 − .50
Female 21.81 4.24 .15 21.00 4.03 − .05 21.92 3.57 .18 20.91 3.82 − .07 18.29 3.56 − .74 19.33 3.50 − .47
Verbal Fluency
Male 51.13 9.61 − .01 55.88 11.63 .38 53.09 11.65 .15 50.04 12.15 − .10 51.91 14.63 .05 42.09 12.83 − .76
Female 54.31 11.51 .25 61.16 9.45 .81 55.13 13.14 .32 51.83 11.05 .04 47.29 12.78 − .33 46.56 9.55 − .39
Symbol Coding
Male 66.74 12.03 .88 62.20 10.50 .52 54.10 11.30 −.12 50.32 11.28 − .43 51.30 8.69 − .35 42.52 11.09 − 1.05
Female 67.69 8.51 .96 65.48 12.15 .78 58.62 11.31 .23 57.00 9.69 .11 49.04 9.69 − .53 45.08 6.44 − .84
Tower of London
Male 18.22 2.30 .41 17.40 3.06 .18 17.33 3.52 .16 16.92 3.17 .05 15.82 4.01 − .25 15.14 5.07 − .44
Female 17.96 2.14 .34 17.60 2.60 .24 16.38 4.81 −.10 16.40 3.47 − .09 15.68 4.32 − .29 16.00 2.17 − .20
Composite Score
Male – – .58 – – .50 – – .07 – – − .33 – – − .32 – – −1.21
Female – – .82 – – .66 – – .42 – – .03 – – − .47 – – − .77
z-scores are calculated using the overall means and standard deviations of all 404 controls.

(r = − .22), Verbal Fluency (r = − .20), Symbol Coding
(r = − .55), and Tower of London (r = − .21), all P b.001.
Planned contrasts suggest that the subjects in their
20s and 30s did not differ significantly on any of the
measures. The composite z-score for subjects in their
20s was 0.5 and for subjects in their 30s was 0.33. This
difference in performance was not statistically signifi-
cant (P N.15). All other age groups were statistically
significantly different from each other. Thus, normative
tables include the following age groups: 20–30s, 40s,
50s, 60s and 70s.
Sex. Women and men had very similar WRAT scores
and education levels. After accounting for age, sex
accounted for only 2% of the variance in composite
scores. For individual measures, the amount of variance
accounted for was as follows: Verbal memory, 7%
(women better); token motor, 4% (women better); for
the other measures 2% or less. However, since the mean
performances between males and females were different,
up to .5 S.D.s on verbal memory overall, and up to .4 S.D.s
in some age groups on the composite score, a decision was
made to organize the normative tables by age and sex,
which will allow investigators to correct for both of these
factors when calculating BACS standardized scores.
Education. The correlation between education level
and the uncorrected composite score was r = .35; the
correlation with verbal fluency was r = .39. The correla-
tions between education and other BACS test measures
were small (b.2). Table 5 indicates that subjects who
reported that they did not have education beyond high
school or did not finish high school performed about
0.27 standard deviations below the group mean, while
those subjects who reported attending or finishing
college and beyond performed 0.27 standard deviations
above the group mean. The differences between high-
and low-education subjects based upon estimates from
WRAT Reading scores was larger, but since this
comparison may have increased differences based
upon current general performance factors, we used the
reported education as the primary comparison for this
analysis. We considered including education-corrected
norms, but decided against this for two main reasons: if
normative tables included stratification based upon high
and low education, the number of subjects per cell
would be too small to yield reliable estimates; also,
using education as an adjustment in schizophrenia will
deflate the magnitude of deficits, referred to as the
matching fallacy (Meehl, 1970). However, to generally
adjust for education levels it is possible to add a T-score of
2.7 for those who did not have education beyond high
school and to subtract a T-score of 2.7 for those who had
some college or higher levels of education. This correction
is not recommended for patients with schizophrenia, who
may never attain their potential education levels.
 R.S.E. Keefe et al. / Schizophrenia Research 102 (2008) 108–115 113

Table 5 The component test T-score will be derived by multi-

BACS composite score based on highest degree obtained and grade plying Cijk by 10 and adding 50:
level estimated from WRAT-reading score
Highest degree N Mean BACS S.D. Tjkl ¼ 104Cjkl þ 50:
Composite Score
Less than High School 9 − 1.146 1.256
The BAC composite score for subject l at time point
GED, High School 17 − .836 .985 k is computed by taking the sum of the six scaled test
High School 175 − .174 .970 scores:
Associates degree or 68 .061 .939
some college 6
Bachelors degree 82 .436 .858 ACkl ¼ R Cjkl
Masters degree 38 .303 .834 j ¼ 1;
Doctorate 13 .201 1.211
dividing by the standard deviation of the sum of the
All High School Grads 201 − .272 1.013
scaled scores from the index population, multiplying by
and less
Greater than 201 .269 .9148 10, and adding 50:
High School
Tkl ¼ 104ACkl =ðS:D:mn Þ þ 50;
WRAT-estimated 183 − .538 .938
grade b=HS where S.D.mn is the standard deviation of the sum of the
WRAT-estimated 220 .448 .815
scaled scores for the index population for sex m and age
grade NHS
category n.
BACS, Brief Assessment of Cognition in Schizophrenia.
Table 6 describes the correlations among the test
WRAT, Wide Range Achievement Test Reading Score.
GED, General Equivalency Diploma. measures and composite score. We also used principal
HS, High School. components analysis to determine whether a composite
score that sums test values weighted according to the
weights of the first principal component might differ
from the standard method described above. However, the
correlation between these two composite measures was
3.1. Calculation of scaled scores, composite z-scores
r = 1.0, suggesting that there was literally no difference
and T-scores
between them.
The raw mean for each version of the Verbal Memory
The primary outcome measure from the BACS is a
test is presented in Table 7, along with the means adjusted
composite score that averages the standardized scaled
for WRAT-Reading score and age. Each version had
scores from each of the six tests. The z-score has a mean
approximately the same number of subjects and there was
of 0 and a standard deviation of 1. The T-score is an
no significant difference between each version (P N.15).
alternative metric for expressing standard scores and has
These analyses suggested that all eight of the verbal
a mean of 50 and an S.D. of 10. Differences in
memory versions were comparable, with less than 0.5 S.D.s
performance associated with variation in age and sex
between the total scores for the easiest and most difficult
were accounted for in the development of algorithms
versions. Version 2 was slightly more difficult than the rest
that calculate the standardized z-score or T-score.
of the versions, with adjusted performance an average of
The scaled score for each test is computed as follows.
1.4 words worse than the other tests. Version 5 was easiest,
Let Xjkl be the raw score on test j (j = 1 to 6) at time
with subjects performing 0.62 words better than the next
point k (1 = baseline; 2 = Visit 10/ET) for subject l.
easiest version. Thus, while all eight versions were deemed
Assume that subject l has sex m (1 = male and 2 =
similar in difficulty, for trials involving six assessments,
female) and is in age category n (1 = 20–39, 2 = 40–49,
versions 1,3,4,6,7,8 were the most similar to one another,
3 = 50–59, 4 = 60–69, and 5 = 70–79 years). The scaled
and are the most highly recommended.
score is computed as follows:


Cjkl ¼ Xjkl  Mjmn =S:D:jmn ; where 4. Conclusions and discussion

Mjmn and S.D.jmn are the mean and standard deviation The Brief Assessment of Cognition in Schizophrenia
for test j, respectively, of the index population for sex m (BACS), is used in various areas of cognition research.
and age category n. We present here normative data from 404 healthy controls
114 R.S.E. Keefe et al. / Schizophrenia Research 102 (2008) 108–115

Table 6
Pearson correlations between BACS measures
Token Digit Verbal Symbol Tower of Composite score Composite score
motor sequencing fluency coding London (uncorrected) (corrected)
Verbal memory 0.30 0.42 0.41 0.54 0.35 0.75 0.62
Token motor 0.20 0.18 0.43 0.21 0.57 0.40
Digit sequencing 0.36 0.40 0.34 0.67 0.65
Verbal fluency 0.40 0.28 0.65 0.59
Symbol coding 0.38 0.78 0.59
Tower of London 0.63 0.60
*Corrected Composite Score is corrected for age and gender.
BACS, Brief Assessment of Cognition in Schizophrenia.

matched to the 2005 United States Census on age, sex, investigators to correct for both of these factors when
race and education. These norms have been organized in a calculating BACS standardized scores. We considered
manner that will allow the calculation of standardized including education-corrected norms, but decided against
scores (z-scores and T-scores) adjusted for age and sex for since the number of subjects in each of the 24 cells created
each test and for overall composite scores. Program files by such a stratification would have been small, yielding
have been developed that users of the BACS can utilize to unreliable estimates. Also, using education as an adjust-
calculate a individual's or group's T-score or z-score for ment in schizophrenia would have artificially deflated the
each measure and for the composite score. There are a severity of deficits (Meehl, 1970). However, for non-
total of 13 programs that can be used to calculate z-scores schizophrenia patients evaluated after a healthy develop-
and T-scores for each measure and the overall BACS mental period, a correction of 2.7 T-score points can be
composite: one for the entire sample and then one for each applied for subjects who did not have education beyond
sex by age stratification. high school and a correction of −2.7 T-score points for
The relationship of the BACS tests and composite those who had some college or higher levels of education.
scores with age, sex and education followed an expected One of the potential weaknesses of this normative data
pattern. Modest age and education effects were evident base was that all of the subjects were ascertained from the
across most cognitive domains. Lower cognitive perfor- same geographical area in and around Durham, North
mance was associated with greater age and less education. Carolina. This area includes people with a great diversity
After accounting for age, sex accounted for only 2% of the of levels of education, and enabled us to mirror the
variance in composite scores. However, for individual education level of the United States Census. In fact, the
measures, the amount of variance accounted by sex varied. WRAT-Reading standardized score was 100, mirroring
Women performed better on tests of verbal memory, and the average of the general population. However, due to
the token motor test. A decision was thus made to organize our subject-selection method of random-digit dialing,
the normative tables by age and sex, which will allow some of the cells organized by age, sex, race, and edu-
cation were difficult to complete. While the numbers in
each of the cells were eventually consistent with the
Table 7 Census, it is possible that these ascertainment methods
Raw and adjusted means for verbal memory test may be biased for certain groups. For instance, certain
Version N Raw S.D. Adj. Stand. Mean Mean WRAT- groups may be less likely to answer the phone during the
mean mean* error age reading score hours we called, and certain individuals may be less likely
1 54 45.81 10.55 44.94 1.11 48.09 48.83 to agree to participate in a research study. Thus, while
2 50 43.56 11.53 43.43 1.15 48.90 47.96 these data reflect the BACS cognitive performance in
3 49 44.24 8.23 44.83 1.16 51.73 47.84 individuals with the same demographic features as the US
4 50 45.82 7.35 45.57 1.15 48.90 47.84
Census, it is not completely certain that this level of
5 49 47.20 10.93 47.84 1.16 53.20 48.29
6 50 46.50 9.16 46.26 1.15 48.56 48.00 cognitive performance reflects that of the entire United
7 50 46.74 9.31 47.22 1.15 48.71 46.90 States population. Further, since the number of subjects in
8 50 45.40 9.43 45.29 1.15 49.86 48.26 any of the cell sizes for a specific age and sex was
WRAT, Wide Range Achievement Test. relatively small (as low as 25), the use of age and sex
*Adjusted for WRAT-Reading Score and age. corrected scores should be done with caution.
 R.S.E. Keefe et al. / Schizophrenia Research 102 (2008) 108–115
Role of funding source
Funding for this study was provided by Eli Lilly and Company
(F1D-US-X281) and Pfizer, Inc. (# 2005-0285). Neither company had
any further role in the study design, data collection, data analysis,
manuscript preparation or in the decision to submit this paper for
publication.
Contributors
Drs. Keefe, Harvey, Goldberg and Gold designed the study,
contributed to writing the protocol, contributed to writing the
manuscript and conducted statistical analysis. Mss. Walker, Kennel,
Hawkins collected the study data and provided feedback about the
study design during the study implementation. Ms. Walker and Miss
Kennel created data tables and performed some of the data analyses.
All authors contributed to and have approved the final manuscript.
Conflict of interest
Dr. Keefe has received grant/research support from Astra-Zeneca,
Eli Lilly, Johnson & Johnson, Pfizer, and NIMH, as well as providing
educational services to Astra-Zeneca, Eli Lilly, Forest Labs,
GlaxoSmithKline, Johnson & Johnson, Pfizer, and Repligen. He has
also served as a consultant and on advisory boards for various
pharmaceutical companies as follows: Abbott Pharmaceuticals
(advisory board), Acadia (consultant), Astra-Zeneca (advisory board,
consultant), Bristol Myers Squibb (advisory board), Cephalon
(consultant), Dainippon Sumitomo Pharma. (consultant), Eli Lilly
(advisory board, consultant), Forest Labs (consultant), GlaxoSmithK-
line (consultant), Johnson & Johnson (advisory board, consultant),
Lundbeck/Solvay/Wyeth (advisory board), Memory Pharmaceuticals
(advisory board), Merck (advisory board, consultant), Orexigen
(advisory board, consultant), Otsuka (consultant), Pfizer (advisory
board, consultant), Repligen (consultant), Saegis (advisory board,
consultant), Sanofi/Aventis (advisory board, consultant), and Xenoport
(consultant). In addition, Dr. Keefe receives royalties from the Brief
Assessment of Cognition in Schizophrenia (BACS) testing battery and
the MATRICS Battery (BACS Symbol Coding).
Dr. Harvey has served as an advisor or consultant to Astra-Zeneca
Pharmaceuticals, Bristol-Myers Squibb, Eli Lilly and Company, Johnson
and Johnson, Inc., Memory Pharmaceuticals, Novartis Pharmaceuticals,
Pfizer, Inc., SolvayWyeth Alliance, Sanofi-Aventis. He currently or
immediately prior has/had grant or contract support from Astra-Zeneca
Pharmaceuticals, Bristol Myers Squibb, Johnson and Johnson, Inc., and
115
Pfizer, Inc. Dr. Harvey receives royalties from the Brief Assessment of
Cognition in Schizophrenia (BACS) testing battery.
Dr. Goldberg receives royalties from the Brief Assessment of
Cognition in Schizophrenia (BACS) testing battery.
Dr. Gold receives royalty payments from the BACS and has served
as a consultant to Pfizer and Solvay.
Acknowledgments
Portions of the text describing the calculation of the BACS composite
score were based on a statistical analytic plan written by Carol Robinson.
References
Green, M.F., Nuechterlein, K.H., Gold, J.M., Barch, D., Cohen, J.,
Essock, S., Fenton, W.S., Frese, F., Goldberg, T.E., Heaton, R.K.,
Keefe, R.S.E., Kern, R.S., Kraemer, H., Stover, E., Weinberger, D.R.,
Zalcman, S., Marder, S.R., 2004. Approaching a consensus cognitive
battery for clinical trials in schizophrenia: the NIMH-MATRICS
conference to select cognitive domains and test criteria. Biol.
Psychiatry 56 (5), 301–307.
Jastak, S., Wilkinson, G.S., 1993. Wide Range Achievement Test-
Revised 3. Jastak Associates, Wilmington, DE.
Keefe, R.S.E., Goldberg, T.E., Harvey, P.D., Gold, J.M., Poe, M.,
Coughenour, L., 2004. The brief assessment of cognition in
schizophrenia: reliability, sensitivity, and comparison with a standard
neurocognitive battery. Schizophr. Res. 68 (2-3), 283–297.
Keefe, R.S.E., Poe, M., Walker, T.M., Kang, J.W., Harvey, P.D., 2006.
The Schizophrenia Cognition Rating Scale (SCoRS): interview-
based assessment and its relationship to cognition, real-world
functioning and functional capacity. Am. J. Psychiatr. 163, 426–432.
Keefe, R.S.E., Sweeney, J.A., Gu, H., Hamer, R.M., Perkins, D.O.,
McEvoy, J.P., Lieberman, J.A., 2007. Effects of olanzapine,
quetiapine, and risperidone on neurocognitive function in early
psychosis: a randomized, double-blind 52 week comparison. Am.
J. Psychiatr. 164, 1061–1071.
Meehl, P.E., 1970. Nuisance variables and the ex post facto design. In:
Radner, M., Winokur, S. (Eds.), Minnesota Studies in the Philosophy
of Science, vol 4: Theories and Methods of Physics and Psychology.
University of Minnesota Press, Minneapolis, pp. 373–401.
U.S. Census Bureau, Current Population Survey, 2005. Annual social
and economic supplement. http://www.census.gov/population/
www/socdemo/education/cps2005.html.