Download as pdf or txt
Download as pdf or txt
You are on page 1of 13

Research News

and Comment
Defining “Highly Qualified Teachers”: What Does
“Scientifically-Based Research” Actually Tell Us?
by Linda Darling-Hammond and Peter Youngs

In July 2002, the U.S. Secretary of Edu- weak and are underprepared for their Proposition 1: Teachers Matter
cation issued the Secretary’s Annual Report jobs. for Student Achievement,
on Teacher Quality (U.S. Department of • Alternative certification programs but Teacher Education and
Education) as required by the 1998 reau- (ACPs) have academically stronger Certification Are not Related
thorization of Title II of the Higher Edu- recruits who are highly effective and to Teacher Effectiveness
cation Act. In this report titled Meeting the have high rates of teacher retention. The Secretary’s report accurately claims
Highly Qualified Teachers Challenge the The report suggests that its recommen- that “researchers have found that some
Secretary essentially argues for the dis- dations are based on “solid research.” How- teachers are much more effective than
mantling of teacher education systems and ever, none of these arguments has strong others” (2002, p. 7). Studies using value-
the redefinition of teacher qualifications to empirical support, and the report does not added student achievement data have found
include little preparation for teaching. Stat- cite the scientific literature that addresses that student achievement gains are much
ing that current teacher certification sys- them: Only one reference among the re- more influenced by a student’s assigned
tems are “broken,” and that they impose port’s 44 footnotes is to a study that was teacher than other factors like class size
“burdensome requirements” for education eventually published in a peer-reviewed and class composition (Sanders & Horn,
coursework that make up “the bulk of cur- journal, and the study’s findings are mis- 1994; Sanders & Rivers, 1996; Wright,
rent teacher certification regimes” (p. 8), represented in the report. Most references Horn, & Sanders, 1997). A recent analy-
the report argues that certification should are to newspaper articles or to documents sis by Rivkin, Hanushek, and Kain (2001)
be redefined to emphasize higher stan- published by advocacy organizations, some attributes at least 7% of the total vari-
dards for verbal ability and content knowl- of these known for their vigorous opposi- ance in test-score gains to differences in
edge and to de-emphasize requirements tion to teacher education.1 teachers.
for education coursework—making stu- Although an accurate review of rigorous The Secretary’s report asserts, however,
dent teaching and attendance at schools of research on teacher qualifications and their that “there is little evidence that education
education optional and eliminating “other relationship to student achievement could school course work leads to improved stu-
bureaucratic hurdles” (p. 19). These con-
provide useful guidance to state policymak- dent achievement” (2002, p. 19), stating
clusions rest on the following arguments,
ers, such a review is not to be found in this that the evidence about “knowledge of ped-
each of which is addressed in turn in this
report. Instead, the Secretary’s report fails agogy, degrees in education or amount of
article:
to meet the Department of Education’s time spent practice teaching”—which are
• Teachers matter for student achieve-
own standards for the use of scientifically the “requirements that make up the bulk of
ment, but teacher education and cer-
based research to formulate policy. The re- current teacher certification regimes”—is
tification are not related to teacher
port cites almost no research that would surrounded by a “great deal of contention”
effectiveness.
meet scientific standards, misrepresents (p. 8). To support the assertion that “vir-
• Verbal ability and subject matter
findings from a large number of sources, tually all” of the studies linking certifica-
knowledge are the most important
components of teacher effectiveness. and includes many unsupported statements tion and improved student outcomes are
• Teachers who have completed teacher about teacher education and teacher certi- “not scientifically rigorous,” the Secretary’s
education programs are academically fication. Whatever the contributions of report cites a report by Kate Walsh (2001),
this report to the debates on teacher qual- written for the Baltimore-based Abell Foun-
ity, an accurate rendering of the research dation,2 which asserts that there is “no cred-
The Research News and Comment section base on these important topics is not one ible research that supports the use of teacher
publishes commentary and analyses on of them. In this article we discuss the re- certification as a regulatory barrier to teach-
trends, policies, utilization, and contro-
versies in educational research. Like the
search base that treats the arguments made ing” (p. 5). Unfortunately, Walsh’s re-
articles and reviews in the Features and in support of the report’s recommenda- port excludes much of the evidence on the
Book Review sections of ER, this material tions and suggest that different conclusions topic, misrepresents many research find-
does not necessarily reflect the views of would derive from a well-grounded ren- ings, makes inaccurate claims about stud-
AERA nor is it endorsed by the organization.
dering of the evidence. ies that have examined the consequences

DECEMBER 2002 13
of preparation, and uses a double standard clusions. Ultimately, she is unable to pro- est and data on this topic just began to re-
in evaluating the research (see Darling- vide well-grounded criticisms of a number turn in the 1990s. Although some newer
Hammond, 2001, 2002). of methodologically strong studies (some data sets provide more useful information
Walsh’s report was written both to pre- of them reviewed here), which show on questions of teacher education and cer-
sent a case against Maryland’s efforts to strong relationships between student tification, federal education data sets are up-
strengthen teacher preparation require- achievement and teachers’ professional dated relatively infrequently and are made
ments and to defend the continuation of a preparation and certification status. available even more slowly. Although the
local short-term alternative route into teach- Indeed, a recent review commissioned age of studies can sometimes influence their
ing that had come under criticism. It also by the Office of Educational Research and applicability to current contexts, and they
attacks the state’s efforts to require courses Improvement that was vetted for scientific must be interpreted with these questions in
in the teaching of reading for all teachers, rigor by a panel of researchers disagrees mind, they do not become invalid merely
characterizing them as additional “barri- with both Walsh’s conclusions and the because they are old. Walsh cites numerous
ers” to the ability to teach. Suggesting that Secretary’s report. This review, which an- studies that are very old—for example,
state and local policymakers are misguided alyzes 57 studies that met specific research Bowles and Levin (1968), Coleman et al.
in their efforts to seek more fully certified criteria and were published after 1980 in (1966), and Massey and Vineyard (1958)
teachers for Baltimore City schools, Walsh’s peer-reviewed journals, concludes that the for the proposition that verbal ability mat-
report ridicules the reports of local jour- available evidence demonstrates a relation- ters (Walsh, 2001, p. 6)—even though she
nalists and a Baltimore community group ship between teacher education and teacher dismisses some of them in her separately
that released data that “bemoaned the fact effectiveness (Wilson, Floden, & Ferrini- published appendix as “too old.”
that more uncertified teachers were teach- Mundy, 2001). The review documents
Sample Size and Methods. Another ar-
ing in the city’s high-poverty, predomi- relationships between teacher qualifica-
gument used to discount many studies is
nantly African-American schools than the tions and student achievement across stud-
the size of their samples. Walsh bemoans
city’s whiter, more affluent schools” (p. 2). ies using different units of analysis and
the lack of experimental research, but she
The report contends that these inequalities different measures of preparation and in then rejects the results of studies with ex-
in access to certified teachers are not prob- studies that employ controls for students’ perimental and matched comparison de-
lematic if certification is discounted as a socioeconomic status and prior academic signs because of their relatively small sample
determinant of achievement. performance. sizes. This is true except when she agrees
To evaluate the weight of evidence in a with the findings of particular studies. For
Defining Scientifically Based Research
field it is often necessary to triangulate example, although Walsh summarily dis-
The Secretary’s report dismisses the impor- findings that used different methods, over misses a set of studies with sample sizes of
tance of teacher preparation by arguing that different time periods, and at different lev- 40 or fewer teachers (p. 25), she cites
the research linking teacher preparation to els of aggregation to see how evidence has Miller, McKenna, and McKenna (1998),
measures of teacher effectiveness is scien- accrued over time and across methods. Of a study that included student achievement
tifically inadequate, referencing Walsh’s course, it is important to do this with at- data from only 18 teachers, for her propo-
report for this assertion. Walsh seeks to tention to the methodological strengths sition that “new teachers who are certified
marginalize much of the scientific research and weaknesses of various studies and lines do not produce greater student gains than
on teacher education by suggesting it is in- of research. We address some of the meth- new teachers who are not certified” (p. 8).
appropriate to cite studies that are older, odological issues that have surfaced as ar- (In fact, as we will discuss, this study did
have relatively small samples, use measures guments for discounting the results of the not include uncertified teachers, but was a
of performance other than student achieve- body of research on teacher preparation. study of an alternative teacher education
ment scores, are aggregated at a level above program.) In the original version of her
the individual teacher or classroom, or Age of Studies. Walsh’s report discounts report,3 she also cites Bullough, Knowles,
have been published in venues other than a large number of studies of teacher educa- and Crow (1989), a study of three student
peer-reviewed journals. Although Walsh tion and certification because they were con- teachers, for her proposition that preservice
rejects or ignores research findings that ducted before 1980. Yet, this is when most training “fades quickly from new teachers’
suggest the influence of teacher educa- studies using experimental or matched com- minds” and Hawk, Coble, and Swanson’s
tion on student learning, she cites com- parison designs were completed. This is in (1985) matched comparison study of
parable research—sometimes the very same part because there was great variability in 36 teachers as evidence for the influence of
studies—when they agree with her asser- entry pathways and much interest in the teachers’ subject matter knowledge on stu-
tions about verbal ability or content topic during the high-demand years of the dent achievement (although she discounts
knowledge (e.g., Ferguson, 1991; Fetler, 1960s and 1970s and because federal fund- its findings for the effects of certification).
1999; Goldhaber & Brewer, 2000; Hawk, ing for educational research was substan- Her arguments that certification rules
Coble, & Swanson, 1985; Monk, 1994; tially larger before 1980 than it has been should be relaxed are made on the basis of
Strauss & Sawyer, 1986). She also cites since. Fewer studies were concerned with anecdotes about three individual teachers
studies that do not meet the criteria for age, the issues of preparation and certification in (pp. 38–40).
size, outcome measures, aggregation level, the 1980s when virtually all teachers were The larger correlational studies on which
or publication venue she uses to discredit certified, and few data sets included mea- Walsh often relies typically do not include
studies whose findings challenge her con- sures of teacher education variables. Inter- variables that measure teacher education

14 EDUCATIONAL RESEARCHER
directly, lack direct controls, and must rely tended construct or in fact be reflecting the practices and outcomes inform medical re-
on statistical manipulations of data to ac- influences of other unmeasured variables. search, so do highly aggregated data at the
count (indirectly) for these other influ- For example, many studies finding strong level of cities, counties, and even countries
ences. This kind of correlational research influences of measures of teacher verbal when researchers seek to understand, for
is, of course, legitimate for staking out ability on student achievement have lacked example, why women in some nations have
broad possibilities in relationships among other measures of teachers’ preparation low levels of breast cancer or men have low
variables, but it has its own limitations. that, when examined in other studies, are levels of heart disease.
Many experimental designs and matched also strong predictors. Furthermore, many Although the size of measured effects of
comparison studies can in fact offer more of the variables that reflect teacher quality different variables can vary at different lev-
solid evidence about effects because the are highly correlated with one another—for els of the system, it is not always clear how
“treatment” they are studying is known and example, teachers’ education levels are typ- the bias operates. Often, the general direc-
the samples can be better controlled than ically correlated with age, experience, and tion of the results holds at different levels of
in studies that use proxies and statistical general academic ability, and certification the system, even if effect sizes differ. For ex-
controls rather than direct observation of status is often correlated with content back- ample, Ferguson and Ladd (1996) found
the phenomena of interest. ground as well as education training and ex- the effects on student achievement of teach-
Medical research, for example, typically perience (e.g., Goldhaber & Brewer, 2000). ers’ test scores, master’s degrees, and expe-
uses small sample experimental research as The effects reflected by any given vari- rience held at both the district and school
the basis for establishing the possibilities of able in a particular study depend on levels in terms of significance and direc-
effects and uses large correlational studies whether other variables that may also mea- tionality. There are advantages and limita-
as rough indicators of possible relation- sure aspects of competence are represented tions for different levels of analyses. On
ships that require further examination. in the estimates. The effect size also de- the one hand, disaggregated data can ex-
Single case studies of clinical findings are pends on other context factors, such as the hibit greater measurement error. On the
part of the medical research base along with range of variability in the measure used, other hand, some analysts have argued that
small, carefully controlled experiments, which can change in different locations omitted variables may bias the coefficients
small and large clinical trials, and corre- and time periods. For example, in some of school input variables upward when
lational studies looking at broad tenden- eras and in some locations virtually all data are aggregated to the district or state
cies. Although medical researchers generally teachers held content degrees or were fully level (Hanushek, Rivkin, & Taylor, 1995).
consider correlational studies to compose certified, so these variables do not strongly However, this generalization does not al-
a weaker source of definitive evidence predict variations in outcomes. When much ways prove true. For example, Summers
about effects than experimental designs, re- more variability is present, these variables and Wolfe (1975) found that selectivity
searchers recognize that mixed methods of are strongly predictive of outcomes. Thus, ratings of each teacher’s undergraduate
research serve complementary purposes. several studies have found strong measured institution were important in explaining
For example, the January 2002 issue of the influences of certification status on student sixth-grade students’ achievement when ex-
prestigious New England Journal of Medi- achievement in states like California and amined at the individual teacher level; how-
cine includes a study with a sample of eight Texas during the 1990s when there were ever, this relationship disappeared when the
patients who received cardiac transplants wide differences in teachers’ qualifications. authors aggregated the college ratings and
(Quaini et al., 2002) and a study of 53 chil- For all of these reasons, it is critical for any other school inputs into school-level aver-
dren infected with E Coli (Chandler et al., review of research to represent a range of ages. This contradicts the assumption about
2002)—neither of which had experimental studies that can shed light on the different the usual direction of aggregation bias.
designs—along with a study of 750,000 relationships of interest using a variety of Of course, omitted variables can bias re-
Norwegian women whose birth outcomes measures. sults at any level of the system. Sometimes,
were examined via medical records (Skjoer- especially when the goal of a study is to
ven, Wilcox, & Lie, 2002). The usefulness Level of Aggregation. Another criticism evaluate broad trends and policy influ-
of small, comparison group studies—as used to dismiss some studies’ findings as ir- ences, it is important to have data aggre-
well as large correlational studies that use relevant is the charge of “aggregation bias.” gated and analyzed at multiple levels. For
grosser measures—is not in the definitive- For example, Walsh dismisses studies that interpreting the weight of evidence on a
ness of their individual findings but in include favorable findings about the value particular issue, the most important ques-
their contribution to a larger body of work of teacher education in which data are ag- tion is whether consistent results are found
from which evidence can be triangulated. gregated at the level of the school or district, at different levels of aggregation. With these
Of course, one of the reasons correla- although she cites similarly aggregated data concerns in mind, we discuss the actual
tional studies must be interpreted with for her conclusion that verbal ability mat- findings of research on the questions raised
caution is that there is always the question ters most (e.g., Coleman et al., 1966; Fer- in the Secretary’s report.
of which direction the correlations may guson, 1991; Strauss & Sawyer, 1986).
point, sometimes referred to as “reverse More important, this critique misses a cru- Research on Teacher Education
causation.” There is also the problem that cial point about how research results accrue and Certification
variables in these studies are frequently and are triangulated to look at possible rela- A variety of teacher experiences and at-
crude proxies for the actual measures of in- tionships among conditions and outcomes. tributes appear to contribute to the effects
terest and may either fail to capture the in- Just as individual-level data about health that teachers have on student learning.

DECEMBER 2002 15
Looking across studies, several aspects of units of analysis reinforces the strength of school certification or are not certified in
teachers’ qualifications have been found to inferences that might be drawn from any their subject area have a negative (though
bear some relationship to student achieve- single study. not statistically significant) impact on sci-
ment. These include teachers’ (a) general ence test scores. (p. 139)
Individual Teacher-Level Data. The
academic and verbal ability; (b) subject The effect of certified teachers on stu-
only study cited in the Secretary’s report
matter knowledge; (c) knowledge about dent achievement was larger in both math-
that was eventually published in a peer-
teaching and learning as reflected in teacher ematics and science than the effect of
reviewed journal is Goldhaber and Brewer’s
education courses or preparation experi- content degrees at the bachelor’s and mas-
(2000) examination of the relationship
ences; (d) teaching experience; and (e) the ter’s degree levels. The fact that the study
between teacher qualifications and stu-
combined set of qualifications measured found a large effect of certification status
dent achievement using data from the
by teacher certification, which includes after controlling for content major sug-
National Educational Longitudinal Stud-
most of the preceding factors (Darling- gests that what certified teachers learn
ies (NELS) of 1988.5 The Secretary’s re-
Hammond, 2000). about teaching adds to what they gain
port cites this study as its only reference for
As the state’s legal vehicle for establish- from a strong subject matter background.
an inaccurate statement that subject matter
ing competence for members of profes- degrees have a greater effect on teacher ef- The Secretary’s report misinterprets the
sions, including teaching, licensing, or fectiveness than certification: findings from this study yet a second time
certification is meant to represent the min- stating, “there was no statistical difference
imum standard for responsible practice. Research has generally shown that high in performance between teachers who at-
Current requirements for licensing include school math and science teachers who tended conventional training programs and
have a major in the subjects they teach received traditional teaching licenses versus
measures of many of the variables we
elicit greater gains from their students
noted, such as basic skills and general those who did not complete such programs
than out-of-field teachers, controlling for
academic ability, knowledge about sub- student’s [sic] prior academic achievement
and were teaching on emergency or tempo-
ject matter, knowledge about teaching and socioeconomic status. These same rary certificates” (2002, p. 8). In fact, Gold-
and learning, and some teaching experi- studies also suggest that possessing an un- haber and Brewer’s study does not include
ence.4 Over the past decade, states have dergraduate major in math and science data about which teachers had attended
taken steps to strengthen their licensure has a greater positive effect on student “conventional” or other training programs
requirements, which are now substantially performance than certification in those or which had received “traditional teaching
stronger than they were 15 years ago. In subjects. (p. 8) licenses.” NELS only included information
most states, candidates for teaching must on the type of certificate teachers held in
In fact, Goldhaber and Brewer (2000)
earn a minimum grade point average or the specific mathematics or science field
found strong influences of teacher certifi-
achieve a minimum test score on tests of taught.
cation on student achievement in high
basic skills, or general academic ability or The study did find that students of the
school mathematics and science, above
general knowledge (or both) in order to be sample’s small number of science teachers
and beyond the effects of teachers’ subject
admitted to teacher education or gain a with temporary or emergency certification
matter degrees. They report:
credential. In addition, they must generally in the science field they were teaching (24
We find that the type (standard, emer- out of the 3,469 teachers in the overall
secure a major or minor in the subject(s)
gency, etc.) of certification a teacher holds sample) did no worse than the students of
to be taught or pass a subject matter test, is an important determinant of student teachers holding standard certification in
take specified courses in education, and, in outcomes. In mathematics, we find that that field; however, both groups of students
some states, pass a test of teaching knowl- students of teachers who are either not cer- did better than the students of uncertified
edge and skill. In the course of teacher ed- tified in their subject (in these data we can-
teachers. Another analysis of these data
ucation and student teaching, candidates not distinguish between no certification
and certification out of subject area) or
(Darling-Hammond, Berry, & Thoreson,
are typically judged on their teaching skill, 2001) shows that in this sample, about
professional conduct, and the appropriate- hold a private school certification do less
well than students whose teachers hold a two thirds of the teachers on temporary or
ness of their interactions with children. emergency certificates were experienced
standard, probationary, or emergency cer-
Studies employing national, state, and tification in math. Roughly speaking, hav- and had education training comparable to
other data sets have reported significant ing a teacher with a standard certification that of the certified teachers, suggesting
relationships between teacher education in mathematics rather than a private school that they had likely completed teacher ed-
and certification measures and student certification or a certification out of sub- ucation programs. The pattern of their
performance at the levels of the individ- ject results in at least a 1.3 point increase qualifications and experiences suggested
ual teacher (Goldhaber & Brewer, 2000; in the mathematics test. This is equivalent many were already licensed teachers from
Hawk, Coble, & Swanson, 1985; Monk, to about 10% of the standard deviation
out-of-state who were typically hired on a
on the 12th-grade test, a little more than
1994); the school (Betts, Rueben, & Da- temporary license while they secured a new
the impact of having a teacher with a BA
nenberg, 2000; Fetler, 1999; Fuller, 1998, state license, and some were experienced
and MA in mathematics [italics added].
2000); the school district (Ferguson, 1991; Though the effects are not as strong in teachers teaching out of their main field,
Strauss & Sawyer, 1986); and the state magnitude or statistical significance, the which was frequently another mathematics
(Darling-Hammond, 2000). The conver- pattern of results in science mimics that in or science field.6 Only a third of this sam-
gence of findings in studies using different mathematics. Teachers who hold private ple were new teachers whose characteristics

16 EDUCATIONAL RESEARCHER
suggested they had a content background ucation; more professional training in how In a second set of studies, Fuller (2000)
with little education training, as the Secre- to work with diverse student populations found that the percentage of properly cer-
tary’s report presumes. In an analysis of co- (a combined measure of training in cul- tified Algebra I teachers in a school was
variance that controlled for students’ pretest tural diversity, teaching limited English positively and significantly associated with
scores and teachers’ degrees and experience, proficient students, and teaching students gains in student achievement after control-
the students of this subsample of teachers with special needs); and more training in ling for student and school characteristics.
had lower achievement than those of the how to develop higher order thinking skills. District-Level Data. Researchers using
more experienced and traditionally trained Similarly, students whose teachers majored data aggregated at the district level also re-
teachers (Darling-Hammond, Berry, & in science or science education and had
Thoreson, 2001). port significant relationships between teach-
more training in how to develop laboratory ers’ scores on certification tests and student
Other research on teacher certification at skills and engage in more hands-on performance. In a study of nearly 900 Texas
the individual teacher-level is consistent learning do better on the NAEP science school districts that controlled for student
with these findings. In a matched compar- assessments. (The NAEP of 1998 asked background and district characteristics, Fer-
ison group study of 36 middle school math- teachers to report either college coursework guson (1991) reports that combined mea-
ematics teachers and 826 students in North or in-service training in these areas.) sures of teachers’ expertise—scores on a
Carolina where teachers were matched by
School-Level Data. Several school-level state licensing examination, master’s de-
years of experience and school setting,
analyses provide further evidence that teach- grees, and experience—accounted for more
Hawk, Coble, and Swanson (1985) found
ers’ certification status is related to student of the interdistrict variation on students’
that the students of fully certified mathe-
achievement. Three recent school-level reading achievement and achievement gains
matics teachers experienced significantly
studies in California found significant in Grades 1 through 11 than students’ race
higher gains in achievement than those
negative relationships between average and socioeconomic status. Of the teacher
taught by teachers not certified in mathe-
student scores on the state examinations qualification variables, the strongest rela-
matics. The differences in student gains
and the percentage of teachers on emer- tionship was found for scores on the state
were greater for algebra classes than gen-
gency permits, after controlling for stu- licensing examination, the Texas Exami-
eral mathematics.
dent socioeconomic status and other school nation of Current Administrators and
Teachers’ education coursework has also Teachers, which is described by the test
characteristics (Betts, Rueben, & Dannen-
been found to add to the influences of sub- developer as measuring basic communica-
berg, 2000; Fetler, 1999; Goe, 2002). All
ject matter knowledge in predicting student tion skills, research skills, and teaching
of these studies also found smaller positive
achievement. For example, using data on knowledge. Master’s degrees also exerted a
relationships between student scores and
more than 2,800 students from the Longi- small but significant influence on student
teacher experience levels, with negative ef-
tudinal Study of American Youth (LSAY), achievement, followed by experience.
fects on student achievement associated
Monk (1994) found that teachers’ college In another district-level study, Strauss
with the proportion of beginning teach-
coursework in the subject field was usually and Sawyer (1986) report that student per-
ers. These studies join a number of others
positively related to student achievement formance in North Carolina districts was
in finding that, among school resources,
in mathematics and science, and education teacher qualifications often appear to have strongly associated with teachers’ average
courses in subject matter methods had a the greatest influence on what students scores on the National Teacher Examina-
positive effect on student learning at each learn and that qualified teachers are un- tions (NTE). The NTE Core Battery fea-
grade level in both fields. In mathematics, equally allocated to students by race, in- tured components measuring teacher’s
these methods courses had “more powerful come, and location. basic skills, general knowledge, and profes-
effects than additional preparation in the Similarly, Fuller (1998, 2000) found sional teaching knowledge. When the au-
content area” (p. 142). Monk concludes that students in Texas schools with greater thors controlled for student, school, and
that, “a good grasp of one’s subject area is a proportions of certified teachers were sig- community characteristics, they found that
necessary but not a sufficient condition for nificantly more likely to pass the Texas As- teachers’ NTE scores had a significant and
effective teaching” (p. 142). sessment of Academic Skills (TAAS), after large effect on students’ performance on the
More recently, Wenglinsky (2000) used controlling for students’ socioeconomic state competency examinations in reading
data from the National Assessment of Ed- status and teacher experience. In one set of and mathematics. In particular, the authors
ucational Progress (NAEP) to examine the studies, he found that the likelihood of el- report that a 1% increase in the district av-
relationships between teachers’ training, ementary school students passing all sub- erage NTE score was associated with a 3%
teaching practices, and student achieve- tests of the TAAS was greater in schools to 5% decline in the district failure rate on
ment, controlling for student characteris- with higher proportions of certified teach- the competency exams. They conclude:
tics and other school inputs. He found ers, controlling for teacher experience, and
Of the inputs which are potentially policy-
that eighth-grade students do better on the that gains in pass rates were related to the
controllable,. . .our analysis indicates
NAEP mathematics assessments when they proportion of properly certified teachers, quite clearly that improving the quality of
have had teachers who engage in more with prior achievement and student demo- teachers in the classroom will do more for
hands-on learning emphasizing higher graphics taken into account. The differ- students who are most educationally at
order thinking and who have a major or ences were significant for Hispanic students risk, those prone to fail, than reducing the
minor in mathematics or mathematics ed- and lower income students (Fuller, 1998). class size or improving the capital stock by

DECEMBER 2002 17
any reasonable margin which could be content knowledge are the most impor- fare reform experiment in Gary, Indiana in
available to policy makers. (1986, p. 47) tant attributes of highly qualified teachers” the 1970s. The data set featured reading
(2002, p. 19). Although there is research and vocabulary achievement data on sev-
Although Walsh (2001) cites this study at
that finds relationships between student eral hundred Black students, most of who
least once and Ferguson’s (1991) study no
achievement and some measures of verbal were from low-income families, as well as
fewer than four times in support of her own
ability and content knowledge, there is no measures of teachers’ verbal ability and ex-
propositions (pp. 5–7), she dismisses their
evidence that these areas of knowledge are perience. Controlling for student back-
findings regarding certification by discount-
more consequential to student achieve- ground characteristics, Hanushek found
ing them for “aggregation bias” (p. 27).
ment than knowledge of teaching. First, that teachers’ verbal ability scores affected
State-Level Data. For a study employ- most of the studies that have included mea- students’ reading score gains but not their
ing state-level data, Darling-Hammond sures of verbal ability or content knowl- vocabulary score gains (Hanushek, 1992),
(2000) examined the relative contributions edge have not included measures of teacher which were more strongly influenced by
of teacher qualifications, other school in- education or certification. Second, in many teacher experience. A study by Murnane
puts, and student characteristics to student cases, the relative effect sizes of these mea- and Phillips (1981) using the same data set
achievement across states on the reading sures are no larger than those of teacher found a significant negative relationship
and mathematics assessments adminis- education and certification measures. between teachers’ verbal ability scores and
tered by NAEP in 1990, 1992, 1994, and students’ Iowa Test of Basic Skills (ITBS)
Evidence of the Importance of Verbal
1996. After controlling for student poverty vocabulary score gains after controlling for
or General Academic Ability
and student language background, this teachers’ experience and degree level, rat-
study found that measures of teacher prepa- The Secretary’s report appropriately claims
ings of their undergraduate institutions,
ration and certification were the strongest that “studies have consistently documented
and students’ socioeconomic status. These
correlates of average student achievement the important connection between a
authors found even stronger positive rela-
in reading and mathematics. The most teacher’s verbal and cognitive abilities and
tionships between teacher experience and
strongly significant predictor of achieve- student achievement” (2002, p. 7) but fails
student performance.
ment was the proportion of well-qualified to note some important attributes of these
Similarly, when Summers and Wolfe
teachers, defined as the proportion hold- studies. The research literature on teacher
(1977) considered the effects of teacher at-
ing both full certification and a major in characteristics has been substantially influ-
tributes on sixth graders’ ITBS composite
the field being taught. The proportion of enced by the measures available in data
score gains, they found that although col-
teachers holding certification exerted an sets during particular time periods. Many lege selectivity ratings appeared significant,
additional small positive effect on achieve- studies have evaluated the effects of teach- scores on the NTE Common Examina-
ment and the proportion on emergency ers’ verbal or general academic ability be- tions—which measured general academic
credentials exerted an additional small cause these variables have been available in ability and included English, mathematics,
negative effect. The study concludes: large data sets since the 1960s. On the other social studies, and science components—
hand, data on teachers’ content preparation were negatively related to students’ achieve-
The strength of the “well-qualified teacher” or teacher education experiences have been
variable may be partly due to the fact that ment gains.
included in large data sets only since the Stronger findings are provided by two
it is a proxy for both strong disciplinary
knowledge (a major in the field taught)
early 1990s. In a recent review, Wayne and studies at the school and district levels.
and substantial knowledge of education Youngs (in press) found five studies that (Walsh also cites these studies despite their
(full certification). If the two kinds of observed relationships between teachers’ presumed “aggregation bias.”) Ehrenberg
knowledge are interdependent as sug- verbal or general academic ability and stu- and Brewer’s (1995) reanalysis of data from
gested in much of the literature, it makes dent achievement that met the standard of the Equality of Educational Opportunity
sense that this variable would be more having controlled for students’ socioeco- study (Coleman et al., 1966) examined the
powerful than either subject matter knowl- nomic status and prior achievement. Four influence of several teacher characteristics
edge or teaching knowledge alone. of these studies employed data sets from the on schools’ average student gain scores
In sum, empirical studies employing 1960s and 1970s and none includes mea- while holding constant school, student,
different units of analysis that have exam- sures of teacher education or certification. and community characteristics. They found
ined the influence of teacher education These studies point out how findings that teachers’ verbal aptitude scores were
and certification on student achievement with respect to the importance of a partic- an important determinant of the school-
have often found significant relationships ular measure of teacher ability are sensitive to-school variation in student gain scores,
between these measures of teacher exper- to the specification of regressions, as the with a smaller contribution of teachers’ ex-
tise and student achievement. influences of verbal ability measures trade perience at the elementary level. Ferguson
off with other variables often used as prox- and Ladd (1996) found a positive, signifi-
Proposition 2: Verbal Ability and ies for teacher quality or expertise—col- cant relationship between teachers’ ACT
Subject Matter Knowledge Are lege selectivity, other academic ability test college entrance examination scores and
the Most Important Components scores, higher degrees, and experience. the achievement gains of third and fourth
of Teacher Effectiveness For example, a study by Hanushek graders from Alabama in reading and
The Secretary’s report asserts, “Rigorous (1992) employed a data set from the Gary mathematics in both school- and district-
research indicates that verbal ability and Income Maintenance Experiment, a wel- level analyses. Class size was an important

18 EDUCATIONAL RESEARCHER
additional predictor at the school level, evidence is not very strong” (Hanushek, for teacher effectiveness than knowledge
and teachers’ degree level was an addi- 1996, p. 116). of how to teach.
tional influence at the district level.
Subject Matter Knowledge Proposition 3: Teachers Who
Although these studies suggest that teach-
There is also evidence on the importance of Have Completed Teacher
ers’ verbal or general academic ability ap-
subject matter knowledge to teaching. Education Programs Are
pears related to student achievement, none
Rowan, Chiang, and Miller (1997) found Academically Weak and
of them include measures of teacher edu-
that students who were taught by a teacher Underprepared for Their Jobs
cation or certification. Thus, they cannot
sustain a claim that verbal ability measures with a bachelor’s or master’s degree in The Secretary’s report also makes several
matter more than measures of teachers’ mathematics or one who had scored well misleading assertions regarding the quali-
professional knowledge. In fact, Ferguson on a brief mathematics quiz had higher fications of the teacher workforce. For ex-
and Ladd’s (1996) findings in their Al- gains in achievement in this subject area, ample, in support of the assertion that
but that the effect was quite small—about “our system allows too many poorly qual-
abama study show smaller influences of
0.015 standard deviations in test score ified individuals into the classroom” (p.
teachers’ ACT scores on student achieve-
gains. In a study using data from NELS 12), the Secretary’s report states that “only
ment (and greater influences of master’s
1988, Goldhaber and Brewer (1997) re- 38 percent [of teachers] have an under-
degrees) than Ferguson (1991) found for
port a greater influence on student achieve- graduate or graduate degree in an acade-
the Texas teacher licensing test, which
ment of teachers’ bachelor’s and master’s mic field outside of a school of education”
includes components that come closer to
degrees in the content area taught (e.g., (p. 12). However, the National Center for
evaluating knowledge that is used for teach-
mathematics or mathematics education) Education Statistics (NCES) data cited in
ing. In an article written nearly 20 years
than was true for undifferentiated degrees. the Secretary’s report show that 95% of
ago, Murnane (1983) observes that evi-
A number of studies show the influences high school teachers and 66% of middle
dence about the influence of verbal ability school teachers in 1998 had earned an aca-
was partly a function of the fact that such of subject matter knowledge in conjunction
demic degree in the subject area they were
scores were among the few teacher vari- with knowledge about teaching. As noted
teaching or in subject area education (e.g.,
ables available in large-scale data sets at earlier, Goldhaber and Brewer (2000) found
mathematics or mathematics education)
that time. In his words, a substantial influence of teachers’ degrees
(1999, p. 12). The Secretary may not have
in the content area or content area edu-
Clearly one should not interpret these re- understood that candidates who complete
cation on student achievement, alongside
sults as indicating that intellectual ability a degree in science education, for example,
even larger effects of teacher certification. In
should be the sole criterion used in re- have generally completed a content major
a multilevel analysis of the LSAY data set,
cruiting teachers or that formal teacher or its equivalent plus additional education
Monk and King (1994) report some evi-
training cannot make a difference. In fact, coursework. At many universities, a sci-
dence of cumulative effects of prior as
the lack of evidence supporting formal ence education major requires as much or
preservice training as a source of compe- well as proximate teachers’ subject matter
more science coursework than a regular
tence may be to some extent a result of coursework on student performance in major because candidates must fulfill dis-
limitations in the available data. For ex- mathematics but did not find the same tributional requirements across the sciences
ample, all databases suitable for examin- effects in science. Also as noted earlier, in as well as in an area of concentration.
ing the correlates of teaching effectiveness another analysis of the same data set, New teachers’ levels of content prepara-
as measured by student achievement gains Monk (1994) found that teachers’ content tion have improved since the 1980s, as
pertain to a single school district. Since preparation, as measured by coursework 38 states now require a content major for
there is less variation in training among in the subject field, was usually positively
teachers within a district than among teachers (U.S. Department of Education,
though rarely significantly related to student 2002, p. 30). Fewer than half of all teach-
teachers in the country at large, these data-
achievement in mathematics and science, ers now receive a bachelor’s degree in edu-
bases do not permit the most powerful
and that coursework in teaching methods cation: Most complete another major and
possible tests of the efficacy of alternative
teacher training programs. (p. 565) had a stronger influence than additional complete a minor, double major, or a cre-
coursework in mathematics. Monk’s find- dential in education or secure a master’s
Even strong advocates of the notion that ing is reminiscent of Begle’s (1979) find- degree. NCES data show that the propor-
academic ability matters are not willing ing from the National Longitudinal Study tion of high school teachers holding a
to make the kinds of sweeping assertions of Mathematical Abilities that teachers’ major or minor in their main teaching
found in the Secretary’s report. For exam- coursework in mathematics methods had field increased noticeably in all core aca-
ple, Eric Hanushek is quoted in the Secre- a stronger effect on student achievement demic fields between 1994 and 1998,
tary’s report (p. 7) for his statement that than additional higher level coursework in reaching 90% or more in each area by
“the closest thing to a consistent finding mathematics for a group of already strong 1998 (1999, pp. 19–20).
among the studies is that ‘smarter’ teachers teachers. These studies do not suggest that Requirements have also changed for ele-
who perform well on verbal ability tests do subject matter knowledge is unimportant. mentary teachers. About 10 states require a
better in the classroom.” The Secretary’s However, they do call into question the Sec- subject area major or concentration. These
report does not include Hanushek’s next retary’s assertion that verbal ability and sub- states as well as those that expect degrees in
sentence, which reads, “Even for that, the ject matter knowledge are more important elementary education or interdisciplinary

DECEMBER 2002 19
fields like liberal studies now require specific dents who prepare to teach no longer take the classroom. The survey results were not
content courses across the curriculum that an education major but complete a separate limited either to new teachers or to gradu-
elementary teachers need to teach (National major and enroll in schools of education for ates of schools of education as the Secre-
Association of State Directors of Teacher a credential or master’s degree. Looking at tary’s report suggests. Instead, they are
Education and Certification, 2001). These the full pool of candidates taking licensing reported for all full-time public school
distributional requirements reverse the his- examinations who prepared to teach or en- teachers, including those who entered with-
torical trend in which elementary teachers tered teaching (and using actual scores out preparation. Using the top two cate-
tended to have little or no coursework in rather than self-reported data), the Educa- gories in the Likert scale, the data reveal
fields like mathematics or science. Interest- tional Testing Service found that among that 95% of all teachers felt moderately or
ingly, Walsh’s report (2001, p. 38) suggests 270,000 test takers in 1995 through 1997, very well prepared to maintain order and
that Maryland should eliminate content re- the lowest pass rates on the Praxis II tests discipline in the classroom; 82% felt ade-
quirements for elementary teachers because were experienced by individuals who had quately prepared to implement new meth-
many of the candidates for the state’s short- never enrolled in teacher education (Gito- ods of teaching; 77% of respondents felt
term alternative route could not meet them. mer, Latham, & Ziomek, 1999). On SAT prepared to implement curriculum and
The Secretary’s report also asserts that and ACT tests, those currently enrolled in performance standards; 69% felt prepared
“Research suggests that students enrolled in teacher education slightly outscored Praxis to use student performance assessment
schools of education are not as academically takers who had never enrolled in teacher ed- techniques; 62% felt prepared to address
accomplished as other university students” ucation. Although special education and the needs of students with disabilities; and
(p. 13). Quoting a study “conducted by physical education majors had SAT scores 57% felt prepared to integrate technology
Education Week” (p. 13) (in fact, it appears below the average for all college-bound se- into instruction. In each of these cate-
the study was conducted by NCES), the niors who took the SAT, elementary educa- gories, fewer than 10% of teachers felt
Secretary’s report recounts the statement tion majors did about as well, and teaching “not at all prepared.” In the lowest cate-
that “only 14 percent of the top quartile of candidates in English, science, mathematics, gory, only 53% felt very well or moder-
1992–93 college graduates entered some social studies, and foreign language had ately well prepared to meet the needs of
type of teacher-preparation program, only higher mean SAT verbal and mathematics limited English proficient students and
12 percent actually taught, and a mere scores than all college-bound seniors who 17% of teacher felt not at all prepared—a
11 percent stayed in the teaching profes- took the SAT. Scores for mathematics and finding that the NCES report notes re-
sion through 1997.” Because even smaller science teachers were substantially higher flects both the newness of this expectation
proportions of the top quartile of college than the overall pool of test takers on the and the fact that only 54% of teachers
graduates go into fields like medicine, law, math SAT, and scores for English, science, actually taught limited English proficient
and engineering, it is difficult to know and foreign language teachers were substan- students (1999, p. 48). Of course, because
what readers are intended to make of this tially higher than the overall pool on the this survey was reported for all teachers, it
statement. The retention rate implied by SAT verbal tests. does not reveal the differences in feelings
this analysis (11 of every 12 entrants Finally, the Secretary’s report erro- of preparation for teachers who experi-
stayed for 5 years) would be a good news neously asserts that “a majority of gradu- enced preparation of various kinds.
story, suggesting retention rates for these ates of schools of education believe that There are data on this latter point. In
prepared high-ability teachers of more traditional teacher preparation programs fact, several recent studies reveal that most
left them ill-prepared for the challenges teacher education graduates believe that
than 90%. In fact, however, this appears to
and rigors of the classroom” (2002, p. 15), their programs prepared them well for
be a misstatement of the actual statistics.
citing the following statistics: classroom teaching. For example, a survey
The real question about qualifications is
of Kentucky teachers (Kentucky Institute
reflected in the following claim: According to NCES data, fewer than 36 for Educational Research, 1997) found
Similar data from NCES also suggest percent of new teachers feel “very well that more than 80% of beginning teachers
that schools of education fail to attract prepared” to implement curriculum and who graduated from Kentucky colleges of
the best students. For example, among performance standards, less than 30 per- education felt well prepared for virtually
college graduates who majored in educa- cent feel prepared to integrate technology
all aspects of their jobs. Similarly, well over
tion, just 14 percent had SAT or ACT into instruction and less than 20 percent
70% of the graduates of the California
scores in the top quartile, compared to feel prepared to meet the needs of diverse
students or those with limited English
State University felt well prepared for vir-
26 percent who majored in the social sci- tually all aspects of their jobs, and those
ences and 37 percent who majored in proficiency.
who had student teaching (just over half
mathematics, computer science, or the This is a very misleading statement. In of the total) felt significantly better pre-
natural sciences. In contrast, 25 percent of
fact, the NCES data cited in the Secre- pared—and were viewed as better prepared
uncertified teachers scored in the top
tary’s report are based on surveys that uti- by principals—than those who had com-
quartile on these tests, as did 33 percent
of private-school teachers. (2002, p. 13)
lized a four-point scale; in the surveys, pleted certification through an internship
teachers were asked whether they felt “very program or who had taught on an emer-
The NCES analysis, however, does not well prepared,” “moderately well prepared,” gency credential without student teach-
represent the range of training routes teach- “somewhat well prepared,” or “not at all ing (California State University, 2002a,
ers now pursue because most college stu- prepared” to carry out various activities in 2002b). Finally, a 1998 survey of 3,000

20 EDUCATIONAL RESEARCHER
beginning teachers in New York City rolled in teacher preparation programs than prehensive report is very helpful in its de-
found that teachers who were prepared in those who did not participate in teacher scriptions of state requirements for such
teacher education programs felt signifi- education. Although this study does not routes, but it includes no citations to justify
cantly better prepared for virtually all tasks specifically identify ACPs, it does indicate this claim. We surmise that the Secretary’s
of teaching than those who lacked prepara- that trained teachers perform best on the sweeping claim may refer to an unrefer-
tion or entered teaching through alternative most widely used licensure tests in the enced statement on page 8 of that report,
programs (Darling-Hammond, Chung, & country. The authors note: referring to California’s intern teaching
Frelow, 2002). program and stating that “the retention for
Current teacher education students have
Proposition 4: Alternative the highest passing rates on the licensure the first five years is 86 percent.”
Certification Programs Have tests. Interestingly, those who report they These data are neither national nor ac-
Academically Stronger Recruits, have never been enrolled in a teacher ed- curate for California. Two other versions of
High Rates of Teacher Retention, ucation program have the lowest passing the retention statistic are offered in an ap-
rates . . . These results make it clear that pendix to Feistritzer and Chester’s report,
and Produce More Successful
teacher education programs have an im- both referring to a paper by a California
Teachers portant impact in preparing their stu- agency consultant (McKibbin, 1999) who
The section of the Secretary’s report headed dents to meet the requirements of licensure.
is quoted as placing retention at 87% (p.
“Alternate Routes to Certification: A Model (p. 24)
424) or 85% (p. 431) over 3 years, not
for the Future” is particularly replete with
A possible source for the Secretary’s as- five. This report’s statistics are based in
misinformation. The report claims that
sertion could be an appendix to Feistritzer turn on an earlier report that studied a
“performance on licensure tests is higher
and Chester’s (2002) state-by-state cata- subset of recently funded California intern
among alternate route teachers than tradi-
logue of alternative routes across the programs representing about one fourth of
tionally trained teachers in most states” (p.
country, which is cited elsewhere in the all such programs in the state, which cited
viii) and later (p. 34) refers to a figure in
Secretary’s report. This appendix quotes a retention rate of about 85% for program
the report for a similar assertion that alter-
William Wale, Director of the Texas State graduates over the period of what ap-
nate route pass rates are higher in 70% of
states. However, the figure does not sup- Board for Educator Certification’s Office of peared to be 1 year (McKibbin, 1998). The
port this assertion: Summarizing state Title Educator Preparation, and includes some retention statistic is based on program self-
II self-reports on this issue, the figure is ti- statistics showing pass rates on an unidenti- reports rather than first-hand data and on
tled “Percent of states where those in al- fied teacher certification exam. These show program graduates rather than total pro-
ternative programs of teacher preparation slightly higher pass rates for ACP candi-
gram participants. An independent analy-
have equal or higher pass rates on state as- dates in 1996–1997 than for traditional
sis of the data set examining all program
sessments than those in traditional pro- program candidates in Texas. However, an-
participants indicated that about 80% of
grams.” The figure shows high rates of other report about Texas’s ACPs (Barnes,
intern program participants (pregradu-
equivalent passage on basic skills and con- Salmon, & Wale, 1989) lists “alternative”
programs ranging from university-based ates) appeared to have remained in teach-
tent tests (80–100%) and low rates on pro- ing after a year, although there was a
fessional knowledge tests (45% of states). 5-year bachelor’s plus master’s degree mod-
els or preservice master’s degree programs substantial range across programs, and
In addition to the fact that the figure does only about 60% remained by the 3rd year
not support the Secretary’s assertion, there (which are called alternative because they
are not undergraduate models) to district- of teaching.
are reasons to question what the data rep- These data are comparable to other
resented in the figure are measuring. The run programs that place teachers in class-
rooms after a few weeks of summer training. studies that have found relatively high at-
report later notes (p. 48) that a number of
Thus, these data make it impossible to draw trition for interns during an initial year of
states did not report on their alternate
any conclusions about the kinds of “alterna- teaching undertaken after a few weeks of
routes and that California—a state earlier
tives” the Secretary favors (i.e., those that preservice training. For example, an earlier
cited for its profusion of alternatives—in-
minimize education training). evaluation of the Los Angeles Teacher
dicated in its report that it did not have
such routes. Furthermore, the report notes Trainee program, California’s largest
Teacher Retention
that pass rates in Massachusetts are 100%. district-run internship program, found
In addition, the Secretary’s report incor- that only 80.3% completed the 1st year
However, a recent study on Massachusetts’ rectly asserts that
alternative route program, Massachusetts of training and only 64.6% completed the
Institute for New Teachers (MINT), found Initial evidence suggests that retention 2nd year and received a clear credential
that 56% of MINT’s 2002 recruits failed rates for teachers certified through alter- the year after (Wright, McKibbin, &
to pass the state content test and their nate routes are higher than for teachers Walton, 1987). Another analysis of this
scores were surpassed by test takers state- who enter the classroom through tradi- same program revealed that 53% of the
wide on every test of content knowledge tional routes. Nationwide, about 85 per- recruits had left the district within the
cent of teachers certified through alternate first 5 years of program operation (Stod-
(Fowler, 2002).
routes remain in the classroom five years
As noted earlier, Gitomer, Latham, and dart, 1992). Comparable attrition rates
later . . . (p. 16)
Ziomek (1999) found that performance on were found for an ACP in Dallas, Texas,
the Educational Testing Service’s Praxis For this proposition, the report cites which found only about 54% of recruits
tests is higher for those who have been en- Feistritzer and Chester (2002). This com- progressed from 1st year to 2nd year sta-

DECEMBER 2002 21
tus without “deficiencies” and only 40% dependent teaching (Darling-Hammond, about 50% of Houston’s new teachers were
planned to remain in teaching (Lutz & Hudson, & Kirby, 1989). uncertified, and the researchers report that
Hutton, 1989), and one in New York City The inclusion of student teaching ap- 35% of new hires lacked even a bachelor’s
(the Teaching Fellows Program), which pears to be one important element in pre- degree; so TFA teachers were compared to
lost more than 15% of its first class by dicting the outcomes of different programs. an extraordinarily ill-prepared group.
Thanksgiving and more than 30% by the For example, a recent report from the Because Houston’s distribution of under-
end of the initial year (Goodnough, 2000). NCES (2000) notes that 29% of newly qualified teachers was intensely concentrated
The Secretary’s report also states that graduated teachers who had not had stu- in schools serving the most disadvantaged
in Texas, “retention rates are higher for dent teaching left teaching within 5 years— students and because the regressions con-
African American teachers who entered an entry strategy that is typical of emergency trolled for proportions of minority and
teaching through an alternate route as hires and some of the shorter term alter- low-income students at the school level as
compared to their traditionally certified native routes—as compared to only 15% well as proportions of low-income and low-
teachers” (2002, p. 16). We noted that of those who had had student teaching. achieving students at the classroom level,
“alternative” routes in Texas include all However, the Secretary’s report suggests TFA teachers were compared largely to the
postbaccalaureate programs, whether that student teaching, along with educa- other underqualified teachers concentrated
extended preservice or streamlined in- tion school coursework, should be “op- in these schools and classrooms. Although
service programs. The Texas data included tional” (p. 19) and urges that ACPs should they have the data to do so, Raymond and
in Feistritzer and Chester’s appendix show not be “larded with a variety of require- colleagues do not report how TFA teachers’
a 5-year retention rate between 1992 and ments” (p. viii). outcomes compared to those of trained and
1997 of 72% for African-American ACP certified teachers or to others with a bache-
Effectiveness of Alternative
recruits versus a 66% rate for traditional re- lor’s degree.
Certification Programs
cruits (2002, p. 419). However, the Secre- The study indicates that minority stu-
tary’s report neglects to note that drop out The Secretary’s report argues that, on the dents in Houston, who were dispropor-
rates are also higher for African-American one hand, “traditional teacher training tionately taught by these underprepared
recruits from alternative programs (18% programs do not necessarily produce grad- teachers, performed increasingly poorly
vs. 15% for traditional recruits). The Sec- uates with superior teaching skills while each year in comparison with their White
retary’s report also neglects to note that at the same time they impose significant peers. The TFA study also reports, al-
these data show 5-year teacher retention costs and challenges on prospective teach- though the Secretary’s report does not, the
rates for ACP recruits that are much lower ers” and, on the other hand, “alternate extraordinarily high attrition rates for
than the 85% “national” retention figure routes to certification demonstrate that TFA teachers: Over the 3 years studied,
cited elsewhere in the report and lower streamlined systems can boost the quan- from 60 to 100% of TFA recruits had left
than those for traditional entrants (62% vs. tity of teachers while maintaining or even after their 2nd year of teaching.
67%). Similarly, drop out rates for ACP re- improving their quality” (p. 19). The re- A recent study in five Arizona school dis-
cruits are higher than those for traditional port cites the findings of a study of Teach tricts did examine the relative effectiveness
entrants (29% vs. 25%) (Feistritzer & for America (TFA) recruits in Houston of TFA teachers as compared to other new
Chester, p. 419). (Raymond, Fletcher, & Luque, 2001) as teachers with different levels of qualifica-
Among ACPs, those that provide more evidence for the success of alternative pro- tions. Using a matched comparison design
extensive supervision and support both grams. The Secretary’s report states that in which teachers were matched by expe-
before and while recruits take over as in- “the evaluation reveals the district’s high- rience level, grade level taught, level of edu-
dependent teachers appear to have better est performing teachers are consistently cation, and school or district, the study
outcomes in terms of retention. For ex- TFA teachers, while the lowest performing found that the students of uncertified teach-
ample, New Haven, California has shown teachers are consistently not TFA teach- ers, including TFA teachers, did signifi-
retention rates above 90% for its intern- ers” (2002, p. 18). This is not, however, cantly poorer than those of comparably
ship program with California State Uni- what the study’s regression analyses actually experienced certified teachers on mathe-
versity—Hayward, which provides a full found. The data reported by Raymond and matics, reading, and language arts tests
program of coherent coursework integrated colleagues show that experienced teachers in (Laczko-Kerr & Berliner, 2002). Another
with full-time student teaching followed by Houston were significantly more successful recent study found that TFA recruits in
intensive mentoring when interns take over than inexperienced teachers, including New York City, like other untrained re-
a part-time teaching load (Snyder, 1999). TFA teachers. After controlling for teacher cruits there, felt markedly less well prepared
Similarly, a RAND report examining a na- experience and school and classroom de- for teaching than graduates of teacher edu-
tional sample of programs found signifi- mographics, TFA recruits were found to be cation programs. These less well-prepared
cantly higher rates of planned retention for about as effective as other inexperienced entrants to teaching reported that they felt
recruits who graduated from alternative teachers in schools and classrooms serving significantly less efficacious in their teaching
programs with more tightly coordinated high percentages of minority and low- and less able to meet their students’ needs,
and extensive preservice components and income students, which, the study shows, less satisfied with their training, less likely
more intensive supervision than those who is where most underqualified teachers in to stay in teaching, and less likely to say
completed alternatives that featured only a the district were placed. In 1999–2000, the they would come into teaching through the
few weeks of summer training before in- last year covered by the study sample, same pathway again (Darling-Hammond,

22 EDUCATIONAL RESEARCHER
Chung, & Frelow, 2002). Although each berg (1996) compared traditionally certified port concludes its peculiar rendering of the
of the three TFA-related studies has limita- teachers in New Hampshire with teachers research on teaching with the following rec-
tions, in combination they illustrate that no in the same state who had participated in a ommendation: “To meet the ‘highly quali-
sweeping claims can be made for the effec- state-sponsored ACP. The alternate-route fied’ teachers challenge, then, states will
tiveness of the program. teachers assumed “full responsibility for stu- need to streamline their certification system
There is other research on alternative dents prior to any preparation, and (had) to focus on the few things that really mat-
routes that goes beyond the TFA program, three years” to acquire 14 state-identified ter: verbal ability, content knowledge, and,
which, as Feistritzer and Chester (2002) competencies through workshops or col- as a safety precaution, a background check
accurately note, is not actually an ACP lege courses (p. 61). Based on surveys of of new teachers” (p. 40).
because its recruits teach on emergency per- 136 principals and more than 200 teach- As our review indicates, these asser-
mits and are not part of a coherent program ers, the author reports that traditionally tions and policy recommendations are
of teacher education (which most states certified teachers were rated by their prin- not supported by scientifically based re-
with alternative certification require). This cipals significantly higher than alternate- search. Although there is evidence that
research is very difficult to interpret be- route teachers on instructional skills and verbal ability and content knowledge con-
cause the design and quality of programs instructional planning, and the tradition- tribute to teacher effectiveness, there is
labeled traditional and alternative varies ally certified teachers rated their own also evidence that teacher preparation—
greatly both within and across states, with preparation significantly higher than did including the student teaching and meth-
some states labeling as alternative postbac- the alternatively certified teachers. ods coursework the Secretary’s report
calaureate programs that other states would By contrast, Miller, McKenna, and deplores—contributes at least as much to
call traditional. Some alternate route pro- McKenna (1998) found no differences outcomes ranging from teacher effective-
grams provide scant preparation while between traditionally certified and alter- ness to teacher retention. And although
others involve extensive coursework, pre- nately certified teachers with regard to in- there is evidence that some well-designed
and in-service professional development, structional practices or student achievement ACPs have strong outcomes, there is also
and school- and university-based induc- in a university-sponsored program. This evidence that programs and entry path-
tion support. Furthermore, many studies program offered 15 to 25 credit hours of ways that skirt the core features of teacher
do not report on the design features of the coursework before interns entered class- preparation produce recruits who consider
programs they have examined. rooms where they were intensively super- themselves underprepared, are viewed as
Thus, it may not be surprising that the vised and assisted by both university less competent by principals, are less effec-
findings on ACPs are mixed: A number of supervisors and school-based mentors while tive with students, and have high rates of
studies have found that ratings of the com- they completed additional coursework teacher attrition. Finally, the recent ad-
petence of alternative program candidates needed to meet full standard state certifica- vances states have made in strengthening
by principals and supervisors are more fre- tion requirements. Because the design of teacher certification requirements have
quently negative than those of “traditional” this program was so different from many begun to be evident in stronger academic
recruits (see Gomez & Grobe, 1990 re- quick-entry alternative route programs, backgrounds and licensing test scores for
garding Dallas; Jelmberg, 1996 regarding Miller, McKenna, and McKenna (1998) college graduates who have prepared to
New Hampshire; Mitchell, 1987 regard- note that their studies teach. These trends suggest that meeting
ing Dallas; CSU, 2002a, 2002b regarding the highly qualified teacher challenge will
provide no solace for those who believe
internship programs in California), while that anyone with a bachelor’s degree can require states to stay the course with re-
others have found that ratings are compa- be placed in a classroom and expect to be spect to the gains they have already made,
rable for candidates from alternative and equally successful as those having com- rather than to reverse course on the basis
traditional programs (Miller, McKenna, pleted traditional education programs . . . of a fictionalized account of what research
& McKenna, 1998). Only two controlled The three studies reported here support says about what effective teachers know
studies of the achievement outcomes of carefully constructed AC programs with
and how they come to know it.
ACP and traditionally trained teachers have extensive mentoring components, post-
been reported, with one finding the stu- graduation training, regular in-service NOTES
dents of traditional teachers showed sig- classes, and ongoing university supervi- 1 Among these are the Fordham Founda-

nificantly larger gains in language arts than sion. (p. 174) tion, which has issued a “manifesto” urging the
those of ACP teachers (Gomez & Grobe, Ironically, this more extensive educa- elimination of teacher education and certifica-
1990), and the other finding student out- tion requirements and the Abell Foundation,
tion coursework and more heavily men-
comes comparable across the two groups, which has advocated for similar measures.
tored clinical training compose the very 2 Teacher Certification Reconsidered: Stum-
although gain scores were not reported “bureaucratic hurdles” the Secretary’s re- bling for Quality, sponsored by the Abell Foun-
(Miller, McKenna, & McKenna, 1998). port suggests should be eliminated (p. 19). dation, is published on the Abell Foundation
When this research is analyzed in terms website at www.abellfoundation.org. The ver-
of program design, it appears that more Conclusion
sion of the report that was publicized and pub-
carefully designed programs yield stronger Having asserted that requirements regard- lished on this website in September 2001 is the
outcomes in terms of teacher effectiveness ing education coursework and student basis for this response. Walsh states that some
and retention than those that provide less teaching are “the Achilles heel of the certi- of the earlier errors and misrepresentations noted
training and support. For example, Jelm- fication system” (p. 31), the Secretary’s re- in a reply to that report (Darling-Hammond,

DECEMBER 2002 23
2001) have been removed from a version since ceding the hemolytic-uremic syndrome. The tember 2002 from http://www.edpolicy.
published in hard copy by the Foundation in New England Journal of Medicine, 346(1), org/publications/documents/updatev2i1.pdf
December 2001 (Walsh & Podgursky, 2001). 23–32. Fuller, E. (1998). Do properly certified teachers
3 The versions of the report published on the Coleman, J. S., Campbell, E. Q., Hobson, matter? A comparison of elementary school per-
Foundation’s website have changed as it has C. J., McPartland, J., Mood, A. M., Weinfeld, formance on the TAAS in 1997 between
been critiqued. These citations were in the ver- F. D., et al. (1966). Equality of educational schools with high and low percentages of prop-
sions made available in August and September opportunity. Washington, DC: U.S. Govern- erly certified regular education teachers. Austin:
2001 and were removed in the version made ment Printing Office. The Charles A. Dana Center, University of
available on the Abell Foundation website in Darling-Hammond, L. (2000). Teacher quality Texas at Austin.
March 2002. and student achievement: A review of state Fuller, E. (2000). Do properly certified teachers
4
Most states require a specified amount of policy evidence. Educational Policy Analysis matter? Properly certified Algebra teachers and
student teaching, and a growing number re- Archives, 8(1). Retrieved from http://epaa. Algebra I achievement in Texas. Paper pre-
asu.edu/epaa/v8n1 sented at the annual meeting of the Ameri-
quire a probationary period, which may feature
Darling-Hammond, L. (2001). The research can Educational Research Association, New
additional mentoring and assessment before
and rhetoric on teacher certification: A re- Orleans, LA.
full certification can be awarded.
5 The Secretary’s report references an article
sponse to “Teacher certification reconsid- Gitomer, D. H., Latham, A. S., & Ziomek, R.
ered.” New York: National Commission on (1999). The academic quality of prospective
about the study published in a Fordham Foun- teachers: The impact of admissions and licen-
Teaching and America’s Future. Retrieved
dation report (Goldhaber & Brewer, 1999) be- from http://www.nctaf.org/publications/ sure testing. Princeton, NJ: Educational Test-
fore the study was published in a peer-reviewed abell_response.pdf ing Service.
journal (Goldhaber & Brewer, 2000). Darling-Hammond, L. (2002). The research Goe, L. (2002, October 14). Legislating equity:
6
In states with the most stringent certifica- and rhetoric on teacher certification: A re- The distribution of emergency permit teach-
tion laws, a teacher fully certified in chemistry sponse to “Teacher certification reconsid- ers in California. Education Policy Analysis
but not in physics may be teaching on a regular ered.” Educational Policy Analysis Archives, Archives, 10(42). Retrieved November 8,
license in one subject and a temporary or emer- 10(36). Retrieved from http://epaa.asu.edu/ 2002 from http://epaa.asu.edu/epaa/v10n42.
gency license in the other, which is why tempo- epaa/v10n36 Goldhaber, D. D., & Brewer, D. J. (1997).
rary certification is most common in science. Darling-Hammond, L., Berry, B., & Thoreson, Evaluating the effect of teacher degree level
A. (2001). Does teacher certification matter? on educational performance. In W. J. Fowler
REFERENCES (Ed.), Developments in School Finance, 1996
Evaluating the evidence. Educational Evalu-
Barnes, S., Salmon, J., & Wale, W. (1989). Al- ation and Policy Analysis, 23(1), 57–77. (pp.197–210). Washington, DC: National
ternative teacher certification in Texas. Paper Darling-Hammond, L., Chung, R., & Frelow, Center for Education Statistics, U.S. De-
presented at the annual meeting of the Amer- F. (2002). Variation in teacher preparation: partment of Education.
ican Educational Research Association. (ERIC How well do different pathways prepare Goldhaber, D. D., & Brewer, D. J. (1999).
Document No. 307316) teachers to teach? Journal of Teacher Educa- Teacher licensing and student achievement.
Begle, E. G. (1979). Critical variables in math- tion, 53(4), 286-302. In C. E. Finn & M. Kanstoroom (Eds.),
ematics education: Findings from a survey of Darling-Hammond, L., Hudson, L., & Kirby, Better teachers, better schools. Washington,
empirical literature. Washington, DC: Na- S. (1989). Redesigning teacher education: DC: Thomas B. Fordham Foundation. Re-
tional Council for Teachers of Mathematics. Opening the door for new recruits to science trieved from http://www.edexcellence.net/
Betts, J. R., Rueben, K. S., & Danenberg, A. and mathematics teaching. Santa Monica, CA: better/goldhab.pdf
(2000). Equal resources, equal outcomes? The The RAND Corporation. Goldhaber, D. D., & Brewer, D. J. (2000).
distribution of school resources and student Ehrenberg, R. G., & Brewer, D. J. (1995). Did Does teacher certification matter? High
achievement in California. San Francisco: teachers’ verbal ability and race matter in the school teacher certification status and stu-
Public Policy Institute of California. 1960s? Coleman revisited. Economics of Ed- dent achievement. Educational Evaluation
Bowles, S., & Levin, H. M. (1968). The deter- ucation Review, 14(1), 1-21. and Policy Analysis, 22(2), 129–146.
minants of scholastic achievement—An ap- Feistritzer, C. E., & Chester, D. T. (2002). Al- Gomez, D. L., & Grobe, R. P. (1990). Three
praisal of some recent evidence. Journal of ternative teacher certification: A state-by-state years of alternative certification in Dallas:
Human Resources, 3, 3–24. analysis 2002. Washington, DC: National Where are we? Paper presented at the annual
Bullough, R., Knowles, J., & Crow, N. (1989). Center for Education Information. meeting of the American Educational Re-
Teacher self-concept and student culture in Ferguson, R. F. (1991). Paying for public ed- search Association, Boston, MA.
the first year. Teachers College Record, 91(2), ucation: New evidence on how and why Goodnough, A. (2000, November 22). Win-
209–233. money matters. Harvard Journal on Legisla- nowing process begins for novice teachers,
California State University. (2002a). First sys- tion, 28(2), 465–498. New York Times. Retrieved from http://
tem wide evaluation of teacher education pro- Ferguson, R. F., & Ladd, H. F. (1996). How nytimes.com.
grams in the California State University: and why money matters: An analysis of Al- Hanushek, E. A. (1992). The trade-off be-
Summary report. Long Beach, CA: Office of abama schools. In H. F. Ladd (Ed.), Hold- tween child quantity and quality. Journal of
the Chancellor, California State University. ing schools accountable: Performance-based Political Economy, 100(1), 84–117.
California State University. (2002b). Preparing reform in education (pp.265–298). Wash- Hanushek, E. A. (1996). School resources and
teachers for reading instruction (K–12): An ington, DC: Brookings Institution. achievement in Maryland. Baltimore: Mary-
evaluation brief by the California State Uni- Fetler, M. (1999). High school staff character- land State Department of Education.
versity. Long Beach, CA: Office of the Chan- istics and mathematics test results. Educa- Hanushek, E. A., Rivkin, S. G., & Taylor, L. L.
cellor, California State University. tion Policy Analysis Archives, 7(9). Retrieved (1995). Aggregation bias and the estimated
Chandler, W., Jelacic, S., Boster, D., Ciol, M., from http://epaa.asu.edu/epaa/v7n9 effects of school resources. Rochester, NY:
Williams, G., Watkins, S., et al. (2002). Pro- Fowler, C. (2002). Fast track . . . Slow going? University of Rochester, Center for Eco-
thrombotic coagulation abnormalities pre- Research Brief. Ed Policy, 2(1). Retrieved Sep- nomic Research.

24 EDUCATIONAL RESEARCHER
Hawk, P. P., Coble, C. R., & Swanson, M. have in common? Social Science Research, 10, production function approach (Research paper
(1985). Certification: It does matter. Jour- 83–100. 6). Philadelphia: Federal Reserve Bank of
nal of Teacher Education, 36(3), 13–15. National Association of State Directors of Philadelphia, Department of Research.
Jelmberg, J. (1996). College-based teacher ed- Teacher Education and Certification. (2001). Summers, A. A., & Wolfe, B. L. (1977). Do
ucation versus state-sponsored alternative The NASDTEC manual on the preparation schools make a difference? American Eco-
programs. Journal of Teacher Education, and certification of educational personnel. nomic Review, 67, 639–652.
47(1), 60–66. Dubuque, IA: Kendall Hunt Publishing. U.S. Department of Education. (2002). Meet-
Kentucky Institute for Education Research. National Center for Education Statistics. (1999). ing the highly qualified teachers challenge: The
(1997). The preparation of teachers for Ken- Teacher quality: A report on the preparation Secretary’s annual report on teacher quality.
tucky schools: A survey of new teachers. and qualifications of public school teachers Washington, DC: U.S. Department of Ed-
Frankfort: Kentucky Institute for Education (NCES 1999-080). Washington, DC: U.S. ucation, Office of Postsecondary Education,
Research. Department of Education. Office of Policy, Planning, and Innovation.
Laczko-Kerr, I., & Berliner, D. (2002). The National Center for Education Statistics. (2000). Walsh, K. (2001). Teacher certification recon-
effectiveness of Teach for America and Digest of education statistics, 1999. Washing- sidered: Stumbling for quality. Baltimore, MD:
other under-certified teachers on student ton, DC: U.S. Department of Education. Abell Foundation. Retrieved September 2001
academic achievement: A case of harmful Quaini, F., Urbanek, K., Beltrami, A., Finato, from http://www.abell.org/pubsitems/ed_
public policy. Educational Policy Analysis N. L., Beltrami, C., Nadal-Ginard, B., et al. cert_1101.pdf
Archives, 10(37). Retrieved from http:// (2002). Chimerism of the transplanted heart. Walsh, K., & Podgursky, M. (2001). Teacher
epaa.asu.edu/epaa/v10n37 The New England Journal of Medicine, certification reconsidered: Stumbling for qual-
Lutz, F. W., & Hutton, J. B. (1989). Alterna- 346(1), 5–15. ity. A rejoinder. Baltimore, MD: Abell Foun-
tive teacher certification: Its policy implica- Raymond, M., Fletcher, S. H., & Luque, J. dation. Retrieved from http://www.abell.
tions for classroom and personnel practice. (2001). Teach for America: An evaluation of org/pubsitems/ed_cert_rejoinder_1101.pdf
Educational Evaluation and Policy Analysis, teacher differences and student outcomes in Wayne, A. J., & Youngs, P. (in press). Teacher
11(3), 237–254. Houston, Texas. Stanford, CA: The Hoover characteristics and student achievement gains:
Massey, H. & Vineyard, E. (1958). Relation- Institution, Center for Research on Educa- A review. Review of Educational Research.
ship between scholarship and first year teach- tion Outcomes. Wenglinsky, H. (2000). How teaching matters:
ing success. Journal of Teacher Education, 9, Rivkin, S. G., Hanushek, E. A., & Kain, J. F. Bringing the classroom back into discussions of
297–301. (2001). Teachers, schools, and academic teacher quality. Princeton, NJ: Educational
McKibbin, M. (1998). Teaching internship pro- achievement. Amherst, MA: Amherst College. Testing Service.
grams: Alternative preparation and licensure Rowan, B., Chiang, F., & Miller, R. J. (1997). Wilson, S. M., Floden, R., & Ferrini-Mundy,
in California: Purposes, procedures and per- Using research on employees’ performance J. (2001). Teacher preparation research: Cur-
formance. Sacramento: California Commis- to study the effects of teachers on students’ rent knowledge, gaps, and recommendations. A
sion on Teacher Credentialing. achievement. Sociology of Education, 70(4), research report prepared for the U.S. Depart-
McKibbin, M. (1999). A report on teaching 256–284. ment of Education. Seattle: Center for the
internship grant programs, 1994–1999: Les- Sanders, W. L., & Horn, S. P. (1994). The Study of Teaching and Policy, University of
sons learned and challenges to face. Sacra- Tennessee Value-Added Assessment System Washington.
mento: California Commission on Teacher (TVAAS): Mixed-model methodology in Wright, D. P., McKibbin, M. D., & Walton,
Credentialing. educational assessment. Journal of Personnel P. H. (1987). The effectiveness of the Teacher
Miller, J. W., McKenna, M. C., & McKenna, Evaluation in Education,8(3), 299–311. Trainee Program: An alternate route into
B. A. (1998). A comparison of alternatively Sanders, W. L., & Rivers, J. C. (1996). Cumu- teaching in California. Sacramento: Califor-
and traditionally prepared teachers. Journal lative and residual effects of teachers on future nia Commission on Teacher Credentialing.
of Teacher Education, 49(3), 165–176. student academic achievement. Knoxville: Uni- Wright, S. P., Horn, S. P., & Sanders, W. L.
Mitchell, N. (1987). Interim evaluation re- versity of Tennessee Value-Added Research (1997). Teacher and classroom context ef-
port of the alternative certification program and Assessment Center. fects on student achievement: Implications
(REA87-027-2). Dallas, TX: DISD Depart- Skjoerven, R., Wilcox, A., & Lie, R. (2002). for teacher evaluation. Journal of Personnel
ment of Planning, Evaluation, and Testing. The interval between pregnancies and the Evaluation in Education, 11, 57–67.
Monk, D. H. (1994). Subject area preparation risk of preeclampsia. The New England Jour-
of secondary mathematics and science teach- nal of Medicine, 346(1), 33–38. AUTHORS
ers and student achievement. Economics of Snyder, J. (1999). New Haven Unified School LINDA DARLING-HAMMOND is the Charles
Education Review, 13(2), 125–145. District: A teaching quality system for excel- E. Ducommun professor of education at
Monk, D. H., & King, J. A. (1994). Multilevel lence and equity. New York: National Com- Stanford University. Her research interests
teacher resource effects in pupil performance mission on Teaching and America’s Future. focus on educational policy and practice in the
in secondary mathematics and science: The Stoddart, T. (1992). An alternate route to areas of teacher quality, school restructuring,
case of teacher subject matter preparation. In teacher certification: Preliminary findings and educational equity.
R. G. Ehrenberg (Ed.), Choices and conse- from the Los Angeles Unified School Dis-
quences: Contemporary policy issues in educa- trict Intern Program. Peabody Journal of Ed- PETER YOUNGS is a postdoctoral fellow
tion (pp. 29–58). Ithaca, NY: ILR Press. ucation, 67, 3. and research associate at Stanford University.
Murnane, R. J. (1983). Understanding the Strauss, R. P., & Sawyer, E. A. (1986). Some He studies policy related to teacher education,
sources of teaching competence: Choices, new evidence on teacher and student com- licensure, and induction.
skills, and the limits of training. Teachers petencies. Economics of Education Review,
College Record, 84(3), 564–589. 5(1), 41–48. Manuscript received August 29, 2002
Murnane, R. J., & Phillips, B. R. (1981). What Summers, A. A., & Wolfe, B. L. (1975). Equal- Revisions received October 8, 2002
do effective teachers of inner-city children ity of educational opportunity quantified: A Accepted October 9, 2002

DECEMBER 2002 25

You might also like