Accountability Plans and the Growth Model Pilot Program"
Daniel Anderson!
Behavioral Research and Teaching, University of Oregon, Eugene, Oregon 2010"
Introduction
With the passage of the No Child Left Behind Act
of 2001 (NCLB; No Child Left Behind, 2002)
schools are required to make adequate yearly
progress (AYP) toward the goal of 100% of
students reaching proficiency. Yet, states vary in
the implementation of AYP policies. Many states
(n = 28) choose only to report AYP data of
‘passing’ or ‘failing’, but others choose to use an
integrated (n = 8) or separate rating system (n =
14). Although NCLB requires all states to report
AYP data, those using separate rating systems
primarily rely on their own metrics within the
state. The majority of states (n = 39) use a status-
based system to measure AYP. This system has
been shown to have a substantial bias against
schools serving students from ethnically diverse
and impoverished backgrounds (Kim &
Sunderman, 2005; Zvoch & Stevens, 2006, 2008).
The remaining states (n = 11) measure AYP
through an approved growth model as part of the
Growth Model Pilot Program (GMPP).
Research Questions
Does the state’s participation in the GMPP
significantly impact the number of schools
making AYP?
Does the type of accountability system
significantly impact the number of schools
making AYP?
Author’s Note
This project was part of a Masters Terminal Project. For the full report,
which includes further analyses, please visit
http://education.uoregon.edu/feature.htm?id=1199 and click on the link
titled “Anderson Terminal Project”.
Materials and Methods
The unit of analysis for this study was states. As part
of NCLB, individual school data within a state are
required to be aggregated to serve as a
representation of the state. No school-level data
were analyzed.
Variables and Analysis
Data, obtained from the Council of Chief State
School Officers (CCSSO) website (CCSSO 2007b),
were analyzed with a two-way, between-subjects
analysis of variance.
Dependent variable
  Percentage of schools making AYP in each
state.All states included with the exception of
NewYork (data not available).
  Obtained percentages ranged from 93%
(Oklahoma, Wisconsin) to 13% (South
Carolina), with a mean of 64.67% and a
standard deviation of 18.67 percentage points.
Independent variables
  State’s participation in the GMPP
o  Dichotomous: coded 1 for no and 2 for yes
  Type of accountability program in the state
o  Three levels: (1) only AYP, (2) integrated,
and (3) separate rating/index system.
o  The label of each state’s accountability
system was obtained from the CCSSO
website (CCSSO, 2007c).
Descriptive statistics for all variables are
summarized in Table 1.
Results
The results are summarized in Table 2. In this
analysis, GMPP participation was confounded with
the type of accountability, making the design
unbalanced, (n = 49, 2) = 1.29, p > .05 (refer to
Table 1 for contingency table). Therefore, unique
sums of squares and unweighted means were
examined in accordance with conventional analysis
of variance logic.
Main effects of GMPP and type of accountability
system were examined because the interaction was
not significant, F(2, 43) = 1.30, p > .05. Overall,
states participating in the GMPP (M = 59.00) did not
have a significantly higher percentage of schools
making AYP than those not participating (M =
65.97), F(1, 43) = 1.02, p > .05. Observed
differences are likely due to random sampling
variability. Further, the type of accountability
system did not significantly affect the percentage of
schools making AYP, F(2, 43) = 0.56, p > .05. The
population within each type of accountability
system could be the same, and difference in
obtained percentages of schools making AYP is
likely due to random sampling variability.
Conclusions
State cut scores potentially played a large role in the
findings of this study, as the variations were not
accounted for. Further, the GMPP is a recent
implementation and it is possible that its effects simply
require more time. An additional weakness is that all
effects were viewed in a comparison analysis. That is, all
observed differences were relative to other states instead
of within a state. It is possible a change may have been
seen using archival data from a state and analyzing the
percentage of schools making AYP pre- and post-GMPP
implementation. Future studies could combine data from
an analysis that ‘maps’ the standard from one state onto a
separate test (see Cronin et al., 2007). This would enable
the researcher, using a quantitative measure of each
state’s standards, to control for the variability. However,
this study did shed light on the relatively small impact
varying state policies (aside from cut score placement)
can have on the percentage of schools making AYP. As
more states begin to participate in the GMPP (Colorado,
Minnesota, Pennsylvania, and Texas have all been
approved for the 2008-2009 school year; U.S.
Department of Education, 2008) it will become
increasingly important to monitor its effects.
Literature cited
Council of Chief State School Officers. (2007a). Statewide student assessment: Mathematics, SY 2007-08.
Retrieved May 10, 2009 from
http://www.ccsso.org/content/pdfs/2007-08_Math-ELAR-Sci_Assessments.pdf
Council of Chief State School Officers. (2007b). Schools identified for improvement, or not making AYP, by
state: 2007-08. Retrieved May 10, 2009, from
http://accountability.ccsso.org/Results/AYP%20Status%202007_2008%20%20032309.pdf
Council of Chief State School Officers. (2007c). State accountability systems and AYP determinations.
Retrieved May 10, 2009, from
http://www.ccsso.org/content/pdfs/State%20Accountability%20Systems%20Beyond%20NCLB.pdf
Cronin, J., Dahlin, M., Adkins, D., & Kingsbury, G. G. (2007). The proficiency illusion. Retrieved February
13, 2009, from http://www.edexcellence.net/doc/The_Proficiency_Illusion.pdf
Kim, J., & Sunderman, G. (2005). Measuring academic proficiency under the no child left behind act:
Implications for educational equity. Educational Researcher, 34, 3-13.
No Child Left Behind Act of 2001, Public Law No. 107-110 (2002).
United State Department of Education. (2008, January 8). Secretary Spellings approves additional growth
models for the 2008-2009 school year. U.S. Department of Education Press Release. Retrieved May
10, 2009, from http://www.ed.gov/news/pressreleases/2009/01/01082009a.html
Zvoch, K., & Stevens, J. (2006). Successive student cohorts and longitudinal growth models: An
investigation of elementary school mathematics performance. Education Policy Analysis Archives,
14, (2) 1-25.
Zvoch, K., & Stevens, J. (2008). Measuring and evaluating school performance: An investigation of status
and growth-based achievement indicators. Evaluation Review, 32, 569-595.
Contact Information
Daniel Anderson
Research Assistant – Behavioral Research and Teaching
175 Education
5262 University of Oregon, Eugene, OR 97403
daniela@uoregon.edu