Progress Monitoring of Mathematics Among Students With Learning Disabilities

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 18

Progress Monitoring of Mathematics Among Students with

Learning Disabilities
Edward S. Shapiro and Lana Edwards
Lehigh University
Naomi Zigmond
University of Pittsburgh
This article describes a statewide project in which weekly progress monitoring of mathematics was imple-
mented with 120 special education students. Curriculum-based measurement probes for computation and
concepts/applications skills were administered weekly over seven months. Students were assessed at lev-
els identified as instructional by their teachers, and goals for performance were set based on recommen-
dations from the research literature. Improvements of 0.38 digits per week (in computation) and 0.38
points per week (in concepts/applications) were found across students. A total of 66% of students
achieved the expected goals in computation and 37% in concepts/applications. These levels of improve-
ment were comparable to those reported in the literature for general education students. Perceived bene-

fits and challenges of conducting progress monitoring in mathematics are discussed.

Many students with learning disabilities interventions for individual students (Deno,
experience persistent difficulties leaming and 1992; Deno, Espin, & Fuchs, 2002).
applying mathematics (Maccini & Hughes, Essentially, the purpose of CBM is to improve
2000; Owen & Fuchs, 2002). student performance. An example and a type
Characteristically, students with leaming dis- of general outcomes measurement approach,
abilities at the end of elementary school per- CBM has been used as an evaluative measure
form basic addition facts only as well as third- in education for over two decades (Fuchs &
grade students without disabilities, most Deno, 1991; Shinn, 1989). More specifically,
demonstrate growth patterns in mathematics of it involves evaluating student progress toward
no better than one year for every two or more the acquisition of basic skills that are globally
years of school, and by the end of high school reflective of instruction within a curriculum
they demonstrate proficiency levels equivalent (Deno et al., 2002; Shinn, Shinn, Hamilton,
to only fifth or sixth grade (Calhoon & Fuchs, & Clarke, 2002; Stecker & Fuchs, 2000).
2003). Considering the challenges students CBM is a standardized procedure conducted
with leaming disabilities typically encounter in repeatedly over time that provides informa-
mathematics, as well as their seemingly labori- tion relevant to making decisions about stu-
ously slow rate of progress, it is critical to iden- dent achievement (Deno et al., 2002; Fuchs
tify an assessment system that would help & Deno, 1991; Fuchs & Fuchs, 1999; Fuchs,
teachers monitor student math performance Fuchs, Hosp, & Jenkins, 2001; Shinn et al.,
and improve mathematics learning outcomes. since the material used in assess-
2002). Also,
Curriculum-based measurement (CBM) is mentis taken directly from the curriculum,
a validated assessment system that teachers CBM is considered to be an &dquo;authentic&dquo; way
can use to set goals, monitor student perfor- to assessstudent performance (Deno et al.,
mance over time, and design instructional 2002; Shepard, 2000). Finally, CBM proce-

15
16

dures can be used in several academic areas problem type. This level of scoring analysis
including reading, writing, and mathematics. reveals whether there are any &dquo;bugs&dquo; or mis-
In math, each CBM is an index of student conceptions in a student’s calculations (Fuchs
performance on the skills targeted for instruc- et al., 1991).
tion across an entire school year. Thus, for Fuchs et al. (1991) explored the use of spe-
each CBM assessment, the teacher specifies cific skill analysis in conjunction with goal
problem types that reflect the math curricu- setting and instructional decision making
lum and creates multiple alternate assess- associated with CBM. The authors randomly
ments that each contain all the problem types assigned 30 special education teachers who
covered in the curriculum (Fuchs, Fuchs, taught self-contained or resource classes in
Hamlett, & Stecker, 1991). CBM math grades 1 through 9 to three different inter-
probes are typically scored according to the vention groups. The first group of teachers
student’s total number of digits correct used CBM performance indicators with skills
(Shapiro, 1996). These CBM scores, or &dquo;per- analysis, the second group used CBM perfor-
formance indicators,&dquo; represent overall profi- mance indicators without skills analysis, and

ciency within the yearly curriculum. Ideally, the third group used neither CBM nor skills
teachers graph these performance indicators analysis. Results supported the use of specific
and use the graphs to (a) monitor student skills analysis within CBM. That is, teachers
goals, (b) judge the adequacy of student who received graphed CBM performance
progress for determining instructional deci- indicators along with a specific skills analysis
sions, and (c) assess the impact of different report planned more specific instructional
instructional interventions (Fuchs et al., interventions for their students and observed
1991 ). more significant gains in student math

Although CBM focuses on a student’s achievement than teachers who just received
progress over time general
on skills within the graphed CBM data. According to Fuchs et al.,
curriculum, a teacher can analyze student’s
a the achievement effect size associated with
responses on a given CBM assessment to CBM skills analysis was large. Compared to
ascertain a comprehensive description of the control teachers, the effect size for the com-
student’s mastered and emerging skills. In bined CBM and skills analysis approach was
math, each skill is assessed during an individ- .67; the effect size for CBM teachers who did
ual CBM assessment probe. Skills analyses not receive skills analysis was .55. Overall, the
can be created by aggregating a student’s per- Fuchs et al. findings appear to suggest the
formance across recent assessments for each potential contribution of skills analysis to
individual skill (Fuchs et al., 1991). In other CBM procedures.
words, a teacher is able to determine specific In addition to investigating the use of spe-
skill areas that students have mastered, as well cific skill analysis in the context of CBM per-
as specific skill areas that require further formance indicators, research on the techni-
remediation and future instruction. For exam- cal adequacy of mathematics CBM systems
ple, an analysis across recent probes might has also been conducted. Math CBM typical-
indicate that a student has mastered two-digit ly focuses on computation skills with a grade-
addition problems without regrouping, but level representative sample of math facts
requires further instruction on two-digit addi- and/or math operations ( i.e., adding, subtract-
tion problems with regrouping as well as a ing, multiplying, and dividing whole num-
review of subtraction. Further, using a digits bers, fractions, and decimals) from a specific
correct scoring procedure provides the curriculum (Fuchs et al., 1991). Research sup-
teacher with an error analysis and informs ports the reliability and validity of using com-
him/her of how students solve a particular putation as a global indicator of math
17

achievement (Fuchs, Fuchs, Hamlett, & concepts/applications CBM probes, Fuchs et


Allinder, 1989; Shapiro, 1996). In addition, al. (1994) tested a total of 140 students in sec-
there is emerging documentation on the use ond-, third-, and fourth-grade classrooms. A
of math computation probes for monitoring total of 12 students with learning disabilities
student growth across time. Using their were included in the sample. Classroom

CBM-computation system, Fuchs et al. teachers were asked to administer


( 1989) found that slopes ranged from .25 at concepts/applications probes to all the stu-
grade 2 to .70 at grade 4 for a sample of 140 dents in their daily mathematics class once a
students in inclusive general education math week for 20 weeks. Teachers followed with
classrooms monitored weekly on math com- high fidelity the administration and scoring
putation across 20 weeks. While these find- guidelines for the weekly concepts/applica-
ings are suggestive of the progress general edu- tion probes outlined by Fuchs et al. (1994).
cation students may make in math, cautious Results indicated strong split-half reliabili-
interpretation is warranted when considering ty on the concepts/applications probes, with
goal setting for students with learning disabil- correlations ranging from .94 to .98. Findings
ities due to the limited number of students also supported the validity of the concepts
with disabilities included in the Fuchs et al. and applications assessment. Correlations
sample. between the concepts/applications probes
-

Although computation has been a focus with computation and a general mathematics
within math CBM, a singular focus on com- battery ranged from .68 to .81. Perhaps most
putation assessment has limitations. First, important when considering the use of the
computation is only one component of a com- concepts and applications probes for progress
prehensive mathematics curriculum (Fuchs et monitoring are findings related to slope.
al., 1994; Thurber, Shinn, & Smolkowski, Fuchs et al. reported that slopes ranged from
2002). Second, many instructional programs .40 at grade 2 to .69 at grade 4. When dis-
focus on highly procedural computation cussing their findings Fuchs et al. recognized
instruction that typically requires students to that the slopes derived from their study pro-
independently negotiate conceptual concepts vide tentative estimates for designing appro-
(Woodward & Montague, 2000). Students priate weekly goals for progress monitoring.
with learning disabilities, therefore, often For example, they note that given the average
struggle in their understanding of core princi- slope of approximately .60 points correct on
ples, concepts, and skill applications the concepts and applications probes at grade
(Jitendra, DiPipi, & Perron-Jones, 2002). In 3, with a standard deviation of .30, one could
response to the documented mathematics target a gain of .60 to .90 points per week
deficits of students with learning disabilities, when setting weekly goals for a third-grade
along with the dominant focus on computa- student. It is important to note, however, that
tion within typical mathematics instruction, these preliminary estimates for rate of growth
Fuchs and colleagues (1994) expanded the are based primarily on the performance of

traditional computation-based assessment to a general education students and a relatively


CBM system that included number concepts, small number of students with learning dis-
counting, applied computation, geometry, abilities participating in inclusive math class-
measurement, charts, graphs, money, and rooms.

problem solving. In other words, they created Finally, research on the use of CBM to
an additional set of CBM concepts and appli- monitor student performance in math sug-
cation probes to reflect broader skills from the gests that students learn more when teachers
mathematics curriculum. make individual adjustments in instruction
To document the technical adequacy of the through the use of CBM procedures (Stecker
18

& Fuchs, 2000). The impact of CBM on math one half the rate of typical students, especial-
achievement is particularly evident when ly in the earlier grades. Based on their find-
teachers more intensively monitor the ings, Deno et al. make specific recommenda-
progress of a few individual students (Fuchs, tions for CBM growth rates in reading that
Fuchs, Hamlett, & Stecker, 1991). For exam- exceed normative levels for students with
ple, Fuchs et al. (1991) observed that teachers learning disabilities.
who used CBM to monitor student math per- Another unanswered question involves the
formance adjusted their students’ instruction- impact of reading demands on the perfor-
al programs approximately 2.5 times over the mance of students with leaming disabilities

course of 20 weeks. This finding was in con- on the concepts/applications probes. Fuchs et
trast to the average .27 times of instructional al. ( 1994), for example, noted that one expla-
adjustments made by teachers who did not nation for differentially higher rates of growth
employ CBM during the same period. for second-grade students on math con-
Overall, when considering the challenges stu- cepts/applications could be the emerging
dents with leaming disabilities face when reading skills of second-grade students early in
leaming mathematics, empirical support for the academic year. In other words, as second-
the importance of collecting progress moni- grade students made improvements in reading
toring data, designing interventions, and across the year, their ability to solve con-

evaluating instructional effectiveness on an cepts/applications problems, problems that


individual basis is critical. require reading, also improved. When consid-
Unfortunately, despite evidence for general ering the serious difficulties students with
CBM efficacy and the impact of CBM perfor- leaming disabilities have with reading, the
mance indicators on student math achieve- relationship between performance on con-
ment, empirical evidence for the use of CBM cepts/applications probes and overall math
to monitor the performance of students with performance deserves further attention
learning disabilities in math is limited. While (Fuchs & Fuchs, 2002).
L.S. Fuchs, D. Fuchs, and colleagues’ research While research has begun to examine how
on the use of CBM in math instruction has the positive impacts of CBM with individual
made substantive contributions to what we students can be extended to a class- and
know about CBM in math, further research is schoolwide scale, emphasis on the National
required to document the growth students Council of Teachers of Mathematics’
with leaming disabilities make in math and Curriculum and Education Standards
how specific intervention decisions impact (NCTM, 1989) and high-level academic
rate of growth on specific skills and skill appli- standards for all students have refocused
cations. For example, an essential question attention on the use of assessment in math
when considering the use of computation and instruction for progress monitoring (Fuchs,
concepts/applications probes in progress mon- 1992; Fuchs & Fuchs, 2001; Fuchs, Fuchs,
itoring is, What are reasonable goals for stu- Hamlett, Phillips, & Kams, 1995). The pur-
dents with leaming disabilities? As men- pose of this article is to discuss specific find-
tioned earlier, current estimates for rate of ings related to the use of CBM computation
growth are based on studies that have includ- and CBM concepts/applications probes in a
ed a majority of general education students statewide project on the use of a general out-
with relatively few students with learning dis- comes-based measurement approach to moni-
abilities. In particular, Deno, Fuchs, Marston, tor student progress. The project was imple-
and Shin (2001) reported data that suggested mented in response to growing recognition of
normative rates in CBM reading for students the need to document and account for stu-
with leaming disabilities were approximately dent progress. Though the statewide project
19

emphasized both reading and math outcomes port ( i.e., self-contained more than 50% of
of students with leaming disabilities, we will their day), 61.1 % (n =
69) receiving part-
emphasize only the findings in math. In the time or resource room support (i.e., special
results and discussion, we specifically address education services between 25-50% of the
critical issues related to progress monitoring day), and 11.5 % (n 13) receiving itinerant
=

such as goal setting and student rate of growth support (fully included, receiving special edu-
on both computation and concepts/applica- cation services for less than 25% of the day).
tions measures. Of the 7 students receiving emotional sup-
port, 4 received part-time support and 3
Method received itinerant support. A total of 16 stu-
Participants dents (all in two districts) were in middle
A total of 15 school districts in the
schools; the remaining 104 students in ele-
Commonwealth of Pennsylvania volunteered mentary school buildings.
to participate in the project 5 from the Materials and Dependent Measures
-

east, 5 from the central, and 5 from the west-


em regions of the state. Each district desig-
Math computation. The blackline masters
nated five special education teachers and a from Monitoring Basic Skills Progress
site coordinator to be part of the project. Each (MBSP) - Math Computation (2nd
edition)
teacher was asked to identify at least two stu- (Fuchs, Hamlett, Fuchs, 1998)
& were used to
assess student progress in math computation.
dents for whom General Outcomes
Measurement progress monitoring would be Each probe consisted of a single sheet of 25
mixed operation problems designed to assess
implemented in reading, math computation,
and math concepts/applications. Teachers mastery of computational skills typical for
were permitted to select the same student for each grade level. For example, problems for
progress monitoring in both reading and grades 1 and 2 consisted of addition and sub-
math. Only data collected for math are traction problems with and without regroup-
reported in this article. ing. Simple multiplication and division facts
Across the 15 districts, a total of 120 stu- were introduced in grade 3 probes, with frac-

dents were monitored in math computation tions and multidigit multiplication as well as
and 109 in math concepts/applications simple division introduced in grade 4 probes.
(teachers in two districts elected not to assess Decimals, complex fractions, and multidigit
concepts/applications) Within each district, division with remainders were introduced in
2-15 students were assessed (mean number of grade 5 and 6 probes. Each computation prob-
students assessed per district 9 in computa-
= lem was scored by counting the number of
tion, 8 in concepts/applications). digits correct in the final answer. The total
Students ranged in grade from 1 to 6 (mean number of digits correct was used as the
= 3.3 grade). Of the 120 students for whom dependent measure to reflect student perfor-
mance on math computation.
progress-monitoring data were collected in
computation, 5.8 % (7) were in programs for Math concepts/applications. The blackline
students needing emotional/behavioral sup- masters from Monitoring Basic Skills Progress

port (i.e., classified as emotionally/behav- (MBSP) - Math Concepts and Applications


iorally disordered). All others (n 113, 94.2
=
(Fuchs, Hamlett, & Fuchs, 1999) were used to
%) were students with leaming disabilities in assess student progress in math
need of learning support. Students with learn- concepts/applications. The probes consisted
ing support needs ranged in level of support, of 18 (grade level 2) or 24 problems (grade
with 27.4 % (n 31) receiving full-time sup-
=
levels 3 to 6) designed to assess mastery of
20

concepts and application of mathematical At each assessment session, students were


skills. The measures specifically cover count- given the amount of time specified by Fuchs
ing, number concepts, names of numbers, et al. (1998, 1999) to complete their progress

measurement, charts and graphs, money, frac- monitoring problems. Probes are set up such
tions, applied computation, and word prob- that students are not expected to complete all
lems. Problems began at grade 2, and contin- problems presented. For computation, stu-
ued to increase in difficulty through grade 6. dents were given 2 minutes for grades 1 and 2
Problems required between one and three probes, 3 minutes for grades 3 and 4 probes,
responses, and varied in type (fill in blanks, and 5 minutes for grade 5 probes. For con-
multiple choice). Each part of a problem cepts/applications, students were given 8 min-
answered correctly was awarded a point and utes for grade 2 probes, 6 minutes for grades 3

the total number of points across the probe and 4 probes, and 7 minutes for grades 5
was used as the dependent measure to reflect probes.
student performance on math concepts/appli- Teachers graphed all data. Teachers were
cation. first instructed to set year-end goals for stu-
dent performance. Data are limited on the
Progress Monitoring Procedures expected levels of performance of typical stu-
dents in progress monitoring of mathematics.
Teachers began by determining the level at
In thisstudy, data reported by Fuchs, Fuchs,
which each student would be monitored.
Hamlett, Walz, and Germann (1993) and by
Looking over the probes to be used for Fuchs et al., (1999) were used to make rec-
progress monitoring, teachers made a &dquo;best
ommendations to teachers for setting weekly
guess&dquo; regarding student instructional levels.
goals for expected performance. Based on
Teachers were asked to collect baseline data these studies and earlier pilot testing by the
and to determine if their estimate of student
investigators, it was recommended that teach-
instructional level was accurate. If their stu- ers set a goal of 0.5 digits per week gain in
dents scored at or near zero level of perfor- math computation and 0.5 points per week
mance in baseline, teachers moved their esti-
gain in math concepts/applications. These
mate of student instructional levels downward
levels of improvement were somewhat arbi-
and again collected baseline data.
trary (approximately 2 digits or 2 points per
Each student was assessed once each week. month), but were viewed as challenging and
Assessments of computation and consistent with performance levels found in
concepts/application were alternated, so that previous research.
students were assessed once every two weeks Teachers were trained to use a decision rule
in computation and concepts/applications. when examining their data. Specifically, they
Students were assessed at a time designated by were instructed to examine their data after 4
each teacher. Instructions recommended by points were collected and every few data
Fuchs et al. (1998, 1999) were used in admin- points thereafter. They were to compare the
istering the probes. Specifically, students were student’s performance against the expected
told that they were to look at each problem, level of performance (i.e., the student’s aim-
first attempt to complete the problems they line). Based on their analysis of the data,
believed that they could do, and then return teachers were instructed to (a) continue cur-
to those they were unsure about. This proce- rent instructional procedures if the trend in
dure ensured that students would demonstrate the student’s performance matched expected
success on the problems that they had level of performance; (b) increase a student’s
learned, maximizing their performance on goal if the student consistently exceeded the
each assessment measure. aimline; or (3) make a teaching change if the
21

student’sperformance was substantially below were made to each of the participating teach-
expected level of performance. Teachers were ers. At these visits, the data for each student

also given the option to continue existing were reviewed, concerns about the data col-

instructional practices even if the student was lection and/or graphing process were
below the aimline if they felt that the student addressed, and teachers were asked to demon-
needed more time to respond to the instruc- strate the data collection process with one
tional practices. Likewise, if students were student. Feedback was provided to correct any
making increasing progress, albeit not quite at errors in the data collection and graphing

the level set as the expected level of perfor- process, and teachers were asked to bring their
mance, teachers were encouraged to continue data to the next training session to be held in
current instructional practices. March.
The progress monitoring process began The March 2003 training session empha-
around the second week of November and sized making instructional changes. Teachers
continued throughout the school year until were provided with ideas and strategies for

approximately May 31. how to vary the instructional process, with a


focus on making simple changes in instruc-
Teacher Training tional process before committing to more
Training in the use of General Outcomes complex changes. At that session, teachers
Measurement (GOM) progress monitoring presented individual cases and shared the
nature of changes they had been making
was developed and conducted by the three
authors. Daylong inservice sessions were based on their data collection.
delivered in October, December, and March A final wrapup session was held in May
2002-2003, with onsite visits to individual 2003, either at each school district or as a
teachers in February as well as a wrapup ses- group inservice, to review students’ progress
sion to review the progress of all students in as well as address additional concerns from

May 2003. The initial session in October was teachers about the progress monitoring
dedicated to introducing the concept of process. At this final session, plans for contin-
GOM in reading and math, teaching teachers uing progress monitoring for the following
how to collect progress monitoring data, and school year were also discussed.
concepts of graphing the data. Teachers were
Results
instructed that they needed to select at least
two students, obtain baselines, and begin the Data were analyzed using several metrics.
progress monitoring process prior to the next Using the ordinary least squares (OLS)
session. method, each student’s performance for both
At the December training session, teachers computation and concepts/applications was
presented the outcomes of their data collec- regressed against assessment sessions. This
tion. The session emphasized the processes of of
resulted in the calculation slopes reflecting
instructional decision making, specifically the rate of change per assessment session.
using decision rules and analyzing data. Given that assessment data for computation
Teachers were taught to examine changes in and concepts/applications were collected
level, variability, and trend when considering every two weeks, the slopes were divided by
graphed student performance. Finally, they 0.5 to reflect the rate of change per week.
were instructed to begin examining their data The degree to which student performance
and to apply decision rules following the matched the goals set by the teacher was also
inservice. calculated. Goals for student performance
In January or February 2003, onsite visits were set based on achieving a gain of 0.5 dig-
22

its or points per week. To determine whether Participants’ performance in concepts/appli-


students met or did not meet the expected cation ranged from -0.5 to + 1.50 points per
rate of growth, a measure called the Goal week, with a mean change across participants of
Score was calculated for each student. The +0.38 points per week (SEM = 0.20).
Goal Score was obtained by subtracting the Multiplying the rate of gain of individual par-
student’s expected slope of performance (i.e., ticipants (0.38 points) by the number of weeks
0.5 digits/week or points/week) from the stu- in a school year (36), it would be estimated that
dent’s actual weekly slope of performance. these students would show a gain of 13.7 points
Values equal to or greater than zero would across the nine months of an academic school
reflect that the student’s rate of performance year.
met or exceeded the expected levels, whereas Participants’ outcomes by teacher-assigned
negative values would reflect performance instructional level are provided in Table 2. In
falling below expected levels. The percentage computation, students who were assessed at
of students whose Goal Scores met or exceed- the first- and second-grade level showed gains
ed the expected level of performance was cal- that were somewhat below the overall average
culated. gain of +0.38 digits or points per week.
Presented below are the outcomes of the Students assessed at third- and fourth-grade
slope calculations across students, the levels of instructional levels showed mean growth of
goal attainment as reflected in the calculated 0.43 and 0.72 digits per week, levels that were
Goal Scores, and two representative case stud- above the average gain across all participants.
ies that reflect the overall findings across stu- In concepts, students assessed in second-,
dents. Data for computation and third-, and fourth-grade levels improved at
concepts/applications are presented separately. rates of 0.36 to 0.44 points per week, very
close to the overall levels found across partic-
Overall Slope of Improvement ipants. In computation, much lower levels of
The average slope of student performance performance were found among the three stu-
in computation and concepts/applications dents working at the fifth-grade level; some-
across participants is displayed in Table 1. In what higher levels were found for
computation, participants ranged in perfor- concepts/applications. However, conclusions
mance from -1.23 to +1.77 digits per week, for students assessed at the fifth grade level
with a mean of change across participants of cannot be drawn because of the limited num-
+0.38 digits per week (SEM = 0.21). ber of students who were assigned to this level
Multiplying the average rate of gain of indi- of probes.
vidual participants (0.38 digits) by the num- Given that the students participating in
ber of weeks in a school year (36), it would be this project were all identified as having spe-
estimated that these students would show cial educational needs requiring learning or
gains of 13.7 digits across the nine months of emotional support, most students were being
an academic school year. assessed at instructional levels that differed by
Table 1 Mean and SEM (Standard Error of the Measure) of Slopes for Computation and
Concepts/Applications Across Participants _ __
23

Table 2 Mean Slope and SEM for Computation and Concepts Across Participants by
Teacher Assigned Instructional Level

one or more years from their chronological students who were five grade levels behind
grade levels. Table 3 displays the performance showed slopes of improvement that nearly
of students based on the difference between equaled overall levels across all participants.
their assigned grade and the instructional lev- The same pattern was not evident for math
els used for monitoring. concepts. The performance of students who
In computation and concepts, all but 25 were at grade level substantially exceeded the

and 10 students, respectively, were being overall level across participants. Both the 36
monitored at least one grade level below their students who were monitored one year below
chronological grade. In computation, students grade level and the one student whose
who were between grade level and two grades instructional level was five grade levels below
below showed rates of improvement that his grade (an 8th grader being monitored at
equaled or were somewhat higher than the the 3rd grade level) made much slower
overall slope across participants regardless of progress (approximately 0.20 points correct
instructional level. Students whose instruc- increase per week). Students who were two,
tional level fell three or four years below grade three, or four
grade levels behind showed
levels showed slower rates of gain than those increasingly higher levels of improvement,
two or fewer grade levels behind, but the ranging from 0.31 to 0.92 points per week.
number of students four grade levels behind The small number of students who were four
was quite small, making any conclusions or five grade levels behind makes any conclu-

about this group tentative. Interestingly, two sions about this group tentative.

Table 3 Mean Slope and SEM Across Participants Based on the Difference Between Grade
and Instructional Level for Progress Monitoring
24

Attainment of Goals 0.70 digits per week. A Goal Score was calcu-
lated by subtracting the value of her slope of
The percentage of students who attained a
Goal Score that was equal to or greater than performance from the weekly expected rate of
the expected levels of performance as set by gain set by her teacher (0.5 digits); her Goal
the instructors ( i.e., a gain of 0.5 digits or Score of +0.20 digits reflected a student who
exceeded the expected rate of gain.
points per week) is Figure 1. As
displayed in
Examination of the pattern of her perfor-
illustrated, in computation, approximately
mance showed that Kelly appeared initially
66% of students attained a rate of improve-
ment that met their goal, whereas in concepts not to be responsive to the instruction deliv-
ered in her math program, with data points
only 37% of students met this goal.
that fell below her aimline (11/21 to 1/6).
Illustrative Cases When her teacher examined the graph after
Two cases are presented to illustrate the data were collected on 1/28, she noticed an
rates of improvement typical across students. increase in Kelly’s performance and decided
to wait to see if the instruction being deliv-
Kelly. Kelly was fourth-grade student
a
enrolled in a part-time special education pro- ered as part of her math program was becom-
gram (25-50% of time in general education ing effective. When the data were examined
setting) for students who need learning sup- on 3/3, the teacher noted substantial
port (i.e., classified as having a learning dis- improvement in Kelly’s performance and
ability). Her teacher elected to monitor her decided that no specific instructional changes
were needed.
progress in mathematics computation at the
third-grade level, where Kelly was currently Niki. Figure 3 reflects Niki’s performance
being instructed. Figure 2 reflects Kelly’s per- in concepts/applications, and is a good illus-
formance in computation. Kelly had a base- tration of a student who made gains through-
line level of 14 digits correct and progressed out the progress monitoring process but who
over the school year to a rate of 25 digits cor- did not reach the level of expected perfor-
rect across the last three sessions. Her rate of mance as set by the instructors. Niki was a

improvement (slope) was calculated to be fifth-grade student being instructed and


Figure 1. Mean percent of students reaching goal as set by instructors.
25

Figure 2. Progress monitoring in computation for Kelly.

assessed in mathematics at the third-grade was not equal the rate set by her teachers,
to
level. Starting at a baseline of 23 points cor- and an earlier instructional change might
rect, the teacher set an improvement goal of have been beneficial, Niki clearly showed
0.5 points per week. Over the course of the gains in her performance once her teacher
seven months during which Niki was moni- paid attention to the probe data.
tored, she showed a gain of 0.15 points per
week. Her Goal Score, calculated by subtract- Discussion
ing the value of her slope from the expected Slope of Student Performance
slope of improvement, was -0.35 points per
week. Although Niki’s performance was In computation and concepts/applications,
below her aimline, from 1/19 through 2/16 students showed overall gains in performance
her teacher noted an increasing trend and across the school year. The level of gain for

elected not to change instruction. Data col- both computation and concepts/application
lected from 3/1 to 3/29 indicated that the was the same, equal to a gain of approximate-

increasing trend was no longer present. ly 14 total digits or total points over baseline
Through an analysis of the error patterns in levels across the academic year, a gain of
the probes, Niki’s teacher recognized that she approximately 0.38 digits or points per four
was struggling in the area of graphing and weeks .
between 3/29 and 4/2 implemented mini- When individual student performance was
lessons as part of her regular instructional examined based on the level at which stu-
activities. As a result, gains in performance dents were being monitored, we found similar
were reflected in the progress monitoring patterns across grade levels for computation
data. That is, Niki raised her performance and concepts. Between the second- and
from a baseline of 23 points to a level of over fourth-grade level, students’ rate of growth
31, 37, and 32 points in each of the last three tended to increase as instructional level
assessment sessions. Although the rate of gain increased. In computation, students moni-
26

Figure 3. Progress monitoring in concepts/applications for Niki.

tored at the fourth-grade level achieved highly responsive to the more intense and
improvement ratesthat were nearly triple focused instruction. However, when the gap
those of students at the second-grade level. between instructional and grade level widens
Similarly, in concepts students at the fourth- to three or four grade levels, students may be
grade level attained the highest slopes, so severely behind that instructional modifi-

although not substantially greater than those cations based on the progress monitoring data
at second- or third-grade instructional levels. alone are not sufficient to close the gap.
The impact of instructional level seemed A second explanation is that the wider gap
even clearer when looking at student perfor- between instructional and grade level reflects
mance when the gap between instructional a student who has a more severe and perva-

and grade level was considered. In computa- sive difficulty in acquiring mathematical
tion, students showed better growth when the understandings. Such students who are being
gap between instructional and grade level was taught well below their actual grade level
two versus one grade, but then showed the might be expected to progress at slower learn-
opposite effect when the gap widened to three ing rates, and this effect was displayed by the
or four grade levels. One explanation for this progress monitoring data collected in this
may be related to the amount of material that study.
a student needs to learn to &dquo;catch up&dquo; to In contrast to computation, students who
his/her chronological grade level. A teacher show wide gaps between instructional and
who is monitoring a student working below grade level in concepts/applications may be
grade level is likely to be intensifying instruc- offering teachers clear indications of where
tion on target skills that are assessed on the more intensive instructionneeded.
is
CBM probes. When the gap is one or two Although only a speculation, anecdotal evi-
grade levels, teachers have fairly prescribed dence from discussions with teachers found
choices of what to teach, and students may be performance on the concepts/application
27

probes easier to directly to instructional


link grade students. Thus, the special education
interventions than performance on the com- students in this project performed at levels
putation probes. Concepts/application probes that met or exceeded the rates of gain expect-
provide students with problems that sample ed of typically performing students at similar
areas of mathematical problem solving. curricular (or instructional) levels in compu-
Problem areas include numeration, decimals, tation and concepts/applications.
geometry, graphing, fractions, time, money, The issue of whether reading skills impact
measurement, and so forth. Teachers adminis- student performance on concepts and applica-
tering these measures can see directly the tions was raised in previous research. Fuchs et
areas in which students are struggling, and al. (1994) speculated that growth on concepts
can quickly adapt or intensify instruction and applications may be related to student
specifically in that area. reading skills. As student reading skills
How do the gains found across students in improved, their performance on concepts and
this project compare to those of general edu- applications improved. For that reason, one
cation students? Fuchs et al. ( 1993 ) and Fuchs
might question whether scores on concepts
et al. (1999) are the best sources for deter- and applications would be a true reflection of
mining the expected outcomes of students in the math performance of students with learn-
general education settings. Both of these pub- ing disabilities or confounded with their read-
lications report on the progress of students in
ing difficulties. Because the sample of students
general education classrooms across grades 1 with learning disabilities in our project
through 6 who were monitored across entire exceeded the performance levels reported for
school years in both computation and con- the Fuchs et al. (1999) sample of mostly gen-
cepts/applications. Fuchs et al. (1993) report eral education students, it did not appear that
realistic CBM weekly rates of improvement in
reading skills had an impact on student growth
computation for grades 1, 2, and 3 of 0.30 dig- in the area of concepts and applications.
its and rates of 0.45 for grades 4 and 5. Fuchs
et al. (1999) reported weekly rates of
Certainly, reading may be a problem for indi-
vidual students, but, overall, it appears that
improvement in concepts/applications of 0.40 with the progress students with learning dis-
for second graders, 0.58 points for third
abilities from this project made on concepts,
graders, and 0.69 points for fourth graders. reading was not an issue. It should also be
Lower slopes were reported for fifth (0.19)
noted that although reading skills are required
and sixth graders (0.12).
for the concepts and applications progress
Comparing these &dquo;normative&dquo; data against
the findings of the present project, the rate of monitoring assessment, there are still prob-
lems such as interpreting graphs, tables, and
gain in computation for students at first-,
third-, and fourth-grade instructional levels figures that do not involve the more extensive
exceeded the level identified as reasonable for reading often required by word problems.
general education students. At second-grade Goal Attainment
instructional levels, students in this project
nearly equaled the reasonable level reported Teachers were instructed to set goals of 0.5
for general education second graders. digits per week for all students. Based on the
Concepts/applications performance of stu- work of Fuchs et al. (1993), that goal level in
dents was also similar to that achieved in the computation is consistent with the recom-
Fuchs et al. (1999) data, except for students mendations of setting challenging weekly
assessed at third, and fourth-grade instruc- rates for grades 1, 2, and 3. In concepts/appli-

tional levels, who performed at a somewhat cations, the goal of 0.5 points per week is con-
lower level than typical third- and fourth- sistent with realistic levels of performance
28

based on the Fuchs et al. (1999) study, the tem for mathematics in their classrooms. The
only data available to make such a recom- use of both computation and concepts/appli-
mendation. In this project, approximately two cations appeared to be important, as the two
thirds of the students met or exceeded these areas represent somewhat different, albeit

goal levels in computation, and one third in related, skill domains, and were found to grow
concepts/applications. Additionally, when at different rates. Finally, the assessment

considering students whose Goal Scores fell schedule used (i.e., alternating weekly assess-
within one SEM, the percentage of students ment between computation and
who met or exceeded the goals increased to concepts/applications) appeared to be a rea-
82.5% in computation and to 50.9% in con- sonable timeframe for data collection.
cepts/applications. These findings suggest Teachers reported that conducting the
that many of the students who did not meet monitoring was feasible within their routine
the goals established by the teacher fell just schedule and that using the probes helped to
short of the expected levels. These very posi- shape their instructional decisions for all stu-
tive results may reflect the progress monitor- dents. For example, one teacher reported that
one of her students who had been struggling
ing project’s emphasis on using data to make
instructional decisions. Because the focus was in math computation was presented with a

on frequent data analysis and making sound


math probe containing fractions, a skill that
instructional adjustments, student achieve- the teacher had not yet introduced. When
ment was likely impacted. As discussed previ-
the student saw the problem, she asked ques-
tions about how one would do the calculation
ously, students leam more when teachers and demanded that the teacher &dquo;teach me
make individual adjustments in instruction
fractions.&dquo; To the teacher’s surprise, the stu-
through the use of CBM procedures (Stecker dent quickly acquired some knowledge of
& Fuchs, 2000). The consultative support and
fractions and demonstrated this knowledge in
training provided to teachers throughout the the next several probes. This type of reaction
project may also have contributed to achieve- was reported by several teachers for students
ment outcomes (Fuchs et al., 1991).
Overall, the recommended goal of increasing
taking the concepts/application probes as
student performance on computation at 0.5 dig-
well; thus, the progress monitoring process
was helping to define instructional content
its per week appears to be reasonable and
and focus for both the teacher and the stu-
attainable by many students with learning and dent. This example is reflective of findings
emotional disabilities. However, the goal of 0.5 from Fuchs, Fuchs, Hamlett, and Stecker
points per week for concepts/application (1990) related to the benefits of adding skill
appears to be a bit too challenging for many stu- analysis to CBM. Just like the above example
dents with special needs. This goal was derived of &dquo;teach me fractions,&dquo; considering mastered,
from the results of the single study reported by
emerging, and untaught skills in the context
Fuchs et al. (1999) that involved only 45-50 of progress monitoring seems to enhance the
students per grade level. Further studies are
quality of information teachers use to design
needed to more clearly establish normative lev- instructional interventions.
els of performance for concepts/application Another benefit of progress monitoring
among general education students. was that teachers used the data in feedback

sessions with students, fellow teachers, and


Considerations and Issues
parents. Several teachers indicated they had
The current project demonstrated that spe- shared the data during parent-teacher confer-
cial education teachers could successfully ences to illustrate student accomplishments,

implement a weekly progress monitoring sys- as well as a mechanism for focused discussion
29

on areas in need of more attention. Teachers ed by one of the authors) to help them record
also reported the usefulness of involving stu- and graph student data. Teachers were also
dents very directly in the progress monitoring offered a commercially available Macintosh
process. For example, some teachers asked software program linked to the probes that
students to graph their own data both with were being used (Fuchs et al., 1999). But only

pencil and paper and on the computer. a few of the teachers took advantage of these

According to these teachers, the motivation- enhancements. Future progress monitoring


al benefits of students recording and graphing projects should look for ways to promote the
their own data were integral to their students’ use of computer technology like Monitoring

achievement. Additionally, some teachers Basic Skills Progress, Basic Math (Fuchs et al.,
began to use the measures in the process of 1999) or Yearly Progress Pro (McGraw-Hill,
developing IEP goals. http://www.mhdigitalleaming.com) to stream-
Although the project had many benefits line the data collection, scoring, and graphing.
for students and teachers, several challenges One of the other challenges encountered
were identified. The teachers who participat- in this project was the difficulty that some
ed in this project were all volunteers. As such, teachers had making time to deliver instruc-
they were motivated to establish the progress tional interventions indicated as being neces-
monitoring system in their classes, and they sary by the progress monitoring data. This was
maintained a high level of implementation a particular problem for teachers who deliv-

fidelity. At the same time, teachers expressed ered special education services within a full
concern about the difficulties they had in inclusion model. Within this model, the spe-
finding time in their students’ schedules to cial education teachers were expected to sup-
incorporate the progress monitoring process. port students in the general education setting
As a result, some teachers elected to moni- by making accommodations to the general
tor only computation or only concepts/appli- curriculum while students were engaged in
cations. Others decidedto assess only reading, the standard classroom instruction. They
foregoing the math assessment entirely. One were not supposed to deliver supplemental or
of the challenges for all teachers doing specific instruction despite what was indicat-
progress monitoring was to make the process ed by progress monitoring.
part of the normal weekly instructional rou- Many of these teachers reported that while
tine. Teachers who could see gains in student they collected the data and recognized when
performance, could recognize the sensitivity instructional modifications were needed, they
of the measures to instruction, and could use could not find the time in a student’s schedule
the data for instructional decision making to deliver such instruction without sacrificing
found ways to incorporate the progress moni- the required support that general education
toring of math into their instructional pro- teachers expected for students who were fully
grams. included. Unfortunately, these observations
Another challenge was related to the are similar to previous research reported by

teachers’ technological capabilities. Most of Fuchs et al. (1995) on the extent to which
the teachers chose to use paper-and-pencil general education teachers engage in special-
graphing methods. Although these methods ized instructional adaptations as result of the
are valuable, the process of progress monitor- use of CBM in math instruction. Thus, Fuchs

ing can be enhanced by using technology such et al. (1995) remarked on the substantial vari-
as computerized generation of graphs. During ability in general education teachers’ use of
the project, teachers were offered some pre- progress monitoring data to make individual
programmed Microsoft Excel software (initial- adjustments in instruction. Such issues con-
ly developed by one of the teachers and adapt- tinue to raise caution regarding the extent to
30

which specialized adaptations made in


are would be collected. The students who were
general education settings for students with selected may not have been students with the
learning disabilities, as well as suggest the most severe problems in mathematics. Indeed,
need for further research on how to imple- our data showed that at least in computation

ment progress monitoring in inclusive mod- and concepts/applications, many students (25
els. This did not appear to be a problem for and 10, respectively) were not deficient in
teachers who were assigned to part-time, mathematics achievement. Future studies
resource, or full-time special education ser- should more carefully select students for
vice delivery models although they, too, had progress monitoring based on identified levels
instructional routines that made it difficult to of mathematics problems.
find time to do new things.
Conclusions
Limitations
Despite the limitations, the outcomes of
This project was not designed to be a care- this project show that the monitoring of com-
fully controlled research study. Therefore, putation and concepts/applications together
explanations for the levels and patterns of can be a powerful tool for examining the

improvement noted among students cannot progress of special education students within
be derived empirically. Although teachers math curricula. Students in this project dis-
were instructed to use the data to make played gains in mathematics skills that were
instructional decisions, and many appeared to equal to or better than those of typical stu-
have done so throughout the project, the dents at equivalent grade levels. Given that
degree to which teachers systematically made the majority of students were performing one
use of the data to drive instruction was not or more grade levels below their chronologi-

well documented. Thus, we do not know if cal grade, progress monitoring served to
some students did not make progress because demonstrate the potential to accelerate learn-
their teachers were not examining their data ing outcomes for students in special educa-
as instructed. It is well established that using tion. Teachers were capable of fully managing
progress monitoring data to make instruction- the system, integrating both data collection
al decisions is the best predictor of successful and analysis into their weekly routine.
student outcomes. However, simply collecting While instructional gains of 1.5 digits per
the data is not sufficient to result in academic month in computation may seem relatively
improvement (Fuchs & Fuchs, 1986). small in terms of overall rate of growth, the
Another limitation of the project is that progress made by students with learning dis-
teachers were all volunteers. As such, they abilities in this project highlights the impor-
were motivated to learn and use the progress tance of viewing growth over an extended

monitoring processes. Given that most of the period of time. Teachers trying to make
teachers were quite naive and unfamiliar with instructional decisions on a daily or weekly
the concepts of General Outcome basis may express frustration at what appears
Measurement prior to the study, we do not to be only slight patterns of student progress.
know the degree to which teacher self-moti- However, our data suggest that the progress
vation and interest resulted in better out- that students with learning disabilities make
comes than one might get with teachers who in math when viewed over extended time is
are less predisposed to implementing the comparable to the progress of nondisabled
progress monitoring process. students at similar instructional levels.
Finally, teachers were free to select stu- Therefore, fairly significant gains in student
dents from whom progress monitoring data math achievement may be linked to the
31

extended of progress monitoring even


use Fuchs, L. S., & Fuchs, D. (2001). Computer applications
to curriculum-based measurement. Special Services in
though only small gains are observed on a the Schools, 17
(1), 1-14.
week-by-week basis. Additional research is Fuchs, L. S., & Fuchs, D. (2002). Mathematical prob-
needed to explore the use of progress moni-
lem-solving profiles of students with and without co-
toring in computation and applications in morbid reading disabilities. Journal of Learning
math, but it appears that committed and (6), 563-573.
Disabilities, 35
deliberate use of progress monitoring over an Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Allinder, R.
extended period of time can have critical M. (1989). Reliability and validity of skills analysis
within curriculum-based measurement. Diagnostique,
impact on student performance - accelerated 14, 203-221.
leaming rates were observed for students with Fuchs, L. S., Fuchs, D., Hamlett, C. L., Phillips, N., &
leaming disabilities. As the challenges many Karns, K. (1995). General educators’ specialized
students with leaming disabilities have with adaptation for students with learning disabilities.
math are considered, the impact of progress Exceptional Children, 61, 440-459.
monitoring and individual instructional deci- Fuchs, L. S., Fuchs, D., Hamlett, C. L., Stecker, P. M.
sion making on math achievement appears (1991). Effects of curriculum-based measurement
and consultation on teacher planning and student
unequivocally worthwhile, especially if achievement in mathematics operations. American
growth is accelerated over time. Educational Research Journal, 28
(3), 617-641.
Fuchs, L. S., Fuchs, D., Hamlett, C. L., Thompson, A.,
References Roberts, P. H., Kubek, P., et al. (1994). Technical
features of a mathematics concepts and applications
Calhoon, M. B., & Fuchs, L. (2003). The effects of peer-
assisted and curriculum-based curriculum-based measurement system. Diagnostique,
learning strategies
measurement on mathematics performance of sec- , 23-49.
19
ondary students with disabilities. Remedial and Special Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, C., &
Education, 24
(4), 235-245. Germann, S. (1993). Formative evaluation of acade-
mic progress: How much growth can we expect?
Deno, S. (1992). The nature and development of cur-
riculum-based measurement. Preventing School School Psychology Review, 22 (1), 27-48.
Failure, 36
(2), 5-10. Fuchs, L. S., Fuchs, D., Hosp, M. K., & Jenkins, J. R.
Deno, S. L., Espin, C. A., & Fuchs, L. S. (2002). (2001). Oral reading fluency as an indicator of reading
Evaluation strategies for preventing and remediating competence: A theoretical, empirical, and historical
basic skill deficits. In M.R. Shinn, H. W. Walker, & analysis. Scientific Studies of Reading, 5
(3), 239-256.
G. Stoner (Eds.), Interventions for academic and Fuchs, L. S., Hamlett, C. L., & Fuchs, D. (1998).
behavior problems II: Preventative and remedial Monitoring basic skills progress: Math computation.
approaches (pp. 213-241). Bethesda, MD: National Austin, TX: Pro-Ed.
Association for School Psychologists. Fuchs, L. S., Hamlett, C. L., & Fuchs, D. (1999).
Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. Monitoring basic skills progress: Concepts and applica-
(2001). Using curriculum based measurement to tions. Austin, TX: Pro-Ed.
establish growth standards for students with learning Jitendra, A., DiPipi, C. M., & Perron-Jones, N. (2002).
disabilities. School Psychology Review, 30, 507-524. An exploratory study of schema-based word-prob-
Fuchs, L. S. (1992). Classwide decision making with lem-solving instruction for middle school students
computerized curriculum-based measurement. with learning disabilities: An emphasis on concep-
Preventing School Failure, 36
(2), 30-33. tual and procedural understanding. Journal of Special
Fuchs, L. S., & Deno, S. L. (1991). Paradigmatic dis- (1), 23-38.
Education,16
tinctions between instructionally relevant measure- Maccini, P., & Hughes, C. A. (2000). Effects of a prob-
ment models. Exceptional Children, 57
, 488-500. lem solving strategy on the introductory algebra per-
Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formance of secondary students with learning dis-
formative evaluation: A meta-analysis. Exceptional abilities. Learning Disabilities Research and Practice,
Children, 53, 199-208. (1), 10-21.
15
Fuchs, L. S., & Fuchs, D. (1999). Monitoring student National Council of Teachers of Mathematics. (1989).
progress toward the development of reading compe- Curriculum and evaluation standards for school mathemat
-
tence : A review of three forms of classroom-based ics. Reston, VA: Author.
assessment. (4), 659-
School Psychology Review, 28 Owen, R. L., & Fuchs, L. S.(2002). Mathematical prob-
671. lem-solving strategy instruction for third-grade stu-
32

dents with learning disabilities. Remedial & Special Learning Disabilities Research and Practice, 15
(3), 128-
Education, 23, 268-278. 134.
Shapiro, E. S. (1996). Academic skills problems: Direct Thurber, R. S., Shinn, M. R., & Smolkowski. K. (2002).
assessment and intervention (2nd ed.). New York: The What is measured in mathematics tests? Construct
Guilford Press. validity of curriculum-based mathematics measures.
Shepard, L. A. (2000). The role of assessment in a School Psychology Review, 31, 498-513.
learning culture. Educational Researcher, 29
(7), 4-14. Woodward, J., & Montague, M. (2002). Meeting the
Shinn, M. R. (Ed.). (1989). Advanced applications of cur- challenge of mathematics reform for students with
riculum-based measurement. New York: Guilford. learning disabilities. Journal of Special Education, 36,
Shinn, M. R., Shinn, M. M., Hamilton, C., & Clarke, 89-101.
B. (2002). Using curriculum-based measurement in
general education classrooms to promote reading Author Note
success. In M. R. Shinn, H. M. Walker, & G. Stoner
(Eds.), Interventions for academic and behavior prob- This project was conducted as part of an
lems II: Preventative and remedial approaches (pp. 113- by the Bureau of Special Education,
initiative
142). Bethesda, MD: National Association of School
Pennsylvania Department of Education, Dr.
Psychologists. Fran Warkomski, Director, and the
Stecker, P. M., & Fuchs, L. (2000). Effecting superior
achievement using curriculum-based measurement: Pennsylvania Training and Technical
The importance of individual progress monitoring. Assistance Network (PaTTAN).

You might also like