Buffalo Public Schools

SUMMER - ELOP ANALYSIS OF MCLASS:DIBELS PERFORMANCE

Summary Thoughts /Observations

Although open to all students, the ELOP disproportionately enrolls Buffalos most challenged students (i.e., students in the intensive intervention/high risk category) Preliminary analysis indicates that ELOP students essentially perform similarly to non-participating students from one year to the next

Research studies commonly find that these types of students are most likely to suffer summer learning losses. Given these expectations, ELOPs ability to keep pace with non-participating students might be viewed as a mitigant of summer learning loss. In other words, participating students might be viewed as performing better than they would have had they not attended the program.
Further analysis could explore whether students are making greater gains at the DIBELS skill level (e.g., on ORF scores) rather than requiring change at the broader instructional recommendation level.

Profile of ELOP Participants

The chart below provides a comparison of the starting skill levels of ELOP participants, in comparison to those Buffalo students who did not attend the summer session.

Profile of ELOP Participants

It is clear that the ELOP students draw disproportionately from the districts lowest performing students. Note how for each grade there are as many, and sometimes more, students performing at the red/intensive level who subsequently attended the ELOP compared to those who did not. Obviously, there are significantly more students who do not participate in the summer program than those who do, and the majority of these students are not surprisingly already performing at grade-level benchmarks (compare the size of the green/benchmark bars in the graph).

Profile of ELOP Participants

The following chart makes this even clearer, by presenting the same comparison using a percentage distribution rather than raw counts of students. The bars skew toward red for ELOP participants, and much more toward green for non-participants.

Comparative DIBELS Performance of ELOP Participants

To compare the performance of ELOP students to those who did not attend, we compared DIBELS instructional recommendations for EOY from the prior grade (i.e., before the summer program) to those for BOY from the current year (i.e., after the summer program). Ideally, participation in ELOP would result in gains for students when they return to school at BOY, at least when compared to those who did not participate. A summary view does not show much difference between ELOP and non-participating students. See the two charts that follow. The first shows the change in DIBELS instructional recommendations from EOY to BOY for ELOP participants. As can be observed, there is not much change from EOY to BOY as students move from one grade to the next, and in several grades, it actually appears to worsen somewhat (see the increase in percentage of red/intensive students).

Comparative DIBELS Performance of ELOP Participants

Comparative DIBELS Performance of ELOP Participants

By contrast, the following chart provides a comparison of DIBELS instructional recommendations for all students not participating in the summer program. While the percentage of students performing at each level differs notably from the ELOP students, their performance mirrors the ELOP students in its general lack of change from end of the prior year to beginning of the current year.

Comparative DIBELS Performance of ELOP Participants

Comparative DIBELS Performance of ELOP Participants

The table that follows shows this comparison at a more granular level comparing the degree to which students have moved from one instructional level to another (similar to an effectiveness report in our Reporting and Analysis Suite). The table places the performance of non-ELOP and ELOP students sideby-side for each grade-level transition, based on their prior years performance level. For example, the first row shows Kindergarten students who performed at the intensive level at EOY, with each column showing the percentage of these students who performed at each level the following BOY (in 1st grade). We see that for non-participating students, 46% of students who ended Kindergarten in the intensive category remained in this category when they returned to school as first graders, while only 41% of ELOP students with the same starting category remained there at BOY.

10

ELOP vs. Non-ELOP Comparison

Performance at BOY 2011-12 Intensive K-to-1st Intensive Strategic Benchmark 1st-to-2nd Intensive Strategic Benchmark 2nd-to-3rd Intensive Strategic Benchmark 3rd-to-4th Intensive Strategic Benchmark 4th-to-5th Intensive Strategic Benchmark 5th-to-6th Intensive Strategic Benchmark 46% 14% 2% 75% 33% 2% 81% 15% 1% 90% 32% 1% 86% 12% 1% 57% 3% 0% Not ELOP Strategic Benchmark 39% 48% 14% 12% 58% 23% 17% 69% 25% 9% 48% 19% 14% 45% 8% 37% 43% 5% 15% 37% 84% 13% 9% 75% 3% 15% 75% 1% 20% 80% 1% 43% 91% 7% 53% 94% Intensive 41% 18% 3% 74% 40% 3% 75% 19% 1% 90% 37% 3% 81% 14% 1% 64% 4% 2% ELOP Strategic 40% 46% 16% 9% 47% 31% 20% 65% 26% 8% 48% 24% 18% 48% 13% 28% 50% 15% Benchmark 19% 36% 81% 17% 13% 66% 4% 16% 72% 1% 14% 73% 1% 38% 86% 8% 47% 84%

Performance at EOY 2010-11

11

ELOP vs. Non-ELOP Comparison

We have highlighted in green the specific instances in which ELOP students appear to outperform non-participating students, which is primarily for students starting the summer at the intensive/high risk level. The table shows that for most grades and risk levels, participating in ELOP does not seem to produce significantly different outcomes by the beginning of the following year.

12