Articles Remedial and Special Education

Volume 30 Number 1
January/February 2009 3-18
© 2009 Hammill Institute on
Disabilities
Preparing Students for College Entrance Exams 10.1177/0741932507314022
http://rase.sagepub.com
hosted at
Findings of a Secondary Intervention Conducted Within http://online.sagepub.com

a Three-Tiered Model of Support

Kathleen Lynne Lane
Vanderbilt University
Jemma Robertson Kalberg
Vanderbilt University
Emily Mofield
Vanderbilt University
Joseph H. Wehby
Vanderbilt University
Robin J. Parks
Vanderbilt University

This study examined outcomes associated with participation in a program, Preparing for the ACT, designed to enhance
student performance (N = 126) on the ACT college entrance exam. This targeted intervention was implemented as part of a
three-tiered model of positive behavior support. Results of descriptive analyses revealed that only academic performance
in the previous academic year was significant in predicting postintervention practice scores. Furthermore, students’ post-
intervention scores were significant in predicting actual ACT scores. However, only in the case of the English subject area
test were academic and behavioral performance predictive of English ACT scores. Results of a quasiexperimental design
used to compare actual ACT performance for students who did and did not participate in the intervention suggested
improved performance for students who did participate in the program, as evidenced by positive effect sizes—an increase in the
percentage of students who met the district target scores and school mean scores that exceeded state mean scores following
intervention participation. Limitations and implications for future research are offered.

Keywords: positive behavior supports; quantitative research methodology; secondary

W ith the call for academic excellence (No Child Left

Behind Act, 2001), combined with the goal of serv-
ing students with special needs in inclusive settings (Fuchs
& Fuchs, 1994; Individuals with Disabilities Education
Improvement Act, 2004), there has been a growing
emphasis on looking at schools as a context for change,
rather than focusing primarily on child concerns (Lane &
Beebe-Frankenberger, 2007; Sugai & Horner, 2002). This
has been accomplished via response-to-intervention (RTI;
Gresham, 2002) and positive behavior support (PBS;
Sugai & Horner, 2002) models, both of which involve
data-driven, three-tiered models of support that contain
primary, secondary, and tertiary levels of prevention.
In both models, interventions of increasing intensity and
depth are provided based on students’ needs. For example,
primary plans, such as violence prevention (Sprague et al.,
2001) and district-developed literacy and academic plans
(Lane & Menzies, 2003; Luiselli, Putnam, & Sutherland,
2002), are implemented with the entire student body.
Primary plans focus on preventing harm by eliminating
conditions that set the stage for learning and behavioral
concerns. Next, secondary prevention efforts typically lend
support for students who are nonresponsive to primary
intervention efforts (10%–15% of students). The final level
of support is the most intensive, reserved for students
with multiple risk factors (3%–5% of students) who
require tertiary prevention supports (e.g., function-based
Authors’ Note: This project was funded by an OSEP-funded, directed
research grant titled Project PBS: A Three-Tiered Prevention Model to
Better Serve All Students (OSEP Award Number, H324D020048) and
NICHD Grant No. P30HD15052 to Vanderbilt University.

Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015 3

4 Remedial and Special Education
interventions, Kern & Manz, 2004). Data are used to
monitor student performance at each level and determine
which students require more intensive, evidence-based
supports.
Another less traditional approach to secondary inter-
ventions is to provide support to subgroups of students
who have historically been less than successful in meet-
ing a given objective; namely, secondary programs may
also be used to support students with common core areas
of concern who may not have yet failed but who have
specific instructional or behavioral needs that are not char-
acteristic of the entire student body’s needs (Lane &
Parks, 2007). For example, if high school freshmen have
historically struggled to negotiate the demands of long-
term assignments, an intervention could be designed to
teach only this group of students the requisite skills asso-
ciated with such tasks (e.g., planning, incremental objec-
tives, etc.; Lane, Wehby, & Robertson, 2007). In this case,
the PBS team does not wait for students to fail on long-
term assignments. Instead, a secondary intervention could
be implemented to reverse the negative consequences
typically experienced by pervious freshman classes by
addressing the appropriate acquisition or performance
deficits (Gresham, 2002).
Many schools, particularly at the elementary level, have
made strides implementing primary (Lane, Robertson, &
Graham-Bailey, 2006) and tertiary (Lane, Umbreit, &
Beebe-Frankenberger, 1999) prevention programs. Yet
less attention has been devoted to secondary prevention
efforts, particularly in middle and high schools (Robertson
& Lane, 2007).
Secondary Prevention Efforts
One challenge facing schools that are developing PBS
models, in particular, is how to design, implement, and eval-
uate secondary interventions during the regular school day.
A systematic review of the literature revealed only a hand-
ful of studies examining the efficacy of secondary interven-
tions implemented within the context of multitiered PBS
models. Most studies identified were conducted at the ele-
mentary level (Lane, Wehby, Menzies, Doukas, et al. 2003;
Lane, Wehby, Menzies, Gregg, et al., 2002; Walker, Cheney,
Stage, & Blum, 2005), with very few secondary interven-
tions implemented within the context of a PBS model at the
middle- and high school levels. For example, Robertson and
Lane (2007) used course grades, behavior screening, and
schoolwide information system (May et al., 2000) data to
identify middle school students (N = 65) who had both aca-
demic and behavioral concerns. Students were assigned
randomly to one of two interventions: study skills only or
study skills plus conflict resolution skills, both of which
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
were taught as an elective course. Results revealed changes
in students’ skill knowledge; however, generalization did
not occur to study skills habits, conflict resolution styles,
grade point average, and risk status.
Results of an electronic and hand search did not iden-
tify any secondary interventions implemented within the
context of a three-tiered PBS model at the high school
level. This absence is concerning given that high school
is the last point of public education before students either
enter the work force or continue on in higher education.
We think a particularly concerning void within the three-
tiered model is the absence of targeted intervention
support to assist high school students in preparing for
college entrance exams. Students’ performance on the
college entrance exams, such as the SAT and ACT, serve
as an important predictor of which students will pursue
the college path (Fuller & Wehman, 2003).
College Entrance Examinations:
A Pivotal Determinant
College admissions committees consider a number
of important factors when determining which students
to admit, one of which is a student’s performance on
college entrance exams (Ehrehnhaft, Lehrman, Obrecht, &
Mundsack, 2001; Fuller & Wehman, 2003). For example,
the ACT is taken by numerous 11th-grade students
across the country, and admissions committees use these
scores to compare the academic achievement of applicants
and to draw inferences about the probability of success-
ful performance at the university level (Ehrehnhaft et al.,
2001). The ACT is a time-limited, multiple-choice test
that includes English, mathematics, reading, science, and
writing subject area tests, with the items assessing content
knowledge as well as students’ ability to infer, analyze,
problem solve, and reason.
Given the important outcomes associated with student
performance on the ACT, it is likely that students will
need to, or should, spend time preparing to take this
exam. Yet a substantial body of literature indicates that
secondary school students in the United States have poor
study skills (Jones & Slate, 1992; Jones, Slate, Bell, &
Sadler, 1991; Jones, Slate, Blake, & Sloas, 1995; Slate,
Jones, & Dawson, 1993). Jones and Slate (1992) report
that high school students exhibit only 40% to 46% of
desired study behaviors as measured by the Study Habits
Inventory (SHI). Kovach, Fleming, and Wilgosh (2001)
found similar results, with 568 high school students
reporting that they performed fewer than 50% of the
study habits delineated in the SHI.
In addition, the extent to which students used effective
study skills declined between 8th and 11th grade because

of heavy reliance on rote memorization (Jones et al.,
1995). Jones and colleagues indicated that the most com-
mon deficits characteristic of high school students were
poor reading skills, inappropriate use of study skills,
and lack of application of organizational strategies.
Unfortunately, many high school study skills programs
fail to address students’ specific acquisition deficits (Jones,
Slate, Mahan, et al., 1993/1994) and do not provide
explicit instruction to help prepare students to take their
college entrance exams. Jones, Slate, Bell, et al. (1991)
contend that it is imperative for teachers to become even
more involved in improving students’ study skills.
This preliminary investigation sought to address this
call by establishing a targeted, secondary intervention,
implemented within the context of a three-tired PBS
model, to assist in preparing 11th-grade students to take
the ACT college entrance exam. Rather than waiting to
identify students who were nonresponsive, as in traditional
models, we conducted a targeted, secondary intervention
for current 11th-grade students given that previous years’
11th-grade students had been less than successful with
their performance on the ACT test. Data on ACT test
scores from the previous 11th-grade classes indicated that
school-site group means fell below the state means. In
brief, 11th-grade students needed additional support
beyond that which was received as part of the primary
plan to perform more successfully on the ACT test.
Consequently, we designed a curriculum to supplement
the primary intervention program to better prepare students
for the ACT test. Furthermore, we wanted to extend
this line of inquiry by examining the extent to which
students with various academic and behavioral perfor-
mance levels responded to the support provided.
Purpose
Thus, this preliminary study examined outcomes asso-
ciated with participation in a program, Preparing for the
ACT, designed to enhance student performance on the
ACT college entrance exam. First, descriptive procedures
were employed to determine the extent to which students’
behavioral and academic characteristics, as measured by
office disciplinary referrals (ODRs) and grade point aver-
age (GPA), respectively, predicted performance on both a
practice ACT test, as well as the actual ACT, for students
who completed the program. Second, a quasiexperimen-
tal, two group, posttest only design was used to compare
actual ACT performance for students who did (11th-grade
students, 2005–2006 academic year) and did not (11th-
grade students, 2004–2005 academic year) participate in
the intervention. We hypothesize that students (a) with
lower rates of ODRs earned during the previous academic
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
Lane et al. / College Entrance Exams 5
year and high GPAs from the previous academic year
would be more responsive to the intervention as measured
by performance on the practice ACT test; (b) with higher
postintervention, practice ACT scores, lower rates of
ODRs, and higher GPAs would have higher actual ACT
scores; and (c) who participated in the intervention pro-
gram would perform better on the actual ACT test than
students who did not participate in the program. Third,
we used descriptive procedures to examine teacher and
student perspectives regarding the goals, procedures, and
outcomes associated with this plan. We hypothesized that
teachers would view the program as more socially valid
as compared to students.
Method
Participants and Setting
Participants were 126 eleventh-grade students, 66
(52.38%) boys, and 60 (47.62%) girls, attending a rural
high school located in Middle Tennessee that was partici-
pating in a longitudinal study of PBS at the high school
level. The majority of students were White (n = 120,
95.24%), with 18 (14.29%) receiving special education
services.
All students attended School F, which enrolled a total
of 716 students. The school was predominately White
(n = 684; 95.53%). School F was above the district mean
of economically disadvantaged students. Data were not
available as to the number or percentage of graduating
seniors who entered 2- and 4-year degree programs. There
were 109 students (15.20%) enrolled in special education
at School F. School F served a total of 188 eleventh-grade
students, all of whom participated in the ACT interven-
tion. Of these students, 136 were (a) enrolled at School F
during the 2004 to 2005 and 2005 to 2006 academic years
and (b) completed the pre- and postintervention practice
ACT test. However, 10 students had patterned responses
(e.g., Christmas tree pattern or marked the same letter for
all items) and were excluded in the analyses. The final
number of participants thus was 126 (see Table 1).
Procedures
School F was participating in a federally funded, lon-
gitudinal study examining the design, implementation,
and evaluation of schoolwide positive behavior support
(SW-PBS) plans at the high school level. See Lane, Wehby,
Robertson, and Rogers (2007) for a detailed description
of the SW-PBS program. The secondary intervention
reported in this article occurred during School F’s second
year of implementation of a SW-PBS plan. Fidelity of
6 Remedial and Special Education
Table 1
Participants Characteristics
Students (N = 126)
Characteristic N %
Gender
Male 66 52.38
Female 60 47.62
Ethnicity
Caucasian 120 95.24
African American 4 3.17
Hispanic 2 1.59
Special education 18 14.20
Special learning disability 9 7.14
Gifted 8 6.35
Other health impaired 1 0.79
the SW-PBS plan was monitored from the teacher (self-
report) and research assistant (RA; direct observations)
perspectives using behavior component checklist. Fidelity
was adequate from both perspectives, with respective
overall annual implementation percentage of 78.70
(SD = 1.91, teacher perspective) and 74.08 (SD = 15.56).
The PBS team held monthly meetings to discuss logis-
tics for implementation and to review schoolwide data to
monitor student progress. At the end of the first year of
implementation, the PBS team identified two key areas for
targeted, secondary intervention, one of which included
designing a program to prepare 11th-grade students for the
ACT tests. During the summer, the PBS team collaborated
with Vanderbilt University to design a curriculum, which
included a prepost assessment and student opinions about
the program, to be implemented during the following
school year as part of regular school practices.
The principal and PBS team at School F decided to
implement an ACT preparation program for all 11th-grade
students as part of their PBS program to be taught by
teachers during homeroom as part of regular school prac-
tices. Specifically, teachers would teach an ACT prepara-
tion program, Preparing for the ACT (described below), to
11th-grade students once per week during homeroom time
and administer ongoing probes to practice for the actual
ACT test taken in the spring. One reason for the focus on
preparing students for the ACT was to increase the likeli-
hood of students being able to attend college, particularly
given that students in Tennessee now had the opportunity
to receive college funding through the Tennessee Hope
Scholarship program that awarded college funding to
students based on ACT scores.
Prior to implementation, the principal organized a meet-
ing with all teachers that would be teaching the ACT
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
preparation curriculum to the 11th-graders during home-
room. The project director attended this initial meeting
and reviewed the curriculum and implementation calendar
with all participating teachers. A sample lesson was pro-
vided to introduce the format of the lesson plans, which
included both scripted and brief outlined versions. All
lessons included procedures to introduce the information,
model the skill, lead guided practice, and provide time for
independent practice with corrective feedback.
Teachers were informed that the curriculum was going
to be taught as part of regular school practices, separate
from any research project, and that teacher participation
in the evaluation study was optional. Specific teacher
duties beyond the regular school day included (a) allowing
project staff to observe instruction for 30 min twice
per month to assess treatment integrity, (b) completing a
similar treatment fidelity form to assess implementation
from the teacher perspective, and (c) filling out a social
validity form to obtain their opinions about the curricu-
lum. The project director read through the consent letter
and answered questions. Teachers were informed that
information collected would be kept confidential and
they could withdraw at any time. All teachers consented
to participate (N = 10).
Intervention Description: Preparing
for the ACT Curriculum
A number of studies indicate the effectiveness of teach-
ing test-taking skills to increase student performance
(Bangert-Drowns, Kulik, & Kulik, 1983; Dreisbach &
Keogh, 1982; Ritter & Idol-Maestas, 1986; Sampson,
1985). Specifically, research on teaching test-taking skills
indicates that when the following guidelines in test-taking
strategy lessons are implemented, students achieve higher
test performance: (a) Students take a pretest to determine
types of questions they need to master; (b) the teacher
provides modeling for choosing the correct answer;
(c) students practice test items that mirror the actual test
they will take; and (d) students apply learned test-taking
strategies on a timed posttest (Scruggs & Mastropieri,
1992). The scope and sequence of the Preparing for the
ACT curriculum, developed by Vanderbilt University’s
Project PBS staff with input from two high school PBS
teams, included these guidelines to help students achieve
maximum test performance on the ACT test (see Table 2).
The goal of the curriculum was to enhance test readiness
by (a) introducing students to the format of the assessment;
(b) providing multiple opportunities to practice exam
questions, (c) teaching general test-taking strategies, and
(d) teaching additional specific test-taking strategies for the
four subject area tests: English, reading, math, and science.
 Lane et al. / College Entrance Exams 7

Table 2
Curriculum Overview
Lesson Topic CBM Probe Lesson Topic CBM Probe

1 Overview: ACT test; the program and 14 English 3: Sentence structure and
curriculum; benefits of strong grammar review
performance on the actual ACT; 15 Math 3: Overview of content areas
and CBM probes. 16 Reading 3: Reading Speed
2 Curriculum-based probe #1 1 17 Science 3: Strategies for reading
3 Test-Taking Strategies research summary passages
4 English 1: Mechanics and rhetoric; 18 Curriculum-based probe #4 4
long-range strategies 19 English 4: writing strategy,
5 English 2: Pacing tactics and organization, transition, and main
punctuation idea statements
6 Math 1: Overview of potential areas 20 Math 4: study strategies for the
(pre-algebra; elementary, ACT math test
intermediated, coordinate, and plane 21 Reading 4: identifying distracters,
geometry; and trigonometry) question types; and shifts in thoughts
7 Math 2: Approaches to algebra problems 22 Curriculum-based probe #5 5
8 Curriculum-based probe #2 23 Science 4: Research summary strategies
9 Reading 1: Description of test; types 2 24 English 5: English style, word choice,
of questions and language
10 Reading 2: Techniques for skimming; 25 Reading 5: reading review and practice;
deciding on a technique marking passages
11 Science 1: Content areas covered 26 Curriculum-based probe #6 6
(Earth or space; chemistry; and
physics) and science skills tested
12 Science 2: Types of data representation
13 Curriculum-based probe #3 3

Note: CBM = curriculum-based measurement.

The curriculum contained 26 lessons, each 30 min in
length, for a total of 13 hours of intervention.
Lesson 1 contained an overview of the ACT test
(ACT, 2004a), the program and curriculum, benefits asso-
ciated with successful performance on the actual ACT,
and self-monitoring procedures as applied to graphed
progress on the four subject area tests as measured by
curriculum-based measurement (CBM) probes. Students
were informed that they would complete mini tests every
5 to 6 weeks with five questions from each subject area
tests. Sample test items obtained from the official ACT
Web site (ACT, 2004) were reprinted with permission
from Preparing for the ACT Assessment. Results were
graphed to monitor student progress and help students
make decisions about how to focus their study efforts.
After the first lesson, students were administered a
full practice ACT test also obtained from the official
ACT (2004a) Web site to familiarize the students with
the format and content of the ACT test and to obtain a
baseline level of performance. The test was adminis-
tered following the standard procedures of an actual
ACT test administration, including all four subject area
tests with a break following the second test. Students
took the test in classrooms and the library located in one
wing of the building. Hall monitors were present to min-
imize noise from other parts of the building. Immediately
following test completion, all materials were collected
and scored.
Table 2 contains a scope and sequence of the remain-
ing lessons and CBM probes. The Preparing for the
ACT curriculum was not designed to serve as an in-
depth review of content-related skills; instead, the cur-
riculum focused on developing students’ familiarity
with test form and test questions. As students evaluated
their own progress through the use of monthly CBM
probes, they were encouraged to seek further assistance
to further develop content-related skills from published
ACT study guides and various ACT Web sites to maxi-
mize their scores. The program closed with a final full
2 hour, 55 min postintervention practice ACT assess-
ment (obtained from the official ACT Web site
reprinted with permission [ACT, 2004a]), which was
administered 2 1/2 weeks prior to the administration of
the actual ACT.

Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015

8 Remedial and Special Education
Intervention Implementation
Following consenting and training procedures, 10
11th-grade teachers taught the ACT curriculum during a
30 min period each Tuesday. Fidelity data were collected
throughout the program from teacher and RA perspec-
tives (details to follow). Social validity data were
obtained at year end.
Measures
Student performance was monitored using the following
measures, collected as part of regular school practices.
Data were collected and entered by an RA. A second RA
assessed reliability of data entry for 25% of the data. The
few identified errors were corrected.
Grade point average (GPA). During the 2004 to 2005
academic year, the district GPA scale was based on a
4-point scale with the grades ranging from A+ (4.0),
B (3.0), C (2.0), D– (1.0), and F (0.00, including pluses
and minuses). During the 2005 to 2006 academic year, a
C– and D– were eliminated from the grading scale,
slightly changing the grade point scale to include a D as
a 1.0 grade point. Mean reliability of entry for quarterly
GPA scores was 100% for the 2004 to 2005 and 2005 to
2006 academic years. Quarterly grade point averages
were averaged into an annual GPA for students’ 10th- and
11th-grade years. Tenth-grade GPAs were used as predictor
variables in models to predict (a) postintervention prac-
tice ACT scores and (b) actual ACT scores. Comparisons
were not made across years because of changes in the
grading scale.
Office disciplinary referrals (ODR). The ODR data
were collected monthly at the student level during students’
10th and 11th grades. Annual ODR rates were computed
by dividing the total number of referrals by the number
of instructional days during each year. Mean reliabilities
for entry of ODR data were as follows: 2004 to 2005 aca-
demic year equals 99.8% and 2005 to 2006 academic
year equals 99.07%. The 10th-grade ODRs were used as
predictor variables in models to predict (a) postinterven-
tion practice ACT scores and (b) actual ACT scores.
ACT actual test. The ACT is a curriculum-based test
consisting of 215 multiple-choice questions. The test is
divided into four subject area tests: English (AES), math
(AMS), reading (ARS), and science (ASS). Actual testing
time is 2 hours and 55 min. The English test measures a
student’s rhetorical (style, organization, and strategy) and
mechanical (sentence structure, basic grammar, and
punctuation) skills. This 45-min test contains five prose
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
passages and consists of 75 multiple-choice questions.
The math test, which is administered in 60 min, contains
60 multiple-choice questions covering math skills
from six areas of study: pre-algebra, elementary algebra,
intermediate algebra, coordinate geometry, plane geom-
etry, and trigonometry. The reading test consists of 40
multiple-choice questions based on four reading pas-
sages (prose, social studies, natural sciences, and human-
ities) to be answered within a 35-min limit. This test
measures comprehension skills such as making infer-
ences, developing generalizations, and analyzing author’s
voice. Finally, the 35-min science test consists of 40
multiple-choice questions designed to measure a
student’s ability to solve scientific problems using rea-
soning skills. This test contains seven passages covering
the following content areas: biology, Earth and space
sciences, chemistry, and physics (ACT, 2004a). Students
receive scores for each of the subject area tests (ranging
from 0–36) as well as a composite score that is the arith-
metic average of the English, mathematics, reading, and
science scale scores (AACT; ranging from 0–36; ACT,
2004a). Higher scores indicate higher levels of mastery.
Most colleges in the United States require either the ACT
or the SAT for admission. The ACT Assessment is now
accepted by virtually all colleges and universities in the
United States. Although the ACT now contains a writing
component, we did not examine writing outcomes as the
intervention curriculum did not address the writing
subject area test.
ACT practice test. Participants took a sample ACT
test from Preparing for the ACT assessment: Test prepa-
ration, reprinted with permission from the official ACT
website (ACT, 2004a). This sample test is a representa-
tion of the authentic ACT assessment consisting of four
subject areas (English, PES; Math, PMS; Reading, PRS;
and Science, PSS) accumulating to 215 multiple-choice
questions. Participants took the full practice ACT (pre-
intervention) 1 week prior to the first lesson (PACT-T1)
and took the full practice ACT (postintervention) 1 week
after the last lesson (PACT-T2). Participants had 2 hours
55 min to complete the pre- and posttests as they would
when taking the actual ACT assessment adhering to the
subject area test time limits specified above. The ACT
practice tests were scored and entered with high reliability
percentages, specifically 99% for the preintervention test
and 100% for the posttest.
Treatment integrity. Treatment integrity was assessed
from two perspectives (teacher and RA) using compo-
nent checklists on which the rater indicated the presence
or absence of 12 items constituting the core intervention

components. Teachers completed a treatment integrity
form after each lesson. Teachers were observed between
10 (37%) and 13 (48%) lessons.
Social validity. Social validity was assessed postinter-
vention by two raters, students, and teachers. Each teacher
rater used a modified version of the Intervention Rating
Profile (IRP-15; Martens, Witt, Elliott, & Darveaux,
1985), whereas students used a modified version of the
Children’s Intervention Rating Profile (CIRP; Martens
et al., 1985) during their final lesson. The wording of each
item was slightly modified to reflect the intervention con-
tent. The IRP-15 was a 15-item survey containing state-
ments to be rated on a 6-point Likert-type scale, ranging
from 1 (strongly disagree) to 6 (strongly agree). Items
included statements related to the perceived ability to meet
the purpose of the intervention, the appropriateness of the
intervention with the student population, and the perceived
acceptability of the curriculum. Total scores ranged from
15 to 90, with higher scores indicating higher rates of
acceptability. The CIRP contained 7 items rated using a
6-point Likert-type scale (1 = I do not agree, 6 = I agree).
Total scores range from 7 to 42, with higher scores sug-
gesting higher acceptability. Both social validity ratings
were entered with 100% reliability. Alpha coefficients
were computed for the IRP-15 and CIRP for the current
sample, with respective values of .95 and .66.
Experimental Designs
Two experimental designs, a descriptive and a quasi-
experimental, were used to address the research objectives.
First, descriptive procedures were employed to examine
the extent to which characteristics of students who par-
ticipated in the program (e.g., disciplinary contacts and
GPA) predicted performance on (a) the ACT practice test
and (b) the actual ACT test. Second, a quasiexperimental
design was used in which actual ACT test performance
for 11th-grade students who did (2005–2006 academic
year) and did not participate (students, 2004–2005 acad-
emic year) was compared using a two-group, posttest only
design.
Results
Treatment Integrity
Statistical analysis. Treatment integrity was assessed
from the RA and teacher perspectives using the procedures
described above. Average session integrity scores were
computed for each class as well as the overall percentage
of implementation for the ACT program. Averages for
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
Lane et al. / College Entrance Exams 9
each rater were computed by summing the daily session
integrity percentage scores (0%–100%) for each lesson
and dividing by the quantity by the number of lessons
taught (teacher scores) or observed (RA scores). An effect
size was computed using the pooled standard deviation
in the denominator to determine the magnitude of the
differences in implementation between RA and teacher
perspectives (Busk & Serlin, 1992).
Findings. Average session integrity scores from the
RA perspective ranged from 62.33 to 95.83 (M = 82.96,
SD = 11.91). Teacher treatment integrity ratings were
moderately higher (M = 89.44, SD = 9.30; range, 69.90–
98.45; effect size = 0.61).
Factors Predicting Practice ACT
Postintervention Test Results
Statistical analysis. Results were analyzed using bivari-
ate correlation and multiple regression procedures. Multiple
regression procedures were used to examine the extent to
which behavioral performance (as measured by ODR) and
academic performance (as measured by GPA) predicted
five outcome variables: practice English (PES-T2), practice
math (PMS-T2), practice reading (PRS-T2), practice
science (PSS-T2), and practice ACT total scores (PACTS-
T2). F values were inspected to determine the significance
of the overall model. Univariate analyses were conducted if
the model was significant to ascertain the unique contribu-
tion of each variable constituting the model. T tests were
conducted to determine the individual contribution of each
predictor variable, controlling for the remaining variables.
Standardized multiple regression coefficients (beta weights)
and unique indices were inspected to ascertain the relative
contribution of each predictor variable in explaining the
criterion variable. Specifically, the unique index of a pre-
dictor variable is the percentage of variance in the criterion
variable accounted for by the given variable over and above
the variance explained by the remaining variables in
the model. Bivariate correlations explored overall relation-
ships between variables, whereas semipartial correlations
determined the relation between the predictor and criterion
variable, controlling for remaining variables (see Tables 3
and 4).
Y = β0 + βODR + βGPA + ε
This model was based on the hypothesis that students
with higher levels of problem behavior, as measured by
the rate of ODRs earned during the previous academic
year, and lower academic performance, as measured by
10 Remedial and Special Education

Table 3
Study 1: Correlation Matrix All Variables
Variable 1 2 3 4 5 6 7 8 9 10

1. ODR-10 1.00
2. ODR-11 0.31*** 1.00
3. GPA-10 0.39**** –0.23* 1.00
4. GPA-11 0.31*** –0.67**** 0.40**** 1.00
5. PES-T1 0.23* –0.36**** 0.43**** 0.59**** 1.00
6. PES-T2 –0.16 –0.40**** 0.26** 0.61**** 0.79**** 1.00
7. PMS-T1 –0.14 –0.36**** 0.45**** 0.62**** 0.78**** 0.71**** 1.00
8. PMS-T2 –0.18 –0.39**** 0.36**** 0.64**** 0.69**** 0.76**** 0.83**** 1.00
9. PRS-T1 –0.25** –0.27** 0.36**** 0.56**** 0.79**** 0.70**** 0.70**** 0.60**** 1.00
10. PRS-T2 –0.21** –0.42** 0.24** 0.60**** 0.70**** 0.84**** 0.65**** 0.72**** 0.74**** 1.00
11. PSS-T1 –0.25** –0.46**** 0.44**** 0.63**** 0.80**** 0.72**** 0.81**** 0.75**** 0.76**** 0.69****
12. PSS-T2 –0.13 –0.31*** 0.32*** 0.56**** 0.62**** 0.72**** 0.71**** 0.80**** 0.70**** 0.76****
13. PACT-T1 –0.24** –0.37**** 0.46**** 0.62**** 0.94**** 0.78**** 0.88**** 0.76**** 0.89**** 0.74****
14. PACT-T2 –0.21* –0.46**** 0.34**** 0.66**** 0.77**** 0.89**** 0.77**** 0.86**** 0.72**** 0.94****
15. AES –0.28** –0.39**** 0.42**** 0.66**** 0.85**** 0.82**** 0.77**** 0.66**** 0.71**** 0.74****
16. AMS –0.11 –0.38*** 0.43**** 0.62**** 0.77**** 0.67**** 0.91**** 0.83**** 0.63**** 0.62****
17. ARS –0.25* –0.33** 0.35*** 0.64**** 0.81**** 0.78**** 0.70**** 0.60**** 0.82**** 0.80****
18. ASS –0.17 –0.31** 0.39*** 0.59**** 0.80**** 0.74**** 0.78**** 0.70**** 0.72**** 0.71****
19. AACT –0.23* –0.38*** 0.42*** 0.68**** 0.88**** 0.83**** 0.84**** 0.75**** 0.79**** 0.79****

Note: N = X; ODR-10 = office discipline referrals from the 10th grade; GPA-10 = grade point average from the 10th grade; PES-T1 = prein-
tervention practice English subtest; PES-T2 = postintervention practice English subtest; PMS-T1 = preintervention practice math subtest; PMS-
T2 = postintervention practice math subtest; PRS-T1 = preintervention practice reading subtest; PRS-T2 = postintervention practice reading subtest;
PSS-T1 = preintervention practice science subtest; PSS-T2 = postintervention practice science subtest; PACT-T1 = preintervention practice ACT
total score [respectively]; PACT-T2 = postintervention practice ACT total score; AES = actual English subtest; AMS = actual math subtest;
ARS = actual reading subtest; ASS = actual science subtest; AACT = actual ACT total score.
*p < .05. **p < .01. ***p < .001. ****p < .0001.

Table 4
Mean Score Comparisons of Practice ACT Scores
Preintervention Postintervention Difference Scores Effect Size
(Time 1) (Time 2) (Time 2 – Time 1) (Time 1 – Time 2)
Variable M (SD) M (SD) M (SD) M (SD)

English 16.98 (5.98) 18.42 (6.84) 1.44 (4.28) 0.22

Math 18.47 (4.48) 19.38 (5.30) 0.91 (2.93) 0.19
Reading 18.17 (6.12) 18.52 (7.72) 0.36 (5.22) 0.05
Science 17.22 (5.13) 18.19 (5.91) 0.98 (4.11) 0.18
Total 17.87 (5.03) 18.64 (6.06) 0.77 (3.52) 0.14

GPA from the previous academic year, would be less
responsive.
Results. In predicting PES-T2, the two variable model
accounted for 7% of the variance in students’ postinter-
vention scores, R2 = 0.07, F(2, 123) = 4.80, p = .0099.
Inspection of semipartial correlations indicated that one
variable, GPA-10 (t = 2.53, p = 0.0128), was a significant
predictor of PES-T2 accounting for 5% of unique vari-
ance after controlling for the other variable (ODR-10) in
the model. There was a significant, positive correlation
between GPA-10 and PES-T2 indicating that students
with higher GPAs in 10th grade had higher postinter-
vention practice English ACT scores (β = .24; r = .26;
p = .0029).
In predicting PMS-T2, the two variable model
accounted for 13% of the variance in students’ postinter-
vention scores, R2 = 0.13, F(2, 123) = 9.17, p = .0002.
Inspection of semipartial correlations indicated that one
variable, GPA-10 (t = 3.73, p = .0003), was a significant
predictor of PMS-T2 accounting for 10% of unique vari-
ance after controlling for the other variable (ODR-10)
in the model. There was a positive correlation between
GPA-10 and PMS-T2 indicating that students with higher

Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015


GPAs in 10th grade had higher postintervention practice
math ACT scores (β = .34; r = .36; p < .0001).
In predicting PRS-T2, the two variable model
accounted for 8% of the variance in students’ postinter-
vention scores, R2 = 0.08, F(2, 123) = 4.98, p = 0.008.
Inspection of semipartial correlations indicated that one
variable, GPA-10 (t = 2.00, p = 0.048), was a significant
predictor of PRS-T2 accounting for 3% of unique variance
after controlling for the other variable (ODR-10) in the
model. There was a positive correlation between GPA-10
and PRS-T2 indicating that students with higher GPAs in
10th grade had higher postintervention practice reading
ACT scores (β = .19; r = .24; p = .006).
In predicting PSS-T2, the two variable model accounted
for 10% of the variance in students’ postintervention
scores, R2 = 0.10, F(2, 123) = 6.82, p = .0016. Inspection
of semipartial correlations indicated that one variable,
GPA-10 (t = 3.36, p = 0.001), was a significant predictor
of PSS-T2 accounting for 8% of unique variance after
controlling for the other variable (ODR-10) in the model.
There was a positive correlation between GPA-10 and
PSS-T2, indicating that students with higher GPAs in
10th grade had higher postintervention practice science
ACT scores (β = .31; r = .32; p = .0003).
In predicting PACT-T2, the two variable model
accounted for 13% of the variance in students’ postinter-
vention scores, R2 = 0.13, F(2, 123) = 8.79, p = .0003.
Inspection of semipartial correlations indicated that one
variable, GPA-10 (t = 3.36, p = .001), was a significant
predictor of PACT-T2 accounting for 8% of unique vari-
ance after controlling for the other variable (ODR-10)
in the model. There was a positive correlation between
GPA-10 and PACT-T2 indicating that students with higher
GPAs in 10th grade had higher postintervention practice
total ACT scores (β = .31; r = .34; p < .0001).
As originally hypothesized, for each subject area test
as well as the overall ACT practice test scores, academic
performance in the previous academic year was significant
in predicting postintervention practice scores. Yet con-
trary to the original hypothesis, behavioral performance
from the previous academic year was not significantly
associated with practice test scores. Although the corre-
lation between ODRs and postintervention scores were
negative, the amount of unique variance associated with
ODRs was not significant in the model (see Table 5).
Factors Predicting Actual ACT Results
Statistical analysis. Results were analyzed using bivari-
ate correlation and multiple regression procedures as
described above. In this model, postintervention practice
scores were included in addition to ODR and GPA scores.
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
Lane et al. / College Entrance Exams 11
We hypothesized that students’ performance on the prac-
tice test would be highly predictive of performance on
the actual ACT test, with a positive correlation between
the two tests. In addition, we hypothesized that students
with higher GPAs and lower rates of ODR would earn
higher actual ACT tests.
Y = β0 + βPost-intervention score + βODR + βGPA + ε
Results. In predicting AES, the three variable model
accounted for 71% of the variance in students’ ACT
English test scores, R2 = 0.71, F(3, 86) = 71.16, p < .0001.
Inspection of semipartial correlations indicated that all
three variables were significant predictors of AES:
PES-T2 (t = 12.47, p < .0001), ODR-10 (t = –2.20, p = .03),
and GPA-10 (t = 2.17, p = .03). Respective contributions
to unique variance are as follows: PES-T1 = 52%; ODR-
10 = 2%, and GPA-10 = 2%. There were positive corre-
lations between PES-T2 and AES scores (β = 0.76; r = 0.82;
p < .0001) as well as between GPA-10 and AES scores
(β = 0.14; r = 0.43; p < .0001) indicating that students
with higher postintervention practice English scores and
with higher GPAs during 10th grade had higher scores
on the English test. Also, there was a negative correlation
between ODR-10 and AES scores (β = –0.14; r = –0.28;
p = .008), suggesting that students with higher rates of
office referrals during 10th grade had lower English test
scores.
In predicting AMS, the three variable model accounted
for 69% of the variance in students’ postintervention
scores, R2 = 0.69, F(3, 86) = 65.06, p < .0001. Inspection
of semipartial correlations indicated that one variable,
PMS-T2 (t = 11.98, p < .0001), was a significant predictor
of AMS accounting for 51% of unique variance after con-
trolling for the other variables (ODR-10 and GPA-10) in
the model. The GPA-10 approached significance, t = 1.70,
p = .09. There was a positive correlation between PMS-T2
and AMS indicating that students with higher practice
postintervention math scores had higher math ACT scores
(β = 0.78; r = 0.83; p < .0001).
In predicting ARS, the three variable model accounted
for 66% of the variance in students’ reading scores, R2 =
0.66, F(3, 86) = 54.99, p < .0001. Inspection of semipartial
correlations indicated that one variable, PRS-T2 (t = 11.33,
p < .0001), was a significant predictor of ARS accounting
for 51% of unique variance after controlling for the other
variables (ODR-10 and GPA-10) in the model. There was
a positive correlation between PRS-T2 and ARS indicating
that students with higher post-intervention practice reading
ACT scores had higher actual reading scores on the ACT
(β = 0.76; r = 0.80; p < .0001).
12 Remedial and Special Education

Table 5
Results of Multiple Regression Analyses on Practice ACT scores
PES-T2 PMS-T2 PRS-T2 PSS-T2 PACT-T2

Beta Unique Beta Unique Beta Unique Beta Unique Beta Unique
Predictor Weights Indices Weights Indices Weights Indices Weights Indices Weights Indices
Variables Value (ta) Value (Fb) Value (ta) Value (Fb) Value (ta) Value (Fb) Value (ta) Value (Fb) Value (ta) Value (Fb)

ODR-10 –0.06 0.003 –0.425 0.002 –0.14 0.016 –0.008 0 –0.09 0.007
(–0.66) (0.44) (–0.46) (0.21) (–1.47) (2.03) (–0.09) (0.00) (–0.99) (0.98)
GPA-10 0.24 0.048 0.341 0.099 0.19 0.03 0.313 0.083 0.31 0.0803
(2.53*) (6.39*) (3.73***) (14.09***) (2.00*) (3.75) (3.36**) (11.80***) (3.36***) (11.29**)

Note: Standardized beta weights and unique indices shown reflect all variables in each model. ODR-10 = office discipline referrals from the
10th grade; GPA-10 = grade point average from the 10th grade; PES-T2 = postintervention practice English subtest; PMS-T2 = postintervention
practice math subtest; PRS-T2 = postintervention practice reading subtest; PSS-T2 = postintervention practice science subtest; PACT-T2 =
postintervention practice ACT total score.
a. For t tests that tested the significance of the beta weights df = 123.
b. For F tests that tested the significance of the uniqueness indices df = 1, 123.
*p < .05. **p < .01. ***p < .001. ****p < .0001.

In predicting ASS, the three variable model accounted
for 54% of the variance in students’ postintervention
scores, R2 = 0.54, F(3, 86) = 33.41, p < .0001. Inspection
of semipartial correlations indicated that one variable,
PSS (t = 8.49, p < .0001), was a significant predictor of
ASS accounting for 39% of unique variance after control-
ling for the other variables (ODR-10 and GPA-10) in the
model. There was a positive correlation between PSS-T2
and ASS indicating that students with higher postinter-
vention practice science ACT scores and higher actual
science ACT scores (β = 0.67; r = 0.72; p < .0001).
In predicting AACT, the three variable model accounted
for 73% of the variance in students ACT total scores,
R2 = 0.73, F(3, 86) = 78.73, p < .0001. Inspection of semi-
partial correlations indicated that one variable, PACT-T2
(t = 13.25, p < .0001), was a significant predictor of
AACT-T2 accounting for 55% of unique variance after
controlling for the other variables (ODR-10 and GPA-10)
in the model. There was a positive correlation between
AACT and PACT-T2 indicating that students with higher
postintervention scores on the practice test had higher
actual ACT scores (β = 0.81; r = 0.85; p < .0001).
As hypothesized, for each subject area test as well as
the overall ACT practice test scores, students’ postinter-
vention scores were significant in predicting actual ACT
scores. However, only in the case of the English test were
GPA-10 and ODR-10 predictive of actual English ACT
scores. As expected, there was a positive relationship
between GPA during 10th grade and ACT scores.
Furthermore, there was an inverse relationship between
ODRs and AET scores, with students who had higher
rates of ODRs had lower English scores on the ACT.
GPA-10 and ODR-10 were not significant predictor vari-
ables in the remaining models (see Table 6).
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
Comparison of ACT Scores: Students Who
Did and Did Not Receive the Intervention
Statistical analysis. Descriptive statistics (frequency
tables, mean score comparisons, and measures of disper-
sion) were examined from the 2004 to 2005 (noninterven-
tion year) and 2005 to 2006 (intervention year) school
years to determine the (a) potential differences between the
academic achievement of the 11th-grade classes to deter-
mine if the two groups were comparable, (b) percentage
of enrolled 11th-grade students who took the actual ACT,
(c) mean scores on each subject area test and the overall
ACT test and the extent to which these mean scores differed
from annual state mean scores, and (d) the percentage of
students who scored at or above a 22—the district target
and the minimum score necessary to obtain state monies
from Tennessee’s Hope Scholarship fund. Effect sizes
were computed using the pooled standard deviation in
the denominator to determine the magnitude of changes in
mean scores between years. Effect sizes were interpreted
according to recommendations offered by Bloom, Hill,
Black, and Lipsey (2006). Rather than relying solely on
fixed numerical interpretations (small = 0.15 σ, medium =
0.45 σ, large = 0.90 σ), we interpreted effect sizes by com-
paring computed effect sizes with (a) effect size distribu-
tions from comparable studies, (b) student attainment of
performance criterion without intervention, and (c) norma-
tive expectations for change.
Findings. Results of an independent t test revealed
nonsignificant differences in GPA indicating that both
11-grade classes had comparable GPAs, with respective
mean scores of 2.83 (SD = 0.85, for 2004–2005 class)
and 2.87 (SD = 0.87, for the 2005–2006 class).
 Lane et al. / College Entrance Exams 13

Table 6
Results of Multiple Regression Analyses on Actual ACT scores
AES AMS ARS ASS AACT

Beta Unique Beta Unique Beta Unique Beta Unique Beta Unique
Weights Indices Weights Indices Weights Indices Weights Indices Weights Indices
Variable Value (ta) Value (Fb) Value (ta) Value (Fb) Value (ta) Value (Fb) Value (ta) Value (Fb) Value (ta) Value (Fb)

PIPS 0.76 0.52 0.78 0.51 0.76 0.51 0.67 0.39 0.81 0.55
(12.47****) (173.17****) (11.98****) (127.55****) (11.33****) (127.93****) (8.49****) (77.42****) (13.25****) (181.73****)
ODR-10 –0.14 0.02 0.01 0.0001 –0.05 0.00 –0.07 0.003 –0.02 0.0005
(–2.20*) (0.37*) (0.21) (0.25) (–0.71) (0.00) (–0.78) (0.64) (–0.38) (0.167)
GPA-10 0.14 0.02 0.12 0.01 0.09 0.006 0.13 0.013 0.08 0.005
(2.17*) (5.23*) (1.70) (2.55) (1.26) (1.58) (1.54) (2.54) (1.24) (1.60)

Note: Standardized beta weights and unique indices shown reflect all variables in each model. PIPS = postintervention practice score for each
subtest predicted; ODR-10 = office discipline referrals from the 10th grade; GPA-10 = grade point average from the 10th grade; AES = actual
English subtest; AMS = actual math subtest; ARS = actual reading subtest; ASS = actual science subtest; AACT = actual ACT total score.
a. For t tests that tested the significance of the beta weights df = 86.
b. For F tests that tested the significance of the uniqueness indices df = 1, 86.
*p < .05. **p < .01. ***p < .001. ****p < .0001.

Furthermore, chi square analyses indicated that both
11th-grade classes were similar with respect to student
ethnicity and special education participation.
Results indicated that a similar percentage of enrolled
students took the ACT during the 2004 to 2005 academic
year and 2005 to 2006 academic year when the secondary
intervention was implemented. During 2004 to 2005, 166
students were enrolled in 11th grade, 90 of whom took the
actual ACT exam (54.22%, n = 90). During the interven-
tion year, 188 students were enrolled in the 11th grade, of
which 99 took the actual ACT exam (52.66%, n = 99).
Nine students were excluded from the analysis as they
either (a) were not present during the previous school year
or (b) had a patterned response on the practice ACT. Thus,
for subsequent analyses, 90 students who completed the
ACT during the 2004 to 2005 school year were compared
to the 90 students who completed the intervention and
took the ACT during the 2004 to 2005 school year.
Mean score comparisons indicated slight increases in
mean scores between the 2 years, with the greatest
improvement on the English test (ES = 0.09) and least
improvement on the science test (ES = 0.02). Overall
ACT scores showed low increase between the 2004 to
2005 (M = 19.66, SD = 4.23) and 2005 to 2006 (M = 20.65,
SD = 5.37) school years (ES = 0.06). Although the mag-
nitude of the increases may appear small, given the age and
grade level of the students, one would not expect high
magnitude improvements (Bloom et al., 2006).
Despite the minimal increases in mean scores, the per-
centage of students who attained the district target objec-
tive (total score = 22) increased by 10%. Furthermore,
whereas school means fell below the state mean scores
on all subject area tests as well as the overall score,
school means during the intervention year (a) were equiv-
alent to the state mean scores on the science test and total
score means and (b) exceeded state mean scores on the
English and math tests. School means scores fell below
the state means on only the reading test (see Table 7).
Social Validity
Statistical analysis. Social validity data were analyzed
using descriptive procedures such as inspection of means
and standard deviations of item level and total scores.
Findings. Students rated the intervention slightly
favorably with an overall mean of 25.43 (SD = 6.33) and
individual scores varying greatly (11 to 42). Inspection
of individual items revealed that students felt the inter-
vention would not negatively affect their reputation with
their friends at school (M = 4.59, SD = 1.66) and the
lessons were not too difficult (M = 4.56, SD = 1.48).
Narrative comments also suggested that students viewed
the program as beneficial (e.g., “being able to see what
the format is”). However, some students viewed the inter-
vention as less than helpful (e.g., “make it more hands on
and less talking for the teacher”).
Teachers rated the intervention overall positively with
an average rating of 54.50 (SD = 15.96). There was
substantial variability amongst the teachers, with total
scores ranging from 29 to 76. Comments also suggested
that teachers found this intervention as helpful, “It was
very beneficial for the students to see and discuss the
various types of problems and questions they will see on

Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015

14 Remedial and Special Education

Table 7 Gregg, et al., 2002; Walker et al., 2005). One similar study
Actual ACT Scores Over Time has been conducted at the elementary level (Robertson &
Preintervention Intervention Effect Lane, 2007), and none have been attempted at the high
Subtest Year (N = 90) Year (N = 90) Size school level, when learning and behavioral concerns are
particularly resistant to intervention efforts (Kazdin, 1987).
English
The absence of targeted interventions in high schools is
M (SD) 19.66 (5.97) 20.91 (6.50) 0.09
Range 7–33 7–35 particularly concerning given that high school is the final
Scores: 22+ 29 (32.22) 40 (44.44) point of public education before students either enter the
State 20.6 20.8 work force or pursue collegial opportunities.
Math Performance on college entrance exams such as the
M (SD) 19.59 (4.29) 20.13 (4.86) 0.04 ACT serves as an important predictor of which students
Range 13–30 12–36 will be admitted to colleges and universities (Ehrehnhaft
Scores: 22+ 29 (32.22) 32 (35.56) et al., 2001; Fuller & Wehman, 2003). Yet a strong body
State 19.7 19.9 of literature indicates that secondary students in the
Reading United States of America lack strong study skills (Jones,
M (SD) 19.72 (5.52) 20.76 (6.54) 0.08 Slate, Bell, et al., 1991; Jones, Slate, Blake, et al., 1995;
Range 9–32 8–36 Kovach et al., 2001; Slate et al., 1993) and many schools
Scores: 22+ 34 (37.78) 36 (40.00)
State 20.8 21.1
do not offer explicit instruction to assist students in prepar-
ing for college entrance exams.
Science This preliminary study sought to address this void of
M (SD) 20.10 (3.59) 20.34 (5.29) 0.02
Range 12–29 9–36
secondary interventions at the high school level and
Scores: 22+ 27 (30.00) 29 (32.22) address the limited study skills characteristic of many
State 20.2 20.3 high school students by examining the efficacy of a
Total
targeted, secondary intervention to assist in preparing
M (SD) 19.96 (4.23) 20.65 (5.37) 0.06 11th-grade students to take the ACT college entrance
Range 12–30 11–35 exam. Furthermore, we extended this line of inquiry by
Scores: 22+ 29 (32.22) 38 (42.22) examining the degree to which students with various aca-
State 20.5 20.7 demic and behavioral performance levels responded to
Note: During the 2004 to 2005 administration, 90 students took the the support provided.
ACT; however, 1 student did not take the science subtest and, conse-
quently, did not have a total ACT score. Student Characteristics Associated
With Performance on the ACT
the ACT.” However, some teachers provided criticism of Based on the notion that students with (a) behavior
the intervention that primarily focused on being asked to problems exhibit behaviors that prompt office referrals
prepare students for all aspects of the exam: “My knowl- and impede instruction and (b) high GPAs are apt to have
edge was limited to Reading/English sections; therefore strong academic and study skills, we hypothesize that
I typically read straight from the packet which caused my students with lower rates of ODRs earned during the pre-
students to zone out.” Overall, the teachers and students vious academic year and high GPAs from the previous
rated the intervention beneficial. academic year would be more responsive to the interven-
tion as measured by performance on the practice ACT
Discussion test. Results partially confirmed our hypotheses.
Although the overall models were significant, only
As RTI and PBS models are implemented across the students’ academic performance in the previous year
country, researchers and practitioners are faced with the (GPA-10) was a significant predictor of postintervention
formidable task of identifying and supporting students English, math, reading, science, and overall test scores,
using targeted interventions. To date the only published after controlling for variance accounted for by ODRs
studies of secondary prevention efforts implemented earned during 10th grade. Behavioral performance during
within the context of three-tiered PBS models have been the prior year (ODR-10) was not a significant predictor of
conducted with elementary age students who were non- postintervention practice test scores after accounting for
responsive to primary prevention efforts (Lane, Wehby, GPA; although the relationships between ODRs and
Menzies, Doukas, et al., 2003; Lane, Wehby, Menzies, postintervention practice test scores were all negative.

Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015


A similar pattern was observed when predicting
student performance on the actual ACT. As originally
hypothesized, students’ postintervention scores were highly
significant in predicting subject area test and overall ACT
scores, accounting for up to 52% unique variance in the
models (English subject area test). Namely, student per-
formance on the postintervention practice test was indica-
tive of performance on the actual ACT test. However,
students’ academic (GPA-10) and behavioral (ODR-10)
performance were only significant in predicting perfor-
mance on the English test. As expected, students with
higher GPAs and fewer ODRs performed better on the
English test. Yet neither of these variables explained a
significant amount of unique variance over and above the
postintervention practice test scores.
In terms of practical implications, there are two key
points. First, given that postintervention practice test scores
are highly predictive of actual ACT test performance, it is
important that students develop the skill sets such as study
skills, content knowledge, and practice opportunities to
bolster practice test scores results. Second, given that
students’ academic performance during 10th grade is highly
predictive of postintervention practice test scores, inter-
vention efforts should ideally begin prior to 11th grade. If
students’ study skills could be developed earlier (i.e., in
10th or even 9th grade) resulting in improved academic
performance in 10th grade, it is likely that student perfor-
mance on both the practice and ultimately the actual ACT
test would improve as well.
Comparison of ACT Over Time
The next objective was to use a quasiexperimental
design to determine the relative, practical benefit of this
intervention by comparing actual ACT performance for
11th-grade students who did (2005–2006 academic year)
and did not (2004–2005 academic year) participate in the
intervention. We hypothesized that students who partici-
pated in the intervention program would perform better on
the actual ACT test relative to the previous years’ 11th-grade
students who did not participate in the program. Findings
support this expectation in three ways. First, mean score
comparisons across the 2 years revealed nominal improve-
ments as measured by effect sizes. Second, the percentage
of students who met the district objective of earning a total
score of 22 or greater increased by 10%. Third, school
mean scores met (science and total score) or exceeded
(English and math) state mean scores during the interven-
tion year, whereas during the year prior to intervention
school mean scores were below the state mean on all four
subject area tests as well as the total ACT score.
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
Lane et al. / College Entrance Exams 15
Results of this quasiexperimental study have impor-
tant practical implications for schools. This program was
feasible in the sense that it demanded few instructional
resources—limited teacher and student time—and no
additional time beyond the school day (Lane & Beebe-
Frankenberger, 2004). Thus, one could argue that the
cost–benefit ratio of implementing such a program is
positive in the sense that it enabled an additional 10% of
11th-grade students to meet the district objective, which
also made these students eligible for monies from the
Tennessee Hope Scholarship (www.collegepaystn.com/
mon_college/hope_scholar.htm). This scholarship offers
$3,800 for 4-year institutions and $1,900 for 2-year insti-
tutions with monies applicable to the cost of attendance.
This is an enormous incentive for students who do not
have the financial resources to attend college.
Two other indices of feasibility are the high levels of
treatment fidelity from teacher (89.44%) and research
assistant (82.96%) perspectives as well as moderate-to-
high social validity ratings from teachers and students. As
expected, teacher ratings were more favorable than
student ratings. Yet the intervention goals, procedures, and
outcomes were rated as moderately favorable by students
and more so by teachers. Gresham and colleagues have
argued that treatment integrity can actually serve as a
behavioral marker for social validity, if teachers imple-
ment the program as designed, then, by definition, they
accept it (Gresham & Lopez, 1996). Lane and Beebe-
Frankenberger (2004) contend that if an empirically
sound, feasible intervention is implemented with fidelity
and produces lasting, desirable changes in student behav-
ior, the intervention is apt to be viewed as socially valid. If
socially valid, the intervention is likely to be used in the
future, producing a circular set of events.
Although the magnitude of the differences between
years may seem low, students become increasingly less
responsive to intervention efforts over time (Bullis &
Walker, 1994). Therefore, the magnitude of treatment out-
comes is not expected to be consistent across the K-12
grade span. For example, evidence from national norm-
ing samplings for several state achievement tests suggest
decreasing effect sizes for annual reading and math
growth as students progress through their educational
experiences (Bloom et al., 2006). For example, reading
growth effect sizes were 1.59 for students transitioning
between kindergarten and 1st grade, 0.35 for students
transitioning between 5th and 6th grades, 0.21 for students
transitioning between 10th and 11th grades, and 0.03 for
students transitioning between 11th and 12th grades.
Effect sizes were slightly lower for math growth with
effect sizes of 1.13, 0.41, 0.15 and 0.00 for the above
16 Remedial and Special Education
named transition periods. Thus, the effect sizes produced
from this study are comparable to growth demonstrated
for normative samples of reading and math growth (Bloom
et al., 2006).
Limitations and Directions
for Future Research
Results of this study should be interpreted in light of
the following limitations and considerations pertaining to
experimental design and outcome measures. First, this is
a preliminary investigation, and additional replications
are necessary to establish the validity of these findings.
Specifically, because of the experimental designs
employed, the inclusion of one school, and the lack of
replication in alternative settings (e.g., urban schools),
results must be examined cautiously. Causal conclusions
cannot be drawn in the absence of a true experimental
design. To draw definitive conclusions regarding outcomes
associated with intervention outcomes, future research
involving more rigorous designs such as true experimen-
tal studies with random assignment of students to inter-
vention conditions (Gersten et al., 2005) or single case
studies with multiple baseline across students (Horner
et al., 2005) is necessary. Generalization of these findings
cannot be established until these findings are replicated in
subsequent studies.
Second, because the intervention was taking place as
part of regular school practices, only a limited number of
outcome measures were collected. Future investigations
could be enhanced by including additional measures to
determine if (a) students’ knowledge of study skills or
study skills habits (Jones & Slate, 1992) actually increased
and (b) these variables (knowledge and habits) actually
predicted performance on the actual ACT. In addition,
it would be interesting to see if other variables, such as
treatment integrity and teacher perceptions of social valid-
ity, mediated intervention outcomes (Lane & Beebe-
Frankenberger, 2004). Larger scale investigations with
larger sample sizes could use hierarchal linear modeling
to determine if such class level variables predicted changes
in student performance.
Third, permission was obtained for the administration
of only one version of the practice ACT test. Thus the pre-
and postintervention practice test was the same version;
alternate versions were not used at the two time points.
Although 8 months lapsed between test administrations,
limitations associated with using the same version should
be considered when examining outcomes of analyses pre-
dicting postintervention practice scores. Future studies
would be improved by obtaining permission to administer
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
alternate versions of the practice tests to determine how
the program influences students’ performance on the prac-
tice ACT. However, the primary objectives of this study
were to (a) determine the extent to which students’ behav-
ioral and academic characteristics (e.g., GPA and ODR)
predicted performance on the actual ACT and (b) compare
actual ACT performance for two cohorts of 11th-grade
students, one that did and one that did not participate in
the intervention. Using the same version of the practice
test would not impact either of these latter objectives.
Fourth, social validity data were collected anony-
mously to increase both the quantity and candor of student
and teacher responses. Future research could be enhanced
by examining student responsiveness in relation to social
validity to determine the extent to which social validity
predicted intervention outcomes as described above
(Lane & Beebe-Frankenberger, 2004). Also, subsequent
research could benefit from collecting social validity
prior to intervention onset to identify any potential pro-
cedural concerns. For example, some of the teachers
indicated that they felt uncomfortable teaching lessons
outside their specific area of expertise. Had this concern
been identified prior to intervention onset, intervention
delivery could have been restructured. One option would
have been to alter the delivery of instruction so that
students rotated through different homerooms so that
math teachers taught all math lessons, science teachers
taught all science lessons, and so on.
Fifth, neither teachers nor students participated in the
intervention development phase. Moving forward, partici-
pation of these stakeholders in designing both the con-
tent and instructional strategies and activities may lead to
higher levels of treatment fidelity, improved social validity
ratings, and ultimately even stronger student outcomes.
Conclusion
Despite the limitations mentioned above, this prelimi-
nary study addresses two voids in the literature by (a) illus-
trating one method of providing targeted, secondary
interventions at the high school level and (b) examining the
efficacy of a feasible approach to preparing 11th-grade
students for the ACT college entrance exam. Furthermore,
this study extends this line of inquiry by using multivariate
procedures to examine the degree to which students’ acad-
emic and behavioral performance predicts intervention
outcomes. We encourage other researchers and educators
to continue to develop this area of interest with an overall
call to develop the methodological rigor used to support
high school students with three-tiered models of support.

For those interested in improving student performance
on college entrance exams specifically, we also encour-
age a focus on a more comprehensive intervention pro-
gram. For example, research conducted by ACT (2005)
suggests that short-term test preparation interventions
may yield small, positive impacts. However, long-term
interventions that included taking the recommended core
college courses (ACT, 2004b) or even taking advanced
courses beyond the core requirements (e.g., additional
mathematics courses) produced greater increases in
composite ACT scores after controlling for students’
prior achievement level (ACT, 2005). Consequently,
future interventions would be wise to also examine the
role of students’ course completed as the predicted ACT
performance.
References
ACT. (2004a). ACT assessment. Test preparation. Retrieved June 11,
2004 from http://www.act.org/aap/testprep
ACT. (2004b). ACT high school profile report: HS graduating class
2004. Iowa City: Author.
ACT. (2005). Issues in college readiness: What kind of test prepara-
tion is best? Iowa City: Author.
Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C. (1983). Effects of
coaching programs on achievement test performance. Review of
Educational Research, 53, 571–585.
Bloom, H., Hill, C., Black, A. R., & Lipsey, M. (2006, June). Effect sizes
in educational research: What they are, what they mean, and why
they’re important. Paper presented at the Institute for Educational
Sciences 2006 Research Conference, Washington, DC.
Bullis, M., & Walker, H. M. (1994). Comprehensive school-based
systems for troubled youth. Eugene: University of Oregon, Center
on Human Development.
Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case
research. In T. Kratochwill & J. Levin (Eds.), Single case research
design and analysis (pp. 187–212). Hillsdale, NJ: Lawrence
Erlbaum.
Ehrehnhaft, G., Lehrman, R. L., Obrecht, F., & Mundsack, A. (2001).
Barron’s how to prepare for the ACT assessment. Hauppauge, NY:
Barron’s Educational Series.
Dreisbach, M., & Keogh, B. K. (1982). Testwiseness as a factor in
readiness test performance of young Mexican-American children.
Journal of Educational Psychology, 74, 224–229.
Fuchs, D., & Fuchs, L. (1994). Inclusive schools movement and the
radicalization of special education reform. Exceptional Children,
60, 294–309.
Fuller, W. E., & Wehman, P. (2003). College entrance exams for
students with disabilities: Accommodations and testing guidelines.
Journal of Vocational Rehabilitation, 18, 191–197.
Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., &
Innocenti, M. S. (2005). Quality indicators for group experimental
and quasi-experimental research in special education. Exceptional
Children, 71, 149–164.
Gresham, F. M. (2002). Responsiveness to intervention: An alternative
approach to learning disabilities. In R. Bradley, L. Danielson, &
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
Lane et al. / College Entrance Exams 17
D. Hallahan (Eds.), Identification of learning disabilities: Research
to practice. Mahwah, NJ: Lawrence Erlbaum.
Gresham, F. M., & Lopez, M. F. (1996). Social validation: A unifying
construct for school-based consultation research and practice.
School Psychology Quarterly, 11, 204–227.
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S. L., &
Wolery, M. (2005). The use of single-subject research to identify
evidence-based practices in special education. Exceptional
Children, 71, 165–179.
Jones, C. H., & Slate, J. R. (1992). Technical manual for the Study
Habits Inventory. Unpublished manuscript, Arkansas Sate
University.
Jones, C. H., Slate, J. R., Bell, S., & Saddler, C. (1991). Helping high
school students improve their academic skills: A necessary role
for teachers. High School Journal, 74, 198–202.
Jones, C. H., Slate, J. R., Blake, P. C., & Sloas, S. (1995). Relationship
of study skills, conceptions of intelligence, and grade level in sec-
ondary school students. High School Journal, 79, 25–32.
Jones, C. H., Slate, J. R., Mahan, K., Green, A., Marini, I., &
DeWater, B. (1993/1994). Study skills of college students as a
function of gender, age, class and grade point average. Louisiana
Education Research Journal, 19, 60–74.
Kazdin, A. E. (1987). Problem solving skills training and relationship
therapy in the treatment of antisocial child behavior. Journal of
Consulting and Clinical Psychology, 55, 76–85.
Kern, L. & Manz, P. (2004). A look at current validity issues of school-
wide behavior support. Behavioral Disorders, 30, 47–59.
Kovach, K., Fleming, D., & Wilgosh, L. (2001). The relationship
between study skills and conceptions of intelligence for high
school students. Korean Journal of Thinking and Problem Solving,
11, 39–49.
Lane, K. L. & Beebe-Frankenberger, M. (2004). School-based
interventions: The tools you need to succeed. Boston: Pierson
Education.
Lane, K. L., & Beebe-Frankenberger, M. (2007). An integrated
intervention model for school-based research. Manuscript in
preparation.
Lane, K. L., & Menzies, H. M. (2003). A school-wide intervention
with primary and secondary levels of support for elementary
students: Outcomes and considerations. Education and Treatment
of Children, 26, 431–451.
Lane, K. L., & Parks, R. J. (2007, March). Using school-wide data to
identify students for targeted interventions across the K-12 grade
span: Illustrations and recommendations. Paper presented at the
Fourth International Conference on Positive Behavior Support,
Boston, Massachusetts.
Lane, K. L., Robertson, E. J., & Graham-Bailey, M. A. L. (2006). An
examination of school-wide interventions with primary level
efforts conducted in secondary schools: Methodological
considerations. In T. E. Scruggs & M. A. Mastropieri (Eds.),
Applications of research methodology: Advances in learning and
behavioral disabilities (Vol. 19). Oxford, UK: Elsevier.
Lane, K. L., Umbreit, J., & Beebe-Frankenberger, M. (1999). A review
of functional assessment research with students with or at-risk for
emotional and behavioral disorders. Journal of Positive Behavioral
Interventions, 1, 101–111.
Lane, K. L., Wehby, J., Menzies, H. M., Doukas, G. L., Munton, S. M.,
& Gregg, R. M. (2003). Social skills instruction for students at
risk for antisocial behavior: The effects of small-group instruction.
Behavioral Disorders, 28, 229–248.
18 Remedial and Special Education
Lane, K. L., Wehby, J. H., Menzies, H. M., Gregg, R. M., Doukas, G. L.,
& Munton, S. M. (2002). Early literacy instruction for first-grade
students at-risk for antisocial behavior. Education and Treatment
of Children, 25, 438–458.
Lane, K. L., Wehby, J., Robertson, E. J., & Rogers, L. (2007). How
do different types of high school students respond to positive
behavior support programs? Characteristics and responsiveness of
teacher-identified students. Journal of Emotional and Behavioral
Disorders, 15, 3–20.
Luiselli, J. K., Putnam, R. F., & Sunderland, M. (2002). Longitudinal
evaluation of behavior support intervention in a public middle
school. Journal of Positive Behavior Intervention, 4, 182–188.
Martens, B. K., Witt, J. C., Elliott, S. N, & Darveaux, D. X. (1985).
Teacher judgments concerning the acceptability of school-based
interventions. Professional Psychology Research and Practice,
16, 191–198.
May, S., Ard, W., III, Todd, A. W., Horner, R. H., Glasgow, A., Sugai, G.,
et al. (2000). School-wide Information System. Eugene: University
of Oregon, Educational and Community Supports.
No Child Left Behind Act, 20 U.S.C. 70 § 6301 (2001).
Ritter, S., & Idol-Maestas, L. (1986). Teaching middle school students
to use a test-taking strategy. Journal of Educational Research,
79, 350–357.
Robertson, E. J., & Lane, K. L. (2007). Supporting middle school
students with academic and behavioral concerns within the con-
text of a three-tiered model of support: Findings of a secondary
intervention. Manuscript submitted for publication.
Sampson, G. E. (1985). Effects of training in test-taking skills on
achievement test performance: A quantitative synthesis. Journal
of Educational Research, 78, 261–266.
Scruggs, T. E., & Mastropieri, M. A. (1992). Teaching test-taking
skills. Brookline, MA: Brookline Books.
Slate, J., Jones, C., & Dawson, P. (1993). Academic skills of high
school students as a function of grade, gender, and academic track.
High School Journal, 76, 245–251.
Sprague, J., Walker, H., Golly, A., White, K., Myers, D. R., &
Shannon, T. (2001). Translating research into effective practice:
The effects of a universal staff and student intervention on indica-
tors of discipline and school safety. Education and Treatment of
Children, 24, 495–511.
Downloaded from rse.sagepub.com at SIMON FRASER LIBRARY on June 8, 2015
Sugai, G., & Horner, R. (2002). The evolution of discipline practices:
School-wide positive behavior supports. Child and Family
Behavior Therapy, 24, 23–50.
Walker, B., Cheney, D., Stage, S., & Blum, C. (2005). Schoolwide
screening and positive behavior supports: Identifying and support-
ing students at risk for school failure. Journal of Positive Behavior
Interventions, 7, 194–204.
Kathleen Lynne Lane is an assistant professor in the Department of
Special Education of Peabody College at Vanderbilt University. Her
research program primarily focuses on designing, implementing, and
evaluating multilevel, school-based interventions to prevent the
development of learning and behavior problems for students at-risk
for EBD and remediate the deleterious effects of existing problems
exhibited by students with EBD.
Jemma Robertson Kalberg graduated from the Special Education
Department of Peabody College at Vanderbilt University in 2006. Her
interests include schoolwide screeners to identify students with or
at-risk for emotional and behavioral disorders, positive behavior
support interventions, and secondary interventions supporting both
academic and behavioral deficits.
Emily Mofield graduated from the Department of Special Education
at Peabody College of Vanderbilt University in 2004. She is a doctoral
candidate in the Department of Teaching and Learning at Tennessee
State University and a special education teacher for Sumner County
Schools in Middle Tennessee.
Joseph H. Wehby is an associate professor in the Department of
Special Education at Peabody College of Vanderbilt University. His
research interests include teacher–student interactions as well as aca-
demic and behavioral interventions for student with emotional or
behavioral disorders.
Robin J. Parks is a special education teacher at the Hoover High
School Freshman Campus in Hoover, Alabama, and is a recent gradu-
ate from the Department of Special Education of Peabody College at
Vanderbilt University.