The Journal of Special Education

Assessing Students With 46(2) 117­–127

© Hammill Institute on Disabilities 2012
Reprints and permission: http://www.
Significant Cognitive Disabilities sagepub.com/journalsPermissions.nav
DOI: 10.1177/0022466910379156

on Academic Content http://journalofspecialeducation

.sagepub.com

Jessica Goldstein1 and Peter Behuniak1

Abstract
Federal and state policies require all students with significant cognitive disabilities to participate in state assessments and
be included in measures of adequate yearly progress. Although these alternate assessments of grade-level content based
on alternate achievement standards have been in place for several years, little is known about the knowledge and skills
of students with significant cognitive disabilities or about the influence of these assessments on student learning. This
study uses an empirically evaluated classification scheme of data from one cohort of students’ participation in a statewide
alternate assessment, Connecticut’s Skills Checklist, to validate claims from teacher focus group interviews. Data from both
components of the study suggest that nearly half of the teachers found the content on the Skills Checklist inaccessible for
their students. Implications of this finding are discussed.

Keywords
validity, assessment, accountability, significant cognitive disabilities

Federal policy requires that all students with significant
cognitive disabilities participate in state assessments and be
included in measures of adequate yearly progress (U.S.
Department of Education, 2003). To meet this requirement,
states developed alternate assessments for this population
based on alternate academic achievement standards (AA-
AAAS) that are aligned to grade-level academic content
standards. Though students with significant cognitive dis-
abilities represent only approximately 1% of the general
population (U.S. Department of Education, 2005), the
diversity of these students’ communication abilities and
learning challenges is quite complex. Developing standard-
ized assessments for this diverse group of students requires
a balance between the standardization required for large-
scale assessments and the flexibility needed for the class-
room teacher to successfully work with this population.
Although a body of literature on technical issues related to
the development of AA-AAAS is slowly emerging (Browder
et al., 2004; Gong & Marion, 2006; Perie, 2007), few stud-
ies exist to describe what is known about students with sig-
nificant cognitive disabilities and their participation in
large-scale assessment programs.
The struggle to balance flexibility for the students and
psychometric rigor is a national issue. The federal gov-
ernment set high standards for the technical quality of
these instruments, which states have struggled to meet. In
2006, 38 states did not meet peer review standards for
their AA-AAAS. By 2008, 15 states were still working on
modifying their assessments to meet federal standards (U.S.
Department of Education, 2009). In a national study of
alternate assessment practices, respondents from every state
reported that the student’s special education teacher admin-
istered or assembled the alternate assessments (Cameto et al.,
2009). Cameto et al. (2009) also reported that the student’s
classroom teacher was allowed to score the alternate assess-
ment in 27 states. These scoring procedures are heavily
dependent on disposition of the teacher (Quenemoen, 2008).
Task design and scoring place special education teachers at
the center of accountability decisions for their students and
have a significant influence on the psychometric quality of
the assessment results (Elliott & Roach, 2007).
As the measurement field grapples with improving the
technical quality of alternate assessments, an effort is
also under way to better understand the nature and needs
of the student population targeted by these assessments.
Results from one study of Learner Characteristics Inven-
tory (LCI; Kearns, Kleinert, Kleinert, & Towles-Reeves,
1
University of Connecticut, Storrs, CT, USA
Corresponding Author:
Jessica Goldstein, University of Connecticut, Neag School of Education,
249 Glenbrook Road, Storrs, CT 06269, USA
E-mail: jessica.goldstein@uconn.edu
118 The Journal of Special Education 46(2)
2006) data from more than 12,000 students from seven
states differentiated students with significant cognitive dis-
abilities according to language use (Kearns, Towles-Reeves,
Kleinert, Kleinert, & Thomas, 2009). Their study indicated
that a majority of the students in this population communi-
cated expressively and receptively at the symbolic level
(i.e., they communicate with a wide variety of intents and
follow directions independently). A second, much smaller
group was at the emergent symbolic level of expressive and
receptive communication, meaning that these students use
pictures, objects, signs, and gestures in addition to oral
speech and require additional cues to follow directions. The
final group was at the presymbolic level of communication.
These students’ communication efforts require interpreta-
tion from the listener or observer, and definition of their
receptive language skills is less clear. An earlier study of
LCI data from three states indicated that most students in
the study had functional reading and mathematics skills but
that those skills were strongly correlated with expressive
and receptive communication skills (Towles-Reeves, Kearns,
Kleinert, & Kleinert, 2009). Almond and Bechard (2005)
identified similar patterns of heterogeneity in their study of
165 students participating in a pilot alternate assessment
program. With regard to communication, almost 40% of the
students were able to communicate using 200 or more words,
but approximately 10% did not use words to communicate.
Research evidence also exists to demonstrate that students
with significant cognitive disabilities are capable of learn-
ing some components of reading (Browder, Wakeman,
Spooner, Ahlgrim-Delzell, & Algozzine, 2006) and mathe-
matics skills (Browder, Spooner, Ahlgrim-Delzell, Harris,
& Wakeman, 2008).
Although these studies provide an important step in
understanding the ways in which this population may access
academic content, there currently exists neither research
evidence nor a theoretical consensus regarding the appro-
priate expectations for academic content mastery by this
student population. Quenemoen (2008) offered,
We do not know as yet what will work the best in
teaching and in assessing students with significant
cognitive disabilities in the academic content. We are
seeing evidence of remarkable achievement, but this
group is so varied in characteristics and the field of
severe disabilities is still divided on what appropriate
outcomes we can and should expect. (p. 23)
Still, it is a federal requirement that all states must assess
these students’ academic content knowledge annually. It is
not surprising that this situation gives rise to questions reg­
arding the extent of students’ exposure to academic content
and the availability of appropriate communication tools.
Browder, Wakeman, and Flowers (2009) suggested, “What
is not yet known is the extent to which all students have
access to the curriculum, appropriate assistive technology,
and other means to demonstrate learning” (p. 333). Little
is known about these students and their learning environ-
ments and the impact of these issues on accountability
decisions.
This study uses teacher focus group interviews and an
empirically evaluated classification scheme of data from a
statewide alternate assessment to examine the accessibility of
the grade-level academic content for all students with signifi-
cant cognitive disabilities. The study was conducted in Con-
necticut with focus groups composed of special education
teachers responsible for implementing the state’s alternate
assessment. Unlike most states in which alternate assess-
ments have been administered only in the past few years,
Connecticut has been employing a system of alternate assess-
ment since 2000. At that time the statewide assessments, the
Connecticut Mastery Test (CMT) in Grades 3 to 8 and the
Connecticut Academic Performance Test (CAPT) in Grade
10, were expanded to include the CMT/CAPT Skills Check-
list (hereafter “the Skills Checklist”). The current version of
the Skills Checklist, implemented in 2006, employs teacher
ratings of student knowledge both for use in improving
instruction and for statewide accountability purposes. We
used existing data from the Connecticut AA-AAAS program
to examine teacher perceptions of student academic content
knowledge based on teacher ratings. The following sections
of the article provide an overview of the CMT/CAPT Skills
Checklist, a description of the research methodology, and the
results of the qualitative and quantitative analyses.
Instrumentation
The Second Generation CMT/CAPT Skills Checklist was
developed in response to 2003 federal guidelines requiring
(a) grade-level-specific alternate assessments that reflected
grade-level content standards, (b) the establishment of
alternate achievement standards in a valid, documented
standard setting procedure, and (c) the adoption of these
alternate achievement standards by the state board of edu-
cation (Behuniak & Amenta, 2009). When the Skills Check-
list was developed in 2005, it was envisioned that it would
transform the educational process for students with signifi-
cant cognitive disabilities. Specifically, it is stated in the
technical manual that teachers will use the instrument to
“inform instruction, monitor student growth and progress,
and document achievement” and that use of the Skills
Checklist will result in “(a) a change in the nature of ins­
truction for students with significant cognitive disabilities,
(b) an improvement in the quality of instruction for these
students, and (c) the greater inclusion of these students in
general education settings” (Connecticut State Department
of Education [CSDE], 2006, p. 9).
Goldstein and Behuniak
The Skills Checklist is a nonsecure, stand-alone working
document that teachers can use throughout the school year
to inform instruction and assess academic skills. All stu-
dents are assessed in reading and mathematics in Grades 3
through 8 and 10; science is assessed in Grades 5, 8, and 10.
The child’s primary special education teacher rates the stu-
dent’s performance on a 3-point scale (0 = does not demon-
strate skill, 1 = developing/support, and 2 = mastered/
independent). When rating an item, any mode of communi-
cation or responding that is typically utilized by the child is
acceptable. Each test item consists of three downward ext­
ensions that address the essence of the content standard and
expected performance statements but in a simplified form
that makes them more accessible for students with signifi-
cant cognitive disabilities. The CMT/CAPT Skills Checklist
is available online at http://www.csde.state.ct.us/public/cedar/
assessment/checklist/index.htm.
The Skills Checklist also includes a section called Access
Skills, which was designed to assess preacademic skills that
students without disabilities typically develop prior to school
entry. The Access Skills section covers receptive communi-
cation, expressive communication, social-interactive com-
munication, basic literacy, and spatial relationships. Scores
from the Access Skills portion of the Skills Checklist are not
used for accountability purposes.
Method
The purpose of this study was to examine teacher percep-
tions of academic content knowledge for students with sig-
nificant cognitive disabilities in the context of the Skills
Checklist. The following research question was of interest:
Do teacher ratings of student academic content knowledge
from the Skills Checklist data validate claims from teacher
focus group interviews that the grade-level academic con-
tent included in the Skills Checklist may not be accessible
for all students with significant cognitive disabilities?
Several methodologies were used to investigate this que­
stion. Focus groups interviews were designed based on the
results of a survey of special education teachers who had
administered the Skills Checklist in 2009. Quantitative analy-
ses of 2009 checklist data were conducted to validate themes
from the focus group data related to assessment administra-
tion and the manner in which teachers evaluate student
knowledge. These methodologies are described below.
Qualitative Method
Patton (2001) suggested that qualitative inquiry is most
appropriate for investigations in which the processes,
impacts, or both are largely unspecified or difficult to mea-
sure. This is particularly true of teachers’ implementation
of the CMT/CAPT Skills Checklist. Development of the
119
focus group interview protocol was based on prior survey
research on teacher use of the Skills Checklist to inform
instruction and monitor student progress. The final protocol
is included in the appendix. Note that the protocol addresses
teachers’ use of the Skills Checklist throughout the year for
multiple purposes. Only issues related to the accessibility of
the academic content for all students with significant cogni-
tive disabilities are addressed in the current article.
Participants were recruited by the consultant responsible
for managing the Skills Checklist program at CSDE. A sub-
set of special education teachers who submitted a checklist
for a student in March 2009 was randomly sampled and
then further limited to a pool of teachers who worked at
schools near the focus group location. Those teachers
received an e-mail from the CSDE stating that the state was
partnering with the University of Connecticut to learn more
about how the Skills Checklist was used at schools and its
impact on the education of students with significant cogni-
tive disabilities. Efforts were made to recruit an equal num-
ber of teachers at each level (elementary, middle, and high
school). However, a greater number of teachers at the ele-
mentary level opted to participate in the research project.
Three focus groups were conducted in June 2009 sepa-
rately with elementary (n = 10), middle (n = 5), and high
school (n = 5) special education teachers who had adminis-
tered the Skills Checklist in the 2008–2009 academic year.
During the focus group sessions, efforts were made to
encourage open and honest responses by assuring complete
confidentiality. Focus groups were video recorded with the
permission of the participants, and each session lasted
approximately 90 minutes. Participants received a gift card
for a nominal amount to a local bookstore as compensation
for their time. The recorded data from the focus groups
were transcribed verbatim, and transcripts were coded using
grounded theory (Corbin & Strauss, 2007). Transcripts
were analyzed on a line-by-line basis by the research team,
and open coding was used to identify key themes in the
data. Several themes emerged from the focus group data; in
this article, we focus exclusively on teachers’ feedback
regarding the range of abilities of students with significant
cognitive disabilities and the appropriateness of the Skills
Checklist for all of the students for whom the instrument
was designed. Specifically, teachers at all levels consis-
tently commented that the content on the Skills Checklist
may not be accessible for the range of students they serve.
Quantitative Method
To validate the focus group data and learn more about the
administration of the assessment, teacher ratings of stu-
dents’ academic content knowledge from the 2009 CMT/
CAPT Skills Checklist were analyzed. Specifically, we attem­
pted to identify those students who had mastered certain
120 The Journal of Special Education 46(2)
areas of content as well as those who were unable to dem-
onstrate academic content mastery, using data for all fifth
grade students (n = 519). The Skills Checklist was first
administered in 2006, when these students were enrolled in
second grade; thus, the Skills Checklist was in place for a
majority of their time in elementary school. In addition,
internal analyses at the state level indicated that younger
students tended to demonstrate mastery on a greater portion
of the Skills Checklist content.
Although checklist ratings submitted to the state were
teachers’ summative evaluations of students’ skills, analy-
ses of these data presented an opportunity to study teacher
perceptions of student academic content knowledge. More-
over, these ratings were assigned by teachers, and this
potentially introduces considerable bias in the data. These
limitations notwithstanding, the notion that certain students
were unable to engage with the academic content explicated
on the Skills Checklist warranted further investigation.
Latent class analysis (LCA; Lazarsfeld & Henry, 1968;
Magidson & Vermunt, 2004) was used to identify patterns
in the teacher ratings. LCA is a statistical technique used to
explore latent variables, which are characteristics of people
or places that cannot be directly observed or measured. Exam-
ples of such latent variables in the social sciences include
phenomena such as economic development, racial prejudice,
and religious commitment. LCA is used to explore classifi-
cation structures in these latent variables through the analy-
sis of variation on observed measures or dependent variables
(McCutcheon, 1987). LCA is a useful analytic technique in
that it offers detailed information about class membership
including the latent class proportions (the proportion of
cases in each latent class) and conditional probabilities (the
probabilities that a case belongs to a specific class given the
pattern of responses to the observed measures).
In this study, the latent variable was student academic
content knowledge and the dependent variables were the
teacher ratings for each content area in mathematics and
reading. Separate analyses were conducted for the mathe-
matics and reading checklist data for the fifth grade stu-
dents. For both subjects, the mean scores for each content
area were used as the observed dependent variables; corre-
lations among the mean scores were indicated in the model.
The language arts domains included reading and respond-
ing, exploring and responding to literature, communicating
with others, and English language conventions/writing; the
mathematics domains included algebraic reasoning, geom-
etry and measurement, numerical and proportional reason-
ing, and probability and statistics. Note that all checklist
items were rated on a 3-point scale in which 0 = does not
demonstrate skill, 1 = developing/support, and 2 = mas-
tered/independent. After the number of latent classes was
established, available additional student data were used to
describe the groups of students. Descriptive statistics included
demographic data, student disability category data, mean
scores for each domain of the Access Skill section of the
Skills Checklist, and several items related to communica-
tion skills from the LCI (Kearns et al., 2006).
Results
The results of the qualitative research based on the focus
group interviews and quantitative analyses based on the LCA
are presented below.
Qualitative Analyses
The focus group interview protocol was designed to inves-
tigate ongoing use of the checklist for both instruction and
accountability. In both contexts, the participating teachers
took great care to point out that the Skills Checklist did not
necessarily accommodate all students with significant cog-
nitive disabilities. One teacher said,
It’s interesting for a test that’s designed for a popula-
tion that is supposedly proportionally so small com-
pared to the CMT and CAPT population, within that
population there’s such a broad range of ability. It’s
hard to use the same tool to assess all the students who
we utilize it for.
This theme is particularly salient because it was add­ressed
at the elementary, middle, and high school levels and refer-
ences to the issue tended to elicit emotional responses from
the teachers.
The teachers took time to establish a picture of their
classroom in communicating their use of the Skills Checklist.
One teacher said,
My students are just learning how to pull their pants
down. Some of them are still learning how to feed
themselves, doing simple exchanges with pictures and
not even understanding that, not even knowing how to
play appropriately with any toy.
Another elementary school teacher described working with
literature with a particular student in her class:
The one that I love the best was the difference
between real and make believe. I have a child who
will pull his pants down. We have this discussion
with the parent that he doesn’t know . . . that his mom
wants him to be able to only pull his pants down
when he’s inside rather than outside. He doesn’t have
that realization anyways. So we went through that
and I said, “How am I ever going to teach real and
make believe?”
Goldstein and Behuniak
Yet another teacher, who was particularly emotional in
describing her students, said,
My class is the medically fragile class and it’s called
that for a reason. We’re running over because
Danny’s. . . he’s not breathing and you have to go
take care of that before we worry about if he’s look-
ing at the picture.
When asked to describe their use of the instrument, the
teachers spoke of a divide between those students who work
on Access Skills and those who can engage more deeply with
academic content. Access Skills were designed to assess
communication and quantitative and preacademic skills that
students without disabilities typically develop prior to school
entry. Access Skills are included in the Skills Checklist at
each grade, but data from this section are not incorporated
into performance level scores for accountability. Many
teachers described classrooms similar to this teacher:
I would say, of the seven students I teach, I use the
Skills Checklist to drive my instruction for three of
them. The other students it’s way too high for. I have
a student who’s currently functioning on the 18-month
level. I feel horrible filling out the checklist because
cognitively none of those things are reasonable for
her. The only thing that is reasonable for her is the
Access Skills.
One teacher said, “I happened to bring a copy of [the Skills
Checklist for] my highest functioning child. I chopped his
name off of it, but he’s got a few things in the Access Skills
and we don’t have anything else.” Yet another middle
school teacher described a subset of her students as having
limited decoding ability and another subset with decoding
skills but limited comprehension skills. When asked how
she uses the Skills Checklist, this teacher responded, “I look
at the Access Skills, but the other part of the test really does
not apply to my students. They’re so far removed from that
level of instruction.” Another teacher elaborated, “I have to
really extrapolate down, down, down to their level to really
picture that in my mind, to see how it would affect my
students.”
The teachers were also asked to discuss the extent to
which they felt that special education teachers were pre-
pared to teach academic content to students with significant
cognitive disabilities. Several teachers mentioned that they
were included in all of the academic professional devel-
opment activities at their schools. Those teachers were
more positive, sharing comments such as, “We’re held
accountable. We’re involved in all of the regular ed.
instruction” and “You’re teaching me how to teach this fifth
grade math and I’m never ever going to get anywhere near
121
it,’ but we’re exposed to everything.” Most, however, were
concerned: “How well are we prepared to teach students
with significant cognitive delays? I don’t think we’re very
prepared at all.” The teachers explained that a community
of practitioners working with a similar population does not
exist for them. One teacher said, “We’re so isolated and
there are so few teachers. I just cross my fingers sometimes,
hope that I’m on the right track.” One teacher explained,
“Those of us who are old are probably not very well pre-
pared. We had one course in mental retardation in college.”
She later added,
It just sort of floored me that the stuff that you kind
of learn through working with these kids that people
don’t grasp. You know, you do first, second, third.
You realize, for example, that they don’t know first
is one, and second is two. And the nuances of low-
ness. You teach them all kinds of separate skills and
then you have to teach them to generalize.
Another said, “I’m a specialist in this area and I’m still
making up my own materials.” In spite of these comments,
another teacher added, “I don’t think you can teach some-
body in a college course what this looks like.”
Within this context, many teachers suggested that two
levels of testing might be more appropriate for these stu-
dents: “One for students who just have Access Skills and
then [one for] the ones where time in the general education
environment is appropriate and beneficial.” This teacher
added that some students have a diagnosis that leads to a
situation in which the student is “either regressing medi-
cally or they’re not going to progress past a certain point.”
A different assessment would be appropriate for these stu-
dents because “it’s important for parents to be able to, and
teachers to be able to focus on the progress, as opposed to
all those zeroes.”
The notion that the Skills Checklist may not be appropri-
ate for all of the students for whom it was designed was an
unanticipated and alarming outcome of the focus group
interviews. However, the small sample size of this portion
of the study left the pervasiveness of this problem unclear.
Data from the 2009 administration of the Skills Checklist
were analyzed to investigate whether such patterns exist in
teacher ratings of student academic content knowledge at
the state level.
Quantitative Analyses
For both subjects, two-, three-, and four-class models were
fit to the data. Only results from analyses of the mathematics
data are presented here because of space limitations (see
Note 1). Measures of model fit and the proportion of students
in each class for each analysis are included in Table 1.
122 The Journal of Special Education 46(2)
Table 1. Model Fit Statistics and Proportion of Students by Class for the Mathematics Data
Statistic Two-class model
Akaike information criterion 966.01
Sample-size-adjusted Bayesian 986.57
information criterion
Entropy .87
Proportion of students by latent
class
Class 1 .65
Class 2 .35
Class 3
Class 4
For mathematics, the Akaike information criterion (AIC)
and sample-size-adjusted Bayesian information criterion
(BIC) measures of model fit decreased as the number of
classes increased; this suggested better fit for a higher num-
ber of classes. However, the three-class model had a higher
entropy value, providing evidence of its superiority over the
other models. In addition, the Vuong–Lo–Rubin likelihood
ratio test for the four-class model was not significant (p =
.19), which indicates that the three-class model is sufficient
to represent the data. Moreover, the three-class model rep-
resents a more even distribution of students across classes
as compared to the four-class model, which facilitates inter-
pretation of the data given the purpose of the analysis. Thus,
the three-class model was selected for the mathematics
data, and students were classified into three groups.
After examination of the model fit statistics for the read-
ing data, a three-class model was selected as well. The AIC,
the sample-size-adjusted BIC, and entropy statistics sug-
gested a four-class model. However, the Vuong–Lo–Rubin
likelihood ratio test was not significant in either the three-
class (p = .18) or the four-class (p = .21) model, suggesting
that a two-class model was sufficient to represent these
data. In the two-class model, however, nearly all of the stu-
dents fell into the first class; further analyses of these clas-
sifications would limit the utility of comparisons across
classes. Moreover, the proportion of students in Class 2 of
the two-class model and Class 3 of the three-class model
was similar, as were the means on the content domains.
This suggested that the three-class model incorporated a
split of the students in larger class of the two-class model.
Use of the three-class model for the reading data allowed
for a parallel structure to the mathematics data. Thus, the
three-class model was selected for the reading data as well.
The three classes represent variable levels of teacher rat-
ings of content mastery and are subsequently referred to as
the students with low ratings, moderate ratings, and high
ratings. For mathematics, the data show that 49% of stu-
dents had mean scores across the downward extensions that
are very close to 0 (M = 0.20, SD = 0.22, n = 256). A rating
of 0 indicates that the student could not demonstrate the
Three-class model Four-class model
842.68 775.43
868.54 806.69
.90 .88
.49 .46
.28 .24
.23 .09
.21
skill explicated in a specific downward extension, even with
teacher support. Stated differently, teachers’ ratings indi-
cated that half of the students did not engage with the math-
ematics content on the Skills Checklist. For reading, 45% of
the students were in the group with low ratings (M =
0.18, SD = 0.17, n = 231). Ratings for the other two groups
of students indicated that teachers viewed students’ skills as
at least developing or supported, but to varying degrees.
Students with moderate ratings (mathematics: M = 0.94, SD
= 0.26, n = 144; reading: M = 0.69, SD = 0.26, n = 205) and
high ratings (mathematics: M = 1.48, SD = 0.26, n = 119;
reading: M = 1.25, SD = 0.26, n = 83) engaged with the
academic content on the Skills Checklist, at least minimally.
Moreover, more students had higher scores in mathematics
than in reading. Replication studies were conducted with
eighth grade data and produced similar findings: Approxi-
mately half of the students had mean scores that were close to
zero (62% for mathematics and 45% for reading). In their
entirety, the quantitative analyses confirmed the findings of
the focus groups: Approximately half of the population did
not engage with the academic content on the Skills Checklist.
Available data were used to describe the three groups of
students. Again, only results from analyses of the mathemat-
ics data are presented here. Review of these data indicated
that the distribution of the demographic variables (gender,
ethnicity, free or reduced-price lunch status, and English
language learner status) is approximately equivalent across
the groups within each subject (see Table 2).
This provides evidence to support the fairness and
validity of the instrument in terms of demographic sub-
group performance. Examination of the distribution of stu-
dents within each disability category across each of the three
groups, however, may highlight some disparities. In review-
ing these data, we consider only the category in which n is
greater than or equal to 20. The mathematics data (see
Table 3) indicate that most of the students with autism and
with multiple disabilities fell in the group with low ratings
(55% and 69%, respectively) and most of the students with
speech-language impairment (46%) were in the group with
high ratings.
Goldstein and Behuniak 123

Table 2. Demographic Data by Mathematics Classification

Low ratings Moderate rat- High ratings All ratings
Demographic category (%; n = 256) ings (%; n = 144) (%; n = 119) (%; n = 519)
Gender
Male 61 62 73 63
Female 39 38 27 35
Ethnicity
Black 20 17 20 19
Hispanic 22 19 18 20
White 56 56 54 55
Asian American 3 6 8 5
American Indian — 1 — 0
Lunch status
Eligible 43 41 47 43
Ineligible 57 59 53 55
English language learner (ELL)
ELL 6 12 13 9
Non-ELL 94 88 87 92

Table 3. Distribution of Mathematics Classification for Each Student Disability Category

Disability category Total n Low ratings (%) Moderate ratings (%) High ratings (%)
Autism 134 55 28 18
Emotional disturbance   4 75 — 25
Hearing impairment   3 33 — 67
Intellectual disability 110 36 38 26
Learning disability 12 17 42 42
Multiple disabilities 116 69 13 18
Orthopedic impairment   0 — — —
Other health impairment 49 35 37 29
Speech-language impairment 26 27 27 46
Traumatic brain injury   2 100 — —
Visual impairment   0 — — —
Missing 63

Students with intellectual disabilities and other health
impairments were approximately evenly distributed across
the groups. Similar trends are evident in the reading data. A
large portion of the students with autism (44%) and a major-
ity of students with multiple disabilities (66%) were in the
group with low ratings. Alternatively, there were few stu-
dents with speech-language impairments in the group with
low ratings (12%).
The additional tables provide a more complete picture of
teachers’ views of these students’ skills. Mean scores for
each domain in the Access Skills section were lower and
more variable for the group with low ratings in both subjects
(see Table 4).
Individual items from the LCI provide greater detail on the
typical modes of communication for each cluster (Table 5).
With regard to communication skills, students in the group
with low ratings were more likely to use cries, facial exp­
ressions, or changes in muscle tone to communicate whereas
students in the group with high ratings were more likely to use
symbolic language to communicate. Similarly, students in the
group with high ratings were more likely to follow directions
without additional cues. In addition, approximately one third
of the students in the group with low ratings use augmentative
communication during instruction, whereas most of the stu-
dents in the other two groups do not.
Discussion
The teachers in the focus group interviews referred to two
types of students with significant cognitive disabilities:
those for whom grade-level academic content is accessible
and those for whom it is not. These findings were confirmed
124 The Journal of Special Education 46(2)

Table 4. Mean Scores and Standard Deviations for Access Skill Domains by Mathematics Classification

Low ratings Moderate ratings High ratings All ratings

(n = 256) (n = 144) (n = 119) (n = 519)
Access skill domains M SD M SD M SD M SD
Expressive communication 1.13 0.52 1.74 0.25 1.88 0.15 1.47 0.52
Receptive communication 1.30 0.52 1.84 0.16 1.93 0.10 1.60 0.48
Basic literacy 1.19 0.63 1.88 0.18 1.95 0.08 1.55 0.58
Spatial relations 1.55 0.49 1.89 0.23 1.95 0.16 1.74 0.42
Social interactions 1.01 0.49 1.52 0.38 1.70 0.31 1.31 0.42

Table 5. Responses for Communication Items From the Learner Characteristics Inventory by Mathematics Classification

Low ratings Moderate ratings High ratings All ratings

Communication items (n = 256; %) (n = 144; %) (n = 119; %) (n = 519; %)
Expressive communication
Communicates primarily 19 1 1 10
through cries, facial expressions,
change in muscle tone
Uses intentional communication, 25 7 5 16
but not at a symbolic language
level
Uses symbolic language to 44 81 87 64
communicate
Receptive language
Uncertain response to sensory 5 — 1 3
stimuli
Alerts to sensory input from 14 — — 7
another person BUT requires
actual physical assistance
Requires additional cues to 50 44 25 42
follow 1–2 step directions
Independently follows 1–2 step 19 44 66 37
directions, does NOT need
additional cues
Augmentative communication
No 54 95 94 69
Yes 39 5 6 20
Missing 30 17 9 11

by analyses of checklist data in mathematics and reading.
Our research indicated that nearly half of the teachers found
the content inaccessible for their students. Moreover, the
data showed these students had poor communication skills,
used augmentative communication as part of their instruc-
tion, and were more likely to have disability classifications
of autism or multiple disabilities. Ratings for the remaining
students suggested teachers felt their students could engage
with the academic content with some support and with
varying degrees of success. Students who could engage
with the academic content had higher ratings of their com-
munication skills and were less likely to use augmentative
or assistive communication devices.
These data highlight the challenges of bringing an aca-
demic curriculum to a population of students who do not
share the same cognitive and communication abilities as the
general student population. Students with significant cogni-
tive disabilities have specific information processing chal-
lenges related to short-term memory and the need for increased
frequency of skill repetition with instructional feedback
(Kleinert, Browder, & Towles-Reeves, 2009). Teachers of
students with significant disabilities encounter many more
obstacles in working to cover the curriculum with their stu-
dents, even a curriculum judged against alternate achieve-
ment standards. The finding that certain disability categories
were more prominent in the group with low ratings is also
Goldstein and Behuniak
of interest. These data are only suggestive of a relationship
between disability category and teacher ratings, and further
investigation into this pattern and its implications for
accountability practice is needed. The present study also
brings focus to teachers’ inadequate preparation to instruct
their students on content of this nature. Several teachers in
the focus groups were quick to note deficits in their own
knowledge and skills. Inclusive accountability practices
have questionable meaning and impact in the absence of
effective instruction. This finding suggests the need for
resources devoted to helping teachers develop instruction to
meet their students’ learning challenges through preservice
teacher training programs as well as professional develop-
ment programs for practicing educators. Certainly, more
research is needed to explore the interplay of these learning
challenges and teachers’ efforts to expose students to an
academic curriculum in combination with functional skills.
The current study provides a preliminary examination of
variability in academic content knowledge for one cohort of
students in one state.
The results of this study suggest that stratification of the
population of students with significant disabilities may be
necessary to improve both the instruction and the assessment
of these students using the Skills Checklist. This is not a new
concept. Browder, Flowers, and Wakeman (2008) conducted
an empirical evaluation of a classification scheme for the
communication abilities of students with significant cognitive
disabilities, based on teacher ratings of 189 students. Their
study provided support for the notion that students with sig-
nificant cognitive disabilities can be classified by symbolic
level (awareness or presymbolic, early symbolic, and sym-
bolic), and they offer this classification scheme as a tool for
academic planning. The authors further suggest that it may be
appropriate to establish alternate expectations for achieve-
ment on alternate assessments based on these levels of com-
munication, as most alternate assessments are biased toward
students who communicate at the symbolic level. Using the
LCI, Kearns et al. (2009) found that students at the presym-
bolic level of receptive and expressive communication were
also identified as having other issues that increased the com-
plexity of their learning needs. These issues included low
levels of social engagement, limited motor skills, sensory
impairment, and health-related issues that affected school
attendance. Their data suggested these students, students
with the “most significant cognitive disabilities,” represented
approximately 10% of the population. The present study,
which incorporates data from both the LCI and Connecticut’s
Skills Checklist, suggests this number may be much higher.
Although the results of the current study can be considered
only preliminary, the finding that the grade-level academic
content of one state’s alternate assessment may not be acces-
sible for a large proportion of the students for whom it was
targeted must be viewed as a cause of concern.
125
Implications and Directions
for Further Research
The Skills Checklist is more than an assessment of academic
progress for students with significant cognitive disabilities.
It is also expected to fill a role as a comprehensive guide for
teachers to instruct students on AA-AAAS. The present
study suggests that these standards may not be accessible for
the entire population of students it was designed to serve.
Specifically, the data indicate that additional classification
of this small population of students may be necessary for
both effective instruction and effective assessment. A dis-
tinction within this population would have important impli-
cations for educators and researchers alike.
This study raises a number of questions regarding rea-
sonable expectations for students at presymbolic levels of
communication. Are alternate assessments based on grade-
specific content an appropriate accountability measure for
these students in their current form? What is the relation-
ship between information processing challenges and the
extent to which the academic curriculum can be covered?
Do these students have sufficient opportunity to learn the
target academic content? Would it be beneficial to focus
instruction, and subsequently assessment, primarily on
“preacademic” skills such as communication and other
adaptive activities for these students? Clearly, more research
is necessary to address these questions.
One alternative to the study of student academic content
knowledge is to focus research on the classroom teacher.
What knowledge and skills must a special education teacher
have to effectively deliver academic content to students with
limited communication abilities? Can we assume that all
special education teachers have such knowledge? Would
alternate assessments based on grade-specific content be an
appropriate accountability measure for this population if
teachers felt comfortable providing grade-specific academic
content instruction for these students? Furthermore, do edu-
cators have sufficient access to assistive technology or aug-
mentative communication devices to allow for efficient
communication with these students? Do educators with
access to these technologies understand how to use them
effectively? Although these questions cannot be answered at
this time, the results of this study provide evidence of the
importance of addressing these issues in the future.
Conclusion
The development and validation of alternate assessments
based on alternate achievement standards for accountability
constitute a nascent field. Connecticut’s Skills Checklist was
created to change the nature of education for students with
significant cognitive disabilities. The quantitative analyses
indicated that approximately half of the target population can
126 The Journal of Special Education 46(2)
engage with the academic content on the Skills Checklist.
However, for the other half of the target student population,
the effects of participation in the alternate assessment are
unclear. In the focus groups, teachers explained that these
students may focus more on communication and other preac-
ademic skills while in school. Our study raises both practical
and conceptual issues that relate to preservice education,
ongoing professional development, availability of resources
within schools, and the broad issue regarding the impact of a
large-scale assessment program on the educational environ-
ment of students with significant cognitive disabilities.
Appendix
Focus Group Protocol
Introduction. UConn investigators will introduce them-
selves to the participants. The facilitator will distribute the
consent forms, explain the purpose of the session, and
explain the procedures to the participants. Participants will
be given time to review the consent form, ask questions as
necessary, and sign the consent form. (15 minutes)
Participant Introductions. Participants will be asked to
provide their first name, the grade range of the students they
teach, and the Checklist training sessions they have attended.
(5 minutes)
Use of the Checklist in the Classroom. Participants
will be asked to describe how the Checklist is used to
design instruction for students with significant cognitive
disabilities in their schools. Specifically, they will be
asked:
1. Do any of you use the Checklist during the fall, before
the testing window? How do you use it?
2. How is this similar or different to the way in which
you use the Checklist during the CMT/CAPT testing
window, from January through March?
3. The Checklist was designed to make grade-level aca-
demic content accessible for students with significant
cognitive disabilities. Do you believe this goal has
been achieved? Why or why not?
4. Can you describe a situation in which the Checklist
affected your decision to expose a particular student
to grade-level content? (15 minutes)
Impact of the Checklist. Participants will be asked to
describe how the Checklist has impacted the educational of
students with significant cognitive disabilities.
1. How has the Checklist impacted the education of stu-
dents with significant cognitive disabilities?
2. Has the Checklist changed the way students with sig-
nificant cognitive disabilities exposed to the general
curriculum? How?
3. Has the Checklist affected teacher or parent expecta-
tions? How?
4. In general, do you feel that special education teachers
are well prepared to teach academic content to students
with significant cognitive disabilities? (15 minutes)
Sharing Student Checklist Data. Participants will be
asked to explain how data from the Checklist is shared with
the planning and placement team during the school year.
1. How well do general education teachers, administra-
tors, and specialists in your school understand the
purpose and uses of the Checklist? To your knowl-
edge, do they receive any formal training on these
issues?
2. Do you share data with the child study team during
the school year? How is this information used by the
child study team?
3. How well do parents understand the purpose and uses
of the Checklist? (10 minutes)
Training for Checklist Administration. Participants will
be asked to discuss their training to use the Checklist.
1. Do you feel the Standard Checklist training program
or the Certified Rater Training program can be im-
proved in any way? (5 minutes)
General Comments. Participants will be asked to offer
any additional comments about the Checklist training pro-
gram or the Checklist administration.
1. Is there anything you would like to see changed in the
way the Checklist is designed or used? (5 minutes)
Debrief. The facilitator will summarize the discussions and
solicit any additional comments or questions from the par-
ticipants. The facilitator will thank the participants for their
time and effort.
Abbreviations: CMT, Connecticut Mastery Test; CAPT, Connecticut
Academic Performance Test.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interests with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) disclosed receipt of the following financial support
for the research, authorship, and/or publication of this article: This
manuscript was supported, in part, by the U.S. Department of
Education Office of Special Education Programs (Grant No.
H373X070003). However, the opinions expressed do not necessarily
Goldstein and Behuniak
reflect the position or policy of the U.S. Office of Special Education
Programs and no official endorsement should be inferred.
Note
1. Analyses of reading data are available from the authors.
References
Almond, P., & Bechard, S. (2005). In-depth look at students who take
alternate assessments: What do we know now? Retrieved from
http://www.measuredprogress.org/assessments/inclusive/
articlespapers.aspx
Behuniak, P., & Amenta, J. (2009). Assessing all students in
Connecticut. In W. M. Schafer & R. Lissitz (Eds.), Alternate
assessments based on alternate achievement standards: Policy,
practice, and potential (pp. 157–170). Baltimore, MD: Brooks.
Browder, D., Flowers, C., Ahlgrim-Delzell, L., Karvonen, M.,
Spooner, F., & Algozzine, R. (2004). The alignment of alter-
nate assessment content to academic and functional curricula.
The Journal of Special Education, 37, 211–223.
Browder, D. M., Flowers, C., & Wakeman, S. Y. (2008). Facili-
tating participation in assessments and the general curriculum:
Level of symbolic communication classification for students
with significant cognitive disabilities. Assessment in Education:
Principles, Policy, and Practice, 15(2), 137–151.
Browder, D., Spooner, F., Ahlgrim-Delzell, L., Harris, A., &
Wakeman, S. (2008). A meta-analysis on teaching mathemat-
ics to students with significant cognitive disabilities. Excep-
tional Children, 74, 407–432.
Browder, D., Wakeman, S., & Flowers, C. (2009). Which came
first: The curriculum or the assessment? In W. M. Schafer
& R. Lissitz (Eds.), Alternate assessments based on alter-
nate achievement standards: Policy, practice, and potential
(pp. 329–332). Baltimore, MD: Brooks.
Browder, D., Wakeman, S., Spooner, F., Ahlgrim-Delzell, L.,
& Algozzine, B. (2006). Research on reading instruction for
individuals with significant cognitive disabilities. Exceptional
Children, 72, 392–408.
Cameto, R., Knokey, A.-M., Nagle, K., Sanford, C., Blackorby, J.,
Sinclair, B., & Riley, D. (2009). National profile on alternate
assessments based on alternate achievement standards (NCSER
2009-3014). Menlo Park, CA: SRI International.
Connecticut State Department of Education. (2006). CMT/CAPT
Skills Checklist Second Generation: Technical manual. Hartford,
CT: Author.
Corbin, J., & Strauss, A. (2007). Basics of qualitative research:
Techniques and procedures for developing grounded theory
(3rd ed.). Thousand Oaks, CA: Sage.
Elliott, S. N., & Roach, A. T. (2007). Alternate assessments of
students with significant disabilities: Alternative approaches,
common technical challenges. Applied Measurement in Edu-
cation, 20, 301–333.
127
Gong, B., & Marion, S. (2006). Dealing with flexibility in assess-
ments for students with significant cognitive disabilities (Syn-
thesis Report 60). Minneapolis: University of Minnesota,
National Center on Educational Outcomes.
Kearns, J., Kleinert, H., Kleinert, J., & Towles-Reeves, E. (2006).
Learner Characteristics Inventory. Lexington: University of
Kentucky, National Alternate Assessment Center.
Kearns, J. F., Towles-Reeves, E., Kleinert, H. L., &
Kleinert, J. O., Thomas, M. K. (2009). Characteristics of and
implications for students participating in alternate assess-
ments based on alternate academic achievement standards.
The Journal of Special Education. Advance online publica-
tion. doi:10.1177/0022466909344223
Kleinert, H., Browder, D., & Towles-Reeves, E. (2009). Models of
cognition for students with significant cognitive disabilities:
Implications for assessment. Review of Educational Research,
79, 301–326.
Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analy-
sis. Boston, MA: Houghton Mifflin.
Magidson, J., & Vermunt, J. K. (2004). Latent class models. In
D. Kaplan (Ed.), The Sage handbook of quantitative methodology
for the social sciences (pp. 175–198). Thousand Oaks, CA: Sage.
McCutcheon, A. (1987). Latent class analysis. Newbury Park, CA:
Sage.
Patton, M. P. (2001). Qualitative research & evaluation methods
(3rd ed.). Thousand Oaks, CA: Sage.
Perie, M. (2007). Setting alternate achievement standards. Lex-
ington: University of Kentucky, National Alternate Assess-
ment Center, Human Development Institute.
Quenemoen, R. (2008). A brief history of alternate assessments
based on alternate achievement standards (Synthesis Report
68). Minneapolis: University of Minnesota, National Center
on Educational Outcomes.
Towles-Reeves, E., Kearns, J., Kleinert, H., & Kleinert, J. (2009).
An analysis of the learning characteristics of students taking
alternate assessments based on alternate achievement stan-
dards. The Journal of Special Education, 42, 241–254.
U.S. Department of Education. (2003). Title I—Improving the aca-
demic achievement of the disadvantaged; Final rule, 68 Fed. Reg.
236 (December 9, 2003). Washington, DC: U.S. Department of
Education, Office of Elementary and Secondary Education.
U.S. Department of Education. (2005). Alternate achievement stan-
dards for students with the most significant cognitive disabilities:
Non regulatory guidance. Washington, DG: Office of Elemen-
tary and Secondary Education.
U.S. Department of Education. (2009). State and local implemen-
tation of the No Child Left Behind Act, Volume V: Implementa-
tion of the 1 percent rule and 2 percent interim policy options.
Washington, DC: Office of Planning, Evaluation, and Policy
Development. Retrieved from http://www.ed.gov/rschstat/
eval/disadv/nclb-disab/nclb-disab.pdf