Assessing Supplemental Instruction: A

Student Perspective
Piloting a CECE Assessment for
Supplemental Instruction
November 30, 2020
Assessment Team: Vanessa Beckham, Nathan Foster,
Jasmine Ivy, Carol McFarland McKee, and Alex Torres
Overview
• Introduction
• Research Questions
• Literature Review
• Theoretical Framework
• Methods Overview
• Findings
• Conclusion
 Introduction
• NIU is currently undergoing a restructuring initiative
to bring all academic support services together.
• Research shows that consistent attendance
in Supplemental Instruction (SI) is directly correlated to
increased student persistence and graduation rates (Malm
et al., 2018). 
• We believe it is vital to collect and use data to identify
program strengths and weaknesses to bring about
positive change while maintaining program successes. 
Research Questions
1. What are students’ perceived impacts of
participating in SI?
2. Why are some students not taking
advantage of SI academic support services?
3. What impact does SI programming have on
students’ perceptions of NIU as a
Culturally Engaging Campus Environment
(CECE)? 
 A Review of Literature
• Corequisite courses have been indicated to significantly improve outcomes
– learning assistance that happens alongside courses as opposed to
prerequisite courses (Barhoum 2018)
• Osborne, Parlier, and Adams (2019) study showed students involved in any
learning assistance programs believed that the assistance impacted them
academically
• International Center for Supplemental Instruction (SI) offers an
accreditation specifically for SI programs
– four areas must be met:
• training and certification of the program coordinator
• initial and ongoing intensive training and observation for SI leaders
• a strong focus on session planning
• attendance of all class sessions by SI leaders and evaluation of program
effectiveness (International Center for Supplemental Instruction, n.d.)
Theoretical Framework
• Our approach = critical theorists, with strong influences from
poststructural/postmodernism.
• Methodology = theoretical framework of the Cultural Engaging
Campus (CECE) Model proposed by Museus (2014).
– provides a more racially and culturally responsive model
– includes two clusters of indicators:
• cultural relevance – includes five indicators that “characterize campus
environments that meaningfully engage and reflect the cultural
backgrounds, communities, and identities of diverse students”
• Cultural responsiveness – consists of four indicators that “reflect
environments in which an understanding of diverse students’ cultural
norms and values undergirds campus learning and support systems
that respond to these students’ needs."
Methods Overview
• Both Quantitative and Qualitative data were
collected
• A survey instrument and focus group questions
were developed to gather perceptions of both
participants and non-participants of SI
• Analysis through the lens of the CECE model
provides insight into student perceptions of SI’s
effectiveness as a culturally responsive support
system
Targeted Participants
• 2,062 individuals who were eligible to participate
in SI across multiple sections of 7 courses in Fall
2019 & Spring 2020
• 749 individuals participated in SI and were
selected to receive survey and invitation to focus
group for SI participants
• 1,313 individuals did NOT participate in SI and
were selected to receive an invitation to focus
group for non-participants
Limitations
• Survey instrument and interview questions were developed
and have not been previously tested
• Compilation of previous assessment survey results was not
readily available
• Inability to obtain student contact information in a timely
manner and demographic data about targeted participants due
to university processes related to FERPA
• Low survey response rate, presumably due to some student
disconnect with campus due to COVID-19 and potential
“online fatigue”
• Lapse of time between participation in SI in Fall 2019 &
Spring 2020 and survey distributed in Fall 2020
The Survey Instrument
• 40 questions + 9 skip-logic questions developed by
team in collaboration with client and course
instructor
• Incorporated elements of past SI surveys and
CECE indicators
• Survey took approximately 9 minutes to complete
 The Survey Organization
• Questions organized in 7 themes for data analysis
– Introduction & Consent
– Familiarity & Understanding of SI
– Expectations & Perceptions of SI
– SI Culturally Engaging Environment
– Student Status & Academic Standing
– Participant Demographic Information
– Survey Closure & Invitation to Participate in
Focus Group
 Survey Distribution & Participation Rate

• Survey invitation & link emailed by gatekeeper to

749 SI participants on 11-5-2020 with reminder
emails on 11-8-2020 and 11-16-2020. Data collected
on 11-19-2020. 12 students completed the survey. 1
student signed up for focus group.
• Focus group invitation & link emailed by gatekeeper
to 1,313 SI non-participants on 11-5-2020 with 2
reminder emails. 3 students signed up for focus
group, 1 student showed.
Demographic Information
• Self-described ethnicity: Black or African American
(2); Hispanic or Latino (4); Asian (2); White (1).
Some respondents did not answer
• Gender: Female (9); Male (2). No responses for other
gender options listed (nonbinary/third gender,
genderqueer, prefer to self-describe, prefer not to say)
• Sexual Orientation: Heterosexual/straight (10*),
Prefer to not say (1)
• Age: 18-20 (10); 25-29 (1)
• Transfer Student: 1
Familiarity/Understanding of SI
• Most students learned about SI from an SI Leader.
Also from campus visits, faculty or instructors,
academic advisors, CHANCE presentations &
advisors. Orientation, NIU website less impactful.
• Half of respondents indicated SI was required.
Others evenly split that 1) SI was optional and 2)
SI was recommended.
• Classes for which SI was most utilized were
MATH 108P, MATH 109P, MATH 110P. Also
some ACCY, LTRE, CHEM, STAT & ENG*
courses were noted.
Expectations/Perceptions of SI
• Half of all respondents met with class instructor 2-
3 times outside of class
• Excluding time spent in class or SI sessions, 42%
studied or did homework for 4-6 hours each week;
42% spent 2-3 hours each week
• 92% of students indicated they enrolled for the
next course in progression the following semester
 Benefits of Participation in SI

Helped me with my                        
understanding of the subject

Helped me improve my grade                  

Helped me build confidence                  

in my abilities

Helped me connect with other            

students in my class

Helped me connect with the  

instructor in my class

Helped me connect with    

informal mentors

0 1 2 3 4 5 6 7 8 9 10 11 12
 Helpfulness of SI group sessions

Extremely helpful                    

Moderately helpful                

Somewhat helpful          

Slightly helpful    

Not helpful at all

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

Open-ended Responses
• “The SI offered a more in depth teaching of each
class subject.”
• “They (SI Leaders) were at one point in time where
I am.”
• “Small group, more personal help.”
• “Someone that can relate since SI leaders are
students as well.”
• “It helped me understand any material the professor
did not clear up.”
• “Easy to understand material.”
Grade Expectations

Quite a bit higher                

A little bit higher      

About the same  

A little bit lower

Quite a bit lower

0 1 2 3 4 5 6 7 8
 Preparedness of SI Leaders

Extremely knowledgeable                    
and well-prepared

Very knowledgeable and                        

well-prepared

Moderately
knowledgeable and well-    
prepared

Slightly knowledgeable
and well-prepared

Not knowledgeable or
well-prepared at all

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5

CECE – Welcomed and Respected

                           
Strongly agree                                

                Welcomed
Agree             Respected

Somewhat agree

Neither agree nor

disagree

Somewhat
disagree

Disagree

Strongly disagree
CECE Indicators
• Respondents strongly agreed (64%) or agreed
(27%) that SI staff members sent information
about where to access academic support
• 73% of respondents strongly agreed or agreed that
SI Leaders or staff checked in on them regularly
and 18% strongly disagreed
• 55% agreed that SI Leaders were interested in
getting to know them
• 60% agreed that SI Staff were interested in getting
to know them
CECE Indicators (cont’d)
• 82% strongly agreed or agreed that if experiencing
an academic challenge, they could go to SI Leaders
for support
• 91% agreed or strongly agreed that if experiencing
an academic challenge, they could go to SI Staff for
support
• 82% strongly agreed or agreed that it was easy to
connect with SI Leaders, and with SI Staff, who
shared similar backgrounds or experiences as them
Focus Groups Themes
• 3 students registered for the non-participant focus
group, and we had 1 attendee.
• 1 student registered and participate in
the participant focus group.

Interview Themes
• Communication is important.
• Helpfulness and respect from SI leaders.
• Grade improvement and accessibility. 
Implications & Recommendations
• Highlighting student participants’ perceived
efficacy of SI program
• Affirming contributions to Culturally Engaging
Campus Environment at NIU
• Expanding participation through comprehensive
marketing plan
• Continuous and ongoing assessment using pilot
assessment model
• Consider pursuing certification and accreditation
through International Center for SI
References
• Barhoum, S. (2018). Increasing Student Success: Structural
Recommendations for Community Colleges. Journal of Developmental
Education. Retrieved September 24, 2020 from 
https://files.eric.ed.gov/fulltext/EJ1210442.pdf 
• Malm, J., Bryngfors, L., & Fredriksson, J. (2018). Impact of
Supplemental Instruction on Dropout and Graduation Rates: An Example
from 5-Year Engineering Programs. Journal of Peer Learning, 11, 76–88. 
• Museus, S. D. (2014). The Culturally Engaging Campus Environments
(CECE) model: A new theory of success among racially diverse college
student populations. In M.B. Paulsen (Ed.), Higher education: Handbook
of theory and research (Vol. 29, pp. 189-227). Springer. 
• Osborne, J. D., Parlier, R., & Adams, T. (2019). Assessing Impact of
Academic Interventions through Student Perceptions of Academic
Success. Learning Assistance Review, 24(1), 9–26.