SOUTHEAST

ACADEMY
HIGH SCHOOL

cOUNSELING PROGRAM EVALUATION

Alejandra V. Gonzalez & Sarahi Salcedo

EDP 596: Program Evaluation
 TABLE OF CONTENTS
Mission, Vision and Executive Summary.......... 3

Program Description........................................... 4

Logic Model ........................................................ 5

Evaluation Questions......................................... 6

Design ................................................................. 7

Results ................................................................ 8

Conclusions & Recommendations ................. 13

References......................................................... 15
MISSION
Southeast Academy High School (SEAHS) Counseling Program, collaborates with parents, staff
and community, to empower young men and women to unlock their potential through a
program of college and career exploration, social development and leadership training within
a structured environment. The school counselor advocates for equity, access and student
success by identifying and addressing barriers, allowing students to become responsible,
productive citizens and lifelong learners. Counseling program embraces personal excellence
in everything we do.

VISION
The counseling department of Southeast Academy prepares self-directed, ethical young men and
women who value the importance of continued academic and career education and are prepared
for responsible leadership. Students will successfully navigate the many complexities of an ever-
changing world, understand what it takes to be an effective leader and team member, and have
the foundation for future success as productive citizens.

EXECUTIVE SUMMARY
Southeast Academy High School (SEAHS) counseling program was evaluated to gain an
understanding of the effectiveness of SEAHS comprehensive school counseling program
and its impact on students. Counseling program is actively working to support students
effectively in line with the intended goals of the school as well as the mission and
vision of the school counseling program. SEAHS is a military and law enforcement
school, where all students participate in a strict environment that prioritizes leadership
development. For that reason, we strongly felt it was essential to examine how the
school counseling program molds itself to support the needs of this school and
community. The focus of this evaluation was based on two evaluation questions that
aimed to determine an overall effectiveness of the counseling program on students’
college/career exploration, social development, and leadership training and the impact
of meeting its intended long-term goals for students. Data was collected through
California Dashboard to examine trends in graduation rates and college/career
readiness. We also examined Norwalk-La Mirada Core Curriculum outcome data to
determine the counseling program’s effectiveness on students’ development. In
addition counselor pre/post-survey and district surveys were collected and analyzed
from years 2018-2022. However, once beginning the data collection process, the
evaluation team came across several barriers to accessing all of the data. Therefore,
our results are only based on the data we were able to recommend. More data,
however, is needed to fully answer our two evaluation questions.
00 21
 02

INTRODUCTION
The main purpose of evaluating SEAHS comprehensive school counseling program was to gain
a better understanding of the different types of services provided to students. In addition, it
allowed us to evaluate if whether the services provided aligned with the counseling
program’s mission and vision statement. In other words, are students successfully becoming
responsible, productive citizens, and lifelong learners as indicated in their mission statement.

LIMITATIONS
Due to COVID-19, data was not collected from the years 2020 to 2021. Based on this we were
unable to determine trends and completely evaluate the school counseling program
effectiveness. In addition, it is important to keep in mind that Southeast Academy High
School’s data is under John Glenn High School’s (JGHS) data umbrella. Data retrieved to
answer the second evaluation question is JGHS College/Career Indicator reports from
California Dashboard. This provides inaccurate and invalid results for the second evaluation
question, but it was necessary to attach because the results provide College/Career readiness
per cultural groups (tables 3-6).

AUDIENCE
The program’s evaluation was developed to showcase SEAHS Counseling program impact and
effectiveness towards students. The information collected is relevant to parents,
administrators, stakeholders, family and school counselors.

PROGRAM DESCRIPTION
SEAHS school counseling program is a data-driven
comprehensive program that ensures equity, access, and student success. SEAHS mission is to
empower young men and women to become responsible, productive citizens and lifelong
learners for success in their college and careers. SEAHS is a Title 1 school in the Norwalk-La
Mirada Unified School District (NLMUSD) in Norwalk, CA. SEAHS’s school counseling program
consists of 1 counselor and 2 counseling interns. The counseling team follows the Multi-Tiered,
Multi-Domain Systems of Support (MTMDSS) framework. Counseling program uses a tiered
approach to support students in their academic, social-emotional, and college/career
development. It is 170:1 student-to-counselor ratio. A logic model was created to indicate inputs,
activities (what we do), outputs (products of inputs and activities), and outcomes (short,
intermediate, and long-term goals). Refer to the following page, Logic Model.
 03
LOGIC MODEL

About 81% of the students identify as Hispanic/Latino,

8% Asian/Pacific Islander, 5% Black, and 6% White. In addition, approximately 87% of the
students identified as disadvantaged, 4% with disabilities, and 1% limited-English-proficiency
students. It is important to note that unlike other high schools, SEAHS falls under the John H.
Glenn High School umbrella. SEAHS offers two pathways, emergency responders and public
services; about 86% of students complete one pathway.

Contextual Events and Circumstances

There are unique contextual events/circumstances that affect this program’s evaluation.
SEAHS is partnered with JGHS. A handful of students go to JGHS to attend specific classes as
well as vice versa. In addition, JGHS has a school counseling program with a different
mission and vision statement, which impacts both JGHS and SEAHS students when
participating in school-wide activities and classroom lessons. Additionally, SEAHS is a
military and law enforcement school.
04
EVALUATION QUESTIONS
1. What is the overall effectiveness of the school
counseling program on students’ college/career
exploration, social development, and leadership
training?
2. What is the impact of the school counseling program
on its intended long-term goals for students
(responsible, productive citizens and lifelong
learners)?

The questions above were developed based on Southeast

Academy High School’s program description and logic model.

DATA UTILIZED
To answer the evaluation questions, we
utilized the following data:
Question 1
1. Norwalk-La Mirada Unified School
District Core Curriculum Action Plan -
Outcome Data
Question 2
1. Graduation Rates
2. A-G Completion Rates
3. Pathway Enrollment and Completion
4. College + Career Indicator - CA
Dashboard
 05
PROGRAM DESIGN
AN INTERRUPTED TIME SERIES (ITS), “MEASURES THE SAME OUTCOME FOR A
TREATMENT GROUP MULTIPLE TIMES BEFORE AND AFTER THE INTRODUCTION OF AN
INTERVENTION” (HALLBERG ET AL., 2018, P. 295).

This program design looks at the impact of a program or intervention over time.
First, one would attain the data before any intervention or program is
implemented for participants (students). The intervention or program is then
implemented, and participants’ outcomes are measured over time. The before and
after data is then utilized to determine if there was an impact from the program
or intervention itself. An interrupted time-series (ITS) design essentially provides
researchers with longitudinal data on the impact of an intervention (Hallberg et
al., 2018). Ultimately, we will be utilizing this program design, as it is the best
design to help us identify the answers we are looking to find in our program
evaluation.

The first evaluation question explores how the school counseling program
impacts students’ college/career exploration, social development, and leadership
training. In contrast, the second question attempts to understand how the school
counseling program impacts students’ ability to be responsible, productive
citizens and lifelong learners. In this case, the treatment group would be the
students at SEAHS who are receiving the services provided by the school
counseling program. The intervention itself is the school counseling program and
its specific services, whether the classroom lessons or other services across the
Tiers (Tier 1-3). ITS design ultimately allows us to see the impact on the school
counseling program over time. We will specifically be looking at data from 2018 -
2022 to analyze the rates in graduation, A-G completion, and pathway
completion.
 06
RESULTS: QUESTION 1
To answer the first question, we originally planned to analyze the outcome data from the Norwalk-La
Mirada’s Core Curriculum Action Plan. This action plan outlines all of the classroom lessons planned for the
year, with their intended outcome goals. However, due to time constraints and unforeseen circumstances,
we were unable to get a hold of the outcome data necessary to answer the first question of our program
evaluation - Norwalk-La Mirada Core Curriculum Action Plan. Outcome data from 2019-2023 is not
available due to the counseling team’s personal constraints and the overall impact of the COVID-19
pandemic. The missing data would have allowed us to analyze the impact of classroom lessons on outcome
data. The counseling team hopes to collect this data in the near future to determine the impact of
classroom lessons on achievement and achievement-related data.

TABLE 1
Front page of NLMUSD Core Curriculum Action Plan

Outcome data is not available for the following lessons:

9th grade: 10th grade: 11th grade Lessons: 12th grade Lessons:
3 College/Career 2 College/Career 2 College/Career 2 College/Career
Readiness lessons Readiness lessons Readiness Lesson Readiness Lesson
1 Academic Lesson 1 Academic Lesson 1 Academic Lesson 1 Social-Emotional
1 Social-Emotional 1 Social-Emotional 1 Social-Emotional Lesson
Lesson Lesson Lesson
 07
RESULTS: QUESTION 2
To answer the second evaluation question, we utilized SEAHS’ long-term data that includes graduation
rates, A-G completion, and pathway completion. This is the only long-term data currently available from
SEAHS.
Graduation rates: Data shows that there was a slight decline in graduation rates from 2020-2022.
Graduation rates resumed to 100% graduation rates in 2023. It is reasonable to infer that COVID-19
pandemic significantly contributed to the temporary decline in graduation rates, as the shift to virtual
learning environments may have impacted students' academic progress during this period.
A-G Completion rates: There was a 10% decrease of A-G completion from 2019 to 2021. From 2021 to
2023 A-G completion rates increased by 9%. Reasons for the increasing and decreasing rates are
unknown, but it is important to acknowledge the impact of the COVID-19 pandemic. Students may
have been less inclined to enroll in A-G courses during the specified period.
Pathway Completion Rates: From 2019 to 2020 pathway completion rates increased by 30%. From
2021 to 2023 there was a decline of 12% in pathway completion.

TABLE 2

Pathway
Graduation rates: A-G Completion:
Completion:
2019- 100% 2019- 42%
2019- 54%
2020-98% 2020-37%
2020-86%
2021-97% 2021-31%
2021-77%
2022-98% 2022-39%
2022-79%
2023- 100% 2023- 41%
2023- 74%
 08
RESULTS: QUESTION 2
To further explore and analyze SEAHS students’ college and career readiness, we intended to utilize data
collected from the California Dashboard - College/Career Indicator (CCI) reports. However, as previously
mentioned SEAHS falls into JGHS’s umbrella. For that reason, SEAHS’s data found on the California
Dashboard is nestled in JHGS’s data. Below, we have provided the California Dashboard College/Career
Indicator (CCI) Reports from 2018 and 2019 for JGHS, which includes SEAHS. There is no data available
from 2020 to 2021 due to COVID-19. In addition, there is no data available yet for 2022.

In 2018, roughly 33.6% students were prepared on the college and career indicator. Of all students, english
learners (6.8%) and students with disabilities (2.2%) were the least prepared, followed by socioeconomically
disadvantaged students at 33.7% prepared, Hispanic students at 34.2%, and homeless students at 32.9%.

TABLE 3 TABLE 4
CCI 2018 CCI 2018

TABLE 5
CCI 2018
 09
RESULTS: QUESTION 2

In 2019, 37.8% of students were prepared on the college and career indicator. 5.3% of students with
disabilities were prepared, demonstrating a 3% increase. In addition, 15.4% of english learners were
prepared, showing a 8.6% increase. 37.4% (3.2% increase) of Hispanic students, 36.5% (3.6% increase)
of homeless students, and 38% (4.3% increase) of socioeconomically disadvantaged students were
prepared on the college and career indicator.

TABLE 5 TABLE 6
CCI 2019 CCI 2019

TABLE 7
CCI 2019
 10

SUMMARY OF EVALUATION FINDINGS

Based on the evaluation findings from SEAHS

counseling team, it is evident that the school
counseling team has prioritized increasing graduation,
A-G pathways and pathway completion amongst
students. The significant increase in graduation rates,
A-G completion, and pathway completion plays a
pivotal role in fostering the development of students
in molding into productive, responsible, and lifelong
learners. This positive trend aligns with the SEAHS
school counselor’s mission/vision statement,
highlighting the school's dedication to preparing
students for lifelong success and well-rounded
development.

In addition, CCI reports indicate a drastic increase

from 2018 to 2019 in “preparedness” in college/career
readiness. Level of preparedness is evident for English
Learners; it is apparent that SEAHS/JGHS school
counseling teams have provided equitable access for
this specific population. It is clear that students with
disabilities continue to face challenges in becoming
ready for college or a career, based on the data
collected. While the data indicates a necessity to
concentrate/prioritize on this particular group, it's
crucial to acknowledge the positive impact the
counseling program has had on students in becoming
prepared for college/career development. Which
essentially contributes to students becoming lifelong
learners, productive, and responsible citizens.
 11
LIMITATIONS
In looking at the data utilized to answer the evaluation questions, its critical to
consider several limitations. Of significance, is the fact that the data on California
Dashboard for SEAHS is included in the JGHS data, even though both function as
separate schools and school counseling programs. Although there was an increase on
the % of students who were more prepared on the college and career indicator from
2018 to 2019, it would be essential to look at the disaggregated data for SEAHS,
without JGHS. We are also missing data points from 2020, 2021, and 2022 which can
provide more insight into the most up to which can provide more insight into the most
up to date information. Additionally, all of the data needed to accurately answer our
evaluation questions were not readily available (i.e. core curriculum action plan
outcome data). Therefore, we were unable to fully determine the imapct of the SEAHS
school counseling program on students’ college/career exploration, social development,
and leadership training.

RECOMMENDATIONS
Our program evaluation sought to determine the impact of the SEAHS school counseling
program on students‘ college/career exploration, social development, & leadership
training, as well as its effect on students ability to be responsible, productive citizens,
and lifelong learners. To answer these two questions, we attempted to analyze the data
gathered from the NLMUSD Core Curriculum Action Plan‘s outcome data, as well as
graduation, A-G completion, and pathway enrollment completion rates from SEAHS, and
the College and Career Indicator data from California Dashboard. Unfortunately, because
we were unable to gather the data from the core curriculum action plan, we strongly
urge future evaluations to look at the longitudinal impact of classroom lessons on
students. This data would give a significant insight into the overall impact of these
lessons, which function as a critical aspect of the school counseling program.
Additionally, currently the SEAHS school counseling team is planning to have alumni
focus groups. These focus groups would give the counseling team insight into students‘
experiences after high school. Future evaluations should strongly consider the
qualitative data gathered from these focus groups to determine the impact of the SEAHS
counseling program on students long-term. Furthermore, it may also be useful for future
evaluations to consider the data gathered from the senior surveys that all students
typically complete as seniors at SEAHS. Due to unforeseen circumstances in the SEAHS
counseling team at the end of the 2022-2023 school year, these surveys were not
administered. The data from these surveys could provide future evaluations with
information regrding students‘ college/career exploration, social development and
leadership training.
 REFERENCES
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2012). Program
evaluation: Alternative
approaches and practical guidelines. Pearson Education.

Hallberg, K., Williams, R., Swanlund, A., Eno, J. (2018). Short

comparative interrupted time series using aggregate school-level data
in education research. Educational Researcher, 47(5), 295-306. DOI:
10.3102/0013189X18769302.
 APPENDIX A

EVALUATION INFORMATION COLLECTION AND ANALYSIS

WORKSHEET

Adapted from Sanders, J.R., & Sullings, C.D. (2006). Evaluating school programs: An educators guide (3rd ed.). Thousand
Oaks, CA: Corwin Press.