HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Healthy Coyotes are Stronger Coyotes

Program Evaluation Plan

Prepared by:
Sarah Alkejek, Maylen Jackson, Athena Montiel, and Vanessa
Rangel

California State University, San Bernardino

June 7, 2017
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

1. INTRODUCTION AND STAKEHOLDER ENGAGEMENT

Evaluation Purpose
• The purpose is to evaluate the efficiency of the presentation of “Healthy Coyotes
are Stronger Coyotes” PSA and Zine to the CSUSB students during the Second
Annual Run like a Mother 5k. Furthermore, evaluate the outcomes, activities, and
all aspects of the program implementation.
• In addition, ascertain if all 200 participant students who attend the walk/
marathon were reached during the presentation.
• Then we will respond based on the results by adding more food resources or
creating more awareness among students to influence and meet the food
insecurity needs on campus.
• The findings will be used to improve the program’s implementation and its
effectiveness. It will also show the need of Healthy Coyotes are Stronger
Coyotes at CSUSB.
• In addition, it will be used to document the level of success in attaining the
objectives, to assess the need to increase more food donations for the DEN,
outside resources, or create more awareness among students.
• This evaluation fits with the student health centers goal because it is eliminating
this key health disparity among college students. Thus, promoting awareness,
resources, and better nutrition to students. To meet the student health centers
goal of “promoting good health on the road to academic and professional
success
Stakeholders
• The individuals and groups interested in the evaluation of our program are the
student health center, our food insecurity team, Dr. Verissimo, the Health
Science department, and the Delivering Emergency Nourishment (DEN) food
pantry.
• The point of view of the evaluation the groups are most interested in, aligns more
with the effectiveness of the program and finding creative ways to make students
aware of available resources on campus such as the DEN to reduce food
insecurity at CSUSB.
• The food insecurity team will hold the role of the planning team (collecting before
and after surveys or kahoot data, or collecting data on any other activity
conducted, analyzing the data, reporting results).
• The student health center role is receiving the results, and they are part of the
planning.
• Dr. Verissimo role is she is the overall supervisor of the activities, and receives
the results.
• The Health Science Departments role will be to support the planning team in the
efforts of the program evaluation and receive all the results.
• The Delivering Emergency Nourishment (DEN) the food pantry role will be
receiving the program evaluation results and providing us with data on enrolled
students.
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Table 1. Stakeholder Assessment and Engagement Plan

Stakeholder Name Interest or Role in the Evaluation
Perspective

{May be an individual or a {program {planning team, external reviewer,

group} participant, staff, etc.}
etc.}

Athena Montiel Staff-MPH Planning team, data collector,

student analyzing the data, reporting results

Vanessa Rangel Staff-MPH Planning team, data collector,

student analyzing the data, reporting results

Sarah Alkejek Staff-MPH Planning team, data collector,

student analyzing the data, reporting results

Maylen Jackson Staff-MPH Planning team, data collector,

student analyzing the data, reporting results

The Health Science Professors, and Receives results and part of planning
Department department staff

Dr. Verissimo Supervisor Receives results, and oversees

activities

CSUSB Health Center Resource/ Receives results and part of planning

support
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Resources/ Receives results, part of the planning,

The Delivering
support and collaborates by giving us data on
Emergency Nourishment
CalFresh enrollment and Den visits.
(DEN) the food pantry

Cultural Competence
• We will engage stakeholders who reflect diversity and are affected by the
evaluation findings by listening to their ideas on how to improve the program.
Moreover, we would only include ideas that would benefit the focus of the food
security program. We would additionally find more ways to improve the program
and affected group through outside resources that focus on food security among
nearby CSUSB communities.
• We will ensure to successfully elicit and incorporate the various perspectives by
ensuring to listen to other student’s ideas such as: single mother’s or father’s or
other in a different situation and through survey outcomes, or program
suggestion. Furthermore, by including others in held meetings, conversations
about our program as long as they benefit the set-out goals and objectives.
• We will account for the influence of context and culture in the evaluation design,
implementation and reporting by establishing an action plan that clearly includes
culture in the evaluation. Culture that relates to food insecurity, and the
individuals who experience it such as students who have families, students living
off campus on their own, or single parents that attend CSUSB. Furthermore, we
will examine how we can be better prepared for any type of influence, and
consider other alternative ways to incorporate that can account for any impact.
2. DESCRIPTION OF HEALTHY COYOTES ARE STRONGER COYOTES
Need
• Our program is trying to meet the nutritional needs of CSUSB students. It strives
to make sure that CSUSB students have the nutritional resources they need in
order to avoid food insecurity.
Context
• The contextual factors that might affect the implementation or effectiveness are
the lack of interest from staff to continue donating, not having enough donations
at the DEN, and a lack of student interest or involvement. Some cultural factors
that could affect the implementation and effectiveness are stigma and negative
emotions, such as fear and embarrassment among other emotions.

Population Addressed
• The population that the activities are intended for is the CSUSB campus as a
whole. The distribution of the zines and 5k run will target students and staff
members since they will provide both groups with essential information about
resources and donations. The PSA is meant for CSUSB students since it shows
them how to make budget friendly foods that are both quick and nutritious in
order to reduce the rate of food insecurity among students.
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Stage of Development
• The program has been in place for half a year. It took a quarter to create the zine
and PSA. Another quarter was used to contact organizations within CSUSB in
order to reach out to volunteers.
• The program is currently in the planning stage because the food insecurity team
hasn’t officially integrated Healthy Coyotes are Stronger Coyotes to the 5k run
and is still prepping for the actual event.
Resources/Inputs
• The food insecurity team (four MPH students) will present the material. No
money is actually needed for the presentation. There will be a booth that will be
used and an arrangement needs to be done in order to get a room that can hold
about 200 participants. A partnership has been made with CSUSB, which will
allow the presentation to take place at the 5k run. The promotion of the event and
program will be done on the CSUSB website. A computer, projector, and printer
will be needed for the PSA and the distribution of zines. Also, a bag of food from
the DEN will be used as grand prize. Food items (canned) and basic cooking
utensils will be used to do the cooking demonstrations.
Activities
• The activities that will take place will be a presentation to about 200 students, the
distribution of the zines, a short cooking segment, a food game, a resource
workshop, and a viewing of the PSA. Two of the graduate students (Vanessa
Rangel and Athena Montiel) will cook live, quick recipes that are both nutritious
and budget friendly. The food items used will be items that are available at the
DEN and at dollar stores. Volunteers will be leading an activity inspired by the
Price is Right. During this activity, students will get the chance to guess the cost
of certain food items (available at the DEN). The winner will win a bag of food
from the DEN. This activity will help students gain knowledge about cheap,
nutritious items and how to eat healthy on a budget. There will also be a
workshop led by Maylen Jackson and Sarah Alkejek in which students will learn
about local resources and other nutrition programs besides the DEN, such as
CalFresh.
Outputs
• The activities produce more awareness of food insecurity problem on campus
and more registrations at the DEN among other nutritional resources. It will also
incite more confidence among CSUSB students, which will make them feel more
comfortable in seeking out resources. Food insecure CSUSB students will also
have better academic performance. In general, CSUSB students will develop
healthier eating habits.
Outcomes
• The program intends to get more students registered at the DEN among other
nutritional resources. It also intends to get more donations from staff members
and create more awareness among the CSUSB campus.
• As a long term outcome, the program intends to improve the quality of life of
CSUSB college students by bringing awareness to food insecurity and promoting
resources
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

• The activities of the program will have worked successfully if the outcomes are
near of what is expected.
Logic Model
• See attached document appendix A.
3. EVALUATION DESIGN
Stakeholder Needs
• The student health center, the DEN, the food insecurity team, and the target
audience.
• They need to learn about the food insecurity disparity that exists among CSUSB
campus and the lack of awareness of the resources the campus offers like the
DEN. In addition, they need to learn about the attribution (if the outcomes were
attributable to the program).
• Intended users view credible outcome indicators, systematic observations,
changes in participant behavior, data collected, methods, and sources where the
evidence came from.
• To mobilize support from the student health center for our program or other
outside food security coalition's. In addition, the findings will be used to solicit
more food donations for the DEN at CSUSB, and to improve the programs
materials.
• Some of the evaluation capacities that will need to be built to engage the
stakeholders are: meetings with outside source (coalitions), conferences, policy
revisions, assessment of the food insecurity individual’s diversity.
• The stakeholders viewing or examining more data feedback on the food insecure
students at CSUSB.
• The stakeholders attending a planning teams meeting where they see how we
advocate for more food donations to stock the DEN.
• The stakeholders attending a community circle meeting where they examine how
the planning team is building stronger leadership skills to create more food
resources for the students and creating a stronger program.
Evaluation Questions
1. To what extent did the materials get delivered to CSUSB students during the 5K
presentation?
2. Did the program succeed in increasing more CSUSB students to visit the DEN?
3. Did the program succeed in increasing more CSUSB students to enroll for
CalFresh?
4. How much do the CSUSB students know about food insecurity?
Evaluation Design
• The design for this evaluation is a pre-post test.
• The rationale for using this design is that we are observing our intervention group
at one point in time (Run like a mother 5K run) before the presentation and after
the presentation. Furthermore, we are focusing on the target audience becoming
aware of food insecurity and available resources on campus. Moreover, we are
also focusing on the level of exposure our target audience has on food insecurity
and the knowledge gained on food insecurity on the day of the 5k run
presentation and this design will allow us to examine this.
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

4. GATHER CREDIBLE EVIDENCE

Data Collection Methods
• The program will rely on primary data to analyze our evaluation questions and
evaluate our program goal. After taking in account the various advantages and
disadvantages of different methods we chose multiple methods including
instruments to be completed by respondent, student surveys, and interviewing
The DEN supervisor.
• In order to assess the PSA presentation and Zine a pre and post test will be
conducted to measure the quality of presentation and to ensure the presentation
is delivering key points and information to the audience. Student surveys would
serve to measure dissemination of information from the presentation which would
lead to raised awareness regarding food insecurity and its impact on college
student and accessible resources at CSUSB. Collaborating with and interviewing
The DEN supervisor to find out if there has been an increase in student
visitations to The DEN, if staff was able to enroll them into long term programs
CalFresh, or if students asked about other resources like information on housing
support.
• Two samples will be needed for data collection: the first sample will include
CSUSB students attending the Run Like a Mother 5K event who will take a pre-
post test and the second sample will include a random selection of CSUSB
students who will take the student survey.

• An easy way to administer the pre-post test would be through Kahoot which is
accessible to any phone with internet. The quality and wording of questions for
the pre-post test, student surveys, and interview with The Den staff will be
created by a team of properly trained MPH students of which two team members
have a background in nutrition.
• The Food Insecurity Team will ensure an adequate sample size is obtained for
the student survey and there is no bias when choosing students to participate in
the survey through specific data collection procedures. All data will be
anonymous due to the stigma and sensitivity related to food insecurity and to
ensure the most honest response.
• Initially the first data will be collected on October 21, 2017 the day of the Run
Like a Mother 5K through a completed questionnaire. Participants will include
students who are attending the walk/ marathon which will include about 200
students. The student survey can be conducted in November of 2017 which
would serve to measure knowledge and awareness regarding food insecurity and
student accessible resources. The student survey will be taken by the Food
Insecurity Team and will include about 200 participants, all students at CSUSB.
The Food Insecurity Team will collaborate with professors in order to gain access
to students who can complete the survey. In October, the Food Insecurity Team
will meet with The DEN supervisor in order to gain access to data regarding
student visits to The DEN and CalFresh enrollments. The collaboration and
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

interview with The DEN staff will still need to maintain the confidentiality of the
evaluations participants. The Food Insecurity Team will not obtain any personal
information and will soley use information regarding the number of student visits
to The DEN and number of new enrollments into CalFresh.
Table 3: Evaluation Questions and Associated Data Collection Methods
Evaluation Question Data Collection Source of Data
Method

1. To what extent did the materials Pre-Post Test CSUSB students

get delivered to CSUSB students conducted by Food attending Run Like a
during the 5K presentation? Insecurity Team Mother 5K

2. How much do the CSUSB students Survey conducted by CSUSB student

know about food insecurity? Food Insecurity Team sample

3. Did the program succeed in Interview conducted The DEN Supervisor

increasing more CSUSB students to by Food Insecurity
visit the DEN? Team

4. Did the program succeed in Interview conducted The DEN Supervisor

increasing more CSUSB students to by Food Insecurity
enroll for CalFresh? Team
5. DATA ANALYSIS AND INTERPRETATION
Indicators and Standards
• Indicators are clearly defined and serve to measure the various program
components related to our program objective. The three main program
components indicators will assess are the PSA and Zine presentation,
dissemination of information regarding to food insecurity among colleges
students, and increased awareness of resources for food insecure students at
CSUSB. The indicators for the PSA and Zine will measure the delivery of
information regarding food security for the intended audience.The dissemination
of information from the presentation will measure an increase in knowledge of
food insecurity among students at CSUSB. An increased awareness of resources
would be measure by an increase in visitations to The DEN and increased
enrollments to CalFresh when visiting The DEN.

• The success of the program will be measure at three different times ensuring the
most effective program evaluation. Initially the first data will be collected on
October 21, 2017 the day of the Run Like a Mother 5K through a pre- post
test completed by 200 students which will require 70% accuracy on the post-
test. The student survey completed by 200 students will be conducted in April of
2018 and require 80% accuracy in order to The student survey will be taken by
the Food Insecurity Team and will include about 200 participants, all students at
CSUSB. Be marked successful. In October the Food Insecurity Team will meet
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

with The DEN supervisor in order to gain access to data regarding student visits
to The DEN and CalFresh enrollments. Success will include a 20% increase in
student visits to The DEN and a 15% increase in CalFresh enrollments which
occurred at The DEN.
Table 4. Indicators and Success
Evaluation Question Criteria or Indicator Standards
(What Constitutes
“Success”?)

1. To what extent did the The quality of PSA and 70% of questions on
materials get delivered to Zine presentation post-test will be
CSUSB students during the 5K answered correctly
presentation?

2. How much do the CSUSB 80% of questions on

Increase of knowledge
students know about food student survey will be
regarding food insecurity
insecurity? answered correctly
among colleges students

3. Did the program succeed in Increased awareness of A 20% increase in

increasing more CSUSB resources for food student visits to The
students to visit the DEN? insecure students at DEN
CSUSB

4. Did the program succeed in Increased awareness of A 15% increase in

increasing more CSUSB resources for food student enrollments
students to enroll for CalFresh? insecure students at that occur at The DEN
CSUSB
Analysis
• Multiple methods will be used in order to effectively measure indicators and
increase the accuracy of our conclusions. A qualitative analysis can be used for
the pre-post test and student survey in order analyze the quality of the
presentation, attitudes towards food insecurity, and awareness regarding the
issue. A quantitative analysis will be done to analyze the difference in student
visits to The DEN and the difference in enrollments to CalFresh that occurred at
The DEN.
Interpretation
• The Food Insecurity team will be involved in drawing, interpreting, and justifying
conclusions and findings will be shared with Dr. Verissimo. The group will then
share findings with the CSUSB Health Science Department, Health Center and
The DEN.
• Findings will be shared with Dr. Verissimo in order to ensure program findings
are concrete. Additionally, findings will be shared with the CSUSB Health
Science Department, Health Center, and The DEN in order to continue raising
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

awareness in hopes to expand our program or create innovative programs to

better service food insecure students.
6. USE AND COMMUNICATION OF EVALUATION FINDINGS
Use
• Evaluation findings will be used by the Food Insecurity Team (Athena Montiel,
Sarah Alkejek, Vanessa Rangel, and Maylen Jackson), CSUSB school officials,
the DEN, Student Health Center, and other program planners.
• The Food Insecurity Team can use the evaluation findings in order to see if the
program was successful and make any necessary changes. CSUSB officials can
use the evaluation in order to determine whether or not they would like to
establish the program as long term. The DEN can use the findings to see how
effective their food pantry is. Other program planners can use the evaluation to
determine whether they would like to adopt a similar program.
• The Food Insecurity Team is in charge of creating and monitoring an action plan
to guide the implementation of evaluation recommendations. A follow up is
needed with CSUSB officials to see if the program meets their standards in order
to implement it at the campus.
• The lessons that should be shared are whether the location for the presentation
was effective, if a bigger room is needed for the presentation, if more zines need
to be produced, if the zine needs to include more information, if the PSA was
effective, and if more volunteers are needed, These can be documented by using
pictures and in written format
Communication
• The evaluation stakeholders I would communicate with are CSUSB officials,
Student Health Center, and the DEN. They would both be invited to meetings, be
kept up to date on the status of the program, and they will also learn whether or
not the program was effective among the targeted audience.
• The Food Insecurity Team will communicate with evaluation stakeholders by
using emails, in-person meetings, and written reports.

• They are appropriate for CSUSB officials because the emails can be used as
reminders and as forms invitations since school officials communicate mostly
through email. In-person meetings will be used to communicate ideas and keep
them updated with the status of the program. Written reports will illustrate the
findings of the program and will also act as official documents that demonstrate
both the potential and effectiveness of the program.
7. EVALUATION MANAGEMENT
Evaluation Team
• The food insecurity team will manage and implement evaluation a month after
the 5K run.
• The food insecurity team will need to consider alternative strategies for data
collection. Also, the team must assess the degree of implication, and limitations
of program guidelines.
• Dr. Verissimo will provide external review of the evaluation plan to provide
feedback.
Table 5. Roles and Responsibilities of the Evaluation Team Member
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Individual Title Responsibilities

or Role

Student Health Analyze and evaluate The Student health center will collect

Center data to assess progress data

on campus to implement food insecurity programs

and support available resources. Also,

educating students about food insecurity

information.

MPH students Create, apply, and Collecting and evaluating data by giving

analyze data for out surveys (questionnaires) and in

evaluation person interviews at 5K run

Volunteers Support MPH students in Helping in gathering data and other

gathering data information regarding food insecurity to

participants at 5K run

Dr. Verissimo External Reviewer Reviews evaluation plan and provides

feedback

Data Collection Management

• Data will be collected at three different times to measure students’ knowledge
and awareness regarding food insecurity and accessible resources.
• MPH students will put up a presentation for 200 students; moreover the food
insecurity team will distribute Zines, and view PSA for participants.
• The food insecurity team will partner up with professors in order to increase
access to students who can receive and complete the survey.
• Student Health Center will oversee the conduct of the evaluation.
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Table 6. Data Collection Plan

Evaluation Question Data Collection Activities Person(s) Due

Method Needed Responsible Date

1.Did students’ Pre/post surveys Presentation, MPH Oct

knowledge of food giving out a PSA, Zine students and 2017-
insecurity and (The questionnaire to Volunteers Oct
DEN) or other students for 2018
resources on campus evaluation
increase after viewing
the presentation and
PSA?
How is it different from
previous
results?

2.Did behavioral Using surveys and PSA, Zine MPH Oct

intentions of the calculating students and 2017-
students change after numbers of Student Oct
giving out Zines and visitations to food Health 2018
viewing PSA? pantry (the DEN) Center
HSCI 614 Program Evaluation Plan Adapted from CDC Asthma Program Evaluation Guide

Communicating and Reporting Management

• The student Health Center, The DEN, and Dr. Verissimo are the audiences for
reporting the progress made on the evaluation and/or evaluation findings.
• The purpose of the communications with this audience is to implement “Healthy
Coyotes are Stronger Coyotes” program to raise awareness and expand our
program or create other innovative programs to better serve food insecure
students on and off campus.
• The most appropriate type of communication method to use with this audience
are in-person meetings, emails, and phone calls for dealing with confidential
information.
• The communication will take place during the evaluation process.
Timeline
• The planning and administrative tasks will occur throughout the evaluation
process.
• Training for data collectors will occur two weeks prior to the 5K run.
• Pilot testing data collection instruments will occur a month prior to the 5K run.
• Formal data collection will occur at the 5K run on October 21, 2017, the analysis
will occur on April 2018, and the interpretation tasks will occur by October 2018.
• After interpretation tasks occur on October 2018 the information will be
disseminated on November 2018.
• Foreseeable bottlenecks or sequencing issues would include: lack of participants
or volunteers, technological problems with (computer, projector, or printer), not
being able to get a room to do the presentation, and having limited time to collect
data.
Wrapping Up
• At the end of the evaluation we will acknowledge the contributions of the planning
team members and others who contributed to the successful implementation of
the plan by having lunch banquet and a recognition ceremony.
• Lessons learned in the course of implementing the evaluation will be
documented by creating an online journal and a confidential notebook detailing
program from beginning to end.
• Relevant documents, instruments, and data will be archived in computer files,
Student Health Center, and Community Engagement (The DEN).