Evaluationproposalfinal Patel

EVALUATION OF FITPAL, GROUP 3 1
Evaluation of the FitPal Intervention
Steven Davis, Sophia Garza, Pablo Herrera, Erin Hudgins,
Sola King, Meg McLaughlin, & Bhumi Patel
PUBH 4333: Evaluation in Public Health
Dr. Beth Lanning
May 1, 2021
Table of Contents
Executive Summary 4
Background 5
Description of Program 6
Mission Statement 7
Theoretical Framework 7
Purpose of Plan 9
Evaluation Goals and Objectives 9
Scope of Plan 15
Theory of Change/Logic Model 16
Proposed Budget 16
Program Staffing 18
Evaluation Tools 18
Formative Evaluation 21
Process Evaluation 22
Outcome Evaluation 25
Impact Evaluation 27
Conclusion and Recommendations 27
References 29
Appendices 31
Appendix A- Process Element Chart 32
Appendix B- Pre- / Post-Program Knowledge and Behavior Survey 34
Appendix C- Pre- / Post-program Survey for Self-Efficacy of Cooking 36

Table of Contents cont.
Appendices
Appendix D- Pre- / Post-Program Participation Self-Assessment Survey 37
Appendix E- Fidelity Checklist 38
Appendix F- Post-Course Comprehension Quiz 39
Appendix G- Participant Tracker Checklist 40
Appendix H- Online Satisfaction Survey 41
Appendix I- Evaluation Gantt Chart 42
Appendix J- Logic Model 43
Appendix K- Focus Group Questions for Participants 44
Appendix L- Focus Group Questions for Partners 45
Appendix M- Theory Utilization Checklist 46

Executive Summary
FitPal is a web-based educational program intended to reduce the cases of obesity in the
disabled adult population of Kent County, Texas. The FitPal intervention plan was designed to
address the issues of obesity in individuals with a disability through obesity prevention,
economic and social empowerment, and self-efficacy training. The purpose of the FitPal
evaluation plan is to measure the effectiveness of the program in reducing the incidence and
burden of obesity within the population of Kent County that is currently both disabled and obese.
The evaluation plan spans two years and aims to assess the program’s success in accomplishing
its short-term, intermediate and long-term goals.
The FitPal Evaluation plan includes elements such as pre- and post-program behavioral
and comprehension surveys, an engagement tracker, focus groups, and satisfaction surveys to
assess formative, process, and outcome evaluation of the FitPal program implementation.
Surveys were sent out by online instructors to program participants via the FitPal online
interface and a focus group was conducted to get feedback on the accessibility and coherency of
the program. FitPal surveys are designed to gauge the level of mastery that program participants
have gained during and after the completion of the FitPal course plan. The engagement tracker
was utilized by the evaluation coordinator to monitor website engagement analytics, reach, and
dose delivered/dose received. Impact evaluation was assessed through the use of post-program
comprehension quizzes, self-reported implementation assessments, and reported vital statistics
from Kent County. Qualitative and quantitative data was used by the evaluation coordinator to
determine if the FitPal program was successful in its respective goals and if further changes need
to be made to the program and its implementation.

Background
Individuals with disabilities suffer disproportionately from obesity compared to the rest
of the population. Factors such as limited mobility, impaired cognitive and bodily function, and
healthcare inaccessibility causes increased rates of obesity within the disabled population. The
levels of obesity in the population diagnosed with a disability are 38.2%, whereas the level of
obesity among individuals without a disability is 26.2% (Disability Impacts All of Us, 2020).
Furthermore, adults in the United States with a disability are 53% more likely to have
diagnosable obesity compared to adults reporting no disabilities (Fox, Witten, & Lullo, 2013). As
a multifaceted condition, obesity can contribute to the development of comorbid health concerns,
such as type II diabetes, high cholesterol, and coronary disease (Disability and Obesity, 2019).
Thus, obesity may decrease the quality of life within individuals with a disability.
In individuals with a disability, the treatment of obesity is complex and costly, creating an
economic impact on the healthcare system. The number of obese individuals with a disability is
nearly 61 million adults in the United States (Disability Impacts All of Us, 2020).
Proportionately, there is a high expenditure on obesity treatment for this population group: an
estimated 44 billion dollars per year in the United States (Disability Impacts All of Us, 2017).
Thus, obesity in the population with disabilities places an economic strain upon taxpayers and
the healthcare system (Disability Impacts All of Us, 2017). This high expenditure could be
diminished through targeted health intervention. To date, most of the documented interventions
have singularly focused on how obesity causes disability (Green, 2010; Thompson, 2010). Many
studies focusing on obesity in the special needs population have been clinically based. This focus
causes a variety of barriers, with the main barrier being reliable transportation to the clinic
(Rimmer, 2005). Thus, an intervention that targets obesity prevention in individuals with
disability is necessary to close this gap in research.
The FitPal intervention was designed to address the issues of obesity in individuals with
a disability through obesity prevention, economic empowerment, and self-efficacy training.
Through educational videos and guidance on the web-based application, the program
administrators will encourage nutrition-related health behavior changes in the target population.
Thus, the FitPal intervention will affect the rate of obesity in the target population, decrease the
economic impact of this condition in the healthcare system, and increase quality of life for
program participants.
Description of Program
FitPal is a web-based educational program intended to reduce the cases of obesity in the
disabled adult population of Kent County, Texas. The online site will showcase videos that
provide the audience with information and guidance regarding nutrition and healthy eating
habits. FitPal is intended to first serve the population of Kent County because of the substantial
proportion of people with physical disabilities compared to the general population of the area.
According to the United States Census Bureau, approximately 29% of the total population in
Kent County reported having a health problem or disability that limits their daily activities (U.S
Census Bureau, 2019). This finding translates to approximately 174 people that would
potentially derive benefit from the FitPal program.
FitPal is composed of five individual classes administered over the course of five weeks.
Course material ranges from the basics of healthy eating to low impact exercise plans.
Additionally, FitPal provides program participants with monitored communication boards to
easily message instructors, a vast collection of healthy recipes, and a “pantry check” feature.
Program participants will be given a wellness check before and after the program, and success
will be monitored through hip-to-waist ratio, weight loss, and levels of perceived satisfaction.
In conjunction with locally established organizations such as “Disabled Advocates of
Kent Country” and Texas Tech University, the FitPal program can quickly become established as
a trusted resource and tool for improving the health of the target population.
Mission Statement
The FitPal program will be implemented by a Texas Tech program leader with a
background in nutrition. The selection of this program leader is based on the institution’s
reputation for exceptional education and a demonstrated commitment to empowering the
surrounding community. The University Mission Statement reads, “The university is committed
to enhancing the cultural and economic development of the state, nation, and world” (Texas Tech
University, 2020, Mission Statement). Additionally, the Texas Tech Department of Nutritional
Science aims to directly impact the community by focusing on increasing the accessibility to
nutritional science for the general population.
Theoretical Framework
The Theory of Planned Behavior and the Self-Efficacy Model were used as a theoretical
framework for the FitPal program. The Theory of Planned Behavior consists of three
components: subjective norm, attitude, and perceived behavioral control (Ajze, 1991). The
utilization of this theory helps program implementers to inform an individual's behavioral control
by promoting healthy behavior and giving the participant perceived autonomy of their actions.
This theory was selected to understand a better attitude in participants towards financing a
healthier diet. In increasing the level of perceived behavioral control throughout the FitPal
program, participants will be empowered to make healthier meal choices. By giving the
participants control of their own eating habits, this theory further combats the preconceived
barriers and norms associated with obesity. Providing the FitPal users with the resources needed
to identify harmful dietary choices, the FitPal program will equip participants with the
knowledge and skills to develop a healthier lifestyle. The theory underscores the need for
intervention.
The Self-Efficacy Model includes the concept of self-efficacy or the belief in one’s own
abilities. Self-efficacy is defined as a personal judgment of “how well one can execute courses of
action required to deal with prospective situations'' (Bandura, 1997). Self-efficacy is a reflection
of confidence in the ability to exert control over one's own motivation, behavior, and social
environment. The Self-Efficacy Model has four concepts: “performance outcomes, vicarious
experiences, verbal persuasion, and physiological feedback”, that are used throughout the FitPal
program (Bandura, 1997). The self-efficacy construct was applied to the FitPal program to
implement better attitudes towards investing in a healthier lifestyle. These four concepts are
incorporated into the FitPal program to improve the participant’s health and increase their
self-confidence through the use of resources provided by the online website, such as available
masterclasses, self-weight tracking, and various wellness checkpoints. Through activities such as
the multiple sessions of nutritional masterclasses and low impact or wheelchair accessible
exercises, the participants will likely begin to improve their confidence and self-efficacy. They
will also begin to see physical progress with their bodies, and they will be encouraged to
continue these practices on their own. These changes represent the “performance outcomes” and
“physiological feedback” constructs of the Self-Efficacy Model. FitPal’s online interface
includes monitored discussion boards that allow participants to communicate with other
members enrolled in the same course and program, as well as instructors to provide feedback.
This interaction creates an environment that fosters mutual encouragement and sharing of
experiences, which is representative of the “verbal persuasion” and “vicarious experiences”
constructs of the Self-Efficacy Model.
Purpose of the Plan
The purpose of the FitPal evaluation plan is to determine the effectiveness of the
program in reducing the incidence and burden of obesity within the population of Kent County
that is currently both disabled and obese. The evaluation will be used by the evaluators to
determine if the FitPal program was effective in providing guidance on how to adopt a healthy
diet and how to increase physical activity while having a disability. Furthermore, the evaluation
plan will be used to rate the efficacy of these components in helping its participants reduce the
risk of obesity-related illnesses. A successful reduction in obesity-related diseases would
decrease the healthcare-related costs that the Kent County government has to cover through
social services. The successful reduction in the incidence of obesity amongst the disabled
population of Kent County, and the benefits that would derive from achieving that, are in
alignment with the mission statement of Texas Tech University to “…enhance the economic
development of the state, nation, and world” (Texas Tech University Mission Statement, 2020).
Evaluation Goals and Objectives
The aim of the FitPal program is to 1) increase self-efficacy, and 2) decrease obesity in
the target population. The evaluation plan is designed to evaluate the implementation and
outcomes of the FitPal program. The formative, process, outcome, and impact evaluations used
to evaluate the participants' progress can be found in the Process Element Chart in Appendix A.
The process evaluation will be measured in the following ways: fidelity, dose delivered, dose
received, reach, recruitment, and context. Each of these elements will help evaluate the success
of the program. The FitPal program was created to increase self-efficacy and awareness of the
components of healthy diet and local resources for healthy food options. Upon completion of the
program, participants will demonstrate increased self-efficacy and awareness.
Formative Evaluation Goals and Objectives
Goal 1: Pilot the FitPal program to ensure the website and materials are easily accessible to
participants.
1.1 Before the start of the program, select 10 program participants to pilot the FitPal
program.
1.2 Before the start of the program, have program participants take a Pre- and
Post-Program Knowledge and Behavior Survey prior to piloting (Appendix B).
1.3: Before the start of the program, compare piloteers results from Pre- and
Post-Program Knowledge and Behavior Survey and assess areas where program
participants did not improve (Appendix B).
1.4: Before the start of the program, have program participants take a Pre- and
Post-Program Survey for the Self-Efficacy of Cooking a Healthy Meal prior to piloting
(Appendix C).
Post-Program Survey for the Self-Efficacy of Cooking a Healthy Meal and assess areas
where program participants did not improve (Appendix C).
1.6: Before the start of the program, have program participants take a Pre- and
Post-Program Participation Self-Assessment Survey prior to piloting (Appendix D).

Post-Program Participation Self-Assessment Survey and assess areas where program
participants did not improve (Appendix D).
Goal 2: Collect feedback and evaluate changes necessary to make the FitPal program
user-friendly for all participants.
2.1: After the piloting phase, conduct a focus group with program participants.
2.2: Ask participants questions in the focus group to collect feedback and opinions on the
program’s usability.
2.3 Ask participants questions in the focus group to collect feedback and opinions on the
readability and comprehension of the FitPal program materials.
2.4. Make any necessary changes to the FitPal program based on piloting pre- and
post-test results and focus group answers.
Process Evaluation Goals and Objectives
Goal 3: To ensure FitPal classes were delivered as intended. (Fidelity)
3.1: By the end of month 23, the evaluator will assess each class based on the Fidelity
Checklist and each class will score a 7 or above on the checklist (Appendix E).
3.2: By the end of month 27, each class will score a 9 or above on the Fidelity Checklist
(Appendix E).
Goal 4: To ensure program participants had access to course material. (Dose delivered)
4.1: By the end of month 12, each participant will have logged into the FitPal program
website 2 times a week.
4.2: By the end of month 23, each participant will have viewed at least 75% of content on
the FitPal website.

Goal 5: Program participants are able to demonstrate course comprehension. (Dose Received
Satisfaction)
5.1: By the end of month 12, 80% of participants score at least a 70% or above on
Post-Course Comprehension Quiz (Appendix F).
5.2 By the end of month 24, 80% of participants score at least a 90% or above on
Goal 6: Program participants completed the course plan in its entirety. (Dose Received Exposure)
6.1: By the end of month 12, 70% of participants will have completed 50% or more of
Post-Class Assessments recorded by the Participant Tracker Checklist (Appendix G).
6.2: By the end of month 24, 90% of participants will have completed 100% of
Post-Class Assessments recorded by the Participant Tracker Checklist (Appendix G).
Goal 7: To ensure program target population received access to the program.
FitPal program participants interact with the FitPal interface on a regular basis. (Reach)
7.1: By the end of month 12, 70% of participants of the intended population will have
logged into the FitPal website at least 3 times a week, every week.
7.2 By the end of month 23, 85% of participants of the intended population will have
logged into the FitPal website at least 3 times a week, every week.
Goal 8: The recruitment process was effective in recruiting members of the target population.
(Recruitment)
8.1: 2 Months before the program start date, both Texas Tech University and Kent County
Department of Health Services will have utilized all forms of recruitment (websites,
social media, and television commercials).

8.2: 1 month before program start date, 3 organizations that already have an established
relationship with the Kent County population will be sharing recruitment materials to
recruit members of the target population.
Goal 9: Program participants demonstrate satisfaction with the course material. (Context)
9.1: By the end of the course, 60% of participants will report a 3 or above on the Online
Satisfaction Survey (Appendix H).
9.2: By the end of the course, 75% of participants will report a 4 or above on the Online
Satisfaction Survey (Appendix H).
Outcome Evaluation Goals and Objectives
Short Term Outcomes
Goal 10: Increase participants' self-efficacy in selecting and preparing healthy meals.
10.1: After the first two courses, 80% of participants will identify components of a
healthy meal through the Post-Program Survey for Self-Efficacy of Cooking a Healthy
Meal (Appendix C).
10.2: After one cooking course, participants will report an increase in cooking confidence
through the Post-Program Survey for Self-Efficacy of Cooking a Healthy Meal
(Appendix C).
10.3: By the end of the program, 70% of participants will be able to identify their top 3
common cravings through the Post-Program Survey for Self-Efficacy of Cooking a
Healthy Meal (Appendix C).
Intermediate Outcomes
Goal 11: Increase participants' knowledge about healthy food options.

11.1: By the end of the program, 75% of participants will identify the “healthiest” meal
out of an array of presented options.
11.2: By the end of the program, 65% of the participants will be able to substitute a
healthy food item for their most commonly craved unhealthy food.
Goal 12: Increase participants' comfort in cooking healthy meals.
12.1: By the end of the program, 75% of participants will report a 7 or more on the
Post-Program Survey for Self-Efficacy of Cooking a Healthy Meal (Appendix C).
12.2: Conduct interviews to assess self-efficacy levels of the participants for identifying
healthy meal options.
Impact Evaluation Goals and Objectives
(Long-term Outcomes)
Goal 13: Program participants learn healthy behaviors to obtain and prepare healthy food
options.
13.1: Six months after program completion, 80% of participants report using services,
such as food pantries to obtain healthy food options.
13.2: Six months after program completion, 80% of participants report cooking healthy
meals at least 3 to 4 times each week.
Goal 14: Program participants increase their overall health and well-being.
14.1: One year after program completion, 70% of program participants report a decrease
in their hip-to-waist ratio on the Post-Course Comprehension Quiz (Appendix F).
14.2: One year after the program completion, 70% of program participants will report
continuing their healthy eating behaviors learned after the program on the Post-Course
Comprehension Quiz (Appendix F).

Scope of Plan
The evaluation plan for the FitPal program consists of formative, process, outcome, and
impact evaluation and will be conducted over two years. The formative evaluation will measure
the website’s content and usability, along with program implementer’s feedback. Feedback will
be collected via focus groups, surveys, and the development of the Evaluation Gantt Chart
(Appendix I). Focus groups will be conducted with 10 program participants and this technique
will be the main form of data collection for the formative evaluation. Process evaluation will be
used to determine program feasibility by evaluating the implementation phase of the program. To
assess the effectiveness of the program, online discussion boards that give participants a chance
to give feedback, will be monitored to measure the participant’s satisfaction with the program.
Outcome evaluation will determine the program’s effect on the participant’s awareness,
knowledge, and skills regarding healthy eating and available resources for adults with
disabilities. The Pre- and Post-Program Knowledge and Behavior Survey will be used to assess
changes in outcome variables while participants’ responses to the educational videos and
embedded questions will be used to gauge improvement in knowledge and skills (Appendix B).
Lastly, impact evaluation will be used to assess the program’s effectiveness in increasing
participants’ self-efficacy, overall health, and well-being. An Evaluation Gantt Chart will be used
to monitor the hiring of employees, the creation of the website, and the overall development of
the program on a bi-weekly basis (Appendix I). The Post-Course Comprehension Quiz will be
used to collect some demographic information on participant’s age, weight, height, and
midsection measurements prior to and after the intervention (Appendix F). In addition, for the
course of two years, participants will receive follow-up Post-Program Knowledge and Behavior
Surveys every six months to assess their health status (Appendix B).
Theory of Change/Logic Model
The theory of change for this program is based on the framework of the Self-Efficacy
Model. The goal of the FitPal program is to increase the target population’s self-confidence by
increasing their knowledge in nutrition and lifestyle behavior. This knowledge will help the
target population to make healthier lifestyle choices. By using the Self-Efficacy Model and the
Theory of Planned Behavior, the program implementers gain the ability to positively shape the
participants’ behavioral control. Teaching the participants important health educational skills,
such as applying for health benefits, identifying components of a well-balanced meal, and
identifying and/or accessing treatment resources, will increase participant confidence in their
own skills and abilities. Through accessing this knowledge, the participants will achieve a
healthier lifestyle by the end of the program. Additionally, program implementers can utilize this
training to highlight that participants have a strong internal locus of control. In improving
efficacy, program participants gain the emotional confidence necessary to obtain and sustain
behavior change. Therefore, assessing changes in self-efficacy and knowledge will lead to
improved behavior and ultimately lead to a reduction of obesity in individuals with disabilities.
The Logic Model, found in Appendix J, is a visual description of the theory of change and the
projected outcomes of this project.
Proposed Budget
The FitPal budget includes durable equipment and paid personnel required to complete
the evaluation portion. In total, $78,095 is allocated in the FitPal evaluation budget. The
itemized budget table for the FitPal intervention can be found below.
Personnel
The Evaluation Coordinator from Texas Tech University will oversee all aspects of the
evaluation plan. The coordinator will receive a stipend of $15,600, or approximately $16.25/hour
at 20 hours a week for the 48-week duration. Four student workers from the Nutrition
Department of Texas Tech University will be hired to administer questionnaires and provide
instructional content to program participants. Each will receive a $15,000 stipend in addition to
relevant coursework experience. Each nutrition department student worker will earn $15.6/hour
at 20 hours a week for the duration of both the intervention and evaluation portions of the
program, approximately 48 weeks.
Durable Equipment
To complete both the intervention and evaluation portions of the FitPal program, workers
will need access to personal laptops. A total of four laptops are needed at a cost of $300 each,
totaling $1,200. $600 will be reserved for equipment upkeep and equipment repairs.
Additionally, the evaluation coordinator will be using NCSS Survey Data Analysis Software to
analyze the pre- and post-program questionnaires. The NCSS Survey Data Analysis Software is
$695. In total, $2,495 will be allocated in the budget for durable equipment.
Program Staffing
Organizing Staffing
The FitPal program will employ one grad assistant from Texas Tech University to act as
the program developer. FitPal will also employ four part-time staff members/instructors, also
known as student assistants, from Texas Tech University who have a background in nutrition.
These student assistants will oversee the development and instruction of course material to
program participants.
Online Staff Members
Online staff members will be responsible for developing and filming course material
before the final launch of the FitPal website. These individuals will administer questionnaires
and surveys to participants as well as encourage the use of online communication, recipe
searching and reevaluating program participants’ pantries.
Evaluation Coordinator
The evaluation coordinator will oversee the entire evaluation. They will screen and hire
online staff members and monitor the implementation process.
Evaluation Tools
The FitPal evaluation team used the Theory of Change program planning model to assess
the change in the incidence of obesity within the disabled population, the burden of disease for
those that are both obese and disabled, and the self-efficacy for the target population for nutrition
and exercise behaviors. The FitPal team will use several evaluation tools to evaluate the
effectiveness of the program.

Formative Evaluation Tools
An entry interview to collect a satisfaction rating from the participants about the survey and how
it can be improved.
1. An Evaluation Gantt Chart to compare the evaluation plan to the implementation plan
(Appendix I).
2. A Pre-Program Knowledge and Behavior Surveys to assess baseline knowledge of
program participants (Appendix B).
3. A Pre-Program Survey for Self-Efficacy of Cooking a Healthy Meal to assess baseline
cooking and nutritional efficacy of program participants (Appendix C)
4. A Pre-Program Participation Self-Assessment Survey to assess program participants’
baseline self-efficacy and perceptions of their nutritional and physical well-being
(Appendix D).
5. A focus group interview to elicit feedback about usability and ease of access to the
program (Appendices K and L).
Process Evaluation Tools
An Evaluation Gantt Chart to compare actual implementation to the implementation plan
(Appendix I).
1. A Participant Tracker Checklist to track the participants’ activities and participation in
the educational videos published by the FitPal team (Appendix G).
2. A data analytics engagement tracker to record post-program health metrics such as
midsection measurement and hip-to-weight ratio loss on the website, recorded by the

3. A data analytics engagement tracker for the program’s cooking and educational videos
(will be tracked by the number of views and/or clicks), recorded by the Participant
Tracker Checklist (Appendix G).
4. A Theory Utilization Checklist to ensure that the constructs of the Self-Efficacy Model
and the Theory of Planned Behavior are addressed in the intervention as planned
(Appendix M).
5. The comparison of the Evaluation Gantt Chart by FitPal staff members to evaluate
fidelity of partner, administrator, and staff training (Appendix I).
6. An Online Satisfaction Survey to track participants’ accessibility to the program
(Appendix H).
Outcome Evaluation Tools
1. A Post-Program Self-Assessment Survey to determine overall nutritional and physical
wellness of each participant (Appendix D).
2. A Post-Program Survey for the cooking course to determine level self-efficacy for
cooking a healthy meal (Appendix C).
3. FitPal Online Satisfaction Survey so the FitPal team can learn how to improve the
program for the participants (Appendix H).
4. Post-Course Comprehension Quiz for the FitPal staff to ensure the information is easy to
read and understand (Appendix F).
Impact Evaluation Tools
1. A Post-Program Self-Assessment Survey to gauge progress of participants' physical
wellness and nutritional self-efficacy (Appendix D).

2. Post-Program Knowledge and Behavior Survey for the cooking course to gauge the new
level of self-efficacy for cooking a healthy meal (Appendix B).
3. Vital health information reported by the Kent County Public Health department to
determine changes in the obesity rate of the target population.
Formative Evaluation
Formative evaluation will be conducted during the planning and testing of the FitPal
program. The formative evaluation will measure the website’s content and usability collected via
focus groups and surveys. A timeline, depicted in an Evaluation Gantt Chart, will be developed
and used to track program milestones (Appendix I).
The main goal of formative evaluation is to determine program feasibility. The website
will be pilot tested with a group of the participants prior to full program implementation. Focus
groups will be conducted with 10 program participants. Participants will have a chance to
interact with the FitPal website and assess the content and usability. Questions asked of the focus
group members will include “What knowledge did you learn from the FitPal website?” and
“What issues did you have interacting and using the FitPal website?” A full list of potential
focus group questions is located in Appendix K. Results from these questions will be used to
guide changes to the program and the website prior to the actual program start. With the website
and its educational materials being such an integral part of the program, it is vital that users can
easily access and fully understand the FitPal materials.
Additionally, feedback from selected individuals within FitPal’s partners, Texas Tech
University and Disabled Advocates of Kent County, will be collected to measure program
materials. Those individuals from partners selected to implement the FitPal program will be
asked questions via small focus groups to receive feedback on FitPal’s content. Selected
individuals will be asked questions such as, “Is FitPal content correct and education for FitPal’s
website?” and “Is information presented in a way that is easily understood for participants?” A
full list of potential questions for this focus group is located in Appendix L. Feedback from
program administrators will be collected to determine any aspects of the FitPal program where
materials are lacking and could be further developed. By collecting feedback from individuals
well-educated in the education and nutrition field, it ensures the FitPal program curriculum and
materials gives program participants the necessary knowledge to be successful and implement
change in their life.
Process Evaluation
Purpose
The purpose of the FitPal process evaluation is to determine if the program was
implemented as intended. The process evaluation will be conducted by a Texas Tech University
graduate student. This graduate assistant, also referred to as the evaluation coordinator, will be
responsible for evaluating program fidelity, dose delivered, dose received, program reach,
program recruitment, and context. The Process Element Chart detailing these items is located in
Appendix A.
Fidelity
Online program instructors were required to have a background in nutrition and exercise
science. The instructors collaborated with each other to develop and lead the FitPal nutritional
lectures. Online instructors will complete the Post-Program Knowledge and Behavior Survey
assessments and the Post-Course Comprehension Quiz to ensure that the information is easy to
read and understand, as well as in alignment with the course curriculum (Appendix B & F). The
evaluation coordinator will monitor the information given over the course plan using the Fidelity
Checklist (Appendix E) to assess and evaluate that the information was delivered as planned by
the online instructors and FitPal partners. Additionally, the Theory Utilization Checklist
(Appendix M) will contain the components of the FitPal program that address the constructs of
the Self-Efficacy Theory and the Theory of Planned Behavior and their intended implementation
method.
Dose Delivered
The FitPal evaluation coordinator will assess the dose delivered by measuring the
adherence to the program outline and the accessibility of the FitPal online components to
program participants. A Participant Tracker Checklist will be developed to ensure that all
program participants enrolled in the FitPal program have access to the program components,
such as the website, the education courses, and the support discussion board at the beginning of
the program (Appendix G). Additionally, the Participant Tracker Checklist will be used by
program evaluators to evaluate if the program remains accessible throughout the program. For
example, the monitoring of the checklist allows program evaluators to ensure that each online
course is accessible to program participants every Monday for five weeks. The evaluation
coordinator will use the checklist to evaluate if there are any issues in accessing and interacting
with FitPal materials that relate to this component. Additionally, throughout the implementation
phase, online instructors will be evaluated by the evaluation coordinator based on their
performance and the quality of the curriculum that is delivered.
Dose Received Satisfaction
Satisfaction of program participants will be documented after each class. Post-Course
Comprehension Quizzes and discussions will be held after each class to determine if participants
gained an understanding of the material, as well as determining their ability to apply program
information to their own lives (Appendix F). To measure the effectiveness of the information
taught, quiz scores will be evaluated and monitored for positive or negative trends. Data
analytics, addressed by the Participant Tracker Checklist (Appendix G), will also be used to
evaluate the overall engagement level of program participants. Components such as the number
of messages sent by participants on the messaging feature, number of video plays, and number of
clicks per day on the FitPal website will be monitored by the evaluation coordinator through the
use of the NCSS Survey Data Analysis Software.
Reach
Attendance will be monitored throughout the program using the Participant Tracker
Checklist (Appendix G). Participants will log into their account before they are able to view any
of the course material or access the quizzes, discussion board, and other website features.
Additionally, the FitPal evaluation coordinator will monitor the number of participants who
complete the FitPal quizzes and program, as well as the amount of online traffic the homepage
receives on a daily and weekly basis, on average.
Recruitment
Recruitment for the FitPal program will be sponsored through Texas Tech University
and the Kent County Department of Health Services. Both organizations will advertise on their
respective websites, social media, and television commercials and the efficacy of this marketing
will be measured using the Participant Tracker Checklist data analytics such as engagement,
views, and clicks (Appendix G). Recruitment advertising will begin four months prior to the
program start and will end two days before the first class begins. Recruiting through
organizations that already have an established relationship with the Kent County population will
be used to circulate information about the FitPal program to the greater community.
Context
Over the course of the FitPal program, program participants will be encouraged to
provide feedback about the lessons they completed. This information will be used to advise
future changes to the program. Additionally, program participants will have an opportunity to
rate their satisfaction levels with the classes, additional features, and the website interface. The
Online Satisfaction Survey is located in Appendix H. Through these surveys, the evaluation
coordinator and online staff members will work together to make changes to the course plan and
interface where needed. Additionally, the evaluation coordinator will monitor the number of
participants who completed all courses and quizzes associated with the FitPal program by using
the Participant Tracker Checklist (Appendix G).
Outcome Evaluation
The purpose of the outcome evaluation is to determine the program’s effect on the
participant’s knowledge, awareness, and skills regarding healthy eating and available resources
for adults with disabilities. Short-term, mid-term and long-term outcomes will be evaluated. The
data necessary for this evaluation will be collected via pre- and post-program knowledge and
behavior surveys, a pre- and post-program survey for self-efficacy of cooking a healthy meal, the
pre- and post-program self-assessment survey, and embedded video course questions.
Participants will complete the pre-test at the time of registration. The Pre-Program Knowledge
and Behavior Survey consists of ten questions intended to determine baseline intervention
awareness, knowledge, and skills of the participants (Appendix B). The post-test will include the
same questions as the pre-test, in a different order to avoid threats to internal validity of the tests,
to gauge the participants' improvement in these domains after completing the program
(Appendix B). The Pre- and Post-Program Survey for Self-Efficacy of Cooking a Healthy Meal
results will measure increases in self-efficacy related to cooking and nutritional literacy of the
program participants (Appendix C). Finally, the results of the Pre- and Post-Participation
Self-Assessment Survey will measure an increase in self-efficacy related to physical well-being
and behavior changes in program participants (Appendix D).
Short-Term Outcomes
The short-term outcome for the FitPal program is to increase participants’ self-efficacy in
selecting and preparing healthy meals. This outcome will be measured through the results of the
Post-Program Knowledge and Behavior Survey and the Post-Program Participation
Self-Assessment Survey (Appendix B and D).
Intermediate Outcomes
The intermediate outcomes are increasing participants’ knowledge about healthy food
options and increasing participants’ comfort in cooking healthy meals. Both outcomes will be
measured using the Post-Program Knowledge and Behavior Survey and Post-Program
Participation Self-Assessment results (Appendix B and D).
Long-Term Outcomes
The long-term outcomes of the FitPal program are to increase overall positive attitudes
toward healthy eating and to reduce the rates of obesity of people with disabilities in Kent
County. The first outcome will be measured using the Post-Program Knowledge and Behavior
Survey, online discussion boards, and the Post-Program Survey for Self-Efficacy of Cooking a
Healthy Meal to gather participants' feedback (Appendix B and C). The second outcome will be
measured using the results from the Post-Course Comprehension Quiz administered to
participants prior to and immediately after the intervention, as well as two years after the
completion of the program (Appendix F).

Impact Evaluation
The purpose of this evaluation is to measure the long-term effectiveness of FitPal in
improving the health status of its target population. This form of evaluation will be used to
determine if changes in behavior (diet and physical activity) and self-efficacy are sustained over
time. The participants who completed the program will be asked to complete the Post-Course
Comprehension Quiz as a follow-up survey after a 1-year and 2-year timeframe following
completion of the program (Appendix F). The data will be analyzed to determine if the behaviors
were sustained. Changes in self-efficacy will be measured by analyzing the reported participant
attitudes regarding the perceived impact of their behavior on their health as measured by the
Post-Program Participation Self-Assessment Survey (Appendix D). In addition, data from the
Kent County Public Health Department will be used to conclude if there are any significant
trends that resulted from participating in the program. Data from the surveys will be used to
make strategic adjustments to the program to increase its effectiveness in helping its participants
achieve sustained health.
Conclusion and Recommendations
The FitPal program was designed to help the disabled and obese population of Kent
County by providing this population with proper guidance and education regarding what a
healthy diet consists of, how to incorporate it into their daily life, and how to increase their levels
of physical activity with the aim of reducing the incidence and burden of obesity. The suggested
evaluation consists of four main parts: a formative evaluation designed to assess the strategy of
the program, a process evaluation to monitor delivery methodology, an outcome evaluation to
estimate the completion of short-term outcomes such as increasing participants’ self-efficacy in
their control of healthy food choices, and an impact evaluation to assess if changes in behavior
and self-efficacy are sustained over time amongst the disabled and obese population of Kent
County. The evaluation will be used as a resource to make strategic adjustments to the program
and increase its effectiveness in helping its participants achieve sustained health.
Recommendations
The most significant potential threats to the validity and reliability of this evaluation plan
are regarding non-response bias. Failure of the participants to respond to survey questions should
be addressed through case deletion, item replacement, or item deletion. Additional threats could
be attributed to evaluation instruments, participant instructions, and data management. Surveys
sent to participants as well as the instructions used should be pre-tested and piloted to the extent
possible. Doing so will allow for the confirmation of an appropriate reading level, and an
adequate length that fits the intended purpose. Additionally, feedback from surveys should be
reviewed constantly to allow the continuous improvement of the program.

References
Behavioral change models. (n.d.). Retrieved February 09, 2021, from
https://sphweb.bumc.bu.edu/otlt/mph-modules/sb/behavioralchangetheories/BehavioralC
hangeTheories3.html
Department of Nutritional Sciences. (2017). Texas Tech University: A Foundation for the Next
Century- Pathway to 2025 Strategic Plan. Retrieved February 6, 2021, from
https://www.depts.ttu.edu/hs/ns/docs/NS-Strategic-Plan-FINAL.pdf
Disability Impacts All of Us Infographic. (2020, September 16). Retrieved February 06, 2021,
from
https://www.cdc.gov/ncbddd/disabilityandhealth/infographic-disability-impacts-all.html
Disability Impacts ALL of US. (2017). Retrieved November 30, 2020, from Centers for Disease
Control and Prevention website:
https://www.cdc.gov/ncbddd/disabilityandhealth/impacts/pdfs/Texas_Disability.pdf
Fox, M. H., Witten, M. H., & Lullo, C. (2013). Reducing obesity among people with disabilities.
Journal of Disability Policy Studies, 25(3), 175-185. doi:10.1177/1044207313494236
Green, D. C. (2010, September 13). Second Circuit Weighs in on Obesity as a Disability - New
York State Human Rights Law (NYCHRL) Issues Remanded. Retrieved February 06,
2021, from
http://www.natlawreview.com/article/second-circuit-weighs-obesity-disability-nychrl-issu
es-remanded
Rimmer JH, Wang E. Obesity prevalence among a group of Chicago residents with disabilities.
Archives of Physical Medicine and Rehabilitation. 2005;86:1461–1464. doi:
10.1016/j.apmr.2004.10.038.
Texas Tech University Health Sciences Center Strategic Plan. (2020). Retrieved February 06,
2021, from https://www.ttuhsc.edu/administration/strategic-plan.aspx
Thompson, A. A. (2009). Obesity as a Disability under the Americans with Disabilities Act
Amendments Act and the Amendments' Effect on Obesity Claims under the Pennsylvania
Human Relations Act: Should Employers Anticipate a Big Change. Duq. Bus. LJ, 12,
259.
U.S Census Bureau (2019). Data.census.gov. Retrieved February 06, 2021, from
https://data.census.gov/cedsci/table?q=kent+county%2C+texas&tid=ACSDP5Y2019.DP
05&hidePreview=false
APPENDICES
Appendix A- Process Element Chart
Process Process Data Sources Tools/ Timing of Data Data Analysis/ Reporting (Purpose &
Evaluation Evaluation Procedures Collection Synthesis Details)
Element Question (months)
Fidelity Were the Fitpal Fidelity Evaluator will Months 5-23, Fidelity will be To ensure that the curriculum
classes Checklist assess each class 26-27 measured on a was delivered in the way
delivered as based on fidelity score from 0-10. previously intended.
intended? checklist.
Dose Did all of the Participant Evaluator will Months 5-23 Measured through To ensure that all participants
Delivered program tracker monitor participant the frequency of had equal access to all course
participants checklist login and activity logins to each class material and supplemental
have access to times based on the and activity time. information.
the course Website participant tracker
material? analytics and website
analytics.
Dose Did all Participant Evaluator will Months 5-23 Measured through To ensure that participants
Received participants tracker monitor class and completion of had equal opportunity to
Exposure complete the checklist assessment post-class complete all course work.
course plan in completion based assessments.
its entirety? on the participant
tracker checklist.
Dose Could Post course Evaluator and Months 24-25 Dose received will To ensure that participants
Received participants comprehension Instructors will be scored on a had lasting comprehension of
Satisfaction demonstrate quiz score and analyze 0-100% grading course material.
course participant quizzes scale of the
comprehension? after completion. post-course
comprehension
quizzes.
Reach Did program Participant Evaluator will Months 5-23 Reach will be To ensure that participants
participants tracker monitor login measured through interacted with and utilized
interact with the checklist information and the frequency of the online interface on a
Fitpal interface timestamps logins and average regular basis.
on a regular through use of the duration time of
basis? participant tracker activity.
checklist.
Recruitment Was the Participant Evaluator will Months 1-4 Recruitment will To ensure that the Fitpal
recruitment tracker assess the number be measured by program recruited as many
process checklist of visits and clicks comparing website members from the target
effective in on the website frequency and population as possible during
recruiting Website during and after consequent the recruitment process.
members of the Analytics the recruitment enrollment.
target period.
population?
Context Did program Satisfaction Evaluator will Months 6-23 Context will be To ensure that participant
participants rating survey assess the level of measured through satisfaction and feedback are
demonstrate participant the average being documented and
satisfaction with satisfaction quantitative implemented in the future.
course material? through analysis of satisfaction ratings
the satisfaction of the program.
rating survey.
Appendix B- Pre- / Post-Program Knowledge and Behavior Survey
1. How many days a week do you eat vegetables? _____
2. How many days a week do you eat fruits? _____
3. How much time do you spend exercising per week?_
____ Less than an hour
____ 1-2 hours
____ 2-3 hours
____ More than 3 hours
4. What is your level of physical activity?
___ None
___ Light (ex: walking slowly, working at standing work station, etc.)
___ Moderate (ex: brisk walk, bicycling light effort)
___ Intense (ex: hiking, jogging, bicycling fast, aerobic dancing, swimming,
running)
5. What is your current weight? ____
6. How much weight have you lost? _____
7. How easy is it for you to find recipes to make healthy food after completing the program?
____ Very easy
____ Somewhat easy
____ Somewhat difficult
____ Very difficult
8. How comfortable are you with your ability to exercise by yourself?
____ Very comfortable

____ Somewhat comfortable
____ Neutral
____ Somewhat uncomfortable
____ Very uncomfortable
9. What kind of exercises do you do? Please select all that apply.
____ Walking
____ Running
____ Weight training
____ Yoga
____ Biking
____ Other (please explain ____________________)
10. Has this program helped you reach your goals? Please explain how it has or has not. If you
have not started this program, you can skip the question.
Appendix C- Pre- / Post-Program Survey for Self-Efficacy of Cooking a Healthy Meal
Please circle the number that describes your confidence level of the following subjects. A 1
indicates that you are not confident in the area and a 10 indicates you are very confident.
1. How confident are you that you can identify a healthy meal on a menu?
No confidence 1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 Very confident
2. How confident would you be to identify healthy food options in a grocery store?
3. How confident do you feel about explaining the nutritional value of fruits and
vegetables?
4. How confident are you in your ability to prepare healthy meals in your kitchen at home?
5. How confident are you in reading food labels?

6. How confident are you in your ability to maintain a well-balanced diet?

7. How confident are you in your ability to know if food is fresh or expired?
8. How confident are you in cooking healthy foods that you think taste good?
In the space below, please write any additional comments or concerns you have about nutrition,
cooking, or maintaining a healthy diet.
Appendix D- Pre- / Post-Program Participation Self-Assessment Survey
1. How would you rate your capacity to make changes to your own health?
Very Bad Bad Neutral Good Very Good
2. How would you rate your nutrition?
3. How many times per week do you exercise for more than 30 minutes?
0 1-2 3-4 5-6 7+
4. How would you rate your ability to maintain good health?
5. How would you rate your knowledge regarding obesity?

Appendix E- Fidelity Checklist

Appendix F- Post-Course Comprehension Quiz
FitPal Questionnaire
Demographics:
Name: ________ Age:_____ Sex:_____
Weight:________ Height: ____ Midsection Measurement:_____
Questions:
1.Which of the following components should be included in a healthy meal? Choose all
that apply.
⬚ Protein ⬚ Sugar ⬚ Dairy ⬚ Whole grains ⬚ Vegetables ⬚ Fruits
2. Rank your familiarity with the following terms. One (1) indicates total familiarity with
the term and five (5) indicates no familiarity with the term.
___ Social Security ___ Calories ___ Macronutrients
3. On average, how many calories should you consume daily?

1,000 – 2,000 2,001 – 3,000 3,001 – 4,000
4. Indicate your level of agreement to the statement: I feel as though I can identify if a meal or
recipe is healthy or unhealthy.
⬚ Completely Disagree ⬚ Slightly Disagree ⬚ Slightly Agree ⬚ Completely Agree
5. Identify foods which should be avoided when possible. Choose all that apply.
⬚ Trans Fats ⬚ Whole wheat ⬚ Sugar based products
⬚ Foods with high sodium ⬚ Vegetables ⬚ Natural Sugars
6. Indicate how comfortable you feel cooking yourself a healthy meal.

⬚ Completely Comfortable ⬚ Slightly Comfortable ⬚ Not At All Comfortable
7. How often do you eat to satisfy a craving?

⬚ Very Often ⬚ Often ⬚ Not Often ⬚ Never
8. Indicate how familiar you are with resources to aid in healthy eating and obtaining
healthy food.
⬚ Completely Familiar ⬚ Slightly Familiar ⬚ Not At All Familiar
9. What is your goal for the rest of the course?

Appendix G- Participant Tracker Checklist
All information in this checklist will be gathered by online data analytics software. It will be
exported to an Excel file for further program analysis and for records.
Pre-Program
❏ Full roster of participants created and basic contact information collected

❏ Program usernames for participants using their provided email
❏ Rollout 1 of account set-up 3 weeks prior to program start
❏ Follow-up email sent to participants reminding them of account set-up 2 weeks prior to
program start
❏ Final roster of participants created and basic contact information collected
❏ Rollout 3 of account set-up 1 week prior to program start
❏ Final reminder email sent to participants day before program start
During Program
❏ Email sent to participants to provide program contact information in case of technical

difficulties or inaccessibility at the end of each week
❏ Monitor assessment surveys for curriculum efficacy
❏ Monitor satisfaction surveys of participants
❏ Monitor data analytics results including:
❏ Engagement level of activities
❏ Number of messages sent by participants
❏ Number of video plays
❏ Number of clicks per day on the FitPal website
❏ Number of quizzes completed by participants
❏ Number of programs completed overall
❏ Marketing efficacy for next round of the program:
❏ Advertisement engagement on the Texas Tech and Kent County
Department of Health Services websites and social media
❏ Advertisement views on the Texas Tech and Kent County Department of
Health Services television commercials
Appendix H- Online Satisfaction Survey

Appendix I- Evaluation Gantt Chart
Timeline
Month
1 2 3 4 5 6-23 24 25 26 27
Recruitment X X X X
advertising
Pre-program X
participation
survey
Satisfaction X
surveys
Post-program X X
participation
survey sent
Data coded, X X
cleaned, and
analyzed
Appendix J- Logic Model

Appendix K- Focus Group Questions for Participants
1. What information did you learn from viewing the FitPal website?
2. Did you have any issues using and interacting with the FitPal website?
3. What are some barriers that block you from using the internet or a computer?
4. When on the website, was there ever a point when you felt lost or overwhelmed?
5. After reading the information on the website, did you fully understand each part?
6. Based on the feeling you got from the survey, would you think that the FitPal team
would be willing to help if you asked?
7. What are some ways that the FitPal team enables you to use the website more often?
Appendix L- Focus Group Questions for Partners
1. Is FitPal content for FitPal’s website correct and educational?
2. Is information presented in a way that is easily understood for participants?
3. When you went through the program, was there a time that you felt the information was
contradicting itself?
4. What are some ways that the FitPal team can make the enrollment and implementation
process easier?
5. Is there anything you would like to see added or taken away from the program or its
website?
6. What are some adjustments you would make to the program?
7. Would you recommend FitPal to a friend or colleague?
8. What was your least favorite and favorite part of the program, its website, or its
curriculum?
Appendix M- Theory Utilization Checklist
Knowledge
___ FitPal programs gives participants education about healthy food options.
___ FitPal program participants learn how to make healthy nutritious meals in their
daily life.
___ FitPal program participants learning how to implement healthy behavior changes in
their everyday life.
___ Fitpal program participants learn how to access healthy food options (ie; food
pantries)
Behavior Change
___ Program participants are able to implement exercises into their everyday life.
___ Program participants are able to choose healthy food options
Self-Efficacy
___ Increase participants' self-efficacy to make healthy nutritious meals.
___ Increase participants' self-efficacy to make healthy lifestyle changes.

Evaluationproposalfinal Patel

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Evaluationproposalfinal Patel

Uploaded by

Copyright:

Available Formats

EVALUATION OF FITPAL, GROUP 3 1

Evaluation of the FitPal Intervention

Steven Davis, Sophia Garza, Pablo Herrera, Erin Hudgins,

Sola King, Meg McLaughlin, & Bhumi Patel

PUBH 4333: Evaluation in Public Health

Dr. Beth Lanning

Evaluation Goals and Objectives 9

Theory of Change/Logic Model 16

Conclusion and Recommendations 27

Appendix A- Process Element Chart 32

Appendix B- Pre- / Post-Program Knowledge and Behavior Survey 34

Appendix C- Pre- / Post-program Survey for Self-Efficacy of Cooking 36

Table of Contents cont.

Appendix D- Pre- / Post-Program Participation Self-Assessment Survey 37

Appendix E- Fidelity Checklist 38

Appendix F- Post-Course Comprehension Quiz 39

Appendix G- Participant Tracker Checklist 40

Appendix H- Online Satisfaction Survey 41

Appendix I- Evaluation Gantt Chart 42

Appendix J- Logic Model 43

Appendix K- Focus Group Questions for Participants 44

Appendix L- Focus Group Questions for Partners 45

Appendix M- Theory Utilization Checklist 46

its short-term, intermediate and long-term goals.

comprehension quizzes, self-reported implementation assessments, and reported vital statistics

to be made to the program and its implementation.

disability is necessary to close this gap in research.

a disability through obesity prevention, economic empowerment, and self-efficacy training.

potentially derive benefit from the FitPal program.

Additionally, FitPal provides program participants with monitored communication boards to

In conjunction with locally established organizations such as “Disabled Advocates of

reputation for exceptional education and a demonstrated commitment to empowering the

nutritional science for the general population.

“physiological feedback” constructs of the Self-Efficacy Model. FitPal’s online interface

experiences, which is representative of the “verbal persuasion” and “vicarious experiences”

constructs of the Self-Efficacy Model.

Purpose of the Plan

risk of obesity-related illnesses. A successful reduction in obesity-related diseases would

Evaluation Goals and Objectives

program, participants will demonstrate increased self-efficacy and awareness.

Formative Evaluation Goals and Objectives

Post-Program Knowledge and Behavior Survey prior to piloting (Appendix B).

participants did not improve (Appendix B).

where program participants did not improve (Appendix C).

Post-Program Participation Self-Assessment Survey prior to piloting (Appendix D).

Post-Program Participation Self-Assessment Survey and assess areas where program

participants did not improve (Appendix D).

user-friendly for all participants.

readability and comprehension of the FitPal program materials.

post-test results and focus group answers.

Process Evaluation Goals and Objectives

Goal 3: To ensure FitPal classes were delivered as intended. (Fidelity)

website 2 times a week.

the FitPal website.

Post-Course Comprehension Quiz (Appendix F).

Post-Course Comprehension Quiz (Appendix F).

Post-Class Assessments recorded by the Participant Tracker Checklist (Appendix G).

Post-Class Assessments recorded by the Participant Tracker Checklist (Appendix G).

Goal 7: To ensure program target population received access to the program.

social media, and television commercials).

recruit members of the target population.

Satisfaction Survey (Appendix H).

Satisfaction Survey (Appendix H).