PHH3061

Week 3
Process Evaluation
 AT 1
• Discuss and show examples
• Due date: 18 April
• Use template and projects on Moodle
Assessment title: Evaluation design exercise
Alignment with learning outcome(s): This exercise is linked to learning outcomes
1-5
Details of task: For each of the projects that are presented as AT1 Project 1 and
AT1 Project 2 in the Assessments folder on Moodle,
write evaluation questions for two of the objectives and for two of the strategies
that are given. Select an evaluation design to answer
each of the four questions developed for each project. Comment on the
strengths and weaknesses of each design, and make note of
any steps that would help these to be conducted in a way that would provide
data of good quality.
Release date (where applicable): week 2
Due date: 18 April
Word limit: 2 pages, plus references, using the template provided on Moodle
Value: 10%
Presentation requirements: Word document, 12 pt. font, min. 2cm margins
Estimated return date: Within 2 weeks
Hurdle requirements (where applicable): Not applicable
Individual assessment in group tasks (where applicable): Not applicable
Criteria for marking: Please see the detailed marking rubric in the AT1 folder on
the Moodle site.
Going back to week 2! Formative evaluation

• Remember…formative evaluation occurs as

part of programme planning and BEFORE any
elements of the programme are implemented.

• Process of participatory planning with

stakeholders and participants – key methods
and materials are identified.
Consulting project participants and partners

• Participants – target group

• Partners – have a role in delivery
 When designing formative questions, use
these…
Participants
• Perceived meaning
• Perceived relevance
• Interest and attractiveness
• Acceptability (cultural
appropriateness)
• Convenience (location,
timing)
PARTNERS
• Time and resources
• Complexity and ease of
deliverance
• Perceived usefulness (value
added)
• Clarity and adequacy of
programme protocols
 Strategy one
Establish working group with Council to
develop neighbourhood food access policy

• Formative ?: Does Council have the necessary

resources to participate in a working group? ?
is it feasible (practical, realistic, achievable,
possible)
• Interview with Council management
 Strategy 2
Work with Council to lobby the Local Member
and Department of Transport about bus
services available for the housing estate.
• ? Does Council have the necessary skills to
lobby the Local Member and Department of
Transport about bus services available for the
housing estate
• Method: Interview with council practitioners
 Strategy 3
Set up home delivery services of fruits and vegetables
in several areas by working with local retailers.
• ?How likely is it that residents of the housing estate
will use a home delivery service
• Method: focus group interview with housing estate
residents
• ?Does the local retailer have the time and resources
to set up a home delivery service?
• Method: Focus group interview with local retailers
 Strategy 4
Engage the Pricewatch group to incorporate healthy
foods onto their list of foods regularly monitored, with
regular publication in local newspapers of healthy vs.
less healthy alternatives
• ? Will the Pricewatch group be able to/willing to…? Can
Pricewater group incorporate..? Will the local
newspaper be willing and interested..?
Discuss these answers and why incorrect?
• What language is appropriate?
• Be specific…Interview…with whom?
 Strategy 5
Conduct workshops with school representatives, followed by
information materials, monitoring and an accreditation system
to increase the proportion of schools which meet healthy
canteen guidelines
? To what extent is the school representatives willing to meet for
workshops? Is it realistic and practical to work with school reps…
Method: ? Meeting with student gov body?
But not: how can the proportion of schools that meet healthy…
would..increase the number…
Can focus on available times for workshops etc.
?Are the local schools willing to participate in this intervention?
Remember-discussions and meetings not proper design methods!
 Strategy six – same apply
• Conduct workshops with sporting clubs and
facilities representatives followed by
information materials and an accreditation
system to increase the proportion which meet
healthy food guidelines
 Strategy 7
Undertake point-of-sale promotions of healthy
foods at schools and sport and recreation facilities.
• Look at criteria and try now!
• Can’t ask yet…will point of sale promotions increase? - impact and
outcome
• We can ask?..are there appropriate resources available to undertake…
• Can’t focus on health outcomes or impact yet!
• Can we ask if it will be effective yet?
• If you ask if it will be feasible..be more specific
• Can ask about resources, etc.
• Remember if you want to do a pilot of a strategy, keep it small…
Based on guidelines, complete the rest!
8. Implement school breakfast program in schools in and
around the housing estate area.
9. Place media articles in local newspapers which draw
consumer attention to the value of healthy food and the
work of the project.
10. Deliver evening seminars and distribute nutrition
information and food preparation tips to parents through
schools and through council newsletters.
11. Undertake cooking demonstrations at neighbourhood
centres and community festivals and open days.
WEEK 3 – PROCESS EVALUATION 1
• Process evaluation is a set of activities
directed towards assessing progress in
PROGRAMME IMPLEMENTATION and once the
programme has actually started.
• It is broad!
• Process evaluation is linked to strategies
 What are some of the reasons that process
evaluation is important in health promotion?

Group work!
Complete first worksheet activity and give
feedback
 Possible answers
• To understand and improve implementation of
the intervention and enhancing best practice
• Provide information that can be used to enhance
programme quality
• Help programmes be accountable to stakeholders
and funders
• To help expand the service/programme
• To show effectiveness of programme
management and how resources are used.
 What aspects of implementation of project
strategies can be implemented?

• Group work
• Refer to reading 3.1
• Complete second activity in worksheet and
give feedback
 Answers
• Context – aspects of the environment – influence
programme – barriers and facilitators
• Reach – % - how may and characteristics
• Dose delivered – HOW MANY number of training
sessions
• Dose received – EXPOSURE active engagement in train
sessions
• Fidelity – quality – is it in line with plan?
• Implementation –
• Recruitment – procedures used to recruit
Components of Process Evaluation
Aspects of process evaluation
• Delivery
• (what is done by project staff)
• Reach
• Exposure – engage!
• Context (barriers and
facilitators)
Process evaluation components
• Fidelity (quality)
• Recruitment
• Dose delivered (number of)
• Reach (participant rate)
• Dose received
• Context
 Reading 3.2
Week 3 case study
• Case study includes all of the components of process,
but we will focus on delivery and context.
• GROUP WORK
• Read the case study – Adolescent Social Action Plan
(read from Programme Evaluation Design and
Methods) and complete the table in your worksheet.
• Prepare to give feedback.
• Make sure to read the notes in your
worksheet
 Answers
Aspects of strategy delivered
• Was the curr implemented
with fidelity/quality to
original design
• Was the curriculum
implemented similarly to
each group (dose delivered)
Data collection method
• Observation and recording
• Focus group with students
• Facilitators
logs/recordkeeping /project
monitoring form
 Answers
Aspects of strategy context evaluated
• What were the barriers and
facilitators to participation?
• What were the competing
and intervening influences
on participation and
exposure?
Data collection methods used (for both)
• Interviews and focus groups
• Examination of documents )
reports, minutes of
meetings
• Surveys
• Observation
 Homework to be completed in class
• Page 13 – worksheets
• Selecting methods for evaluation strategy delivery and
context
• Consult your Weeks 2-5 Project Example (Outer Western
Sydney Food Project).
• Develop questions concerning delivery and context that
correspond to at least three of the strategies and select
methods.
• Group 1: strategies 1 -3. Group 2: strategies 4-6, Group 3:
strategies 7-9, Group 4: strategies 10-11 and group 5 –
strategies 1-3 etc.
Evaluation terminology