FAMILY RESEARCH NETWORK

FORUM
Fri 26 March 2010

1
 Evaluation Processes of
NUR Drop-In-Centres:
A Case Study of a Community
Leaders’ Forum ((CLF)) Project
j
Sharifah Sakinah Alkaff (Mdm)
Director, Youth Development
Yayasan MENDAKI

Presentation Outline

1. Yayasan MENDAKI

2. Community Leaders forum (CLF): An Overview

3. CLF Strategic Thrusts

4. NUR Drop-in-Centre: Evaluation Process

5
5. E l ti Fi
Evaluation Findings
di

6. Challenges and Learning Points

7. Moving Forward and Conclusion

2
 Yayasan MENDAKI

o Founded in 1982 to VISION

address low educational
achievement within the Community of
community. Excellence

MISSION
o Expansion of assistance to
areas of family, youth and To navigate, empower and
employability. p
position the
Malay/Muslim
o MENDAKI: agent of change Community at the
in the community. forefront of excellence

Community Leaders’ Forum (CLF):

An Overview
o Set up in 2003 by Malay Members of Parliament to discuss issues and
strategies to uplift the community.

o The Community Leaders’ Forum (CLF) aims to:

1. provide a platform for collaboration within the MMVS

2. foster community engagement

3. enhance the capacity of the MMVS

o The CLF is supported by the four Sectoral Networks of Education,

Youth, Family and Employability.

o Yayasan MENDAKI is Secretariat to the CLF.

3
 CLF Programme Map

Youth Devt Family Devt

Network Network
•Youth In Action •Family
Integrated Excellence Circles
• Programme for •Core Parenting
Teenagers Community Series
Leaders’
Education Forum Employability
p y y
Network Network

•Tiga M- Maju •Job Ready

Minda Matematika •Job Steady
•M-Power

CLF – Strategic Thrusts

1. Maximise resources
o Leverage
g on national initiatives
o Align existing programmes to achieve better synergy
o Focus on upstream efforts

2. Build capacities
o Research to understand issues affecting
g communityy
o Conduct training and sharing of best practices
o Evaluate CLF programmes to determine effectiveness
and relevance
o Empower youths and families with relevant skills

4
 INTEGRATED PROGRAMME FOR
TEENAGERS (NUR)
o Initiated in 2006 to provide holistic intervention for
teenagers & their parents

o Services included:
– Helpline (NURteensLINE)
– NUR On-
On-The-
The-Move (Public education
through road shows in schools)
– Drop
Drop--in-
in-Centres (NUR DIC)
– Sheltered home for unwed pregnant
teenagers

Implementation Approach
Enablers:
1. MENDAKI
Staff

2. CLF
2
Research & Development Partners
Planning of Content

Beneficiaries: 3. Students
NUR DIC
To provide safe and
1. Students supportive 4. Parents
Selling Delivery of
environment for
Results Content
youth
2. Parents

Evaluation

5
 Importance of Our Evaluation

1. Support for programmes

o Government Grants
o Community Funds/Donations

2. Transparency and accountability

3. Enhance capacities
o Ensure relevance
o Identify gaps for improvements

Evaluation – Logic Model

6
 Evaluation Process
Key Considerations

1 Criteria
1.
o CLF programmes that were implemented for at least 2 years
o Pilot programmes
2. Data Source
o Enablers & Beneficiaries
3. Methodology
o Qualitative & Quantitative
o Target sample size of least 30% of beneficiaries
4. Level of Outcomes
o Initial Outcomes : Knowledge, attitudes, skills (1 – 6mth)
o Intermediate Outcomes : Behaviour (6 – 12mth)
o Impact Outcomes : Condition / status(> 12mth)

Evaluation Objectives

INITIAL OUTCOME (1-6mths)

o To evaluate effectiveness of Youth Workers (YW) in NUR

Drop In Centres (DIC)
- Why ?
- Determine the level of confidence in YWs to discuss
issues
- Identify service gaps for further improvement
- How ?
- Questionnaires and Feedback from clients
- Self assessment of youth workers

7
 Evaluation Objectives

INITIAL OUTCOME (1-6mths)

o To evaluate effectiveness of publicity efforts

- Why ?
- Measure the level of outreach and cases
handled by NUR DICs
- Identify relevant channels for publicity

- How ?
- Questionnaires and Feedback from clients

Evaluation Objectives

INTERMEDIATE OUTCOME (6-12mths)

o T
To measure the
th effectiveness
ff ti off Y
Youth
th W
Workers
k
- Why ?
- Clients’ compliance/commitment to counselling
- Level of awareness on parental responsibilities

- How ?
- Youth Workers’ reports on clients

8
 Evaluation Objectives

IMPACT OUTCOME (>12mths)

o To determine the impact of services rendered

- Why ?
- Measure any positive change in behaviour
- Measure improved communications with parents

- How ?
- Questionnaires and Feedback from clients

Evaluation of NUR Drop-in-Centre

INITIAL OUTCOME FINDINGS

o To measure the o The level of service rendered by Youth Workers was

effectiveness of the rated higher in 2009 as compared to 2008.
Youth Workers o This is a reflection on the level of confidence that
clients placed on NUR DICs and our youth workers

100 100 100

100
96.3
95.2
96
92.6
92
%

88

84

80
Y W inter es ted to Y W is c onf ident in
lis ten to is s ues dis c us s ing is s ues

2007 2008 Jan- Jun 2009

9
 Evaluation of NUR Drop-in-Centre

INITIAL OUTCOME FINDINGS

o To measure the o Steady increase noted in the number of referrals from

schools and agencies
effectiveness of the
publicity efforts from o Indicates success in publicity efforts and confidence
Jan to June 2009 in NUR DICs

22.1
Schools/Agencies 13.8
12
60.5
Direct Call 61.5
49.2
6.9
Walk-Ins
Walk Ins 10 4
10.4
25.2
10.5
NURteensLINE 14.3
13.6

0 10 20 30 40 50 60 70
%

2007 2008 2009

Evaluation of NUR Drop-in-Centre

INTERMEDIATE OUTCOME FINDINGS

o To measure the o Improvement in the effectiveness of NUR DIC Youth

Workers since 2007; the largest increase being the
effectiveness
ff ti off th
the
percentage of clients committed to seek counselling
Youth Workers
o 100% of Clients’ expectation were met whilst 97%
would use the services again

69.6
Complied to counselling session 68.1
67.9

73.9
Committed to seek counselling 47.7
47 7
48.7

Improved awareness on parental 71.1

60.2
responsibilities
59.8

0 10 20 30 40 50 60 70 80
%

2009 2008 2007

10
 Evaluation of NUR Drop-in-Centre

IMPACT OUTCOME FINDINGS

o To measure the number
o Overall, there was improvement in the services
of youth that turn into
rendered since inception in 2006
socially-well adjusted
persons o Increase in the number of clients who managed to
achieve the desired outcomes to remain in school,
improved communication with parents and positive
change in behaviour
9 5 .5
100
7 8 .3
80 7 3 .1 7 0 .5
5 8 .8 6 1 .3
60

%
40

20

0
2008 2009

Re ma in in s c h o o l/e mp lo y me n t
Dis p la y imp r o v e c o mmu n ic a tio n w ith p a r e n ts
Po s itiv e c h a n g e in b e h a v io u r

Challenges

No Challenge Measures taken

1 Data C
D Collation
ll i
a. Collecting data  Strengthen post-
from clients counselling processes
through youth
workers

b. Collecting data  Sharing of evaluation

from Drop-in- findings with partners
Centres to
MENDAKI  Set and disseminate
subsequent evaluation
requirements

11
 Challenges

No Challenge Measures taken

2 Performance
Indicators
a. Ensuring KPIs are  On-going annual reviews
relevant and to ensure relevancy of KPIs
benchmarked and programme

Learning Points

No Learning
g Point Action Plan

1 The need for after-

service monitoring
a. Tracking behavioural  Referrals to relevant CLF
changes in the youths programmes under the four
Sectoral Networks

12
 Learning Points

No Learning Point Action Plan

2 The need to address
disparity in the level of
youth worker services  Formal
a. Capacity of youth training/certification
workers
 Monthly sharing of best
b. Casework
b C k practices
management  Clinical supervision: Youth
Worker Coordinator
 Youth worker auditor

Moving Forward

Ensure (4) Clarify Outcome

((1)) KPIs
Relevancy I t
Improvements Indicators

Programme
Evaluation

Measure Identify Data

((3)Evaluation
) (2) Data requirements
Effectiveness
Eff i

Planning is key to Programme Evaluation

13
In Conclusion…

Through our rigorous

evaluation processes,
processes
MENDAKI ensures that NUR
Drop In Centre continues to
be:
1. Relevant to current
youth issues
2. An effective outreach
and intervention
programme

Thank You!

14