Professional Documents
Culture Documents
A Sri Lankan Perspective
A Sri Lankan Perspective
A Sri Lankan Perspective
V.SIVAGNANASOTHY SECRETARY
MINISTRY OF TRADITIONAL INDUSTRIES AND SMALL ENTERPRISE DEVELOPMENT SRI LANKA sivagnanasothy@hotmail.com
Road Map
Development Evaluation - Sri Lankan Perspective Focus, types and approach to evaluation Evaluation Capacity Development Institutionalization of Evaluation Criteria for Selection and Evaluation Methodology Key attributes of good evaluation and quality standards Evaluation Feedback Planning, Budgeting and Policy making Process Institutionalizing Managing for Development Results
Focus of Evaluation
What works? What does not work? And Why? Under what context it works? or It does not work?.
On-going Evaluation / Mid Term /Formative Evaluation - Projects may run into problem during implementation - Requires fresh look (leads to modification, adjustments and revision of program).
Ex- Post Evaluation / Summative Evaluation - Some years after the implementation is completed. - Leads to decisions concerning expansion / adoption and future Impact Evaluation Considerable years after implementation is completed focusing on outcomes and long term impacts
Approaches to Evaluation
Goal Based Evaluation Goal Free Evaluation (Michael Scriven) Utilization-Focused Evaluation (Michael Patton)
ECD Initiatives in Sri Lanka The ADB TA for Strengthening Post Evaluation Capacity (1991/92) to MPI
Introduction of methodology, techniques, procedures for PE (manuals and guidelines) On the job training of senior government officials Sensitization of policy makers and senior government officials Dissemination of evaluation findings and establishment of feedback arrangements Development of Computerized Evaluation Information System ( EIS) for storing and retrieving Post Evaluation Findings Introduced Evaluation Module in SLIDA/PIM to orient government officials
Evaluation Information System (EIS) to support evidenced based decision making and learning A Data Base of Evaluation Information
Relevance
Efficiency Effectiveness
Impact
Sustainability
Rating Value
1. Relevance
20%
2. Efficacy
25%
3. Efficiency
20%
4. Sustainability
20%
15%
Highly successful (HS): Overall weighted average (OWA) is > 2.5 and none of the 5 criteria has a score of less than 2;Successful (S): OWA is between 1.6S 2.5 and none of the 5 criteria has a score of less than 1; Partly
Evaluation Methodology
Rating
Highly successful Successful Partly Successful Unsuccessful Judgment of the Achievement
Scoring
Evaluability Assessment
To assess whether a programme is evaluable. Determinants of evaluability are ;
1. Clarity of the intended programme model or theory, Programme goals, objectives and performance criteria/indicators are defined and agreed. 2. Availability of data on performance 3. Intended uses of evaluation information's are well defined.
Inputs
Funds Labor Technical Support Equipment
Output
Functional Scheme Strengthened FOs
Outcome
Improved Agriculture production through improved water Management
Impact
Improved Income and standard of living of the farmers
Evaluation Questions
A series of questions formulated to represent What one wants to know through the evaluation. Must Should and Nice to know
Evaluation question should be designed for each evaluation criteria.
Importance of Feedback
Utility of any evaluation is a prime criterion for judging
its worth, regardless of the technical, practical and ethical merit Making Evaluation Report Effective Evaluation to have an impact, needs to be disseminated and communicated to ensure behavioral changes and action Linking to the Planning, Policy Making, Budgeting and Programme Management
Dissemination Mechanisms
Abstracts and summaries a short description of the project or activity
evaluated and synopsis of key findings and issues, conclusion, recommendations made and lessons learned Annual Reports provide an opportunity to highlight findings, trends and lessons, to synthesize recent experience from a number of studies and to direct management attention to particular needs Synthesis Report Lessons learned over a number of years in particular sectors (eg health, roads) or on cross cutting issues ( eg; women in development, environment)
Bibliographies listing of evaluation reports manually or in computerized database Automated systems automated databases which increase the
accessibility of information from evaluations for various users, including project designers, policy and planning staff, management and evaluators themselves ( EIS). Feedback Seminars promote discussion of issues and lessons arising out of evaluations Press Conference & Presenting Evaluation Results
Institutional Mechanisms
Linkages to Senior Management evaluation reports be submitted to senior management ( higher authorities) for formal approval Linkages to Policy Development linkages between agency evaluation and policy units. Evaluation staff to be involved directly in new policy discussion Linking to Planning Link evaluation to project concept and project submission Linkages to Program Management- project approval documents must indicate how evaluation lessons have been taken into account in the design and planning of new activities. Linkages to Agency Procedures lesson from evaluation experience are incorporated into future agency policy and practices (eg; appraisal procedures) Linkages to Training lessons from evaluation are sometimes used in such programs ( e.g.; training on project design)
Synthesis
Inexpensive way of drawing from existing data ( make use of what is existing already) Translate technical findings into policy (Communication approach) Single evaluation findings may be challenging or unexpected, unlikely to be acted Enhanced the ability of evaluation to feed into planning, budgeting and policy making process
Institutionalize Feedback
People often strongly tempted to believe that the link between two activities of (a) building a stockpile of evaluation reports and (b) feedback are automatic Feedback calls for different kinds of skills ( more those of the communicator than the analyst) During feedback one is more likely to lose friends than to gain them. The courage to say what users may not want to hear is the characteristic of a honest evaluation function Feedback is not just distributing reports. Feedback has to be planned for and organized with as much care and determination
Institutionalize Feedback
Too often evaluation reports are written without adequate regard to who the audience is ( Supply-led rather than demand-led). Different audiences have different needs. If you want people to read your report you need to make them attractive ( eye-catching title) Evaluation reports should feed at a time when important decision are being taken, and not afterwards. Senior officials are too busy to read the reports. One page summaries of key findings are more effective. Computerize data base (EIS) is useful. Getting feedback institutionalized within the organization as a normal function is important
Early involvement of Audience(s) (Date they need Information, the areas of interest, formats etc.)
Framing the evaluation questions to meet the needs of audiences Tailoring the contents of Evaluation report to the Evaluation Audience(s) Delivering the Message Tailoring report format, Style and language (non-technical language)
Cover both Positive and Negative messages (Present Positive first and then Negative. People will be defensive when you place criticisms) Attractive to the Audience
Story-telling Use simple language (Avoid jargons) Print Quality
Results Focus
Inputs Activities/ Outputs No. of clinics built Outcomes/ Impacts Quality of health service delivery How many girls and boys are better educated Health Sector Funds disbursed
Education Sector
Funds disbursed
Focus on Results in all phase of development (Planning, Implementation, and Post Implementation)
Defining MfDR
shared vision, clear goals and measurement of results
KPIs
Increased in number of Palmyrah plants from 11Mn to 20 Mn in 2016 (MC 10 year Dev.Framework p6) through re planting and new planting. Jaggery Production (kg) from 6 Centres - Jaffna, Mannar, Trincomalee, Kilinochchi, Mullaitivu and Vavuniya Pulp Production (Litres) from 4 Centres - Jaffna,Vavunia, Mannar and Trincomalee Soft Drinks Production (bottles) from 3 Centres Jaffna, Colombo, Vavuniya Fibre Production (Kg) from 6 Centres Jaffna, Mannar, Mullaitivu,Kilinochchi, Katpity and Hambantota
Baseline 2009
11Mn
Target
2010
11.5
2011
12
2012
13
7496
9500
10500
11500
4251
8000
9600
10500
16756
45000
49000
54000
5660
7500
8250
9000
2010
23%
2011
22%
2012
21%
21.6%
Incidence of EPI target Diseases (TB) rate per 100,000 population Immunization coverage of infants against measles % of women in childbearing age practicing modern family planning methods Human Resources for Health Doctor Population ratio (Doctors per 100,000 population) Nurse/Population ratio (Nurses per 100,000 population)
48 97.1% 52.8%
42 100% 54%
40 100% 55%
38 100% 56%
55.1
75
78
80
157.3
160
165
170
Analysis
Where are we now? Performance Measures How did we do? Actions
Fundamental Prerequisites
Political will and Policy environment Govt.Policy on MfDR: Budget Call 2010; NARC Administrative Reform Agenda; Finance Commission Budget Call. Champions to lead the change management process Leadership at different levels of Government (Change Agent) MfDR Strategy and Action Plan Adoption of a process approach-consensus building Buy-in (LM/NB/NPD/AG)