Development Project

Planning 6

Monitoring and Evaluation

Data Sources
 Ground Rules

 The point is
1) for you to get benefit from the process
2) not to disturb other people

 What are the rules?

 This Session

 Recap – Indicators

 Mathematics

 Data Systems and Collection

 Levels of Indicators
 What is an Indicator ?

a variable …
that measures one aspect of a project
that is directly related to the program’s
objectives.
 DATA SYSTEMS

Components of a clearly defined Data System:

o multiple, well defined indicators
o a variety of appropriate data sources
o baseline and target values
o feasible data collection plan and budget
• specified frequency

• identified responsibility
LEVELS OF DATA

 Policy or Program

 Population

 Service Environment

 Client

 Spatial / Geographic
 DATA SOURCES AND TOOLS
POLICY/PROGRAM LEVEL

Gives information about the ‘big picture’

 The context, the nation or region
 How countries compare to each other
 DATA SOURCES AND TOOLS
POLICY/PROGRAM LEVEL
Sources:
 Official documents (e.g. government reports,
legislative and administrative documents)
 National budgets or other accounts data
 Policy inquiries
 Reputational rankings (e.g. program effort scores)
Tools:
 Index questionnaires (give rankings e.g. morbidity)
 Special and contract studies
 DATA SOURCES AND TOOLS
SERVICE ENVIRONMENT LEVEL

Gives information about how the services

are being run
 DATA SOURCES AND TOOLS
SERVICE ENVIRONMENT LEVEL
Sources:
 Administrative records (e.g., service statistics, HMIS data, financial)
 Service delivery point information (e.g., audit information,
inventories, facility survey data)
 Staff or provider information (performance / competency
assessments, training records, staff/provider data, quality of care
data)
 Client visit registers/compilations
Tools:
 Health Service Information Systems
 Facility sample surveys
 Performance monitoring reports
 Service Delivery Point records
 HEALTH MANAGEMENT
INFORMATION SYSTEMS (HMIS)
An important way of monitoring routine data over time is
through a Health Management Information System.
HMIS is a system for routine reporting on service delivery.
Often it is national. Best if data collected from a full set of service
delivery points, and on topics like:
o Costs
o Stockouts
o Births
o Mortality
o Morbidity
o Numbers of clients seen, referred (inpatient; outpatient)
o Numbers of clients by types of service
 DATA SOURCES AND TOOLS
INDIVIDUAL LEVEL

Gives information about how individual

people are being served
 DATA SOURCES AND TOOLS
INDIVIDUAL LEVEL
Sources:
 Case surveying (e.g., epidemiology of disease)
 Medical records
 Interview data
 Provider-Client interactions (check the clinical,
technical or interpersonal skills)
Tools:
 Case reports
 Client register analysis
 Patient flow analysis
 Direct observation
 DATA SOURCES AND TOOLS
POPULATION LEVEL

Gives information about everyone in

the country
o How things change over time
o how countries compare
 DATA SOURCES AND TOOLS
POPULATION LEVEL
Sources:
 Government Census Office
 Vital registration systems (e.g., birth and death
certificates)
 Sentinel systems – watch a specific thing (e.g. H2N1)
 Sample households or individuals
 Special population samples (demographic or
occupational group, or geographic sector)
Tools:
 Birth certificates
 Household/Individual/Special surveys
 Census forms
 DATA SOURCES AND TOOLS
SPATIAL / GEOGRAPHIC

Sources:
 Satellite imagery and aerial photography
 Digital line graphs and elevation models
 Cadastral maps (land ownership)
Tools:
 Global Positioning System GPS
 Computer software programs GIS
Take a break

• Back in 15 minutes
 Some Maths

 A couple of statistical ideas

 PERCENTAGES

 Consist of a fraction – the number measured

divided by the maximum possible
times by 100 (per cent = for each hundred)
 Number counted = “Numerator”
 Number possible = “Denominator”

Numerator
% = x 100
Denominator
 Example: PERCENTAGES

 Number of projects at HU Ratanakiri: 7

 Number of EU funded projects: 4
 What percentage of projects at HU RTK ate
EU funded?
 Numerator = 4, Denominator = 7

4
% = x 100 = 57%
7
 Percentile

 A percentile tells you where an individual or

subgroup fits into a larger group as a
percentage

 If you say “Sam’s IQ is at the 66th percentile”

it means that his IQ is in the first 66% of the
population
 Normal Distribution

 Many population statistics rely on the rule

that the measure results will scatter in a
“normal distribution”

 There is an average (“mean”) with half

above and half below

 Most individuals are close to the mean

 Normal Distribution

Percentile 0.1 2.2 15.8 50 84.1 97.7 99.9

 Quintile

 Divide a group into five equal groups

according to a ranking e.g income
 Each group is a quintile

 Enables comparisons between locations

 e.g someone in Cambodia earning $2 a
week and someone in the USA earning $40
a week will both be in the lowest quintile for
income.
Example: Disabilities by income quintile
(Indigenous Australia)
 DIFFERENT DATA SOURCES -
SAME INDICATOR
Different data sources can be used to measure the same indicator. In
some cases, changes to the metric are required depending on
data sources selected.
 % of live births attended by a trained TBA in last 12 months
Option 1 – using health service data:
 Numerator: # of live births in the district attended by trained TBA
in last 12 months
 Denominator: # of live births in the district in last 12 months
Option 2 – using information from village visits:
 Numerator: # of women having a live birth in last 12 months
reporting being attended by a trained TBA
 Denominator: # of women having a live birth in last 12 months
 DIFFERENT DATA SOURCES -
SAME INDICATOR
When a choice is available look at the + and - of different data sources.
% of children 12-23 months receiving all three polio vaccines
 Numerator: # of children 12-23 months receiving all three polio
vaccines
 Denominator: # of children 12-23 months
Data sources:
 Option 1: numerator from individual client records at facilities,
denominator from census
 Option 2: numerator from HMIS, denominator from census
 Option 3: numerator and denominator from sample household
survey (interview with mother)
 Option 4: numerator and denominator from sample household
survey (review of child’s immunization card)
 Some MEASURE Indicators
Population
Health
Environment
 The Handouts come from USAID MEASURE
program, which promotes good M&W practice

 Disaggregate: to separate into parts

 how this indicator is specific to one group
 Value-Added Indicators

 Value-added indicators show groups of

people or sectors that may not have
been targeted in the intervention but
have an impact from the intervention.
 Value-Added Indicator: Example

 The project may intend to increase the use

of condoms by providing condoms to local
merchants and providing merchants with
sales training.
 In the process, the livelihood of that
merchant is helped by providing an
alternative stream of income.
 The diversification of livelihood for that
merchant is a value-added result.
 Integration Indicators
 Integrated programs have several advantages over
stand-alone population, child health, or environment
programs.
 Integrated programs are cost-effective compared to
the cost of single-sector programs.
 Integrated programs also recruited a greater number
of men to family planning efforts and a greater
number of women and adolescents to environment/
conservation efforts. Integrated programs also
improved the perceived valueof family planning
efforts by packaging them with health interventions.
 Example: Number of cross sector linkages formed
Global
Fund
Guide

Lists
many
indicators
Thankyou
Produced by Tony Hobbs
Health Unlimited,
Ratanakiri, Cambodia
www.healthunlimited.org

With the support of

Australian Volunteers International
www.australianvolunteers.com

© 2009 HU. Use with Acknowledgement