Measuring Woreda Transformation

Outline

• Why measuring WT? Importance?

• Components of the measurements?
• Indicators for measurement
• Indicator selection process
• Pathway or category of statuses
• Composite measures
• Data sources/Data sets
• Overview of the dashboard
Why measuring WT?

• The need to measure progress of woredas,

facilities, institutions,
• Need to measure progress of the health
sector in advancing the transformation
agendas
• Composite measurements to help
leadership gauge the sector’s progress
towards the transformation agendas
• Measures the progress of the health sectors
against all the six WHO building blocks of
health system
Components of the measurements

HSTP I HSTP II
• Transformation agenda • An Objective
• Model Kebele • Quality and equity
• HP-PCHU • Information revolution
• Management standard • Leadership and governance
• CBHI • MCCP
• Health financing
Components of the measurements

 Five transformation agendas

 Mapped into six WHO building blocks
 Measures health system performance

Quality & Information Leadership & Health

MCCP
Equity Revolution Governance Financing
• Service • Health • Leadership • Health • Financing
Delivery Information & workforce
• Access to Systems governance
essential
medicines
Indicator selection process
• Series of consultations with program and M&E experts to select indicators
• Major criteria: Measures the progress in the TA, Have data source/can be collected easily
• First draft list of indicators were proposed (91)
• Data sources identification:
• Some dropped (difficult to capture data)
• Some data elements not collected through routine system
• Majority collected through routine system DHIS-2
• New data sets developed on DHIS-2 to capture data elements not in DHIS-2
• Weight determination for the indicators (level of effort, impact, cost)
• Composite scores and cut off points were determined
• Dashboard drafted and tested with actual data
Indicators (58 indicators)
Woreda health office
Health post (11) Health center (42)
(17)

• Qua&Equity: 7 • Qua&Equity: 32+1 • Qua&Equity: 1+1

• IR: 3 • IR: 3+1 • IR: 3+1
• Leadership & gov: 1 • Lead & gov: 2+1 • Lead & gov: 2+1
• MCCP: 1 • MCCP: 3+1
• Health financing: 1 • Financing: 3+1
Composite measures

Scores and weighting for each transformation agendum

Transformation agenda Health post Health Center Woreda level

Quality and equity 50% 45% (6%) 20% (10%)
Information revolution 20% 15% (6%) 15% (6%)
MCCP __ 10% 10% (4%)
Leadership and governance 30% 25% (5%) 50% (15%)
Health financing __ 5% 5% (2%)
Supervisee institution score No Yes Yes
Pathway or category of statuses
• Health institutions pass through levels of performance to reach model level

Model
Medium >85%
Performing
Low
performing 70-85%
Very low
performing 50-70%
<50%
Indicator: Health Post Level

Quality and equity indicators (50%) (Seven indicators)

 Proportion of model households(20%)

 Eg. Kebele with a total of 800 HHs and 650 currently model HH for a specific period, then the
proportion of model HHs will be (650÷800)x100%=81%

 Proportion of households having sanitation facilities (6%)

 Eg. If a kebele has a total of 800 HHs and 720 of them have basic sanitation facilities for a
specific period, then the proportion of HH with latrine will be (7200÷800)x100%=90%

 Dropout rate (Penta 1 to MCV1) (6%)


Indicator: Health Post Level

Quality and equity indicators (50%) (Seven indicators)

 Proportion of under-five children with pneumonia received antibiotic treatment (6%)

Number of under 5 children treated for pneumonia * 100%
Estimated number of under 5 children with pneumonia

 HP Outpatient attendance per capita (6%)

Total number of outpatient visits * 100%
Total catchment population

 Availability of essential Drugs for (6%)

Σ (tracer drugs x months available) * 100%
Σ tracer drugs x Σ total number of months in time period
Indicator: Health Post Level
Total Information Revolution (20%) (three indicators)

 Reporting timeliness (8%)

Number of reports sent according to schedule * 100%
The number of reports expected

 Proportion of conducted LQASs from the expected in the period (4%)

Number of health facilities that conducted LQAS * 100%
Total number of health facilities

 Reporting Completeness (8%)

Total number of reports received during a given time period * 100%
The number of reports expected

Leadership/Governance (30%) (One indicator)

 HP Reform implementation score (30%)

 Indicator: Health center level: Q&E (45%) (33 indicators)
Contraceptive acceptance rate (CAR) 1%
Proportion of pregnant women that started ANC early (less than 12 weeks) 1%
ANC 8 coverage 1%
Skilled Delivery Coverage 1%
Early PNC Coverage 1%
Drop-out rate from ANC1 to Delivery 1%
Proportion of Pregnant, Laboring and lactating Mothers tested for HIV (PMTCT) 1%
Proportion of Pregnant Mothers screened for Sphylis 1%
Fully immunization coverage for under one year children 1%
Dropout rate (Penta 1 to MCV1) 1%
 Indicator: Health center level: Q&E (45%) (33 indicators)
Proportion of under-five children with pneumonia received antibiotic treatment 2%
Proportion of pregnant women received IFA 90 plus 1%
Proportion of children aged 6–59 months who received 2 doses of vitamin A supplement 1%
Proportion of children <2 years who participated in Growth Monitoring and Promotion 1%
Recovered/cure rate for management of complicated SAM in children 0-59 months 1%
ART retention rate 1%
%of patients on ART with a suppressed viral load (<50 copies/ml) in reporting period 1%
Percentage of HIV-positive pregnant women who received ART to reduce the risk of Women-
to child-transmission (MTCT) during pregnancy, L&D and PNC 1%
Proportion of all forms of TB cases notified and treated from community Referral 1%
Percentage of new and relapse TB cases that were notified and treated during the reporting
period among the estimated TB cases in the same period 1%
TB Treatment Success rate (TSR) among all forms of TB cases 1%
Confirmed malaria cases (microscopy or RDT) per 1000 persons per year 1%
 Indicator: Health center level: Q&E (45%) (33 indicators)
Proportion of women aged 30-49 screened for cervical Ca 1%
Proportion of adults screened for HPN 1%
Proportion of adults screened for DM 1%
OPD Attendance per capita 1%
Patient satisfaction score 1%
Percentage of client with 100% prescribed drug filled 1%
Essential laboratory test availability 1%
Essential Drug Availability 2%
Clinical audit score 2%
EPHCG implementation score 5%
Average HP/Kebele quality and equity KPI score 6%
 Indicator: Health center level: others
Information revolution
IR-Structure and implementation of HIS score 3.00%
IR-Data quality score 3.00%
IR-Data use score 4.00%
Average HP/Kebele Information Revolution score 5.00%
MCCP
Staff satisfaction rate 5%
Proportion of health professionals with active professional license 5%
Leadership and governance
EHCRIG implementation score 15%
Community score card score 5.0%
Average HP/Kebele leadership/governance score 5%
Health Financing
Proportion of reimbursed amount from total spent 5.0%
 Indicator: Woreda level
Q&E
OPD attendance per capita 10%
Average PHCU and primary hospital quality and equity (KPIs) score 10%
Information revolution
IR-Structure and implementation of HIS score 3.00%
IR-Data quality score 3.00%
IR-Data use score 4.00%
Average HP/Kebele Information Revolution score 5.00%
MCCP
Staff satisfaction rate 2%
Attrition rate 2.00%
Health workers density (physician, nurses, midwives, HOs) per 1000 population 2.00%
Average PHCUs and primary hospital MCC; Health workforce score 4%
 Indicator: Woreda level
Leadership and governance
WoHO management standard Score 20%
Enhanced multi-sectoral engagement score 6%
Coordination for integrated implementation of transformation agendas 4%
Average PHCU and primary hospital Leadership/Governance score 15%
Health Financing
Proportion of households enrolled in CBHI 4%
Proportion of fee-waived?? 2%
Budget allocation (Share of health budget from the total government budget) 1%
Budget utilization rate 1%
Average PHCU and primary hospital health financing score 2%
Cascading of measurements
Health post

PHCU

Indicators
Woreda level
Cascading of measurements …
Has two levels
• Health post to PCHU
• Health post: Total Quality and Equity Score (KPIs): 50% (max)
• PHCU: Average HP/Kebele quality and equity KPI score: 6% (max)

– Example: if the average HPs Q&E score for HPs in a PHCU is 40% (that is
40% out of the maximum 50% allocated for Q&E), the score for the average
HP Q&E score is calculated as (40%/50%)*6%=4.8%
Cascading of measurements
Has two levels
• PCHU to woreda
• PHCU: Total Quality and Equity Score (KPIs): 45% (maximum score)
• Woreda: Average PHCU quality and equity KPI score: 10%

– Example: if the average HP Q&E score for PHCUs in a woreda is 30% (that is
30% out of the maximum 45% allocated for Q&E), the score for the average
HP Q&E score is calculated as (30%/45%)*10%=6.67%
Weight and calculating scores
Criteria to assign weight
 The level of investment need (economical)
 How impactful is the intervention in influencing the health status
 Ease or difficulty of the intervention to implement
 Level of effort to implement the intervention
Weight and calculating scores
Value of indicators is either directly or inversely proportional to the score
 Directly proportional indicators: when the value increases, the
score increases. This happens when the increase in the value of
an indicator is a desirable outcome.
 Example. Proportion of households;
 Weight assigned to this indicator is 20%
 If the value of the indicator for a health post/kebele is 80% for a certain quarter,
then the score will be 80%*20%=16%
Weight and calculating scores
 Inversely proportional indicators: when the value increases, the
score DECREASES. This happens when the increase in the value
of an indicator is NOT a desirable outcome.
 Example. Dropout rate (penta1 to MCV1);
 Weight assigned to this indicator is 5%
 8% drop out rate is 3% above the cut-off, and 2% less than the other cut off point
(10%), then the value is calculated as [1-3%/5%]=40%, if the maximum score is
5%, then, then score of the indicator is 40%*5%=2%
Indicators with inverse relationships
• Dropout rate (Penta 1 to MCV1)
• Drop-out rate from ANC1 to Delivery
Weight and calculating scores: Cut off points
• These are values assigned to indicators and indicate that the score of the
indicator will not affected by values of the indicators beyond the cut off point.
• When the indicator’s value of beyond the cut off point, the indicator will get
the maximum weight assigned to it.
• Example: cut off point for OPD attendance per capita is 2.5 visits per year. If a
certain facility or administrative level gets a value above 2.5. visits/year, then,
that health institution will get the maximum score assigned to that indicator.
• If a HO’s OPD attendance per capita is 2.7 for a certain quarter, then the score
for that indicator is 5% (maximum value).
• If less the values is <2.5, say 1.5, then the score will be calculated as
(1.5/2.5)x5%. 5% is the maximum weigh. Then the score will be 3%.
Weight and calculating scores: Cut off points
• When calculating the scores of an indicator whose value is inversely proportional
to the indicator, we need to note the following.
• If the value is less than the cut-off point set for that indicator, then the indicator
will be given the maximum score.
• If the cut of point on the other side is set and if the value of indicator is above or
equal to the cut off point, then the indicator will get zero score.
• Example drop-out rate for penta1 to MCV1 is 8%, the cut off point is 5% on the
positive side and 10% on the negative side: then the value of the indicators is
calculated as,
• 8% drop out rate is 3% above the cut-off, and 2% less than the other cut off point (10%), then
the value is calculated as [1-3%/5%]=40%, if the maximum score is 5%, then, then score of the
indicator is 40%*5%=2%
• If the value of the indicator was greater or equal to 10%, then it will get
zero, whereas if its value is <5%, it will get the maximum score
Other issues
Effect of data completeness on score
Entry of denominators: population data entry
Zero value: ??
Data sources

Existing DHIS-2 data sets: (enlist data sets in DHIS-2)

New DHIS-2 data sets: (Enlist data sets in DHIS-2)
Thank You!