Define, explain and discuss the monitoring and evaluation

purpose, plan and frameworks

7.1.1. Monitoring & Evaluation Defined

Monitoring is the systematic collection, analysis and use of
information from programs for three basic purposes:(1) Learning from the
experiences acquired (learning function); (2) Accounting internally and
externally for the resources used; and (3) the results obtained (monitoring
function) and taking decisions (steering function).
Meanwhile, Evaluation is assessing an ongoing or completed
program or policy as systematically and as objectively as possible. The object
is to be able to make statements about their relevance, effectiveness,
efficiency, impact and sustainability. Based on this information, it can be
determined whether any changes need to be made at a program or policy
level, and if so, what they are, what went well, where is there room for
improvement.
Thus, Evaluation has both a learning function - the lessons learned
need to be incorporated into future proposals or policy - and a monitoring
function - partners and members review the implementation of policy based
on objectives and resources mobilized. Monitoring and evaluation are
complementary
7.1.2. M&E Purpose
The primary aim is to have a strong M&E and review system in
place for the national health strategic plan that comprises all major disease
programs and health systems.
A robust monitoring and evaluation (M&E) system is required to
assess the effect of integrated service delivery. Appropriate indicators, data
collection systems and data analysis to support decision-making help guide
successful implementation of integrated services and measures the effect on
both service delivery and use of services (FP/Immunization Integration
Working Group, n.d.).
The national M&E plan and system should address all components
of the framework and lay the foundation for regular reviews during the
implementation of the national plan.
 Existing country health-sector review processes are key events to
assess progress and performance. Country M&E systems should generate
the information needed for global monitoring while minimizing the reporting
burden for countries. The Joint Assessment of National Health Strategies
(JANS) and health systems funding platform provide new opportunities to
align all partners around these principles (World Health Organization, 2010).
Progress of any medical institution are monitored and evaluated
through various activities such as monitoring reports, HMIS, surveys and
evaluation studies. The state has undertaken various activities as a part of
established monitoring and evaluation system. According to the National
Health Mission (2014), strategies for operationalizing the framework should:

 be primarily country-focused but also offer the basis for global

monitoring;
 address M&E needs for multiple users and purposes, including
monitoring program inputs, processes and results, tracking health
systems performance and evaluation;
 facilitate the identification of indicators and data sources, provide tools
and guidance for data analysis, and show how the data can be
communicated and used for decision-making
 bring together the monitoring and evaluation work in disease-specific
programs with cross-cutting efforts such as tracking human resources,
logistics and procurement, and health service delivery.

7.1.4. M&E Framework

Monitoring and evaluation (M&E) is a core component of current
efforts to scale up for better health. Global partners and countries have
developed a general framework for M&E of health system strengthening
(HSS).
The framework builds upon principles derived from the Paris
declaration on aid harmonization and effectiveness and the IHP+, putting
country health strategies, and the related M&E processes such as annual
health sector reviews, at the center.
The core is the strengthening of a common country platform for
M&E of HSS, which should result in better alignment of country and global
M&E systems and can be used both for monitoring the health systems funding
platform, as well as for tracking the performance of specific programs. The
framework addresses indicator selection, related data sources, analysis and
synthesis practices (including quality assessment), performance review,
communication and use (World Health Organization, 2009).
Figure 7.1: WHO Framework for Monitoring and Evaluation of Health Systems
Reform/Strengthening

The World Health Organization’s M&E of Health Systems Strengthening

Framework (Figure7.1) presents the indicator domains and considerations for
data collection, synthesis and use along the pathway for achieving health
impact at scale.
 For monitoring medical services, indicators should be tracked to
assess processes and results associated with the various indicator domains.
Monitoring these indicators provides information on the strengths and
weaknesses of implementation, and can help to identify “red flags” where
further investigation and adjustments may be needed.
It should be noted that shifts in outcome and impact indicators may
not be directly attributable to integrated service delivery efforts, as there are
many other factors which influence these indicators.
However, where possible, it can be useful to collect these data in
order to understand the broader health context within a country, and the ways
in which packages of interventions can lead to impact over time
(FP/Immunization Integration Working Group, n.d.).

Relationship between M&E with HMIS Indicators

An indicator can be defined as a variable whose value changes. It is

a measurement that measures the value of the change in meaningful units
that can be compared to past and future units. It focuses on a single aspect of
a program or project – i.e., an input, output or the overarching objective.
There are different HMIS indicators which can be used for
monitoring of key aspects of the health system performance. These are from
among the five broad categories – Reproductive health, Immunization,
Disease prevention and control, Resources utilization and Data Quality which
are shown in Table 7.1 below, as enumerated by USAID (2013):

Table 7.1: Key Performance Areas and Indicators

7.2.1. Indicators for monitoring FP/Immunization integration

Table 7.2 below provides specific indicators, data sources, and
purpose of tracking each indicator in Reproductive Health area, as listed by
the FP/Immunization Integration Working Group. This table includes a variety
of quantitative indicators, it is also important to complement collection of these
data with the use of qualitative techniques in order to better understand
nuances of the integration processes and solicit feedback on the approach.

Table 7.2: Indicators in the Reproductive Health Area

7.2.2. Indicators for monitoring FP/Immunization integration
Table 7.3: Health System Indicators and Data Sources
7.3.1. The Maternal Survival Strategy and HMIS indicators
Campbell and Graham (2006) discuss the following about Maternal Survival
Strategies and HMIS indicators:
The Maternal Survival Strategies lays down a framework for achieving the fifth
Millennium Development Goal of reducing maternal mortality. Given the
complexity of the country contexts and the determinants of maternal health,
none of the maternal survival intervention alone can reduce the maternal
mortality rate. Rather, evidences support packaging of health facility oriented
interventions is highly effective and has high coverage of the intended target
group.
In order to routinely monitor the progress towards implementation of a highly
effective package of maternal survival interventions, the HMIS is designed to
provide albeit some of the core input, process and output indicators. HMIS
indicators related to pregnancy care interventions are:

 1st antenatal care attendances

 4thantenatal care attendances
 Cases of abnormal pregnancies attended at out-patient departments
(OPD) of health facilities
 Institutional cases of maternal morbidity and mortality due to
Antepartum hemorrhage (APH), hypertension and edema reported by
In-patient departments (IPD) of health facilities
 Cases of abortion attended at health facilities
 Cases of medical (safe) abortions conducted at health facilities

HMIS indicators related to intrapartum care:

 Deliveries by skilled attendance (at health facilities)

 Deliveries by Health Extension Workers (HEW) (at home of Health Posts)
 Institutional cases of maternal morbidity and mortality due to Obstructed
labor

HMIS indicators related to post-partum care:

 1st postnatal care attendance

 Institutional cases of maternal morbidity and mortality due to Postpartum
hemorrhage (PPH) and Puerperal sepsis

HMIS indicators related to inter-partum (between pregnancies) period

 Family planning method acceptors (New and Repeat)

 Family planning methods issued by type of method
Though not a complete set to monitor every facet of maternal survival
strategies, these HMIS indicators duly capture data related to pregnancy,
intra-partum and postpartum care, sufficient to give a broad indication of the
ongoing performance of the package of maternal survival interventions, and
having the ability to instigate further investigation if problems/issues are
identified using these HMIS indicators.
The following illustration relates the HMIS indicators with the Maternal
Survival Strategies framework published in the Lancet. The HMIS
indicators related to specific sub-strategies are shown in green shaded
boxes.

Figure 7.3: Maternal Survival Strategies7.3.2. The Child Mortality and

Child Survival Interventions
Ethiopia is one of those countries who have made great strides towards
reducing the under-5 mortalities based on Ethiopia Maternal and Child Health
Data (2012).
However, under-5 mortalities still remain high at 106 per 1000 live births (LB)
in 2010 and the country faces the challenge of reducing it to 61/1000 LB by
2015.
The EDHS 2011 estimated under-5 mortalities to be 88 per 1000 LBthat is a
47% decline from 166/1000 LB in 2000. Diarrhea, pneumonia, measles,
malaria, HIV/AIDS, birth asphyxia, preterm delivery, neonatal tetanus and
neonatal sepsis are the major causes of under-5 deaths in Ethiopia, with
under-nutrition attributing to over one third of these deaths.
Figure 7.4:Under-five causes of deaths and mortality rate
Ethiopia is implementing interventions targeting under 5-year-old children
through:

 Universal Immunization Coverage,

 Nutrition program,
 Integrated Management of Childhood Illnesses and the Community
Case Management of Childhood Illnesses
 Through Health Development Army to – improve water, sanitation and
hygiene– Malaria prevention through Integrated Household Spraying
and distribution of Insecticide Treated Nets (ITN) (USAID, 2013).

In the context of the above these child survival interventions, the related HMIS
indicators are:

 Number of treatments for children under five provided by health facility

by disease : Diarrhea, dysentery, pneumonia, measles, malaria,
neonatal tetanus
 Number of infants immunized for measles
 Latrine coverage
 Latrine coverage
 Safe water coverage
 Household with ITN

7.3.3. The STOP TB Program

With the vision to have a TB free world, the goal of the STOP TB Program
(STP) is to dramatically reduce the global burden of TB by 2015, in line with
the Millennium Development Goals and the Stop TB Partnership targets of the
World Health Organization (2006). One of the main objectives of the program
is to achieve universal access to high-quality care (i.e. universal access to
high quality diagnosis and patient centered treatment) for all people with
TB(including those co-infected with HIV and those with drug-resistant TB).
TB case detection and successful completion of the treatment/cure of the TB
remains at the core of the Stop TB Strategy. Hence one of the targets linked
to the MDGs and endorsed by the Stop TB Partnership is by 2050 to reduce
prevalence and deaths due to TB by 50% compared with a baseline of
1990.The following flowchart puts the HMIS indicators (in green shaded
boxes) in the context of the STOP TB Program.
Figure 7.6: Routine Monitoring data on TB program captured through HMIS

HMIS Indicators to Monitor STOP TB Program

 TB patients on DOTS
 Number of new smear pulmonary TB cases enrolled in the cohort
 TB Case Detection
 Number of New smear positive pulmonary TB cases detected
 Number of new smear negative pulmonary TB cases detected
 Number of new extra pulmonary TB cases detected
 HIV – TB – Co-infection
 Proportion of newly diagnosed TB cases tested to HIV
 HIV+ new TB patients enrolled in DOTS
 TB Treatment outcome
 Treatment completed PTB+
 Cured PTB+, Defaulted PTB+, Deaths PTB+
 KEY POINTS TO REMEMBER

 The primary aim is to have a strong M&E and review system in place for
the national health strategic plan that comprises all major disease
programs and health systems.
 Monitoring and evaluation (M&E) is a core component of current efforts to
scale up for better health. Global partners and countries have developed
a general framework for M&E of health system strengthening (HSS).
 There are different HMIS indicators which can be use for monitoring of
key aspects of the health system performance. These are from among the
five broad categories – Reproductive health, Immunization, Disease
prevention and control, Resources utilization and Data Quality
 HMIS is a source of routine data that is necessary for monitoring different
aspects of various health programs implemented in the country. The
HMIS indicators have been carefully selected to meet the key information
needs of monitoring the performance of various health programs and
services and provide a snapshot of the available health resources.