Professional Documents
Culture Documents
Sop For Hmis Jun 2017
Sop For Hmis Jun 2017
STANDARD OPERATING
PROCEDURES (SOP)
Haron NJIRU
UNICEF Somalia
HMIS Consultant |njiru@outlook.com
STANDARD
OPERATING PROCEDURES
DOCUMENT OUTLINE
List of tables and figures ....................................................................................................................... 2
List of abbreviations .............................................................................................................................. 3
Definition of terms ................................................................................................................................. 3
1.0 Introduction ............................................................................................................................... 5
1.1 Situation analysis ....................................................................................................................... 5
1.2 About the document ............................................................................................................. 5
1.3 Proposed dissemination plan ................................................................................................ 5
2.0 Data Management System ........................................................................................................ 6
2.1 Preamble ................................................................................................................................ 6
2.2 Data protection ...................................................................................................................... 6
2.3 Data security .......................................................................................................................... 7
3.0 Data Collection and Reporting .................................................................................................. 9
3.1 Data collection tools .............................................................................................................. 9
3.2 Data collection procedure ................................................................................................... 10
3.3 Data reporting procedure .................................................................................................... 11
3.4 Late reports ...........................................................................................................................13
4.0 Data Quality Audits ................................................................................................................... 11
4.1 Overview ............................................................................................................................... 11
4.2 DQA objectives ...................................................................................................................... 12
4.3 DQA data management ........................................................................................................ 12
4.4 Procedures for conducting DQA exercise .......................................................................... 14
4.5 DQA assessment report ........................................................................................................15
4.6 HMIS scorecard .................................................................................................................... 16
4.7 Staff roles and responsibilities in DQA ............................................................................... 16
5.0 Data Demand and Use ............................................................................................................. 18
5.1 Data utilization at Facility level............................................................................................ 18
5.2 Data analysis beyond the Facility Level .............................................................................. 20
5.3 Typical procedure for Data analysis and Presentation ...................................................... 24
6.0 SOP Responsibilities ................................................................................................................ 29
7.0 Future considerations .............................................................................................................. 30
7.1 HMIS reforms ....................................................................................................................... 30
7.2 Electronic Data capture ....................................................................................................... 30
8.0 Revision History ........................................................................................................................31
9.0 References ................................................................................................................................31
10.0 Annexes .................................................................................................................................... 32
Annex 1: District data log ................................................................................................................. 32
Annex 2: Regional data log .............................................................................................................. 34
Annex 3: National data log .............................................................................................................. 36
Annex 4: DQA Template .................................................................................................................. 38
Annex 5: HMIS Scorecard ................................................................................................................ 39
Annex 6: HMIS Data visualization ................................................................................................... 40
Annex 7: HMIS Supportive Supervision Tool .................................................................................. 41
Annex 8: Mentorship Guidelines and SOP ...................................................................................... 42
Annex 9: DQA Log ............................................................................................................................ 44
LIST OF ABBREVIATIONS
DEFINITION OF TERMS
AWD Acute watery diarrhoea refers to passage of 3 or more loose or watery stools per day
with or without dehydration.
Data Raw, unorganized facts that need to be processed. Data can be something simple and
seemingly random and useless until it is organized.
Data analysis Organising and manipulating data using statistical concepts to generate information.
Data quality How well the information system represents the reality (accurate and reliable).
DDU Efforts to build capacity and improve data utilization and demand by all stakeholders
DQA Data quality assessment refers to a statistical evaluation of a data set to determine its
validity and adequacy for its intended use.
Evaluation Periodic assessment of the efficiency, impact, relevance and sustainability of the
project’s actions.
HMIS This is a routine integrated system of collection, collation, analysis, presentation,
dissemination and utilization of relevant health‐related information. It is specially
designed to assist in the management and planning of health programmes. a
Information This is data that is processed, organized, structured or presented in a way that makes
it useful.
MFL This is a list that contains all the health facilities in a defined geographical region.
Monitoring Continuous assessment that aims at providing stakeholders with early detailed
information on progress of program activities.
a WHO (2004). Developing health management information systems: a practical guide for developing countries
1.0 INTRODUCTION
All public health facilities (HFs) are provided with standard registers and summary forms
for HMIS dataa. Inside covers of the revised HMIS tools contain standard guidelines on the
collection, reporting of HMIS data. The tools are also identical irrespective of the level of
service delivery, except primary health units (PHUs) which have unique registers b. With
the nationwide introduction of DHIS, the country has shifted from Excel/Access reporting
to DHIS online reporting thus eliminating the need for manual aggregation of data. The
focus is now on data quality, data analysis, presentation and use. There is a Monitoring
and Evaluation framework to guide monitoring of the Health Sector Strategic Plan (2013 -
2016), however, there are no guidelines for data quality audits (DQA) and data analysis.
This document serves multiple purposes: It describes the THESE PROCEDURES APPLY
procedures for data collection, data management and TO BOTH PUBLIC AND
PRIVATE HEALTH FACILITIES
security procedures for routine health management
information system (HMIS). HMIS data should be collected, recorded and managed in
accordance with the Ministry of Health policies to protect patient confidentiality. This
document aims at ensuring that quality data is collected in a more efficient way by
describing how to carry out operations correctly and consistently. Following these
procedures will help in achieving uniformity in carrying out HMIS activities. This document
should therefore be available at every unit where HMIS data is either being generated,
aggregated or analysed, be it a public or private HF. It describes the steps required to
prepare for and conduct DQA and presents templates to be used in DQA process. Gives
guidelines for basic data analysis, interpretation and presentation using selected
indicators from the national HMIS indicators list. It complements the national HMIS
tool’s procedures and training manual, hence the two should be used together.
The SOP should be disseminated in a hierarchical cascade. The national HMIS teams should
be trained first; the team should then train the regional teams, who should train the district
teams. District teams should train and mentor HF teams to complete the cascade.
aData from private HFs is largely underreported making it impossible to know the precise burden of morbidity and mortality.
bInitially, there were different types of registers and summary forms for HCs and Hospitals even when the service mix was uniform which
made data aggregation for analysis and subsequent utilization cumbersome.
2.1 PREAMBLE
1. There should be a trained HMIS data focal person at all levels where HMIS data is
either generated, aggregated or processed.
2. Each facility should have a unique code from the master facility list (MFL)
maintained by national HMIS office. The code should be indicated on all HMIS tools
at the HF. Having an MFL is important for establishing the exact number of HFs, and
for resource allocations including supportive supervision.
3. All HMIS data shall be collected using standard health facility registers. The inside
cover of each register clearly explains what data should go to each column. This is
hoped to minimize data collection and reporting errors.
4. Data should be summarised using standard monthly summary forms. The forms
contain guidelines on how to summarize data from the various registers.
Data generated through the HMIS system should be securely stored at all levels.
1. Data should be securely stored at all levels, preferably metallic cabinets or lockable
wooden cabinets.
2. The most current health files (last 2 years) should be kept within the records office
while files older than 2 years should be safely archived.
3. Data should be stored chronologically and alphabetically using clear labels to ease
retrieval. E.g. January 2016 reports should be adjacent December 2015 reports while
Bay region should appear before Sool in the filing cabinet.
4. Anyone seeking access to the aggregated HF data should obtain permission from
the regional or district health offices or the HF in-charge. Such request should
include the reason why data is being requested, the variables and period of interest.
5. Anyone seeking access to patient-level records from HF registers should obtain
written permission from the national HMIS office. Such request should include the
reason why data is being requested, the variables and period of interest.
Exceptions to this requirement include supportive supervision / mentorship visits
by relevant authorities.
1. Electronic data should only be stored on devices that are routinely and securely
backed up:
5. Access to the data should be limited to authorised personnel. Each user of the
system should have a password protected individual account.
7. Users should be advised against sharing their database login details with anyone.
8. The database should have an audit trail for any edits made to the data (this is inbuilt
in DHIS). The trail should include original value, date of the change and user details,
and a brief explanation of why the change was made.
9. New staff should be taken through an induction program to familiarize them with
the HMIS system and data safety.
**
NO DATA SHOULD BE CHANGED OR DELETED THROUGH THE
VERIFICATION AND VALIDATION PROCESS WITHOUT PROPER
DOCUMENTATION.
1. HMIS data should be collected using the standard HMIS tools developed and
produced by the Ministry of Health. As a basic requirement, each health facility
should have all the necessary registers. These are shown in Table 1.
2. For ease of reference, and to reduce the overlap of register numbers across service
delivery levels, data recording tools have a unique number depending on the level
of service delivery. These are shown in Table 2.
3. Organizations and implementing partners whose services and activities utilise the
existing health facilities should report through the respective HFs using the
national HMIS registers.
4. Existing HMIS tools should not be updated or edited without the approval of the
Ministry of Health.
5. For harmonisation and standardization, no additional data collection and reporting
tools should be introduced without the approval of the Ministry of Health.
6. Any changes or updates to the system must be documented as part of a change
control process and the relevant parts of the system revalidated.
7. All health workers should be trained on the data collection and summary tools.
Data for HMIS should be collected from each PHU, HC, RHC and Hospital. All persons
involved in the collection and management of patient-related information must ensure
that the uses of those data do not "compromise" patients’ confidentiality.
1. Client data should be first captured in the designated patient medical cards / charts
by the health care provider.
2. The provider should immediately transfer the data recorded on medical forms into
appropriate registers (see Table 1 on page 9).
5. At end of each month, staff should add up page totals to get monthly summary.
6. The clinician should compute the page summaries at bottom of each fully
completed page of the register. This should be done at the end of each day for all
the completed pages. Figure 1 below is an illustration of page summary section.
7. MoH and IP should strengthen this process through routine supportive supervision,
mentorship, DQAs and flip-chart reference sheets.
Each register and data reporting tool should have Version Number and Version date
clearly printed on the cover page. This reduces chances of using outdated registers.
The inside of the front cover of every register / reporting tool should:
1. Specify where the registers should be placed in the HF
2. Describe what data should go to which column or cell
3. State if the tool is to be used together with any other tool e.g. tally sheet.
Data should be reported using the designated MoH standard reporting tools. This should
be done within the stipulated time to ensure availability of data for timely decision making.
Table 3 shows the general data flow for monthly reports and the reporting deadlines.
Where DHIS is in use, data is keyed into DHIS at the HF, district or regional level. This should
happen within 5 days after receiving data. Accordingly, all data should be available in DHIS by
the 15th Day of every month.
3. The facility in-charge should cross-check the reports for errors before submitting to
district. Use DQA Forms and Data Validation Rules (Table 7) as a guide.
4. If the report has no errors, the in-charge should sign the requisite sections of the
report. If any errors are identified, request form to be done again, and review
afresh. Ensure all expected reports have been prepared for submission.
5. Two copies of the report should then be submitted to the district HMIS by 5th day
of the subsequent month. Data entry to DHIS should occur by 10th of the month.
6. The facility should retain a copy of the report for their records as well as for data
quality audits (DQAs) by district/regional officers and implementing partners.
4. All data queries should be resolved before submitting the data to the next level, or
before entering data into DHIS.
5. The district HMIS report should then be compiled using the standard Ms Excel
template. Skip this step if using DHIS.
6. District Monthly Excel file, the District Data Log and Annex 1b should then be
submitted to the regional HMIS officer by 7th day of the month.
4. All data queries should be resolved before submitting the data to the next level, or
before entering data into DHIS.
5. The regional HMIS report should then be compiled using the standard Ms Excel
template. Skip this step if using DHIS.
6. Region’s Monthly Excel File, and copies of District, Regional Data Logs and Annex
2b should then be submitted to national HMIS office by 10th day of the month.
1. The National HMIS coordinator or his/her designee should ensure that s/he has
received all expected reports from all the regions and update the National monthly
data log (see Annex 3 for the National data Log template).
3. S/he should contact any regions whose reports are either missing or incomplete for
clarifications before compiling the national HMIS report.
5. National HMIS report should then be compiled and shared with various
stakeholders and feedback provided to regions and districts. The analysis is done
using DHIS visualization features, pivot tables (exported to Excel) or both.
Any queries identified after running DHIS validation rules should be discussed with
respective facilities and resolved: HFs should recheck registers and make necessary
adjustments on summary forms. Such changes should be documented in DHIS for
reference. This is done by double-clicking the value that is being modified.
1. In case of delayed facility reports, the HMIS officer supporting the HF should
contact the HF requesting status of the report within a day after the deadline.
2. If it’s not possible to receive the report for compilation in good time, the officer
should update the data log accordingly with the available reports, and submit.
3. Reports submitted after the deadline should still be entered into DHIS.
4.1 OVERVIEW
Routine monitoring activities of health programmes generate important data for patient
care and programme improvement. This data should therefore be of high quality. Data
quality assessment help identify errors and provide detailed information of the facilities’
data and mentorship needs to strengthen good data practices at health facilities.
This SOP contains detailed instructions for conducting HF data quality audits (DQAs). It
looks at completeness of data records and compares facility register records with
documentation on the monthly HMIS and DHIS reports.
The DQA process verifies key data characteristics, facilitates the process of evaluating set
data standards and provides a score against data qualities. Table 5 below shows the various
dimensions of data quality.
Characteristic Definition
Accuracy This is one of the components of data quality and refers to whether
the data values stored are the correct values. To be correct, a data
values must be the right value and must be represented in a consistent
and unambiguous form.a
Completeness Measure of availability of all data fields within a set of data collection
tool. Are all fields in monthly reporting form complete? Does report
capture all patients in the register during the reporting month?
Consistency Logical coherence among related aspects of data. e.g. Number of ANC
women given Iron/Folate for treatment should not be greater than
number of anaemic pregnant women. Please see Table 7
Validity The ability of data to reflect events/outcomes that is measured. E.g.
Screening children’s nutritional status by other means besides MUAC
and weight/Height is not valid.
Reliability The ability of data to show similar results when indicators are
measured more than once, using similar ground/characteristics. It is
the extent to which we can rely on the source of the data, therefore
the data itself. E.g. The Total Number of HIV infected clients should be
consistently extracted from all the HIV testing points (e.g. VCT
register, ANC Register and Delivery Register), not just some of the
registers.
3. To appraise data reliability against data summaries on MoH registers and forms.
5. To assess the accuracy of data through double checking all data variable fields
against source documents.
HMIS data is aggregated from different sources, the design of DQA is to compare
reported data against these sources. Due to time-consuming nature of DQAs (4 hours on
average), the DQA team only focuses on selected data elements for each program area.
The arbitrary list of DQA indicators and their source documents are shown in Table 6.
Table 7: Data validation rules for DHIS data entry and general mentorship
The following steps should be completed before leaving the office to go to the field to
conduct the facility DQA. These steps are appropriate and crucial for a successful DQA.
3. Refer to the DQA process checklist to be sure that all steps are completed:
preparation, conduct and follow-up of DQA.
4. Using the MoH monthly HMIS report, and DHIS review the predetermined
indicators for DQA and populate the template with DHIS data prior to the visit.
Identify any missing reports earlier to plan future mentorship support of the
facility in documenting and reporting using the MoH tools. Copies of any
missing reports should be delivered to the HF during the DQA visit.
1. Brief IP & HF staff and MoH officials about purpose and steps of conducting DQA.
2. Open the DQA Excel tool and enter the facility name, DQA date, and names of the
staff conducting the DQA into the appropriate cells (Annex 4).
3. Obtain copies of MoH HMIS monthly reports and relevant registers.
4. Start the DQA process addressing all DQA areas documented in Table 6 above.
Give time for short discussions that may arise during the DQA exercise.
5. Enter data in the Excel sheet. Percent error will be calculated automatically.
6. Complete the HMIS supportive supervision tool (Annex 7).
7. At the end of the DQA, hold a joint meeting with the facility staff to discuss in
detail the results and plan how to resolve the identified gaps or inconsistencies.
8. A copy DQA report with action plan should be kept by HF for reference. IP also
keeps a copy for review and preparation for tailored mentorship to the facility.
9. The HF documents the just concluded QA in their QA monitoring log. The
monitoring log should contain the following information: Date of DQA, DQA
score, action items and dates when the action items are resolved.
MoH and IP shall help facilitate feedback to the facilities through graphs and tables
that will summarize the findings of the DQA. M&E staff should use these sheets during
their presentation of DQA findings to health facility staff and clinical staff. Ideally,
these pages should be printed out for the facility staff to review and post on their walls.
DQA timelines
DQA should be conducted at each facility every three months. This provides an
opportunity to review gaps, weaknesses and see the strengths that will help shape the
strategies for conducting tailor made mentorship or any training needs.
1. Conduct DQA every three months (or earlier for immediate follow up DQA).
2. Data for the current month should not be included in the DQA process since it is likely
to be incomplete and therefore not representative.
3. Examine register entries for current month for completeness and appropriate
mentorship.
4. DQA feedback and follow up action items to be agreed immediately after completing
the DQA exercise.
5. During each DQA exercise, the team should review monthly reports for previous 3
months against source documents, as well as DHIS entries.
District and Regional HMIS officers, and program managers together with IPs should
support and mentor facilities on proper recording and reporting to ensure that all the
MoH reports are complete, accurate, reliable and timely. See Annex 8 for Mentorship
Guidelines and Standard Operating Procedures.
There should be a written report of every DQA. The report should include specific guidance
for facility staff on how to improve data collection and recording, it should indicate the
timelines for joint review of the set priority activities. In addition, the DQA team should
generate a HMIS scorecard highlighting performance levels (See Section 4.6 on page 16).
1. Present and discuss the DQA report: A written final DQA report should be shared with
facility staff within 7 days after the DQA.
2. Include DQA reports in quarterly data review meetings: Two or three slides in the
quarterly data review presentation should be dedicated to DQA. The slides
summarise HF levels and district/regional DQA results and should always include
measures on how to improve collection and recording of data.
3. Mentor staff on implementation of corrective measure: After the facility DQA report
is shared and discussed with facility staff, mentorship on implementation of
corrective measures should be provided by appropriate MoH and IP staff. Follow-up
is critical for improving data. Continued mentorship is expected to result in improved
DQA results over time.
To improve the quality of HMIS data, a scorecard has been developed to assess and
improve the health management and information system. The scorecard is one of the
outcomes of a data quality audit. The scorecard assists in comparative assessment of HMIS
performance at HF, district, region and national level. The scorecard uses data quality
aspects from DHIS (timeliness, completeness, data values and data validation rules), HF
registers records and Monthly Summary Forms to show relative position of a service
delivery unit (HF, District, Region, State) using a composite index. The composite index
combines a number of indicators in a standardised way in order to assess the overall
performance of the level.
The HMIS scorecard is generated using the DQA template (Annex 4). The template
generates a score for tools in each program area, and an aggregate score for the health
service unit. The performance results are colour-coded into 3 groups as shown in Table 8.
Collaboration of IP staff, district/region MoH staff and HF staff is essential for effective
implementation of DQAs. This section describes the roles of these players in DQA process.
Descriptive epidemiology covers time, place, and person. This is achieved using
the 5W's of descriptive epidemiology:
Health data should be presented in a way that enables decision makers answer
these 5 questions. Data analysis and appropriate presentation is key.
HMIS data should be presented in a form appropriate for the needs of various
stakeholders. As such, data should be analysed at all levels to generate outputs that will be
provided as feedback to improve health service delivery. To achieve this, capacity should
be built in data use core competencies.
Figure 2 is an illustration of data demand and use (DDU) capacity improvement. The goal
of DDU capacity-enhancement efforts is increased use of data for decision-making and
advocacy, and improved health outcomes.
Source: MEvala
Figure 2: An approach to enhance DDU capacity to improve data demand and use
The following procedures are recommended for improving DDU capacity at various levels:
1. Facility staff involved in data management should be trained on basic data analysis,
presentation and interpretation skills.
2. HFs should identify 5 priority indicators (from the national HMIS indicator list) for
tracking and visualisation (see Table 9 for possible list of priority indicators).
3. HFs should hold monthly data review meetings to review the 5 indicators, prior to
submission of the Monthly reports. This is the first stage in DDU.
4. HF data analysis should at the minimum include trend analysis. Analysis of the
trends should help facility management assess progress towards set targets.
Indicator
1 % pregnant women utilizing ANC services
Numerator: Women who made first ANC visit during the month
Denominator: Estimated number of pregnant women in the community
2 ANC dropout rate
Numerator: Women who made ANC1 MINUS women with 4 or more ANC visits
Denominator: Women who made first ANC visit during the month
3 % ANC women who are tested for HIV and know their status
Numerator: Women making first ANC visit
Denominator: Women who get tested for HIV during the first ANC visit
4 % of deliveries conducted by skilled birth attendants
Numerator: Deliveries conducted by a skilled birth attendant in a health facility
Denominator: Estimated number of deliveries in the catchment population
5 % delivering women attending PNC
Numerator: Number of women who made first PNC visit during the month
Denominator: Estimated number of deliveries in the catchment population
6 % children under 1 immunized with Penta 3
Numerator: Children <1-year-old provided with Penta 3 vaccine
Denominator: Estimated number of <1 children in catchment population
7 Immunization dropout rate
Numerator: Children <1 who received Penta 1 MINUS Children <1 who got Penta 3
Denominator: Children <1 who received Penta 1 vaccination during the month
8 % children aged 6-59 months treated with ORS and Zinc for AWD
Numerator: Children with AWD who are treated with ORS and Zinc
Denominator: Number of Children diagnosed with AWD during the month
9 % of patients with malaria who are treated with A+L
Numerator: Number of patients treated with A+L
Denominator: Number of patients with a positive RDT or positive malaria Slide
10 % of malnourished children seen in OPD
Numerator: Number of children who are moderately or severely malnourished
Denominator: Children screened for malnutrition
1. HMIS data received from facilities through DHIS THE ULTIMATE GOAL IS NOT
should be regularly analysed and feedback provided TO GAIN INFORMATION, BUT
to facilities. Information products emanating from TO IMPROVE ACTION
such data are useful for updating dashboards, generating talking points,
identification of success stories and areas of improvement.
2. There must be a data quality check prior to data analysis. Quality checks should
include checking for missing data, missing reports, numbers that are outside the
expected range (outliers) and any inconsistencies. Any errors identified should be
corrected using the source documents. See Section 4.0 for DQA procedures.
3. Such analysis should include trend analysis (to show seasonal variations),
comparison of performance across geographical regions, and efforts to identify
possible causes of the observed trends e.g. weather conditions, government
policies etc. Chi-square test of associations and simple logistic regressions can be
performed at this level to test for the existence and strength of any associations.
5. An annual HMIS report should be prepared for sharing with relevant stakeholders.
To make sense of data, officers managing data should have an understanding of basic
epidemiological concepts and skills that will help them to analyse and interpret the data.
Below is a brief introduction to the basic terms that will be relevant when analysing routine
HMIS data.
Illustration
The weight of 4 children in OPD is 11, 15, 16, 18 kilograms
Sum of the numbers = 11+15+16+18 = 60
Length of the list = 4 children
Average = 60 / 4 = 15 kilograms
Completeness Completeness rate measures how sufficient is the data that’s available.
& Can be at the tools level or at a facility/region/state level. This is also
Timeliness rate referred to as reporting rate. Timeliness on the other hand refers to
being done or occurring at a favourable or useful time.
Composite This is an indicator formed when individual indicators are compiled into
indicator a single index.
Illustration: Vingt Village has 12,000 children under the age of 5 years.
If only 6000 of these children are dewormed, then the coverage for
the service is 6000/12000 = 50%
Dropout rate This refers to the percentage that fails to complete a service as
required. In immunization program, dropout rates represent the
proportion of children who receive at least one vaccine in a schedule,
but do not receive the next dose in the series.
Fraction This is a quantity that is not a whole number. It’s part of a whole. It is
also known as a proportion. The top number (the numerator) says how
many parts three are while the bottom number (the denominator)
says how many parts the whole has.
Illustration: The fraction 5/10 means that we are looking at 5 units out
of a total of 10 units. 5 is the numerator while 10 is the denominator.
Indicator This is a single measure that is reported on regularly and that provides
relevant and actionable information about population health and/or
health system performance and characteristics. An indicator can
provide comparable information, as well as track progress and
performance over time.
Maximum Maximum refers to the highest value in a list of values, while Minimum
Minimum is the lowest value.
Illustration
The weight of 4 children in OPD is 11, 15, 16, 18 kilograms
Maximum weight = 11 kilograms
Minimum weight = 18 kilograms
Median The middle number in a list when the entries are sorted in an ascending
or descending order.
Illustration
The weight of 5 children in OPD is 11, 15, 16, 18, 20 kilograms . In this
case the median weight is 16 kilograms
Range This is the difference between the lowest and highest values.
Illustration
The weight of 4 children in OPD is 11, 15, 16, 18 kilograms
Lowest weight = 11 kilograms
Highest weight = 18 kilograms
Range = 18 – 11 = 7 kilograms
Performance This is a measure of how a service delivery level (HF, district, region
etc.) is faring either against set targets (or expected coverage) or
against set standards.
In addition to the routine analysis of HMIS data that is presented in the DHIS dashboards,
the following steps are recommended when preparing for quarterly data review meetings.
It expected the results of the HMIS data analysis would be shared during the data review
meeting possibly in terms of a PowerPoint presentation.
Purpose: This section of HMIS SOPs describes the typical procedure for carrying out
statistical analysis of HMIS data. The document is meant for use by MoH HMIS officers.
Create analysis database: Since DHIS data is continuously being updated by regional and
facility HMIS officers, it is necessary to note the date of analysis. Any data queries arising
should be referred to the HMIS officer for resolution, and the new date noted.
Conduct a DQA
- Check reporting rates to confirm all reports have been received.
- Run validation rules to confirm al data fields have been correctly entered
- If both are acceptable, decide on an analysis and presentation plan.
- If not drill down to flag out HFs with missing and outlier variables.
- Contact HF in-charges to recheck the correct values from source documents
- Proceed with analysis one the DHIS data is cleaned.
Which errors can the HMIS officer correct and which ones must be referred to
HF for confirmation? Questionable results and extreme outliers must not be
changed without written confirmation from the facility in-charges. The same
corrections should be effected on the respective monthly reporting forms at the
HF level, and marked on the forms at the data entry point for consistency and
proper paper trail, before effecting the changes in DHIS. Data should be rejected
if facility more than 80% of facility entries do not pass validation rules.
Data analysis plan (Table 10 gives a general guide. Modify indicators as needed)
- Review your data analysis plan to identify what data is available or not
- Identify data sources (some data analysis is already done by DHIS for others you
may have to download as pivot tables and manipulate data in Excel)
Health facility staff: - Appoint a focal person for HMIS data reporting
- Accurate data generation and timely submission of monthly
reports.
- Utilize data by active visualization and data use for decision
making
HMIS officers: - District HMIS officers to support and mentor health facility staff
- Regional HMIS officers to support and mentor district teams and
HF staff
- Zonal HMIS officers to support and mentor Regional and district
team
- Initiating and participating in the DQA processes.
Ministry of Health - Ensure standard registers data tools are available at HFs
- Insist usage of standard tools for all reporting purposes.
- Ensure staff are adequate and sufficiently trained.
- Sharing HMIS data with national and international stakeholders.
- Oversee periodic review and update of these SOPs.
STANDARDIZE
For harmonized reporting, HMIS data collection and
INTEGRATE
reporting tools have been reduced to a minimum, SIMPLIFY
similar registers have been merged depending on the COMPUTERIZE
DECREASE DATA COLLECTION BURDEN
level of services a facility is equipped to provide. E.g.
OPD registers for hospitals and HCs are now the same. In addition, the introduction of DHIS
as the national online reporting system w.e.f January 2017 has now shifted the focus to
data quality. However, some parallel reporting exists especially for donor funded vertical
programs e.g. HIV and Tuberculosis.
Electronic medical registers and records is one of the latest developments in HMIS where
traditional paper registers and data tools are either partly or completely replaced with
electronic versions. EMRs have the advantage of speed, accuracy and efficiency. As such,
EMRs is the future of HMIS.
9.0 REFERENCES
DQA Standard Operating Procedures for Somalia (Muhsin Sheriff, Jul 2014)
EPHS M&E Training Plan (Muhsin Sheriff)
HMIS Training manual for Somalia (Haron Njiru, 2016)
HUD Exchange. HMIS Data Quality Monitoring Toolkit (Aug 2010)
MEASURE Evaluation Technical Brief (Nov 2015)
MEASURE Evaluation, Data Quality Assurance Tools. http://tiny.cc/s8suly
Mentorship for ME systems for Somalia (Muhsin Sheriff, 2013)
MoH Kenya Data Quality Assurance - ICAP Kenya. SOPs (Muhsin Sheriff, June 2011)
MoH Rwanda. Health centres and Posts. Standard Operating Procedures for Management of
Routine Health Information (2016)
Monitoring and Evaluation within the Joint Health and Nutrition Programme. DRAFT Standard
Operating Procedures (January 2014)
Somali National HMIS indicators (Haron Njiru, 2015)
Supportive supervision guidelines for Somalia (Muhsin Sheriff, 2013)
The District Health Information System (DHIS) as support mechanism for strengthening the
health care system (Christa van den Bergh, Mar 2009)
UNICEF Agora course: Immunization Coverage
UNICEF Agora course: Research and Evidence at UNICEF
UNICEF. Methodological Brief No.10: Overview: Data Collection and Analysis Methods in
Impact Evaluation
WHO. Health Measurement and Accountability Post 2015: Five-Point Call to Action (Jun 2015)
10.0 ANNEXES
Reports expected during the month Reports received during the month Is report complete? (closely examine) Total
(Yes = 1, No = 0) (Yes = 1, No = 0) (Yes = 1, No = 0)
Surgery / Inpatient
Surgery/ inpatient
Surgery/inpatient
Name of health facility
Incomplete
Maternity
Maternity
Maternity
(Include all HFs in the District)
Missing
Total
Total
Total
HRH
HRH
HRH
OPD
OPD
OPD
ANC
ANC
ANC
PNC
PNC
PNC
Lab
Lab
Lab
EPI
EPI
EPI
Log completed by: Date:
Designation: Signature:
Number of HFs offering the service HF reports received this month Number of HFs with incomplete reports Total
Name of District
Surgery / IPD
Surgery/IPD
Incomplete
Maternity
Maternity
Maternity
(Include all districts in
Surgery
Missing
the Region)
Total
Total
Total
HRH
HRH
HRH
OPD
OPD
OPD
ANC
ANC
ANC
PNC
PNC
PNC
Lab
Lab
Lab
EPI
EPI
EPI
Log completed by: Date:
Designation: Signature:
Number of HFs offering the service HF reports received this month Number of HFs with incomplete reports Total
Name of Region
Incomplete
Maternity
Maternity
Maternity
Surgery
Surgery
Surgery
Missing
(Include all Regions)
Total
Total
Total
HRH
HRH
HRH
OPD
OPD
OPD
ANC
ANC
ANC
PNC
PNC
PNC
Lab
Lab
Lab
EPI
EPI
EPI
Log completed by: Date:
Designation: Signature:
Number of regions: INSTRUCTIONS: Please type T for timely report, L for late reports, and N if a Region does not submit any report
This is the word version of the DQA tool. Accompanying Excel file automatically calculates the scores
NAME OF HEALTH FACILITY
DATE OF DQA
NAMES OF STAFF CONDUCTING DQA
Please complete table below using values obtained from HF register, Monthly Form and the DHIS.
Indicate analysis (%) in the last 2 Columns if values in the 3 sources are Available and/ or Matching.
SUMMARY
Total indicators selected
Total indicators with available entries (# & %)
Total indicators with matching entries (# & %)
In absence of complete DQA, DHIS validation rules, and the reporting and timeliness rates will be
used to assess the quality of data. The score would be the average of the two values.
For HF DQA, please do a visual inspection of all the forms submitted to identify the proportion of
missing data elements, use the supportive supervision form and mentor accordingly. When using
DHIS and Paper forms, check the delays in data entry based on DHIS date and Date on the forms.
INSTRUCTIONS
This is the paper version of the HMIS scorecard. The Excel version automatically calculates the
scores per program area. Enter % values that were available and / or matching for every program
area, and the % of validation rules passed per program area.
The final score for unit would be the average of all entries in Column i. the final score is then coded
Red, Yellow or Green as shown in the table at the bottom of the page.
Note: This scorecard is used to assess the status of HMIS data. However, scorecards can also be
used to assess the quality of clinical services and overall performance of a program.
Information products should be Provided at the point of service delivery. They should
communicate key messages from data including the place, person and time to allow for
effective decision making.
This will be discussed with the regional officers and finalised during the data
analysis and presentation trainings in July/August 2017.
Possible solutions:
- Development of wall chart A3 templates for HFs to track and report their
performance
- Arrangements to access/email HF dashboards from DHIS to the facility in-
charges.
- Frequency and format / templates for data review meeting
2. Reporting period
(a) Does the facility in-charge know the deadline for reporting? Yes [ ] No [ ] (b) Which deadline: ………..
3. Accuracy: Check accuracy of data for the LAST 2 MONTHS and record in table below:
Month 6: Month 5: Average
# Indicator
Register Report Variance* Register Report Variance* Variance **
A OPD: # of malaria cases confirmed by RDT or Slide (choose ONE)
B OPD: Total # of patients treated at the facility
C Nutrition: # children screened–severely malnourished
D EPI: # of children Under 1 year received Penta 1
E EPI: # of children Under 1 year received Penta 3
F EPI: # of children Under 1 year received Measles
G ANC: # of clients made 2nd ANC visit
H ANC: # pregnant women screened for syphilis
I Deliveries: # of normal deliveries at health facility
J PNC: Clients with PNC within first 2 days
Total == == == == == ==
Score = (Total / 10) * 100 == == == == == ==
* Variance = Register – Report; ** Average variance = (Month 5 + Month 6) / 2
4. Display of Data
Please list indicators that are displayed in the wall of the health facility
a. d.
b. e.
c. f.
5. Choose 5 registers at random and check how many have updated page summaries: …….. OUT OF 5 = ……%
6. How many of the registers have columns completed as per HMIS Tools guidelines? …….. OUT OF 5 = ……%
7. Does the facility have all required Registers & Forms depending on its range of services? Yes [ ] No [ ]
8. If computers are used: (a)Are antiviruses updated? Yes [ ] No [ ] (b) Are there regular backups? Yes [ ] No [ ]
9. Are the following SERVICES / EQUIPMENT / SUPPLIES available?
(Discuss with PHC coordinator if there is need to combine other issues during supervision)
Service / Equipment/ Supplies Available? (Yes / No) Functioning? (Yes / No) Comment
a.
b.
c.
10. Any other comments on Supervision Activities from Supervisor or facility staff and your final remarks:
The Somali health sector is committed to high quality health situation requires a mentorship program to improve capacity of the
services provision. However this is not possible without good health system to be able to respond to the health needs of the
quality monitoring systems. This includes adequate documentation population appropriately and adequately. This section provides
tools, trained human resource and resources for oversight of the tips on the mentorship system while focusing on health systems
health systems. There is an existing supportive supervision monitoring and evaluation.
concept which is lacking in implementation and effectiveness. This
‘High-quality monitoring and evaluation is not only a donor requirement; it is how the impact of interventions is measured and
attainment of goals assessed’
Mentorship (and supportive supervision) is a joint effort capacity of health facilities to achieve their aims. Mentorship is a
(partnership) between the mentor and health worker to improve relationship between a mentor and mentee to bring about
motivation, knowledge, skills and performance of the latter in his exchange of learning and cause development. Systems and skills
work situation. Mentorship and supportive supervision have are required in data collection, collation, aggregation,
several commonalities. However it is differentiated significantly in summarising, validation, cleaning, submission, data analysis,
the sense that it is less hierarchical, more hands on and improves interpretation, display and dissemination.
skills of health workers while producing significant improvement in
‘Enabling the public health workforce to collect and use data to guide programs and policies’
Goal: The goal of the data systems mentorship is to provide health workers with
Importance of mentoring staff
the necessary knowledge and skills to undertake appropriate tasks required for
• Contributes to the development of the
effective running of health facilities and systems.
organization's talent
• Helps new staff members acclimate to a
Objectives: To improve knowledge and skills of health workers on:
new role and the organizational culture more
quickly
1. Assessing data systems at health facilities.
The need for data and information. • Promotes diversity
2.
3. Data Quality Audits (DQA). • Provides broader perspective on the
4. Data collection and recording using the MoH registers and tools. challenges facing the organization,
5. Data cleaning. divisional, regional and country offices
6. Data reporting (using MoH monthly reporting tool). • Creates a greater sense of involvement
7. Data flow. • Offers insight into the culture of the
8. Simple data analysis to provide information for decision making pertinent to organization
the health facility. • Supports an innovative work environment
9. Data presentation.
10. Skills on data use and decision making, as well as Effective mentors
11. To improve knowledge and skills of IPs on UNICEF reporting tools. • Serve to role model effective organizational
behaviours and attitudes
THE MENTOR AND MENTEE • Give actionable advice and feedback
• Resist the temptation to solve the problems
Who can be a mentor? of those they mentor
• Challenge the people they mentor to
Anyone who has undergone the necessary mentorship training, has knowledge, develop a plan for success
skills and experience in the field under focus, and possesses the characteristics • Create a foundation of support
of a mentor. See box to the right. • Suspend judgment
MENTORSHIP PROCESS
How to mentor
1. Communicate
2. Listen patiently
3. Work plan
4. Acknowledge achievements
5. Identify areas of strengths and weaknesses
6. Identify challenges and opportunities in the work environment
7. Identify and utilise appropriate resources
8. Document all mentorship processes, agreements, progress and plan
Data system mentorship encourages continuous capacity building of health workers to provide high quality data records, reports,
dissemination and use. Mentorship is required to strengthen the systems and skills necessary for HMIS. Focus should therefore be on the
following systems and skill areas:
1. Collection 6. Cleaning
2. Collation 7. Reporting
3. Aggregation 8. Data analysis
4. Summarizing 9. Interpretation
5. Validation 10. Data display, presentation and dissemination
1. Assess needs for individual health facilities based on DQA and health facility assessment reports.
2. Identify staff and specific topic requiring capacity building.
3. Preparation:
a. Identify a mentor for the topic from the in-country team.
b. Agree on date and topic of mentorship visit with facility in-charge ensuring that the relevant staff will be available on that date.
c. Ensure you will have at least 2-3 hours of dedicated time with the relevant staff.
d. Review past mentorship activity reports, results and needs.
e. Ensure familiarity with the topic.
f. Prepare short exercise to improve skills. Ensure data from the identified health facility is used for the exercise.
g. Ensure all necessary resources are available (stationery, funds, transport, communication equipment and airtime).
h. Ensure all team members are dedicated and available for the visit.
4. Mentorship visit:
a. On the day of visit, plan to reach the facility well in advance to allow at least 2-3 hours of dedicated time with the relevant staff.
b. On arriving at the health facility:
c. Introduce yourself, by name, designation and agency, to the health facility in charge and all staff.
d. Inform them about the purpose of the visit and that you had prior appointment.
e. Obtain all staff name and designation especially the HMIS data point person. This may be the nurse at the facility.
f. Agree on point person (HMIS officer/nurse) with who will be the mentee.
g. Review tools. Ensure all needed registers are available.
h. Perform a quick DQA and assess data tools, data flow, monthly data reports, data display and use.
i. Identify and agree on strengths and weaknesses.
j. Develop work plan for mentorship and improvement plan with timelines.
k. Provide mentorship for knowledge and skills building.
l. Debrief with all staff.
m. Agree on next visit date and areas for mentorship.
5. Post-mentorship visit
a. Prepare mentorship visit report; attach the mentorship plan agreed with the facility staff and share with supervisor.
b. Discuss with IP team and report at quarterly UNICEF/MoH review meetings.
c. Follow-up on mentorship plan and prepare for next visit.
INSTRUCTIONS
This log should be maintained for every HF, District and Region, and an overall one for national
level DQAs. It should be updated every time a DQA is conducted so as to be able to track the
quality trends at that particular HF, district, region etc. The log should be filed together with the
DQA template used and the DQA report.