Data Quality Assurance Framework (DQAF) for the

Global Health Expenditure Data Production Process

Health accounts are records of a country’s health system expenditures over a given period and are an
important policy tool for evaluating progress towards universal health coverage. Health accounts can
improve the financing of health systems by monitoring the extent to which resources are allocated
efficiently and equitably. This helps ensure health services are available and affordable when people need
them.
The Global Health Expenditure Database (GHED) compiled by World Health Organisation
(WHO).makes available health expenditures from countries around the world. To ensure that GHED data
are of good quality a Data Quality Assurance Framework (DQAF) covering the GHED production
process and its products has been introduced. Its benefits include:
 providing a systematic mechanism for facilitating the ongoing identification of quality
problems and possible actions for their resolution;
 giving greater transparency to the processes by which statistics are produced and their
quality is assured and thereby reinforce WHO’s image as a trustworthy provider of good
quality health expenditure indicators;
 providing a mechanism for the exchange of ideas on quality assurance with other producers
and users of statistics, at international and national levels.
The DQAF has five components.
1) The Principles Governing International Statistical Activities formulated by the UN Committee for
the Coordination of Statistical Activities.
2) The set of quality dimensions in three groups, those relating to data quality, those relating to process
quality, and those relating to institutional quality. The last two groups are important because well
designed processes within an appropriate institutional setting provide the basis for data quality.
3) The set of quality guidelines, comprising good practices for assuring quality.
4) The quality assurance and improvement program, comprising a set of procedures for ensuring that
quality is regularly assessed and appropriate quality improvement actions are implemented.
5) The quality self-assessment checklist, comprising a set of questions based on the quality guidelines
and serving as starting point for self-assessment.
More Information
The first four components are summarised in the following pages. The fifth component and a full copy of
the entire document are available from the Costs, Effectiveness, Expenditure and Priority Setting Unit,
World Health Organisation: nha@who.int.

DQAF for the GHED Production Process Page 1 of 5

 Principles Governing International Statistical Activities

1 High quality international statistics, accessible 6 Individual data, or small aggregates subject to
for all, are a fundamental element of global national confidentiality rules, are kept strictly
information systems. confidential and used exclusively for statistical
purposes or for purposes mandated by legislation.
2 The public has a right to be informed about the
mandates for the statistical work of the 7 Erroneous interpretation and misuse of statistics
organisations. are to be immediately appropriately addressed.
3 To maintain the trust in international statistics, 8 Standards for national and international
their production is to be impartial and strictly statistics are based on sound professional criteria
based on the highest professional standards. while also meeting the test of practical utility and
feasibility.
4 Concepts, definitions, classifications, sources,
and methods and procedures are chosen to meet 9 Coordination of international statistical
professional scientific standards and are made programmes is essential to strengthen the quality,
transparent for the users. coherence and governance of international
statistics, and avoid duplication of work.
5 Sources and methods for data collection are
appropriately chosen to ensure timeliness and 10 Bilateral and multilateral cooperation in
other aspects of quality, to be cost-efficient and to statistics contribute to the professional growth of
minimise the reporting burden for data providers. the statisticians involved and to the improvement
of statistics in the organizations and in countries.

Quality Dimensions
Relevance is the degree to which the data address the
purposes for which they are sought by users.
Accuracy is the degree to which the data correctly
estimate what they are designed to measure.
Reliability is the closeness of initially released values
to values subsequent released for same reference
period. Coherence is the degree to which the data are
mutually consistent with other data.
Timeliness is the length of time between data
availability and the event or phenomenon they
describe.
Punctuality is the existence of and adherence to a data
dissemination schedule.
Interpretability is the ease with which users can
understand and properly use the data.
Accessibility is the measure of how readily the data can
be discovered, located and accessed.
Sound Method and Systems is use of international
standards and best practices through all process stages.
Cost-efficiency is the measure of the costs incurred and
resources expended relative to the benefits of the data.
Objectivity, impartiality and transparency are aspects
of the environment that engender trust.
Credibility is the confidence that users have in the
organisation and its data, in other words, its brand
image.

DQAF for the GHED Production Process Page 2 of 5

 Quality Guidelines

The guidelines are organised in two broad

groupings:
1. The guidelines for the GHED production
process itself, which are subdivided into eight
groups based on the eight phases defined in the
Generic Statistical Business Process Model
published by United Nations Economic
Commission for Europe.
2. The guidelines for the methodological, systems
and organisational activities that are part of the
statistical infrastructure that supports the
production process.
A selection of the guidelines (of which there
are about 1000 are presented below.

Specify Needs and Manage User Relations

 Regularly review user requests, queries and comments and identify new user needs.
 Consult with key users on proposed changes to statistical products.
Design Production Process and Supporting Infrastructure
 Define and justify the scope and content of the data collected.
 Identify and review all possible available data sources and select the most appropriate.
Build Statistical Procedures and Systems
 Use the latest ICT to build efficient data collection tools.
 Configure smooth workflows in which all activities fit together without gaps or duplication.
 Test all new procedures and systems before putting them into production.
Collect Data and Manage Provider Relations
 Extract data electronically from national, international, commercial databases.
 Ensure that providers are familiar with the reasons for data collection and minimise burden.
Process Data
 Devise and implement efficient and effective procedures and systems for data verification,
editing, harmonisation, and estimation including a full range of validity checks and edits.
 Collect and analyse operational metadata, e.g. number of countries for which data modelled.
 Facilitate integration and coherence of the data with data from other production processes.
Analyse Data
 Compare with data for previous reference periods and confront with data from other sources.
 View data from all perspectives and ensure an in-depth understanding before dissemination.
Disseminate Statistical Products
 Maximize interpretability of the data by accompanying them with appropriate metadata.
 Ensure products are timely and punctual by adhering to a release schedule.
 Manage communications with users, ensuring that queries are recorded and replied ro.
DQAF for the GHED Production Process Page 3 of 5
Manage and Evaluate Quality
 Conduct an annual quality self-assessment of the production process and products.
 Identify and implement quality improvements based on the results of quality evaluation.
Assure Professionalism, Impartiality, Objectivity, and Transparency
 Ensure data are produced on an objective basis, determined by statistical considerations.
 Explain major changes in methods and data revisions to users.
 Correct errors in data outputs as soon as possible after they are detected, and inform users.
Manage Metadata
 Identify and document the various types of metadata, and the needs for and uses of them.
 Ensure metadata are recorded, preferably automatically, at the time they are created.
Manage ICT Systems and Services
 Map the sequences of activities constituting each phase and automate to extent possible.
 Ensure systems are tested before use and problems recorded and notified to the contractor.
Manage Performance
 Measure, analyse and minimise the work effort involved in each phase of the process.
 Ensure good communication channels within the GHED team and with providers and users.

Quality Evaluation and Improvement Programme

The quality assurance programme comprises three types of quality evaluation:
 monitoring process quality and performance;
 annual quality self-assessment of the process, products and supporting services;
 external quality evaluation of the process, products and supporting services.

Monitoring process quality and performance

Quality and performance indicators (QPIs) are a means to quickly check ongoing operations, to monitor
performance with respect to target objectives, and to identify sources of operational errors and correct
them on the spot. They also provide information for an evaluation at the end of the process. QPIs are very
carefully chosen. Too few QPIs, or the absence of QPIs for key procedures, result in ineffective
monitoring. Too many QPIs, or ill-chosen ones, overload the production procedures and are a waste of
resources.
Quality and performance targets are a mechanism for focusing attention on quality improvement. They,
too, are carefully chosen. Too many overload the process; ill-chosen targets lead to poor resource
allocation.
Quality Gates are manual inspection points where clearance is required before the process can continue.
Given they are resource intensive they are carefully designed and judiciously used. They are based on a
process review and identification of points of greatest quality risk.
Quality Self-Assessment
A quality self-assessment is conducted at the completion of each production cycle. Its objectives are to
measure the quality of the process and products, and hence to identify weaknesses and to propose quality
improvements to be implemented before the next production cycle. It is facilitated by use of the quality
assessment checklist. It involves meetings of the production team to review the QPIs and any other
measures of process performance, meetings with key users to review the products and their uses, and
meetings with key data providers to review the data sources and acquisition methods.

DQAF for the GHED Production Process Page 4 of 5

External Quality Evaluation
An external quality evaluation is appropriate on an occasional basis, or if user or stakeholder concerns
about the quality of products or processes reach a high level. The aims of the evaluation are to obtain an
objective view of the quality of the process and its products and hence to identify any structural
weaknesses quality improvements to address them. It typically starts with and elaborates the most recent
quality assessment.

DQAF for the GHED Production Process Page 5 of 5