1 Philippine Methods Guide for

2 Health Technology Assessment

3 First Edition
4 Department of Health - Philippines
5 2020
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

1
 1 Acknowledgements

2 Working Group

3 HTA Unit Members

4 Marvinson S. Fajardo, RN, MD, MPM
5 Anne Julienne M. Genuino, RPh, MSc
6 Anna Melissa S. Guerrero, MD, MPH (HTA)
7 Sarah May L. Obmaña, RPh
8 Christelle Joy L. Reyes, RPh
9 Deinzel R. Uezono
10 Yves Miel H. Zuñiga

11 Contributors
12 Ioana Ursu

13 Peer Reviewers
14 Dr. Maria Carinnes Alejandria, PhD
15 Dr. Yot Teerawattananon
16 Dr. Aleli Kraft, PhD
17 Alia Luz
18 Imperial College London Team
19 Dr. Hilton Lam
20 Dr. Marissa Alejandria

21 Development Partners
22 HITAP
23 EU
24 WHO
25 UNICEF
26 IDSI
27
28 Gratitude is extended to *experts/individuals, organisations, institutions* who participated in the
29 consultations and who were instrumental in the development of these guidelines.
30
31
32
33
34
35
36
37

2
 1 Conflict of Interest Declaration
2 *insert relevant declarations of any contributor*
3
4 Contact Information
5 DOH/PBC Address
6 hta.phl@gmail.com / hta@doh.gov.ph
7 Contact numbers
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

3
 1 Table of Contents
2 Acknowledgements 2
3 Conflict of Interest Declaration 3
4 Contact Information 3
5 Abbreviations 6
6 Glossary 8
7 Foreword 9
8 Message of the Secretary of Health 11
9 Message of the Undersecretary of Health 12
10 CHAPTER 1 13
11 BACKGROUND 13
12 1.1. What is a health technology? 13
13 1.2. What is Health Technology Assessment (HTA)? 13
14 1.3. What is the legal and policy framework of HTA in the Philippines? 13
15 1.4. What is the general process of HTA in the Philippines? 13
16 1.5. What is the purpose of this document? 14
17 1.6. What is the scope of this document? 15
18 1.7. Who is the target audience of this document? 15
19 1.8. What is the process of development of this document? 16
20 1.9. How will this document be updated? 16
21 CHAPTER 2 17
22 METHODOLOGICAL STANDARDS IN EVALUATION 17
23 2.1. Basic Methodological Framework of HTA in the Philippines 17
24 2.2. Components of an HTA Report 18
25 2.3. Defining the HTA Decision Problem 19
26 2.4. Scoping and protocol development 23
27 2.5. Assessment 24
28 2.5.1. CLINICAL ASSESSMENT 24
29 2.5.1.1. Location and selection of studies 25
30 2.5.1.2. Critical Appraisal of Clinical Evidence 31
31 2.5.1.3. Synthesis of Clinical Evidence 32
32 2.5.1.4. Algorithm of Clinical Assessment Stage 34
33 2.5.2. ECONOMIC ASSESSMENT 35
34 2.5.2.1. What is an economic evaluation? 35

4
 1 2.5.2.2. Rationale behind the use of economic evaluation 35
2 2.5.2.3. Components of the Economic Evaluation 36
3 2.5.2.4. Selection of the Type of Economic Evaluation 38
4 2.5.2.6. Cost Estimation 39
5 2.5.2.7. The Philippine Reference Case 41
6 2.5.2.8. Modelling 46
7 2.5.2.8.1. Model Development 49
8 2.5.2.8.2. Input Parameter Estimation 50
9 2.5.2.8.3. Model Validation 50
10 2.5.2.9. Presentation of Results 51
11 2.5.2.10. When is a health technology considered to be “cost-effective” in the Philippines 52
12 2.5.2.11. Appraisal of Economic Evaluations 52
13 2.5.2.12. Budget Impact Analysis 52
14 2.5.2.3. ASSESSMENT OF ETHICAL, LEGAL, and SOCIAL IMPLICATIONS (ELSI) 58
15 2.5.2.4. ASSESSMENT OF HEALTH SYSTEM IMPACT 59
16 REFERENCES 60
17 Annexes and Appendices 62
18 Annex 1 - Methodologic Principles of HTA 62
19 Annex 2 - Conflict of Interest Declaration Form for Assessment Team 63
20 Annex 3 - HTA Scoping Tool 66
21 Annex 4 - Data Sources: Relevant databases 69
22 Annex 5 - Data Sources: Input for HTA 70
23 Annex 6 - Clinical Assessment Protocol Template 78
24 Annex 7 - Critical Appraisal Tools 79
25 Annex 8 - Sample Data Extraction Table 80
26 Annex 9 - Sample Costs Table 81
27 Annex 10 - Adjusting for inflation and currency changes 82
28 Annex 11 - Summary of the Philippine Reference Case 83
29 Annex 12 - Duration of Assessment 84
30 Annex 13 - Links to Required Attachments 85
31 Annex 14 - Commonly used tools and methods for ethical evaluation in HTA 86
32
33
34

5
1 Abbreviations
Acronym Description

AO Administrative order

BIA Budget impact analysis

CBA Cost-benefit analysis

CE Cost-effectiveness

CEA Cost-effectiveness analysis

CMA Cost-minimization analysis

CUA Cost-utility analysis

DALY Disability-adjusted life year

DOH Department of Health

ELSI Ethical, Legal, and Social Implications

FEC Formulary Executive Council

HTA Health technology assessment

HTAC Health technology assessment council

HTAU Health technology assessment unit

HRQOL Health-related quality of life

ICER Incremental cost-effectiveness ratio

MA Meta-analysis

NMA Network meta-analysis

PhilHealth Philippine Health Insurance Corporation

PICOT Population, Intervention, Comparator, Outcome, Timeframe

PNF Philippine National Formulary

PNFS Philippine National Formulary System

PSA Probabilistic sensitivity analysis

QALY Quality-Adjusted life year

RA Republic act

RCT Randomized clinical trial

6
 SR Systematic review

UHC Universal health care

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

7
1 Glossary
2
3
4
5

8
 1 Foreword
2
3 The Philippine HTA Methods Guide is a vital document that aims to provide guidance to
4 all those involved in conducting health technology assessment (HTA). HTA serves to inform the
5 funding and coverage decisions of the Department of Health (DOH) and the Philippine Health
6 Insurance Corporation (PhilHealth) through evidence-based recommendations of the Health
7 Technology Assessment Council (HTAC).
8
9 The role of HTA in the Philippines has grown and evolved over the years with its first
10 stages of development seen in PhilHealth in the early 2000’s informing coverage decisions for
11 medicines and medical procedures. In recent years, HTA was used by the Formulary Executive
12 Council (FEC) responsible for determining the inclusion of drugs in the PNF incorporating formal
13 methods of systematic reviews and economic evaluations to determine the value for money and
14 pricing of new medicines and vaccines.
15
16 The DOH envisions to bring effective, efficient, accessible, affordable and quality
17 healthcare to all Filipinos as mandated by the Universal Health Care (UHC) Act. Hence, it is
18 important that the use of health interventions and technologies within the national healthcare
19 system is based on rigorous assessments that follow common standards to ensure that they bring
20 value in improving health outcomes of Filipinos.
21
22 For the first time, the DOH is making explicit the concept of the ‘reference case’ which
23 specifies the methods and technical aspects of health technology assessments to be considered
24 by the HTAC and its subcommittees. The development of the reference case draws from the
25 experience in assessing medicines for the Philippine National Formulary (PNF) to make
26 recommendations for other health technologies such as medical and surgical procedures,
27 screening and diagnostic interventions, medical devices and other health technologies that are
28 relevant to national healthcare needs. The reference case will therefore be useful to academics,
29 health professional bodies, patient organizations, industry, and other sponsors when submitting
30 proposals for public funding to ensure that they adhere to the submission, assessment, and
31 reporting standards. While the ‘reference case’ may facilitate a consistent and predictable
32 approach in determining the value of health technologies specific to our local context, it only
33 serves as a guide and it is possible that strict adherence to the reference case may not be possible
34 in all situations.
35
36 With the increasing public demand for new health interventions and the modernization of
37 health services under UHC, there is also a clear need to expand the scope of HTA in the
38 Philippines to produce good quality evidence for other emerging health technologies such as
39 biologics, targeted therapies, orphan drugs, hospital medical equipment and devices, diagnostic
40 and screening tools, and preventive and promotive health services. However, it must be
41 recognized that other health technologies may require different types of evidence on
42 effectiveness, costs, and social preferences as well as different methodological standards to
43 assess their potential value to our health care system.
44

9
 1 The contents of this document should be interpreted with the HTA Process Guide which
2 separately details the step by step procedures in undertaking HTA from topic selection to
3 dissemination of the decision, as well as the roles and responsibilities of the different accountable
4 government bodies and stakeholders.
5
6 This HTA Methods Guide does not aim to cover in detail the specific assessment
7 standards for each broad classification of health technologies but instead endeavors to provide
8 broad guidelines in the conduct of systematic reviews and economic evaluations and the
9 production of standard HTA reports for healthcare decision makers and other target audiences in
10 the health system. It is expected that future guidance will be issued to include more technical
11 aspects and special considerations relevant to other health technologies.
12
13 The DOH, in partnership with stakeholders in the academe, health professional bodies,
14 the industry, patient groups and the public shall continue to review and enhance the current HTA
15 process and methodological standards for assessing a broad range of health technologies to
16 ensure that they remain up-to-date, relevant, and responsive to present and future Philippine
17 healthcare needs.
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

10
 1 Message of the Secretary of Health
2
3 (Electronic signature)
4 Dr. Francisco Duque III, MSc
5 Secretary of Health
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

11
 1 Message of the Undersecretary of Health
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

12
 1 CHAPTER 1
2 BACKGROUND
3

4 1.1. What is a health technology?

5 As defined in the implementing rules and regulation of the RA 11223 or the UHC Act, a health
6 technology refers to the application of organized knowledge and skills in the form of devices,
7 medicines, vaccines, procedures and systems developed to solve a health problem and improve
8 quality of lives (Philippines, 2019). Health technologies include pharmaceuticals, devices,
9 procedures and organizational systems used in health care.

10 1.2. What is Health Technology Assessment (HTA)?

11 Health technology assessment (HTA) is defined under the RA 11223 or the UHC Act as a
12 systematic evaluation of properties, effects, or impact of health-related technologies, devices,
13 medicines, vaccines, procedures, and all other health-related systems developed to solve a health
14 problem and improve the quality of life and health outcomes, utilizing a multidisciplinary process
15 to evaluate the clinical, economic, organizational, social, and ethical issues of a health
16 intervention or health technology.
17
18 1.3. What is the legal and policy framework of HTA in the Philippines?
19 The following laws and national policies serve as the legal basis of HTA as a tool for healthcare
20 decision makers in making evidence-based decisions on the funding and use of health
21 technologies in the Philippine healthcare system:
22 ● RA 10606: National Health Insurance Act of 2013 mandated the utilization of HTA in
23 guiding PhilHealth’s coverage of health services (Congress of the Philippines, 2013).
24 ● AO 2016 - 0034: The New Implementing Guidelines of the Philippine National
25 Formulary System (PNFS) required that any medicine considered for inclusion in the
26 PNF shall be assessed its benefit-risk assessment (safety and efficacy), cost-
27 effectiveness, affordability and public health relevance. (Department of Health, 2016).
28 ● AO 2018-0026: Framework for the Use of HTA to Guide Coverage Decisions in
29 Support of UHC provided an explicit framework for a systematic and consistent use of
30 HTA to guide health coverage decisions of both the DOH and PhilHealth (Department of
31 Health, 2018).
32 ● RA 11223: Universal Healthcare Act institutionalized HTA as a fair and transparent
33 priority-setting mechanism that shall be recommendatory to the DOH and PhilHealth for
34 the development of policies, programs, regulations, and the determination of a range of
35 entitlements such as drugs, medicines, pharmaceutical products, other devices,
36 procedures, and services (Congress of the Philippines, 2019).

37 1.4. What is the general process of HTA in the Philippines?

38 The following figure outlines the general process of conducting an HTA in the Philippines. The
39 Philippine HTA Process Guide contains the step-by-step process and the roles of the different
40 stakeholders involved in each step:

13
 1

2
3 Figure 1. General process of conducting HTA in the Philippines

4 1.5. What is the purpose of this document?

5 The purpose of this document is to provide mandatory guidance to researchers in conducting
6 HTA based on the prioritized topics (mainly Assessment stage as illustrated in the HTA process
7 flow above) and producing a standardized HTA report that will be used to inform all public
8 funding, coverage, and resource reallocation/ optimization decisions made by the DOH and
9 PhilHealth.
10
11 The specific objectives of this document are as follows:
12 ● To provide clear guidelines to researchers on methodological standards in evaluating the
13 clinical and economic value of health technologies as well as other relevant aspects on
14 the adoption of health technologies including sources of data in conducting assessments

14
 1 ● To ensure the consistency of methods and reporting standards of researchers so that HTA
2 reports meet the needs of the HTA Council and health care decision makers for timely,
3 comprehensive, reliable, relevant, rigorous and accurate evidence on health technologies
4 ● To give guidance to health care professionals, patients, and the general public on the
5 decision framework of the HTAC and healthcare decision makers
6
7 1.6. What is the scope of this document?
8 HTA aims to produce recommendations for healthcare decision makers on the use of the following
9 health technologies:
10 ● Medicines
11 ● Vaccines
12 ● Medical devices
13 ● Medical and surgical procedures
14 ● Screening procedures and diagnostics
15 ● Promotive and preventive health interventions
16 ● Traditional and complementary medicine
17 ● Other health interventions and technologies such as systems, organizations, and
18 delivery systems of care
19
20 This document, however, only applies to pharmaceutical products to be listed in the PNF,
21 different health technologies and services delivered by DOH public health programs,
22 vaccines covered under the National Immunization Program, and the benefit packages
23 developed and covered by PhilHealth.
24
25 1.7. Who is the target audience of this document?
26 The target audience of this document are researchers aiming to produce an HTA report,
27 whether they are internal or external to the DOH and PhilHealth. This also serves as guide to
28 the HTA Council and decision-makers on the methodological standards that are required in HTA
29 to inform their recommendations and decisions. While HTA Reports may be done internally by
30 the HTA unit or externally commissioned academics, the DOH also allows the industry and other
31 stakeholders to provide submissions adhering to the methodological standards for good quality
32 analysis as contained in this Methods Guide.
33
34 Nonetheless, since HTA implementation has to align itself with various processes involving
35 existing policies and programs within the DOH and PhilHealth which covers, implements and
36 monitors the delivery of various health technologies, the target audience of the HTA report itself
37 would include not only the HTA Council and major healthcare decision makers but also the
38 following key stakeholders:
39 ● National DOH program managers
40 ● Health authorities from municipal, city, and provincial health governments
41 ● Healthcare providers
42 ● Healthcare organizations
43 ● Patient organizations

15
 1 1.8. What is the process of development of this document?
2 This document draws on the experience gained on the use of HTA by both the DOH and
3 PhilHealth for funding decisions on medicines and other health technologies. This was also based
4 on early versions developed in partnership with academe (Lam et al., n.d.-a) from the University
5 of the Philippines-Manila (UPM) in 2016 and development partners such as the European Union
6 – Health Sector Reform Contract (EU-HSRC) in 2018 (Ursu, 2018), the United Nations Children's
7 Fund (UNICEF), International Decision Support Initiative (iDSI) and the World Health Organization
8 (WHO).
9
10 Moreover, this Methods Guide incorporates recent global developments in methodological
11 standards for conducting HTA in more advanced HTA systems but with particular interest to
12 recent normative guidelines developed for low- to middle-income countries (LMICs) such as the
13 iDSI Reference Case for Economic Evaluation.
14
15 The work proceeded as follows:
16 ● Review of literature based on publicly-available resources from major HTA agencies and
17 HTA Networks (EunetHTA, INAHTA)
18 ● Review of local HTA and HTA-related reports to identify methodological weaknesses and
19 address them in this document
20 ● Identification of gaps in the current HTA methods in the Philippines in consultation with
21 clinical, health economics, and methods experts
22 ● Round table discussions and public consultation with stakeholders to ensure that the
23 methodological standards consider practical issues and limitations in the Philippines while
24 meeting the needs of HTAC and decision makers
25 ● Differences in views and methodological variations were resolved through consensus with
26 appropriate stakeholders
27
28 1.9. How will this document be updated?
29 This is the official first edition of the HTA methods guide for the Philippines. A review of this
30 document shall be done on a regular basis to draw on advances in HTA methodologies,
31 continuously improve based on past experience, and adapt to the dynamic nature of the Philippine
32 healthcare context. The review shall be rigorous, transparent, and consultative.
33
34
35
36
37
38
39
40
41
42

16
 1 CHAPTER 2
2 METHODOLOGICAL STANDARDS IN EVALUATION

3 2.1. Basic Methodological Framework of HTA in the Philippines

4
5 Figure 2. Basic Methodological Framework of HTA in the Philippines
6
7 In conducting a health technology assessment, assessment teams should be guided by a clear
8 framework on the relevant questions to be addressed in assessing the overall value of a health
9 technology to the national health care system which includes the clinical, economic, social,
10 ethical, legal, organizational and other relevant aspects on the use of the health technology such
11 as patient preferences and values. Figure 2 illustrates the basic methodological framework for
12 each prioritized topic for evaluation by the HTA Council.
13
14 a. Research questions, in Population-Intervention-Comparator-Outcome-Timeframe
15 (PICOT) format, shall be formulated from topics nominated by topic sponsors (e.g., DOH
16 and PhilHealth, professional bodies, industry, patient organizations) and subsequently
17 prioritized by the HTA Core Committee.
18
19 b. For all prioritized topics, a study protocol for clinical assessment stating the research
20 questions, objectives, and methods shall be developed by the assessment team
21 composed of technical researchers, a clinical expert, and an information specialist. The
22 study protocol will be drafted with the assistance of HTA Subcommittee members and
23 relevant clinical experts. The result of the clinical assessment shall serve as the basis of
24 the second stage of the assessment which is the economic evaluation.
25
26 c. Cost-effectiveness analysis (CEA) shall be conducted for health technologies that exhibit
27 significant and meaningful clinical outcomes compared to the comparator. For health

17
 1 technologies which exhibit non-inferior or comparable clinical benefits versus the
2 comparator, a cost-minimization analysis (CMA) shall be conducted.
3
4 d. Economic evaluations shall include both a CEA or a CMA and a budget impact analysis
5 (BIA) to assess both the value for money of a health technology and its affordability or the
6 feasibility of publicly funding the health technology relative to the Philippine context.
7
8 e. All HTA Reports should be contextualized for the Philippines and must include the
9 analysis of ethical, legal, social, and organizational and health system implications when
10 so necessary to facilitate the understanding of various factors, issues, and barriers
11 relative the adoption of health technologies from the local perspective. This may include
12 consideration of relevant national and international laws and policies that may influence
13 the uptake of the health technology in both the national and local settings.
14

15 2.2. Components of an HTA Report

16 A standard HTA report should include the following sections as contained in Table 1.
17
18 Table 1. Parts of a Standard HTA Report*
Sections
I Executive Summary
● Concise overview of context, research question, methods, key results,
conclusion, and recommendations
● List of authors and contact details for further information
● Statement regarding conflict of interest
● Checklist of confidential information
II Health problem and clinical management options
● Burden of disease (both magnitude and severity)
● Current management options
○ Local Guidelines and Practice
○ Accessibility of treatment options (based on 5A’s: Availability, Adequacy,
Accessibility, Affordability, and Appropriateness)
○ Existing government policy and reimbursement mechanism
III Description, technical characteristics, and use of the health technologies
● Proposed intervention
● Comparator/s
IV Clinical effectiveness and safety
● Best available evidence comparing the clinical performance in terms of benefits
and harms of the proposed intervention when compared with its comparator/s
● Therapeutic conclusion on the superiority or non-inferiority of the proposed
intervention
V Economic evaluation
● Cost-minimization analysis OR;
● Cost-utility/Cost-effectiveness analysis
● Budget Impact Analysis
VI Ethical analysis

18
 Ethical considerations including population factors and health equity issues which may
impact on the use of the health technology among all eligible patients and
consequently the fair distribution of health outcomes across the Philippine population
VII Legal aspects
Legal and policy statutes in the Philippines and relevant international
treaties/agreements which promote, facilitate or prohibit the adoption and use of the
health technology
VIII Social aspects
Social, cultural, religious and other factors which affect the acceptability of the adoption
and use of the health technology in the Philippines for the general population and/or
specific subgroups of the population

Patient preferences and values which need to be considered by decision makers such
as the acceptability of the health intervention and its convenience of use which may
impact on various aspects of the lives of patients as well as their families and
caregivers
IX Health System Impact
Likely organizational and health system impacts which may influence the adoption or
implementation of the health technology in the national context or in particular localities
of the country (e.g. infrastructure, human resource needs, training requirements,
changes in benefit schedule of Philhealth)
X Relevant attachments (e.g., protocol, electronic copies)
1 *based on the domains of assessment in the HTA Core Model® (European Network for Health
2 Technology Assessment, 2018)
3
4 a. Recent and good quality synthesis of relevant clinical, economic, legal, social, ethical and
5 health system evidence is required for all health technologies prioritized for assessment
6 by HTAC to demonstrate its potential value relevant to the Philippine context. The key
7 aspects of the evidence which must form the HTA Report are as follows:
8 ○ Clinical evidence – clinical effectiveness and safety information of the health
9 technology
10 ○ Economic evidence – costs, cost-effectiveness analysis, budget impact analysis
11 concerning the use of the health technology
12 ○ Ethical considerations – equity and fairness of coverage decisions
13 ○ Legal and policy framework
14 ○ Social and cultural context – social acceptability and cultural factors including
15 patient and caregiver preferences and values
16 ○ Health system impact
17
18 b. The project duration of assessment per stage is shown in Annex 12.
19

20 2.3. Defining the HTA Decision Problem

21 Not all policy questions are answerable by HTA. HTA questions, in our context, aim to respond
22 to national-level service coverage or disinvestment decisions to ensure proper allocation of
23 resources.

19
 1
2 It attempts to answer the following questions:
3 ● Does the technology work? Is it safe?
4 ● Is there meaningful improvement in health status relative to its cost?
5 ● Which patients benefit the most?
6 ● Can we afford to pay for all people who might need the technology?
7 ● What other considerations make this technology important?
8
Examples of policy questions answerable by HTA:
1. What is the optimal screening strategy for colorectal cancer in the Philippines in
terms of type of interventions (e.g. fecal occult blood test, sigmoidoscopy or
colonoscopy), frequency of the test, and population coverage (e.g. age, gender,
health risk)?
2. Should the subcutaneous form of the drug be subsidized instead of its intravenous
form for cancer patients?
3. Should the eye health program cover eye convergence testing for particular groups
of children?
4. Should DOH continue the coverage of adjuvant trastuzumab versus chemotherapy
alone for HER2-positive early-stage breast cancer patients? And at what price?

Examples of policy questions not directly answerable by HTA:

1. Should DOH approve the conduct of a primary study to determine the
effectiveness of Wolbachia to reduce dengue incidence?
2. What is the significance of medical cannabis in the economy of the Philippines?
9
10 After clarifying the policy question, a focused HTA research question should be
11 developed. This requires, at the minimum, specifying the target population (P),
12 intervention (I), comparator (C), relevant outcomes (O), and timeframe (T) to form the
13 PICOT question.
14
15 The research question should be delineated across 5 elements (PICOT):
P: Population of interest The targeted population with a certain disease or health
condition that may likely benefit from the introduction of the
new technology

Note:
● Important characteristics such as age, sex, disease
severity, comorbidities, setting (i.e., in-patient or
outpatient) should be included to the extent
possible, to further specify the population.

I: Intervention/s The health technology/ technologies considered for

assessment

Note:

20
 ● The role of the intervention/s in the current clinical
pathway should be defined.
● For all health technologies, the following must be
defined: treatment intensity, setting (e.g., primary
care, health centre, or home), co-interventions, and
duration of use. It should also be specified if it is a
replacement to the current treatment, an adjunct, or
sequential therapy.
● Additional details for drug interventions: dosage
form, route of administration (e.g., intravenous or
oral), dose

C: Comparator/s Current health technology/ technologies or standard/s of

care or most prevalent practice used in the Philippines in
the targeted population which may be replaced by the new
intervention

A “do nothing” comparator should be explored if it is the

current practice in the Philippines.

Note:
● Substitute, adjunct, adjuvant for drugs
● Comparators may not always be alternative
interventions but can be different ways of
administering the same intervention, such as
different regimens or treatment sequences.
● Comparators may be drug/s listed in the PNF,
services currently offered by PhilHealth, treatment
indicated in clinical practice guidelines by
professional societies, or widely accepted treatment
among clinicians.

O: Outcome Defining the relevant outcome to measure in order to

establish the effect of the intervention

Note:
Clinical outcomes
● The clinical benefit should have been measured on
internationally recognized measuring scales and
wherever possible, the trial outcome(s) should be a
hard endpoint(s) (e.g., survival) and clinically
meaningful.
● Specify primary and secondary outcome.
● Surrogate points are acceptable only to the extent
based on clear epidemiological evidence that there
is a direct link to the primary outcome of interest and
to which physicians find them clinically meaningful
and should have been validated by the literature
and in the stakeholder consultation.

21
 Economic outcomes
● Incremental cost-effectiveness ratio (ICER) and
Cost-effectiveness acceptability curves (CEAC);
Budget impact (to be discussed further in later
chapters)

ELSI and Health System outcomes

● Relevant ethical, legal, social, and health system
implications depending on the health technology

T: Timeframe The time it takes to demonstrate the chosen outcome/s and

the duration of time in which the participants will be
observed

Note:
Timeframe of outcomes may vary for the clinical, economic,
ELSI, and health system research questions.
1
2
3
4 For example:
Sample HTA policy question:
Should the Philippine government introduce pneumococcal conjugate vaccine in the
expanded program on immunization (EPI) for the prevention of childhood mortality and
mobility due to invasive pneumococcal diseases (IPD), clinical pneumonia, and acute otitis
media (AOM)?
(Haasis et al, 2015:
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0131156)

5
6 Sample HTA research questions
7 Clinical research question:
8 ● What is the clinical effectiveness of pneumococcal conjugate vaccine (I) compared to
9 status quo (C) in preventing mortality and morbidity due to IPD, clinical pneumonia, and
10 AOM? (O) of children under 5 years old (P) over their life-time (T)?
11
12 Economic research question:
13 ● Does pneumococcal conjugate vaccine represent good value for money in the
14 Philippines for preventing mortality and morbidity due to IPD, clinical pneumonia, and
15 AOM (O) of children under 5 years old (P) over their lifetime (T)?
16 ● What are the budget implications of introducing pneumococcal conjugate vaccine in
17 the EPI program?
18
19 Ethical, Legal, Social, and Organizational or Health System research question:

22
 1 ● Are there any ethical, legal, social, and organizational implications of introducing
2 pneumococcal conjugate vaccine in the Philippines?
3

4 2.4. Scoping and protocol development

5 a. Scoping refers to the process of defining the overall scope or focus of the health
6 technology assessment in terms of at least the following components:
7 ● population of patients who will benefit from the intervention
8 ● health technology or intervention of interest
9 ● appropriate comparators relevant to the local practice or context
10 ● clinically meaningful outcomes
11 ● timing and setting where the intervention will be used
12 ● any other consideration that will likely impact the results of the assessment
13 such as appropriate perspective, equity issues, etc.
14
15 It is important to undertake this step to formulate the most appropriate policy question that
16 needs to be answered and ensure that relevant considerations are included in the
17 technology assessment/report which shall form the basis of HTAC recommendations.
18
19 b. Scoping is done through the review of both published and unpublished literature on the
20 topic of interest which involves formal scientific database searches (for clinical
21 effectiveness and safety data), official government and health organization databases
22 (such as DOH and PhilHealth for relevant local policies) including websites of other
23 international HTA agencies (for HTA reports), grey literature, and local standard
24 practice guidelines (for treatment pathway) to understand and formulate the most
25 appropriate research questions relevant to the Philippines including contextual factors
26 associated with the use and value of the health technology of interest. Other sources of
27 information in scoping the HTA report are as follows:
28 ○ Epidemiological data from the DOH Epidemiology Bureau or international
29 organization reports such as the Institute for Health Metrics and Evaluation
30 (IHME) and WHO
31 ○ Published and unpublished analyses from local hospitals or medical
32 societies
33
34 It is recommended to refer to official websites of major international HTA agencies to
35 determine if assessments on the topic have been already conducted or in the process of
36 being performed by other HTA bodies to avoid duplication and also understand the use
37 and experience in the adoption of the health technology in other health systems.
38
39 c. Stakeholder consultations with clinical experts, DOH program managers, industry
40 representatives, patients, healthcare organizations, and other relevant health system
41 partners should be done to ensure that all relevant considerations are included in the HTA.
42 The panel of stakeholders should agree on the HTA policy question (i.e. PICOT).
43
44 d. Guide questions in scoping the different components of the HTA Report are
45 summarized in Appendix 3.

23
 1
2 e. All data collected in the literature search and meeting inputs will be summarized and
3 presented to HTAC to finalize and approve the HTA policy question.
4
5 f. HTA Core Committee shall endorse the approved topics and policy question to the
6 respective subcommittees which shall then delegate the topics to the assessment teams.
7
8 g. The HTA Policy and Evaluation Unit through its internal and external Assessment Teams
9 shall then draft the study protocol with the scope of the assessment through a review of
10 literature on the specific health technologies, expert advice or contact of
11 manufacturers/companies as necessary. The protocol development shall include scoping
12 with relevant stakeholders to refine and validate the assessment methods and scope.
13
14 h. The study protocol is approved by the HTAC Subcommittee and published in the DOH
15 website to show the anticipated time frame for completing the appraisals to stakeholders.
16

17 2.5. Assessment
18 a. Assessment refers to the application of formal scientific methods of evidence synthesis
19 to assess the clinical, economic, health system, ethical, legal and social impact of covering
20 or disinvesting a particular health technology in the local Philippine context.
21
22 b. The general steps in the assessment stage include a review of clinical evidence which
23 involves evidence synthesis and risk of bias assessment of the included studies, an
24 assessment of costs, and economic evaluation to determine the value for money of the
25 health technology, in terms of both its costs and outcomes, relative to the perspective of
26 the purchasing agent which may be DOH or PhilHealth.
27
28 c. The assessment may also include other important aspects such as the ethical, legal,
29 social, and health system implications with the use of the health technology in the
30 local Philippine context which may entail additional study methodologies including case
31 study, field surveys, key informant interviews, focused group discussions, policy and legal
32 analysis among others.
33
34 d. Members of the assessment team must meet the required criteria for authorship
35 (refer to the uniform requirements of authorship by the International Committee of Medical
36 Journal Editors), depending on the expertise required by the type of the clinical
37 assessment (i.e., rapid review, systematic review).
38

39 2.5.1. CLINICAL ASSESSMENT

40 a. The objective of the clinical assessment stage is to identify and synthesize all eligible
41 clinical studies which report on the benefits and harms relevant to the PICOT clinical
42 research question.

24
 1 b. The clinical value of the health technology as regards to whether it is non-inferior or
2 superior to the comparator, as well as the size of relative treatment effect,
3 confidence intervals, and statistical p value should be reported.
4 c. Safety outcomes and risks should also be reported relative to its comparator if they are
5 clinically meaningful and/or they increase or lead to additional health expenditures as a
6 result of treating the adverse effects.
7 ● Data on the costs of the management of side effects should be collected and used
8 as input in the economic evaluation.
9 d. For the purposes of HTA, well-designed and well-conducted systematic reviews with
10 or without meta-analysis are considered the best source of evidence due to the
11 rigorous and comprehensive approach to search, appraise and synthesize all relevant
12 studies and the larger statistical power resulting from the combination of several studies
13 (if applicable), compared to single studies.

14 2.5.1.1. Location and selection of studies

15 a. To systematically locate relevant studies that should be included in the assessment, an
16 extensive literature search should be conducted using relevant scientific databases. A
17 protocol (Annex 6) has to be developed a priori to minimise the potential for bias.
18 b. An expert from the relevant HTAC Subcommittee and a member of the Methods
19 Working Group shall oversee the development of the study protocol and provide
20 feedback with regard to its consistency with the acceptable methodological standards and
21 its responsiveness to the decision needs of the HTAC.
22 c. As part of the protocol, a search strategy should be formulated with search terms drawn
23 from the ‘population’ (P) and ‘intervention’ (I) components of the research question using
24 keywords with their synonyms and free text terms as well as controlled vocabulary or
25 Medical Subject Headings (MeSH) in particular databases such as PubMED/MEDLINE.
26 The PRESS Checklist for Search Strategies, a tool for peer review electronic literature
27 search strategies which can be found at
28 https://www.sciencedirect.com/science/article/pii/S0895435616000585, may be used as
29 a guide (McGowan et al., 2016).
30 d. The search strategy should be as comprehensive as possible with the search preferably
31 performed first in the MEDLINE (OVID) platform then subsequently translated and
32 performed in the other scientific databases such as:
33 ○ Cochrane Register of Controlled Trials (CENTRAL)
34 ○ Cochrane Database of Systematic Reviews (CDSR)
35 ○ EMBASE (OVID)
36 ○ National Health Service Economic Evaluation Database (NHS EED)
37 ○ HERDIN
38
39 Other scientific databases which may also be searched as appropriate to the HTA
40 topic are shown in Appendix 4.
41
42 e. It is recommended to also refer to official websites of major international HTA agencies to
43 determine if clinical assessments on the topic have been already conducted or in the

25
 1 process of being performed by other HTA bodies to avoid duplication and also understand
2 the use and experience in the adoption of the health technology in other health systems.
3 The search strategy must include grey literature to identify unpublished studies including
4 conference abstracts and proceedings that were not identified through the electronic
5 database search.
6 f. Date and language restrictions should be avoided in the search strategy to ensure a broad
7 and optimized search. However, in specific cases, search filters or restrictions may be
8 applied to target a specific population, time period, or study design depending on the
9 inclusion criteria of the study and the breadth of the literature on a particular health
10 technology.
11 g. Search results must be downloaded to a reference management system (e.g.,
12 Mendeley, Endnote) for ease of retrieval of citations. Data extraction tables (e.g., through
13 Google tools such as sheets and drive, Microsoft Excel) must also be used for appropriate
14 documentation of the search (date, source, terms used, number of results). A sample
15 data extraction table may be found in Annex 8.
16 h. Each study must be assessed to determine whether it meets the inclusion criteria of the
17 review. A log of ineligible studies should be maintained with the rationale for why studies
18 were included or excluded using ineligibility coding. Having more than 1 reviewer assess
19 all records retrieved by the search strategy is recommended to increase the validity of the
20 study selection process.

If the systematic search has identified a recent SR addressing the research

question that is applicable to the local setting, this may be adopted upon quality
appraisal. An SR is considered recent if its last search was conducted three years before
the present time.

However, conducting a systematic review or rapid review is needed if the

systematic search has not identified a recent SR that may adopted or no relevant SR
exists at all (and several primary or synthesized studies exist).

The next sections shall elaborate on systematic review and rapid review.

21
22 Systematic Review
23 a. A systematic review (SR) attempts to collate all empirical evidence that fits pre-specified
24 eligibility criteria in order to answer a specific research question. It uses explicit, systematic
25 methods that are selected with a view to minimizing bias, thus providing more reliable
26 findings from which conclusions can be drawn and decisions made (Antman 1992; Oxman
27 1993 as cited in (Higgins et al., 2011).
28 b. In appraising or conducting systematic reviews, appropriate study designs for different
29 types of health technologies should be used in synthesizing the clinical evidence. While
30 randomized controlled trials (RCTs) are considered the ‘gold standard’ in evaluating the
31 efficacy of drugs and other therapeutic interventions, these may not be feasible to conduct
32 for other health technologies. Table 2 summarizes the appropriate study design for each
33 type of question (Dans et al., 2017).

26
 1
2 Table 2. Appropriate study designs
Type of question Study design

Therapeutic Effect SR of RCTs

Preventive Effect SR of RCTs or cohort studies

Adherence or Compliance to SR of cohort studies

Treatment

Safety or Harm SR of cohort or case-control studies

Diagnostic Accuracy SR of cohort studies or studies reporting

“sensitivity and specificity” or “likelihood
ratios” (LRs)
3
4 ● While there are different levels of evidence for different clinical questions and
5 health technologies, the levels of evidence work to guide healthcare providers and
6 decision makers that study designs with Level I evidence generally have more
7 reliable results and minimal bias compared to those with Level 4 evidence (please
8 refer to the table below). While systematic reviews and RCTs are positioned on
9 top of the hierarchy, other study designs such as observational studies may offer
10 the best source of objective evidence for some types of health interventions where
11 randomization and/or blinding may not be feasible or where studies lack a control
12 group. The quality of the evidence must be appraised to assess methodological
13 rigor using validated appraisal tools (to be discussed in later sections). If designed
14 and conducted properly, observational studies may provide more reliable evidence
15 than a poorly designed systematic review or RCT.
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

27
 1 Table 3. Hierarchy of evidence (Harbour R. & Miller, J., 2001)
Level Criteria

1++ High-quality meta-analyses, systematic reviews of RCTs, or RCTs with a very low risk
of bias
1+ Well conducted meta-analyses, systematic reviews of RCTs, or RCTs with a low risk
of bias
1- Meta-analyses, systematic reviews of RCTs, or RCTs with a high risk of bias
2++ High-quality systematic reviews of case-control studies or cohort studies, or high-
quality case-control or cohort studies with a very low risk of confounding bias, or chance
2+ Well conducted case-control or cohort studies with a very low risk of confounding, bias,
or chance

2- Case-control or cohort studies with a high risk of confounding, bias, or chance

3 Non-analytic studies (e.g. case report, case studies)

4 Expert opinion

RCTs= randomized controlled trials

2 ● Expert evidence or expert opinion are acceptable sources of information when

3 published research evidence is missing or inadequate. Experts may include
4 clinicians, patients, or patient group representatives who may have contextual
5 information, insights, values and preferences as well as experiences on the health
6 condition or health technology of interest. It is important to obtain from the experts
7 the evidence which forms the basis of their opinion, document it through formal
8 submission or transparent recording of the discussion, and manage conflicts of
9 interest, if any.
10
11 c. The general steps of conducting an SR are outlined below. Additionally, Table 4 lists
12 the minimum requirements in conducting a systematic review based on standard practices
13 by Cochrane (Higgins and Green, 2011) and CRD (Center for Reviews and Dissemination,
14 2008).
15 1. Scoping the literature
16 2. Forming a team, declaring, and managing conflicts of interest
17 3. Formulation of the research question and development of the protocol
18 4. Systematic search of the literature
19 5. Critical appraisal of included studies
20 6. Data extraction
21 7. Data synthesis
22 8. Interpretation and presentation of the results
23

28
1
2 Table 4. Table of Minimum Requirements for a Systematic Review of Clinical Evidence
Protocol registration PROSPERO or Cochrane Database of Systematic
Reviews

Timeline 12 - 24 months (with a need to update as new evidence

comes up; if not finished within 2 years, it has to be
withdrawn)

Data Sources ● MEDLINE (OVID or PubMed)

● Cochrane Library
● EMBASE

Other potential data sources are listed in Annex 4.

Study selection, critical Independent review of at least 2 authors with

appraisal and data extraction specified method of resolving disagreements

In cases where quantitative pooling of data is possible, a meta-analysis should be

conducted, and the following information should be explicitly specified:

● Outcomes to be pooled
● Method of pooling measures of treatment effect (random or fixed effects model)
● Test for the presence of heterogeneity (Chi² test, I² statistic, Q statistic)
● Method of exploring heterogeneity (meta-regression, subgroup analysis, sensitivity
analysis)
● Method of detecting publication bias (Funnel plot, Deek’s test)

Additionally, if a network meta-analysis is more appropriate, the following information

should also be explicitly specified (Chaimani & Salanti, 2013):

● Evaluation of transitivity assumption

● Model used (hierarchical, meta-regression, multivariate analysis, two-stage
approach etc.
● Method to estimate heterogeneity (Markov Chain Monte Carlo (MCMC), restricted
maximum likelihood (REML) etc.)
● Statistical approach for evaluating network inconsistency (loop-specific, node-
splitting, Lu & Ades model, design-by-treatment etc.)
● Additional analysis for important effect modifiers (e.g., network meta-regression)
● Presentation of information
○ Presentation of the evidence base (network graph, matrix, table etc.)
○ Presentation of results (forest plot, rankogram, SUCRA plot etc.)
○ Presentation of inconsistency in the network (net-heat plot, density plot)
○ Presentation of heterogeneity in the network (graph)

Method of evaluating the To rate the quality of evidence for each outcome reported
quality of evidence and guide the development of recommendations, the
GRADE (Grading of Recommendations Assessment,
Development and Evaluation) system must be

29
 adopted. Summary tables may be generated using
GRADEPro. For each outcome, the following
characteristics must be assessed:
● Risk of Bias
● Consistency
● Precision
● Directness
● Reporting Bias

Quality assessment may also consider looking into the

appropriateness of the study design, choice of outcome
measure, statistical issues, generalizability, and
applicability.

Reporting format PRISMA (Preferred Reporting Items for Systematic

Reviews and Meta-Analyses) including PRISMA Flow
Diagram to map the number of records identified,
included, excluded, and present the reasons for
exclusion. PRISMA-NMA should be used if a network
meta-analysis is involved.
1
2 For more guidance, resources are available online at:
3 ● Cochrane: https://training.cochrane.org/handbook
4 ● Centre for Reviews and Dissemination (CRD):
5 https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf.
6
However, in particular situations e.g. public health emergencies, there is a need to
balance the need for rigor and practical considerations on the decision timelines of
policy makers. Hence, conducting a de novo systematic review may not always be
feasible. This is where a rapid review may be considered. The decision of whether to
conduct a rapid or systematic review should be informed by the results of an initial scan
of the available evidence and advice from the HTAC on the appropriate track for the
review.
7
8
9 Rapid Review
10 a. Rapid reviews (RR) are a form of knowledge synthesis that follows the systematic review
11 process, but the components of the process are simplified or omitted to produce
12 information in a timely manner (Khangura, 2012 as cited in (Dobbins, 2017a). They may
13 vary in scope, methodology, and timeline of preparation (Tricco et al., 2017). A staged
14 search is usually employed in rapid reviews with the objective to first identify existing
15 systematic reviews before studies with other study designs. Table 5 outlines the
16 differences of rapid reviews to systematic reviews.
17
18
19

30
 1
2 Table 5. Rapid Review vs Systematic Review (Dobbins, 2017, Tricco A.C., 2017)
Rapid Review Systematic Review

Timeframe (theoretical) 4-12 weeks 8-12 months

Number of databases At least 1 database At least 3 databases

Grey Literature May or may not be searched Must be searched

Search Strategy Common restrictions include Ideally no restrictions but if

language, region, year, any restriction is applied (i.e.,
publication status, study year, language), these should
design be justified

Independent Review Optional Required (a minimum of two

reviewers and a third
reviewer to resolve
disagreements)

Evidence Synthesis Various approaches Narrative +/- meta-analyses

Risk of Bias Assessment Yes Yes

3
4 b. Types of Rapid Reviews
5 1. Review of Systematic Reviews
6 2. Updating Systematic Reviews with Primary Studies or newly published studies
7 3. Review of Primary Studies only

8 2.5.1.2. Critical Appraisal of Clinical Evidence

9 a. Critical appraisal is the use of a systematic method to evaluate the methodological
10 quality of studies assessing both their strengths and limitations therefore leading to the
11 judgment on their internal validity and the reliability of the study’s findings and conclusions.

12 b. Critical appraisal is an important step in HTA to determine the extent to which biases in
13 the design and conduct of the study are likely to have affected the results and therefore
14 potentially overestimate or underestimate the effects of the interventions. This step shall
15 guide the overall interpretation of data and the formation of conclusions and
16 recommendations with consideration to the strength of the evidence presented.

17 c. While there are different critical appraisal tools currently available to assess different types
18 of studies, the tools recommended to be used in local studies are shown in Appendix
19 7. These tools were selected based on the appropriateness to the study designs,
20 comprehensiveness in terms of the domains of bias included in the checklists,
21 transparency in the process of development, ability to provide descriptive information on
22 the assessment of the domains (rather than scores) and practicality and familiarity of use.
23 It is recognized that these instruments may be updated, and other tools may be developed

31
 1 with the advancement in methodological standards. Hence, the list of critical appraisal
2 tools will be updated as the Methods Guide continues to evolve.

3 2.5.1.3. Synthesis of Clinical Evidence

4 a. A narrative synthesis is done to summarize the results of all included studies taking into
5 account the similarities and differences across studies especially on the important clinical
6 outcomes and their measures of effect with their respective p values and confidence
7 intervals. Results should also be presented in summary tables.
8 ○ In certain cases, reviewers should be cautious in reporting the lack of effect of a
9 health technology because this may simply be due to the lack of evidence or lack
10 of statistical power to detect an effect.
11 b. A description of methodological quality of the studies should be part of the
12 narrative synthesis. A description and justification of the ratings for each risk of bias
13 domain should be provided to support the overall assessment of the quality of the included
14 studies.
15 c. Where statistical pooling of quantitative data across included studies is appropriate and
16 possible, a meta-analysis (MA) is ideal to provide an overall estimate of the effect size.
17 This applies to cases where both methodological and clinical heterogeneity are
18 acceptable.
19 d. In performing a meta-analysis, a widely-used standard statistical software such as Stata
20 and RevMan should be used. Refer to Table 4 above for all the important information that
21 must be specified in conducting a MA.
22 e. In cases where there are no studies showing direct comparisons of the effects of the
23 intervention and relevant comparators, indirect and mixed treatment comparisons
24 (network meta-analyses, or NMA) may be done. Other situations which may warrant the
25 conduct of NMA are the following:
26 ○ There is no direct evidence between the interventions of interest;
27 ○ There is insufficient direct evidence; or
28 ○ There are more than two interventions that need to be compared.
29 f. NMA includes ‘adjusted indirect comparisons’ but also refers to more complex evidence
30 analysis such as 'mixed treatment comparisons'.
31 ○ An adjusted indirect comparison refers to the synthesis of data from trials in
32 which the technologies of interest have not been compared directly with each other
33 in head-to-head trials but have been compared indirectly using a common
34 comparator. Figure 3 shows a trial network consisting of three sets of independent
35 trials. The first set shows a direct comparison of A versus B while the second set
36 shows two additional trials for adjusted indirect comparison of A versus B with C
37 as common comparator.
38

32
 1
2
3 Figure 3. Direct versus an indirect comparison of trials
4
5 ○ Mixed treatment comparisons include both head-to-head trials of technologies
6 of interest (both interventions and comparators) and trials that include 1 of the
7 technologies of interest. Figure 4 shows a trial network consisting of both direct
8 (solid lines) and indirect (dashed lines) comparisons.
9

10
11 Figure 4. Mixed treatment comparisons
12
13 g. A clear description of the methods of synthesis and the rationale for how RCTs
14 were identified, selected, and excluded from the NMA is needed. The methods and

33
 1 results of the individual trials included in the NMA and a table of baseline characteristics
2 for each trial must be documented.
3 h. The direct and indirect components of the NMA should be clearly identified and the
4 number of trials in each comparison stated. Results from pairwise meta-analyses using
5 the direct comparisons should be presented alongside those based on the full network
6 meta-analysis.
7 ○ Uncertainty associated with the lack of direct evidence when considering estimates
8 of relative effect derived from indirect sources only should be taken into account.
9 ○ If there is doubt about the relevance of a particular trial or set of trials, sensitivity
10 analysis should be presented in which these trials are excluded (or if absent from
11 the base-case analysis, included).
12 ○ The heterogeneity between results of pairwise comparisons and inconsistencies
13 between the direct and indirect evidence on the technologies should be reported.
14 If inconsistency within an NMA is found, attempts should be made to explain and
15 resolve these inconsistencies.
16 i. When sufficient relevant and valid data are not available for including in pairwise or
17 network meta-analyses, the analysis may have to be restricted to a narrative
18 overview that critically appraises individual studies and presents their results. In
19 these circumstances, the HTAC will be particularly cautious when reviewing the results
20 and in drawing conclusions about the relative clinical effectiveness of the treatment
21 options.
22 j. Results from subgroup analyses investigating the treatment effect, especially
23 heterogeneous results, across different subgroups of patients or trials should also
24 be presented, if applicable.

25 2.5.1.4. Algorithm of Clinical Assessment Stage

26 In summary Figure 5 shows the overall framework in assessing clinical evidence as elaborated in
27 sections 2.5.1.1 to 2.5.1.3. Conclusion on clinical evidence for the intervention of interest shall
28 then be drawn from either the appraisal of an existing systematic review or the conducted SR or
29 RR of the assessment team.
30

34
 1
2 Figure 5. Algorithm of Clinical Assessment

3 2.5.2. ECONOMIC ASSESSMENT

4 2.5.2.1. What is an economic evaluation?

5 An economic evaluation is the comparative analysis of alternative courses of action in terms of
6 both their costs (resource use) and consequences (outcomes and effects) (Drummond et al.,
7 2015).

8 2.5.2.2. Rationale behind the use of economic evaluation

9 As new health technologies are introduced into the health system, the ability to pay for them is
10 outpaced by rapidly escalating healthcare costs, increasing public demands, and many competing
11 health priorities. Tough choices, however, must be made to ensure fair, efficient, and equitable
12 allocation of resources by policy makers to ensure maximal benefits for all Filipinos.
13
14 With the goal of the DOH to maximize health outcomes across the population, economic
15 evaluations are important to inform priority-setting and resource allocation decisions in healthcare.
16 It is essential for making such priorities and decisions by providing information on the comparative
17 costs and outcomes of various policy options that are most relevant to the Philippine context. The
18 results of economic evaluations may also inform price negotiations of the government for
19 innovative health technologies such as drugs and medical devices. When conducted in a

35
 1 transparent, rigorous, and consistent manner, this can assist policy makers in dealing with
2 complex decisions with greater objectivity and accountability to stakeholders.

3 2.5.2.3. Components of the Economic Evaluation

4 The conduct of an economic assessment should allow for the determination of the relative costs
5 and benefits of a health technology as well as the budgetary implications of its adoption to the
6 national health system from the perspective of the government payor, namely DOH and
7 PhilHealth. An economic evaluation includes the following components:
8 1. Rapid review of existing economic evaluations relevant to the economic PICOT to
9 summarize the current economic evidence
10 2. De novo economic evaluation using Philippine-specific input parameters in terms of
11 epidemiology, efficacy/ effectiveness, expected health outcomes, and costs
12 3. Budget impact analysis (BIA) to assess the affordability to DOH and PhilHealth and aid
13 decision makers on fiscal planning and implementation (to be discussed further in
14 section 2.5.2.12)
15
16 A. RAPID REVIEW OF ECONOMIC EVALUATIONS
17 Rapid review of existing economic evaluations on the health technology may be done to
18 initially scan the literature, summarize, and assess the quality of existing economic
19 evidence and assess their applicability to the decision problem in the Philippines. The
20 objective of this stage is to gather evidence on the cost-effectiveness of the health
21 intervention in diverse settings and examine the population, disease pathway, health
22 outcomes, costs, and methods considered in the existing studies. A well-conducted rapid
23 review of economic evaluations may guide the planning of the local EE in terms of the
24 economic models employed as well as in addressing the gaps and weaknesses of current
25 economic evaluations.
26
27 1. Objectives of the rapid review
28 The rapid review of economic evaluations is undertaken to identify, summarize,
29 and critically appraise available economic literature relevant to the research
30 question. The results of this review help the evaluator in:
31 ○ Building the rationale on the need to conduct a local EE based on what
32 conclusions can be drawn from the existing body of knowledge
33 ○ Gathering existing decision models where the local EE’s decision model
34 may possibly be adopted or adapted from
35 ○ Determining potential sources of information used to populate the model
36 using experience from the previous studies
37 ○ Identifying the current gaps of knowledge as well as strength and weakness
38 of the previous studies which, ideally, should be aimed to be addressed by
39 the new EE
40
41 2. Literature Search Strategy
42 In addition to the usual medical databases used in systematic searching, the
43 preferred databases to systematically search for EEs include the following:

36
 1
2 o Center for the Evaluation of Value and Risk in Health at Tufts Medical
3 center (https://cevr.tuftsmedicalcenter.org/databases)
4 o National Health Service Economic Evaluation Database (NHS EED)
5 (https://www.crd.york.ac.uk/crdweb/Homepage.asp)
6 o EconLit (https://www.aeaweb.org/econlit/)
7
8 3. Study selection
9 Selection of studies in an RR of EEs follow the usual study selection approach of
10 a review where the set eligibility criteria built from the PICOT of the review are used
11 as basis for the inclusion and exclusion of studies from the total search. The
12 excluded studies and their corresponding reason for exclusion should be properly
13 accounted for using a set ineligibility coding. The results of the study selection
14 should also be reported using the PRISMA flow diagram.
15
16 4. Data extraction
17 Standard data extraction sheets should be designed and used to extract the
18 following information from the included studies: study characteristics and
19 methodological details (e.g., author, year and title of publication, first author
20 affiliation, setting, funding source, EE type, PICOT, perspective, time horizon,
21 discounting rate, sensitivity analyses used; input data sources for clinical
22 (epidemiologic and efficacy data) economic and utility parameters; as well as key
23 findings and conclusion. Information on the decision models used should also
24 extracted to compare and contrast the model assumptions and structure such as
25 the health states, cycle length, and transitions.
26
27 5. Quality assessment
28 The methodological quality of the included studies should be assessed using the
29 critical appraisal tool for EEs by Drummond et al., 2015 and the reporting standard
30 Consolidated Health Economic Evaluation Reporting Standards (CHEERS)
31 [http://www.equator-network.org/wp-content/uploads/2013/04/Revised-CHEERS-
32 Checklist-Oct13.pdf]. Rather than summary scores, the strengths and weaknesses
33 of each included study drawn from the results of this appraisal tool and their
34 implications in the overall validity of the findings are reported.
35
36 B. DE NOVO ECONOMIC EVALUATION
37 ● De novo economic evaluations are conducted internally by HTA unit staff or
38 externally by commissioned academics when there is a need to assess the value
39 for money of health technologies compared with alternatives relevant to the
40 Philippine setting.
41 ● It is possible that good quality economic models already exist which may be
42 applicable to the local decision problem in which case, adapting the economic
43 model may be done using Philippine-specific input parameters in the analysis.
44

37
 1 C. BUDGET IMPACT ANALYSIS
2 To be discussed in section 2.5.2.12.
3

4 2.5.2.4. Selection of the Type of Economic Evaluation

5 a. The type of economic evaluation will depend on the decision problem as defined after the
6 scoping and the availability of relevant data (e.g., QALY data).
7 b. The preferred type of economic evaluation in the reference case is a cost-utility
8 analysis (CUA) to meet the need of decision makers to compare the costs and outcomes
9 of health interventions against the appropriate comparators. It is preferred since it allows
10 the comparison of health gains and costs across different diseases and interventions and
11 therefore allows decision makers to make efficient allocation of resources while
12 maximizing health gains.
13 c. A cost-effectiveness analysis (CEA) reporting benefits in terms of natural health units
14 should be reported alongside the CUA to further characterize the clinical benefit profile of
15 the health technology although it does not allow for broad comparisons across diseases
16 and interventions. CEA reporting benefits in terms of natural health units alone may be
17 acceptable in situations where HRQOL data are not available, provided that clinically
18 meaningful outcomes are used.
19 d. Cost minimization analysis (CMA) may be sufficient where the intervention and the
20 comparator have been shown to be equivalent or not significantly differently in
21 terms of clinically relevant health outcomes.
22
23 Table 6. Types of Economic Evaluation.
Type of Economic Evaluation Cost Benefits

If the clinical effect of the proposed health technology is equivalent or non-inferior to the
comparator:

Cost Minimization Analysis (CMA) Philippine Not applicable

● An analysis that focuses on
Peso
calculating and comparing costs to
project the least costly health
technology (Haycox, 2009).

If the clinical effect of the proposed health technology is superior to the comparator:

Cost Benefit Analysis (CBA) Philippine Philippine Peso

● An analysis that expresses all Peso
outcomes (health and non-health)
valued in monetary rather than natural
or utility units.
● This is generally not used because of
the controversy in assigning a
monetary value on life.

38
 *Cost Effectiveness Analysis (CEA) or Philippine Outcome expressed as either
*Cost Utility Analysis (CUA) Peso natural units (e.g., life years
● CEA: An analysis that measures the gained, cases averted) for
incremental cost per extra unit of CEA or preference-based
health outcome achieved (incremental measures (e.g., QALY, DALY)
cost/unit of health outcome). [note:
for CUA
health outcome must be clinically
meaningful]
● CUA: An analysis used to determine
cost in terms of utilities, especially
quantity or quality of life.

*preferred for HTA purposes

1
2 a. As economic evaluations are data- and resource-intensive structured approaches to
3 assess the comparative costs and outcomes of health technologies, it is important to
4 standardize the sources of input parameters to be used in the analysis to ensure
5 consistency in data standards, provide context-specific inputs, and produce reliable and
6 robust economic models that truly capture the specific decision needs of policy makers in
7 the Philippines.
8 b. Appendix 5 maps the different sources of available local and international data from official
9 government databases and other sources on different inputs required in the conduct of
10 economic assessments consisting of epidemiological, clinical, resource use, costs,
11 information on equity and access, and HRQOL measures. By providing this map of HTA-
12 related information, the assessment team is expected to use appropriate information in
13 economic models as well as address the current challenges of missing, varying, and low-
14 quality data.

15 2.5.2.6. Cost Estimation

16 a. Costing, as defined by the Global Health Costing Consortium (Vassall et al., 2018), is the
17 estimation of the cost of health interventions or services in a specific context (i.e., location,
18 time period, population). The objective of this section is to guide the process of measuring
19 the costs that will be used as inputs in the economic evaluation and budget impact
20 analysis.
21
22 1. Define the method that will be used to estimate resource use
23 ● The methods for estimating the quantities of inputs, the data sources,
24 criteria for allocating shared costs, and any exclusions made should be
25 described comprehensively.
26 ○ Some approaches to resource measurement are gross- or micro-
27 costing approach, bottom-up or top-down allocation methods, and
28 step-down or activity-based costing.
29 ● Cost structures from prior studies may be used as basis.

39
 1 ○ Limitations from using cost estimates derived from
2 research/trial/demonstration settings should be carefully weighed
3 before using them. Research-driven activities (e.g., for protocol
4 adherence and reduction of loss-to-follow up) may have an impact
5 on the effect size observed and removal of the costs associated to
6 them may introduce bias to the incremental cost-effectiveness ratio
7 (ICER).
8 ○ The use of local clinical practice guidelines (CPGs) are
9 recommended to guide the estimation of frequency of
10 administration or use, duration of treatment etc. of a health
11 technology.
12 ○ It is recommended that whether or not research costs are included,
13 disaggregation of the research and non-research costs should be
14 clearly presented.
15 ● Where necessary, primary data collection should be done.
16 ● Any deviation from the ideal scope according to set purpose, due to lack of
17 available data, should be reported so that bias can be ascertained.
18
19 2. Define the sampling strategy
20 ● The sampling frame, method, and size should be determined by the
21 precision demanded by the costing purpose and designed to minimize
22 bias.
23 ○ Wherever possible, a random or stratified approach should be
24 employed. Stratified sampling by facility size or type, or location
25 type (urban vs. rural) may be a practical alternative in some cases.
26 Convenience sampling should be avoided because this is likely
27 biased.
28 ○ In general, a larger sample is preferred in obtaining cost data for
29 precise estimates.
30
31 3. Define the data source for estimating service use
32 ● For example, human resource costs may be estimated through focus group
33 discussions, interviews with providers or patients, examination of patient
34 records, time sheets, direct observation of practice, and work sampling.
35 ● Formal analytical approaches used to address bias resulting from
36 misreporting or incomplete data should be reported and justified.
37
38 4. Define the timing of data collection
39 ○ Consideration should be given to the timing of data collection to minimize
40 recall bias and other differences over time.
41 ○ Some studies suggest that a two- to three-month recall period still provides
42 reliable estimates. Recall periods longer than three months should be
43 justified. The recall period may vary, however, depending on what is

40
 1 measured. In some cases, it may be appropriate to assign a long recall
2 period (e.g., studies involving hospitalizations).
3 ○ Prospective contemporaneous recording of activities is preferred to
4 improve data precision. However, retrospective data collection may be
5 acceptable if reliable written records are available. The choice of
6 prospective vs. retrospective data collection may also depend on whether
7 the costs of the intervention or service output changes over time. Costs of
8 interventions where provision or service use exhibits seasonal variation
9 (due to changes in service volume and resource use) should be measured
10 for at least a one-year period.
11 ■ For new programs, costs incurred during both the start-up and
12 implementation phases should be captured. The latter may be
13 obtained through a field visit after the program has been running for
14 three to six months.
15

16 2.5.2.7. The Philippine Reference Case

17 a. The ‘reference case’ specifies the methodological standards considered by the HTAC in
18 making judgments on the value of health technologies to patients and the wider health
19 system. By clarifying the methods that should be employed by the assessment team,
20 transparency is expected to be provided to all stakeholders on all evidentiary
21 requirements.
22 b. In making explicit the concept of the reference case, it is expected that consistency is
23 achieved in HTA Report submissions and clear decision points are consistent with the
24 methodological standards set out in this document.
25 c. While it is ideal to abide with the reference case analysis as much as possible, it is
26 also recognized that some cases may necessitate divergence from the reference
27 case (e.g., non-availability of data, emergency situations) and in which case, the
28 use of methods that do not follow the reference case must be justified.
29
30 Table 7. The Philippine Reference Case for Health Economic Evaluation
Principle Methodological Specification

Decision Problem The economic evaluation should clearly specify the rationale behind
conducting the study and the policy question in terms of the
population, intervention, comparator/s, outcome/s, setting,
perspective and time horizon.

Perspective The cost effectiveness analysis should preferably be conducted from

a publicly-funded healthcare payor perspective, i.e., taking into
account all costs and outcomes related to the health care system.

Comparator Similar comparator/s used in the clinical assessment (ie, current

standard of care or most widely used intervention or no comparator if
this is the current practice)

41
Measure of Health For the cost-effectiveness analysis, health effects should preferably
Outcome be expressed as utilities in the form of quality-adjusted life years
(QALYs). This measure captures both the positive and negative
effects of a health technology on the length and quality of life and is
generalizable across disease states. Its generalizability guides
investment decisions by facilitating comparisons across different
diseases. Furthermore, QALY is the preferred measure of health
outcome because it is based on specific societal preference. In the
Philippines, an established value set is available for the Filipino
population based on a nationwide study conducted by Dr. Hilton
Lam in YEAR.

Disability adjusted life years (DALYs) may also be acceptable for

practical considerations especially where QALY measures are
unavailable for specific health conditions in the Philippines.

QALY is determined by the following equation:

QALY = Health State Utility x Time in State (Years)

Quality of Life (QoL) on the other hand, is an outcome measurement

that focuses on how the patient experiences an illness rather than on
clinical assessment. Outcomes are adjusted for quality by using utility
values or weights which in turn reflects the desirability of or preference
for a particular state of health. Preferred health states receive higher
utility weights.

EQ-5D-5L is the preferred instrument for the measurement of generic

preference-based health-related quality of life (HRQoL) because it is
the most translated and most widely used validated tool in economic
evaluations; and, is practical in capturing relevant key domains of
health.
● To make a case that the EQ-5D-5L is inappropriate for a
particular condition, qualitative empirical evidence on its lack of
content validity should be provided, demonstrating that key
dimensions of health are missing. This should be supported by
evidence that shows that EQ-5D-5L performs poorly on tests of
construct validity and responsiveness in a particular patient
population. This evidence should be derived from a synthesis of
peer-reviewed literature.
○ In these circumstances, alternative health-related quality
of life measures may be used and must be accompanied
by a carefully detailed account of the methods used to
generate the data, their validity, and how these methods
affect the utility values.

Measurement of All associated costs for the resources used in the model that is
Costs relevant to the perspective taken should be presented in detail, for
both the proposed intervention and its comparator/s.

This includes but is not limited to the following:

42
 ● Direct costs
○ Healthcare costs
■ costs related to the condition of interest and
incurred in additional years of life gained as a
result of treatment
■ cost of any additional products used during the
administration phase or management of side
effects (if applicable)
■ cost of preparation, clinic day visits, and usage of
monitoring equipment, hospitalization (if
applicable)
■ costs or savings as a result of a change in
infrastructure due to the introduction of the
technology (if applicable)
○ Non-healthcare costs (for societal perspective)
■ Patient and family out-of-pocket expenses (e.g.,
transportation, caring)
● Indirect costs
○ Productivity loss

For technologies requiring administration based on weight or cycle of

treatment, an average Filipino men/women weight should be
considered, and the cost should be presented by administration
and by cycle.

For certain forms of treatment (e.g., Tuberculosis or

Antiretroviral treatment), consider disaggregating costs into
separate time periods where they vary over time.

For capital costs (inputs with a useful life of more than one year),
the estimates should be depreciated. The method for depreciation
and the useful life (length and data source) should be reported for
each major capital input.

Quantities of resources must be reported separately from their

unit costs/prices. All unit prices and costs should be presented in
Filipino pesos with a consistent year of analysis (which should be
stated and be as close as possible to the evaluation date) by using
relevant indexes. The index used and the original price year should
be clearly indicated. For more guidance on adjusting for inflation and
currency changes, see Annex 10.

All steps taken to calculate the included costs should be presented in

such a way that allows the calculations to be independently verified.
All input data and cost calculations should be captured in an
Excel document, in separate spreadsheet(s). For each such input,
the source (and its electronic link where applicable) and calculation
method, if applicable, should also be present.

43
 Funding sources, especially for direct medical costs, must be
carefully accounted for and detailed in the presentation and
analysis of costs (e.g., donor funds).

If applicable (i.e., societal perspective), shadow prices used to value

inputs with no available market prices (e.g., volunteer time, household
time) should be reported and the minimum value based on a proxy or
hypothesized market should be used.

In addition, all alternative costs, their sources, and any

assumptions should be detailed. If multiple estimates are identified,
the estimate used in the base case should be justified and alternative
plausible estimates in sensitivity analyses should be presented.

Time Horizon The time horizon for estimating clinical and cost effectiveness should
be sufficiently long to reflect all important differences in costs or
outcomes between the technologies compared.

A lifetime time horizon is required when alternative technologies

lead to differences in survival or benefits that persist for the remainder
of a person's life. For a lifetime time horizon, it is necessary to
extrapolate data beyond the duration of the clinical trials and to
consider the associated uncertainty.

A time horizon shorter than a patient's lifetime is justified if there

is no differential mortality effect between treatment options, and the
differences in costs and health-related quality of life relate to a
relatively short period (e.g., in the case of an acute infection which
has no long-term sequelae).

Discounting Costs and benefits occurring in the future should be discounted to

the net-present value at an annual base case discount rate of seven
percent (7%) based on computations from the Ramsey Formula.

For a time horizon greater than 30 years, the discount rate shall
be decreased as the time period increases. The table below lists
the suggested schedule of discount rate declines based on global
practice:
Period of Years Discount Rates

0-30 7%

31-75 5.33%

76+ 3.66%

Discount rates at 1.67% decrease, computed in proportion to the decline

used by UK Treasury Green Book (2018).

44
 Sensitivity analysis should be performed at three and ten percent (3
and 10%) (applying the same rate to costs and benefits at a time).
(Haacker, 2019)

Heterogeneity The effects and costs of the health technologies assessed on

sub-populations should be explored (e.g., different baseline risk,
geographical location etc.). This enables decision-makers to consider
whether the proposed intervention should be made available to
groups of individuals with a greater capacity to benefit.

Relevant subgroups should be identified a priori based on the

available evidence (e.g., pre-specified subgroups within a clinical
trial). However, post hoc subgroup analysis may also be accepted in
cases where external evidence suggests differences among
subgroups where they have not been pre-specified.

Uncertainty As discussed in the iDSI Reference Case (NICE International, 2014),

all economic evaluations reflect a degree of uncertainty and it is
important that all types of uncertainty are appropriately presented to
the decision-maker. The characterization of this uncertainty enables
the decision-maker to make a judgement based not only on a likely
estimate of the incremental costs and effects of an intervention, but
on the confidence that those costs and effects accurately represent
reality.

Methodological uncertainty which pertains to the methodological

disagreement among analyses (e.g., method of costing, discount rate)
is not included here with the assumption that the economic evaluation
followed the steps and the reference case outlined in this methods
guide.

The following are the three (3) types of uncertainty to consider:

● Modelling or Structural uncertainty - uncertainty with
the functional form of the model (e.g., categorisation of
different states of health and the representation of
different pathways of care)
● Parameter uncertainty - uncertainty with the true
numerical values of input parameters (i.e., mean health
and cost inputs) in the model
● Sources of values to inform parameter estimates -
uncertainty with different estimates of key parameters
(e.g., estimates of relative effectiveness)

The identified uncertainties should be explored and quantified where

possible through appropriate sensitivity analyses. Both deterministic
and probabilistic sensitivity analyses should be employed. The choice
of sensitivity analyses approaches to deal with uncertainty should be
clearly stated and a rationale provided:

45
 Deterministic Sensitivity Analysis:
● Univariate sensitivity analysis
It is assumed that all inputs used in the model are estimated with
a degree of imprecision. Therefore, the model results are
presented as a tornado graph after varying each parameter in
a sensible range, one at a time.
● Multivariate sensitivity analysis
Model results are estimated varying multiple parameters at the
same time to analyse how the combined variations affect the
results.
● Scenario analysis
Scenarios are useful to test particular subsets of multivariate
analyses. For example, the scenarios may represent the most
optimistic and the most pessimistic parameter combinations.
● Threshold analysis
Especially for key parameters of the model, critical values of
parameters should be identified. The critical value is defined as
the threshold parameter value for which the conclusion of the
analysis would change, e.g. no longer considered cost-effective.

Probabilistic sensitivity analysis (PSA)

In contrast to the deterministic approaches mentioned above
where only point estimates are used, PSA introduces a
stochastic element into the model and incorporates information
regarding the likely parameter values. As such, each input
parameter is assigned a specific sampling distribution. The
choice of distribution depends on the type of parameter and the
available data.

Model values are then drawn at random simultaneously, which

allows representing joint parameter uncertainty. The ICER is
calculated predefined number of times (i.e., 10.000 times). The
sample of ICERs from the PSA should be presented as an
empirical distribution of the analysis results.

PSA is presented in two forms:

● CE plane showing the PSA iteration plots
● CE acceptability curve with the Philippine threshold
clearly identified

When the purpose of a probabilistic sensitivity analysis is to

guide decisions about acquisition of information to reduce
uncertainty; results should be presented in terms of expected
value of information.

1 2.5.2.8. Modelling
2 a. A decision model is developed to assist decision-makers in making choices relating to the
3 evaluated options. The objective of a decision model is to obtain a clearer understanding
4 of the relationships between incremental costs and their consequences.

46
 1 b. The model should mimic the patients’ evolution from the current status of health to the
2 final one (cured or death). It should simulate as much as possible the real life setting of
3 the disease. The comparator chosen will be the same one(s) identified and used in the
4 clinical assessment.
5 c. The three common structures used in economic evaluations are decision trees, Markov
6 models (state-transition models), and dynamic transmission model.
7 ○ A decision tree is a form of analytical model, in which distinct branches are used
8 to represent a potential set of outcomes for a patient or patient cohort (York Health
9 Economics Consortium, 2016). Decision trees are used to model interventions that
10 have few and distinct outcomes that can be measured at a specific time point or
11 over short time horizons.
12

13
14 Figure 6. Decision tree for breast cancer screening options (Petrou & Gray, 2011)
15
16 ○ For more complex frameworks involving stochastic processes, a markov
17 model (state-transition model) is more commonly used. It is well-suited
18 for chronic conditions. It uses disease states to represent all possible
19 consequences of an intervention of interest. These are mutually exclusive
20 and exhaustive and so each individual represented in the model can be in
21 one and only one of these disease states at any given time.

47
 1
2 Figure 7. Markov Model based on the CUA of Adjuvant Trastuzumab Therapy for HER2-Positive
3 Early-Stage Breast Cancer in the Philippines by (Genuino et al., 2019)
4
5 ○ For the analysis of interventions against an infectious disease concerning
6 the population of interest, a dynamic transmission model or dynamic
7 model is used (Caro et al., 2012). This type of modelling incorporates
8 externalities such as indirect effects from averted infections due to
9 vaccination. This differs from static models that assume a constant risk of
10 infection. In a dynamic model, the infection risk is dependent on the number
11 of infectious agents at a given point in time.

48
 1
2 Figure 8. Dynamic Transmission Model based on the CEA of the universal rotavirus vaccination
3 program by (Lam et al., n.d.-b) (a: age, t: time, Im: infected mild, Is: infected severe, VEm: vaccine
4 efficacy for mild disease, VEs: vaccine efficacy for severe disease)

5 2.5.2.8.1. Model Development

6 a. Before constructing a model, models from published economic evaluations should first be
7 reviewed. An appropriate model detected from the preliminary review may be adopted or
8 modified to better align it with the decision problem, modeling objective, and scope of the
9 assessment. If no appropriate model was found in existing literature, a model should be
10 constructed that adequately represents the disease progression and addresses the
11 decision problem.
12 b. An explicit process which includes expert consultations should be undertaken in the
13 development of an appropriate model structure, ensuring that it reflects current disease
14 knowledge.
15 c. The description and assumptions of the model used should be clearly stated. Assumptions
16 and mathematical formulas should be aligned with the overall clinical review and size of
17 benefit resulting from the clinical assessment.

49
 1 d. The model will be provided by the applicant (e.g., industry sponsors) or developed in-
2 house by the HTA unit or commissioned academics, in Excel format. Given the need to
3 ensure accountability and respond to any future audit questions, web-based programs
4 and software applications to which access is only provided for a limited time, or
5 calculations are not transparent are not considered acceptable. The applicant may also
6 submit the Excel calculations in PDF to ensure their version of the model is correctly
7 maintained in the history of the application.
8 e. All Excel documents delivered will ensure that all input data and their sources, as well as
9 mathematical formulas, are available and can be modified if needed. No (password)
10 protected electronic documents will be accepted.

11 2.5.2.8.2. Input Parameter Estimation

12 a. Disease progression probabilities used to model the progression of the disease will be
13 based on local data wherever reliable databases are present (government or official
14 registries are preferred); should local registries not be present, data from Asian studies
15 will be preferred over other international ones. If Asian/international probabilities used,
16 they should be confirmed with local practitioners in a panel consultation. The applicability
17 of the data should be justified and there should be a discussion if it is a potential source
18 of uncertainty within the model.
19 ○ Transition probabilities inform the movement of patients between health states in
20 decision trees or state transition models.
21 ○ Should other transition probabilities be needed to describe further the progression
22 of the disease or condition as a result of an intermediate outcome event, the same
23 transition probabilities will be applied for all compared pathways, regardless of
24 treatment allocation.
25 b. For preferred sources of input parameters, kindly refer to Annex 5.

26 2.5.2.8.3. Model Validation

27 a. Model validation is important to see how accurate a model can make relevant predictions.
28 The following are the five main types of validation based on the Good Modeling Practices
29 released by International Society for Pharmacoeconomics and Outcomes Research
30 (ISPOR) (Eddy et al., 2012).
31 1. Face validation - validation of the model, its assumptions, and applications as
32 judged by experts relevant to the problem area; this may be done by conducting a
33 stakeholder consultation
34 2. Internal validation or Technical validation or Verification - examination of
35 the mathematical calculations performed through the verification of individual
36 equations and their accurate implementation in code
37 3. Cross validation or Convergent validation or Model corroboration -
38 comparison with other models and determination of the extent to which they
39 calculate similar or different results
40 4. External validation - simulation of a real scenario (e.g., clinical trial) and
41 comparison of the predicted outcomes to the real-world outcomes
42 5. Predictive validation - comparison of forecasted outcomes with actual events

50
 1
2 b. All validation types should be conducted based on the context of their particular
3 application. For instance, although predictive validation is the most desirable type, it is
4 time-consuming and therefore not recommended for immediate decisions.
5
6 A summary of the decision analysis modeling process includes the following:
7 1. Adaptation or construction of a model that depicts the relevant options and possible
8 outcomes of these options, validated with a panel of experts;
9 2. Estimation of probabilities;
10 3. Estimation of costs and outcomes;
11 4. Calculation of the expected value of costs and outcomes for all options;
12 5. Identification of the option with the greatest expected value (most desirable option or
13 alternative); and
14 6. Sensitivity analyses to handle uncertainties
15

16 2.5.2.9. Presentation of Results

17 As a result of running the model, the conclusion of the base-case should be presented, stating
18 which intervention is the most cost-effective one.
19
20 The result of the cost-effectiveness or cost-utility analysis is best summarized in the form of an
21 incremental cost-effectiveness ratio (ICER). ICER is a value defined as the difference in cost
22 between two interventions, divided by the difference in health outcome obtained. Simply put, ICER
23 is the additional cost per additional benefit delivered by the new technology compared to the old
24 technology.
25
26 This is a computed statistic that has the following formula:
27 ICER =Cost new– Cost old
28 Benefit new – Benefit old
29
30 The results should be presented in the form of:
Incremental
Health Incremental Incremental
Cost C/E ratio
Outcome cost outcome
(ICER)
Old health
PHP Xx - - -
technology
New health
PHP Xx PHP xx PHP
technology
31
32 The “Consolidated Health Economic Evaluation Reporting Standards (CHEERS)
33 statement” (Annex 13) with its checklist may be used as a reference to check the items to
34 include when reporting economic evaluations of health interventions. This was developed by a

51
 1 task force supported by the International Society of Pharmacoeconomics and Outcomes
2 Research (ISPOR).

3 2.5.2.10. When is a health technology considered to be “cost-effective” in the

4 Philippines?
5 Interventions with an ICER falling below or at the Philippine CE threshold are considered cost-
6 effective. However, other decision criteria like responsiveness to magnitude, severity and equity;
7 effectiveness and safety; household financial impact; and, affordability and viability may be
8 brought to bear in the decision to recommend cost-effective or cost-ineffective interventions for
9 possible coverage.

10 2.5.2.11. Appraisal of Economic Evaluations

11 Drummond’s checklist (Annex 13) will be used to appraise submitted economic evaluations.

12 2.5.2.12. Budget Impact Analysis

13 a. The budget impact analysis (BIA) is a financial approach designed to estimate, over a
14 defined time horizon, the financial consequences of adopting a health intervention. The
15 objective of this analysis is to increase the awareness of DOH or PhilHealth policymakers
16 with regards to the financial impact of introducing a new technology, and to aid in budget
17 or service planning of government and/or social insurance. Therefore, All BIAs shall use
18 government payor perspective. This is required, along with the CEA or CUA.
19 b. Lower annual percent changes in the budget is always preferred. However, higher
20 percent changes may be acceptable by the payor, if the payor would be able to find ways
21 to increase its annual budget in order to accommodate the expected increases due to the
22 implementation of the new technology.

52
1
2
3 Figure 9. ISPOR Budget Impact Schematic
4
5 c. The BIA methods set out in this document is guided by the ISPOR Principles of Good
6 Practice for Budget Impact Analysis as shown in Figure 9 and Table 8 (Sullivan et al.,
7 2014).
8
9 Table 8. Performing a BIA
1 Establish The budget holder or government payor’s perspective should be
perspective used.

2 Establish time At least a one-year assessment, and ideally up to 3-5 years

horizon forecasts should be provided based on the base case price and
discounted scenarios.

3 Identify the The volume of patients should be clearly defined. With a government
eligible payor perspective, the population should be adjusted for the level of
population use of government facilities. This takes into account the planned
delivery mechanism health system level/facility types/integration with
other services where relevant).

A distinction should be made based on the full or gradual

implementation of the new treatment. In the latter, subsequent
calculations should be based on the mix of patients on the current
treatment and of patients on the new treatment.

53
 In some cases, a distinction should also be made between maximum
potential number of patients that may benefit from the new treatment,
as opposed to the number of patients for which there is a positive
cost-effective resolution (i.e., only a subset of patients has been
deemed cost-effective, as opposed to the whole exposed population
included in the studies).

Changes to the eligible population (i.e., delay of disease progression,

reduction of mortality etc.) within the established time horizon should
be taken into account.

Table 9 must be populated with the necessary data. Preferred

sources of data are epidemiologic studies and government records
(e.g., utilization rates of government facilities, Philhealth claims).
Other data sources are the following:
● Data from clinical trials specific to or extrapolated to specified
population
● Uptake, usage, and adherence data from international
sources, from similar populations, and with similar practice
patterns
● Market research data
● Expert opinion and surveys for practice patterns

In any case, a summary of the process in deriving the numbers

should be described.

4 Estimate the If there is no current intervention that the payor pays for, then this will
costs of treating be zero.
the disease with
current If there is an intervention that the payor currently pays for, then this
treatment will be the current cost for the delivery of the current treatment/s. Cost
of treatments used off-label may be included. The estimates should
account for variations in usage and cost-relevant details concerning
usage such as monitoring and management of side effects or
treatment failures. The cost of required diagnostic test/s to identify
eligible individuals should also be estimated.

Refer to Annex 5 for sources of standard treatment costs.

Note: The cost values and items used in the CEA/CUA and BIA
should be consistent.

5 Estimate the Similar to #3, all relevant costs of the new treatment should be
costs of treating estimated (e.g., diagnostic test, procurement price, delivery costs,
the disease with storage, administration costs, management of side effects or
the new treatment failures).
treatment
Note: The cost values and items used in the CEA/CUA and BIA
should be consistent.

54
 6 Compute the Compute for the difference in total costs of the current treatment
changes in total versus the new treatment. The estimated total cost for each treatment
costs should have been multiplied by the proportion of the eligible
population using or expected to use that treatment.

7 Compute the Compute for the percent changes in total costs of the current
annual percent treatment versus the new treatment.
changes
1
2 d. The report should contain a list of values used with corresponding sources,
3 formulas, assumptions and explanations. A summary table containing these should
4 be included in the report. Confidential information should be highlighted in the report.
5 e. A costing template that is user friendly should be provided with the budget impact report.
6 The use of readily available software such as spreadsheet software is highly
7 recommended. The template will allow the budget holder to conduct their own analyses
8 and use their own parameters and assumptions.
9 f. The results of the computations for both the current and new health technology should be
10 summarized in Table 9 below.
11 g. Changes in the value due to inflation or deflation or changes in prices due to patent
12 expiration should be accounted for in the calculations whenever possible.
13 h. An estimation of the impact to household and out-of-pocket expenses, third-party payers,
14 and external donors may also be done as an additional analysis.
15 i. Use of graphs in the main report is encouraged. Use of graphical information would
16 likely facilitate decision making. Data tables used to construct the graphs should be
17 included in the appendix.
18
19 Assumptions and Uncertainty
20 ● All assumptions should be subjected to sensitivity analysis. Scenario rather than
21 probabilistic analysis is preferred. Directed changes to parameter values testing realistic
22 scenarios created in consultation with the decision maker is likely to be more useful to
23 decision holders over testing the entire range of possible values. Therefore, the BIA Table
24 above should be submitted for a base case, and at least one low assumption case and
25 one high assumption case.
26 ● The sensitivity analysis should point out assumptions or parameters that highly
27 affect cost estimates to provide the decision maker a better picture of the potential
28 impact. This would also allow the decision maker to focus on testing or deciding upon
29 certain assumptions and parameters.

30
31
32
33
34
35

55
1 Table 9. Details of the health technologies under evaluation (this table may be edited depending
2 on relevant information concerning the health technologies assessed)
Approved name of health
technology

Give the (anticipated) indication(s) in the

Indications and any restriction(s) Philippines.
of use

Data Source

Procurement cost*

Anticipated frequency of use per

course of treatment

Average length of a course of

treatment

Anticipated number of repeat

courses of treatments per year

Total Cost

*When the marketing authorisation or anticipated marketing authorisation recommends

the intervention in combination with other treatments, the list price of each intervention
should be presented.
3
4
5
6
7
8
9

56
 1
2 Table 10. Expected Budget Impact

Year 1 Year 2 Year 3 Year 4 Year 5

Eligible
population for
treatment with
[new
treatment]

Population
expected to
receive [new
treatment]

Total cost with

current
treatment

Total cost with

[new
treatment]

Net budget
impact
(difference in
total costs)

% changes in
total cost

3
4
5
6
7
8
9
10
11
12

57
 1 2.5.2.3. ASSESSMENT OF ETHICAL, LEGAL, and SOCIAL IMPLICATIONS (ELSI)
2 a. Beyond the clinical and economic aspects of a health technology, HTA takes into account
3 the ethical, legal, and social implications (ELSI) associated with the use or non-use of a
4 health technology. This aims to increase the relevance and applicability of assessments
5 in a given context.
6 b. Ethical and societal effects of a certain technology are likely to vary depending on the
7 structure, functioning, and cultural norms of a specific setting. An interactive,
8 participatory approach involving relevant stakeholders in a real discourse (through
9 a consultation) should be done to improve the applicability of the HTA. Literature
10 searches must also be done to identify ethically and socially relevant issues.
11
12 At minimum, all research should conduct literature review but still subject to other more
13 extensive data collection methods, as appropriate. Please refer to Annex 14 for specific
14 methods.
15
16 Equity considerations
17 Equity in Health implies that ideally everyone should have a fair opportunity to attain their full
18 health potential and that no one should be disadvantaged from achieving this potential (World
19 Health Organization, 2019). Qualitative methods may be employed in assessing the equity
20 implications of a proposed intervention because they are effective in identifying factors and
21 contexts, such as patient preferences, patient world view, social norms, socioeconomic
22 status, gender roles, ethnicity, and religion that affect access to or quality of care received.
23 Considering the implications of these differences is important to ensure that inequities are
24 reduced if possible and that they are not increased. The three most common qualitative methods
25 are participant observation, in-depth interviews, focus group discussions, and online
26 ethnography (Family Health International, n.d.). At the most basic level, a description of
27 particular groups which may be disproportionately affected positively or negatively by a
28 decision should be made (NICE International, 2014). Identification of such groups shall include
29 basic information which may include the following:
30
31 ● number of members;
32 ● status of group (with registered organization or not);
33 ● identified sources of support (funders, political supporters, etc); and
34 ● Such other variables which could potentially impact discourses
35

36 The report on equity will be organized according to PROGRESS-Plus.

37 PROGRESS-Plus, an acronym developed by Cochrane (The Cochrane Collaboration, 2019),

38 may also be used to further define factors that stratify health opportunities and outcomes:
39 • Place of residence
40 • Race/ethnicity/culture/language
41 • Occupation
42 • Gender/sex
43 • Religion

58
 1 • Education
2 • Socioeconomic status
3 • Social capital
4 • Plus other possible factors such as disease status or disability
5 1. personal characteristics associated with discrimination (e.g. age, disability)
6 2. features of relationships (e.g. smoking parents, excluded from school
7 3. time-dependent relationships (e.g. leaving the hospital, respite care, other
8 instances where a person may be temporarily at a disadvantage)
9
10 More thorough qualitative studies may be done when deemed necessary by an expert panel.
11
12
13 Legal Implications
14 a. Legal issues should be carefully accounted for in the entire process of conducting an HTA.
15 This goes beyond managing conflicts of interests and obtaining informed consent in the
16 conduct of the research. Relevant local and international laws concerning regulation of
17 particular health technologies (e.g., patent, market entry, off-label use, data protection,
18 product liability), patient rights, specific disease-based laws (e.g., Cancer, Orphan
19 diseases), and policies of relevant health sector agencies should be looked into (Widrig &
20 Tag, 2014).
21 b. It must be recognized however that even in recent times, there is a shortage of accepted
22 methods in incorporating these considerations in an assessment. In our setting, the ELSI
23 checklist developed by EUnetHTA (Annex 13) may be used as one of the tools to assess
24 and report the ethical, legal, and social issues surrounding the implementation of a
25 proposed health technology (European Patients’ Academy, 2016). More importantly, it
26 must be considered at all stages of an evaluation - from the design, analysis, and reporting
27 of the results.
28

29 2.5.2.4. ASSESSMENT OF HEALTH SYSTEM IMPACT

30 a. The capacity of both public and private sector to implement a health technology should
31 also be assessed. This includes availability, feasibility or and capacity of human
32 resources, service capacity or facilities, and the potential to impact other roles of existing
33 divisions or organizations. Like ELSI, qualitative methods such as in-depth interviews,
34 focus group discussions with identified key stakeholders (e.g., DOH program
35 managers, hospital administration, local government units, other implementers who will
36 be affected by the adoption and management of the health technology), and surveys
37 should be done to assess health system impact.
38
39

59
 1 REFERENCES
2
3 1. ASSASI, N., TARRIDE, J. E., O’REILLY, D., & SCHWARTZ, L., (2016). Steps toward
4 improving ethical evaluation in health technology assessment: a proposed framework. BMC
5 medical ethics, 17(1), 34. https://doi.org/10.1186/s12910-016-0118-0
6 2. CARO, J. J., BRIGGS, A. H., SIEBERT, U. & KUNTZ, K. M. (2012). Modeling Good Research
7 Practices—Overview: A Report of the ISPOR-SMDM Modeling Good Research Practices
8 Task Force-1. Value in Health, 15, 796-803.
9 3. CENTER FOR REVIEWS AND DISSEMINATION (2008). Systematic Reviews: CRD’s
10 Guidance for Undertaking Reviews in Healthcare. CRD, University of York.
11 4. CHAIMANI, A. & SALANTI, G. (2013). Comparing Multiple Interventions Stream 2 (Statistical
12 Issues in NMA). Cochrane.
13 5. DANS, A. L., DANS, L. F., & SILVESTRE, M. A. (2017). Painless evidence-based medicine.
14 John Wiley & Sons.
15 6. DOBBINS, M. (2017). Rapid Review Guidebook. Hamilton, ON: National Collaborating Centre
16 for Methods and Tools.
17 7. DRUMMOND, M. (2015). Methods for the economic evaluation of health care programmes.
18 Oxford University Press.
19 8. EDDY, D. M., HOLLINGWORTH, W., CARO, J. J., TSEVAT, J., MCDONALD, K. M., &
20 WONG, J. B. (2012). Model Transparency and Validation: A Report of the ISPOR-SMDM
21 Modeling Good Research Practices Task Force-7. Value in Health, 15(6), 843-850.
22 doi:10.1016/j.jval.2012.04.012
23 9. EUROPEAN NETWORK FOR HEALTH TECHNOLOGY ASSESSMENT. (2018). HTA Core
24 Model [Online]. Available: https://www.eunethta.eu/hta-core-model/ [Accessed].
25 10. EUROPEAN PATIENTS’ ACADEMY. (2016). Ethical, social, and legal issues (ELSI) in HTA
26 [Online]. Available: https://www.eupati.eu/health-technology-assessment/ethical-social-and-
27 legal-issues-elsi-in-hta/#Ethical_Legal_and_Social_Issues_ELSI_checklist [Accessed].
28 11. GENUINO, A. J., CHAIKLEDKAEW, U., THE, D. O., REUNGWETWATTANA, T. &
29 THAKKINSTIAN, A. (2019). Adjuvant trastuzumab regimen for HER2-positive early-stage
30 breast cancer: A systematic review and meta-analysis. Expert Review of Clinical
31 Pharmacology, 12(8), 815-824. doi:10.1080/17512433.2019.1637252
32 12. GONZALES, M.C. (2013). Positibo: Achieving Well-Being Among Young People Living with
33 HIV in Cebu. Aghamtao, Volume 22: 22-39.
34 13. HAACKER, M., HALLETT, T. B., & ATUN, R. (2019). On discount rates for economic
35 evaluations in global health. Health Policy and Planning. doi: 10.1093/heapol/czz127
36 14. HAASIS, M., CERIA, J., KULPENG, W., TEERAWATTANANON, Y., & ALEJANDRIA, M.
37 (2015). Do pneumococcal conjugate vaccines represent good value for money in a lower-
38 middle income country? A cost-utility analysis in the philippines. Value in Health, 18(7).
39 doi:10.1016/j.jval.2015.09.1549
40 15. HARBOUR, R., & MILLER, J. (2001). A new system for grading recommendations in evidence
41 based guidelines. BMJ, 323(7308), 334-336. doi:10.1136/bmj.323.7308.334
42 16. HAYCOX, A. (2009). What is cost-minimisation analysis? [Online]. Available:
43 http://www.bandolier.org.uk/painres/download/whatis/What_is_cost-min.pdf [Accessed].
44 17. HIGGINS, J.P.T., THOMAS, J., CHANDLER, J., CUMPSTON, M., LI, T., PAGE, M.J.,
45 WELCH, V.A., (EDITORS). (2019). Cochrane Handbook for Systematic Reviews of
46 Interventions version 6.0 (updated July 2019). Cochrane. Available from
47 http://www.training.cochrane.org/handbook
48 18. JAVIER, J.R., SUPAN, J., LANSANG, A., BEYER, W., KUBICEK, K., & PALINKAS, L.A.
49 (2014). Preventing Filipino mental health disparities: Perspectives from adolescents,
50 caregivers, providers, and advocates. Asian American Journal of Psychology, 5(4), 316–324.
51 doi: 10.1037/a0036479

60
 1 19. LAM, H., TAI, B. G., VILLARUZ-SULIT, M. V., RIVERA, A., TUMANAN-MENDOZA, B.,
2 ESTRADA, M. A., CHENG, K. J., MAYBITUIN, C. & SY, B. n.d.-a. Methods Manual.
3 20. LAM, H., WU, D. B., RIVERA, A. S., ALEJANDRIA, M. M., JACINTO BLAS MANTARING, I.,
4 SISON, O. T., SANTILLAN, M. C. & LADIA, M. A. J. n.d.-b. Assessing the cost-effectiveness
5 of a Universal Rotavirus Vaccination program for the Philippines using a dynamic transmission
6 model. Value in Health, 18(3). doi:10.1016/j.jval.2015.03.1308
7 21. MACK, N., & WOODSONG, C. (2005). Qualitative research methods: A data collector's field
8 guide. FLI.
9 22. MCGOWAN, J., SAMPSON, M., SALZWEDEL, D. M., COGO, E., FOERSTER, V. &
10 LEFEBVRE, C. (2016). PRESS Peer Review of Electronic Search Strategies: 2015 Guideline
11 Statement. Journal of Clinical Epidemiology, 75, 40-46. doi:10.1016/j.jclinepi.2016.01.021
12 23. NICE INTERNATIONAL (2014). The Gates Reference Case.
13 24. PETROU, S., & GRAY, A. (2011). Economic evaluation using decision analytical modelling:
14 design, conduct, analysis, and reporting. Bmj, 342. doi: 10.1136/bmj.d1766
15 25. PHILIPPINES (2019). Republic Act No. 11223: Universal Health Care Act, Republic of the
16 Philippines.
17 26. PHILIPPINES (2019). Republic Act No. 11223: Implementing Rules and Regulations,
18 Republic of the Philippines
19 27. PHILIPPINES (2018). Administrative Order No. 2018-0026: Framework for the use of Health
20 Technology Assessment to guide coverage decisions in support of Universal Health Care,
21 Republic of the Philippines
22 28. PHILIPPINES (2016). Administrative Order No. 2016-0034: The new implementing guidelines
23 of the Philippine National Formulary System, Republic of the Philippines
24 29. PHILIPPINES (2013). Republic Act No. 10606: National Health Insurance Act of 2013,
25 Republic of the Philippines
26 30. SULLIVAN, S. D., MAUSKOPF, J. A., AUGUSTOVSKI, F., CARO, J. J., LEE, K. M., MINCHIN,
27 M., ORLEWSKA, E., PENNA, P., BARRIOS, J.-M. R. & SHAU, W.-Y. (2014). Budget Impact
28 Analysis—Principles of Good Practice: Report of the ISPOR 2012 Budget Impact Analysis
29 Good Practice II Task Force. Value in Health, 17(1), 5-14. doi:10.1016/j.jval.2013.08.2291
30 31. THE COCHRANE COLLABORATION. (2019). PROGRESS-Plus [Online]. Available:
31 https://methods.cochrane.org/equity/projects/evidence-equity/progress-plus [Accessed].
32 32. TREASURY. The Green Book. Central Government Guidance on Appraisal and Evaluation.
33 (2018). Available: https://www.gov.uk/government/publications/the-green-book-appraisal-
34 and-evaluation-in-central-governent [Last viewed on 2 February 2020]
35 33. TRICCO, A. C., LANGLOIS, E. V. & STRAUS, S. E. (2017). Rapid reviews to strengthen
36 health policy and systems: A practical guide. World Health Organization
37 34. TURNER, H.C., LAUER, J. A., TRA, B, X., TEERAWATTANANON, Y., & JIT, M., (2019).
38 Adjusting for Inflation and Currency Changes Within Health Economic Studies. Value in
39 Health, 22(9), 1026-1032. doi:10.1016/j.jval.2019.03.021
40 35. URSU, I. (2018). HTA Methods Manual. Department of Health - Philippines.
41 36. VASSALL, A., SWEENEY, S., KAHN, J. G., GOMEZ, G., LORI BOLLINGER, MARSEILLE,
42 E., HERZEL, B., PLOSKY, W. D., CUNNAMA, L., SINANOVIC, E., BAUTISTA, S., GROUP,
43 H. T. A., GROUP, G. S., HARRIS, K. & LEVIN, C. (2017). Reference Case for Estimating the
44 Costs of Global Health Services and Interventions.
45 37. WIDRIG, D. & TAG, B. (2014). HTA and its Legal Issues: A Framework for Identifying Legal
46 Issues in Health Technology Assessment. International Journal of Technology Assessment in
47 Health Care, 30, 587-594.
48 38. WORLD HEALTH ORGANIZATION. (2019). Health Equity [Online]. Available:
49 https://www.who.int/topics/health_equity/en/ [Accessed].
50 39. YORK HEALTH ECONOMICS CONSORTIUM. (2016). Decision Tree [Online]. Available:
51 https://www.yhec.co.uk/glossary/decision-tree/ [Accessed].

61
 1 Annexes and Appendices
2
3 Annex 1 - Methodologic Principles of HTA
4 Building on the process principles of HTA, the methodologies employed in the production of an HTA
5 report shall adhere to the following principles:
6
7 A. GENERAL PRINCIPLES IN THE CONDUCT OF HTA
8 The regulations also explicitly require that the HTA process adheres to the following principles:
9
10 1. Ethical soundness. The HTA process is grounded on moral standards and principles as defined
11 by relevant Philippine laws, international agreements and covenants. It includes managing conflicts
12 of interest and ensures that all actors and stakeholders have equal opportunity to contribute and
13 these contributions are equally accounted and treated objectively.
14
15 2. Inclusiveness and preferential regard for the underserved and unserved. The HTA process
16 involves deliberate, structured consultations with relevant parties, such as community members
17 and end-users, with particular attention to the underserved. Societal values are acknowledged in
18 the acceptance of nominations for health technologies.
19
20 3. Evidence-based and scientific defensibility. The HTA process utilizes evidence that underwent
21 systematic appraisal and preferentially uses local data. Recognizing that this is not always possible,
22 the HTA process encourages contextualization of foreign data by proactively seeking
23 multidisciplinary experts and applying relevant methods. The HTA process is regularly updated
24 based on developments in this field.
25
26 4. Transparency and Accountability. All steps in the HTA process must be standardized, consistent
27 and explicit. All actors and stakeholders are well-informed and acquainted on the proceedings,
28 have a clear idea of their roles and responsibilities. The HTA process ensures that proceedings of
29 activities are publicly disclosed in a manner that is easily accessible, clear and understandable.
30
31 5. Efficiency. The HTA process ensures proper coordination among the stakeholders and
32 consolidation of information to avoid redundancy of actions and delays of output. Technical and
33 administrative staff are adequate in number, well adept and competent in fulfilling the tasks in a
34 timely manner. Applications are efficiently directed, assessed and managed through the relevant
35 steps. Administrative costs are kept at a minimum, without compromising the quality and rigor of
36 the HTA process.
37
38 6. Enforceability. The HTA process is executed with strict observance to the guidelines and
39 procedures. Human and financial resources required for implementation are readily available to
40 ensure feasibility and sustainability of the HTA process.
41
42 7. Availability of remedies and due process. Proponents are informed of the status of
43 applications and appeals, including supporting facts and reasons, in a clear and timely manner.
44 Embedded in the HTA process is a standardized appeals mechanism, where guidelines are
45 clearly communicated, thus empowering all stakeholders to utilize. The HTA process enables
46 resolution of conflict.

62
1 Annex 2 - Conflict of Interest Declaration Form for Assessment Team
2

3
63
1
64
1

2
3

65
 1 Annex 3 - HTA Scoping Tool
2
3 In conducting an HTA, there should be a defined scope of the content and focus of the report to
4 provide a clear framework from the beginning on the relevant questions that needs to be
5 addressed in assessing the overall value of a health technology to the Philippine health system.
6
7 This tool serves as a guide to assessment teams in developing the scope of the report through a
8 review of both published and grey literature and consultation with relevant stakeholders such as
9 potential users of the health technology, clinical experts, DOH program managers, industry
10 representatives, patients, healthcare organizations, and other relevant health system partners.
11
12 Questions to be addressed:
Area of focus Guide questions

Population (P) ● What patient population, health condition or disease is

being addressed by the health technology in the
Philippines?
● What is the local incidence or prevalence of the condition
being addressed by the health technology? What is the
proportion of the diseased population likely to use the health
technology?
● Who will likely use the health technology in terms of age,
gender, ethnicity, level of risk/severity, place of residence or
other determinants relevant to the health technology?
● Is there a particular subgroup of the diseased population
likely to gain the most benefit from the health
intervention/technology?
● Is there a particular subgroup of the diseased population
likely to face harm or risks from using the health
intervention/ technology?

66
 Intervention (I) ● Is the health technology licensed for used in the Philippines
by the FDA? (i.e., for drugs, vaccines, medical devices)
● Will the health technology be used for prevention,
screening, diagnosis, treatment, monitoring of the
progression of the disease, guidance in treatment selection,
knowing the prognosis, rehabilitation or other purposes?
● What is the potential place of the health technology in the
current clinical pathway in the Philippines? How might the
health technology change the current treatment or
management of the disease?
● In what particular health setting or level of care will the
health technology be likely used (e.g., home or community,
primary care, general hospital, specialty hospital,
inpatient/outpatient care, ambulatory care)?
● What is the required expertise (e.g., nurse, general
practitioner, primary care provider, specialist) to facilitate
the use of the health technology?
● What is the type/classification, indication, mechanism of
action, mode of administration or delivery,
dose/frequency/timing of use of the health technology?
● What are the expected health benefits of the intervention to
patients and healthcare providers?
● What are the expected risks or harms that may arise from
the use of the health technology?

Comparator (C) ● How is the disease currently being treated/managed in the

Philippines?
● Is there a local clinical guideline available which describes
the current standard of care or other alternative treatments
available in the Philippines?
● Are there existing variations in how the disease/condition is
being treated or managed in the local setting?

Outcome (O) Clinical Outcomes

● What are the measurable and clinically meaningful health
outcomes that should be considered in assessing the health
technology (e.g., morbidity, mortality, survival, patient
admissions/readmissions, episodes of disease health-
related quality of life)?

Economic Outcomes
● What are the costs related to treatment of the target health
condition?
● What are the costs relevant to the use of the health
technology for the targeted disease?
● What is the budget impact of implementing the health
technology?

67
 Timeframe (T) ● When is the health technology used by or administered to
patients in the disease trajectory or clinical pathway (i.e.
emergency, recovery, early stage or late stage, acute or
chronic stage)?
● How long is the health technology used to produce a
clinically meaningful health outcomes?

Ethical, Legal, ● Are there ethical issues relevant to the Philippine context
and Social that should be considered in the assessment report (e.g.,
Implications (ELSI) need for informed consent of targeted patients, implications
on fairness and equity, other populations likely to be
affected or disadvantaged with the use of the health
technology)?
● Are there existing laws, statutes, or policies in the
Philippines which may impact on the implementation of the
health technology for general or specific situations in the
Philippines? Do they allow for, mandate or prohibit against
the use of the health technology?
● Does the general population or specific subpopulations find
the use of the health technology acceptable or
controversial?
● Are there population factors that need to considered in the
assessment that may affect the equity in distribution of
health outcomes as a result of using the health technology?
(refer to PROGRESS-Plus framework on p. 64 which
includes determinants of health equity)
● Are there patient-relevant values or outcomes that should
be considered in the assessment of the health technology
(e.g., ease of use to patient or caregiver, acceptability)?

Health System ● Are there organizational changes and additional costs

Impact likely needed in the local health system or health setting
required in adopting the health technology?
● Is training of health facility end users needed to use the
health technology?

68
 Annex 4 - Data Sources: Relevant databases
HTAs in Other Settings Clinical Evidence

NICE (UK) PubMed/MEDLINE

CADTH (Canada) Cochrane Library
HITAP (Thailand) EMBASE
GEAR HERDIN

Ongoing Trials:
https://clinicaltrials.gov/
http://apps.who.int/trialsearch/

Unpublished local trials:

Coordinate with FDA or local manufacturers

Topic-specific databases:
• BIOSIS Previews (Biology and pharmacology)
• AMED (Allied and Complementary Medicine)
• CINAHL (Nursing and Allied Health)
• PsycINFO (Psychology)
• HuGE (Human Genome Epidemiology)
• International Pharmaceutical Abstracts
• Occupational Therapy Journal of Research Index
• Applied Social Sciences Index and Abstracts

69
 Annex 5 - Data Sources: Input for HTA
DATA SOURCE AGENCY/BODY DISAGGREGATION WEBSITE

EPIDEMIOLOGIC

Demographic and health indicators

Population and vital Philippine Census Philippine Age, gender, http://openstat.psa.gov.ph/Database/Pop

statistics (i.e. births, of Population and Statistics geographic location, ulation-and-Migration
deaths, marriage) Housing Authority socioeconomic status
https://psa.gov.ph/population-and-
Civil Registration housing
Service Geographic location
(i.e. province, city, https://psa.gov.ph/content/vital-statistics-
Vital Statistics barangay) report-vsr
Report

https://www.doh.gov.ph/sites/default/files
Philippine DOH /publications/Philippines%20projected%
Population 20pop%20by%20Prov%2CCity%2CBara
Projections ngay%202018-2022.pdf

Poverty incidence Philippine Poverty Philippine Age, gender, https://psa.gov.ph/content/proportion-

and indicators Statistics Statistics geographic location, poor-filipinos-registered-210-percent-
Authority (PSA) socioeconomic status first-semester-2018

Natality Philippine Health DOH Age, gender, https://psa.gov.ph/vital-statistics/table-

Statistics Epidemiology geographic location year/2017
Bureau

Causes of morbidity Philippine Health DOH Age, gender, https://www.doh.gov.ph/sites/default/files

and mortality Statistics Epidemiology geographic location, /publications/2016_PHILIPPINE-
Bureau etiology HEALTH-STATISTICS.pdf

70
 DATA SOURCE AGENCY/BODY DISAGGREGATION WEBSITE

Fertility, maternal Philippine Health DOH- Age, gender, https://www.doh.gov.ph/sites/default/files

health and child Statistics Epidemiology geographic location, /publications/2016_PHILIPPINE-
health Bureau socioeconomic status HEALTH-STATISTICS.pdf

National Philippine https://psa.gov.ph/sites/default/files/PHIL

Demographic Statistics IPPINE%20NATIONAL%20DEMOGRAP
Health Survey Authority (PSA) HIC%20AND%20HEALTH%20SURVEY
%202017_new.pdf
https://psa.gov.ph/statistics/children
Child Poverty
Database

Communicable diseases

Notifiable and other Philippine Health DOH- Age, gender, DOH website or upon request
communicable Statistics Epidemiology geographic location
diseases (e.g., Bureau
Tuberculosis, Philippine
Dengue, HIV/AIDS, Integrated
Vaccine preventable Disease
diseases etc.) Surveillance and
Response
(PIDSR)

Event-based
Surveillance and
Response (ESR),
Surveillance
Reports

Antimicrobial Antimicrobial Research Institute Geographic location, https://arsp.com.ph/reference/recent-

resistance patterns Resistance of Tropical type of bacterial publications/
Medicines (RITM) pathogen

71
 DATA SOURCE AGENCY/BODY DISAGGREGATION WEBSITE

Surveillance
Program (ARSP)

Noncommunicable diseases

Nutritional status National Nutrition Food and Age, gender, http://122.53.86.125/NNS/8thNNS.pdf

(i.e., malnutrition, and Health Nutrition geographic location
obesity) Survey Research institute
(FNRI)

Tobacco smoking National Nutrition Food and Age, gender, http://122.53.86.125/NNS/8thNNS.pdf

and Health Nutrition geographic location,
Survey Research Institute type of tobacco
(FNRI) product, https://www.who.int/tobacco/surveillance
Global Adult knowledge/attitude /survey/gats/phl_country_report.pdf?ua=
Tobacco Survey and behaviours 1
(GATS): Country
Report https://www.doh.gov.ph/sites/default/files
/publications/FinalGYTS_CountryReport.
Global Youth pdf
Tobacco Survey:
Country Report

Cancer GLOBOCAN International Age, gender, type of http://gco.iarc.fr/today/

Association of cancer, geographic
Cancer Registries location, type and
(IACR) site of cancer
Philippine Cancer http://www.philcancer.org.ph/wp-
Facts and Philippine Cancer content/uploads/2017/07/2015-PCS-Ca-
Estimates 2015 Society Facts-Estimates_CAN090516.pdf

Hypertension/ National Nutrition Food and Age, gender, http://122.53.86.125/NNS/8thNNS.pdf

diabetes and Health Nutrition geographic location
Survey

72
 DATA SOURCE AGENCY/BODY DISAGGREGATION WEBSITE

Research Institute
(FNRI)

Injury National Injury DOH- Age, gender, type of https://www.doh.gov.ph/sites/default/files

Surveillance Epidemiology injury, geographic /publications/ONIESS_FACTSHEET_20
System (NEISS) Bureau location 18_Q4.pdf

End stage renal Philippine Renal DOH-National Age, gender, https://nkti.gov.ph/services/specialty-

disease (ESRD) Disease Registry Kidney and geographic location centers/renal-disease-control-program-
(PRDR) Transplant redcop
Institute (NKTI)

ECONOMIC INDICATORS

Gross domestic BSP Online Bangko Sentral Industrial origin, http://www.bsp.gov.ph/PXWeb2007/data

product (GDP)/ Statistical ng Pilipinas (BSP) share of expenditures base/SPEI/inc_exp_accts/inc_exp_accts
Gross national Database _en.asp
income (GNI)
Philippine http://openstat.psa.gov.ph/Database/Eco
Annual National Statistics nomic-Accounts/National-Accounts-of-
Accounts Authority (PSA) the-Philippines/Annual-National-
Accounts

Income, Family Income Philippine Age, gender, https://psa.gov.ph/sites/default/files/FIES

consumption and and Expenditure Statistics geographic location, %202015%20Final%20Report.pdf
expenditures Survey Authority (PSA) socioeconomic status
http://openstat.psa.gov.ph/PXWeb/pxwe
Annual National b/en/DB/DB__1E/?tablelist=true
Accounts
http://openstat.psa.gov.ph/PXWeb/pxwe
b/en/DB/DB__2B__NA__AN__HF/?table
list=true

73
 DATA SOURCE AGENCY/BODY DISAGGREGATION WEBSITE

Consumer Price BSP Online Bangko Sentral Consumer group/item http://www.bsp.gov.ph/PXWeb2007/data

Index and inflation Statistical ng Pilipinas (BSP) base/SPEI/prices/prices_en.asp
rates Database
Philippine http://openstat.psa.gov.ph/PXWeb/pxwe
PSA National Statistics b/en/DB/DB__2M__PI/?tablelist=true
price database Authority (PSA) https://psa.gov.ph/statistics/survey/price/
summary-inflation-report-consumer-
price-index-2012100-june-2019

Peso exchange Reference Bangko Sentral Country http://www.bsp.gov.ph/statistics/statistics

rates Exchange Rate of ng Pilipinas (BSP) _exchrate.asp
the Philippines

Health expenditures Philippine Philippine Financing agent, https://psa.gov.ph/pnha-press-

National Health Statistics financing scheme, release/data
Accounts Authority (PSA) type of healthcare
providers

CLINICAL EFFECTIVENESS

Systematic reviews Cochrane Cochrane Topic, year of https://www.cochranelibrary.com/cdsr/ab

of health Database of publication out-cdsr
technologies Systematic
reviews

Global clinical trial International World Health Trial phase, country http://apps.who.int/trialsearch/
registry Clinical Trials Organization
Registry Platform (WHO)
(ICTRP) Trial phase, country https://clinicaltrials.gov/
US Food and
Clinicaltrials.gov Drug
Administration https://www.clinicaltrialsregister.eu/ctr-
(US FDA) search/search

74
 DATA SOURCE AGENCY/BODY DISAGGREGATION WEBSITE

EU Clinical Trials European

Register Medicines
Agency (EMA)

PHL Clinical trial Philippine Health Philippine Food Trial phase http://registry.healthresearch.ph/
registry Research and Drug
Registry (PHRR) Administration
(FDA)
http://drugdiscovery.pchrd.dost.gov.ph/
Philippine Drug Philippine Council
R&D Database for Health
Research and
Development
(PCHRD)

COSTS

Public drug tender Drug Price DOH- Geographic location, https://dpri.doh.gov.ph/

prices of DOH and Reference Index Pharmaceutical hospital
DOH Hospitals (DPRI) Division

Wholesale, Electronic Drug DOH- Geographic location, https://dpw.doh.gov.ph/

distribution and retail Price Monitoring Pharmaceutical type of drug outlet
prices of essential System (EDPMS) Division
drugs and Drug Price
Watch

Hospital services Philhealth case Philhealth https://www.philhealth.gov.ph/benefits/

billing rates for medical
and surgical Medical procedures:
procedures https://www.philhealth.gov.ph/circulars/2
017/annexes/0019/AnnexA-
MedicalCaseRates.pdf

75
 DATA SOURCE AGENCY/BODY DISAGGREGATION WEBSITE

Surgical procedures:
https://www.philhealth.gov.ph/circulars/2
015/annexes/circ08_2014/Annex2_Listof
ProcedureCaseRatesRevision1.pdf

Philhealth Z-benefit Database listing Philhealth https://www.philhealth.gov.ph/benefits/

packages cost of
catastrophic
benefit packages
reimbursed by
PHIC

Salaries of health Salary Grade DOH Type of health Available upon request
professionals table of the professional
Department of
Health

Health expenditure UHC Medium- DOH Health program, https://www.doh.gov.ph/sites/default/files

program term Health priority disease, /publications/MTEP%202019-
Expenditure expenditure 2022%20Update%20for%20CY%20202
Program classification 0%20Budget%20Preparation.pdf

If there are important reasons to use different unit prices from those recommended by the DOH or PhilHealth (i.e., national
registries from Medical Societies containing various fees practiced in the public or private sector), each should be
justified and its source or generation should be described.

UTILIZATION/SERVICE USE

Coverage of Field health DOH- Age, gender, https://www.doh.gov.ph/sites/default/files

essential primary Service Epidemiology geographic location, /publications/FHSIS_Annual_2018.pdf
care services Information Bureau
system (FHSIS)

76
 DATA SOURCE AGENCY/BODY DISAGGREGATION WEBSITE

https://psa.gov.ph/sites/default/files/PHIL
National IPPINE%20NATIONAL%20DEMOGRAP
Demographic and HIC%20AND%20HEALTH%20SURVEY
Health Survey %202017_new.pdf

Water and sanitation National Philippine Age, gender, https://psa.gov.ph/sites/default/files/PHIL

Demographic Statistics geographic location, IPPINE%20NATIONAL%20DEMOGRAP
Health Survey Authority (PSA) socioeconomic status HIC%20AND%20HEALTH%20SURVEY
%202017_new.pdf

Hospital services Philhealth claims Philhealth Age, gender, Available upon request in PhilHealth
database diagnosis (ICD-10),
geographic location

Hospital and National sales IQVIA Type of drug outlet; Available upon request
pharmacy drug sales audit (NSA) Company, https://www.iqvia.com/locations/philippin
data innovator/generic; es
geographic location,

Public drug Drug price DOH Geographic location, https://dpri.doh.gov.ph/

procurement data reference index hospital

QALY/DALY

HRQOL Philippine EQ-5D- DOH- Age, gender, Available upon request

5L Study Pharmaceutical geographic location,
Division ethnicity

DALY WHO Global World Health Age, gender, https://www.who.int/healthinfo/global_bur

Health Estimates Organization geographical den_disease/GHE2016_YLL-2016-
2016 (WHO) location, etiology country.xls

77
 Annex 6 - Clinical Assessment Protocol Template
Title

● Authors
Protocol Information
● Contact Information

● Description of the condition

● Description of current available treatment and proposed
intervention
Background
● Description of how the proposed intervention might
work
● Significance of the review

Research Question or
Specific Objectives

● Inclusion criteria (PICO)

● Exclusion criteria
● Data Sources (e.g., electronic databases, grey
literature)
○ At least 2 electronic databases
● Search Terms/Strategy, including filters (if applicable)
Methods ○ MeSH (Medical Subject Headings)
○ Boolean terms
● Data Collection and Analysis (including number of
reviewers, process of resolving disagreements)
● Presentation of Study Selection (i.e., PRISMA)
● Assessment of Risk of Bias
● Dealing with Missing Data

References

About the Article
● Contributions of Authors
● Declarations of Interest
● Sources of Support
○ Internal sources
○ External sources

78
 Annex 7 - Critical Appraisal Tools
Type of Evidence Critical Appraisal Tool

Summaries/Guidelines AGREE II

Overview of Reviews Ballard and Montgometry Checklist

Systematic Reviews and Meta-Analyses AMSTAR 2

Randomized Controlled Trials Cochrane Risk of Bias (RoB 2)

Non-randomized Studies ROBINS-I

Other Types of Studies CASP

79
 Annex 8 - Sample Data Extraction Table
First author, Study Outcomes
Country Population Intervention Comparator Findings
Year Design measured

*may add rows as necessary

80
 Annex 9 - Sample Costs Table
Unit cost Quantity Total Cost Source of Data

Model stage

Element 1

Element 2

Total

81
 Annex 10 - Adjusting for inflation and currency changes
Inflation decreases the purchasing power of a currency which means that it can cost more to
provide the same quantity of goods and services over time. Therefore, costs obtained from
varying time periods and across different countries should be adjusted for inflation to make them
comparable with each other.

The method used for adjustment should reported and justified in sufficient detail especially the
following information (Turner et al., 2019) Underlined is the preferred choice.:
Choice of measure of inflation ● Consumer Price Index (CPI)
● Gross Domestic Product (GDP) implicit price
deflator (recommended by the 2003 WHO-
CHOICE guide to CEA)

*GDP implicit price deflators and CPIs are available from

the International Monetary Fund: World Economic Outlook
and the World Bank.

Choice of output currency ● Local currency

● US dollars ($)
● International dollars (I$)

Choice of adjustment method ● Exchanging US dollars or international dollars to

local currency then inflating using local inflation
rates
● Inflating US dollars or international dollars using
US inflation rates and then exchanging to the local
currency
● Mixed Approach: splitting the costs into tradable
and non-tradable resources in which:
○ Method 1 will be used on tradable
resources
○ Method 2 will be used on non-tradable
resources

*The World Bank can be a source of exchange rates.

*The Global Health Costing Consortium prefers Method 3
or the mixed approach within their reference case for
estimating the costs of global health services and
interventions.

82
 Annex 11 - Summary of the Philippine Reference Case
Principle Methodological Specification

Perspective Publicly-funded Healthcare Payor perspective; Societal perspective if

deemed necessary or as an additional analysis

Comparator Similar comparator/s used in the clinical assessment

Measure of Health
Preferably QALY; DALY for practical considerations
Outcome

Measurement of All associated costs relevant to the perspective taken for both the
Costs proposed intervention and its comparator/s

Time Horizon A time horizon that is sufficiently long to reflect all important differences
in costs or outcomes between the technologies compared
● A lifetime time horizon is required when alternative
technologies lead to differences in survival or benefits that
persist for the remainder of a person's life.
● A time horizon shorter than a patient's lifetime is justified if
there is no differential mortality effect between treatment options,
and the differences in costs and health-related quality of life
relate to a relatively short period (e.g., in the case of an acute
infection which has no long-term sequelae).

Discounting 7% per year for both costs and outcomes

● Sensitivity analysis should be performed at
three (3) and ten percent (10%).

Period of Years Discount Rates

0-30 7%

31-75 5.33%

76+ 3.66%

Heterogeneity Subgroup analysis

Uncertainty Uncertainties should be explored and quantified where possible through

both deterministic and probabilistic sensitivity analyses

83
 Annex 12 - Duration of Assessment

Scoping and Protocol Development Duration (weeks)

Protocol development and quality assurance 4

● Involves review of related literature, stakeholder consultation/s,
and refinement of PICOT question

Clinical Assessment

Critical appraisal of a current synthesized evidence 1

Rapid review 1-12

Systematic review (with or without MA/NMA) 24-52

Economic Assessment

Cost-minimization analysis (CMA) + Budget Impact Analysis (BIA) + 8-24

(Ethical, Legal, Social Impact (ELSI) and Health System Implications
(HSI)

Cost-effectiveness analysis (CEA) or Cost-utility analysis (CUA) + 27-90

Budget Impact Analysis (BIA) + (Ethical, Legal, Social Impact (ELSI)
and Health System Implications (HSI)

84
 Annex 13 - Links to Required Attachments

CHECKLIST LINK

PRESS Checklist for https://www.sciencedirect.com/science/article/pii/S089543561

Search Strategy 6000585
Optimization

PRISMA Checklist for http://prisma-statement.org/PRISMAStatement/Checklist.aspx

SR/MA

http://www.prisma-
PRISMA-NMA
statement.org/Extensions/NetworkMetaAnalysis.aspx

CHEERS Checklist for https://journals.plos.org/plosmedicine/article/file?type=supple

CEA/CUA mentary&id=info:doi/10.1371/journal.pmed.1001860.s001

Drummond’s Checklist for https://www.nlm.nih.gov/nichsr/edu/healthecon/drummond_list

Assessing Economic .html
Evaluations

https://www.eupati.eu/health-technology-assessment/ethical-
EUnetHTA ELSI Checklist social-and-legal-issues-elsi-in-
hta/#Ethical_Legal_and_Social_Issues_ELSI_checklist

INAHTA Checklist for HTA http://www.inahta.org/wp-

reports content/uploads/2014/04/INAHTA_HTA_Checklist_English.pdf

85
 Annex 14 - Commonly used tools and methods for ethical evaluation in HTA
Lifted from: Assasi, N., Tarride, J. E., O'Reilly, D., & Schwartz, L. (2016). Steps toward improving ethical evaluation in health
technology assessment: a proposed framework. BMC medical ethics, 17(1), 34. https://doi.org/10.1186/s12910-016-0118-0

TOOL DESCRIPTION STRENGTHS CHALLENGES

A. Ethics literature review and appraisal

Methodological approaches for the

systematic retrieval of ethical The proposed
Encourages a
information are discussed in two search terms or
Methodologies separate literature
articles. These articles provide strategies might not
for the search search relevant to
recommendations for good practice be sufficient for
and retrieval of ethical questions,
in selection of sources of ethical retrieval of all
information on using the common
information, designing and executing relevant ethical
ethical issues retrieval framework
ethics-specific search strategies, issues. Additional
in HTA for effectiveness
quality check of search results, and targeted searches
assessments.
reporting information retrieval might be necessary.
process.

An article by Strech discusses the

appropriate criteria for appraisal of
Addresses some
empirical research required for No detailed
important
ethical reasoning. He suggests four guidelines or case
challenges of
appraisal criteria related to the studies are provided
considering
relevance of study questions, for how to apply the
empirical data in
selected outcomes and measure, appraisal criteria.
ethical analysis.
study design and generalizability of
Tools for study results.
critical
appraisal of
Mertz et al propose a set of
empirical Designed based on
structured quality criteria which can
ethics research an in-depth
be used as a checklist to guide
analysis of existing The practicality of
empirical ethics researchers and
empirical ethics the criteria is not
appraisers in the following four
research and the tested in real life
domains: research methodology,
opinion and empirical ethics
scientific and social relevance of the
experience of research practice.
research project, interdisciplinary
experts in the field
research practice, and research
of medical ethics.
ethics.

Judgment about the

McCullough et al offer a tool to help validity and quality
clinicians (particularly of ethical analyses
Designed based on
A tool for obstetrician/gynecologists) in critical and arguments
the standards of
critical appraisal of normative bioethics requires some level
critical appraisal of
appraisal of literature. The tool includes four of knowledge about
argument-based
normative questions about the focus of the ethical reasoning.
ethics and
medical ethics study, validity and soundness of the This might not be an
evidence-based
literature study results, as well as their easy task for the
medicine.
implication and usefulness in clinical target audience of
practice. the tool, i.e.,
physicians.

86
TOOL DESCRIPTION STRENGTHS CHALLENGES

The proposed
search algorithms
are not definitive
Strech e t al propose a 7-step Practical
and might need
approach for systematic reviews of recommendations
some modifications
empirical bioethics literature. The are provided for
depending on the
stepwise process involves definition each step.
context and review
of review questions, development The application of
questions.
and execution of search strategies, the proposed
Data analysis and
assessment of relevance and quality approach is
presentation may
of identified studies, and analysis and illustrated with an
require some level
Guidelines for presentation of data. example.
of knowledge and
systematic
skills in synthesis of
reviews of
qualitative data.
ethical
evidence
Structured based
Strech and Sofaer also offer a Performing a
on the common
methodology for systematic reviews “systematic” review
steps of a
of non-empirical reason-based based on this model
systematic review
bioethics literature. Their model might be time-
process.
provides instructions for formulation consuming.
Provides a detailed
of review questions and study This type of review
description of
selection criteria, identifying eligible requires some level
operational steps,
literature, data extraction and of knowledge about
and examples of
synthesis, as well as presentation of ethical reasoning. be
how to apply the
the review results. time-consuming and
model in practice.

B. Stakeholder analysis

This tool is a four quadrant matrix

that classifies stakeholders in relation
to the power that they hold and their The stakeholders
Highlights the
level of interest in the technology. interests,
Stakeholder importance of
Power classification can be based on perceptions
Power/Interest actors and interest
the ability of stakeholders to define or positions, and
grid groups in the
influence health care systems and influence are subject
technology
services, change the way services to change
are provided, or guide the public
opinion.

Procedures for
A SWOT analysis (Strengths,
Can be used to performing a SWOT-
Weaknesses, Opportunities, and
stimulate and analysis are not
Threats) can help in understanding
Stakeholder organize thoughts clearly defined.
the interests of key stakeholders, the
SWOT and discussions in The analysis is
actions they can take to support and
stakeholder prone to subjective
the risks that they pose to
analysis. biases of the
implementation of the technology.
assessors.

C. Public/stakeholder engagement

87
TOOL DESCRIPTION STRENGTHS CHALLENGES

A methodology document published

by the National Coordination Centre
for Health Technology Assessment Summarizes and
(UK) presents the results of a compares various
systematic review of qualitative and techniques in a
quantitative approaches to involving single document.
No single best
the public in in HTA. The document Uses pre-defined
technique or group
identifies and describes details of the sets of criteria to
of techniques for
techniques that can be used to obtain evaluate
public engagement
public preferences and makes methodological
is recommended by
recommendations regarding the use issues of different
this document.
of different techniques. Some of the techniques (e.g.,
Exploring Users of the tool
commonly used methods identified in validity,
public values may require
this document are as follows: reliability/reproduci
and background
− Quantitative techniques, including bility,
preferences knowledge and
ranking (e.g., simple ranking, generalizability,
specific skills that
qualitative discriminant process, and acceptability to
enable them to
conjoint analysis) rating (e.g., visual respondents, or
choose and conduct
analogue scale) and choice-based cost) identified
an appropriate
(e.g., standard gamble, time-trade- methodologies.
public engagement
off, discrete choice conjoint analysis Provides examples
technique.
and willingness to pay) methods. of how the
− Qualitative techniques, including techniques have
individual interviews, focus group been used in
discussions, Delphi technique, research practice.
citizen’s juries, consensus panels,
and nominal group techniques.

D. Identification and analysis of ethical issues

Takes into account

several ethical
This approach consists of 6 steps, perspectives and Users of the tool
whereof one step covers 7 main analytical may require some
The Socratic
questions and 33 explanatory and approaches. level of ethical
approach
guiding questions. This checklist is Can be used by knowledge in order
(Hofmann’s
designed t for identification of and HTA practitioners to use appropriate
guiding
reflecting on ethical data throughout who may be less approaches to
questions)
the HTA process, and for reflexive familiar with ethical answer the
dialogue with stakeholders. analysis. questions.
Facilitates ethical
analysis.

AECs describe the details of the Designed to

information that is outlined by the provide structured
The way in which
HTA core basic units of the HTA Core Model information
AECs should be
model’s (assessment elements). Each AEC required for
used as a part of the
assessment provides information on the element, answering the
assessment is not
element cards its importance and transferability for generic question
fully addressed in
(AECs) different applications (diagnostic, defined by each
the model.
surgical, pharmaceutical or screening assessment
technologies), and appropriate element.

88
TOOL DESCRIPTION STRENGTHS CHALLENGES

sources of information and research Useful when

methodologies to address the producing HTA
question defined by the element. reports based on
The ethical domain of the Core the HTA Core
Model includes 19 elements related Model.
to the 19 ethical issues on the topics
of beneficence/non maleficence (4
AECs), autonomy (4 AECs), respect
for persons (3 AECs), justice and
equity (3 AECs), legislation (2 AECs),
and ethical consequences of HTA (3
AECs).

Ethical matrix is an analytical tool to

Facilitates ethical
aid ethical analysis of technological
analysis by
options The matrix uses a tabular
simplifying and
format to identify ethical impact of a
structuring ethical
particular technology on different
discussion
stakeholders. The table lists a set of
Raises awareness
prima facie moral principles, typically
of a wide range of May become large,
the four Beauchamp and Childress’s
ethical concerns complex and difficult
moral principles (autonomy,
Helps researchers to manage, when
beneficence, non-maleficence, and
and decision- too many moral
Ethical matrix justice), along one axis and different
makers to avoid principles are listed
stakeholder groups along the other
bias towards a or diverse groups of
axis. Relevant facts and values are
specific moral stakeholders are
usually listed in each cell of the
principle. identified.
ethical matrix. Ethical matrix can be
Can be used in
used either to identify ethical
both expert-led and
considerations around the technology
participatory/deliber
or to quantify and compare the
ative ethical
impact of the technology on different
evaluation
principles using semi-quantitative
processes.
scores (e.g., ranging from -2 to +2).

A summary table of consequences of

Allows for
using and not using a particular
highlighting key
healthcare technology is
impacts of a
recommended in the HTA core model
particular
as an open framework for performing
technology on
ethical analysis. This table
various domains of
summarizes key benefits and
HTA.
adverse impacts of implementing of Cannot be used as a
Consequences Can be used by
the technology or otherwise on substitute for careful
table decision-makers to
various stakeholder groups. ethical reflection
compare
A consequences table summarizing
anticipated ethical
positive and negative impacts of the
issues around
technology on all domains of HTA,
alternative
along with references to the quality of
technologies in
their evidentiary sources, is also
relation to other
proposed as a part of the HTA core
domains.
model’s reporting template.

89
TOOL DESCRIPTION STRENGTHS CHALLENGES

E. Computerized support tools for aiding ethical analysis

EthXpert is a computer program

Does not focus on
designed to help the user in In some cases, the
a specific audience
summarizing and structuring ethical use of the these
or any specific
problems, describing potential inter- computer programs
EthXpert contexts.
relations between the interests of can be difficult and
Therefore, can be
different stakeholders, and analyzing time consuming,
applied to ethical
the impact of alternative technologies especially when one
evaluation in HTA.
on various stakeholders’ interests. needs to include all
details about
Illustrates the flow complex ethical
Ethos is a computer program that
of data collection problems, or too
provides a framework for organizing,
and analysis in a many different
storing and analyzing ethical
map format. perspectives.
information needed for problem
ETHOS Enables the user to The use of the
solving or decision-making. The
add or remove software may
program allows for ethical analyses
information through require investment
using different ethical theories and
an iterative in resources.
approaches.
process.

90