See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/5902939

Decision-analytical modelling in health-care economic evaluations

Article  in  The European Journal of Health Economics · November 2007

DOI: 10.1007/s10198-007-0078-x · Source: PubMed

CITATIONS READS

47 778

2 authors:

Xin Sun Thomas Faunce

Sichuan University Australian National University
173 PUBLICATIONS   4,504 CITATIONS    197 PUBLICATIONS   1,523 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Impact of free trade agreements on medicines in Australia View project

ANU Summer Research Scholarship Project View project

All content following this page was uploaded by Xin Sun on 01 January 2014.

The user has requested enhancement of the downloaded file.

Eur J Health Econ
DOI 10.1007/s10198-007-0078-x
ORIGINAL PAPER
Decision-analytical modelling in health-care economic evaluations
Xin Sun Æ Thomas Faunce
Received: 10 July 2006 / Accepted: 12 September 2007

Springer-Verlag 2007
Abstract
Objective Decision-analytical modelling is widely used
in health-care economic evaluations, especially in situa-
tions where evaluators lack clinical trial data, and in
circumstances where such evaluations factor into reim-
bursement pricing decisions. This paper aims to improve
the understanding and use of modelling techniques in this
context, with particular emphasis on Markov modelling.
Methods We provide an overview, in this paper, of the
principles and methodological details of decision-analyti-
cal modelling. We propose a common route for practicing
modelling that accommodates any type of decision-
analytical modelling techniques. We use the treatment of
chronic hepatitis B as an example to indicate the process
of development, presentation and analysis of the Markov
model, and discuss the strengths, weaknesses and pitfalls of
different approaches.
Conclusions Good practice of modelling requires careful
planning, conduct and analysis of the model, and needs
input from modellers and users.
X. Sun (&)
Department of Clinical Epidemiology and Biostatistics,
McMaster University, 1200 Main Street West, HSC 2C7,
L8N 3Z5 Hamilton, ON, Canada
e-mail: sunx26@mcmaster.ca
X. Sun
Chinese Evidence-Based Medicine Center,
West China Hospital, Sichuan University, Chengdu, China
T. Faunce
College of Law and Medical School,
Australian National University, Canberra, Australia
Electronic copy available at: http://ssrn.com/abstract=1408882
Keywords Decision analysis
Markov model

Economic evaluation
Cost-effectiveness analysis
Introduction
Several research designs usually underpin economic (or
cost-effectiveness) evaluations in health-care regulatory
systems, for example, for assessing the community value of
alleged innovation in new pharmaceuticals and medical
devices. These evaluations may rely on clinical trial-based
studies, cohort studies, database studies and decision-
analytical modelling [1]. Trial-based cost-effectiveness
evaluation is becoming increasingly emphasised as a pri-
oritised precondition (after safety, quality and efficacy
evaluation) for central government drug reimbursement
programs in Australia and veterans affairs and other
specialised agency programs in the United States [2–4].
Database analysis may be used to rapidly inform policy
making if its scope is adequate, but it usually presents
methodological concerns with low robustness of results. It
is less commonly used in health-care cost-effectiveness
decision-making.
Decision-analytical modelling has been widely used for
expert investigation of the cost-effectiveness (or economic)
profiles of health-care programs. However, concerns have
been raised that such models are low in validity due to
limited capacity for simulating the manifold complexities
of the ‘real world’. However, modelling remains an
important and essential aspect of economic evaluation and
is likely, we argue, to grow in significance as more gov-
ernments commence to link cost-effectiveness evaluations
with central government price negotiations [5, 6]. The
ongoing value of modelling, in this context, is recognised
in the following aspects:
123
 X. Sun, T. Faunce

• Extending the results of a clinical trial. Clinical trials use of cost-effectiveness evaluation of health-care technol-
are usually too short in length of follow-up to capture ogies, with particular emphasis on Markov modelling.
economically important outcomes. Modelling can help
cost-effectiveness evaluators in this situation to extend
the short-term data into the future, including extend- Decision-analytical model in economic evaluations:
ing the outcomes and costs, and translating efficacy key issues
outcomes into health economic outcomes.
• Extrapolating intermediate clinical endpoints to final Steps in decision-analysis modelling
outcomes. In some diseases, clinical trials can be only
useful to report intermediate endpoints, while physi- There is a common route of modelling practice in eco-
cians and health economists are interested in clinically nomic evaluations (Fig. 1). In relation to the research
and economically relevant endpoints. However, the question, the model structure, to which both health out-
relation between intermediate outcomes and long-term comes and cost data are attached, is developed and
prognoses is unlikely to be linear, and costs due to long- validated, expected costs and health outcomes are calcu-
term consequences are not accounted for in the clinical lated, the robustness of results is examined, and the
trials. In this case, modelling is useful in extrapolating implications of results are interpreted.
the results of trials to long-term consequences.
Identifying the objectives and developing the model
• Combining evidence into head-to-head comparisons. It
structure The research objectives are specifically defined
is common for medicines to be compared with placebos
at the beginning of modelling, following which the
within trials for the purpose of safety, quality and
research question is developed. The research question
efficacy registration, while policy makers and regula-
should clearly state the competing interventions, target
tors assessing the community value of medicines are
population and outcome measures. The perspective of the
more interested in their comparative efficiency with
study and the length of follow-up should also be specifi-
existing effective medicines in the same therapeutic
cally defined. The structure of the model is developed on
class. In such cases, head-to-head comparisons need to
the basis of an understanding of the nature of disease
be artificially synthesised by modelling.
progression, required exactness of results and the user’s
• Modelling before clinical trials are available. Sculpher
perspective. However, with restricted availability of data,
et al. [7] reviewed the use of modelling in development
compromises need to be made when developing the model
of health technologies, and indicated its value in four
structure. As the model is a process of simulating disease
different stages of technology development. Modelling
progression, it is extremely important that the model
was recognised to fulfil three aspects prior to clinical
structure be validated to meet the quality criteria.
trials: generating hypotheses that could be tested by
trials, identifying key variables, and analysing the value Assigning probabilities The probabilities are assigned
of trials. Although these studies are not routinely used to the model after the model is validated. Sources of
and reported in medical literature, the industry and probabilities include medical literature (for example,
government authorities show great interest in their
potential, especially given increasing industry pressure
for ‘conditional’ approval [8].
Identifying the research objectivesand questions
Several decision-analytical models are widely used in
economic evaluations, including the decision tree and Developing the modelstructure
Markov models, discrete event simulation [9] and mathe-
matical modelling [10]. Among these, the decision tree and Assignment of probabilities to the model
Markov models are the most frequently used. As the
simplest form of modelling technique, decision-tree anal- Assignment of utilities (and costs)
Modelling refinement
ysis has been well developed for economic evaluation
[11, 12]. However, it is limited when addressing compli- Calculation of health outcomes and costs and
incremental analyses
cated conditions of disease management. The Markov
model involves a sophisticated process of simulating Sensitivity analysis
progressive and recursive disease, empowering it to address
more challenging problems. This paper aims to provide
Presentation and interpretation of results
an overview of the state-of-the-art decision-analytical
modelling techniques to facilitate the understanding and Fig. 1 The flow chart for developing a decision-analytical model

123

Electronic copy available at: http://ssrn.com/abstract=1408882

Decision-analytical modelling in health-care economic evaluations
meta-analyses, clinical trials, observational studies), math-
ematical models, and expert opinions. The practical
difficulties in assigning probabilities are either that the data
on probabilities can not be obtained from the medical liter-
ature, or that those available could not be used in the patients
of interest because of differences in patient characteristics. In
such cases, models or expert opinions are generally applied
to obtain the probabilities. Some techniques have been used
to obtain probabilities in mathematical models, including
survival models, multivariate logistic regression analysis
(for individualized probabilities), Bayesian analysis [13],
and neural networks [14]. Expert opinions are collected and
analysed to obtain the probabilities by Delphi or modified
Delphi methods [15, 16].
With the burgeoning of evidence-based medicine in
medical research, hierarchical evidence has been proposed
for use in determining the effects of health-care interven-
tions, that is, those studies with stronger validity should be
prioritised in assessing the effects. Randomised trials and
overviews of randomised trials appear to be the best
evidence to assess the effects of health-care interventions,
while observational studies, studies without controls and
expert opinions are prone to produce biased estimates of
effects. Accordingly, the estimates from overviews of
randomised trials and individual trials are prioritised to use
for probabilities, with observational studies following.
Modelled probabilities and expert opinions usually suffer
from the low validity.
Assigning utilities and costs The final step of developing
the model is to assign a value to the relevant health states.
The assignment of health utilities and costs is performed
separately. The utility, ranging from 0 to 1, is multiplied
with the natural unit of measure (e.g., life-years gained) to
produce the quality-adjusted life expectancy resulting from
the investigated health-care intervention or product. Three
major techniques are used to measure the utility: visual
analogue scale (VAS), standard gamble (SG) and time
trade-off (TTO) [17]. These are supplemented by less used
techniques such as magnitude estimation (ME) and person
trade-off (PTO). In an overview of the health-state mea-
sures, Brazier [18] argued that, although outcomes for
health states were not diffentiated by using different tech-
niques, SG and TTO should be used for their theoretical
validity. Costs must also be assigned to each state of the
model. Costs of different states usually vary, depending on
the resource use and unit prices. Costs can be input either
in weighted form or as single costs. The use of a weighted
form enables adjustment and has the advantage of detailed,
and thus more transparent reporting, although it requires
intensive documentation and calculation.
Calculating health outcomes and costs and incremental
analysis Both expected health outcomes and costs are
calculated separately by a process referred to as ‘folding
back’. The folding-back process usually involves a great
deal of multiplication and addition and is difficult to deal
with by hand. Software such as DATA, and its later ver-
sion, TreeAge, is available for calculation of expected
results. Such folding back may be processed in a simple
decision tree. The expected health outcomes of the chance
node are the sum of the products of health utilities in each
branch multiplied by the probabilities of that branch. The
expected outcomes of the decision node are equal to
the node with the highest branch. Incremental analyses
need to be performed for health outcomes and costs, and
incremental cost-effectiveness ratios (ICERs) should be
calculated.
Sensitivity analysis The expected results from such
models, referred to as base-case results, are usually
obtained from average estimates of inputs (including
probabilities, utilities, and costs). Sensitivity analysis is
conducted to examine the robustness of results. If the result
is sensitive to a variable, the magnitude of its effect on the
results is investigated, and, if necessary, revisions are made
to estimates of the appropriateness of the model structure.
Briggs et al. [19] identified a few approaches to sensitivity
analysis that can be classified as deterministic and
stochastic in nature. In deterministic analysis, simple sen-
sitivity analysis, extreme analysis and threshold analysis
point estimates or range estimates are applied to assess the
effect of variables on results. Brennan et al. [6] proposed an
analysis of effects of model structure on the results.
Probabilistic analysis (Monte Carlo simulation) incorpo-
rates the probability distribution of key variables to
produce the distribution of expected results [20, 21].
Practice of decision-analytical modelling: guidelines
and critical appraisal checklist
The practice of decision-analytical modelling has received
markedly increased interest in the medical literature in
recent years, leading to academic and bureaucratic con-
cerns and scrutiny about its wide application in health-care
evaluations for innovative medicines and medical devices.
This is particularly true where government regulators
making cost-effectiveness reimbursement decisions worth
millions of dollars have to make decisions in the absence of
good head-to-head clinical trial data. Thus, there is a need
for modellers to ensure/guide the good practice of model-
ling, and for model users to critically appraise the quality
of practice.
Philips et al. [22] reviewed 15 existing guidelines on
good practice of analytical modelling published in medi-
cal journals. They summarised the quality of the items
123

proposed, and developed and synthesised a practice
guideline. It was found that the existing guidelines reported
attributes differently, some items being consistent across
guidelines, while the other items conflicted. Philips et al.
further synthesised a current best practice guideline from
the existing examples. Their guideline contains three
domains (structure, data, consistency), covering 21 attri-
butes. For reasons of space, we do not go into detail about
these.
Markov modelling in economic evaluation
The Markov model overcoming ‘surrogate’ outcome
data: an example
To give an example of the value of Markov modelling in
health-technology assessment, consider the hypothetical
treatment of chronic hepatitis B (CHB). An idea that
immediately arises to a cost-effectiveness assessor is that
CHB is a chronic disease, which requires long-term follow-
up. The relevant clinical and economic endpoints of most
policy interest are life-years gained and quality-adjusted
life years (QALY), while the measures in clinical trials are
clearance of HBeAg and HBeAg seroconversion that are
obtained at 1 year of follow-up (Fig. 2). These ‘surrogate’
measures, though indicating prognosis of CHB, may not
equate with the long-term outcomes as effectively as
QALY. Therefore, the surrogate results need to be further
extrapolated by assessors.
To extrapolate the 1-year results, each health state at the
end of 1 year will be further progressed to its possible
consequences. At the end of a second year, the possibility
of patients being in different health states will be calculated
in the same way. In this manner, the results of the third
year and so on are obtained until the end of the follow-up.
HBeAg-
treatment
0.905
HBeAg+
HBeAg+ CHB
HBeAg-
regular care
0.896
HBeAg+
Fig. 2 A hypothetical decision tree for HBeAg-positive (HBeAg+)
chronic hepatitis B. This decision tree assesses the effects of
treatment on clearance of HBeAg in patients with HBeAg+ chronic
hepatitis B, with 1-year follow-up. Patients who achieve clearance of
123
X. Sun, T. Faunce
Clearly, such a recursive process would require inten-
sive decision-tree modelling, and the nodes and branches of
the decision trees would increase exponentially over the
follow-up. Thus it is nearly impossible to use the simple
tree to simulate the process, especially when the time
horizon is long and the tree involves multiple health states.
Therefore, a more complex model than a recursive process
of clinical prognosis is needed. The Markov model is an
ideal modelling technique to achieve this purpose [23].
The conceptual framework of Markov model
Markov model and Markov states
Previous publications [23, 24] described the Markov model
from a variety of disparate political, sociological and
economic perspectives. Here, we provide an overview of
the Markov modelling technique by emphasising the rele-
vant policy issues and, where necessary, using the practical
examples relevant to health-technology assessors.
Markov modelling is a technique that allows presenta-
tion and analysis of random processes over time. It is
usually used to simulate disease progression in a defined
period of time, and is particularly suitable for diseases that
are chronic and recursive in nature. The principle of
Markov modelling is that the disease of interest is divided
into discrete states of progression, in a stochastic manner,
over a certain defined time interval (termed ‘Markov
cycle’). The length of the time interval is called the cycle
length. Through the defined time horizon, the health utili-
ties and costs, which are attached to each discrete state, are
accumulated. An example of a Markov modelling process
is depicted in Fig. 3.
In the Markov model, each possible health condition is
termed a ‘Markov state’. During the modelling process,
0.95
0.23
HbeAg+
0.90
#
0.891
# cirrhosis
0.75
0.06
0.95
0.09
HbeAg+
0.90
#
0.891
# cirrhosis
0.75
0.06
HBeAg are assumed not to develop cirrhosis, thus no choice node is
developed at the state of HBeAg–. CHB Chronic hepatitis B, HBeAg
hepatitis B e antigen
Decision-analytical modelling in health-care economic evaluations
HBeAg-
CHB
HBeAg+
CHB
Cirrhosis Death
Fig. 3 The Markov modelling process of treatment arm for chronic
hepatitis B. With a cycle of 1 year, the initial patients who have
HBeAg+ chronic hepatitis B (grey oval) might have different
prognostic outcomes at the end of 1-year. Some may stay in the
same health state, while others may experience clearance of HBeAg
(HBeAg– CHB), or progress to cirrhosis and even death. The
proportion of patients progressing to different states depends on the
probabilities. For the next round, those patients who have been in
different states may experience similar changes, based on the disease
progression routes (indicated by arrows) and the probabilities. In this
pattern, the patients in different states would be re-distributed at the
end of each cycle until the end of the follow-up
patients may stay at the same state, or transit to another at
the end of a cycle. In some cases, patients may transit
interchangeably between states within a cycle. Such tran-
sition of states depends on the clinical prognosis (or natural
history) of the disease. All the above states can be grouped
into transition states. However, the Markov process ulti-
mately needs to posit a state that patients cannot leave.
Such a state is called the ‘absorbing state’. After a number
of cycles, patients would enter this state.
In addition to the routinely defined states, the Markov
model also uses another two less applied but useful states:
temporary and tunnel states. Temporary states are used
when the event has short but important effects, and when
the patients can only stay at the state, at most, for one
cycle. The use of temporary states enables assessors to
assign state-specific transition probabilities and adjust
utilities and costs. One example of using such a temporary
state is modelling of a stroke event, which could initially
require hospitalisation and intensive care, thus leading to
significant impact on costs and prognosis. The tunnel state,
in which patients can only transit in a fixed sequence, is
appropriately analogous (given the nature of life-threaten-
ing disease) to passing through a tunnel, and would be used
when a temporary state would last more than one cycle.
The Markov states should represent the clinically and
economically important events that occur to patients over
time. Another equally important factor that needs to be
carefully defined is cycle length. Usually, the cycle length
should be short enough to consider the changes of clinical
effects and resource use in patients between the cycles.
The transition between states in the Markov model
depends only on their definition in that cycle. The model
has no intrinsic property that memorizes the history of
previous events to determine transitions. This feature of the
model is called ‘memorylessness’ and is also termed
the ‘Markovian assumption’. This is a major limitation of
the Markov model, as the disease prognosis for each patient
usually depends on the previous history of diseases.
Modellers should be aware of such limitations during
modelling, and justify whether such a limitation would
significantly affect the validity of the model in the context
of their evaluation. However, solutions may be found to
avoid such limitations. These may include a combination
of more complex states to model the disease history, and
the use of time-dependent probabilities.
Transition probabilities
As previously mentioned, in the Markov model, patients
transit between the states according to the predefined path
in each cycle. The chance that the patients move between
states within a cycle is termed ‘transition probability’.
Theoretically, if the model has n states, there would be
n · n possible transition probabilities between states,
yielding an n · n transition matrix. However, many of
these probabilities are set to zero, since there would be no
transition between the states.
In the Markov model, multiple cycles are employed to
model the disease prognosis. Two sets of transition prob-
abilities are used to characterise the type of Markov
modelling in this respect. A Markov model that assumes
the transition probabilities are constant over time is called a
Markov chain model. Mathematically, this assumption has
the advantage that outcomes can be calculated by using
simple matrix algebra.
However, transition probabilities (for example, that of
mortality) change with age, sex, and other relevant patient
characteristics. The more commonly used transition prob-
abilities in the health-care system tend to be time-
dependent, which takes into consideration the disease
progression in the real world. The advantages of time-
dependent transition probabilities include that they can get
around the Markovian assumption, and reduce the chance
of over- and underestimation of results. A comparison of
the use of constant probabilities with time-dependent
probabilities in the Markov model is examined by using
different probabilities for the transition of HBeAg-positive
CHB to cirrhosis over 40 years. As a result, there is a 10%
difference in mortality rates between the models using
constant and time-dependent probabilities.
One practical problem in practicing Markov modelling
is identification and application of transition probabilities.
The transition probabilities are obtained from published
medical literature. However, the estimates of rates of
123
 X. Sun, T. Faunce

events for a period of observation may not be equal to Initial stage HBeAg - HBeAg + Cirrhosis Death
Cycle 0 (n=0) (n =1000) (n = 0) (n = 0)
transition probabilities needed for each cycle. Therefore, it Cycle total utility: 0.95 utility: 0.90 utility: 0.75 utility: 0.00
is wrong to simply copy and apply data (for example rate of (utility/costs) costs:€ 100 costs:€ 500 costs:€ 900 costs: € 50
Incremental HBeAg - HBeAg + Cirrhosis Death
events) from the medical literature. Modellers should judge Cycle 1 (n = 230) (n = 710) (n = 60) (n = 0)
that the length defined is equal to the defined cycle length U = 451 QALYs
C =€ 216×10 3
230× 0.95
230×€ 100
710× 0.90
710×€ 500
60 × 0.75
60×€ 900
0 × 0.00
0× € 50
in the model. Otherwise, a formula can be employed to Incremental HBeAg - HBeAg + Cirrhosis Death
Cycle 2 (n = 282) (n = 580) (n = 129) (n = 9)
translate the rate of events to transition probability. U = 887 QALYs 282× 0.95 580× 0.90 129× 0.75 9 ×0.00
C =€ 434 ×10 3 282×€100 580×€ 500 129×€ 900 9×€ 50

p ¼ 1  expðrtÞ Incremental HBeAg - HBeAg+ Cirrhosis Death

Cycle 40 (n = 198) (n = 56) (n = 51) (n = 695)
where p is wanted transition probability and r is the overall U = 284 QALYs
C = € 130 ×10
3
198× 0.95
198×€ 100
56× 0.90
56 × € 500
51× 0.75
51× € 900
695 × 0.00
695 × € 50
rate over the time of t.
Fig. 4 An illustrative example of a Markov cohort simulation. This
example is also adopted from the CHB model, which runs for 40
cycles to calculate the distribution of patients in different states. The
Presentation of the Markov model numbers in brackets are the number of patients in different states of
each cycle. The patients moving to the state of death increases over
Three possible methods are available to present the Markov time. This figure will be further discussed in the section ‘‘Attaching
process. These are the matrix algebra solution, cohort utilities and costs to the model’’ about the cost and outcomes
evaluation (as shown in grey boxes)
simulation and Monte Carlo simulation. The matrix algebra
solution, although mathematically appealing, is not able to
handle time-dependent transition probabilities, which exist cycle, the bigger the effect that the correction has on the
widely in medical fields. Practically speaking, the matrix total estimates. However, Briggs et al. [24] argued that
algebra solution is too complex for routine use by non- such a correction might not be so important in economic
mathematicians. evaluations, since calculation of incremental costs and
benefits would not be affected.

Markov cohort simulation

Cohort simulation is the most straightforward for present-
ing Markov process and is easily appreciated by health-
care modellers, assessors and policy makers. Starting this
simulation, a hypothetical cohort of patients (e.g., 1,000
people), who are preliminarily distributed in different
states, transit from one state to another each cycle, and
form a re-distributed cohort of patients in different states at
the end of each cycle. During each cycle, the data on how
many patients have stayed in different states are recorded.
These processes are repeated for a number of cycles to
obtain summary results on patients time spent in different
states. An illustrative example is presented in Fig. 4. The
number of patients in each state is divided by the total to
calculate the probabilities of patients being in the states at a
given time.
In the Markov cohort, it is assumed that transitions
happen at the end of each cycle. However, in reality,
patients’ transition is a continuous process, which may
occur during any time of the cycle. To address this, a half-
cycle correction is used, which assumes that state transi-
tions occur, on average, half-way through the cycle. This
approach is analogous to the estimation of survival in the
survival curves. This correction approach is important to
evaluate health outcomes, particularly when the cycle
length is very long. Usually, the longer the length of the
Markov Monte Carlo simulation
An alternative simulation method is Monte Carlo simula-
tion, or so-called individual simulation. It is assumed that a
large number of patients enter the model individually, and
go through the states in a random manner. After repeating
many cycles of this behaviour, the overall results are
obtained by summing the outcomes for all patients who go
through the model. An example is shown below to indicate
the processes of Monte Carlo simulation (Fig. 5). It is clear
that most patients will be absorbed when they go through
the model. However, in this simulation, it is not necessary
that all patients ultimately go into the absorbing state after
the defined number of cycles.
The differences between cohort simulation and Monte
Carlo simulation are (1) patients in cohort simulation move in
a given path through the model, while individual patients in
Monte Carlo simulation transit stochastically, and (2) calcu-
lation methods for expected outcomes are different between
these two approaches, which will be discussed further in the
section ‘‘Markov modelling in economic evaluation’’.
As the Monte Carlo simulation enables the patients to
transit through the model randomly, it has the advantage of
producing ‘distribution profiles’ of patients with random
deviation. The uncertainty of the model can be presented
with such a ‘distribution profile’.

123
Decision-analytical modelling in health-care economic evaluations

Cycle 0 HBeAg - HBeAg + Cirrhosis Death where ns and us are number of patients and utilities of a
patient A
certain health state in cycle i and s is the number of
Cycle 1 HBeAg - HBeAg + Cirrhosis Death states within the cycle. This also applies to the cost
•••••• estimation.
Cycle 20 HBeAg - HBeAg + Cirrhosis Death In the Monte Carlo simulation, the total health outcomes
••••••
and costs of an individual are the sum of consequences and
Cycle 30 HBeAg - HBeAg + Cirrhosis Death
costs of states that the individual experiences transiting
••••••
through the model. For example, in Fig. 5, the formula for
Cycle 40 HBeAg - HBeAg + Cirrhosis Death
health outcomes of patient A is:
Fig. 5 An illustrative example of a Monte Carlo simulation. Again X X
adopted from the hepatitis B model, this simulation process simulates Un ¼ ui ¼ 0:95ðcycle 1Þ þ


þ 0:95ðcycle 20Þ
the patients’ clinical pathways in a random manner. Each patient who
enters the model will have random disease progression. In the þ


þ 0:75ðcycle 30Þ þ 0:00ðcycle 40Þ
example, three patients have different prognoses in the model
(symbolised by three different lines). After many cycles, these where Un is expected health outcomes of patient n
patients have different outcomes (expressed here as QALY, as the cycle length is 1 year),
ui is the utility in cycle i (the utilities here are adopted from
Fig. 4). Such a calculation method can be also applied to
Markov modelling in economic evaluation costs. After repeating this behaviour a defined number of
times, total health outcomes and costs can be obtained by
The Markov model is widely used in economic evaluation summing the health outcomes and costs of all patients. This
to estimate costs and health outcomes of health-care can be expressed as the following formula:
interventions, particularly for those diseases of chronic and X X X
recursive nature [25, 26]. Utotal ¼ m  Un ¼ m  ui

where Utotal is the overall health outcome of all patients

who go through the model and m is the number of patients
Attaching utilities and costs to the models who entered the model.

In economic evaluation, utilities and costs are attached

separately to the model, and the methods of attaching Comparison of results from two simulation approaches
values are the same. However, the methods for attaching
values differ in Markov cohort simulation and Monte Carlo Although the methods for estimating costs and health
simulation. outcomes are different, the results from these two
In the cohort simulation, the patients stay at the different approaches are similar. We performed the Markov cohort
health states through a cycle, and their health outcomes and and Monte Carlo simulations using a hypothetical model of
costs are calculated separately for the states in that cycle. treatment for chronic hepatitis B. The results are presented
The total values for that cycle are thus obtained by sum- in Table 1.
ming the health outcomes and costs in different health
states. An example is shown to illustrate the process
(Fig. 4). Health utilities and costs are calculated indepen- Table 1 Comparison of results from two approaches to a hypothet-
dently for each cycle, and the accumulated health utilities ical model
and costs are calculated by summing the cycle totals. The
Approach Costs (€) QALYs ICERs (€/QALY)
formula for calculating health outcomes (or costs) in cohort
simulation can also be expressed mathematically. If m is Cohort simulation
the number of cycles, and Ui is the total health outcome in Treatment 9,070 19.90
cycle i, then the overall health outcomes (Utotal) can be Regular care 7,508 18.02
calculated by formula (1): Difference 1,562 1.88 830.85
Monte Carlo simulation
X
Utotal ¼ m  Ui : ð1Þ Treatment 8,253 ± 5,249 19.57 ± 12.95
Regular care 6,737 ± 5,028 17.74 ± 13.24
Further, Ui can be calculated by using Eq. (2): Difference 1,516 1.83 828.42
X X QALYs Quality-adjusted life years, ICERs incremental cost-effec-
Ui ¼ ðns  us Þ ð2Þ tiveness ratios

123

Clearly, the absolute values for costs and QALYs pro-
duced by the two approaches are somewhat different, but
the differences in costs and QALYs are nearly the same,
thus so are the ICERs. Also, Monte Carlo simulation pro-
duced the deviation of the average of costs and QALYs,
reflecting the uncertainty of the estimated results. How-
ever, such uncertainty is just a representation of the random
nature of the model structure.
Dealing with uncertainty in Markov modelling
Types of uncertainty
Markov modelling is a process that combines multiple
sources of evidence into a pre-specified structure, to per-
form analysis by using a variety of analytical methods. It
is an unavoidable fact that Markov modelling always
involves uncertainty. Generally, four types of uncertainty
can be identified in Markov modelling:
• Parameter uncertainty—the data on probabilities, health
consequences (e.g., utility weights), and resource use
(sometimes, unit prices).
• Analytical uncertainty—the choice of time frame, the
costing methods, outcome valuation methods, and
inclusion of indirect costs.
• Generalisability of results—questions raised by the
inclusion of patients with differing characteristics or
from different clinical settings.
• Structural uncertainty—whether the developed struc-
ture can simulate the disease progression and the
possible outcomes.
To examine the robustness of results when these four
aspects are varied, sensitivity analyses are always used in
Markov models. For analytical uncertainty, a ‘reference
case’ on methodology handling is usually applied, partic-
ularly in guides and recommendations [27, 28], to
minimize the methodological variability. For structural
uncertainty, sensitivity analysis can be performed to
modify the structure of the model to identify possible
structural factors that affect the robustness of results.
However, less attention is usually paid to this in published
modelling studies. It has been argued that such uncertainty
should also be considered during sensitivity analysis.
Another type of uncertainty is generalisability, which
has important implications when the models include
patients with different confounding factors (e.g., ages, sex,
risk factors), or when the modelling is performed based on
data from various settings (e.g., multinational resource-use
data) [29]. Briggs [20] reported the effects of patient
characteristics on cost-effectiveness results intensively. It
was found that some patient parameters were crucial to the
123
X. Sun, T. Faunce
cost-effectiveness results. A distinctive example was the
intervention of statins on patients with different cholesterol
levels [30], in which incremental cost-effectiveness ratios
(ICERs) were separately reported in relation to the cho-
lesterol levels. The stratified analysis found that the ICERs
differed substantially in patient groups. In fact, the causes
for the differences are easily understood, since the dis-
tinctions in crucial confounding factors not only affect the
patient prognosis, and thus the health outcomes, but also
the resource use. However, such variability in patient
characteristics is not exactly a matter of uncertainty. It
relates more to the specification of patients at the stage of
design. Modellers should be aware of the confounding
factors, and specify the patient populations, and pre-specify
the analysis at the design stage.
Last, but most commonly discussed, is ‘‘parameter’’
uncertainty. Generally, parameters are obtained from
the sampling patients. The estimates and variances (for
example, standard errors) of the parameters are usually
produced to represent the uncertainty. When the sampling
estimates are not available, expert opinions and literature
review of published studies are used. To deal with
uncertainty, point- and range-sensitivity analyses are
routinely used [31]. Point-sensitivity analysis measures
the responsiveness of the outcomes to the input value at a
fixed point, and range-sensitivity analysis evaluates the
difference in outcome when an input value is varied
within its range. Brennan et al. [6] also explained the
point- and range-sensitivity analyses from another per-
spective. Such routinely used methods have been well
established. However, these approaches have inherent
disadvantages. Even though the key variables are readily
identified through the analysis, we are not certain how
confidently we could say that the key variables change
the cost-effectiveness frontier. As the estimates for the
parameters are obtained from the samples, the random
deviations of the estimates of results (e.g., ICERs) could
also be projected during modelling. More recently, an
approach has been used that employs the simulation to
yield the distribution of estimates to represent the variance
of results. This process is Stermed probabilistic sensitivity
analysis [32, 33].
Probabilistic sensitivity analysis
Two distinct methods are available in probabilistic sensi-
tivity analysis: first-order simulation and second-order
simulation. They differ in their approaches to performing
simulation. In first-order simulation, an individual patient
transits through the model, which is repeated a finite
number of times to yield the distribution of estimates. This
is very similar to first-order uncertainty. In a second-order
Decision-analytical modelling in health-care economic evaluations
simulation, different values of parameters are used to
project the distributions during the analytical process. The
concept of first-order simulation is separate from second-
order simulation, as the latter does not mean performing
the sensitivity analysis on the basis of the first-order
uncertainty. Rather, it mainly reflects the random feature of
the parameters, and thus the estimates (for example, ICER
statistics).
The first-order simulation can only represent the
inherent uncertainty of the model, rather than the
parameters’ randomness. Therefore, it would not be suit-
able to address the uncertainty raised by the parameters.
In addition, Briggs [20] furthers the issue on the inter-
pretation of results from the first-order simulation.
Although the average value of the estimates from different
numbers of simulations (e.g., 1,000 vs. 10,000 simula-
tions) is nearly the same, the variance of the estimates
may vary greatly based on the sampling size. This is a
practical problem that confronts health-care assessors and
modellers.
In second-order simulation, the parameters are assumed
to have specific probability distributions. During sensitivity
analysis, the values of parameters are drawn on the basis of
the distributions to calculate the estimates. The 95% con-
fidence interval of estimates is represented by the 2.5 and
97.5 percentile values. The standard error can also be
calculated by the following formula:
SE ¼ ðupper limit  lower limitÞ=2  1:96:
For different parameters, the distributions differ depending
on the nature of the data. However, for reasons of spaces,
this will not be explained in detail. Readers with interest
may refer to the publication by Brigg and Gray in this
regard [34].
Interpretation of results from the probabilistic sensi-
tivity analysis might be a problem, in particular, when the
estimates intervals cross more than one quadrant of the
cost-effectiveness plane. In this case, negative estimates
would be negative; however, this result can be obtained
from either a case in which the intervention of interest
dominates the control (positive effect difference vs. neg-
ative cost difference), or in which the intervention was
dominated (negative effect difference vs. positive cost
difference). In addition, it still remains difficult for health-
care policy makers to make a decision when ICERs are
presented. Although net-benefit analysis might help to
address the problem [35], it may be arbitrary to set a
willingness-to-pay ratio, an approach which has received
criticism [36]. A better vehicle to interpret the ICER
statistic and the policy making is the acceptability curve
[37, 38], which is used to inform users of the probability
that the intervention is affordably within a range of health
budgetary constraints [39].
Discussion
Decision-analytical modelling is an unavoidable and most
useful technique in health economic evaluations. This is
particularly true where expert cost-effectiveness assessors
must make fiscally expensive government reimbursement
decisions in the frequent absence of good head-to-head
clinical trial comparisons.
However, it must be recognised that the validity of
modelling is always facing challenges over a few major
practical difficulties, including structure variability and
multiplicity of data for probabilities, health utilities and
costs. The development of model structure is a compre-
hensive reflection of the current status of understanding of
the nature of disease, data quality and availability, users’
perspectives and requirement for exactness of results, of
which the first two are the essential elements. Theoreti-
cally, the modelling should be performed to simulate the
disease progression and relevant outcomes as closely
as possible. However, this would make modelling an
unmanageable practice due to excessive technical com-
plexity, causing a large number of technical problems and
costing unnecessary effort. More practically, the develop-
ment of the model structure needs to balance the
complexity of the nature of the disease and the sophisti-
catedness of the analysis. Modellers should reduce the
tediousness of the model, where possible, to accommodate
decision-making in an environment of best available evi-
dence. In order to achieve this, the model structure needs to
be validated by incorporating efforts from multiple sources.
Efforts will at least need to be made by modellers, users
and clinicians, so that the model can meet the clinical and
methodological criteria and have relevance for policy
making.
The availability and quality of data are the other
essential elements limiting the validity of modelling. It
would be extremely difficult to develop a robust model
without sufficient data. The unavailability or low validity
of data is most likely to cause significant deviations of
estimates from truth. Gathering sufficient, high-quality data
is technically difficult, but might be unavoidable. Dis-
crepancies also exist on how to deal with the unavailability
of data; one side favours the use of assumed/modelled data,
whereas the other prefers to disregard the condition. No
conclusions have been made to address which is more
appropriate.
The development of models is tricky work and requires
modellers equipped with sufficient experience. However,
even the most experienced modeller may not develop a
model that adequately addresses the cost-effectiveness of
health-care interventions of interest. As a more appropriate
vehicle to assess cost-effectiveness, clinical trials enable
more accurate estimates of health-care interventions to be
123

obtained. However, there is not sufficient empirical evi-
dence to indicate when and to what degree the modelling
and trial estimates differ. Therefore, more research is
needed in this regard to indicate when modelling can be
validly performed.
Another concern of modelling is the uncertainty due
to methodological, structural and parameter variability.
Although a few techniques can be used to reduce uncer-
tainty, the variability could inherently affect the robustness
of results. More importantly, the involvement of multi-
plicity of data further complicates this issue. Therefore,
efforts would be needed to reduce the variability of the
study, including careful planning and development of the
model and acquisition of health and cost data.
The development, valuation, presentation, analysis and
interpretation of the model are not easy tasks. Good practice
of modelling usually requires careful planning, conduct and
reporting. However, as modelling is a very sophisticated
technique that demands great skill, the validity of modelling
is usually affected by the experience of the modellers.
Therefore, sufficient training and an easy-to-follow guide
are important to improving the practice of modelling.
In summary, modelling is an important and unavoidable
aspect of economic evaluation of health-care programmes.
As one of the modelling techniques, the Markov model is
used to simulate the recursive and chronic progression of
diseases. Development and application of the Markov model
need to be balanced between the availability of data, the
understanding of the nature of the disease and the necessity
for the specificity of results. Efforts should be spent as much
as possible to attain the best available estimates (evidence)
for probabilities, utilities and costs. Presentation and valu-
ation of the model can differ depending on the method
chosen (cohort simulation vs. individual simulation). How-
ever, both produce very similar results and require that
sensitivity analyses be performed. Probabilistic sensitivity
analysis should always be preferred for including confidence
intervals of economic statistics and for producing cost-
effectiveness acceptability curves (for the sake of good
practice of policy making). The practice of modelling
requires not only input from modellers, but also needs efforts
of more parties, including at least users and clinicians, so as
to improve the validity and relevance.
Acknowledgements We are grateful to two anonymous reviewers
for the helpful comments. We also thank Mike Clarke from the
University of Oxford for kind provision of a place for Sun to conduct
work on this paper.
References
1. Sun, X., Orlewska, E., Li, Y.P., Cheng, G., Yang, H.: Methods for
economic evaluation of marketed medicines II: Conducting
123
X. Sun, T. Faunce
primary economic evaluation. Chin J Evid Based Med 6, 218–223
(2006)
2. Commonwealth Department of Health Housing and Community
Services: Guidelines for the Pharmaceutical Industry on Prepa-
ration of Submissions to the Pharmaceutical Benefits Advisory
Committee. Including Major Submissions Involving Economic
Analyses (draft). Canberra (2006)
3. Food and Drug Administration: Principles for the Review of
Pharmacoeconomic Promotion. Rockville: FDA (1995)
4. Brennan, A., Akehurst, R.: Modelling in health economic eval-
uation: What is its place? What is its value? Pharmacoeconomics
17(5), 445–459 (2000)
5. Buxton, M.J., Drummond, M.F., Van Hout, B.A., Prince, R.L.,
Sheldon, T.A., Szucs, T., Vray, M.: Modelling in economic
evaluation: an unavoidable fact of life. Health Econ 6, 217–227
(1997)
6. Brennan, A., Akehurst, R.: Modelling in health economic eval-
uation. What is its place? What is its value? Pharmacoeconomics
17, 445–459 (2000)
7. Sculpher, M., Drummond, M., Buxton, M.: The interactive use of
economic evaluation as part of the process of health technology
assessment. J Health Serv Res Policy 2, 26–30 (1997)
8. Chilcott, J.B., Brennan, A., Booth, A., Karnon, J., Tappenden, P.:
The role of modelling in prioritising and planning clinical trials.
Health Technol Assess 7, 1–125 (2003)
9. Karnon, J.: Alternative decision modelling techniques for the
evaluation of health care technologies: Markov processes versus
discrete event simulation. Health Econ 12, 837–848 (2003)
10. Orlewska, E.: The cost-effectiveness of alternative therapeutic
strategies for the management of chronic hepatitis B in Poland.
Value Health 5, 405–421 (2002)
11. Wright, J.L., Wessells, H., Nathens, A.B., Hollingworth, W.:
What is the most cost-effective treatment for 1 to 2-cm bulbar
urethral strictures: societal approach using decision analysis.
Urology 67, 889–893 (2006)
12. Chambers, M.G., Koch, P., Hutton, J.: Development of a deci-
sion-analytic model of stroke care in the United States and
Europe. Value Health 5, 82–97 (2002)
13. Spiegelhalter, D.J., Abrams, K.R., Myles, J.P.: Bayesian
Approaches to Clinical Trials and Health-Care Evaluation.
London: Wiley (2004)
14. Breese, J.S.: Construction of belief and decision networks.
Comput Intel 8, 624–647 (1992)
15. Powell, C.: The Delphi technique: myths and realities. J Adv
Nurs 41, 376–382 (2003)
16. van Steenkiste, B.C., Jacobs, J.E., Verheijen, N.M., Levelink,
J.H., Bottema B.J.A.M.: A Delphi technique as a method for
selecting the content of an electronic patient record for asthma.
Int J Med Inf 65, 7–16 (2002)
17. Salomon, J.A., Murray, C.J.L.: A multi-method approach to
measuring health-state valuations. Health Econ 13, 281–290
(2004)
18. Brazier, J., Deverill, M., Green, C., Harper, R., Booth, A.: A
review of the use of health status measures in economic evalu-
ation. Health Technol Assess 3 (9) (1999)
19. Briggs, A., Sculpher, M., Buxton, M.: Uncertainty in the eco-
nomic evaluation of health care technologies: the role of sensiti-
vity analysis. Health Econ 3, 95–104 (1994)
20. Briggs, H.A.: Handling uncertainty in cost-effectiveness models.
Pharmacoeconomics 17, 479–500 (2000)
21. Nuijten, M.J.C.: Incorporation of statistical uncertainty in health
economic modelling studies using second-order Monte Carlo
simulations. Pharmacoeconomics 22, 759–769 (2004)
22. Philips, Z., Ginnelly, L., Sculpher, M., Claxton, K., Golder, S.,
Riemsma, R.: Review of guidelines for good practice in decision-
Decision-analytical modelling in health-care economic evaluations
analytic modelling in health technology assessment. Health
Technol Assess 8(36) (2004)
23. Sonnenberg, F.A., Roberts, M.S., Tsevat, J., Wong, J.B., Barry,
M., Ken, D.: Toward a peer review process for medical decision
analysis models. Med Care 32, JS52–JS64 (1994)
24. Briggs, A., Sculpher, M.: An introduction to Markov modelling
for economic evaluation. Pharmacoeconomics. 13, 397–409
(1998)
25. Shamir, R., Hernell, O., Leshno, M.: Cost-effectiveness analysis
of screening for celiac disease in the adult population. Med Decis
Making 26, 282–293 (2006)
26. CDC Diabetes Cost-effectiveness Group: Cost-effectiveness of
intensive glycemic control, intensified hypertension control, and
serum cholesterol level reduction for type 2 diabetes. JAMA 87,
2542–2551 (2002)
27. Canadian Coordinating Office for Health Technology Assess-
ment: Guidelines for economic evaluation of pharmaceuticals:
Canada, 3rd edn. Ottawa: Canadian Agency for Drugs and
Technologies in Health (2006)
28. Weistein, M.C., O’Brien, B., Hornberger, J., Jackson, J.,
Johannesson, M., McCabe, C.: Principles of good practice for
decision analytic modeling in health-care evaluation: Report
of the ISPOR task force on good research practices-modeling
studies. Value Health 6, 9–17 (2003)
29. Manca, A., Rice, N., Sculpher, M.J., Briggs, A.H.: Assessing
generalisability by location in trial-based cost-effectiveness
analysis: the use of multilevel models. Health Econ 14, 471–485
(2005)
30. Pharoah, P.D., Hollingworth, W.: Cost effectiveness of lowering
cholesterol concentration with statins in patients with and without
View publication stats
pre-existing coronary heart disease: life table method applied to
health authority population. BMJ 312, 1443–1448 (1996)
31. Nuijten, M.J.C., Hardens, M.: Measuring sensitivity in pharma-
coeconomic studies: an integration of point-sensitivity and range
sensitivity. Pharmacoeconomics 12, 555–564 (1997)
32. Heitjan, D.F., Li, H.: Bayesian estimation of cost-effectiveness: an
importance-sampling approach. Health Econ 13, 191–198 (2004)
33. Vanness, D.J., Kim, W.R.: Bayesian estimation, simulation and
uncertainty analysis: the cost-effectiveness of ganciclovir pro-
phylaxis in liver transplantation. Health Econ 11, 551–566 (2002)
34. Briggs, A.H., Gray, A.M.: Handling uncertainty when performing
economic evaluation of healthcare interventions. Health Technol
Assess 3(2) (1999)
35. Stinnett, A.A., Mullahy, J.: Net health benefits: a new framework
for the analysis of uncertainty in cost effectiveness analysis. Med
Decis Making 18(Suppl 2), S65–S80 (1998)
36. Gafni, A., Birch, S.: Guidelines for the adoption of new tech-
nologies: a prescription for uncontrolled growth in expenditures
and how to avoid the problem. Can Med Assoc J 148, 913–917
(1993)
37. Fenwick, E., Marshall, D.A., Levy, A.R., Nichol, G.: Using and
interpreting cost-effectiveness acceptability curves: An example
using data from a trial of management strategies for atrial
fibrillation. BMC Health Serv Res 6, 52 (2006)
38. Sendi, P.P., Briggs, A.H.: Affordability and cost-effectiveness:
Decision-making on the cost-effectiveness plane. Health Econ 10,
675–680 (2001)
39. Bala, M.V., Mauskopf, J.A.: Optimal assignment of treatments to
health states using a Markov decision model: an introduction to
basic concepts. Pharmacoeconomics 24, 345–354 (2006)
123