Chapter 8: Vehicles for Economic Evaluation

- Pages 274- 282 only

Explanatory versus Pragmatic RCT’s
- Studies that have been designed primarily for clinical purposes (ex, to support the
licencing application for new pharmaceuticals) are often described as explanatory in
nature
o Seek to estimate the efficacy of interventions in ideal or experimental settings
- Strength of efficacy studies is that they have potentially very high level of internal
validity
o Also have a clear definition of the interventions being compared and the relevant
patient population
 Can be advantageous when different studies are being compared using
meta-analysis
- An alternative option is to design trails specifically as a vehicle for economic evaluation
- Alternative to explanatory orientation (can the intervention work?) is the pragmatic
orientation (does the intervention work?)
- General aim of a RCT undertaken to support economic evaluation is to be more
pragmatic
o Intention is to offer some compromise between the goals on internal validity and
generalizability
- Pragmatic trail retains the concept of subjects being randomly allocated to treatments but
has fewer restrictions on how patients are recruited and followed up after randomization
o Seeking to increase generalizability
o Aims to evaluate the effectiveness or cost-effectiveness of an intervention under
something closer to the real world conditions that would prevail once the
intervention is in routine use
- Pragmatic-explanatory continuum exists
o PRECIS was developed to assess and display the position of any given trial within
this continuum
o Help assess the degree to which designs of a trail is aligned with the trails stated
purpose or either supporting decision makers or providing explanations
- The 10 domains of the PRECIS tools relate to:
1. The criteria used to establish the eligibility of trial participants
2. How prescriptively the protocol defines the use of the experimental
intervention
3. The level of clinical expertise in applying and monitoring the new
intervention
4. How prescriptively the protocol defines the use of the control intervention
5. The level of clinical expertise in applying and monitoring the control
intervention
6. How intensively trail participants are followed up on
7. The type of primary outcome used
8. How much attention is given to measuring compliance with therapies and
whether attempts are made to improve compliance
9. How much attention is given to measuring practitioners adherence to the
trial’s protocol and whether attempts are made to improve adherence
 10. The analysis of the primary outcome
The Domains of the PRECIS tool
Participants
- Participant eligibility criteria
o Pragmatic Trial:
 All participants who have the condition of interest are enrolled, regardless
of their anticipated risk, responsiveness, comorbidities, or past compliance
o Explanatory trial
 Stepwise selection criteria are applied that restrict study individuals to
those previously shown to be at highest risk of unfavourable outcome,
further restrict to those who are thought likely to be highly responsive to
the experimental intervention, and include just those high-risk, highly
responsive study individuals who demonstrate high compliance with
pretrial appointment keeping and mock intervention
Interventions and expertise
- Experimental intervention- flexibility
o Pragmatic trial:
 Instructions on how to apply the experimental intervention are highly
flexible, offering practitioners considerable leeway in deciding how to
formulate and apply it
o Explanatory trial:
 Inflexible experimental intervention, with strict instructions for every
element
- Experimental intervention- practitioner expertise
o Pragmatic trial:
 The experimental intervention typically is applied by the full range of
practitioners, and in the full range of clinical settings, regardless of their
expertise, with only ordinary attention to dose setting and side effect
o Explanatory trial:
 Experimental intervention is applied only by seasoned practitioners
previously documented to have applied that intervention with high rates of
success and low rates of complications
 In practice settings where the care delivery system and providers
are highly experienced in managing the types of patients enrolled
in the trial
 Intervention is closely monitored so that its dose can be optimized
and its side effects treated, co-interventions against other disorders
often are applied
- Comparison intervention- flexibility
o Pragmatic trial:
 Usual practice or the best alternative management strategy available,
offering practitioners considerable leeway in deciding how to apply it
o Explanatory trail:
 Restricted flexibility of the comparison intervention, may use a placebo
rather than the best alternative management strategy as the comparator
- Comparison intervention- practitioner expertise
 o Pragmatic trial:
 The comparison intervention typically is applied by a full range of
practitioners and in the full range of clinical settings, regardless of
expertise, with only ordinary attention to their training, experience and
performance
o Explanatory trial:
 Practitioner expertise in applying the comparison intervention is
standardized to maximize the chances of detecting whatever comparative
benefits the experiments intervention might have
Follow-up and Outcomes
- Follow up- intensity
o Pragmatic trial:
 No formal follow-up visits of study individual
 Administrative databases are searched for the detection of outcomes
o Explanatory trails:
 Study individuals are followed with many more frequent visits and more
extensive data collection than would occur in routine practice regardless of
whether patient experiences any event
- Primary trial outcome
o Pragmatic trial:
 Primary outcome is an objectively measured clinically meaningful
outcome to the study participants
 The outcome does not rely on central adjudication and is one that can be
assessed under usual conditions
 Special tests or training are not required
o Explanatory trials:
 The outcome is known to be direct and immediate consequences of the
intervention
 Outcome is often clinically meaningful but may sometimes be a surrogate
marker of another downstream outcome of interest
 May also require specialized training or testing not normally used to
determine outcome status or central adjudication
Compliance and Adherence
- Patient compliance with prescribed intervention
o Pragmatic trial:
 There is unobtrusive (or no) measurement of participant compliance
 No special strategies to maintain or improve compliance are used
o Explanatory trial:
 Study participant’s compliance with the intervention is monitored closely
and may be a prerequisite for study entry
 Both prophylactic strategies (maintain) and rescue strategies (to regain)
high compliance are used
- Practitioner adherence to study protocol
o Pragmatic trial:
 There is unobtrusive (or no) measurement of practitioner adherence
 No special strategies to maintain or improve adherence are used
 o Explanatory trials:
 There is close monitoring of how well the participating clinicians and
centres are adhering to even the minute details in the trial protocol and
manual of procedures
Analysis
- Analysis of primary outcome
o Pragmatic trial:
 The analysis includes all patients regardless of compliance, eligibility, and
others (intention-to0treat analysis)
 The analysis attempts to see if the treatment works under the usual
conditions with all the noise inherent therein
o Explanatory trial:
 An intention-to-treat analysis is usually performed
 This may be supplemented by a per-protocol analysis or an analysis
restricted to compliers or other subgroups in order to estimate maximum
achievable treatment effect
 Analyses are conducted that attempt to answer the narrowest, mechanist,
question
 Whether biological, educational or organizational
Summary:
- Pragmatic trials have a relatively short follow-up
o Also have difficulty of comparing more than 2 or 3 options and the fact that other
trial evidence may exist and need to be considered in an economic evaluation
- Pragmatic trials may not overcome the challenges of achieving generalizability
- If a pragmatic trial is used to inform decisions relating to a somewhat different
population the study could be more difficult to interpret than an explanatory trial
Observational Studies
- In observational studies patients receive treatments on the basis of routine decisions
resulting from the interaction of patients, their clinicians, and the health system more
generally
- Contrasts with the randomized allocation of patients to treatments
- These studies are considered to be extremely pragmatic on the spectrum
- They generally impose little or no protocol restrictions on routine practice
- Usually called real world studies
- Disadvantage of such studies are their lack of randomization
- Groups are unlikely to be similar in terms of patients observed and unobserved
characteristics because they may be related to the mechanism of selection
- Characteristics are also prognostic (they predict patient outcomes)
o Any estimate of treatment effectiveness in such a study will be confounded as a
result of this imbalance
- Generating a reliable estimate of treatment effects requires that there is enough variation
between patients with similar characteristics in the interventions they receive
- They can provide valuable evidence of the cost effectiveness of interventions
o Value of these studies is high when RCT are impractical
 Policy initiatives rarely introduced in a way that experimental designs are
feasible
 o Use of routine observational data can provide a vehicle for evaluation
- Selection process in observational studies provide an analytical challenge
o The extent of which depends on the design of the study
- Studies can be prospective or retrospective
o Prospective is also known as in de novo
 Advantages
 Full range of patients’ baseline characteristics can be identified and
measured in the study
o Socio-demographic and clinical details at the point that
interventions are selected
 May also be a scope to establish and collect data relating to
o Factors that may explain why patients are given particular
types of treatments
 Planning and collecting these types of data maximizes the
opportunity to use statistical techniques to adjust for how patients
are selected for specific treatments
 Possibility of including a range of interventions that a cost-
effective analysis would seek to compare
 Many non-randomized prospective studies take the form of treatment
registers
 A collection of baseline and follow-up data on patients receiving
particular treatment
o Sometimes they relate to one treatment only and these offer
little basis to estimate relative cost effectiveness
 They lack information on the characteristics and outcomes of patients on
older therapies or without active treatment
 Potentially limiting their value as a vehicle for cost-effectiveness
analysis and for estimating treatment effects of the full range of
interventions
o Retrospective observational studies may lack some of the baseline characteristics
of patients and treatment decisions and may not include the full range of
comparators
 More difficult to model the selection process into the study then with a
prospective design
 They have the potential advantage of being relatively low cost sources of
data
 Much interest in recent years in the use for research purposes of
information collected routinely by health systems
 Data is collected for years in administrative purposes