Bayesian Decision Making in Clinical Trials

Swarnendu Chatterjee

gsk.com
Introduction

Clinical trials are essential for evaluating the safety and efficacy of
new medical treatments, drugs, or interventions. They provide
critical evidence that guides medical practice and regulatory
decisions.

Traditional Frequentist Approach

• The frequentist statistical framework has been widely used in clinical trials.
• Common methods include p-values, confidence intervals, and hypothesis testing.
• However, this approach has limitations, such as fixed sample sizes and lack of
flexibility.
The Bayesian Paradigm
A Shift in Perspective

Bayesian statistics is a statistical approach that

incorporates prior knowledge or beliefs into the analysis of
data.

Unlike frequentist statistics, which focuses on population

parameters, Bayesian statistics estimates probability
distributions for these parameters.

This shift allows for a more nuanced understanding of

uncertainty and a more informative interpretation of
results

3 28 June 2024
Bayes’ Theorem

4 28 June 2024
Illustration of Bayes’ Theorem

Bayes’ theorem (also referred to as Bayes’ rule) is a mathematical

relationship between the prior probability and the posterior probability
conditional on data. In general terms, Bayes’ theorem is expressed as
follows

𝑃 ( 𝐵| 𝐴 ) 𝑃 ( 𝐴 )
𝑃 ( 𝐴| 𝐵 )=
𝑃 (𝐵)
• where and are events of interest, and and are probabilities of event and , respectively.
• denotes the probability of event happening on the condition that event has already
happened.
• Similarly, denotes the probability that the event happens conditional on information that
event has already happened.
Example

• OraQuick In-Home HIV Test, the only FDA-approved HIV test for
self-testing at home
• Assume you perform the test and get a positive result.
• What you really want to know is “what is your probability of truly
having HIV if you test positive?”
• Using the mathematical expression above, is the event of testing
positive, and is the event of having the disease.
• is the probability of testing positive, and is the probability of having
the disease (i.e., HIV).
• is the same as sensitivity , i.e., the probability that someone will test
positive given they have the disease
Example-Continued

• The OraQuick label notes that in a prospective clinical study, this test
was observed to have 92% sensitivity and 99.98% specificity.
• This means that one false negative result would be expected out of
every 12 test results in truly HIV infected individuals, and one false
positive result would be expected out of every 5,000 test results in
truly uninfected individuals.
• One may be tempted to conclude that if you test positive (event
occurred), you have a 92% probability of having HIV (event ).
• This logic is WRONG, though unfortunately not uncommon.

This Photo by Unknown Author is licensed under CC BY-NC

Example—Continued

• The probability of testing positive if you have a disease (,

or 92% in this case) is not the same as the probability of
having a disease if you test positive ().
• The test’s sensitivity and specificity DO NOT give us an
answer to the key question: what is the probability that an
individual indeed has HIV if their test is positive ()?
• This is known as the positive predictive value (PPV) of
the test

8 28 June 2024
Example-continued

• The answer to this question depends on another very important

parameter: the prevalence of the disease in the population.
• To figure out the probability of having the disease if you test positive,
we need to know the overall prevalence of HIV in the population (),
or prior information, using Bayesian terminology.
• We can arrive at the answer to this question using the HIV test
example.
• The PPV of OraQuick is about 65%,
• which means that even after a positive test result, one is not sure
whether the patient really has HIV, and
• one would want to obtain an additional confirmatory laboratory-
based test.
Introduction to the Components of a Bayesian
Approach

10 28 June 2024
Priors
The prior distribution is a
key part of Bayesian
inference and represents
the information about an
uncertain parameter before
the data are collected.
Priors can have different
degrees of informativeness,
Conceptually, the prior
ranging from a non-
specifies plausible values
informative prior (if we
and the uncertainty of these
have no prior data) to a
parameters.
weakly informative prior to
an informative prior.
Methods to choose the Data-driven methods (e.g.,
The prior is constructed
prior range from expert using data from completed
before the start of a clinical
opinion-based to data- clinical trials) are usually
trial.
driven methods. preferred.
Likelihood

It describes the data- The likelihood is an

The likelihood is the
generation process as a expression involving both
probabilistic model for the
function of the parameters of the observed data and the
data.
interest. parameters of interest.
Posterior

When we obtain data from our The posterior distribution is the

trial, Bayes’ theorem gives us a probability distribution of the It describes the distribution of
mathematical way to combine the unobserved parameters of parameters considering the newly
prior and the likelihood to get the ultimate interest, given the obtained data
posterior distribution. observed data.
Prior-Likelihood-Posterior
Model components within Frequentist and Bayesian frameworks

Frequentist Bayesian

The uncertainty of parameters are managed with probability

The parameters are fixed but unknown and we estimate directly, i.e., by considering these parameters as random with
them using the data collected their own probability distributions

Uncertainty is addressed by taking a sample of In Bayesian models, before collecting the data, a prior
distribution is assumed for the parameters which would
patients and evaluating the sampling distribution. characterize the uncertainty of parameters

The estimates of uncertainty around our observed Once the data are obtained, the prior distribution is updated using
estimates are quantified as if we were repeating the Bayes’ theorem to get the posterior distribution, which will be
sampling process of patients over and over used to calculate the updated probability distributions and
credible intervals of interest
Bayesian Methods: Facilitating Interpretations

In a clinical study evaluating a new treatment in a controlled trial (noted for Treatment and for Control), the prior could
be the chance that the new treatment outperforms the control with respect to a particular endpoint.

Denote the Probability that Treatment is better than Control as .

Assuming equipoise before starting the trial, the chances of either treatment being better is or and

After the trial is over, one may use the data from the trial and the prior to determine if one is convinced that Treatment is
better than Control overall (e.g., exceeds ).

If results from the trial lead to a conclusion that the treatments are not all that different, it would be unlikely to advance
the new treatment for consideration.
Interpretations

Bayesian thinking facilitates a common-sense interpretation of statistical conclusions.

Bayesian methods can help answer our research questions because Bayesian methods allow one to estimate the probability that the
hypothesis is true considering the data.

In the Bayesian framework, we estimate the probability of the effect based on the observed data.

Consequently credible interval, by definition, contains the true parameter with probability.

By comparison, a frequentist’s confidence interval means that the true response would be contained within of the confidence intervals
produced by repeating the same experiments.

Those learning statistics often confuse the definition of a frequentist confidence interval with that of a Bayesian credible interval.
Key concepts within frequentist and Bayesian frameworks

Frequentist Bayesian
Estimating probability of the observed data given (assuming) that the null
hypothesis is true
Estimating the probability that the hypothesis is true given the data
P value=probability (data can be more extreme than what is observed | null
hypothesis)
Posterior probability of the hypothesis of interest=probability (hypothesis |
Provides probability that observed effect can be by chance if null is true data)

Provides probability of the treatment effect based on the observed data and
Parameter of interest is fixed but unknown and data are random. prior

Parameters are random, the posterior distribution given the data describes
Parameter is estimated by the data along with the measure of error in estimation
(standard error)
the distribution of the parameter once the data is observed

95% Confidence interval: The true effect size would be contained within 95% of 95% Credible Interval: 95% Credible Interval contains the true parameter
the Confidence Intervals produced over repeated Experimentations with 95% probability given the observed data
Unlocking Advantages
Why Use Bayesian Statistics for Medical Devices?

Flexibility in Trial
Incorporation of Prior Improved Efficiency: More Informative
Design: Allows for
Knowledge: Integrates Can potentially require Results: Provides
adaptive trial designs
existing scientific smaller sample sizes probability distributions
that adjust parameters
understanding with new compared to frequentist for parameters, offering
based on accumulating
data, leading to richer methods for the same a clearer understanding
data, leading to more
insights. level of certainty. of uncertainty.
efficient trials.

19 28 June 2024
A Real-World Example
Bayesian Analysis in a Pacemaker Trial

Scenario: A clinical trial is evaluating a new pacemaker design for its ability to
reduce hospital readmissions due to heart failure.

Prior Knowledge: Previous studies suggest that existing pacemakers typically

achieve a 20% reduction in readmissions, with some uncertainty around this
estimate.

Bayesian Approach:

• The prior information about the reduction rate (20%) is incorporated into a probability distribution (e.g., a
normal distribution).
• Data on hospital readmissions from the current trial is then factored in through the likelihood function.
• The resulting posterior distribution provides a more refined estimate of the new pacemaker's effectiveness,
considering both prior knowledge and new data.

20 28 June 2024
Bayesian Interim Decision Making

21 28 June 2024
What is Interim Decision-Making?

• Accumulating Data
• Collect data from enrolled patients.
• Update prior distributions with observed data.
• Posterior Estimates
• Obtain posterior distributions for treatment effects, safety
endpoints, etc.
• Decision criteria based on credible intervals or posterior
probabilities.
• Adaptive Changes
• Modify trial parameters:
• Sample size adjustments.
• Treatment arm allocation.
• Stopping rules (early efficacy, futility, safety).

22 28 June 2024
How do Bayesian Models Help?

• Bayesian models provide a principled way to update our beliefs

about the unknown parameters as new data arrives.
• This is achieved through Bayes' theorem, which allows us to
calculate the posterior distribution of the parameters given the
observed data.
• The posterior distribution reflects our updated knowledge about
the parameters, considering both the prior information we had
before the experiment and the new data that has been collected.

23 28 June 2024
Benefits of using Bayesian Models for Interim Decision-Making

• Efficiency
• Smaller sample sizes due to adaptive designs.
• Faster decision-making.
• Ethical Considerations
• Stop trials early if evidence supports efficacy or futility.
• Minimize patient exposure to ineffective treatments.

24 28 June 2024
Bayesian Decision-Making Process

• Define pre-specified decision boundaries

• Use prior information to create a prior distribution
• Update the prior with interim data to form the posterior
distribution
• Compare the posterior distribution to the decision boundaries
• Make decisions based on pre-defined rules (e.g., stop for
efficacy, futility, continue)

25 28 June 2024
Challenges and Considerations

• Complexity
• Bayesian models require expertise.
• Communication with stakeholders.
• Transparency
• Clear documentation of interim decisions.
• Regulatory acceptance.

26 28 June 2024
 Bayesian Dose Finding

28 June 2024
27
Traditional Method

• Limitations of 3+3 Design

• The 3+3 design (commonly used in oncology) is simple but has
drawbacks:
• Fixed dose escalation rules.
• Insufficient use of available data.
• Difficulty in selecting the true Maximum Tolerated Dose (MTD).
• Challenges in MTD Selection
• Identifying the MTD with precision is crucial.
• Traditional designs may lead to suboptimal dose
recommendations.
• Bayesian methods address these limitations.
Bayesian Logistic Regression Model (BLRM)

• Combining clinical expertise and statistical modeling.

• Incorporating historical data and trial-specific information.
• Estimating posterior DLT (Dose-Limiting Toxicity) rates.
• Dose recommendations based on targeted DLT rates.

29 28 June 2024
Continual Reassessment Method (CRM)

• Iterative Bayesian model for

dose escalation.
• Adaptive dose selection
based on observed toxicity.
• Balancing safety and
efficacy.
• Widely used in early-phase
trials.

30 28 June 2024
Advantages
Utilizing All Available Data

• Bayesian methods allow joint modeling of efficacy and

toxicity.
• Incorporating both endpoints for dose recommendations.
• Improves precision in MTD estimation.

31 28 June 2024
Advantages
Modeling Time to Toxicity and Efficacy

• Time-to-event models
capture dynamic patient
responses.
• Differentiating endpoints
enhances decision-making.
• Examples: Time to tumor
response vs. time to adverse
events.

32 28 June 2024
 Bayesian Computation
MCMC

28 June 2024
33
Markov Chain

• It’s a mathematical process that transitions from one

state to another within a finite number of possible
states.
• It’s like Snakes and Ladders, where moves are
determined entirely by dice, can be modeled as
Markov chains.
• In these games, the next state (or position on the
board) depends only on the current state and the
outcome of the dice roll, not on the sequence of states
that preceded it.

28 June 2024
Monte Carlo

• It’s like guessing the

number of jellybeans
in a jar by making a
lot of random guesses
and averaging them
out.

28 June 2024
Markov Chain Monte Carlo

• Imagine you’re in a room filled with hills and valleys (these represent
different solutions to a problem).
• It’s pitch dark and you’re trying to find the highest hill (the best solution).
• You start walking (this is the Markov Chain part) but you can only feel the
slope of the ground beneath your feet (you only know your current state).
• You make random steps (this is the Monte Carlo part),
• if uphill, you go,
• if downhill, you decide based on how steep it is.
• Over time, you’ll end up visiting the high hills (good solutions) more
often and hopefully, find the highest one.
• This method helps in finding good solutions to complex problems where
direct calculation is difficult.

28 June 2024
Uses of MCMC

• High-dimensional problems: If your problem involves many variables,

MCMC can be a good choice. It’s often used in Bayesian statistics where you
might have many parameters to estimate.
• Complex probability distributions: MCMC is useful when you’re dealing
with complex probability distributions that you can’t easily sample directly
from.

28 June 2024
 Regulatory Guidance

28 June 2024
38
FDA and Bayesian Statistics
A Framework for Responsible Use

The FDA recognizes the potential benefits of

Bayesian statistics in medical device trials.

Key aspects of the guidance include:

• Clearly defined prior distributions and justification for their
selection.
• Transparent communication of the impact of prior distributions
on the results.
• Sensitivity analyses to assess the robustness of findings to
variations in prior assumptions.

39 28 June 2024
FDA Guidance
Ensuring Robustness and Transparency

Prior Distributions:
• Selection should be based on scientific rationale and relevant data sources.
• Impact of different prior choices should be explored through sensitivity
analyses.

Transparency:
• Clearly document the chosen prior distributions and their justification.
• Communicate how prior information influences the final results.

Sensitivity Analyses:
• Evaluate how the posterior distribution and conclusions change under
alternative prior assumptions.
• Demonstrates the robustness of the findings to variations in prior beliefs.

40 28 June 2024
 gsk.com

28 June 2024
41