MAKING RECOMMENDATIONS ABOUT DIAGNOSTIC

TESTS AND STRATEGIES: WHAT DO EXPERTS SAY?

Reem A. Mustafa

Wojtek Wiercioch, Jan L. Brozek, Matthew Ventresca, Nancy Lloyd,

Holger J. Schnemann. Du-Diagnosis expert group

Du-Diagnosis expert group

Stefan Sauerland
Toni Tan
Hanan Bell
Robin Habour
Roger Chou
Rosanne Leipzig
Michael Bettman
Martin Reed
Davina Gershi
Marjukka Mkel
Maija Saijonkari
Richard Mendelson

Michelle Mujoomdar
Amir Qaseem
Massimo Gion
George Browman
Paul Glasziou
Karen Steingart
Andrea Rita Horvath
Andrew Don-Wauchope
Glyn Elwyn
Murray Krahn
Denis Remedios
Ina Kopp
Susan Norris

Disclosure of Interests (last 3 years)

< Reem Mustafa>
I certify that, to the best of my knowledge, no aspect of my current
personal or professional situation might reasonably be expected to
affect significantly my views on the subject on which I am
presenting, other than the following*:
Grant: German Insurance Fund

What do we know?
Current practices in developing guidelines
about the use of diagnostic tests and
strategies (DTS) are out of step with the
conceptual discussion among experts.

What do we know?
Current practices in developing guidelines
about the use of diagnostic tests and
strategies (DTS) are out of step with the
conceptual discussion among experts.

How
Diagnostic
are DTSTest
results
Accuracy
presented?
(DTA)

What do we know?
Current practices in developing guidelines
about the use of diagnostic tests and
strategies (DTS) are out of step with the
conceptual discussion among experts.
Recommendation, Level 1
Multidetector CT (MDCT) scans should be used to screen for and diagnose
thoracolumbar spine fractures.
Evidence:
There is significant Class II and Class III data that demonstrate the superior
sensitivity of MDCT scan in comparison with plain films for the diagnosis of
thoracolumbar spine fractures

Our goal
Identify the essential factors to consider
when making recommendations about
DTS.
Find out how are these factors
considered.

What did we do?

We conducted semi-structured 90 minutes indepth interviews with experts in assessing
evidence and producing guidelines about DTS.
We invited all experts to a Conference call to
discuss and confirm our findings.

23 Experts!

WHO
USPSTF
SIGN
Canada, USA, UK, Germany,
NICE
Finland, Italy, Scotland, Australia
IQWiG
and others (LMIC)
ACP
Canadian partnership against cancer
Canadian Association of Medical Biochemist
Canadian Association for Radiology
clinical research, community
European Society of Radiology
health, public health, GP, family
Global summit on radiological quality
and safety
medicine,
Geriatrics, oncology,
National Academy of Clinical Biochemistry
(NACP) pathology,
pharmacology,
Royal Collage of Radiology
radiology, laboratory medicine,
biochemistry, nursing, decision
Australian College of Medicine
support tools, HTA and health
Diagnostic Imaging Pathways
National Agency for Regional Health Serviceseconomics
in Italy
National Health and Medical Research Council of Australia (NHMRC)
others

Experts say!
Questions in Diagnostic CPGs:
Should we perform a test?
Which test to use? Radiology vs biomarkers.
Which test should we use next?
Other questions: when is the best time to
test? What is the best cut-off? ..etc.

Experts say!

Its not just the

test, its what
happens after the
test

Although DTA was the factor most commonly

considered by organizations when developing
recommendations, experts agreed that DTA is rarely
sufficient and relying on DTA may be misleading.

Experts say!
We should consider:
Complications and harms from performing the test
Rates of inconclusive results
Additional benefits of the test that may affect the
management pathway
Resource implications
Ethical considerations
Legal considerations
Feasibility and organizational considerations
Patient and societies values and preferences.
The link between the test results and patient important
outcomes including treatment efficacy, side effects and
natural progression history.

When is DTA enough?

only when all
otherthis
things
Difficult to think of a situation where
are equal
would be true

win-win situation
When the goal is to establish a diagnosis for a
condition or role out a diagnosis
When there are obvious inferences about
downstream
consequences.
U/S if the mother
Genetic testing, wherePrenatal
R/O
is not planning to change
or diagnosis help in future
planning and does notanything based on the
results
affect management

Experts say!
Creating flowcharts or a decision tree to
illustrate the use of a diagnostic test, with
treatment options, if any, and outcomes of
those treatment options following a specific test
result helps improve transparency and facilitate
the process of developing recommendations.

Asymptomatic
women

HPV

Look at the test in

practice, not in
theory

Test +
(TP & FP)

Cyto

Test (TN & FN)

Test +
(TP & FP)

Cryo
eligible?

No

Treat with
LEEP

Outcomes*

Colposcopy
+/- biopsy

Test +
(TP)

Yes

Treat with
CKC

Outcomes*

Treat with
Cryo

Outcomes*

Eligible for
Cryo

Te
(T

Not eligib
for cryo

Treat with
Cryo

Treat with
CKC

Outcomes*

Outcomes*

Experts say!
There is general agreement that guidelines,
HTA and decision support tools, should be
evidence-based can inform each other.
Performing formal decision analysis and
mathematical modeling is limited by time and
resources required as well as frequently
lacking high quality evidence that informs
these models which may lead to inaccurate
conclusions.

Experts say!
One of the main challenges faced by guideline
panels making decisions about diagnostic tests
is insufficient evidence
The guideline
development process
is question-driven, not
evidence-driven

Experts say!
Formal decision modeling can be a useful
framework for organizing the clinical, cost, and
preference data relevant to the use of
diagnostic tests.
If evidence is insufficient should we bother?
Although decision modeling requires
It is
the only
wayforyoure
resources,
it is useful
integrating these
into decision-making,
identifying
goingfactors
to get
an answer that
evidence gaps, and high priority research
makes sense
areas

Experts say!
Even when costs are not formally considered,
they will be considered informally and not
systematically.
As costs vary from place to place, if economic
modelling is used, it should be reported in a
It is always considered. Its may
way that allows other jurisdictions to use the
be a factor in deciding to
model by applying
local data.
commission
a review of the
evidence and develop a
guideline about a test in the first
place.

Summary of what experts say!

Developing guidelines about the use of DTS requires
considering other factors beyond DTA
Implementing this demand is challenging
Difficult to think of a situation where DTA would be
sufficient
Creating flowcharts or a decision trees helps improve
transparency and facilitate the process of developing
recommendations
Performing formal decision analysis and mathematical
modeling is limited by time and resources required as
well as frequently lacking high quality evidence
Resources are always considered
Further development and testing of a framework that
can guide this process is needed

Thank you
Questions, Feedback, Suggestions?