Observational Outcomes or Use of Guideline Recommended Treatments
Invited Commentary
Observational Outcomes or Use of Guideline
Recommended Treatments?
Which Demonstrates Better Care?
Paul Heidenreich, MD, MS
As new models of care delivery are developed, particularly
when the model is designed to reduce costs, it is imperative
to examine any effects on the quality of care. An example is
Medicare Advantage, Medicare’s expanding managed care
model that pays a lump sum
Related article
per month for treating a Medi-
care patient. One study cal-
culated that Medicare pays 8% less for patients in Medicare
Advantage than for similar patients in fee-for-service (FFS; tra-
ditional) Medicare,1 although this was not the conclusion of
the Medicare Payment Advisory Commission.2 The quality of
care differences between Medicare Advantage and FFS Medi-
care are also unclear.
In this issue of JAMA Cardiology, Figueroa et al3 examine
the quality of cardiac care provided by Medicare Advantage and
compare it with FFS Medicare. They were able to do this using
clinical data from the Practice Innovation and Clinical Excel-
lence (PINNACLE) registry of the American College of Cardi-
ology National Cardiology Data Registries. The PINNACLE reg-
istry collects deidentified clinical data from cardiology practices
through extracting electronic medical records. The authors are
to be commended for using registry data as opposed to rely-
ing on claims data to evaluate quality of care. While claims data
can answer certain questions, the additional information from
vital signs, contraindications to medications, and laboratory
values provides a more complete measure of care and patient
severity of illness.
They found that among outpatients with coronary artery
disease, those who were enrolled in Medicare Advantage were
slightly younger but had more comorbidities than those in FFS
Medicare. Importantly, patients in Medicare Advantage were
more likely to receive recommended secondary prevention
treatments, including β-blockers and angiotensin-converting
enzyme inhibitors (or angiotensin receptor blockers) than those
in FFS Medicare. However, the authors noted that this im-
provement in the process of care did not translate into an im-
proved intermediate outcome of blood pressure control.
What should we conclude when treatment improves but
we do not see a significant improvement in outcomes? It is
tempting to let outcomes be the arbiter of quality consistent
with the classic Donabedian model of care in which structure
is the setting for processes of care that lead to clinical
outcomes.4 Groups, such as the Center for Medicare and Med-
icaid Services (CMS) and the National Quality Forum, have cho-
sen to emphasize, encourage, and financially reward out-
come measures over process measures. We are also told that
patients care more about outcomes over processes. This makes
sense when one is considering his or her own outcome vs his
jamacardiology.com
© 2019 American Medical Association. All rights reserved.
Downloaded from jamanetwork.com by Macquarie University user on 02/21/2019
Invited Commentary
or her own process of care. But are we misleading patients in
suggesting that the differences between risk-adjusted out-
comes (such as hospital 30-day mortality rates) will apply to
them? Furthermore, should patients and payers ignore the dif-
ferences in the process of care once outcome data are avail-
able when selecting clinicians or health systems?
One of the appealing features of an outcome measure is
that an outcome is influenced by all the relevant differences
in the quality of care, not just those that are measurable. How-
ever, outcomes, including observed to expected outcomes, are
also influenced by case mix, the way data are measured (eg,
overcoding or undercoding), and random variation. While one
can attempt to adjust for case mix, it is difficult to adjust for
the method of comorbidity measurement and impossible to
exclude random variation. Thus, much of the difference in out-
comes may be noise as opposed to signal. This signal-to-
noise ratio is only going to worsen as the quality of care con-
tinues to improve, making the differences in outcomes between
clinicians mostly noise.
So which is the better measure of quality, observed out-
comes or the processes of care that are shown to improve out-
comes in randomized clinical trials? A recent study of acute
myocardial infarction (MI) mortality has suggested that both
recommended processes of care and prior 30-day risk-
adjusted mortality independently predicted future mortality
with similar effect sizes.5 This report argued that CMS should
adopt a composite measure of prior mortality and the process
of care in judging quality.
A substantial problem in comparing processes of care and
outcomes is the power disparity. When processes and out-
comes are compared in the same study, it is rare to have the
study powered for the outcome measure. Thus, even if con-
founding is eliminated, there is often a process effect without
a corresponding outcome effect due to limited power. In
Figueroa et al,3 the differences in medication use were rather
small (1.8% absolute difference for β-blockers and 5.6% dif-
ference for angiotensin-converting enzyme inhibitors and an-
giotensin receptor blockers). It is unclear what change in mean
blood pressure would be expected with such small changes in
medication use. This difference is easier to calculate from ran-
domized clinical trials. In patients at intermediate risk in the
Clopidogrel to Understand Current Events Trial (CURE) trial,
the adherence to clopidogrel was 79% with an absolute 2.1%
reduction in major adverse cardiac events (MACE), which im-
plies a reduction of 0.03% in the MACE rate per 1% increase
in actual clopidogrel use.6 Thus, an intervention leading to a
3% increased use in appropriate patients would be expected
to result in only a 0.08% decrease in MACE. This effect size
(Reprinted) JAMA Cardiology Published online February 20, 2019 E1
 Invited Commentary Observational Outcomes or Use of Guideline Recommended Treatments

was recently found in a randomized clinical trial of adher-
ence improvement that was able to significantly improve
adherence to P2Y12 inhibitor therapy by 3.2% (83.8%-87.0%).
It was noted that the improved adherence in this trial of
11 000 participants did not translate to an improvement in
MACE. 7 However, such a study would require close to
250 000 patients to achieve the same power used in CURE to
detect the expected 0.08% decrease in MACE with improved
adherence.
Are patients even paying attention to published outcome
data? This seems unlikely, although it may be just lack of pa-
tient awareness. California hospitals that were found to have
low or high acute MI mortality saw no clear change in volume
for acute MI-related conditions or procedures after a public re-
lease of mortality data.8 In a study of data from New York State,
only 5% of patients were aware that the state tracked the risk-
adjusted outcomes of interventional cardiologists.9 How-
ever, once the program was described to them, patients were
significantly more likely than physicians to think that the
publication of physician-specific mortality for percutaneous
coronary intervention can provide an accurate measure of phy-
sician quality.
In summary, as tempting as it may be, we should not let the
lack of an observational outcome supersede a benefit in a pro-
cess of care that is shown to improve outcomes in randomized
clinical trials. The low-quality signal-to-noise ratio for obser-
vational outcome measures, combined with the huge sample
size that is needed to show an outcome difference given a small
process of care difference, makes a null outcome finding diffi-
cult to interpret. We will be more accurate and helpful to health
system users by combining process and outcome measures
when assessing and labeling quality of care.

ARTICLE INFORMATION docs/default-source/reports/mar18_medpac_ Effects of clopidogrel in addition to aspirin in

Author Affiliation: Veterans Affairs Palo Alto
Health Care System, Palo Alto, California.
Corresponding Author: Paul Heidenreich, MD, MS,
Veterans Affairs Palo Alto Health Care System, 111C,
3801 Miranda Ave, Palo Alto, CA 94303 (heiden@
stanford.edu).
Published Online: February 20, 2019.
doi:10.1001/jamacardio.2019.0050
Conflict of Interest Disclosures: None reported.
REFERENCES
1. Baker LC, Bundorf MK, Devlin AM, Kessler DP.
Medicare Advantage plans pay hospitals less than
traditional Medicare pays. Health Aff (Millwood).
2016;35(8):1444-1451. doi:10.1377/hlthaff.2015.1553
2. Medicare Payment Advisory Committee
(MedPAC). The Medicare Advantage program:
status report, in report to the Congress: Medicare
Payment, March 2018. http://www.medpac.gov/
entirereport_sec_rev_0518.pdf?sfvrsn=0. Accessed
January 9, 2018.
3. Figueroa JF, Blumenthal DM, Feyman Y, et al.
Differences in management of coronary artery
disease in patients with Medicare Advantage vs
traditional fee-for-service Medicare among
cardiology practices [published online February 20,
2019]. JAMA Cardiol. doi:10.1001/jamacardio.2019.
0007
4. Donabedian A. The quality of care: how can it be
assessed? JAMA. 1988;260(12):1743-1748. doi:10.
1001/jama.1988.03410120089033
5. Doyle JJ, Graves JA, Gruber J. Evaluating
measures of hospital quality. NBER working paper
no. 23166. https://www.nber.org/papers/w23166.
Accessed January 9, 2018.
6. Yusuf S, Zhao F, Mehta SR, Chrolavicius S,
Tognoni G, Fox KK; Clopidogrel in Unstable Angina
to Prevent Recurrent Events Trial Investigators.
patients with acute coronary syndromes without
ST-segment elevation. N Engl J Med. 2001;345(7):
494-502. doi:10.1056/NEJMoa010746
7. Wang TY, Kaltenbach LA, Cannon CP, et al. Effect
of medication co-payment vouchers on P2Y12
inhibitor use and major adverse cardiovascular
events among patients with myocardial infarction:
the ARTEMIS randomized clinical trial. JAMA. 2019;
321(1):44-55. doi:10.1001/jama.2018.19791
8. Romano PS, Zhou H. Do well-publicized
risk-adjusted outcomes reports affect hospital
volume? Med Care. 2004;42(4):367-377. doi:10.
1097/01.mlr.0000118872.33251.11
9. Fernandez G, Narins CR, Bruckel J, Ayers B, Ling
FS. Patient and physician perspectives on public
reporting of mortality ratings for percutaneous
coronary intervention in New York state. Circ
Cardiovasc Qual Outcomes. 2017;10(9):e003511.
doi:10.1161/CIRCOUTCOMES.116.003511

E2 JAMA Cardiology Published online February 20, 2019 (Reprinted) jamacardiology.com

© 2019 American Medical Association. All rights reserved.

Downloaded from jamanetwork.com by Macquarie University user on 02/21/2019