The Accuracy of Substituted Judgments in Patients with
Terminal Diagnoses
Daniel P. Sulmasy, OFM, MD, PhD; Peter B. Terry, MD, MA; Carol S. Weisman, PhD;
Deborah J. Miller, RN, PhD; Rebecca Y. Stallings, MHS; Margaret A. Vettese, RN, PhD;
and Karen B. Haller, RN, PhD
Background: Patients' loved ones often make end-of-life
treatment decisions, but the accuracy of their substituted
judgments and the factors associated w i t h accuracy are
poorly understood.
Objective: To assess the accuracy of judgments made by
surrogate decision makers; ascertain the beliefs, practices,
and clinical and sociodemographic factors associated w i t h
accuracy of surrogates' decisions; assess the preferences of
patients for life-sustaining treatments; and compare dif-
ferences in accuracy across diagnoses.
Design: Cross-sectional paired interviews.
Setting: Outpatient practices of three university hospitals.
Patients: 250 patients w i t h terminal diagnoses of conges-
tive heart failure, AIDS, amyotrophic lateral sclerosis, lung
cancer, and chronic obstructive pulmonary disease (50 pa-
tient-surrogate pairs in each group) and 50 general med-
ical patients and their surrogates.
Measurements: The accuracy of surrogate predictions
was measured by using scales based on 10 potential treat-
ments in each of three hypothetical clinical scenarios.
Results: Preferences varied according t o mode of treat-
ment and scenario. On average, surrogates made correct
predictions in 66% of instances. Accuracy was better for
the permanent coma scenario than for the scenarios of
severe dementia or coma w i t h a small chance of recovery
{P < 0.001). In a binary logit model, the accuracy of substi-
tuted judgments was positively associated w i t h the patient
having spoken w i t h the surrogate about end-of-life issues
(odds ratio [OR], 1.9 [95% CI, 1.6 t o 2.3]), the patient
having private insurance (OR, 1.4 [CI, 1.1 t o 1.7]), the
surrogate's level of education (OR, 1.5 [CI, 1.2 to 1.9]), and
the patient's level of education (OR, 1.7 [CI, 1.4 t o 2.2]).
Accuracy was negatively associated w i t h the patient's be-
lief that he or she would live longer than 10 years (OR, 0.6
[CI, 0.5 t o 0.7]), surrogate experience w i t h life-sustaining
treatment (OR, 0.4 [CI, 0.3 t o 0.5]), surrogate participation
in religious services (OR, 0.67 [CI, 0.50 t o 0.91]), and a
diagnosis of heart failure (OR, 0.6 [CI, 0.5 t o 0.8]). Age,
ethnicity, marital status, religion, and advance directives
were not associated w i t h accuracy.
Conclusions: The accuracy of substituted judgments is
associated w i t h multiple clinically apparent patient and
surrogate factors. This information can help clinicians
identify conditions under which substituted judgments are
likely t o be accurate or inaccurate and can help target
populations for education designed t o improve the accu-
racy of surrogate decision making.
This paper is also available at http://www.acponline.org.
Ann Intern Med. 1998;128:621-629.
From Georgetown University Medical Center, Washington, D.C.;
and the Johns Hopkins Medical Institutions, Baltimore, Mary-
land. For current author addresses, see end of text.
W hen patients face end-of-life decisions and
are unable to speak for themselves, loved
ones are often asked to make substituted judg-
ments. Ethicists have argued that this is an impor-
tant way of respecting patient autonomy (1, 2), but
preliminary studies indicate that substituted judg-
ments may not be very accurate (3-11). This con-
cern has led some observers to question the useful-
ness of asking for substituted judgments (8, 12).
If the concept of substituted judgment is to re-
main clinically useful, health care professionals need
a better sense of the circumstances under which
substituted judgments made by surrogates are accu-
rate or inaccurate. Such data may alert clinicians to
instances in which patients are at risk for having
their wishes misrepresented and may aid in the
design of education campaigns to improve the ac-
curacy of substituted judgments.
Little is known about the factors associated with
accurate substituted judgments, especially among
patients with terminal diagnoses (8, 11). In a pilot
study of 50 general medical patients (13), we found
that accuracy was positively associated with discus-
sions between patients and surrogates and nega-
tively associated with religiosity. Building on this
experience, we studied 250 terminally ill patients
and their legal surrogates and a comparison group
of 50 general medical patients and their surrogates.
Our objectives were to 1) assess the accuracy of the
substituted judgments made by the surrogates; 2)
ascertain the beliefs, practices, and clinical and so-
ciodemographic factors associated with accuracy; 3)
assess the preferences of patients for life-sustaining
treatments; and 4) compare differences in accuracy
across diagnoses.
Methods
Patients
Between November 1993 and February 1996, pa-
tients were recruited from the outpatient practices
of Johns Hopkins Hospital and the Hopkins Bay-
view Medical Center, Baltimore, Maryland, and
© 1998 American College of Physicians 621
Georgetown University Medical Center, Washing-
ton, D.C. Eligible patients were older than 17 years
of age, spoke English, and had one of the following
conditions known to be associated with a less than
50% chance of surviving for 2 years: New York
Heart Association class III or IV congestive heart
failure with no possibility of transplantation (14),
advanced HIV infection according to the Centers
for Disease Control and Prevention case definition
for AIDS (before the use of protease inhibitors)
(15), amyotrophic lateral sclerosis with respiratory
compromise (16), unresectable non-small-cell lung
cancer (17), or oxygen-dependent chronic obstruc-
tive pulmonary disease with dyspnea at rest (18, 19).
Treating physicians were asked to exclude patients
whom they thought they were likely to live longer
than 2 years and patients who were cognitively in-
capable of participating. The comparison group
comprised general medical patients who were older
than 64 years of age, spoke English, and were ex-
pected to live longer than 2 years.
To have a power of 0.80 to detect a 15% differ-
ence between patient preferences and surrogate pre-
dictions at a = 0.05, we needed 263 patient-surrogate
pairs. Rounding up to 300 gave a power of 0.75 to
detect a 25% difference in agreement between di-
agnostic groupings at a = 0.05.
Surrogates were chosen according to Maryland's
legal hierarchy of surrogates for incapacitated pa-
tients: durable power of attorney, guardian, spouse,
adult child, parent, sibling, other relative, or friend
(20). For uniformity, surrogates of patients recruited
at Georgetown University Medical Center were also
chosen according to the Maryland hierarchy.
Attending physicians were contacted 2 weeks be-
fore scheduled patient visits. Patients were initially
informed of the general nature of the study and
asked to supply the phone number of the surrogate
who was highest in Maryland's legal hierarchy. At
the time of the appointment, details of the study
were described and informed consent was obtained.
Surrogates were interviewed within 48 hours before
Table 1 . Clinical Scenarios
Please imagine that each situation happens to you in addition to your
present medical condition.
Scenario 1: Imagine that you are permanently unconscious and, in the
opinion of your physicians, have no hope of waking up and can feel
no pain.
Scenario 2: Imagine that you are in a coma and can feel no pain. The
physicians think that with aggressive treatment:
You have a high chance of dying (4 in 5).
You have a small chance of coming out of the coma but not being able
to speak or understand others (1 in 5).
You have a very small chance of coming out of the coma and being
able to speak and understand others (less than 1 in 100).
Scenario 3: Imagine that you are not in a coma, but you develop a brain
disease (such as Alzheimer disease) that cannot be cured and renders
you unable to recognize people or to speak. Assume that you can
feel pain.
622 15 April 1998 • Annals of Internal Medicine • Volume 128 • Number 8
or after the patient interview. Patients and surro-
gates were asked not to speak with each other about
the interviews until both interviews were completed.
Informed consent was obtained from all partici-
pants. The study was approved by the institutional
review boards of the Johns Hopkins Medical Insti-
tutions and Georgetown University Medical Center.
Interview Schedule
Structured interviews consisted of closed-ended
questions about sociodemographic factors, previous
discussions about end-of-life decisions, advance di-
rectives, and past experiences. Patients were asked
about their treatment preferences, and surrogates
were asked to predict patient preferences. Sociode-
mographic information included age, sex, ethnic
group, exposure category for HIV infection, level of
education, socioeconomic factors, insurance status,
and religious beliefs and practices.
We elicited preferences for various medical in-
terventions in the setting of three hypothetical clin-
ical scenarios that would render patients unable to
make decisions for themselves: permanent coma,
coma with a small chance of recovery (the "small
chance" scenario), and severe dementia (Table 1).
For each scenario, the patient was asked to report
his or her preferences for intensive care, intubation,
cardiopulmonary resuscitation, feeding tube place-
ment, nasotracheal suction, surgery, hemodialysis,
esophagogastroduodenoscopy, phlebotomy, and chest
roentgenography. All scenarios and interventions
were described briefly and simply to ensure under-
standing. For example, the feeding tube question
was worded, "Would you want to be fed by a feed-
ing tube (that is, a tube put through your nose or
through the skin over your stomach)?"
Requests for a limited trial of an intervention
were interpreted as "yes" responses. Patients and
surrogates were urged to give their best judgment.
"Unsure" responses were recoded as "yes" re-
sponses because they would be clinically interpreted
that way. The interview schedules had been previ-
ously developed and tested for reliability (13) and
were based on the Medical Directive of Emanuel
and Emanuel (21). Slight modifications were made
for our protocol; the revised instruments showed
high internal consistency, with Kuder-Richardson
statistics of 0.97 for the permanent coma scenario,
0.94 for the small chance scenario, and 0.93 for the
severe dementia scenario. The complete set of in-
struments is available from the authors on request.
Surrogate Accuracy in Matching Patient
Preferences Scale Scores
We constructed a Surrogate Accuracy in Match-
ing Patient Preferences Scale (SAMPPS) for each of
the three scenarios (SAMPPS-1 for permanent
coma, SAMPPS-2 for small chance, and SAMPPS-3
for severe dementia). This scale assigns a score of 1
when the surrogate's prediction correctly matches
the patient's preference for each scenario-treatment
pair. Mismatches are assigned a score of 0. The sum
of the correct answers constitutes the SAMPPS
score. For each of the three scenarios, a perfect score
is 10 and a complete mismatch has a score of 0.
Statistical Analysis
Categorical variables were analyzed by using the
chi-square test or the Fisher exact test. Two-tailed t-
tests were used for paired continuous variables.
Analysis of variance was used to compare continu-
ous variables across multiple groups. Differences be-
tween surrogate and patient demographic character-
istics were examined by using the McNemar test for
paired comparisons. The McNemar test was also
used to assess any pattern in mismatches between
patient and surrogate preferences. Differences in
patient preferences for treatment according to mo-
dality or scenario were assessed by using the Coch-
ran Q test.
Although K scores have often been reported in
similar studies, we do not report them because of
concern about the appropriateness of K as a test of
surrogate accuracy (11, 22), and the "paradox of
kappa" (23). We presented elsewhere an alternative
to K for measuring surrogate accuracy (24).
Factors Associated with the Accuracy of
Substituted Judgments
A skewed distribution was found for SAMPPS
scores. The Friedman two-way analysis of variance
was used to compare SAMPPS scores across scenar-
ios, and the Kruskal-Wallis one-way analysis of vari-
ance was used to compare SAMPPS-1 scores across
diagnoses.
A binary logit model (25) was constructed using
the 10 binary items (correct/incorrect) per patient-
surrogate pair of the SAMPPS-1 score. Indepen-
dent variables were screened for association with
SAMPPS-1 score by using the Spearman rank cor-
relation and Kruskal-Wallis tests, as appropriate.
Variables that did not contribute significantly to the
multivariate model were eliminated in a stepwise
manner. When diagnostic categories were stratified
by age (>65 years or <65 years), the log odds
differed significantly by age group for patients with
amyotrophic lateral sclerosis, but no such interac-
tion was found between education and insurance.
Model diagnostics to identify influential observa-
tions were examined (26), revealing 9 outliers that
were subsequently removed from the final model.
Thus, the final model reports on only 291 patient-
surrogate pairs. The model was validated by using
the independent variables found to be significant in
15 April 1998 • Annals of Internal Medicine • Volume 128 • Number 8
the SAMPPS-1 model to predict the SAMPPS-2
and SAMPPS-3 scores.
Results
Participants
Interviews of both patient and surrogate were
completed for 75% of pairs that met study inclusion
criteria. Of interviews that were not completed,
51% were not completed because of patient refusal;
25% were not completed because of patient refusal
to allow surrogate contact; 15% were not completed
because of surrogate refusal; and 9% were not com-
pleted for other reasons, chiefly our inability to
arrange for a surrogate interview within 48 hours of
the patient interview. Compared with participants,
nonparticipants were older (68 and 60 years of age;
P< 0.001), more likely to be female (65% and
39%; P < 0.001), more likely to receive Medicare
(24% and 10%; P < 0.001), and more likely to be in
the chronic obstructive pulmonary disease or gen-
eral medicine groups (P < 0.001). Participants and
nonparticipants did not differ significantly by ethnic
group. Patients and surrogates were interviewed on
the same day in all but 31 cases.
Surrogates were younger, better educated, more
likely to be married, and more likely to be female
than patients (Table 2). They were likely to be
spouses (57%) or adult children (22%). The surro-
gate that we interviewed, chosen according to Mary-
land's legal hierarchy, was the person the patient
would have chosen (the patient's preferred surro-
gate) in 90% of cases. Patients with AIDS were the
youngest, the most likely to be of minority ethnicity,
the most likely to have not graduated from high
school, and the most likely to lack private insurance
(Table 3).
Substituted Judgments
Surrogate accuracy did not differ significantly ac-
cording to whether patients were interviewed on the
same day (SAMPPS-1 score, 7.4) or on different
days (SAMPPS-1 score, 7.0) (P > 0.2). Overall, sur-
rogates correctly predicted the wishes of patients for
particular treatments in specific scenarios in 66% of
instances. Surrogates were better at predicting pa-
tients' wishes for more invasive procedures. For ex-
ample, in the permanent coma scenario, surrogates
were correct about ventilator care in 84% of in-
stances and about cardiopulmonary resuscitation in
79% of instances compared with 57% of instances
for phlebotomy and 62% of instances for chest
roentgenography (P < 0.001).
As shown in Figure 1, accuracy scores did not
differ significantly according to patient diagnosis,
623
Table 2. Characteristics of 300 Study Patients and thought that the patient would want the treatment,
300 Surrogates*
and only 36% of these patients did want it.
Characteristic Patient Surrogate P Value
Mistaken Substituted Judgments
Mean age (range), y 60(22-84) 55(18-83) <0.001t
Women, % 39.0 73.3 <0.001* For most modes of treatment, when the surro-
White persons, % 72.7 72.3 NS* gates were wrong, they were no more likely (by the
Marital status, %
Married 58.7 75.3 McNemar test) to state that the patient would want
Widowed, separated, or divorced 26.7 12.0 treatment when the patient did not want it than
Never married or living with
significant other 14.7 12.7 <0.001* they were to state that the patient would not want
Education, % treatment when the patient did want it. The only
Less than high school 22.0 14.7
High school graduate or some exceptions across all three scenarios were the less
college 50.3 53.7 invasive procedures of nasotracheal suctioning,
College graduate or more than
college 27.7 31.7 <0.05* phlebotomy, and chest roentgenography. In the per-
Religion, % manent coma scenario, for example, the McNemar
Protestant 56.7 58.7
Catholic 27.0 28.3 statistic was significant only for suctioning (P =
Jewish, other, or none 16.3 13.3 NS* 0.02), phlebotomy (P < 0.001), and chest roentgen-
Type of insurance, %
Private insurance, health ography (P < 0.001). For these treatments, the sur-
maintenance organization, rogate's tendency was to state that the patient would
or other 74.3
Medicare only 9.7 want treatment when the patient did not want it.
Medicaid only 16.0

* NS = not significant, Factors Associated with Accuracy

t Paired f-test.
* McNemar test.
but they were significantly better for the permanent
coma scenario (mean SAMPPS-1 score, 7.4) than
for the small chance scenario (mean SAMPPS-2
score, 6.2) and the severe dementia scenario (mean
SAMPPS-3 score, 6.3) (P < 0.001 by Friedman two-
way analysis of variance).
Surrogates generally chose for patients what they
would choose for themselves (in 89% of instances
for the permanent coma scenario, 65% of instances
for the small chance scenario, and 72% of instances
for the dementia scenario), and this preference was
usually the same as the patient's preference. For
example, of the 51 surrogates who would choose
ventilator support for themselves in the small
chance scenario, 75% thought that the patient
would want the treatment, and 61% of these pa-
tients did want it. Of the 249 surrogates who would
not want ventilator support themselves, only 37%
To sort out the weight and independence of fac-
tors associated with the accuracy of substituted
judgments, we developed a binary logit model by
using the responses for the permanent coma sce-
nario as the dependent variable (Table 4). The sur-
rogates of 16 patients with amyotrophic lateral scle-
rosis who were older than 64 years of age did very
well, with an average SAMPPS-1 score of 9.5 out of
10 correct, whereas the surrogates of younger pa-
tients with amyotrophic lateral sclerosis and those
of patients with congestive heart failure were only
about half as accurate as the surrogates of patients
in the general medicine reference group. Accurate
substituted judgments were associated with patients
who had private insurance, patients and surrogates
who had high school diplomas, patients who had
spoken in detail about their wishes to surrogates,
patients who expected to die in 10 years or less,
surrogates who never attended religious services,
and surrogates who had never had personal experi-
ence with a ventilator. Factors that were not signif-

Table 3. Character istics and Diagnoses of 300 Patierits

Characteristic Diagnosis P Value

New York Heart Association AIDS Amyotrophic Unresectable Chronic Obstructive General Internal
Class III or IV Heart Failure and Lateral Sclerosis Non-Small-Cell Pulimonary Disease Medicine
No Possibility of Transplantation Lung Cancer

Mean age, y 64 37 55 63 67 73 <0.001*

Women, % 30 36 32 38 52 46 0.18t
Ethnic minority, % 36 62 8 18 12 28 <0.001t
High school graduate, % 84 68 96 70 74 76 0.01t
Protestant, % 58 66 70 40 46 60 0.02t
Private insurance, % 90 24 90 90 74 78 <0.001t

* Analysis of variance,
t Chi-square test.

624 15 April 1998 • Annals of Internal Medicine • Volume 128 • Number 8


Figure 1. Accuracy of substituted judgments according to diagno-
sis and scenario. Mean Surrogate Accuracy in Matching Patient Prefer-
ences Scale scores are given for three hypothetical scenarios. Responses of
50 surrogates of 50 patients with congestive heart failure (CHF), AIDS,
amyotrophic lateral sclerosis (ALS), unresectable non-small-cell lung cancer
(CA), and chronic obstructive pulmonary disease (COPD) were scored. Scores
of 50 general internal medicine (GIM) patients are also shown. Scores did
not differ significantly by diagnosis. Overall accuracy was higher for the
permanent coma scenario than for the small chance and severe dementia
scenarios (P = 0.001 by Friedman two-way analysis of variance).
icant in univariate analysis or that became insignif-
icant when other factors were adjusted for included
patient and surrogate sex, ethnicity, and religious
denominations; relationship of the surrogate to the
patient; whether the patient had an advance direc-
tive; and the surrogates' degree of satisfaction with
their understanding of the patients' wishes.
To assess the validity of the final model, which
was developed using the SAMPPS-1 responses (per-
manent coma scenario) as the outcome variable, we
substituted SAMPPS-2 (small chance) and SAMPPS-3
(severe dementia) as the outcome variables. The
global fit of the model was significant for these two
scenarios, with chi-square values of 118.4 (P <
0.001) and 101.9 (P < 0.001), respectively, for - 2
log likelihood. All independent variables from the
permanent coma model except for the surrogate's
personal experience with ventilator treatment and
type of patient insurance remained significant in at
least one of the two other models.
To discern whether surrogates who attended re-
ligious services were "imposing their values" on pa-
tients, we explored surrogates' preferences for
themselves according to their church- or temple-
going behavior. Although numbers were small, sur-
rogates who attended religious services were not
significantly more likely to want ventilator treatment
for themselves than were surrogates who never at-
tended religious services (13% and 6% for the per-
manent coma scenario; P = 0.18). However, surro-
gates who attended religious services were significantly
more likely than nonattenders to think that the
patients would want ventilator treatment in the per-
manent coma (17% and 2%; P = 0.007), small
chance (46% and 30%, P = 0.04), and severe de-
mentia (39% and 21%; P = 0.02) scenarios.
Practices and Beliefs
Seventy-five percent of patients reported that
they had thought about advance directives, and 63%
reported having spoken to someone about end-of-
life issues. Only 21% of patients who had spoken to
someone about end-of-life issues reported that they
had spoken to a physician or nurse, and only 44%
had spoken with their legal surrogate. For patients
who had spoken about their wishes, hospitalization
was the cue that prompted this conversation only
4% of the time. Events in the lives of family or
friends (47%) were more commonly cited as rea-
sons for such conversations than events in the pa-
tients' own lives (36%).
Of patients who had not spoken with anyone
about end-of-life issues, 65% intended to do so.
Major barriers included not wanting to upset loved
ones (41%), not believing that they were ill enough
(32%), and uncertainty (30%).
Thirty-three percent of patients had living wills,
and 31% had durable powers of attorney for health
care. Patients in the general internal medicine
group were less likely than patients with terminal
diagnoses to have living wills (18%; P = 0.02) or
durable powers of attorney for health care (16%;
P = 0.01). Of patients who had living wills or dura-
ble powers of attorney for health care, 13% and
15%, respectively, reported having learned about
these documents from a health care professional.
These patients also learned about these documents
from lawyers (19% and 22%, respectively), family
and friends (29% and 19%), the media (31% and
22%), and other sources (28% and 32%).
Table 4. Factors Associated with the >Vccuracy of
Substituted Judgments in Peirmanent Coma
Scenario: Binary Logit Model *
Factor Odds Ratio (95% CI)
Surrogate has experience with life-sustaining
treatmentst 0.41 (0.30-0.55)*
Surrogate does not go to church or temple 1.48(1.12-1.96)*
Patient has spoken about end-of-life care in
detail with surrogate§ 1.89(1.56-2.28)*
Patient thinks that he or she will live for more
than 10 years 0.62(0.51-0.75)*
Patient has private insurance 1.35(1.06-1.73)*
Patient has high school diploma 1.73(1.38-2.19)*
Surrogate has high school diploma 1.50(1.16-1.92)*
Patient diagnosis
ALS and >65 years of age 5.40(2.14-13.60)*
ALS and <65 years of age 0.51 (0.36-0.71)*
New York Heart Association class III or IV
heart failure and no possibility of
transplantation 0.60(0.45-0.81)*
AIDS 1.07(0.77-1.48)
Unresectable non-small-cell lung cancer 1.22(0.89-1.67)
Chronic obstructive pulmonary disease 1.16(0.85-1.59)
General internal medicine (reference group) 1.00
* ALS = amyotrophic lateral sclerosis,
t Cardiopulmonary resuscitation, intubation, or feeding t iibe treatment.
*P<0.01.
§ Surrogate and patient have spoken about preferences fc>r specific life-sustaining treat-
ments.

15 April 1998 • Annals of Internal Medicine • Volume 128 • Number 8 625


Figure 2. Percentage of patients who would choose each inter-
vention according to scenario and intervention. Percentages are the
proportion of patients (n = 300) who would choose the following interven-
tions in three hypothetical scenarios: intensive care (ICU), ventilator treat-
ment (Vent), cardiopulmonary resuscitation (CPR), feeding tube placement
(Feed), surgery to fix a correctable problem (Surg), hemodialysis (Dial),
esophagogastroduodenoscopy (EGD), nasotracheal suction (Suet), phlebot-
omy (Phlb), or chest roentgenography (CXR).
Overall, 27% of 249 patients with terminal diag-
noses thought that they would be cured. This belief
was especially common among patients with unre-
sectable non-small-cell lung cancer (54%) and
AIDS (32%) compared with patients with chronic
obstructive pulmonary disease (12%), amyotrophic
lateral sclerosis (16%), and congestive heart failure
(22%) (P < 0.001). Only 10% of the patients with
626 15 April 1998 • Annals of Internal Medicine • Volume 128 • Number 8
terminal diagnoses judged their own life expectancy
to be less than 2 years. Twenty-six percent of pa-
tients thought that they would live longer than 10
years.
Preferences
Patient preferences varied according to treatment
and scenario. As shown in Figure 2, 13% of patients
would choose ventilator treatment if they were in a
permanent coma. However, three times as many
patients (39%) would choose such treatment if they
were told that they had even a less than 1% chance
of recovery to their present medical state, and 33%
would choose ventilator treatment if they were de-
mented rather than comatose.
Across all scenarios, more patients would choose
less invasive procedures (such as chest roentgenog-
raphy and phlebotomy) than more invasive proce-
dures (such as cardiopulmonary resuscitation and
ventilator support) (P < 0.001). For example, in the
permanent coma scenario, 20% of patients would
want cardiopulmonary resuscitation and 13% would
want ventilator support, whereas more than 30%
would want chest roentgenography and phlebotomy.
Surrogates were also less likely to prefer ventila-
tor support for themselves in the setting of perma-
nent coma (12%) than in the small chance (17%) or
the severe dementia (20%) scenarios (P = 0.003).
Across all three scenarios, surrogates were less
likely to choose ventilator support for themselves if
they were white or never attended religious services.
Discussion
Substituted judgment is an imperfect method for
assessing the true wishes of patients who lack deci-
sion-making capacity (3-11, 13). One way of grap-
pling with this imperfection would be to ascertain
the conditions under which substituted judgments
are likely to be inaccurate so that clinicians can
learn when to raise their index of suspicion that
patients may be at risk for erroneous substituted
judgments. In addition, health care professionals
might identify potentially remediable factors associ-
ated with poor substituted judgments and design
targeted interventions to improve the process. Our
study takes initial steps toward these goals.
Substituted Judgments
Our results suggest that the substituted judg-
ments made for patients with terminal diagnoses are
not better than those made for general medical
patients. These findings show the need for more
careful interpretation of substituted judgments for
terminally ill patients and for improvements in the
process.
 In addition to our pilot study (13), two other
studies have attempted to elicit factors associated
with the accuracy of substituted judgments. The
study by Suhl and colleagues (8) was limited by
small numbers (50 participants) and a lack of multi-
variate analysis. The study by Layde and coworkers
(11) examined preferences only for cardiopulmo-
nary resuscitation, explored a limited number of
independent variables, sampled only hospitalized
patients, and was unusual in that 74% of patients
wanted cardiopulmonary resuscitation.
Our study corroborates the results of previous
studies showing that substituted judgments are more
accurate in the setting of an explicit discussion be-
tween patient and surrogate about patient prefer-
ences (8, 11, 13). In addition, we found associations
with many other factors. For example, more accu-
rate substituted judgments are associated with better-
educated patients and surrogates.
As we reported elsewhere (13), the accuracy of
substituted judgments is inversely related to the sur-
rogate's church or temple attendance, independent
of religious denomination. This counterintuitive
finding does not seem to reflect an imposition of
the surrogates' wishes for themselves on the pa-
tients. Surrogates who attend religious services are
actually more likely to report that they do not want
the treatment for themselves but would choose it
for the patient. Surrogates who attend religious ser-
vices may be more likely to feel a sense of duty to
choose treatment for their loved ones even if they
would not choose it for themselves; may be moti-
vated by religiously mediated feelings of compas-
sion, altruism, or guilt; or may be unable to "let
go." We did not ask questions that would allow us
to explore these hypotheses.
Optimistic patient estimates of their own prog-
noses for survival were associated with increased
preferences for treatments and inaccuracy in substi-
tuted judgments. It is understandable that surro-
gates might err in their estimates of patient wishes
if they have different views of the patient's life
expectancy.
Surrogates of patients with congestive heart fail-
ure who are not candidates for transplantation
made more inaccurate substituted judgments com-
pared with the reference group of general medical
patients. Predicting mortality is generally difficult
(27), but predictions may be even more difficult for
congestive heart failure than for other diagnostic
categories because death in congestive heart failure
is usually sudden (28). Perhaps this greater prog-
nostic uncertainty makes it more difficult for pa-
tients to make choices or for surrogates to predict
those choices. Alternatively, cardiologists may be
more reluctant to discuss end-of-life issues because
of this uncertainty or for other reasons.
15 April 1998 • Annals of Internal Medicine • Volume 128 • Number 8
The surrogates of older patients with amyotro-
phic lateral sclerosis were remarkably accurate. Few
patients with amyotrophic lateral sclerosis choose
ventilator support (29), and older patients with
amyotrophic lateral sclerosis report intense psycho-
logical distress (30). Loved ones of older patients
with amyotrophic lateral sclerosis may be especially
adept at reading these signals.
It is interesting that, when they were wrong, sur-
rogates were no more likely to say that a patient
would want treatment when the patient actually did
not than they were to say that the patient would not
want treatment when the patient actually did. This
suggests that surrogate errors are more random
than systematic and may be more amenable to ed-
ucational intervention.
Patient Beliefs, Practices, and Preferences
As other studies have shown (31, 32), although
most patients had thought about end-of-life issues,
relatively few had actually prepared advance direc-
tives or had spoken to their surrogates about their
wishes. Health care professionals were infrequently
the source of patient knowledge about living wills or
durable powers of attorney.
Terminally ill patients, especially those with un-
resectable non-small-cell lung cancer and AIDS,
made estimates of their own survival prognoses that
far exceeded those in the published literature. De-
spite this optimism, most patients (as other re-
searchers have found [30, 31]) preferred to forgo
life-sustaining treatments. Our study provides a
more nuanced understanding of this phenomenon
by showing that patients with terminal diagnoses are
much more likely to forgo invasive procedures than
to forgo less invasive procedures.
The clinical scenario also influenced preferences.
Almost three times as many terminally ill patients
would opt to continue ventilator support if they
were told that they had a less than 1% chance of
recovery than if they were told that they would
never recover from a coma. This finding raises ques-
tions about the importance attached to suggesting
any hope of survival and about how prognostic es-
timates are framed (33).
Limitations
Our study sampled only five terminal conditions
and may not represent all patients with terminal
conditions. We cannot be absolutely sure that "con-
taminating" discussions did not take place between
patients and surrogates, but our data show that
accuracy did not differ significantly if patients and
surrogates were interviewed on different days. Fi-
nally, all scenario studies are somewhat artificial,
and it is not known how substituted judgments
might change in actual clinical settings. However,
627
this matter can be studied only hypothetically be-
cause one cannot know the minds of persons who
lack decision-making capacity.
Conclusions and Implications
Reliance on surrogate decisions seems inescap-
able. Living wills are not universally used, and their
specific provisions still require interpretation. If cli-
nicians remain committed to the belief that patient
autonomy is not completely abrogated when pa-
tients lose their decision-making capacity, there re-
mains little choice but to continue to ask those who
know the patient best what they think the patient
would want. The information gathered in our study
suggests several testable strategies for grappling
with this imperfect process.
The accuracy of substituted judgments is associ-
ated with multiple clinically apparent patient and
surrogate factors. By determining these factors, our
study helps clinicians know when to raise their index
of suspicion that a patient's loved one may not be
representing the patient's true preferences. Under
these circumstances, the clinician may wish to take
additional steps, such as probing more deeply or
involving other relatives.
Several testable strategies for intervention may
help surrogates make more accurate substituted
judgments. For instance, health care professionals
might actively urge patients to talk to their loved
ones about their wishes for end-of-life care. Substi-
tuted judgments may be improved if clinicians have
more frank discussions with terminally ill patients
about their prognoses. These interventions might
take place in physicians' offices, in community set-
tings, or in support groups for patients with terminal
diagnoses. Because surrogate attendance at religious
services is associated with inaccurate substituted
judgments, community education efforts designed to
improve the quality of surrogate decision making
should perhaps target churches and temples in par-
ticular (34). Further study is needed to ascertain
whether any of the aforementioned strategies will
be effective in improving substituted judgments.
Acknowledgments: The authors thank Ruth Faden and Edmund
Pellegrino for their advice during the design and analysis of this
project and Stephanie Poe and Andrea Kalfoglou for interview-
ing patients.
Grant Support: By the National Institute for Nursing Research
(NIH-RO1NR03045-01A1). Dr. Sulmasy is supported by a Soros
Faculty Scholar Award of the Project on Death in America.
Requests for Reprints: Daniel P. Sulmasy, OFM, MD, PhD, Cen-
ter for Clinical Bioethics, Room 236, Building D, Georgetown
University Medical Center, Washington, DC 20007; e-mail
sulmasyd@gunet.georgetown.edu.
Current Author Addresses: Dr. Sulmasy: Center for Clinical Bio-
ethics, Room 236, Building D, Georgetown University Medical
Center, Washington, DC 20007.
628 15 April 1998 • Annals of Internal Medicine • Volume 128 • Number 8
Dr. Terry: Johns Hopkins University, Division of Pulmonary
Care, 5501 Bayview Circle, Asthma and Allergy Center Building,
Baltimore, MD 21224.
Dr. Weisman: Department of Health Management and Policy,
School of Public Health, University of Michigan, 109 South Ob-
servatory, Room M3138, Ann Arbor, MI 48109.
Dr. Miller: Center for Nursing Research, Johns Hopkins School
of Nursing, 1830 East Monument Street, Room 233, Baltimore,
MD 21205.
Ms. Stallings: Johns Hopkins School of Hygiene and Public
Health, International Health/Human Nutrition, 615 North Wolfe
Street, Room 2041, Baltimore, MD 21205.
Dr. Vettese: Johns Hopkins School of Nursing, 1830 East Mon-
ument Street, Room 417, Baltimore, MD 21205.
Dr. Haller: Department of Medicine, Johns Hopkins University
School of Medicine, 1830 East Monument Street, Room 9061,
Baltimore, MD 21205.
References
1. Substantive and procedural principles of decision making for incapacitated
patients. In: United States. President's Commission for the Study of Ethical
Problems in Medicine and Biomedical and Behavioral Research. Making Health
Care Decisions: A Report on the Ethical and Legal Implications of Informed
Consent in the Patient-Practitioner Relationship. Washington, DC: US Gov Pr
Office; 1982:177-82.
2. Buchanan A, Brock DB. Deciding for Others: The Ethics of Surrogate Deci-
sion Making. New York: Cambridge Univ Pr; 1990.
3. Tomlinson T, Howe K, Notman M, Rossmiller D. An empirical study of
proxy consent for elderly persons. Gerontologist. 1990;30:54-64.
4. Zweibel NR, Cassel CK. Treatment choices at the end of life: a comparison
of decisions by older patients and their physician-selected proxies. Gerontol-
ogist. 1989;29:615-21.
5. Uhlmann RF, Pearlman RA, Cain KC. Physicians' and spouses' predictions
of elderly patients' resuscitation preferences. J Gerontol. 1988;43:M 115-21.
6. Ouslander JG, Tymchuk AJ, Rahbar B. Health care decisions among elderly
long-term care residents and their potential proxies. Arch Intern Med. 1989;
149:1367-72.
7. Libbus MK. Russell C. Congruence of decisions between patients and their
potential surrogates about life-sustaining therapies. Image J Nurs Sch. 1995;
27:135-40.
8. Suhl J, Simons P, Reedy Tf Garrick T. Myth of substituted judgment.
Surrogate decision making regarding life support is unreliable. Arch Intern
Med. 1994;154:90-6.
9. Hare J, Pratt Cf Nelson C. Agreement between patients and their self-
selected surrogates on difficult medical decisions. Arch Intern Med. 1992;152:
1049-54.
10. Seckler AB, Meier DEf Mulvihill M, Paris BE. Substituted judgment: how
accurate are proxy predictions? Ann Intern Med. 1991;115:92-8.
11. Layde PM, Beam CA, Broste SK, Connors AF Jr, Desbiens N, Lynn J, et
al. Surrogates' predictions of seriously ill patients' resuscitation preferences.
Arch Fam Med. 1995;4:518-23.
12. Baergen R. Revising the substituted judgment standard. J Clin Ethics. 1995;
6:30-8.
13. Sulmasy DP, Haller K, Terry PB. More talk, less paper: predicting the
accuracy of substituted judgments. Am J Med. 1994;96:432-8.
14. Madsen BK, Hansen JF, Stokholm KH, Brons J. Husum D, Mortensen
LS. Chronic congestive heart failure. Description and survival of 190 consec-
utive patients with a diagnosis of chronic congestive heart failure based on
clinical signs and symptoms. Eur Heart J. 1994;15:303-10.
15. Jacobson LP, Kirby AJ, Polk S, Phair JP, Besley DR, Saah AJ, et al.
Changes in survival after acquired immunodeficiency syndrome (AIDS): 1984-
1991. Am J Epidemiol. 1993;138:952-64.
16. Gay PC, Westbrook PR, Daube JR, Litchy WJ, Windebank AJ, Iverson
R. Effects of alterations in pulmonary function and sleep variables on survival
in patients with amyotrophic lateral sclerosis. Mayo Clin Proc. 1991;66:686-
94.
17. Rossing TH, Rossing RG. Survival in lung cancer. An analysis of the effects
of age, sex, resectability, and histopathologic type. Am Rev Respir Dis. 1992;
126:771-7.
18. Burrows B, Earle RH. Prediction of survival in patients with chronic airway
obstruction. Am Rev Respir Dis. 1969;99:865-71.
19. Dubois P, Jamart J, Machiels J, Smeets F, Lulling J. Prognosis of severely
hypoxemic patients receiving long-term oxygen therapy. Chest. 1994; 105:
469-74.
20. Maryland Annotated Code. Health-General, 5-605 C.
21. Emanuel LL, Emanuel EJ. The medical directive. A new comprehensive
advance care document. JAMA. 1989;261:3288-93.
22. Coughlin SS, Trock B, Criqui M H , Pickle LW, Browner D, Tefft MC. The
logistic modeling of sensitivity, specificity, and predictive value of a diagnostic
test. J Clin Epidemiol. 1992;45:1-7.
23. Feinstein AR, Cichetti DV. High agreement but low kappa: I. The problems
of two paradoxes. J Clin Epidemiol. 1990;43:543-9.
24. Sulmasy DP, Terry PB, Weisman CS, Miller DJ, Stallings R. Vettese MA,
et al. End-of-life substituted judgments: sensitivity, specificity, and predictive
value [Abstract]. J Gen Intern Med. 1997; 12 (Suppl 1):105.
25. Breslow NE, Day NE. Fitting models to grouped data. In: Davis W, Breslow
NE. Statistical Methods in Cancer Research, v 2. Lyon: International Agency
for Research on Cancer; 1987:120-76.
26. Pregibon D. Logistic regression diagnostics. Annals of Statistics. 1981;9:705-24.
27. Lynn J, Harrell F Jr, Cohn F, Wagner D, Connors AF Jr. Prognoses of
seriously ill patients on the days before death: implications for patient care
and public policy. New Horiz. 199;5:56-61.
28. Lynn J. Caring at the end of our lives [Editorial]. N Engl J Med. 1996;335:
201-2.
29. Moss AH, Casey P, Stocking CB, Roos RP, Brooks BR, Siegler M. Home
ventilation for amyotrophic lateral sclerosis patients: outcomes, costs, and
patient, family, and physician attitudes. Neurology. 1993;43:438-43.
15 April 1998 • Annals of Internal Medicine • Volume 128 • Number 8
30. McDonald ER, Wiedenfeld SA, Hillel A, Carpenter CL, Walter RA.
Survival in amyotrophic lateral sclerosis. The role of psychological factors.
Arch Neurol. 1994;51:17-23.
31. Emanuel LL, Barry MJ, Stoeckle JD, Ettelson LM, Emanuel EJ. Advance
directives for medical care—a case for greater use. N Engl J Med. 1991;324:
889-95.
32. Gamble ER, MacDonald PJ, Lichstein PR. Knowledge, attitudes, and be-
havior of elderly persons regarding living wills. Arch Intern Med. 1991; 151:
277-80.
33. Rybash JM, Roodin PA. The framing heuristic influences judgments about
older and younger adults' decision to refuse medical treatment. Applied Cog-
nitive Psychology. 1989;3:171-80.
34. Voorhees CC, Stillman FA, Swank RT, Heagerty PJ, Levine DM, Becker
D. Heart, body, and soul: impact of church-based smoking cessation inter-
ventions on readiness to quit. Prev Med. 1996;25:277-85.
629