See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/311781263

Development of a Predictive Model for Drug-Related Problems in Kidney

Transplant Recipients

Article  in  Pharmacotherapy · December 2016

DOI: 10.1002/phar.1886

CITATIONS READS

10 3,206

10 authors, including:

Kelly Covert Caitlin Musgrave Mardis

East Tennessee State University University of South Carolina
12 PUBLICATIONS   160 CITATIONS    11 PUBLICATIONS   219 CITATIONS   

SEE PROFILE SEE PROFILE

James Fleming Prince Mohan

Transplant Genomics Medical University of South Carolina
62 PUBLICATIONS   612 CITATIONS    28 PUBLICATIONS   161 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Donor Hepatic Steatosis View project

Improving Transplant Medication Safety Through a Pharmacist-Empowered, Patient-Centered, mHealth-Based Intervention: TRANSAFE Rx Study View project

All content following this page was uploaded by James Fleming on 17 August 2018.

The user has requested enhancement of the downloaded file.

 Development of a Predictive Model for Drug-Related
Problems in Kidney Transplant Recipients
Kelly L. Covert,1* Caitlin R. Mardis,2 James N. Fleming,3 Nicole A. Pilch,3 Holly B. Meadows,3
Benjamin A. Mardis,3 Prince Mohan,4 Maria Posadas-Salas,4 Titte Srinivas,4 and David J. Taber,5,6
1
Department of Pharmacy Practice, Bill Gatton College of Pharmacy, East Tennessee State University, Johnson
City, Tennessee; 2Transplant Service Line, Medical University of South Carolina, Charleston, South Carolina;
3
Department of Pharmacy Services, Medical University of South Carolina, Charleston, South Carolina;
4
Department of Transplant Nephrology, Medical University of South Carolina, Charleston, South Carolina;
5
Department of Surgery, Medical University of South Carolina, Charleston, South Carolina; 6Department of
Pharmacy, Ralph H. Johnson VAMC, Charleston, South Carolina

STUDY OBJECTIVE Drug-related problems (DRPs) are associated with increased rates of infection, rejection, and
graft loss in kidney transplant recipients. This study aimed to develop a model to predict which patients are
at highest risk of DRPs to streamline pharmacists’ workflow in a chronic kidney transplant clinic.
DESIGN Prospective observational study.
SETTING Chronic kidney transplant clinic at a large, tertiary care, academic hospital.
PATIENTS Two hundred thirty-seven adults seen in the kidney transplant clinic between September 16,
2015, and November 30, 2015, who were at least 90 days posttransplantation at the time of their
clinic visit.
MEASUREMENTS AND MAIN RESULTS Prospective data detailing DRPs and a survey assessing baseline char-
acteristics and patient-related outcomes were used to generate a predictive model to identify patients
at risk of having six or more DRPs; the cutoff of six DRPs provided a threshold for identifying a
subset of high-risk patients on whom the transplant pharmacists could focus their efforts. DRPs
were categorized as nonadherence, overdosing or underdosing, duplication of therapy, preventable
adverse drug reaction, missing medication, erroneous medication, conflicting provider information,
undermonitoring or lack of monitoring, and wrong medication received. In total, 865 unique DRPs
were identified, and the most common were erroneous medication, missing medication, and nonad-
herence, accounting for 38%, 21%, and 16% of the DRPs, respectively. A nine-variable model with a
sensitivity of 62.5% and specificity of 66.7% (area under the receiver operating characteristic curve
of 0.720) was developed to identify patients at risk of having six or more DRPs. The model included
the following variables: age, Medicaid for prescription insurance, current employment status, medi-
cation affordability, difficulty or lack of difficulty obtaining medications from the pharmacy, nega-
tive impact of medications on quality of life, medication nonadherence, poor rating of current
health status, and moderate or poor medication understanding.
CONCLUSION These results demonstrated that a straightforward, 5-minute survey completed by renal
transplant recipients prior to their clinic visit may be capable of effectively determining those at risk of
having six or more DRPs, potentially allowing use as a screening tool for transplant pharmacists’ work-
flow prioritization. External validation is needed before this tool can be used in the outpatient setting.

*Address for correspondence: Kelly L. Covert, Maple

Avenue, Building 7, Room 312, Johnson City, TN 37614;
e-mail: covertk@etsu.edu.
Ó 2016 Pharmacotherapy Publications, Inc.
160 PHARMACOTHERAPY Volume 37, Number 2, 2017
KEY WORDS transplant, drug safety, outcomes, pharmacy practice, renal.
(Pharmacotherapy 2017;37(2):159–169) doi: 10.1002/phar.1886
Drug-related problems (DRPs) have been
widely defined as any events or circumstances
involving drug treatment that actually or poten-
tially interfere with a patient experiencing opti-
mal outcomes of medical care.1 DRPs have been
a topic of considerable clinical research and
efforts since the Institute of Medicine published
“To Err Is Human” in 2001, which reported that
up to 98,000 patients die annually as a result of
medication errors.2 Since this report, more recent
literature has shown that nearly 400,000 patients
die annually secondary to medical errors.3
Transplant recipients are inherently at high
risk for DRPs given the complexity of their med-
ication regimens, multiple comorbid conditions,
and use of narrow–therapeutic index medica-
tions. Despite the propensity for DRPs, there is a
scarcity of published literature regarding the
incidence, risk factors, and effects of DRPs in
renal transplant recipients.4, 5 The literature that
does exist in renal transplant patients, as well as
data extrapolated from the surgical population,
suggests that DRPs in this population have been
associated with an increased risk of infection,
rejection, and graft loss.6–9
Pharmacists are uniquely equipped to identify
DRPs, and their positive impact on intervening
on behalf of patients with DRPs and reducing
rates of DRPs has been demonstrated in numer-
ous populations.10–13 Pharmacists have been
recognized as essential members of the inter-
disciplinary transplantation team in the United
Network of Organ Sharing (UNOS) bylaws since
2004, but neither UNOS nor Centers for Medi-
care and Medicaid Services (CMS) accreditation
standards specify the degree to which transplant
pharmacists must or should be involved in the
long-term outpatient care of transplant recipi-
ents.14 A recently published national workforce
survey of transplant pharmacists indicated that
77% of responders have some involvement in
the outpatient setting, but this involvement is
often limited to the patient’s first clinic visit
following discharge after transplantation. The
identified national median of 1.4 transplant
pharmacist full-time equivalent (FTE) positions
(range 0.1–7.1) per 100 transplant recipients
does not allow for services spanning all phases
of care, but the incorporation of the trans-
plant pharmacist into long-term ambulatory
management has been identified as a goal of the
profession.15
At our institution, transplant pharmacists have
recently extended coverage to long-term kidney
transplant clinics that serve patients who are at
least 3 months posttransplantation. Tradition-
ally, pharmacists’ roles in the clinic have
included completing medication reconciliations,
providing patient education and counseling,
identifying DRPs, and making therapeutic rec-
ommendations to providers. Unfortunately, the
patient volume within the chronic transplant
clinic at our institution averages 40–50 patients
per 6-hour clinic day, which creates a significant
workflow strain on our transplant pharmacists.
Given that this workflow strain has been identi-
fied both locally and nationally, the immediate
aim of this study was to design and validate a
patient-completed instrument that could be used
real time in the clinic setting to identify patients
at highest risk for DRPs, with a longer-term goal
of using the tool to prioritize patient care and
guide pharmacy workflow in the clinic.
Methods
Study Design, Setting, and Patients
This was a prospective observational study
conducted in kidney transplant recipients at a
750-bed tertiary care, academic hospital (Medi-
cal University of South Carolina, Charleston,
SC), which was approved by the local institu-
tional review board. All adult patients seen in
the renal transplant clinic between September
16, 2015, and November 30, 2015, who were at
least 90 days posttransplantation at the time of
their clinic visit were eligible for inclusion in
the study. Enrollment in the study was based on
pharmacist availability to conduct a visit at the
time the patient was transitioned into the clinic
room. Data were compiled by collection of
patient responses to the administered study
questionnaire as well as detailed chart review.
Patient Self-Administered Survey
The variables included in the survey were
chosen based on a thorough literature review to
identify factors that have been previously
 PREDICTIVE MODELING IN RENAL TRANSPLANT Covert et al
validated to identify patients with potential
DRPs.16–21 To ensure that the survey was of suf-
ficient length to identify risk factors for DRPs
but succinct enough to be completed in a short
period of time, the potential variables identified
in the literature were assessed by the investiga-
tors. This assessment included an evaluation of
the feasibility of reliably obtaining the variable
in question, the strength of the literature citing
the variable, and the patient demographics of
our center. A consensus among study investiga-
tors was required for variable inclusion in the
survey. The risk factors that were evaluated
included financial and social support, past med-
ical history, education background, and adher-
ence assessments. The medication adherence
questions were based on the Morisky Medica-
tion Adherence Scale.17 For variables in which a
validated questionnaire has not yet been estab-
lished, a “yes/no” answer choice was used
rather than a Likert scale given the concern for
interpatient variability and the challenges that
variability would pose when using the question-
naire to guide workflow interventions in the
future. The survey was self-administered and
intended to be delivered electronically using
iPads, with data automatically gathered into the
Research Electronic Data Capture (REDCap)
database system (Center for Clinical and Trans-
lational Services, Chicago, IL). REDCap is a
free, secure, Web-based site that can be used
for real-time data collection. Due to logistic
issues, some surveys were also conducted on
paper, with data entered into the REDCap data-
base following completion. A survey was con-
sidered completed and applicable for inclusion
in the study if the patient answered more than
50% of the survey questions. The full survey is
available in Appendix S1.
Drug-Related Problems
After completion of the survey, an encounter
occurred with a transplant pharmacist who was
blinded to the survey results. The transplant
pharmacist conducted a medication history ses-
sion to identify DRPs. During the study period,
a total of nine transplant pharmacists (specialists
and residents) participated in completing
medication histories. DRPs were categorized as
nonadherence, overdosing or underdosing,
duplication of therapy, preventable adverse drug
reaction, missing medication, erroneous medi-
cation, conflicting provider information, under-
monitoring or lack of monitoring, and wrong
161
medication received. These DRPs were devel-
oped and categorized from published literature,
specifically a taxonomy developed by Basger and
colleagues in 2015.22 There were some modifica-
tions made to this classification system given the
outpatient nature of the study as well as the
patient population. Table 1 displays the types of
DRPs identified during the pharmacist encoun-
ter, the definition of each DRP, and an example
of each.
The goal of the model was to predict patients
who were likely to have six or more DRPs, thus
identifying a subset of high-risk patients on
whom the transplant pharmacists could focus
their efforts. The delineation cut-off of six or
more DRPs was decided upon based on the fre-
quency of these events identified in the study as
it correlated to the expected number of patients
that a transplant pharmacist could feasibly
encounter during a standard clinic day. An aver-
age clinic day serves 40–50 transplant recipients.
Based on the level of care provided by the trans-
plant pharmacist, the investigators determined
that one pharmacist would be capable of con-
ducting a thorough medication history session
for 8–10 patients, or 20% of patients. Of the
patients included in the study, 20% had six or
more DRPs. Thus, the cutoff of six DRPs pro-
vided a threshold that would both be impactful
from a medication intervention standpoint as
well as feasible from a workflow standpoint for
the pharmacist.
Statistical Analysis
Baseline demographics were compared
between groups by using the t test or Mann–
Whitney U test for continuous variables,
whereas categorical variables were analyzed by
using the Pearson v2 test. After initial analysis of
the data to assess distribution, normality, and
colinearity, predictive modeling was conducted
by using binary logistic regression. Initially, all
variables gathered from the self-administered
instrument were entered into the model as
covariates, with the primary outcome being six
or more DRPs identified at the encounter. Back-
ward elimination was conducted to remove vari-
ables that were not predictive (p>0.2). As
variables were removed, sequential fully nested
iterative models were assessed for goodness of
fit by using the Hosmer–Lemeshow test and area
under the receiver operating characteristic area
under the curve (AUC) to develop parsimonious
models with the best predictive abilities. Internal
162 PHARMACOTHERAPY Volume 37, Number 2, 2017
Table 1. Drug-Related Problems, Definitions, and Examples
Drug-Related Problem Definition Example
Nonadherence Intentionally or unintentionally taking a Patient taking mycophenolate mofetil 500 mg
medication in a manner differently than twice/day rather than 1000 mg twice/day
was prescribed
Overdosing or Overdosing or underdosing based on renal Valganciclovir dosed inappropriately for
underdosing function, vital signs, or time from transplantation patient’s renal function
Duplication of Multiple concurrent medications with the same Patient taking both metoprolol and carvedilol
therapy mechanisms of action or that serve the same because two separate providers had each
pharmacologic purpose prescribed a b-blocker
Preventable adverse Adverse drug reaction occurred due to either Patient experienced a cough after lisinopril
drug reaction inaccurate or incomplete medication instructions had been restarted at an outside facility
or a known allergy or intolerance after it had been discontinued in clinic
Missing medication Discrepancies (either omissions or erroneous Antihypertensive added by an outside
erroneous medication additions) between patient’s reported medications provider that had not been added to the
and the medications recorded on the electronic electronic medication list.
medication record Sodium bicarbonate had been discontinued
by the provider but was still listed as “active”
on the electronic medical record
Conflicting provider Multiple providers providing conflicting Patient instructed to discontinue
information pharmacotherapy plans to the patient sulfamethoxazole-trimethoprim by one
physician and told to continue the drug
by another
Undermonitoring or Scheduled laboratory tests or other monitoring Undermonitoring of scheduled viral loads
lack of monitoring parameters were omitted (cytomegalovirus and BK virus)
Received wrong An incorrect medication was dispensed from the Patient was provided the wrong strength of
medication outpatient pharmacy furosemide

validation of models was conducted using boot-
strapping, with 1000 iterations to assess for the
robustness and bias of estimates. A 2-sided p-
value of <0.05 was deemed statistically signifi-
cant. SPSS statistical software, version 23, was
used for the analyses (IBM Corp., Armonk, NY).
included, the majority received tacrolimus as
their baseline calcineurin inhibitor (95%) and a
mycophenolate product as their baseline adjunc-
tive agent (93%); all patients were maintained
on a minimum daily dose of 5 mg of pred-
nisone.

Results Patient Survey Results

Baseline Demographic and Transplant
Characteristics
From September 16, 2015, to November 30,
2015, 570 chronic renal transplant recipients
were scheduled to be seen in the clinic. Of
these, 237 patients were enrolled in the study.
Seven patients who were initially enrolled in the
study either could not or chose not to complete
the survey and were thus excluded from further
analysis. Table 2 displays the baseline character-
istics of the study cohort. The median age was
55 years (interquartile range 47–63), with 51%
being male; 57% of patients included were Afri-
can American. A medical history of hypertension
was present in 93% of patients, and a history of
diabetes mellitus was present in 42% of patients.
The majority received grafts from deceased
donors, and an interleukin-2 antagonist was
used for induction immunosuppression in 55%
of transplant recipients. Of the 237 patients
Notable findings of the self-administered sur-
vey included a high percentage of unemployed
patients (76%) and those who relied on Social
Security Disability Insurance as their source of
income (58%). When patients were asked to
assess their current health status and medication
burden, 78% of patients reported good, very
good, or excellent health status, and 67% of
patients reported taking eight or more medica-
tions/day. Patients generally endorsed adherence
to and understanding of their medication regi-
men, with 84% of patients reporting complete
adherence and 85% of patients reporting good
or very good medication understanding.
Drug-Related Problems
The DRPs identified during this study are
shown in Table 3. A total of 865 DRPs were
recorded, with a mean rate of 3.6 DRPs/patient.
The most common DRPs reported were
 PREDICTIVE MODELING IN RENAL TRANSPLANT Covert et al 163
Table 2. Baseline Demographic and Transplant Character- Table 3. Drug-Related Problems Reported by Pharmacists
istics of the 237 Study Patients at Outpatient Clinic Visits
Characteristics Data No. (%)
Demographic characteristics of Patients
Recipient age at time of 55 (47–63) No. (%) of with the
visit (yrs), median (IQR) Drug-Related Drug-Related
Female 115 (49) Problems Problem
Race Problem Type (n=865) (n=237)
White 94 (40) Nonadherence 138 (16) 86 (36)
African American 134 (57) Overdosing or underdosing 27 (3) 24 (10)
History of hypertension 221 (93) Duplication of therapy 34 (4) 25 (11)
History of diabetes mellitus 99 (42) Missing medication 182 (21) 100 (42)
Medicaid 65 (27) Erroneous medication 330 (38) 149 (63)
Highest level of education Preventable adverse drug 1 (0.1) 1 (0.004)
Less than high school 24 (10) reaction due to known
High school diploma or GED 96 (41) allergy or intolerance
Some college 111 (47) Preventable adverse drug 1 (0.1) 1 (0.004)
SSDI as income source 82 (35) reaction due to incomplete
Transplant characteristics or inaccurate medication
Retransplant 21 (9) instructions
Type of transplant Conflicting information 39 (5) 31 (13)
Deceased donor 180 (76) from providers
Living donor 34 (14) Undermonitoring or lack 111 (13) 58 (36)
Multiple organs 23 (10) of monitoring
Extended-criteria donor 20 (8) Received wrong medication 2 (0.2) 2 (1)
Delayed graft function 46 (19)
Induction agent
IL-2 antagonist 131 (55) Table 4. Distribution of Drug-Related Problems
rATG 80 (34)
Alemtuzumab 5 (2) No. of No. of
Unknown 17 (7) Drug-Related Patients Percentage Cumulative
Other 4 (2) Problems (n=237) of Patients Percentage
Cold ischemic time (hrs), 15 (10–22) 0 24 10.1 10.1
median (IQR) 1 33 13.9 24.1
Years from transplant to 1.96 (0.76–4.14) 2 41 17.3 41.4
survey administration, median (IQR) 3 37 15.6 57
Data are No. (%) of patients unless otherwise specified. 4 33 13.9 70.9
IQR = interquartile range; GED = general education development; 5 20 8.4 79.3
SSDI = Social Security Disability Insurance; IL-2 = interleukin-2; 6 12 5.1 84.4
rATG = rabbit antithymocyte globulin. 7 15 6.3 90.7
8 4 1.7 92.4
9 5 2.1 94.5
erroneous medications, accounting for 38% of 10 8 3.4 97.9
> 10 5 2.1 100
all DRPs, with 63% of patients being identified
as having an erroneous medication listed on
their electronic medical record medication list. When comparing the two groups’ medication
Other common DRPs identified in the study information, more patients in the high-DRP
included a missing medication, nonadherence, group used Medicaid for prescription insurance
and undermonitoring or lack of monitoring. The (39% vs 24%, p=0.025). In addition, signifi-
distribution of the number of DRPs in our study cantly more patients in the low-DRP group had
population is displayed in Table 4. diabetes that was diet controlled (62% vs 43%,
A comparison of survey results for patients p=0.022), whereas more patients in the high-
with fewer than six DRPs (low-DRP group) ver- DRP group required the use of insulin (41% vs
sus patients with six or more DRPs (high-DRP 27%, p=0.022). Finally, medication adherence
group) is displayed in Table 5. More patients in was lower and medication burden typically
the low-DRP group reported being currently higher in the high-DRP group. More patients in
employed than in the high-DRP group (22% vs the high-DRP group endorsed difficulty affording
12%, p=0.007). When rating health status, the medications (33% vs 19%, p=0.028) and diffi-
low-DRP group had a significantly higher rating culty obtaining medications from the pharmacy
of current health status than did the high-DRP (31% vs 15%, p=0.011). The high-DRP group
group (p=0.007). also reported medication administration had a
164 PHARMACOTHERAPY Volume 37, Number 2, 2017
Table 5. Survey Results Stratified by Number of Drug-Related Problems
< 6 Drug-Related ≥ 6 Drug-Related
Survey Question Problems (n=188) Problems (n=49) p-Valuea
Patient demographics
Recipient age (yrs), median (IQR) 57 (47–63) 60 (46–64) 0.428
Race
White 75 (40) 15 (31) 0.371
African American 103 (55) 29 (59)
History of hypertension 163 (87) 37 (76) 0.080
History of diabetes mellitus prior to transplantation
No 110 (59) 27 (55) 0.947
Yes, and I used insulin 50 (27) 15 (31)
Yes, and I used oral medications 13 (7) 3 (6)
Yes, and I used insulin and oral 2 (1) 0 (0)
medications
Yes, but it was controlled without 5 (3) 1 (2)
medication
Highest level of education
Less than high school 20 (11) 4 (8) 0.894
High school diploma or GED 76 (40) 20 (41)
Some college 88 (47) 23 (47)
Currently employed 42 (22) 6 (12) 0.007
Source of income (select all that apply)b
Currently employed 33 (18) 6 (12) 0.272
Disability 105 (56) 32 (65) 0.113
Retirement 54 (29) 13 (27) 0.487
Spouse 25 (13) 5 (10) 0.808
Other 8 (4) 2 (4) 0.668
Rating of current health status
Excellent 16 (9) 4 (8) 0.007
Very good 67 (36) 9 (18)
Good 2 (1) 20 (41)
Fair 71 (38) 11 (22)
Poor 27 (14) 2 (4)
Medication and prescription information
Prescription insurance (select all that apply)b
Medicare 143 (76) 32 (65) 0.151
Medicaid 46 (24) 19 (39) 0.025
Private 80 (43) 14 (29) 0.07
Use of coupon card or discounted 35 (19) 8 (16) 0.836
copay
Number of medications daily scheduled medications
2–4 8 (4) 0 (0) 0.512
5–7 49 (26) 14 (29)
8–10 60 (32) 16 (33)
> 10 68 (36) 16 (33)
How many times per day do you take medications?
2 Times 134 (71) 37 (76) 0.335
3 Times 18 (10) 6 (12)
4 Times 13 (7) 1 (2)
5 or More times 8 (4) 1 (2)
Number of antihypertensive medications taken per day
0 35 (19) 8 (16) 0.407
1 63 (34) 14 (29)
2 53 (28) 12 (24)
3 20 (11) 7 (14)
4 6 (3) 2 (4)
5 0 (0) 0 (0)
≥6 1 (0) 2 (4)
Do you currently use insulin or medications to help control your blood sugar?
No 116 (62) 21 (43) 0.022
Yes, and I use insulin 51 (27) 20 (41)

(continued)
 PREDICTIVE MODELING IN RENAL TRANSPLANT Covert et al 165
Table 5 (continued)

< 6 Drug-Related ≥ 6 Drug-Related

Survey Question Problems (n=188) Problems (n=49) p-Valuea
Yes, and I use oral medications 9 (5) 2 (4)
Yes, and I use both insulin and oral 2 (1) 3 (6)
medications
Medication understanding and adherence
Number of times/wk a dose of scheduled medication is missed
Never 160 (85) 36 (73) 0.032
1–2 Times weekly 19 (10) 9 (18)
3–4 Times weekly 0 (0) 0 (0)
5–6 Times weekly 0 (0) 0 (0)
≥ 7 Times weekly 0 (0) 1 (2)
There are times when you cannot 36 (19) 16 (33) 0.028
afford your medications
You have difficulty obtaining medications 28 (15) 15 (31) 0.011
from the pharmacy
How well do you understand what your
medications are for?
Not well 1 (0) 2 (4) 0.083
Moderately well 13 (7) 7 (14)
Well 48 (26) 12 (24)
Very well 116 (62) 23 (47)
How confident are you in managing your transplant medications?
Not confident 2 (1) 0 (0) 0.366
Moderately confident 3 (2) 3 (6)
Confident 46 (24) 13 (27)
Very confident 124 (66) 29 (59)
A friend or family member helps you manage 61 (32) 15 (31) 1
your transplant medications
You have difficulty reading or understanding 7 (4) 5 (10) 0.069
instructions on your medications
Taking medications prevents you from doing 29 (15) 14 (29) 0.024
the things you want to do in life
Data are No. (%) of patients unless otherwise specified. Patients were allowed to skip any questions they chose not to answer.
IQR = interquartile range; GED = general education development.
a
Values in bold are statistically significant with p<0.05.
b
Percentages may total > 100% because patients had the opportunity to select more than one response.

more negative impact on their daily lives (29%
vs 15%, p=0.024) than did the low-DRP group.
Predictive Models
Figures 1 and 2 display the AUCs for the two
predictive models generated from the results of
this study. Figure 1 is a model that includes 12
variables identified as risk factors for six or
more DRPs. The variables included in this
model were age, lack of or unknown baseline
calcineurin inhibitor, use of Social Security Dis-
ability Insurance as income, use of Medicaid as
prescription insurance, poor health status rating,
medication burden, current antidiabetic regimen,
antihypertensive medication burden, and medi-
cation affordability, adherence, and understand-
ing. This model has an AUC of 0.724 and a
61.5% and 66.3% sensitivity and specificity,
respectively. The Nagelker R2 for the model was
0.189, and the Hosmer–Lemeshow test p value
was 0.444, suggesting model robustness. Fur-
thermore, the omnibus test of model coefficients
was 0.027, suggesting statistically significant fit.
Figure 2 is a simplified model that includes nine
variables. The variables included in the simpli-
fied model are age, Medicaid for prescription
insurance, current employment status, medica-
tion affordability, difficulty or lack of difficulty
obtaining medications from the pharmacy, medi-
cation administration negatively impacting
patient quality of life, medication nonadherence,
poor current health status rating, and moderate
or poor medication understanding. This model
has an area under the ROC curve of 0.720 and a
sensitivity and specificity of 62.5% and 66.7%,
respectively. The Nagelker R2, Lemeshow test p
value, and omnibus test of model coefficients for
the simplified model are similar to those of the
full model. Table 6 displays the variables
included in the final nine-variable model. Only
use of Medicaid for prescription insurance was
166 PHARMACOTHERAPY Volume 37, Number 2, 2017
Figure 1. Area under the receiver operating characteristic
(ROC) curve for the full predictive model for patients at
risk of six or more drug-related problems. The following
variables were included in the model: lack of or unknown
baseline maintenance calcineurin inhibitor, use of Social
Security Disability Insurance as income, use of Medicaid
for prescription insurance, poor rating of current health
status, number of scheduled medications taken per day,
currently receiving an antidiabetic drug regimen, number
of antihypertensive medications, and patient rating of
medication affordability, adherence, and understanding.
CI = confidence interval.
found to be statistically significant (odds ratio
3.145, 95% confidence interval 1.390–7.112,
p=0.006). Variables that were deemed not pre-
dictive (p>0.2) included race, education, use of
a coupon card for prescriptions, social support
with medication management, and antihyperten-
sive medication burden.
Discussion
Our study demonstrates that a straightfor-
ward, 5-minute survey administered to kidney
transplant recipients before their clinic visit may
be capable of predicting those at highest risk of
DRPs. This tool has the potential to be used as a
screening method for transplant pharmacists’
interventions and could positively impact clinic
workflow.
The variables assessed in the survey were cho-
sen based on risk factors for DRPs that had been
previously established in the literature. A 2001
report noted a lack of medication regimen
knowledge, high medication burden, and
advanced age (> 60 yrs old) as risk factors for
medication errors.23 Specifically, patients aged
Figure 2. Area under the receiver operating characteristic
(ROC) curve for the simplified predictive model for
patients at risk of six or more drug-related problems. The
following variables were included in the model: age,
Medicaid for prescription insurance, current employment
status, medication affordability, difficulty or lack of
difficulty obtaining medications from the pharmacy,
medication administration negatively impacting patient
quality of life, medication nonadherence, poor rating of
current health status, and moderate or poor medication
understanding. CI = confidence interval.
70–79 years old had the highest incidence of
medication errors (15.1%), and the 60–69-year-
old age group had the second highest incidence
of errors (10%). It is important to note that the
patient medication errors included in this analy-
sis were not solely errors in transplant recipi-
ents, but the data are certainly pertinent to the
renal transplant population, as each of these
variables were found to be predictive of DRPs in
the renal transplant population studied. In addi-
tion, nonadherence has been identified consis-
tently in the literature as a risk factor for DRPs
in both transplant and nontransplant patients.19–
21
Complexity and cost of a transplant patient’s
medication regimen, education, and family and
social support have all been identified as barriers
to medication adherence.21 In our study, nonad-
herence was a predictive risk factor for DRPs in
both the full model and the simplified model. A
2007 meta-analysis also noted poor patient-per-
ceived health status and low income as risk fac-
tors for medication nonadherence, which, once
again, held true in our predictive model.24 In
addition, patient perceptions of medication
importance and medication availability have
 PREDICTIVE MODELING IN RENAL TRANSPLANT Covert et al 167
Table 6. Variables Included in the Final Predictive Model of Six or More Drug-Related Problems
95% Confidence
Risk Variable Reference Variable Odds Ratio Interval p-Valuea
1-yr increase in age 22-yr-old individual 1.011 0.98–1.044 0.484
Unemployed Currently employed 1.745 0.738–4.129 0.205
Medicaid insurance Medicare or private insurance 3.145 1.390–7.112 0.006
Inability to afford medications Can afford medications 1.885 0.978–4.503 0.153
Difficulty obtaining No difficulty obtaining medications 1.693 0.703–4.078 0.240
medications from the from the pharmacy
pharmacy
Taking medications prevents Taking medications does not prevent 1.699 0.692–4.174 0.248
you from doing the things you from doing the things you want
you want to do in life to do in life
One category worsening in Excellent rating of current health 1.407 0.915–2.165 0.120
health status (excellent, very status
good, good, fair, and poor)
Misses at least one medication/wk No missed medications/wk 1.695 0.602–4.774 0.318
Poor or moderate medication Understands medication regimen 2.379 0.839–6.749 0.103
understanding very well or well
a
Values in bold are statistically significant with p<0.05.

each been independently identified as risk fac-
tors for medication nonadherence and DRPs.25
In our survey, we specifically assessed if patients
had difficulty obtaining their posttransplantation
medications, and medication availability was a
predictive factor noted in the simplified model.
Finally, a 2012 study noted African American
race, pretransplantation diabetes mellitus, gov-
ernment-funded prescription insurance, and
delayed graft function all as significant risk fac-
tors for DRPs.6 Interestingly, government-funded
prescription insurance was the only risk factor
found to be statistically significant in our study,
whereas the other variables in the simplified
model were included based on their ability to
improve the model’s predictive value. This find-
ing is different than those in the previously pub-
lished literature but may be secondary to
variations in the patient populations studied.
There have been several predictive models
used in the literature to identify high-risk
patients. A 2010 study detailed the implementa-
tion of a questionnaire for kidney transplant
recipients to explore variables that impact
patient knowledge level.19 The instrument con-
sisted of questions regarding medications, graft
rejection, and lifestyle. There was a significant
positive correlation between length of time since
diagnosis of kidney disease and increased knowl-
edge. Furthermore, patients who had experi-
enced posttransplantation complications had
significantly lower scores on the knowledge
questionnaire. A notable difference in that study
is that the questionnaire specifically assessed
patient knowledge, whereas our study aimed to
assess and predict risk of DRPs. Furthermore,
the patient population in that study was signifi-
cantly different from the patients seen at our
facility in that they were primary white, and dia-
betic nephropathy and hypertensive nephroscle-
rosis accounted for only 11% and 15%,
respectively, of the patients’ causes of end-stage
renal disease. Our study had a high percentage
of African American patients, and diabetes and
hypertension were the most common causes of
end-stage renal disease.
Another well-known predictive model is the
Morisky Medication Adherence Scale.18 This
scale was developed to assess the predictive
validity of a medication adherence tool in low-
income, minority patients with hypertension.
After the development of the eight-item ques-
tionnaire, the authors noted that the tool had
93% sensitivity and 53% specificity at predicting
adherence. Clearly there are several key differ-
ences in the patient population that those
authors studied compared with the patients at
our center, and most notable is that thatstudy
was not conducted specifically in transplant
recipients. However, the authors did note that
social support, stress, knowledge regarding
hypertension treatment, and coping skills were
all associated with treatment adherence, which
is consistent with the available literature in the
transplant population as well as with the find-
ings of our study.
Identifying patients at high risk for DRPs is of
paramount importance, and the DRPs identified
in our study were similar to those found in the
prior published literature.26 Our study identified
multiple risk factors for DRPs, some of which
are modifiable. The nine-variable model
168 PHARMACOTHERAPY Volume 37, Number 2, 2017
identified age, use of Medicaid prescription
insurance, current employment status, medica-
tion affordability, difficulty or lack of difficulty
obtaining medications from the pharmacy, medi-
cation administration negatively impacting
patient quality of life, medication nonadherence,
poor current health status rating, and moderate
or poor medication understanding as predictive
factors for six or more DRPs. Within this model,
medication affordability, ability to obtain medi-
cations from the pharmacy, negative impact of
medications on quality of life, nonadherence,
poor health status rating, and medication under-
standing are all modifiable risk factors, and
there have been multiple studies capturing the
impact of pharmacy services on these risk fac-
tors.12, 27 As such, this study certainly supports
the presence of transplant pharmacists in the
outpatient setting to identify and intervene on
behalf of patients with six or more DRPs. How-
ever, there are several challenges to providing
effective pharmaceutical care in the ambulatory
care setting, particularly in light of the imbal-
ance between FTEs to patient care needs. A
2015 workforce survey identified that 61% of
transplant pharmacist respondents felt under-
staffed to provide coverage during all phases of
a transplant recipient’s care.15 The workflow
discrepancy between available pharmacy services
and optimal patient care also underscores the
need to identify patients at highest risk for DRPs
so that pharmacy workflow can be optimized.
This study does have several limitations that
warrant discussion. The largest limitation is the
current lack of external validation. Until exter-
nal validation is completed, this survey should
not be used in a clinic setting. A long-term goal
of the research team is to externally validate the
survey and begin using it in the chronic kidney
transplant clinic. A second limitation of this
study is the inherent variation in DRP reporting
by the nine pharmacists in the clinic during the
time of the survey. Although great measures
were taken to ensure that the research team and
participating pharmacists had clear and consis-
tent definitions of the DRPs included in the
study, interrater variability is inevitable. Further-
more, whereas the numbers of DRPs both in
total and per patient were assessed, DRP severity
was not assessed given concern for significant
interrater variability among the transplant phar-
macists. Finally, the DRP cutoff chosen in this
study was based on center-specific resources,
patient needs, and our center’s workflow chal-
lenges. As such, this cutoff may not be
appropriate for all transplant centers and should
be individualized.
Conclusion
This study demonstrates that a straightfor-
ward, nine-question patient survey may be cap-
able of accurately predicting patients at risk of
having six or more DRPs and has the potential
to guide transplant pharmacists’ workflow in the
outpatient setting. The majority of the variables
included in the survey are modifiable and can be
addressed via a pharmacist intervention. In addi-
tion, with a concordance level of nearly 75%,
the model can accurately classify patients with
six or more DRPs in three of four patients.
Although further research is needed to deter-
mine a more precise measure to estimate
patients at highest risk of DRPs, this model is a
significant step in the right direction given the
paucity of literature in the transplant population.
In addition, the research team is actively evalu-
ating the impact of this tool on longer-term out-
comes, such as hospital readmissions, emergency
department visits, and graft dysfunction.
Acknowledgments
The authors would like to thank Dr. Sara Strout,
Dr. Matthew Van Cuyk, and Dr. Ethan Sebring for
their assistance in the prospective data collection per-
iod of this study.
References
1. Koh Y, Kutty FB, Li SC. Drug-related problems in hospitalized
patients on polypharmacy: the influence of age and gender.
Ther Clin Risk Manag 2005;1(1):39–48.
2. Kohn LT, Corrigan JM, Donaldson MS. To Err is Human:
Building a Safer Health System. Washington, DC: National
Academy Press; 2001.
3. Makary MA, Daniel M. Medical error—the third leading cause
of death in the US. BMJ 2016;353:1756–1833, doi:10.1136/
bmj.i2139. Published 03 May 2016.
4. Friedman AL, Geoghegan SR, Sowers NM, Kulkarni S,
Formica RN. Medication errors in the outpatient setting:
classification and root cause analysis. Arch Surg 2007;142
(3):278–83.
5. Abbott KC, Viola RA, Agodoa LY. Hospitalized poisonings
after renal transplantation in the United States. BMC Nephrol
2002;3:10.
6. Taber DJ, Pilch NA, Bratton CF, McGIllicuddy JW, Chavin
KD, Baliga PK. Medication errors and adverse drug events in
kidney transplant recipients: incidence, risk factors, and clini-
cal outcomes. Pharmacotherapy 2012;32(12):1053–60.
7. Musgrave CR, Pilch NA, Taber DJ, et al. Improving transplant
patient safety through pharmacist discharge medication recon-
ciliation. Am J Transplant 2013;13:796–801.
8. Bates DW, Leape LL, Petrycki S. Incidence and preventability
of adverse drug events in hospitalized adults. J Gen Intern
Med 1993;8(6):289–94.
9. Benkirane RR, Abougal R, Haimeur CC, et al. Incidence of
adverse drug events and medication errors in intensive care
 PREDICTIVE MODELING IN RENAL TRANSPLANT Covert et al
units: a prospective multicenter study. J Patient Saf. 2009;5
(1):16–22.
10. Leape LL, Cullen DJ, Clapp MD, et al. Pharmacist participa-
tion on physician rounds and adverse drug events in the inten-
sive care unit. JAMA 1999;282(2):267–70.
11. Buckley MS, Harinstein LM, Clark KB, et al. Impact of clini-
cal pharmacy admission medication reconciliation program on
medication errors in “high-risk” patients. Ann Pharmacother
2013;47(12):1599–610.
12. Chisholm-Burns MA, Lee JK, Spiven CA, et al. US pharma-
cists’ effect as team members on patient care: systematic
review and meta-analysis. Med Care 2010;48:923–33.
13. Kripalani S, Roumi CL, Dalal AK, et al. Effect of a pharmacist
intervention on clinically important medication errors after
hospital discharge: a randomized trial. Ann Intern Med
2012;157(1):1–10.
14. Alloway RR, Dupuis R, Gabardi S, et al. Evolution of the role
of the transplant pharmacist on the multidisciplinary trans-
plant team. Am J Transplant 2011;11(8):1576–83.
15. Taber DJ, Pilch NA, Trofe-Clark J, Kaiser TE. A national sur-
vey assessing the current workforce of transplant pharmacists
across accredited U.S. solid organ transplant programs. Am J
Transplant 2015;15:2683–90.
16. McAdams-DeMarco MA, Grams ME, Hall EC, et al. Early hos-
pital readmission after kidney transplantation: patient and cen-
ter-level associations. Am J Transplant 2012;12:3283–8.
17. Harhay M, Lin E, Harhay MO, et al. Early rehospitalization
after kidney transplantation: assessing preventability and prog-
nosis. Am J Transplant 2013;13:3164–72.
18. Morksy DE, Ang A, Krousel-Wood M, Ward HD. Predictive
validity of a medication adherence measure in an outpatient
setting. J Clin Hypertens 2008;10(5):348–54.
19. Urstad KH, Anderson MH, Øyen O, et al. Patients’ level of
knowledge measured five days after kidney transplantation.
Clin Transplant 2011;25:646–52.
View publication stats
169
20. Vlaminck H, Maes B, Evers G, et al. Prospective study of late
consequences of subclinical non-compliance with immunosup-
pressive therapy in renal transplant patients. Am J Transplant
2005;4:1509–13.
21. Chisholm MA. Enhancing transplant patients’ adherence to
medication therapy. Clin Transplant 2002;16:30–8.
22. Basger BJ, Moles RJ, Chen TF. Development of an aggregated
system for classifying causes of drug-related problems. Ann
Pharmacother 2015;49(4):405–18.
23. Phillips J, Beam S, Brinker A, et al. Retrospective analysis of
mortalities associated with medication errors. Am J Health Syst
Pharm 2001;58:1835–41.
24. Dew MA, DiMartini AF, DeVito Dabbs A, et al. Rates and risk
factors for nonadherence to the medical regimen after solid
organ transplantation. Transplantation 2007;83(7):858–73.
25. Rajpura JR, Nayak R. Role of illness perceptions and medica-
tion beliefs on medication compliance of elderly hypertensive
cohorts. J Pharm Pract 2014;27(1):19–24.
26. Friedman AL, Geoghegan SR, Sowers NM, Kulkami S, For-
mica RN. Medication errors in the outpatient setting: classifi-
cation and root cause analysis. Arch Surg 2007;143(3):278–83.
27. Chisholm-Burns MA, Spivey CA, Garrett C, McGinty H, Mul-
loy LL. Impact of clinical pharmacy services on renal trans-
plant recipients’ adherence and outcomes. Patient Prefer
Adherence 2008;2:287–92.
Supporting Information
The following supporting information is available in the online
version of this paper:
Appendix S1. Kidney Transplant Clinic Questionnaire.