IMPROVING PHARMACY

STUDENT COMMUNICATION
SKILLS THROUGH
DELIBERATE PRACTICE OF
A SIMULATED DRUG
INFORMATION REQUEST
(RESEARCH IN PROGRESS)
Divya Desai, PharmD
Regulatory Pharmaceutical Fellow in Drug Information
Purdue University | Eli Lilly and Company | FDA
November 4, 2021
ABOUT ME
 Overview of deliberate practice theory,
Background application of theoretical framework to
the research project

OUTLINE Methods Overview of the study conducted

Qualitative
Feedback from pharmacy students
Results
BACKGROUND
OVERVIEW
Drug Information Assignment

• Students enrolled in PHRM 848 “Principles of Drug Information and

Literature Evaluation” receive a phone-mediated simulated drug
information request with the main objective of collecting relevant patient-
specific background information to accurately answer the drug information
question posed.

Original Feedback

• Students in previous years requested a practice verbal drug information

request
• Adding a practice drug information question formally requested by the
curriculum committee review of PHRM 848 in the spring of 2021
RESEARCH TEAM
THEORETICAL FRAMEWORK
Deliberate Practice Theory
• Deliberate practice is not mere mindless repetition of
a certain task, but a focused approach to training
aimed at reaching a well-defined goal based on
incremental improvement through:
• (a) repetitive performance of intended cognitive or
psychomotor skills
• (b) rigorous skills assessment
• (c) specific informative feedback
• (d) better skills performance
Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, van der Vleuten CP, Scherpbier AJ. BMC Med Educ. 2011;11:101. Published 2011 Dec 6. doi:10.1186/1472-6920-11-101
THEORETICAL FRAMEWORK

Why do I look “good” in the mirror, but “bad” on camera?

http://make-happy.blogspot.com/2012/03/87366-mirror-image.html
THEORETICAL FRAMEWORK

Chamunyonga C, Edwards C, Caldwell P, Rutledge P, Burbery J. Therapy. J Med Imaging Radiat Sci. 2018;49(3):237-242. doi:10.1016/j.jmir.2018.04.024
The scientific study of expert levels of performance: General implications for optimal learning and creativity by K. A. Ericsson in High Ability Studies, 9, p. 90. Copyright 1998 by European Council for High Ability.
THEORETICAL FRAMEWORK
In medical education, deliberate practice theory has been successfully applied in the area of
simulation, where learners have unique opportunities for repeated practice of clinical skills and
integration of both reflection and feedback:
• Practicing history-taking skills with a standardized patient
• Taking repeated stabs at peripheral-line placement on a plastic arm
• Resuscitating a simulation mannequin more than once
• Reviewing one’s own performance on video to augment reflection and facilitate feedback

However, there is limited literature evaluating the use of deliberate practice theory to
enhance drug information skills

Could this framework be applied to improving communication

skills in pharmacy school?

Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, van der Vleuten CP, Scherpbier AJ. BMC Med Educ. 2011;11:101. Published 2011 Dec 6. doi:10.1186/1472-6920-11-101
Mitchell SA, Boyer TJ. Deliberate Practice in Medical Simulation. In: StatPearls. Treasure Island (FL): StatPearls Publishing; May 10, 2021.
APPLICATION OF FRAMEWORK
What specific skills do we want to improve in pharmacy students?

Communication
ASHP &
Guidelines: Inter-
Pharmacist’s professional
AACP role in drug Collaboration
Entrustable information
Professional
Activities

Haines ST, Pittenger AL, Stolte SK, et al. Core Entrustable Professional Activities for New Pharmacy Graduates. Am J Pharm Educ. 2017;81(1):S2. doi:10.5688/ajpe811S2
Ghaibi S., Ipema H., Gabay M. ASHP guidelines on the pharmacist's role in providing drug information. Am J Health-Syst Pharm. 2015; 72:573–7.
APPLICATION OF FRAMEWORK
Steps of Deliberate Practice Theory Applied to Research
• Repetitive performance of intended cognitive or
psychomotor skills
• Practice simulated drug information request in lab
• Rigorous skills assessment
• Assessment utilizing standardized rubric
• Specific informative feedback
• Formative verbal feedback and self-reflection
• Better skills performance

Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, van der Vleuten CP, Scherpbier AJ. BMC Med Educ. 2011;11:101. Published 2011 Dec 6. doi:10.1186/1472-6920-11-101
Chamunyonga C, Edwards C, Caldwell P, Rutledge P, Burbery J. Therapy. J Med Imaging Radiat Sci. 2018;49(3):237-242. doi:10.1016/j.jmir.2018.04.024
METHODS
ULTIMATE QUESTION

Research Question

• Does using the learning theory of

deliberate practice for a simulated drug
information request improve pharmacy
student communication skills?
AIMS OF RESEARCH

Specific Aims

• Assess student perceptions of the

incorporation of deliberate practice for a
simulated drug information request
• Assess student performance following
incorporation of deliberate practice for a
simulated drug information request
TIMELINE

Survey published
Week of 9/6/21 Week of 9/13/2
9/21/21

A simulated drug information

An identical simulated assignment
assignment was administered to All students enrolled in PHRM
(but with a different case question)
students enrolled in PHRM 84000 84800 received an e-mail from the
was administered to students
“Pharmacy Skills Lab” in which they course TA (not affiliated with the
enrolled in PHRM 84800 “Principles
practiced retrieving health-related research) with a direct hyperlink to
of Drug Information and Literature
information through a practice drug an electronic survey instrument
Evaluation”. Students were
information request. Students were designed to collect information
evaluated using the same
given verbal feedback by volunteer regarding student perceptions of the
standardized rubric and a formal
practitioners trained to give deliberate practice of the simulated
grade was assigned as part of the
standardized feedback based on a drug information question.
course.
rubric.
Please circle (or highlight) the box that corresponds to the student’s level of performance for each competency.
SIMULATED REQUEST
Hi this is Dr. [say your name] and I need to know the starting dose of nabilone.

Background: The requester is the patient’s primary care physician wanting to prescribe
nabilone but is unclear of the appropriate dose.

Required Background Questions:

What is the indication for nabilone?
How old is the patient?
What is the weight of the patient?
What is the status of the patient’s renal and liver function?
What other medications is the patient taking now?
What are the doses of the medication the patient is taking?
Does the patient take any OTC or herbal medications?
What other disease states/medical problems does the patient have?
Does the patient have any allergies?
LOGISTICS
• In order to effectively implement this project a number of
volunteers were needed to formulate drug information
questions and act as the “caller”
• Eight questions created for the practice and graded
assignments
• Eleven instructors volunteered to become “callers”
for the practice drug information question
• Twenty-one instructors and TA’s volunteered to
become “callers” for the graded assignment

• The “Professional Skills Laboratory” was utilized for

the practice assignment, where students would enter
break out rooms and receive their practice call.
• Due to time constraints and a concurrent activity in
lab, each student was limited to 7 minutes to perform
their background information collection and receive
feedback.
INCLUSION CRITERIA AND INCENTIVES
Inclusion Criteria

• Purdue University P2 pharmacy students enrolled in PHRM 848 “Principles of Drug

Information and Literature Evaluation” and PHRM 840 “Professional Skills Laboratory”

Study Design

• A 23-question retrospective pre-post survey was administered to students in which they

rated their confidence in their own drug information and communication skills as well as
their satisfaction with the practice assessment. Grades from the practice assessment and
the graded assessment were collected to measure student performance.

Incentives

• Students who participated in the survey received 5 points of extra credit on the graded drug
information request.
DATA ANALYSIS
Retrospective Pre-Post Survey Instrument

• Student perceptions assessed at three time points (before practice request, after practice
request, and after graded request) utilizing a Likert scale to evaluate opinion statements
(Strongly Disagree, Disagree, Neither Agree Nor Disagree, Agree, and Strongly Agree)

Grade Analysis

• Comparison of the average score following the deliberate practice to the average rubric
score following the graded assignment

Cross-cohort grade analysis

• Comparison of the average rubric score for the graded assignment 2021 to grades
from 2020 and 2019 (i.e., previous cohorts that did not experience the deliberate practice
assignment intervention). All grades will be reported in aggregate using descriptive
statistics.
 STUDENT PERCEPTION SURVEY
Part 1: Student Perception of Drug Information and Communication Skills
Questions 1-5 AACP Entrustable Professional Activities (EPA) and ASHP Guidelines regarding the pharmacist’s
role in providing drug information (application of the systematic approach):
Survey Question Question Framework
1. I am confident in my ability to communicate AACP EPA - Interprofessional Team Member Domain: Collaborate as a member
with other health care professionals. of an interprofessional team
2. I am confident in my ability to actively listen to
other health care professionals. ASHP Drug Information Provision Guidelines
Anticipate and evaluate the DI needs of patients and health care professionals.
3. I am confident in my ability to determine if a AACP EPA Patient Care Provider Domain:
drug information request is related to a specific Collect information to identify a patient’s medication-related problems and health-
patient. related needs
4. I am confident in my ability to identify the true ASHP Drug Information Provision Guidelines
question being asked and the information need Define the true question and information need. Identify the true question and
of a drug information request. information needed by asking probing questions of the requestor.
5. I am confident in my ability to collect relevant ASHP Drug Information Provision Guidelines
patient-specific information from other health Obtain complete background information. Obtain more complete background
care professionals when receiving a drug information, including examining the medical record for patient data, if applicable,
information request. to individualize the response to meet the requestor’s need
Haines ST, Pittenger AL, Stolte SK, et al. Core Entrustable Professional Activities for New Pharmacy Graduates. Am J Pharm Educ. 2017;81(1):S2. doi:10.5688/ajpe811S2
Ghaibi S., Ipema H., Gabay M. ASHP guidelines on the pharmacist's role in providing drug information. Am J Health-Syst Pharm. 2015; 72:573–7.
 STUDENT PERCEPTION SURVEY
Part 2: Student Satisfaction with Practice Drug Information Request
Questions 6-11 outline student’s satisfaction with the practice assessment using the five foundational elements
necessary for instructors and learners to accomplish deliberate practice:
Survey Question Deliberate Practice Theory Foundational Element
6. The practice exercise motivated me to perform well on the Motivate the learners
graded assignment
7. The practice exercise improved my proficiency in my verbal Provide clearly defined learning objectives for specific tasks
communication skills .
8. The practice exercise improved my grade on the verbal drug Define precise, measurable metrics of performance
information request assignment
9. The practice exercise effectively mimicked the graded Engage in focused, repetitive practice of skills
assignment
10. After the practice exercise I was Deliver real-time, constructive, actionable feedback.
given valuable constructive feedback regarding my
communication skills
11. The practice exercise helped me to identify my Deliver real-time, constructive, actionable feedback.
communication strengths and weaknesses

Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, van der Vleuten CP, Scherpbier AJ. BMC Med Educ. 2011;11:101. Published 2011 Dec 6. doi:10.1186/1472-6920-11-101
 STUDENT PERCEPTION SURVEY
Part 3: Student Satisfaction with Practice Drug Information Request
Questions 12-16 outline student’s satisfaction with the practice assessment using a validated open-source
National League for Nursing (NLN) tool covering simulation (a 5-item instrument designed to measure student
satisfaction (five items) with the simulation activity. Reliability tested using Cronbach's alpha: satisfaction=0.94)
Survey Question NLN Tool Questions
12. The practice assignment was helpful and effective at The teaching methods used in this simulation were helpful
increasing my confidence in my communication skills and effective
13. The practice assignment promoted my overall learning The simulation provided me with a variety of learning
experience materials and activities to promote my learning the medical
surgical curriculum
14. I enjoyed how the practice assignment was I enjoyed how my instructor taught the simulation
conducted
15. The practice assignment motivated me to perform well The teaching materials used in this simulation were
on the graded assignment motivating and helped me to learn
16. The way the instructors conducted the practice The way my instructor(s) taught the simulation was suitable
assignment was suitable to the way I learn to the way I learn
Adamson, K.A., Kardong-Edgren, S., & Willhaus, J. An Updated Review of Published Simulation Evaluation Instruments. Clinical Simulation in Nursing, 2013;9:393-400.
STUDENT PERCEPTION SURVEY
Part 4: Student Demographics
To determine elements of confounding that may have affected the survey results, demographics were collected
Survey Question
Is English your first language?

What is the highest level of education you achieved prior to your admission to pharmacy
school?
Do you have prior work experience in any field of pharmacy?
Approximately, how many MONTHS of cumulative paid work experience do you have in
any field of pharmacy? (Please input a WHOLE NUMBER of months below)
Students outlined previous work experience practice area and intended area of practice
upon graduation
What improvements would you suggest for the practice exercise to make the exercise
more suitable to your style of learning?
RESULTS
 “The practice exercise was helpful in preparing me for the graded
Positive Feedback

exercise. Unfortunately I think I let my own nerves get the best of me

during the graded exercise because I knew it was being graded and I
did mess up more during the actual graded one versus the practice
one. But the practice did give me insight on how the exercise was
going to go and I believe that was extremely helpful.”

“I thought the practice exercise was absolutely perfect and gave

students the ability to make mistakes and learn from them before we
were graded on it.”

“I don’t see any improvements that are necessary, but I do like the
practice diq that we had in lab. I hope you continue giving those
practice sessions for future students.”
“I wish that they would given us written feedback so that we can have it to
further practice. I was asking my practice prescriber questions and they
said all of the feedback would be given to me written , however it was never
uploaded and I did not get a chance to write down all of the verbal
feedback that he told me.”

Constructive Criticism
“More discussion is needed after the exercise.”

“I believe having a little more time for the entire call would be beneficial
because I noticed my constructive feedback portion was a bit rushed
because my room was behind schedule. I still got valuable feedback, but I
felt like I had to rush to help the room get back on time.”

“I would've liked to have been able to do a second practice DI question just

to be sure that I was able to use the feedback I had receive from the first
correctly.”
NEXT STEPS

Analyze
Analyze grades Present
Data
survey (cohort research
Analysis
results and cross- at AACP
cohort)
THANK YOU!
• Research Series mentors • Volunteer callers
• Dr. Margie Snyder • Dr. Delaney Strong • Dr. Nashel Patel
• Dr. Molly Nichols • Regina Solomito • Dr. Peggy Huang
• Dr. Katelyn Hettinger • Brandon Short • Dr. Hyder Khatri
• Nathan Macatangay • Dr. Ashley Idusuyi
• Lab Coordinators • Naomi Walker • Dr. Peter Kokkinias
• Dr. Chelsea Baker • Halley Wilson • Dr. Michael Barnes
• Prof. Jason Reed • Dr. Steven Scott • Dr. Amy Bao
• Dr. Katie Comanici • Dr. Shannon Alexander
• Dr. James Tisdale • Dr. Mo Oginni
• Dr. Cindi Koh-Knox Sharp
• Dr. Alex Isaacs
• Dr. Evan Young
• Dr. Andrew Lee
• Dr. Anes Karic
QUESTIONS?