Research in Social and Administrative Pharmacy xxx (xxxx) xxx

Contents lists available at ScienceDirect

Research in Social and Administrative Pharmacy

journal homepage: www.elsevier.com/locate/rsap

Use of Rapid Assessment Procedures when analyzing qualitative data in

pharmacy research
Chelsea Phillips Renfro a, Zoe Rome b, Justin Gatwood c, Kenneth C. Hohmeier c, *
a
Department Clinical Pharmacy and Translational Science, University of Tennessee Health Science Center, College of Pharmacy, Memphis, TN, USA
b
University of Tennessee Health Science Center, College of Pharmacy, Nashville, TN, USA
c
Department of Clinical Pharmacy and Translational Science, University of Tennessee Health Science Center, College of Pharmacy, Nashville, TN, USA

A R T I C L E I N F O A B S T R A C T

Keywords: Qualitative research analytics and methodology are a useful part of many research projects. However, qualitative
Qualitative research data analysis may be time intensive causing delays in results. This is especially problematic in time-sensitive
Qualitative methods projects where there an urgent need for results and a rapidly evolving situation being studied, such as during
Rapid assessment procedures
health crisis or early stages of project implementation. An emerging body of literature around the use of Rapid
Rapid qualitative assessment
Rapid turn-around qualitative research
Assessment Procedures (RAP) suggests that this method of qualitative assessment provides more efficient coding
Thematic analysis and categorizing of data without comprising rigor. The objectives of this manuscript are to: 1) describe how RAP
Coding can be used in pragmatic healthcare research studies and 2) provide an example of when RAP was applied to a
Medication therapy management qualitative research study in the healthcare setting. RAP includes 5 core features: 1) use in combination with
quantitative outcomes or process data (mixed methods approach), 2) quick timeline from start to finish (weeks to
months), 3) population of interest participation in planning and implementing the research, 4) team approach to
research process, and 5) iterative cycle of data collection and analysis. Use of RAP provides key stakeholders and
decision makers the ability to generate solutions to problems faster than ever before without compromising rigor,
a method needed now more than ever. The progression of healthcare and clinical management is moving at an
unprecedented rate, and RAP allows researchers to stay ahead by providing quicker results for better outcomes.

Introduction instruments, field observations, or content analysis on newspaper

Research analytics and methodology is an essential part in deter­
mining outcomes of any project or trial. Methodologies are often cate­
gorized at quantitative or qualitative. Quantitative results are classified
as a numerical value while qualitative results provide an open-ended,
subjective component to results collected.1,2 Additionally,
mixed-methods approaches where quantitative and qualitative data
analyses are combined are becoming popular in research conducted in
the healthcare setting.3 Qualitative research in the healthcare setting
has seen a lag in adoption due to skepticism regarding the rigor as many
researchers are used to using quantitative research methods to produce
an outcome or validity and reliability evidence.4 However, adoption of
qualitative research methods has increased as many researchers have
recognized the value of understanding participant viewpoints on ac­
tions, events, and relationships.4 While the most common examples of
qualitative methodology are conducting focus groups and interviews,
other examples include asking open-ended questions on survey
articles.5
There are several different types of in-depth, traditional qualitative
research approaches that can be used when working with qualitative
data such as content analysis, thematic analysis, grounded theory, or
phenomenology.1 However, the foundation of all qualitative analysis is
coding and categorizing the data to identify significant patterns.5 While
coding or categorizing data is one of the most important steps in the
qualitative data analysis process, it is also one of the most
time-consuming and cognitively demanding aspects of the process. The
use of software to assist with qualitative analyses can help with coding
and categorizing data, however the software does not create the cate­
gories, code transcripts, identify patterns or draw meaning from iden­
tified patterns which can be very time consuming.6 For time-dependent
research, this creates the larger problem of analyzing data rigorously
and also within the allotted time frame.7
Rapid assessment procedures (RAP) provides the innovation needed
to solve this problem. However, it is important to note that rapid should

* Corresponding author. University of Tennessee Health Science Center, College of Pharmacy, 301 S. Perimeter Park Drive, Suite 220, Nashville, TN, 37211, USA.
E-mail address: khohmeie@uthsc.edu (K.C. Hohmeier).

https://doi.org/10.1016/j.sapharm.2021.05.013
Received 21 November 2020; Received in revised form 19 May 2021; Accepted 20 May 2021
Available online 6 June 2021
1551-7411/© 2021 Published by Elsevier Inc.

Please cite this article as: Chelsea Phillips Renfro, Research in Social and Administrative Pharmacy, https://doi.org/10.1016/j.sapharm.2021.05.013
C.P. Renfro et al. Research in Social and Administrative Pharmacy xxx (xxxx) xxx

not be interpreted as rushed, but as an approach that emphasizes both
rigor and analytical efficiency. It is important to note that RAP has gone
under many different names such as rapid qualitative assessment and
rapid turn-around qualitative research, however the process is the
same.7 RAP is an evidence-based methodology defined as “team-based
qualitative inquiry using triangulation, iterative data analysis and
additional data collection to quickly develop a preliminary under­
standing of a situation from the insider’s perspective.”7 This analytic
method, when compared to traditional qualitative methods, is a rela­
tively underutilized method of decreasing turnaround time for results
and outcomes to become implemented and effect change. An example of
this would be public health situations such as the COVID-19 pandemic
where newly implemented programs need to be quickly assessed. By
using RAP, researchers can have the opportunity to make changes in real
time as results are provided in a shorter timeframe when compared to
traditional qualitative methods, potentially creating even better
outcomes.8,9
Rapid Assessment Procedures methodology
Designing a study using RAP to analyze qualitative data has common
core features that can be considered: 1) use in combination with quan­
titative outcomes or process data for a mixed methods approach, 2)
quick timeline from start to finish (weeks to months), 3) population of
interest participation in planning and implementing the research, 4)
team approach to research process (includes planning, data collection,
interpretation of findings, and presentation of results) and 5) iterative
cycle of data collection and analysis.10 It is important to note that while
RAP can be used in combination with quantitative approaches, it can be
used by itself. However, this should be considered when developing the
study methods. To perform an iterative cycle of data collection and
analysis, one analytic context that can be used is the Sort and Sift, Think
and Shift approach which requires researchers to “diving in” by reading,
reviewing, recognizing, and recording your observations during data
analysis. “Stepping back” allows you to reflect, re-strategize, and
re-orient after “diving in” analysis.11 While this iterative approach al­
lows for data to be analyzed while being collected, the same procedures
for developing and conducing focus groups or interviews would be used
as compared to traditional qualitative research. Use of the Sort and Sift,
Think and Shift approach has been described below in an example of
how RAP can be used in the pharmacy practice setting. When
approaching RAP, it is important to create an inventory of data contents.
To do this, a systematic approach should be applied to move data
collected into a templated summary.12 The process to create this sum­
mary contains five main steps, however it is important to note that this
process can be tailored (e.g., summary template and matrix can be set up
different based on research team preferences and needs) to meet the
research team’s needs and the goals of the project. In the first step, a
neutral domain name that corresponds with each interview question
should be created (Table 1). During the second step, the neutral domain
can then be used to develop a summary template for use by the research
team (Fig. 1).
A section should be included at the end for any other observations (e.
g., a participant’s tone during the interview) that do not fit into the
neutral domains as well as important quotations. Third, the summary
template should be pilot tested with members of the research team to
determine if the domains are intuitive, missing, or incorrectly labeled.
All researchers who will be using the template should practice creating a
summary using the same transcript/data to assess for consistency among
researchers. For the fourth step, the transcripts/data should be divided
across the team after establishing consistency across researchers. It is
important to note that when creating summaries, researchers should
make sure they are brief (max of 2 pages), thorough, organized, useful,
and readable. The final step is transferring the summary into a matrix to
streamline the process of noting similarities, differences, and trends.13
The matrix format should be set up based on the purpose of analysis (e.
g., by role, by site). Table 2 shows an example of how a matrix could be
organized. The matrix allows you to quickly identify content in any
domain, assess gaps in data, and develop memos or summaries based by
domain, sites, or respondents. After finishing RAP, go back to the data
and “dive in” by asking questions to identify any potential stories
beyond the identified codes. Then “step back” and reflect on connections
identified.11,12 The objectives of this manuscript are to: 1) describe how
RAP can be used in pragmatic healthcare research studies and 2) provide
an example of when RAP was applied to a qualitative research study in
the healthcare setting.
Application of Rapid Assessment Procedures in the healthcare
setting
Several studies implemented in the healthcare setting have used RAP
methods to quickly move results to make changes. One study by Solo­
mon et al. assessed HIV prevention programs within a community
mental health center.14 Rapid assessment procedures were used to
analyze data collected from interviews and focus groups. Through use of
RAP, areas for improvement in the program were able to be identified
such as the failure between case managers to maximize their relation­
ship with the patients to explain the importance of sexually transmitted
infection and HIV education to a high-risk population. Determining this
need over a few weeks versus a year long study provides an opportunity
to protect patients now as compared to them needing to receive
potentially preventable treatment in the future.
In another study by Moloney et al., RAP was used to develop case
studies and derive lessons learned from providing clinical interventions
at the beginning of the COVID-19 pandemic.15 As the world faces an
unprecedented outbreak, this study implemented rapid analytics to
progress clinical treatment at a quicker pace. Early findings from the use
of RAP found that primary and secondary prevention strategies for
COVID-19 could be incorporated into current treatment for patients who
were at greatest risk for COVID-19 exposure. COVID-19 has brought the
need for rapid qualitative analysis to the forefront of scientific research
and Moloney et al. showed how healthcare within the United States and
abroad can capitalize on it.
Application of Rapid Assessment Procedures in pharmacy
practice
Intervention overview

To support and improve primary care transformation through in­

Table 1
Example of neutral domain development.
Sample Interview Question
How well does the TennCare MTM Pilot fit with existing work
processes and practices in your setting?
To what extent might the implementation take a backseat to
other high-priority initiatives going on now?
How do you feel about the TennCare MTM Pilot being used in
your setting?
To what extent is there a strong need for the TennCare MTM
Pilot?
clusion of medication therapy management (MTM), Tennessee’s state
Medicaid program (TennCare) implemented the TennCare MTM Pilot
Domain
Program.16 This program focused on implementing a patient-centered
Compatibility approach to MTM by integrating this pilot into an overarching Tenn­
Care Primary Care Transformation initiative which includes
Relative priority
patient-centered medical homes (PCMH). The Primary Care Trans­
Beliefs formation initiative was designed to improve primary care through
proactive management and coordination of a patient’s healthcare
Tension for experience. Nested within this initiative, the MTM pilot focused on four
change
areas: 1) patient attribution through risk stratification, 2) reporting and

2
C.P. Renfro et al. Research in Social and Administrative Pharmacy xxx (xxxx) xxx

Fig. 1. Example summary template with neutral domains.

documentation using a shared care coordination platform, 3)

Table 2
value-based reimbursement, and 4) evaluation of MTM specific quality
Example matrix.
metrics. MTM services included medication therapy evaluation and
Demographics Relative Priority Tension for Change monitoring, patient and prescriber education, care plan development,
Role, years in role, Interviewee Degree to which and conduction of a comprehensive medication review. The payment
time at perception of individuals at model for this program was established on a per month case rate based
organization importance of pharmacy/PCMH
on targeted disease state or risk category.
implementation perceives status quo
within pharmacy needs to be changed
Interview Ambulatory care - Vaccines top - Competition leads
Participants
1 pharmacist; Role priority - MTM to desire for
time: 1.5 years; growth area, high differentiation from
Organization time: importance competitors Participants included the implementation team (e.g., pharmacists,
2 years physicians, nurses, support personnel, and administrators), imple­
Interview Community - MTM high priority - - Pharmacist mentation facilitators (e.g., University researchers, TennCare
2 pharmacist; Role Included in metrics experience with
time: 10 years; for the pharmacy poorly managed
personnel), and managed care organizations (i.e., medical insurance
Organization time: team program companies contracted with TennCare to administer the Medicaid pro­
3 years beneficiaries and sees gram to TennCare beneficiaries) in the state of Tennessee. The PCMH
pilot program as an served as the center of the MTM pilot as it was intended to be the central
opportunity to be
location for the patient’s entire medical and prescription record. Phar­
change agent
Interview Community - Large emphasis - Organization’s l top macists practiced either within the PCMH or as a pharmacist at an
3 pharmacist; Role placed on medication management team external but nearby community pharmacy. Of note, the program span­
Time: 2 years; sync and vaccines recent turnover ned over 500 miles across 3 distinct regions of the state (east, middle,
Organization time: over MTM leading to fear of new and west) and covered rural, urban, and suburban areas. In addition,
2 years initiatives and
protection of status
three separate managed care organizations (MCOs) administered bene­
quo fits for TennCare members. Key informants were recruited for the study
via three separate emails sent over four weeks from implementation
facilitators (TennCare and UTHSC) to all participants within the
program.

3
C.P. Renfro et al.
Methodology
To explore the adoption, feasibility, acceptability and appropriate­
ness of an implementation strategy to support the MTM pilot program in
Tennessee. RAP was used to analyze qualitative data and provide real-
time feedback on the MTM pilot program’s implementation to Tenn­
Care in order for programmatic changes to be made. One member of the
research team conducted nine in-depth, key informant interviews with
nine participants as part of an early implementation assessment. The
Consolidated Framework for Implementation Research (CFIR) provides
37 constructs organized into domains associated with effective imple­
mentation. Constructs under the inner setting, characteristics of in­
dividuals, and process were used to develop a semi-structured interview
guide17 Interviews were audiotaped and transcribed. Field notes made
during the interviews were added to the transcriptions.
To conduct RAP, researchers first created a neutral domain that
corresponded with each interview question (Table 1). Next, the research
team created a summary template to be used during data analysis
(Fig. 1). The summary template was piloted with one transcript to allow
for the research team to assess the usability, make any necessary
changes, and develop consistency among the researchers conducting the
analysis. Then, the transcripts were divided among the research team to
develop summaries. A matrix (Table 2) was used to visualize data across
participants and CFIR constructs. After completion of the analysis, the
research team stepped back to reflect and re-strategies on the findings
(Table 3). Formal RAP reports were delivered to TennCare personnel
quarterly, with informal updates presented monthly during standing
meetings.
Implications from using Rapid Assessment Procedures
The program officially started July 2018. Interviews were conducted
over the first quarter of 2019 to explore implementation processes,
characteristics of individuals implementing the program, MTM pilot
characteristics, and the inner setting or the PCMH or pharmacy. This
data was compared against quantitative data including program reach
and adoption as defined by rate of MTM claims billed per month and
quantity of National Council for Prescription Drug Programs (NCPDP)
numbers (i.e., discrete providers) billing per month.
RAP was used to provide real-time updates to the program (Table 3).
For the purposes of formal interim and annual reports, as well as data
dissemination via peer-reviewed manuscripts, traditional qualitative
data analysis was used per the sponsor organization’s request. This
approach gave the project sponsor critical feedback as often as monthly
and in real time, while still having the option to report to the legislature
and Centers for Medicare and Medicaid Services results undergoing
traditional qualitative research analysis. Neutral domains were drafted
based on the Consolidated Framework for Implementation Research,17
but was then further customized in collaboration with TennCare to
match their internal language and to further facilitate communication.
Table 3
Sample rapid assessment procedure timeline from medicaid medication therapy management pilot project.
Mar- Apr- May- Jun- Jul-
19 19 19 19 19
Implementation Stage Early Implementation
Qualitative data X X X X
collection
Oral RAP briefings X X X X
Quarterly reports on X X
overall evaluation
status using RAP
Formal Interim Reporta X X
Formal Annual Reporta
a
Use of traditional qualitative data analysis methodology.
Research in Social and Administrative Pharmacy xxx (xxxx) xxx
Meetings within the research team refined the summary template over
several meetings in early and mid-2019 to match data being collected.
Data was transferred to a matrix to better visualize the data across
multiple neutral domains and interviewees. The matrix too was
customized based on feedback within the University research team and
TennCare. From start to finish the length of time to use RAP for this
program was four months.
After an initial spike in participation through the end of 2018 and
early 2019, a decline in both reach and adoption of the program was
noted through summer of 2019. Monthly meetings between TennCare
and University personnel allowed researchers to refine their analysis
approach to meet real-time questions asked by TennCare personnel
during those critical first two quarters of 2019. Four key programmatic
changes were identified based on this rapid assessment, presented to
TennCare, and changes were implemented during the fourth quarter of
2019 through the third quarter of 2020. These changes included:
pharmacist reimbursement rate increases after a nationwide review of
pharmacist MTM fee schedules, re-sequencing of care coordination tool
training during credentialing process to increase the speed of program
enrollment, duplicate documentation workaround using existing soft­
ware limitations, and an eligibility expansion. Importantly, these bar­
riers and the corresponding supporting data were not only used by the
implementation facilitation team to make more informed decisions, but
as concrete data from which key decision makers within the Tennessee
Government could then approve those changes. Although several of
these barriers and suggested changes were noted anecdotally by stake­
holders within the project (e.g., conference calls with pharmacists,
emails between implementation facilitators), it was the formal qualita­
tive inquiry and use of RAP which provided both the adequate level of
rigor and speed of analysis to allow for programmatic improvements to
take place.
Future use of RAP
RAP offers a new tool in the arsenal of qualitative researchers, but
there remain some unanswered questions and areas for future research.
RAP may offer considerable reductions in time for analysis, but may also
increase overall time spent in write-up of results.18 The degree to which
time may be saved in data analysis varies greatly and may be a function
of both the techniques applied and the researchers themselves.18,19 This
should be of no surprise as data analysis is still heavily dependent upon
manual coding procedures – meaning that even if researchers have more
efficient procedures to code it does not guarantee that those researchers
will spend less time on coding.
In our experience, a mutual understanding of the urgency for quick
and rigorous data is key. The researchers and program sponsors which
formed the project team were aware of the need for quick turnaround of
data for program improvement and monitoring. This facilitated urgency
in data interpretation while still relying on established RAP methods to
ensure analytical integrity. It is our belief, that this mutually understood
Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun- July-
19 19 19 19 19 20 20 20 20 20 20 20
Middle Implementation
X X X X
X X X X X X X X X X X X
X X X X
X X
X
4
C.P. Renfro et al.
urgency within the team allowed for efficient data analysis and report­
ing, whereas RAP used for program assessment that is less obviously
timebound or where urgency is not established may lead to increased
analysis time.
Despite wide variations in time savings through the use of RAP, there
is strong evidence to support consistent overlap in results between RAP
and traditional qualitative research methods, such as thematic ana­
lysis.18–20 Even at this early stage in the development, use, and study of
RAP methods, there appear to some general consensus around its
application and usefulness. For instance, in health emergencies in the U.
S. and abroad where there is both an urgent need for results and a
rapidly evolving situation being studied (e.g., COVID-19, Ebola).19 As
was reported in the present paper, RAP is also useful for certain pro­
grammatic assessment – especially during early implementation periods
when there is a need to quickly and fluidly understand implementation
context from the field. Regardless of the reason to apply RAP, to make
the most of this methodology it would seem that a mutually understood
sense of urgency on the part of the research team performing the analysis
is a critical factor for its successful application.
Conclusion
The benefit of using RAP is the potential to see changes made in real
time that can effect patient healthcare immediately. We see the possible
benefits in the profession of pharmacy in many areas, including disease
prevention programming, medication therapy management and even
guideline changes for the current global pandemic. Use of RAP provides
key stakeholders and decision makers the ability to generate solutions to
problems faster than before without compromising rigor, a method
needed now in this rapidly changing healthcare environment. The pro­
gression of healthcare and clinical management is moving at an un­
precedented rate, and RAP allows researchers to stay ahead by providing
quicker results for better outcomes.
Author statement
The first author name is lead. The last author listed is senior and
corresponding. All other authors are listed in order of contributions.
Contributions to the paper are described using CRediT taxonomy.
Writing – Original Draft: CR, ZR, and KH; Writing – Review & Editing:
CR and KH; Conceptualization: CR and KH.
This is original work that has not been submitted to any other
journal, nor has any of the manuscript duplicated any material previ­
ously published. None of authors have any financial or other relation­
ships they would like to disclose that may lead to a conflict of interest.
This paper was read and approved by all authors. The research has been
approved by the University of Tennessee Health Science Center (UTHSC)
institutional review board (IRB).
Declaration of competing interest
Three authors of this publication (CR, JG and KH) received funding
Research in Social and Administrative Pharmacy xxx (xxxx) xxx
from the State of Tennessee Medicaid (TennCare) Program. The other
authors declare no other relevant conflicts of interest or financial re­
lationships. The authors alone are responsible for the content and
writing of this article.
References
1. Rosenthal M. Qualitative research methods: why, when, and how to conduct
interviews and focus groups in pharmacy research. Curr Pharm Teach Learn. 2016;8:
509–516.
2. Miles B, Huberman A, Saldana J. Qualitative Data Analysis. third ed. Thousand Oaks,
CA: Sage Publications; 2014.
3. Creswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research.
Thousand Oaks, CA: Sage Publications; 2014.
4. Castleberry A, Nolen A. Thematic analysis of qualitative research data: is it as easy as
it sounds? Curr Pharm Teach Learn. 2018;10:807–815.
5. Patton MQ. Qualitative Research and Evaluation Methods. third ed. Thousand Oaks:
Sage Publications; 2002.
6. Roberts K, Wilson R. ICT and the research process: issues around the compatibility of
technology with qualitative data analysis. Forum Qualit Sozialforschung/Forum:
Qualit Social Res. 2002;3.
7. Beebe J. Rapid Assessment Process: An Introduction. first ed. Walnut Creek, CA:
AltaMira Press; 2001.
8. Vindrola-Padros C, Chisnall G, Cooper S, et al. Carrying out rapid qualitative
research during a pandemic: emerging lessons from COVID-19. Qual Health Res.
2020 Aug 31, 1049732320951526.
9. Holdsworth LM, Safaeinili N, Winget M, et al. Adapting rapid assessment procedures
for implementation research using a team-based approach to analysis: a case
example of patient quality and safety interventions in the ICU. Implement Sci. 2020;
15.
10. McNall M, Foster-Fishman PG. Methods of rapid evaluation, assessment, and
appraisal. Am J Eval. 2007;28:151–168.
11. Maietta RC, Mihas P. Sort & sift, think and shift: let the data be your guide. In:
Presentation at Research Talk Fall Seminar Series in. NC: Chapel Hill; 2017.
12. Hamilton A, Maietta R. Rapid turn-around qualitative research. In: Presentation at
14th Annual Qualitative Research Summer Intensive in. NC: Chapel Hill; 2017.
13. Averill JB. Matrix analysis as a complementary analytic strategy in qualitative
inquiry. Qual Health Res. 2002 Jul;12(6):855–866.
14. Solomon PL, Tennille JA, Lipsitt D, Plumb E, Metzger D, Blank MB. Rapid
assessment of existing HIV prevention programming in a community mental health
center. J Prev Interv Community. 2007;33(1-2):137–151. https://doi.org/10.1300/
J005v33n01_11.
15. Moloney K, Scheuer H, Engstrom A, et al. Experiences and insights from the early US
COVID-19 epicenter: a rapid assessment procedure informed clinical ethnography
case series. Psychiatry. 2020;83(2):115–127.
16. Woeppel J, Clark R, Underwood L, et al. The Tennessee medication therapy
management program: a hybrid type 2 effectiveness-implementation trial study
protocol. Res Soc Adm Pharm. 2020;16(3):315–320. https://doi.org/10.1016/j.
sapharm.2019.05.018.
17. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering
implementation of health services research findings into practice: a consolidated
framework for advancing implementation science. Implement Sci. 2009 Dec;4(1):1–5.
18. Taylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S. Can rapid approaches to
qualitative analysis deliver timely, valid findings to clinical leaders? A mixed
methods study comparing rapid and thematic analysis. BMJ Open. 2018 Oct 1;8(10),
e019993.
19. Vindrola-Padros C, Vindrola-Padros B. Quick and dirty? A systematic review of the
use of rapid ethnographies in healthcare organisation and delivery. BMJ Qual Saf.
2018;27.
20. Johnson GA, Vindrola-Padros C. Rapid qualitative research methods during complex
health emergencies: a systematic review of the literature. Soc Sci Med. 2017 Sep 1;
189:63–75.