See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/369332509

Assessment of the Relationship Between Protocol Adherence, Study

Complexity and Personnel in Surgical Clinical Trials

Article in Therapeutic Innovation and Regulatory Science · March 2023

DOI: 10.1007/s43441-023-00506-4

CITATION READS

1 78

4 authors, including:

Ogheneyoma Akpoviroro Myriam Castagne

Geisinger Health System Boston University
27 PUBLICATIONS 6 CITATIONS 16 PUBLICATIONS 15 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Myriam Castagne on 28 September 2023.

The user has requested enhancement of the downloaded file.

Therapeutic Innovation & Regulatory Science
https://doi.org/10.1007/s43441-023-00506-4

ANALYTICAL REPORT

Assessment of the Relationship Between Protocol Adherence, Study

Complexity and Personnel in Surgical Clinical Trials
O. Akpoviroro1 · M. Castagne‑Charlotin1 · N. P. Joyce1 · M. A. Malikova1

Received: 10 November 2022 / Accepted: 24 February 2023

© The Author(s), under exclusive licence to The Drug Information Association, Inc 2023

Abstract
Background In order to improve quality and efficiency of surgical trials, we assessed protocols complexity and examined
whether it influenced the conduct, as measured by the number and types of deviations that occurred during the execution
phase. Knowledge of these facts and performance of research team would allow to effectively mitigate the occurrence of
deviations.
Methods Thirty-five research protocols were rated according to a previously established complexity scoring model. Statisti-
cal analyses were performed to examine associations between protocol complexity, number of protocol/informed consent
amendments vs. number/types of protocol deviations; as well as correlations with phase of the study, type of investigational
product, personnel changes/experience level were assessed.
Results Assessment of complexity score in Pearson’s correlation test with the number of protocol deviations showed weak
correlation, suggesting that other factors can influence protocol adherence. There was no correlation observed between
number of deviations and type of study by investigational product category. In examining association between protocol
deviations and number of subjects enrolled a trend was observed towards increased number of deviations once more subjects
have entered the study. The higher number of protocol deviations was associated with increased number of protocol amend-
ments (p = 0.0396), and there was no statistical significance observed between number of deviations and informed consent
amendments (p = 0.5083). There was a moderate correlation detected between increased number of protocol deviations and
total number of investigators on the study.
Conclusion Protocol adherence can be improved with effective training and retention of research coordinators, investiga-
tors and frequent internal auditing to address discrepancies and effectively implement corrective actions. Upfront training
of research personnel, with subsequent monitoring of performance metrics throughout the execution phase can reduce the
total number of protocol deviations, ensure data integrity and improve quality of research conducted. Engagement of all
stakeholders upfront, including clinical site personnel, can help develop well-designed clinical trial protocol, avoid time
consuming and costly protocol and informed consent amendments at execution phase and ensure higher quality of research
conducted, while allowing to meet objectives of the trial in a more efficient manner.

Keywords Protocol complexity · Protocol adherence · Study protocol amendments · Deviations management · Research
personnel training · Risk and change management

Introduction

For several decades, one of the most frequently noted by

* M. A. Malikova
mmalikov@bu.edu; Marina.Malikova@bmc.org
1
Department of Surgery, Chobanian and Avedisian School
of Medicine, Boston University, Boston Medical Center, 85
East Concord Street, Boston, MA 02118, USA
Food and Drug Administration (FDA) clinical trial enforce-
ment actions on FDA’s inspection observations 483 form
and warning letters have been failure to follow investiga-
tional plans and/or study protocols [1, 2]. Specifically, these
included lack of compliance with study protocol, manuals/
guidelines and/or standard operating procedures (SOPs)
that have had or had a potential to severely impact subjects’

13
Vol.:(0123456789)

safety, rights, or well-being, and/or negatively impact integ-
rity of data collected and quality of conducted trials. The
increasing number of clinical trials and their complexity
make it challenging to detect clinical safety signals and
quality issues in a timely manner [3, 4]. Studies assess-
ing protocol complexity have shown that more complex
research study protocols are associated with greater costs,
both monetarily and also in terms of staff work and effort
[4–6]. Protocols with a greater complexity have also been
associated with poorer study outcomes, especially with
regard to subject recruitment, retention, and the quality of
data subsequently collected [5, 6], and possibly, these studies
were more dependent on staff productivity and efficiency of
research conducted [5, 6].
Scientifically sound and well-designed clinical trial pro-
tocol, with input provided upfront from all stakeholders, is
essential to meet objectives of the trial and ensure validity
and quality of research performed.
Based on our experiences in conducting clinical trials,
we hypothesized that more complex protocols would lead to
a greater number of errors during the execution phase, and
we planned to examine patterns of study protocol deviations
for their occurrence and type, as a measure of such errors.
Deviations are considered to be a measure of adherence
to study protocol [7, 8]. Therefore, it seems that studies with
complex study protocol would be associated with a greater
number of deviations, due to difficulties associated with
adhering to stringent protocol requirements and/or more
elaborate procedures at the execution phase, which leads to
more chances of introducing variability.
We sought not only to test this hypothesis, but also to
examine other factors which can contribute to the lack
of study protocol adherence and lead to deviations; such
as experience of research personnel, study protocol and
informed consent amendments. We aimed to identify oppor-
tunities by which deviations could possibly be mitigated in
future studies.
Materials and Methods
Complexity of study protocols was assessed using a scor-
ing model based on parameters previously developed by
our group [9]. This complexity model was built based on a
10-tiered system of study parameters or factors, each with
a possible maximal complexity score of 2, increasing in
complexity from 0. A score of 0 is indicative of processes
and activities that are standard of care practices, a score of
1 is related to moderately complex processes, and a score
of 2 is associated with highly complex processes, such as
the requirement for study-specific training for handling and
application of investigational product, or the use of drugs or
devices that are considered to have high-risk safety profiles.
13
Therapeutic Innovation & Regulatory Science
Based on this scoring model, any single research study
could potentially have a maximum complexity score of 22
[9]. The study parameters according to this model included
the following: the number of study arms, complexity of the
informed consent process, enrollment feasibility, complexity
of subject registration and randomization, nature of investi-
gational product and complexity of its administration, length
of the treatment phase, requirement for interdepartmental
coordination, complexity of data collection, duration of the
follow-up phase, ancillary clinical tests, and ancillary ques-
tionnaires [9]. The study protocol was considered routine/
standard, simple in design if complexity score, based on
parameters described above, was ranked between 0 and 4;
and moderate, if scores were 5–13. The study protocol was
ranked high in complexity, if complexity scores were deter-
mined to be between 13 and 22.
In total, all 35 study protocols for randomized clinical
trials conducted in the department of surgery during past
10 years were reviewed and scored; these protocols included
studies from the following clinical areas: vascular/endovas-
cular surgery, wound care/podiatry, trauma/critical care,
bariatric and thoracic surgery. The phase of each study and
also the type of the investigational product were also noted.
Once complexity scores of the studies were determined, they
were further reviewed to determine the number and types
of deviations that occurred during the execution phase of
each study. Statistical analyses were performed to determine
the presence of any associations between protocol complex-
ity and the following factors: (1) the number and types of
protocol deviations, (2) the phase of the study, (3) the type
of investigational product or procedure; and (4) change of
research personnel.
In addition, an internal audit of these 35 prospective, ran-
domized, surgical clinical trials was performed. Adherence
to study protocol and compliance with current regulatory
requirements was examined based on the rate of protocol
deviations within the context of study personnel changes
(e.g., investigators, study coordinators/research nurses). To
elucidate the major causes of protocol deviations in analyzed
studies, each deviation was assigned to one of the following
categories as summarized in the Table 1.
Statistical analyses were performed to examine potential
correlations between number of protocol deviations and the
following aspects: (1) study complexity, (2) number of sub-
jects enrolled, (3) phase of clinical trial, (4) nature of inves-
tigational product (e.g., drugs–small molecule or biologic,
medical devices), (5) research personnel, and (6) number
of informed consent and study protocol amendments. Spe-
cifically, the Pearson correlation coefficient (r) was used to
examine relationship between variables. This statistical test
is the most common way of measuring a linear correlation
[10]. It is a number between (r) = –1 and (r) = 1 that meas-
ures the strength and direction of the relationship between
Therapeutic Innovation & Regulatory Science
Table 1  Number of protocol
deviations in drug/biologic and Category of protocol deviation
device clinical trials per each
category, which were used in Study visit out of window
the study protocol adherence
Missed study visit
analysis
Prohibited concomitant medication taken
Deviation related to the test article
Study procedure not performed or late completion
Eligibility criteria not met
Other
Total
two variables [10]. If Pearson correlation coefficient (r) = 0,
there is no relationship between the variables and no correla-
tion. If it is between 0 and 1, there is a positive correlation
observed, meaning when one variable changes, the other
variable changes in the same direction. If this coefficient is
between 0 and –1, the correlation is negative, and when one
variable changes, the other variable changes in the opposite
direction. For instance, when Pearson correlation coefficient
(r) > 0.5 strong positive correlation between examined vari-
ables exists, if (r) is between 0.3 and 0.5—moderate; and
between 0 and 0.3 positive correlation is observed [10, 11].
Results
We have analyzed 35 prospective, surgical clinical trials with
simple, parallel group randomizations, which were based
on interactive voice response systems (IVRS) or web-ena-
bled platform for randomization procedures. As shown on
Fig. 1A, examined portfolio consisted of phase 2 (N = 6),
phase 3 (N = 18) and phase 4 studies (N = 11); with major-
ity been industry-sponsored (N = 26) (Fig. 1B). Based on
previously described study protocol complexity model [9]
studies were ranked as high (N = 10), moderate (N = 20) in
complexity, and routine/standard or simple studies (N = 5)
as illustrated on Fig. 1C. Further assessment of complex-
ity score in Pearson’s correlation test with the number of
protocol deviations showed weak correlation between these
two variables (Pearson correlation coefficient, r = 0.01295,
p = 0.9411) (Fig. 2A). Also, the weak correlation was
observed between complexity score of study protocol and
the number of subjects enrolled in the trial (r = 0.06760,
p = 0.6996 (Fig. 2B). In examining association between pro-
tocol deviations and number of subjects enrolled a trend was
observed towards increased number of deviations once more
subjects have entered the study, but the correlation between
these two variables was also weak (r = 0.28311, p = 0.0994)
(Fig. 2C).
A descriptive analysis of deviations was based on
our experiences in these surgical clinical trials, and it
Number of deviations in Number of deviations
drug/biologic clinical trials in device clinical trials
70 111
20 113
2 3
17 8
174 199
49 25
41 62
373 521
demonstrated that the most common causes for protocol non-
compliance in both drug and device studies were missed
visits, visits out of window, and study procedures not per-
formed or performed late, which was consistent between
different studies (Fig. 3). Closer assessment of the category
“study procedures not performed or performed late” showed
that protocol deviations are most frequently associated with
photography (e.g., assessing outcomes by capturing pho-
tos of the wounds procedural deviations and/or photos not
taken), functioning of investigational drug or device ser-
vices, laboratory testing, and other protocol required proce-
dures such as obtaining quality of life questionnaires.
We also used Wilcoxon rank-sum test to examine pro-
tocol deviations (Fig. 4A) and protocol complexity scores
(Fig. 4B) per investigational product categories (e.g., drug or
medical device), which demonstrated that statistically signif-
icant difference was observed in complexity scores between
drug and device trials (p = 0.0109). There was no correlation
observed between number of deviations and type of study
by investigational product category (e.g., drug versus device
trials) (p = 0.5143).
Distribution of Wilcoxon scores in Kruskal–Wallis test
for study protocol complexity scores per phase of clinical
trials throughout development lifecycle of biomedical prod-
ucts (e.g., earlier phases to post-marketing surveillance) was
assessed (Fig. 5), and Chi-square test showed no statistical
significance in complexity of conducted studies by phase
(p = 0.2018). Slightly higher number of protocol deviations
was noted for earlier phases of clinical trials (Median = 19.5
for phase 2 studies) as compared to later phases of clinical
development (Median = 9.0 and 11.0 for phases 3 and 4 stud-
ies, respectively). However, these results were not statisti-
cally significant (p = 0.7446).
It was demonstrated previously by Krafcik B.M. and
Malikova M.A. that onboarding of new research personnel
consistently showed an increase in the number of deviations
per active patient within the following months, which gener-
ally tapered off over time [7]. As demonstrated by Taekman
et al., departures from study protocol (e.g., deviations, vio-
lations) during clinical trial conduct were associated with
13

Figure 1  Number of studies analyzed by phase (A), source of funding (B), and complexity of the protocol (C).
the performance “learning curve” of investigators and per-
sonnel on the study, especially at the beginning of recruit-
ment and with higher, rapid rate of enrollment [12]. Authors
emphasized that recognition of the “learning curves” in
medical skill acquisition has enhanced patient safety through
improved training techniques [12].
Assessment of study protocol deviations in connection
with research personnel, involved in clinical trials conduct
(i.e., number of research coordinators, number of investi-
gators) showed slightly moderate correlation (r = 0.38797,
p = 0.0213) between increased number of protocol devia-
tions and total number of investigators, which were involved
in the study from start up to close out phase (Fig. 6B).
In Person correlation test, there was no association found
between number of protocol deviations and total number of
study coordinators involved in the clinical trial (r = 0.16982,
p = 0.3294) (Fig. 6A). Spearman test showed no correlation
between experience of study coordinators and number of
protocol deviations (r = 0.01780, p = 0.9192), which points
out to efficiencies of upfront training programs implemented
for research coordinators at our organization to ensure pro-
tocol adherence and improve quality of research conducted
[13]. This was a very important initiative in order to address
13
Therapeutic Innovation & Regulatory Science
personnel turnover and support business continuity within
our organization for clinical research conducted.
In addition, we utilized Kruskal–Wallis test to examine
number of protocol deviations and principal investigator
experience, which demonstrated there was no statistically
significant difference observed between number of devia-
tions and investigator experience (p = 0.9603). The analy-
sis variable “investigator experience” in this test for prin-
cipal investigator experience (Median = 180 months) was
144 months for lower quartile (N = 16 trials) and 240 months
(N = 19 trials) for upper quartile which informed us that we
are dealing with relatively experienced and mature group of
investigators in our practice. Change of principal investigator
in relationship to protocol deviations was also not statisti-
cally significant (p = 0.6035). In Spearman correlation test
there was a moderate association observed between total
number of investigators, involved in the study and number
of protocol deviations (r = 0.38361, p = 0.0229).
Noteworthy, Spearman test also revealed moderate
association between total number of subjects enrolled and
increased number of study protocol deviations (r = 0.44885,
p = 0.0068), which intuitively can be explained by the
fact that once more subjects were enrolled the degree of
Therapeutic Innovation & Regulatory Science
Figure 2  Assessment of complexity score in Pearson correlation test with the number of protocol deviations (A) and number of subjects enrolled
in the trial (B); and examining correlation between protocol deviations and number of subjects enrolled.
variability increased with regards to their adherence to
study protocol and procedures, in addition to study person-
nel effects.
Lastly, the relationship between study protocol and
informed consent amendments versus number of protocol
deviations was explored by applying Kruskal–Wallis test.
It revealed that higher number of protocol deviations was
associated with increased number of protocol amendments
(p = 0.0396) (Fig. 7A), and there was no statistical signifi-
cance observed between number of deviations and informed
consent amendments (p = 0.5083) (Fig. 7B).
Discussion
Well-designed study protocol, careful planning of study con-
duct with adequate oversight, training of research personnel
on study protocol/procedures and role-based training can
serve as a solid foundation to improve adherence to study
protocol, ensure data quality, integrity of results and human
subjects’ protection, especially in the era of digitalization
and heavy reliance on remote monitoring/auditing of qual-
ity and data integrity of clinical trials [14]. The level of
detail in the protocol should be specific enough to achieve
consistency across different participating sites during the
execution phase, but also provide flexibility when needed.
The timing of assessments/study procedures and stringency
of eligibility criteria are a delicate balance to be achieved
between asking for too little versus too much. If these crite-
ria and precise requirements are too strict, it can cause trial
results not reflect “real world” clinical practice, as well as it
can trigger too many protocol deviations, which in turn can
affect data integrity and validity of results. Providing appro-
priate windows around dates/times of assessments would
allow for flexibility, while still providing scientifically valid
data. Additionally, if there are sponsor/CRO/vendor’s pref-
erences with regards to methods/approaches to procedures/
techniques, these should be addressed upfront and processes/
methodology should be clearly stated and synchronized
between all stakeholders involved in the study protocol,
SOPs, study manuals, guidelines, instead of being implied,
or assumed.
According to recently conducted research [15–18], con-
ducting pre-launch clinical trial simulations [19] with site
investigators, coordinators and patients as part of early
engagement with key stakeholders is critical to identify
13

Figure 3  Analysis of the categories of protocol deviations per test articles: drug (A) and device (B) clinical trials.
site-specific processes and procedures that can improve
adherence to study protocol and procedures during study
execution [15–19].
In addition, engagement of all stakeholders upfront,
including input provided by site personnel and patients’
feedback as potential study participants [17–19], can help
develop well-designed clinical trial protocol, avoid time
consuming and costly protocol and informed consent
amendments at execution phase and ensure higher quality
of research conducted, while allowing to meet objectives of
the trial in a more efficient manner.
13
Therapeutic Innovation & Regulatory Science
Despite best efforts and intentions of everyone involved in
the conduct of a clinical trial, protocol deviations still occur.
As part of building quality assurance (QA) and risk-based
quality management (RBQM) systems into clinical research
operations, we recommend that personnel changes should
be addressed upfront by providing study protocol, GCP
and role-based training to develop core competencies in the
conduct of clinical research [13]. There is a learning curve,
when people just start working on a given clinical trial.
Therefore, close monitoring of new personnel’s conduct
in regards to protocol deviations by study managers and/
Therapeutic Innovation & Regulatory Science
Figure. 4  Distribution of mean Wilcoxon scores for protocol deviations (A) and protocol complexity score (B) per trial category in terms of
investigational product been tested: 1- drugs, 2- medical devices.
Figure 5  Distribution of mean Wilcoxon scores for study protocol
complexity scores per phase of clinical trials: 2- phase 2, earlier in
development trials; 3- phase 3, large comparative trials; and 4- phase
or more experienced personnel at the site level is advised.
In addition, changes of personnel (i.e., adding new investi-
gators to the existing study, changing principal investiga-
tors, transitions to new or additional study coordinators/
research nurses, etc.) can introduce variability as different
people may interpret study documentation slightly differ-
ently, have different tolerance level for risks involved, and/
or perform study related procedures in a different way. In
case if pattern of deviations is identified for a specific indi-
vidual, refresher training on a study protocol/procedures,
study manuals, guidelines should be offered at execution
phase promptly in order to avoid repetitive occurrence and
effectively implement corrective/preventive actions (CAPA)
to ensure compliance with study protocol, organizational
policies/procedures and current regulations.
4, post-marketing surveillance studies. Note: phase 1, first in human
studies were not present in the analyzed data set.
We also recommend that a trial-specific protocol devia-
tions handling plan (PDHP) will be developed prior to
the first patient enrollment. The PDHP should be part of
a broader quality by design (QbD) risk management plan
for a particular clinical trial to ensure consistency across
all systems and stakeholders involved in the study [13]. It
should describe an approach for preventing, detecting, tack-
ing, classifying and managing protocol deviations. Such
plan should be reviewed and updated as new information
becomes available (i.e., safety profile of drug/device under
study changes, protocol and/or informed consent amend-
ments are issued, patterns of protocol deviations are detected
at several sites, etc.) [13] All key personnel involved in the
trial at the sponsor, CRO, third party vendors and site levels
should be trained on these plans upfront. Ongoing, timely
13

Figure 6  Assessment of study protocol deviations in connection with research personnel, involved in clinical trials conduct: A number of
research coordinators, B number of investigators.
Figure 7  Distribution of mean Wilcoxon scores for protocol devia-
tions and study protocol amendments A in binary analysis with
Median = 4.0; cut off categories labeled as “1” if number of protocol
amendments was > 4.0 and “0” if number of amendments per study
and periodic oversight should be provided at execution phase
to ensure adherence and to implement mitigation strategies
to minimize negative impact on subjects’ safety and data
integrity.
Principal Investigator (PI) must personally conduct the
study and provide an oversight for all activities performed
within given study protocol, which includes among other
responsibilities review of screening data with formal, docu-
mented approval for enrollment, assessment of diagnostic
reports and all adverse events with assignment of causality
[20]. The PI also must review all protocol deviations and
determine the impact on human subject protection, safety,
data quality and integrity [20].
Real-time remote or on-site monitoring of study data and
regulatory documentation through electronic data capture
13
Therapeutic Innovation & Regulatory Science
protocol was ≤ 4.0; B scores distribution for protocol deviations and
informed consent amendments in binary analysis with Median = 4.0;
cut off categories labeled as “1” > 4.0 and “0” ≤ 4.0 amendments per
informed consent.
(EDC) systems, clinical trial management systems (CTMS)
and electronic trial management files (eTMF) can confirm
adherence to study protocol, manuals, guidelines, GCP, and
regulatory requirements. Risk-based management dash-
boards and signal detection tools allow more closely moni-
tor performance of the study teams, assess more proactively
protocol deviations in real-time, and improve quality of key
study data, as well as timely implementation of corrective
and preventive issues to ensure more effectively subjects’
safety and data integrity. Appropriate and timely reporting
of protocol deviations to sponsors, IRB/IEC, and if neces-
sary regulatory authorities, relies on the effectiveness of
protocol deviations handling plan and sensitivity of signal
detection tools. It is essential to understand the lifecycle of
protocol deviations and develop best practices for classifying
Therapeutic Innovation & Regulatory Science
and managing them in PDHPs. Conducting effective and
regular training of site personnel should be placed on a fore-
front of clinical research, as well as engaging them early
in the development of study protocol and learning about
site-specific processes, procedures and policies are key to
improving adherence to study protocol and meeting other
sponsor’s expectations at execution phase.
Establishing ongoing quality improvement systems and
analyzing lessons learned with respect to protocol deviations
will allow to build and implement more robust risk manage-
ment strategies, protect human subjects, preserve integrity of
the data and improve quality of clinical research conducted.
Limitations
Although we collected and analyzed data over 10 year’s
period of time, we realized that our observations and find-
ings are gathered at only one academic clinical site, which
may have unique characteristics attributable to our research
practices/processes and personnel at a safety-net, inner city
hospital.
Furthermore, our study populations consist of people
who are often socio-economically disadvantaged, which
can affect their ability to access healthcare systems, as well
as participation in clinical trials and adherence to all study
protocol procedures and requirements without contributing
to increased rate of study protocol deviations. This aspect is
very important to be investigated further to create equitable
and inclusive, diverse environment in clinical research con-
ducted. Furthermore, understanding of these patient popu-
lations’ specific needs, individual burdens, improving their
access to clinical trials with innovative products, creating
more patient-centric solutions to ensure adherence to study
protocol, will preserve data integrity and improve partici-
pants’ outcomes on these studies.
Therefore, larger prospective studies are needed in differ-
ent research settings (e.g., academic versus tertiary hospi-
tals, urban versus more rural, etc.) to examine unique charac-
teristics at more sites, elucidate further observed trends and
examine relationships between different variables to improve
conduct of clinical trials, ensure data quality and integrity.
Conclusions
Preparation for potential difficulties in prospective trials can
lead to earlier mitigation of risks, quality improvement in
data obtained, and increased efficiency of studies conducted.
Further research is needed to investigate quality and fre-
quency of training for research personnel needed to decrease
number of protocol deviations, improve protocol compli-
ance and preserve data integrity. Potentially, these aspects
can be improved with more effective training and retention
of research coordinators, investigators and more frequent
internal auditing to address any discrepancies and increase
study protocol adherence. Close supervision, upfront train-
ing of research personnel with subsequent monitoring of
performance metrics throughout the execution of the study,
especially with study protocol and informed consent amend-
ments implementation, can reduce the total number of pro-
tocol deviations, ensure data integrity and improve quality
of research conducted.
Author Contributions
Study concepts and design; Guarantor of integrity of the entire study:
MMA. Data collection; Literature research: AO, C-CM, JNP. Statistical
analysis and interpretation: MMA. With significant help and expertise
of Sing Chau Ng acknowledged. Manuscript preparation: MMA, AO.
Manuscript editing: MMA.
Funding
This is unfunded quality improvement project.
Data availability
Data were collected as part of ongoing quality assurance and improve-
ment process for conducted clinical research projects.
Declarations
Conflict of interest
The author have no conflicts of interest to declares.
References
1. Bioresearch Monitoring Program Information (BIMO) Inspec-
tions Metrics. Available from https://​www.​fda.​gov/​scien​ce-​resea​
rch/c​ linic​ al-t​ rials-a​ nd-h​ uman-s​ ubjec​ t-p​ rotec​ tion/b​ imo-i​ nspec​ tion-​
metri​cs. Updated Sept 29th, 2022. Retrieved October 28th, 2022.
2. US Food and Drug Administration. Inspections, compliance,
enforcement, and criminal investigations: enforcement actions,
warning letters. Available from https://​w ww.​f da.​g ov/​i nspe​
ctions-​compl​iance-​enfor​cement-​and-​crimi​nal-​inves​tigat​ions/​
compl​iance-​actio​ns-​and-​activ​ities/​warni​ng-​lette​rs Updated Oct
26th, 2022.
3. Koneswarakantha B, Ménard T, Rolo D, et al. Harnessing the
power of quality assurance data: Can we use statistical modeling
for quality risk assessment of clinical trials? Ther Innov Regul Sci.
2020;54:1227–35. https://​doi.​org/​10.​1007/​s43441-​020-​00147-x.
4. Getz KA, Campo RA. New benchmarks characterizing growth in
protocol design complexity. Ther Innov Regul Sci. 2018;52(1):22–
8. https://​doi.​org/​10.​1177/​21684​79017​713039.
5. Getz KA, Wenger J, Campo RA, Seguine ES, Kaitin KI. Assessing
the impact of protocol design changes on clinical trial perfor-
mance. Am J Ther. 2008;15(5):450–7. https://​doi.​org/​10.​1097/​
MJT.​0b013​e3181​6b9027.
13

6. Morin DJ. Use of proxy variables to determine the impact of pro-
tocol complexity on clinical research site productivity. Ther Innov
Regul Sci. 2019;53(1):52–8. https://​doi.​org/​10.​1177/​21684​79018​
769290.
7. Krafcik BM, Malikova MA. Implementation of quality and risk
management strategies in wound care trials. Int J Clin Trials.
2016;3(3):89–97. https://​doi.​org/​10.​18203/​2349-​3259.​ijct2​01627​
91.
8. Mehra M, Kurpanek K, Petrizzo M, et al. The lifecycle and
management of protocol deviations. Therap Innov Regul Sci.
2014;48(6):762–77.
9. Malikova MA. Optimization of protocol design: A path to effi-
cient, lower cost clinical trial execution. Futur Sci OA. 2016.
https://​doi.​org/​10.​4155/​fso.​15.​89.
10. Hotelling H. New light on the correlation coefficient and its trans-
forms. J R Stat Soc B (Methodol). 1953;15(2):193–232. https://​
doi.​org/​10.​1111/j.​2517-​6161.​1953.​tb001​35.x.
11. Rousseau, R., Egghe, L., Guns, R. Chapter 4: Statistics. In Becom-
ing metric-wise: A bibliometric guide for researchers. Chandos
Publishing, 2018, 67-97. ISBN 9780081024744. https://​doi.​org/​
10.​1016/​B978-0-​08-​102474-​4.​00004-2.
12. Taekman JM, Stafford-Smith M, Velazquez EJ, et al. Departures
from the protocol during conduct of a clinical trial: A pattern from
the data record consistent with a learning curve. BMJ Qual Saf.
2010;19:405–10.
13. Malikova MA. Building quality assurance (QA) and risk-based
quality management (RBQM) systems into clinical research oper-
ations: An academic site perspective. J Clin Trials Regul Affairs.
2017;1(1):1–8.
13
View publication stats
Therapeutic Innovation & Regulatory Science
14. Koneswarakantha B, Ménard T. Statistical modeling for qual-
ity risk assessment of clinical trials. Ther Innov Regul Sci.
2022;56:433–41. https://​doi.​org/​10.​1007/​s43441-​022-​00388-y.
15. Price A, Albarqouni L, Kirkpatrick J, et al. Patient and public
involvement in the design of clinical trials: An overview of sys-
tematic reviews. J Eval Clin Pract. 2018;24:240–53. https://​doi.​
org/​10.​1111/​jep.​12805.
16. Planner C, Bower P, Donnelly A, et al. Trials need participants
but not their feedback? A scoping review of published papers
on the measurement of participant experience of taking part
in clinical trials. Trials. 2019;20:38. https://​doi.​org/​10.​1186/​
s13063-​019-​3444-y.
17. Sine S, de Bruin A, Getz K. Patient engagement initiatives in clini-
cal trials: Recent trends and implications. Ther Innov Regul Sci.
2021;55:1059–65. https://​doi.​org/​10.​1007/​s43441-​021-​00306-8.
18. Stergiopoulos S, Michaels DL, Kunz BL, et al. Measuring the
impact of patient engagement and patient centricity in clinical
research and development. Ther Innov Regul Sci. 2020;54:103–
16. https://​doi.​org/​10.​1007/​s43441-​019-​00034-0.
19. Lim S, McKinnell D, Pierson M, et al. THU0355 clinical trial
simulation identifies factors for patient engagement in systemic
lupus erythematosus and lupus nephritis trials in African. Am Ann
Rheumatic Dis. 2016;75:315.
20. ICH GCP (2011) Title 21 CFR 312.56(b), Subpart D. Responsi-
bilities of sponsors and investigators: part 312 investigational new
drug application.
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.