Clinical Trials http://ctj.sagepub.

com/

The other side of clinical trial monitoring; assuring data quality and procedural adherence
George W Williams
Clin Trials 2006 3: 530
DOI: 10.1177/1740774506073104

The online version of this article can be found at:

http://ctj.sagepub.com/content/3/6/530

Published by:

http://www.sagepublications.com

On behalf of:

The Society for Clinical Trials

Additional services and information for Clinical Trials can be found at:

Email Alerts: http://ctj.sagepub.com/cgi/alerts

Subscriptions: http://ctj.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.com/journalsPermissions.nav

Citations: http://ctj.sagepub.com/content/3/6/530.refs.html

Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011

Clinical Trials 2006; 3: 530–537 ARTICLE

The other side of clinical trial monitoring;

assuring data quality and procedural
adherence
George W Williams

Background Data monitoring can mean different things. It can mean statistical
methodologies for clinical trial monitoring, interim data analysis, monitoring for
quality control or assurance or safety reporting to regulatory agencies.
Purpose The various facets of data monitoring will be discussed and reviewed
from primarily an industry perspective.
Methods By careful attention to the design and conduct of a clinical trial, the
expense of monitoring can be markedly reduced. Careful attention should be given
to the qualifications of investigators in the selection of clinical sites and central
facilities. Site personnel must be adequately trained. The sponsor should utilize
appropriately qualified individuals to supervise the overall conduct of the trial. The
monitor should visit the investigator at the site of the investigation frequently
enough to ensure acceptable quality. The monitor is responsible for inspecting the
case report forms at regular intervals. Quality control should be applied to each
stage of data handling to ensure that all data are reliable and have been processed
correctly. The auditor will assess whether the site is being monitored in accordance
with the monitoring plan. The determination of the extent and nature of monitor-
ing should be based on considerations such as the objective, design and complex-
ity of the trial. Statistical sampling may be an acceptable method for selecting the
data to be verified. The monitor should ensure that adverse events are reported.
Study data will be monitored on an ongoing basis to ensure patient safety. The
sponsor may utilize a Data Monitoring Committee to protect the validity of a trial.
Conclusions Discussions between industry, academia and regulatory groups
regarding the optimal extent and methods for monitoring of clinical trials are
encouraged. Clinical Trials 2006; 3: 530–537. http://ctj.sagepub.com

Introduction ensuring that it is conducted, recorded and reported

Monitors are employed by sponsors to review the
conduct of clinical studies to assure that clinical
investigators abide by their obligations for the proper
conduct of clinical trials. Hence, monitoring is the act
of overseeing the progress of a clinical trial and of
in accordance with the protocol, standard operating
procedures (SOPs), good clinical practice (GCP) and
applicable regulatory requirements. The extent to
which a clinical trial is conducted according to the
protocol will have a major impact on the credibility
of the results. Careful monitoring can assist in identi-

Amgen, Inc., Thousand Oaks, California, USA

Author for correspondence: George W Williams, Amgen, Inc., One Amgen Center Drive, MS # 24-2-C, Thousand Oaks,
California 91320-1799, USA. E-mail: gewillia@amgen.com
Based on a presentation at the 10th International Symposium on Long-Term Clinical Trials, Keble College, Oxford, UK,
September 2005.

© Society for Clinical Trials 2006 10.1177/1740774506073104

Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011


fying difficulties early and minimizing their subse-
quent occurrence or recurrence. There are many
groups with monitoring responsibility – eg, the spon-
sor, the investigators, the medical monitor, the regu-
latory authorities, Institutional Review Boards (IRBs)
and Data Monitoring Committees. However, the
final accountability for the conduct of a clinical trial
is with the study sponsor and study investigators.
Instead of endorsing a single approach to study
monitoring, the International Conference on
Harmonization (ICH) E-6 [1] indicates that every
study needs an adequate monitoring/auditing plan,
appropriate to the study. Onsite monitoring can
range from the industry model (frequent, all sites)
to no onsite monitoring at all with central monitor-
ing in conjunction with procedures such as investi-
gator training, investigator meetings and extensive
written guidance to assure the appropriate conduct
of the trial in accordance with GCP [2].
We will now consider many of these aspects in
more detail and consider some of the cost implica-
tions of different approaches.
Impact of design on monitoring
Although it may be impossible to avoid all errors
and inconsistencies in data generated in clinical tri-
als, careful attention should be devoted during the
design, conduct, and analysis stages of clinical trials
to minimize these problems.The scientific integrity
of the trial and the credibility of the data from the
trial depend substantially on trial design. The com-
plexity of the protocol bears directly on the quality
of the data. By careful attention to the design and
conduct of a clinical trial, the expense of monitor-
ing can be markedly reduced [3,4].
Protocols should be realistic in the amount of
data to be collected and the ability of patients
and investigators to comply with protocol require-
ments. The quantity of data that must be collected
in a study is flexible, depending on the stage of
development of the drug and the nature of the trial
[2]. The volume of noncritical data required from a
clinical trial should be kept to a minimum. Only
important information should be collected. Too
much noncritical data only leads to confusion
rather than helping the data management and
statistical aspects of the trial and often does not
contribute substantially to addressing the main
objectives of the trial. The more data that are
collected, the more cumbersome and complicated
the case record forms become, and the greater the
likelihood of confusion and error [5].
The increasing complexity of trials includes
interactive voice response systems for randomiza-
tion assignments, imaging centers, central laborato-
ries, contract research organizations (CROs) and so
http://ctj.sagepub.com
Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011
Monitoring of clinical trials 531
on. These complexities impact the monitoring of
clinical trials.
Investigator/site selection
Careful attention should be given to the qualifica-
tions of investigators in the selection of clinical
sites and central facilities for participation in the
clinical trial. Many factors need to be considered
in the selection of investigators, including their
expertise, scientific training, ability to recruit
patients and time to devote to the trial.
It is the responsibility of the investigator and his
site personnel to ensure the adequacy of the
research facility for the conduct of the trial and the
accuracy of the collected data. All the different facil-
ities involved in the study need to be considered, for
example, laboratory areas and the pharmacy, as they
may generate study data that need to be examined.
Training and pre-investigation meetings
Site personnel must be adequately trained to con-
duct a study in order to successfully produce qual-
ity data. Training of investigators, data managers
and laboratory personnel is critical and should
include appropriate attention to guidelines for good
clinical practices. Training sessions and certification
procedures will promote standardization and mini-
mize errors. Periodic retraining and recertification is
required. It is important to document training of
trial personnel.
The monitor’s responsibilities include verifying
that the investigator receives the current investiga-
tor’s brochure, all trial documents and all trial sup-
plies required to support proper trial conduct and
compliance with regulatory requirements.
Although a visit to each site can be productive,
bringing the investigators and coordinators to a cen-
tral location can encourage the concentration and
interaction desirable to enact the protocol effectively
[4]. Topics to be included are: protocol review, the
efficacy and safety profile of the treatment, safety
monitoring standards and adverse event reporting
procedures, case report form (CRF) completion and
query resolution, GCP/ICH compliance issues,
monitoring issues/regulatory concerns, and drug
accountability. Investigators are instructed regarding
the importance of accurate/clean data, avoiding pro-
tocol violations, and keeping patients in the study.
Monitors and training of monitors
The sponsor should utilize appropriately qualified
individuals to supervise the overall conduct of the
Clinical Trials 2006; 3: 530–537
532 GW Williams
trial, to handle the data, to verify the data, to con-
duct the statistical analyses, and to prepare the trial
reports. A monitor need not be a person qualified to
diagnose and treat the disease or other condition
for the treatment under study, but review of the
study data should include a medically credentialed
individual with experience in clinical trials and the
treatment population. The factors to be considered
in determining the number of monitors and the
education, training or expertise necessary should
include: the number of investigators conducting
the study, the number and location of the facilities
in which the study is being conducted, the com-
plexity of the study and the nature of the disease
or other condition under study. Monitoring work-
shops will introduce the study to the monitors [6].
Monitoring plan
The monitoring plan is designed to ensure that the
clinical team members monitor the study in a con-
sistent manner and in accordance with principles of
Good Clinical Practice, ICH guidelines and SOPs. A
standardized, written procedure, sufficiently
detailed to cover the general aspects of clinical
investigations, may be used as a basic monitoring
plan and supplemented by more specific or addi-
tional monitoring procedures tailored to the
individual clinical investigation. The monitoring
plan is dynamic and should be revised and updated
as necessary.
Clinical site visits
As noted in the Food and Drug Administration
(FDA) guidances [6], the monitor should visit the
investigator at the site of the investigation fre-
quently enough to ensure that the facilities con-
tinue to be acceptable; the study protocol is being
followed; changes to the protocol have been
approved by the IRB; accurate, complete and cur-
rent records are being maintained; accurate, com-
plete and timely reports are being made to the
sponsor and the IRB; the investigator is carrying out
the agreed upon activities and has not delegated
them to other previously unspecified staff; and that
subjects’ rights are being protected through a thor-
ough informed consent process.
In preparation for the site visit, the monitoring
plan should be reviewed. Safety monitoring reports
from the safety reporting database or serious adverse
events reported by the site or any applicable regula-
tory agency safety reports (eg, Investigational New
Drug (IND) safety reports) since the last monitoring
visit should be reviewed. The status of outstanding
data should be assessed CRFs, clinical database
Clinical Trials 2006; 3: 530–537
Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011
discrepancy reports for unresolved data queries).
Documentation and training requirements for new
staff should be determined. Reconciliation and
discrepancy reports of investigational product ship-
ments to and from the site should be determined.
There should be follow-up with the investigators as
appropriate relative to the status of IRB/IEC
approval of the protocol/protocol amendments and
informed consent forms.
Typically, the first monitoring visit is to occur
within two weeks of the first subject’s randomization.
If no subjects are registered within the first month, a
visit should occur within four to six weeks to assist
with recruitment. Following the first monitoring
visit, routine site visits will occur approximately
every six to eight weeks. Monitoring requirements
and frequency at each clinical site may be modified.
When problems are sensed through either a site visit
or information transmitted to the sponsor, an
unplanned site visit is scheduled.
Special laboratories (exercise testing facility,
echocardiography lab) must be visited and assessed
for compliance.
Source document verification
The monitor is responsible for inspecting the case
report forms at regular intervals throughout the
study to verify adherence to the protocol; complete-
ness, accuracy, and consistency of the data; and
adherence to local regulations on the conduct of
clinical research. The monitor should have access to
subject medical records and other study-related
records needed to verify the entities on the case
report forms.
Typically, for studies conducted to support regis-
trational filings, CRF data will be source verified for
all enrolled subjects. Monitors will verify 100% for
each subject screened that an informed consent
form is on file with appropriately dated signatures
and that all pages are present. Monitors will verify
100% that all of the inclusion and exclusion crite-
ria are met for each subject enrolled into the study
and will verify primary endpoint data.
Informed consent
The clinical manager will be responsible for review-
ing the draft informed consent form (ICF). The spon-
sor must review all ICF drafts prior to submission to
the IRB, including any changes requested by the sites’
IRB and verify that the latest version of the informed
consent includes the essential elements and complies
with the sponsor’s sample informed consent form
requirements. The sponsor assures that IRB approval
has been obtained prior to the enrollment of subjects
http://ctj.sagepub.com

in the study. The sponsor assures that informed con-
sent was obtained from all subjects in the study.
Patient recruitment
In trials with a long-time scale for the accrual of sub-
jects, the rate of accrual should be monitored and, if
it falls appreciably below the projected level, the rea-
sons should be identified and remedial actions taken
in order to protect the power of the trial and address
any concerns about quality. If the rate of accrual is
above projections at specific clinical sites, it is impor-
tant to verify that eligible patients are enrolled and
that the site has capacity to maintain patients on
the study. The monitor should ensure that all with-
drawals and dropouts of enrolled subjects from the
trial are reported and explained on the CRFs.
Drug accountability
Monitors are responsible for verifying for the investi-
gational product that: 1) storage times and condi-
tions are acceptable, and that supplies are sufficient
throughout the trial; 2) investigational product is
supplied only to subjects who are eligible to receive it
and at the protocol specified doses; 3) subjects are
provided with the necessary instruction on properly
using, handling, storing and returning the investiga-
tional product; 4) receipt, use and return of the inves-
tigational product at the trial sites are controlled
and documented adequately; and 5) disposition of
unused investigational product at the trial sites com-
plies with applicable regulatory requirements and is
in accordance with the sponsor requirements.
Site visit reports and follow-up
At the conclusion of a monitoring visit, the moni-
tor should: 1) communicate deviations from the
protocol, GCP and regulatory requirements to the
investigator; 2) report evidence of detected or sus-
pected scientific misconduct immediately to the
clinical manager and the clinical quality assurance
group for recommended actions; and 3) ensure
query clarifications are made and retained by the
investigator.
In follow-up to a monitoring visit, the monitor-
ing visit report should be completed. A monitoring
report is a written report from the monitor to the
sponsor after each site visit and/or other trial
related communication according to the sponsor’s
SOPs. Reports should include a summary of what
the monitor reviewed and the monitor’s statements
concerning the significant findings/facts, devia-
tions and deficiencies, conclusions, actions taken or
http://ctj.sagepub.com
Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011
Monitoring of clinical trials 533
to be taken and/or actions recommended to secure
compliance.
The detection of fraud and misconduct is a mat-
ter of concern for everybody involved in clinical
trials. It requires careful judgement by the monitor
including scrutiny of all original documentation for
correctness. As Walsh [7] has noted, trends and pat-
terns are most easily identified, however, when all
the data from a particular center are assembled and
reviewed at the same time. It is important when
designing monitoring systems that provision is
made for this kind of statistical review.
Laboratories
If local laboratories are used, their participation in
internal and external quality control, quality assur-
ance and accreditation schemes should be evalu-
ated by the study monitors. Central laboratories are
frequently used. There should be periodic review of
documentation and results from internal and exter-
nal quality control programs implemented at each
core laboratory to ascertain compliance with stan-
dards specified in the protocol. Computer systems
and interfaces between instruments and databases
should be validated.
Internal quality control often includes replicate
values obtained from aliquots of the same laboratory
specimens or repeat readings. External quality con-
trol may involve the submission of duplicate labora-
tory specimens or resubmission of records such as
electrocardiograms. The external program may also
include the submission of known standards for
analyses or reading to detect secular trends [3].
Database cleaning and report
preparation
Quality control should be applied to each stage of
data handling to ensure that all data are reliable
and have been processed correctly. To ensure the
quality of clinical data across all subjects and sites,
a clinical data management review will be per-
formed on subject data received by the sponsor.
During this review, subject data will be checked for
consistency, omissions and any apparent discrepan-
cies. The methods apply to the accumulating data
without any unblinding of treatment allocation in
individual patients or treatment groups.
As Knatterud et al. [3] have noted, there should be
checks of the data analyses. There should be consid-
eration of independently reproducing tabulations
and statistical calculations including correct selec-
tion of variables from the analysis file, proper defi-
nition of variables and cut points and correct
formulation of transformations of original variables.
Clinical Trials 2006; 3: 530–537
534 GW Williams
Auditing
It is important to note the basic difference between
quality control and quality assurance in clinical trials.
All of the day-to-day process controls that are accom-
plished by the internal and on-site monitoring staff
and directed toward the control of the clinical study
collectively comprise the quality control of the study.
On the other hand, quality assurance involves
independent review or auditing of key processes or
pivotal studies in order to establish that the trial has
been conducted in accord with GCP, ICH and
SOP [1].
Typically, the responsibility for routinely con-
ducting source document verification (SDV) for all
patients involved in a clinical study clearly rests
with the study monitor (a quality control activity).
The auditor’s responsibility is to look at a proportion
of the source document verification that has been
undertaken to ensure that the system works and is
appropriately documented (a quality assurance
activity). The auditor will assess whether the site is
being monitored in accordance with the protocol
specific monitoring plan and applicable SOPs, if the
monitoring plan is adequate for the study and site
(the frequency of monitoring visits to support the
number of subjects enrolled), and if the monitor is
effectively monitoring and managing the site to
maintain protocol and GCP compliance [5]. Quality
assurance audits are performed during enrollment
and early in the trial so as to provide feedback to
study teams regarding potential problems with the
site, the protocol or the management of the study.
Costs and extent of monitoring and
extent of data cleaning
The determination of the extent and nature of
monitoring should be based on considerations such
as the objective, purpose, design, complexity, blind-
ing, size and endpoints of the trial. In general, there
is a need for on-site monitoring, before, during and
after the trial. However, in exceptional circum-
stances the sponsor may determine that central
monitoring in conjunction with procedures such as
investigator training and investigator meetings, and
extensive written guidance can assure appropriate
conduct of the trial in accordance with GCP [1].
In a web model of clinical trial management,
much of the study monitoring is performed elec-
tronically allowing monitoring of aspects of study
conduct in real time. Problems are identified earlier
making them easier to resolve. Auditors need con-
siderably less time at sites, reducing cost and
increasing study efficiency [8].
Statistically controlled sampling may be an accept-
able method for selecting the data to be verified. The
Clinical Trials 2006; 3: 530–537
Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011
size and composition of the sample needs to be deter-
mined. While the use of sampling may be acceptable
when conducting SDV, there may be occasions when
checking the data in totality is appropriate. For exam-
ple, if the number of patients at a center is relatively
small, it might be sensible to check all data during an
on-site visit. Alternatively, data could be checked
100% until confidence in the quality has been
gained, with subsequent sampling of the data.
The sample can be dependent on the nature and
volume of data generated. A common approach is
often to identify different categories of data, for
example, critical and non-critical data. Critical data
are those that are central to the objective of the
study, and which must be corrected. These would
generally be few in number. The non-critical data
should, of course, also be correct, but if an item of
such data is in error, it is not critical to the study
outcome. Non-critical data must be checked for a
proportion of the CRFs, for example, 25% [5].
Data validation is the cleaning of trial data after
they are entered into a computer database in order
to ensure that they attained a reasonable quality
level. While data quality is centrally important to
the success of clinical trials, perfect data are unnec-
essary and indeed, may be difficult if not impossible
to obtain. As Fong [9] has noted, it has been gener-
ally agreed that high quality data are data used to
arrive at the same conclusion as perfect data. An
acceptable data quality level is one that can be con-
sidered good enough to not materially effect the
results or conclusions. However, an acceptable data
quality level has never been defined in regulatory
guidances or the literature. Nevertheless, it is critical
to optimize data quality by identifying errors from
different sources before any conclusions are drawn
from statistical analyses.
These comments made by Clarke in a textbook
on clinical data management are relevant:
‘Defining, setting up, and applying validation is a
time-consuming and labor intensive process, so it is
important to assess the value of the effort put into
validation against the resulting improvement in the
data . . . Validation should ease when there is a fair
degree of certainty that any remaining errors will
not materially affect the results or conclusions of
any analyses or the reliability of the trial’ [10].
As Eisenstein et al. [11] have noted, data meas-
urement and capture errors (ie, errors not evident to
investigators and monitors) are more frequent than
data processing errors. Inter-observer variability in
clinical observations consistently exceeds transcrip-
tion variability (from the medical record to the
CRF). Although data capture errors are more diffi-
cult to detect and correct, they may be reduced
through education. Reducing a trial’s complexity
may be an effective strategy to reduce costs and
increase data integrity.
http://ctj.sagepub.com

Adverse event reporting
Study investigators, of course, have the front-line
responsibility for identifying potential adverse
effects experienced by study participants, adjusting
the intervention accordingly and reporting the
experience to the sponsor [12]. Regulations require
that United States FDA be promptly notified of seri-
ous and unanticipated adverse experiences with the
use of investigational articles. The monitor should
ensure that adverse events, concomitant medica-
tions and comorbidities are reported in accordance
with the protocol on the CRFs.
Monitoring for safety
Study data will be monitored on an ongoing basis
to ensure patient safety. These reviews will include
all available data on incidence of adverse events,
serious adverse events, fatalities and events leading
to withdrawal. Ongoing review of interval and/or
cumulative line listings of reported adverse experi-
ences by medical monitors and/or independent
experts are conducted.
Data monitoring committees
Continuous monitoring of blinded safety data is
the responsibility of the study team. In some cases
blinded monitoring of safety data is not sufficient,
and, in order to ensure the safety of clinical trial
subjects, it is also necessary to periodically review
clinical trial data (both efficacy and safety) in an
unblinded fashion. When it is necessary for the
investigator and sponsor to be blinded to treatment
assignments, the sponsor may utilize a committee
known as a Data Monitoring Committee (DMC) to
protect the validity of a trial and its conclusions.
The DMC members are individuals independent of
the sponsor. A data monitoring committee is
generally considered to be a group of experts in rel-
evant subject matter who review the accumulating
data from a clinical trial on an interim basis and
who make recommendations to the trial sponsor/
organizer regarding the appropriateness of trial
continuation and the need for modifications of any
protocol design or trial conduct procedures [13].
Interim monitoring of the data from clinical tri-
als is critical to ensure the safety of the individuals
participating and to maintain vigilance over the
conduct of the trial. Should, for example, the treat-
ments being tested produce some unexpected toxic
effect or an unexpectedly high rate of an antici-
pated adverse effect, the trial organizers would need
to carefully consider whether to modify the trial in
some way – reduce the dose, tighten the eligibility
http://ctj.sagepub.com
Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011
Monitoring of clinical trials 535
criteria, prohibit certain concomitant medications
or even stop the trial altogether. Another major pur-
pose of monitoring is to determine whether there is
any value to continuing the trial. If the results are
so overwhelmingly positive or negative at an
interim point in the trial that the ultimate conclu-
sion can be readily predicted, early termination of
the trial could be considered. Similarly if it has
become clear that obtaining a definitive answer to
the trial question is unlikely – for example because
the event rate is far lower than had been antici-
pated and increasing the sample size to account for
this is not feasible – then termination could be con-
sidered in this situation as well. A final important
reason for interim monitoring is to maintain the
quality of the trial conduct. If the accrual is lagging
or the number of ineligible patients is high or the
dropout and or non-compliance rate are high the
organizers will want to know this while there is still
an opportunity to make adjustments [13].
An important advantage of an independent DMC
has to do with maintaining the confidentiality of
the interim data. Awareness of interim comparisons
can damage the trial by affecting accrual rates, drop
out rates, endpoint assessment and other aspects of
trial conduct. Limiting this information to the DMC
minimizes this danger while still providing assur-
ance that the trial can be modified without intro-
ducing bias or stopped if concerns arise [13].
Stopping rules
Formal statistical methods of interim analysis to
avoid inflated type 1 errors have been developed.
These include sequential analysis, group sequential
designs and analyses possibly utilizing spending
functions, stochastic curtailment, repeated confi-
dence interval approaches, and Bayesian methods.
Formal guidelines are not typically defined for toxi-
city and accrual monitoring. Because of the com-
plexity of randomized clinical trials these statistical
procedures are intended to provide helpful guide-
lines rather than rigid rules about whether early trial
termination should occur. Recommendations about
trial termination or continuation must be based on
a global consideration of all available data from the
trial including information on primary and second-
ary efficacy measures, the frequency and severity of
adverse effects and quality of trial conduct, along
with relevant information external to the trial.
Risk assessment/pharmacovigilance/
pharmacoepidemiology
Risk assessment during product development should
be conducted in a thorough and rigorous manner.
Clinical Trials 2006; 3: 530–537
536 GW Williams
However, it is impossible to identify all safety
concerns during clinical trials. Once a product is mar-
keted, there is generally a large increase in the num-
ber of patients exposed, including those with
co-morbid conditions and those being treated with
concomitant medical products. Therefore, post-mar-
keting safety data collection and risk assessment
based on observational data are critical for evaluating
and characterizing a product’s risk profile and for
making informed decisions on risk minimization.
Pharmacovigilance includes all observational
(non-randomized) postapproval scientific and data
gathering activities relating to the detection, assess-
ment, and understanding of adverse events associ-
ated with medication exposure. This includes the
use of pharmacoepidemiologic safety studies.
Good pharmacovigilance practice is generally
based on acquiring completed data from spontaneous
adverse event reports, also know as case reports.
Spontaneous case reports of adverse events submitted
to the sponsor and the FDA, and reports from other
sources, such as the medical literature or clinical stud-
ies may individually or collectively constitute poten-
tial signals of adverse effects associated with
medications. The quality of the individual case char-
acterization is critical for appropriate causality assess-
ment between the product and the adverse event.
Signals warranting additional investigation
can be further evaluated through carefully designed
observational studies of the product’s use in the real
world. Such studies could include pharmacoepi-
demiologic safety studies, registeries, and surveys.
Pharmacoepidemiologic safety studies include
cohort studies that may be prospective or retrospec-
tive, uncontrolled or case-control or nested case-
control, and so on. Clinical trials are impractical in
almost all cases when the event rates of concern
are less common than 1:2000–3000. Because phar-
macoepidemiologic safety studies are observational
in nature, they are more subject to confounding,
effect modification, and other bias, which may
make results of these types of studies more difficult
to interpret than results of clinical trials. It is almost
always prudent to conduct more than one study, in
more than one environment and even using differ-
ent designs. Triangulation on the safety observa-
tions from more than one study helps to further
define the product safety profile. In recent years, a
number of pharmacoepidemiologic safety studies
have been conducted in automated claims data-
bases (health maintenance organization (HMO),
Medicaid) [14].
Conclusions/recommendations
When we monitor we obtain better data which
leads to better quality decisions. As Lewis [15]
Clinical Trials 2006; 3: 530–537
Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011
noted, ‘if there are alternative, less expensive and
more efficient ways than those currently used by
the pharmaceutical industry to ensure that the
quality of the trial is sufficient to support the claims
that rest upon it, then we should adopt them
widely and modify guidelines appropriately . . .
There may be some room for simplification in the
case of double-blind superiority trials using an
objective endpoint such as mortality. However, for
trials that aim to demonstrate non-inferiority the
currently recommended levels of monitoring
should be maintained. The risks arising from lower-
ing the standards would be too great, namely the
potential use of less effective products under the
mistaken belief that they are equally effective’.
As Eisenstein et al. [11] note, strategies exist that
can significantly reduce total trial costs without
compromising integrity. Through a focused
approach to identifying critical trial elements, data
collection forms can be streamlined, data manage-
ment can be more intelligent and onsite monitoring
can be reduced. As Peto and Baigent [16] note, col-
lecting less information may require larger patient
numbers but may lead to more definitive answers.
Large amounts of defensive documentation or exces-
sive audits may substantially reduce the reliability
with which therapeutic questions are answered.
Discussions between industry, academia, and
regulatory groups regarding the optimal extent and
methods for monitoring of clinical trials are
encouraged. Adjustments to the current intensity of
monitoring typical of the industry model would
be facilitated by broad leadership from regulatory
bodies.
References
1. ICH E6. Good clinical practice: consolidated guideline, Step
5 as of May 1996 (http://www.ich.org).
2. Temple R. Policy developments in regulatory approval.
Statist Med 2002; 21: 2939–48.
3. Knatterud GL, Rockhold FW, George SL et al.
Guidelines for quality assurance in multicenter trials.
Controlled Clinical Trials 1998; 19: 477–93.
4. Califf RM, Karnosh SL, Woodlief LH. Developing
systems for cost effective auditing of clinical trials.
Controlled Clinical Trials 1997; 18: 651–60.
5. Schuyl M, Engel T. A review of the source document
verification process in clinical trials. Drug Information
Journal 1995; 29: 1291–99.
6. Guidance for Industry. Guideline for the monitoring of
clinical investigations, FDA January 1998, revised
November 1998.
7. Walsh RC. Systematic measures for the prevention and
early detection of investigator fraud. Drug Information
Journal 1994; 28: 1161–65.
8. Marks RG, Conlon M, Ruberg SJ. Paradigm shifts in
clinical trials enabled by information technology. Statist
Med 2001; 20: 2683–92.
9. Fong DYT. Data management and quality assurance.
Drug Information Journal 2001; 35: 839–44.
http://ctj.sagepub.com

10. Clarke PA. Data validation pg. 189–212. In Rondel RK,
Varley SA, Webb CF eds. Clinical data management. John
Wiley & Sons, 1993.
11. Eisenstein EL, Lemons PW, Tardiff BE et al. Reducing
the costs of phase III cardiovascular clinical trials. Am
Heart J 2005; 149: 482–88.
12. Guidance for clinical trial sponsors on the establishment and
operation of clinical trial data monitoring committees,
November 2001 (www.fda.gov/cber/guidelines.htm).
Last accessed 6/20/07.
http://ctj.sagepub.com
Downloaded from ctj.sagepub.com at UNIV OF PENNSYLVANIA on September 2, 2011
Monitoring of clinical trials 537
13. O’Neill RT. Regulatory perspectives on data monitoring.
Statist Med 2002; 21: 2831–42.
14. Guidance for Industry. Good pharmaccovigilance practices
and pharmacoepidemiologic assessment, and premarketing risk
assessment. U.S. Food and Drug Administration, March
2005 (www.fda.gov/cder/guidance/). Last accessed 6/20/07.
15. Lewis JA. The European regulatory experience. Statist
Med 2002; 21: 2931–38.
16. Peto R, Baigent C. Trials: the next 50 years. Br Med J
1998; 317: 1170–71.
Clinical Trials 2006; 3: 530–537