Professional Documents
Culture Documents
White Paper: From Here To Risk-Based Monitoring
White Paper: From Here To Risk-Based Monitoring
White Paper: From Here To Risk-Based Monitoring
Contents
1. Understanding Risk-Based Monitoring Solutions in Full Limitations of Previous Comparisons The Importance of Real-Time, Forward-Looking Management Suggested Framework for Comparison 2. Monitoring Each Study on its Own Terms The Starting Point: a Risk-Based Monitoring Plan The Importance of Midstudy Adjustments 3. Looking Backward: Error Detection and Correction Introduction Audits Central Statistical Monitoring Focused Continuous Central Monitoring The Old Standby: Source Data Verification 4. Monitoring in the Moment: Real-Time Quality Management Quality Metrics Site Performance Metrics Key Risk Indicators Fixed Performance Indices The Speed of Monitoring Adjustments 5. Ensuring Future Success: Error Prediction and Prevention Beyond Reactive Monitoring Error Prediction Based on On-Screen Data Checks Dynamic Performance Indices Indirect Leading Indicators 6. Planning the Right Transition for You Implementing a Solution In-House Selecting an Outsourced Solution 7. Summing Up 1 1 1 2
2
2 3
4
4 4 4 5 6
7
7 7 7 7 8
9
9 9 9 10
10
10 11
12
WHITE PAPER
WHITE PAPER
Tailoring the Approach for Each Study Looking Backward: Correcting and Detecting Past Errors Monitoring in the Moment: Real-Time Quality Management Ensuring Future Success: Error Prediction and Prevention Making the Transition: Issues in Adoption and Implementation
This paper takes the view that SDV, while extremely important, is one form of error correction and detection.
WHITE PAPER
Critical data Key Risk/Quality/Performance Indicators (KRIs/KQIs/KPIs) Potential error types and associated remedial actions and action triggers Allowable error rates I nitial SDV approach, including an algorithm for selecting subjects and data to SDV and % SDV for subject groups, data types, ICFs, study procedures, etc. The basis for adjusting the SDV approach during the study The basis for scheduling monitoring visits and adjusting the visit schedule Section 4 provides detailed information on a variety of metrics to consider when authoring a risk-based monitoring plan. For simplicity, this document will use Key Risk Indicators to denote measures that industry practices sometimes calls Key Quality Indicators or Key Performance Indicators.
Risk assessment, including risk to analysis Choice of metrics Derived from past trials? Specific to study? Flag critical fields? Set Acceptable Quality Levels/ error rates for each: Primary endpoint? Secondary endpoint? Data Field?
Adjustments during trial
Monitoring basics Frequency Intensity Monitoring Method Central (remote) Onsite Mixed SDV targeting (see separate table) Metrics Change Key Risk Indicators? Change index components and weighting? Correlate indices and components with actual quality/performance?
When a CTMS supports use of Key Risk Indicators, most monitoring approaches utilize the initially identified set of KRIs throughout a study. However, an alternative approach offers greater flexibility, allowing adjustments to KRIs during the trial based on continuous assessment of the predictive value of each indicator for actual site performance and data quality. Such adjustments may change weights for each component metric or extend to replacing one metric initially designated as a KRI with another that has proven a better predictor of site performance and data quality. To support such flexibility in KRIs, a risk-based monitoring system may incorporate a model that assesses the predictive power of various metrics, enabling changes in indices and dashboards that continually focus the study team on the activities that have proved most important to the success of the trial. WHITE PAPER
Audits
While it is important to detect errors before a regulatory submission, error detection during an audit shortly before database lock may come too late to save a study. Such an audit may produce information on site performance that is as definitive as a medical examiners determination of the cause of death. Audits are included here because a risk-based monitoring plan likely includes triggers for audits. Audits are clearly all about error detection rather than correction.
WHITE PAPER
Identify as many potential problems as possible in advance Establish processes for detecting such problems Maintain relentless vigilance for signs of any developments or events that might compromise data quality.
Rules-based analytic monitoring encapsulates experience with previous trials. Each rule reflects a previously observed pattern associated with a specific type of error. For example, a rule may flag repeated values for vital signs for a single subject or repeated values for all subjects on a single date, both highly unlikely to result from actual measurements. The value of central rules-based analytic monitoring depends on:
Completeness of the library of rules Applicability of the rules to the current study Frequency and timeliness of the rules-based analysis.
As with central statistical monitoring, the value of rules-based analytic monitoring as a tool for achieving quality goals is limited unless the analysis occurs early and often. Inclusion of rules-based monitoring in this section rests on the assumption that rules-based checks will happen frequently or continuously. Checklist 2 summarizes key elements and considerations in central monitoring.
WHITE PAPER
WHITE PAPER
Quality Metrics
The ability to set target Acceptable Quality Levels (AQL) for data related to an endpoint or for a specific field and then to track actual quality continuously during the trial provides a foundation for achieving quality goals. This approach incorporates tracking of such measures of variability of data for individual fields or for the set of fields that define a primary or secondary endpoint. Metrics of this type focus monitoring attention in areas of greatest importance to the analysis and regulatory submissions. Streaming information is ideal for effective management to achieve quality goals. If management relies on periodic reports, errors can escape detection and proliferate between reports. Table 4 summarizes metrics useful in realtime quality management as well as some important considerations involved in selecting such metrics.
If streaming, currency
If periodic, frequency
Data quality levels vs AQL by:
Time to query resolution Time from patient visit to CRF entry Protocol deviations Protocol deviation under/over reporting AEs under/over reporting (by comparison with study median) SAE queries AE queries Time to SAE reporting Enrollment rate Screenfail rate Discontinue rate
Continuous Adjustments
The ideal process for error detection and correction is continuous rather than periodic, utilizing streaming information for early detection of problems and precise planning to enable immediate response. The ability to make continuous adjustments depends on:
Specifications of clear decision criteria (triggers) in the monitoring plan Specification of specific remedial actions for foreseeable types of quality issues Availability of actionable information (as opposed to raw data or tabular reports) relating to triggers Readiness to execute predefined responses A feedback loop to track the effectiveness of responses and make further adjustments if necessary Vigilance for signs of unforeseen issues For some types of data, accumulation of sufficient information for meaningful interpretation. WHITE PAPER
WHITE PAPER
risk assessment. Risk-based monitoring based on dynamic performance indices ensures that risk-based monitoring is functioning as intended.
Consist of most predictive elements proportionally weighted Components adjusted based on actual predictivity Weights adjusted based based on actual predictivity Provide trending and predictive information
Indirect measures
Available without visiting site? Known independently of site reporting? Predictive of direct measures
10
may become available months or years after recognition of the need for additional features. Furthermore, a commercial software vendor will likely prioritize revisions based on the size and importance of customers and the number of customers requesting similar functionality. Furthermore, the transition to any new technology always brings the risk of picking the wrong product or vendor. It is common during technology transitions to invest in a product that proves less capable than expected or a vendor that makes design choices that prove suboptimal as the technology and market evolve. This argues for a thorough, deliberate selection process and a strategy of maintaining as much flexibility as possible.
Identify desired feature set Compare offerings Select commercial package Workarounds for missing features? Timeline Installation Process development Pilot project Revisions based on lessons learned Deployment Staff training Cultural transformation from 100% SDV Maintream use
Outsourced
Identify CROs with: Full understanding of risk-based approaches Appropriate technology Track record Trained staff Completed cultural transformation Ability to individualize to sponsor and study Send RFP
WHITE PAPER
11
7. Summing Up
Risk-based monitoring is an important advance with the potential to promote substantial improvements in data quality, patient safety and resource allocation. However, risk-based monitoring approaches must contend with a mindset developed in the age of paper trials. In these early days of the transition to risk-based monitoring, pharma companies, CTMS vendors and CROs are typically adopting rudimentary quality-management-based approaches that are likely to produce modest improvements over historical monitoring approaches that relied almost entirely on SDV. In many cases, the limitations of existing systems, the complexity and cost of required technology upgrades and the challenges of cultural transformation are leading to cautious approaches with modest benefits that may actually increase costs without producing substantial improvements in data quality. To reap the full benefits of risk-based monitoring, the industry must leverage the best of modern management approaches and computing technology. This requires a shift to management based on real-time metrics, precise focus on the information most critical to study success and predictive elements that allow anticipatory rather than reactive or even proactive management techniques. While larger companies navigate a tortuous migration path from 100% SDV for all trials to the right monitoring approach for each study based on its unique requirements, smaller companies can enjoy substantial competitive advantage by adopting products or services from partners that are leading the transition to effective risk-based monitoring.
Presentations to address sponsor inquiries about risk-based monitoring In-depth consultation on potential risk-based monitoring solutions Full study execution based on a proven risk-based monitoring technology and processes utilized by a staff with great experience and thorough training in risk-based monitoring techniques.
To discuss how Health Decisions can assist you with the transition to risk-based monitoring, please contact us at agile@healthdec.com.
WHITE PAPER 2013 Health Decisions. All rights reserved. All brand names are trademarks or registered trademarks of their respective company.
12