Professional Documents
Culture Documents
V&V White Paper
V&V White Paper
Overview: The FDA requires medical device companies to verify that all the design outputs meet the design inputs. The FDA also requires that the final medical device must be validated to the user needs. When there are so many more design inputs and outputs than specific user needs, why do companies spend so little time verifying the device and so much time and money on validation? And what is the role of risk management in determining the amount of testing required? This presentation will demonstrate that if developers conduct more complete verifications of design outputs and risk mitigations, validations can be completed in a shorter time, more reliably, and more successfully.
Introduction: In our experience working with medical device manufacturers to improve their Quality Management Systems and to gain regulatory clearance of new devices, we have found that Design Verification is an often underutilized tool for ensuring success during the latter stages of device development efforts. In many cases, limited verification efforts represent a lost opportunity to significantly reduce the scope of validation efforts and thereby reduce time to market and development costs. This paper explores the use and misuse of Design Verification and how device manufactures can get the most out of their verification efforts. But first, some background . . .
Background: Medical device manufacturers have been working to align their design and development systems with the FDAs design control regulations (21 CFR 820.30) since their release in 1996. The regulations are structured to ensure that device manufacturers maintain control over their designs throughout the development process and that the marketed device is safe and effective for its intended use. At their most basic level, the regulations require that manufacturers: Clearly state what they intend to produce (Planning and Design Inputs); Develop a design that meets those needs (Design Outputs and Design Review); Confirm that the design meets the original intent (Verification); Confirm that manufactured product can be produced reliably and achieve the desired result (Transfer and Validation); and
Page 1 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
It would be hard to argue that any of these steps should be excluded from any development effort. Whether you are building a bridge, a cell phone, or a surgical tool, this combination of steps has a logical flow that most engineers would simply consider to be good practice. The challenge arises when you move from the conceptual world, where product development can be seen as single stream of water flowing down a hill: steadily moving from one point to the next without interuption; to the real world, full of rapids, eddies, branching streams, and (occasionally) deep pools of water that dont seem to be moving at all. In this turbulent environment, its can be difficult to maintain a disciplined design approach particularly when business schedules demand rapid progress and any delays in moving on to the next development phase risk affecting project milestones and development staff deployment plans. These challenges are multiplied when the device has multiple components and integration points that must be managed throughout the development process. Much has been written about the importance of early stage planning and problem solving to speed timetomarket and reduce development costs and the value of clear Design and Development Plans and welldefined Design Inputs cannot be overstated. An equal amount of attention has been focused on Design Transfer and Validation activities. At this late stage of a development effort, the project scope will have expanded to include more active involvement of Clinical, Operations and Marketing staff. In addition, the cost of validation studies and the make or break aspect of these studies require a great deal of attention and company resources. Problems at the validation stage will have serious implications for the success of the project and especially for small companies could determine the fate of the whole company. Compared to these early and latestage activities, the importance of Design Verification tends to be overlooked. There are several reasons why: Definitions: Many practitioners (and established Quality Systems) continue to have trouble defining exactly what verification is, and how it fits into the design control process. The term is often considered to be synonymous with validation and the development of combined V&V plans that do not establish a clear distinction between the objectives of the two efforts dont help. [Note: when discussing verification with the FDA, its best not to talk about your V&V plan increasingly, inspectors preferences are to address the two activities separately.]
Timeframe: Verification is often an ongoing effort conducted throughout the development of outputs. So, a great deal of time may pass between when the first and last output is verified.
Page 2 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
Approaches: There are a variety of ways to complete the verification for an output, so it can be difficult to communicate that all of these activities, as a group, represent the design verification.
The problem with this lack of attention is that it weakens a critical link in the design control chain, affecting the strength of the overall design control effort. Poor design verification can lead to problems during design transfer and validation, and can reduce your ability to track down and correct problems if (and when) they occur. Just as importantly, it may represent a lost opportunity to optimize validation activities, reduce time to market, and increase your overall confidence in the safety and efficacy of your products. Key Concepts: To make sure that were aligned on the proper use of the term verification, here are a few key points from the FDAs Design Control Guidance For Medical Device Manufacturers: Definition: Verification means confirmation by examination and provision of objective evidence that specified requirements have been fulfilled [820.3(aa)].
Types of Verification Activities: Verification activities are conducted at all stages and levels of device design. The basis of verification is a threepronged approach involving: tests, inspections, and analyses. Any approach which establishes conformance with a design input requirement is an acceptable means of verifying the design with respect to that requirement. In many cases, a variety of approaches are possible . . . the manufacturer should select and apply appropriate verification techniques based on the generally accepted practices for the technologies employed in their products.
Table 1 provides some examples of the types of verification activities that device manufacturers often employ.
Page 3 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
o o o
A key distinction between design verification and design validation activities is that verification only requires that a single unit be assessed. What constitutes that single unit will vary depending on the intent of the verification. It might be one batch of raw material (for material performance tests), one machined part (for first article inspection), on one package sample (for integrity tests). The intent of verification is to confirm that the design outputs (i.e., the materials or components specified in design documents) meet the design input requirements. Validations, on the other hand, require that studies be conducted to ensure that the device can be manufactured to meet design specifications on a consistent basis (i.e., process validation), and to ensure that the finished device is safe and effective for its intended purpose (i.e., design validation). For the process validation, multiple devices must be manufactured and evaluated to confirm that the production process is capable of producing devices within specifications on a consistent basis. For the design validation, multiple devices must be used to treat multiple patients to confirm that the treatment is safe and effective.
Page 4 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
Page 5 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
Page 6 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
Why are these failures occurring? One possibility is that by viewing verification as a regulation, device developers are losing sight of the fact that conducting verifications is simply good engineering. For example, it makes no sense to move forward with the development of a component before you are sure that the material properties meet the performance and safety requirements established in the design inputs. This meeting the regulations mindset can lead engineers to view design verification simply as a task that must be checked off by QA instead of valuable tool for building knowledge about their device. Failures occur because sometimes the checkoffs get missed. Even when all the boxes are checked, the regulation mindset can drive engineers to focus only the minimum number of verification activities needed to satisfy the design control requirements. While this approach may help satisfy shortterm budget or schedule constraints, the development tea m will lose the opportunity to learn potentially important information about the performance and behavior of its device and its components. If that learning is put off to the validation stage, the cost of failure can grow dramatically. What To Do? There are a variety of tools that developers can use to improve the effectiveness of their design verification process. In this paper we discuss three tools that can help ensure that verification activities are appropriate and complete and, if well documented, can provide regulators with sufficient confidence in the design that excessive validation can be avoided. The three tools are Traceabilty Matrix Failure Modes and Effects Analysis (FMEA) Fault Tree of Design Inputs
Traceability Matrix: The traceability (trace) matrix is a basic design control tool that all device developers should use throughout the design control process to establish clear linkages between Design Inputs, Outputs, Verifications, Validations and Risk Analyses. Too often the trace matrix is left for QA to complete just in time for the final design review. However, if it is begun as soon as Design Inputs are approved and managed throughout the development effort, the trace matrix provided an excellent roadmap guiding developers through key steps of the design control process and ensuring that required documentation is created and controlled.
Page 7 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
Table 2: Traceability Matrix at Design Inputs Stage Traceability Matrix Design Design Input Design Risk Verification Process Design Req. (Requirement) Output Analyses Validation Validation (Specification) No. 1.1.1 Can be ETO sterilized
Comments
Table 3 illustrates how a row of a trace matrix might look when being reviewed at the Final Design Review. At this point the design outputs have been approved and the requirement that the material in question is appropriate for ETO sterilization is established in Drawing No. 1.2. The matrix also identifies that identified risks associated with the device have been mitigated (in part) through the use of this material. The references to design FMEA 1.3 and process FMEA 5.4 identify two places where this material is addressed. Verification was achieved by confirming that the specified material is included in the purchasing billofmaterial (BOM 2.3). The Process Validation column identifies that production of the material was assessed in an operation qualification (OQ Study 3.5) and a performance qualification (PQ Study 4.3). Finally, it shows that a design validation (Sterilization Study 2.1) was conducted to confirm that the final manufactured device could in fact be effectively sterilized using ETO. Table 3: Traceability Matrix at Final Design Review Traceability Matrix Design Design Input Req. (Requirement) No. 1.1.1 Can be ETO sterilized
Risk Verification Process Design Analyses Validation Validation dFMEA 1.3 pFMEA 5.4 BOM 2.3 OQ Study 3.5 PQ Study 4.3
Comments
Page 8 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
Page 9 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
While risk identification and mitigation is clearly the primary objective of the risk management activity, these analyses can also be used to focus verification and validation efforts. For example, consider a device that includes two components, one affects how the device is held by the user and the other comes into contact with the patient. Both components have a similar number of physical dimensions to verify, but risks associated with the userfacing component are Broadly Acceptable, while the patient facing component risks are ALARP. While a First Article inspection of the userfacing component may be appropriate, a more thorough analysis of the patientfacing component may be warranted. This analysis could include a tolerance analysis or the inspection of multiple pieces to provide a better indication of the potential capability to produce the component within specification. While the additional efforts described above may not be required at the verification stage (conducting a First Article would be sufficient to claim that the verification is complete), the ALARP risk classification is an indication that validation of this component may be challenging. The more you learn about this component during verification, the better prepared you will be once you get to the validation. If fact, if the verification is thorough, it may eliminate the need for some aspects of validation saving time and money later in the development effort. The key point is that incomplete or minimal verifications may not provide a sufficient understanding of the likely performance of a device during validation, and excessive verification can be a waste of resources. To effective focus your verification efforts, risk analyses, and FMEAs in particular, can provide Page 10 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
While more challenging to structure and assess than an FMEA, the FTA provides a better understanding of the risks associated with the system. Since the performance of the system is the objective of validation activities, FTA is an ideal tool to prepare for validations. With the top of the fault tree representing the intended use of the device, this tool allows device developers to fill out the tree
Page 11 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept
Page 12 of 12
MEDIcept, Inc. 200 Homer Avenue Ashland, MA 01721
11/2010 May not be reprinted or copied without expressed permission from MEDIcept