Technical Guidance Series (TGS)

Risk management for

manufacturers of in
1
TGS–07
vitro diagnostic
medical devices

Draft for comment 25 September 2017

© World Health Organization 2017
All rights reserved. Publications of the World Health Organization can be obtained from WHO Press, World
Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791
4857; e-mail: bookorders@who.int). Requests for permission to reproduce or translate WHO publications –
whether for sale or for non-commercial distribution – should be addressed to WHO Press, at the above address
(fax: +41 22 791 4806; e-mail: permissions@who.int).
The designations employed and the presentation of the material in this publication do not imply the expression
of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any
country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries.
Dotted lines on maps represent approximate border lines for which there may not yet be full agreement.

The mention of specific companies or of certain manufacturers’ products does not imply that they are
endorsed or recommended by the World Health Organization in preference to others of a similar nature that
are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by
initial capital letters.
All reasonable precautions have been taken by the World Health Organization to verify the information
contained in this publication. However, the published material is being distributed without warranty of any
kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the
reader. In no event shall the World Health Organization be liable for damages arising from its use.

Contact: Irena Prat, EMP Prequalification Team Diagnostics

WHO – 20 Avenue Appia – 1211 Geneva 27 Switzerland
 WHO Prequalification – Diagnostic Assessment: Technical Guidance Series
WHO The World Health Organization (WHO) Prequalification Programme is coordinated through
Prequalification the Department of Essential Medicines and Health Products. The aim of WHO
– Diagnostic
prequalification of in vitro diagnostic medical devices (IVDs) is to promote and facilitate
Assessment
access to safe, appropriate and affordable IVDs of good quality in an equitable manner.
Focus is placed on IVDs for priority diseases and their suitability for use in resource-limited
settings. The WHO Prequalification Programme undertakes a comprehensive assessment of
individual IVDs through a standardized procedure aligned with international best regulatory
practice. In addition, the WHO Prequalification Programme undertakes post-qualification
activities for IVDs to ensure their ongoing compliance with prequalification requirements.

Procurement of Products that are prequalified by WHO are eligible for procurement by United Nations
prequalified agencies. The products are then commonly purchased for use in low- and middle-income
IVDs
countries.

Prequalification IVDs prequalified by WHO are expected to be accurate, reliable and able to perform as
requirements
intended for the lifetime of the IVD under conditions likely to be experienced by a typical
user in resource-limited settings. The countries where WHO-prequalified IVDs are
procured often have minimal regulatory requirements. In addition, the use of IVDs in these
countries presents specific challenges. For instance, IVDs are often used by health care
workers who lack extensive training in laboratory techniques, in harsh environmental
conditions, without extensive pre- and post-test quality assurance (QA) capacity, and for
patients with a disease profile different from those encountered in high-income countries.
Therefore, the requirements of the WHO Prequalification Programme may be different
from the requirements of high-income countries, and/or of the regulatory authority in the
country of manufacture.

About the The Technical Guidance Series was developed following a consultation, held on 10–13
Technical March 2015 in Geneva, Switzerland, which was attended by experts from national
Guidance regulatory authorities, national reference laboratories and WHO prequalification dossier
Series
reviewers and inspectors. The guidance series is a result of the efforts of this and other
international working groups.

Audience and This guidance is intended for manufacturers interested in WHO prequalification of their
scope IVD. It applies in principle to all IVDs that are eligible for WHO prequalification for use in
WHO Member States. It should be read in conjunction with relevant international and
national standards and guidance.
The Technical Guidance Series guidance documents are freely available on the WHO
website.

3
Contents
Contents .................................................................................................................................... 4
1 Abbreviations and definitions ............................................................................................ 6
1.1 Abbreviations .................................................................................................................... 6
1.2 Definitions ......................................................................................................................... 6
1.2.1 Definitions related to risk management .................................................................. 6

1.2.2 General definitions .................................................................................................. 7

2 Introduction .................................................................................................................... 10
2.1 Standards, guidance and WHO prequalification assessment ......................................... 10
2.2 Key concepts ................................................................................................................... 11
2.3 Cautions........................................................................................................................... 12
2.3.1 Issues observed during WHO prequalification assessment .................................. 13

2.4 Risk management in a regulated environment............................................................... 13

3 Risk management process ............................................................................................... 14
3.1 Responsibilities................................................................................................................ 15
3.1.1 Top management................................................................................................... 15

3.1.2 Departments .......................................................................................................... 16

3.2 Policies and planning....................................................................................................... 18

3.3 Training for risk management ......................................................................................... 19
3.4 Risk management file ...................................................................................................... 19
3.4.1 Demonstrating regulatory compliance .................................................................. 21

4 Tools and methods .......................................................................................................... 22

4.1 FMEA ............................................................................................................................... 23
4.1.1 The start-up work .................................................................................................. 24

4.1.2 The FMEA process ................................................................................................. 26

4.1.3 Risk evaluation ....................................................................................................... 29

4.1.4 Risk grid.................................................................................................................. 30

5 Risk control options ......................................................................................................... 32

6 Risk management – selected activities ............................................................................. 33
6.1 Quality management system .......................................................................................... 33
6.2 Risk management and design control ............................................................................. 34

4
 6.2.1 Design risk assessment .......................................................................................... 37

6.3 Verification and validation .............................................................................................. 39

6.4 Analytical and clinical performance studies (design validation)..................................... 40
6.5 Change controls............................................................................................................... 41
6.5.1 Equipment change ................................................................................................. 42

6.5.2 Change of supplier of a critical component of an IVD ........................................... 42

6.5.3 Change of intended use ......................................................................................... 42

6.6 The IFU and other labelling ............................................................................................. 43

6.6.1 IFU .......................................................................................................................... 43

6.6.2 Package labelling ................................................................................................... 47

6.7 Manufacturing................................................................................................................. 48
6.7.1 Suppliers ................................................................................................................ 48

6.7.2 Manufacturing processes ...................................................................................... 52

6.7.3 Safe documentation .............................................................................................. 54

6.7.4 Process changes in manufacturing ........................................................................ 54

References ............................................................................................................................... 55

Acknowledgements

The document Risk management for manufacturers of in vitro diagnostic medical devices
was developed as part of the Bill & Melinda Gates Foundation Umbrella Grant and the
UNITAID grant for “Increased access to appropriate, quality-assured diagnostics, medical
devices and medicines for prevention, initiation and treatment of HIV/AIDS, TB and
malaria”. The first draft was prepared in collaboration with Dr Julian Duncan, London,
England, and with input and expertise from Ms Jeanette Twell. This document was
produced under the coordination and supervision of Kim Richards and Deus Mubangizi
WHO/HIS/EMP, Geneva, Switzerland.

The draft guidance was posted on the WHO website for public consultation on 25
September 2017.

5
1 Abbreviations and definitions
1.1 Abbreviations

CAPA corrective and preventive action

CLSI Clinical and Laboratory Standards Institute
FTA fault tree analysis
FMEA failure mode and effects analysis
FRACAS failure reporting and corrective action system
IFU Instructions for Use
GHTF Global Harmonization Task Force
IMDRF International Medical Device Regulators Forum
ISO International Organization for Standardization
IVD in vitro diagnostic or in vitro diagnostic device
KPI key performance indicators
QA quality assurance
QC quality control
QMS quality management system
R&D research and development
RPN risk prioritization number
RDT rapid diagnostic test
SQA supplier quality agreements
TMV test method validation
US CFR United States Code of Federal Regulations

1.2 Definitions

The definitions below related to risk management of in vitro diagnostic devices (IVDs) are
transcribed from ISO 14971:2007 Medical devices – application of risk management to
medical devices (1) and are generally used in this guidance. When a source other than ISO
14971 is used, the source is indicated.

1.2.1 Definitions related to risk management

6
 1 Harm: Physical injury or damage to the health of people, or damage to property or the
2 environment
3 Hazard: Potential source of harm
4 Hazardous situation: Circumstance in which people, property, or the environment are exposed
5 to one or more hazard(s)
6 Residual risk: Risk remaining after risk control measures have been taken
7 Risk: Combination of the probability of occurrence of harm and the severity of that
8 harm. (Note: The definition of risk in (2) is broader and more generally applicable
9 than this, and is used by preference in this guide.)
10 Risk analysis: Systematic use of available information to identify hazards and to estimate the
11 risk
12 Risk assessment: Overall process comprising a risk analysis and a risk evaluation
13 Risk control: Process in which decisions are made and measures implemented by which risks
14 are reduced to, or maintained within, specified levels
15 Risk estimation: Process used to assign values to the probability of occurrence of harm and the
16 severity of that harm
17 Risk evaluation: Process of comparing the estimated risk against given risk criteria to determine
18 the acceptability of the risk
19 Risk management: Systematic application of management policies, procedures and practices to
20 the tasks of analysing, evaluating, controlling and monitoring risk
21 Risk management plan: For the particular IVD being considered, the manufacturer shall establish
22 and document a risk management plan in accordance with the risk management
23 process
24 Severity: Measure of the possible consequences of a hazard
25 Safety: Freedom from unacceptable risk

26 1.2.2 General definitions

27 The following definitions are used throughout this guide.

28 Design input: The physical and performance requirements of an IVD that are used as a basis for
29 IVD design.
30 Source: (3), definition (f).

7
31 Evidence: Information that can be proved true based on facts obtained through
32 observation, measurement, test or other means.
33 Source: Modified from (4), definition 3.8.1.
34 Instructions for Use (IFU): Information supplied by the manufacturer to enable the safe and
35 proper use of an IVD.
36 Note: Includes the directions supplied by the manufacturer for the use,
37 maintenance, troubleshooting and disposal of an IVD, as well as warnings and
38 precautions.
39 Source: (5), definition 3.30.
40 In the United States, the acronym IFU occasionally stands for “indications for
41 use”, and the acronym IU stands for “intended use” or “indications for use”. The
42 ISO definition and requirements (5) for IFU cover the intended use and the
43 precise method of use.
44 Intended use Use for which a product, process or service is intended according to the
45 specifications, instructions and information provided by the manufacturer.
46 Source: (1), definition 2.5.
47 Note 1: The clinical use for which the procedure was designed.
48 Note 2: The concept includes definition of the measurand, the target condition
49 and the clinical use of the measurement procedure, which may include
50 screening, diagnosis, prognosis, and/or monitoring of patients. (these notes are
51 from the Clinical and Laboratory Standards Institute (CLSI) website
52 http://htd.clsi.org.)
53 WHO note: The concept includes the physical, economic and resource limitations
54 in the environments of intended use.
55 In vitro diagnostic (IVD): A medical device, whether used alone or in combination, intended by
56 the manufacturer for the in vitro examination of specimens derived from the
57 human body solely or principally to provide information for diagnostic,
58 monitoring or compatibility purposes.
59 Note 1: IVDs include reagents, calibrators, control materials, specimen
60 receptacles, software, and related instruments or apparatus or other articles and
61 are used, for example, for the following test purposes: diagnosis, aid to
62 diagnosis, screening, monitoring, predisposition, prognosis, prediction,
63 determination of physiological status.
64 Note 2: In some jurisdictions, certain IVDs may be covered by other regulations.
65 Source: (5) and definition 3.27.
66 IVD reagent: Chemical, biological or immunological components, solutions or preparations
67 intended by the manufacturer to be used as an IVD.
68 Source: (5), definition 3.28.
69 This guide uses the terms IVD and IVD reagent interchangeably.

8
 70 Life cycle: All phases in the life of a medical device, from the initial conception to final
71 decommissioning and disposal.
72 Source: (1), definition 2.7.
73 Measurand Quantity intended to be measured
74 NOTE 1 The specification of a measurand in laboratory medicine requires
75 knowledge of the kind of quantity (e.g., mass concentration), a description of the
76 matrix carrying the quantity (e.g., blood plasma), and the chemical entities
77 involved (e.g., the analyte).
78 NOTE 2 The measurand can be a biological activity
79 Source : (5), definition 3.39
80 Performance claim: Specification of a performance characteristic of an IVD as documented in the
81 information supplied by the manufacturer.
82 Note 1: This can be based upon prospective performance studies, available
83 performance data or studies published in the scientific literature.
84 Source: (5), definition 3.51.
85 “Information supplied by the manufacturer” includes but is not limited to:
86 statements in the IFU, in the dossier supplied to WHO and/or other regulatory
87 authorities, in advertising or on the Internet.
88 Referred to simply as “claim” or “claimed” in this document.
89 Process Set of interrelated or interacting activities which transforms inputs into outputs

90 Note 1: Inputs to a process are generally outputs of other processes.

91 Note 2: Processes in an organization are generally planned and carried out under
92 controlled conditions to add value.
93 Source: (4) definition 3.4.1.
94 Risk: Effect of uncertainty on objectives.
95 Note 1: An effect is a deviation from the expected — positive and/or negative.
96 Note 2: Objectives can have different aspects (such as financial, health and
97 safety, and environmental goals) and can apply at different levels (such as
98 strategic, organization-wide, project, product and process).
99 Note 3: Risk is often characterized by reference to potential events and
100 consequences, or a combination of these.
101 Note 4: Risk is often expressed in terms of a combination of the consequences of
102 an event (including changes in circumstances) and the associated likelihood of
103 occurrence.
104 Note 5: Uncertainty is the state, even partial, of deficiency of information related
105 to understanding or knowledge of an event, its consequence, or likelihood.
106 Source: (2) definition 1.1 and (4) definition 3.7.9

9
107 State of the art: What is currently and generally accepted as good practice. Various methods can
108 be used to determine” state of the art” for a particular medical device.
109 Examples are:
110  standards used for the same or similar devices,
111  best practices as used in other devices of the same or similar type,
112  results of accepted scientific research.
113 State of the art does not necessarily mean the most technologically advanced
114 solution.
115 Source: (1) paragraph D.4.
116 Top management: Person or group of people who direct(s) and control(s) a manufacturer at the
117 highest level
118 Source: (1) definition 2.26.

119 2 Introduction
120 2.1 Standards, guidance and WHO prequalification assessment

121 Risk management is essential to the competent manufacture of a safe in vitro diagnostic
122 medical device (IVD). Evidence of appropriate risk management within a quality
123 management system must be provided in dossiers supplied to the World Health
124 Organization (WHO) for prequalification assessment (6), including through formal dossier
125 review and information from the dossier request during manufacturing site inspection (7).
126 If no dossier is submitted for formal dossier review, as in the case of the abridged WHO
127 prequalification assessment, evidence of a risk management process will be reviewed
128 during inspection of the manufacturing site and at the stage one inspection, if applicable.

129 This guidance is intended as an aid for manufacturers of IVDs in compiling a product
130 dossier for submission to WHO and in preparation for the site inspection aspect of the
131 WHO prequalification assessment. The guidance does not cover instrumentation,
132 analysers nor software, except in a general way. It must be read in conjunction with the
133 internationally accepted regulations, requirements and guidance documents.

134 The principal international standard for risk management of IVDs is ISO 14971:2007
135 Medical devices – Application of risk management to medical devices (1), together with
136 the harmonized European version, EN ISO 14971.

10
137 ISO 31000 (8) and its supporting standard, ISO 31010:2009 Risk management – Risk
138 management techniques (9) and guide, ISO Guide 73:2009 Risk management –
139 Vocabulary (2) are generic standards providing the framework for managing (analysing,
140 evaluating, controlling and monitoring) all types of risk including, but not specifically for,
141 those relevant to IVD design, manufacture and use. They are powerful adjuncts to
142 ISO 14971 (1) when considering the complete life cycle of an IVD, including commercial,
143 financial and manufacturing aspects. In addition, they provide essential information on
144 the development and application of any aspect of risk management.

145 Other internationally recognized guidance on the techniques and tools for risk
146 management is available from the CLSI (10,25) and GHTF (12). The CLSI guide (11) is
147 concerned, like ISO 14971 (1), with safety at the point of use, whereas CLSI EP23-A (10)
148 covers many aspects of safety in clinical laboratories. The guidance from GHTF [12] is also
149 primarily concerned with safety in use and considers all stages of the product life cycle
150 from that viewpoint.

151 2.2 Key concepts

152 Four characteristics of successful risk management underlie the methods and practices:

153  Risk management is specific for a process (.e.g. development of a product, use of a
154 product, design of labelling, installation of equipment, managing a change of use
155 to a product, evaluation of a production procedure) and not generic. Although
156 lists of factors to consider with respect to use of an IVD are available (13, 14),
157 these factors need to be evaluated and appropriately extended depending on the
158 intended use, the measurand, the environment of use and the specific features of
159 the IVD involved.

160  Risk management is a whole life cycle activity and is not retrospective. It begins
161 with the conception of the IVD and ends only when the IVD is withdrawn from the
162 market. Lessons learnt can be applied to subsequent IVDs. This means that risk
163 management of an IVD is not a one-time process: it is iterative. The risk
164 management documentation must be revisited and revised, within change
165 control, as knowledge and circumstances alter, including as post-market
166 information becomes available.
11
167 Note: For an IVD that was developed and marketed prior to the enforcement of risk
168 management regulatory requirements, the development of a product specific risk
169 management file might have to be retrospective. Updates and revisions, however,
170 should be made in “real time”.

171  Risk management is an important activity helping to produce market-leading,

172 technically innovative products and in driving cost reduction.

173  Risk management is a company-wide activity and is not restricted to a specific

174 department.

175 2.3 Cautions

176  Different groups of similarly qualified people assessing the same system, process
177 or problem with the same risk management methods will produce different lists of
178 hazards and will assign different priorities (section 4.1.3) to those that they find in
179 common (15).
180  The different tools used in risk management (section 4) produce different lists and
181 priorities of hazards for the same subject (16).
182  The validity of the weightings (criticality and risk prioritization number, section
183 4.1.2) calculated in FMEA is disputable (17, 18): because “the concept of
184 multiplying ordinal scales to prioritize failures is mathematically flawed” (18).
185  Risk management can be time-consuming (and hence expensive) and without top
186 management support is unlikely to be useful, despite being a regulatory
187 requirement and one that is valuable in practice.
188  Risk management documentation must reflect the input from all departments
189 (including the quality department) involved in the IVD development, manufacture
190 and user environment of the organization.

191 Despite the cautions listed above, risk management, however practised, is a necessary,
192 important and useful tool in the production of safe and innovative IVD. It is important to
193 have the outcomes in mind, to think constructively, and not to become obsessed with the
194 assessment processes and marginal quantitation. With appropriate training within the
195 organization and skilledfacilitation of risk management, groups of suitably experienced

12
196 people can pool their knowledge. They can then reason in a constructive, structured
197 manner about ways to eliminate potential harm, solve current problems, produce novel
198 solutions and optimize manufacturing and financial factors.

199 2.3.1 Issues observed during WHO prequalification assessment

200 Some deficiencies, only a few examples of which are outlined in Box 1, will result in
201 noncompliance with the requirements of ISO 13485:2016 (13). Of greater concern is that
202 these deficiencies indicate the possibility of unsuspected problems with the IVD resulting
203 from poor risk management of the development and verification processes.

Box 1 Examples of issues identified during WHO prequalification assessment

 FMEA are submitted in the form of a tick-list related to the Essential

Principles (13) with little analysis of the particular measurand, IVD format
or use of the IVD.
 Risk analysis content is poorly written; sometimes it is not even possible
to discern the IVD format or measurand in question.
 The risk management documentation has not been updated properly
during the product life cycle; for example risk assessment for IVDs that
have been marketed in high-resource settings has not been changed
before entering into commerce in more challenging environments.
 Insufficient consideration has been given to the skills required of users
and the reproducibility of the results obtained with the IVD in their
hands, their physical environment and potentially interfering substances
that may be present.

204 2.4 Risk management in a regulated environment

205 Risk management has long been an expectation in IVD design, manufacture and
206 commercialization. ISO 13485:2016 requires “a risk based approach to the control of the
207 appropriate processes needed for the quality management system”, clause 4.1.2 b.
208 Compliance with ISO 13485:2016 and ISO 14971: 2007 satisfies the basic regulatory
209 requirement for risk management of most authorities. However, the standards might not
210 reflect the state of the art nor all of the scientific and business aspects of the IVD life

13
211 cycle. As noted previously, the IVD regulatory requirements are predominantly safety
212 related but the organization will also want to manage business, manufacturing and
213 environmental risks.

214 The primary responsibility for the preparation of the risk management plans for any IVD
215 development project or subproject in its early phases is generally contrlled by research
216 and development department. As the project progresses, the department of
217 manufacturing, and then customer services, normally take the lead roles. The QA
218 department is usually responsible for ensuring that risk management is documented in
219 the QMS of an IVD manufacturer. This includes preparing the policies (in association with,
220 and with the approval of, top management), procedures (in association and agreement
221 with multiple departments) and documenting allocated responsibilities. The QA
222 department is involved with all risk management activities throughout the organization
223 but not necessarily in a leading role.

224 3 Risk management process

225 The required risk management process from an IVD safety viewpoint is set out in
226 ISO 14971 (1), and as a flow diagram in Annex B of that document. This information is
227 summarized here in Figure 1. The general descriptions will be expanded with examples of
228 risk management at various stages of the life cycle in later sections of this guide.

14
 Figure 1 Risk management process for IVDs

 intended use and identification of

Risk assessment
characteristics related to the safety of
Risk analysis the IVD
 identification of hazards
 estimation of the risk(s) for each
hazardous situation
Risk evaluation
 risk control option analysis


Risk management
implementation of risk control
measure(s)
 residual risk evaluation
Risk control
 risk–benefit analysis
 risks arising from risk control
measures
 completeness of risk control
Evaluation of
acceptability of residual
risk
Risk management
report
Production and post-
production information
229 Risk management is a process that involves feedback as knowledge of the topic being
230 managed increases, as is shown in the process flow diagram.

231 The process summarized in Figure 1 is concerned primarily with safety of an IVD, but this
232 general process of risk management is applicable to any aspect, for example, financial,
233 commercial, regulatory or manufacturing. Reference can be made to the ISO 31000 series
234 (2, 8, 9) for substantial guidance on these processes. Further information is available in
235 the WHO Prequalification mock dossiers (20–22).

236 3.1 Responsibilities

237 3.1.1 Top management

238 1. Responsibility for establishing the criteria for acceptability of residual risk: this is a
239 key responsibility that cannot be delegated, although developing the criteria should

15
240 be a team effort. The documented criteria with their justifications and verification
241 must be recorded in the files associated with the IVD – the design control files or
242 the risk management files. The risk management report for an IVD must be
243 approved and signed by top management, in particular the statement of
244 acceptability of the overall residual risk.
245 2. Review of the suitability and progression of the risk management process at
246 planned intervals. This would normally be included in policies regarding routine
247 reviews of the QMS and design progression (19). Organizing the review is commonly
248 an activity prepared for top management under the control of the management
249 representative (19).

250 3. Appropriate resources and suitably qualified personnel for risk management must
251 be provided throughout the product life cycle. Sufficient numbers of competent
252 personnel must be trained in risk management and have sufficient time and the
253 physical resources needed to perform the tasks required of a risk management
254 team. Without top management’s total support, risk management may be poorly
255 performed, as the activities require significant resources (23).

256 3.1.2 Departments

257 It is important that all departments within the company are involved in all risk
258 management planning and subsequent activities, except certain aspects that only affect a
259 few departments. These exceptions would need to be documented in the overall quality
260 policies and the justification for excluding some departments would be noted in the risk
261 assessment. For example, a safety risk assessment of the chemicals for a manufacturing
262 process might not require the involvement of the marketing and financial departments.

263 Everyone involved in the development and implementation of the risk management
264 process must understand the objectives – i.e. they must have an understanding of the risk
265 management plan, the available risk management tools and the methods of assigning the
266 level of risk. High-level technical knowledge of each process being evaluated is not
267 necessary for every individual on the team; however, there must be at least one person
268 with such knowledge involved.

16
269 Specific training for risk management has proven invaluable in ensuring sufficiently
270 knowledgeable individuals who can conduct the meetings, collect and present the
271 information arising in a systematic way and write effective reports. They should also be
272 able to apply, in a consistent manner, the organization’s methods for assignment of
273 degree of probability, severity and detectability to the hazards that have been identified.

274 The extent of training needed will vary between those who prepare company policies and
275 plans (high-level), those who lead risk management meetings, and those who attend risk
276 management meetings to provide insight into specific processes. To meet the
277 requirements of ISO 13485:2016 (19) on risk management, formal training is essential
278 organization-wide and many training companies already exist to provide this, either in
279 person or via the Internet.

280 Evidence of successful training in the principles and application of risk management must
281 be readily available for assessment during an audit. This would include training records,
282 interviews with relevant staff members and observation of the effectiveness of
283 implementation of risk management within the organization.

284 Example: Job description requirements for a senior risk manager

285 The risk manager will need to be able to:

286  Develop and maintain a strategic risk management policy, framework, annual plan
287 and budget for risk management activities that will help achieve the objectives of
288 the organization and meet stakeholder expectations.
289  Manage communication about risk management activities throughout the
290 organization including timely reporting to top management.
291  Collate and analyse the results of risk assessments and contribute to managing the
292 actions required.
293  Ensure risk management activities meet current regulatory requirements, both in
294 the country of manufacture and countries of distribution, including audit readiness.
295  Manage risk management training requirements within the organization, attract
296 and retain suitably qualified personnel.
297  Maintain current knowledge of risk as applied to the IVDs of the organization or
298 similar IVDs both nationally and internationally.
17
299  Promote a proactive and performance-based risk aware culture within the
300 organization.

301 3.2 Policies and planning

302 ISO 14971 (1) requires documentation showing that risk management activities are
303 planned in detail. This standard lists activities that must be covered by a risk management
304 policy and plan and Annex F gives a comprehensive guide to planning. Detailed guidance
305 for preparing quality plans is available in ISO 10005:2005 (24), ISO 31000 (8) and
306 ISO 31010 (9). Together these documents cover all aspects of policies and plans for risk
307 management. CLSI QMS02 (11) provides guidance on the contents of - and the
308 relationship between - policies, plans and procedures and is helpful in the preparation of
309 clear documentation that allows good traceability.

310 There will usually be an overall plan for risk management of an IVD covering all phases of
311 its life cycle and more specific plans for the management of the risks for each phase. Risk
312 management policies must always contain definitions of the occasions in the life cycle of a
313 product when risk management activities will take place. At a minimum these will be:

314  when an IVD or one of its related processes is being designed, after the design
315 inputs have been obtained.

316  before and after design verification and validation studies and prior to launch of a
317 new or modified product.

318  before using an existing IVD in a different way, for example with new specimen
319 types, new environments of use (such as when an IVD that has been used in a
320 high-resource setting is to be used in a low resource setting, or one that has been
321 used in major clinical laboratories is to be used at primary level testing sites), or
322 new intended users (such as extending from professional use to self-testing).

323  before and after any change to the manufacturing processes, whether as an
324 improvement of any kind or in response to problems.

325  before and after any field safety corrective actions – whether following complaints
326 from users or for other reasons.

18
327  at regular, frequent, defined intervals during the commercial life of a product to
328 ensure that no information has been overlooked.

329 The risk management policy must include methods for assigning degrees of severity to
330 effects of potential failure modes and also to the categories of probability and
331 detectability (see 4.1.2 ).

332 3.3 Training for risk management

333 Risk management is central to the development and maintenance of a quality system and
334 at all stages of the life cycle of an IVD as outlined in ISO 13485 (19). Hence, a
335 comprehensive knowledge of the philosophy and tools of risk identification and analysis
336 must be available within an organization commercializing an IVD.

337 Responsibilities listed within a risk management policy would include preparing the risk
338 management plans, organizing the meetings, preparing the reports, updating the risk
339 analyses, preparing any change control documentation and ensuring timely completion of
340 tasks identified as a result of the analyses. Responsibilities would also include post-
341 commercialization activities such as searching sources (for example, the Internet and
342 reviewed literature) for information that may affect the IVD risk management, integrating
343 feedback (for example from customers and the manufacturing process) and checking
344 continuing compliance with regulatory requirements.

345 3.4 Risk management file

346 The results of risk management activities must be collected in a risk management file,
347 which can be managed in various ways. For example, the information could be held with
348 the design control documentation, which might be an electronic system (possibly a
349 database or custom software). However, the file format must allow ready access to all
350 interconnected aspects of the risk management of each process, and to the relationship
351 of that process to other processes, within the overall risk management plan for the IVD.

352 The risk management file will provide traceability. It will include such information as the
353 risk analyses on the hazards identified and a record of the risk assessment and evaluation.
354 It will also include information on risk controls (including verification of adequacy) and
355 summarize the acceptability of residual risk in a risk management report. Top

19
356 management must sign the documentation of the acceptability of any overall residual
357 risk.

358 The file contents might not be held in a single place. However, references to and locations
359 of associated documentation must be included. A link to meeting minutes and to top
360 management reviews and approvals must also be included. This documentation must be
361 retrievable without delay for review by auditors.

362 Example: Table of contents of a risk management file. (The risk management file might
363 not physically contain all items listed in the table of contents, but must include links or
364 instructions on how to locate the information quickly.) Contents will be replicated for each
365 of the risk evaluations at key points in the life cycle of the product, e.g. for design input,
366 design verification, design validation, IFU validation, post-market surveillance.

367 Table of contents

368  Description of the IVD including intended use; safety data sheet
369  Description of risk management scope, timeline and tools to be used
370  Design and development risk management documentation
371  Risk management plan
372  Hazard identification, analysis and evaluation (biological, physical and
373 environmental)
374  Residual risk acceptance or risk mitigation (criteria and outcome)
375  Verification and validation of risk control measures
376  Production risk documentation: risk assessment of each production process
377  Post-production data analysis report (for example using data from manufacturing,
378 customer feedback etc.)
379  List of participants in risk management teams; minutes of meetings including
380 attendees, action items etc.
381  Risk management summary report including risk–benefit statement
382  Sign-off by stakeholders and top management

20
383 3.4.1 Demonstrating regulatory compliance

384 “The manufacturer shall establish, document, and maintain throughout the life cycle an
385 ongoing process for identifying hazards associated with a medical device, estimating and
386 evaluating associated risks, controlling these risks and monitoring the effectiveness of
387 controls and shall include:

388 – Risk analysis

389 – Risk evaluation

390 – Risk Control

391 – Production and post-production information”(1).

392 As previously mentioned, the basis for requirements can be found in ISO 13485 (19) and
393 ISO 14971 (1). These standards offer the broad outline of the principles and practices that
394 are required. In addition, the annexes provided, particularly in ISO 14971 (1), together
395 with many other resources such as those in the reference list, are very useful to suitably
396 trained and qualified personnel.

397 Within this context, the intent of the standards is used by WHO to assure compliance with
398 internationally recognized best practice in the manufacture of IVDs and to meet the
399 needs of WHO and its stakeholders. WHO’s technical guidance (including this guidance
400 document), together with published procedures and mock dossiers available on the WHO
401 website (http://www.who.int/diagnostics_laboratory/evaluations/en/) are intended to
402 assist manufacturers in their understanding of this approach.

403 Throughout this document there are examples of frequently encountered occurrences of
404 failure to comply with requirements set out in the two main standards used by WHO
405 ISO 14971 (1) and ISO 13485 (19). These examples come from dossiers submitted to, and
406 on-site inspections performedon behalf of, the WHO Prequalification Programme and are
407 given here to assist manufacturers in avoiding similar deficiencies and nonconformities.

408 Examples: The main examples of inadequate risk management resulting in

409 nonconformities include:

21
410  Resourcing (and approval of outcomes) of risk management activities by top
411 management is underestimated and hence insufficient to meet compliance
412 expectations.
413  The qualifications of personnel performing risk management activities are
414 inadequate.
415  Risk management activities are superficial in nature, generic, and do not consider
416 all of the layers of complexity required in risk management as outlined in the
417 standards.
418  Risk management is not applied to all aspects of the product life cycle as it must
419 be according to ISO 13485:2016 and according to best practice. It is rare to find
420 risk management applied to manufacturing and business processes as would be
421 expected to ensure product quality and continuity of supply.
422  The risk management documentation submitted with the dossier is incomplete.
423  Documentation, in terms of a risk management file, is scattered and not easily
424 accessible: it is not retrievable within a reasonable time frame (within one hour) at
425 on-site inspections.
426  Traceability of risk management activities throughout the whole life cycle of the
427 IVD is inadequate.
428  Review and updating of the risk management assessment at timely intervals is not
429 performed.
430  The “worst-case” environment of the end user is not considered.
431 Comment 1: As with all submissions to WHO, accuracy of information and data
432 (truthfulness) is considered to be an essential requirement.
433 Comment 2: Evidence of effectively implemented risk management across all aspects of
434 the life cycle of an IVD presented to WHO for prequalification builds significant trust in
435 the manufacturer’s ability to provide a high quality IVD.

436 4 Tools and methods

437 ISO 31010 (9) has a comprehensive list of risk assessment and problem-solving tools, their
438 applicability and their strengths and weaknesses. Both ISO 31010 (9) and CLSI EP18-A2
439 (25) provide excellent guidance.

22
440 The techniques most commonly used in the IVD industry are:

441  failure mode and effects analysis (FMEA)

442  fault tree analysis (FTA)
443  failure reporting and corrective action system (FRACAS)

444 When using these tools, it must be remembered that:

445  FMEA deals with single failure modes

446  FTA can lead to discovery of some effects caused by two or more failure modes
447 occurring simultaneously
448  FRACAS can be used to feed information into the other two tools about existing
449 failure modes with known causes and effects

450 This interdependency explains why the tools are most effective when used together. In
451 addition, usage, descriptions and techniques for the “seven basic tools” for problem
452 solving (including those of risk management) can be found in most manuals about quality
453 processes including that of the American Society for Quality (26).

454 4.1 FMEA

455 FMEA will be described in detail as it is the most commonly used basis for hazard
456 discovery and quantification during the processes of IVD design, manufacture and use.
457 However FMEA is not always the most appropriate basis for risk management. The
458 techniques described in ISO 31010 (9) are of more general applicability because they are
459 concerned with risk defined as “effect of un certainty on objectives” (2 and4), not simply
460 with potential harm as in ISO 14971 (1). For example, management of the risks in
461 preparation of the IFU is probably not efficiently based on an FMEA: the potential harms
462 arising from the IVD and its use should have been assessed during the various design and
463 development evaluations. Development of the IFU, especially in an established IVD
464 manufacturing organization, is more likely to relate to provision of correct and complete
465 information in a user accessible fashion and regulatory compliance than in identifying,
466 quantifying and minimizing the effect of hazards as defined in (1). Similarly some aspects
467 of performance evaluation and supplier management are best risk assessed using
468 techniques from (9) other than FMEA (but probably not those involved with defining

23
469 criteria for incoming goods inspections) because the related hazards should have been
470 evaluated during design and development of the IVD. Whatever the techniques of risk
471 management, their use and outcome must be recorded for compliance with (19).

472 4.1.1 The start-up work

473 The following sections are written predominantly from an ISO 14971˗compliant safety
474 stance (1). However, the principles and comments are applicable to risk assessment for
475 most processes in the IVD life cycle including R&D, manufacturing and post-market
476 surveillance. These sections also apply to most business-related activities when read with
477 reference to guidance in ISO 31010.

478 The first step in preparing an FMEA is to obtain a complete description of the process –
479 the scope – with as much detail as practicable. Often a flowchart of the process can be
480 prepared from the scope, giving a detailed overview of the activities in the process in the
481 sequence in which they take place, as this makes the subsequent analysis easier to
482 document.

483 A template for the FMEA should have been developed from the policy and planning for
484 risk management in the QMS of the organization. Conventionally this is prepared as an
485 electronic spreadsheet with a worksheet with headings as in Figure 2 (25).

Existing situation After action taken

Potential cause of failure mode
Item in scope: characteristic

Effect of failure mode

Risk priority number

Action to be taken
Failure mode

Detectability

Action taken
Occurrence

Occurrence
Criticality

Criticality
Controls
Severity

Severity

RPN

486 Figure 2 Example of an FMEA template

487 There are examples of spreadsheets in the WHO HIV self-test sample dossier (20), and the
488 two nucleic acid testing sample dossiers (21, 22). The spreadsheet must have a start-up

24
489 worksheet with details of QMS factors such as version control, participants in the various
490 meetings, signatures and dates. The organization’s risk grid (see section 4.1.4) is also
491 usually added to the spreadsheet.

492 Comment: There are many other ways of capturing hazard data. However, the FMEA
493 format seems to be the most frequently used and is easy to read and understand.

494 It is generally the task of the risk management leader to assemble the scope
495 documentation, draw up the flowchart of the process and from that to prepare the
496 spreadsheet using the template. The risk management leader then completes as much as
497 possible of the start-up information and also the first column (“Item in scope:
498 characteristics”) of the main worksheet.

499 With this preparation completed, a team of people from across the organization who
500 have diverse knowledge of the topic should be assembled to develop the risk assessment.
501 The team should include representatives from various levels of seniority and all should
502 have basic training in risk management and FMEA processes. They should be well
503 informed about the agenda of the meeting, having reviewed the scope document and the
504 flowsheet and considered independently the specific risk management needed.

505 A useful aid to simplify and hasten progress is to project the worksheet onto a screen and
506 complete it electronically as the meeting progresses. After the meeting the leader can
507 add to the worksheet from his or her notes, agree about follow-up work with other staff
508 and if necessary use it to report to senior management.

509 Risk management is iterative and spreadsheets will be reworked several times during the
510 lifetime of a process, whether that process is commercialization of an IVD or risk
511 management of a factory process. Controls will be added, new failure modes discovered,
512 changes in probabilities and detectability calculated, new characteristics added (for
513 example submission of an IVD to a new regulatory authority or addition of a new
514 intended use). Use of a spreadsheet helps in managing this activity, as extra rows and
515 worksheets can be added, all within a document change control system.

25
516 4.1.2 The FMEA process

Prepare for the meeting

Input documents
Flow sheets
Worksheets
Circulate to attendees

Identify failure modes

For each item in the scope and in
logical order

Identify the effects of the failure

mode

Determine severity

Identify the potential causes of each

failure mode Calculate criticality and risk
Use problem solving tools priority number
Possibly several potential causes for
each failure mode
Ensure the real (root) cause is identified
Risk evaluation and
control measures
Determine probability of
occurrence of HARM from each
cause
Re-evaluate FMEA with
new controls
? Determine probability of detection New hazards?
with current controls Severity cannot be changed
517

518 Figure 3 The FMEA process

519 The assessment group starts work by creating lists of ways that each item in the scope
520 could fail (or become noncompliant or nonconforming). These are the failure modes and
521 must be as comprehensive as possible. Hazards in normal use not caused by a failure
522 mode of the IVD (although perhaps caused by a failure mode of the design), such as
523 contamination of the user with specimen, must also be considered during the design and
524 use FMEA processes.

525 For each failure mode the group must next generate a list of all the effects of that type of
526 failure on the output of the process concerned. When compiling this list every aspect that
527 might be affected should be taken into consideration, for example, manufacturability,
528 safety of the user or the patient, continuity of supply, cost and so forth. These listings are
26
529 the effects of the failure modes. The following sections give guidance on this for each
530 major point in the life cycle of an IVD.

531 The next action, determining the severity of all the possible harms of each of the effects,
532 presents difficulties. It is common practice to give each harm a score between 1 and 5 or
533 between 1 and 10 (higher being more serious) but different groups will nearly always
534 assign different degrees of severity to the same harm for the same process, sometimes
535 markedly different (15, 17). One approach is to take the best opinions available from the
536 members of the assessment group and calculate the median of the values. Another
537 possibility is to obtain a consensus value through group discussion. (See reference (17) for
538 apparently more rigorous methods but which still face the same difficulty.)

539 Some failure modes might have more than one effect. In that case it is possible that only
540 the most serious effect needs to be considered further. However, in view of the difficulty
541 of assigning severity, this could present problems. This is particularly likely if the range of
542 severity values assigned to an effect is wide but the value finally chosen is low (relative to
543 other effects from the same failure mode). However, failure modes with effects on a
544 critical outcome of the process (for example safety, continuity of supply, user perceptions
545 or cost in a manufacturing environment) are usually given high scores.

546 All the potential causes of each failure mode must next be listed (generally using routine
547 problem solving tools, for example Ishikawa (fishbone) diagrams) and bearing in mind
548 that there might be more than one potential cause for each failure mode. The potential
549 causes are given an estimated probability of occurrence using the current state of the
550 process and its existing controls if any. As with severity, the probability of occurrence can
551 be debatable. It might be possible to obtain probabilities from similar processes, failures
552 and causes, but it is more likely that a consensus view will be necessary. A procedure for
553 estimating qualitative frequencies and probabilities from available data is presented in
554 CLSI QMS11 (27, Appendix D) and a thorough approach is described in (28). This
555 probability is known as P1 (see Annex E of ISO 14971), it is the probability of the
556 occurrence of the hazard. Risk, however, is defined in relationship to harm, and the
557 existence of a hazard or hazardous situation does not always lead to harm – usually a
558 second effect or event must occur to bring about the harm. The probability of this second

27
559 effect is known as P2. Thus the probability of harm being caused by the hazard is
560 composed of the combined probability of the hazard and the second event, P1*P2, which
561 is lower than the probabiity of either event alone. This overall probability of causing
562 harm is given the score (from 1–5 or 1–10, higher being more probable) to be used in the
563 occurrence column of the spreadsheet and in the variious risk calculations. As discussed
564 above assigning such probabilities and scores is subjective!

565 Examples:

566 1.) Consider an effect being a false-positive result, a harm of this is a misdiagnosed and
567 maltreated patient. A recombinant protein might occasionally have an impurity,
568 depending on the efficacy of its manufacture, that could be incorporated into an
569 assay. The probability of the presence of the impurity is P1, the probability of the
570 hazard. An individual tested in the assay might have antibodies that react with the
571 impurity (the probability of this second event is P2) giving rise to a false-positive
572 reaction, probabiility P1*P2, which is the probability of occurrence of the harm. In
573 the presnce of either the impurity or the individual with the antibody, but not both,
574 no harm will occur. This is frequently the case in HIV and HCV testing, when the
575 recombinant proteins purified from E. coli cultures are not subjected to stringent
576 evaluation prior to use and some individuals have strong anti-E. coli reactivity.

577 2.) Consider an effect being a technician becoming infected as a result of using the IVD. A
578 potential cause being exposure to patient body fluid because of a poorly designed
579 sample entry port. The hazard is that the user might touch the specimen, the
580 hazardous situation arises if this happens (probability P1), and with probability P2 the
581 patient has an infection (not necessarily that being tested) that affects the technician.
582 The probability of harm is then P1*P2 – coincidence of both hazard and second event.

583 3.) Two events are not always directly involved. Consider an IVD which has been subject
584 to maltreatment during transport so that it no longer detects reactive specimens (the
585 probablity is P1, the hazard). If the IVD has a control line which becomes visible when
586 an assay is performed but which only monitors flow, not function, and the presence
587 of the line is said in the IFU to validate the assay: harm (wrong diagnosis and its
588 consequences) will be caused with probability P1.

28
589 In IVD manufacturing organizations, detectability (probability of detection) of the failure
590 mode is occasionally taken into consideration during an FMEA. As with severity and
591 probability, opinions on detectability can differ; indeed whether detectability should be
592 included in risk management at all is subject to debate (29). If the failure can be detected
593 (for example by the user, or by the operator during manufacturing) the effect should not
594 occur, but the process of detection itself should form part of the risk evaluation of the
595 control mechanism. For example, could a visually impaired user notice that a control line
596 was unusually weak? The probability of detection is given a score of 1–5 or 1–10. A score
597 of 1 is assigned if the failure will always be detected (100%) and a score of 5 or 10 when
598 there is no possibility of detection. Whichever methods of assigning severity, probability
599 and detectability are used, it is important to validate them as thoroughly as possible and
600 to re-evaluate the FMEA as knowledge increases.

601 Finally, the risk priority number (RPN: severity × occurrence × detectability) and the
602 criticality (severity × occurrence) might be calculated for each effect of the failure modes
603 (see Box 2).

Box 2 Explanatory notes on probability × severity × detectability

 Probability (occurrence) is related to likelihood, occurrence or

frequency of the HARM arising. A probability estimate can be
quantitative (using data and statistics) or qualitative (based on
experience and considered opinion). Each hazard and hazardous
situation will give rise to a risk estimate.
 Severity measures the possible consequence of a hazard.
 Detectability (probability of detection) of the hazard or hazardous
situation before it leads to harm and so reduces the likelihood of
harm and reduces the estimated risk.

604 4.1.3 Risk evaluation

605 Once the FMEA has been completed, the risks can be placed in order of criticality or
606 priority (possibly with aid of Pareto diagrams) and also measured against the company
607 risk grid. The risks that are unacceptable can be dealt with by introducing appropriate
608 controls (see section 5 and later in this section) and subsequently reviewed by further
29
609 FMEA for newly introduced hazards. Decisions can then be made about how to proceed
610 with risks that are unacceptable. Every measurement or action by QC or QA should be
611 the outcome of, and traceable to and a risk evaluation.

612 For IVDs that are to be CE-marked under the EU directive on In vitro diagnostic medical
613 devices (30), risk must be reduced “as far as possible” through safe design and
614 construction. This is usually interpreted as meaning that the cost of ameliorating a risk
615 cannot be used as a factor in deciding acceptability. While CE marking is not a
616 prerequisite for acceptance to the WHO Prequalification Programme, if an IVD is
617 CE-marked, the dossier would be expected to prove compliance with this aspect.
618 ISO 14791 (1) is more lenient in that risk must “as low as reasonably practicable”.

619 4.1.4 Risk grid

620 The most common form of risk evaluation is against a risk grid similar to that shown in
621 Table 1, which uses a grading scheme with scores of 1–5. The risk grid is usually
622 incorporated as a worksheet on the FMEA spreadsheet together with a table of actions
623 similar to those shown in Table 2, which must be taken subsequent to the FMEA. The
624 detailed action is added to the “action to be taken column” either after discussion at the
625 FMEA meeting or by the risk management leader in consultation with technical staff.

Severity of effect
Table 1 Risk Grid 5 4 3 2 1
Critical Major Moderate Minor Minimal
5 Frequent High High High High Medium
Probability of occurrence

4 Probable High High High Medium Low

3 Occasional High High Medium Medium Low

2 Rare High Medium Medium Low Low

1 Improbable Medium Low Low Low Low

30
 Table 2 Actions to be taken following the FMEA

Table of actions dependent on zone in risk grid

Outcome zone Risk assessed Action to be taken
Process must be redesigned, or, if that is not
High Harm A possible, a mode of control must be added. This
must be agreed by top management.
Process to be redesigned or a control added. This
Medium Harm B
may need agreement by top management.
Low Harm C Control to be added.
626 How the grades for severity and probability are assigned and what action should be taken
627 must be defined in the QMS policies and might vary from process to process. There is no
628 regulatory standard and very little guidance on quantifying any of the factors, nor for
629 determining the acceptability of overall risk for an IVD.

630 For safety aspects, ISO 14971:2007 (1) suggests that severity could be classified
631 qualitatively using three grades as shown in Table 3 (1).

632 Table 3 Qualitative classification of severity using three grades

Term Description of the harm

Significant Death or loss of function or structure
Moderate Reversible or minor injury
Negligible Will not cause injury or will injure slightly

633 But more usually severity is classified in at least five grades as shown in Table 4.

634 Table 4 Qualitative classification of severity using five grades

Severity Description of the harm

5 Critical Life-threatening, loss of limb, threat to community
4 Major Severe lasting effects, requiring medical action
3 Moderate Short-term effects, requiring medical action
2 Minor No lasting effects
1 Minimal Slight or no effects to users or patients

635 For an IVD these harms might be caused by:

636  Misdiagnosis.
637  physical, chemical or microbiological failure mode of the device itself.
638  a hazard present in the use of the IVD with no failure mode.

31
639  use of the device in an unintended fashion (“off-label use”).

640 Annex D of ISO 14971 (1) provides guidance on estimation of risk.

641 No regulatory standards have been established for probability and detectability of risk for
642 IVD. The usually accepted levels for probability are listed in Table 5.

643 Table 5 Commonly used probability levels for risk assessment of IVDs
Level Description
5 Frequent 1:100
4 Probable 1:1 000
3 Occasional 1:10 000
2 Rare 1:100 000
1 Improbable 1:1 000 000
644 Whatever characteristics are selected for severity, occurrence of harm and detectability
645 of the hazardous situation, the QMS must include a policy on how to choose and justify
646 them, together with the justification of acceptability of overall residual risk (ISO 13485
647 (13)). Given the variability in outcome of assessing risk quantitatively (see the comments
648 above and references in section 2.3) the policy must encourage the exercise of caution
649 and be carefully and comprehensively written. Use of a qualitative risk grid and action
650 table as described above avoids the need to calculate RPN or criticality scores and this in
651 turn avoids much debate about calculating “exact” values and their interpretation.

652 5 Risk control options

653 Once risks have been quantified using the tools described in previous sections they must
654 be controlled. ISO 14971 (1) prescribes the options for risk control and the order in which
655 they must be applied:

656  Safety by design comes first and foremost and is usually interpreted as meaning
657 “inherently safe design and construction” (30). For this reason, risk management
658 must start from the inception of an IVD (or any process) so that the design can be
659 planned safely and controls do not need to be built in later on to overcome design
660 flaws that should have been avoided. Experience shows that risk management at
661 the design input stage can also lead to much greater user satisfaction, as ideas for
662 novel and functional features often arise during the risk management meetings.

32
663  Protective measures in the IVD itself or in the manufacturing process are features
664 that prevent a failure mode in the first place. They are not features such as run
665 controls (31) that warn users that a failure has already occurred.

666 Example: A shield over the specimen addition port of a rapid diagnostic test to prevent
667 contact with potentially infectious material.

668  Information for safety. Well-written and validated IFU provide control of risk, but
669 this is the least effective method of control. It is not the same as a warning, which
670 alerts users to the existence of risk but neither prevents nor ameliorates it.

671 Run controls might be viewed as providing information for safety but they are
672 weak controls in that, if activated, indicate the failure of that test run.

673 Control measuress must be evaluated to ensure they do not present new hazards. They
674 must be verified and validated in the user environments.

675 Example: Many run controls for rapid diagnostic tests indicate successful flow of
676 reagents only and do not confirm that the test would have detected a positive
677 specimen. Such a run control might not indicate thermal inactivation of the IVD, and
678 so give a false sense of security to a user. Limitations must be clearly stated.

679 6 Risk management – selected activities

680 6.1 Quality management system

681 As noted in section ‎2.4 ISO 13486:2016, unlike earlier versions of the standard,
682 specifically states that the QMS must be developed using risk management principles.
683 Quality management and risk management of all processes of the organization must be
684 linked and have feedback interchange mechanisms between them. All steps in the risk
685 management process must be performed in accordance with the organization’s quality
686 manual. For example, document control will apply to risk management activites and
687 include document identification, version control, traceability, review intervals,
688 identification of responsible personnel, links to competence and training records of
689 personnel, records of meetings and links to management review, among others. The risk
690 management documentation – the risk management file – is managed within the

33
691 manufacturer’s QMS and is an essential component of the risk management
692 documentation linking and feeding back to the quality manual

693 Input leading to changes in the quality manual and the QMS from risk management of
694 activites within the system might include for example:

695  potential lack of staff competence determined during particular risk

696 management activities and any training needs documented.
697  Overall risk to the organization from customer feedback mechanisms such as the
698 CAPA system; problems found when using particular transportation suppliers or
699 modes; lack of clarity in IFU and subsequent unintended uses; reviews of
700 literature related to the manufacturer’s products, analytes, assay methods or
701 failure to notify regulators and WHO about critical change to a product.
702  Information from a manufacturing risk assessmentto assess the need for a
703 general change in documentation (e.g. styles, use of language, font size in
704 policies, procedures) within the QMS that might be needed to minimize risk
705 from lack of readability by non-technical personnel, sometimes through the use
706 of photographs, diagrams or translation into the language in use on the
707 manufacturing floor.
708  Overall risk to the organization caused by poor suppliers found from particular
709 failings in the supplier audit methods noticed during supply-risk analyses

710 6.2 Risk management and design control

711 Risk management is an integral part of design control. The two processes have the same
712 goal from a manufacturer’s viewpoint: to produce a safe, efficacious,
713 regulatory-compliant product with wide customer appeal, good profitability and
714 continuity of supply.

715 A typical design and development flowchart with integral risk management is shown in
716 Figure 4. This is a suggested process flow only but is typical of current practices. A risk
717 management process is associated with each critical stage of the life cycle of the product,
718 within the overall plan (see section 3.2). The risk management performed as the product
719 is withdrawn from commerce (mainly user satisfaction and business oriented but also

34
720 summarizing the life cycle of the IVD) is not shown in Figure 4, but the design information
721 obtained from this activity should be used in the development of any future IVD.

35
Design inputs
User requirements
Regulatory requirements
Manufacturing requirements and capabilities
Management expectations Design and development plan written
including
"Customer" requirements document risk managment plan
Design will be validatedi against this
Design change control begins when this document is approved

Design risk analysis

Product specifications
Numeric design requirements for R&D
Product will be verified against this

R&D phase 1 User, patient, manufacturing risk

(under design change control) analyses started,
Format and instrumentation developed re-evaluated regularly
Processes developed and qualified. Manufacturing documentation started
Guard bands for all process parameters defined and validated
QA and QC parameters and materials defined, sourced and documented
Calibrators and internal controls developed and metrologically traceable
Beginning of stability work for in-process intermediates and final device
IFU initiated

R&D phase 2
Transitioning to factory
Change control begins
Instruments and material suppliers finalised and audited
Instrumental PQ, OQ in the factory
Pilot batches made, tested against putative QC
Interfering substances evaluated, efficacy of microbiocides proven
Process documentation finalised and approved
All aspects of device and specimen stability using devices made to approved specifications
IFU finalised and approved
QA and QC specifications finalised and approved Pre-verification and validation risk analyses

Final IFU risk analysis

R&D phase 3
Design verification using material made in the factory to approved documentation
(R&D evaluation of all aspects of product specification document
e.g.performance, repeatability, reproducibility , lot to lot variability, all aspects of stability)

R&D phase 4
Design validation using material made in the factory at scale to approved
documentation
(User evaluation of all aspects of product specification document and customer
requirements document)
e.g.performance, repeatability, reproducibility , lot to lot variability,
functionality of IFU, training manuals, software)
Normally done in three user-labs with three independent lots of reagent
Pre-commercial launch risk analyses
Production of:
Declaration of acceptable residual overall risk

Manufacture
CE marking declarations and inspections
Full scale manufacture under change control
On-market surveillance
Review of scientific literature for independent clinical evaluations

Continuous on-market monitoring and risk re-assessment

Figure 4 Design control and risk management

36
722 6.2.1 Design risk assessment

723 This is the first stage of risk management for the whole life cycle of the IVD. It begins as
724 soon as the concept of the product is finalized, after the customer inputs and
725 requirements have been obtained. “Customer” in this context is any entity that the
726 potential product will be affected by or will affect. Customers will be both outside the
727 organization, for example patients, users, distributors, purchasers and regulators and
728 within the organization, for example finance department, patents managers, QA,
729 manufacturing, R&D, sales, marketing and customer support. Design input from all of
730 these sources will define the intended use and lead, in turn, to defining the design
731 requirements of the proposed IVD.

732 Once the design requirements are available, the basic features of the proposed IVD will
733 become apparent and the design risk management planning can begin. Normally the
734 outline of the plan will be described in the QMS and policies, but the details will need to
735 be defined. These include determining responsibilities overall and identifying the
736 participants in the initial risk assessment meetings. Design risk management planning will
737 consider everything that is known about the potential product, in line with and taking into
738 account the Essential Principles (13, 14). This will be focused specifically on the proposed
739 IVD and its uses and is not a generic assessment.

740 The initial design risk management meeting should address as a minimum the factors
741 listed in Table 6.

742 Table 6 Factors related to the specific IVD to be considered in initial design risk
743 management

Factor Example characteristics to consider

Method of obtaining specimen from patient, type, storage, transport,
Specimen likely interfering substances and cross-reacting materials in that
specimen type
Environment, age, sex, likely concurrent and similar illnesses,
Patient potential pharmaceutical treatments, vaccinations, potential drugs
and other social factors
Environment (especially temperature, humidity, altitude, microbial
User flora), training, skill level, social factors (intolerance of some
constituents)

37
 Factor Example characteristics to consider
Known failure modes with a similar IVD using the same format (for
example prozone effect, lack of or poor specimen/reagent flow,
IVD reversal of specimen/reagent flow, insufficient specimen addition,
cross-contamination, failure to link result to patient, contamination
with enzymes)
Known failure modes with all formats of IVD for the measurand (for
example cross-reaction with other substances, vaccination status of
Measurand patient, drug treatment interference, confusion with disease that has
similar symptoms, specific problems in certain populations or age
groups)

744 Many of these factors will interact with other factors. For example, the patient’s
745 environment is likely to affect the possible concurrent illnesses and possible treatments
746 unrelated to, but potentially affecting, the proposed IVD. This assessment requires
747 considerable knowledge of the disease, technicalities of the measurand, the device
748 formats available and state of the art for the proposal. The initial design risk management
749 meetings must also take into account the internal requirements of the organization such
750 as business and commercial factors, manufacturing factors including training and
751 equipment and regulatory issues.

752 The control factors coming from the design risk assessment should lead to features of the
753 design of the IVD, to requirements for many of its performance features, to supplier
754 considerations and to in-built controls. Many risks should be removed by appropriate
755 design of the format of the device, choice of reagents and method of addition of
756 reagents. This process might lead to at least some features that will give the product
757 commercial and possibly societal advantages and provide more than a “me too” device
758 that would be perceived by potential purchasers as competing simply on cost.

759 As the design is developed, new ideas will be generated, new problems will arise and the
760 risk assessment and controls will need to be re-evaluated. Once the main design risk
761 management process, usually an FMEA supported by an FTA, has been completed just
762 after design initiation, subsequent amendments are simple to document. Review of the
763 design risk management together with the progress of the whole project at the regular
764 design control review meetings (19) is usually sufficient.

38
765 As the various phases of the design progress, risk assessment will become necessary for
766 manufacturing processes, the IFU, verification and validation work and pre-launch
767 matters (see Figure 4 and subsequent sections in this guide). All of these assessments
768 should be specified in the QMS policies and will provide information especially related to
769 safety and QA matters and to the main design risk management process, although they
770 are to some extent independent of it.

771 Any changes to the design, manufacturing process or IFU at any stage and for any reason
772 (for example because input requirements cannot be met, supplier change, regulation
773 change, CAPA, a new intended use, a new population of patients or users of the IVD, or a
774 new environment of use) must trigger input to the design risk management files and a
775 new assessment, even if only to report and justify that there is “no change” in risks.

776 6.3 Verification and validation

777 Design verification, in accordance with the manufacturer’s QMS, will confirm that the
778 design output meets the design input requirements. Results of the design verification will
779 usually be documented in, for example, a design history file. Verification occurs at
780 multiple stages and includes testing, inspection and analysis of results. Verification will
781 include hazards associated with handling and use, environmental effects, packaging
782 integrity tests, biocompatibility testing of materials to be used, bio-burden testing and
783 comparison with existing similar designs.

784 Design validation confirms that products conform to user needs and are suitable for their
785 intended use. Design validation must include risk analysis. Consideration must be given to
786 the robustness of the process. That is, expected variations of components, materials,
787 manufacturing processes and the user and user’s environment must be taken into
788 account. Validation must use routine production units tested under actual or simulated
789 conditions.

790 Process validation confirms that a process produces a result or product that consistently
791 meets requirements (and hence that the product consistently meets the correct
792 specifications).

39
793 Comment: The absence of “re-validation” (ongoing verification) programmes for
794 equipment and processes and inadequate qualifications of personnel performing such
795 tasks have resulted in nonconformities being recorded during WHO PQ inspections.

796 6.4 Analytical and clinical performance studies (design validation)

797 The risk management methodology and tools already described apply to performance
798 studies.

799 Existing WHO guidance and mock dossiers support a risk management approach to
800 analytical performance studies to confirm intrinsic performance capabilities relative to
801 design specifications and to clinical performance studies to confirm that the expected
802 performance of the IVD is achieved in its intended use by intended users (19).

803 The WHO publication Principles of performance studies (Technical Guidance Series 3)
804 refers to risk management in the context of analytical and performance studies
805 throughout (33). In addition, the sample dossiers (20, 21, 22) have information that can
806 be used when preparing for risk management of the processes.

807 The risk management plan, assessment, evaluation and control actions related to the
808 studies must be documented and executed.

809 Note: “Clinical performance studies should ensure that the rights, safety, and well-being
810 of subjects participating in a clinical performance study must be protected … That is, each
811 clinical performance study should generate new data, the benefits to health must
812 outweigh risks to study participants and any risks must be minimized, and confidentiality
813 must be respected” (33 Study rationale 4.6.1).

814 Furthermore, “The risk assessment conducted as part of product development should
815 also include a component that accounts for any hazards posed (to user and/or patient) by
816 the product during the course of the clinical study” (33 Study method 4.6.3).

817 As an example, assessment would need to include the following:

818  a well-defined study protocol (to include risk assessment and plans, assess data
819 collection, amendments and changes protocol).

40
820  monitoring of study (monitoring plan that considers complexity of study design,
821 clinical complexity of study population, geography of study location, experience
822 of investigators, relative safety of the product and data collection methods).
823  recruitment risk (selection of sites and patients, adequacy of medical records,
824 informed consent risk).
825  risk of deviation from the protocol (rules for cessation of study).
826  data collection risk (quality of data and staff turnover).

827 6.5 Change controls

828 Changes made to, for example, any design, manufacturing processes or intended use of
829 the product must be implemented and documented in line with the manufacturer’s QMS
830 change control requirements. The QMS will have procedures for the identification,
831 documentation, validation (or where appropriate verification), review and approval of
832 changes before implementation. The reason for and justification of the change and any
833 retraining required must also be documented.

834 Many changes will be related to the product design and so the risk management will likely
835 be based on hazard evaluation using FMEA as the main tool, closely linked to the overall
836 design risk management. Other changes, for example to labelling, would be expected to
837 be linked to risk management documentaton already established for both the design and
838 the process concerned for that IVD. Risk evaluation of change must be initiated before
839 any changes are made so that the planning for the change will take into consideration any
840 risk to be minimized and ensure that appropriate verification or validation of the process
841 and IVD is performed in a timely fashion. Any changes must be assessed and
842 documented in relation not only to the product or process that is changed but also taking
843 into account the possible repercussions across subsystems and the system as a whole
844 because modification of one aspect of a process might well introduce hazards elsewhere.

845 The following paragraphs exemplify some of the change processes which are frequently
846 found to be poorly managed during inspections by WHO PQ. The changes are often not
847 managed in compliance with (19) and any risk evaluation is not documented. Aspects of
848 particular concern are absence of the following: traceability, justification for change,
849 verification and validation of the change and notification to the user of the change.

41
850 6.5.1 Equipment change

851 Criticality of the equipment or process will affect the level of risk management of change
852 control. High-risk equipment or processes will require a higher level of qualification,
853 change control, maintenance and monitoring.

854 In addition, the category of equipment change will affect the risk management activities.

855  If it is an “identical” replacement, then assessment may be limited to

856 demonstrating that the equipment is identical as defined in the manufacturer’s
857 procedures, and documenting the process within the change control procedures,
858 recording specifications and operating parameters to demonstrate they are
859 identical. An abridged functional qualification may suffice.
860  If the equipment has the same dimensions, uses the same methodology and has
861 the same performance characteristics, then the activities described for identical
862 replacement plus additional performance testing may suffice.
863  A “true” change (neither of the above) would require a full risk management
864 approach covering the whole life cycle of the equipment and the products
865 concerned.

866 6.5.2 Change of supplier of a critical component of an IVD

867 Example: A recombinant protein is purchased from a new supplier.

868  The processes and the intermediate products tmust be re-validated to ensure
869 the new protein meets all of the requirements.
870  The design must be re-validated as the change will potentially affect assay
871 stability, sensitivity and specificity (ideally validation by users although this
872 depends on the risk evaluation).
873  The change must be notified to regulatory bodies as it is the change of a
874 critical component.

875 6.5.3 Change of intended use

876 Example: Risk management of change of the types of anticoagulants for plasma used in
877 the IVD: a change in the IFU is required.
878  The design must be re-validated, at least partly, because:

42
879 – the change will affect the intention in the IFU.
880 – the stability of the new plasma type must be documented.
881 – performance claims must be maintained.
882  The change may need to be notified, certainly if plasma types are restricted
883 and perhaps if increased.

884 6.6 The IFU and other labelling

885 Risk management for some processes such as the design, safety in use and development
886 of most of the manufacturing and QA procedures for an IVD can be based efficiently on
887 an FMEA and associated tools for quality management and improvement. However, as
888 noted previously (section 4) ISO 31010 (9) provides a comprehensive list of techniques
889 which might be better than FMEA for managing risks for processes for which the basic
890 hazards should be well understood and documented from the design and manufacturing
891 assessments.

892 6.6.1 IFU

893 As the IFU is the main communication between user and manufacturer it is regarded with
894 particular importance by WHO PQ. The contents of the IFU and their validation must be
895 evidence based. The following must be read in conjunction with TGS 5 “Designing
896 Instructions for use for in vitro diagnostic medical devices” (34) which presents WHO
897 expectations for IFU, and is consistent with international regulation (5 and 35). For
898 established manufacturing companies the use of checklists and properly facilitated
899 brainstorming as described in (9) should be sufficient to control the risks to be managed
900 during the IFU development process, perhaps with organization of the output using FMEA
901 like spreadsheets but with different column headings. The main residual risks related to
902 using, storing and disposing the IVD, its specimens and accessories should be available
903 from the design and manufacturing groups, the risks from not satisfying the regulatory
904 requirements laid out in the ISO 18113 (5) series from the company’s regulatory group
905 and risks related to the supply and physics of the IFU from the logistics managers. Each of
906 these departments plus the customer support group should be present at the IFU risk
907 management meetings of which records must be maintained (19). A specific risk,
908 relevant for WHO, is that the users’ language might not be included in the instructions for

43
909 use or the level of language might be not be appropriate for the level of training of the
910 intended user. As a consequence, diagrammatic aids for use are provided frequently but
911 this needs considerably thought to ensure they are appropriate for all intended users in
912 all environments. The culture of the users must be evaluated in each intended setting to
913 ensure the information is appropriate and accurate. Whatever “simple” user aids are
914 made available it is critical that the risk management process ensures that the intended
915 method and the depicted method are identical, changed in tandem as necessary, and
916 validated relative to the clinical evaluation of the product.

917 Although the risks will be product dependent and risk management must make specific
918 reference to the IVD concerned there are recognized, repeated deficiencies (and non-
919 compliances with (5)) found in the IFU presented to WHO PQ. These deficiencies might
920 be found from the IFU itself, in comparison of the claims in the IFU with the data in
921 dossiers, or during on-site inspections. The following table, which can help with the risk
922 management process by contributing to checklists, brings together the principle issues
923 found (many of which should have been dealt with and made acceptable during IVD
924 development). Reading through the table it will be apparent that risk management for
925 the IFU needs to evaluate that all statements and numerical data required from (5, 3 and
926 35) are present and that there is evidence in the design history file for each. In addition it
927 will be apparent that consideration must be given to users who might have different
928 educational, language and technical skills from those with equivalent roles familiar to the
929 manufacturer.

Vulnerability Description Detail Examples

method inappropriate IVD, specimens,
disposal
or not defined accessories
harmful chemicals
warnings none or inadequate biocides,
Safety
thiomersal, azide
SOP invalid,
inactivation of positive
control specimens equipment
specimens not proven
inadequate
diagnostic,
intended use not no statement of screening,
Intent
defined purpose quantitation,
prognostic

44
Vulnerability Description Detail Examples
no restricting statement
of what has been paediatric, elderly,
validated – or no treated
exclusions for
population to be populations not tested
tested not validated in point of care;
claimed medical clinics,
setting laboratories for
screening,
diagnosis
evaluated only by
expert local
not stated or no
intended users laboratories or
supporting data
only by
manufacturer
IgM not detected,
strains of target
known limitations of
organism not
method, reagents,
limitations detected
patient type: some or
(malaria,
all not stated
cholera, HIV),
treated patients
plasma types not
specified or not
validated
specimen types no statement of what
has been validated –
or no exclusion of
types not tested
not evaluated on specificity not
performance appropriate tested
Claims populations appropriately
only by expert
users or in
not evaluated by
precision manufacturers
intended users
own facility and
staff
not proven rigorously based on QC panel
not on the
stability
partial or no stability critical
data for specimens specimens

45
Vulnerability Description Detail Examples
Reading time not
validated at
timings and not validated rigorously
beginning and
volumes or not at all
end of assigned
life
EIA controls set
too high
not proven to warn of
line controls fail
controls damaged IVD with
to mirror state of
lower performance
active
constituents
font too small
type face
users and unsuitable
legibility poor environments not use of white space
considered inappropriate
ink smudges with
time
complex language
no consideration of
poor translations
educational level of
languages in text
intelligibility poor intended users in
not appropriate
intended
for intended
environment
areas of use
not in accord with (36)
symbols used
or absent
Documentation pictures inaccurate
IFU, “simple” method number of drops
mismatches in
guide, training angle of addition
instructions
manuals differ different methods
(times, volumes)
product changes
not present or not
product code nor reflected in
under version control
the product code
version of IFU
provided not the
same as, or
IFU content not traceable through untraceable to,
validation the change system that used in
performance
verification and
validation
Regulation ISO 18113 series requirements not met sections missing

46
 Vulnerability Description Detail Examples
actual OEM, inadequate
manufacturer not addresses,
clear contact details
no audit of
printers
supplier control specifications not
appropriate or
not checked
CAPA data not
IFU not updated as applied between
improvements
appropriate IVD for different
measurands
930 A spreadsheet related to a checklist like this could have further columns indicating what
931 measures need to be taken and by whom.

932 6.6.2 Package labelling

933 Checklists and facilitated brainstorming (9) should be appropriate for managing risks
934 related to package labelling for IVD because the hazards related to transport, storage and
935 use of the IVD need to be well understood from its development processes. WHO PQ
936 expects that package labelling will be suitable for users in environments that might be
937 more extreme than in manufacturers’ home countries and where transport operatives
938 might not read English. It is particularly important that the labelling on the outer package
939 for transport is clear, of a size providing easy legibility under all conditions and uses
940 international symbols (36) appropriately and correctly. Restrictions related to handling
941 and allowable temperatures must be very obviously displayed. Regulatory requirements
942 for labelling are set out in the ISO 18113 (5) series and in (35). The following table lists
943 some of the common, repeated failings of package labelling submitted to WHO PQ. It is
944 intended as a resource for a manufacturer’s checklist but each IVD, in each pack size,
945 must be risk managed individually by the manufacturer. Any statement on labelling (e.g.
946 transport conditions, expiry dating) must be supported by evidence in the design history
947 file of the IVD.

Issue Detail Examples

47
 Font too small
Only English on shipping labels:
Labels obscure
No symbols used
Symbols used incorrectly
Clarity Ink poor quality and smears during transport or when
Ink used not
handled in use
permanent
Ink fades in strong light
Labels detach from
Glue of poor quality: heat labile or water soluble
containers
Evidence for stability only available for one bottle
size although different kit sizes use different bottles
Claims Stability not valid
Shelf-life or transport studies performed using
unlabelled materials
No product code
Not fully compliant No lot number
with (36) and (5) No expiry date
Regulatory or (35) No storage conditions
No single use symbol
Supplier audits
inadequate
948 6.7 Manufacturing

949 6.7.1 Suppliers

950 Control of both suppliers and goods supplied have always been expectations according to
951 ISO 13485; however ISO 13485:2016 now explicitly states that the criteria for their
952 control and management must be proportionate to the risk associated with the medical
953 device (19). Formal, documented risk management in relation to suppliers and goods
954 (which should always include assessments of continuity of supply in addition to the
955 assessment of quality of the goods supplied) is hence a regulatory requirement. The
956 policies and procedures of the organization should specify how this risk management will
957 be performed and documented, and these activities must form part of the design control
958 of the IVD. The mechanisms for these risk management activities are as described
959 previously, probably with an emphasis on ISO 31000 (8) and the methods it specifies.

960 Following from this assessment will be the documented rationale of the extent and
961 frequency of supplier audits and the nature and methods of verifying the incoming goods
962 against the specifications (The incoming goods QA specifications, developed to minimize
963 effects of identified hazards.) As with all risk management, the process is iterative: once

48
964 controls have been developed, the system must be re-assessed to ensure that no new
965 hazards have been created. Then, once the system is operative, it must be continuously
966 reviewed to ensure that it is functional and does not need amendment.

967 In addition to the use of suppliers for basic materials for the manufacture of IVDs,
968 outsourcing of services and component manufacturing is increasingly common. Particular
969 hazrds are associated with outsourcing of manufacturing processes of either subsystems
970 or complete systems. Good risk management can help maintain the balance between
971 quality and cost. This, together with the increased regulatory requirements of risk
972 management (19), make it essential to have a robust risk based approach for evaluating
973 new and existing suppliers.

974 The steps to be taken are as follows:

975 a. Use the already identified critical control points of the product noted.
976 b. Identify specifications that the supplier needs to meet. Document how these are
977 to be evaluated (for example initial product check, audit of supplier) and
978 monitored (for example incoming product QC data).
979 c. Prepare a qualification plan to:
980 – assess the supplied product risk.
981 – assess the supplier risk.
982 – schedule audits.
983 – ensure effective follow-up of nonconformities.
984 – determine frequency of formal review of supplier.
985 d. Prepare supplier quality agreements (SQA). Note: SQA apply to both internal
986 suppliers (for example subsidiary providers under a single ownership) and to
987 external suppliers.

988 The SQA should indicate, for example:

989 – commitment by the supplier to quality.

990 – agreement on how quality will be monitored.
991  frequency and scope of audits defined; allow purchaser’s auditors
992 to review audit reports from other external auditors, especially
993 when related to QMS regulatory approvals.

49
 994  access to supplier QMS management review.
995  change control notification obligation, for example timely
996 notification of changes to product design, manufacturing
997 equipment, critical personnel, QC changes, change in vendor of raw
998 materials etc.
999  the product acceptance criteria defined.
1000  for nonconforming product, a description of actions taken by the
1001 supplier within their QMS.
1002  complaints from other customers of the supplier
1003  field corrective actions – supplier’s involvement and
1004 responsibilities.
1005  environmental controls.
1006  distribution of product, for example shipping conditions
1007 (temperature, humidity, dust, vibration), packaging.
1008 e. Use of tools.
1009 – use FMEA or other such tools.
1010 – develop supplier risk scorecards to rank suppliers based on past and
1011 current performance.
1012 – define key metrics and key performance indicators (KPIs)
1013 f. Demonstrate compliance to auditors and/or regulators. Documentation and
1014 qualified personnel need to be available for interview to support the following:
1015 – qualification plan for each supplier (referred to in supplier audit plans).
1016 – approved supplier list that includes explanation of criticality of supplier,
1017 how this is determined and what impact this has on risk management.
1018 – supplier audits:
1019  that qualifications of auditors are suitable for the task.
1020  objective evidence that specific requirements for the
1021 manufacturer’s particular IVD have been met; compliance with
1022 relevant technical standards may also be required.
1023  well-documented audit findings (reports contain sufficient detail to
1024 illustrate thoroughness of audit).

50
1025  supplier corrective action plan (CAP) or corrective action request
1026 (CAR) is documented and follow-up is completed in an appropriate
1027 time frame.
1028  triggers for action are defined – for example, critical
1029 nonconformities may initiate an immediate meeting with the
1030 supplier, an additional audit or cessation of supply. Justification for
1031 actions taken must be documented.
1032  Evidence that top management has been informed of findings (as
1033 required by the manufacturer’s QMS) and any decisions for action
1034 or no action were taken by qualified personnel.
1035 – Traceability at all steps is essential. Documentation must be readily
1036 accessible for each supplier. For example, although a spreadsheet of KPIs
1037 of all suppliers may be available, and all audit reports kept together, all
1038 data related to a single supplier must be readily available in an assembled
1039 single supplier-specific file for review by an external auditor for regulatory
1040 and compliance purposes. Note that by assembling all notifications of
1041 nonconforming material and other reports from a single supplier, trends
1042 may become evident.
1043 – The risk to the manufacturer’s QMS posed by a poorly performing supplier
1044 must be considered broadly for example the problem could occur with
1045 another supplier and could thus be prevented.
1046 – Evidence of good communication between the supplier and manufacturer
1047 must be available. That is, it should be shown that the supplier is quick to
1048 respond to the manufacturer’s quality concerns, for example with a
1049 corrective action plan; cooperative when scheduling audits; and readily
1050 provides evidence of QMS compliance, for example, certification and audit
1051 reports from EU notified bodies or the US Food and Drug Administration.

1052 Comment: The work of external auditors, although becoming more

1053 harmonized, varies in quality. Regulatory approvals such as certification are
1054 not valuable unless they are supported by the review of the actual audit
1055 reports on which they are based. The manufacturer’s auditor can thus assess

51
1056 the auditors’ skills and the thoroughness of the external audit and hence the
1057 validity of regulatory approvals provided by a supplier.

1058 6.7.2 Manufacturing processes

1059 Each manufacturing process should be risk assessed to ensure the safety of
1060 manufacturing staff together with the capability of the process to produce the planned
1061 results leading to a consistent, safe product. These top level assessments should take into
1062 account the local health and safety regulations as well as the expectations of ISO 14971
1063 (1). There should also be an assessment of:

1064  the materials used in relation to safety, to local environmental regulations and
1065 possibly to ISO 14001 (32).

1066  equipment (purchase, maintenance and cleaning), training needs as

1067 communicated in the manufacturing section of the input documentation, and
1068 cost.

1069  the written procedures to be followed by the operators (legibility under

1070 manufacturing conditions, intelligibility, completeness, and the presence of any
1071 warnings and precautions).

1072  any in-process controls necessary to ensure consistency within the process and
1073 the methods for those controls.

1074  potential effects of differences in the scale of the process (especially in stability
1075 and specificity of the final IVD), and the necessity for validation of different scales
1076 of manufacture.

1077  the necessity for validation or verification of the product of the process, and the
1078 methods for performing those activities.

1079 The mechanism for process risk management is most often an FMEA led by the
1080 manufacturing department with input at least from R&D and QA. A flow diagram of the
1081 process being evaluated is essential as is detailed technical knowledge of the materials
1082 being used, the capability of the manufacturing department (before and after any
1083 controls introduced as a result of the assessment) and the reasons for the specifications
1084 for the product of the process. Figure 5 shows fragments of the risk management
52
1085 documentation for a manufacturing process. It shows only a part related to the actual
1086 manufacturing steps, not all the aspects listed above. Note that the work instructions are
1087 written in a structured fashion that relates easily to a flowsheet and then to the FMEA.
1088 This makes the process easy to manage, explain and update as necessary.

1089 It should be possible to trace each control action back to the hazard that it controls and
1090 forward to the validation of the control and the test methods involved. (See the Test
1091 Method Validation guide, TGS-4 in this series (37))

Fragment of a process instruction showing the hierarchy of actions for a single step
1. Ink Jet Mark the plates
Remove plates from boxes
Transport to coating
Cover print site with label
Print label
Check the label and sign for it
Ink Jet mark whole batch
Stack marked plates
Put stacks in cabinets
Record cabinet number

2. Prepare for coating

Record room temperature
Check bed of instrument
Collect coating reagents
Record time of spiking

3. Set the dispensed volume

Adjust the dispenser to 110 l
Load 40 plates, no frames
Calibrate the dispenser

Fragment of the flowchart for the same process, showing several steps

53
 Fragment of the FMEA document for the same process

Figure 5 Risk management for a manufacturing process

1092 6.7.3 Safe documentation

1093 It is essential that each procedure is assessed for the safety of the process concerned.
1094 This should be a priority of management of manufacturing and needs emphasis. In
1095 particular, appropriate warnings (about hazards to operator safety) and cautions (about
1096 hazards to equipment) must be included and evaluated.

1097 6.7.4 Process changes in manufacturing

1098 Any change to a process must be managed within the change control system of the QMS
1099 but the policies concerning change must also make reference to re-assessing the effect of
1100 any changes on the established risk profile. This applies even to minor changes to a
1101 manufacturing process. Such changes must be managed, details recorded and the risk
1102 management documents updated accordingly, even if the outcome is that there is no
1103 change to the risks.

1104 Example: Nonconformities noted during WHO inspections of production lines have
1105 included the following:

1106  A lack of staff well trained in risk management techniques and hence the
1107 inadequate application of such techniques to the production line. This has
1108 resulted in observing unsafe practices of personnel, for example not
54
1109 wearing hearing protection and unsafe handling of infectious materials, as
1110 well as hazard to product quality, for example inappropriate handling of
1111 labile biological materials.
1112  Well-structured and sufficiently detailed batch manufacturing records
1113 (BMRs) play an important role in reducing manufacturing risk. Analysis of
1114 BMRs by suitably qualified personnel was poorly performed leading to lost
1115 opportunities for preventive action to maintain quality. Quality control data
1116 in BMRs (checking of the output being within specifications and trend
1117 analysis of the data) were not adequately reviewed and thus trends
1118 towards failure modes were not detected.
1119  Reporting on deviations (products not meeting specifications) and
1120 appropriate follow-up actions were often inadequate.

1121 References
1122 1. ISO 14971:2007 and EN ISO 14971:2012 Medical devices – Application of risk management to
1123 medical devices. Geneva: International Organization for Standardization; 2007.

1124 2. ISO Guide 73:2009 Risk management — Vocabulary. Geneva: International Organization for
1125 Standardization; 2009.

1126 3. United States CFR 21- Code of Federal Regulations Title 21. Sec. 820.3 Definitions.

1127 4. ISO 9000:2005 Quality management systems – Fundamentals and vocabulary. Geneva:
1128 International Organization for Standardization; 2005.

1129 5. ISO 18113-1:2009 In vitro diagnostic medical IVDs – Information supplied by the manufacturer
1130 (labelling) – Part 1: Terms, definitions and general requirements. Geneva: International
1131 Organization for Standardization; 2009.

1132 6. WHO PQDx_018: Instructions for Compilation of a Product Dossier. Geneva: World Health
1133 Organization; 2014.

1134 7. WHO PQDx_014: Information for Manufacturers on the Manufacturing Site(s) Inspection
1135 (Assessment of the quality management system). Geneva: World Health Organization;
1136 2017.

55
1137 8. ISO 31000:2009 Risk management — Principles and guidelines. Geneva: International
1138 Organization for Standardization; 2009.

1139 9. ISO 31010:2009 Risk management – risk assessment techniques. Geneva: International
1140 Organization for Standardization; 2009.

1141 10. CLSI: Laboratory quality control based on risk management, approved guideline (CLSI
1142 document EP23-A). Wayne, PA: Clinical and Laboratory Standards Institute; 2011.

1143 11. CLSI: Quality management system: development and management of laboratory documents:
1144 approved guideline (CLSI document QMS02-A6, sixth edition). Wayne, PA: Clinical and
1145 Laboratory Standards Institute; 2013.

1146 12. GHTF/SG3/N015:2005 Implementation of risk management principles and activities within a
1147 quality management system. Global Harmonization Task Force (GHTF); 2005.

1148 13. GHTF/SG1/N068:2012 Essential principles of safety and performance of medical devices.
1149 Global Harmonization Task Force (GHTF); 2012.

1150 14. ISO/DIS 16142-2:2016 Medical devices – Recognized essential principles of safety and
1151 performance of medical devices – Part 2: General essential principles and additional
1152 specific essential principles for all IVD medical devices and guidance on the selection of
1153 standards. Geneva: International Organization for Standardization; 2016.

1154 15. Shebl NA, Franklin BD, Barber N. Is failure mode and effect analysis reliable? J Patient Saf.
1155 5;2009;86–94.

1156 16. Potts HWW, Anderson JE, Colligan L, et al. Assessing the validity of prospective hazard analysis
1157 methods: a comparison of two techniques. BMC Health Serv Res. 14;2014:41–50.

1158 17. Ashley L, Armitage GJ. Failure mode and effects analysis: an empirical comparison of failure
1159 mode scoring procedures. Patient Saf. 6;2010:210–215.

1160 18. Shebl NA, Franklin BD, Barber N. Failure mode and effects analysis outputs: are they valid?
1161 BMC Health Serv Res. 12:2012;150–160.

1162 19. ISO 13485:2016 Medical devices – Quality management systems – Requirements for regulatory
1163 purposes. Geneva: International Organization for Standardization; 2016.

56
1164 20. WHO Prequalification: Sample product dossier for a qualitative test for HIV-1 and HIV-2: 'Risk
1165 analysis and control summary' 5.4 and Annexes II, III, IV and V. Geneva: World Health
1166 Organization
1167 (http://www.who.int/diagnostics_laboratory/guidance/160613_sample_product_dossier
1168 _for_qualitative_nucleic_acid_test_hiv1_2.pdf?ua=1).

1169 21. WHO Prequalification: sample product dossier for a quantitative nucleic acid test to detect
1170 HIV-1 RNA’: 'Risk analysis and control summary' 5.4 and Annexes II, III, IV and V. Geneva:
1171 World Health Organization
1172 (http://www.who.int/diagnostics_laboratory/guidance/161026WHO_QuantHIV_sample_
1173 dossier.pdf?ua=1).

1174 22. WHO Prequalification: Sample product dossier for an IVD intended, for HIV self-testing
1175 Geneva: World Health Organization
1176 (http://www.who.int/entity/diagnostics_laboratory/guidance/160613_sample_product_
1177 dossier_for_intended_hiv_self_testing.pdf?ua=1).

1178 23. Franklin DB, Shebl NA, Barber N. Failure mode and effects analysis: too little for too much? J
1179 Qual Saf. 21:2012; 607–611.

1180 24. ISO 10005:2005 Quality management systems — Guidelines for quality plans. Geneva:
1181 International Organization for Standardization; 2005.

1182 25. CLSI: Risk management techniques to identify and control laboratory error sources; approved
1183 guideline. Second edition CLSI document EP18-A2 (ISBN 1-56238-712-X). Wayne, PA:
1184 Clinical and Laboratory Standards Institute; 2009.

1185 26. American Society for Quality (ASQ) (http://asq.org/learn-about-quality/seven-basic-quality-

1186 tools/overview/overview.html).

1187 27. CLSI: Nonconforming event management. Second edition (CLSI guideline QMS11) Wayne, PA:
1188 Clinical and Laboratory Standards Institute; 2015.

1189 28. Kusselman I. Pennechi F., UPAC/CITAC Guide: Classification, modeling and quantification of
1190 human errors in a chemical analytical laboratory. Pure Appl Chem. 88;2016:477–515.

57
1191 29. Krouwer J. (https://jkrouwer.wordpress.com/2016/11/01/westgards-detection-and-iqcp/,
1192 accessed 6 January 2017).

1193 30. Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on
1194 in vitro diagnostic medical devices.

1195 31. ISO 15198:2004 Clinical laboratory medicine – In vitro diagnostic medical devices – Validation
1196 of user quality control procedures by the manufacturer. Geneva: International
1197 Organization for Standardization; 2004.

1198 32. ISO 14001:2015 Environmental management systems – Requirements with guidance for use.
1199 Geneva: International Organization for Standardization; 2015.

1200 33. WHO prequalification: ‘Principles of performance studies’ Technical Guidance Series (TGS) 3.
1201 Geneva: World Health Organization
1202 (http://www.who.int/diagnostics_laboratory/guidance/160613_tgs3_principles_for_perf
1203 ormance_studies.pdf?ua=1).

1204 34. WHO prequalification: “Designing Instructions for use for in vitro diagnostic medical devices”
1205 TGS 5. Geneva: World Health Organization

1206 35. US FDA “In Vitro Diagnostic Device Labeling Requirements” 2014
1207 https://www.fda.gov/medicaldevices/deviceregulationandguidance/overview/devicelabe
1208 ling/invitrodiagnosticdevicelabelingrequirements/default.htm

1209 36. ISO 15223-1:2016. Medical Devices - Symbols to be used with medical device labels, labelling
1210 and information to be supplied - Part 1: General requirements. Geneva. International
1211 Organization for Standardization; 2016.

1212 37. WHO prequalification: ‘Guidance on Test Method Validation for in vitro diagnostic medical
1213 devices’ Technical Guidance Series (TGS) 4 Geneva: World Health Organization

58