Guideline WHO - Guidance On Test Method Validation of in Vitro Diagnostic Medical Devices PDF

Technical Guidance Series (TGS)
for WHO Prequalification – Diagnostic

Assessment
Guidance on Test Method

Validation of in vitro diagnostic TGS–4
medical devices
Draft for comment 20 December 2016

© World Health Organization 2016
All rights reserved. Publications of the World Health Organization can be obtained from WHO Press, World Health Organization,
20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857; e-mail: bookorders@who.int).
Requests for permission to reproduce or translate WHO publications – whether for sale or for non-commercial distribution –
should be addressed to WHO Press, at the above address (fax: +41 22 791 4806; e-mail: permissions@who.int).
The designations employed and the presentation of the material in this publication do not imply the expression of any opinion
whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of
its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border
lines for which there may not yet be full agreement.
The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or
recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and
omissions excepted, the names of proprietary products are distinguished by initial capital letters.
All reasonable precautions have been taken by the World Health Organization to verify the information contained in this
publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The
responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization
be liable for damages arising from its use.
Contact: Irena Prat, EMP Prequalification Team Diagnostics
WHO - 20 Avenue Appia - 1211 Geneva 27 Switzerland

Technical Guidance Series for WHO prequalification – Diagnostic assessment:
Guidance on Test method validation of in vitro diagnostic medical devices TGS–4
Contents
Contents 3
Acknowledgements 5
1 Definitions 6
2 Introduction 8
3 Scope 8
4 Terminology for test method validation 8
4.1 Explanation of the terms characterisation, verification and validation .................... 8
4.2 Explanation of the terms accuracy, trueness and precision ................................... 10
5 Uses of test method validation in the lifecycle of the IVD 11
6 Test methods 11
6.1 Categories of test methods ..................................................................................... 11
6.2 Statistics and test methods ..................................................................................... 11
6.3 Quantitative and qualitative assays ........................................................................ 11
6.4 Specimen panels and test methods ........................................................................ 12
7 Variability in the test method 12
8 Planning for test method validation 13
9 Examples of test methods and their validation 14
9.1 Validation of test methods related to cleaning processes ...................................... 14
9.2 Validation of test methods for raw materials ......................................................... 16
10 References 20
Page 3 of 21 Draft for comment 20 December 2016

WHO Prequalification – Diagnostic Assessment: Technical Guidance Series

WHO The World Health Organization (WHO) Prequalification Programme is coordinated
Prequalification through the Department of Essential Medicines and Health Products. The aim of WHO
– Diagnostic
prequalification of in vitro diagnostic medical devices (IVDs) is to promote and facilitate
Assessment
access to safe, appropriate and affordable IVDs of good quality in an equitable manner.
Focus is placed on IVDs for priority diseases and their suitability for use in resource-
limited settings. The WHO Prequalification Programme undertakes a comprehensive
assessment of individual IVDs through a standardized procedure aligned with
international best regulatory practice. In addition, the WHO Prequalification Programme
undertakes post-qualification activities for IVDs to ensure the ongoing compliance with
prequalification requirements.
Procurement of Products that are prequalified by WHO are eligible for procurement by United Nations
prequalified (UN) agencies. The products are then commonly purchased for use in low- and middle-
IVDs
income countries.
Prequalification IVDs prequalified by WHO are expected to be accurate, reliable and be able to perform
requirements
as intended for the lifetime of the IVD under conditions likely to be experienced by a
typical user in resource-limited settings. The countries where WHO-prequalified IVDs
are procured often have minimal regulatory requirements. In addition, the use of IVDs
in these countries presents specific challenges. For instance, IVDs are often used by
health care workers without extensive training in laboratory techniques, in harsh
environmental conditions, without extensive pre- and post-test quality assurance
capacity, and for patients with a disease profile different to those encountered in high
income countries. Therefore, the requirements of the WHO Prequalification Programme
may be different to the requirements of high-income countries, and/or of the regulatory
authority in the country of manufacture.
About the The Technical Guidance Series was developed following a consultation, held on 10-13
Technical March 2015 in Geneva, Switzerland which was attended by experts from national
Guidance regulatory authorities, national reference laboratories and WHO prequalification dossier
Series
reviewers and inspectors. The guidance series is a result of the efforts of this and other
international working groups.
Audience and This guidance is intended for manufacturers interested in WHO prequalification of their
scope IVD. It applies in principle to all IVDs that are eligible for WHO prequalification for use in
WHO Member States. It should be read in conjunction with relevant international and
national standards and guidance.
The TGS guidance documents are freely available on the WHO web site.

Acknowledgements
The draft document “Technical Guidance Series for WHO Prequalification – Diagnostic Assessment:
Guidance on Test method validation of in vitro diagnostic medical devices” was developed with support
from the Bill & Melinda Gates Foundation and UNITAID. This draft was prepared in collaboration with Dr
J Duncan, London, United Kingdom; D Healy; R Meurant, WHO; and with input and expertise from Dr V
Alcón; D Lepine; IVDD Section, Medical Devices Bureau Health Canada, Ottawa, Canada; Dr S Hojvat,
MD, USA; and Dr S Norman, CA, USA. This document was produced under the coordination and
supervision of Robyn Meurant and Irena Prat, Prequalification team – Diagnostic Assessment, WHO,
Geneva, Switzerland.

1 Definitions
1 The section below provides definitions which apply to the terms used in this document.
2 Accuracy: The closeness of agreement between a test result and the accepted reference value.
3 (‎1)
4 Lot: Defined amount of material that is uniform in its properties and has been produced
5 in one process or series of processes.
6 NOTE: The material can be either starting material, intermediate material or finished
7 product. (‎2)
8 Characteristic: Distinguishing feature
9 Note 1 to entry: A characteristic can be inherent or assigned.
10 Note 2 to entry: A characteristic can be qualitative or quantitative. (3)
11 Note 3 Characterisation: a description of the distinctive nature or features of
12 something. (3)
13 Control material: Substance, material or article used to verify the performance characteristics of an in
14 vitro diagnostic medical device. (4)
15 Control procedure: Activities at the point of use to monitor the performance of an IVD medical device.
16 Note 1 In the IVD medical device industry and in many laboratories that use IVD
17 medical devices, these activities are commonly referred to as quality control.
18 Note 2 Quality control may monitor all or part of the measurement procedure, from
19 the collection of samples to reporting the result of the measurement. (4)
20 In vitro diagnostic medical device (IVD): A medical device, whether used alone or in combination,
21 intended by the manufacturer for the in vitro examination of specimens derived
22 from the human body solely or principally to provide information for diagnostic,
23 monitoring or compatibility purposes.
24 Note 1 IVDs include reagents, calibrators, control materials, specimen receptacles,
25 software, and related instruments or apparatus or other articles and are used, for
26 example, for the following test purposes: diagnosis, aid to diagnosis, screening,
27 monitoring, predisposition, prognosis, prediction, determination of physiological
28 status.
29 Note 2 In some jurisdictions, certain IVDs may be covered by other regulations. (‎5)
30 In vitro diagnostic reagent/IVD reagent: Chemical, biological or immunological components, solutions,
31 or preparations intended by the manufacturer to be used as an IVD. (‎2)
32 Life-cycle: All phases in the life of a medical device, from the initial conception to final
33 decommissioning and disposal. (‎6)
34 Limit of detection, detection limit: Measured quantity value, obtained by a given measurement
35 procedure, for which the probability of falsely claiming the absence of a component
36 in a material is β, given a probability α of falsely claiming its presence.

37 Note 1 IUPAC recommends default values for α and β equal to 0.05.

38 Note 2 The term analytical sensitivity is sometimes used to mean detection limit, but
39 such usage is now discouraged. Modified from (7)
40 Limit of quantitation, quantitation limit: Lowest value of measurand in a sample which can be
41 measured with specified measurement uncertainty, under stated measurement
42 conditions. (7)
43 Measurand: Quantity intended to be measured. (‎7)

44 Objective evidence: data supporting the existence or verity of something
45 Note 1 Objective evidence can be obtained through observation, measurement, test,
46 or by other means. (‎2)
47 Performance claim: Specification of a performance characteristic of an IVD medical device as
48 documented in the information supplied by the manufacturer.
49 Note 1 This can be based upon prospective performance studies, available
50 performance data or studies published in the scientific literature. (2)
51 WHO Note “Information supplied by the manufacturer” includes but is not limited
52 to: statements in the instructions for use, in the dossier supplied to WHO and / or
53 other regulatory authorities, in advertising, on the internet referred to simply as
54 “claim” or “claimed” in this document.
55 Precision: The closeness of agreement between independent test results obtained under
56 stipulated conditions. (‎1)
57 Quality: Degree to which a set of inherent characteristics of an object fulfils requirements. (3)
58 WHO Note: for the purpose of this document these requirements include fitness-for-
59 use, safety and performance.
60 Quality assurance: Part of quality management focused on providing confidence that quality
61 requirements will be fulfilled. (‎3)
62 Ruggedness (robustness): A measure of an analytical procedure’s capacity to remain unaffected by
63 small but deliberate variations in method parameters and provides an indication of
64 its reliability during normal usage. (‎8)
65 Standard method: A method that is (metrologically) traceable to a recognized, validated method.
66 Non-standard method: A method that is not taken from authoritative and validated
67 sources. This includes methods from scientific journals and unpublished laboratory-
68 developed methods. (‎8)
69 Trueness: The closeness of agreement between the average value obtained from a large series
70 of test results and an accepted reference value. (‎1)
71 Validation: Confirmation by examination and provision of objective evidence that the
72 requirements for a specific intended use have been fulfilled. (‎3)
73 Verification: Confirmation through the provision of objective evidence that specified
74 requirements have been fulfilled. (‎3)

2 Introduction
75 The purpose of test method validation is to ensure that a method consistently provides results fit or
76 adequate for a specific purpose. Testing must have a useful purpose and the result from the test must
77 be shown to be meaningful and to give the expected (and appropriate) information. In order to ensure
78 meaningful results, the test method must be validated; otherwise the measurement has little purpose
79 and no economic value. By using validated test methods, a manufacturer can have confidence that
80 claims made in respect to the quality and performance of an IVD are supported by objective evidence.
3 Scope
81 This document is intended to provide guidance on the validation of the test methods used in
82 manufacturing of an IVD. Sometimes test methods are referred to as analytical methods but in the
83 context of establishing the design, the development and manufacture of an IVD, “test method” is the
84 more commonly used and a more appropriate description since not all testing is analytical. Minimal
85 specific guidance relating to test method validation is available for IVD manufacturers despite the
86 abundance of guidance for the analytical chemistry or pharmaceutical industries (e.g. those from
87 Eurachem (10), Eurolab (11), ICH (12), WHO (13) and FDA (14)) or for clinical laboratories compliant with
88 ISO 15189 (15). This document provides information on validating the test methods used by
89 manufacturers of IVDs in their research and development (R&D), quality control and quality assurance
90 laboratories; it must be read as an adjunct to those formal guides mentioned previously.
91 This document is not intended to give guidance on validation of the IVD itself. For this, it is
92 recommended to refer to “TGS3 Principles of performance studies” (16) in this series. Qualification of
93 instrumentation is outside the scope of this document although the test methods used in qualification
94 must be validated (17). This document does not outline statistical methods for analysis of the required
95 data.
4 Terminology for test method validation
96 4.1 Explanation of the terms characterisation, verification and validation

97 Although internationally accepted definitions exist for the terms characterisation, verification and
98 validation, the following explanation is provided to give greater clarity with relation to test method
99 validation.
100 Characterisation, verification and validation are essential terms. For the purposes of this guidance
101 document “characterisation of a test method” refers to an experimental procedure and the
102 documentation of its characteristics. It is undertaken in order to provide objective evidence of what a
103 method is capable of consistently achieving under defined conditions. The characteristics of the assay
104 are the numerical values proven and documented for each of the method attributes such as sensitivity,
105 specificity, limit of detection etc. Each attribute should be evaluated using an appropriate, validated test
106 method.
107 Verification is the documentary proof that particular specifications have been met. When designing and
108 developing an IVD, relevant attributes such as cost, and those for performance such as precision,
109 sensitivity and stability are identified and given numerical specifications in design input documentation.
110 It is subsequently the role of the R&D department to design an IVD that will meet those specifications.

111 The R&D department consequently identifies valid test methods to demonstrate that the specifications
112 have been met (verification) in the new design. Once design has been established, further numerical
113 specifications are produced by the R&D department to ensure that the specifications of each attribute
114 will be met consistently in routine production to ensure quality manufacturing. These new specifications
115 are assigned to control critical production points and may include those for acceptance of raw materials,
116 in-process materials, cleanliness of equipment, qualification of instrumentation and for the finalised IVD
117 to verify its manufacture. Again, it is also the role of the R&D department to identify appropriate test
118 methods to monitor these specifications. An example of verification is related to incoming goods
119 inspections; each time a raw material is purchased its properties will be verified against the specification
120 using a validated test method.
121 Validation is the documentary proof that the particular requirements for a specific intended use can be
122 consistently fulfilled (9). VIM (7) defines validation as “verification against needs for a specific use” (i.e.
123 the specification for that use). Within this guide, consistency is essential: it is an expectation that every
124 lot of an IVD will behave as all other lots and will continue to meet design inputs. To ensure this, it is
125 necessary to have validated test methods for measuring and/or monitoring specifications that will
126 consistently produce results fit for purpose. The test methods must be validated to ensure that the
127 results of measuring and/or monitoring are meaningful. For example, the need for accurate
128 measurement of a raw material weighed in micrograms will not be achieved by using a weighing device
129 with tolerance measured in grams. A test method using such an instrument would not be valid for the
130 intended use. Thus, for the example provided, a test method should be specified that has the necessary
131 accuracy and precision for measuring such weights, and an instrument and procedure identified that will
132 consistently achieve this requirement during its use. The test method is then validated to produce
133 results fit for purpose.
134 Validation of a test method is distinct from its characterisation. Characterisation is documentation of
135 some or all of the features of the method; validation is ensuring that the relevant characteristics are
136 appropriate for the specific intended use. Validation of a method to be used widely, and for standard
137 methods, often begins with complete characterisation. However, for each specific intended use it is
138 likely that only a subset of the characteristics will be relevant and must be evaluated.
139 Published guides to test method validation provide the broad characteristics of assays as set out in Table
140 1.

141 Table 1: Examples of characteristics of assays

Characteristic Attributes
Trueness bias, recovery, accuracy, matrix effects

Precision repeatability, intermediate precision, reproducibility
Selectivity specificity, interferences
Sensitivity traceability, working range, linear range, limit of detection, limit of
quantitation, uncertainty at clinically significant threshold values
Reproducibility precision under defined conditions: repeatability, ruggedness, robustness
Stability reagents, analyte (specimen types)
Productivity† speed, hazards, cost
† Productivity is not usually mentioned in test method validation texts but is important in
manufacturing environments. Although cost must not be a factor considered in risk minimization (6), it
should be a consideration in choice and validation of test methods.
142 Usually only a small selection of the possible characteristics and attributes will ever be studied for a test
143 method specifically developed for a single purpose.
144 4.2 Explanation of the terms accuracy, trueness and precision

145 The terms accuracy, trueness and precision have specific meaning for technical documentation.
146 Trueness and accuracy are the values obtained for the method under investigation and relative to a
147 value accepted as truth, being established through the use of an accepted traceable calibrator or derived
148 by testing using an accepted reference measurement method on the same item as measured by the
149 method under test. Without an accepted value neither trueness nor accuracy can be given for a test
150 method, only a percent (positive and/or negative) agreement.
151 Trueness is a measure of the closeness of agreement between the accepted value and the average of a
152 large [infinite] number of results from a test or assay method under review. It is expressed as a bias: “the
153 result from this test method has a bias of ±units”. Trueness is a characteristic of the method.
154 Precision is a comparison of the results obtained on the same test method. It does not encompass
155 comparison to another value obtained using a method associated with trueness. It is a measure of the
156 closeness of agreement between independent test results obtained under stipulated conditions using
157 the same test method and encompasses concepts such as repeatability and reproducibility, depending
158 on the specified conditions. It is expressed in terms of a standard deviation or related measures “the
159 precision (or repeatability or reproducibility etc.) of this test method under these conditions is ±y-units”.
160 Precision is a characteristic of the method.
161 Accuracy is a measure of the closeness of agreement between the accepted value as documented and
162 the result of a measurement using the item (i.e. test equipment etc.). It is a characteristic of that single
163 measurement and has components from both trueness and precision of the test method. Each time the
164 test method is performed the accuracy of the measurement is likely to be different because of
165 experimental error and the imprecision of the test method. If a test method requires the documented
166 result to be the average of several individual measurements, the accuracy is related to that average; the

167 fact of limited replication does not convert accuracy to trueness although it may improve the accuracy.
168 Accuracy is expressed in terms of bias: “the accuracy of that measurement was -z-units” (‎1, ‎7).
169 Both precision and trueness for a particular method, and subsequently the accuracy of an assay by that
170 method, can be influenced by the concentration of the measurand. As such, when characterising a
171 method, knowledge of the performance of the assay should be obtained over a range of foreseeable
172 measurand concentrations, to ensure the validity of any assumptions regarding the performance of the
173 method.
5 Uses of test method validation in the lifecycle of the IVD

174 Testing in the R&D phase of the life-cycle of a commercial IVD is often to ensure that the work in
175 progress will meet the input requirements or to verify that the test development meets those
176 requirements. Design requirements such as a claim of lack of interference from similar analytes will need
177 to be supported by validated test methods.
178 During production, testing is usually employed to ensure that the material being tested meet its
179 specifications. Test methods will use classical analytical chemistry or biochemistry to evaluate the quality
180 of materials coming into, or synthesised by the factory e.g. commercial chemicals, enzymes,
181 recombinant proteins, peptides or nucleic acids. Validation of the test methods will ensure that the
182 correct attributes are measured appropriately.
6 Test methods
183 6.1 Categories of test methods

184 Test methods can be categorized as standard or non-standard. Standard methods are metrologically
185 traceable to a recognized, validated method and do not require additional characterisation by IVD
186 manufacturers. Pharmacopoeia and various national regulations document approved standard methods.
187 In contrast, non-standard methods must be individually characterised and validated for the intended
188 use. However, all methods must be assessed as appropriate for the specific intended use, and must be
189 verified as being used correctly (6, 13, 19).
190 6.2 Statistics and test methods

191 It is recommended to seek expert statistical advice during the planning stage of all experiments to
192 ensure that sufficient numbers of specimens are tested to provide statistically powered results. These
193 are required to justify any claim, and to provide reasonable estimates of uncertainty.
194 Frequently statistical differences will be found that have no practical consequence. For that reason
195 practical differences, or limits of confidence, should always be defined before experiments are
196 performed.
197 6.3 Quantitative and qualitative assays

198 Most test methods will produce numeric, quantitative results, but some assays can only produce
199 qualitative output: the binary result of analyte present or analyte absent relative to a particular cut-off
200 value. For qualitative assays some of the characteristics listed in section 4 Table 1 cannot be enumerated
201 without applying advanced statistical methods. For guidance on this issue see Valcárcel et al. 2002 (20).

202 Some IVDs are intended only to produce qualitative results in users’ environments but it is usual (and is
203 probably essential) that the test methods used in manufacture (quality assurance, quality control) will
204 provide a quantitative result. In most cases qualitative assays can be adjusted to provide a quantitative
205 result, either from an instrumental reading, e.g. for an enzyme immunoassay, or against a graduated
206 reference scale (semi-quantitative reporting of a present/absent result as is the case with many rapid
207 diagnostic tests) . If a quantitative result cannot be obtained then experiments and results must be
208 designed to be analysed by appropriate qualitative statistical methods. Documenting an outcome as
209 merely positive or negative without giving an uncertainty estimate is rarely sufficient, particularly for
210 test methods intended to characterise an IVD (e.g. for stability, precision, sensitivity) or for release-to-
211 sale testing.
212 6.4 Specimen panels and test methods

213 Test methods used to verify design and consistent production will frequently involve the choice and use
214 of panels of specimens in order to determine and/or monitor quality characteristics of an IVD. Panels
215 must be designed and specimens selected to ensure that data generated usefully demonstrates that the
216 specifications have been met. Designing a valid method to assess sensitivity of antibody detection for
217 example, will need to take into account the fact that testing of dilutions of a strong positive specimen
218 will not produce results that reflect the performance of the assay with respect to seroconversion
219 sensitivity. Similarly, the panel composition for release-to-sale and stability testing must employ panels
220 utilising specimens demonstrated to reflect the state of an IVD relative to real, critical specimens to be
221 valid. It is useful to note that test methods used for an IVD in both the design and development phases
222 as well as during production can have various applications, for example the experiments undertaken at
223 release-to-sale can be also be used with adjusted criteria in proving the stability of the IVD.
7 Variability in the test method

224 A critical attribute of all test methods is that they must be less variable than the parameter being
225 evaluated (i.e. have a higher precision). The variability of the test method must not conceal variability in
226 that which is tested. This requirement is usually studied as “gauge R&R” (Gauge Repeatability and
227 Reproducibility, refer to Burdick et al (21)) but the process and methodology applies to any measuring
228 system, not just to gauges. As a rule of thumb, the gauge should have a variance of less than 20% of the
229 variance of the “test-piece”. In this context, it is unreasonable to make claims based on one lot of an IVD
230 evaluated in one or two similar laboratories, regardless of how many individual specimens are tested. It
231 is important to understand the variability between lots of IVD and the test method must be capable of
232 revealing it. For instance, it is an accepted published practice to use three lots of an IVD to demonstrate
233 stability (22), however, no guidance is provided on what actions are required when significant lot-to-lot
234 variability is identified during stability testing. Shelf-life should be assigned statistically taking into
235 consideration the variance between lots (see “TGS-2 Guidance Document for establishing stability of in
236 vitro diagnostics assays and components” in this series (23)). Due to the requirement of high precision of
237 the test method, using a previous lot of the IVD as a simple comparator is unlikely to meet the
238 requirements of being a validated test method unless there is sufficient knowledge and control of the
239 variability associated with each lot. The concerns regarding variability apply equally to all claims,
240 including specificity and sensitivity.

8 Planning for test method validation

241 Figure 1: Test method validation process
242 The flow of the test method validation process is Define
243 shown in Figure 1. The steps will be described in the purpose of the
244 detail in the examples of test method validation to testing
245 follow.
Identify the
246 Understanding the real, intended purpose of the test required
247 method is critical. The US FDA (24) states that: characteristics of
248 “Design input is the starting point for product design. the test method
249 The requirements which form the design input
250 establish a basis for performing subsequent design Assign numerical
251 tasks and validating the design. Therefore, values to the
attributes
252 development of a solid foundation of requirements is
253 the single most important design control activity”.
254 ISO 13485 (25) requires that “design and
Select or develop
255 development outputs shall meet the input
the test method
256 requirements”. Compiling requirements and
257 comparing outputs to inputs is essential for activities
258 that require detailed planning and execution. NO
Compare
select a different
259 Once the input requirements which define the exact performance with
method
requirements
260 purposes for the test method are documented and or redevelop
261 agreed (e.g. to ensure a particular claim will be met
262 for the whole shelf-life of the IVD), the required
263 characteristics of the test method can be specified Is the
method fit for
264 and given measurable attributes, usually following
purpose?
265 risk assessments and development work.
266 The following are examples of input requirements: YES
267 ability to detect an increase of 0.5% in the invalid
268 result rate of an IVD; ability to detect 10% loss of Use method
269 sensitivity for a particular epitope; evidence that
270 infectivity of a positive control material is reduced by >100-fold.
271 The method can then be developed and validated against these predetermined needs. Without the
272 predetermined needs the method cannot be validated, merely characterised.
273 The analysis of incoming raw materials is usually supported by well-established standard methods
274 and/or published routine test method validation. There are usually a greater array of test methods (e.g.
275 chemical analyses, protein, peptide or nucleic acid sequencing or terminal analyses, spectroscopy,
276 chromatography, electrophoresis, electro-blotting) than for routine quality control or quality assurance
277 of an IVD itself but the link between specifications, predetermined test method characteristics, the
278 capability of the method and the utility of results must still be documented.
279 For quality assurance and quality control of an IVD, the test method frequently requires decisions on the
280 attributes to be measured and numbers of specimens for testing panels. The finalised test methods and
281 specimens to be used are usually developed together during the R&D for an IVD. Given the important

282 relationship between the chosen test method and the testing panel, it is almost always too late to try to
283 find appropriate specimens for the panels after R&D is completed.
9 Examples of test methods and their validation

284 This section gives examples of test methods and their validation for some aspects of IVD manufacture.
285 The examples were generated following WHO analysis of the deficiencies in evidence provided dossiers
286 submitted to support prequalification. The analysis concluded poor understanding of the importance of
287 test method validation, and its necessity in support of claims (for example on lot to lot reproducibility,
288 stability, specimen types). A manufacturer will need to evaluate each phase of work, processes and
289 materials and adapt the procedure outlined in section 08 to the particular IVD. The examples in this
290 document are neither authoritative nor complete. However, if the test method is not valid and
291 documented, the claim is not supported.
292 9.1 Validation of test methods related to cleaning processes

293 Introductory discussion
294 This example of test method validation is of validation of the methods used in verifying cleanliness, not
295 validation of the cleaning process itself. The specifications of what constitutes “clean” must be
296 ascertained on a case by case basis. This is usually from risk analysis based on chemical knowledge of the
297 reasons the cleaning process is necessary and some experimental evidence: why cleaning is essential,
298 the cleaning agents used and the probable subsequent uses of the cleaned item. Once the specifications
299 for cleanliness are known, proven and documented, the requirements of the test methods can be
300 defined.
301 As a simple example consider cleaning a vessel used for preparing conjugates, last used for a conjugation
302 of a monoclonal antibody with an enzyme and now to be used for preparing other conjugates.
303 a) Define the purpose of the testing
304 The residual conjugation chemicals, antibody and enzyme and subsequently any cleaning agents must all
305 be removed in order not to contaminate the next solutions in the vessel.
306 The vessel will be cleaned with pressurised hot water containing an organic anionic detergent followed
307 by alkali and acid rinses and finally rinsing with distilled water and drying.
308 b) Identify the required characteristics
309 The characteristics required are trueness, sensitivity, selectivity and precision for each of the possible
310 analytes.
311 The criteria for successful cleaning could be based in a standard operating procedure requiring vigorous
312 extraction of the cleaned vessel with a defined volume of distilled water prior to any drying stage in
313 order to avoid artefacts from an unclean vessel appearing clean because of difficulty in detecting dried-
314 on contaminants. The methods must be able to detect any contamination of the water that could in
315 principle affect subsequent use of the vessel.
316 c) Assign numerical values to the attributes
317 Typically the specification for the water after rinsing could be: less than 10 ppm of total organic carbon,
318 less than 5 ppm of residual protein, less than 1 ppm of residual detergent, less than 1 µM in conjugation
319 related reagents and a conductivity of less than 0.5µS.

320 These are typical specifications for equipment used for fermentation or protein handling and experience
321 shows that if they are achieved the vessel is likely to be acceptably clean for these purposes. However
322 the utility of the cleaning process with these specifications would need to be validated (26) for the
323 specific use before finalising and validating the test methods.
324 d) Select or develop the method(s)
325 The extraction of the water from the vessel is part of the test method: it is required to demonstrate
326 during validation by R&D that a second, similar extraction would contain unmeasurable amounts of the
327 potential contaminants and, independently, that nothing of practical importance would leach into the
328 next solution to be used in the vessel. This aspect of the work requires different methods to those used
329 for the routine verification. These more sensitive methods are required to be validated in the R&D phase
330 of the work (validated for use in the conjugation solution).
331 The validation of the (non-standard) test methods used in this type of work is thoroughly exemplified in
332 the formal pharmaceutical test method validation guides. Validation of the total organic carbon
333 measuring system originates from the manufacturer’s specification and the subsequent performance
334 qualification.
335 As the measurements are made in almost pure water it would not be necessary to verify lack of
336 interference from other constituents of the matrix. Similarly if the conductivity meter was specified as
337 being capable of accurate readings superior than the requirement, further validation would not be
338 necessary.
339 Residual detergent would be measured using an instrument, for example high performance liquid
340 chromatography (HPLC). The method chosen would require characterisation of the sensitivity, accuracy
341 and precision for this purpose and the specific detergent involved. Functionality and conformation of
342 proteins would not survive the acid and alkali washes so any contaminating protein would be measured
343 by a chemical technique, defined in the standard operating procedure for the cleaning process. The
344 definition of the test method is essential as each method of protein quantitation (Lowry, biuret, binding
345 of various dyes and HPLC) provides marginally different results for specific proteins. The sensitivity of the
346 method would be demonstrated to be capable of meeting the requirement and the precision to show
347 that the stated level of protein could be determined with sufficient accuracy. Chemical methods may be
348 used to analysis cleanliness relating to the conjugation reagents. The test method must be defined and
349 the precision and sensitivity in the matrix of distilled water demonstrated to be appropriate.
350 e) Compare performance with requirements and use the methods if adequate
351 Once the methods have been characterised and proven to meet the required numerical attributes they
352 can be used in routine verification of the cleaning process.
353 Over time as the process is shown to consistently produce cleanliness within the required
354 specification, testing would be minimised: i.e. the process itself would be validated
355 (consistently providing cleanliness fit for purpose). However this can only be done by prior
356 use of validated test methods.

357 9.2 Validation of test methods for raw materials
358 9.2.1 Routine commercial materials

359 Most commercial chemicals (salts, acids, alkalis, sugars) have standard analytical methods from
360 pharmacopoeia, (needing no further evaluations except for verification of proper use and documentary
361 evidence of the required level of quality in the materials. The scope of testing for routine commercial
362 chemicals requires individual assessment. However this should be easy with reputable suppliers.
363 9.2.1.1 Components

364 Components to accompany the IVD such as sachets of drying agents, specimen collection devices and
365 tubes, transfer pipettes or dropper bottles will require testing (and documentation) against the specific
366 requirements of the IVD.
367 Example: validation of an incoming test for transfer pipettes used to drop specimen into an IVD.
369 The purpose of the testing is to demonstrate that across the lot of transfer pipettes, the volume
370 delivered meets the specification provided by the R&D department during the development of the IVD.
371 The specifications provided by the R&D department would have been validated to demonstrate that all
372 the claims of the assay (sensitivity, specificity, precision, etc.) are met through the assigned life of the
373 IVD using the pipettes which provide a volume within the specification.
375 The required characteristics are the trueness and the precision of the lot of transfer pipettes, for each
376 specimen type claimed.
377 Further specifications that need to be evaluated for such pipettes may include: orifice diameter and
378 overall length (measures of trueness and sensitivity required), ability to deliver discrete drops easily by
379 untrained individuals (i.e. an in-use precision measure). Each of these requires a validated specification,
380 numerical limits and consequently a test method validated as giving the required information. The
381 following example below is only for the volume measurements.
383 The specification for volume delivered into the IVD, validated by the R&D department, might be “not
384 less than 30 µL and not more than 45 µL of specimen to be delivered in two drops from the pipette”
385 which in the instructions for use would be translated to “add two drops of specimen using the dropper
386 pipette provided”. The specification of the pipettes would be “to deliver 35-40 µL ± 2 µL in two drops
387 and evaluated across the lot”. This would have been validated by R&D for each specimen type claimed.
388 From that specification, the requirement of the test method would be a bias of < 1 µL in the range 30 –
389 45 µL and a precision of < ± 0.8 µL (variance ≈20% of that allowed in the volume specified for the
390 pipettes).
391 d) Select or develop the method
392 The test method (weighing drops of water) is unlikely to introduce bias or imprecision beyond that in the
393 specification (based on the assumption that properly maintained and calibrated weighing
394 instrumentation is accurate) so the most important validation aspect is the relationship between drops

395 of water and drops of each specimen type from the pipettes. The number of randomly selected pipettes
396 and the proportion of lots to be tested are calculated on the basis of acceptable risks (27) as confidence
397 in the supplier increased.
398 It is unlikely that the quality assurance incoming goods inspection team have access to the specimens
399 claimed for the IVD (e.g. fresh whole blood, fresh serum, cerebrospinal fluid). Hence any volume
400 measurements on a substitute liquid (e.g. water) with volume estimated by weight must be validated.
401 Drop volumes and variances of different liquids differ due to density and surface tension effects.
402 The exact method and required specifications would be documented in a standard operating procedure
403 in addition to data recording, monitoring requirements and a reference to the validation of the test
404 method.
405 e) Compare performance with requirements
406 The required characteristics of the test method can be measured and compared against the specification
407 (the precision and the number of pipettes to be tested). Consequently the method may be used if found
408 to be fit for purpose.
409 As can be seen, method validation at this level require planning, experimental work and documentation
410 beyond merely defining the method (“weigh some drops”) and the specification for the component.
411 9.2.1.2 Package labels, instructions for use, vials, stoppers etc.
412 Ancillary materials (e.g. printed matter, packing materials, containers, stoppers) will at a minimum, need
413 inspection prior to acceptance. Inspection is a test method. As usual, the requirements of the test
414 method (the inspection) must be defined once the purpose of the testing is understood. The attributes
415 of the test method can be evaluated (e.g. the pre-defined consumer risk and hence the proportion of the
416 incoming delivery to examine, the capability of the inspectors to distinguish and record the attribute)
417 and the method validated to consistently assure appropriate quality. Regardless of inspection method
418 chosen, its capability to detect flaws at the required level of risk must be documented as must be the
419 standard operating procedure for performing the inspection.
420 9.2.2 Constituents critical to IVD performance

421 Critical constituents must be decided on a risk assessment basis. Nitrocellulose membranes, some
422 detergents and all complex biological reagents (peptides, proteins, and oligonucleotides) are assumed to
423 be critical (unless there is evidence to the contrary).
424 Testing critical constituents typically involves techniques such as HPLC, spectroscopy, mass-
425 spectrometry, sequencing and various forms of electrophoresis. All instrumentation is assumed to be
426 correctly documented, qualified and operated and the instrumentation itself will not contribute to bias
427 or uncertainty in the example below. The latter assumptions are usually true for the biological systems
428 described here; it is the nature of the measurements made that requires validation. Assessment of
429 critical constituents should be more detailed than for non-critical constituents (even if the critical
430 materials are obtained commercially). A commercial supplier cannot know the exact use of the material
431 and can only give a general certificate of analysis (COA).
432 9.2.2.1 Example of acceptance testing for a low molecular weight constituent
433 Some detergents (those containing a polyether bond e.g. Tween, Triton) easily and quickly generate
434 peroxides (28).

436 Peroxides can disrupt enzyme activity and the conformation of recombinant proteins and some
437 peptides. For this reason it may be considered necessary to monitor the peroxide content of detergents
438 used in an IVD, either at the incoming goods check or often just prior to use.
440 The characteristics required are sensitivity (range and uncertainty at specific concentrations), trueness
441 (accuracy, bias) and precision.
443 R&D should have proven the stability of the IVD in studies using various lots, some of these lots at the
444 end of their shelf-lives (18, 23). Preliminary stability experiments should lead to knowledge of the
445 maximum permissible concentration of peroxide, (specified as, for example, < 6.0 µM) in the detergent
446 used routinely in the manufacture of the IVD.
447 An example of specifications for the test method derived from the R&D requirements could be “no bias,
448 sensitivity of 5 µM ± 1 µM at a concentration near 55 µM “(i.e. ability to distinguish between 50 and 55
449 µM and to allow for 10-fold dilution of a stock solution), with a peroxide specification of <55 µM in the
450 stock solution: to give an acceptable margin of safety relative to the permissible concentration and the
451 test method variance.
452 d) Select or develop the method
453 Several suitable methods for measurement of peroxide in aqueous detergent solutions are available.
454 However, as they are not standard methods, they necessitate characterisation and validation of the
455 required sensitivity, bias and precision near the permissible concentration of peroxide in solutions of the
456 particular detergent.
457 e) Compare performance with requirements
458 Clear specifications and justification of both method and expected result are required.
459 9.2.2.2 Acceptance testing of molecules with defined structures

460 For short peptides and oligonucleotides, the COA from an established and reputable supplier usually
461 gives sufficient structural detail (e.g. proof of sequence and terminal residues (usually by mass
462 spectrometry) and freedom from synthetic artefacts and residues (usually by HPLC)). As result, further
463 acceptance measurements are usually not required. However, for peptides containing cysteine (or
464 cystine) residues it might be necessary to monitor the state of the received material to ensure lack of
465 oxidation (or reduction), requiring a validated method for measurement of sulphydryl content.
466 Monoclonal immunoglobulin G class antibodies (but not polyclonal antibodies) are normally robust in
467 production and a COA of identity and purity is generally sufficient. Polyclonal antibodies, which vary in
468 avidity and precise epitopic dependence from animal to animal, may require similar functionality testing
469 to that suggested in the following for recombinant proteins. This is also true for immunoglobulin M class
470 antibodies, whether monoclonal or polyclonal.
471 9.2.2.3 Acceptance testing of recombinant proteins and polynucleotides

472 Recombinant proteins and polynucleotides require more complex testing than for molecules with a
473 simple sequence. The functionality of recombinant proteins and polynucleotides is frequently

474 dependent on the conformation of the macromolecules and specificity depends on the precise
475 impurities present (among other things). The material’s specification must include requirements for
476 functionality, for purity based on similarity of contaminants between lots, measures of sequence
477 integrity and molecular conformation.
478 The test methods involved in preparing satisfactory COAs, or of providing evidence of satisfactory in-
479 house preparations, are much more complex than those in the elementary examples given above.
480 However, validation of the methods follows exactly the same principles. Usually the methods are well
481 known analytical procedures. It may be the case that they are not “standard” methods but adequately
482 known and characterised so that if used appropriately they do not require further validation.
483 A problem observed in many submissions to WHO prequalification is that either the methods are not
484 used at all, or the output is not appropriate for the task. The following section discusses these major
485 issues. It does not provide detail of the process of validation, but an expectation of how the well-known
486 test methods will be used.
487 Both purity and conformation are critically lot dependent and are not usually documented in sufficient
488 detail in a commercial COA to provide objective evidence of inter-lot reproducibility. A standard
489 commercial COA for a recombinant protein usually provides a result for purity from a gel after
490 electrophoresis (e.g. “>95 %”) without specifying the exact concentrations and molecular weights of the
491 impurities (so allowing lot-to-lot variation within the specification and hence potential for specificity and
492 stability issues). A COA will also usually provide a molecular weight which is determined from a gel or a
493 Western blot. However, neither technique are adequately sensitive to demonstrate minor post-
494 translational modifications, nor capable of providing any information about conformation (allowing
495 potential sensitivity and selectivity issues). Quantitation of results from both stained and blotted gels is
496 not reproducible without special techniques and gives only approximate values (29). A COA should
497 always give some measure of uncertainty in the stated values of both molecular weight and quantity. A
498 competent COA should also include amino- and carboxy- terminal amino acid analyses to ensure
499 absence of minor proteolysis during purification.
500 Choice of correct methods, and knowledge of their limitations, is a major deficiency in most WHO
501 prequalification submissions. There is insufficient proof that there is no lot-to-lot variability, neither in
502 those critical materials nor in the final IVD made from them.
503 Before committing to purchase or process a substantial amount of a new lot of recombinant protein, a
504 careful manufacturer will need to check that the new lot will detect the difficult specimens
505 (seroconversion, latent stage, unusual serotype specimens) with the sensitivity and specificity claimed
506 for the IVD and with approximately the same utilization (devices per milligram) as the lots used to
507 validate the IVD itself. Proficient manufacturers develop testing of identity, integrity and functionality of
508 polynucleotides to be used in IVD.
509 These tests for complex critical reagents can only be specified for commercial or for in-house reagents
510 by the manufacturer of the IVD, since the requirements are unique to the IVD and its validated claims.
511 Nevertheless, the methods used must be validated as suitable for use.
512 The sensitivity, specificity and utilization measurements on new lots of recombinant proteins are usually
513 made by preparing the IVD on a small scale and testing against defined panels of specimens proven to
514 monitor the stated parameters with satisfactory efficiency. Test method validation is to ensure that the
515 panels do indeed monitor the expected parameters.

10 References
516 1. ISO 5725-1:1994: Accuracy (trueness and precision) of measurement methods and results —
517 Part 1: General principles and definitions. Geneva, Switzerland: International Organization for
518 Standardization; 1994
519 2. ISO 18113-1:2009. In vitro diagnostic medical IVDs – Information supplied by the manufacturer
520 (labelling) – Part 1: Terms, definitions and general requirements. Geneva, Switzerland:
521 International Organization for Standardization; 2009.
522 3. ISO 9000:2015. Quality management systems – Fundamentals and vocabulary. Geneva,
523 Switzerland: International Organization for Standardization; 2015.
524 4. ISO 15198:2004. Clinical laboratory medicine – In vitro diagnostic medical IVDs – Validation of
525 user quality control procedures by the manufacturer. Geneva, Switzerland: International
526 Organization for Standardization; 2004.
527 5. GHTF/SC/N4:2012 (Edition 2). Glossary and Definitions of Terms Used in GHTF Documents.
528 Global Harmonization Task Force (GHTF) Steering Committee; 2012.
529 6. ISO 14971:2007. Medical devices – Application of risk management to medical IVDs.
530 International Organization for Standardization; Geneva, Switzerland: 2007.
531 7. ISO/IEC Guide 99:2007: International vocabulary of metrology -- Basic and general concepts and
532 associated terms (VIM). Geneva, Switzerland: International Organization for Standardization;
533 2007.
534 8. US FDA Volume II: Methods, Method Verification and Validation ORA-LAB.5.4.5 October 2003,
535 Revised August 2014. http://www.fda.gov/ScienceResearch/FieldScience/ucm171877.htm
536 9. United States CFR - Code of Federal Regulations Title 21. Sec. 820.3 Definitions. Washington DC,
537 United States of America; 2010.
538 10. Magnusson, B., and Örnemark, U. (eds.) Eurachem Guide: The Fitness for Purpose of Analytical
539 Methods – A Laboratory Guide to Method Validation and Related Topics, (2nd ed. 2014). ISBN
540 978-91-87461-59-0. (http://www.eurachem.org)
541 11. Eurolab: Validation of Test methods. General principles and concepts: EL1545/96
542 (http://www.eurolab.org)
543 12. ICH: Validation of analytical procedures: text and methodology {Q2(R1)} November 1996
544 (http://www.ich.org/products/guidelines/)
545 13. WHO Technical Report Series, N°937, Appendix 4, Analytical method validation, of Annex 4
546 Supplementary guidelines on good manufacturing practices : validation - WHO, 2006
547 14. US FDA Center for Drug Evaluation and Research (CDER): Bioanalytical Method Validation, May
548 2001 & September 2013 (draft update)
549 15. ISO 15189:2012. Medical laboratories -- Requirements for quality and competence. Geneva,
550 Switzerland: International Organization for Standardization; 2012
551 16. WHO Prequalification Team – Diagnostic Assessment. Technical Guidance Series (TGS).
552 Principles for Performance studies TGS–3. Geneva: World Health Organization; 2016. Available

553 at: http://www.who.int/diagnostics_laboratory/guidance/technical_guidance_series/en/,

554 accessed 15 July 2015.
555 17. CLSI. Quality Management System: Equipment; Approved Guideline. CLSI document QMS13-A
556 Wayne, PA: Clinical and Laboratory Standards Institute; 2011
557 18. CLSI. Evaluation of Stability of In Vitro Diagnostic Reagents: Approved Guideline. CLSI document
558 EP25-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2009
559 19. CLSI. User Estimation of Precision and Estimation of Bias CLSI document EP15-A3 Wayne,
560 PA: Clinical and Laboratory Standards Institute; 2014
561 20. Valcárcel, M., Cárdenas, S., Barceló, D. et al. Metrology of qualitative chemical analysis, KI-NA-
562 20-605-EN-C, ISBN 92-894-5194-7; 2002 Available free of charge from:
563 http://bookshop.europa.eu/en/metrology-of-qualitative-chemical-analysis-pbKINA20605/
564 21. Burdick, R.K., Borror, C. M., & Montgomery, D. C. (2005). Design and analysis of gauge R and R
565 studies: making decisions with confidence intervals in random and mixed ANOVA models. ASA &
566 SIA Math: ISBN 0898715881
567 22. ISO 23640:2011. In vitro diagnostic medical IVDs - Evaluation of stability of in vitro diagnostic
568 reagents. Geneva, Switzerland: International Organization for Standardization; 2011.
569 23. WHO Prequalification Team – Diagnostic Assessment. Technical Guidance Series (TGS).
570 Establishing stability of an in vitro diagnostic for WHO Prequalification TGS–2. Geneva:
571 World Health Organization. 2016. Available at:
572 http://www.who.int/diagnostics_laboratory/guidance/technical_guidance_series/en/,
573 Accessed 15 July 2016.
574 24. US FDA Center for Devices and Radiological Health (CDRH): Design control guidance for
575 medical device manufacturers March 1997. Available at:
576 http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/uc
577 m070627.htm Accessed 20 December 2016
578 25. ISO 13485:2003. Medical devices – Quality management systems – Requirements for regulatory
579 purposes. Geneva, Switzerland: International Organization for Standardization; 2003.
580 26. WHO Technical Report Series, N°937, Appendix 3, Cleaning validation, of Annex 4
581 Supplementary guidelines on good manufacturing practices : validation - WHO, 2006
582 27. ANSI/ASQ Z1.4–2003 (R2013): Sampling Procedures and Tables for Inspection by Attributes,
583 2013. Available at: https://www.ansi.org/ Accessed 20 December 2016.
584 28. Jaeger J, Sorensen K, Wolff SP. Peroxide accumulation in detergents. J Biochem Biophys
585 Methods. 1994;29:77–81. doi: 10.1016/0165-022X(94)90058-2
586 29. Taylor, SC, Posch, A. (2014) The Design of a Quantitative Western Blot Experiment BioMed
587 Research International, volume 2014, Article ID 361590 8 pages, 2014
588 http://dx.doi.org/10.1155/2014/361590

Guideline WHO - Guidance On Test Method Validation of in Vitro Diagnostic Medical Devices PDF

Uploaded by

Copyright:

Available Formats

You might also like

Guideline WHO - Guidance On Test Method Validation of in Vitro Diagnostic Medical Devices PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Guideline WHO - Guidance On Test Method Validation of in Vitro Diagnostic Medical Devices PDF

Uploaded by

Copyright:

Available Formats

Technical Guidance Series (TGS)

for WHO Prequalification – Diagnostic

Guidance on Test Method

Draft for comment 20 December 2016

Contact: Irena Prat, EMP Prequalification Team Diagnostics

WHO - 20 Avenue Appia - 1211 Geneva 27 Switzerland

Page 3 of 21 Draft for comment 20 December 2016

WHO Prequalification – Diagnostic Assessment: Technical Guidance Series

Page 4 of 21 Draft for comment 20 December 2016

Page 5 of 21 Draft for comment 20 December 2016

Page 6 of 21 Draft for comment 20 December 2016

37 Note 1 IUPAC recommends default values for α and β equal to 0.05.

43 Measurand: Quantity intended to be measured. (‎7)

Page 7 of 21 Draft for comment 20 December 2016

4 Terminology for test method validation

96 4.1 Explanation of the terms characterisation, verification and validation

Page 8 of 21 Draft for comment 20 December 2016

Page 9 of 21 Draft for comment 20 December 2016

141 Table 1: Examples of characteristics of assays

Trueness bias, recovery, accuracy, matrix effects

144 4.2 Explanation of the terms accuracy, trueness and precision

Page 10 of 21 Draft for comment 20 December 2016

5 Uses of test method validation in the lifecycle of the IVD

183 6.1 Categories of test methods

190 6.2 Statistics and test methods

197 6.3 Quantitative and qualitative assays

Page 11 of 21 Draft for comment 20 December 2016

212 6.4 Specimen panels and test methods

7 Variability in the test method

Page 12 of 21 Draft for comment 20 December 2016

8 Planning for test method validation

Page 13 of 21 Draft for comment 20 December 2016

9 Examples of test methods and their validation

292 9.1 Validation of test methods related to cleaning processes

Page 14 of 21 Draft for comment 20 December 2016

Page 15 of 21 Draft for comment 20 December 2016

357 9.2 Validation of test methods for raw materials

358 9.2.1 Routine commercial materials

363 9.2.1.1 Components

Page 16 of 21 Draft for comment 20 December 2016

420 9.2.2 Constituents critical to IVD performance

435 a) Define the purpose of the testing

459 9.2.2.2 Acceptance testing of molecules with defined structures

471 9.2.2.3 Acceptance testing of recombinant proteins and polynucleotides

Page 18 of 21 Draft for comment 20 December 2016

Page 19 of 21 Draft for comment 20 December 2016

Page 20 of 21 Draft for comment 20 December 2016

553 at: http://www.who.int/diagnostics_laboratory/guidance/technical_guidance_series/en/,

Page 21 of 21 Draft for comment 20 December 2016

You might also like