(In cooperation with EAL, European cooperation for Accreditation of Laboratories) EL1545/96 VALIDATIO O! TE"T #ET$OD" GEE%AL &%I'I&LE" AD 'O'E&T" December 1996 2 !O%E(O%D EAL and Eurolab have a Permanent Liaison Group PLG!" #hich is a $orum #here EAL and Eurolab are discussin% matters o$ mutual interest& 'he PLG consists o$ (ve members $rom each or%ani)ation& 'his document has been prepared in the PLG and endorsed b* both or%ani)ations& 'he document is intended to %ive %eneral vie#s on certain issues related to the validation o$ test methods and should be seen as a common understandin% and position o$ EAL and Eurolab& +n order to de(ne and describe the activities behind the concept ,-alidation o$ test methods, more detailed %uidance documents are needed& 'his document should be seen as a basis $or such %uidance documents& . VALIDATIO O! TE"T #ET$OD" )* Introduction 'he de(nition used $or /validation/ in the +01 standard 2432 is /con(rmation b* e4amination and provision o$ ob5ective evidence that the particular re6uirements $or a speci(c intended use are $ul(lled/& 'his de(nition %ives the impression o$ con(ned and #ell7de(ned e4act! operations& 'est methods are normall* developed $or an intended application ran%e& 'he re$erence to particular re6uirements must in man* cases be interpreted in a 8e4ible #a* as the re6uirements can be o$ %eneral nature& 9oth standardi)ed and non7standardi)ed test methods are covered& 'he* can be e4act or associated #ith lar%e uncertainties& Even novel test methods #ill be considered& 'he validation o$ a test method becomes in this conte4t a #a* o$ demonstratin% that the method is (t $or its intended purpose& 'he (tness $or purpose includes an assessment and a balancin% o$ technolo%ical possibilities" ris:s and costs& 'here are ver* $e# papers in the open literature dealin% #ith the %eneral principles o$ test method validation& 1n the other hand" a lot o$ detailed descriptions o$ the validation o$ speci(c test methods are available& A brie$ overvie# o$ the concepts" aims and procedures in validation is %iven in this document& +* General principles to be used in ,alidation +n the validation process an estimate is made o$ the representativeness" repeatabilit* and reproducibilit* o$ the test method& 'he de(nitions are %iven in anne4 1& +n the validation process the ultimate aim is to secure that the test methods are %ood enou%h #ith respect to representativeness" reproducibilit* and repeatabilit*& ;o# much e<ort should be spent on validation must be decided on a case b* case basis& +$ lar%e economic values as #ell as considerable health" sa$et* and environmental issues are involved" much more emphasis must be paid to the validation o$ the test methods& 'he $re6uenc* o$ use o$ the test method should also be considered #hen determinin% the e4tent o$ validation& 'he total conse6uences o$ #ron% results are o$ course lar%er $or methods in e4tensive use than $or test methods used occasionall*& 'he validation o$ test methods covers to a lar%e e4tent the uncertaint*" repeatabilit* and reproducibilit* o$ the test method& As the $actors a<ectin% the results and contributin% most to the uncertaint* chan%e $rom one technical sector to another or even $rom one test method to 4 another" a universal solution cannot be %iven& Guidance on the e4pression o$ uncertainties can be $ound $or e4ample in the international ,Guide to the e4pression o$ uncertaint* in measurement, and EAL %uidance document ,E4pression o$ uncertaint* in 6uantitative testin%,& 0tandardi)ed test methods should be considered validated $or their intended application ran%e and thus %ood enou%h $or that purpose althou%h their repeatabilit* and reproducibilit* are not :no#n in detail& 'he testin% laborator* must" ho#ever" chec: that the* appl* the method correctl*& =or non7standardi)ed test methods it is up to the testin% laboratories to determine ho# $ar the* %o in de(nin% the level o$ repeatabilit* and reproducibilit*& 'o develop a representative test method" ade6uate :no#led%e is re6uired o$ the practical use o$ the test results and o$ the real service conditions o$ the ob5ect o$ the test& 9ased on such :no#led%e" the /representative/ properties to be determined b* the test ma* be identi(ed& 'he $actors a<ectin% the test results and their uncertaint* ma* be %rouped into three main cate%ories> +nstrumental and technical $actors 7 samplin% 7 homo%eneit* 7 test method 7 e6uipment ;uman $actors Environmental $actors 7 testin% environment Instrumental and technical factors are related to the constructional and $unctional characteristics o$ the test and measurement e6uipment" as #ell as to other technical operations involved in the test e&%& samplin%" preparation o$ samples" test ob5ect homo%eneit*!& 'heir e<ect ma* be minimi)ed and :ept under control b* the $ollo#in% provisions> 7 de(ne the e6uipment as precisel* as necessar* 7 provide a clear description o$ the test procedure as #ell as the e6uipment operation 7 establish procedures $or operational control and calibration 7 ensure #here applicable traceabilit* o$ measurements to the 0+ units& ?henever practical" the above provisions should be included in the description o$ the test method& @e$erences to internal procedures or applicable standards should be included& 5 $uman factors are related to the competence o$ the sta< and ma* be controlled throu%h> 7 education/basic :no#led%e 7 on 5ob trainin%/practical e4perience& 'he 6uali(cation re6uired $or the personnel emplo*ed $or a %iven test ma* be speci(ed in the test method or re$erence can be made to the applicable internal procedures& En,ironmental factors are associated to the environment #here the test is per$ormed& Amon% others the e<ect o$ the $ollo#in% parameters must be assessed and properl* controlled> 7 atmospheric conditions temperature" pressure" humidit*! 7 pollution/contamination 7 other environmental characteristics e&%& EAB!& 'he e<ect o$ the above parameters should be described in the test method or re$erence to other applicable documents should be made& ;o#ever" $or ne# test methods this in$ormation is o$ten not available& +n some cases the data base $or method validation is so lar%e that statistical methods should be applied& 'he validation process must consider the e4pected or re6uired uncertaint* o$ the test results and their intended use& Britical threshold values e&%& in health and environment! cannot %enerall* be technicall* 5usti(ed #ith a small uncertaint*& ;o#ever" i$ a le%al limit is set" there must be test methods suited $or the purpose& @e$erence is made to a recent +LAB Guide& 'he re6uired depth o$ the validation process depends also on the maturit* o$ the test method and the prevalence o$ its use& 1ne can distin%uish bet#een the $ollo#in% cate%ories> 7 novel methods 7 methods used b* several laboratories 7 modi(cation o$ established methods 7 standardi)ed methods& 'he #a*s" in #hich the validation is per$ormed in the di<erent cases" need not be clearl* di<erentiated& +$ the (tness $or purpose concept is maintained" it is o$ten possible to validate at reasonable cost but #ith a hi%her de%ree o$ uncertaint*& 'he novel methods are (rst developed in one sin%le laborator*" o$ten on the basis o$ a special re6uest $rom a customer or on ideas created in the laborator*& 'hat customer cannot pa* $or a #ide ran%e validation nor can the laborator* itsel$& 'he aim o$ the validation o$ test methods must al#a*s be to demonstrate that the method is (t $or the intended purpose and that the results have an acceptable uncertaint*& +t is important that the rules o$ validation o$ test methods do not prevent 6 the natural technolo%ical development $rom ta:in% place& 'he laborator* does not e4pect althou%h it does #ant! outside (nancial help $or validation o$ novel methods and in man* cases tries to protect its ne# development $rom %oin% to its competitors or $rom becomin% %enerall* available to all& ?hen a certain number o$ laboratories #or: in the same area" cooperation and inter7laborator* comparisons can be arran%ed& 'he coordination o$ such activities is an e4tra economic burden& +n order to speed up the process" e4ternal (nancin% is needed& 'he testin% laboratories need to update their e4istin% test methods& 'he 8e4ible scope o$ accreditation as a%reed bet#een EAL and Eurolab #as also intended to allo# modi(cations to be made to accepted accreditation covered! test methods& 'his re6uires validation procedures applicable to method modi(cations& +t is up to the laboratories to describe their procedures $or validatin% modi(ed test methods& 'he most thorou%h validation procedure is re6uired $or test method standardi)ation purposes& 'he #or: needed is considerable and covers pro(cienc* testin%" the determination o$ $actors a<ectin% the uncertaint*" measurin% ran%e" etc& 'he (nancial burden cannot be laid on the laboratories but on the standardi)ation or%ani)ations& 0tandardi)ed test methods must be considered suCcientl* validated $or their intended application ran%es& +$ the* are not" the* should be #ithdra#n& 'he validation o$ test methods consists o$ t#o interrelated steps> i! suitabilit* o$ the test to solve the problem customer needs! ii! demonstration o$ the technical capabilit* o$ the test method #ithin the speci(ed test ran%e i&e& measurin% the ri%ht properties #ith a suCcientl* reliable method& 'he suitabilit* or representativeness o$ a test method is in man* cases an attribute #hich is diCcult to de(ne especiall* $or tests related to product acceptance& 'he test methods must be such that the results obtained correlate #ith the per$ormance characteristics and operational e4perience o$ the product& -* Validation procedure 9oth testin% laboratories and accreditation bodies are loo:in% $or procedures and %uidelines $or plannin% and controllin% the test method validation process& ;o#ever" the discussion above has clearl* indicated that one sin%le procedure cannot be developed& Bonse6uentl*" a palette o$ di<erent choices o$ validation techni6ues has to be developed& ;o# detailed the validation #ill be" depends on the circumstances needs" costs" possibilities" ris:s" etc&!& D 'he validation o$ the test methods is" o$ course" o$ interest also to the accreditation bodies& 'he principle to be applied should be that the laborator* describes the #a* it is validatin% the test methods and the accreditation bod* should ma:e the 5ud%ement i$ the procedure used is acceptable in that case& 'he di<erent validation possibilities are built up around 7 utili)ation o$ calibration 7 intercomparisons includin% the use o$ re$erence materials and re$erence methods 7 #ell 6uali(ed sta< and their pro$essional 5ud%ement 7 simulation and modellin% 7 other approaches& Aethod validation is o$ten based on the combined use o$ validation procedures& 'he validation used can be /direct/ or comparative& 'he selection o$ the validation procedures should also be 5usti(ed on a cost7bene(t basis as lon% as the (tness7$or7purpose is maintained& =ocusin% the e<ort on the most critical $actors a<ectin% the test method #ill lead to a di<erent solution $or the validation o$ /e4act/ ph*sical and chemical test methods as compared to that $or product or sub5ective testin%& =or e4ample" in the validation o$ er%onomics and sensor* test methods not all possibilities are applicable& As said above di<erent validation procedures ma* be $ollo#ed" their e<ectiveness and applicabilit* dependin% on the t*pe o$ test considered& 'he* can be characteri)ed as /scienti(c/ or /comparative/> 2 ."cienti/c approach. +n the scienti(c approach the assessment o$ the representativeness" repeatabilit* and reproducibilit* o$ the method is per$ormed #ith re$erence to the di<erent constitutive elements and $eatures& Evidence should describe the representativeness o$ the selected properties and the associated uncertaint*& 'his can be based on in$ormation published in the scienti(c and technical literature or on ad hoc investi%ations per$ormed b* the laborator* developin% the method& 'he laborator* shall demonstrate that relevant in8uencin% $actors instrumental and technical" human" environmental! have been anal*)ed and that the* are under control #ithin the uncertaint* associated #ith the method& .'omparati,e approach. 'he test method is assessed b* comparin% its results to those obtained b* means o$ another alread* validated test method" #hich has been developed $or the same purposes& +$ this is not possible" the per$ormance characteristics o$ the method ma* be assessed throu%h interlaborator* comparisons& 'he method is /valid/ i$ the results obtained b* the di<erent laboratories $all #ithin the e4pected uncertaint* limit& Deviations be*ond such limits ma* indicate e&%& a lac: o$ control o$ the in8uencin% parameters& 'he causes o$ this behaviour should be clari(ed and the method is to be rede(ned accordin%l*& 'he interlaborator* comparison does not al#a*s provide a comprehensive validation o$ the representativeness o$ the method" #hich ma* be accurate and stable" thou%h ph*sicall* /#ron%/& 'he acceptance procedure $or ne# or modi(ed test methods is either i! determined internall* in the laborator* ii! a%reed upon bet#een the customer and the laborator* or iii! accepted b* the authorities and/or accreditation bodies& A hi%her de%ree o$ reliance is needed #hen sa$et*" health and lar%e economic values are involved& Balibration has been emphasi)ed as an important element in the method validation process" but it is not necessaril* the most dominatin% $actor& 'he understandin% o$ the testin% method #ith its s*stematic and random errors is crucial& A scienti(c approach to anal*)e sources o$ error as #ell as the competence o$ the personnel doin% that 5ob is o$ %reat importance& 'he laborator* should al#a*s describe the #a* the validation o$ test methods is done and this description should be a part o$ the 6ualit* s*stem/manual #hen appropriate& As simpli(ed validation procedures /$ast/ validation methods! must be used in man* cases" the capabilit* to use pro$essional 5ud%ement in assessin% #hether the validation is comprehensive enou%h becomes pronounced& ;o#ever" even #hen tal:in% about simpli(ed or $ast validation procedures" the validation must be done #ith such a depth that the method is (t $or the intended use and acceptable to the 9 customer and/or authorities& +t is clear that the de(nition o$ the use and scope o$ the method and assumption o$ uncertaint* should not be misleadin% and too optimistic& ?hen the use o$ ne# test methods becomes more e4tensive" #or: describin% the e<ect o$ chan%es in test parameters can be initiated in order to sho# the robustness o$ the method& Prenormative research should also be initiated& 'he need $or ne# or improved test methods arises #hen #e lac: methods or the e4istin% ones are not complete" %ood or eCcient enou%h& 'here is no need $or the laborator* communit* to develop ne# methods i$ e4istin% ones can be considered ade6uate& 13 AEEEF 1 DE!IITIO" %epeatabilit0 (of results of measurement) Bloseness o$ the a%reement bet#een the results o$ successive measurements o$ the same measurand carried out under the same conditions o$ measurement& -+A! Eotes 1& 'hese conditions are called repeatabilit* conditions 2& @epeatabilit* conditions include> 7 the same measurement procedure 7 the same observer 7 the same measurin% instrument" used under the same conditions 7 the same location 7 repetition over a short period o$ time .& @epeatabilit* ma* be e4pressed 6uantitativel* in terms o$ the dispersion characteristics o$ the results& %eproducibilit0 (of results of measurements) Bloseness o$ the a%reement bet#een the results o$ measurements o$ the same measurand carried out under chan%ed conditions o$ measurement& -+A! Eotes 1& A valid statement o$ reproducibilit* re6uires speci(cation o$ the conditions chan%es& 2& 'he chan%es conditions ma* include 7 principle o$ measurement 7 method o$ measurement 7 observer 7 measurin% instrument 7 re$erence standard 7 location 7 conditions o$ use 7 time& .& @eproducibilit* ma* be e4pressed 6uantitativel* in terms o$ the dispersion characteristics o$ the results& 4& @esults are here usuall* understood to be corrected results& 11 1ncertaint0 (of measurement) Parameter" associated #ith the result o$ a measurement" that characteri)es the dispersion o$ the values that could reasonabl* be attributed to the measurand& 9+PA/+EB/+=BB/+01/+GPAB/+GPAP/1+AL!& Eotes 1& 'he parameter ma* be" $or e4ample" a standard deviation or a %iven multiple o$ it!" or the #idth o$ a con(dence interval& 2& Gncertaint* o$ measurement comprises" in %eneral" man* components& 0ome o$ these components ma* be evaluated $rom the statistical distribution o$ the results o$ series o$ measurements and can be characteri)ed b* e4perimental standard deviations& 'he other components" #hich can also be characteri)ed b* standard deviations" are evaluated $rom assumed probabilit* distributions based on e4perience or other in$ormation& .& +t is understood that the result o$ the measurement is the best estimate o$ the value o$ the measurand" and that all components o$ uncertaint*" includin% those arisin% $rom s*stematic e<ects" such as components associated #ith corrections and re$erence standards" contribute to the dispersion& Validation Bon(rmation b* e4amination and provision o$ ob5ective evidence that the particular re6uirements $or a speci(c intended use are $ul(lled +01 2432!& Eotes 1& +n desi%n and development" validation concerns the process o$ e4aminin% a product to determine con$ormit* #ith user needs& 2& -alidation is normall* per$ormed on the (nal product under de(ned operatin% conditions& +t ma* be necessar* in earlier sta%es& .& 'he term /validated/ is used to desi%nate the correspondin% status& 4& Aultiple validations ma* be carried out i$ there are di<erent intended uses& 12 Veri/cation Bon(rmation b* e4amination and provision o$ ob5ective evidence that speci(ed re6uirements have been $ul(lled +01 2432!& Eotes 1& +n desi%n and development" veri(cation concerns the process o$ e4aminin% the result o$ a %iven activit* to determine con$ormit* #ith the stated re6uirements $or that activit*& 2& 'he term /veri(ed/ is used to desi%nate the correspondin% status& 1. AEEEF 2 %E!E%E'E" AD 2A'3G%O1D #ATE%IAL +01 5D25 1926!> Precision o$ test methods 7 Determination o$ repeatabilit* and reproducibilit* $or a standard test method b* inter7 laborator* tests& +01 2432 1994!> Hualit* mana%ement and 6ualit* assurance 7 vocabular*& 9+PA/+EB/+=BB/+01/+GPAB/+GPAP/1+AL 1995!> Guide to the e4pression o$ uncertaint* in measurement& EAL7G 2. 1996!> E4pression on uncertaint* in 6uantitative testin%& EAL7G 12 1995!> 'raceabilit* o$ measurin% and test e6uipment to national standards& =orstIn Jarl 1991!> A vie# on the assessment o$ the technical competence o$ testin% laboratories& Eordtest 'echnical @eport 149" 46p& +LAB 1996!> Guide an assessment and reportin% o$ compliance #ith speci(cation" based on measurements and tests in a laborator*& -+A 199.!> +nternational vocabular* o$ basic and %eneral terms in metrolo%*& 14 ;>Kte:stit1KvalidatK.&12&1996 15