28/06/2015

MethodValidation

Select English(UnitedKingdom)

content
MethodValidation
KeyRequirements
IntermediatePrecisionDetermination
EstimationofBias
LimitsofDetection(LOD)
QualityControlandDetectionLimitMaintenance
KeyRequirements

Therecanbeconsiderableconfusionastowhatisactuallyrequiredofalaboratoryinrespectofitsdemonstratingthatithasimplementeda
methodproperlysoastoobtainreliableresults.MuchofthisconfusionarisesbecauseofsomeunfortunatewordingofISO17025.When
section5.4ofthestandard,whichdealswithmethods,referstovalidationitisonlyinthe

contextofvalidationofnonstandardmethods.Clause5.4.5.2showsanexample.Manylaboratorieshavebeentemptedtointerpretthisto
meanthatstandard methods require no validation and, if applied exactly as published and within their intended scope, can simply be
assumedtogivereliableresults.Thisisblatantlyabsurdandwillcertainlynotbeacceptedbyanyreputableaccreditationbody.Skillsof
staffandlocalinstrumentperformanceareobviousfactorswhichmightcompromiseamethodevenifitisappliedexactlyaspublished.

Theproblemreallyarisesbecausethetermvalidationisusedwithavarietyofshadesofmeaning.AsappliedinISO17025itisclearly
meanttorefertotheextensiveworkneededtobedonewhenacompletelynewtestmethodisdeveloped.Suchamethodwillneedfull
validationtoshowitsscopeandlimitationsofapplication,toestablishallrelevantperformancecharacteristicsandtoshowitisrobustin
applicationunderdifferingcircumstances.Inthecaseofstandardmethods,whichhavealreadyundergonethisfullvalidationaspartofthe
exercisetohavethemacceptedasstandards,alaboratorycantakemuchofthevalidationforgranted.Totakeonlyoneexampleprovided
thetestmethodisappliedtosampleswithinthescopeofthepublishedmethoditcanbeassumedthatthemethodissuitableforthattype
ofsample.

Ineffectthelaboratorycanassumethatastandardmethodisfitforpurposeingeneraltermsbutwillstillneedtocarryoutanexerciseto
confirmthat the method is working properly as implemented and to determine those performance characteristics which depend on local
circumstances. This, much more restricted exercise, is also often also called validation. Undoubtedly this restricted investigation of a
method is not what ISO 17025 means by the term validation. It is, therefore, quite wrong to interpret the standards implication that
validationisnotrequiredforstandardmethodsasmeaningthatnothinginthewayofconfirmingtheireffectiveimplementationisrequired.
It is hoped that the next edition of ISO 17025 will be explicit as to what is required of laboratorys in respect of validating their
http://www.labnetwork.org/en/textileandleather/133methodvalidation

1/8

28/06/2015

MethodValidation

implementationofstandardmethods.Inanyeventatpresentaccreditationbodieswillnormallyprovideguidanceastowhattheyrequirein
thisrespect.

Theminimumrequirementsforestablishingsatisfactoryperformanceofstandardmethodsbythelaboratorywillbeasfollows.

The intermediate precision of the method covering typical sample types and the range of measurement. Typically a method is
investigated at two points in the measurement range, at approximately 20% and 80% of the calibrated range, and for at least two
sampletypes.
Thebiasofthemethodneedstobeinvestigatedbyanalysisofsamplesforwhichtheresultisknowntowithinaconfirmedlevelof
uncertainty.
If the method is concerned with establishment of levels of, for example, a toxic residue, then the limit of detection and limit of
quantitationwillberequired.
Thevastmajorityofthemethodsappliedintextileandleathertestingarewellestablished,published,standardmethodsmanyofthemof
them with international application. Hence the restricted type of validation is all that is required. Often the terms verification or
conformationofperformancecharacteristicsisappliedtothisexercisetodistinguishitfromfullvalidationofnovelmethods.
Backtotop
IntermediatePrecisionDetermination

SampleRequirements
Inordertodetermineintermediateprecisionasamplewhichishomogeneousinthepropertybeingmeasuredisrequired.Thetestcanthen
berepeatedwiththesame,orusuallyreplicate,samplessoastoobtainanestimateofthestandarddeviationofthetestresultsinroutine
use.

Intextileandleathertestingthereareactuallytwotypesofsampleinvolvedindifferenttest.

Samples used in physical testing these samples are normally in the form of test pieces prepared from a bulk and are destroyed
duringthetest.Inthiscaseprecisioncanonlybeestimatedusingtestpiecespreparedtobeasconsistentaspossibleintherelevant
property.Inevitablytheresultingintermediateprecisionwillincorporateanyelementofinhomogeneityinthebulkfromwhichthetest
pieceisprepared.
Samplesusedtodeterminecontaminantsorresidues,e.g.chromium(VI),formaldehydeinthiscaseahomogenisedbulksamplecan
beprepared,e.g.bycryogenicgrindingofaleather.Ifsampleswithappropriatelevelsofthetargetanalytesarenotavailablethen
these can be generated by spiking of samples with low or zero natural levels of the target. Samples of powdered materials
generatedforprecisionstudiesmustbecarefullycheckedforhomogeneitybeforeuse.
The strategy for obtaining test pieces from bulk samples so as to have a maximum possibility of homogeneity between the test pieces
needstobecarefullyplanned.Suitablesamplescanbegeneratedbypreparinganappropriatenumberoftestpiecesfromasinglesheetof
leatherorfabric.

Use sample types which reflect the range of normal application of the method and which also reflect two points in the normal
calibration range. If there is a wide range of types then more than one sample type may be needed to reflect the spectrum of
matricesroutinelytested.
Chooseasheetorfabricwhichisasuniformaspossible.Tightlywovenhighqualityfabricsarethebestchoicefortextilesandfor
leathersthinnersheetsintendedforgarmentorupholsteryratherthanfootweararepreferable.Ifpossiblethesizeshouldbesufficient
for50to60testpieces.Conditionthesheetfor24hoursasfornormalsamples.
Cutas many test pieces as possible from the centre of the piece. Avoid using the extreme edges of the piece and any selvedge
whichmightbeatypical.
Selectadjacenttestpiecesfromthecentreofthepiece.Theobjectiveistotaketestpieceswhicharemostlikelytobethesamein
properties.
Inthecaseofyarnstakeallofthesamplesfromconsecutivelengthsinthecentreofalargereelorcone.
http://www.labnetwork.org/en/textileandleather/133methodvalidation

2/8

28/06/2015

MethodValidation

OrganisationandConductofPrecisionEstimate

The interest is in the precision of the method when used under the conditions of the routine work of the laboratory, the intermediate
precision.
Thismeansthattheprecisionstudyshouldreflecttheroutineworkingascloselyaspossiblesoitshouldusethefullrangeofoperators
authorisedforthemethodandbespreadoveraperiodofatleasttwoweeks.
The estimate should also attempt to cover the full numerical range of the method. Normally this should be taken as meaning that a
minimumoftwomaterialsaretoberun.Oneshouldbeatapproximately20%ofthecalibrationrangeandtheotheratapproximately80%.
Animportantpointtonoteisthattheseparateprecisiondeterminationsmustreflectalloftheprocessesinvolvedinthetest.Forexample,if
normaltestingrequiresthataninstrumentbestandardisedeachdaythentheprecisionestimatemustbecarriedoutsothateachsampleis
runonadifferentinstrumentstandardisation.Samplesrunonthesamestandardisationarelikelytounderestimatetherealvariabilitysince
anyeffectofrestandardisationisomitted.Thedatafromtheprecisionstudyshouldbeprocessedtogiveameanandstandarddeviation.

NumberofReplicates

Inaprecisiondeterminationreplicatesamplesneedtobetestedsotheinevitablequestionarisesastowhatisareasonablenumberof
replicates to work with. The object of the exercise is to determine the mean and standard deviation for the method, or in statistical
terminology,thepopulationmeanandpopulationstandarddeviation.Inpracticewecanneverdothissincewewouldhavetomeasurean
infinitenumberofsamplessoweestimatethesevaluesbyrepeattestingofafinitenumberofsamples.

Themoresampleswetestthebetterestimateofthepopulationmeanandstandarddeviationisprovidedbythevalueswemeasurebutan
examination of the relationship between the number of samples tested and the uncertainty in the estimate of the population mean and
standarddeviationshowsthataminimumofsevenreplicatesarerealisticallyrequiredandalsothatthereislittleimprovementaftertento
fifteenreplicates.

Onthisbasisroutinemethodverificationshoulduseatleastsevenreplicatesandnomorethaneleven.
Backtotop

EstimationofBias
DeterminationofBias

UsingReferenceMaterials
Inthissectionthetermreferencematerialisusedtomeananymaterialforwhichwehaveanestablishedvalueforthetestparameterand
with a quoted uncertainty. Wherever possible a formally certified reference material should be used but this is not always a possibility.
Indeedcertifiedreferencematerialsinleatherandtextiletestingarenotwidelyavailable.

The exercise of checking for bias of a method in practical terms is similar to the determination of precision. Indeed if precision is
determinedusingareferencewithknownvaluesoftheparameterbeingmeasuredthenthesamedatasetcanyieldprecisionandbias.

Theguidelinesgivenforprecisiondeterminationshouldbefollowedandseventoelevenmeasurementsmadeonthereference.This
willyieldameanvalueandstandarddeviationwhichmustbecomparedwiththequotedvaluesforthereference.

http://www.labnetwork.org/en/textileandleather/133methodvalidation

3/8

28/06/2015

MethodValidation

Inmakingthiscomparisonweneedtotakeintoaccountourownprecisionofmeasurementandthequoteduncertaintyinthevalueforthe
referencematerial.Amethodcanbeconsideredaccurate,i.e.freefrombiasifthefollowingcriteriaaremet:

Wherethetermshavethefollowingmeanings:

refIsthestandarduncertaintyquotedforthereferencematerial.Mostcertificateswillquotethe95%confidenceinterval.Thisshouldbedividedby2togetthestandard
uncertainty.

s/xIstheintermediateprecisionwhichyouhavedeterminedforyourmethodexpressedasastandarddeviation.
Isthemeanofthemeasurementsmadeonthereferencematerialbyyou.
cIsthecertifiedoragreedvalueoftheparameterinthereferencematerial.
Putinwordsifthedifferencebetweenyourmeanvalueandthecertifiedvalueislessthanorequaltotwicethecombineduncertaintiesfor
yourmeasurementandthecertificationvaluethenthereisnoevidenceforbiasatthe95%confidencelevel.

Example

Amethodforthedeterminationoftensilestrengthoffibreshasbeendemonstratedtohaveanintermediateprecisionof3Mpaexpressed
asastandarddeviation.Acertifiedreferencesampleisavailablewithacertifiedvalueof2030.5Mpa.The 0.5 is stated to be a 95%
confidenceinterval.

Repeattestingofthereferencematerialreturnsameanvalueof199Mpa.
Thestandarddeviationforthecertifiedvalueis0.5/2=0.25Mpa.
Hencethecombineduncertaintyis(0.252+32)=3.01Mpaandthecomparisontermis2x3.01=6.02Mpa.
Thedifferencebetweenthecertifiedvalueandthemeasuredvalueis(199203)=4
Theequationshowsthatthereisnoevidenceofsignificantbiasinthemethodsinceclearly
6.024+6.02
Thefollowingpointsneedtobenoted:

Theminimumbiaswhichcanbedetectedislimitedtotwicethestandarduncertaintyofthereferencematerialsoitisimportantto
chooseareferencematerialwithassmallanuncertaintyaspossible.
Whencarryingoutthemeasurementsonthereferencematerialyoumustensurethatthemethodisdeliveringtheprecisionthatyou
have determined at verification. This means applying normal quality control procedures and rejecting any measurements of the
referencewhichfailyournormalqualitycriteria.
IfyourdemonstratedprecisiondependsonthevalueoftheparameterremembertouseavalueforSwhichistypicaloftherangeof
valuesinwhichthereferencemateriallies.
UsingInterLaboratoryComparisons

Where there are no reference materials with universal acceptance, a common situation in textiles and leather testing, an alternative
approachforinvestigatingbiasistosendatestitemsfromarangeoftypicalsamplestoseveralotherlaboratorieswhomakethesame
measurementonaregularbasisandthentocomparedata.
http://www.labnetwork.org/en/textileandleather/133methodvalidation

4/8

28/06/2015

MethodValidation

Ideallytwosampleswithdifferentvaluesofthemeasurandforeachmatrixtypetypicallytestedshouldbesentforcomparison.

The number of laboratories used should be as large as possible but in practice at least two will be required. If one laboratory returns a
resultwhichdoesnotagreewithyoursitisimpossibletoresolvetheconflictwithoutdatafromanotherlaboratorywhichwill,hopefully,be
consistentwithoneofthepreviousresults.

The data required from the other laboratories should be their measurement result plus their estimate of the uncertainty of that
measurement.Thesefigurescanthenbeusedintheequation:

Thisisessentiallythesameequationasusedtodeterminewhetherthevalueobtainedforareferencematerialwasunbiasedexceptthat
thecertifiedparametershavebeenreplacedbyvaluessuppliedbythecollaboratinglaboratories.Thetermshavethefollowingmeanings:

lab Is the standard uncertainty quoted for their measurement by the collaborating laboratory. The laboratory may supply the 95%
confidenceinterval.Ifsothisshouldbedividedby2togetthestandarduncertainty.SeereferenceA.

s/xIstheintermediateprecisionwhichyouhavedeterminedforyourmethodexpressedasastandarddeviation.

Isthemeanofthemeasurementsmadebyyouonthesamplesubmitted.
clabValuereportedbythecollaboratinglaboratory.

Youmust,ofcourse,carryoutaseparatecalculationforeachofthecollaboratinglaboratoriesanddeterminewhetheryouagreewiththem
all. If there are one or more laboratories with which you do not agree it is a good idea to carry out the above calculation for these
laboratoriesincomparisonwithalloftheotherparticipantstodeterminewhetherthelaboratorieswithwhichyouhaveadisagreementare
generallyoutwiththeconsensus.

Theoverallpatternofagreementanddisagreementhasthentobestudiedsothatyoucanmakeajudgementastowhetheryourdatais
anomalous.

Similarinformation can be obtained by participation in proficiency testing (PT) but suitable PT might not be available at the time of the
validation.AnotherproblemwithuseofPTformethodvalidationisthatproficiencyschemestendtoonlyprovidearestrictedsetofsample
typesandcoverageofmeasurementrangeateachround.AtypicalPTroundmaywellincludeonlyonesample.
Backtotop
LimitsofDetection(LOD)

Limit of detection is only an issue in textile and leather testing for tests, usually chemical, where levels of residues or contaminants or
compositionarebeingmeasured.

http://www.labnetwork.org/en/textileandleather/133methodvalidation

5/8

28/06/2015

MethodValidation

As a method, especially an instrumental method, is pushed to lower and lower levels there comes a point where the variability of the
measurementissuchthatitisimpossibletodistinguishbetweenrandomfluctuationsinthevaluemeasuredandanactualsignal.Inshort
thenoiseonthedeterminationisbecomingcomparablewiththemeasurement.

Note that by noise we mean not only instrument noise but any other contributions due, for example, to low levels of interfering signals
generatedbyothersubstancesbeingdetected.Forthisreasonthelimitofdetectionmustbedeterminedbyfollowingthecompletemethod
throughsoastotakeintoaccountnotonlyinstrumentalfactorsbutalsoinputsfromthepreparationofthesampleforanalysis.

Thelimitofdetectionforamethodcanbegenerallydefinedasthesmallestsignalwhichcanbedistinguishedreliablyfromtheblank.In
practicetherearetwopossibleerrorconditionsinthisprocess

Wemaydecide,wrongly,thatanoisesignalisduetoanalyteandsoreportatargetaspresentwhenitis,infact,absent
Wemaydecidethatthereisnosignaldistinguishablefromtheblankwhentheanalyteis,infact,presentandsoreportthetargetas
absentwhenitisinfactthere.
Clearlythisbegsthequestionastohowreliablyisdefined.Unfortunatelythereisnoconsensusonthisintheworldanalyticalcommunity
butwewillapproachitintermsofthecommonestpractice.

ThekeyinformationwhichweneedtoestablishLODisthevariabilityontheblank.Thisiswhatwemustdetermineexperimentally.Wewill
thenbeabletoobtainameanvaluefortheblankandthestandarddeviationontheblank.

Thelimitofdetectionisthencommonlydefinedbytheequation:
LOD=yb+3xsB

WhereyBisthemeanvaluemeasuredfromtheblankandsBisthestandarddeviationonthatvalue.Thisshouldbebasedonaminimum
ofsevenmeasurements.
Inwordswearesaying that the LOD is equal to the average result for the blank plus three standard deviations. In statistical terms this
quantifiesreliableasitcanbeshownthattheprobabilityofmakingawrongdecisionbasedonthisLODis7%.

Aminorvariationontheequationaboveissometimesusedasfollows:
LOD=yb+3.28xsB
Thechangesofmultiplierfrom3to3.28changestheprobabilityofawrongdecisionfrom7%to5%.Putanotherwaythisisthepointat
whichweare95%confidentthatthedecisionwemakeisright.Alevelof95%confidenceiswidelyregardedasthetargettobesetfor
decisionsbasedonstatisticalfactors.

Notethatnormallythevalueobtainedfortheblank,yBwillbeveryclosetozero.ThisexplainsthesimplifieddefinitionofLODasthree
timesthestandarddeviationontheblank.

PracticalDeterminationofLOD
InprinciplethedeterminationofLODissimpleenough.Wesimplycarryoutsevendeterminationsontheblankandcalculatethemeanand
standarddeviation.Notethattheblankneedstobeamethodblanknotsimplyablankcalibrationsolutionasitmustpassthroughthe
entiremethodincludinganyworkup.Thematrixoftheblankalsoneedstobeascloseaspossibletoarealsample.

http://www.labnetwork.org/en/textileandleather/133methodvalidation

6/8

28/06/2015

MethodValidation

OneofthedifficultieswhichariseswithLODdeterminationsisthatmanyinstrumentalmethodswillgiveidenticalzerosforablank.Whatis
happening here is that the instrument is highly stable but does not have adequate resolution to distinguish small variations in the blank
determination.Thiswillresultinameasuredstandarddeviationontheblankofzeroandthereforeanessentiallyinfinitelimitofdetection.
Clearlythisisunrealistic.

Inthesecircumstancesweneedtogeneratealowsignalsothatwecanactuallyseesomevariation.Thebestwaytodothisistoprepare,
e.g.byspiking,orobtainasamplewithalowlevelofthetarget.OftheorderofthreetimestheexpectedLODshouldbeaimedatand
certainlynomorethantentimes.Thissamplecanthenberepeatanalysedseventimesandameanandstandarddeviationmeasured.

TheLODcanthenbecalculatedassumingthatyBiszeroandsBisthestandarddeviationonthevalueforthelowsample.

LimitsofQuantitation
Itisoftenargued,withjustification,thatlongbeforeyoureachtheLODthemethodprecisionwillhavedeterioratedtoapointwherethe
uncertaintyonthevaluesreportedishigh.Insimpletermsthereisarangeofmeasurementsbetweenalimitofquantitation(LOQ)andthe
LODwherewecanbesurethatthetargetispresentbutwherethereliabilityofanynumberispoor.Inthisregionweareabletoreportthat
thetargetispresentbutataleveloflessthantheLOQratherthanspecifyingavalue.BelowtheLODwewouldreportNotDetectedbut
wewouldneedtospecifywhatthisactuallymeant,i.e.lessthentheLOD.

TheLOQistypicallysetat:
LOQ=yB+10xsB
Inotherwordsbasedontentimethestandarddeviationontheblank.
Backtotop
QualityControlandDetectionLimitMaintenance

Clearlyitispossiblethatthedetectionlimitmightdeteriorateforatestmethodovertime,e.g.duetoinstrumentperformancechanges,and
thelimitestablishedatvalidationceaseberealisedinpractice.Thismightgoundetectedunlessspecificarrangementsaremadetocontrol
the situation. A spectrometer which has deteriorated in its performance might still calibrate perfectly but the value of absorption
measurementsmightbereduced.

It is critical, therefore, that absolute instrument performance be checked by appropriate system suitability checks. For example with a
colorimetertheactualabsorbanceofacheckstandardneedstobemonitored,forachromatographthepeakareaforareferenceneedsto
betrackedandsoon

Theseinstrumentsarenormallysetuptoreportdirectlyinconcentrationsotheywillgivearesultevenifresponseisdepressed.However
in this case the limit of detection will be worse than that established at validation. Hence the need to monitor the actual output of the
instrumentnotthederivedconcentrationvalues

additionalinformation
Textile and Leather Related Links
UncertaintyofMeasurement
KeyFunctionsandTestMethods
MethodValidation
Design,Development,Layout
EquipmentandConsumables
StaffSkillsandtraining
http://www.labnetwork.org/en/textileandleather/133methodvalidation

7/8

28/06/2015

MethodValidation

ProficiencyTesting

http://www.labnetwork.org/en/textileandleather/133methodvalidation

8/8