Measurement Systems Analysis - Ey il etem ate celg)Cope seion A onc ‘rotan Ppa nergy leroatin Pape td Teminy @ Section A Introduction, Purpose and Terminology Introduction ‘Measurement data ar used more often and in more ways than ever before For instance, the decision 16 adjust manufacuring process is now ‘commonly tosed on measurement data. The data, of some sat calculated ffom them, are compared with statistical conto its fr the process, nd if {he comparison indicates that the process is ot of statistical conte then a3 Adjustment of tome Kind is made Otherwise, the process allowed to ran ‘without adjustment. Another use of measurement data ist determice if 2 ‘Slgnficant relationship exists between two or more variables For example, ‘may be suspected that serial dimension ona molded plastic par i related i to the temperature ofthe feed material, That possible relationship coud be i died by using 8 sass procedure called egession analysis o compare measurements of the crea! dimension with measurement of the | temperature of the Fed material j Studies tht explore such relationships are examples of what Dr. W. E Deming called anatytie sud. In general, an analyte study is one tat increases knowledge about the system of causes that affect the proces. ‘Analytic studies are among the most important use of measurement dala ‘beatae they lead ukimately to beter understanding of processes. ‘The benefit of using a data-based procedure is largely determined by the _guality of the measurement data wd. Ite daa galt 8 low, the beef of {he procedie i ely tobe low Similarly, ihe quality ofthe daa igh, ‘Oe Benefits likely obe high also. “To ensure thatthe benefit derived from using measurement data is great, enough to warrant the cost of obtaining it avetion needs 0 be focused oo the quality ofthe data, Tie qualiy of meant data is defined by the stitial properties of Quality of ape masemens oie fo a esc oyten peaig edt Sable condiions. For instance, suppose tht 9 measurement 5sem, ne ‘operating under stable condoned to oi several mesure of Data s"cerin character Ifthe mesurements ae ll cos (0 the master C ‘ale forthe characteristic, then the quality ofthe data ssid to be “high Similarly, if Some, or all, of the measurements are “for sway” fom the ‘maser value thn the quality ofthe daa i sido be “ow” ‘The stitial properties most commonly used to characterize the quality of, data are the Bar and variance ofthe measurement system, The propety ‘alled ba refers o the location ofthe data relative to a reference (rte) ‘ale, and the property called variance refers tothe spread of he dt ‘One ofthe moat common reasons fr low-quality data is too much variation, ‘Much ofthe vanation in ae of measurements may be duet the interaction between the measurement system and Ii environmen. For insiance,apa Sen A ‘Seen Pree Teminlgy ‘measurement system used lo measure the volume of liquid in tank may be [oive toe ambient temperature of the environment in which its wed. In that case, variation inthe data may be doe ether to changes inthe volume ‘orto changes in the ambient temperature. That makes interpreting the dats tore dificult andthe measurement system, therefore, less desirable 1 the interaction generates too much variation, then the quality ofthe data tay be 40 low tht the dat are not useful. For exorple, «measurement fyaten with «large amount of variation may not be appropriate for use in tmalyzing a manufacturing process because the measurement system's ‘aaton may mask the varton fo the manufacturing process. Much ofthe ‘wor of managing @ measurement system is directed at monitoring and ‘Contoling vation, Among other thiags, tis means tat emphasis needs 10, be placed on Teaming how the measurement sysem interacts with its ‘cvronment so tat only data of aceepable quality are generated. Pupose “The purpose ofthis document is to present guidelines for assessing the ‘qaly of « measurement system. Although the guidlines are genera hough to be used for any measurement system, they are intended primarily forthe mearrement systems wed in the indusal world. This documents ‘ot ininded to be 8 compendiom of analyses forall messurement systems Ts primary focus is mensurcment sysioms where the readings can be repletd on each pat. Many ofthe analyses ae seful wih other (pet of Iretsuement systems andthe mmarual does contin ieferences od fuggestions. It is recommended Gul competent statistical resources. be onsued for more complex of unusual situations not discussed bere. CCstomer approval is required for measurement systems analysis methods ‘ot covered inthis manual a ‘The discussion ofthe analysis of measurement system can become confusing tt misleading without an extalished et of terms to refer tothe common ‘sical properteg and related elements of the measurement sytem. This ‘ection provides a summary of ech rms which are sed in his manta In Gis document, the folowing terms ae use: ‘+ Meavurement i defined as "he sssignment of numbers [or values) to materia things to represent the relations among ther with respect to paiular properties” This definition was first given by C. ena (1963), The process of assigning the number is defined as the measurement proces, and the value asigned is defined asthe rmeasorement value @ Caper —Seion A Intedion, Pej Fen + Gage is any device wed to obtain measurements; frequently sed 10 refer specifically to the device weed on the shop foo, incles {olno-go devices aso, see Reference List. ASTM E45696). + Measurement System is the collection of instruments or gages, standards, opentions, methods, fimures, software, penoandl, caviroamcot and assumption used to quantify unit of measure oF fix assessment to the, feaueé characterise being measured the ‘complete process wsod iS obtain measurements From these definitions i follows that a measurement process may be viewed asa manuficturng process that produces numbers (dota) for is out ‘Viewing a measurement system this way is tsefil becase it allows ut 10 bring to bear all the concept, philosophy, aad tools that have already demonstrated their wefunes inthe area of stitial process cootol ‘Summary of Terms* Standard Accepted basis for comparison + Criteria for acceptance Known valu, within stated limits of unceraity, accepted asa true value + Reference value ‘A standard should bo an operons daft: a defion which wit ‘ald ho some rosuta whan appld by to suppue oF customer, wah the same mearing yesterday, oda, ad tomorrow Basic equipment S&S ‘+ Diserimlnation, readability resoltios Alias: smallest readable wot, measurement resolution, scale limi of detection nit ¥ Aninherent property fixed by design % Smalest sale unit of measure or outpt for an instrument Always reported a a ait of measure ¥ 10tot ne of bumb + Erteetive revolution The sensitivity ofa measurement system to process variation for a parca aplication ‘See Caper, Secon E or terminology defiitons and dceson.‘Choptert— Seon A Inacio, Peps Teminoboey ‘Smallest put that results in usable ouput signal of Always repored ae a unitof measure + Reference value Accepted valu of natifct % Requires an operational definition Used as the surrogate forthe wae vale + Traevalue Actual value ofan rife. Unknown and unknowable Location variation ous) Meer + Bias i Difference betwoen the observed average of measurements and the refereace value Rayner eror component ofthe measurement system 2 + subaty 1 Thangs bi oer ine Y Avsable measurement process is in stastial contol with respect location _— v Alas: Dat + Lineartty ce au Y The change in bias over the normal eperating range ] 7 eee i = operating ange ¥ Reystemati eror component of te measurement system ¥ “Closeness” tothe te value oF to an accepted reference value ASTM includes the effect of location and with errs »@ hapa Secton A Ineoscton Pane Tern) Width variation + Preetslon? ¥, “Closeness” of repeated ceadingso each ober - Arandom er component of te measurement system 4+ Repeatbiity Variation in_meaturemenis obtined. with one measuring inguument when used sever! ines by aa apprise: while ‘measuring the dential charac o the sre part, ‘The varinion in successive (shor-tewr) tals under fixed and my defined condiions of meacurement Commonly refered toa E.V.~ Equipment Variation Testrument (gage) capbily or potential ¥- Within-syem variation + Reproducibility Besa -Variton inthe average ofthe measurements made by diferent sppraisrs using the sume gage when measuring a characteristic omove part @ Dan For product and proces qualifcation,eror may be appraise, environment (ie), or metiod Commony refered to as A.V. — Apprier Variation Berwoen-syem (conditions) variation ASTM £45696 includes repeatbiiy, lbortory, and avroomental effects as well a appraiser effects ass + GRRor GageRER + Gage repeaabiity and reproducibility: the combined estimate of measurement system repeatability and reprodutility Measurement system capably; depending on the method used, may or may not include the effects o time + Measurement System Capability ¥ ‘Shortierm existe of measurement sytem variation (eg, "GRR" ocuding graphics) Te ASTM documet, ere so sich thing asthe precision ofa measurement syste, eh precision cant be epesened by a single umbercaper Secton A ‘edo, Pape ed Temilony ‘+ Measurement System Performance ¥ Longer exiate of measurement ynem varnton(eg.long- | 7 term Control Chart Method) 9 Traceability + Senasty Soll np tha rent i a detectable ot signal Y Responsiveness of the measrement system 10 changes in ‘peaured fete + Determined by gage design (crimination), inberent quality {Grin Equipment Manufacture) inservice mainennee nd ‘Speting condition of he iste! end andar Rivas tepoted as awit of measure National “A DL * ommneney ‘Measurement nage The depres of change of repeaubilty ove time Institutes 2 Aiconastot meanest proces Is in stalin cone with tte rape to wih eraity) + Unlormity A te cern eceee er % Homogeniy of repetbiiy Traceability ‘System variation ‘Measurement system vraon canbe characterized as: + Capabitty 7 Vavabilt in readings taken over a shor period of tine + Performance Variability ia veadings taken over Tong pevodof time ‘Based on ol variation + Uncertainty Anestimated range of values about the measured value in which ‘the tue vale is Believed to be consined [Ai charactozations ofthe tial vation of tho measurement system tasue that tho ajslom fe sable and consistent. For example, the Eemgonenis of varalon can ince eny combination of tne hams Sowmint8 1 The measurement em must be stable (nd consistent Standards and ape Sion A Ieee Puen emoagy ‘The Nationa Instiane of Standards and Technology (NIST) is the principal [Navonal Measurements Institute (NMI) inthe United Stats serving under the US. Deparment of Commerce. NIST, formerly the Notional Bureau of ‘Standards (NBS), serves as the highest level authority for mevology ia the US. NIST's primary responsibly is to provide measurement services and ‘mains measurement stancards that asst U.S. industry in aking raceable ‘meamurements which ultimately asst in trade of products and services, [IST provides these services directly to many types of indstes, but primarily wo thoteindostes that require the highest level of accuracy for their products and that incorporate sate-ofaheart measurements in their processes, ‘Mort ofthe industrialized counties throughout the wold maintain their own Mis and similar to NIST, they alo provide a high level of metrology andards or measurement tervices for thir respective counties. NIST ‘works eolaboratively with these other NMIs to assure measurements made in one country do” aot differ from those wade in another. This is sccomplished through Mutual Recognition Arrangements (MRAS) and by Performing interlaboratory comparisons between the NMIs. One thing 10 ‘otis ta te capabilities of these NMls will var from country to country fod not all types of mesturements ae compared on a regular bass, so differences can exist. This is why i imporant to endesiand 19 whom ‘measurements ae traceable and hw traceable the ae ‘Traceability isan import concept in the trade of goods and services. “Measurements that ae uaceale tothe same or sila standards will agree ‘ore clocey than those that ee not trceable. Thi helps reduce the need for ‘ete, rejeton of good product, and ecceptance of bal produc. “Traceability is defined by the 1SO Intemational Vocabulary of Basle and General Terms in Metrology (VIM) a5 “The property of measurement orth vue ofa standard whereby it can be related to stated references, usualy national or interatonal standard, ‘Brough on unroton chan of comparison ll having sowed uncertainties. “The tncesbilty ofa measurement wil ically be established tough 2 chain of comparisons back 10 Ue NMI. However, in many instances in Industry, the traceability of measurement may be liked bark to an agreed ‘pon reference value oc “consensus standard” between a customer and a stpplie. The eaceablliy linkage of these consensus standards tothe NMI tay oot always be clearly understood, so ultimately itis erica thatthe ‘measurements ar traceable ote extent that satisfies customer needs. With the advancement in measurement technologies and the sage of sate-ofthe- an measurement systems in industry, the definition 1 where and how 9 ‘measurement i traceable ian ever evolving concept.Catt Seton Inde, epee ad Teil Viordengh ace ‘Sc csem : . set Rates age Incite Mod Conprsr cum oe cs t t FewreGoge Miron Calibration Systems ‘manuictring companies, et. to assure that their reference standards are properly calibrated and dietly waceable tothe standards maintained by the ML. These goverament and private industry organization will then use thelr standards to provide calraon and measurement series 10 thet ‘customers™ metrology of gage laboratories, calibrating working ot ther primary stacdards, This Hnkage oc cain of events limately finds ts way ‘nto. te. factory floor and then provides the basis for_ measurement traceability. Measurements tat can Be connected back to NIST through this Unbroken cain of measurements ae suid to besraceable ro NIST. [Not all oxganizations have metrology of sige laboratories within thir facilities therefore depend on outside commersaiependent laboratories to provide traceability calibration and. measurement services. This an Eccepable and pproprite means of ataining wceabiliy wo NIST, provided thatthe capability of the commeriaVindependent laboratory can be assured ‘Brough processes such as sbortory accrediation. ‘A caliration system is se of operations tut establish, under specified ‘conditions, the relaonship between a measing device and a traceable ‘Sandard of Known reference valve and uncaring. Calibration may also include steps to detect, come, report, or eliminote by adjustment any ‘iscrepancy in accuracy ofthe mensting device being compared “The calibration system determines measurement tcesbility 10. the measurement syste through the use of calibtion methods and standards “Traceability i the chain of calibration events originating wth te calibration Esch calibration event includes al . sandards, measurement and est equipmect being. verified, calibration ‘methods and procedures, records, and qualified personnel True Value nodes, Pupec nd Terminology ‘An organization may have an itermal calibration Ibortory ot organization ‘which controls and mainais the elements ofthe calibration events These {ermal laboratories will maintain a laboratory scope which lists the specifi calibrations they ae capable of performing as well asthe equipment and ‘methodsprocedues used to perfor the calibrations. ‘The calibration system is part of an onpanizaton’s quality management system and therefore shouldbe includ i any inteaal audit requirement Measurement Assurance Programs (MAPs) can be used to vey the ‘ccepabliy ofthe measurement processes wed throughout the calibration ‘ysem. Generally MAPs wil ache verification ofa measurement sytem results through a secondary independent measurement ofthe sume feature or parancter. Hodependent measurements imply thatthe traceability of the Fecondary measurement proces ip. derived from a separie chain of Calibration evens from those used fr the ntl measurement. MAPS may Slt include the we of statistical process contol (SPC) to tack the long-term, stbiliy ofa measurement proces. Noe: ANSUNCSL 2540:3 and 1SO 10012 each provide models for many of the elements of calraton systom. ‘When the calibration event is performed by an extemal, commercial, oF independent calibration service supplier, the service supplies calibration ‘system can (or may) be verified through accrediation 19 ISOMEC 17025, ‘When a qualified laboratory isnt avalale fora given pice of equipment, Operation |) Output Unomy, indy ha oly viewed he meanest ad sma oiviy rc bon, Equpcn we moe este Soe ipa te shanti, be mre ce Wage Te Fos fpr wit pemisn fom Serene Sms Asal by. | SRE ea rata oe «+ SOR Ri Ae da or Ph) Ren Ml Eon 2 B(hap - Seon ‘Tee Memon Peet Statistical Properties of Measurement Systems ‘wefulness of the instrument, its compatibility with the proces and envionment, and its usability was rly questioned. Consequenly these ‘ages were often not used propery or simply not wed. ‘The measurement and analyse activity ea process -a meaturement process. ‘Any and ll of the management, statics, and logis echniques of proces ‘conto an be applied to ‘This means thatthe customers and their needs must frst be identified. The customer, the owner ofthe process, wants to make a comet decison with ‘minimum effor. Management mist provide the resources to. purchase ‘quipment which i necessary and sufficient to do thi. Bot purchasing the best oF the latest measurement technology will not necessarily gurebice cont production process contol decisions. ‘Equipment is only one pat ofthe reauremest process. The owner of the process must know howto comecly wee this equipment and Bow to analyze fd interpret the resus. Management must therefore also provide cleat operational definitions and sandards at well taining and suppor. The ‘owner of the proces has, in tam, the obligatin to manor and contol the ‘measurement process to assure sable and corectreuls which includes @ ‘oral measurement systems analysis perpetve ~ the study of the #368, procedure, se, and environment Le, normal eperatng conditions ‘An ideal measurement system would produce only “comect” measurements ‘ach imei is used. Each measurement would awaye agree with standard? ‘A measurement sysem that could produce mearuremens ike that would be ‘sad to fave the ststieal properties of zero variance, 20 bios, and 2270 probability of misclssifying any prot i measured. Unfortunately, ‘measurement systema with such desiabe suatsiealproperts seldom exist and so process managers are typically forced lo use measurement systems that have less desirable statistical properties. The quality of measurement system is usually detcomined solely by the statistical properties ofthe data it produces over ime. Other properties, sch as cos, ease of se, eae also {enporant in hat they contribute to he overall desirably of «measurement system. Bu isthe statstical properties ofthe data produced that determine the quality ofthe measurement system. Statistical properties that are most important for one se ae not necessarily the most important properties for another use. For instance, for some uses of 1 coordinate measuring machine (CMM), the most important stalistical ‘ropes are “small” bas and variance. A'CMM with those properties will {enerte measurements that ae “close to the certified values of standards {hat are traceable. Dats obained fom such a machine can be very ust for salyzing a mancticuring proces. But, no mater how "aral” the bis ‘variance of the CMM ry be the meastrement system which sce the CMM ‘may be unable fo do an accepuable job of disciminatng between good and bad product because ofthe additonal soures of variation introduced by the oer elements ofthe measurement system. 7 Foc fale discussion onthe mate of anda se Ou ofthe Cri, W.Edwants Deming, 1982, 1986, Sources of Variation "Tee mensreeat anlt ms always consider prac per -Secin Tie ern at ‘Management has the responsibility for idemiying the staisical properties that are the mot important forthe ultimate we of the dia. Managerent i also responsible fr ensuring thal those properties are used asthe basis for selecting 2 measurement system. To accomplish this, operational definitions ‘ofthe statistical properties, as well a accepable methods of meaturing them, ae required. Although each measurement sytem may be required to have ifeent satsical properties, shere-are ceralo fundamental properties that define a “good” measaremént system. These Include: 1) Adequate discrimination and sensitivity. The increments of measure should be small relative to the process variation or speciation limits forthe purpose of measuremeat. The commonly Known Role of Tens, o 1-1 Roe, sates that instrument discrimination should Aivide the tolerance (or process variation) ito ten pars or more, “This rule of thumb was intended as a practical minimam rating point for gage selection, 2) The measurement system ought to bein satisical conrolFP6PF ‘Tis means that under repeatable conditions, the variation inthe measurement system is due to common cases only and not due to special causes. This can be refered to as staitial silty and ie best evaluated by graphical methods, 3) For product control, variability ofthe measurement systam must be stall compared to the specification limits. Assess the measurement ‘tem to the Feature tolerance, 4) For process contol, the variability of the measurement system ought to demonstrate effective resoltion and te small compared 10 ‘manufacturing process variation. Asses the measurement sytem f2 the sigma proces variation andor Total Variation from the MSA sty. ‘The statcal properties ofthe maacurement system may change 38 the woms being measured vary. Iso, hen te lerpest (wore vaon| ‘of ns measurement este i smal lave othe eatr of ear ho ‘Process vaiaton othe specication mis Similar tall processes, the measurement system is impacted by both rand and systematic soures of variation, Thete sources of variation are de (2 common and special cautes, In onder to contol the measurcmen system 1) deat the potential sources of variation, 2) liminae (whenever possible) or monitor these sources of variation Ahough the specific causes will depend on te situation, some typical Sources of variation can be idelifed. There are various methods of and ata sige, 1scopes reign cain es suc win sth a case IS aw Peace E SBR] re aon S198 ie wpa ei etl een of 4b vefith | puemececte, kote wera ae i Src” | Sipe agar Sue Moan si 1 pen avietmen Ths may ages oot irons Couple nesrement rem tc notes ob wed 9 ey nb Siete i i FR | co wa oly evel by May Hon els mora wi enw, Bh vere Laban the Bead Corpor MS Append for here crt model PLSMOEA 6 covey ese voraiy ‘Measurement System Variability id Speen modatity ox cence * understandingSept Seton ‘Tho Effects of Measurement System Variability Effect on Decisions Because the measurement system can be acted by various sources of ‘aration, repeated readings on the same part db not yield he sume, identical ‘result Readings vay from eachother duet eammon and special eases. ‘The effcs of the varios sources of variation the measurement system shouldbe evaluated over a shor and long period of ime. The measurement -pulem capably is the measurement system (random) erat over & shor Petiod of time. His the combination of enor quantified by linearity, {tiformity, repeatability and reproducibili. The mearurement system performance, as with process performance, isthe effect ofall sources of Watton over tine, This is Secomplished by determining wheter our cess isin statistical contol (Le, suble an consistent; variation is due ‘nly @ common causes), on target (a0 bias), and bas actepable variation (gage rpeabilty and reproducilty (GRR) over the range of expecied renuls, This adds stability and consistency to the measurement sytem capably, ‘Because the ouput of the messurement system is used in making a decision about ho product andthe process, cumulative efecto a {he source of voriaton is olen cafed mecsuremant system err. oF sometimes ost"eor" ‘After measuring a part, one ofthe actions that can be tken isto determine the sas of tht part. Historically, it would be determined ifthe pot were scceplable (within speciietion) ‘or unnceepable (ouside speciation. ‘Another common scenario ie the clasifesion of parts imo specific ‘atgores (piston sizes), For the rest of tho discussion, 2s an exampla, tho two category shuaton wil be used cut of speccaten (bed) and in speciation (Cye0c"). Tis does nt rest the appeaton of the scission to oher ‘slogorzain ocbvtos. Farber classifications may be reworkable,slvageable or scrap. Under & ‘product conrol pilasophy this classfeton setivity would be the primary Feason for measurjeg a par. But, wih a process control philsophy, interest {is focused on whether the pat variation is due common esis o special causes inthe process, Patvopby overt roduc cota sth part ina speci ctegary? roe comto Tete proen vain anbe ad aceepable? Control Philosophy 2 Driving Interest, Effect on Product Decisions 3 _. oe oper = Second ‘TeNetaret st ‘The next section deals wih the effet of the measurement erat 0a the product decision. Following that isa section which adresses its impact 00 the process decision. In order o beter undertand the effect of measurement sytem err on ‘product decisions, consider the case where ll ofthe varsity fm multiple ‘eadings of single partis due to the gage repenablity and reproducibility. ‘Thats the measurement proces isn state coal and has zero bias ‘A wrong. decision will sometimes be made whenever any pat ofthe above ‘measurement distribution overlaps a specification limit. For example, a good par wil sometimes be called “bed” (Qpe I ero, producers risk or fale ‘tarm) i ce ust ay ‘Aad = ba part wil sometimes be called “good” (ype I enor, eomumer’s Fk or mis ra) if J or AN = | WoTE: False Alorm Rate + Miss Rate = Eror ato RISK is the chance of making @ decision which wil be dotrimontal fo an Individual or process ‘That i, with respect to the specification limits, the potential to make the ‘wrong decision about the part exists only when the measurement Sjstem ceorintersectthe specification ints, This gives ee ditnt aes‘pe Nenreso ees Effect on Process. Decisions where: 1 Bad par wit anys be called bad Potential wrong decison can be made 1M Good pare wl nay be alia good Since he goal is to maximize CORRECT decisions regarding product status, there are we choles: 1) Improve the production process: reduce the varsilty of the process oo that no parts Wil be produced inthe I or “shaded” seat ofthe graphic above. 2) Improve the measurement system: reduce the measurement system error to reduce the size ofthe I areas eo tha ll pats being produced wil fll within area IT and thos minimize the Fisk of making a wrong decison. ‘This discussion assumes tht the measurement proces i in static contol and on target. If either ofthese assumptions is volsted then thee is litle nce that any observed value would lead to correct decision, ‘With process cont, the folowing nods to be established ‘© Sttisia! contol + Onuarget + Acceptable variability ‘As exphined in'the previous section, the messurement eror can cause incorect decisions about the produc. The impact on proces decisions would bea follows: 1 Calling common cause a special ease 1+ Calling a special cause a common case Messurement system variability can affect the decision regarding. the stabil, taget and variation of «process. The basi eaionhip between he sca andthe observed process variation i 20 hap Sectin 8 ‘Tie Nesaremt Pret Oh, = variance of the measurement ystern “The opal index” Cp is dened as _ToleranceRange % 6a ‘The relationship between the Cp index ofthe observed process and the Gp indices of the actual process and the measurement system is Serato ASTMENT.50 ‘recommended ™hae Seon Duerimiaaton Discrimination is the amount of change fom a reference valve tht an insrument can detect and fuithfuly iadiae. This is also refered to as readability or resolution. ‘The measure ofthis ability is typically the value ofthe smallest graduation on the sale ofthe instrument. Ifthe istument bas "coree” graduations, then a alFgaduation canbe used, ‘A general ule of thumb isthe measuring instrument discrimination ought to ‘beat least one-tenth ofthe range to be meaured. Traditionally this range has ‘been taken 1 be the product specification. Recenlly the 10 oI rae ie being Interpreted 1 mean thatthe measuring equipment is able to discriminate to st least onetenth of the process varaion. This is consistent wih. the philosophy of contioual improvement (ie, the process focus isa customer designated age). Figure IE 2: Discrimination The above rule of thumb can be considered asa starting point to determine the discrimination since it does ot iocide any ther element of the measurement system's variability. Because of economic and physical imitations, the measurement system will not perceive all pars of a process distribution as having separate o diferent ‘measured characterises. Instead the measured charctriatie wil be grouped by the measured values into data categories All pars in the same data caegary will have the same vale forthe measured charters 1 the measurement system lacks dscimination (scesivity or effective resolution), it may not be an appropriste system to identify the process ‘aration of quantify individual part characteristic vues, If hati De case, ‘beter measurement techniques shouldbe ated, ‘The discrimination is unaccepable for analysis if it cannot detect the ‘vatiation ofthe process, and uraceepable for corral ft cannot detect the Special cause variation (See Fie 6 ® ‘Number of Categories Control T Data Category Cane wet foetal ony The process vation sal | 6 Unecepale fr eiating wher compared a be oct partes not Specticatons + | © Only inne wheter the + Thelos fneenistatover |” pes s peacing (he capeed proces Si oroting + Themain source of variation ‘oer sean "2-4 Data Categories + Canbewsed withsemi- | © Generally enaceepable for ‘arabe cont ecniques |” suman proces parameter, ‘aed onthe procese td inde sce oly ‘deebtion Provides couse sites ‘+ Canproduce insensitive abe cone ars 5 or more Data Categories Can benacd with varabes | © Recommended Figure 3: Impact of Number of Distinct Categories (nde) ofthe Proccss Distribution on (Control and Analyss Actives Symptoms of inadequate discrimination may appear in the range chat. Figure FE 4 contains two sets of contol charts dived from the sme dat ‘Conto! Chart (2) shows the orginal measurement othe nearest howsandth ‘of an inch. Contzol Chat (8) shows these dita rounded off to the nearest hundredth of an inch. Corto Chart () appear o be out of contol due > the aticially tight limits. The 2er0 ranges are more 2 product of the rounding off than they ar an indication ofthe subgroup variation ‘A go indication of inadequate discrimination canbe Sen on the SPC range ‘hart for process variation. In pariculr, when the range chart shows only ‘one, two, or three posible values forthe ange within the contol im, te ‘measurements ae being made with inadequste discrimination. Also, ifthe ‘ange chat shows four posible vals forthe range within contol limite and ‘more than one-fourth ofthe range re 2eo, hen te mearurement are being, ‘made with inadeqatedeciminaion, Another good ideation of inadequate ‘ierminaton ison a normal probability plot whee the data wil be stacked {nto buckets insted of lowing along the 45 degree lin, ”Ie ert —Srson Seren ene Memories XbariR Chart Reig to Figure LE 4, Contol Chan (2), there are only two posible ‘values for the range within the control limits (values of 0.00 and 0.01). g Drsorinination = .00% ‘Therefore, the rule comely identifies the reason forte lek of contol as ew a ETH ee . inadequate discrimination (sastvity or effective resolution). A ‘This problem can be remedied, of couse, by changing the ability to detect, ew + oo the veriion within the subgroups by increasing the Wscrimination of the measurements. A measurement system will have adequate dacriminaion if its apparent resolution js small relative to the process vaiaion. Thus a recommendation for adequate dacimination would be for the epparen A resolution 10 be st most one-tenth of total process sit sigma standard | ‘eviation instead of the traditional ile which ie the appatent reson best toe one-tenth ofthe tolerance spread. i teveor Eventually, there are situations that reach a stable, highly capable process g tiga ttble,“boirelat menue yt att ane Lio fon — | cl. civ raion ray bude ater epovenet 4 5 of te measurement system becomes imprcticl. In these special cases, ‘measurement plnaing may require allemaive process” monitoring techniques. Customer approval wll typically be required fr the alternative i process monitoring technique XbariR Chart Discrimination =.01 B teva Figure LE 4: Process Control Chart" 3 Figare LE 4 was developed wing da fom Evaluating The Mearrement Proce, by Wheder and Lyin te (Cpt 1989, SPC Press, le, Kaori, Teese. “ ”Blas i the diffrence between the true value (reference value) and the 9 observed average of measurements on the same characteristic onthe same For most measurement processes, the til measrementvriaon is usual Measurement [ony mame es eee cn ly engl cement team Su Process the standard metods of measurement systems analy. fac hee are j= BIAS =| (s,s ottaton oe eal ee Variation smeasremeat systems hat ar nt normaly ditboted. When this bappens, conapind of te combined ees ofall sed nonmaliy is sssumod, the ‘MSA mochod. may overestimate the Sources of varition, known ot unknown, freasrement sytem err. The mensrementanalyt mist cope and ! woe tontbuions tthe ttl ener tends 0 Correct evaluations for non-normal earirement sysens et consinenly and predictably al res L of repeated applications of te same b measurement proces at the tine of ie i ecnaronan HRY Ratner ae | ‘Average Possible causes for excessive bia are: ‘Instrument needs calibration Location + Wom instrument, equipment or fixture Width + Wom or damaged master, err in master + improper calibration or use ofthe seting master Figure HE 8: Charactrites of the Measurement Process Variation ‘+ Poor quality instrument - design or conformance Linearity eror 1 Wrong gage foe he application ‘+ Different measurement method - setup loading, clamping, echique 1 Measuring he wrong characteristic Accuracy Location + Distorion (age ope) ns Accuracy is a generic concept of exacnes felted to the closest of Variation agcemor brwten the serge of uw or more mets eule ant * Bavicrnot vpn, bani, Wert hens ‘eterenee value. The mearrement proces most bein a sat of stasical “+ Violin ofan sumption, eo in an applied constant onl thers he sey of te pce a o meaning In some organizations scury is wed interchangeably with bias. The 60 {ernest Orson fr Standards) ste ASTM (Ameen Sei for Teng and Matra) ase elem secur to enace bth bis and repetabiyIn oder faved cenfson which cul el fom ‘ing the Wor acc, ASTM recommends tha ony the tom Bbw 1th deserporof ton eno. Ths pole wil bs flowed ints et Bas Blas is often veered to as “accuracy.” Becaute “accuracy” bas several ‘meaning in literature, is use a an alterna for “bias” ino recommended, ‘+ Application ~ part size, poston, opersor sil, fatigue, observation ‘ror (eadabilty, parallax) ‘The measurement procedure employed inthe calibration process (i, using “masies") should be as intial as posible to the normal operations measurement procedure ” q stapart Section Cert Secin Neseronar ies Naser es — Satori al in ae mamma ominvne . ‘measurement system on the same master of parts when measuring a single SS Sra andes pl en oul ee ame f= BIAS =| Value 1 Value N Note that unacepiable linearly can come in a varity of favors. Do not ‘sume a constant bis Possible causes for instability include + Instrument needs calibration, reduce the elation interval + Wor instrument, equipment or fture 1+ Noma aging o obsolescence ‘+ Poor maintenance ~ sr, power, hydraulic, fiers, comosion, ras, cleanliness ‘+ Womor damaged maser, error in master + improper calitration or use of the sting master Lneaity- Nonoatant Bas Observed Reference Values Leary Neneertant as + argu tama gn enters 4 + stam dopo ol ets umes | § + it vere eed sep dg caine q + Distortion (gage or pat) } + cnvoumenl f= ene my, vain en i L__ '* Violation of an essumption, error in an applied constant ° Reference Values ‘+ Application - part size, position, operator skill, fatigue, observation F aa aS ete caso yee : une ender aan nel nary | © eer taecrn spin ee Th ditac of ts Groat spud pean ence > perma, pe, yl la, cone, a ‘range of the equipment is called linearity. Lincarity can be thought of as 0 E ‘cleanliness on =m See ae : 7 Ware damped mae err ia mates) ~ aimedhap —Seion Metecnet ies Width Variation ~~ Repestabitiy ‘+ tnproper calibration (oot covering he operating range) or use of the seting masters) ‘+ Poor quality instument~ design ot conformance ‘+ Instrarnent design or method lacs robustness ‘© Wrong gage forthe aplication Different measurement method ~setp, loading, clamping, technique + Distortion (gage or pan) changes with part size ‘+ Eavironment~ temperature, humid, vibation, cleanliness + Violation of en assumption, eoria an applied constant ‘+ Application - pa sie, postion, operate kil, fig, observation ‘ror renabity, paral) Precision ‘radtonally, precision describes the net effect of discrimination, sensitivity and repeatability over the operating rrge (iz, rage and tie) of the measurement system In” some” oranizations precision is wed interchangeably with repeatability. In fac, preison is mest often used to deseribe the expected variation of repeated measurements over the range of measurement that range may bese or time (Le, a device i a8 precise st the low range as high range of measurement or “as precise today a yesterday". One could say precision is repetabliy what linearity ef as (alihough the fist 1s random and te olber systematic eros). The ‘ASTM defines precision in broader sese to include the variation fom Aiferentreadiogs, sage, people, lbs or conditions Repeatabity| ‘This is taditionally refered w as te wihin appraiser” varisbiliy. Repeaubiliy is the variation ia measurements obsined wih one measurement Instrument when used several imes by ene appratser while measuring the ientieal characteristic onthe same part. Thiel the isherent ‘aration or capil ofthe equipment fief. Repentability is commonly ‘refered to af equipment variation (EP), although this is misleading. tn fc, ‘repeatability i the common cause (andon ere varition from sbccesive lwals under pe Secon ‘repens Mess Sy Sy bbe recorded to oneal the smallest gradation o linit of sensivity aod resolution, For analog device, ifthe smallest Scale graduation is 0.0001” then the measurement resus should ‘be recoded to 0.00005". ‘The study should be managed snd observed by a person who understands the importance of conducting a reliable dy ‘When developing Phase 1 or Phase 2 est programs there ar sever fctors hat need tobe considered What effect does the appiser have on the measurement process If possible, the apprise who normaly use he ‘measurement device shouldbe incloed inthe study Each appoiser should use the procadr ~ inci a sts = they normaly use 10 lai readngs. The ast of ry Giterencas. between methods the sportier ‘uso wil be relecte in the Reprod ofthe massurermant sym. 1s appraiser calibration of he measurement equipment likely to ‘be significant cause of variation? If, the appraisers should recalibrate the equipment before each group of readings. How many sample pars and repeated readings are required? “The cumber of pans required wil depend upon the significance of the characteristic Being measured and upon the level of coafidence required in the esimate of measurement system ‘Altough the number of apprise. tals and pars may be \aried when using the rocammendod pracoes doussed in “The manner in which a sudy i conducted is very important All analyses preveoted in this manual assume statiscal independence” ofthe individual readings. To minimize the likelihood of misleading results, the following ‘eps need to be taken 1) The mearurements shoule be made in a random order™ to ensure that any dei or changes that could occur will be spread randomly throughout the sudy. The appaiers should be ‘aware of which numbered parts Being checked ia order 10 void any posible knowledge biss. However, the person ‘conducting the sudy should know which numbered pr is being ‘checked and reord the data accordingly. that is Appriser A, Par I rst a Appraiser, Par 4, second wal, te {tis manual, be numbor of appalers, tls and part should ‘emai constant between Phase 1 ond Phase 2 let programs ‘rbetwoon sequent! Phas 2 ess for common messurament ‘sysioms, Mebiaining commonaly between lst programe and Sauer tts wa nrvecongrcon aban aos 2) In reading the equipment, measurement values should be recorded tothe practical leit ofthe instrument discrimination, Mechanical devices must be read and recorded to the smallest nit of scale. dicrimintion, For eleevonie readouts, the ‘measurement plan must eblsh a cormon policy for recording the right most significant digit of daply. Anbog device should ‘There ism conlatonberween readings See Chapt I Seine B,"Randomiation and Stina Independerce” * 1%put Secten ‘epanen or Mesrenen Syte Sty Section D Cupet—SeieaD Jasbir Rea Analysis of the Results ‘Assembly or Fixture Error Location Error Width Error ‘The results should be evaluated to determine ifthe measurement device is acceptable for its intended application. A measurement system should be Stale before any adinal analy is valid Acceptability Celterla~ Gage Assembly and ature Error ‘An improperly designed Fixture or poorly assembled gage will iocrease ‘measurement evor. Ths is aormally found when the measurements indicate or spay proces instability or ot-f-conto condions. This may be due to ‘excesive gage variation ox poor repeatability and poor GRR values In genera, the fist thing to do when an spparent meaturement eeu exist to review the assembly and setup instructions to make sure the gage was ‘properly assembled, (NOTE: this will not be in the fstvtins) and, for ‘example the clanp/probes are positioned property and have the right loud ‘lao, for automated measurement, verify the program follows required of ‘expected protocol. If problems are found in any ofthese areas, reset or repair the gage and fixtures, then rerun the measurement evaluation. ‘Aceeptabity Criteria Location Error ‘Location eor is normally defined by enalyzng bia and linearity. Jo general, the bias or linearity enor of s measurement sytem it unacceptable if it fe tlgulfeanly diferent fom aero or exceeds the ‘maximum permissible er exalished by the gage calfration procedure, In Such eases, he measurement sytem should be recalibrate ‘or an offset correction applied o minimize tis enor Acceptabilty Crterts - Width Error ‘The erteria as to whether a measurement eytem’s variability is ststactory tre dependent upon the peceaage of te manuécturing production proces variability othe part tolerance that is consumed by measuremeat system avntion. The fu! acceptance eri for specific measurement stems depend on the measurement system's environment aad purpose and ehoald be agreed to by the cusomer. (See Chapter I Section B- "The Effecs of Measurement Sper Varisbiliy”) When beginning to evaluate on organization's: measurement systems, ican be usell lo set proses on which measurement systems to filly focus. ‘Sine the fina! (oll) variation is based | on a combination of the process and measurement variation, (ora = Vohams * lax). when SPC. Is. being appled for process contol or collecting process data, and the contl chat Indicates: that te process Is slabla and the total valaton le 4 acceptable, the measurement system can be considered‘oape it - SecenD ‘Aatnefer Renta ‘acceptable, for this use and does not require separate ro- ‘evaluation, If an outot-contol condlton or noncanformance Is {ound in this stuaton, the frst thing that should be done is to ‘evalua the measurement system, For measurement systems whose purpose i to analyze a proces, a general sguielines for measurement sytem aceepubity is as follows: Ga Deion Connie ‘Under10 | Genely considered obeas | Recommend, pel weil wen eying es oF pement” | sceepubemessuement system. | cau pu or when Ughened procs conta S| reid Opec | Maybe sepals frsone | Decision held be sed upon, er ample, imporance open! | spplenion ofsppliaton mearement cot of eanurement eve, canot rework or resin Should be approved th etme ‘O1e30 | Condado beumcoqabie | Every elfen sbouldbe madeo improve te nessrement percent aren This condion maybe dressed by he we of a2 sprepinemessuen satya xp ting the ‘rope elt of eel readings ofthe same pat ‘Sarasin oro rete Gal mereent vernon ‘Table H-D 1: GRR Criteria Addivonal Width Error Metric Another statistic of the measurement sytem variability isthe number of distinct categories (nde). This statistic eats the numberof categories into which te mescuereat process can be divided. This value should be rater than or equal oS. Cautior The use of the GRR guidelines as thrahold ertera elone is NOT an acceptable practic for determining the acceptably of «measurement system, 7 if etal proces in wie! conto! for both the mean and varson chars an the eal varistion seep hen cn be somumed ier (1) bot the acl proce and mesure watsby ae eeepble fe (2 the measure vation srt sceeplbe with rene fo the yocervoaion (which textenely Sal br bah ae in stasis cone, ee ens the process proauing accepubl rod. nese 2) {he existence af sonanforance or oat of conl canton coud be fle iam (oe the pod ‘eepable but the evolution pt we) which woul cane on evan of bth the mesure! sem nd proces, See Chap, Section E, "Measurement iter” apertt- Seon ‘ess fee Applying the guidelines os the thvesholds assumes that the eased ‘sales are deterministic estimates of the measurement system's variably (ich they are ot). Specifying the guidlines a the thesbold criteria ean ‘dive the wrong bebavior. For example the supplier may be creative ia achieving a low GRR by eliminating rel life sources of variation (eg, part to gage interaction) or manipulating the measurement study (e8, producing ar outside the expected process varitio). ‘Comments on the Application and Gage Acceptability ‘When looking st GRR and measurement variation it is important 1 ook at exch application individually, 19 see what is required and how the rmeasurement is going to be sed. For example: the required precision of Temperature meafurement may be differet fr dissimilar applications. A oom thermostat ean regulate the temperature for himan comfort and i ‘economically priced, but ray have a GRR vpwards to 30%. It eccepabe for this applcatoe. But in a laboratory, where small variations lenperstoe can impact test results, 2 more sophisiesed temperance ‘measurement and cotol is required. Thi hermosa wl be more expensive and is required to have less variability (ie, have lower GRR). “The final accoptance of a massuremant system should not come own fo single st of Indes. The longitem peromance of Be ‘mosauremont sysiem shoul so be reviewed, for example, ing ‘raphical analysis over tne.c Caer ‘syafacree ‘Recomended Prac Rpt Messen yt CHAPTER III Recommended Practices for Replicable ‘i 4 Measurement Systemshae -Sen A ape Secon A tate Tor Pct tome Ta Presa ’ Section A Example Test Procedures Examples of specific test procedures are presented in this chapter. The procedures are simple to use and can be readily applied in & production envionment. As discussed previously, the tet procedure which shoud be tied to understand a measirement sytem and to quantify is varity ‘depends on the sources of varation which may affect the measirement system. In many situations the major soures of variation are de 10 the instrument (gage/equipment, person (appraiser), and method (measurement procedure). The test procedures inthis caplr are sulfiient fr tis type of ‘measurement system analysis. “The procedures are appropriate to use when: Only two factors or conditions of measurement (i. appraisers and por) plus measurement system repestbly” ae being sudied The effet of the variability iin each pat is negligible ‘There iso statistical interaction between appraisers and pans ‘The pars donot change functioaly or dimensionally dug the study, ie are replicable ‘A satisical design of experiment can be conducted andlor subject mater knowledge used to determine if these procedures are appoprise fr any specie meaaurement systemae Secon A ‘Ee Tat Peete Verte Mesure Syren Sel Gln Section B Variable Measurement System Study Guidelines ‘This section consis inplementason uidelins forthe measurement techniques described in Chapter, Section EA thorough review of Chapter 1, Seetion Eis recommended to ensure proper application of these guidelines. Guldelines for Determining Stability Conducting the Study 1) Obiain a sample and establish is reference value(s eave 1 a trnceable standard. If one is not avallble, select production ‘art that falls in the mid-range of the production measuremen's 4d designate ita the master simple for sbility analysis. The ‘known reference value isnot required for racking measurement system sablty ‘mg, It may be desimble to have maser samples forthe low end, the dP high cod, and the mid-range of the expected measurements, ‘Separate measurements and ootol charts ae recommended fot cash 2) On periodic basis (diy, weekly), measure the master sample tivce to five times. The sample size and frequency should be based on knowledge ofthe measurement rystem. Factors could {clade how often recalibration or repair hasbeen required, how ‘fequenly the measurement system ic used, and how stressfl the ‘opering conditions are. The readings eed wo be taken at a fering times 10 represent when the measurement system Is a fcruly being wed. Tis will account for warm-p, ambient of ‘other factors thet may change during the ay. 3) Plot the daa on an 4 R or J &s contol chart inte onder. Analysis of Results - Graphical 4) Establish control mite and evaluate for outofcontol or ‘unsable conditions using standard conto chart analysis, ‘Caton shouldbe we wher a production master could experience excessive wear dt use, materi and ‘anling Ths my rege modying the production pr, sucha pling fo extend he eof he master aspte ‘ere Means Sytem uy —Gsdelines Analysis of Results — Numerical (Other than normal contol chart analyses, thee is no specific mumerial analysis or index for stability.” I the measurement process i subl, dhe data can be used to determine the ‘as ofthe meaurement ayer. ‘Also, the standard deviation of the msnsurements can be used as an fpprosimation for the measurement stems repeatability. This can be compared with that ofthe process to daermine ifthe measurement system repeatability is suitable forthe application. eeign of Experiments of otner anatyical problem sting techrlquos may be requied to determine the pe contutors 10 tho lack of eaturementsysar stabi. Example - Stability ‘To determine if the sabiliy of a rew measurement insrument was sccepabl, the process team selected par near the middle ofthe ange of the production process. Tis part wat seat to the measurement lab 10 toss the reult btained from the eapicl analysis ie Je his eas, it does not mater wht ration 5 bas of, since theresa linearity problem. Possible causes for linearity problems can be found in (Chapter, Seton E, "Location Variation: ‘the measurement system has a lneaty problem, it needs tobe ealbrated to achieve zero bias through the modification ofthe hardware, software oF both If te bins cannot be adjusted to ze bins Groughout the measurement system range, it stl can be used fr produeyproces contol but not analysis as lng asthe measurement sytem rein table. Since his has high isk of appraiser oor, t shoul be used only wat be concurrence ofthe custamer. Guidelines for Determining Repeatability and Reproductbilty** ‘Average and Range method Gichuding the Consol Chart method) + ANOVA method Except forthe Range method, the study data design is very similar for each ofthese methods. The ANOVA method is prefered because it measures the operat to par interaction gauge eror, whereas the Range andthe Average td. Range methods does not include this variation. As presented, all tmethods Ignore withn-part variation (such a5 roundness, dametrc tape, flmess, et, as discuseed in Chapter TV, Section D in thei analyses, “Tho ANOVA approach can dant apprtearpatniaracton but it an also evaluste other soureas of variation which fs the reeson ‘shy wes incu. Hatorcly, the assumption 16 modo tat te Imvac ls ze, a which caso th rests o both approaches ar ‘cuivalon Wit) "hat sai, the ANOVA’ approach Is proferod Because of Rs lexi i ine user hes sccass lo a epproptats ‘computer program. ira, tha X bar and R approach appropriate ‘end ean be done manually or via computer program. ‘See Caper, Seton , for an operational defntion and discussion of pote nue. 101a (per -seie ‘Vatican Sym Sey - Cuties 1 2 3 s However, the tot! measurement system includes not only the gage itself ond it related bin, repeatability, te, ut alse could nce the variation ofthe pats being checked. The determination of how to handle within-pat ‘ariaton neds to be based ona rational uederstanding ofthe intended use of the par andthe purpose ofthe measuremes Fnaly al ofthe techaigues in his section are subject othe prerequisite of ‘stitial stably. Ambough roproducilty. fs util inorreed as appraiser arian, there are sluations whan this varaton la v0 fo other Sources’ of variation. For” ouarple, wits vomo inprocess ‘oesuremont sytem thro am na human appraisal the pats aro handed, fatrod and moasurod bythe samme equipment, ‘on repreducbiy fs 20° Le, only a ropealbity stay hooded. i, howover, mile ures. are Used, tron the reproducty ste Devout van. ‘The Range method isa modified variable gage study which will provide a «quick appresimation of measurement vaiabliy. Tis method will provide rly the overall pice ofthe measurement system. It doesnot decompose the variability into repestabiley and reproduiiiy.Iis ypcaly used asa ‘quick check to verify that the GRR hes not changed. This approach hes the polelal to celect_an unacceptable rmossurement system" 60% of tho te wih 0 sori size of § ‘2nd 60% of te te with a sampe ae a 10, ‘The Range method typically uses two appriers and five pars forte study. Tn this std, both appraisers measure each pat once. The range foreach par. is the abrolute iflernce beoreen the meiauement obtained by appraier A tnd the measurement obtained by appraner B. The sum of the ranges is ound and the average range (ZR) is calculated. The total measurement is found by muliplying the average range by Je whee is found in Appendix C, with m= 2 and g »rumber of pars. “Appraler A ‘Appraber 3 Range (A.B) 08s oo 005 ons, ono 0s 100 os 0s os oss 10 oso 060 10 om = (2) () ( (Process Sanaré Devon » 0.077 frm revi sud) 15.% HGR = 100 + ( as Process Sandard Deviation Tabten Gage Study (Range Method) To determine what percentge of the process sanded devition the Ieatrementvaaon comes, conven te GRE ts pce ‘lying by 100 and diving by the proces sand devion, ta he grape (ee Table IE 6) the proces Sanda! dviton fr is charter O77 herr, Process Standard Deviation Now that the 26GRR for the measurement system is determined, an interpettion ofthe eau should be mode. la Table MI- 6, the SSGRK 1 determined to be 75.7% andthe contusion ith the measerementysem fs in need of improvement sone = 10 + (, a ‘Average and Range Method ‘The Average and Range method & fs an aproach which wil provide an estimate of both repeaubilty and reproduce fora measroent ‘pst, Unt te Ranged. ts apps wl sow ees system's variation tobe composed nt two aepante componen, {speatabiity and reprodcily.” However vataon ee w Be neato tetween he appar andthe pn/gage sno secoune forint sesh be wed detemine he interaction between the gag and appa, isch exits 13ape uh —Secin soos SS Mecano conducting the Study pe uber of appraiser, ins and pars may be vcs the acon repens he pian cones fer oni ee Gn on sen en Teed pre 8 FB in aml of n> 10 pr a epee alo 1) Oieed range of proces varain ute as A.B, Ce and umber he pas 1 ‘ 2 RE yrs tthe umber ar ot visible ote appraisers. {sn Cope Secon re tbe age if is spa of he normal measurement tem 9 ee ee Anu vo rnd oe Serene nro eran C measur the ttn pars witout sexing ech eg cr ren ows 6nd Tepe ibe ele sing diferent random okt of measures. 9 Boat in rows 2,7 and 12. Record the data in the spproprinte Pot example fin pee mesure pat 7 hen cod Sse tae nbeld pat 7 Ite wal ar ned, 2 Se cyl eee dain tove 3, Bed 13 Mt «9 Segs4 nd 5 may be changed tothe folowing when lrbe Pst 2¢ Sette uaeaiy fpr makes ccs 4 at opprasee A measure the Fst pat and record the reading in Hae TLet oppaiser B measure the fist par and record the (ding in rom 6 Let appraiser C measure the fist part and {rord the eading in ow. 1 let appraiser A repeat reading on he fist part and record the ating in ow 2, appraiser B record the repeat reading i row 7, ‘Sd aprnse € record the repeat reading row 12. Repeat his ‘Scand ener he ela in rows 3,8, and 13, bre ia are fabe used 1 Aadertve ead may be wed if he appraiser on iat ‘Ma Tet appraiser A rmeasre all 10 pars and coer the reading in SN Then have appraiser A repeat the reading ina diferent ord 1 der he resls rows ? and 3. Do the sare with appraisers B oc Mi —_———_— ought to be > 15 for a minimal level of confidence in the ress. Rea eT ISLS nt pment yt me ee eel ee ey sen ion cat ates oo set ue keene o SOS sane vo CongleI~Seten Vata Mesurnen Sym dy Gdns, Gage Repeatability and Reproducibility Data Collection Sheet ‘Appraver ira 134] 047] 080] 002] 059/031 a7 075] 020] 127 056] 0.17 Range 1 [oe 047] 1.10] 001|-056] 020] 047] 06a] 2 | o2s/-122| 094] 103] -120/ 022/ 055] aoa) 212/162 3 | oo7]-o68| 134] 020] 1.28| 006{ os|-034| 219] -120] Averse Range 1 [os] 130] oss] 0.14[-146] 029] o02| 046] 1.77] -1.40] 2 | -oas]-119] 100/ 020] -.07|-67| oos|-os6| s4s| 177 3 035] 096] o67| 0,11{ 1.45] -049] 021] 040] 187/216 ‘Table HEB 6a: Gage Repeatability and Reproducibility Data Collection Shest losaa tse nyt Analysis of Results - Graphical“ ‘The we of graphical wool i very important. The specific gaptical tools sed depend on the experimental design employed to collet the dat. A ‘tematic sreening ofthe data for apparent special causes of variations by ‘Sing graphical ool abould precede any cher saitical analysis “The following are some of the techniques which have proven to be usefl (Gee aleo the Analysis of Variance Method), “The daa from te measurement system aralyss canbe displayed graphically ty consol chars "The Hea of using contol chars to answer questions ‘concerning the measurement system has been used by Wester Elect (see Reference List TAT Static! Quality Control HandBboo). ‘The svenges ofthe ule readings by ech appr on eich prt ate Observed Vac ~ Port Average ‘The histogram plot provides a quick visual overview of how the emo is 1 istibated. Issues such a. whether bias or lack of conisteay exist in ‘he metsuremens taken by. the sppmisers, can be identified even before the data ae analyzed. (a cere ee se ga oy gee oh ee co Re ae es meh ete Poe mms ‘Nove that he" o eck ofthe histogram ara wih respect o each ote usSay Cutis ‘The averages of the multiple readings by each appraiser on each part ae 1 Plot of loted with the reference value or overall part averages as the index (see gira Hae ee Occ mae ‘See “+ Linearity (ifthe reference value is used) ‘Consistency in Hnearty between appraisers Comparison XY Plots 128 1.25 128 125 ope Sesion 8 ‘ore Menno Sy ty Glan, ofthe multiple readings by each eppmiser on each pat are piloted aguas each other with the appraisers ex indices. Ths plot compares the values obuined by one appaser to thse of another (see Figure MILB. 14), If tere were perfect agreement benween appraisers, the ploted pois ‘would desribe a stig line trough the origin and 45°10 the axis, Appr wt] ve Appr(Shp Secten Vat MconrmesSytem Say — Cuties <———— The Gage Repeatability and Reproductiliy calculations are shown in Numerical Figures II-B 15 and 11 16, Figue II-B 5 shows the data collection sheet Calculation’s «8 which all study results are recorded. Figure II-B 16 displays a report Sheet on which all identifying information is to be recorded and all Calculations made according fo the petribed formula, Reproduce bank forms are avalable In tho Semple. Forms ‘tection. Tho procedure for dong ino calelabons sar deta have ‘oon cailectd le follow: (Toe following refers to Figure II-B 15) 1) Subtact the smallest reading fom the ages reading in rows 1,2 and 3; enter the result in row 5.Do te se for rows 6,7, and and If, 12, and 13 and enter result in rows 10 snd. 1S, respectively. 2) Entries in rows 5, 10 and 15 are ranges and therefore always positive valu, 3) Total ow 5 and divide te total bythe number of pas sampled to chain the average rage fo the frst appraisers als. Do the same forrows 10and 15 o obain Rand Ri 4) Transfer the averages of rows 5,10, and 15 (R,, RR.) or0w 17, Ad them ogether and divide by the numbe of appraisers and cer rests B (average o al ge) 5) Eater B (average valu) in rows 19 and multiply by De “* 1 get the upper contol imi. Note: Dy it 3.27 if two trials are weed. The ‘value of the Upper Contol Limit (UCL ) of te individual ranges is entered in row 19. Note: THe value of Lower Contol Limit (Lc) fr less than seven wialsis equal w 20, 6) Repeat any readings that procuced a range greater than the caleulted UCL using the same appraiser and pat as original sed, oF discard those values andre-average and compute and the fimiting value UCL, basee upon the revised sample size ‘Comect the special cause tht produced the out-of-control condition, Ifthe data were pled and analyzed using a contol ‘char as discussed previously, ts condition would have already ‘ben comected and would not occ her 7), Sum the rows (rows 1, 2.3, 67.8 11,12, and 13). Divide the sum in each row by the umber ef parts sampled and enter these ‘ales inthe rightmost cohumn hibeled “Average”. Se Seal Process Conve! (SPC) Reference Manual, 208, o ober atisical ference source for x able otters. $y tape Sates 8 Vara Menten yen ly Gans 8) Add the avrg in rows 1,2 and 3 and vite he ttl bye umber of tls ed enter the vale in tw 4 in eR, lose Repel this fr ows 6,7 and nd 2 and 13, and ete he rela in the blocks for, and. in rove 9 and ta reectiely. 9) ater the mic a rniou aveages of ows 4,9 and 1 in the appropri spe in row I8 and decrnin We difeees Enter this iternee inthe space beled oe into 18 10) Sum te measurement for ech via fr each, il for each part and vite he tol bythe ember of measures rune of ta ee ‘umber of eppalses). Ene the aul ow 16 os Provided forgo avenge 10) Subtaetthe sles pat average fom the gst pan 2 and ener the emit in the pace bled, intow Ry eke mtg: of pan svenges ’ (The following refers to Figure IN-B 16) 12) Tan the calcu values of B, Ray and R, tthe baks rove one eo side ofthe faa 1) Pest te asain ude be enn ee “Mesuenen ‘Unit Analysis” on the left side of the form. o 19 Perm te alelatons ander e cola ened“ Tot ‘Variation on the right side of the form. Taal 15) Check the results to make sure no eor have ben made—__—_ ape secs — Seine Sym Sty Coie Vate Meron Sine -——— ‘Gage Repeatability and Reproducib rt ‘and Reproducibility Data Collection Sheet fran Na & Name a Gage Repeatability and Rep! ty |Charscterisies Gage No eee Spesticons Gege Type: “Wate mae Fromdunstect R-0.3617 _Xnqy 0.4848 . Roast ifs 096 Meamreient Ua Ansa 25 Tal Volto 77) Repenbiy- Equipment Varition 7) a 068 a = Rxx, Key = 10016" : 7 03417 x oss08 Tm], 0 jo.28t007.14610) io fo 03903 ozo 2 [awa] liane 51 Rawee] 035] 092 023] 06] ox6] 027] ot] R- onee 3 [asses sR ae Rody — Appin Varn Cy 7 2 ois AY = Naan XK) = EV fey ‘may = 100 Lavery, ' 3 080 = Vlo444.05259) - (0201084 /10%9)) ~1oojoaaeeart. 14610) > [rear [ane = oases waza [oem] 27 [3] -aneen tol renee] 018] 075 aaa] o36| o71| 039] 016] Rosia stme r= mu [xs [oro [osm Repesebiiy & Reprod (GRR) 7 . one Vers av ‘eank = 100887) 7 r apo = fom one) [ru] = too{= o30evs9.148%0) 4 > o30s7s 2 [awn] 280% 15 _wamge|_o10[ 042] a2] o00| 030] o38| 020| 010] 042] 087] R= oszs 3 Joss ‘6 F- .oow ae «lesser | 977 = orem, ual 0100/81 s01e| oat] ree] axel oats] asea] a0] srs ee a oo sen | |e = 1.0486 amlare |e eee 10] CR, ~esear UR, «0819 >(F, ~ 0320)/( FOF APPRAISERS =3)= 03497 Reo || [Tent varnion rH sBliGai 18] pt = e03)— loin X = 42549) Fagy = 04088 1 yw aa = Pr & [aans 0)e (Reommxto,-2501-uct, «0806 Hoses soneet] [5 [ome] me = (75) 0, 13 eo 758 ft UKs mem enfin Cie ove a we t = tao 10 Joaree] = 140c.t04sar 30675) jot i in ey acne cares Rept hs eg sig naps nd it ogi wee : caoeaes scab strong tecape Had einige sing enon Ltt eenenmnntiieesetnimriettoncn id oe: Figure i 6: Gage Repatabityo a Figure IH-B 15: Completed GR&R Data Collection Sheet : Reproducibility Report us . 9‘Shap sein ‘ese Meanoenen Sytem Su ~Cuieline ‘Analysis of Results —Numerical ‘he Gage Repeal and Rprodiclly Dota Colleton Siew Mot ees S'S eed B16 wl roves watod fre eechea wate othe sy Gta The sales wil stone ‘arnt at pce’ of poser ata fr tl mexnenen Se i's compose npeccbiy, rt, nad prt wean, Ta ‘Sommice sets compar wad compen he re of opie mat On the Ie side of the fom (Fue LB 16 ner Mexnceent Uni Paap te sontnd Gens caked fer ech enpnet of — The repeaubiliy oF equine vation (EY or i Semin ‘multiplying the average range (R ) by a constant (K;,). K, depends upon the tuner of is aed a he gooey ud et the vee of tc is ied fm Appendix Cd edepedent on te ane of als (n) nd he pb of pats imesh ner fara) axed {Capen than for cottage ve of) The mpecilty or ape via (47 or ,) it detemind by mpage maim eeag apie dierece (K own ) by 8 Cont): pends pone sai of poser wed he ima ade vee of wc oie fom Append C. ds Scyodcl ont umber af opis (wand I soe re tome esata Since ope tates contami ye Taipan vrai, an be ase by bang ¢fatin of Be SRizeet ton thet he spar ion 1) cay ee where n= numberof parts and = numberof til I a negative valve is calculated under ta square root sign, the appraiser vation (AV) defaults to zero, ‘The measurement system variation fr repeatability and reproducibility (GRR or oy is calculated by adding the square ofthe equipment variation andthe square ofthe appraise variation, and taking the square root 2 follows: AV = l( Rear * Kx Gar = Evy + (avy ‘There are generally four diferent approaches to determine the process variation which is used to analyze the acceptability of the measurement aviation: The mea aus nthe example were developed a they were computed manually, the els were fared an founed tone dttnal decal ag Ansys by computer programs shoud maintain the Interment ales oe maxima pein ofthe computrprogamming language. The els from 8 ait Cemmputr program may difer fom the example euls ithe second br rater decimal lace but the Gat ats wil emia he se. vie Cope SesionB Meaneenent Syn Say = eet 1) ing proces variaon foes vain, ten fom te pre in he GR dy © tie when the sled sample represents the expec roces varia (peered epton) 2) surogate process aii 0" ee lice spe epee rote sot vale but an cxsing proces wd soar poe ‘aiaon is available " 2 Poor py get valve © "tae when sulin samples represen the proces are at trailed stg ose poet ‘aviation is aot svalabe ore now ree expel to ‘have less variability then an existing process. “! 4) speifeationlece © When the measurement sytem ist be used 10 son the Poses andthe proces has Pp <1.) ‘The par variation (parttopan; part variion without measurement vatation) (PY ot o,) is determined by maltpying the range of part ‘averages (R, bya constant (K). Ky depends upon th nibs of pars cd ‘in the gage stay and isthe inverse of which is obtained fom Appendix Cd is dependent on the mur of prs (mand (fa his situation y= 1 since there i only one rage calculation, ‘The total variation (TV or o) fom the study i then cated by sumeiag the aur of bth he peti andreprodubiy variation nde pat variation (PP) and taking the square rot ss follows: acm TH = SiGRRy = (PrP ‘slog Historical Variation Information To ase this proach the information must be fom a process tha i in ‘tistical contr Ifthe process variation is known an its vl based on 60. then it canbe used in place ofthe tol study variation (PY) eaenited fom he ge stay dat. This Seog by peroming oe fwing Ty = beseeveriation 2) PY = fervy ~ GRRPTs Caper Secten Vek Meneeeoet Sytem Sty Oseos Using Pp (or Ppk target value i “Towse he Pp option, use the following TV inthe GRR analysis: USL~LSL _ USL LSI ov PP Pv = firvy - GRR H slag the Tolerance (Specification Range). See eee Yo amt a x poucon process wih 2 Fp of 1.09, OEM ae meee tarsen slat Same omar apenas eke ya ti a ps se a eaten ‘Towse this option, use the following TV inthe GRR analysis: nd USL-LSL 1 ee sot PY = YTV - (GRR)? Indices i the variability for each factor in the gage study is determined, it can be oe i ee coor Se ee fee me \ "% Total Variation.” i “The percent the equipment variation (94E¥) consumes ‘of the total variation | (TV)is calculated by 100{EV/T7), The percent that the other factors consume { STine tora variation canbe similarly calculated as follows: ! %AV = 100[AVATY YGRR = 100 [GRRITY) Seren ‘THE SUM OF THE PERCENT CONSUMED BY EACH FACTOR WALLNOT EQUAL 100%. percent cif “The results ofthis percent ttl variation geed to be evaluated to determi the mensurcent sytem i accepuable frit intended sppicaton. [yar xa ee Csr, Fd and GM, PPAP Moma m2 (V7, Etthor of both approaches can be taken depending on Sha ntnde uae of he moesiroment system andthe dean ‘The final step in the murercal analysis isto determine the number of tne: categories that can be reliably disGeguiched by the measurement sytem. ‘This isthe numberof non-overlapping 97% confidence interval that will span the expected product variation (Yorn) Given that the graphical anlysis has not indicated any special cause ‘variation, the re of thumb for gage repeatability ond. reproducibility (G6GRR) may be found in Chapter I, Seton D. For analysis, the mde the maximum of one or the ealulated valve truncated to th inter This result shouldbe greater than or equal to 5 To aval a nde = 0, which s posible wih tuncaion alone, some computer programs wit ound up the calculated reeulL. This ean result In iferencas {inal reports when the same dala is evaluated by offre programs ‘When using the Pp approach to 7¥, te eaeultion for nd is == nde TV? = PV# + GRR? or PV? = TV? - GRR* then pv = 14M ot, Aree = 144 ‘Analysis of Variance (ANOVA) Method ‘Analysis of valance (ANOVA) i a standard satsical technique and can be ‘sed fo analyze the mensurement error and otber sources of variably of dun in a measurement aysiems study. Inthe analytic of variance, the variance ein be decomposed into four categories: pars, appraisers, {neracion between parts and appraisers, and replication ero due to the esse. The imporance of the nmber of dint categories (de) in contol and alse acts i dicated in (Chapter Seton E, "Measurement sus” especllly Figure 3) Cacaaton of reviewed in Chapter Ty, Secon B, “Anais of Rest - Namen” 1eee ect ‘The advantages of ANOVA techniques ab compared with Average and Range methods are: + They ae capable of handling any esperimental set-up + Can estimate the variances more accutely ‘© Bxtrct more information (such ss interaction betwoen parts and sppraiser effet) fom the expeinental dat ‘The dinadvantages are thatthe numerical computations are more complex and users rue certain degre of statatial knowledge to interpret the feauli The ANOVA method as descrited in the following sections is advised expecially fa computer is aailab, ‘The method of collecting the data is imporant in an ANOVA method. Ifthe dota are not eolete ina random manner, this can lead to a source of bins ‘alee A simple way to asare a balanced éesign for (1) pas, (2) appraisers, tnd. (7) Wals is through randomization. One common approach 10 randomization ito write dl on lip of paper to denote the measurement for the frat appraise o the fst pa. Do his ep to A() forte measurement By the fret appraiser on the part. Follow the same procedure for the next sppraser upto and including the * appisr. The similar notation willbe ‘ied where By, Cy denotes the measurement for second and thin appraiser ‘on the frat par. One all nk combinations ae writen, then the slips of paper ‘in be pt ina hat or bow. One at ate, a lip of paper is selected. These ombinmtions (4, Bj, .) are the measuring order in which the gage study snl be performed. Once allt combinations we acl, icy we pt buck into te at andthe procedure ie followed sain, This i doe fra toa ofr times to determine the order of experiments foreach repeat ‘There are alterate approaches to generat random sample. Care should be exercised 0 differentiate among randon, haphazard and convenience ‘somping.” In general, all efforts need to be taken asturesatisical independence swith the sy. Conducting the Study ‘The data can by collected in a random anne using a form sia to Table 11 Ga. For our example, there are ten parts and thre appraisers and the experiment tas been performed in random order tre ines foreach pr nd ‘ppmiser combination ee —aeat 2 ites ioe ‘oo yey, Belen the Measurement Proce, Scoed Eatin, 1989, 27 {SiS 20 Feb 1996) war an American mathematician who wat «health physics wid the Coa om 194} ding WW I He bec 0 tecopiel exper i paraderandom umber $$$ Shep Saon Vaiahe Meseenet Syn Sy cant Graphical Analysis ‘ayo gpl meds given athe dcso of Gap Ani ‘ove cont edn te gpa of ta elect spares ANOVA any Tae metots can be ted cof wd pov Bak isp he aes nds nyc) cree lta ctr ae ee erste rere nate Sanne en neg Powe Seeeiet Os comstamits Wee eg Ciara saiauie emai med il ates ag sane in Sea eee incre Secctel ecto Seema cr omit ice eg emote itiaurraccemattmemrats SRE pyaar hon re ee rel eee Figure 4B 17: Interaction Pit st rin ee tp an ‘et est sr eae (scammer mace ae cartier Ce dara mets ihe ele i's eect ee Pl chp gst eaten arora hen ance oanlarcantinte asas eed Vole Figure IN-B 18: Residual Plot Numerical Calculations ‘Athough the values can be exalted manuly, most people will we ‘Computer program to generate what is called the Analyis of Variance (ANOVA) able eee Appendix A) “The ANOVA table ere i composed of five columa (sce Tobe LB 7). + Source column isthe cause of variation, i + DF column ie he degre of freedom assointed withthe sour. ‘+ SS.orsum of squares column is the deviation around the mean of i the soure. MS 0 mean square column isthe surn of squares divided by depres of freedom. © Frotio columa, ealeulated to determine the statistic! ‘Significance ofthe soure valu, i The ANOVA dable is used to decompose the tot variation into four Compoosais part, apprises, intrection of appraizers and pasts, and " ‘epeatablty be to the nstunent | For analysis purposes epsive variance componcas ar se to 20 ' “This information is used to determine the measurement system characteristics ssinthe Average and Range Method tie Mame Sn ta os a 7 @D rT 598 mae ree 9 m9 same ae pomierbyPat 0380 aime eae Eeiomet 205 ose sa seen 7 Signieat at = 005 eed ‘Table LB 7: ANOVA Table ‘Table LB 7 shows the ANOVA calculstons for the example data from Figure II-B 15 assuming balanced two-factor factorial design. Both factors sre considered tobe random, Table IB 9 thows the comparison ofthe ‘ANOVA method withthe Average and Range method. Table II-B 10 sbows ‘he GRR repor forthe ANOVA raeod. % Toa Vernon srbaen é i) Pam 19953 3a epi) : leoosiass 47022608 ng ‘ apriven pao nr-0 ° ° Govzacion | Sytem ‘ore 0302375 ns 18 | aooin | (Pep to) | Peas PY Loa 60 na i) x0 Toul Vaion 1-108 ono | “Table HLB 4: ANOVA Analy % Variation & Contributes (Estimate of varlance based on model witht Interaction)pa pete sem Sy - ine lone + PP ef TatVariation so( em) % Contribution (to Total Variance) Analysis of GRR Studie: “ rnedod aed ANOVA method wil ov rc conning csr esac 3m or ae Yao. For example if repeal is ge compu 10ers the eons maybe + Theinsrumee neds manianse The gage may ned tobe rdergned tobe mor gi {Te clamping or lcaton fr eng neds tobe improve Theres xesivewihin-partvariato. Ar eproduciy is lage compared to rpeatbiliy, ten pone cates touldbe ‘The appraise ees t be eter ined in ow to te sd ea the eae nomen. + Calon othe geal went ls. ‘iano some son maybe nde 1 ely the pra ue the EAE ME comin us @® Cape teen 8 ‘Varubie Mexnenet ten Say On, ™ ‘ower ‘Sia: Dew. Upper ‘ot Toul a soc" sweet Veriton aE a ons an 0240 vs wv ons 230 ir 2a Inreraction m= 5 oo Ger 0366 6 0303 267 rg ios oa anova ca own 200 om 184 aw nD om 001 m9 ‘reracrion e ° . ° Gee om ox Lon ns ” one 560 + nie average and ange metod te rt rton component came! be nied Table 9: Comparison of ANOVA and Average and Range Methods Pan No & Nae age Nae: Dat: (Chanctrsice Gage No Perfamed by ‘Specenton Gage Type svo.pev. etoTAL PERCENT VARIATION __ CONTRIBUTION Repan (E7) oa Tes ua Repredeilty 47) on m9 aa ‘Agpraserby Pan (47) ° ° ° Gee ox no 2 Par(Pr) 112 Ex m2 ‘Measurement System Is acceptable for Process Control and Anahyes, ote Tolennce = WA. ‘Total variation TH) 108s Numer of dnt da cateore (nds) = 4 ‘Table LB 10: GRR ANOVA Method Report ch Contdence init 19ESCA Syn sy Ones 130 @ 5 Coupe -Sectenc tint Mean Sys Say Section C Attribute Measurement Systems Study ‘Avebute meacurement aytems ae the cits of measurement systems where the measurement value is one of faite number of eategores. Tis it ‘contrasted to the Variables mesurement system which can result in & continuum of values. The most common of these is a go/no-go gage which ‘nas only two posible resus. Otberaftibue systems, for example visual sandards, may result in five to seven classifications, such as very good, ‘00d, fi, poor, very poo. The enalyses described inte preceding chapters ‘annot be used io evaluate such systems | , As discussed in Chapter I Seton B, there i «quantifiable rik when using | m7 #4 any measurement systems in making deisions, Stace the largest risk i the a category boundaries, the most appropiie analysis ‘would be the ‘quantification of the measurement sysce Variation witha gage performance are, (See Chater IV Section F) Risk Analysis Methods Jn some attibute situations itis not feasible o get suiient pats with variable reference vales. In such cases, the risks of making wrong of ‘nconsisten™ decisions canbe evalusted by using ‘Hypothesis Test Analyses + Signal Detection Theory Since these methods do not quantify the measurement system vai, they shouldbe used only withthe consent ofthe customer. Selection and se ‘of such techniques should be based on good statistical practices, an understanding of the potential sources of variation which can affect the product and measurement process, andthe effect ofan incorrect decision onthe remaining processes andthe fal customer. ‘The sources of variation of atibute systems should be minimized by wing the results of human fctrs and ergonomic research este tae ln pt production proces i in stitial coo perfomance Approaches: indices of Fp = Fpk = 0.5 which is unacceptable. Because the process is ‘producing nonconforming product,» coatalament ston is reid fo cull, theumaceepable parts fom te prodaction seam. Thisincads te comparison of mip appraisers ryrr ' hag Seton ‘Aas Meserent Syoma Sty Chaps Scion ‘Anse Means Sys Sy Lsl=45 USL=.55 ‘eared wih «vale gauge we oer means ach a exper 12 ‘determine which samples are good or defective, * = The oppressed, with ech appa maki Deer opr making re daca on te ‘An acceptable decision was designated with a one (1) and an unacceptab Soother cers Ont emer erhacta e etieeamen seen ; ie ait naw seat See i noe LE oe eee Figure II-C 1: Example Process with Pp = fpk =0.50 Fer the containment stv te proces tam seetd an atte gage tat Hypothesis Test Analyses - Cross-Tab Method | compares each part toa specific st flint and aceps the pat if he iis Since the team did not know the reference trv cals: bers jects the part aown 2 go-go gage) Mo eg am i nt ow te reac dct te prs ey ‘ees of this type are st upto accpt aad reject based on 8 St of maser paring cach ofthe appraisers tothe other ors Unlike a variable gage, hs atu gage cannot inate how god or ‘The crose-abulation process analyzes disuibution dat fortwo oe more stegrial variables, The results ~ presented i arate format fom s Tow bad'a par is, but ely tat the pat ih acepied or meted (ie. 2 Categories) As fol gages, hs bute age will bave “On awe where cotgeney ale hai laste th ierdependeney between vaable frrong decision ean be made (ace Figu IL-C 2 below and Chapter I, Section B). Figure IH-C2: The “Gray” Areas Associated withthe Measurement System Since this bas not yet been documented by the team, it needs to study the measurement system. However, to address the areas of risk around the fpeciieation limits, the team chose approximately 25% ofthe pats st oF lose tothe lower specication limit and 25% of the pans at or close to the ‘pper specification limit In some cases where it dificult to rake such Dart the team may decide to use «lower percentage recognizing that this fay increase the vaibility ofthe results. If not possible to make pars. | loge to the specification Unite the team should reconsider the use of fl Stebute gaging for thi process. As appmpriste fr each characteristic the parts should be independently measured wih variable gage wth accepiable — Variation (eg, a CMM). When measurng a tue attribute that cannot be Croton save in many statins analss software packages and is wit sd is used in predates pvt ‘abe fneons,——— rape Seon ¢ ‘inte Manes Sate Sy ‘Aeitse Metsat Sy Say 37 ‘The frst step i to summarize the observed data. Reviewing Table II-C 1, @ the team examined the dats for pairs of observers, counting when they agreed and when they disagreed for each set of evaluations, Tht is; for the valuations, there are 34 times where A-1 = 1 and B-1= I; there ae 32 tines where A-2 1 and B-2 «1; and there are 31 mes where A3 = Land B-3 = ‘ors ttal agreement of 97. The table below was constructed io summarice the data disteoution forthe observer pair AB, Similar ables were prepared for abserver pars °C and A*C 0566157 057030 SUBST gst sts ‘oaa7ai7 OS1S572 OHS 0555018 asa7ot Oase518 OSI7T 0531939 0519650 awe 0520055. Oar OAS, OS45500 O50 ‘Os iale2 0559581 050206. osz16e2 OSS 0561657 050) 2887617 0.49656 ‘a5H69E ‘Osa0r7 abe Oannrer ‘Oarr6H7 OS01137 Osi379. 066575 O63 OeTar B15) Basse O87 Sty ‘aat3803 rd 90 [400 | rota 1.00 3] or] 100 Toul a7] 103] 480 ‘The second sep is to estimate the expected data distibtion. What isthe ‘probability that an observer pair will agree or diagree on an observation ately by chance? In 150 cbserations Observer A rejeced the pan 30 tines snd Observer B rejected the pa 7 times: Po 4M13020313 Pas ™ 30/150 0.333 ‘Since the two observers ae independent, the probability that tey ages that the partis bed is piven bye £ 240 1.80) = pas pas = 0.104 ‘The expected number of times for Observer A und Observer B agre the pat {s bad i estimated by maliplyng the combined probably by ie mumbo of ‘observations 150 apie) 150 x (4/150) x (50150) = 15.7 Ae ‘The team made similar estimations ofeach category pair for cach observer ito complet the fllowing tables true Study Data Setager seen ‘Nett Meare Sens Sy ‘Are crosebuteton . sme | tow ao a ee |e Emewmcon | s7| saa | 00 Tae =} ar] womnccom| na | ear| woo [aa —— Seat 7] | 8 Eomanwcon | vo] ros0| 00 8c crotabon afm] tas [er fleet |] Cowon] wo] a0 ao i Tata sp] 8 Eonenscom| sso | eso | v0 i Tat Sa p=] ‘ Sommcom| sro| wal 100 creation oo aaa eee Smontcon| wel ase | 00 a0 ass =] a] — i Smmmecons| uo | seo | r008 Ta sty 1 Eommcon| sol veo] 100 “Table HU-C 2: Cross tabulation Srudy Rests ‘To determine the level ofthis agreement the tam uses the (Cohen) apo which measures the agreement Between te evaluations of two mters when both are rating the same abject, A valve ofl indicaes perfect agreement. A ‘al of O indicates that agreement is no beter than chance. Kapa is only “tvalable for ables in which boh variables use the same category values and ‘oth variables have the same numberof categories. Bhs we a amber of wate wich canbe uae 0 deennineiterrer agreement iret stisics ae ppp fr dfteen pes of mesusnen, See Reference List oclug: Bland, J. M. and A, D.G. (seh Cohen J (1960): Evert, B (1996 Fle, 1 (0971); Keppendor,K. 2004, Sal, FE, Downey, RG. nd ahey, MA (1980, Shroat, and Fle, (1979; and Utes, ohn S (1987) 136 —$—$—<—<< hae Scion Aiba Messen Syma Kapa ia mer of mere agreement te if te court in he slgeal cols (eat tn coe he tame a) fn ‘expected by chance alone. eee Let py the sum ofthe observed proportion in he diagonal els ‘Pe the sum ofthe expected proportion in he diagonal ells then appa is meas rather than tet. Is sie is judged by wing on ‘eympoi andr enor fo const a atic, Ager a hua that values of kopp petr tan 075 inte god occ agement (rita mci hppa: vases thn 040 nae poor speenens appa ako 20 acco he sza of cggouent beeen be | toe netabe bey epee se ™ Upon eaeuating the kappa measures forthe ‘wih ie following: prises, the team came up a c - 78 a6 29 78 z ‘Table HL-C3: Kappa Summary “This analysis indicates that cach oer. the appraisers show good agreement between This nls is ecessay oe ere ae ny irc ao 2ppnisen batt dosnt bow wel he uve ye sts nd pars fom bd. For ths mate tea had te prs alc weg = {Sie meee em od el heel Seem eee Wi on ian nt: goof shone ‘comparing each appraiser to the reference decision, se) ‘Asi al ech tego auton, age um on x leat, ae mamber pars coering the ei spectum fps ‘Wie he observations a mead onan icp sale ihe pp eo con be ued ober near speenen Agreement brute inf iba Srsmen ae By tember efenegoey nich ne sie Moret Seren as need “7- chr seenc mina Manna Sys By 3 apr Secie eer ems The process tam thn clea the effeinenes of th mesrwremen sem cme @® ae number of correct decisions oe Afficttveness = Topportniie for adeclsion [recom | wea | xo| 500 we 3 | eso] woo apa a Ra emerecon| 320) e997 _190 Source Azpriver A hopraver8_hopraserC | Appraiser A Appraiser Apraizr © ta ef We Tessiepeces [Sp = = = = = [Tat ‘Exgacnd Count se 192.0 et Matched 2 45 40. az 45 0 Fe Nepahe oe oe a a 3% False Positive (ssproker based towerd acorptanca) o | 1° ReFcroneuteton toed 7 smut 5] a] ae — | — ari] — ar | | to Catena Score [er a a | a 3 7 os 95% LC! Tal TEE ax TH 73% ca Tsco| 129 | m0) awe +e | 08 teoticom| sno| _roo}_son0 Sa TS Ba? TTT Eee Sea = 5 ih var Ewcom| | seo] 1000 ol npaced a = Lo Bin Agreement 3 2 son uct a ca rar cretion Ealetted Sere 7 7a Yaesu ca 3 Ta] sow | + a | me [eet cna oa} sa : (et age nha ts a on Ed - (ype ogres ona als wn th own standard Beecid Covel o8 {3)All appraisers agreed wahin and botwoen themsetves boat 102 130 {@)All appraisers epreed wchin & batween themaelves AND agreed wah the reforance Te = wel wo] (SoerSaUST an cber ane concn al San macy ancl Comparisons of Appraisers to Reference ‘Table HL 5: Stady Enectiveness Table Mabie tes of hypothesis beweca cach pir of apres can be o Lo Sndused wih he mal pets ‘Tre tou te ete decision: He: The effectiveness ofboth appraisers is the same ch appraise c ‘Since the calculated score of each appraiser falls within the confidence A B a interval of the other, the team conclodes that they eannot reject the nll 8 2 ns 5 tppobeies Tus enact coats Gon he fap menue ccdas ech of appraise as od apenct For fer asi, one ofthe tem meres trong ott flow —remaeaeamimmiooae | Suny mes mm at tt wit oe ihf: —- i opt -Secen ten ya “Anmibete Measurement Systems Study ae ) se es tone “pH ese Chapter, Seton Benker eb Deion Teves | Mintate | Faken | QD) sod (6 le Cy Cu or Py a) ten tal a a Rate ave many (or any) samples in this ‘Thi pace oi se The mena whe ‘capability improves , the required random sample for emey Acceptable for the appraiser 290% 3% 55% Seer eae “ manne) sate tate ample atove te des wee Py P= a expected prc ay ; perfomance of approximately 13% noc Ses se : sw | si toma of eponincly 38 aeomansy S eae Se i — ‘ene spoch og amples sed spe whe ve sic ay ten epee ge ae ensure tht the effet of appraise varity seen, Sawiete ‘Table LC 6: Example Effetiveness Cetra Guldelines Concerns 1) There are no terse deiton citron sce io tei on acceptable ik above guidelines hurite and cvelped bused oie | = needs improvement <10% 23% t Sse te orton yey ne a wi mm Seca ary at ct Neto ni cr ate cog fl ee i Estectiveness | Miss Rate | False Alerm [Process and final cusomer. This ia subject mater decision nan a i roe ag A 84% 63% 49% 2) The analysis above is datn dependent. For example, if AO aa cronies cect pe ren few piel al areata \ c 0% 125% 88% @ rmeasureonnt system, (the “gay” areas) of the r ‘Table HI-C 7; Study Effectiveness Summary ‘These resus showed thatthe measurement system had diferent levels of performance in Effectiveness, Miss Rateand False Alarm Rate depending on {he appraiser. No single appraiser tad accepable resus in all ree categories. No single appricer had unacceptable results in all tree categories. Do the acceptance guidelines need 10 be changed for this | process? Are these risks acceptable? Do the appraisers nced better aning? (Could the testing environment be improved? Most importantly what did the remcs: ‘customer think about the measurement system and these results ~ what were Figure IC 3: Example Process with Fp» Ppk = 1.33 thei expecttions? With this new situation, it would be concluded that ll the appnisers were aceptable since there would be ao detson Does te customer accept these ik levels? Sample Size ‘The question alvays arises: “How many samples should be used in the study?” To the dismay of most people the answer is “enough. The purpose | cof any measurement study (variables or attibule) is to understand the fl properties of the meesurement system. A sufficient number of samples shouldbe selected to cover the expected operating range (see alto Chapter It t Section C). With atibute measurement systems, the area of interes are the i ‘ ! 40 Mthap Seton ‘este Mecsas Say etc Se Sy fon resus seaneption of what he cose sbi 9 2) Tage ay pir Bes Tbe IC 4: @D signal oatction Approach . °F Crosstalton ‘An agate approach is to ie Signal Detection ; RetValve Code Ret Value Code | theon" to determine an approximation of the width of ete osgeset = 0803008 + | the region, lea and fom this, the measurement cw eat ae oserees = osoaess + | system GRR. This requires that cach of the sample 20m | 313% osreaso = osnzaes + | pars can be erahited. offline by variables sywmener | onan | 20m) a3 osrosso > 0801132 + | Measurement system. This reference vu Shown in Tao er | eon| onox | corm Kits sae | thecotum Ref Value in Table IC . ace + qa =>] Osetis7 x 0488005 + | Steps: Seooner | won | soo0% | woos assoete “x caaeres + dx || serene x oas7e1s + | 1. ecemine te Tolemnce (pet ange: om fe inspection iso fede be nonconforning oeeone x nae + Fae Since oP okt te wpe eft comer asa meas of he Opuiost ox: Oasaier + sts 330 ne pare Tis eran the probably of sosworr x oases ist as0 Urtecivns o of ates _« ‘rile ted escalate he GRR. the part bod abad part) gosta ase ess heme! Gsateee + asaio’ x" | Cul poses is improved to Fp = Prk = 1.00, he oszoms 5 oases x | compare sie sytem ore mig ow ee se? bad 9001 OASEST x «| © fp, compure ransom’ ym eras, probaly te OsTTarMS04aoee bad) Osissra Oaarear |_| tembehnerinmest eric the pose orthe whence “To ein Gsoeo%s + 0412459 | isis example he P= 5 ee Figure IC 1.00 te y 05238 - | proces pnb lence td he marr ‘ute sbeld therefore be compared to tlernce ‘Table INC 8: Table HL-C 1 sorted by Ret Value For the data in Table I-C 1 pr(clled bad| bad)*Pr(bad) (ba called bad) = =a) (bad) + Pr clled bad 200d) _swionm) } 2) Rank oder the data (and adjacent els) Gen highs lowest based t Pr(bad| called bod) = Se oa + 20°(9973) ‘on the individual Reference Vals (see Table II-C 8; Note: this table ‘is been divided into two columns to save space). \ tedted) = 3) yh epinng nde pont of to ae ts Table Px{bod| aed bad) erste te par ical bad ere oa IL 8 tis shown by the coke Coe icp itis truly bed. ah ‘aL oxtot ocksnee hat ly | + =ccepted with total agreement. evar dt ination even ! 0 A mn eee ececn Tenge ierwerteie’ x = dsapeement ‘The wih of these zone is what we are tying to determine, and the avenge width of these zones will be used fo compare the See Reference Lis Baker 1975Zone t= pase frotame ne Zone t= paso saa Zone = queso ‘without 00% Trondag ac ioe Tim measurement system tthe specification tolerance, 10 the proses {sigma width (depending onthe stuaon). at Reterng to the above graphic, let den. = dance between the lt at accepted by all appraisers toe ft par rejected by all ” 4) Lat diz = distance beesn the lst pr sceied by > ‘appraisers in Zone III to the first ‘part rejected by all appraisers zee Lexa dtace awen he at pa see yal apie ef 1a Seda rath spss) ) ‘Analytic Method apt Secon rie Meares Sy sy {AS with any measurement system, the stability of the process shoald be re net srouding the USL bo, Stan: eninge amon SS Cevifed and if ascesay, monte. Fr stibte meanrement ste, beamed yep ee 8 A rue contol ang a cota ample oer tines common ay of Nowe the aes ov ‘ering ai For an atibute measurement system, the concept ofthe Gage Performance ‘Carve (ice Chapter 1V, Section F) is used for developing a measurement system dy, whic i sed to ates the amount of repeatability and bias of the measurement syste. This analysis canbe used on both single and double limit measurement systems. For a double limit measurement sytem, only ‘one limi need be examined withthe esumpion of linearity and uniftrmity ‘of ero, For convenience, the lower Kimi wil be used for discussion. In general the atibute measurement system study consists of cbaining the referece values for sever selected pars. These parts are evaluated = ‘numberof Ges, (m), with the total numberof accepts (2, foreach pat being recorded. From the resus, repeatability and bas can be assessed. ‘The frat stage ofthe stribute study is the pat selection. I is essential that the reference value be known for each part wed in the study. Eight pars should be telecied at as neatly equidistant intervals as practical. The = eclipse _— Snore rena TE op teetoe Seeeeee a we womens ® shomavrsmenees eee 7H oa tn this example (p-126) where the tolerance is 0.10, ise = OATOSS2 ~ 0.446697 = 0824135 diay = 0.566152 ~ 0.542708 = 0923448 For the total study, the smallest part must have the value a 0; the acest pt, a = 20; and the six other pars, Isa 519. If these erteia are not fated, more pers with known feference values, (4), mst be run tough the gage tnt the above condition ae met If, or he smallest value +0, ‘hen staller and smaller pars ar taken and evaluated uati a= 0.1, forthe largest valu a» 20, then larger and larger pasar en until a= 20. Ifsix d= 00237915, | ned cep nH i ofthe pats donot have 1 << 19, additional pas canbe taken at selected orthe estimated 4G RR i, pins throughout the range. These points ar kena the midpoits ofthe GRR = 1% | prt measurements already measured inthe dy. The fist imterval at the @ = rement sytem with an seal Siac tis example was pected by a measurement tem wih nett SORA = 25%, this estimate will lead to Ieasuremen stem cans bewed nl ore ata formation able hs tec Msc omg oan = (end starts fom the largest measurement where a= 0. For the a = 20 end, ‘he fit interval starsat the smallest measrement where = 20. For best results, samples should be taken at both the a = O and a ~ 20 ends and ‘worked toward the mide ofthe prt range. Ifncessr, the procedure ean be repeated unt the criteria ae met ‘Once the deta collection eiteria have been stisted, the probabilities of ‘cceplance must be calculated foreach prt sing the fllowing equations: RSG Tia neste ne ig op 4 Lag! ne emer the estimate Retericos, a0 yS>0s a2 y2n0s ‘he adjsnens cover he isaces where 1 s.@ 1, For the insaness Thece Oat 2 = eco for he largest reference vale with = 0,2 ii which PY = 0.025, For the instances where a= 20 hen Pf = 1 except for the Hr ! nals efeence vale wit a= 20a which R= 0975. a ent tena epee ore ii 7PC) can be develc ‘Although the GPC can be presented graphically Mm (cr cn dev sion eC gee 5 eae He ay See eins ere acer ee " line of best fit is drawn through these points. The bins is equal to the lower afte i re Pecan Zt bias Lower Specification Limit Xr (at F; = 0) ‘The epi i termine by nding te diteenecs of the refence Wie oauremttscomeponding Fz" 0999 and 0.008 and dig ‘by an adjustment factor of 1.084. HX, (atP, = 0995) ~_X, (at, = 0005) 108 To deemine fhe Was is ighifenty dffest Gon 20, the lowing dite ned : 6.078 _x_|Bias| it nmr 1 is cateated vale is greater than 2093 (lann, ten the bins is sigan ifere fom 20, “An exanple wl clarify the ait dy data collection and cacuaton of , fepeabiiy sod bis. The 08 ese (oinig) fcr specie fora sae size of 2 ad wat Te Jon fr pon wh 0 99% cepenabiy range. To conve the rei 6 sim SiS andmaepiy iy was determined trough & range, divide bY 146 ope Seta ¢ ‘Aste Meee Sy Exampl Anti gas being do mee s ison at as wena of 0010. The gage san eon 100% soma npecon gage as ‘Mle by ree and is, To pero esti say aot prs with reference valoes eines of 0002 fom -0016 to 0002 ee rn ‘rough te gage 20 tne exch, The amber feces for ach prt se x “oor Sous cami “a0 “as cone coon on BEBEBaus Since there are wo reference values with 1 3 samen Table1V-A 1: Methods Based on Type of Measurement System. St: Single Part*, Single Measurement Per Cycle Application: ¢ 8) Measurement sysems in which the partis not changed by the Ieasurememt procest; ie, measurement systems tha ar” bom 2 Instruments ‘This allows the comparison of multiple instruments, hl | Applicaton: 5) Measurement systems in which the port is aot changed by the measurement process; i, measurement systems hat are non 3 Instruments ‘+ Repeated readings ac taken over specified time intervals. ‘Analyze sume a8 V2 (se also Lipson & Sheth, ec 13.2). ‘The shelf life ofthe characteris (property) is known and extends beyoed the expected duration of tho stuly; ie, tbe measured characteristic does not change ove the expected period of ws. + Pars (pecimens) cover the expected range ofthe process variation ofthe characterise, ‘Analyze by determining the degradation model foreach sample part: { + Consistency of degradation (i'm 22) Vi: Linear Analysis, ; Assumptions Repeated readings are taken ove specified time intervals. I ‘© The degradation inthe measurerent system is known (documented) | te havea linear response over the specified time intervals a) 1 The shelf life ofthe characte (property) is known end extnds beyond the expected duration of the sy; Le, the measured characteristic des nt change over the expected period of we ‘Pars (specimens) cover the expected range of the process variation | ofthe character, ‘Analyze by linear regresslon: I + Consistency of degradation (ifm 2 2) | i V7a—V7 with a Homogencous Sample Analyze by lioear regression: This tan V8: Time versus Characteristic (Property) Degradation i. V6 & V7 cn be modi to etemie i te deri time (tll ’ bound study: Og S ite) or activity dependent,rapt «acon using Ee of cnn Wain an Vann, Section D Recognizing the Effect of Excessive Within-Part Variation ‘Understanding the souresof variation ofa measurement system i important for al measurement applications bit becomes even more eral where there is significant within-partvarition. Within-par variation, such as taper of ‘utotround, can eause the mesarement system evaluation 10 provide misleading ress, Tiss because unaccounted within-pan variation affects the estimate of repeatabily, reproducibly, or bth. Thi, the within-pat variation can appear asa significant component of the measurement system ‘aration. Understanding the within-part variation present nthe product wil result in amore meaningful understanding of the suitably of the ‘measurement system for the ask hand. Examples of within-pactvarstion which may be encountered are: roundness (Girl runout, conceniiy, ape, fess, profile ylindiciy, et.” It 's possible that more than one of these characteristics may be presenta he same time within the same pet (composite emer). The srengh of each characteristic and thir itedependeneies may compound the data and the resultant understanding of the measuremeat system. Recognition and tanalysis of these addtional soures of variation during # measurement system sudyis paramount o understanding the actual measurement system variation. “A D.O.E, ANOVA. or otber more sophisicaed satiical technique may be required to analyze this situation. Whatever mettodology Is chosea shouldbe approved bythe customer supplier quality sepresenativ. ‘Also, decisions that product design engineers make can unwitingly affet hhow a partis measured, how a fixture may be designed, andthe result can alec the outcome of measurement ror. An example might bes plastic part that has a erica fea on the paring line (a paring line gypicaly bat ‘excess plastic material where the two halves ofthe mold join, ond tus is 4, ‘uncontolled surface), These fctors ought to be considered during Design FMEA. 2 Many of tee fener contol in the eign by way of Geomere Dinersonng and Tlerancing (@DRT). GDAT provides un operaonaly defined method wo check par ina fonconl tana. General, 2 functional check an atibute check, Whee varie data requed, ses can ase so ings page eign fora faneina chek to il arbi data. Ths ay suntan be done y singe fnctonl ppt es hla, ‘xe fra CMM stty. However, ven tas don tie atthe are ote pr fy tad ‘epenably inthe sae lean (i doesn the estan MSA stay Should genet hi).——— Cape -sadt 4 vn tne a Me SS me want vn — section € ete ing aii npn nest oy ction Loser i cae actoacwrem) sot exes — @/3) Average and Range Method — Additional Treatment (ent confounded. ‘There is an additional consideration worthy of mention relative to the Average and Range method of measurement system assessment. ”* ‘The primary purpose ofthis graphical approch i the same as other well designed measurement system analyses: to determine ifthe Measurement process is adequate to measure the manufacturing procest variation andor vauat produet conformance Ate all gages doing the same job? Areal appraisers doing the same ob? + 1s the measurement system variation sceptble with respect othe process variation? ‘+ How good are the data obtained from the messuement process or inte how many non-overlapping groups o:clegoris can the data be divigea 1) Cae should be taken to follow the “Prepution for Measurement Procedural Steps Systems Stay", Chapter, Serion C 2) Have each appraiser check each sample forthe chanctrstie being ‘Ride. Record the fia cheeks onthe tap data row of conto cha (GeeFiguetver 13). 3) Repet the checks and record the data onthe second dat row of the contol chat Vote: Do not allow the appar to see ther original ‘reading while making ths second check} The dts Should now tow "wo checks on he se prt by each spraser. Analyze the data by calculating the average (2) and range (R) for ach subgroup. 5) Plot the range values on the ange char and clue the average I ‘ange () Gace al sub-group ranges (8) oral appraiser). Draw : this verge range onthe chart Ure the Dy lice forn = 2 to least the contol it forthe ange chart, Draw i hs Kimi and . etme fall values arin coool. Yall anges are in contol, all appraisers are doing the se obi one appraise is out of contr his metbou differs foe the otters, I If all oppraisees have some out-of-control ranges the measurement system is sensitive to appraiser techn and needs improverneat wo obtain useful data “Te contol cat exape is then with persion fom "Evang the Mesrement recs” by Wiel & Lyiy ee Reference i).