Measurement Systems Analysis

Measure

Analyze

Improve

• Champion Training Control

Six Sigma Champion Training 1

 Learning Objectives

• Understand the language of Measurement

• Show the importance of Measurement
• Walk away knowing how to perform a Gage
R&R and how to interpret results
• Share some lessons learned

Six Sigma Champion Training 2

Since Measurement systems represent
a sub-process within a process...

• They are subject to Variation.

• What could be the source of this variation?

• Why do Measurements Vary?

CLASS EXERCISE : Break into teams. Do a

cause-effect Diagram to determine the causes
of variation in Measurement.

Six Sigma Champion Training 3

Sources of Measurement Variation

Measurement System C&E Matrix

Measurements Materials Men

Cleanliness Procedure
Mechanical Integrety Fatigue
Temperature
Attention
Dimension
Wear Calibration Error
Weight Interpretation
Corrosion Speed
Electrical Instability
Coordination
Hardness
Know ledge
Conductivity Dexterity
Algorithm Instability
Density Vision
Measurement System Error

Temp Fluxctuation Operator Technique Wear

Standard Procedure Stability

Line Voltage Variation Resolution
Sufficient Work time
Calibration
Vibration Maintenance Standard
Precision
Calibration Frequency
Cleanliness Design
Operator Training Temperature
Humidity Ease of use Cleanliness
Environment Methods Machines
Six Sigma Champion Training 4
Possible Sources of Process Variation
Observed Process Variation

Actual Process Variation Measurement Variation

Long-term Short-term Variation Variation due Variation due

Process Variation Process Variation w/i sample to gage to operators

Repeatability Accuracy Stability Linearity Reproducibility

We will look at ―repeatability‖ and ―reproducibility‖ as

these are the primary contributors to measurement error.

Six Sigma Champion Training 5

 Knowledge to be obtained

• How big is the measurement error?

• What are the sources of measurement
error?
• Is the gage stable over time?
• Is the gage capable for this process?
• How do we improve the measurement
system?
Six Sigma Champion Training 6
 Sources of Variation
Product Variability
(Actual variability)

Measurement
Variability

Total Variability
(Observed variability)

Six Sigma Champion Training 7

Effects of Measurement Error
Measurement
System Bias —
Averages Determined through
―Accuracy Study‖

 total   product   measurement

Measurement System
Variability Variability —
Determined through
―R&R Study‖

 2total   2product   2measurement

Six Sigma Champion Training 8

 Work Around Gage Error
If you want to decrease your gage error take advantage
of the standard error square root of the sample:

x
x  n = sample size
n
Example: Gage error of 50% can be cut in half if your
point estimate is a sample of 4 data points.

THIS IS USED AS A SHORT TERM APPROACH TO PERFORM A

STUDY, BUT YOU MUST FIX THE GAGE.

Six Sigma Champion Training 9

 Terminology

• Location related terms:

– True value
– Bias
– Linearity
• Stability (over time)
• Variation related terms
– Repeatability
– Reproducibility
– Linearity
Six Sigma Champion Training 10
• True value:
– Theoretically correct value – unknown and unknowable
– Reference standards
– NIST standards
• Bias
– Distance between average value of all measurements
and true value
– Amount gage is consistently off target
– Systematic error or offset
Six Sigma Champion Training 11
 BIAS Definition

BIAS — Is the difference between the

observed average of the measurement
and the reference value. The reference- Reference
value is the value that serves as an Value
agreed-upon reference. The reference
value can be determined by averaging
several measurements with a higher level
(e.g., metrology lab) of measuring
equipment.

Warning: Don’t assume

your metrology
reference is gospel. Observed
Average Value
ACCURACY IS THE SAME AS BIAS
Six Sigma Champion Training 12
 Linearity
Difference in the accuracy values of a gage
through the expected operating range of the gage

Good Linearity Bad Linearity

Regression Plot Linearity is Not Good

55 55

45 45

35 35
Trials

Trials
Y = 0.934227 + 0.994959X Y = 0.245295 + 0.99505X
25 25
R-Squared = 0.981 R-Squared = 0.982

15 15

5 5

10 20 30 40 50 10 20 30 40 50

Standard Standard

Six Sigma Champion Training 13

 Stability

• The distribution of measurements remains

constant and predictable over time for both mean
and standard deviation
• Total variation in the measurements obtained with
a gage, on the same master or master parts,
when measuring a single characteristic over an
extended time period.
• Evaluated using a trend chart or multiple
measurement analysis studies over time
Six Sigma Champion Training 14
 Stability (drift) Definition

Time-2
Stability — Is the total variation in
the measurement obtained with a
measurement system (test / gage ) on
the same master parts when measuring
a single characteristic over an
extended time period.
Magnitude
Time-1

time
Points to the frequency of Mean center Calibration Stability

Six Sigma Champion Training 15

• Total variation in the measurement system
• Measure of natural variation of repeated
measurements
• Terms: Random Error, Spread, Test/Retest
error
• Repeatability and Reproducibility

 2
MS    2
G
2
O

Six Sigma Champion Training 16

 Repeatability
G
• The inherent variability of the measurement system
• Variation in measurements obtained with a gage when
used several times by one operator while measuring a
characteristic on one part.
• Estimated by the pooled standard deviation of the
distribution of repeated measurements R
G 
d 2*
• Repeatability is less than the total variation of the
measurement system

Six Sigma Champion Training 17

 Repeatability Definition

Repeatability — The variation in measurements

obtained with one measurement instrument
when used several times by one appraiser while
measuring the identical characteristic on
same part.

REPEATABILITY

Six Sigma Champion Training 18

 Reproducibility
O
• Operator variability of the measurement system
• Variation in the average of the measurements
made by different operators using the same gage
when measuring a characteristic on one part
• Must be adjusted for gage variation
R

• Reproducibility is less than
O  the
* total variation of
d2
the measurement system

Six Sigma Champion Training 19

 Reproducibility Definition
Operator-B
Reproducibility — Is the
variation in the average of the
measurements made by different
appraisers using the same measuring
instrument when measuring the
identical characteristic on the same Operator-C
part.

Operator-A

Reproducibility
Six Sigma Champion Training 20
 The Nature of Process Variation

1 2 3 4 5

Precise but not Accurate

1 2 3 4

Accurate but not Precise

Rule of thumb:
. . . . . .Test equipment MUST be a least 10 times
more accurate & precise then what’s being tested
Six Sigma Champion Training 21
Measurement System Discrimination
Least count should be at most one-tenth of the total
process capability or tolerance (6 sigma)
– Process capability 10 Max Least count 1
Part to Part variation must be greater than the smallest
unit of measure
Range control chart provides best indication of inadequate
discrimination
– Occurs when only 1,2, or 3 possible values for the
range within the control limits exists
Number of Distinct Categories equals part sigma/ total
gage sigma  1.41.

Six Sigma Champion Training 22

 Measurement System Capability
 Gage Capability

. * MS
515 Usually expressed
GR& R  as percent
Tolerance
 Addresses what percent of the tolerance or process
capability is taken up by measurement error.
 Best case: 10% Acceptable: 30%
 Includes both repeatability and reproducibility
– Operator  Unit  Trial Experiment

Six Sigma Champion Training 23

 Measurement Error
Effect on Capability Index Cp

USL  LSL
• We know that Cp 
Act where  Act   Obs   MS
2 2

6Act

• Therefore: USL  LSL

CpAct 
6  2Obs   2MS

Six Sigma Champion Training 24

 R&R Effect on Capability
70% 60% 50%

6.0

5.0 %R&R

4.0 0%
Actual Cp

40%
10%
3.0 20%
30%
30%
2.0
40%
10% 50%
1.0
60%
0.0 70%
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0

Observed C p

Six Sigma Champion Training 25

 Types of R&R Studies

• Variable Gage R&R

– Numbers
– Units of measure
• Attribute Gage R&R
– Subjective (cosmetic defects)
– Scatter of defects
– feel/visual

Six Sigma Champion Training 26

 The Inspection Exercise
Task: Count the number of times the 6th letter of the alphabet
appears in the following text.

The Necessity of Training Farm Hands for First

Class Farms in the Fatherly Handling of Farm Live
Stock is Foremost in the Eyes of Farm Owners.
Since the Forefathers of the Farm Owners Trained
the Farm Hands for First Class Farms in the
Fatherly Handling of Farm Live Stock, the Farm
Owners Feel they should carry on with the Family
Tradition of Training Farm Hands of First Class
Farmers in the Fatherly Handling of Farm Live
Stock Because they Believe it is the Basis of Good
Fundamental Farm Management.
Six Sigma Champion Training 27
 Convert Data

Try To Always Convert Attribute To Variables

 End Disk Height

Examples:  Likert Scale
 Leak Rate (go/no go)
 Mass Spec

Six Sigma Champion Training 28

 Basic Terms

EV= Equipment Variation (Repeatability)

AV= Appraiser Variation (Reproducibility)

R&R= Repeatability & Reproducibility

PV= Part Variation

TV= Total Variation of R&R and PV

K1-Trial, K2-Operator, & K3-Part Constants

Six Sigma Champion Training 29

 Gage R&R study

 Generally two or three operators

 Generally 10 units to measure

 Each unit is measured 2-3 times by each

operator

Six Sigma Champion Training 30

 Preparation for a Measurement
Study
• Determine if reproducibility is an issue. If it is, select the
number of operators to participate.
• Operators selected should normally use the measurement
system.
• Select samples that represent the entire operating range.
• Gage must have graduations that allow at least one-tenth
of the expected process variation.
• Insure defined gaging procedures are followed.
• Measurements should be made in random order.
• Study must be observed by someone who recognizes the
importance of conducting a reliable study.
Six Sigma Champion Training 31
 Procedure for Performing R&R
Study
• Calibrate the gage, or assure that it has been calibrated.
• Have the first operator measure all the samples once in random
order.
• Have the second operator measure all the samples once in
random order.
• Continue until all operators have measured the samples once
(this is Trial 1).
• Repeat above steps for the required number of trials.
• Use GR&R form to determine the statistics of the study.
– Repeatability, Reproducibility & %GR&R
– Standard deviations of each of the above
– % Tolerance analysis
• Analyze results and determine action, if any.
Six Sigma Champion Training 32
 Variable Gage R&R

Guidelines
% R&R Results
<5% No issues
 10% Gage is OK
10% – 30% Maybe acceptable based upon importance
of application, and cost factor
Over 30% Gage system needs improvement/corrective
action

Six Sigma Champion Training 33

 Gasket Thickness Study
PT1 PT2 PT3 PT4 PT5 PT6 PT7 PT8 PT9 PT10 OP/TRIAL
0.65 1.00 0.85 0.85 0.55 1.00 0.95 0.85 1.00 0.60 A1
0.60 1.00 0.80 0.95 0.45 1.00 0.95 0.80 1.00 0.70 A2
0.55 1.05 0.80 0.80 0.40 1.00 0.95 0.75 1.00 0.55 B1
0.55 0.95 0.75 0.75 0.40 1.05 0.90 0.70 0.95 0.50 B2
0.50 1.05 0.80 0.80 0.45 1.00 0.95 0.80 1.05 0.85 C1
0.55 1.00 0.80 0.80 0.50 1.05 0.95 0.80 1.05 0.80 C2
Xbar & R Minitab Example
Using Aiag49:mtw Data File
Specification: 0.6 - 1.0 mm
Process Variation: 1.6 mm
Reference QS Measurement System Analysis Manual
Six Sigma Champion Training 34
 Gage name:
Date of study :
Gage R&R (Xbar/R) for Thickness Reported by :
Tolerance:
Misc:

Xbar Chart by Operator Operator*Gasket Interaction

1.1 1 2 3 1.1 Operator
1.0 1.0 1
Sample Mean

0.9 3.0SL=0.8796 0.9 2

Average
0.8 X=0.8075 0.8
-3.0SL=0.7354 3
0.7 0.7
0.6
0.6
0.5
0.4 0.5
0.3 0.4
Gasket 1 2 3 4 5 6 7 8 9 10

R Chart by Operator Response by Operator

0.15 1 2 3 1.1
3.0SL=0.1252 1.0
Sample Range

0.10 0.9
0.8
0.7
0.05
R=0.03833 0.6
0.5
0.00 -3.0SL=0.000 0.4
Operator 1 2 3

Components of Variation Response by Gasket

1.1
200 %Total Var 1.0
%Study Var 0.9
Percent

%Process 0.8
%Toler 0.7
100
0.6
0.5
0 0.4
Gage R&R Repeat Reprod Part-to-Part Gasket 1 2 3 4 5 6 7 8 9 10

Six Sigma Champion Training 35

Gage R&R Study for Thickness – XBar/R Method
Source Variance StdDev 5.15*Sigma

Total Gage R&R 2.08E-03 0.045650 0.235099

Repeatability 1.15E-03 0.033983 0.175015
Reproducibility 9.29E-04 0.030481 0.156975
Part-to-Part 3.08E-02 0.175577 0.904219
Total Variation 3.29E-02 0.181414 0.934282

Source %Contribution %Study Var %Tol %Process

Total Gage R&R 6.332 25.164 58.77 14.69

Repeatability 3.509 18.733 43.75 10.94
Reproducibility 2.823 16.802 39.24 9.81
Part-to-Part 93.668 96.782 226.05 56.51
Total Variation 100.000 100.000 233.57 58.39

Number of distinct categories = 5

Six Sigma Champion Training 36

 Calculation Explanation

 5.15 Sigma = 5.15  the factor standard deviation. 5.15 was

developed empirically to approximate the gage population
distribution variation.
 % Contribution = Percent contribution of each factor based
upon the variance. Repeatability = 100  repeatability variance/
total variation variance.
 % Study Variation = 5.15  the factor standard deviation
divided by 5.15  the total variation standard deviation.
Repeatability = 100  5.15 repeatability standard deviation/ 5.15
total variation standard deviation.
 % Tolerance = 5.15  the factor standard deviation divided by
the tolerance. Repeatability = 100  5.15 repeatability standard
deviation/tolerance.

Six Sigma Champion Training 37

 Calculation Explanation

% Process Variation = 5.15 x the factor standard

deviation divided by the process variation. Repeatability =
100 x 5.15 repeatability standard deviation/ process
variation.

Number of Distinct Categories = part standard deviation

divided by the total gage R&R standard deviation times
1.41.

Six Sigma Champion Training 38

 Attribute Data Example
Using A_grr.xls file

• Metric is the % error against known population deemed

good by local experts
• Attribute legend can be the defect codes
• If appraiser % is less than 100% training is required, focus
on area of weakness
• 100% is the target for screen effectiveness
• Use this to prove measurement system capability prior to
task assignment
• Select the 5 vital few (80-20 rule) to conduct GR&R

Six Sigma Champion Training 39

 Attribute Gage R & R
Effectiveness
Instructions:
1) The following spreadsheet is used to calculate an Attribute GR&R Effectiveness, in which up to
30 samples can be evaluated, using 2 or 3 operators.
2) In the worksheet fill in the appropriate information in the Scoring Report section and
enter the type of Attributes you are evaluating in the Attribute Legend section. YOU MUST ENTER
THE INFORMATION IN THE ATTRIBUTE LEGEND SECTION OR THE SPREADSHEET
WILL NOT WORK . The attributes can be either alpha or numeric, e.g. Yes, No; pass, fail;
go, stop; or 1, 2. You must be consistent throughout the form and spell properly, anything
will work as the spreadsheet compares what is in each cell.
3) If you or an expert has selected samples to be evaluated and you know what attributes these
samples are, enter this information in the Attribute sample column. This will enable you to determine
how well each operator can evaluate a set of samples against a known standard. You do not
need to enter information in this column for the spreadsheet to work.
4) You do not have to specify how many operators or the # of samples that you will be evaluating
during the test. Simply enter the data into the spreadsheet under the specific operator. Remember
the attributes must be spelled properly or the spreadsheet will not analyze the data correctly.
5) To print a copy of the report click on the Print Report icon.
6) To delete the data in the spreadsheet, click on the Delete Data icon.
7) To delete all and begin a new test, click on the Delete All icon
8) To see a Demo of the Attribute GR&R Effectiveness spreadsheet, click on the Demo icon.
Move around the spread sheet to see the data. When you are finished click the Delete All icon
to delete all data to begin entering your own data.
Six Sigma Champion Training 40
Attribute Gage R & R Effectiveness
SCORING REPORT
Attribute Legend DATE:
1 pass NAME: 3/10/96
2 fail PRODUCT: Allied Employee
SBU: 3313 Spark Plug
TEST CONDITIONS: F&SP

Known Population Operator #1 Operator #2 Operator #3 Y/N Y/N

Sample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
% APPRAISER SCORE(1) -> #DIV/0! #DIV/0! 0%
% SCORE VS. ATTRIBUTE(2) -> Known Known Known

SCREEN % EFFECTIVE SCORE(3) -> #DIV/0!

SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE (4) -> #DIV/0!

Six Sigma Champion Training 41

Attribute Gage R & R Effectiveness
SCORING REPORT
Attribute Legend DATE: 3/10/96
1 pass NAME: Allied Employee
2 fail PRODUCT: 3313 Spark Plug
SBU: 3313 Spark Plug
TEST CONDITIONS: F&SP

Known Population Operator #1 Operator #2 Operator #3 Y/N Y/N

Sample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree
1 pass pass pass pass pass fail fail N N
2 pass pass pass pass pass fail fail N N
3 fail fail fail fail pass fail fail N N
4 fail fail fail fail fail fail fail Y Y
5 fail fail fail pass fail fail fail N N
6 pass pass pass pass pass pass pass Y Y
7 pass fail fail fail fail fail fail Y N
8 pass pass pass pass pass pass pass Y Y
9 fail pass pass pass pass pass pass Y N
10 fail pass pass fail fail fail fail N N
11 pass pass pass pass pass pass pass Y Y
12 pass pass pass pass pass pass pass Y Y
13 fail fail fail fail fail fail fail Y Y
14 fail fail fail pass fail fail fail N N
15 - - - - - - -
16 - - - - - - -
17 - - - - - - -
18 - - - - - - -
19 - - - - - - -
20 - - - - - - -
21 - - - - - - -
22 - - - - - - -
23 - - - - - - -
24 - - - - - - -
25 - - - - - - -
26 - - - - - - -
27 - - - - - - -
28 - - - - - - -
29 - - - - - - -
30 - - - - - - -

% APPRAISER SCORE(1) -> 100.00% 78.57% 100.00%

% SCORE VS. ATTRIBUTE(2) -> 78.57% 64.29% 71.43%

SCREEN % EFFECTIVE SCORE(3) -> 57.14%

SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE(4) -> 42.86%

Six Sigma Champion Training 42

 Calculation
Known Population Operator #1 Operator #2 Operator #3 Y/N Y/N
Sample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2 Agree Agree
1 1 1 1 1 1 2 2 FALSE FALSE
2 1 1 1 1 1 2 2 FALSE FALSE
3 2 2 2 2 1 2 2 FALSE FALSE
4 2 2 2 2 2 2 2 TRUE TRUE
5 2 2 2 1 2 2 2 FALSE FALSE
6 1 1 1 1 1 1 1 TRUE TRUE
7 1 2 2 2 2 2 2 TRUE FALSE
8 1 1 1 1 1 1 1 TRUE TRUE
9 2 1 1 1 1 1 1 TRUE FALSE
10 2 1 1 2 2 2 2 FALSE FALSE
11 1 2 1 1 1 1 1 FALSE FALSE
12 1 2 1 1 1 1 1 FALSE FALSE
13 2 2 2 2 2 2 2 TRUE TRUE
14 2 2 2 1 2 2 2 FALSE FALSE
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
% Appraiser Score 85.71% 78.57% 100.00% 42.9% 28.6%
Agreements 12 11 14 6 4
Total Sample 14 14 14 14 14

Six Sigma Champion Training 43

 Attribute Gage R&R Workshop

• Perform attribute Gage R&R study using M&M's.

– Determine the defects by looking for consistent color in
the M&M's, clear markings (M's), and roundness.
– Use 3 operators/inspectors
• Complete attribute GR&R analysis and report results (30
minutes).
• Improve inspection criteria, rerun attribute Gage R&R
study/analysis and report results (30 minutes).

Six Sigma Champion Training 44

 Do You Understand?

• The language of Measurement ?

• The importance of Measurement?
• How to perform a Gage R&R Study and
how to interpret results ?
• Use Minitab to analyze GR&R results?

ANY QUESTIONS OR COMMENTS

Six Sigma Champion Training 45