QUALITY TOOLS &

TECHNIQUES

Q T T
MEASUREMENT SYSTEM
ANALYSIS (MSA)
By: HakeemUrRehman
Certified Six Sigma Black Belt (SQII Singapore)
IRCA (UK) Lead Auditor ISO 9001
MSTotal Quality Management (P.U.)
MSc (Information & Operations Management) (P.U.)
IQTMPU

INTRODUCTION TO
MEASUREMENT SYSTEM ANALYSIS
Also known as Measurement System Evaluation (MSE)

Anytime you measure the results of a process you will observe some variation.
This variation comes from two sources:
Parts made by any process
Method of making measurements
Thus, measuring the same part repeatedly does not result in identical
measurement.
So far we have learned that the heart and soul of SixSigma is that it is a
datadriven methodology.
How do you know that the data you have used is accurate and precise?
How do you know if a measurement is a repeatable and reproducible?

How good are these?

MEASUREMENT SYSTEM
ANALYSIS: Definition
A measurement system may be defined as the
collection of instruments or gages, standards,
operations, methods, fixtures, software, personnel,
environment and assumptions used to quantify a unit
of measure or fix assessment to the feature
characteristic being measured; the complete process
used to obtain the measurement.
(Automotive Industry Action Group AIAG 2002 Standard)
3

MEASUREMENT SYSTEM ANALYSIS

Whenever you measure anything, the variation that you observe can
be segmented into the following components

Observed Variation
Measurement System Error

Unit-to-unit (true) Variation

Precision

Repeatability

Reproducibility

Accuracy

Stability

Bias

Linearity

All measurement systems have error. If you dont know how much of the
variation you observe is contributed by your measurement system, you cannot
make confident decisions.

ACCURACY Vs PERCISION

Two categories of measurement error.

ACCURACY refers to how close measurements are to the
"true" value,
while PRECISION refers to how close measurements are
to each other.
5

PERCISION METRICS
A precise metric is one that returns the same value of a
given attribute every time an estimate is made.
Precise data are independent of who estimates them or
when the estimate is made.
Precision can be partitioned into two components:
Repeatability
Reproducibility

Repeatability and Reproducibility = Gage R+R

6

PERCISION METRICS
Repeatability is the variation in measurements obtained
with one measurement instrument used several times by
one appraiser while measuring the identical characteristic on
the same part.
Y

Repeatability

For example:
Manufacturing: One person measures the purity of multiple samples of the
same vial and gets different purity measures.
Transactional: One person evaluates a contract multiple times (over a
period of time) and makes different determinations of errors.
7

PERCISION METRICS
Reproducibility is the variation in the average of the
measurements made by different appraisers using the same
measuring instrument when measuring the identical
characteristic on the same part.
Reproducibility
Y

Operator A
Operator B

For example:
Manufacturing: Different people perform purity test on samples from the
same vial and get different results.
Transactional: Different people evaluate the same contract and make
different determinations.
8

ACCURACY METRICS
BIAS = Observed average value Reference (True) value
Bias, is the difference between the true value (reference value) and the observed average
of measurements on the same characteristic on the same part. (AIAG, 2002)
It answers the question: "How accurate is my gage when compared to a
reference value?"
Low

Nominal

High

+e

B i a s (y)

LINEARITY:
Linearity is an indication that gauge response
increases in equal increments to equal increments of
stimulus, or, if the gauge is biased, that the bias
remains constant throughout the course of the
measurement process.
Linearity examines how accurate your measurements
are through the expected range of the
measurements. It answers the question: "Does my
gage have the same accuracy across all
reference values?

-e

0.00

*
*

Reference Value (x)

y = a + b.x
y: Bias, x: Ref. Value
a: Slope, b: Intercept

STABILITY (or DRIFT):

Stability (or Drift) is the total variation in the measurements obtained with a measurement system
on the same master or parts when measuring a single characteristic over an extended time period.
(AIAG, 2002)
Control Charts may be used to monitor the stability of a measurement system
A signal of special cause variation on the charts could indicate the need for calibration of the

measurement system

MEASUREMENT SYSTEM
ANALYSIS

MEASUREMENT SYSTEM
ANALYSIS USING MINITAB
MINITAB offers several commands to help you
determine how much of your process variation
arises from variation in your measurement system.
Gage R&R (Crossed), Gage R&R (Nested)
examine measurement system precision.
Gage Linearity and
linearity and accuracy.

Bias

examines

gage

MEASUREMENT SYSTEM ANALYSIS (Cont)

BIAS AND LINEARITY (EXAMPLE):
A manufacturer wants to know if a thermometer is taking
accurate and consistent readings at five heat settings (202,
204, 206, 208, and 210). Six readings are taken at each
setting.
To find out if the thermometer is taking biased
measurements, subtract the individual readings from the
reference value. The bias values for measurements taken at
heat setting 202 are calculated in the below table.
Thermometer
reading

Actual
temperature

BIAS

202.7

202

0.7

202.5

202

0.5

203.2

202

1.2

203.0

202

1.0

203.1

202

1.1

203.3

202

1.3

The temperature readings at the

202 heat setting are positively
biased; the thermometer gives
readings that are higher than
the actual temperature.
12

MEASUREMENT SYSTEM ANALYSIS (Cont)

BIAS AND LINEARITY (EXAMPLE) (Cont):
To interpret the linearity of the thermometer data, determine if
the bias of the thermometer changes across the heat settings.

If the data do not form a horizontal line on a scatter plot,

linearity is present.

The scatter plot shows that bias

changes as the heat settings
increase. Temperatures for lower
heat settings are higher than the
actual temperatures, while readings
for higher heat settings are lower
than the actual temperatures.
Because bias changes over the heat
settings, linearity is present in this
data.
13

GAGE LINEARITY AND BIAS STUDY

EXAMPLE: A plant foreman chose five parts that represented the expected range of the
measurements. Each part was measured by layout inspection to determine its reference (master)
value. Then, one operator randomly measured each part twelve times.
You obtained the process variation (16.5368) from a Gage R&R study using the ANOVA method.
Minitab displays the process variation in the Session window (Total Variation row of the Study Var
(6 * SD) column).
Open the worksheet GAGELIN.MTW

Choose Stat Quality Tools Gage Study Gage Linearity and Bias Study

14

GAGE LINEARITY AND BIAS STUDY:

EXAMPLE (Cont)
INTERPRETATION RULE:
In (Gage Bias) Section; if
Average PValue < 5%
So, Gage is Bias
In (Gage Linearity) Section;
if Slope PValue < 5%,
So Gage is producing
Nonlinear Results

Good Gage must have more

linearity than bias

15

Types of MSAs
MSAs fall into two categories:
Attribute
Variable

Attribute
Pass/Fail
Go/No Go
Document Preparation
Surface imperfections
Customer Service Response

Variable
Continuous scale
Discrete scale
Critical dimensions
Pull strength
Warp

Transactional projects typically have Attribute based measurement

systems.
Manufacturing projects generally use Variable studies more often, but do
use Attribute studies to a lesser degree.

GAUGE REPEATABILITY &

REPRODUCIBILITY (R & R) STUDIES
Gage repeatability
and
reproducibility
studies
determine how much of your observed process
variation is due to measurement system.

17

GAUGE REPEATABILITY & REPRODUCIBILITY

(R & R) STUDIES USING MINITAB
Gage repeatability and reproducibility studies determine how much of your observed
process variation is due to measurement system variation. MINITAB allows you to
perform either crossed or nested Gage R&R studies.
Use Gage R&R Study (Crossed) when each part is measured multiple times by
each operator.
If all operators measure parts from each batch, then use Gage R&R Study
(Crossed).
Use Gage R&R Study (Nested) when each part is measured by only one
operator.
If each batch is only measured by a single operator, then you must use Gage R&R
Study (Nested). In fact, whenever operators measure unique parts, you have a
nested design.

MINITAB provides two methods for assessing repeatability and reproducibility:

Xbar and R, and ANOVA. (ANOVA is better than Xbar and R method)
The Xbar and R method breaks down the overall variation into three
categories: part-to-part, repeatability, and reproducibility.
The ANOVA method goes one step further and breaks down reproducibility
into its operator, and operator-by-part (An Operator*Part interaction means that two or
18
more operators may measure different parts differently) components.

Gage R&R Study (Crossed)

METHODGage R&R Study (Crossed): ANOVA Method
The percent
contribution
from that represent the expected range of the process
EXAMPLE:
Ten parts
were selected
Part-To-Part
is larger than
that of the ten parts, three times per part, in a random
variation.
Three operators
measured
order. Total Gage R&R, telling you that
much
of the variation
is due to
Open the
worksheet
GAGEAIAG.MTW
differences
between
parts.
Choose Stat
> Quality
Tools
> Gage Study > Gage R&R Study (Crossed).
There are large
differences between
parts, as shown by the
non-level line.
the differences between operators are small

Operator B measures
compared to the differences between parts, but are
parts inconsistently. significant. Operator C appears to measure slightly
lower than the others.

most of the points in the X-bar and R

chart are outside the control limits,
indicating variation is mainly due to
differences between parts.

GUIDELINES FOR MEASUREMENT

SYSTEM ACCEPTABILITY

According to the Automobile Industry Action Group

(AIAG), you can determine whether your measurement
system is acceptable using the following guidelines.
If the Total Gage R&R contribution in the %Study Var
column (% Tolerance, %Process) is:
Less than 10% the measurement system is
acceptable.
Between 10% and 30% the measurement
system is acceptable depending on the application,
the cost of the measuring device , cost of repair, or
other factors.
Greater than 30% the measurement system is
unacceptable and should be improved.
If you are looking at the %Contribution column, the
corresponding standards are:
Less than 1% the measurement system is
acceptable.
Between 1% and 9% the measurement system is
acceptable depending on the application, the cost of
the measuring device, cost of repair, or other
factors.
Greater than 9% the measurement system is
unacceptable and should be improved.
According to the AIAG ,
when the number of distinct categories is 5 or more
it represents an adequate measuring system.

Here are the Automotive Industry

Action Groups definitions for Gage
acceptance
% Tolerance
or
% Study
Variance

% Contribution

System is

10% or less

1% or less

Ideal

10% - 20%

1% - 4%

Acceptable

20% - 30%

5% - 9%

Marginal

30% or greater

10% or greater

Poor

20

Gage R&R Study (Crossed)

METHODGage R&R Study (Crossed): ANOVA Method
Two-Way ANOVA Table With Interaction
Source
Part
Operator
Part * Operator
Repeatability
Total
Alpha to remove

DF
SS
MS
F
P
9 88.3619 9.81799 492.291 0.000
2
3.1673 1.58363 79.406 0.000
18
0.3590 0.01994 0.434 0.974
60
2.7589 0.04598
89 94.6471
interaction term = 0.25

Gage R&R
Source
VarComp
Total Gage R&R
0.09143
Repeatability
0.03997
Reproducibility
0.05146
Operator
0.05146
Part-To-Part
1.08645
Total Variation
1.17788

%Contribution
(of VarComp)
7.76
3.39
4.37
4.37
92.24
100.00

Between 10% and 30% the measurement

system is acceptable depending on the
application, the cost of the measuring device ,
cost of repair, or other factors
number of distinct categories is 5
represents an adequate measuring system

If p-value for Operator * Part is > 0.25, Minitab

omits this from the full model. Notice there is
an ANOVA table without the interaction because
the p-value was 0.974.
Two-Way ANOVA Table Without Interaction
Source
DF
Part
9
Operator
2
Repeatability 78
Total
89

SS
MS
F
P
88.3619 9.81799 245.614 0.000
3.1673 1.58363 39.617 0.000
3.1179 0.03997
94.6471

Between 1% and 9% the measurement system

is acceptable depending on the application, the
cost of the measuring device, cost of repair, or
other factors. (AIAG)
Study Var %Study Var %Tolerance
Source
StdDev (SD) (6 * SD)
(%SV)
(SV/Toler)
Total Gage R&R
0.30237
1.81423
27.86
22.68
Repeatability
0.19993
1.19960
18.42
14.99
Reproducibility
0.22684
1.36103
20.90
17.01
Operator
0.22684
1.36103
20.90
17.01
Part-To-Part
1.04233
6.25396
96.04
78.17
Total Variation
1.08530
6.51180
100.00
81.40

Number of Distinct Categories = 4

Gage R&R Study (Crossed)

METHODGage R&R Study (Crossed): XBar & R Method
EXAMPLE: Ten parts were selected that represent the expected range of the process
variation. Three operators measured the ten parts, three times per part, in a random
order.
Open the worksheet GAGEAIAG.MTW
In the Components of Variation graph, a low percentage
Choose Stat > Quality Tools >ofGage
Study
> Gage
R&R
Study
(Crossed).
variation
(7.13%)
is due
to the
measurement
system
(Gage R&R), and a high percentage (92.87%) is due to
differences between parts. (See your Session Window)
Most of the points in the XBar Chart are outside the
control limits when the variation is mainly due to part-topart differences.

22

Gage R&R Study (Crossed)

METHODGage R&R Study (Crossed): XBar & R Method
INTERPRETATION:
EXAMPLE: (Cont)
Session window output
XBar and R method with GAGEAIAG data
Source
Total Gage R&R
Repeatability
Reproducibility
Part-To-Part
Total Variation

VarComp
0.09357
0.04073
0.05284
1.21909
1.31266

%Contribution
(of VarComp)
7.13
3.10
4.03
92.87
100.00

Process tolerance = 8
Study Var
Source
StdDev (SD) (6 * SD)
Total Gage R&R 0.30589
1.83536
Repeatability
0.20181
1.21087
Reproducibility 0.22988
1.37925
Part-To-Part
1.10412
6.62474
Total Variation
1.14571
6.87428

Look at the %Contribution column in the Gage R%R Table.

The measurement system variation (Total Gage R&R) is
slightly smaller than what was found for the same data with
the ANOVA method.
The % Study Var column shows that the Total Gage R&R
accounts for 26.70% of the study variation; again slightly
smaller than what was found using the ANOVA method. In
some cases, there is a greater difference in the two
methods because the ANOVA method considers significant
Operator by Part interactions whereas the XBar and R
method does not.

%Study Var
(%SV)
26.70
17.61
20.06
96.37
100.00

Between 1% and 9% the

measurement
system
is
acceptable depending on the
application, the cost of the
measuring device, cost of
repair, or other factors. (AIAG)

%Tolerance
(SV/Toler)
22.94
15.14
17.24
Between 10% and 30% the
82.81
85.93
measurement
system
is

Number of Distinct Categories = 5

number of distinct categories is 5 represents an adequate

measuring system

acceptable depending on the

application, the cost of the
measuring device , cost of repair,
or other factors

EXERCISE
GAGE R&R STUDY (CROSSED) USING:
i. ANOVA METHOD
ii. XBAR & R METHOD
Three parts were selected that represent the
expected range of the process variation. Three
operators measured the three parts, three
times per part, in a random order.
Open the file GAGE2.MTW

Gage R&R Study (Nested)

METHODGage R&R Study (NESTED): XBar & R Method
EXAMPLE: Three operators each measured five different parts twice, for a total of 30
measurements. Each part is unique to operator; no two operators measured the same part. You
decide to conduct a gage R&R study (nested) to determine how much of your observed process
variation is due to measurement systemvariation
Look at .the Components of Variation Graph - located in
Open the worksheet GAGENEST.MTW
upper left corner. Most of the variation is due to
Choose Stat > Quality Tools > Gage Study
> Gage system
R&R Study
measurement
error (Nested).
(Gage R&R), while a low
percentage of variation is due to differences between parts.
Look at the X-Bar Chart - located in the lower left corner.
Most of the points in the X-Bar chart are inside the control
limits when the variation is mostly due to measurement
system error.

Gage R&R Study (Nested)

METHODGage R&R Study (Nested):
INTERPRETING THE RESULTS:
EXAMPLE: (Cont)

Look at the %Contribution columns for Total

Gage R&R (Nested) for Response

Gage R&R and Part-to-Part. The percent
Source
DF
SS
MS
F
P
contribution for differences between parts
Operator
2
0.0142 0.00708 0.00385 0.996
Part (Operator) 2 22.0552 1.83794 1.42549 0.255 (Part-To-Part = 17.54) is much smaller than the
percentage contribution for measurement
Repeatability
15 19.3400 1.28933
system variation (Total Gage R&R = 82.46).
Total
29 41.4094
Gage R&R
The %Study Var column indicates that the Total
%Contribution
Gage R&R accounts for 90.81% of the study
Source
VarComp
(of VarComp)
variation. So, most of the variation is due to
Total Gage R&R 1.28933
82.46
measurement system error; very little is due to
Repeatability
1.28933
82.46
differences between part.
Reproducibility 0.00000
0.00
A 1 in number of distinct categories tells you
Part-To-Part
0.27430
17.54
that the measurement system is not able to
Total Variation
1.56364
100.00
Process tolerance = 10
Source
Total Gage R&R
Repeatability
Reproducibility
Part-To-Part
Total Variation

Study Var
StdDev (SD) (6 * SD)
1.13549
6.81293
1.13549
6.81293
0.00000
0.00000
0.52374
3.14243
1.25045
7.50273

Number of Distinct Categories = 1

distinguish between parts.

%Study Var %Tolerance

(%SV)
(SV/Toler)
90.81
68.13
90.81
68.13
0.00
0.00
41.88
31.42
100.00
75.03

QUESTIONS