GAUGE REPEATABILITY AND REPRODUCIBLILITY: • Linearity – assess with equipment manufacturers data and
APQP/PPAP EVIDENCE WORKBOOK STATISTICAL
RATIONALE AND USER REFERENCE GUIDE
INTENT:
The intent of this reference guide is to provide an overview of the
statistical tools used within the APQP/PPAP Evidence Workbook
and the statistical methodology underpinning the sequence of
these tools. The instructions of how to populate the APQP/PPAP
Evidence Workbook can be found within the APQP/PPAP Evidence
Workbook itself. There is an instructions sheet for each of the
APQP/PPAP Evidence Workbook sheets requiring population.
Additional instructions can be found in the APQP/PPAP Evidence
Workbook training. The APQP/PPAP Evidence Workbook and
associated training can be accessed on the Ford Supplier Portal on
the APQP/PPAP Requirements for GPDS page (APQP/PPAP
Requirements for GPDS (ford.com)).
GAUGE DISCRIMINATION:
The first piece of information that needs to
be entered into the Capability Evidence
sheet within the APQP/PPAP Evidence
Workbook is the gauge resolution. Note:
All columns that require user input have a
light blue column heading.
G AUGE RESOLUTION :
Gauge Resolution which can be also referred to as the least
significant digit - LSD is the increments of measure provided by the
gauge. Per the AIAG MSA guide, a measurement instrument’s
discrimination should be at least one-tenth of the range to be
measured (tolerance). This is also known as the 10-bucket rule. For
example, if a length of a part is 10mm with a tolerance of +/- 0.1mm
providing a total tolerance of 0.2mm then multiplying 0.2mm by
10% is 0.02. This means that the gauge needs to be able to measure
a difference of at least 0.02mm.
The image of the glasses provides an example of gauge resolution.
The scale on the left has a resolution of 1cm. Using this scale, we
measure the height of the glasses to be 2cm. The scale on
the right has a resolution of
0.1cm and therefore
we can measure the
glasses to be 1.7cm.
Better resolution
gives you more accurate measurements. It is critically important
that the measurement device selected for the measurement of a
special characteristic is fit for purpose. Often devices are selected
to measure the magnitude of characteristics and the gauge either
does not have enough resolution or the gauge is difficult to use for
the specific characteristic and generates reproducibility errors
when handled by different operators. The previous graphic
illustrates these points.
Version 1.0 – 6 April 2023
VARIABLE GAUGE R&R:
As per the Ford Customer Specific Requirements for PPAP released
in 2023, the ANOVA method for analysis of gauge R&R is now
mandatory. The ANOVA method is superior to the Average and
Range Method as it includes part to operator interaction, whereas
the Average and Range Method does not. Many statistical packages
are available to evaluate gauge R&R using the ANOVA method, in
addition Ford has developed an Excel based ANOVA template to aid
suppliers who do not have a package available. This can be
accessed through the Ford Supplier Portal under Miscellaneous
Tools (Quality System and Supplier Documents (ford.com)).
Per the Ford Customer Specific Requirements for PPAP; variable
GR&R studies require: 10 parts, 3 operators and 3 repeats.
Deviation from this requirement needs STA approval.
A CCURACY D EFINITION :
Accuracy is the closeness of agreement between an observed
value and an accepted reference value or standard. Low accuracy
reflects a systemic issue in the measurement such as a gauge out
of calibration, worn etc. Accuracy is measured as the amount of
error in a measurement
in proportion to the total
size of the measurement.
One measurement is
more accurate than
another if it has a smaller
relative error.
P RECISION (R EPEATABILITY )
D EFINITION :
Is the closeness of agreement between individual measurements
or results. Precision relates to the variance of repeated
measurements. A measuring device with a low variance is more
precise than another having a higher variance.
A measurement system may be precise but not accurate at the
same time. The image of the targets illustrates accuracy and
precision.
R EPRODUCIBILITY D EFINITION :
Is the variation in the same measuring device when it is used by
different operators to measure the same parts. Causes of poor
reproducibility include poor operator training.
V ARIABLE GR&R D EFINITION :
Gauge R&R (GR&R) studies determine how much of the observed
process variation is due to measurement system variation due to:
• Measurement System Repeatability
• Measurement System Reproducibility
GR&R does not assess:
• Accuracy – assess with calibration
• Bias – assess with calibration
• Stability – assess with equipment manufacturers data
A TTRIBUTE GR&R D EFINITION :
calibration.
The following image summarises both repeatability and
reproducibility.
%
S TUDY ,
% the attribute
T OLERANCE , NDC D EFINITION , +/-2 T OTAL STD DEV :
Ford Customer Specific Requirements for PPAP released in 2023
requires both % Tolerance and % Study to be reported but the
primary metric used for acceptance of GR&R analysis is %
Tolerance. Reporting of +/- 2 Total Gauge R&R Standard
Deviations is also required to understand the 95% confidence
(uncertainty) of any one measurement. This value can be used in
conjunction with engineering judgment to help assess the distance
between the edge(s) of the process distribution and the
specification limit(s). % Tolerance is mandated over % Study
Variation, as % Tolerance is
relatively unaffected by the parts
selected for the GR&R study,
whereas % Study Variation and
Number of Distinct Categories
(NDC) is significantly affected.
% Study and NDC is only meaningful if the parts used for the GR&R
study are representative of a production run with all known sources
of variation as it uses the gauge variation as a ratio of the spread of
parts selected. Both % Study and NDC are correlated to total
variation. % Tolerance is not correlated to total variation.
If the gauge does not pass % Tolerance, it cannot perform the job
of determining if the parts are within the specification.
Tolerance, % Study and NDC, need to be reported in the
APQP/PPAP Evidence Workbook. % Tolerance must meet the
criteria to be able to proceed through the APQP/PPAP Evidence
Workbook. GR&R as a percent of tolerance criteria:
• <10% is acceptable
• ≥10% and ≤30%, contact the STA site engineer to determine if
the GR&R is acceptable.
• >30%, it is unacceptable, and the organisation shall
implement containment actions and a corrective action plan
to improve measurement capability until the GR&R
requirements are met.
ONE - SIDED TOLERANCE :
The % Tolerance calculations for one-sided tolerances are
different to the above double-sided tolerance calculation but the
acceptance percentages are as above.
Upper specification limit with no lower boundary: In these cases,
calculate % tolerance by dividing 3 gauge R&R standard deviation
by the
difference
between the upper specification limit and the mean of the data.
Lower specification limit with no boundary: In these cases
calculate the % Tolerance by dividing 3 gauge R&R standard
deviation by
the difference
between the mean of the data and the lower specification limit.
Ford Customer Specific requirements for PPAP require attribute
GR&R studies to have a minimum of 50 parts, 3 operators and 3
trials. Deviation from this requirement needs STA approval.
The parts used for the GR&R study should be variably measured
using an appropriate measurement device (e.g. CMM) prior to
conducting
GR&R. The
parts to be
used for the
attribute
study should be distributed as per the following graphic. This will
test the gauge throughout the tolerance range and at the
specification limits addressing the key risk of using an attribute
gauge.
The criteria
for attribute
GR&R
acceptance
are:
• Gauge
must
reject all parts that are outside of the specification limits.
• All Kappa values should be greater than 0.75.
For additional information refer to the Ford Customer Specific
Requirements for PPAP.
GAUGE R&R IMPORTANT POINTS:
1. Gauge resolution must be acceptable before proceeding with
GR&R analysis.
2. Do not proceed past the GR&R columns in the APQP/PPAP
% Evidence Workbook if gauges have not been shown to meet the
acceptance criteria for either variable or attribute GR&R.
3. Any measurement analysis (i.e., capability analysis) conducted
using a gauge that has poor Gauge R&R will be incorrect.
STATISTICAL PROCESS CONTROL (SPC):
Statistical process control (SPC) is a
methodology for monitoring a process to
identify special causes of variation and signal
the need to take corrective action when it is
appropriate. When special causes are
present, the process is deemed to be out of
control. If the variation in the process is due
to common cause alone, the process is said
to be in statistical control. SPC is a proven
technique for improving quality and productivity.
A process must be stable and in-control before process capability
can be calculated. Instability affects the process in unpredictable
ways. Capability which is calculated on a process that is not stable
is not valid, because the calculations are not based on a
predictable process. Therefore, future results of the process
cannot be predicted.
The APQP/PPAP Evidence Workbook requires that you select the
control chart that has been used to demonstrate process stability
and control. Moreover, identify if any points were not in statistical
control. Note: if any points were not in statistical control, then you
will not be able to proceed through the remainder of the sheet.
Actions will need to be taken to address the cause of the out-of-
control conditions. Please consult your STA engineer for assistance.
Page 1 of 2
T YPES OF VARIATION :
Common Cause: variations are natural variations that are a part
of the process.
Special Cause: variations are not part of the normal process
variation and are due to unusual, or out-of-the-ordinary events.
CONTROL CHART STRUCTURE :
Control charts graphically represent process variation and are
ordered by time. The Average Chart tells us about the process
location. The Range Chart tell us about the process stability. The
positions of the upper control limit and the lower control limit lines
are determined by the data. The lines are calculated at +/-3
standard deviations away from the centre line. The control limits
are determined by the natural variation of the process. The control
limits show the range that the process can randomly fall within.
The average of different subgroups made in production order will
have similar subgroup averages. They will not be identical as they
are averages from different parts, but there is a limit to the
magnitude of difference due to random chance. This is how the
control limits are determined. If the process is stable, then it is
highly probable that the points will fall within the control limits. If
the points fall outside of the control limits, then there is a very good
chance that there is a special cause present in the process.
Control limits do not equal Specification limits. Specification limits
should be based on the voice of the customer and established
during the design process. These limits define the range of
customer satisfaction. Control limits are based on the voice of the
process and reflects how the process is running.
OUT OF CONTROL INDICATIONS :
Imagine that you are driving on a highway with several lanes. This
is your road to establishing stability for your process. The guardrails
on each side represent the +/-3σ control limits. The lanes allow you
to drive at different speeds. These lanes (different sigma intervals)
represent the slight differences in parts within your production
process caused by common cause variation. If there was an
obstacle on the road this would cause you to deviate away from
your average trajectory into another lane and potentially outside of
the +/-3σ control limits (guardrails). The same could happen in a
process, if the settings on a machine change or a material type is
changed etc. Special causes are unpredictable, but they are
assignable causes or patterns which need to be understood and
reacted to; otherwise, a process can drift further out of control and
in this example the car would run off the road. The following
Version 1.0 – 6 April 2023
general rules are applicable to both variable and attribute control
charts.
Test 1: Point outside of the control limit: This is the most
recognized rule. The cause of a point outside of the 3σ control limit
indicates a process shift. The special cause in this instance must be
understood and addressed prior to proceeding with any further
analysis and PPAP. This is why it is included in the APQP/PPAP
Evidence Workbook.
In a 3-sigma process that is perfectly centred, 2,700 parts per
million are expected to exceed the 3 sigma limits. This means that
1 subgroup out of 370 subgroups will exceed the 3-sigma limit
without an identifiable special cause. This means the chance of any
points outside of the control limits before launch due to common
cause are extremely small and any such case must be assumed to
be a special cause and be investigated and understood.
The remaining general rules are useful to help problem solving
activities.
Test 2: Nine points in a row on the same side of the centre line.
Test 3: Six points in a row either increasing or decreasing.
Test 4: Fourteen points in a row alternating up and down.
Test 5: Two out of three points more than 2σ from the centre line
(on the same side).
Test 6: Four out of five points more than 1σ from the centre line
(on the same side).
Test 7: Fifteen points in a row within 1σ from the centre line (either
side).
Test 8: Eight points in a row more than 1σ from the centre line
(either side).
SPC IMPORTANT POINTS:
1. A process cannot be said to be stable and in-control if both
charts have points outside of control limits.
2. Parts plotted in control charts must be in production order.
3. Control limits do not equal specification limits.
4. A process must be stable and in-control before process
capability can be calculated. If the process is not statistically
stable process capability estimates have no predictive value as
the process parameters can change, therefore, the process
capability estimates are estimates of past capability and cannot
predict future capability (the primary goal).
STUDY PARAMETERS:
The Ford Customer Requirements for PPAP
require a minimum of 125 measurements to
establish process capability. The 125
measurements are to be obtained using
rational sampling methods from a
population of 300 parts. Note: Any deviation from this
requirement requires STA management approval.
RATIONAL S AMPLING :
A rational sample is a collection of process output measurements
(a subgroup) with only common cause variation affecting the
samples. To ensure this, a small number of consecutively produced
parts are measured. These parts are produced under the same
process conditions (i.e., not interrupted by tool changes, operator
changes, material changes etc), as these may introduce special
cause variation.
Control charts (SPC) require rational sampling to effectively meets a Ppk of 1.67 but the Weibull distribution shows a Ppk of 1.27
differentiate between Special Cause and Common Cause variation. which is not acceptable.
RATIONAL SUBGROUP STRATEGIES:
The type of distribution that a process follows is governed by the
At pre-production (i.e., TT build etc) an initial baseline for each
special characteristic needs to be established. This can be achieved
using 25 subgroups of 5. Typically, 300 part Phase 0 PPAP run is
required and thus a rational sampling plan is to measure 5 parts
and skip the next 7 parts which would total 25 subgroups:
• Measure parts 1 to 5;
• Skip parts 6 to 12;
• Measure parts 13 to 17,
• Skip parts 18 to 24 and so on.
Rational sampling requirements post launch are dependent on the
process, the characteristic and historical performance. Please
discuss the specifics of your process with the STA Site engineer.
physics of the process. Leveraging historical data or process
DISTRIBUTION ANALYSIS: experts to determine the expected distribution is very helpful if the
data is not following a normal distribution. Blindly fitting a
The APQP/PPAP Evidence Workbook requires
distribution is not acceptable as the cause of non-normal data
that the process distribution type is identified.
maybe due to special cause. Most processes follow a normal
The process distribution type needs to be
distribution. Below is a table that illustrates different processes
identified prior to the calculation of capability
and the expected process distributions.
so that the correct distribution is used for
process capability calculations. Failure to use DISTRIBUTION ANALYSIS IMPORTANT POINTS:
the correct distribution type will significantly
affect the accuracy and validity of process
capability. Standard process capability
equations (Ppk) assume normal distributions (Bell shaped curve)
calculations may be significantly inaccurate if data is not normal.
Normality tests such as p-values and probability plots are the most
1. A distribution type needs to be determined prior to proceeding
common methods used to determine what type of distribution data with any process capability calculations.
follows. These tests compare the shape of the calculated curves to 2. Distribution types are determined by the physics of the process.
Consulting with process experts and using historical process
the actual distribution of process data points and provide a value
data is helpful to confirm the distribution type.
indicating how well the distribution fits the curve. Consider the
3. Fitting a distribution that does not match the expected
below data, both the probability plot and the p-values show that
distribution indicates that special causes are present that need
the data does not follow a normal distribution and that in fact it to be identified.
follows a Weibull distribution. A p-value that is less than 0.05
4. Utilise Ford STA or Master Black Belt (MBB) to help when fitting
indicates that the data does not follow that specific distribution. non-normal distributions.
CAPABILITY RESULTS:
Capability refers to the ability of the
process to consistently produce a
product meeting customer
expectations. Capability is based on a
set of data taken over a period and measures all the effects of the
process. Process capability calculations make no sense if:
• the process is not in stable and in control.
The probability plot shows how well the data points fall within a • the incorrect distribution type has been used.
distribution; in this case most of the points from the data set fall Process capability is a measure of the actual process variation as a
within the red lines showing that the data fits the Weibull ratio of the specification (customer requirements). Ford requires
distribution best. If the distribution type was not verified before suppliers to use the Ppk index, which must be greater or equal to
proceeding with calculating process capability the team would
1.67 at PPAP.
mistakenly think that the process is capable, but when analysis If a Ppk 1.67 cannot be achieved, then 100% measuring or mistake
using the correct distribution would show the process is not proofing is required until process stability and acceptable
capable. Below the normal distribution shows that the process capability are achieved.
Page 2 of 2