Variation Management of Key Characteristics

(AS9103) Evaluation

Variation Management of Key Characteristics (AS9103) Evaluation Tool

This matrix tool is designed to facilitate the evaluation of supplier systems relative to the expectations of SAE AS9103, “Variation
Management of Key Characteristics.” AS9103 is imposed contractually through Boeing Quality Management System Addendum 1.
Addendum 1 is typically called out whenever the supplier has key characteristics defined by Boeing specification document, drawing or
dataset, or by SCD. Key characteristics may also be identified through supplier implementation of AS/EN/SJAC 9100 paragraph 7.3.3.e:
“[Design and development outputs shall] identify key characteristics, when applicable, in accordance with design or contract
requirements.” Capability is indicated for those supplier systems that meet minimum expectations as defined by the “shaded boxes” of
the matrix. Each shaded box contains a reference to a requirement contained in AS9103. A score for the assessment is calculated only
when it is part of a PSC evaluation (see below).

Boeing suppliers are expected to use this matrix periodically for self-assessment purposes. Companies desiring to develop an advanced
AS9103 system may find this matrix to be a valuable guide, particularly when combined with Boeing documents AQS Guidelines and
D1-9000-1, AQS Tools, available on the Boeing “Doing Business” website:
http://www.boeing.com/companyoffices/doingbiz/supplier/index.html

Preferred Supplier Certification

This same matrix is used for evaluations when the supplier is participating in the Boeing Preferred Supplier Certification program.
Boeing personnel will review the supplier’s system relative to the expectations of this tool, and score the matrix appropriately. The
supplier’s system must meet the minimum expectations (shaded boxes) for Bronze level qualification in the Preferred Supplier
Certification program. Silver and Gold level expectations are substantially higher, with correspondingly higher minimum scores. Please
note: PSC evaluations are the only instance in which scores are assigned to this matrix.

Copyright © 2006 The Boeing Company. All rights reserved Fifth Edition 10/04/2006 Page 1 of 4
Variation Management of Key Characteristics (AS9103)
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
1. Variation Management (General requirements) ---- The analysis of products and processes, implementation of improvement projects, or the
existence of customer defined Key Characteristics may indicate that variation management activities are required. Variation control methods other
than SPC may be used, but measurable evidence must demonstrate that the controls are effective. For Boeing defined Key Characteristics, the
requirement to control and reduce variation may be waived in situations where it is impossible or prohibitively expensive to meet the control and
capability requirements of AS9103 Section 5. These exceptions must be documented by the organization and approved by Boeing.
Little or no documentation on Documentation exists for all Variation management Variation management Documentation and
identified Key Characteristics. identified Key Characteristics, documentation exists for core documentation contains lessons understanding of processes
including mfg. process elements manufacturing processes in learned. leads to further improvement.
that influence variation in KCs addition to KCs.
1 Documentation of KC or
as well as their control
process information techniques and measurement
methods.

Ref. AS9103, 5.2

2 Need for variation
reduction and KCs
3 KC management
4 Variation management
activities
Organization has not considered When analysis indicates, KCs Organization examines for part- Advanced analytical methods Variation is managed through
the need for variation reduction. have been identified. family or process Key are used to identify Key Process process control.
Characteristics. Parameters.
Ref. AS9103,5.2c
Little or no variation Plan in place to manage Key characteristic activities are Key characteristics are Advanced variability reduction
control/reduction activities. identified key characteristics. given a high priority. periodically reviewed for tools are used to avoid variation
suitability. in product and/or process
Ref. AS9103 5.5 development.
Little or no variation Variation management activities Demonstrable improvement Part families and/or process Demonstrable improvement in
management activity observed performed on identified KCs exists from part level variation variation management activities part family and/or process
on Key Characteristics. until control and capability management activities. implemented. variation management activities.
established, unless agreed to by
the customer.

Ref. AS9103, 5.1, 5.3e & 5.6

Little or no measurable evidence Measurable evidence
of control. demonstrates that the selected
variation control method is
5 Verification of control effective. Appropriate
method and process monitoring methodology
monitoring implemented to ensure
continued performance.
Organization performs regular
validation of control method
effectiveness. Monitoring
methodology drives
improvement activities.
Monitoring methodology
implemented that ensures
process performance through
control of process settings.
Comprehensive database of
control techniques and methods
is consistently used to establish
effective and efficient process
controls.

Ref. AS9103, 5.1& 5.4

Little or no standardized
processes exist.
6 Variation control methods
Variation control methods such Key Processes are standardized Demonstrated improvement is Process settings are well
as tooling, control of process through the use of variation evident from use of variation controlled to ensure consistent
settings, standard processes control methods. control methods. processes.
and mistake proofing may be (e.g. Mistake proofing is used to
used to ensure process stability ensure non-conforming product
and capability. is not passed on.)

Ref. AS9103, 5.4

Copyright © 2006 The Boeing Company. All rights reserved Fifth Edition 10/04/2006 Page 2 of 4
Variation Management of Key Characteristics (AS9103)
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
2. Variation Control Methods ---- Statistical process control charts can be an effective means of controlling variation and obtaining process
knowledge. This process knowledge will help establish mistake proof processes, develop up stream process controls, contribute to process and
product redesign, and eventually alleviate the reliance on control charts. The requirements in this section are conditional on the selection of SPC as
the variation control method.
Control charts do not represent Measurements used for control Summaries of control chart data Lessons learned from data Control chart data is analyzed at
normal production output. charts represent the normal are readily available to all being used on similar a more detailed level (i.e. by
1 Data collection and usage production output. employees. products/processes. shift, machine, etc).

Ref. AS9103, 5.3a

Control Chart chosen is Control Chart chosen is Control charts periodically Process shifts and trends are Advanced methods of control
2 Control chart selection inappropriate for application. appropriate for application. reviewed for appropriateness. investigated (e.g. Western are used (e.g. Target charts,
and usage Electric rules) Multivariate charts, EWMA).
Ref. AS9103, 5.3a
Errors in control limit calculation. Control limits computed Control limits ‘frozen’ with Control limits are reviewed Control Limits are recalculated
appropriately and reflect the appropriate data and the data periodically for needed changes. for processes that exhibit
3 Calculation of control current process. used in control limits is readily statistically significant change.
limits identifiable.
Ref. AS9103, 5.3a
Little or no Control Chart usage. When similar KCs from different Reduction of control chart Process control is used Process control yields
products are combined on the frequency is evident as a result extensively in lieu of charts for improvements in other, non-Key
same control chart, the of Process Improvement. each Key Characteristic. Characteristics.
4 Control charting characteristics shall have similar
processes variability and be traceable to
the specific part or product.

Ref. AS9103, 5.3.c

Out-of-control conditions are Out-of-control conditions Corrective action consistently Processes are stable and Processes are designed to
identified, but no action taken. identified, and investigated. addresses the root cause of the predictable. minimize or eliminate out-of-
5 Process control and Action is taken to remove or out of control condition. control conditions.
minimize the effect of these
correction causes.
Ref. AS9103, 5.1
Potential sources of variation Sources of variation are Sources of variation are Advanced Statistical Techniques Advanced Statistical Techniques
are not consistently identified for identified and controlled for identified and controlled for all are sometimes used to identify are routinely used to identify and
6 Identification of variation incapable Key Characteristics. incapable Key Characteristics. (including capable) Key and correlate sources of correlate sources of variation
sources Characteristics. variation with Key with Key Characteristics.
Characteristics.
Ref. AS9103, 5.1 & 5.2

Copyright © 2006 The Boeing Company. All rights reserved Fifth Edition 10/04/2006 Page 3 of 4
Variation Management of Key Characteristics (AS9103)
Expectation Column 1 (1 point) Column 2 (2 points) Column 3 (3 points) Column 4 (4 points) Column 5 (5 points)
3. Variation Management Activities ---- The identification, control, and optimization of key processes impacts the performance measures. When a
process is not in control, the out-of-control condition must be investigated and corrective action shall be taken. A process is considered capable if its
Cpk exceeds 1.33. If the process is not capable, the organization will identify and control sources of variation and take corrective action. (D1-9000-1
contains information that is useful in diagnosing sources of variation as well as providing guidance in techniques to reduce measurement error and
process variation.)
Data collected but capability Process capability shall be Reporting of process capability Capability measures calculated Management uses process
measure not established. established for Key to management on a regular for Process Key Characteristics. capability database for process
1 Establishing capability
Characteristics basis. improvement efforts.
measures
Ref. AS9103, 5.1 & 5.3.a
Little or no action plans for For Cpks (or equivalent Action plans for capability Majority of processes have Cpks Capability values show stability
incapable processes. measures) that are less than improvement show progress (or equivalent measures) of over time.
2 Process capability and 1.33, corrective action plans are over time. >1.33.
improvement in place for improvement.

Ref. AS9103, 5.1 & 5.3b

Reduced frequency of Process capability or equivalent Process capability and Opportunities to reduce Opportunities to reduce
inspection is not related to fallout rate are calculated inspection frequencies are inspection frequency based on inspection frequency based on
3 Reduced frequency of process capability. whenever capability is used to reported to management on a capability are occasionally capability are reviewed on a
inspection reduce inspection frequency. regular basis. reviewed. frequent, scheduled basis.

Ref. AS9103 5.3d

Little or no activity related to KC requirements are flowed to Log of KCs flowed to KC status, both in-house and at There is an active program to
ensuring customer KC subcontractors, and all KCs are subcontractors exists; status of subcontractors, is reported to work with subcontractors to
4 Ensure customer KC requirements are met managed to conform to all KCs in house and at management on a regular basis. improve the control and
requirements are met customer requirements. subcontractors is recorded. capability of their KCs.

Ref. AS9103 4.0

Inconsistent records of
capability trends.
5 Use of capability data
Capability trends are
consistently recorded.
Lesson learned database is
shared so as to improve similar
parts / processes.
Process capability information is
used in choosing the
appropriate manufacturing
process/equipment (i.e. shop
loading, capacity loading, etc.).
Savings from improvements are
calculated and reviewed. Plans
link capability improvement to
cost-benefit ratio.

Little or no gage variation
studies are performed.
6 Gage variation studies
Gage variation studies are
performed for incapable Key
Characteristics when analysis
indicates, and corrective action
is taken as needed.
Gage variation studies are
routinely performed for all
(including capable) Key
Characteristics, and corrective
action is taken as needed.
Gage variation studies are Gage variation studies are
routinely performed prior to any performed as a regular part of
data collection. the calibration/certification
process.

Copyright © 2006 The Boeing Company. All rights reserved Fifth Edition 10/04/2006 Page 4 of 4