AS 3960—1990

Australian Standard

Guide to reliability and

maintainability program
management
This Australian Standard was prepared by Committee QR/5, Reliability and
Maintainability. It was approved on behalf of the Council of Standards Australia on
20 July 1989 and published on 16 February 1990.

The following interests are represented on Committee QR/5:

Australian Electrical and Electronic Manufacturers’ Association
Australian Nuclear Science and Technology Organisation
Australian Organization for Quality
Department of Defence
Federal Chamber of Automotive Industries
Federation of Automotive Products Manufacturers
Institute of Quality Assurance
Institution of Engineers, Australia
Institution of Radio and Electronic Engineers, Australia
Telecom Australia

Review of Australian Standards. To keep abreast of progress in industry, Australian Standards are subject
to periodic review and are kept up to date by the issue of amendments or new editi ons as necessary. It is
important therefore that Standards users ensure that they are in possession of the latest editi on, and any
amendments thereto.
Full detail s of all Australi an Standards and related publications wil l be found in the Standards Australia
Catalogue of Publi cati ons; this informati on is supplemented each month by the magazine ‘The Australi an
Standard’, which subscribing members receive, and which gives detail s of new publications, new edit ions
and amendments, and of withdrawn Standards.
Suggesti ons for improvements to Australi an Standards, addressed to the head offi ce of Standards Australia,
are welcomed. Noti fi cati on of any inaccuracy or ambiguity found in an Australi an Standard should be made
without delay in order that the matter may be investigated and appropriate action taken.

This Standard was issued in draft form for comment as DR 87226.

 AS 3960—1990

Australian Standard

Guide to reliability and

maintainability program
management

First publi shed in part as AS 1211.2—1972.

AS 1211.1 fir st published 1977.
AS 1211.3 fir st published 1977.
AS 12112.2—1972, AS 1211.1—1977 and
AS 1211.3—1977 revised, amalgamated and redesignated
AS 3960—1990.

PUBLISHED BY STANDARDS AUSTRALIA

(STANDARDS ASSOCIATION OF AUSTRALIA)
1 THE CRESCENT, HOMEBUSH, NSW 2140
ISBN 0 7262 5892 X
AS 3960—1990 2

PREFACE

This Standard was prepared by the Standards Australia Committee on Reliability and
Maintainability, under the direction of the Quality and Reliability Standards Board to supersede
AS 1211, Reliability of electronic equipment and components, Part 1—1977: Terminology,
Part 2—1972: Reliability concepts, and Part 3—1977: Reliability program for equipment.
It is one of a number of Standards to be prepared on various aspects of reliability and
maintainability.
It is based largely on BS 5760, Reliability of constructed or manufactured products, systems,
equipments and components: Part 1: Guide to reliability and maintainability programme
management. Consideration was also given to IEC Publication 300, Reliability and maintainability
management, and this Standard is compatible with that publication.
This Standard extends the scope of the AS 1211 series beyond the confines of electronic hardware
and increases the detail in which the subject is treated. The presentation of material has been
re-formatted so as to provide clear guidance on the separate tasks required for effective reliability
and maintainability program management.

 Copyri ght STANDARDS AUSTRALIA

Users of Standards are reminded that copyri ght subsists in all Standards Australi a publications and soft ware. Except where the
Copyri ght Act all ows and except where provided for below no publications or software produced by Standards Austr alia may be
reproduced, stored in a retri eval system in any form or transmitt ed by any means without pri or permission in wri ti ng fr om
Standards Australi a. Permission may be conditi onal on an appropriate royalt y payment. Requests for permission and information on
commercial soft ware royalti es should be dir ected to the head off ice of Standards Australi a.
Standards Australi a wil l permit up to 10 percent of the technical content pages of a Standard to be copied for use
exclusively in-house by purchasers of the Standard without payment of a royalty or advice to Standards Austr alia.
Standards Australi a wil l also permit the inclusion of its copyri ght material in computer soft ware programs for no royalt y
payment provided such programs are used exclusively in-house by the creators of the programs.
Care should be taken to ensure that material used is fr om the current editi on of the Standard and that it is updated whenever the
Standard is amended or revised. The number and date of the Standard should therefore be clearly identif ied.
The use of material in pri nt form or in computer soft ware programs to be used commercially, with or without payment, or in
commercial contracts is subject to the payment of a royalty. This policy may be vari ed by Standards Austr alia at any ti me.
 3 AS 3960—1990

CONTENTS

Page

FOREWORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

SECTION 1. SCOPE AND GENERAL

1.1 SCOPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 REFERENCED DOCUMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 DEFINITIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

SECTION 2. RELIABILITY AND MAINTAINABILITY PROGRAM

2.1 GENERAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 PROGRAM ACTIVITIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

SECTION 3. SPECIFICATION OF RELIABILITY AND MAINTAINABILITY

3.1 GENERAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 WRITING RELIABILITY AND MAINTAINABILITY CLAUSES
IN A SPECIFICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 SPECIFICATION OF RELIABILITY AND MAINTAINABILITY
IN PRACTICE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

SECTION 4. ASSESSMENT AND PREDICTION OF RELIABILITY

AND MAINTAINABILITY

4.1 GENERAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 25
4.2 RELIABILITY ASSESSMENT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 25
4.3 RELIABILITY PREDICTION BY MODELLING . . . . . . . . . . . . . . . . . . . . . . 25
4.4 PROVISION OF RELIABILITY DATA . . . . . . . . . . . . . . . . . . . . . . . . . . .. 25
4.5 RELIABILITY GROWTH TESTING . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 26
4.6 RELIABILITY DEMONSTRATION AND TESTING . . . . . . . . . . . . . . . . . .. 26
4.7 MAINTAINABILITY PREDICTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 27
4.8 MAINTAINABILITY DEMONSTRATION AND TESTING . . . . . . . . . . . . .. 28
4.9 COMPLIANCE ILLUSTRATION BY MEANS OTHER THAN TESTING . . .. 28

SECTION 5. PRODUCTION, FLOW, ANALYSIS AND INTERPRETATION

OF RELIABILITY AND MAINTAINABILITY DATA

5.1 GENERAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . .. .. . .. . 29
5.2 DATA INPUT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . .. . . ... . .. . 29
5.3 DATA SOURCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . .. . ... . . .. . 34
5.4 DESIGNING THE DATA COLLECTION FORM . . . . . . .. . . . .. . .. .. . .. . 35
5.5 VALIDITY OF DATA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . ... . .. . 35
5.6 COLLECTION AND FLOW OF RELIABILITY DATA . .. . . . .. . .. .. . .. . 36
5.7 ANALYSIS OF DATA . . . . . . . . . . . . . . . . . . . . . . . . .. . . . .. . .... . .. . 36
5.8 FAILURE CLASSIFICATION . . . . . . . . . . . . . . . . . . . .. . . . .. . .... . .. . 37
5.9 INTERPRETATION AND PRESENTATION OF DATA . .. . . . .. . .. .. . .. . 37

APPENDICES

A TERMS FOR RELIABILITY AND MAINTAINABILITY . . . . . . . . . . . . . . . . . 38

B EXPLANATION OF RELIABILITY AND MAINTAINABILITY
TERMINOLOGY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
C INDEX OF TERMS FOR RELIABILITY AND MAINTAINABILITY . . . . . . . . . 45
AS 3960—1990 4

FOREWORD

A standardized systematic approach to reliability and maintainability of manufactured and constructed

products is essential to ensure that pertinent statements are consistent and unambiguous in all
communications. As well as providing a basis for the preparation of reliability and maintainability
programs, this Standard can be used in preparing documents concerned with the specification of reliability
and maintainability or the reporting of reliability and maintainability data or tests. Maintainability is
applicable for systems and equipments that are repaired following failure.
It is vital that reliability and maintainability be considered in a manner similar to the other characteristics
of a system, equipment or item, from its design conception to the end of its working life, and that the
reliability and maintainability experience of one generation of products, engineers or technology is used
by the next. This requires that each factor affecting reliability and maintainability at any stage in the life
of an item be identified and considered in its relationship to the other factors. A reliability and
maintainability program is the formalization of this procedure. It may be necessary to divide such a
program into two parts, one of which may be the concern of the designer and manufacturer, and the other
the concern of the user. It is most important that consideration be given to the reliability and
maintainability of items or parts which are used by one manufacturer but produced by another, i.e.
subcontractor evaluation and vendor appraisal (see AS 1821, AS 1822, AS 3901 and AS 3902).
Satisfactory reliability and maintainability have traditionally been achieved by the use of codes of practice
or methods of working that have been shown by experience to give good results for established products.
However, with the increase in pace of change and performance expectations, it is no longer sufficient to
consider reliability and maintainability in purely qualitative terms when their achievement often demands
the specification of quantitative criteria. Accordingly, the onus should be on all parties (i.e. the designer,
supplier, purchaser or any other responsible authority) to specify quantitative requirements wherever
possible.
With the need to satisfy competing constraints, including stricter health and safety requirements, the advent
of new materials and pressure to reduce costs, reliability and maintainability should be expressed in
quantitative terms if they are to be given their proper weight. For example, a product may be required to
achieve at least a certain useful life, not to exceed a certain failure rate or to have an extremely high
availability. Quantitative statements relating to reliability and maintainability should then be made which
may be accompanied by stipulations that certain codes of practice be followed.
One aspect of quantitative reliability and maintainability that requires emphasizing is that often reliability
and maintainability can be predicted even for products of new design and high cost, such as a ship’s
propulsion system or a new kind of nuclear reactor pressure vessel, but only if there is sufficient
knowledge of the processes leading to failure, including data based on the statistical analysis of
observations made.
The designer’s or user’s confidence that reliability and maintainability requirements will be met depends
on the following:
(a) Adequacy of the data about failures.
(b) Correct interpretation of such data in the existing environment.
(c) A disciplined approach to the specification and design of the product or service.
Data should be obtained about failures from the following sources:
(a) Reliability testing.
(b) Field data.
(c) Knowledge of the physical processes leading to failure.
A reliability and maintainability program sets out to provide products with acceptable levels of reliability
and maintainability at an acceptable cost rather than products with improved reliability and maintainability
regardless of cost. For this reason the quantitative approach is essential. The qualitative approach can be
effective in improving reliability and maintainability but the improvement can not be quantified.
In general terms and philosophy this Standard is equally applicable to all parts of a system, including the
software. The subject of software reliability and maintainability is considered to be a separate discipline,
and the techniques of software reliability and maintainability are not dealt with in detail in this Standard,
except where they interface at the system level.
 5 AS 3960—1990

However, it is important to recognize that software as a product is different from all other engineering
products in that it is not tangible, and it does not wear out. Software exhibits faults, but, unlike hardware
faults, they originate in undetected errors in the software specification, design logic and the coding process.
These errors generate faults only when that part of the program is executed and specific (input) conditions
exist. The errors mature into failures when they affect the response of the system. Such software-induced
system failures exhibit time dependency because the range of system inputs may vary with time, and the
process of a mistake maturing into a failure is of uncertain duration and depends on the application of the
system. Software reliability and maintainability assurance is therefore not a question of showing, avoiding
or compensating for changes such as wear, but of attempting to detect or limit the effect of mistakes in
design.
While systems have always had a logic content, the advent of computers has afforded the opportunity to
incorporate very large, complicated and often subtle logic structures. These structures often exceed the
capability of the human mind to perceive them in total and thus the difference in scale of the logic now
possible brings a greater chance of human error at all stages of the life cycle.
AS 3960—1990 6

STANDARDS AUSTRALIA

Australian Standard
Guide to reliability and maintainability program management

SECTION 1. SCOPE AND GENERAL

1.1 SCOPE. This Standard provides guidance on 1.2 REFERENCED DOCUMENTS. The following
reliability and maintainability program management documents are referred to in this Standard:
of manufactured and constructed products. It
AS
discusses the essential features of a comprehensive
program for the planning, organization, direction and 1057 Quality assurance and quality control—
control of resources to produce systems, equipment Glossary of terms
and components which will be reliable and 1199 Sampling procedures and tables for
maintainable. In management terms it is concerned inspection by attributes
with what has to be done, and why, and when and
how it has to be done, but it cannot be specific about 1821 Suppliers quality systems for design,
who should do it and where, because organizations development, production and installation
and projects vary widely.
1822 Suppliers quality systems for production
Section 2 reviews the essential features of a and installation
comprehensive reliability and maintainability
program, setting out a logical framework in which the 2490 Sampling procedures and charts for
activities described in the other Sections can take inspection by variables for percent
place. defective
Section 3 describes the steps that should be followed 2529 Collection of reliability, availability and
when a specification of reliability and maintainability maintainability data for electronics and
is drafted. The Section provides guidance on the similar engineering use
inclusion of reliability and maintainability clauses in
specifications relating to the performance, 2530 Presentation of reliability data on electronic
construction, testing and installation of manufactured and similar components
products, and indicates the nature of the reliability
and maintainability statements appropriate to each 2990 Quality systems for engineering and
specification. construction projects
Section 4 is an introduction to the means by which 3900 Quality systems—Guide to selection and
quantitative values can be assigned to the reliability use
and maintainability of systems, equipment and
components at various stages in their life cycle, and 3901 Quality systems for design/development,
the factors that influence that assignment. production, installation and servicing
Section 5 describes the processes of assimilation and 3902 Quality systems for production and
utilization of the data on which assessment and installation
prediction of reliability and maintainability are based.
BS
Appendix A provides the user with a supplementary 5760 Reliability of systems, equipments and
list of terms for reliability and maintainability, taken components—
from IEC Publication 271, which are not found in Part 2: Guide to the assessment of
AS 1057. reliability
NOTES:
IEC
1. Appendix B provides an explanation of reliabili ty and
maintainabil it y terminology. 271 List of basic terms, definitions and related
mathematics for reliability
2. Appendix C provides a complete index of terms for reliabili ty
and maintainabili ty.
3 Many but not all of the program elements described in this 1.3 DEFINITIONS. For the purpose of this
Standard are appli cable to the achievement of reli abil it y and Standard the definitions given in AS 1057 and the
maintainabil it y of services. terms listed in Appendix A of this Standard apply.

COPYRIGHT
 7 AS 3960—1990
SECTION 2. RELIABILITY AND MAINTAINABILITY
PROGRAM
2.1 GENERAL.
2.1.1 Life cycle concept. The reliability and
maintainability activities to be implemented for each
phase of the product life cycle need to be selected
within the context of the total life cycle of the
product. Decisions made at any time have an impact
on product reliability, maintainability and cost at that
time, and in subsequent phases of the product life.
The product life cycle has a number of distinct phases
as follows:
(a) Definition phase—in which the concept and the
need for the product are decided and its basic
requirements are defined, usually in the form of
a product specification agreed upon between
manufacturer and user.
(b) Design and development phase—in which the
product hardware and software are created to
perform the functions described in the product
specification. This phase will normally include
the assembly and testing of a prototype product,
under laboratory simulated conditions or in
actual field trials, and the formulation of detailed
manufacturing specifications and instructions for
operation and maintenance.
(c) Production phase—in which the design is put
into production.
(d) Installation and commissioning phase—in which,
in the case of large complex products, the
installation of the product on a particular site
may be regarded as an extension of the
production phase.
(e) Operation and maintenance phase—in which the
product is operated for the period of its useful
life. During this phase, essential preventive and
corrective maintenance actions are taken and
product performance is monitored. The useful
life of a product ends when its operation
becomes uneconomic because of increasing
repair costs or other factors, or the product
becomes technically obsolete.
(f) Disposal phase—in which the product reaches
the end of its planned useful life, or the
requirement no longer exists for the product, and
it is disposed of, destroyed, or, if economically
feasible, modernized.
2.1.2 Aim of a reliability and maintainability
program. The aim of a reliability and
maintainability program as part of a quality system is
to ensure that adequate and effective effort is brought
to bear on reliability, and maintainability (for
maintained products), during all phases of the
life-cycle of an item; and to ensure activities which
contribute to reliability and maintainability are
properly integrated with other contract/specification
activities. The program should provide continuous
study of both quantitative and qualitative
requirements throughout all phases of a project;
reliability and maintainability assessments should be
updated; specified requirements should be verified;
and the reliability and maintainability activities
should be integrated with other elements of the
development, production and operation program.
COPYRIGHT
In many projects there are no sharp demarcations
between the definition, design and development,
production and operation phases. However, for
clarity, distinctions are drawn in this Section. The
activities appropriate to the different phases are as
given in Table 1.
Activities are also shown in flow diagram form in
Figure 1. They are explained in the text of this
Section in relation to overall project management
responsibilities and time scale. Some of these
activities have a direct bearing on the reliability and
maintainability achieved in service but they are
contained in other program elements. The reliability
and maintainability program should therefore be
prepared and implemented in conjunction with the
other parts of the quality program and safety
program.
NOTE: Examples of requir ements for a quali ty system are given
in AS 1821, AS 1822, AS 2990 and AS 3900.
2.1.3 General considerations on maintainability.
In common usage, maintainability is concerned with
two very different concepts—
(a) hardware; where the purpose is to retain the item
in, or restore it to its required state; and
(b) software; where the purpose is to correct
inadequacies.
In hardware systems, maintainability has a direct
influence on system availability. Reliability and
maintainability should be planned together throughout
design and development and integrated wherever
possible to maintain a balance with life cycle costs.
In systems containing software, maintainability refers
to the ease with which the software can be modified
in order to bring it in line with the original
requirements. The maintainability program should
ensure that software maintenance can be performed
with the minimum effect on the reliability of the
system.
2.1.4 Cost considerations. It may be neither
necessary nor desirable to fully implement all the
activities described in this Section. A suitable
compromise should be reached whereby reliability
and maintainability needs are balanced against all the
other factors. Whole life costs are the sum of the
initial acquisition costs and costs of subsequent
ownership. For complex equipment the ownership
cost can be a high proportion of whole life costs.
Furthermore, a properly integrated reliability and
maintainability program, as illustrated in Figure 2,
can show an overall cost benefit to the manufacturer
in terms of reduced costs of technical support, repair
and replacement, and warranties. It can also lead to
increased competitiveness and goodwill and have a
vital significance in the prevention of product liability
claims. It follows, therefore, that there should be
continuous communication between the design
authority/manufacturer and the customer/user after the
customer has formulated his reliability and
maintainability requirements. The procedures set out
in this Section give guidance on implementing an
effective reliability and maintainability program,
which should be integrated with design, development
and production activities, thus allowing its overall
AS 3960—1990 8

objectives to be achieved in the most cost-effective
way. Maintainability is only applicable to systems
and equipment that are subject to repair and
maintenance. The product development and
production activity decision making process should
consider the impact of—
(a) the complexity, the development risk and
function of the product, and the hostility of the
environment in which it is to be used;
(b) the availability of reliability and maintainability
data;
(c) the value of the appropriate reliability and
maintainability (if applicable) characteristic for
the principal phases in the total life;
(d) demonstration and guarantees required;
(e) facilities, workload and schedule;
(f) reliability and maintainability aspects of
operation and safety;
(g) maintenance, logistics support and future
requirements;
(h) design, development and manufacture; and
(i) ownership costs, e.g. operation, maintenance,
energy, etc.
2.1.5 Relative effectiveness of program activities.
It is not possible to predict accurately the relative
effectiveness of each activity on improving reliability
and maintainability. The choice of activities and the
resources to be expended on each should be based on
past experience of similar projects. The following
statements are offered as a general guide:
(a) The earlier a design change can be agreed, the
lower the total cost is likely to be. The cost of
making a design change at the production or
usage stage is normally many times the cost of
doing so at the initial design and development
stage.

TABLE 1
RELIABILITY AND MAINTAINABILITY PROGRAM ACTIVITIES DURING
THE PRINCIPAL PHASES OF A PROJECT
Phase Acti viti es Subclause
Defi niti on Feasibil it y study 2.2.1.1
Statement of reli abil it y and maintainabili ty 2.2.1.1
objectives and requir ements
Reli abil it y and maintainabili ty specifi cati on and 2.2.1.2
contract formulation
Design and development Analysis of part s, materi als and processes 2.2.2.3(b)
(i ncluding init ial Analysis of established and novel features 2.2.2.3(c)
manufacture) Fail ure mode, eff ect and crit icalit y analysis 2.2.2.3(d)
Incident sequence analysis (fault tree analysis) 2.2.2.3(e)
Stress and worst case analysis 2.2.2.3(f)
Reli abil it y prediction 2.2.2.3(g)
Redundancy analysis 2.2.2.3(h)
Human factors 2.2.2.3(i)
Design review 2.2.2.3(j)
Design audit 2.2.2.3(k)
Design change control 2.2.2.3(l)
Maintainabil it y analysis 2.2.2.3(m)
Maintainabil it y predicti on 2.2.2.3(n)
Maintainabil it y design crit eria 2.2.2.3(o)
Safety program 2.2.2.4
Test plans 2.2.2.5(a)
Part s and sub-assembly testing 2.2.2.5(b)
Perf ormance and envir onmental testing 2.2.2.5(c)
Accelerated testing 2.2.2.5(d)
Endurance testing 2.2.2.5(e)
Reli abil it y growth testi ng 2.2.2.5(f)
Development reliabili ty demonstrati on testi ng 2.2.2.5(g)
Maintainabil it y test and demonstrati on 2.2.2.5(h)
Data coll ecti on, analysis and feedback 2.2.2.5(i)
Producti on Preservati on of reli abil it y achievement 2.2.3.1
Qualit y conformance veri fi cati on 2.2.3.2
Screening (r un-i n, bed-in or burn-i n) of components
and assemblies 2.2.3.3
Producti on reliabili ty demonstrati on testing 2.2.3.4
Maintainabil it y in producti on 2.2.3.5
Additi onal software check 2.2.3.6
Installati on and System acceptance 2.2.4.1
commissioning Commissioning tests 2.2.4.2
Reli abil it y growth 2.2.4.3
Reli abil it y and maintainabili ty demonstrati on 2.2.4.4
Data coll ecti on 2.2.4.5
Reli abil it y and maintainabili ty assessment 2.2.4.6
Operation-Usage and Data coll ecti on, analysis, feedback and redesign/ 2.2.5.1
maintenance modifi cati on
Maintenance 2.2.5.2

COPYRIGHT
 9 AS 3960—1990

FIGUR E 1 RELIABILITY AND MAINTAINA BILITY PR OGRA M CONCE PT

COPYRIGHT
AS 3960—1990 10

FIGUR E 2 TYP ICAL RE LIAB ILITY AN D MAINTAINAB ILITY PROGRAM

COPYRIGHT
 11
(b) For complex items, design evaluation and
review, use of proven parts and processes, and
the use of redundancy or derating within the
design can greatly enhance overall system
reliability and maintainability. Large scale use of
system redundancy can greatly improve overall
system reliability and maintainability, but at an
increase in initial cost and maintenance load. It
should be noted that the useful system
redundancy may be limited by common mode (or
common cause) failures.
(c) Reliability testing, including corrective action,
carried out during the development and
production phases, is an effective means of
increasing the reliability and maintainability of
complex products as well as increasing
confidence that the required level has been
attained.
(d) Reliability improvement depends on several
factors, such as the effectiveness of procedures
for identifying, reporting and taking action on
failures, the way the program is managed and the
use to which failure analyses are put.
(e) The achievement of good maintainability in
design depends upon the thorough analysis of the
requirement through prediction, modelling and
allocation, followed by adherence to specific
design techniques. Maintainability demonstration
is applied iteratively during the design and
development phase to provide confidence that
maintainability requirements will be met.
2.1.6 Training. Personnel should have training and
experience relevant to their reliability and
maintainability tasks.
Reliability and maintainability disciplines should be
incorporated in training programs for personnel who
will be responsible for operating, maintaining and
supporting the product. The training and support
programs should consider the types and levels of
training that will be needed by the different
personnel.
Training requirements should be planned early in the
reliability and maintainability program. Training and
support programs should be co-ordinated with product
development planning, maintenance policy, and the
operational requirements of the product.
2.2 PROGRAM ACTIVITIES.
2.2.1 Definition phase.
2.2.1.1 Feasibility study and statement of objectives
and requirements. At the definition phase, the
originator of the requirement should consider the
reliability and maintainability characteristic that will
be required of the product in its different phases, and
the factors which will influence this. These factors
include—
(a) the complexity of the product;
(b) the state of development of similar products and
of the parts to be used including the availability,
accuracy and relevancy of reliability data;
(c) the method of use of the product, such as duty
cycles, maintenance, frequency of inspection and
expected life;
(d) the expected environmental conditions, which
should include operating and non-operating
conditions, packaging, transport and storage;
COPYRIGHT
AS 3960—1990
(e) the constraints imposed by other requirements,
e.g. performance, size, mass, safety and costs;
(f) provision for changes in requirements during the
life of the product; and
(g) maintenance and logistic support philosophy.
For complex products it may be necessary to carry
out a preliminary study to assess the reliability and
maintainability likely to be attainable under the
conditions expected, in order to set a sensible target.
This study should be in accordance with Section 4.
2.2.1.2 Specification and contract. When the
feasibility of each relevant reliability and
maintainability characteristic has been determined and
its value set and agreed on, it should be stated as the
specified reliability and maintainability that is to be
achieved and demonstrated for the product. This
requirement or objective should then be included in
a specification or contract. The specification should
take into account factors such as those given in
Clause 2.2.1.1 of this Standard, in as much detail as
is necessary, and it should be drawn up in accordance
with Clause 2.1.1. Arrangements should be made in
the overall program to allow for the reliability and
maintainability elements to be reviewed in terms of
the overall program, at predetermined intervals.
2.2.2 Design and development phase.
2.2.2.1 Establishment of product design guidelines
and codes. Design guidelines and codes should be
compiled for use by design and development
engineers. These will include the producer’s and
customer’s preferred or required design disciplines
and maintenance policy, and a listing of all relevant
Industry and Government Standards and Regulations,
Codes and Practices.
2.2.2.2 Evaluation of costs. Consideration of
product cost is a significant element of each product
design and development program. An evaluation of
life cycle cost of design alternatives should be part of
the program.
2.2.2.3 Analyses and other activities.
The reliability and maintainability plan should
establish a preliminary schedule for these analyses
which takes into account major program milestones
and the costs of performing these analyses.
Analyses and other activities are as follows:
(a) Allocation of reliability and maintainability
objectives. Reliability and maintainability
targets should be allocated to subsystems so that
the system’s reliability and maintainability
requirements can be achieved in an optimal
manner; considerations, for example are–product
development time, system availability and life
cycle cost.
(b) Parts, materials and process analysis. The
design stage should include an analysis of the
reliability and maintainability implications of all
parts, materials and processes called up in the
initial design. As far as practicable, parts or sub-
assemblies whose reliability and maintainability
characteristics (at the stress levels expected) are
known, should be selected. Wherever possible,
the number of different part types should be
minimized by use of standard products.
For systems containing software, the designer
should select (based on known reliability and
maintainability data) existing software, proven
languages and compilers, etc.
AS 3960—1990
(c) Established and novel features analysis. The design
should be analysed to identify those features that
are established practice and those that are novel,
including features that use novel components, novel
software techniques, etc or use established
components in novel ways. The reliability and
maintainability of each innovation in the design
should be assessed by analysis or testing in order to
justify the proposed innovation objectively.
(d) Failure mode, effect and criticality analysis
(FMECA). The design should be analysed in order
to determine possible modes of failure and their
effects on system operation. The primary objective
of such analysis is to discover critical failure areas
and design characteristics. It can be carried out
either by starting at the system level and expanding
downwards, or from the component level upwards,
to a level commensurate with necessity, time
available and required assurance. Within the
analysis, each potential failure should be considered
in the light of the probability of occurrence and
categorized as to its probable effect on the
successful operation of the system or item.
This will aid in allocating resources for corrective
design action, reliability and maintainability, and
development engineering. Such analysis should be
a major consideration in design reviews and should
always be used to provide acceptance criteria for
test planning and the establishment of check out and
diagnosis procedures when required. Depending
upon the nature of the system, major sub-units may
require separate analysis, often to meet specific
legal or safety code requirements, e.g. ships,
bridges, aircraft, high pressure vessels in process
plants and real time computing systems.
Elements of the analysis are as follows:
(i) Failure mode analysis—a study of the system
and the working interrelationships of its
components under various anticipated
conditions of operation (normal and abnormal)
in order to determine probable failure location,
mode and mechanism.
(ii) Failure effect analysis—a study of the
potential failures that might occur in any
section of the system, in order to determine
the probable effect of each on all other
components, or sections, and on operational
success.
(iii) Failure criticality analysis—a study of the
potential failures in any section of the system
in relationship to other sections of the system,
in order to determine the severity of each
failure effect in terms of a probable safety or
environmental hazard, unacceptable
degradation of performance, or loss of
operation or availability of the system.
(iv) Fault recovery analysis (applicable to
software)—a study of the extent to which a
fault can be bypassed or corrected
automatically.
(e) Incident sequence analysis (fault tree analysis). In
cases where a major system failure can be
identified, it may be helpful to use a fault tree
analysis to identify possible causes. This consists of
an analysis of possible causes starting at system
COPYRIGHT
12
level and working down to component level,
identifying all possible causes.
NOTE: This technique is often used for the development of
maintainability programs and maintenance procedures.
(f) Stress and worst case analysis. This is an analysis
to ensure that the design will not cause
overstressing of components due to specified (or
postulated abnormal) combinations of input
conditions, output loading, environmental conditions
and parts’ tolerances. This has the purpose of
ensuring that safety and reliability and
maintainability margins are adequate. Stress
conditions may be specified by regulating codes of
practice, e.g. boiler and pressure vessel codes,
lifting and winding gear codes, civil air worthiness
and in-flight requirements and motor vehicle safety
legislation requirements. A complete analysis can be
time consuming and costly, and priority should be
given to possible trouble areas indicated by the
failure mode, effect and criticality analysis.
NOTE: In systems containing software the analogous activity
will be ensuring that, under conditions of specified (or postulated
abnormal) stimuli from the outside world, no part of the system
will operate outside permitted time bounds, capacity bounds, etc.
(g) Reliability prediction. Using an appropriate
mathematical model the relevant reliability and
maintainability characteristics of the product should
be predicted from published, laboratory, field parts
or subsystem reliability and maintainability data, for
the purpose of providing an indication as to whether
the product’s target reliability and maintainability
specification can be met.
(h) Redundancy analysis. An analysis to ascertain the
need for redundancy should be performed on design
of items, failure of which may be critical in terms
of safety, costs, down time, etc. The analysis
consists of a review of any aspect of the design that
can be replicated in order to provide an alternative
means of performing the required function in case
of failure. This analysis should consider the possible
options in terms of effect on overall reliability and
maintainability, balanced against other features such
as cost, size and mass. The type of redundancy
selected will be determined largely by the effects of
failures against which redundancy is provided, in
which case the designer should consider the
provision of indicators, warnings or alarms to draw
attention to a failure on one path. This is essential
in the case of critical items where redundancy is
active or automatic. The effect of tandem working
upon individual item reliability and maintainability
(reduced stress), the increased maintenance load and
the reliability and maintainability of changeover
should also be considered in reaching a decision on
whether to add redundancy and, if so, what form it
should take (active or standby).
(i) Human factors. The possibility of human error
leading to unreliability and maintenance errors
should be considered over and throughout the life
cycle of the product. In particular, careful
consideration should be given to the way in which
reliability and maintainability of an item or system
may be reduced by the following:
(i) Communication. Good communication is
essential for reliability and maintainability
throughout the life-cycle of a system. One of
the most difficult and sophisticated communi-
 13
cation phases occurs when a design
specification is being created, when it
should be visible or communicable to those
involved and affected, e.g. designers,
maintainers, and quality controllers.
(ii) Design. Care should be given to the
structure of the design, the design teams
and the design interface so that at every
level the logic of the design can be
perceived. Nevertheless, in the design phase
human error produces faults. To eliminate
such faults, methods of fault detection and
well-structured procedures are required e.g.
drawing inspection, computer program
walk-through (check), simulation and task
analysis.
(iii) Operation. Careful consideration should be
given to the way in which the reliability
and maintainability of the product may be
influenced by the performance and attitude
of the operator. This requires an analysis of
the operator’s function and information
needs, the way in which the information is
provided, the operation response required,
the physical actions required to respond and
the response time allowed. Where the
analysis reveals unsatisfactory features,
design changes may be necessary. In some
cases it may be possible to quantify the
element of human (personnel) unreliability
and maintainability in the total system, but
in general this analysis is qualitative.
(j) Design review. The design program should
include formal reviews at appropriate stages of
design, development, production and in-service
usage, to evaluate achievement of the reliability
and mai nt ai nabi l i t y and mai nt enance
requirements. The review(s) should be a formal
systematic study of the design, to be carried out
jointly by specialists from the supplier’s(s’) and
purchaser’s organization(s) and should take
account of such considerations as—
(i) current reliability and maintainability
estimates and achievements, identification
of principal items inhibiting reliability and
maintainability, and reviews of reliability
and maintainability effort;
(ii) potential design/production/installation
problem areas;
(iii) reliability and maintainability analysis
reports;
(iv) proposed design trade-offs, solutions to
reliability and maintainability problems,
and status of previous review actions;
(v) effects of reliability and maintainability
engineering on design;
(vi) the extent of software in the system and the
effect on reliability and maintainability of
the design approach adopted for the
software; and
(vii) evaluation of significant differences
between the present and proven designs.
Minutes should be kept of these formal reviews,
and provision should be made for the issuing of
progress reports.
COPYRIGHT
AS 3960—1990
(k) Design audit. A critical examination of the
design should be made in order to ensure that
there will be an acceptable level of reliability
and maintainability in operational use. It should
be carried out by engineers independent of the
design process, and should cover all aspects of
the manufacturing process from the design
concept to testing, installation, operation, and
degradation of performance in service and
maintenance. The audit should identify any
design weaknesses requiring modifications but it
should not offer a solution to the problems; the
latter is the province of the designer.
(l) Design control and design change control. A
systematic procedure for review of evaluations,
analyses, appraisals and assessments of each
change during the evolution of the design should
be established to ensure that ‘reliability and
maintainability growth’ during initial design and
development phases is shown to be adequate and
satisfactory. All changes to the design after it
has reached the point of release for production or
installation are regarded as engineering changes.
These also should be governed by design change
control measures to ensure reliability and
maintainability is not unacceptably reduced by
later design changes made for reasons not
concerned with reliability and maintainability,
e.g. ease of production, value analysis and value
engineering.
(m) Maintainability analysis. Maintainability
analysis uses information from feasability
analyses, failure mode and effect analyses, and
trade off studies to provide design guidance for
maintainability. The analysis should include
assessment of accessibility, interchangeability,
modularity, standardization, operator/maintainer
requirements, test and maintenance requirements,
spares provisioning and maintenance policy.
(n) Maintainability prediction. Maintainability
prediction enables an early assessment of the
maturity of the design and enables early
decisions concerning the compatibility of a
proposed design with specified requirements or
the choice of other alternatives. Mathematical
models are available to aid the prediction
process.
(o) Maintainability design criteria. This involves
the development and application of design
criteria and guidelines, with the following aims:
(i) Providing adequate accessibility, work
space, and work clearance.
(ii) Reducing the need for and frequency of
maintenance activities.
(iii) Reducing maintenance downtime.
(iv) Reducing maintenance support costs.
(v) R e du ci ng ma i n t e na nc e p er so nn el
requirements.
(vi) Reducing potential for maintenance error.
(vii) Providing a built-in test capability.
NOTE: The above describe those activities which, during the
design and development (including initial manufacturing) stages,
can be carried out by the supplier, by an independent assessor,
or may be undertaken by the purchaser. Because of the
complexity of some systems, the level at which the above are
carried out should be carefully selected, taking into account
costs, complexity of analysis and potential benefits.
AS 3960—1990
2.2.2.4 Safety program. Where it is necessary to carry
out a separate safety program, the reliability and
maintainability analyses should be used to provide
inputs. Much duplication of effort can be avoided if
common inputs and methods are used wherever
practicable.
A safety program is a sequence of activities conducted
during design, development, production, installation and
usage, aimed at identification, elimination or control of
hazards.
2.2.2.5 Testing and demonstration. Reliability and
maintainability verification tasks are carried out during
the design and development phase to verify the adequacy
of the design. They include both analysis and testing,
taking into consideration the product performance history
and the associated hardware and software failure
characteristics, where applicable. Types of tests are
outlined as follows:
(a) Test plans. Testing is a normal part of any product
development for many reasons, e.g. performance,
function and ergonomics. The function of reliability
and maintainability program management is to make
maximum use of all such testing, and to determine
the amount and type of additional testing necessary
both for reliability growth and for reliability and
maintainability demonstrations. There are, however,
special cases where reliability testing and
maintainability demonstration is not possible before
the item goes into service; reliability and
maintainability can then only be predicted from an
adequate knowledge of statistical data or the physics
of failure. This procedure is possible in principle,
but it may be difficult in practice.
Methods of dealing with this situation are discussed
in Section 4. All testing should be integrated and
monitored centrally, and a uniform data collection
and presentation system should be applied.
A test plan should be prepared which identifies the
aim and procedures of the test and the required
facilities and resources, together with a time
schedule and assigned responsibilities. It is essential
to appreciate that every success and failure that
occurs is relevant reliability and maintainability
information, but that, to make full use of this data,
each such event should be fully documented. All
failures on test should be fully analysed even if,
superficially, they seem to be caused by factors not
related to the product under test.
It is also essential to appreciate that testing to
expose design weaknesses should be planned to
generate failures. Weaknesses will only be
demonstrated by failure, and therefore a test that
does not generate any failures has not provided any
information on which to base improvements. All
tests should therefore be as severe as can be
considered compatible with the planned operation of
the products, and the maximum possible test time
should be spent at the limits of expected
environmental stress, or at overstress (see
Section 4).
When testing systems that contain software, it is
important to realize that a large proportion of the
faults may be built into the software during the
design and development phase. Testing should
therefore be organized to stress the design in ways
likely to demonstrate errors.
COPYRIGHT
14
(b) Parts and sub-assembly testing. The results of tests
on components or sub-assemblies during
development should be considered applicable to the
reliability and maintainability program, and should
be covered by the data collection system. Particular
attention should be given to components or
sub-assemblies whose failure is critical.
(c) Performance and environmental testing.
Performance and environmental tests, often referred
to as ‘type tests’ or ‘qualification tests’, are part of
most development work, and are often called for by
mandatory parts of specifications and sometimes by
statutory obligations. However, though they can
provide some useful success and failure data, they
are not usually of long enough duration, nor are the
samples large enough, to provide high confidence in
reliability and maintainability. Type tests should,
therefore, be considered as part of the reliability and
maintainability program in so far as they produce
information which can be combined with that
obtained from tests of longer duration to modify or
supplement the statistical data, and provide a basis
for reliability and maintainability improvement
modifications.
(d) Accelerated testing. Products may be subjected to
accelerated testing, in which stresses applied are
more severe than those encountered in normal use,
in order to speed up ageing and thereby obtain
degradation and failures in less time. It is a
technique that reduces testing time, and hence costs.
These tests are also valuable when used
qualitatively, e.g. to validate a failure mode, effect
and criticality analysis. Two methods of applying
this form of testing are usual: constant stress and
step stress. Care should be taken in the
categorization of failures induced by the accelerated
test, as these modes may be different from those
likely to be experienced under expected operation
conditions, and therefore they may not be relevant.
Thus again, it is important that the physics of
failure be understood.
A common way in which computer system testing
is accelerated is to increase the rate of activity of
input conditions so that the probability of
occurrence of events that may cause failure is
increased (as opposed to actually overloading the
system which is a changed mode of failure). In
testing plant control systems, for instance, the
operation of warning sensors may be simulated in
many combinations over, e.g. a two-hour period,
where in practice they may only be rarely brought
into play, e.g. monthly or at even longer intervals.
(e) Endurance testing. Wherever endurance testing is
applied as part of a development program to prove
operational performance over a period or to obtain
wear-out or fatigue information, the results should
be used to augment the failure data file. It should be
possible to combine these tests with reliability
demonstration tests, either by superimposing the
testing or by combining the data statistically if the
test environments and samples are similar or
identical.
It should, however, be remembered that the
reliability characteristics of development models
may undergo significant changes in consequence of
the test-analyse-and-fix (TAAF) program (see
Clause 2.2.2.5(f)). In the case of systems containing
 15
software, endurance testing may be carried out to
identify the successful elements and the failures
in the software. The method of data
accumulation has to be spread out over a period
of time on one sample, rather than over several
samples.
(f) Reliability growth testing. It is often necessary
to supplement general development test activities
by allocating units specifically for the purpose of
reliability growth testing. The object of such
testing is to reveal sources of failure and to
promote reliability improvements or growth by
progressively eliminating them. The process is
also known as ‘test-analyse-and-fix’.
The allocated products are operated in an
appropriately stressed environment for an
extended period and are monitored frequently.
When failures occur, the units are repaired and
the test continues with separate failure
investigation and action, as required. A unit may
comprise any product ranging from a complete
system to a component. The principal steps in
the reliability growth testing procedures are as
follows:
(i) Stimulation of latent faults by exposure for
a sufficient period (possibly some hundreds
or thousands of hours) to an environment
that is severe but maintained with specified
limits, coupled with realistic functioning
and frequent monitoring of the product.
(ii) Diagnosis of each failure, repair of the
associated fault, and continuation of testing.
(iii) Analysis of each failure to determine, if
possible, the basic cause.
(iv) Elimination of failures by corrective action
including design modifications where
necessary.
(v) Proving the effectiveness of (iv) in
preventing further similar failures without
undesirable side effects.
(vi) Incorporation of approved modifications
into other products.
It cannot be over-emphasized that the success of
a reliability growth test depends as much upon
the feedback loop and follow-up action as upon
the test program (see Clause 2.2.2.5(i)).
(g) Development reliability demonstration testing.
This form of testing should be planned as an
integral part of the development stage of any
new product as it will provide early indication of
the likelihood that the item will pass the
production phase demonstration tests. It should
also be integrated with any reliability
assessment, analysis and evaluation. It is
essential that development products be made
available for testing, and this should be
considered in the planning or manufacture of
development models, and their allocation. The
decision as to how much reliability testing
should be carried out during development should
be based on the following:
(i) The amount of sub-assembly testing
planned or carried out.
COPYRIGHT
AS 3960—1990
(ii) The severity of the reliability requirement
in relation to the predicted reliability.
(iii) Economic considerations.
The test parameters to be applied (sample size,
test duration, decision rules for pass or failure of
test, environment, cycling, item parameters to be
measured) should be determined on the basis of
the production reliability demonstration
requirements (see Clause 2.2.3.4). It should be
remembered that the reliability characteristics of
development models may be significantly
different from those of production models.
The information obtained from the reliability
demonstration testing (including prototype or
type approval test, etc) should be used to update
the reliability assessment.
NOTE: Whilst it is possible to carr y out a development
reli abil it y demonstr ation test, in the case of systems
containing soft ware, no great confidence can be placed on the
result because of the problems wit h creati ng an appropriate
environment for the test.
(h) Maintainability test and demonstration. The
primary function of a maintainability test and
demonstration is to verify that maintainability
has been designed-in and built-in to a system or
equipment. Up to this point in development, the
elements of the maintainability program have
been analytical in nature, and do not reflect
practical experience with the actual hardware.
Therefore, it is essential to confirm the
maintainability analysis by testing the hardware
in an operational or simulated operational
environment, and by performing actual
maintainability tests and demonstrations
involving the prime equipment and its associated
logistic resources (i.e. support equipment, tools,
technicians, technical data). Statistical methods,
using relatively small samples of maintenance
tasks, can be used to provide statistical
confidence that the maintainability requirements
have been met.
(i) Data collection, analysis and feedback. The
extent of test documentation and data recording
will depend on any contract data requirements
and on management objectives. In any case the
reliability and maintainability program should
incorporate a system of reporting of test events
and results which ensures the traceability of data
and the documentation of conditions under which
the data was collected, the assumptions made,
the permissible preventive maintenance during
testing, the rules for determining which incidents
are to be considered as relevant failures and the
procedures for analysis of test incidents and for
the recording of corrective action.
The following is a basic check list for data
collection analysis and feedback:
(i) Date and time.
(ii) Nature of test (environmental conditions,
test equipment, test software configuration,
etc).
(iii) Component identification (type number,
name, serial number, revision or issue
number, etc).
(iv) Sub-assembly.
(v) Assembly.
AS 3960—1990
(vi) The operating history, e.g. in hours, cycles run
or throughput.
(vii) Mode of operation (if applicable), e.g. full
output, half output, or nature of function
performed.
(viii) Failure mode.
(ix) Failure cause.
(x) Failure effect.
(xi) Classification of failure (consequence and
severity, relevant or non-relevant, systematic
or random).
(xii) Corrective action for test continuation.
(xiii) Diagnosis and repair time (expressed as active
or total repair time or total down time).
(xiv) Cross-reference to previous occurrences of the
same type of failure.
(xv) Corrective action proposed to prevent
recurrence.
(xvi) Name(s) and position(s) of personnel
conducting the analysis.
To be effective, all data should be analysed, and the
analysis fed back to design and production so that the
necessary corrective action can be planned and
implemented as quickly and economically as practicable.
The analysis and particularly the categorization of
failures, in terms of relevance, responsibility and
proposed corrective action, should ideally be prepared in
conjunction with the appropriate design, development,
production and quality staff.
2.2.2.6 Transition from development: changes in
production method. Methods suitable for production
may differ from those used for the initial manufacture of
the prototypes or test trial items on which all the
development reliability testing has been done. Each such
change should be regarded as a design change and the
change control procedure of Clause 2.2.2.3(l) should be
used.
In systems containing software it is important to note
that changes of tools, e.g. compilers, may produce a
significantly different product (in terms of reliability).
2.2.3 Production phase.
2.2.3.1 Preservation of reliability achievement. The
reliability and maintainability program should state
procedures to be followed during the production of
systems and equipments in order to prevent departure
from the achieved and demonstrated reliability. The
proceduresdeveloped should include those manufacturing
process controls that are to be applied during production.
Procedures for reliability acceptance and endurance tests
should be included in the reliability program. Suitable
procedures should be established to control any
manufacturing process that affects product reliability.
Examples of such controls are quality conformance
verification measures and quality control procedures in
accordance with the agreed quality level.
2.2.3.2 Quality conformance verification. It is generally
accepted that quality control of components and
processes is essential in order to achieve and maintain
reliability. However, to ensure that quality control is
fully effective, it should be integrated with the reliability
program and its planning. Acceptance sampling
procedures for attributes and variables should be adopted.
COPYRIGHT
16
These are described in AS 1199 and AS 2490. Sampling
procedures are not relevant to software and 100% quality
conformance verification is recommended.
Documentation, terminology and methods of categorizing
failures should be common to all parts of the program.
In so far as quality control also attempts to identify and
take action on potential failures, in addition to those
which have already occurred, and to report on aspects
that affect properties other than reliability, other
procedures may be used. However, a close link should be
maintained, preferably through the failure analysis,
between the reporting of potential failures (e.g. material
faults prior to working) and actual failure reporting from
the field.
2.2.3.3 Screening (run-in, bed-in or burn-in) of
components and assemblies. The extent of screening
required, and the stage at which it is to be applied, may
be specified in whole or in part. The following points
should be considered in stating screening requirements:
(a) The screening performed at components level by the
parts supplier.
(b) The cost of repair or rejection at each stage of
manufacture.
(c) The processes involved at each stage of
manufacture.
(d) The stress levels to be applied: these should not be
lower than the stress levels applicable to the
production reliability demonstration (see
Clause 2.2.3.4).
(e) The duration of screening tests to be applied: this
should be dependent on the stress levels applied and
the nature of the failure rate/time curve.
As screening relates principally to the early failure
period, the data derived from it should not be added
to those derived from long duration testing for the
purpose of reliability assessment. The data should
be used, however, in the same way as any other
data for reliability improvement proposals and for
modifying manufacturing processes. If items
subjected to production reliability tests are subjected
to screening prior to the tests, it is essential that all
production items be screened to the same extent.
2.2.3.4 Production reliability demonstration testing.
Production reliability demonstration tests should be
planned as the assurance, on final release, that the
finished product has met the reliability specification
requirement, i.e. that the achieved and demonstrated
reliability requirement has been maintained prior to
delivery.
As with the development stage testing, products should
be selected for the tests as part of the production plan.
The testing should be as realistic as practicable and
should aim at reproducing the maximum stress levels and
operating conditions to be expected in customer service.
However, it is not always possible to obtain good
correlation between test and service reliability, and it
may be more economical to use accelerated testing
techniques (see Clause 4.6.3). Demonstrations should be
representative of the specified reliability parameter (e.g.
time, distance, cycles). The test specification should
include at least the following:
(a) The required sample sizes and sampling procedure
(see AS 1199 and AS 2490 for recommendations).
 17
(b) Full details of stress levels and environmental
and operating conditions.
(c) Types of test, durations, accept/reject criteria and
decision risks (confidence level).
(d) Parameters to be monitored.
(e) Maintenance procedures (corrective and
preventive).
(f) Specific failure definitions in amplification of
basic definitions.
The types of test may cover qualification tests of
early production items, sampling tests on later
production batches, and endurance tests to determine
the existence of any wear-out failure modes.
Clause 4.6 covers the use of demonstration test
statistics in the assessment of reliability.
2.2.3.5 Mai nt ai nabi l i t y i n product i on.
Maintainability is a design characteristic that is not
sensitive to the production process. Maintainability
program effort is minimal during the production
phase, in comparison to the reliability effort.
Maintainability demonstrations are seldom performed
cost-effectively during the production phase. Activity
is mainly directed towards—
(a) ensuring that maintainability achievements are
not degraded;
(b) implementing any maintainability enhancements;
and
(c) ensuring that modifications to the system or
equipment do not degrade maintainability.
2.2.3.6 Additional software check. It is important
that there is a plan to ensure that the production
phase for systems containing software does not
contribute to a reduction in reliability. The following
points should be considered, bearing in mind their
cost and complexity:
(a) Sample screening of the material of software
distribution.
(b) 100% validation of the software copying process.
(c) 100% validation that the hardware and software
complements match.
2.2.4 Installation and commissioning phase.
2.2.4.1 System acceptance. The installation phase of
the reliability and maintainability program should be
planned and controlled so that the reliability and
maintainability from the production phase is not
degraded. Procedures and instructions for conducting
acceptance inspection, testing of systems and
components by verifying compliance with the initial
specification and design should be provided.
2.2.4.2 Commissioning tests. Reliability and
maintainability testing on systems and sub-systems
should be performed to define existing weaknesses
and problem areas. This should involve several levels
of testing and, for complex installed systems, these
may include—
(a) cabling tests;
(b) sub-system functional tests;
(c) software validation tests;
(d) integration tests; and
(e) final system overall functional tests.
Test procedures should describe and control the
distinct areas of testing, specifying the test equipment
and calibration requirements. The test program itself
should describe in detail all adjustments and
COPYRIGHT
AS 3960—1990
operations to test the system successfully. The quality
system should provide assurance that the detailed
requirements for the commissioning tests are met, and
that the procedures laid down are followed precisely,
and any variations are properly recorded and
witnessed.
2.2.4.3 Reliability growth. Any design weaknesses
that become evident as a result of the commissioning
tests should be brought to the attention of the
designer so that the appropriate action may be taken.
Reliability growth implies that any corrections should
improve the final system overall reliability.
2.2.4.4 Reliability and maintainability demonstration.
Reliability and maintainability demonstration tests
should be primarily directed at demonstrating that the
reliability and maintainability specifications have
been achieved. The maintainability aspect should
further demonstrate the ease of maintenance, the need
for spares allocation and the prediction of mean times
to repair/restore.
2.2.4.5 Data collection. The test program should
contain detailed data sheets to ensure that all desired
data, both input and output, are recorded. The data
sheets should also include spaces for recording
information such as environmental conditions, dates,
precise system configuration, test engineer, quality
personnel identification, operational times, failure
events and other data to permit the reconstruction of
the tests if required.
2.2.4.6 Reliability and maintainability assessment.
The detailed data sheets should ensure that all failures
are recorded and that the failure reports provide
sufficient information for an analysis to define the
failure adequately. Inherent reliability weaknesses and
problem areas should be analysed and defined. These
should be reported to the designer for the appropriate
action to be taken.
2.2.5 Operation-usage and maintenance phase.
2.2.5.1 Information and data collection, analysis,
feedback and redesign/modification. During the
warranty, guarantee or hand-over stages of the
product, reliability and maintainability information,
such as in-service information, provides a vital
feedback function to the producer. The operation
phase of the item should be considered to be part of
the reliability and maintainability program, even
though this now depends mainly on the user. The
manufacturer, and in certain instances the user,
should consider the operation to be an extension of
reliability and maintainability demonstration testing.
Therefore, data collection, analysis and feedback
should be continued, and followed up by redesign and
modification, if further reliability improvement is
required. Even if it is not planned to introduce further
changes in the item, the analysis of operational
failure data can be used to ascertain the correlation
between reliability testing and operational failure
data, so that future demonstration specifications can
be better related to the operational requirement.
Ideally, failure reporting and analysis during
operation should be as comprehensive as during
reliability demonstration. In practice, however, most
field failure reporting systems give less than
satisfactory results. This is because of the difficulty
of obtaining meaningful data on the conditions under
which failure occurred.
AS 3960—1990
Even with the failed product in hand it can be
difficult to identify the causes which contributed to
the failure. It is important, therefore, that the
reporting system be structured so that the specific
information requirements are stated and not left to
chance.
For expensive products the cost of field failure
reporting and analysis can often be recovered by
further reliability improvement.
2.2.5.2 Maintenance. Maintenance of equipment in
operation obviously affects reliability, and
maintenance should be based on the reliability
characteristics of the item. Basic preventive
maintenance (lubrication, cleaning, inspection, etc)
reduces the rate and severity of such failure causes as
wear and corrosion. Preventive maintenance should
also be related to the existence of known
time-dependent failure modes (fatigue, wear,
COPYRIGHT
18
corrosion, etc) so that maintenance is minimized
while still providing adequate protection. Ideally,
such preventive maintenance should be performed just
prior to, or early in, the increasing failure rate period,
i.e. at the onset of wear-out. This can be evaluated
through condition monitoring.
For some types of engineering equipment subject to
fatigue, wear, corrosion, etc, the failure rate increases
from the beginning of usage, and in such cases the
most cost-effective preventive maintenance policy
should be determined by analysis of failure statistics
and other relevant data. However, optimum
preventive and corrective maintenance policies can be
formulated only if adequate reliability recording and
analysis are performed during operation. Where
systems contain software, corrective maintenance of
the software element will invariably result in a
change to the system design.
 19
SECTION 3. SPECIFICATION OF RELIABILITY AND
MAINTAINABILITY
3.1 GENERAL.
3.1.1 Types of specification. The form of a
reliability and maintainability specification can vary
considerably. It may comprise a statement that
defines the required test program to meet the
purchaser’s requirements, or it may be a formal
specification that defines the complete management
structure and sets standards for all aspects from
operating environment, demands for life testing, to
details defining environmental or functional testing.
A specification may, for example, be prepared by an
ultimate user, as a statement of the user’s needs, or it
may be a design specification prepared by a supplier
in conjunction with the user, as a detailed statement
of the quality level for production items.
The detail written into a specification depends on the
nature of the product, its purpose and the market for
which it is intended.
Reliability can be specified by a series of
specifications each of which derives from the
preceding stage as shown in Figure 3. This method
can be very effective because only the relevant
clauses appear in each document, and the reliability
requirements do not have to be sought elsewhere.
There is a risk, however, that this technique may
result in reliability clauses that do not clearly define
the standards required. The alternative is to prepare
a single document that covers all aspects of reliability
requirements, and is referred to generally as the
‘reliability specification’. The preparation of a
separate reliability specification in its simplest form
has the advantage of providing a check list that
ensures that no important features are omitted in the
preparation of any other specifications.
The risk of omission will be minimized by having
good liaison and review procedures to examine each
specification before it is published to ensure that it
is—
(a) precise;
(b) complete; and
(c) unambiguous, particularly as regards acceptance/
rejection criteria corresponding to the desired
functions and the reliability of the product in
question.
From the point of view of a system user, failure to
perform as required, or unreliability, can be
considered as being caused by either–
(i) a failure of hardware, user instruction, etc; or
(ii) a residual error in the design logic of the
systems–possible with hardware, more probable
with software.
As systems increase in size and complexity,
unreliability due to (ii) above is likely to be an
increasing percentage of the total unreliability
exhibited by the system.
It has to be emphasized that the design phase does
have a considerable impact on system reliability and
maintainability. To minimize unreliability due to
design errors, consideration should be given to the
application of a quality system to the design phase
COPYRIGHT
AS 3960—1990
such as one of those described in AS 1821, AS 2990
or AS 3901.
Calculation or estimation of a value for (ii) is at
present a particularly imprecise science. The
di ffi cul t y shoul d not , however, prevent
acknowledgement of the factor of unreliability due to
design error.
3.1.2 Purpose of reliability and maintainability
clauses. Reliability and maintainability clauses state
the reliability and maintainability required, without
which the item may not be procured or purchased.
They detail targets for design, development or
production, installation and commissioning, and state
the reliability and maintainability that is expected in
service.
Reliability and maintainability clauses normally
contain three elements, as follows:
(a) The objective or required value of the relevant
reliability and maintainability characteristics,
expressed in performance terms.
(b) The conditions of use, storage and maintenance,
and the life of the item, during which this
reliability and maintainability is required.
(c) The means by which the required reliability and
maintainability are to be, or have been, assured.
3.1.3 Qualitative versus quantitative approach to
reliability and maintainability. In the past,
reliability and maintainability clauses have often been
included in specifications in a qualitative rather than
a quantitative sense. Satisfactory reliability and
maintainability have been thought of as subjective
characteristics, achieved by the use of suitable codes
of practice or methods of working established over
decades of observed experience. For certain products,
such as simple consumer products, such a qualitative
approach may still suffice; for others, and especially
where a product is designed and developed to meet a
particular purchaser’s requirements and where
inservice availability and cost of maintenance are
significant, a quantitative approach is necessary.
In such cases it is essential to make quantitative
statements relating to reliability and maintainability;
for example, one product may be required to have not
more than a specified maximum failure rate, or
another may be required to be repaired in a specific
time frame without specialist test equipment. In
general, it is good practice to follow the steps
recommended in Clause 3.2.1 in all circumstances,
adopting a qualitative approach only in the special
cases where it is impossible, unnecessary or
uneconomic to be quantitative.
3.1.4 Quantitative reliability clauses. Before any
quantitative reliability statement can be made, the
following need to be specified:
(a) A task (time or other measure of usage), for
which a probability can be expressed.
(b) The performance at which the item’s function
ceases to be satisfactory, i.e. the criterion of
failure.
(c) The conditions under which the product is to
function.
AS 3960—1990 20

Target Function Product Materials Process Inspection Test A cceptance H andling, Installation U se Maintenance D isposal
specification specification specification specification specification specification specification specification storage and specification specification specification specification
transport (Manual) (Manual) (Manual) (No tice)
specification
Storage and use Limitation of D esign C hoice of Assembly C hecks during Endurance and Inspection and Procedures for U npacking and Instructions for Preventive and
conditions the features critical materials and methods, assembly/ environmental conformity/ packing and installation setting up, corrective
performance, for the components for treatments and construction tests reliability test transportation procedures operating, maintenance
Lifespan ratings and achievement of compromise procedures and on data necessary to ensure that w ith lowest risk using, procedures
(durability) characteristics the reliability between w hich minimize completion of Accelerated for product the conditions of introducing controlling and w hich give
to give a objectives and production/ the introduction details critical
tests FMECA acceptance, encountered by external adjusting the maximum
R eliability compromise requirements construction, of failure for failure analysis approval and the equipment causes of product which availability
characteristics between level convenience mechanisms certification do not exceed failure are consistent consistent w ith
(Failure rate, of function and and reliability U se of FMEC A R esistance to those for w hich w ith the reliability
MRBF , etc) reliability objectives Information misuse it is rated reliability characteristics
objectives objectives and
Maintainability Storage of The object is to give the lowest
and D efinition of parts to avoid protect the chance of
maintenance failure classes deterioration reliability misuse
objectives likely to induce objectives and
failure requirements in
R esistance to the target
Misuse specification

NOTE: This figure does not call up design or management techniques to be used.

FIGUR E 3 RELIABILITY AND MAINTAINA BILITY CONTEN T IN SP EC IFICATIONS FOR MAN UFAC TURE D AN D CONSTRU CTED PR ODUC TS

COPYRIGHT
 21
A product is designed, and the initial reliability
requirement is stated, to satisfy certain performance
criteria and conditions of use and maintenance. If
these criteria and conditions are not compatible with
those that occur during in-service use by the
customer, the quantitative reliability levels at each
stage will not necessarily bear any relation to each
other. Thus, at whatever product phase the
specification and its reliability clauses are written,
those who are working to its requirements should be
able either to control the ultimate conditions of use
and maintenance and the failure criteria that will be
applied, or to predict them with sufficient accuracy
and in sufficient detail.
Unless the ultimate conditions of use are controlled,
predicted, or arbitrarily chosen for assessment, a
quantitative measure of reliability cannot be made.
The only alternative is to use a purely qualitative
approach.
3.1.5 Problems in applying the quantitative
approach. Where adequate failure data are available,
there is normally no difficulty in writing a
quantitative specification of reliability. Special
problems may, however, be introduced by certain
features of the product or of the project program,
such as—
(a) a requirement for extremely high reliability (e.g.
a low failure rate or high mean time between
failures);
(b) a requirement for extremely long life;
(c) high cost or shortage of test samples; or
(d) the size or complexity of the equipment or item.
The presence of these problems does not, however,
invalidate the approach described in the following
clauses; their effect is to cause difficulty in obtaining
reliability assurances as discussed more fully in
Clause 3.2.9.
The designer’s confidence that the reliability
requirements will be met is dependent on the
adequacy of the data available concerning failure.
This data can only be obtained from reliability
testing, from field data, or from knowledge of the
physical processes leading to failure. The greater the
volume of data, the greater the confidence. A small
volume of data, therefore, requires considerable
engineering judgement in its interpretation.
3.1.6 Qualitative approach. When reliability is
specified qualitatively, the methods used to assure
reliability should be clearly described in the
specification. The criteria against which reliability
may be judged should be stated. Where possible, it is
advisable for the customer to agree on the
specification with the supplier as well as on the
extent of reliability assurance to be carried out. If the
reliability data are to remain valid, it is essential for
the customer to ensure that the relevant operating and
maintenance conditions in particular are met. Thereby
the greatest influence on reliability will be achieved.
3.1.7 Quantitative maintainability clauses. A
complete statement of maintainability requirements
will cover four broad areas, as follows:
(a) Maintainability characteristics to be achieved by
the item design.
(b) Constraints to be placed on the deployment of
the item which will affect its maintenance.
COPYRIGHT
AS 3960—1990
(c) Maintainability program requirements to be
accomplished by the supplier to assure that the
delivered item has the required maintainability
characteristics.
(d) Provision of maintenance support planning.
A maintainability specification typically covers the
various aspects of maintainability requirements at the
operational level. However, since maintainability
affects maintenance and support costs and
maintenance times at different maintenance levels,
statements should be included in the specification
covering requirements needed at all levels affected by
the maintenance policy.
3.1.8 Qualitative maintainability requirements.
The qualitative approach considers specification of
design disciplines and the degree to which the item
concurs with a specific maintenance and support
policy. Where the qualitative requirement contains
numerical values, it should be qualified by a
statement of the degree to which it should be met.
This could be done by proportions of cases or events,
by confidence levels or by other probabilities.
Otherwise, compliance must be judged by inspection
or document review. Such policies could include
statements such as the following:
(a) Repair shall be performed by personnel of stated
skill level.
(b) Repair shall be performed by replacement of
recoverable units.
(c) Replaceable parts shall be plug-in units.
(d) Maintenance shall be performed according to
defined and established procedures.
(e) Failed part isolation shall be performed by
built-in test equipment for 95% of all cases.
Examples of qualitative aspects for which
requirements may be specified are–
(i) maintenance skill level requirements;
(ii) need for special tools or test equipment;
(iii) need for adjustments;
(iv) parts standardization;
(v) clear subsystem function identification;
(vi) visual inspection access;
(vii) built-in test facilities;
(viii) properly marked test points;
(ix) colour coding and labels as appropriate;
(x) use of plug-in units;
(xi) use of captive fasteners;
(xii) use of handles on replaceable units;
(xiii) scope and range of technical manuals; and
(xiv) human factor limitations in the design of the
item.
When writing a specification, not only should
requirements be stated on how things should be done,
but in most cases there are also constraints on ways
of meeting requirements which, for various reasons,
cannot be accepted (for example, no preventive
maintenance will be allowed during certain periods of
time). Failure to specify constraints might give the
designer the opportunity to fulfil his maintainability
requirements at the expense of the user (e.g. by
requiring more expensive instrumentation and the use
of special tools).
AS 3960—1990
Constraints needed in a specification depend on the
nature of the maintainability requirements discussed
above. It is advisable to explicitly examine the need
for each objective stated. Generally, such constraints
will include a basic maintenance and support policy
with limits for critical resources like expensive
instrumentation, high cost spares, and skill and
number of personnel at each level in the organization.
Some statement on the amount and the nature of
preventive maintenance will often be included in the
specification. Special emphasis should be given to
constraining critical resources to be held on site.
Emphasis should also be given to constraining periods
of time relative to expected utilization of the
equipment, when preventive maintenance can be
carried out. Constraints may also be introduced by
reference to other specifications.
3.2 WRITING RELIABILITY AND MAINTAIN-
ABILITY CLAUSES IN A SPECIFICATION.
3.2.1 Necessary clauses.
3.2.1.1 Reliability considerations. A written
reliability specification irrespective of whether it is a
single document or based on a series of documents,
should contain clauses dealing with the following:
(a) The function or functions of a product.
(b) The criteria for failure of the product.
(c) The reliability characteristic or characteristics
(e.g. MTBF, MTTF, etc) that are appropriate to
the circumstances.
(d) The required value of the reliability
characteristic and, if known, the distribution of
failures in time.
(e) The time during which, and the conditions in
which, the product is required to perform its
function or functions.
(f) The means by which reliability assurance is to be
attained.
3.2.1.2 Factors for consideration. There should be
clauses in a reliability specification to take account of
the following:
(a) Environmental conditions—the conditions to
which the product may be subjected during
transport, storage or use. Such conditions may
include—
(i) extremes of heat, cold, pressure and
humidity;
(ii) exterior use, in which case rainproofing and
resistance to dust and sunlight, etc, are
important;
(iii) shock and vibration;
(iv) the electromagnetic environment;
(v) the chemical environment; and
(vi) the biological environment.
(b) Stress conditions—the type of stress testing
required and the nature and direction of the
applied loading.
(c) Lifespan/durability—the objective or required
serviceable life of the product, given that it is
maintained as specified, and at the specified
intervals.
(d) Envisaged use time—the envisaged use time per
year or per month, depending on the predicted
operating life of the product being specified.
COPYRIGHT
22
(e) Resistance to misuse—the means of protecting the
product, during any phase of its total life, against
the effects of misuse.
(f) Maintenance—the maintenance requirements and
procedures that should be specified to the user.
Maintenance periods and time out of operations
(down time) necessary to carry out maintenance
should be specified. Maintenance may be provided
by the supplier or procedures may be given in the
form of a manual.
(g) Storage life—maximum stored life (shelf life) and
any protection required during storage.
(h) Supplier’s reliability testing—any testing such as
life testing or environmental testing which the
supplier chooses to protect/ensure the reliability of
the product.
(i) Customer or user testing—any kind of acceptance
testing or trial laid down by the customer as a
condition of supply.
3.2.2 Function of an item. It is necessary to have a
clear statement of the function or functions of an item
since any reliability specification is based on the failure
of an item to perform its function. In some cases the
function may be obvious, e.g. a nut or an electric light
switch, but in others a detailed specification should be
given.
3.2.3 Criteria for failure. The criteria for failure may
follow implicitly from a definition of the function of the
item but, in circumstances where this is not so, it is
essential that all these criteria be stated explicitly.
Failures should be divided into categories with the object
of helping to understand the problem involved in
assigning the area of responsibility for action. (See
Appendix A for the classification of failures.)
3.2.4 Choice of a reliability characteristic. Reliability
characteristics are quantitative statements. Reliability
may be approached from the standpoint of the success or
failure of a product to perform its function over a given
period of time. Time may be replaced by distance,
cycles, throughput or other usage-related parameters.
Reliability characteristics in common use include failure
rate, mean life or mean time to failure (for non-repaired
items) and mean time between failures (for repairable
items). Examples of reliability requirement statements are
as follows:
(a) Equipment should operate successfully for X hours
on Y% of the occasions on which it is required.
(b) There shall be Z% confidence that an equipment
will not fail more frequently than X times in Y
equipment running hours.
(c) The mean life of a population of similar items
should be equal to or greater than Y hours with a
standard deviation of S hours.
(d) An equipment should have a Y% probability of
completing an X kilometre task without a failure.
If the failure pattern is known, any of these statements is
satisfactory since it may be interpreted mathematically,
employing probability theory and statistics. Information
on the density function also simplifies demonstration
procedures, as every test can be related to available
information and also makes it possible to estimate the
confidence attached to the result. If the failure pattern is
unknown, requirements may be stated in these terms and
it will then be necessary either to establish the pattern
from appropriate field data or to assume a pattern for the
purposes of assurance.
 23
3.2.5 Required value of the reliability characteristic.
The required value is the numerical value of whatever
reliability characteristic has been selected. Where
possible or appropriate, the underlying frequency
distribution relating to the required reliability
characteristic should be stated. However, it frequently
happens that specific assumptions about the failure
distribution and the relationships between the parameters
of the distribution and the environment cannot be made
owing to a lack of data, e.g. the only available data may
be results from tests carried out on a small prototype or
pre-production sample. In such cases statistical
procedures, known generally as ‘non-parametric’ or
‘distribution-free’, have been shown to yield conservative
reliability estimates when compared with those calculated
from actual distributions. However, non-parametric
procedures can provide a quantitative indication useful
for initial estimation when test data are limited to the
results from a small sample.
3.2.6 Choice of a maintainability characteristic.
Quantitative maintainability characteristics are used to
express maintainability in numerical terms.
Maintainability is generally approached from the
standpoint of returning an equipment to an operating
condition following failure (corrective maintenance) or
keeping the system from failing (preventive
maintenance). The most common objective is related to
the time an item is in a non-operable status due to
maintenance. The aim is to keep this time as short as
possible. There is a wide range of requirements used to
this effect. The difference between them is that they
express different priorities between related item
properties and that they allow for different flexibility
concerning trade-offs to be made later in the program. It
is important that such factors are considered before a
specific requirement is included in the specification.
Active repair time is often used to specify maintainability
and includes the following sub-elements:
(a) Diagnosis (failure detection, localization of cause,
etc).
(b) Technical delays (typical technical delays include
setting time, cooling, interpretation and application
of information, interpretation of displays, read out,
etc).
(c) Restoration (disassembly, interchange, reassembly,
alignment, etc).
(d) Final check (testing procedures as necessary).
3.2.6.1 Characteristics. A variety of other
maintainability characteristics may be specified for the
item. Some other types of requirements associated with
different classes of maintenance time and verification
methods, with their characteristics, are—
(a) active maintenance time (mean, median, maximum);
(b) active corrective maintenance time (mean, median,
maximum);
(c) active preventive maintenance time (mean, median,
maximum);
(d) routine inspection interval;
(e) maintenance cost per operating hour (mean);
(f) number of hours labour per operating hour (mean);
(g) number of personnel per maintenance action
(mean); and
COPYRIGHT
AS 3960—1990
(h) maintenance support cost for the life cycle (mean).
3.2.6.2 Examples. Typical examples that reflect the
incorporation of quantitative maintainability requirements
into specifications are as follows:
(a) The mean time to repair at intermediate level shall
be X minutes. Y% of all maintenance tasks shall be
completed in less than Z minutes.
(b) Preventive maintenance shall not be required.
(c) Maintenance reliability, the probability that an
equipment is capable of performing its functions
following a satisfactory maintenance checkout shall
be greater than X%.
(d) All operator level maintenance tasks shall be
completed in less than Y minutes without the use of
special tools.
3.2.7 Required value of the maintainability
characteristic. As for reliability, where possible and
appropriate, the underlying frequency distribution relating
to the required maintainability characteristic should be
stated. In some cases two values of the maintainability
characteristic may be specified, which will better
determine the distribution. For example, as well as
specifying a mean time to repair (MTTR) for an
equipment, the maximum time to repair (i.e. the longest
repair time) may be specified.
3.2.8 Operating regime and conditions. It is
important to state the period in the life of the product to
which the chosen reliability and maintainability
characteristic applies and to state the conditions in which
the item will operate, including the stress conditions.
The term ‘stress’ is used throughout this Section to
denote the intensity of the applied stress in its broadest
sense. The product may be loaded by internal or external
forces, by electrical or mechanical effects, by chemical
or biological agents, by temperature or by other factors
that affect reliability and maintainability such as human
error.
Maintenance and operational procedures may be
important in their effect on reliability. Where this is the
case, such procedures should be stipulated in the
reliability specification. Alternatively, where appropriate,
the supplier should stipulate to the purchaser the
maintenance and operational procedures needed to
achieve the required degree of reliability.
3.2.9 Reliability and maintainability assurance.
Evidence should be provided giving confidence that the
reliability and maintainability requirements are satisfied.
There are three ways of obtaining assurance regarding a
product’s reliability, as follows:
(a) By stipulating that reliability and maintainability
testing be carried out on the product or by making
existing test or field reliability data available.
(b) By stipulating that a reliability and maintainability
assessment of the product be carried out, based on
a knowledge of—
(i) their failure mechanism;
(ii) the failure data; or
(iii) both of these.
AS 3960—1990
If neither of these ways is sufficient, confidence can
be progressively generated by using analytical
methods described in Clause 2.2.2. Use of such
methods will ensure that all reasonable steps have
been taken to achieve reliability and maintainability
in the design. Assurance that these activities have
occurred can be obtained by a design audit.
When reliability and maintainability testing is
specified in a contract, the provision of satisfactory
demonstration testing results is normally a condition
of contract. However, where reliability and
maintainability assessment are specified, the
purchaser shoul d require rel iabi li t y and
maintainability predictions using methods of
assessment acceptable to both the purchaser and the
supplier. Similarly, failure mode, effects and
criticality analysis (FMECA) or fault tree analysis
should be agreed on between them. A design audit
should be carried out by the purchaser or his
COPYRIGHT
24
nominee, but provision should be made in the
contract for necessary assistance from the supplier. A
further point to be noted is that reliability assurances
or a warranty given by the supplier may depend on
certain recommended operational and maintenance
procedures being followed by the purchaser.
3.3 SPECIFICATION OF RELIABILITY AND
MAINTAINABILITY IN PRACTICE. In practice
it may be difficult to fulfil all the conditions
necessary to write completely satisfactory reliability
and maintainability clauses in a specification.
However, if all the steps that are recommended in the
previous clauses are followed, areas of doubt will be
exposed and action may be taken to raise the level of
confidence that the reliability and maintainability
requirement will be met.
 25
SECTION 4. ASSESSMENT AND PREDICTION
OF RELIABILITY AND MAINTAINABILITY
4.1 GENERAL.
4.1.1 Aims of reliability assessment. Reliability
and maintainability assessment is the process by
which quantitative values are assigned to reliability
and maintainability. An essential part of a reliability
program is the prediction and measurement of the
reliability and maintainability of a product.
Reliability and maintainability assessment is required
when—
(a) establishing the reliability and maintainability
required of a product;
(b) predicting the reliability and maintainability of a
product that is still in the design, development or
pre-manufacturing stage; and
(c) establishing whether a product that is in service
has performed, or is performing, in such a way
as to satisfy the specified value of the reliability
characteristic, and whether it is likely to continue
to do so for the rest of its design life.
The customer for an item may require demonstrations
that the reliability and maintainability requirements
have been or will be met, which is a major aim of a
comprehensive reliability program. During the design,
development and manufacturing phases, reliability
and maintainability assessments are a predictive
process that relies on techniques, such as the use of
models, failure mode analysis, evaluation of
components, operating stresses, materials and
processes, and on the testing of development models
and prototypes. It also relies on the provision of
failure data in order to investigate specific failure
modes and thus provide information for remedial
action. In service, the user may wish to measure
reliability and maintainability by observation.
During the design and manufacturing stages, and
during the service life of a product, adequate failure
data are essential. Guidance is given in Section 5 on
the production, flow, analysis and interpretation of
reliability data. Reliability assessment requires firstly
a reliability model for the product to represent its
reliability in an appropriate mathematical form, and
secondly the necessary reliability data.
4.1.2 Rel i ab i l ity an d mai n tain ab i l i ty
characteristics. The choice of a suitable reliability
characteristic for a specification, and the selection of
its required value are discussed in Clauses 3.2.4 and
3.2.5. The choice of maintainability characteristics
and their required values are discussed in
Clauses 3.2.6 and 3.2.7.
4.2 RELIABILITY ASSESSMENT. Reliability
assessment can vary enormously in the techniques
used, from the very simple to the lengthy and
complex. Methods by which reliability data are
collected for a particular item that can be tested will
vary, because development testing may initially be
the only source of data, whilst test data on production
items will subsequently be available. After the
product goes into service, the source of data will be
field experience, which is discussed at greater length
in Clause 5.3.
When the nature of a product is such that it cannot be
tested before being put into service, it is necessary to
COPYRIGHT
AS 3960—1990
obtain data by testing separately the components of
the item or, if this is not possible, to adopt the
physics of failure method or to obtain relevant failure
data from other circumstances.
Specifications often demand that a reliability
demonstration be carried out. This may be interpreted
as the process by which the actual or predicted
reliability of an item may be compared with the
requirement. The process of establishing a
requirement, and that of observing or predicting the
reliability of a product, are both examples of
reliability assessment.
4.3 RE LIAB IL IT Y PREDICTION BY
MODELLING. The value of reliability prediction
lies in its use for selecting design options.
One method of predicting the reliability of a system
is by determining its individual component
reliabilities and combining them, using a
mathematical model of the complete system.
A mathematical model consists of one or more
equations or a logic matrix which defines the
characteristics of the system.
The mathematical model should include, where
required—
(a) failure and repair characteristics of the items;
(b) components/sub-assemblies/assemblies arranged
in parallel and standby redundant groups, and in
dependent and independent series groups; and
(c) allowance for operating stresses.
In addition to meeting these requirements, the model
will be constrained by a number of more practical
considerations. Where a design is developing rapidly,
options have to be selected quickly and a lengthy
prediction process is impracticable. The model should
therefore be constructed quickly and be readily
altered to accommodate changes in the design. The
mathematical treatment of reliability modelling is
discussed in BS 5760:Part 2.
Reliability models for the software elements of
systems are being developed, but none have gained
widespread acceptance. There are no standard
methods comparable with those used for hardware
prediction. BS 5760: Part 2 was written for systems
not containing software, and the methods outlined
are, therefore, not necessarily applicable to the
software elements of systems.
4.4 PROVISION OF RELIABILITY DATA. The
data used in reliability predictions can be gathered
from various sources.
The acquisition of field data on the product itself is
obviously the best source since there is no substitute
for field experience. Reliability statistics can be
obtained from the recorded operating data. Where a
product is under development, data can be obtained
from development or production tests.
AS 3960—1990
Where there is neither test nor physics of failure data
available for a product, reliability data can be
obtained from data on products of comparable
function, but not necessarily of the same design or
subject to the same characteristics, i.e. function,
design, operating conditions, size, complexity, quality
systems, etc. Users of this method should take
account of any characteristics that are not
comparable, recognizing that any values derived are
therefore subject to uncertainty, and this uncertainty
should be identified. Sources of reliability data are
discussed in Clause 5.3.
4.5 RELIABILITY GROWTH TESTING.
4.5.1 General. Reliability growth testing involves
deliberate stress testing designed to stimulate failures
during development, analysis and corrective action.
Clause 2.2.2.5(f) summarizes the purpose and features
of reliability growth testing.
4.5.2 Preparation. The development program should
ensure the provision of the necessary equipment and
services, suitably phased with all other development
and reliability program activities. The following are
typical of these provisions:
(a) Units allocated for testing.
(b) All test equipment (including any which is
‘special-to-type’).
(c) Environmental chambers for combining vibration
with other environmental stresses.
(d) Ancillary services, e.g., special power supplies,
cooling air.
(e) Manpower for the program, including
maintenance of the facilities and supervision
outside normal hours.
Further, an active preparation period (typically from
2 months to 6 months, according to size of project)
before testing will be required for the following tasks:
(i) Drafting a test procedure document for
agreement by the customer, showing each item in
the program, including fault follow-up and
estimated test time. It is important to include
precise definitions of failures in their various
categories.
(ii) Design and building of any special test
equipment.
(iii) Preparation of the environmental and associated
facilities and installation of the units to be
tested.
(iv) Accomplishment of pre-conditioning or burn-in
(Clause 2.2.3.3).
4.5.3 Results of reliability growth testing. The
improvement in reliability resulting from reliability
growth testing can be estimated from—
(a) the final instantaneous reliability (usually stated
as a failure rate or probability of success); and
(b) the growth ratio (ratio between initial and final
instantaneous failure rates).
Instantaneous failure rate can only be estimated by
assuming a mathematical growth model that
adequately fits the distribution of observed failures in
the time period. However, the results will not be
significant if only a few failures are revealed in all
the equipment under test.
It is important that the model fits the data reasonably
and such models should only be used with
discrimination and caution.
COPYRIGHT
26
A comprehensive series of reports should be prepared
at regular intervals describing progress towards
reliability achievement and giving updated
assessments of the problems involved.
4.5.4 Factors governing reliability growth testing
effectiveness. Project management should be aware
of the many factors that constrain the effectiveness of
growth testing and that govern the improvement in
reliability ultimately achieved. These may typically
include—
(a) the number of units allocated for testing and the
extent to which they are representative;
(b) the resources available to sustain testing at the
planned rate (including engineering and spares
back-up for units, test facilities and test
equipment);
(c) the effectiveness of failure analysis and the
feedback loop; and
(d) the resources and lead time available to develop
and introduce design changes prior to design
freeze for production (or to introduce
modifications after design freeze).
Compromises will continually be necessary to achieve
optimum growth; for example, testing may be
continued with incomplete serviceability of
equipment, and with modifications introduced on an
‘opportunity’ basis only. This may affect the accuracy
of assessments of reliability and growth and any such
reservations should be stated. However, the growth
process itself has to take precedence over the
accuracy of monitoring it.
4.6 RELIABILITY DEMONSTRATION AND
TESTING.
4.6.1 General. Reliability demonstration is the
process by which it is shown that an item has
characteristics that meet a particular requirement. It
is important to be able to describe that the product
can do this: this is termed ‘compliance’. Normally
this is indicated by the use of appropriate test
procedures during the development, pre-production,
production and operational phases.
4.6.2 Aims of a test program. Generally, the aims
of a test program are to—
(a) ensure as far as practicable that the product
meets the specified performance or operational
requirements, including reliability;
(b) ensure as far as practicable that manufacturing
faults and faulty parts are eliminated;
(c) highlight systematic errors so that deficiencies in
design are corrected;
(d) contribute information that can be used to
determine reliability characteristics; and
(e) verify that changes, such as those affecting
design, including value analysis, modification,
manufacturing processes or sources of supply
have not reduced reliability below an acceptable
level.
These aims normally apply whatever the stage in the
life of the product to which they refer.
It is not always possible or economical to test
complete systems; it may be necessary to test
components, or specific design features and to use the
data in a reliability model to predict the behaviour of
the complete system.
 27
4.6.3 Choice of test program. There is a wide variety
of test programs. Some of the main types are as follows:
(a) Condition terminated tests (Bernoulli trials). A
condition terminated test terminates when a
predetermined condition is met or when failure
occurs.
(b) Fixed time or sequential tests. Fixed time or
sequential tests are based on specified acceptable
values of the reliability characteristic, and their rules
enable decisions to be made on whether the
equipment is to be accepted or rejected, or whether
the test is to be continued.
(c) Accelerated testing. The primary purpose of an
accelerated test is to provide failure data more
quickly than if the product were tested under
normal conditions. To be valid, an accelerated test
should not alter the modes of failure. Correlation
between failure rate data obtained during accelerated
testing and that obtained during normal life testing
is not always possible and should only be attempted
when a relationship can be unquestionably
determined.
4.6.4 Evaluation of test data using Bayesian methods.
The analysis of reliability test data, which may be
limited to a small number of observed failures, will
evaluate the true reliability only at a low level of
confidence. Bayesian statistics may be used in this
situation, using prior information obtained from earlier
tests on the same or similar items, sometimes combined
with judgement, to produce an estimate of reliability.
4.6.5 Proof test. Proof testing is concerned with
demonstrating fitness for purpose by validating design
hypotheses, assumptions and other criteria in relationship
to the design itself, and also in relationship to the
conversion of that design into the artefact, and hence
with the validation of the method of manufacture. It may
consist of pressure testing, climatic testing, vibration
testing, static loading of structures, crack testing in
turbine blades, etc. Data gained during proof testing may
contribute to any overall statement on the system
reliability.
4.6.6 Suitability of statistical methods for analysis of
test results. It is often found that, for reliability analysis
of hardware systems, the negative exponential
distribution is an appropriate basis for statistical tests.
However, this cannot be generally assumed.
If the distribution departs significantly from this
assumption, tests based upon the negative exponential
distribution are inappropriate and another distribution
may have to be utilized. There may be no alternative to
estimating the failure distribution function from the test
data. As the outcome cannot be foretold, all time tests
should be so designed that distribution estimation is
possible as an alternative analysis without repeating the
tests. The statistical level of confidence regained from
testing will influence the test duration.
There is no currently accepted distribution for software
analysis.
4.7 MAINTAINABILITY PREDICTION.
4.7.1 Maintainability prediction. This is the estimate
of the maintenance workload (preventive and corrective)
associated with the proposed design. Maintainability
predictions should be accomplished immediately
following the definition of the basic system. This is the
earliest time when sufficient data is available to perform
COPYRIGHT
AS 3960—1990
a meaningful quantitative evaluation of design
characteristics in terms of performance and maintenance.
At this early stage of the system design process, the
maintainability predictions can still influence the design
approach. As the system design progresses to the detailed
level, more complete design information becomes
available and consequently the estimation of system
maintainability characteristics becomes more accurate.
The estimate should be updated continuously as the
design progresses to provide the visibility necessary to
ensure that the specified requirements have a high
probability of being achieved. Predictions are applicable
to all programs and all types of systems and equipment.
However, they are particularly pertinent in programs
where risks are high or unknown, and the failure to
achieve the maintainability requirements is highly
undesirable.
4.7.2 Prediction advantages. A significant advantage
of using maintainability prediction is that it highlights for
the designer, those areas of poor maintainability which
justify product improvement, modification, or a change
of design. Another useful feature of a maintainability
prediction is that it allows the user to make an early
assessment of whether the predicted down time, the
quality and quantity of personnel, tools and test
equipment are adequate and consistent with the needs of
the systems operational requirements.
4.7.3 Techniques. The effectiveness of maintainability
prediction as an evaluation tool depends on the technique
and accuracy of input data. This in turn is based on the
applied knowledge and insight of the analyst. There are
a considerable number of maintainability prediction
techniques presently in use. The procedures vary
depending upon the specific need for measurement,
differences in imposed requirements, peculiarities of the
equipment being measured, and individual or company
preferences.
4.7.4 Basic assumptions and interpretations. Every
maintainability prediction procedure depends upon the
use of recorded reliability and maintainability data and
experience which has been obtained from comparable
systems and components under similar conditions of use
and operation. It is also customary to assume the
principle of transferability. This assumes that data which
accumulates from one system can be used to predict the
maintainability of a comparable system which is
undergoing design, development or study. This procedure
is justifiable when the required degree of commonality
between systems can be established. Usually during the
early design phase commonality can only be inferred on
a broad basis. As the design becomes more refined,
commonality is extendable if a high positive correlation
is established relating to equipment functions,
maintenance task times and levels of maintenance.
History has shown that the advantages greatly outweigh
the burden of making a maintainability prediction.
4.7.5 Elements of maintainability prediction
techniques. Each maintainability prediction technique
utilizes procedures which are specifically designed to
satisfy its method of application. All maintainability
prediction methods are dependent upon at least two basic
parameters—
(a) failure rates of components at the specific assembly
level; and
(b) repair time required at the maintenance level
involved.
AS 3960—1990
4.7.5.1 Failure rates. There are many sources which
record the failure rates of parts as a function of use
and environment. Failure rates are used in
maintainability prediction to provide an estimate of
the relative frequency of failure of those components
utilized in the design. Similarly, the relative
frequency of failure of components at other
maintainable levels can be determined by employing
standard reliability prediction techniques using parts
failure rates. Another use of failure rates is to weight
the repair times for various categories of repair
activity, in order to provide an estimate of its
contribution to the total maintenance time.
4.7.5.2 Repair times. Repair times are determined
from prior experience, simulation of repair tasks, or
data secured from similar applications. Most
procedures break the maintenance action into a
number of basic maintenance tasks whose time of
performance is summed to obtain the total time for
the maintenance action.
4.8 MAINTAINABILITY DEMONSTRATION
AND TESTING.
4.8.1 General requirements. Maintainability
specifications written into a contract are in effect
only targets or goals, unless there is an actual
assessment of the maintainability parameters of the
developed system/equipment.
The primary function of maintainability test and
demonstration is to ‘verify maintainability’ that has
been ‘designed-in’ and ‘built -in’ to t he
system/equipment. Up to this point in development,
the tasks of the maintainability program have been
analytical in nature, providing a confidence that both
the quantitative and qualitative maintainability
requirements would be met.
4.8.2 Maintainability testing program. There can
be three phases to a maintainability testing program.
Maint ai nabil i t y veri ficat i on i s conducted
incrementally during development on mock-up
models and early hardware designs, with the intention
of provi di ng progressive assurance t hat
maintainability requirements can be achieved and that
earlier modelling and allocation were accurate.
Maintainability demonstration occurs at the end of
development, to determine whether contractual
requirements have been achieved. The demonstration
is performed on as close-to-production hardware as
possible (i.e. final prototype or pre-production item),
conducted in an environment which simulates, as
closely as possible, the operational and maintenance
environment specified for the item. The environment
should be representative of the working conditions,
tools, support equipment, repair parts, facilities and
technical publications that are required during
operational service.
Maintainability evaluation occurs in the field
environment. Its objectives are to evaluate the impact
of the actual operation, maintenance and support
environment on the maintainability, to evaluate
COPYRIGHT
28
correction of deficiencies detected during the
maintainability demonstration, and to demonstrate
depot level maintenance tasks when applicable. All
evaluation items should be production or production
equivalent items.
4.8.3 Maintainability demonstration. To fulfil
maintainability demonstration requirements for a
typical program, a contractor is obligated to
demonstrate that equipment meets the specified
maintainability requirements. The accomplishment of
such a demonstration in a realistic operational
environment is often impractical. In certain instances
demonstrations can be accomplished in an
environment that closely approximates a true
operational situation. In other words, contractor
demonstration may be conducted at the customer’s
facility, employing customer personnel, on equipment
installed and ready for operational use. However,
such demonstrations are generally scheduled at
specific times and faults are simulated in the
equipment in order to simulate maintenance
requirements. Although this type of situation does not
completely reflect normal user operations (since
failures are induced, and the subsequent
demonstrations planned, eliminating some of the
randomness normally involved), it can provide a close
simulation.
4.8.4 Test conditions. Test conditions for formal
maintainability demonstrations include—
(a) maintainability requirements;
(b) maintenance policy;
(c) demonstration model configuration;
(d) test environment;
(e) test personnel;
(f) technical data;
(g) support equipment; and
(h) spare parts.
4.8.5 Maintenance task selection. The assurance
that the proposed demonstration reflects the
maintainability of the total system depends on the
maintenance task selection process. This process
involves the identification of a representative sample
(based on the expected percentage contribution
toward total maintenance requirements) of
maintenance tasks to be demonstrated. The process
does not include actual random (unplanned) failures
occurring during the test, but it does include a variety
of induced failures to ensure adequate coverage.
4.9 COMPLIANCE ILLUSTRATION BY MEANS
OTHER THAN TESTING. If, for some reason,
reliability and maintainability demonstration testing
is not practicable, reliability and maintainability
prediction techniques using verifiable data obtained
from research programs or other sources, if possible,
should be considered.
 29
SECTION 5. PRODUCTION, FLOW, ANALYSIS AND
INTERPRETATION OF RELIABILITY AND
MAINTAINABILITY DATA
5.1 GENERAL.
5.1.1 Benefits. The knowledge of the performance
of a product during its life is necessary for the
following reasons:
(a) Effective action can be taken to improve
reliability. Periodic reliability reports are
necessary to determine whether reliability goals
have been achieved. Where modifications to
improve the reliability or the life of a product in
service are necessary, such reports are useful in
monitoring the effects of the modifications.
Where warranty or guarantee clauses are in
operation, reliability data can be used to monitor
trends in claims. Reliability data feedback is
vital to effective product quality assurance.
(b) Improvements can be incorporated in future
designs. The design of future generations of
products can be improved if a detailed
knowledge of achieved reliability is obtained
from the present generation of the products. This
can lead to commercial advantages to both
customer and manufacturer.
(c) Improvements can be made to design
documentation in order to improve the efficiency
of carrying out software changes (or repairs).
(d) An effective and efficient support organization
(which can forecast spares requirements, control
stores inventories and software maintenance) can
be provided.
(e) Where hazardous operations are involved, the
degree of hazard or risk may be reduced by
improvement of the design or operating
procedures. Collection and analysis of data are
important parts of modern safety programs.
(f) Operational procedures, operational documentation
and maintenance schedules can be improved.
5.1.2 Organization. The systematic means of
collecting, recording, processing and analysing data
should be centred on a suitable department within the
organization concerned. Such a department can be
justified on an economic basis and should be capable
of dealing with the large volume of information that
may flow to and from field agencies. Some of the
activities of this department may be concerned with
matters other than reliability. This should be
encouraged so that the benefit of close consultation
with those concerned with other activities may be
reaped. The data feedback loop should embrace the
customer, end user, the manufacturer’s own design
and production functions and also subcontractors, or
suppliers.
Many organizations have an established information
collection system that provides management with data
for normal control purposes. Consideration should be
given to the extra cost of adapting any such system so
that additional data on reliability, maintainability and
availability (as described in this Section) can be
collected.
COPYRIGHT
AS 3960—1990
The advantages that may be gained from subscribing
to one or more of the data bank services should be
considered.
It is important that reports of failures, unreliability,
etc, reach the maintenance and support organization
without delay.
5.1.3 Effectiveness of communication. It is
important that communication and co-ordination and
training techniques are aimed at creating a responsive
and responsible approach to data collection, data
analysis and corrective action.
5.2 DATA INPUT.
5.2.1 Reporting systems. The reporting system
should primarily be based on what is needed rather
than what is available. However, job cards, log books,
permits and quality documentation may contain useful
data and should be analysed for inclusion.
The utilization, evaluation and interpretation of data
will be difficult without a system which extracts all
the significant facts from large amounts of material.
Computers may be appropriate to store and retrieve
reliability data, especially where large quantities of
data are to be handled, but in each case cost
effectiveness should be considered.
Data to be recorded for an event should include data
on the nature of the event and additional details taken
from routine operating records. These routine records
should ideally include the following:
(a) Specification and description of products
(location, nature, design, size, current
configuration, etc).
(b) Operating history, including installation data
covering duty, maintenance regime, condition
monitoring, calibration data tests and
environment.
(c) Fault/failure history that should include a
complete listing of events, causes, consequences,
event times, returns to operation, maintenance
data, test data, calibration data, etc.
It is important to clearly define items, e.g. system,
components and t he boundaries impl ied.
Configuration changes should be taken into account.
The precision and limitations of the data should also
be clearly indicated.
5.2.2 Specification and description. The
importance of the specification is to enable the
intentions behind the design and operation of the item
to be appreciated and its function to be understood.
Items are sometimes misused, e.g. the absence of a
clear specification of limitation of use can severely
affect reliability. Together the specification and
description wil l enable identificat ion and
classification to be carried out. Items within a class
should differ only by minor modifications or
amendments. Figure 4 gives an example of a typical
data sheet for an item which is a product of the
manufacturing industry.
AS 3960—1990 30

INFORMATION FOR RELIABILITY DATA STORE

(To be completed for all it ems on entr y to test or to service)

* D R N DATA REF. NO.

* I D C ITEM DESCRIPTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. .. . .. .. .. ... . .. . .. .. . .. .. . .. .. . .. .. .. .. .. .. . ... ...
. .. . .. .. .. ... . .. . .. .. . .. .. . .. .. . .. .. .. .. .. .. . ... ...

* L O C LOCATION

* M A N MANUFACTURER (Name and address) . .. . .. .. .. ... . .. . .. .

. .. . .. .. .. ... . .. . .. .. . .. .. . .. .. . .. .. .. .. .. .. . ... ...

* D E S DESIGNER (Name and address) . .. . .. .. .. ... . .. . .. .. . .. .

. .. . .. .. .. ... . .. . .. .. . .. .. . .. .. . .. .. .. .. .. .. . ... ...

* S P E SPECIFICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

* S U B SUBSTANCE CONTAINED (i. e. Type of liquid, gas, etc.)

. .. . .. .. .. ... . .. . .. .. . .. .. . .. .. . .. .. .. .. .. .. . ... ...

* I N S INFORMATION SOURCE (Lit . Ref., etc.)

. .. . .. .. .. ... . .. . .. .. . .. .. . .. .. . .. .. .. .. .. .. . ... ...

* D T Y DATA YEAR (Last year of surveillance)

* I N T INFORMATION TYPE

* A P P APPLICATION

* E N V ENVIRONMENT

* I N R INSPECTION REQUIREMENT . . . . . . . . . . . . . . . . . . . . . . . . . . .

* M T P MAINTENANCE TYPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

* M I V MAINTENANCE INTERVAL VALUE

* M I U MAINTENANCE INTERVAL UNIT . . . . . . . . . . . . . . . . . . . . . . . . .

* M O D MODEL NO. (Type, No., etc)

RA NG E/SIZE (Up to 4 recommended working values)

VA LUE UC OD E SIG UN IT UN IT COMMEN T

* R S 1 . .. . .. .. .. ... . . ( . .. . .. .. .. ... . .. . .. .. . .. .. . .. .. )

* R S 2 . .. . .. .. .. ... . . ( . .. . .. .. .. ... . .. . .. .. . .. .. . .. .. )

* R S 3 . .. . .. .. .. ... . . ( . .. . .. .. .. ... . .. . .. .. . .. .. . .. .. )

* R S 4 . .. . .. .. .. ... . . ( . .. . .. .. .. ... . .. . .. .. . .. .. . .. .. )

FIGU RE 4 TYPICA L LIBR AR Y FILE DATA SHE ET

COPYRIGHT
 31 AS 3960—1990

History data sheet (t o be completed for all it ems on entr y to test or to service.) DRN

Item descripti on Installati on Inventory No.

Event Event Serial no. or Failure Failure Repair Outage Operati ng Plant Descripti on of event
no. date plant it em no. mode cause time time time status

Assumptions

FIGUR E 5 EXA MPLE OF HISTORY DATA SH EE T

COPYRIGHT
AS 3960—1990 32

Locati on and 1
A
report no.
Non-conformance 8
Use black ink B
report no.
Circumstances of failure including symptoms C Date of failure 16

(1) Quantified operating conditions Project and

(2) Symptoms D build
(3) Stress levels standard
(4) Mode of inspection and time of failure
(5) Duration of service Equipment 22
E
(6) Environmental conditions seri al no.
F System/section 28
G Seri al no. 32
H Unit /drg no. 38
Section 1

I Seri al no. 41
J Sub unit /drg no. 47
K Seri al no. 50
L Board drg no. 56
M Seri al no. 60
Component and
manufacturer or
modifi cati on no. 66
N
68
79
D R Ori ginator Date P Circuit reference 90
MRB decision and local corr ecti ve action Q Test state 93
Section 2

R Time into test 94

Signature Time 1 98
Details of investigation, related cases and repair Time 2 102
S
Time 3 106
Time 4 110
T Test specifi cati on
U
Section 3

Investigation responsibilit y 115

V
Survey report no. 116
Formal M R B/L.T.C/D.I .P.
Final decision

W Actual fault 122

X Classifi cati on 123
Y Signif icance 124
MRB/LTC/DIP/Rel 125
Z Future preventi ve action
Signature Date ECR No. 129
135
Section 4

Extr a
160
Project
Data 185

FIGU RE 6 TYPICA L FORM FOR RE CORD ING FAILUR E INFORMATION

COPYRIGHT
 33 AS 3960—1990

Job card

Item name Ori ginator’ s name What has failed?

Item code Ori ginator’ s section Where is it?
Ori ginator’ s dept. What is wrong with it?

Dept. reference Ori ginator’ s code When was it found? Date, ti me and meter reading
How was it found? Routine check
On start ing up
During operation
Observed
During maintenance

Work started . ... . .. . .. . ... . . ... ... . .. . ... ... . . .. .. Repair er’s report

Work completed . ... . .. . .. . ... . . ... ... . .. . ... ... . . .. ..

Net working time . ... . .. . .. . ... . . ... ... . .. . ... ... . . .. ..

Man-hours skilled . ... . .. . .. . ... . . ... ... . .. . ... ... . . .. ..

Man-hours unskilled . ... . .. . .. . ... . . ... ... . .. . ... ... . . .. .. Item exchanged Meter reading after change . . . . . . . New serial no. . . . . . . . . . . . . . . . . . .

Could item continue in use? YES NO Failure due to Maloperation Design Wear

Other . ... . .. . .. . ... . . ... ... . .. . ... ... . . .. .. .. . .. .. .. . . ... .

Repairs carri ed out by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other it em aff ected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Spares used . ..................

Code . ... . .. . .. . ... . . ... ... . .. . ... .. .. .. . .. .. . .. . Other report references . . . . . . . . . . . . . . . . . . . . . . . . . . . Part no. . . . . . . . . . . . . . . . . . . . . . . .

Other . ... . .. . .. . ... . . ... ... . .. . ... .. .. .. . .. .. . .. . Unit . .................. ......

Quantity . .................. ...

Cause of any delay . .............

FIGUR E 7 EXA MPLE OF JOB CA RD WITH SPA CE FOR FAILURE DATA

COPYRIGHT
AS 3960—1990
5.2.3 Operating history. Reliability, maintainability
and availability can be derived from the operating
history of an item. The operating history consists of
an operating log, backed up with instrumentation
where appropriate, inspection records and
maintenance/modification records which may include
a job card system. The history can be defined in
terms of events or changes of state that occur. There
are two main changes of state, namely–
(a) change to operating capability; and
(b) change of configuration.
The description of operating capability may include
operation to full or part rating or not functional. It
may be necessary to state why a product is not
functioning, e.g.—
(a) it is under repair due to failures;
(b) it is under routine maintenance; or
(c) it is not required to operate.
A description of the operational environment is most
important.
The means of recording may take various forms but
a set of standardized headings should be used.
Numerical codes are only worth considering for
computerized systems, otherwise coding and decoding
may be time-consuming.
5.2.4 Failure history. A standardized method of
reporting failures should be used. A typical history
data sheet is shown in Figure 5; Clause 2.2.2.5(i) lists
failure data recording attributes.
Consideration should be given to the use of data
terminals for feeding failure information into a
computer at the source of the failure. A typical form
of computer format for failure information is shown
in Figure 6.
Failure information may form part of an existing job
card system currently employed in an organization. It
should be designed to enable an organization to
assess the factors itemized in Clause 5.3. Figure 7
gives a typical example of such a job card.
NOTE: Failure classif ication is explained furt her in Clause 5.8
and defi niti ons of failure classifi cati ons are contained in
Appendix A.
5.3 DATA SOURCES.
5.3.1 Guidelines. The recommendations of Section
5 provide for setting up data collection schemes
which can be applied either during investigations of
samples of equipment, or on a more widespread basis
by large maintenance and inspecting organizations.
It is considered that, if the recommendations are
followed, accuracy and completeness of reporting,
which are paramount if such schemes are to be of
value, will be promoted by the collection of
st andardized informat ion. Moreover, such
standardization will facilitate the interchange of
information between users and manufacturers.
Caution should be exercised in combining data from
unrelated sources.
NOTE: To obtain optimum result s from the coll ecti on of data,
it is recommended that the entir e program of reporti ng, analysis
and disseminati on of result s be closely co-ordinated.
5.3.2 Past experience. Complete field data on items
being used in design are unlikely to be available. This
may be due to lack of adequate recording during the
life of a product or simply to the lack of history of a
COPYRIGHT
34
newly developed product. In such instances, it is
possible to obtain some information from sample
testing carried out during design and development.
It should be recognized that, in many cases, a new
complex product is not revolutionary in design.
Innovations, requiring new data, may occur in only a
few aspects in the design. It is therefore important
that full account be taken of the experience gained
with similar or identical products.
5.3.3 Design and development. Notwithstanding
Clause 5.3.2, some designs contain a higher
proportion of innovation than others. The care and
attention given to reliability should increase with the
proportion of new design if the product is to be
acceptably reliable.
Collection of information and data during the
development stage is vital. The whole purpose of
design and development is to provide such
information in order to progressively resolve the
factors of uncertainty surrounding the design, its
intended purpose and use, and its manufacture.
Some background information is essential when
evaluating data from the following sources:
(a) Prototype testing, either as part of a larger
product or individually on a test rig under
conditions more or less simulating actual use.
The function of these tests is to verify the design
or a particular detail, feature or characteristic of
the design, in terms of the expected use.
(b) Development testing on pre-production products.
These tests concern products built to the verified
design but they may still be proving
manufacturing methods and processes.
(c) Evaluation of the design features concerned.
5.3.4 Production. Information is usually readily
available in quantity, but the quality is often
questionable as assembly errors may frequently mask
normal process factors. Sources of production data
are as follows:
(a) Initial tests aimed at production validation. The
products are often tested under real conditions.
(b) Routine production tests to ensure that the
process is still under control, and routine production
checks carried out for quality control. The latter are
often concerned with monitoring those variables that
define parameters for reliability.
5.3.5 Factory test. The factory test is, or should be,
a prime source of data. Testing of sub-assemblies,
modules, etc, is normally a formal process backed by
procedural documentation from customers. Personnel
should be trained to use the normal failure reporting
forms.
5.3.6 Guarantee or warranty reports–product
liability test reporting. Guarantee or warranty
reports are perhaps the ideal means of extracting data
from customers, by involving their direct interests.
The level of warranty claims is one index of a
successful or unsuccessful product. Warranty
reporting should form part of the normal data
collection system and employ a common style format.
It should, however, be recognized that warranty
reporting applies, by definition, to the warranty
period, and that usually the life of a product in
service is longer than this.
 35
5.3.7 Supply of replacement parts. Spares usage
data provide a useful means of monitoring the
reliability of the product, and the collection of these
data should be encouraged. The typical job card
shown in Figure 7 includes a space for a ‘spares
used’ record.
5.3.8 Material or component supply. In material
or component supply the sources of data are—
(a) the specification of materials supplied for the
manufacture of the products;
(b) specifications defining the reliability and
performance of bought-out components; and
(c) results of goods inward inspection of material or
components from suppliers.
5.3.9 Repair department. Reporting by service or
repair organizations should provide positive
identification of the failed product, accompanied by
coded failure identification and description of the
failure. The original field failure reports should be
updated on receipt at the repair facility and be
followed up by official reports.
It is usual to employ suitably qualified personnel for
diagnosis in this area, and valuable data, in terms of
assignable cause, are normally available. Relevant
personnel are usually trained in preparing technical
submissions. Suitable arrangements should be made
to cater for items recycled in a production facility as
distinct from a separate repair facility.
5.3.10 Field installation, demonstration or
commissioning tests. In large systems such as power
stations, this phase can be significant for reliability
and maintainability. This may be the first time that
many of the components within a system, and whole
systems have been integrated; the consequent
interactions may be unexpected and can provide a
useful source of data. It should be borne in mind that
all failures being reported may be based on
configurations significantly different from those
planned for operation. The analysis of the report will
provide valuable insights into the likely reliability to
be achieved during operation.
Field installation, demonstration or commissioning
tests are usually well documented because contract
acceptance may be involved. Reports should attempt
to include both failure and usage data on the same
form so that support spares demand can be cross-
checked with failure reports, thus ensuring that the
‘support inventory’ is adjusted in a timely manner.
5.3.11 User reporting system. The usefulness of
the user reporting system as a service will depend on
how well it is organized, its efficiency, and of course
the relationship between the supplier and the user.
Visiting field engineers can often extract information
from a user which is of considerable value to the
supplier. Any such data should be fed back to the
design department accompanied by a copy of any
final test/commissioning test report and the field
engineer’s observations. In addition reports on
reliability and performance are sometimes available
from independent bodies.
5.3.12 Field surveys. Some manufacturers employ
specialist teams to carry out field surveys on the
reliability of their products.
5.4 DESIGNING THE DATA COLLECTION
FORM. The following points should be kept in mind
when designing a data collection form:
COPYRIGHT
AS 3960—1990
(a) The form should be simple and convenient to
use. It should carry a serial number and be
uniquely coloured for rapid identification.
Typical formats are shown in Figures 5, 6 and 7.
(b) It should be clearly stated that event date/data
relationship is required, i.e. the date the form
was completed, the date the failure was found,
the date of repairs, replacement, etc.
(c) The recording of the serial number is essential
for products whose life history is considered
important.
(d) It is essential for reliability purposes to record
time (or other measure of usage) based data.
Some products have elapsed time meters
installed, but care should be taken to ensure that
the times recorded are directly related to the item
being reported on. Multi-mode systems should be
fitted with meters for each major mode of
operation. The same method applies where log
cards or books are used for running time records.
(e) It is useful to provide a space for recording the
new stores items used to rectify the fault.
(f) A space for any narrative back-up report is
valuable but its use should not be mandatory.
Users should be encouraged to express
themselves freely in this part of the report,
including an indication of the probable cause of
failure, and to make recommendations. In
systems containing software, particular emphasis
should be placed on encouraging users to submit
a comprehensive narrative report.
(g) The minimum number of copies should be
demanded.
(h) The user should be provided with the facility to
classify the impact of the reported failure upon
the operation of the item/system e.g.
critical–major–minor.
(i) There should be a space on the form to allow the
user to identify the configuration at the relevant
time, e.g. at the time of a failure or a change to
the configuration.
NOTE: An example of a data collection form appli cable to
electr onic and similar engineeri ng use may be found in
AS 2529.
5.5 VALIDITY OF DATA.
5.5.1 Product manufacturer. Prototype items are
rarely identical with those that are eventually
produced. Results obtained from testing them should
be extrapolated with extreme caution if they are to be
used for assessing the field potential of the product.
The validity of results obtained from tests under
simulated conditions depends on how well the
simulation approximates real conditions. It is
important to ensure that spurious failure modes are
not induced under test, giving a misleading picture.
As a result of evolution, the life of a design can be
very short, so that when adequate data have been
collected, the results are no longer directly relevant.
This, and changes made to an item during
development, can create difficulties when relating test
data to current product specification.
In production testing, rapid feedback of results is
very important, but life tests are usually lengthy, as
the name implies. Methods for reducing the length of
tests will be needed (this applies equally to other
areas of testing), but care should also be taken to
AS 3960—1990
ensure the relevance of these tests. In general, in order
to reduce the time occupied by tests, accelerated tests are
employed, but in order to be valid an accelerated test
should not alter the basic modes for mechanisms of
failure, or their relative prevalence.
Data from endurance tests conducted by the
manufacturer and independent certified test houses may
be utilized. However, such data should relate to a known
manufacturing period and should preferably be capable
of verification by the purchaser. The question of the
validity of the test conditions remains.
For some products or installations the specifications will
include detailed descriptions of the conditions to which
the item will be subjected.
The design of adequate tests presents a problem if the
environment in which the product will be used is
either—
(a) unknown, e.g. electronic components; or
(b) known in a sense, but difficult to describe, e.g. in
the case of motor vehicles.
5.5.2 Materials or component supplier. Care should
be taken when incorporating data provided by material or
component suppliers into reliability calculations because,
for example—
(a) the supplier may misinterpret, or be unaware of its
proposed use;
(b) poor communication may exist between the supplier
and the user;
(c) the supplier may be optimistic; or
(d) supporting information may be lacking.
5.5.3 Field data retrieval programs. The integrity of
data obtained from the field is critically dependent upon
the expertise of the reporter or the efficiency of the
diagnostic tools used.
Often only failure data are recorded, especially for small
mass-produced items. Life of unfailed items may be
unknown. Reliability estimates may therefore show
worse reliability than is actually being achieved.
Conditions under which products fail may be completely
unknown or poorly documented. As these conditions
form an integral part of the reliability definition, lack of
knowledge concerning them makes estimation difficult.
On the other hand, for large items or installations, the
operational environment and the configuration is likely
to be well documented.
It may be difficult to distinguish between weakness and
misuse failures. Moreover, serviceable items may be
reported as having failed as a result of faulty diagnosis,
bad maintenance, etc and products that have not failed
may be mistakenly removed from service.
Except in the case of products with open or coded
manufacturing dates, it is often impossible to connect
failure reports with date of manufacture. Also, unless
appropriate records are kept, it is difficult to relate a
failure to the date of entry into service (as distinct from
manufacturing date, which is rarely the same). It may
also be difficult to obtain accurate data on other
measures of usage, e.g. number of motor starts, number
of runs of a computer program, etc.
For mass-produced products, the amount of data that
could be collected is very large, but the cost of collection
compared with the value of the product may also be
COPYRIGHT
36
large. The amount of data actually gathered will be
influenced by cost of collection and transmission.
5.6 COLLECTION AND FLOW OF RELIABILITY
DATA. The timely analysis of all discrepancy or failure
reports by a team formally constituted by management
enables the basic or underlying causes of failure in parts,
materials, processes and procedures to be determined.
The analysis should include failures in design,
manufacture, procurement, quality control, maintenance
and operation. The resulting failure analyses should be
fed back to design, production and management
personnel for action.
It is essential that the route by which data are fed back
is clearly defined. Some means should be established to
co-ordinate all those concerned with the investigation of
failures and the follow-up actions necessary.
It is essential that reports from all other sources, e.g.
from the operating authority, user or maintenance-repair
organization, are brought together and co-ordinated
before analysis to avoid multiple reporting of a single
fault.
It is generally accepted that the overwhelming majority
of failures, down time or cost associated with a product
is usually associated with relatively few modes of failure.
It may be cost effective to collect fairly coarse data on
everything, sort them and so pinpoint the major problem
areas under each of the three headings: failure rate, down
time and cost. These areas can then be more thoroughly
investigated by acquiring more comprehensive data. In
this way limited resources can be directed in a
cost-effective manner.
5.7 ANALYSIS OF DATA.
5.7.1 Quantitative data. The nature and quality of the
data should be firmly established. The degree of
sophistication of the analysis should never be more than
the intrinsic accuracy which the basic information allows.
Appropriate characteristics of reliability should be
chosen, e.g. useful life, mean time to first failure, mean
time between failures, percentile life, failure rate and
availability.
A relevant statistical analysis method should be chosen,
which may range from merely counting numbers of
failures to establishing assessed characteristics.
The assumptions made for the analysis should be
carefully verified, e.g. whether a constant failure rate is
applicable.
Care should be taken to qualify the chosen reliability
characteristic by the use for the words ‘observed’,
‘assessed’, etc, as appropriate.
The statistical distribution characteristics and overall
rates of both failures and repairs will be vitally affected
by the conditions of use, e.g. the maintenance schedule
and the extent to which the schedule is fulfilled. It is
important to take into account statistical uncertainties.
5.7.2 Qualitative data. The descriptive information
should be analyzed in terms of failure mechanism, design
or manufacturing procedures and characteristics. If no
quantitative data are available, this information will serve
to indicate the nature of any problem that may exist and
the seriousness of the effects, but not the extent of the
problem.
If quantitative data do exist, the qualitative information
should serve to give greater appreciation of the problem.
 37
5.7.3 Requirements specifications. In complex
systems, especially those containing software, it is
common for failure reports to record that the system
does not perform ‘as reasonably expected’ by the
user, even though the system meets its specified
requirements.
This arises from the scale of the task in totally
specifying the requirements of the user. Judgement
should therefore be used as to which of these failure
reports should be included in the analysis.
5.8 FAILURE CLASSIFICATION. When using
failure reports for any purpose it is necessary to
define what constitutes a failure; it is also desirable
to classify the failure in terms of its relevance to the
task in hand.
The two main purposes of classifying failures are—
(a) to determine those types of failure that should be
taken into account in order to estimate the
reliability characteristics; and
(b) to determine those types of failure that call for
corrective action.
Failures are divided into two broad classifications as
follows:
(i) Relevant failure.
(ii) Non-relevant failure.
NOTE: See Appendix A of this Standard for failure
classifi cati ons.
Depending on the precise purpose of the
assessment and the agreement of all interested
parties, the non-relevant category could include
the following:
(A) Failures during run-in, burn-in or screening.
(B) Failures induced by operator error, although
these may be reduced by design.
COPYRIGHT
AS 3960—1990
(iii) Failures caused by faulty peripheral equipment,
test equipment, etc.
(iv) Secondary failures.
(v) Failures already classified, for which steps
towards corrective action have been taken and
proven effective.
5.9 INTERPRETATION AND PRESENTATION
OF DATA. The main problem associated with
interpretation of reliability data is the requirement for
the extrapolation of results to other situations; this
problem arises, for instance, because—
(a) differences usually exist between test conditions
and field conditions of use, or between the times
used in the tests and those used in the field;
(b) it may be necessary to use results from tests (rig
or field) on one product mark, grade or type to
assess another product mark, grade or type; or
(c) an assessment of the potential of a new design
may be required from historical data.
The analysis should demonstrate, as far as possible,
the effects and validity of extrapolation, and should
take account of the levels of understanding of the
intended users, and of the uses to which the data may
be put.
Quantitative data should be summarized in terms of
relevant reliability values. It should be presented as
collections of single values, in tabular form or in
graphs, the latter showing the relationship between
reliability and time or other measure of usage. To the
extent applicable to the product under consideration
the data should be presented in accordance with
AS 2530. The source of the data and the
configuration of the product should be stated. The
analytical method should be mentioned as a reference
or described fully as an appendix to the main report.
Qualitative data should include written descriptions,
photographs, drawings and actual specimens, if
possible.
AS 3960—1990 38

APPENDIX A
TERMS FOR RELIABILITY AND MAINTAINABILITY
(This Appendix forms an integral part of this Standard.)

A1 SCOPE. This Appendix provides a listing of terms for reliability and maintainability which
are not found in AS 1057. The terms have been taken without amendment from IEC Publication 271.
A complete index of terminology is found in Appendix C.

A2 CLASSIFICATION OF FAILURES AS TO CAUSE.

A2.1 Misuse failure—failure attributable to the application of stresses beyond the stated
capabilities of the item.
A2.2 Inherent weakness failure—failure attributable to weakness inherent in the item itself when
subjected to stresses within the stated capabilities of the item.

A3 CLASSIFICATION OF FAILURE AS TO SUDDENNESS.

A3.1 Sudden failure—failure that could not be anticipated by prior examination or monitoring.
A3.2 Gradual failure—failure that could be anticipated by prior examination or monitoring.

A4 CLASSIFICATION OF FAILURE AS TO DEGREE.

A4.1 Partial failure—failure resulting from deviations in characteristic(s) beyond specified limits,
but not such as to cause complete lack of the required function.
NOTE: The limit s referred to in this category are specifi ed for this purpose.
A4.2 Complete failure—failure resulting from deviations in characteristic(s) beyond specified
limits such as to cause complete lack of the required function.
NOTE: The limit s referred to in this category are special li mits specif ied for this purpose.
A4.3 Intermittent failure—failure of an item for a limited period of time, following which the
item recovers its ability to perform its required function without being subjected to any external
corrective action.
NOTE: Such a failure is often recurrent.

A5 CLASSIFICATION OF FAILURE IN COMBINATION OF SUDDENNESS AND DEGREE.

A5.1 Catastrophic failure—failure which is both sudden and complete.
A5.2 Degradation failure—failure which is both gradual and partial.
NOTE: In time, such a failure may develop into a complete fail ure.

A6 RELIABILITY CHARACTERISTICS.
A6.1 Observed reliability.
(a) Of non-repaired items—for a stated period of time, the ratio of the number of items which
performed their functions satisfactorily at the end of the period to the total number of items in
the sample at the beginning of the period.
(b) Of repaired item or items—the ratio of the number of occasions on which an item or items
performed their functions satisfactorily for a stated period of time to the total number of
occasions the item or items were required to perform for the same period.
NOTE: The cri teri a for what constit utes satisfactory function need to be stated.
A6.2 Assessed reliability—the reliability of an item determined by a limiting value or values of
the confidence interval associated with a stated confidence level, based on the same data as the
observed reliability of nominally identical items.
NOTES:
1. The source of the data needs to be stated.
2. Result s can be accumulated (combined) only when all condit ions are similar.
3. The assumed underl ying distr ibution of failures against time needs to be stated.
4. It should be stated whether a one-sided or a two-sided interval is being used.
5. Where only one limit ing value is given, this is usually the lower limit .
A6.3 Extrapolated reliability—extension by a defined extrapolation or interpolation of the
observed or assessed reliability for durations or conditions different from those applying to the
observed or assessed reliability.
NOTE: The validit y of the extr apolation needs to be justi fi ed.

COPYRIGHT
 39 AS 3960—1990

A6.4 Predicted reliability—for the stated conditions of use, and taking into account the design of
an item, the reliability computed from the observed, assessed, or extrapolated reliabilities of its
parts.
NOTE: Engineeri ng and statisti cal assumptions have to be stated, as well as the bases used for the computation (observed
or assessed).

A7 MEAN LIFE.
A7.1 Observed mean life—the mean value of the lengths of observed times to failure of all items
in a sample under stated conditions.
NOTE: The cri teri a for what constit utes a failure need to be stated.
A7.2 Assessed mean life—the mean life of an item determined by a limiting value or values of the
confidence interval associated with a stated confidence level, based on the same data as the observed
mean life of nominally identical items.
NOTES:
1. The source of the data needs to be stated.
2. Result s can be accumulated (combined) only when all condit ions are similar.
3. The assumed underl ying distr ibution of failures against time needs to be stated.
4. It should be stated whether a one-sided or a two-sided interval is being used.
5. Where only one limit ing value is given, this is usually the lower limit .
A7.3 Extrapolated mean life—extension by a defined extrapolation or interpolation of the
observed or assessed mean life for stress conditions different from those applying to the observed
or assessed mean life.
NOTE: The validit y of the extr apolation needs to be justi fi ed.
A7.4 Predicted mean life—for the stated conditions of use, and taking into account the design of
an item, the mean life computed from the observed, assessed or extrapolated mean life of its parts.
NOTE: Engineeri ng and stati stical assumpti ons need to be stated, as well as the bases used for the computation (observed
or assessed).

A8 FAILURE RATE.
A8.1 Observed failure rate—for a stated period in the life of an item, the ratio of the total number
of failures in a sample to the cumulative observed time on that sample. The observed failure rate
is to be associated with particular and stated time intervals (or summation of intervals) in the life
of the items, and with stated conditions.
NOTES:
1. The crit eria for what consti tutes a failure need to be stated.
2. Cumulati ve ti me is the sum of the ti mes during which each individual it em has been performing its required functi on
under stated condit ions.
A8.2 Assessed failure rate—the failure rate of an item determined by a limiting value or values
of the confidence interval associated with a stated confidence level, based on the same data as the
observed failure rate of nominally identical items.
NOTES:
1. The source of the data needs to be stated.
2. Result s can be accumulated (combined) only when all condit ions are similar.
3. The assumed underl ying distr ibution of failures against time needs to be stated.
4. It should be stated whether a one-sided or a two-sided interval is being used.
5. Where only one limit ing value is given, this is usually the upper limit .
A8.3 Extrapolated failure rate—extension by a defined extrapolation or interpolation of the
observed or assessed failure rate for durations or conditions different from those applying to the
observed or assessed failure rate.
NOTE: The validit y of the extr apolation needs to be justi fi ed.
A8.4 Predicted failure rate—for the stated conditions of use, and taking into account the design
of an item, the failure rate computed from the observed, assessed or extrapolated failure rates of its
parts.
NOTE: Engineeri ng and statisti cal assumptions need to be stated, as well as the bases used for the computation (observed
or assessed).
A8.5 Assumed failure rate—the failure rate of an item, based on experience in relevant
technology, when observed or assessed failure rates are not available.
A8.6 Failure rate level—for the assessed failure rate, a value chosen from a series of failure rate
values and used for stating requirements or for the presentation of test results.
NOTES:
1. In a requirement, it denotes the highest permissible assessed failure rate.
2. In interpreti ng test result s, it is the value in the series immediately higher than the assessed fail ure rate.

COPYRIGHT
AS 3960—1990 40

A9 MEAN TIME TO FAILURE.

A9.1 Observed mean time to failure—for a stated period in the life of an item, the ratio of the
cumulative time for a sample to the total number of failures in the sample during the period, under
stated conditions.
NOTES:
1. The crit eria for what consti tutes a failure need to be stated.
2. Cumulati ve ti me is the sum of the ti mes during which each individual item has been performing it s required functi on
under stated condit ions.
3. This is the reciprocal of the observed failure rate duri ng the peri od.
A9.2 Assessed mean time to failure—the mean time to failure of an item determined by a limiting
value or values of the confidence interval associated with a stated confidence level, based on the
same data as the observed mean time to failure of nominally identical items.
NOTES:
1. The source of the data needs to be stated.
2. Result s can be accumulated (combined) only when all condit ions are similar.
3. The assumed underl ying distr ibution of failures against time needs to be stated.
4. It should be stated whether a one-sided or a two-sided interval is being used.
5. Where only one limit ing value is given, this is usually the lower limit .
A9.3 Extrapolated mean time to failure—extension by a defined extrapolation or interpolation
of the observed or assessed mean time to failure for durations or conditions different from those
applying to the observed or assessed mean time to failure.
NOTE: The validit y of the extr apolation needs to be justi fi ed.
A9.4 Predicted mean time to failure—for the stated conditions of use, and taking into account
the design of an item, the mean time to failure computed from the observed, assessed or extrapolated
mean times to failure of its parts.
NOTE: Engineeri ng and stati stical assumpti ons need to be stated, as well as the bases used for the computati on (observed
or assessed).

A10 MEAN TIME BETWEEN FAILURES.

A10.1 Observed mean time between failures—for a stated period in the life of an item, the mean
value of the length of time between consecutive failures, computed as the ratio of the cumulative
observed time to the number of failures, under stated conditions.
NOTES:
1. The crit eria for what consti tutes a failure need to be stated.
2. Cumulati ve ti me is the sum of the ti mes during which each individual it em has been performing it s required functi on
under stated condit ions.
3. This is the reciprocal of the observed failure rate duri ng the peri od.
A10.2 Assessed mean time between failures—the mean time between failures of an item
determined by a limiting value or values of the confidence interval associated with a stated
confidence level, based on the same data as the observed mean time between failures of nominally
identical items.
NOTES:
1. The source of the data needs to be stated.
2. Result s can be accumulated (combined) only when all condit ions are similar.
3. The assumed underl ying distr ibution of failures against time needs to be stated.
4. It should be stated whether a one-sided or a two-sided interval is being used.
5. Where only one limit ing value is given, this is usually the lower limit .
A10.3 Extrapolated mean time between failures—extension by a defined extrapolation or
interpolation of the observed or assessed mean time between failures for duration or conditions
different from those applying to the observed or assessed mean time between failures.
NOTE: The validit y of the extr apolation needs to be justi fi ed.
A10.4 Predicted mean time between failures—for the stated conditions of use, and taking into
account the design of an item, the mean time between failures computed from the observed, assessed
or extrapolated failure rates of its parts.
NOTE: Engineeri ng and stati stical assumpti ons need to be stated, as well as the bases used for the computati on (observed
or assessed).

A11 DATA CONCEPTS.

A11.1 Test data—data from observations during tests.
NOTE: All condit ions should be stated in detail , for example: ti me, str ess condit ions and failure or success crit eria.

COPYRIGHT
 41 AS 3960—1990

A11.2 Field data—data from observations during field use.

NOTE: The time, str ess conditi ons and fail ure or success cri teri a should be stated in detail.
A11.3 Accelerated test—a test in which the applied stress level is chosen to exceed that stated in
the reference conditions in order to shorten the time required to observe the stress response of the
item, or magnify the response in a given time. To be valid, an accelerated test shall not alter the
basic modes and mechanisms of failure, or their relative prevalence.
A11.4 Acceleration factor—the ratio between the times necessary to obtain the same stated
proportion of failures in two equal samples under two different sets of stress conditions involving
the same failure modes and mechanisms.
A11.5 Failure rate acceleration factor—the ratio of the accelerated testing failure rate to the
failure rate under stated reference test conditions. Both failure rates refer to the same time period
in the life of the tested items.
A11.6 Step stress test—a test consisting of several stress levels applied sequentially for periods
of equal duration to one sample. During each period, a stated stress level is applied and the stress
level is increased from one period to the next.
A11.7 Screening test—a test, or combination of tests, intended to remove unsatisfactory items or
items likely to exhibit early failures.
A11.8 Laboratory reliability test—a reliability compliance or determination test made under
prescribed and controlled operating and environmental conditions which may or may not simulate
field conditions.
A11.9 Field reliability test—a reliability compliance or determination test made in the field where
operating, environmental, maintenance and measurement conditions are recorded.

A12 DESIGN CONCEPTS.

A12.1 Redundancy—in an item, the existence of more than one means of performing a given
function.
A12.2 Active redundancy—that redundancy where all means for performing a given function are
operating simultaneously.
A12.3 Standby redundancy—that redundancy where the alternative means for performing a given
function are inoperative until needed.
A12.4 Storage life—the length of time an item can be stored under specified conditions and still
meet the specified requirements.
A12.5 Wear-out—the process of attrition which results in increase of the failure rate with
increasing age, expressed in time, cycles, kilometres, events, etc, as applicable to the item.
A12.6 Rating—the recommended limiting value of an operating condition or parameter.
A12.7 Burn in—the operation of an item prior to its ultimate application, intended to stabilize its
characteristics and to identify early failures.
A12.8 De-bugging—the operation of a complex item prior to its use, to detect and replace parts
which are defective or expected to fail, and to correct errors occurring in production.

A13 TIME CONCEPTS.

A13.1 Active preventive maintenance time—the period of preventive maintenance time during
which preventive maintenance actions are performed on an item either manually or automatically,
including the time due to technical delays inherent in the maintenance action.
A13.2 Administrative time—the period of time during which maintenance actions are pending or
are prepared but are not yet initiated or are suspended.
A13.3 Undetected failure time—the period of time between the instant at which failure occurs and
its recognition. Dormant failures could occur during this period.
A13.4 Maintenance time—the period of time during which maintenance actions are performed on
an item either manually or automatically, including the time due to delays inherent in the
maintenance action.
NOTES:
1. The inherent delays could, for example, include those due to the design or to prescribed maintenance procedures.
2. Maintenance action may be carri ed out whil e the it em is perf orming a required functi on.

COPYRIGHT
AS 3960—1990 42

A13.5 Preventive maintenance time—the period of maintenance time during which preventive
maintenance is performed on an item, including the time due to delays inherent in the preventive
maintenance action.
NOTES:
1. Test procedures that are involved as part of preventive maintenance ti me may be performed either manually or
automati call y.
2. Preventi ve maintenance time does not include ti me taken to maintain a removed item which has been replaced as part
of the preventive maintenance acti on.
A13.6 Corrective maintenance time—the period of maintenance time during which corrective
maintenance is performed on an item, including the time due to delays inherent in the corrective
maintenance action.
NOTES:
1. Test procedures that are involved as part of corr ecti ve maintenance time may be perf ormed eit her manuall y or
automati call y.
2. Corr ecti ve maintenance ti me does not include time taken to repair a removed item which has been replaced as part of
the corrective maintenance acti on.

A14 Q-PERCENTILE LIFE.

A14.1 Observed Q-percentile life—the length of observed time at which a stated proportion (Q%)
of a sample of items has failed.
NOTES:
1. The crit eria for what consti tutes a failure should be stated.
2. The Q-percenti le li fe is also that lif e at which (100 - Q)% reli abil it y is observed.
A14.2 Assessed Q-percentile life—the Q-percentile life determined as a limiting value or values
of the confidence interval with a stated confidence level, based on the same data as the observed
Q-percentile life of nominally identical items.
NOTES:
1. The source of the data should be stated.
2. Result s can be accumulated (combined) only when all condit ions are similar.
3. The assumed underl ying distr ibution of failures against time should be stated.
4. It should be stated whether a one-sided or two-sided interval is being used.
5. Where one li miti ng value is given this is usually the lower li mit.
A14.3 Extrapolated Q-percentile life—extension by a defined extrapolation or interpolation of the
observed or assessed Q-percentile life for stress conditions different from those applying to the
assessed Q-percentile life and for different percentages.
NOTE: The validit y of the extr apolation should be justi fi ed.
A14.4 Predicted Q-percentile life—for the stated conditions of use, and taking into account the
design of an item, the computed Q-percentile life based on the observed, assessed or extrapolated
Q-percentile lives of its parts.
NOTE: Engineering and stati stical assumption should be stated, as well as the bases used for the computation (observed
or assessed).

A15 MAINTENANCE CONCEPTS.

A15.1 Level of maintenance—the type of maintenance actions to be carried out at a stated
indenture level of an item.
NOTE: The cri teri a for deciding a level of maintenance can be the complexit y of the item’s construction, accessibil it y to
part s of the it em, skill level of maintenance personnel, test equipment facili ti es, safety considerations, etc.
A15.2 Line of maintenance—the position in an organization at which the maintenance of an item
is to be carried out at stated levels of maintenance.
NOTE: The posit ion is characterized by the skil l of the personnel, the facili ti es available, the location, etc.

COPYRIGHT
 43 AS 3960—1990

APPENDIX B
EXPLANATION OF RELIABILITY AND MAINTAINABILITY
TERMINOLOGY
(This Appendix does not form an integral part of this Standard.)

B1 GENERAL–PRINCIPLES. Some of the definitions contained in Appendix A of this Standard

are more precise than are sometimes found in current literature. It is therefore advisable to be aware
of restrictions in the meaning of some definitions compared with current use.
The selection and arrangement of the terms involve several principles of which the following are
the most important:
(a) The starting point for the choice and definition of each term is a clarification of the concept
involved.
(b) The terms are logically related to one another as far as possible, and their definitions are
consistent.
(c) Where several versions of a term are possible, each is given a separate title and its relationship
to the other versions is described.
(d) Due to the widely varied circumstances to which reliability and maintainability terms can be
applied, the words ‘under stated conditions’ are used to remind readers that in every case the
relevant conditions have to be stated to whatever extent is judged necessary.
(e) The terms are grouped under conceptual headings rather than in alphabetical order to enable
a common arrangement to be used for any language.

B2 FAILURE TERMS. The word ‘failure’ is the basic term denoting the termination of required
performance. It is applied to parts and equipment in all circumstances.
In practice, some kinds of failures are more important than others, and it is necessary to distinguish
between them. This is done by adding an adjectival modifier to the word ‘failure’. For example,
complete failure, misuse failure and so on.
A failure term applies only to the item under consideration. Thus a complete failure of a particular
part may cause only a partial failure of the equipment in which it is used.

B3 THE NEED TO USE THE APPROPRIATE ADJECTIVAL MODIFIER. When dealing with
data, it is important to use the appropriate failure term. For example, in the case of failure data from
equipment in the field, it is important to distinguish between parts which fail as the result of misuse
and those whose failure is due to a weakness in the part itself. Failure statements should therefore
indicate which kind of failure is being considered.

B4 GENERAL STATEMENT ON THE PROBABILISTIC NATURE OF RELIABILITY.

Reliability is concerned with the probability of future events based on past observations. Any
reliability characteristic term may thus be used in respect of what has been observed and what may
happen, the latter use being quantified in terms of a probability.

B5 RELIABILITY CHARACTERISTICS. Reliability characteristics are quantities used to

express various aspects of reliability in numerical terms. The emphasis of a particular reliability
characteristic may be either from the viewpoint of success (i.e. performing a function) or of failure.
The characteristics used include reliability, mean life, failure rate, mean time between failures and
mean time to failure. The observed versions are estimates of the ‘true values’ of the characteristics,
but not necessarily the ‘best estimates’ in the statistical sense.

B6 VERSIONS OF RELIABILITY CHARACTERISTICS. The significance of reliability

characteristics–for example, failure rate–depends on the amount of data collected, the statistical
treatment and the technical assumptions made in particular circumstances. A different name is used
when statistical treatment differs, but in any case the set of assumptions and circumstances should
be stated. For example, the term ‘observed failure rate’ relates to the failure rate observed under a
given set of circumstances, and by definition the number of specimens, duration and failure
definition should all be stated. If these data are subjected to appropriate statistical treatment, the
result is the limiting values of a confidence interval with a stated confidence level. This is called
the assessed failure rate.

COPYRIGHT
AS 3960—1990 44

Where the adjective ‘assessed’ is used, this is to be understood in the statistical sense.
Two approaches to publication of reliability characteristics are possible. The first, which is used in
this Standard, is one in which each term (e.g. failure rate) is associated with an adjective (e.g.
assessed). The second is where, the basic reliability characteristics are defined individually and the
adjectives are explained separately. When numerical reliability data are quoted, the results of both
these approaches will be identical if they are used correctly, since both require complete information
(e.g. stress and time conditions, and failure definitions).
B7 ACHIEVED AND REQUIRED RELIABILITY CHARACTERISTICS. The four versions
(observed, assessed, extrapolated and predicted) of a reliability characteristic may be used for what
has been achieved or what is required. Thus a statement of a requirement may be used as a target
for the observed, assessed, etc failure rate.

B8 PRACTICAL APPLICATION OF APPENDIX A. In practice, some of the terms in Appendix

A have to be given greater precision by being defined in accordance with the characteristics of the
item under consideration. Thus the criteria of a partial failure of a transistor, for example, would
usually be given as the limit values for, or limit changes in, gain or leakage current. In some cases,
further distinctions may be desirable, for example, major and minor partial failures. Distinctions of
this kind may be made by building up on Appendix A of this Standard as a foundation.

B9 MEANING OF THE TERM ‘ITEM’. The words ‘an item’ are also used to denote any part,
sub-system, system or equipment that can be individually considered and separately tested. The word
‘item’ is also used to denote items, population of items, sample, etc, where the context justifies its
use.
Where the expression ‘repaired item’ is used, it means an item which is intended to be repaired
when it fails. Similarly the term ‘non-repaired item’ means an item which is not intended to be
repaired when it fails.

B10 ‘HARDWARE’ TERMS. The choice, definition and relationship of the principal ‘hardware’
terms such as part, equipment, system, etc are strongly influenced by the viewpoint adopted in
different countries, and by different organizations in one country. It has been found impracticable
to provide a universally acceptable set of ‘hardware’ terms and definitions. Therefore, in any
practical reliability study involving ‘hardware’ terms, the definitions and hierarchy of the terms used
should be stated and should not be changed during the study.

B11 TIME CONCEPTS. In definitions where ‘time’ is used, this parameter may be replaced by
distance, cycles or other quantities or units as may be appropriate. This concept may cover any
duration of observation of the considered items, either in actual operation or in storage, readiness,
etc, but it generally excludes down time due to a failure.

COPYRIGHT
 45 AS 3960—1990

APPENDIX C
INDEX OF TERMS FOR RELIABILITY AND MAINTAINABILITY
(This Appendix does not form an integral part of this Standard.)

accelerated test AS 3960 A11.3

acceleration factor AS 3960 A11.4
active preventive maintenance time AS 3960 A13.1
active redundancy AS 3960 A12.2
active repair time AS 1057 10.3.14
administrative time AS 3960 A13.2
assessed failure rate AS 3960 A8.2
assessed mean life AS 3960 A7.2
assessed mean time between failures AS 3960 A10.2
assessed mean time to failure AS 3960 A9.2
assessed Q-percentile life AS 3960 A14.2
assessed reliability AS 3960 A6.2
assumed failure rate AS 3960 A8.5
audit AS 1057 3.4
availability AS 1057 11.1
burn in AS 3960 A12.7
catastrophic failure AS 3960 A5.1
complete failure AS 3960 A4.2
condition monitoring AS 1057 10.2.4
conditioning AS 1057 9.6.1
constant failure rate period AS 1057 9.5.3
corrective maintenance AS 1057 10.2.3
corrective maintenance time AS 3960 A13.6
de-bugging AS 3960 A12.8
degradation failure AS 3960 A5.2
de-rating AS 1057 4.2.14
down time AS 1057 10.3.2
durability AS 1057 9.2.2
early failure period AS 1057 9.5.2
endurance test AS 1057 9.6.2
environment AS 1057 9.6.3
environmental test AS 1057 9.6.4
extrapolated Q-percentile life AS 3960 A14.3
extrapolated failure rate AS 3960 A8.3
extrapolated mean life AS 3960 A7.3
extrapolated mean time between failures AS 3960 A10.3
extrapolated mean time to failure AS 3960 A9.3
extrapolated reliability AS 3960 A6.3
fail safe AS 1057 4.1.5
failure AS 1057 9.3.1
failure analysis AS 1057 9.3.5
failure cause AS 1057 9.3.2
failure mechanism AS 1057 9.3.4
failure mode AS 1057 9.3.3
failure rate AS 1057 9.5.1
failure rate acceleration factor AS 3960 A11.5
failure rate level AS 3960 A8.6
fault tree analysis AS 1057 9.3.6
field data AS 3960 A11.2
field reliability test AS 3960 A11.9
free time AS 1057 10.3.6
gradual failure AS 3960 A3.2
inherent weakness failure AS 3960 A2.2
intermittent failure AS 3960 A4.3
laboratory reliability test AS 3960 A11.8
level of maintenance AS 3960 A15.1
line of maintenance AS 3960 A15.2

COPYRIGHT
AS 3960—1990 46

maintainability AS 1057 10.1.1

maintenance AS 1057 10.2.1
maintenance time AS 3960 A13.4
mean life AS 1057 9.7.1
mean time between failures (MTBF) AS 1057 9.7.3
mean time to failure (MTTF) AS 1057 9.7.2
mean time to first failure (MTTFF) AS 1057 9.7.4
mean time to repair (MTTR) AS 1057 10.3.16
misuse failure AS 3960 A2.1
non-relevant failure AS 1057 9.4.8
non-required time AS 1057 10.3.5
observed Q-percentile life AS 3960 A14.1
observed failure rate AS 3960 A8.1
observed mean life AS 3960 A7.1
observed mean time between failures AS 3960 A10.1
observed mean time to failure AS 3960 A9.1
observed reliability AS 3960 A6.1
operating time AS 1057 10.3.3
partial failure AS 3960 A4.1
predicted Q-percentile life AS 3960 A14.4
predicted failure rate AS 3960 A8.4
predicted mean life AS 3960 A7.4
predicted mean time between failures AS 3960 A10.4
predicted mean time to failure AS 3960 A9.4
predicted reliability AS 3960 A6.4
preventive maintenance AS 1057 10.2.2
preventive maintenance time AS 3960 A13.5
primary failure AS 1057 9.4.5
quality system AS 1057 2.2.1
rating AS 3960 A12.6
redundancy AS 3960 A12.1
relevant failure AS 1057 9.4.7
reliability AS 1057 9.2.1
reliability compliance test AS 1057 9.6.5
reliability determination test AS 1057 9.6.6
reliability growth testing AS 1057 9.6.7
required time AS 1057 10.3.4
screening test AS 3960 A11.7
secondary failure AS 1057 9.4.6
serviceability AS 1057 10.1.2
standby redundancy AS 3960 A12.3
standby time AS 1057 10.3.7
step stress test AS 3960 A11.6
storage life AS 3960 A12.4
sudden failure AS 3960 A3.1
test data AS 3960 A11.1
testability AS 1057 10.1.3
undetected failure time AS 3960 A13.3
up time AS 1057 10.3.1
useful life AS 1057 9.5.5
wear-out failure AS 1057 9.4.1
wear-out AS 3960 A12.5
wear-out failure period AS 1057 9.5.4

COPYRIGHT
This page has been left intentionally blank.