Revision of Australian Defence Standard Def (Aust) 5679

Tony Cant∗ , Brendan Mahony∗ , Brenton Atchison†

∗
Information Networks Division
Defence Science and Technology Organisation
PO Box 5111
Edinburgh 5108, South Australia
Email: [Tony.Cant, Brendan.Mahony]@dsto.defence.gov.au

†
Invensys Rail Systems Australia
10 Brandl Street
Eight Mile Plains 4113, Brisbane, Australia
Email: Brenton.Atchison@invensys.com

Abstract the further development of Def (Aust) 5679, with sup-

The Australian Defence Standard Def (Aust) 5679,
entitled “The Procurement of Computer-Based
Safety-Critical Systems”, was originally published in
August, 1998. Def (Aust) 5679 is currently undergo-
ing revision. This paper describes the standard and
discusses the issues that have been highlighted during
the revision process.
Keywords: Safety Critical Systems, Software, Assur-
ance, Trust, Safety.
1 Introduction
The Australian Defence Standard Def (Aust) 5679,
entitled “The Procurement of Computer-Based
Safety-Critical Systems”, describes procedures and
practices to be carried out during system development
that are intended to provide sufficient assurance of
system safety (AEA 1998). It was written by the De-
fence Science and Technology Organisation (DSTO)
in response to what were (and still are) perceived
as deficiencies in extant military and civilian safety
standards, following extensive consultation with gov-
ernment, industry and academic experts, both within
Australia and overseas. Def (Aust) 5679 was ori-
ginally published by the former Army Engineering
Agency (AEA) in August, 1998.
Def (Aust) 5679 has been applied to a number of
Defence projects within Australia. This practical ex-
perience has been very beneficial, and has exposed
areas where there may be difficulty applying the
standard. In 2001, the Defence Materiel Organisation
(DMO) contracted the Software Verification Research
Centre (SVRC) of the University of Queensland to
provide support and advice on safety management to
DMO projects, to review international safety stand-
ards, and to carry out a critical review of Def (Aust)
5679. This project (called “DefSafe”) resulted in sev-
eral technical studies on Def (Aust) 5679, comparing
it to international safety standards, and highlighting
areas where the standard needed improvement.
The Standardisation Branch of the DMO has
formed a Standardisation Working Group to guide
Copyright
2005,
c Australian Government Department of De-
fence. This paper appeared at the 10th Australian Workshop
on Safety-Related Programmable Systems, Sydney. Confer-
ences in Research and Practice in Information Technology, Vol.
XX. Tony Cant, Ed. Reproduction for academic, not-for-profit
purposes is permitted provided this text is included.
port from Invensys Rail Systems Australia (IRSA)
under a contract established in June 2004. The Work-
ing Group consists of representatives from the DMO,
DSTO and Invensys, as well as representatives from
Australian Army and Navy Technical Regulatory Au-
thorities. The revision process took the DefSafe re-
ports as a starting point, as well as the experience
gained by the use of Def (Aust) 5679 on the Safety
Case for the autopilot for the Nulka decoy (Nulka is a
joint Australian/US ship defence system). The pro-
cess involved close interaction between Invensys and
DSTO. The Working Group had regular meetings to
review recommendations and to discuss key procure-
ment issues. It should be noted that consensus on
certain issues could not be reached by the Standard-
isation Working Group, and that some key recom-
mendations made by IRSA were not adopted in the
revised standard.
This paper describes the structure of the revised
standard, as well as the revision process and issues
raised.
2 Background
The current revision of Def (Aust) 5679 is being car-
ried out against the background of a major paradigm
shift in military safety standards: the move to goal-
based standards. These are standards that focus on
broad principles and requirements to be placed on de-
velopers, and are not prescriptive about the processes,
methods and tools that should be used during system
development. This trend began in the USA under
the umbrella of acquisition reform, which was motiv-
ated by the desire to place responsibility for systems
(including safety) with defence contractors. Acquis-
ition reform also involved the replacement of milit-
ary standards by civilian ones where possible. Thus,
for example, the US Military Standard MIL-STD
882C (US DoD 1993) evolved into MIL-STD 882D
(US DoD 2000) that focused on broad requirements
for system safety and did not mandate a particular
process for achieving these requirements (although
some guidance was provided in a non-normative an-
nex to the standard). This standard is currently be-
ing revised; draft versions of MIL-STD-882E have
apppeared (US DoD 2004).
More recently, the UK MoD has published Is-
sue 3 of the Interim Defence Standard 00-56 (UK
MoD 2004) This is a goal-based standard that starts
from a number of high-level principles, and amplifies
 Customer
Developer
End Users
Figure 1: Safety Management Agents
these into requirements on system developers. It is
very much less prescriptive than the previous issue of
the standard, and was produced following wide con-
sultation. One major change is that Safety Integrity
Levels (SILs), although used in earlier versions, are
no longer mandated by the standard. However, some
guidance on the use of safety integrity requirements -
to define the variation of the degree of rigour of sys-
tem analysis with potential safety risk - is provided
in Annex C.
Def (Aust) 5679 is also based on fundamental
safety principles. However, in the authors’ opinion
the time is not yet ripe for a purely goal-based safety
standard, for the following reasons. Safety critical
systems engineering is still in its infancy and thus
in many cases system developers will need to seek
further guidance beyond that given in a goal-based
standard in order to produce a safety case. Such
guidance may be difficult to obtain and may also be
inconsistent with the spirit of the standard. Also, it
will take some time to determine whether or not the
adoption of goal-based standards is successful. Thus
it is not intended that Def (Aust) 5679 follow overseas
trends and become purely goal-based.
Rather, the objectives of the revision are to ad-
dress specific technical issues and feedback from ex-
perience to assist its application to Australian De-
fence projects. The challenge is to make the stand-
ard generic (so that it applies to as many systems as
possible), practical (with a clear description of the re-
quirements on developers and evaluators) and effect-
ive (in that application of the standard will provide
sufficient assurance).
In the following sections, the structure of Def
(Aust) 5679 will be described in detail.
System
Safety Evaluator
Case
Auditor Certifier
3 Safety Management
Def (Aust) 5679 provides requirements and guidance
on safety program management, including planning,
the safety lifecycle, monitoring and reviews.
A key feature of the safety management require-
ments is the identification of agents and their roles
and responsibilities (see Figure 1). The Customer is
the organisation (usually a Systems Program Office)
within the DoD that is procuring the system. The
Developer is the organisation (usually a commercial
one) that is responsible for producing the system re-
quired by the Customer according to the Standard. In
most cases, a Certifier will need to accept the system
as being safe and suitable for service. The Auditor
provides oversight (on behalf of the Certifier) of the
safety management process. The Evaluator is an in-
dependent organization responsible for checking the
detail and validity of the Developer’s arguments that
the system will meet its safety critical requirements.
The Customer will often seek to involve End Users,
who provide input on the safety aspects of user re-
quirements and on user interface and performance is-
sues.
4 Technical Safety Case
Def (Aust) 5679 is a system-level standard that
provides detailed requirements and guidance on the
structure of the Safety Case for a safety-critical sys-
tem. It focuses on assurance activities: these are sys-
tem development and analysis activities that provide
evidence that the system meets its safety require-
ments. The main mechanism for structuring the
Safety Case is to require a component-level design
(components may be implemented using software,
hardware or human operators).
 Def (Aust) 5679 was one of the first international
standards to mandate a technical safety case deliver-
ing a structured argument and supporting evidence.
The technical safety requirements of Def (Aust) 5679
are based on defining the necessary contents of the
safety case.
Figure 2 gives an overview of the Safety Man-
agement Process, showing the relationship between
the various stages of System Development and Safety
Case Development.
The key stages of Safety Case Development are
described in the following sections.
5 Preliminary Hazard Analysis
The aim of Preliminary Hazard Analysis (PHA) is to
define the system boundary and identify all possible
accidents, and their severities, that may be caused by
a combination of the states of the system, environ-
mental conditions and external events.
First of all, the system is described in terms of the
interfaces between the system and its environment.
The collection of System Functions defines the scope
of the system (and hence what can be changed or de-
signed by the Developer), which will generally include
operator procedures. In other words, the System
Functions taken together define the System Bound-
ary. It is crucial that the System Boundary is clearly
understood and accepted by the Safety Management
Agents. Factors that are beyond the scope of sys-
tem control are called external and are determined
through reference to the Operational Context.
The next step is to identify all accidents that could
conceivably arise from a combination of system and
external conditions, as well as their severities, defined
as show in Table 1.
SEVERITY DEFINITION
Catastrophic Multiple Loss of Life
Fatal Loss of Life
Severe Severe Injury
Minor Minor Injury
Table 1: Def (Aust) 5679 accident severities.
System Hazards are top-level states of the system
from which an accident could conceivably result, pos-
sibly involving a further chain of events external to
the system (called an accident sequence) as depicted
in Figure 3. In this diagram, events [E] are shown
as square boxes; accidents [A] are terminal events.
States are shown as circles - these may be hazard (H)
states or states external (X) to the system. Hazard
states should be defined in the most convenient way,
even if this means that a number of hazards must be
realised to enable a given accident or that a number
of accidents may be enabled by a given hazard.
For each accident sequence, the Operational Con-
text should be referenced to determine the strength
of external factors that would perform hazard control
or damage limitation functions. Such external factors
must be assessed as one of Low, Medium, or High. In
general, engineering judgement is used to make these
assessments. Such arguments must be made explicit
in the Safety Case, and need to be carefully checked
by the System Safety Evaluator.
In a limited number of cases (i.e. where the acci-
dent sequence includes random external events such
as environmental factors, the behaviour and condition
of hardware etc), it may be possible to ascribe a prob-
ability to the occurrence of these external factors, and
hence the conditional probability of an accident oc-
curring given that a System Hazard has been realised.
As a guide, in the case where these conditional acci-
dent probabilities can be calculated, it is reasonable
to assign strengths as follows:
• Low for probabilities greater than 10−2 ;
• Medium for probabilities between 10−2 and 10−4 ;
and
• High for probabilities less than 10−4 .
It is very important to note that the standard does
not allow probabilities to be assigned to the chance
of the System Hazard itself being realised (this is dis-
cussed further in Section 12.4).
Each System Hazard generates a System Safety
Requirement (SSR), which expresses the fact that the
hazard should not occur. To each SSR we assign one
of six Levels of Trust (LOTs), named T1 ,... , T6 .
The LOT is a function of the severity of the result-
ing accidents and the strength of external mitigating
factors contributing to an accident sequence, and is
calculated in accordance with Table 2. For each ac-
cident sequence involving the corresponding System
Hazard, a default LOT is assigned to an SSR based on
the severity of accident. If no external factors can be
assigned to the accident sequences involving the cor-
responding System Hazard, then the LOT remains at
the default value for that severity. On the other hand,
if for each relevant accident sequence a strength of ex-
ternal factors can be assigned, then the LOT can be
reduced from its default value.
The LOT is therefore a measure of how badly we
want the SSR to be satisfied; i.e. it is a measure of
the assurance required that the SSR is met by the
system.
6 System Hazard Analysis
System Hazard Analysis (SHA) derives Component
Safety Requirements (CSRs) for each component of
the system.
First of all, the Developer must produce a high-
level description of the system, called the Component-
Level System Design (CLSD). The CLSD consists of a
decomposition of the system in terms of Components
that combine to carry out the system functions. The
CLSD must define how each Component is intended
to be implemented, that is, whether a function of the
Component is carried out in software, hardware, or
by operator procedures.
System Hazard Analysis details all possible
Component-Level Hazards that could result in Sys-
tem Hazards. Fault tree analysis may be used to do
this. Each such hazard corresponds to a Component
Safety Requirement (CSR).
The CSRs serve to focus the System Safety As-
surance effort on those parts of the system where it
is most needed. Once Component Safety Require-
ments have been documented they can serve as top-
level safety specifications for system Components.
6.1 Hardware
This standard identifies two broad classes of hardware
Components: simple and complex. The behaviour of
simple hardware may be directly determined by ex-
haustive testing, while for complex hardware it is ne-
cessary to infer behaviour through design analysis.
 System Development Safety Case Development

System Definition
Preliminary
Hazard Analysis

Review &
Preliminary Evaluation
Design
System Hazard
Analysis

Review &
Evaluation

Safety Criticality
Assessment
Design Review &
Development Evaluation

Design Assurance

Implementation Review &

Evaluation

Implementation
Assurance
Integration and
Installation Review &
Evaluation

System Integration
Assurance

Review &
Evaluation

Final Review and

Post-Development
Evaluation

Figure 2: Def (AUST) 5679 Safety Management Process.

Examples of simple hardware include: mechanical
interlocks, door latches, canisters, bulkheads, etc. For
simple hardware Components, reliable, thoroughly
tested and robust equipment must be used, and ana-
lysed with respect to safety.
Complex hardware Components may be further
classified as custom and non-custom components.
Custom hardware Components are those that are
designed specifically for the system under devel-
opment or are otherwise limited in use, such as:
application-specific integrated circuits (ASICs), pro-
grammable logic controllers (PLCs) and program-
mable gate arrays (PGAs).
The design of custom hardware Components must
use a structured architecture approach in which ele-
ments with well-defined interfaces are combined, and
must be supported by tools.
Non-custom hardware Components are general
purpose devices, such as: microprocessors, disk-drives
and other computer peripherals and PDAs.
Non-custom hardware Components present a chal-
lenge for safety assurance. Adequate assurance can-
not be gained through testing alone and detailed
design information may not be available. Assurance
must be gained by the use of generic, widely proven
components.
6.2 Software
Software allows the reuse of widely proven hardware,
usually in the form of microprocessors, in the imple-
mentation of a very wide class of system functions.
It should be recognised that implementation of
system functions by software means that it is easy to
introduce unmanageable complexity and hinder pre-
dictability of Component behaviour. Software does
not have to obey any physical laws and its behaviour
is not continuous. Behaviour can change radically
with small changes in input values. Thus, although
the purpose of software is to customise the behaviour
of general purpose computing hardware, it is concerns
 System

E1 H1

E3 X2 A1

E2 H2

H3

E4 X3 A2

X1

Figure 3: Example accident sequences.

DEFAULT EXTERNAL
LEVEL MITIGATION
SEVERITY Low Medium High
Catastrophic T6 T6 T5 T4
Fatal T5 T5 T4 T3
Severe T4 T4 T3 T2
Minor T3 T3 T2 T1

Table 2: Def (Aust) 5679 Levels of Trust.

about the design of the software, rather than the re-
liability of the hardware, that tend to dominate.
Contracts for the production of software Compon-
ents must specify conformance to a software devel-
opment standard. Sound software engineering and
structured programming must be practised both in
software design and coding.
Software must be as simple as possible in structure
and in execution. The aim is to minimise sources of
unpredictability of software in operation, and maxim-
ise the opportunity for independent verification of its
correctness by the use of tools. Other coding practice
constraints may have to be applied as a consequence
of the hazard analysis and consideration of the hard-
ware environment in which the software operates.
The programming language chosen shall allow the
assurance of safety in accordance with the relevant
Safety Assurance Levels for critical requirements of
Components containing software, as discussed in Sec-
tion 9.
6.3 Operators
The way in which human operators interact with
other Components of the system is very important
to overall safety. Thus, operators must be considered
to be part of the system.
The design involved in operator Components con-
sists of defining standard and emergency procedures
that those operators must follow in interacting with
the system and the training they must undergo before
using it.
In addition to the core activities of design, atten-
tion should be paid to the human-computer interface
and the procedures and training required of personnel
that operate, install or maintain parts of the system,
with special focus on the safety-related functions of
the system.
Human factors analysis, especially the potential
for operator induced hazards, must be carried out as
part of safety analysis and design of operator proced-
ures and operator interfaces.
Operator tasks should be designed to maximise
performance through consideration of a number of
factors, including the physical and cognitive demands
of the task, the conditions under which the task is per-
formed, the nature of information presented to make
decisions and the mechanisms by which actions are
taken. The limitations and strengths of human per-
formance should be considered in allocating operator
tasks and designing their interaction with a computer
system.
7 Safety Criticality Assessment
The next stage of the Safety Case is called Safety Crit-
icality Assessment. This assigns a Safety Assurance
Level (SAL) to each CSR. The SAL of a CSR pre-
scribes the system development and safety analysis
techniques that need to be applied during develop-
ment to help ensure that a given system Component
will achieve the required level of assurance. As with
LOTs, there are six SALs S1 , ... , S6 .
The SAL that is assigned to a CSR may be reduced
by the application of protective measures (such as in-
terlocks) in the system design that reduce the chance
of the corresponding hazard resulting in an accident.
The use of protective measures means that there are
multiple, independent CSRs that prevent the occur-
rence of an accident, either by prevention of a single
System Hazard or prevention of multiple System Haz-
ards that are combined within the same accident se-
quence.
If no specific protective measures have been taken,
then the SAL of a given CSR is the same as the LOT
of the SSR from which the CSR was derived. The
default SAL of a CSR may be reduced if protective
measures are in place that prevent the occurrence of
a corresponding Component Hazard from resulting in
an accident.
Although protective measures are an important
method for containing the consequences of failures
within the system, there is a limit to their use. A
certain amount of trust may be placed in protect-
ive measures and the corresponding arguments in the
Safety Case as to how the assurance of safety will be
increased by these measures. However, the indiscrim-
inate use of arguments for safety that rely on protect-
ive measures can result in components being assigned
an assurance level far below their corresponding con-
tribution to possible hazards.
The amount of trust that is gained by the use of
such measures is not offset by the trust that is lost
by excessive reduction of SALs. This is because the
assurance gained by activities carried out at higher
SALs (such as design modelling, verification etc) out-
weighs the assurance gained by the use of protect-
ive measures and hazard mitigation arguments. Ac-
cordingly, the SAL of a CSR shall be no less than
two levels lower than the LOT Level of the SSR from
which it was derived.
Def (Aust) 5679 provides detailed guidance on the
various kinds of protective measures, and how SALs
can be reduced by their use. Such measures include:
functional redundancy, data redundancy, design re-
dundancy, safety kernels, etc.
8 Component Design Assurance
The aim of Component Design Assurance is to analyse
Component Design and the process of Component
Development, and provide sufficient assurance that,
if the Component is implemented as designed, the
Component Safety Requirements are met.
Assurance of system safety is established through
the application of specific development and analysis
activities for each System Component according to
the SALs of associated CSRs (see Table 3).
8.1 Component Models
In order to reason about the behaviour of a Compon-
ent and show that it meets its CSRs, a model of the
Component Design must be formulated. The level of
formality of the language used to formulate the model
of a Component is specified in Table 3 (the SAL used
is the highest for the CSRs associated with the com-
ponent).
8.2 Safety Requirements Specification
The Component Safety Requirements identified by
System Hazard Analysis shall be specified as asser-
tions about the Component’s interface with other
Components. The level of formality of the specific-
ation language used is prescribed in Table 3.
It is sufficient to express Component Safety Re-
quirements with a SAL of S1 as informal English lan-
guage statements. For SALs S3 or higher, Component
Safety Requirements are expressed in a formal spe-
cification language. Safety requirements with a SAL
of S2 require some form of semi-formal specification,
which involves the use of structured English language
statements, possibly augmented with diagrams and
mathematical notation.
8.3 Design Verification
Design Verification proofs must be carried out for
each system Component against its CSRs. The degree
of formality of the proofs is prescribed in Table 3.
For the SALs S1 or S2 , it is sufficient that such
proofs be expressed using informal clear and logical
English arguments. For SALs S5 or S6 , proofs need to
be expressed using a formal mathematical logic. Such
proofs are called formal. The SALs S3 and S4 require
some form of rigorous proof. Such proofs are mathem-
atically convincing but may omit some of the detailed
justification. For example, the proof could consist of a
high-level formal argument but with lower-level steps
justified informally.
9 Component Implementation Assurance
The aim of Component Implementation Assurance is
to provide detailed evidence that the implementation
of a given component will meet its CSRs. Assurance
of system safety is established through the applica-
tion of specific development and analysis activities
for each System Component, according to the SALs
of associated CSRs.
The requirements for software components are
summarised in Table 4 and include the following:
1. Defensive programming techniques, which in-
clude the use of safe subsets of high-level lan-
guages, in which only program constructs with
well-defined semantics are allowed.
2. Flow analysis to study aspects of code such as
control flow, information flow and data use ana-
lysis. It must be supported by the use of static
analysis tools.
3. Code specification, in which the CSRs are decom-
posed into corresponding Unit Safety Require-
ments (USRs) representing critical properties of
source code units. The specification of USRs may
be presented into informal, semi-formal or formal
language, as for the CSRs.
 S1 S2 S3 S4 S5 ,S6
System Component Model Informal Informal Semi-Formal Formal Formal
Specification of CSRs Informal Semi-formal Formal Formal Formal
Design Verification Informal Informal Rigorous Rigorous Formal

Table 3: Def (Aust) 5679 Safety Assurance Design Attributes.

S1 S2 S3 S4 S5 S6
Defensive High Level High Level Safe Safe Safe Safe
Programming Language Language Subset Subset Subset Subset
Flow Analysis Optional Optional Yes Yes Yes Yes
Code Informal Informal Semi- Semi- Formal Formal
Specification Formal Formal
Code Informal Informal Rigorous Rigorous Formal Formal
Verification Object
Code
Software Basic Medium Medium High High High
Safety Testing

Table 4: Def (Aust) 5679 Safety Assurance Level Attributes for Software.

4. Code verification to prove that each software unit
satisfies its USRs. The proofs may be presented
using informal, rigorous or formal arguments.
5. Software safety testing, broadly classified into
unit testing and integration testing. The pur-
pose of unit testing is to demonstrate that soft-
ware units satisfy their corresponding USRs. The
purpose of integration testing is to show that the
complete component satisfies its corresponding
CSRs.
In practice, the assurance evidence may combine
to form a sound argument. For example, justification
that the software safety testing need cover only a spe-
cific part of the software, may be provided by a flow
analysis showing that the remaining part does not
interact, and defensive programming demonstrating
that the entire software is exception free and cannot
indirectly modify critical variables.
10 System Integration Assurance
The aim of System Integration Assurance is to provide
sufficient assurance that the System Safety Require-
ments are met by the integrated and installed system.
System Safety Testing is performed on the in-
tegrated System to demonstrate that the collection
of Components satisfy System Safety Requirements.
System Safety Testing should exercise the system in
accordance with its expected operating demands. It
may be partially performed in a simulated environ-
ment prior to Installation, or performed on the in-
stalled system, when safe to do so.
Installation Safety Assurance provides evidence of
the safety of each System Installation, including the
manufacture and assembly of all components, use of
any customisation data, and the installation itself.
11 Evaluation
Def (Aust) 5679 requires that an independent System
Safety Evaluation be carried out to assess the tech-
nical validity of the Safety Case. Effective System
Safety Evaluation is absolutely essential for the assur-
ance of safety; it relies on diversity and independence
in the review of technical arguments.
The Evaluation is carried out in well-defined
stages, after each of the Safety Case activities de-
scribed above. The results of evaluation then feed
back into system and Safety Case development. It is
important to avoid the extreme situations where, on
the one hand, System Safety Evaluators become part
of the development process (and thus compromise in-
dependence) and, on the other hand, Evaluation is
done only at the very end of development (this could
be very costly if a safety issue is not addressed in the
Safety Case, and could have been picked up by the
Evaluator).
The Auditor must take the lead in resolving any
disputes that may arise between the Customer, De-
veloper and Evaluator.
12 Issues Addressed
The Def (Aust) 5679 revision process has identified
a number of potential issues with the standard that
have been debated at length. Many of these issues
represent fundamental challenges for computer-based
safety engineering. Others highlight certain key prin-
ciples of Def (Aust) 5679 that differentiate it from
some other safety standards. This section discusses
some of the issues and the current status of their res-
olution.
12.1 Non-Development Systems
The most pressing issue with the application of Def
(Aust) 5679 is the treatment of Non-Development
Systems, which are systems that have been already
developed overseas for other Defence customers and
potentially modified to suit local conditions and re-
quirements. Procurement of such systems is necessary
practice for the Australian Department of Defence.
Non-Development Systems may have been developed
in accordance with an international safety standard or
no safety standard at all, but rarely with Def (Aust)
5679 compliance in mind. Convincing evidence may
be difficult to elicit, and such systems present great
challenges for procurement.
The procurement of non-development systems
poses three major issues for Def (Aust) 5679. First,
the avoidance of probabilistic targets and ALARP
(see Section 12.3) means that a safety risk assessment
must be conducted in Def (Aust) 5679 terms in order
for the standard to be applied. Second, since the Def
(Aust) 5679 assurance requirements are deliberately
more rigorous than other standards, it is likely that
non-development systems will not meet it, even if they
have been accepted overseas. Third, Def (Aust) 5679
currently provides no means of incorporating field ser-
vice evidence into a safety argument. The revision
process has suggested a means by which this could
be achieved and has applied it to the limited scope of
non-custom hardware components. However, broader
application requires further consideration.
The Standardisation Working Group has acknow-
ledged that the procurement of Non-Development
Systems presents fundamental questions of the role of
an indigenous safety standard and its enforcement in
the Australian Defence environment. These questions
are equally applicable to the role of standards more
generally and the ability of a procurement authority
to enforce standards in the modern age of global sys-
tems development. In part, this is a likely motivation
for the trend towards goal-based standards seen else-
where.
12.2 Strength of Assurance
The strength of the assurance demanded by safety
standards ranges considerably: from the lowest as-
surance, such as MIL-STD 882C (DoD93 1993),
through intermediate assurance (such as IEC 61508
(IEC 2005) and RTCA DO-178B (RTCA 1992)), to
very high assurance, such as the original version of
the UK DefStan 00-55, which contained extremely
demanding requirements for Safety-Critical Software.
The strength of assurance that is adopted is to some
extent a question of engineering judgement about
what best practice should be. Def (Aust) 5679 adopts
a strength of assurance intended to be high enough
to give confidence in system safety, but not so high as
to place unreasonable demands on developers. The
use of high-assurance techniques can also provide in-
creased understanding of system requirements and
design; thus, it can lead to cost savings for later stages
of the project.
The revision process raised the issue of whether the
Def (Aust) 5679 assurance requirements represent a
standard that can be met at acceptable cost, particu-
larly given the propensity for procurement of overseas
and previously developed system. However, in view
of the above, and on the advice of the Def (Aust)
5679 editor, the Standardisation Working Group has
decided to make no major changes to the assurance
requirements.
12.3 Risk Tolerability
Def (Aust) 5679 does not use risk tolerability criteria.
In particular, it does not invoke the ALARP (As Low
as is Reasonably Pr icable) principle so familiar in
the UK context through the Health and Safety at
Work Act (HMSO74 1974). The ALARP principle
states that there is a region of risk between strictly ac-
ceptable and strictly unacceptable levels where risks
should be reduced where practicable. ALARP is in-
timately bound up with issues such as how to measure
risk, what weight to give risk reduction activities and
ultimately becomes bound up with cost estimation.
It is also problematic for software. Def (Aust) 5679
makes no use of it and, as a consequence, does not
allow the acceptance of specific risk levels by a higher
authority.
A related issue is that Def (Aust) 5679 is not in-
tended to be tailored. The processes described in it
are mandatory. Many standards do allow for tailor-
ing: developers or procurement agencies may decide
to lower assurance requirements; they may decide to
use their own risk tolerability criteria; they may de-
cide not to use an independent evaluator, and so on.
The ability to tailor standards is easily abused. The
authors do not believe that tailoring is appropriate
and it is not permitted in Def (Aust) 5679.
12.4 Probabilities
Def (Aust) 5679 makes only limited use of probab-
ilities. They are permitted only when assessing the
chance of external events as contributions to accident
sequences (and may then be used as a basis for re-
ducing LOTs). However, the use of probabilities for
system hazards in computer-based systems is prohib-
ited by the standard.
There are several reasons for avoiding the use of
probabilities. First, it is considered to be inappro-
priate to associate probabilities to design failures, in-
cluding conditions arising from software components.
Second, it is believed that probabilities can be used to
obscure unsound safety arguments arising from erro-
neous or optimistic data and independence claims. A
third reason is that the use of failure probabilities may
cause confusion between claims of system reliability
and evidence that the system is safe. A fourth reason
is that the standard does not require full analysis of
subcomponents such as processors and operating sys-
tems, making it difficult to achieve accurate probabil-
istic calculations taking account of such subsystems.
Finally, it is not clear that the state-of-the-art per-
mits the development of probabilistic arguments to
the level of formality required by other parts of the
standard.
One limitation arising from the avoidance of prob-
abilities is that the standard provides no framework
for assessing events such as the failure of physical ma-
terials and certain human behaviours, which are best
modelled in probabilistic terms.
It is accepted that the treatment of such random
events is an important aspect of safety engineering.
However, in light of the concerns raised above, it
is not yet clear how best to address these matters.
The main options are either to associate probabil-
istic targets with SSRs and flow the targets top-down
through all the system design stages, or else to syn-
thesise probabilistic estimates bottom-up in a separ-
ate stage following final implementation. The former
seems to place undue constraints on design choices
and also to risk confusion between assurance and re-
liability engineering activities. The latter seems to
offer too little guidance to the design process. The
view of the Def (Aust) 5679 editors is that the latter
option is preferable as it is of great importance that
failure rates not be equated with assurance levels.
12.5 System Integration and Deployment
The current revision of Def (Aust) 5679 provides
scope for the analysis and assurance of systems up
to the development of system components. However,
experience from early application of the standard re-
vealed the need for the technical safety case to ex-
plicitly address system integration and deployment.
Accordingly, the revision process has added require-
ments and guidance to ensure that the safety case
include test evidence from system integration and ac-
ceptance testing. For this purpose, the standard has
distinguished the development of an initial system
from the manufacture and deployment of multiple
system instances, including the potential for custom-
isation of each instance. While the generic system
safety case must demonstrate safety of the general
design, each instance need only demonstrate safety of
the manufacture, installation and customisation.
12.6 Operation and Support
Def (Aust) 5679 deliberately targets the procurement
of systems and does not cover the safety of systems
once accepted into service. For this strategy to be ef-
fective, the standard must clearly identify those oper-
ating and support procedures necessary to maintain
system safety in service. The system must be de-
signed to minimise the risk that these procedures will
be violated, and the standard must ensure that re-
sponsibility for meeting the safety procedures is trans-
ferred to the operational body when the procurement
safety case is accepted. While these issues were ad-
dressed to some extent in the original Def (Aust)
5679, the revision process added requirements and
guidance to make this more explicit. However, a more
holistic view of Defence procurement has been devel-
oping over recent years, and Def (Aust) 5679 will need
to pay greater attention to whole of lifecycle issues in
the future.
13 Key Revisions
In this section we discuss some of the key changes to
the standard as a result of the revision process.
13.1 Accident Sequences
The analysis of accident sequences, according to the
published version of the standard, was a matter of
some concern in practice.
A particular point of confusion was the question of
correspondence between accidents and hazards. For
example, is it legal to have more than one hazard lead-
ing to a given accident (via differing sequences) or to
have more than one hazard leading to the same acci-
dent? These issues have been clarified in the revised
version.
The published version of the standard did not
allow qualitative estimates of external mitigating
factors when determining the LOT required for a
given SSR. This was of concern in practice for two
reasons. Firstly, it denied the value of and failed to
encourage the use of external mitigating factors that
cannot be clearly quantified. Secondly, it could be
seen as encouraging the attribution of probabilities
that could not be justified, a practice in contraven-
tion of the general spirit of Def (Aust) 5679.
13.2 Assurance Framework
The development and analysis processes required for
each safety assurance level were reviewed in detail and
some small, but significant, improvement were made.
For example, the notions of semi-formal specification
and rigorous proof were a point of some confusion and
have been more carefully defined in the revision. The
role and practice of testing in software assurance has
been significantly revised, leading to a more stream-
lined process that is better in line with modern testing
practices. The contribution of physical reliability to
system safety has been more clearly explained.
It should be noted, however, that the overall
strength of assurance has not changed in the revision,
as discussed above.
13.3 Tailoring
There were several clauses in the published version
that gave the safety management agents the scope
to substantively modify the strength of assurance
delivered by application of Def (Aust) 5679. Such
clauses allowed scope for significant abuse of the spirit
of the standard and have been removed.
13.4 Changes in Terminology
A number of changes of terminology have been made
in the revised version of Def (Aust) 5679. Some of
the more significant ones are listed in Table 5.
14 The Next Step
The intention is to issue a public draft revision of
Def (Aust) 5679 late in 2005 for review and comment
by Defence and industry, as well as the international
safety community. In particular, comment will be
sought from members of The Technical Cooperation
Panel (TTCP) Focus Area JSA-TP-4-3 on Safety-
Critical Systems. This is a collaborative group in-
volving Defence Research and Development and pro-
curement representatives from the UK, USA, Aus-
tralia and Canada. Defence safety standards are reg-
ularly discussed in this forum.
These comments will then be collated and presen-
ted to the Standardisation Working Group for its fur-
ther consideration. It is expected that the revised
standard will be published by the end of 2005.
For further information on Def (Aust) 5679, please
contact Dr Tony Cant (DSTO).
15 Acknowledgements
The DSTO contribution to this work was carried out
under the Critical Systems Development Task JTW
04/061, which is sponsored by the Standardisation
Branch of the Defence Materiel Organisation. The
ISRA contribution was carried out under contract to
the DMO.
References
Army Engineering Agency (1998), Def (Aust) 5679:
The Procurement of Computer-Based Safety-
Critical Systems.
US Department of Defense (1993), Military Stand-
ard MIL-STD-882C: System Safety Program Re-
quirements.
Department of Defense (2000), Military Stand-
ard MIL-STD-882D: Department of Defense -
Standard Practice for System Safety.
US Department of Defense (2004), Draft Military
Standard MIL-STD-882E: Department of De-
fense Standard Practice for System Safety.
UK Ministry of Defence (2004 ), Safety Management
Requirements for Defence Systems: Part 1 (Re-
quirements), Part 2 (Guidance).
Her Majesty’s Stationery Office (1974), Health and
Safety at Work etc Act.
RTCA Inc (1974), Software Considerations in Air-
borne Systems and Equipment Certification.
 Original Term New Term Reason
Safety Critical Safety Critical The term “computer-based” was dropped to
Computer-Based Systems emphasize the system nature of the standard,
Systems and to remove the focus on software.
Safety Integrity Safety Assurance The notion of Safety Integrity Level is used
Level Level in many standards and has a rather differ-
ent meaning from that invoked by Def (Aust)
5679, which uses the term as a measure of
strength of assurance.
Safety Integrity Safety Criticality Again, the term “safety integrity” was felt to
Assessment Assessment be misleading
Post-Development Non-Development The term “post-development” could be taken
Systems Systems as meaning ”after the development of the cur-
rent system”, which was not what was inten-
ded.

Table 5: Terminology.

International ElectroTechnical Commis-

sion (1974), Functional Safety of Elec-
trical/Electronic/Programmable Safety-Related
Systems - IEC 61508.