TECHNICAL BROCHURE

401
WG
Functional Testing B5.32
of IEC 61850 based systems

Members:
Iony Patriota, Convener (BR), Aitzol García (ES),
Pascal Postec (FR), Marcelo Paulino (BR),
Alex Apostolov (US), Tetsuj Maeda (CH),
Dennis Holstein (US), Fred Steinhauser (AT),
Stephen Thompson (GB), Hyuk Soo Jang (KP),
Jian-cheng Tan (CA)

Corresponding Members:
Damien Tholomier (FR), Marcus Steel (AU),
Gareth Baber (GB), Benemar Alencar (BR),
Ubiratan Carmo (BR), Benton Vandiver (US),
Ricardo Cartaxo (PO), Allan Cascaes (BR),
Ren Yanming (CN), Sun Bo (CN), Byung-Tae, Jang (KR)

Over the last few years considerable work has been
carried out in the development of a new standard IEC
61850, “Communication Networks and Systems in
Substations”. Its introduction has raised questions on
how to test their functionality and performance, as
conformance and interoperability have already been
standardized. Contrary to previous “Substation
Automation Systems (SAS)” technologies, these sys-
tems use distributed hardware and software, with asyn-
chronous and real time behavior that challenge current
functional test practices.
To discuss these issues, Cigré WG B5.32 was commis-
sioned to produce a TB addressing the needs of utilities,
integrators and testers on functional testing of IEC 61850
based systems. IED (Intelligent Electronic Device) ven-
dors, tool suppliers, and system integrators can use it to
ensure their products and services provide the capabili-
ties needed and specified by users. It does not attempt to
comprehensively address all possible testing scenarios,
such as non-functional requirements like maintainabil-
ity, reliability, availability, (US)bility, security, etc. Only
functionality and performance are dealt with.
Functional tests for any system must be designed to
check its behavior, comparing with an accepted or for-
mal specification. As the IEC did not standardize func-
tions, a method is proposed based on UML and XML.
UML Use Cases are suggested as suitable artifacts to
describe functions, from a user point of view. Other
UML diagrams are suggested to specify real-time per-
formance requirements, and tools needed for func-
tional testing. FMEA and HAZOP, as defined by IEC,
are suggested to verify the fault coverage of test plans.
F u n c t i o n a l R e q u i re m e n t s
Functional requirements express a specification
from a user point of view, and as the standard against
which functional test results should be compared for
approval. Traditionally, SAS functions have been
defined by utilities in a textual form, supplemented
with tables and pictures, a method not yet formalized.
IEC 61850 does not define functions, only their inter-
face to the communication network. Functions are
performed by the interaction of several Logical Nodes
(LNs), so the specification may require their definition
and exchanged messages. SAS functions are designed
using SCL (Substation Configuration Language),
which may differ from a user functional specifi-

No. 247 - December 2009 E L E C T R A 51


cation, as users are not expected to specify SAS directly
in SCL. Being a design specification, SCL contains
details needed for functional testing, such as network
addresses of logical nodes, servers and messages, and
points for signal injection and monitoring. It should be
available at the time of testing as a design specification.
UML is a standard means by which complex con-
ceptual structure and functionality can be represented
and communicated using easy to grasp diagrams and
tables. It is recommended that a subset of UML artifacts
be used to describe SAS functions, including Use Case,
Communication, Sequence, Activity, Deployment and
State diagrams.
Test Requirements
Once functions are specified, and before functional
tests are initiated, at least the following documentation
must be available: functional specification (FICS file)
supplemented optionally by other UML artifacts, sys-
tem specification (SCD file), IEDs configuration (CID
files), and conformance declarations (PICS, MICS and
PIXIT). In addition, all IED must have passed factory
acceptance, conformance and interoperability tests, as
required by IEC 61850-10.
Functional tests, being directed to users, shall com-
ply with several additional requirements: they must be
simple, repeatable, documented, automatable, human
readable, customer oriented, tool independent, sup-
plier independent, extensible, systematic, and stan-
dardized. In addition, the SUT must comply with
customer specifications and possibly with a mixing of
devices from different suppliers. To comply with all
these requisites, a standard method should be adopted
to guarantee that enough tests are performed to ascer-
tain desired test coverage.
Test Coverage
Test coverage is a measure of completeness of a test
plan, expressed by the set of features tested, like possi-
ble failures discovered or functions tested. FMEA and
HAZOP are two methods standardized by IEC to ana-
lyze possible failures a system may present, and to avail
the fault coverage of any proposed test plan. In these
methods, Functional Failures represent abnormal
TECHNICAL BROCHURE
401
states or results from system functions, varying from a
WG
complete loss of the function, to a partial degradation
B5.32
of its expected performance level. Components are
defined as any part of a system where failure modes can
originate. Physically, SAS systems are composed mainly
by IED servers and network components and function-
ally by sets of logical nodes, both connected by network
components like switches, hubs, etc. Failure Modes
represent abnormal events that may occur on these
components and their connections. Failure modes of
physical components are usually related to environ-
mental or physical causes, like short-circuits, broken or
weary pieces, etc. Software components like logical
nodes may present novel failure modes, such as: wrong
parameters; wrong code or software bugs; wrong or
absence of input/output signal/messages; or wrong
timing (delay) for input/processing signals/messages.
For testing purposes, logical nodes are treated as black
boxes, so that their failure modes are limited to loss or
degradation of an expected external behavior.
Functional Test Tools
Testing tools must have the capability to exercise
specific functional requirements of IEC 61850 based
systems. The TB follows the UML Testing Profile to
model the required test components. Functional test-
ing tools should also support the principles of Black
Box testing, where the internal logic of the components
is not known. They should also support tests per-
formed in a top-down or bottom-up approach, to
address the needs of integration and factory acceptance
testing. So, in addition to input and output
Functional Test Setup
No. 247 - December 2009 E L E C T R A 53

signals, they should be able to monitor and generate
messages between the different components of a dis-
tributed function. Following these concepts, the next
picture shows a UML package diagram of a general
architecture for a test system, connected to an SAS.
The SAS is seen as a network of logical nodes,
linked to a substation process, and accessed by local
and remote operators. The package representing the
Tester should be able to intercept the interfaces among
users and process equipment to the SAS, and to have
access to specific logical nodes through the station and
process networks, by importing the SCL files from the
SAS package. The following is a brief description of
each package forming the Tester.
Functional Test Specification
The proposed test architecture uses XML as the test
script language, associated to a user oriented test tem-
plate (next Table). Any test is divided in six phases:
Connection, Setup, Start, Stop, Disconnection, and
Verdict. Each phase is described as a script, composed
of commands sent to the test tools and from this to the
SAS or SUT (System Under Test). Each command
encodes a message sent to an object derived from a
class of the test architecture.
In the first phase, Test Connections have to be
declared so that the Test Tool can select the object
Table – Functional Test Case Template
TECHNICAL BROCHURE
401
modules and connections to perform the test. In the
WG
Test Setup phase, for each component needing initial-
B5.32
ization, a message is sent to the appropriate object,
instantiating a method of the process simulator class.
The Test Start phase applies analogue and digital sig-
nals to the SUT. The Test Stop phase removes all exter-
nal signals to the SUT. Before Test Disconnection of all
components from SUT, the tool must collect all neces-
sary events, reports, etc. Finally, the Test Verdict phase
checks all specified conditions from the records. A ver-
dict is a pass or fail conclusion about some results col-
lected from the test.
Conclusions
Testing the functionality of IEC 61850 based sys-
tems requires techniques used mainly by distributed
software developers and maintenance engineers. In this
report a method has been described using UML with
XML, and IEC standards for FMEA and HAZOP. In the
TB, a complete example shows the proposed method
applied to a simple station with an IEC 61850 SAS.
Appendices contain the suggested XML Schema and a
Functional Test Specification Template to support its
documentation. It is expected that this report will be
followed by additional refinements in the proposed
method, by its implementation by tool suppliers, and
to the standardization of methods of specifying SAS
functions and testing by IEC. ■
No. 247 - December 2009 E L E C T R A 55