Distributed Network Protocol - DNP3 - Security Interoperability Testing 2012

Distributed Network Protocol (DNP3) Security
Interoperability Testing 2012

1026561
11687472
11687472
Distributed Network Protocol (DNP3) Security
Interoperability Testing 2012
1026561
Technical Update, December, 2012
EPRI Project Manager

G. Rasche
ELECTRIC POWER RESEARCH INSTITUTE

3420 Hillview Avenue, Palo Alto, California 94304-1338 ▪ PO Box 10412, Palo Alto, California 94303-0813 ▪ USA
11687472800.313.3774 ▪ 650.855.2121 ▪ askepri@epri.com ▪ www.epri.com
DISCLAIMER OF WARRANTIES AND LIMITATION OF LIABILITIES
THIS DOCUMENT WAS PREPARED BY THE ORGANIZATION(S) NAMED BELOW AS AN ACCOUNT OF
WORK SPONSORED OR COSPONSORED BY THE ELECTRIC POWER RESEARCH INSTITUTE, INC. (EPRI).
NEITHER EPRI, ANY MEMBER OF EPRI, ANY COSPONSOR, THE ORGANIZATION(S) BELOW, NOR ANY
PERSON ACTING ON BEHALF OF ANY OF THEM:
(A) MAKES ANY WARRANTY OR REPRESENTATION WHATSOEVER, EXPRESS OR IMPLIED, (I) WITH
RESPECT TO THE USE OF ANY INFORMATION, APPARATUS, METHOD, PROCESS, OR SIMILAR ITEM
DISCLOSED IN THIS DOCUMENT, INCLUDING MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE, OR (II) THAT SUCH USE DOES NOT INFRINGE ON OR INTERFERE WITH PRIVATELY OWNED
RIGHTS, INCLUDING ANY PARTY'S INTELLECTUAL PROPERTY, OR (III) THAT THIS DOCUMENT IS
SUITABLE TO ANY PARTICULAR USER'S CIRCUMSTANCE; OR
(B) ASSUMES RESPONSIBILITY FOR ANY DAMAGES OR OTHER LIABILITY WHATSOEVER (INCLUDING
ANY CONSEQUENTIAL DAMAGES, EVEN IF EPRI OR ANY EPRI REPRESENTATIVE HAS BEEN ADVISED
OF THE POSSIBILITY OF SUCH DAMAGES) RESULTING FROM YOUR SELECTION OR USE OF THIS
DOCUMENT OR ANY INFORMATION, APPARATUS, METHOD, PROCESS, OR SIMILAR ITEM DISCLOSED IN
THIS DOCUMENT.
REFERENCE HEREIN TO ANY SPECIFIC COMMERCIAL PRODUCT, PROCESS, OR SERVICE BY ITS
TRADE NAME, TRADEMARK, MANUFACTURER, OR OTHERWISE, DOES NOT NECESSARILY
CONSTITUTE OR IMPLY ITS ENDORSEMENT, RECOMMENDATION, OR FAVORING BY EPRI.
THE FOLLOWING ORGANIZATION, UNDER CONTRACT TO EPRI, PREPARED THIS REPORT:
EnerNex LLC
This is an EPRI Technical Update report. A Technical Update report is intended as an informal report of
continuing research, a meeting, or a topical study. It is not a final EPRI technical report.
NOTE
For further information about EPRI, call the EPRI Customer Assistance Center at 800.313.3774 or
e-mail askepri@epri.com.
Electric Power Research Institute, EPRI, and TOGETHER…SHAPING THE FUTURE OF ELECTRICITY
are registered service marks of the Electric Power Research Institute, Inc.
Copyright © 2012 Electric Power Research Institute, Inc. All rights reserved.
11687472
ACKNOWLEDGMENTS
The following organization, under contract to the Electric Power Research Institute (EPRI),
prepared this report:
EnerNex LLC
620 Mabry Hood Road Suite 300
Knoxville, TN 37932
Principal Investigator
G. Gilchrist
This document describes research sponsored by EPRI.
The DNP3 Secure Authentication specification and the DNP3 Secure Authentication Test
Procedures are being developed with the assistance of:
• The Distributed Network Protocol Users Group Technical Committee,
http://www.dnp.org.
• The International Electrotechnical Committee Technical Committee Working Group 15:
Utility Data Communications Security, and Working Group 3: Telecontrol
• The Institute of Electrical and Electronic Engineers Power Engineering Society
Substations Committee Working Group C14
• The Smart Grid Interoperability Panel Cyber-Security Working Group, sponsored by the
U.S. National Institute of Standards and Technology
This publication is a corporate document that should be cited in the literature in the following
manner:
Distributed Network Protocol (DNP3) Security Interoperability Testing 2012. EPRI, Palo Alto,
CA: 2012. 1026561.
11687472 iii
11687472
PRODUCT DESCRIPTION
This project encourages the adoption of the Distributed Network Protocol (DNP3) Secure
Authentication specification through three approaches:
• Supporting development and review of the specification by industry experts in
supervisory control and data acquisition (SCADA) and cryptography
• Aligning the specification with international standards performing similar functions, so
that there is a common industry-wide approach to SCADA security
• Developing procedures for testing the specification to reduce potential barriers to new
adopters and improve interoperability between new implementations
Results and Findings
The DNP3 Technical Committee reviewed the draft DNP3 Secure Authentication Test
Procedures, which consists of 275 tests of approximately a page each, and revised the document
to address the review comments.
The Institute of Electrical and Electronics Engineers (IEEE) balloted the DNP3 Secure
Authentication Specification Version 5 developed in 2011 and released it as IEEE Standard
1815-2012.
The International Electrotechnical Commission (IEC) balloted and released Edition 2 of IEC
62351-5, which had been aligned with DNP3 Secure Authentication version 5 in 2011.
The IEC working group revised IEC 60870-5-7, based on the revised IEC 62351-5, and released
it to the IEC for ballot.
Challenges and Objectives
This project involves coordinating updates to four different documents among several different
standards-development working groups. The primary challenge continues to be ensuring that as
the documents are reviewed by each of these groups, all necessary changes are propagated into
the appropriate specifications. A secondary challenge is trying to coordinate the processes of the
different groups, which all have different timings.
Another challenge is encouraging the implementation of the revised DNP3 Secure
Authentication Version 5 specification by vendors. Most interested vendors released Version 2
products in 2011, so there will be a delay before widespread adoption of the new standard.
The DNP Technical Committee is largely a volunteer organization with only limited resources
for developing a large, complex specification such as test procedures, especially a specification
requiring specialized knowledge like cryptography. Furthermore, the coordination between the
various organizations that must approve the specification is very labor-intensive. Without EPRI
support, this type of development might take years rather than months.
Applications, Values, and Use
The DNP3 protocol is the most widely used utility communications protocol in North America.
It has recently been released as the IEEE 1815 standard and is recognized in the National
Institute of Standards and Technology (NIST) Smart Grid Interoperability Framework as one of
11687472 v
the key standards to be used in smart grid deployments. Ensuring that DNP3 communications are
secure is therefore an important goal for the power industry.
With a comprehensive set of test procedures, vendors and utilities alike can ensure that a
particular implementation is correct and interoperable. Ensuring that these test procedures also
comply with international standards helps to ensure the specification’s adoption across the
industry.
Approach
The DNP3 Secure Authentication specification has been selected as the core document driving
the rest of the affected specification since the feedback from the DNP3 Technical Committee is
the fastest. This document is used to drive the other specifications, which have longer
development cycles.
The DNP3 Technical Committee developed the DNP3 Security Test Procedures framework by
creating a one-to-one correspondence between each requirement stated in the DNP3 Secure
Authentication specification and an appropriate section of the Test Procedures document.
Keywords
Security
Authentication
Cryptography
Key
Credential
Certificate
SCADA
Telecontrol
11687472 vi
CONTENTS
1 HISTORY AND BACKGROUND ............................................................................................ 1-1
Value to the Industry ............................................................................................................ 1-1
Goals and Objectives ........................................................................................................... 1-1
Relationships Between Deliverables .................................................................................... 1-2
Previous Accomplishments .................................................................................................. 1-3
2 CURRENT STATUS ............................................................................................................... 2-1
Status of DNP3 Secure Authentication Test Procedures ..................................................... 2-1
Status of the DNP3 Secure Authentication Specification and IEEE Std. 1815 .................... 2-1
Status of IEC 62351-5 Edition 2 ........................................................................................... 2-2
Status of IEC 60870-5-7....................................................................................................... 2-2
3 NEXT STEPS .......................................................................................................................... 3-1
DNP Secure Authentication Test Procedures ...................................................................... 3-1
Edition 2 of IEC 62351-5 ...................................................................................................... 3-1
Edition 1 of IEC 60870-5-7 ................................................................................................... 3-1
A TEST PROCEDURE REVIEW NOTES ................................................................................. A-1
11687472 vii
11687472
LIST OF FIGURES
Figure 1-1 Relationships between the Specifications and their Current Status .........................1-2
Figure 1-2 History of the DNP3 Secure Authentication Specifications (top) and IEC
62351-5 (bottom) .......................................................................................................................1-3
Figure 1-3 Interaction Between Working Groups and Documents 2008-2012...........................1-4
11687472 ix
11687472
1
HISTORY AND BACKGROUND
Value to the Industry
The Distributed Network Protocol (DNP3) is the most widely used utility communications
protocol in North America. Over 75% of power utilities have reported using it as part of their
Supervisory Control and Data Acquisition (SCADA), substation automation or feeder
automation projects. Originally released in 1994, DNP3 has been continuously updated over the
years by an active community of suppliers, users and integrators within the DNP Users Group.
Unfortunately, DNP3 as originally defined was not secure. Like most utility protocols developed
at the time, it was designed from the start to be highly resistant to noise and other communica-
tions failures found on power system feeders, but had no protection against deliberate cyber-
attacks. In today’s security context this omission was critical because DNP3 is frequently used
by utilities over vulnerable communications links, such as unencrypted radio systems. It is
sometimes carried over third-party wide area networks whose owners may not implement
adequate security. And even communications links that were traditionally thought to be secure,
such as leased lines, have been shown to sometimes be vulnerable.
Therefore, ensuring that DNP3 communications are secure is an important goal for the power
industry. Furthermore, because of DNP3’s popularity, it is a project that can provide a huge
benefit through relatively modest investment in a single technology.
Goals and Objectives

This project encourages the adoption of security for DNP3 through three approaches:
• Support development and review of the specification by industry experts in SCADA and
cryptography.
• Develop procedures for testing the specification to reduce potential barriers to new
adopters and improve interoperability between new implementations.
• Align the specification with international standards performing similar functions, so that
there is a common industry-wide approach to SCADA security.
To meet these goals, this project has four primary milestones in 2012:
• Balloting completed of the DNP3 Secure Authentication specification as a part of IEEE
Std. 1815-2012.
• Review and revision of the existing draft of the DNP3 Secure Authentication Test
Procedures.
• Release of Edition 2 of IEC 62351 Part 5: Data and Communications Security – Part 5:
Security for IEC 60870-5 and derivatives
11687472 1-1
• Release for ballot of Edition 1 of IEC 60870-5-7: Telecontrol Equipment and Systems –
Part 5-7: Security Extensions to IEC 60870-5-101 and IEC 60870-5-104 protocols
(Applying IEC 62351-5)
It was hoped that this project would also include conformance and interoperability testing of
DNP3 Secure Authentication implementations in 2012. However, although software library
implementations of DNP3-SAv5 are available, no products were available for testing in 2012 so
this part of the project could not proceed.
Relationships Between Deliverables

Figure 1-1 illustrates the relationships between these documents and summarizes their current
status. Their current status is described in more detail in section 2.
Edition 2
passed
ballot
IEC 62351-5
Reviewed
by Working Vendors IEEE Std.
Group, beginning to 1815-2012
revised and
Std.
implement Released
submitted Version 5 1815
for CDV
IEC 60870-5-7 DNP3 Secure

Authentication
Reviewed by
Tech Committee
DNP3 Secure Authentication and Revised
Test Procedures
Figure 1-1
Relationships Between the Specifications and Their Current Status
The DNP3 Secure Authentication specification was developed in parallel with the IEC 62351-5
specification, as shown in Figure 1-2. The intent was that IEC 62351-5 would describe a generic
authentication mechanism that could be applied to both DNP3 and IEC 60870-5, the two most
commonly used utility communications protocols at the time the development was initiated. By
eliminating duplication of the effort to secure these protocols, it was intended that the work could
be accelerated and that the resources of the industry could be efficiently applied to the problem.
11687472 1-2
Figure 1-1 shows how the DNP3 Secure Authentication specification and IEC 60870-5-7 are
both based on IEC 62351-5, which in turn is based on fundamental cryptographic standards
defined by the U.S. National Institute for Standards and Technology (NIST) and the International
Standards Organization (ISO). Since work has been able to advance more quickly on the DNP3
specification, the IEC specifications are being revised to match the DNP3 changes rather than the
other way around.
Previous Accomplishments
The top half of Figure 1-2 shows events and releases of the DNP3 specification, while the
bottom half shows events in the development of IEC 62351-5.
Key distribution NIST
and ITI CSWG
cryptographic review of
EPRI Project review ver 2 / 5
Begins
Symmetric key
Testing Draft
distribution
Frame- Tests
proposed
DNP Tech work
Review “Bulletin of Tests
Version Ver Ver Ver Ver Ver Ver
Starts Reviewed
Intent” 1.0 1.5 2.0 2.2 3.0 4 5
2004 2005 2006 2007 2008 2009 2010 2011 2012
Requests for:
• Return of public Ed. 2
Edition 1 Edition 2
62351-5 key distribution passed
Released without delayed by with key
ballot
• Authenticate key distribution IEC / NSA distribution
people, not mechanism patent issues sent to
devices ballot
Figure 1-2
History of the DNP3 Secure Authentication Specifications (top) and IEC 62351-5 (bottom)
EPRI’s sponsorship of the work began in 2008, when version 2.0 of the DNP3 Secure
Authentication specification was released by the DNP Users Group and Edition 1 of the IEC
62351-5 Technical Specification (TS) was ready for release by the IEC. While release of the
IEC specification was delayed due to patent declaration issues (which have now been resolved),
EPRI helped the DNP Users Group to develop the remote key distribution mechanism which had
been missing in previous versions, and to have the specification reviewed by the Information
Trust Institute at the University of Illinois.
11687472 1-3
In 2011, the following work took place as a part of the EPRI project, as shown in Figure 1-3.
• DNP3 Secure Authentication Version 4 (SAv4) was developed to address comments
from implementers on Version 3.
• The Cyber Security Working Group (CSWG) of the Smart Grid Interoperability Panel
(SGIP) reviewed SAv4 and found a vulnerability in it.
• DNP3 Secure Authentication Version 5 was developed to address the vulnerability and
align with IEC 62351-8. It was included in the version of IEEE Std 1815-2012 submitted
for ballot.
• IEC 62351-5 Edition 2 was revised to match DNP3-SAv5 and submitted for national
ballot.
• IEC 60870-5-7 was revised to match DNP3-SAv5 and submitted for working group
review.
IEC TC57 WG3 DNP Technical Committee IEC TC57 WG15 IEEE Sub Comm C12 WG SGIP Cyber-Security WG
IEC 60870-5-7 DNP3 Secure IEC 62351 IEEE Std. 1815 Reviews
Authentication
Version 2
2008-08
ITI Review,
IEC 62351-5
Key Distribution
Edition 1 Edition 1
Working Draft 2009-08
Version 3 IEEE Std.
Review 2010-03 1815-2010
2011
Implementer
Comments Catalog of
Stds Review
Version 4
2011-03
IEC 62351-8
Vulnerability
Edition 1
Review
2011-02
Common
Certificate Fix Encryption
Vulnerability
Version 5
2011-10 Submitted for Submitted for
Submitted to WG ballot review and ballot
2012
IEC 62351-5 IEEE Std.
Edition 1 Edition 2 1815-2012
Committee 2012-?
Draft for Vote Review
Review
Figure 1-3
Interaction Between Working Groups and Documents 2008-2012
11687472 1-4
2
CURRENT STATUS
This section describes the work completed this year and the current status of this project as of
December 2012.
Status of DNP3 Secure Authentication Test Procedures

The following steps were completed on this deliverable in 2012:
1. The draft of the test procedures developed in 2011 was completed as of 2012-02-06.
2. The draft was reviewed by members of the DNP Technical Committee.
3. The draft was revised according to the review comments.
4. An informal review of a test set prototype was performed.
The DNP Technical Committee reviewed the draft during six different teleconferences,
completing the review of approximately 80 of the 275 tests by the end of August 2012. To
facilitate completing the review by the end of the year, Andrew West, the chair of the DNP
Technical Committee, and Grant Gilchrist, the primary author of the specification and the test
procedures, reviewed the remaining tests. They presented some of the review items for resolution
by the rest of the committee at the committee’s annual face-to-face meeting in Knoxville in
October and at further teleconferences. All 275 test sections were reviewed by December 1, 2012
and the review notes as of November 22 are attached as an Annex to this technical update. By
the end of December 2012, the test procedures will be revised but remain to be approved by the
DNP Technical Committee.
Status of the DNP3 Secure Authentication Specification and IEEE Std. 1815
The following steps were completed on this deliverable in 2012.
1. IEEE Std. 1815-2010 including DNP3 Secure Authentication version 2, was approved in
January 2012 by the Governing Board of the Smart Grid Interoperability Panel (SGIP)
for inclusion in the SGIP Catalog of Standards.
2. The draft IEEE Std. 1815-2012, which includes Secure Authentication Version 5,
completed ballot on March 11 with approximately 260 comments.
3. The comments on the ballot were resolved by members of IEEE Substations Committee
Working Group C12 and the IEEE technical writers.
4. The draft standard was submitted to the IEEE Standards Association review committee
(RevCom) on April 26, 2012.
5. The IEEE-SA review committee approved IEEE Std. 1815-2012 on June 8, 2012.
6. The final version was published Oct 10, 2012.
11687472 2-1
The EPRI contribution to this work in 2012 was to resolve the ballot comments having to do
with DNP3 Secure Authentication and facilitating some of the C12 Working Group meetings.
Status of IEC 62351-5 Edition 2

1. The IEC national committees completed voting on April 13, 2012 on the Draft Technical
Specification (DTS) that was developed in 2011, incorporating the work from DNP3-
SAv5. The ballot passed with 21 out of 22 countries voting in favor.
2. EnerNex wrote replies to the 11 comments from the voting process.
3. EnerNex facilitated the review of these comments and the final draft by IEC Technical
Committee 57 Working Group 15.
4. The document was approved by the IEC TC57 secretariat on September 1, 2012
Final formatting and publishing of the Technical Specification (TS) began on November 19,
2012.
Status of IEC 60870-5-7

1. The IEC TC57 Working Group 3 met on February 23 and March 6 to review the revised
Committee Draft for Vote (CDV) that was developed in 2011, incorporating the work
from DNP3-SAv5 and IEC 62351-5 Edition 2.
2. EnerNex resolved all the Working Group 3 comments from the CDV review. All the
comments were resolved shortly after the review meetings, except for one, having to do
with fitting long security messages into the 255-byte IEC 60870-5 data link frame. It took
several months to resolve the final issue and develop a proposal for chaining messages,
based on the similar solution used by DNP3.
3. A draft was submitted to the WG3 and WG15 members on October 3, 2012. It was
reviewed by WG15 on October 11 and by WG3 on November 20.
4. The draft CDV of IEC 60870-5-7 has been submitted to the IEC for vote by national
committees. Assuming it passes, there will be one final balloting stage known as Final
Draft International Standard (FDIS) to be completed in 2013.
11687472 2-2
3
NEXT STEPS
This section describes the next steps in the development of this project that should be completed in
2013.
DNP Secure Authentication Test Procedures

Steps should include:
• Final approval of the revised test procedures by the DNP Technical Committee.
• Deployment of equipment in the EPRI Substation Laboratory for implementing the tests.
• Organization of volunteer implementations for the first conformance and interoperability
tests.
• Pilot conformance tests of the volunteer implementations.
• Development of agreements for interoperability tests. Note that interoperability testing
differs from conformance testing in that different implementations communicate with
each other rather than with a test set.
• Interoperability testing between implementations.
Edition 2 of IEC 62351-5

Steps should include:
• Publishing the final document.
Edition 1 of IEC 60870-5-7

Steps under the current project will include:
• Resolving comments to the CDV submitted by the National Committees.
• Discussing the resolutions with IEC TC57 Working Group 3.
• Making changes to the document based on the comment resolutions.
• Releasing the revised document as an International Standard (IS). There may be an
additional round of review called Final Draft International Standard (FDIS).
11687472 3-1
11687472
A
TEST PROCEDURE REVIEW NOTES
This section provides the notes from the review of the DNP3 Secure Authentication Test
Procedures as of Nov 22, 2012
11687472 A-1
DNP3 Document Review
Reviewed Document’s Title: Date This Review Last Edited:
DNP3Spec-SecureAuthenticationTestProcedures 2012-11-22
Reviewed Document’s Version: Edited by:
2012-02-06 Grant Gilchrist
Cmnt Reference T/
Name Reviewer’s Comments Reviewer’s Proposed Changes Editor’s Comments
Num Identifier 1 E2
1. GG 2.1 E Section references are incorrect Fix Accepted
2. AW 1.1 E “Shall” implies binding – is this correct Delete “shall” Accepted
Don’t need any references that aren’t directly Change to just reference the IEEE 1815
3. AW 1.2 E Accepted
used. standard
4. AW 2.1 E “roughly” is colloquial Change to “approximately” Accepted
5. AW 2.1.3 E The correct term is DNP3-XML Change all occurrences Accepted
6. AS 1.3 E Missing definition of PIXIT Add Accepted
Need to clearly identify which version of the

7. PA 1 T Add a new section Accepted
standard is being tested (SAv5)
Is there a test for the Device Attribute object that
8. AW 1 T Check Accepted
shows the version?
Need text to explain that we are not talking about
9. RK 2.2.5 T testing the same software on two devices, but two Add text Accepted
different profiles on the same device
1
Reference Identifier’s include section, paragraph, table or figure numbers to identify the specific location within the original document.
2
T/E indicates whether the change is technical (T) or editorial (E).
11687472 A-2
Cmnt Reference T/
Num Identifier 1 E2
Add as an Annex. Must include which
10. AAW Missing T Description of what a test log should look like Accepted
profiles the tests were performed on.
Add as an Annex. Would like to have
Description of what the certificate should look
11. RK Missing T check boxes that show which optional Accepted
like
tests were performed.
12. AW 2.3.1 item 3 T “DNP3 and IP”, not “DNP or IP” As noted Accepted
Add the word “selectively” to show that it does

13. AW 2.3.1 item 7 T As noted Accepted
not always prevent transmission
14. GG 2.3.1 item 8 T Is it really necessary to force CRC failures Check to see if that’s true Accepted
15. GG 2.3.1 item 10 T Do we actually test against non-secure devices Check to see if that test exists Accepted
2.3.1 items 14- Specifically mention both “manually

16. AW T Is it useful to insert an error automatically Accepted.
16 entered” and “error” values.
2.3.1 items 21-
17. AW T Note that these are not required if not testing TLS As noted Accepted
22
18. AW 2.3.2 item 5 T Incorrect value, or manually-entered? Specifically need “incorrect” value. Accepted.
2.3.2 item 6
19. GG T Don’t need this requirement Remove Accepted.
and 8
20. GG 1.2 E Add NIST SP-800-22 to the references As noted Accepted.
Note that statistics are tested even if the logging is

21. AW 2.4 E As noted Accepted.
not
22. AW 3.1, 4.1 E Reword to use the default configurations As noted Accepted.
3.1 step 8, Change to AES-256/SHA-256 to match the

23. AAW T As noted Accepted.
12c, 14, 30 default config
24. GG 3.1 step 12c T Add certification data length As noted. Accepted.
11687472 A-3
Cmnt Reference T/
Num Identifier 1 E2
Verify that the confirmation object from the
3.1 step 16
25. GG T master is actually in the same request as the key As noted. Accepted.
and 17
change itself.
26. GG 3.2 T Restart test numbering As noted. Accepted.
Provide more detail that the test includes changing

27. AAW 3.1 E As noted Accepted.
the Update Key and the Session Keys
Add a comment to the IEEE 1815 spec that the
28. GG 3.2.1 T Not a change for this document.
length of the challenge data is not specified.
29. GG 3.2.2 step 30 T Step is incomplete Complete sentence Accepted.
3.3.1 step 6/7, Note that the CSQ is only 0 if nothing happened
30. AW step 10/11 E between the reset of the session keys and the As noted Accepted.
3.4.1 step 7/8 control operation.
3.3.1 step 6/7,
step 10/11 Use the default configuration of AES-256/SHA-
31. GG T As noted Accepted.
256
3.4.1 step 7/8
32. AW 3.4.5 step 21 T The CSQ should perhaps be 3? Investigate
Note in the preamble that this only works if there

33. AW 3.4 E As noted Accepted.
were no event objects previously.
Don’t need to test all the non-critical functions, Change “each” to “at least one” for the
34. GG 4.1 T Accepted.
just a sample. other non-critical functions to be tested.
35. GG 4.1 T Critical messages received statistic is not tested Remove from list of statistics tested. Accepted.
For instance, this is not an aggressive

Make it clear what is meant by “critical non-
36. GG 4.1.2 E mode message, it is a regular DNP Accepted.
security ASDU”
message that happens to be critical
Note that <0> Unknown and <1> Default may be
37. AAW 5.1.2.2 T Add tests for these cases Accepted.
invalid users under some conditions.
11687472 A-4
Cmnt Reference T/
Num Identifier 1 E2
Not clear exactly which MAC algorithms are List very specifically which ones are
38. AW 4.1.2.1 E Accepted.
mandatory permitted, noting there are 4 values
Refer to the table in section 12.1.2 of the
The list of mandatory critical functions is not
test procedures regarding which functions
39. AW 4.1.2.2 E found in the test procedures, but in the Accepted.
are critical, and that the PIXIT lists them
specification itself.
all as supported.
The test asks the test set to be configured after Swap steps 4 and 5 so the configuration
40. AW 4.1.3 T Accepted.
sending the first message occurs first.
41. CA 4.1.3 T Is the function code 0x21 correct? Investigate Accepted.
Is it necessary to test roll-over of the CSQ when Investigate. This may need to be a change
42. AW 4.1.4 T Accepted
the value increases by more than 1? to the specification.
Investigate whether roll-over requires a
Is it possible to test roll-over if the Challenger is
43. AW 4.1.4 T very long test or cannot be tested in a Accepted.
verifying that each new CSQ increments by 1?
conformance or integration test.
Change this test to verify that the next Challenge
44. AAW 4.1.4.1 T CSQ increments by 1 only if the Aggressive Mode As noted Accepted
CSQ was correct and the MAC was correct.
Add verification that authentication can only be
45. PA 4.1.5 T As noted. Accepted
disabled locally.
Note that the statistics to be checked are the ones
46. AW All tests E As noted. Accepted.
on the DUT
There were two messages received: the critical Note “received messages” increases by 2,
47. GG 4.2.3.3 step 21 T Accepted.
ASDU and the Reply not 1.
Correct step 16 to “critical” and change
There was a cut-and-paste error: step 16 should step 15 to note that the second critical
48. CB 4.2.4.2 step 16 T Accepted.
verify that the “critical” ASDU was not acted on. message must be distinguishable from the
first, e.g. a different point number.
Make “3. Discards critical ASDU
49. GG 4.2.6 step 3 E This is not a step, it’s a heading. Accepted.
if Reply times out” into 4.2.6.1
11687472 A-5
Cmnt Reference T/
Num Identifier 1 E2
50. GG 4.2.7.1, 4.2.7.2 E Note that logging is optional “If the DUT supports logging, check…” Accepted.
Missing a test for a CSQ that does not increment

51. AW 4.3.2.1 T Add tests. Accepted.
correctly, and a replay attack.
May not be clear that all the statistics should be
Add a section somewhere in the Test
52. AW 4.3.2.2 E checked, not just the ones that are listed, and that Accepted.
Approach chapter to discuss this.
they should increase by the amount shown.
Add text to the default configuration
If the session keys change periodically in the
53. AW 4.3.2 T section, and note at the beginning of the Accepted.
middle of the test, the test will be messed up.
test
Note that it is possible to send a non-critical
function using an Aggressive Mode Request.
What should we test?
Add tests for both (A) and (B). Write a
A. Non-critical function in VALID aggressive
54. AAW 4.4 T technical bulletin to clarify how these Accepted.
mode request; process the non-critical function
cases should be handled.
B. Non-critical function in INVALID aggressive
mode request; process the non-critical function
anyway.
This case is different because the
Aggressive Mode Request is VALID.
Why is this case handled differently than some That means the sender is authentic and
55. AW 4.4.3 T others that are similar when waiting for Reply? really meant to abort the original critical
ASDU. However, aggressive mode is
disabled so the DUT sends an Error
message.
The statistics do not show that two messages are
Change the discarded messages statistic
56. GG 4.4.3 T discarded: the original ASDU and the Aggressive Accepted.
count to (2).
Mode Request
The behavior in this case is inconsistent with the
philosophy of 4.4.3, which is that the pending Write a technical bulletin to make the
57. AW 4.4.5 T Accepted.
ASDU should be discarded if the Aggressive behavior consistent.
Mode Request is valid.
11687472 A-6
Cmnt Reference T/
Num Identifier 1 E2
Add a test to increment
Unexpected Messages statistic if
the CSQ is not incremented
There are no responder tests to verify that the from Challenge to Challenge.
CSQ increments, or that the CSQ is different
from previous Challenges, or that the Challenge Make a change to the SA
data is different from previous Challenges. Add tests to check the CSQ and Challenge specification to require a
58. AW 5.1.2 T
data and ignore them. Responder to check that the
Furthermore, there are no rules in the Responder CSQ increments and update the
section of the specification saying that these Unexpected Messages statistic.
checks are necessary.
Note that this protects only
against brute force or DOS
attacks.
No need to tie the MAC algorithm to a particular
59. AW 5.1.1.2 E medium, and we don’t want them to think they Remove the notations in brackets. Accepted.
should only test it if it’s on a particular medium.
Finish sentence to say that the test should
60. GG 5.1.1.3 E Truncated sentence be skipped if the DUT does not support Accepted.
multiple users.
Accepted.
Missing a test for ignoring an unexpected Make a change to the SA

61. GG 5.1.1.4 T Add a test. specification requiring a
Challenge.
Responder to ignore unexpected
Challenges and increment the
appropriate statistic.
The extra Challenge is a received message and Change the expected number of messages
62. GG 5.1.1.4 T Accepted.
should be counted. received to (3).
Should the Critical Messages Sent counter
63. GG 5.1.1.4 T increment for the unexpected Challenge? It is not Allow either (1) or (2). Accepted.
really another critical message sent, but by the
11687472 A-7
Cmnt Reference T/
Num Identifier 1 E2
definition of the statistic, it should be acceptable.
64. 5 Note that Default <1> is used if the test set is

AW 5.1.2.1 step 3 E As noted. Accepted.
. simulating a master.
5.1.2.1 step 7, Keep it as a marker to note that the test set
65. GG T This step is invalid Accepted.
12, 16, 21, 25 will not respond.
Say “for an outstation, any non-zero
Instead of “will not recognize” be more specific
66. GG 5.1.2.2 step 9 E number is invalid. For a master it is a Accepted.
about the User Number
non-configured user.”
As noted. Also add text in the
67. GG Annex A T Fill in Annex A. introduction explaining what Annex A Accepted.
contains.
5.1.4 As noted. Also provide a list of the

Test not only the mandatory MAC algorithms, but
68. AW T mandatory MAC algorithms in the Accepted.
5.2.3 all the algorithms supported by the device.
introduction.
Cause the test set to send the proper answer to the
69. GG 5.2.2 T As noted. Accepted.
critical message from the DUT.
Do not note the statistics again for each repeat of
70. GG 5.3 step 8 T As noted. Accepted.
the test.
In step 13, require the tester to record the
CSQ and in steps 16 and 20 and 25 and 28
71. GGG 5.4.2 step 16 T The CSQ should be 4, not 3 as shown. write the text in relation to the recorded Accepted.
value. Explain this process at the top of
5.4.
Don’t just test for decreasing CSQs but also for
72. AW 5.4.4 T As noted. Accepted.
anything but the expected value.
Enable Aggressive Mode because it was disabled
73. CB 5.4 step 1 T As noted. Accepted.
in the previous test.
11687472 A-8
Cmnt Reference T/
Num Identifier 1 E2
The text is mixed up between Max Reply

74. GG 6.1 step 3 T Divide up the text correctly. Accepted.
Timeouts and the length of the Reply Timeout.
Add text to deal with the case where there is not

75. AW 6.1 E an offline configuration tool, e.g. the defaults are As noted. Accepted.
on the running system, not on the tool.
Add test for the case that the master restarts but
76. CB 6.1.2 T the outstation has not restarted, and therefore Add new test. Accepted.
returns an OK key status.
Add preconfiguration requirement that the master
Add pre-configuration steps, and add a
77. GG 6.1.3 T should poll at 10-second intervals. This test Accepted.
missing step for the first poll.
seems to assume that configureation.
Repeat should be steps 11 through 19, not 11
78. GG 6.1.4 T As noted Accepted.
through 17.
79. GG 6.1.6 T Correct invalid reference As noted Accepted.
Make it clearer exactly when the session key

80. AW 6.1.6 E Add more explanatory text. Accepted.
change shouldn’t happen.
Modify text under “Purposes”. See
Is the statement “whether waiting for a Reply or
changes to 6.2.2. to verify that the key
81. JB 6.2 T not” correct? The Challenger state machine Accepted.
isn’t changed until no longer waiting for
appears to disagree.
Reply.
As noted. In step 21, also verify that the
Should wait until the Key Change Timeout minus key is changed at the Session Key Change
82. GG 6.2.2 step 17 T Accepted.
HALF of the Reply Timeout. Timeout plus HALF of the Reply
Timeout.
Between steps 47 and 48, should wait for a Reply
83. GG 6.2.6 step 47 T As noted. Accepted.
Timeout to occur before the Session Key Change
Missing one more Reply Timeout in before the Add steps and increment the number of
84. GG 6.3.3 T Accepted.
Session Key Change timeout will occur. Reply Timeouts in the statistics (to 4).
11687472 A-9
Cmnt Reference T/
Num Identifier 1 E2
Missing a test for a TCP timeout (or data link
85. JB 6.3 T Add steps. Accepted.
timeout)
Sentence at the top of the section is incorrect;
86. CF 6.4 T Reword. Accepted.
copied from previous section.
Change the title to “Ignores Key Status if
87. AWGG 6.4.3 E Using “confirmation” in the title is confusing. waiting for confirmation of Session Key
Change”. Also change spec.
G120v5 is “Authentication Session Key Status”,
88. AWGG 6.4.3 E Change name throughout.
not “Authentication Key Status”
There is no test for the master receiving a Key
Status Request that is valid except the KSQ is
89. AWGG Missing T incorrect. If the MAC is valid but the KSQ is not Add a test.
incremented, the master should just accept the
new KSQ.
It is not clear that the (0x20) and (0x83) are
90. AWGG Throughout E Change to (FC=0x20) and (FC=0x83).
function codes.
6.4.4 and
Whenever the test set sends a Key Status object,
91. AWGG throughout, T Change the step
specify the User Number (usually the Default)
e.g. step 36
6.4.3 step 22
and 6.4.5 step This step should specify the MAC algorithm and
92. AWGG E Change the step
47 and challenge data as in 6.4.3
elsewhere
Clarify that the message must have a MAC, but
93. AWGG 6.4.6 step 61 E the master will know that any MAC value is Change the step
invalid.
Change “increased” to “changed” because it is
94. AWGG 6.4.6 step 65 E Change the step.
more difficult to mis-read.
Need four occurrences to exceed Max
95. AWGG 6.4.7 step 72 T Change text to “three more times”
Authentication Failures.
11687472 A-10
Cmnt Reference T/
Num Identifier 1 E2
This step should check the Discarded Messages Discarded Messages should increment 8
96. AWGG 6.4.7 step 82 T
statistic also. times
Change text to “Repeat steps 68 through
97. AWGG 6.4.7 step 77 T Wrong number of repeats
76 two more times”
Once the keys have been successfully reset due to Add steps to test one more rekey due to
98. AWGG 6.4.7 step 81 T a key change timeout, it should be possible to authentication failure (steps 68 through
rekey due to authentication failure again. 76).
Critical messages rxed = 12
Auth failures = 12
Statistics counts are off by one because of
99. AWGG 6.4.7 step 82 T Error messages = 12
“exceeded”
Session Key Changes = 3
Rekeys due to auth failure = 4
100. AWGG 6.4.7 step 79 E The DUT may transmit non-security messages Reword the step
Not necessary to disconnect – there is a restart in

101. AWGG 6.5 step 1 T Remove step 1 and step 6.
step 5.
6.5.1 step 16,
6.5.2 step 18 Note in the step, “Before the Reply
6.5.2 step 24 Timeout occurs…”. Add a warning at the
These steps must be completed before a Reply
102. AWGG T beginning of 6.5 that several tests must be
6.5.3 step 32 Timeout
performed quickly before a Reply Timeout
6.5.3 step 33 occurs.
6.5.4 step 34
Reply Timeouts 6 or 4
Unexpected Messages 4 or 2
103. AWGG 6.5.3 step 33 T Statistics are incorrect
Authentication Failures 4 or 2
Discarded Messages 4 or 2
Delete “If a Reply Timeout has expired
since the last step”. Change to “Wait for
104. AWGG 6.5.3 step 33 T Not clear how long the tester has to wait
the DUT to send….. This may require
waiting for a Key Change Interval”
11687472 A-11
Cmnt Reference T/
Num Identifier 1 E2
Note this issue at the beginning of the test
Some masters may have a timeout that would kill and say that it may be necessary to disable
105. AWGG 6.5.4 E
the control if it was queued for too long. such a timeout or set it too a long period,
etc.
Remove the “disconnect the devices”,
106. AWGG 6.5.5 step 44 E Change the step and delete step 47.
unnecessary.
May not be a good idea to have the timers be Change the Session Key Change Interval
107. AWGG 6.5.5 step 45 T
exactly the same. to 35 sec instead of 30.
108. AWGG 6.5.5 step 52 E Invalid reference error Change reference to say step 51
109. AWGG 6.5.5 step 52 T Add a step to wait until the timeout expires Change text
Keep the repeat of 53 through 57, then

replace step 59 with the steps we want to
This step should do 53 through 55, not 55 through
110. AWGG 6.5.5 step 58 T see; another key status request, another
57
key status (NOT_INIT), and a key change,
all for User <8>.
Remove this step – statistics are not checked in
111. AWGG 6.5.4 step 35 T Remove step
remainder of 6.5.
112. AWGG 6.6 E “Compiling” is confusing Replace “compiling” with “collecting”
6.6 steps 7, 8
and elsewhere
113. AWGG in document, E Add the word “valid” for the replies Change text
e.g. 6.7 steps
3, 4
“If higher data rates can be achieved and the Key
Change Interval can be reduced, the overall Change text. Also add “NOTE” in front
114. AWGG 6.6 step 9 E
duration of this test can therefore be reduced, of “At 128 bits every 2 seconds,…”
which is preferred.”
Requirement 3 should read “two sequential
115. AWGG 2.3.3 T Change text
messages”, not just “two messages”
11687472 A-12
Cmnt Reference T/
Num Identifier 1 E2
6.5.5,
6.5.6,
Note this test requires optional support for
116. AWGG 6.6 step 12, T Add note
multiple users
6.7
“Configure the DUT to require authentication

117. AWGG 6.7 step 1 T Change note
from data responses if the DUT supports it.”
Note that the first time through, the Session Keys
118. AWGG 6.7 step 3, 5 E should be set for the Default User, “and the other Change text
configured users (if supported)”.
119. AWGG 6.7.2 step 9 T Verify that the RESTART IIN has been cleared Change text
Change “skip this test” to “skip to step 23”

120. AWGG 6.7.5 step 20 T Change text
because the confirmation must still be sent.
121. AWGG Throughout E It’s not clear what “skip this test” means Change to “skip to test number…”
122. AWGG 6.7.7 E Reword so it’s possible to have just a single user Change text
“Place system in the default configuration

123. AWGG 6.8 step 1 T Change step to use default configuration
as described in 2.6”
Not clear that the Challenge goes in the same Change “followed by” to “included in the
124. AWGG 6.8.1 step 7 T
message same message by”
Test later on assumes Aggressive Mode is
125. AWGG 6.8 step 1 T Disable Aggressive Mode in this step
disabled
126. AWGG 6.8.4 step 18 E Step is fragmented Remove “If the DUT…”
6.8.4 step 22
127. AWGG 6.8.5 step 28 T Aggressive Mode Request is not g120v1 Change to g120v3
6.8.7 step 40
11687472 A-13
Cmnt Reference T/
Num Identifier 1 E2
Establishes keys but does not perform the first
128. AWGG 6.8.4 step 19 T Add steps to perform the first challenge
challenge
6.8.4 step 24 Tests are too confusing and perhaps incorrect if
Add tests specifically for DUTs that do
129. AWGG 6.8.5 step 28 T the DUT does not require event data to be
not require event data to be authenticated.
6.8.6 step 32 authenticated
After discussion with Tech Committee,

decided to change the state machine so the
Master will discard the unsolicited
response if it is waiting for an
This test does not appear to be valid. The
Authentication Reply. This is cell E19 of
130. AWGG 6.8.7 T outstation should not be sending a UR if there is
the Challenger state machine. Also note
an unconfirmed response.
that this contradicts Rule 6 of 4.6.6 of
IEEE 1815-2012, but Rule 8 says the
outstation should not have issued the UR.
Change the test procedures to match.
After discussion with the technical
committee, decided to change 7.5.3.4 (c )
as follows: if the master knows that it is
about to send a critical message and the
Is this a valid test? Is section 7.5.3.4
keys are invalid, it shall change the keys
131. AWGG 6.9.1 T Authentication Errors (c ) of the specification
first instead of sending the message.
what we should be doing?
There are at least three ways it could
know: (a) assume based on the list of
mandatory functions (b) pre-configured (c)
learning from what is challenged.
No test for either outstation or master if we
receive a critical message and the Session Keys
132. AWGG Missing T Add tests to 6.1 and 8, or modify 6.9.1
have not yet been set. Should send an Error
message.
Change “two more times” to “three more
133. AWGG 6.9.1 step 7 T Wrong number of timeouts to fail session keys
times”
134. AWGG 6.9.1 step 10 T Wrong number of timeouts to fail authentication Change “two more times” to “three more
11687472 A-14
Cmnt Reference T/
Num Identifier 1 E2
rekeys times”
After discussion with the technical

Is section 7.5.3.4 Authentication Errors (d) a valid
135. AWGG 6.9.2 T committee, decided to delete 7.5.3.4 item
specification?
(d) from the SA spec.
Change to “Cause the DUT to send one of
The master might not support all the function
136. AWGG 6.9.3step 44 E the supported function codes from the
codes
following list.
6.8.1 through
137. AWGG E These tests are hard to follow without diagrams. Add a diagram for each test.
6.8.7
8.1.1 step 4, The MAC Algorithm must be <0> if there are no
138. AWGG T Change the MAC Algorithm value to <0>
step 10 previous Session Keys.
Change to “The MAC field is not included
139. AWGG 8.1.1 E The phrase “There is no MAC Value” is confusing
in the object”
140. AWGG 7 Section still needs to be reviewed.
Where did the required lengths of the Challenge

141. AWGG 8.1.1 T Investigate.
Data come from?
Delete the MAC algorithm checking (e.g.
step 5, step 7) from 8.1.1 and just check
MAC Algorithm and MAC value cannot change
142. AWGG 8.1.1 T the first initial Key Status Request without
until the Session Keys have been initialized.
a MAC, and the multiple users. The MAC
algorithms will be checked in 8.2.
No specification of how long the challenge data Add a step to specify the length of the
143. AWGG 4.1.3 T
should be. challenge data as 4 octets.
No need for Challenge Data to be 32 or 20 octets
144. AWGG Throughout 8 T Change to 4 octets
for a Session Key Status object.
This step should be done earlier because 8.1.2 Move step 21 and 22 to start of 8.1.2 (step
145. AWGG 8.1.3 step 21 T
sends 12 Key Status Requests. 12). Check reference in step 26.
11687472 A-15
Cmnt Reference T/
Num Identifier 1 E2
Add “If supported” to the test for the
This only tests for an Error message on an
146. AWGG 8.1.3 step 25 T alternate connection. Also verify the Error
alternate connection.
was sent on the same link, IF supported.
147. AWGG 8.1.5 step 27 T This is not Session Key Data, it is Challenge Data. Change text
Change the Expected Session Key Change

Interval to 3 minutes (180 sec) and the
8.1.5 step 28, Error message will occur if too many Key Status
148. AWGG T Session Key Change Interval to 2 minutes
step 31 Requests are sent.
(120 sec) and keep the Maximum Session
Key Status Count at 255.
Note in both steps that the Challenge Data
8.1.5 step 30, It’s not clear where the Challenge Data is being
149. AWGG E is being collected from the Key Status
31 collected.
Object.
Note that it will take over 8 and a half
With only 4 octets of Challenge Data it will take a hours and that if the devices can support a
150. AWGG 8.1.5 step 32 T
lot longer. higher message change rate, scale all the
times accordingly.
After discussion with the Technical
Committee, decided that g120v5 (Key
Status) object should have the MAC
calculated not only across the previous
The MAC should not be the same MAC each Session Key Change object, but across the
151. AWGG 8.1.6 step 39 T
time! Key Status object itself. Note also that the
calculation is across the latest Key Change
*for this user*, not just the latest Key
Change. This helps reduce denial of
service attacks.
152. AWGG 8.2 E Typo “orrectly” Fix typo.
Aggressive mode must be turned off for this to

153. AWGG 8.2.3 T Change 8.2.1
work
154. AWGG 8.2.3 T This test is incorrect. The DUT should throw Replace steps 23-25 with:
away the critical message and stop waiting for a
11687472 A-16
Cmnt Reference T/
Num Identifier 1 E2
Reply. - send the DUT a Key Status Request
before the Reply Timeout would have
occurred.
- verify the DUT responds with a Key
Status immediately
- verify the following statistics
Unexpected Messages (only 1, not 2)
Session Key Changes (1)
Critical Messages Received (1)
Messages Discarded (1)
Failed Authentications (1)
After discussion with the Technical
The specification has a serious vulnerability Committee, decided that for a Rx Invalid
associated with this test case. The spec says the Key Change event in Security Idle, the
outstation should invalidate the keys if the Key outstation shall discard the Key Change
155. AWGG 8.2.4 T
Change Message is not authentic. Instead, it message and increment the Discarded
should just discard the Key Change Message and Messages and Authentication Failed
continue. statistics. This is Row 25, column C of
the Challenger State Table (Table 8).
The specification currently says that Unexpected
Messages should be updated, but this is incorrect. When the specification is updated, remove
156. AWGG 8.2.5 T
The Key Change message was expected, but it the Unexpected Messages (3) verification.
was not authentic.
8.3.1 step 8 Delete steps 8 and 13, just say “do not
157. AWGG T “disconnect” is incorrect and unnecessary.
and 13 respond”
158. AWGG 8.3.2 step 23 T Max Authentication Failures isn’t exceeded.
times”.
159. AWGG 8.3.3 step 25 T Max Authentication Rekeys isn’t exceeded
times”.
8.3.3 step 25
160. AWGG T Need to change the keys Change the repeat to go back to step 16.
and 26
11687472 A-17
Cmnt Reference T/
Num Identifier 1 E2
Totals will be:
Session Key Changes (5)
Reply Timeouts (4)
161. AWGG 8.3.4 step 32 T Statistics are incorrect
Authentication Failure (15)
Rekeys Due to Authentication Failure(4)
…but check them on each repeat
162. AWGG 8.3.4 step 29 E Note that this information will be used for step 33 Change text
In step 33, only wait for half a Reply

Timeout before the Expected Key Change
This test will not work because the outstation will Timeout.
163. AWGG 8.3.5 T ignore the Key Status Request as long as it is In step 36, wait for the rest of the Reply
waiting for a Reply. Timeout.
Then just issue a status request.

This is an invalid test. It is impossible to
increment the number of Key Status Requests
164. AWGG 8.3.7 T Remove this section.
received while waiting for a Reply Timeout
because they will be discarded.
165. AWGG 8.4.1 E This section is redundant with 4.3.1.2 Leave as is.
The Successful Authentications statistic should Verify that the statistic incremented.
166. AWGG 8.4.3 step 26 T
increment.
Begin the section by recording the
167. AWGG 8.4.3 step 26 T Statistics were not recorded
statistics
Change (2) To (1) for Critical Messages
168. AWGG 8.4.3 step 26 T Only one critical message was received.
Received.
Change steps to use “N” and
Not sure if this test is valid. Why would an “X” for sequence numbers.
169. AWGG 8.4.4 T outstation generate a UR if it was waiting for an Discuss with DNP Technical Committee
Authentication Reply.
1. Create a new event in the
11687472 A-18
Cmnt Reference T/
Num Identifier 1 E2
Challenger state machine
called “Ready to send UR
requiring Aggressive Mode
Confirm”
2. In “Security Idle” state,
send the UR and move to
“Wait for Reply”
3. In “Wait for Reply”, queue
the UR and stay in “Wait
for Reply”.
4. Split the “Rx Critical
ASDU” event in “Wait for
Reply” state.
5. If waiting for an Aggressive
Mode App Confirm, cancel
the UR and send a
challenge, otherwise
discard the ASDU.
Change the challenge sent by the master

8.4.5 step 47, and subsequent AMs from the outstation
170. AWGG T CSQ shouldn’t necessarily be 1 in both directions
51 so that they are designated “Y”, “Y+1”
etc. instead of 1,2,3…
171. AWGG 8.4.5 step 50 T No check for CSQ Verify that CSQ=1
172. AWGG 8.4.5 step 51 T Must generate more events in order to cause a UR Add step.
Need to verify that the new unsolicited is not a Check the application sequence number in
173. AWGG 8.4.5 step 51 T
retry. step 42 and verify it changes in step 51.
174. AWGG 8.4.5 step 56 T There may be other Internal Indications Set Specify only Class 1, 2, 3 indications.
175. AWGG 8.4.5 step 57 T Verify this step has CSQ=Y+2 As noted.
176. AWGG 8.4.5 step 57 E There might be a Challenge object in the Add a note that there might be one, and it
11687472 A-19
Cmnt Reference T/
Num Identifier 1 E2
Response. would have the CSQ of 3 if there was.
Critical messages sent should be 3 (challenge

Send picture to Steve McCoy and/or
received and two AMs sent). Critical Messages
177. AWGG 8.4.5 step 58 T Charlie and ask how many Critical
Received should be 4 (two challenges sent, two
Messages Received would be counted.
AMs received)
Note that CSQ may be 4 or 5 depending on
178. AWGG 8.4.6 step 62 T whether the previous response in step 56 As noted.
requested an aggressive mode confirm.
8.4.6 step 64, To be true retries, the application sequence Change the wording so the app sequence
179. AWGG T
66 number should be the same. number is the same.
180. AWGG 8.4.6 step 70 T whether step 56 requested an aggressive mode As noted.
confirm.
If it’s truly an application retry, should the Critical
181. AWGG 8.4.6 T Ask implementers what they did.
Messages Sent be incremented?
182. AWGG 8.4.6 step 70 T whether step 56 requested an aggressive mode As noted.
confirm.
183. AWGG 8.4.7 T Need to check statistics so step 81 will work. Add step.
Typo: Configure the DUT as Configure the

184. AWGG 8.5.1 step 1 E Fix.
DUT…
Require that the statistics be reported as event
185. AWGG 8.5.1 step 1 T (Class 1,2 or 3) data, so that they will also appear Change step.
in a Class 0 response.
The statistics should each be incremented by one, As noted. Also acceptable for Messages
186. AWGG 8.5.5 step 18 T
i.e. 11. Sent to be one less, or 10.
Remove “if supported by the DUT” since both
187. AWGG 8.5.5 step 21 T As noted.
object variations are required.
11687472 A-20
Cmnt Reference T/
Num Identifier 1 E2
Typo: “does not support Update Key Change Change to “does not support a particular
188. AWGG 7.1 E
Method” Update Key Change Method”
189. AWGG 7.1.1 step 4 T Missing the Certification Data Length Change to 20.
Does not verify the actual Public Key. Public Key Replace with “Verify there is no Public
190. AWGG 7.1.1 step 4 T
Length appears twice. Key included”
For asymmetric methods, the Certification Data Change this step to include the possible
191. AWGG 7.1.1 step 5 T will not be a MAC, but a signature, and the length methods and their Certification Data
will be different. Lengths.
Should not refer to “certificate message” because Change to “sent by the authority to the
192. AWGG 7.1.1 step 8 T
the comms between authority and master DUT”
Should also test all the fields that have NOT
193. AWGG 7.1.1 step 8 E Add verification to this step.
changed compared to step 4.
“Update Key Length to the length required
194. AWGG 7.1.1 step 10 E Unclear grammar by the asymmetric key change
algorithm…”
195. AWGG 7.1.1 step 14 E Unclear grammar “A valid IEC 62351-8 certificate”
7.1.1, 7.1.2,
7.1.3 and The mandatory Key Change Method is <4> AES-
196. AWGG T Change from <3> as the default to <4>
many other 256 / SHA-256-HMAC, Key Length 256
places
7.1.1, 7.1.2, Add a step to send it the expected
Need to send the DUT the appropriate response to
197. AWGG 7.1.3 step 8, T Response and verify that nothing “bad”
the User Status object.
14, 24 happens.
The text about “hundreds of milliseconds should
198. AWGG 7.1.3 step 26 E be next to the “Reply Timeout”, not the “Max Move text.
Reply Timeouts”.
The other test section is 7.2.6. Delete the
Need another test section for the Reply Timeout
199. AWGG 7.1.3 E steps in this section that test the Reply
steps.
Timeout.
11687472 A-21
Cmnt Reference T/
Num Identifier 1 E2
Typo: “increment” is not part of the statistic
200. AWGG 7.1.4 step 42 E Delete.
name.
Add steps. Include table regarding the
201. AWGG 7.2.1 step 6 T Does not verify Challenge Data Length minimum required length for each
Method.
At 256 bits per message and 1 second per
202. AWGG 7.2.2 step 19 E Note how long it’s going to take. retry, should take about 3900 seconds or
just over an hour.
Replace with changing the DUT back to
203. AWGG 7.2.3 E Steps 22 through 24 are not necessary.
the default configuration.
Add a step that states, “If the master does
7.2.1 step 5 not automatically proceed with sending an
Missing a test for masters that do not want to
204. AWGG T Update Key Change Request, cause it to
7.2.4 step 32 change the Update Key immediately.
do so as specified in the PIXIT”. Also add
a section to the PIXIT.
Change to “Update Key Change
205. AWGG 7.2.5 step 37 E Incorrect message name
Confirmation”
Title is confusing, all the subtitles also. “User
206. AWGG 7.2 E Change to “Waits for Null Response” etc.
Change Response” sounds like a particular object.
Missing a test for an unexpected unsolicited
207. AWGG 7.3 T Add test.
response in the middle of the key change.
Change text to be more generic, e.g.
7.3.1 step 3 “The certificate should” is incorrect; the authority
“Cause the authority to send the following
208. AWGG and E and the master do not necessarily communicate
user status change information to the
throughout 7 using certificates.
master…”
7.3.1 step 6,
step 7 No information about what should be in the Add “common” user, User Name Length,
209. AWGG E Update Key Change Request or Update Key and Key Change Method = <4>, User
7.3.2 step 19, Change Reply Number (USR), KSQ etc.
20
210. AWGG 7.3.1 step 8 T “Default value of <128>” is wrong and confusing Add or refer to the table showing the
11687472 A-22
Cmnt Reference T/
Num Identifier 1 E2
correct key lengths for each Key Change
Method
Check that the length actually matches the length
of the field
“Encrypted Update Key Data is the user
ID “common” received from the
Authority in step 3, the new random
Missing information from the expected Encrypted
Update Key, and the Challenge Data
212. AWGG 7.3.1 step 8 T Update Key Data, and the Update Key was not
provided by the outstation in step 7
provided in step 3 as stated.
Update Key Change Reply (g120v12)
object, and any padding data required by
the selected algorithm”
Add text to say the g120v13 and g120v15
objects are in the same message.
Add text to check that the test set can
calculate the hash in the g120v15 object
based on:
Missing text saying that the g120v15 object must - the outstation’s text name
213. AWGG 7.3.1 step 8,9 T (preconfigured)
be in the same message
- master Challenge Data (step 6)
- outstation Challenge Data (step 7)
- the KSQ (step 7)
- the User Number (USR) (step 7)
- any padding data
Default configuration is missing the outstation
214. AWGG 2.6 T Add the name, e.g. “outstation”
name
As noted, especially that the two pieces of
Need to specify the contents of the g120v15
215. AWGG 7.3.1 step 10 T random data are reversed and it is the
object.
“common” user name, not the outstation.
7.3.2 step 13
Provide a step number to skip to, or a
216. AWGG and E It is not clear what “skip this test” means.
heading number.
throughout
11687472 A-23
Cmnt Reference T/
Num Identifier 1 E2
Refer to table for the correct Update Key
217. AWGG 7.3.2 step 14 E Could be clearer Length, and swap the order of the two
bullets
7.3.2 step 21 Missing text saying that the g120v14 object must Add text to say that the g120v13 and
218. AWGG T
and 22 be in the same message g120v14 objects are in the same message
Signature of the following using the
User’s private key (decoded with the
User’s Public key)
- outstation name (preconfigured)
219. AWGG 7.3.2 step 22 T Need info on what is in the signature
- outstation Challenge Data (step 20)
- the KSQ (step 20)
- the USR (step 20)
- the Encrypted Update Key Data from
g120v13 (same message as g120v14)
Hash of the following using the new
Update Key:
- user name “common”
Need to specify the contents of the g120v15 - outstation Challenge Data (step 20)
220. AWGG 7.3.2 step 23 T
object.
- the KSQ (step 20)
- the USR (step 20)
Note that the MAC value in g120v15 can
7.3.3 steps 32, Add “valid” to the description of the messages
221. AWGG E be random since a valid one can’t be
34 received.
created.
7.4, 7.4.1,
222. AWGG T Name object retransmitted is incorrect Change to just “Update Key Change”
7.4.2, 7.4.6
7.4.1 step 9 Merge the two steps so the objects are sent
223. AWGG T These two steps should be a single message.
and 10, and 12 in the same message.
224. AWGG 7.4.1 steps 5, E Add “valid” to the description of the messages As noted.
11687472 A-24
Cmnt Reference T/
Num Identifier 1 E2
7, 8, 9/10, 12,
13
Note this should be a repeat of the message in step
8
7.4.1 step 14 Only perform once, no repeats. Statistic
226. AWGG T No need to repeat multiple times
and 15 only counts once.
7.4.2 steps 22,
227. AWGG 24, 25, 26/27, E Add “valid” to the description of the messages As noted.
28, 29, 30
7.4.2 step 26, Merge the two steps so the objects are sent
27 and 29 in the same message.
7.4.1 step 31 Only perform once, no repeats. Statistic
229. AWGG T No need to repeat multiple times
and 32 only counts once.
7.4.3 steps 38,
230. AWGG 40, 41, 42, 43, E Add “valid” to the description of the messages As noted.
44
7.4.3 step 42, Merge the two steps so the objects are sent
43 in the same message.
Add a step: “Wait for a Session Key Change
232. AWGG 7.4.3 step 44 T Interval”. The changing of Update Keys and As noted.
Session Keys is not necessarily linked.
Change to “either <1> OK or <2>
NOT_INIT, depending on whether the
233. AWGG 7.4.3 step 46 T Will the Session Key status be NOT_INIT?
outstation has timed out waiting for a
Session Key Change.”
234. AWGG 7.4.3 step 48 E Verify that the Key Status MAC is valid. As noted.
7.4.4 steps 58,

235. AWGG E Add “valid” to the description of the messages As noted.
59, 60, 61
11687472 A-25
Cmnt Reference T/
Num Identifier 1 E2
7.4.4 step 64
236. AWGG E Statistics should be bulleted, not numbered. As noted.
7.4.5 step 69
The state machine goes back to Security Idle after
Repeat 7.4.4 instead of just sending more
237. AWGG 7.4.5 T an invalid confirmation, so all of 7.4.4 must be
invalid confirmations.
repeated.
Change to 4 Error Messages Sent and 5 of
238. AWGG 7.4.5 T Statistics count is wrong.
all the other statistics listed.
7.4.6 steps 76,
78, 79, 80, 83
7.4.6 step 80,
Merge the two steps so the objects are sent
240. AWGG 81, 83, 84, 87, T These two steps should be a single message.
in the same message.
88
241. AWGG 7.4.6 T Count of repeats is wrong Remove steps 83, 84, 85
7.5.1 step 9,
21/22, 25/26
7.5.1 steps 21, Merge the two steps so the objects are sent
22 and 25, 26 in the same message.
Missing a test for an asymmetric Key Change Repeat the test for one asymmetric Update
244. AWGG 7.5.1 T
Method Key Change Method, if supported.
245. AWGG 7.5.2 E Check the statistics after each Error As noted.
7.5.2 step 32,

49, 50, 51
7.5.2 steps 50 Merge the two steps so the objects are sent
and 51 in the same message.
248. AWGG 7.5.3 steps 59, E Add “valid” to the description of the messages As noted.
63, 65, 69, 70,
11687472 A-26
Cmnt Reference T/
Num Identifier 1 E2
71/72, 76, 77
7.5.3 steps 71 Merge the two steps so the objects are sent
and 72, 76 in the same message.
7.5.3 step 60, Specify that they are URs with event data,
250. AWGG T These need to explicitly be URs.
66, 73 CON bit set
Note that this means that the master transmits
251. AWGG 7.5.3 step 78 E As noted.
three separate Confirms.
These should be sub-steps of step 81. It’s not
7.5.4 step 82
252. AWGG T necessary to wait for a Reply Timeout if you As noted.
and 83
haven’t issued a control.
7.5.4 step 87
253. AWGG T These should be sub-steps of step 86 As noted.
and 88
7.5.4 step 93
254. AWGG T These should be sub-steps of step 92 As noted.
and 94
7.5.4 steps Merge the two steps so the objects are sent
90/91 and 94 in the same message.
256. AWGG 7.5.4 step 92 T This should read “third pass”, not “second pass” As noted.
This should be “was requested” not “were

attempted”
7.5.4 various
steps
This note is unnecessary – it must have been done
259. AWGG 7.5.4 step 96 E Delete.
already.
115, 116, 121 in the same message.
This should be a Response, not an Unsolicited
261. AWGG 7.5.6 step 126 T Change the text.
Response
11687472 A-27
Cmnt Reference T/
Num Identifier 1 E2
7.5.6 step 132,
262. AWGG T This should be a Response, not a Confirm. Change the text.
139
7.5.6 step
Merge the two steps so the objects are sent
263. AWGG 137/138 and T These two steps should be a single message.
in the same message.
142
7.5.7 step Merge the two steps so the objects are sent
156/157, 161 in the same message.
265. AWGG 9.1.1 step 4 T The default Key Change Method is <4>, not <3> Change step.
Add explanations to all enumerated values, e.g

266. AWGG 9 T As noted.
<4>.
Change the steps so each Key Change
Method supported by the DUT is tested,
9.1.1 step 6 to This test assumes that one and only one Key
267. AWGG T (including any asymmetric methods) and
8 Change Method is supported at a time.
then one Key Change Method that is NOT
supported by the DUT is tested.
268. AWGG 9.1.1 step 9 E “otherwise test 9.1.1 is complete” As noted.
269. AWGG 9.1.1 step 10 E Refer to the table of key lengths versus Methods As noted.
It is incorrect to say “User Status Change IEC

270. AWGG 9.1.1 step 12 E Delete “User Status Change”
62351-8 certificate”
Missing test to verify the outstation supports the
271. AWGG 9.1 (missing) T Add test
use of mandatory Key Change Method <4>
Missing test to verify the outstation permits the
272. AWGG 9.1 (missing) T Key Change Method <3> AES-128/SHA-1 to be Add test
disabled.
Use the wording, “select a Key Change method
that is not supported by the DUT”.
274. AWGG 9.1.1 step 12 T Don’t know what “the note in section 9” refers to. Change to “according to the description
11687472 A-28
Cmnt Reference T/
Num Identifier 1 E2
for g120v8”.
Delete the bullet. Use the text, “signed by

There is no “Key Change Method” in the the Authority using the signature
275. AWGG 9.1.1 step 12 T
certificate. algorithm that is specified by the selected
Update Key Change Method.”
There must be at least one ID certificate sent to Specify that the Certificate Type must be
276. AWGG 9.1.1 step 13 T
the outstation; send one first. <1> ID Certificate in this step.
Verify the object contains the certificate
277. AWGG 9.1.1 step 13 T The object must also contain the actual certificate from step 12, and the length is correct.
Make the changes to the default

Add steps to make sure the outstation has all the configuration in 2.6. Especially need the
278. AWGG 9.1.1 missing T
necessary pre-shared information configured outstation name and the Authority’s Public
Key.
9.1.2 step 24- Don’t need these steps to verify the success case;
279. AWGG T Delete steps.
25 done in the previous section.
9.1.2 step 34
and 43,
9.1.3 step 57
9.1.4 step 66
and 69
9.1.5 step 73 Delete that text. (search for all
280. AWGG T Don’t know what “the note in section 9” refers to.
and 76 occurences)
9.1.6 step 82
and 85
9.1.7 step 89
and 95
Etc.
Specify that the Certificate Type must be
281. AWGG 9.1.2 step 44 T Don’t need both ID and Attribute certificate
<1> ID Certificate in this step.
11687472 A-29
Cmnt Reference T/
Num Identifier 1 E2
9.1.2 step 46-
282. AWGG T Unnecessary to re-test the unsupported modes Delete the steps
47
Get a valid certificate attempting to
Can’t change the user name without invalidating change the key for a not yet installed user.
283. AWGG 9.1.2 step 48 T
the signature. Then verify the error is <11> unknown
user.
9.1.3 step 58 “Choose an unsupported Key Change
284. AWGG T <4> is not an invalid Key Change Method
and 59 Method”
9.1.4 step 68 Add a note saying it will be tested later

Remove comment about how to test that a user (by changing Session Keys). Add tests to
285. AWGG 9.3.4 E
has been added. section 9.3 for Add, Delete and Change,
verifying the operation has taken place.
Add a step, referencing the PIXIT. Skip to
This test should only be run if the device supports
286. AWGG 9.1.4 T 9.1.7 if the DUT does not support multiple
more than one user.
users.
287. AWGG 9.1.5 step 72 T Change the reference to a link Link to section 9.1.4
Delete this test because role changes are tested in As noted. Add text in 9.1 that the
288. AWGG 9.1.6 T
9.4 changing of roles is tested in 9.4.
If the DUT supports multiple users, add a
user and let it expire. If the DUT does not
support multiple users but permits
Have to determine whether it’s possible to test
289. AWGG 9.1.7 T “Common” to expire, do so. IF the DUT
expiry.
does not support multiple users and does
not permit “Common” to expire, skip
9.1.7.
g120v10 does not permit an expiry time of less Change the text to explain this test will
290. AWGG 9.1.7 step 88 T
than one day take a long time using v10.
9.1.4, 9.1.5,
291. AWGG T Missing tests for the g120v8 Add tests
9.1.6, 9.1.7
11687472 A-30
Cmnt Reference T/
Num Identifier 1 E2
The spec says that the role does not start to expire Discuss with the DNP Technical
292. AWGG 9.1.7 T
until the Update Key has been changed. Committee.
Discuss with the DNP Technical
Committee whether a device that supports
This test can’t be run on DUTs that don’t permit
293. AWGG 9.1.7 T Update Key Changes but does not support
additions.
multiple users must permit a User Status
Change, including role and expiry time.
9.1.7 steps 91-
294. AWGG E These steps should be bullets. Change them to bullets of step 90.
93 and 97-99
In step 7, specify that the Update Key
295. AWGG 9.2.1 steps 3-6 T These steps are not necessary. Change Request should name a user that
does not exist on the outstation.
296. AWGG 9.2.2 step 9 E Change the reference to a link Link to section 9.1.7
Provide more information on what is in the

297. AWGG 9.2 E Name the user.
Update Key Change Request
298. AWGG 9.2.3 T Don’t need “Test User” for this test Use “Common”
9.2.3 steps 15- Don’t need steps to add a new user if the
299. AWGG T Delete.
20 outstation doesn’t support it.
The first time, the User Number will be 1 (for
300. AWGG 9.2.3 step 22 T Change to “User Number is 1”
“Common”)
Add steps to the test adding a new user and
301. AWGG 9.2.3 step 22 T receiving a different User Number if the outstation As noted.
supports adding additional users.
Verify that the message contains Challenge Data
and the length value is correct
9.2.4 steps 28-
303. AWGG T Don’t need steps to add the “Common” user Delete.
30
11687472 A-31
Cmnt Reference T/
Num Identifier 1 E2
Some devices may not support more than one
304. AWGG 9.2.5 T Add step to skip this test if not supported.
user.
305. AWGG 9.2.4 step 34 T Test that KSQ increases As noted.
306. AWGG 9.2.5 T Missing test for g120v8 for creating the new user Test at least once.
307. AWGG 9.3.1 steps 4-6 T Don’t need steps to add the “Common” user Delete.
9/10 in the same message.
309. AWGG 9.3.1 step 11 T Must verify that the MAC is valid Add text.
9.3.2 steps 17-

310. AWGG T Don’t need steps to add the “Common” user Delete.
19
312. AWGG 9.3.2 step 24 T Must verify that the MAC is valid Add text.
9.3.2 step 26- No need to test g120v8 because that is associated

313. AWGG T Delete.
27 with User Status
314. AWGG 9.3.3 step 28 T This test is not necessary for the “Common” user Delete.
Instead, ensure the test set has the name and the
315. AWGG 9.3.3 step 28 T Add step.
Update Key for the outstation configured.
316. AWGG 9.3.3 step 30 T Verify the content of the Reply Check USR and random data
Add text. Note that it should not be an

317. AWGG 9.3.3 step 32 T Verify the content of the Confirm (MAC) echo of the test set’s MAC, and should
correctly reverse the random components.
It doesn’t matter what the Key Status is in this Note that the status may be <2>
318. AWGG 9.3.4 step 35 T
step. NOT_INIT, or may be <1> OK.
11687472 A-32
Cmnt Reference T/
Num Identifier 1 E2
319. AWGG 9.3.4 step 38 E Note that there was (1) Session Key Change. As noted.
Do not repeat all of 9.3.1, just steps 1-11 (i.e.

don’t repeat all the Methods yet)
9.3.5 steps 42- Don’t need to do the User Status Change to
321. AWGG T Delete.
44 change the key of the “Common” user.
Error code should be <10> Invalid Certification
Data
324. AWGG T Delete.
9.3.5 step 48 Better not to change the Encrypted Update Key

326. AWGG T Length – corrupting the Encrypted Key Data is a Change the step as noted.
9.3.6 step 60 better test
9.3.5 step 49 Need repeated steps to test corruption of the

327. AWGG T signature or the hash that is separate from the Add steps.
9.3.6 step 61 Encrypted Key Data
328. AWGG 9.3.6 step 62 T Error code should be <9> Invalid Signature As noted.
Add steps. Invalid USR or invalid IDA

Need separate tests for invalid KSQ, invalid USR,
could be error <11> Unknown User.
invalid encrypted IDA, invalid encrypted RB, but
329. AWGG 9.3.5, 9.3.6 T Invalid KSQ or invalid RB would need to
with a valid signature or valid MAC. Would be
be some other error code (just some non-
software errors.
zero Error Code).
330. AWGG T Delete.
11687472 A-33
Cmnt Reference T/
Num Identifier 1 E2
331. AWGG 9.3.7 step 72 T The Error Code should note be <9> or <10> Change to <1> Authentication Failed.

332. AWGG T Delete.
334. AWGG T Delete.
335. AWGG 9.4 T Missing test for SINGLEUSER role. Add a test.
Discuss with DNP Tech Committee.

Should roles (a) all be mandatory (b) all
Is it necessary to support multiple users to do except SINGLEUSER which is optional
336. AWGG 9.4 T
these tests? (c) SINGLEUSER is only allowed if you
only permit one user. Grant and Andrew
like (a) and (c) but not (b).
Test that the specified AOR is not
337. AWGG Missing T Need a test for the Area of Responsibility.
configured in the outstation, and that it is.
Add a section discussing the assignment of roles
to users (see comment AWGG336), and a set of
338. AWGG 9.4 T As noted.
steps for selecting the user to be used for each
“roles” test.
Must repeat all the Roles tests using g120v10 and
g120v8.
Remove the steps up to the Null Response, and
9.4.1 through
340. AWGG T refer to the introductory section from each of the As noted.
9.4.10
other sections to select the appropriate user.
Combine the steps that send a g120v13 and
9.4.1 through
341. AWGG T g120v15 objects into a single step sending a single As noted.
9.4.9
fragment.
11687472 A-34
Cmnt Reference T/
Num Identifier 1 E2
Add sections for :
“How to verify the user can monitor data”
“How to verify the user can operate controls”
“How to verify the user can transfer data files”
9.4.9 step 127, Don’t include <32768> in this test since this is
343. AWGG T As noted.
129 SINGLEUSER
As noted. Will need to change spec to add
344. AWGG 9.4.9 step 126 T Change to “a non-zero Error Code”
an “Invalid Role” error code.
Missing a test to verify that the user status does
345. AWGG 9.4 T not actually change until the Update Key Change Add test for Role, Expiry Interval
is confirmed.
Verify that the SCS increments each time a new
g120v10 is transmitted.
Verify that the SCS
(IECUsersRoles.statusChangeSequenceNumber)
347. AWGG 7.1.1 step T increments each time a new g120v8 is transmitted. As noted.
Note that the certificate’s serial number can
increment instead.
Missing a test that the outstation rejects a User
Status Change (g120v10 or g120v8) with an SCS
348. AWGG 9.1 T Add test.
that gets smaller or increases by more than half of
the range.
349. AWGG 7.1 T Add a test to delete a user Add test.
7.1.1 step 5 Repeat and verify that the User’s Public Key
350. AWGG T Add test.
7.1.1 step 15 changes
Verify the signature uses the outstation’s public
351. AWGG 7.3.2 E The test set will verify.
key.
Move the test for randomness of User’s Public
352. AWGG 10.1.7, 7.1 E As noted.
Key to 7.1
11687472 A-35
Cmnt Reference T/
Num Identifier 1 E2
Add a test to verify that the master sends an
353. AWGG 7.1 T appropriate expiry time when selected at the Add test.
Authority.
Make sure all these tests are performed using both
g120v10 and g120v8.
Verify that each time the role, expiry time, or
355. AWGG 7.1 T Public Key changes, the Update Key is also As noted.
changed.
Add a section: “How to change the user Public
Key”
Delete this section because the tests have been
357. AWGG Section 10 E As noted.
moved into 7.1.
11.1.1 and
358. AWGG E Clarify “skip this test” As noted.
anywhere else
Configure the DUT for a TCP/IP
359. AWGG 11.1.1 step 3 E Remove “but”. connection to the test set. Disable the use
of TLS.
11.1.2 step 9-
360. AWGG T Do not use “hard” numbers for KSQ. Verify values in terms of “X”, for KSQ.
12
“If the DUT can initiate a connection,
11.1.1 step 6 cause it to do so and verify it does so using
Either a master or an outstation can initiate a
361. AWGG T TCP port number 20000. If the DUT
11.1.3 step 28 connection.
accepts connections, initiate a TCP/IP
connection to the DUT.”
362. AWGG 11.1.2 T Disable aggressive mode for these tests As noted.
Should repeat five more times because default in

363. AWGG 11.1.3 T Change “two” to “five”.
2.6 is Max Authentication Failures = 5.
Add TLS_RSA_WITH_AES_128_SHA because
it is the default and so should be permitted.
11687472 A-36
Cmnt Reference T/
Num Identifier 1 E2
Find out the difference between TLS change They are the same thing (says Herb Falk).
365. AWGG 11.2.7 T
cipher timeout and the renegotiation timeout. Delete 11.2.7.
“Verify that the act of checking the CRL
366. AWGG 11.2.8 step 37 T Clarify what is being tested. does not drop the existing TLS security
connection and TCP/IP connection.”
11.2.10 step
367. AWGG T Typo: DTU Fix.
42
Attempt another connection using a different
368. AWGG 11.2.10 T As noted.
revoked certificate found in the CRL.
Need a test that it does not accept a certificate
369. AWGG 11.2.13 T Add test.
from a CA that is not configured at the DUT.
Is “user name” the correct name for the name in a
370. AWGG 11.2.13 T Change to “subject name”
certificate?
13.4.14 TLS Change “user name” to “subject name” because
371. AWGG T As noted.
parameters that is what it is called in the RFC.
Test for Attribute Certificates as well as ID
372. AWGG Missing T Add test.
Certificates
11.3.1, 11.3.2,
373. AWGG E Clarify what is meant by “skip this test” As noted.
11.4, 11.4.4
“but with the same destination DNP3 address” Is
374. AWGG 11.3.2 E Delete.
not necessarily true.
Must disconnect both channels for the test to
375. AWGG 11.3.2 step 6 T “Disconnect both channels”
work.
Wait for a reply timeout from step 10 before
repeating the Read
“Reconnect the link during the second Reply
Timeout”
378. AWGG 11.3.2 T Make the Reply Timeout longer to make Increase to 20 seconds.
11687472 A-37
Cmnt Reference T/
Num Identifier 1 E2
reconnection easier.
379. AWGG 11.4 E Typo: DTU Global search and replace.
380. AWGG 11.4 E “upstream protocol” is not very clear “upstream communication link”
“upstream” is toward test set(s) acting as

381. AWGG 11.4 E Define “upstream” and “downstream” masters. “downstream” is toward test
set(s) acting as IEDs.
The DUT may not support more than one user on Skip 11.4.1if the DUT does not support
382. AWGG 11.4.1 step 2 T
the upstream side. more than one user on the upstream side.
Clarify that the User Number <1> in the example
is a different entity on each association.
“Verify the DUT correctly responds to
“initializes” sounds like the outstation side of the
384. AWGG 11.4.2 step 9 E Session Key initialization for each of the
DUT initiates the key change, which is incorrect.
upstream users”.
“Configure the DUT so that each of the
upstream users (up to six) has a
corresponding downstream user. Divide
385. AWGG 11.4.2 step 5 E The upstream side may not support multiple users the upstream users among multiple
downstream associations (e.g. six
upstream users on three downstream
associations)”
Tell the tester to write down the mapping of the
users.
11.4.2 step 8 Tell the tester to write down the numbers assigned
387. AWGG E As noted.
and 9 to the users on each association.
Disable aggressive mode so it does not occur until
388. AWGG 11.4 step 1 T As noted.
11.4.3.
389. AWGG 11.4.2 – 11.4.3 T It’s not clear what functionality is required in data Discuss with DNP technical committee
concentrators. Is it permitted to have multiple whether it would be useful to make
11687472 A-38
Cmnt Reference T/
Num Identifier 1 E2
users to the master and only the default to the multiple users or remote Update Key
IED, for example? change mandatory in one direction or the
other for data concentrators in the interests
of interoperability.
Missing test for the data concentrator to create
390. AWGG 11.4.2 T Add test.
users on downstream IEDs.
Missing information from the specification about
Discuss with DNP Technical Committee
391. AWGG 11.4.3 T how to perform remote Update Key Change
how it should be done.
through a data concentrators.
Require that this test is always run using DNP3
with security turned off on the upstream side; As noted. Can be done with a “repeat”
392. AWGG 11.4.4 step 31 T
additional tests using non-DNP3 protocols are step.
optional.
Add a note that it is better to perform tests in
393. AWGG 2.1.3 E sections 12 and 13 first in order to verify the As noted.
documentation before proceeding.
Make the requirement that four certificate
394. AWGG 12, 12.1.4 E authorities must be supported for TLS, not As noted.
necessarily for all parts of the specifications.
Add a statement that these test procedures are for
395. AWGG 1.1 E version 5 and the DUT should be configured for As noted.
only that version.
Add an item for “How to configure the system for
396. AWGG 13.4 E As noted.
a particular version of Secure Authentication”
Add caption and use it for reference in
397. AWGG 12.1.2 E Table is missing a caption
12.1.2 step 5.
Missing a test for whether the device
398. AWGG 12.2 T documentation states whether it supports multiple Add test.
users.
399. AWGG 12.2.2, 12.2.8 T SHA-1 is no longer mandatory. Remove SHA-1.
11687472 A-39
Cmnt Reference T/
Num Identifier 1 E2
The outstation name is not needed if you don’t
400. AWGG 13.1.4 T Add an “if…”
support Update Key Change.
It is not necessary to know how the random
13.1.8, 13.1.9, numbers are generated as long as they are
401. AWGG T Delete this requirement.
13.2.8 sufficiently random, or how the keys are generated
as long as they are random.
Change to “operate only in challenge-
13.1.11,
402. AWGG E “purely” is colloquial reply mode with aggressive mode
13.2.10
disabled.
List the application layer function codes
403. AWGG 13.1.13 E May confuse “no ack” with data link layer
specifically
13.1.15, “security logs” is not used anywhere else in the
404. AWGG E Delete “security”
13.2.15 document
405. AWGG 13.1.20 E Time synchronization is never tested anywhere Remove.
Change text to “Whether they are assigned

Knowing how the USRs are assigned is useful randomly or sequentially or according to
406. AWGG 13.2.7 E mainly because then the tester can know how to some other convention, whether they are
configure them if necessary. configured and if so, how to configure
them.”
Change title to “How to Cause a Critical
Function to be Sent” with description
407. AWGG 13.3.1 E This item is confusing “How to make the DUT send at least one
type of message that will be sent in
aggressive mode because it is critical”
408. AWGG 13.3.3 E Typo: “oustation” Fix.
Delete the corresponding Master and

409. AWGG 13.4.2 T This section is redundant
Outstation items.
410. AWGG 13.4.3 T The reply timeout is not necessarily just a security Discuss with DNP Technical Committee
parameter; is it also subject to the rule that it can’t
11687472 A-40
Cmnt Reference T/
Num Identifier 1 E2
be changed remotely?
Delete the Annex and all references to it.

See if it’s vital to be able to repeat the
This is complex to create. It makes more sense to
411. AWGG Annex A T same random data and MAC calculation.
trust the test set.
If so, add a requirement on the test set to
be able to do so.
11687472 A-41
11687472
11687472
Export Control Restrictions The Electric Power Research Institute, Inc.
Access to and use of EPRI Intellectual Property is granted (EPRI, www.epri.com) conducts research and
with the specific understanding and requirement that development relating to the generation, delivery
responsibility for ensuring full compliance with all applicable and use of electricity for the benefit of the public. An
U.S. and foreign export laws and regulations is being independent, nonprofit organization, EPRI brings
undertaken by you and your company. This includes an
together its scientists and engineers as well as
obligation to ensure that any individual receiving access
hereunder who is not a U.S. citizen or permanent U.S. experts from academia and industry to help
resident is permitted access under applicable U.S. and address challenges in electricity, including
foreign export laws and regulations. In the event you are reliability, efficiency, health, safety and the
uncertain whether you or your company may lawfully obtain environment. EPRI also provides technology, policy
access to this EPRI Intellectual Property, you acknowledge and economic analyses to drive long-range
that it is your obligation to consult with your company’s legal
research and development planning, and supports
counsel to determine whether this access is lawful.
Although EPRI may make available on a case-by-case research in emerging technologies. EPRI's
basis an informal assessment of the applicable U.S. export members represent approximately 90 percent of the
classification for specific EPRI Intellectual Property, you and electricity generated and delivered in the United
your company acknowledge that this assessment is solely States, and international participation extends to
for informational purposes and not for reliance purposes. more than 30 countries. EPRI's principal offices and
You and your company acknowledge that it is still the
laboratories are located in Palo Alto, Calif.;
obligation of you and your company to make your own
assessment of the applicable U.S. export classification and Charlotte, N.C.; Knoxville, Tenn.; and Lenox, Mass.
ensure compliance accordingly. You and your company
Together…Shaping the Future of Electricity
understand and acknowledge your obligations to make a
prompt report to EPRI and the appropriate authorities
regarding any access to or use of EPRI Intellectual Property
hereunder that may be in violation of applicable U.S. or
foreign export laws or regulations.
© 2012 Electric Power Research Institute (EPRI), Inc. All rights reserved.
Electric Power Research Institute, EPRI, and TOGETHER…SHAPING THE
FUTURE OF ELECTRICITY are registered service marks of the Electric
Power Research Institute, Inc.
1026561
Electric Power Research Institute

3420 Hillview Avenue, Palo Alto, California 94304-1338 • PO Box 10412, Palo Alto, California 94303-0813 • USA
11687472800.313.3774 • 650.855.2121 • askepri@epri.com • www.epri.com

Distributed Network Protocol - DNP3 - Security Interoperability Testing 2012

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Distributed Network Protocol - DNP3 - Security Interoperability Testing 2012

Uploaded by

Copyright:

Available Formats

Distributed Network Protocol (DNP3) Security

Interoperability Testing 2012

EPRI Project Manager

ELECTRIC POWER RESEARCH INSTITUTE

Goals and Objectives

Relationships Between Deliverables

IEC 60870-5-7 DNP3 Secure

2004 2005 2006 2007 2008 2009 2010 2011 2012

Status of DNP3 Secure Authentication Test Procedures

Status of IEC 62351-5 Edition 2

Status of IEC 60870-5-7

DNP Secure Authentication Test Procedures

Edition 2 of IEC 62351-5

Edition 1 of IEC 60870-5-7

1. GG 2.1 E Section references are incorrect Fix Accepted

2. AW 1.1 E “Shall” implies binding – is this correct Delete “shall” Accepted

4. AW 2.1 E “roughly” is colloquial Change to “approximately” Accepted

5. AW 2.1.3 E The correct term is DNP3-XML Change all occurrences Accepted

6. AS 1.3 E Missing definition of PIXIT Add Accepted

Need to clearly identify which version of the

Add the word “selectively” to show that it does

2.3.1 items 14- Specifically mention both “manually

20. GG 1.2 E Add NIST SP-800-22 to the references As noted Accepted.

Note that statistics are tested even if the logging is

3.1 step 8, Change to AES-256/SHA-256 to match the

26. GG 3.2 T Restart test numbering As noted. Accepted.

Provide more detail that the test includes changing

29. GG 3.2.2 step 30 T Step is incomplete Complete sentence Accepted.

32. AW 3.4.5 step 21 T The CSQ should perhaps be 3? Investigate

Note in the preamble that this only works if there

For instance, this is not an aggressive

41. CA 4.1.3 T Is the function code 0x21 correct? Investigate Accepted.

Missing a test for a CSQ that does not increment

Missing a test for ignoring an unexpected Make a change to the SA

64. 5 Note that Default <1> is used if the test set is

5.1.4 As noted. Also provide a list of the

The text is mixed up between Max Reply

Add text to deal with the case where there is not

79. GG 6.1.6 T Correct invalid reference As noted Accepted.

Make it clearer exactly when the session key

Not necessary to disconnect – there is a restart in

Keep the repeat of 53 through 57, then

112. AWGG 6.6 E “Compiling” is confusing Replace “compiling” with “collecting”

“Configure the DUT to require authentication

Change “skip this test” to “skip to step 23”

“Place system in the default configuration

After discussion with Tech Committee,

After discussion with the technical

140. AWGG 7 Section still needs to be reviewed.

Where did the required lengths of the Challenge

Change the Expected Session Key Change

152. AWGG 8.2 E Typo “orrectly” Fix typo.

Aggressive mode must be turned off for this to

In step 33, only wait for half a Reply

Then just issue a status request.

Change the challenge sent by the master

Critical messages sent should be 3 (challenge

Typo: Configure the DUT as Configure the

7.4.4 steps 58,

7.5.2 step 32,

This should be “was requested” not “were

Add explanations to all enumerated values, e.g

268. AWGG 9.1.1 step 9 E “otherwise test 9.1.1 is complete” As noted.