Submitted to UN TF-CS/OTA by Tesla, Inc. v0.

Interpretation document and recommendation from Tesla on Cyber Security

regulations of UNECE WP.29 GRVA

United Nations ECE/TRANS/WP.29/201x/xx

Economic and Social Council Distr.: General
DD MM YYYY

Original: English

Economic Commission for Europe

Inland Transport Committee
World Forum for Harmonization of Vehicle Regulations
xxx session
Geneva, DD–DD MM YYYY
Item XXX of the provisional agenda
Draft new Regulation on software updates

Interpretation document and recommendation from Tesla on Cyber Security

regulations of UNECE WP.29 GRVA

Submitted by Tesla Inc. after feedback on the Test Assessment Pilot phase
(April through June 2019)

1
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

Contents
Page
5. Approval .......................................................................................................................................... 4
6. Cyber Security Management System (CSMS) Certificate of compliance......................................... 4
7. Specifications..................................................................................................................................... 5
8. Modification and extension of the vehicle type ................................................................................ 7

2
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

Introduction

This document was written by Tesla representative in order to share feedback on the Cyber Security regulation, after
concluding the Test Assessment in the pilot phase of the Cyber Security regulation.

As we went through the test assessment phase, we realized that the cybersecurity regulation needed improvements to
ensure the manufacturer was appropriately equipped to evaluate, analyze, and respond to cyber security threats.
Instead, we found that the test assessment led to divergent views of how the both the cybersecurity and OTA
regulations are to be interpreted, as well as how the manufacturer should be evaluated against the regulation. Below
are listed our top-level concerns, with more details on each provision throughout the rest of this document.

1. Lack of objective requirements and a defined assessment framework (ex: Section 7.2 / 7.3 in CS regulation)

Type approval regulations and the audits conducted by approval authorities generally are objective with precise
requirements outlining clear targets that the manufacturer must meet. They are almost exclusively focused on
evaluating very specific and externally visible vehicle behaviors. We believe that the OTA and more significantly
the cybersecurity regulations are too abstract in this regard, and do not provide sufficient guidance to type-approval
authorities and inspection services as to what manner they are supposed to evaluate compliance.

The regulation also does not adequately provide a framework to objectively evaluate the security of a vehicle
system. This creates opportunities for subjective assessment and different outcomes from one testing facility to
another (see Section 7.2.2.2 for a more complete example). In the latest interpretation document, the UK feedback in
is a first step towards appropriate guidelines. Similarly, the NCSC assessment framework is well actionable, and
focusing on what matters: ensuring OEMs are well equipped in their cyber security strategy
(https://www.ncsc.gov.uk/collection/nis-directive/cyber-assessment-framework)

Considering the unique considerations that apply in view of cybersecurity management, and in view of our
experience that the lack of objective criteria to be met encourage auditors to ask for specific implementation details,
we believe that it is not the responsibility of this regulation or the testing facilities to test that a feature or a particular
system is correctly implemented. The goal is to ensure that the manufacturer has appropriate processes in place to
assess its systems, and review risk assessments and mitigations. For a specific vehicle type approval, the result of the
manufacturer's internal process should be reviewed, not the product itself.

We recommend that the criteria in the standard be refined (similar to how the UK annotations were approached) so
that the auditing process is normalized and objective. The viability of processes that are in place can, for example,
be tested through a case example. Most importantly, the regulation or the interpretation document should provide a
set of outcomes and goals for each provision, so the provision can be clearly assessed and understood.

2. Type Approval and Extension requirements (ex: Section 8.1 in CS regulation)

We believe the current regulations and interpretations extensively consider the context of a traditional OEM "model-
year" release rate, but do not factor in the perspective of a company that releases software at a faster rate than once
or twice per year - similar to the model used by every other modern internet-connected consumer device. The
current regulations and interpretations do not fully consider the overhead introduced to a manufacturer that releases
software at a frequent pace. We believe is the direction the entire industry will move in, especially to address
security and safety considerations. Any requirement to submit documentation, or wait for approval, for every
individual software release will compromise velocity and agility; and works against the goals of a cyber security
strategy. The regulation should be designed so it is based on evaluation of the manufacturer's internal processes,
appropriate records of activities, and trust between manufacturers and authorities.

We recommend that the manufacturer needs to keep the authority to decide which updates warrant reporting and/or
approval, based on an internal assessment. The results of this assessment can be provided to the Type-Approval
Authority when a certification is deemed necessary. In addition, the OEM can document and submit to auditing the

3
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

process for managing updates and determining which need to be reported or approved. Type approval assessment
will be recorded at the manufacturer’s facilities.

3. Confidentiality and IP protection (ex: Section 3.2.2 in CS regulation) 

The cyber security assessment regulation emphasizes an evaluation of internal processes to design for security,
evaluate and respond to threats, as applied to a vehicle type. The review from testing facilities should be on the
internal processes and test/threat assessments that were performed by the OEM, not on IP-protected information
such as source code or architecture design. According to the regulation, design details should only be shared if
deemed necessary to describe specific risk mitigation. As defined in section 3.2.2:

3.2.2. In cases where information is shown to be covered by intellectual property rights or to constitute
specific know-how of the manufacturer or of their suppliers, the manufacturer or their suppliers
shall make available sufficient information to enable the checks referred to in this Regulation to
be made properly. Such information shall be treated on a confidential basis.

The lack of an explicit process by which an audit is conducted is at the apex of why this discrepancy exists. Once
objective criteria are established (as mentioned above), the standard should prescribe the method by which
compliance is measured. For each requirement, the auditor should be able to verify that the OEM is in compliance
without needing to document the design. The process should describe what is expected to be in the report. Without
an audit process explicitly defined, we cannot provide appropriate feedback on whether the process is reasonable.
An example of an audit process where the outputs are clearly defined can be found in 8110.49, section 2-9.

We recommend that the process of evaluation and the responsibilities of each party (testing facility, approval
authority, manufacturer) are more clearly defined:
 The CSMS approval provides an assessment and validation of the manufacturer’s internal processes
towards Cyber Security.
 The CS Type Approval should be based on the results of the processes above, via documentation on risk
assessment, threat analysis, mitigation, and incident response. The design of the vehicle system or specific
features shall only be shared, if deemed necessary to justify the documentation provided.
 Only if the documentation is missing or insufficient should the testing facility require to request more
details on implementation or design of security features.

On Security and Software Updates

We believe Tesla to be in a unique position today, with a complete fleet of connected vehicles, comprehensive
software update capabilities, and under constant scrutiny by the security community. The approach we take is one of
continuous improvement. We, and the rest of the security research community, will constantly be learning new
things about encryption protocols, operating systems, networks, and processors - and the best way to ensure ongoing
security of our systems is to constantly update our software based on what all of us have learned. At Tesla, we
believe that in order to design and build inherently secure systems, manufacturers must work closely with the
security research community to benefit from their collective expertise and diversity of thought - anything less fails to
put consumers’ best interests at the center of product creation, distribution and security. Our goal is to ensure that all
Tesla owners constantly benefit from the brightest minds in the global community.

Our approach is currently unique in the automotive industry, in terms of the scope of over-the-air software updates,
as well as their frequency. However, we are confident that the rest of the industry will move in these same
directions, as software and internet-connected functionality becomes more pervasive in vehicles and increasingly
demanded by customers. Tesla is actively interested in sharing the best practices we've painstakingly developed over
our formative years, and in helping to ensure that these technologies are appropriately and productively regulated.
At Tesla, we strongly believe that product security follows closely from sound and disciplined engineering, and have
developed organizations, tools, and processes that help us achieve both. Our recommendations below represent
what we believe will most effectively accomplish our shared goal of demonstrably secure products, within the
design, development, validation, and deployment frameworks that Tesla and the rest of the industry will use to
produce them.

4
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

Legend

Tesla Feedback

5
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

1. Scope

Not included in this document

2. Definitions

Not included in this document

3. Application for approval

Not included in this document

4. Marking
Not included in this document

5. Approval

Not included in this document

6. Cyber Security Management System (CSMS) Certificate

of Compliance

Not included in this document

6
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

7. Specifications

7.1. General specifications

7.1.1. The requirements of this Regulation shall not restrict provisions or requirements of other UN
Regulations.

7.1.2. The vehicle manufacturer may refer to [the Recommendation / Resolution on Cyber Security] in
their assessment of cyber security risks and the mitigations, as well as when describing the
processes employed.

7.2. Requirements for the Cyber Security Management System

7.2.1. For the preliminary assessment the Approval Authority or Technical Service shall verify that the
vehicle manufacturer has a Cyber Security Management System in place and shall verify its
compliance with this Regulation.

7.2.2. The Cyber Security Management System shall cover the following aspects:

7.2.2.1. The vehicle manufacturer shall demonstrate to an Approval Authority or Technical Service that
their Cyber Security Management System considers the following phases:
- Development phase;
- Production phase;
- Post-production phase.

7.2.2. The vehicle manufacturer shall demonstrate that the processes used within their Cyber Security
Management System ensure security is adequately considered. This shall include:

a) The processes used within the manufacturer’s organization to manage cyber security;
b) The processes used for the identification of risks to vehicle types;
c) The processes used for the assessment, categorization and treatment of the risks identified;
d) The processes in place to verify that the risks identified are appropriately managed;
e) The processes used for testing the security of the system throughout its development and production
phases;
f) The processes used for ensuring that the risk assessment is kept current;
g) The processes used to monitor for, detect and respond to cyber-attacks on vehicle types;
h) The processes used to identify new and evolving cyber threats and vulnerabilities to vehicle types;
i) The processes used to appropriately react to new and evolving cyber threats and vulnerabilities.

Tesla Feedback

It is unclear from the requirement above what would be an objective pass/fail criterion for each of the
subsections of this requirement. The raises the following question:
 Is the expectation that an audit will review each of these processes for correctness of intent or
just that the process exists?
o If it is for ‘correctness of intent’: the lack of rationale for each of the subsections and
the lack of explicit methods of compliance in the content of the process makes the

7
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

review of compliance subjective by nature.

o If it is for ‘existence of the process’: the value of the audit should be questioned. If the
goal of the audit is to ensure that an OEM is applying best practices, then just having a
process does not seem adequate.

Furthering the example, in the preliminary report provided for the audit conducted onsite, the following
was stated in section 7.2.2.2 a):
“It’s clear a process is in place but it’s not clear based on the roles described in the provided
document who is responsible for managing cybersecurity processes in the different phases, or
who is the owner of the overall processes.”

This section was scored as “Process implemented but documentation is incomplete”.

There is nothing in the standard that implies that these are explicit requirements. This is not to say that we
are in disagreement with the finding, but that the finding itself is subjective, given that it is written with
nothing to measure against. Another auditor may interpret compliance differently.

The UK annotations to the interpretation document prescribe a more objective set of criteria to be
measured against. Tesla recommends that the criteria in the standard be refined (similar to how the UK
annotations were approached) so that the auditing process is normalized and objective.

Furthering the example, the UK annotations to section 7.2.2.2 a) provides a better context of the intention
of the regulation as well as examples of documentation that would show compliance. This would facilitate
a normalized, objective review when conducted by an independent auditor. This also enables Tesla to
provide feedback on whether the requirements defined are reasonable/correct.

Recommend to:
Define clear objective criteria for each bullet under 7.2.2.2 to qualify goal and method for assessment.
Refer to NCSC as guidance on how to expose clear outcomes and objectives without making
recommendations on how to achieve them.

Example
https://www.ncsc.gov.uk/collection/nis-directive/cyber-assessment-framework/caf-objective-a

8
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

7.2.2.3. The vehicle manufacturer may refer to [the Recommendation / Resolution on cyber security] when
describing the processes they have employed.

7.2.2.4. The vehicle manufacturer shall be required to demonstrate how their Cyber Security Management
System will manage dependencies that may exist with contracted suppliers and service providers
in regards of the requirements of paragraph 7.2.2.2

7.3. Requirements for vehicle types

7.3.1. Before the assessment of a vehicle type for the purpose of type approval is carried out the vehicle
manufacturer shall demonstrate to the Approval Authority or Technical Service that their Cyber
Security Management System has a valid CSMS Certificate of Compliance relevant to the vehicle
type being approved.

7.3.2. The Approval Authority or Technical Service shall verify that the manufacturer has taken the
necessary measures relevant for the vehicle type to:

(a) Collect and verify information required under this regulation, through the full supply chain;

(b) Document appropriate design and test information;

9
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

(c) Implement appropriate security measures in the design of the vehicle and its systems;

Tesla Feedback

It is not the responsibility of the Testing Facility or Authority to review how a feature, product or
design was implemented. It is their responsibility to evaluate the product of the manufacturer’s risk
assessment, test results and mitigation. Design details should only be provided as required to clarify
the documentation provided by the manufacturer, for ex:
 Definition of subsystems, their boundaries and interfaces;
 Limits between systems, and which systems have impact on cyber security.

It is more appropriate to document processes for risk evaluation, and the outputs of those processes
for evaluation to the vehicle type. There is no reason to include vehicle architecture and design to
this exercise.

Recommend change to:

7.3.2. The Approval Authority or Technical Service shall verify that the manufacturer
has taken the necessary measures relevant for the vehicle type to:
(a) Collect and verify information required under this regulation, through the full
supply chain;
(b) Document risk assessment, test results and mitigations applied to the vehicle
system, including appropriate design information supporting the risk assessment;
(c) Verify implementation of security measures in the design of the vehicle and its
systems;

Item (c) should be accomplished through review of test results. If not available, it may be achieved
via code inspection or design audit, as a last resort measure if other documents cannot be provided.
The documentations provided by the OEMs should already reflect the assessment, necessary design
and mitigations.

7.3.3. The vehicle manufacturer shall demonstrate the risk assessment for the vehicle type in terms of the
vehicle systems, the interactions of the different vehicle systems and the entire vehicle.

Tesla Feedback

Level of information requested by testing facilities during the pilot phase was inappropriate.
Recommend referring to results of the security assessment process, instead of design & test
information.

7.3.4. The vehicle manufacturer shall demonstrate how the design of critical elements of the vehicle type
are protected against risks identified in the vehicle manufacturer’s risk assessment. Proportionate
mitigations shall be implemented to protect such elements.

10
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

Tesla Feedback

Level of information requested by testing facilities during the pilot phase was inappropriate.
Recommend referring to results of the security assessment process, instead of design & test
information.

7.3.5. The vehicle manufacturer shall demonstrate how they have implemented appropriate and
proportionate measures to protect dedicated environments on the vehicle type (if provided) for the
storage and execution of aftermarket software, services, applications or data.

Tesla Feedback

Level of information requested by testing facilities during the pilot phase was inappropriate.
Recommend referring to results of the security assessment process, instead of design & test
information.

7.3.6. The vehicle manufacturer shall describe what testing has been performed to verify the
effectiveness of the security measures implemented and the outcome of those tests.

8. Modification and extension of the vehicle type

Tesla Feedback

This section is missing the requirements of when an OEM needs to consider a new/extended vehicle
type with respect to security. Assuming that 8.1.1 implies when this is appropriate, the term
“appreciable averse effect’ is subjective. This could be interpreted as ‘The OEM needs to file for an
extension every time a software update is sent out’. The criteria by which this is applicable needs to
be defined objectively.

The lack of discussion in the interpretation document on this topic is worrisome. Without explicit
objective criteria, Tesla is unable to provide appropriate feedback on whether the proposed Type
Approval/modification/extension is reasonable.

11
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

8.1. Every modification of the vehicle type shall be notified to the approval authority which granted
the approval. The Approval Authority may then either:

Tesla Feedback

It is unmanageable for every modification to be reviewed by the approval authority, as all

subsystems in the vehicle may be subject to scrutiny from a cyber security standpoint.

Recommendation:
 CS Type Approval should include a definition of the subsystems, interfaces and limits.
 CS certification should include a review of the process for risk assessment, and type
approval assessment.
 Future modifications are to the discretion of the OEM, including a self-evaluation of which
changes may impact type approval, and only submit to authorities if so.

Proposal to change to:

“For every modification or update to the vehicle system, the manufacturer makes an assessment of
the impact on the vehicle type. If the modification negatively impacts functionality or architecture as
defined in the prior Type Approval documentation, the manufacturer shall notify the approval
authority which granted the approval.”

8.1.1. Consider that the modifications made are unlikely to have an appreciable adverse effect and that in
any case the vehicle still complies with the requirements; or

Tesla Feedback

Needs clear and objective requirement. “Appreciable adverse effect” is subjective

Proposal to change to:

“Consider that the modifications made still comply with the requirements and documentation of
prior type approval; or”

8.1.2. Require a further test report from the technical service responsible for conducting the tests.

Tesla Feedback

What test report is this referring to? Is this a complete reassessment of the system?

Proposal to change to:

“Review the OEM risk assessment and test results, and require the manufacturer to subject to a type
approval extension.”

12
Submitted to UN TF-CS/OTA by Tesla, Inc. v0.5

8.1.3. Confirmation or extension or refusal of approval, specifying the alterations, shall be

communicated by means of a communication form conforming to the model in Annex 2 to this
Regulation. The Approval Authority issuing the extension of approval shall assign a series number
for such an extension and inform there of the other Parties to the 1958 Agreement applying this
Regulation by means of a communication form conforming to the model in Annex 2 to this
Regulation.

9. Conformity of production

Not included in this document

10. Penalties for non-conformity of production

Not included in this document

11. Names and addresses of Technical Services responsible

for conducting approval test, and of type approval
authorities

Not included in this document

13