Virtual homologation

of an ALKS according
to UNECE R157

| Co-authored by

White paper

Abstract
Proving that an automated driving risk is acceptable requires extensive investigation into the challenge
of handling real-world traffic. UNECE R157 is the first regulation to formalise the idea of virtualising major
parts of the validation and homologation process. This white paper discusses a recommended process
for the virtual homologation of an Automated Lane Keeping System (ALKS) by following the provisions in
UNECE R157, including test approaches to prove system safety.

TÜV SÜD
Contents
INTRODUCTION 3

PROVING SYSTEM SAFETY 6

RELIABILITY OF SIMULATION RESULTS 10

CONCLUSION 14

About TÜV SÜD expert

Christoph Miethaner
Test Methods Engineer for Highly Automated Driving at TÜV SÜD
Christoph is an automated driving specialist with four years of industry experience and
participates in various committees that promote the advancement of these complex
automotive technologies. He is a member of the openSCENARIO working committee which
focuses on the virtual development, test and validation of driver assistance functions,
automated and autonomous driving. He is also a member of the committee ISO/TC22/SC33/
WG09 for test scenarios of automated driving systems, and DIN NA 052-00-33-09 AK,
which is the national mirror committee of ISO/TC22/SC33/WG09. He is a member of the
Methods, Models and Effectiveness Calculation working committee at P.E.A.R.S (Prospective
Effectiveness Assessment for Road Safety), and represents the international not-for-profit
association CITA (international motor vehicle inspection committee) at UNECE VMAD
(Informal Working Group on Validation Methods for Automated Driving at United Nations
Economic Commission for Europe).

About dSPACE expert

Jann-Eve Stavesand
Head of dSPACE Consulting
Jann-Eve supports customers worldwide to define test strategies for complex automotive
systems, helping them to overcome the challenges of developing safety-critical systems.
He is a specialist in testing and quality assurance of complex automotive software and
systems, including their approval and homologation. He sits on various standards committees
in Germany and was involved in the development of ISO 26262:2018 - "Road vehicles –
Functional safety", focusing on the software and processes elements. He is also involved
with the standardisation of Safety of the Intended Functionality (SOTIF).

2 Virtual homologation of an ALKS according to UNECE R157 | TÜV SÜD

Introduction
The high complexity of an automated
driving task has prompted the idea
of virtualising major parts of the
validation and homologation
process. UNECE regulation R157,
for the approval of vehicles with
ALKS, is the first regulation to
formalise these ideas. The authors
of UNECE R157 not only defined
a fixed set of test cases, but also
augmented the prescribed set of
tests by making it a requirement to
achieve higher-level principles, such
as acceptable risk.
CERTIFICATION AND SAFETY ARGUMENT PROCESS
AND BASIS FOR VERIFICATION
Regulations
Methods
KPI
Scenario approach
TÜV SÜD | Virtual homologation of an ALKS according to UNECE R157
Proving that a risk is acceptable
requires extensive investigation into
the challenge of handling real-world
traffic. Currently, the prevailing
opinion among experts is that this task
can only be performed on a virtual
basis. Therefore, several types of
testing are required (from simulation
to real-world tests, see figure below).
The following sections describe
the virtual evaluation method used
to determine whether a risk is
acceptable for an ALKS, as per the
UNECE R157 guidelines.
Argument for certification authorities
Argument for safety
Verification results
Selection
Scenario
catalogue
Verification
Test management framework (end-to-end)
ODD
Design
Risk Analysis
According to these guidelines,
simulation plays a crucial role for
safety argumentation. Validating
certain safety measures by simulation
means that it must also be proven
why the results of the simulation are
reliable. The section “Reliability of
simulation results” gives an overview
of the different methods to achieve
trustworthiness of a virtual test setup.
In addition, we compared simulated
traffic situations with equivalent
real-world test drives to analyse the
accuracy of the simulation.
Exploration of unknown
Field observation
Selection
Trustworthiness
Validation
Test execution plan /
V&V Strategy
3
Status quo – ALKS UNECE R157
A key aspect of UNECE R157 that
is unique to the regulatory and
standardisation framework is the
definition of a reasonable risk.
UNECE R157 Annex 4 Clause 2.16.
defines an unreasonable risk as
“[...] the overall level of risk for
the driver, vehicle occupants and
other road users which is increased
compared to a competently and
carefully driven manual vehicle.”
Compared to existing sources, such
as ISO 26262-1:2018 Clause 3.176,
which defines the same term as
“risk judged to be unacceptable in
a certain context according to valid
societal moral concepts,” UNECE
R157 redefines the threshold for an
acceptable risk to a comparison that
can be measured and argued to a
certain degree. This is a key enabler
for the approval of vehicles with
higher levels of automation, since
it acknowledges the presences of a
residual risk and provides guidance
on how this risk can be quantified.
Compared to the traditional
homologation process, where
real-world test drives at the end
of the development process form
the basis for homologation, UNECE
R157 demands independent audits
of relevant processes and their
implementation. Furthermore,
several assessments are required
throughout the product lifecycle. For
details, refer to the figure below [1].

CONCEPT FOR CERTIFICATION – THE THREE PILLARS:

PHASES OF TYPE APPROVAL
MAIN POINTS FOR CERTIFICATION

Real-world ■ Overall impression of system

test drive behaviour on public roads
■ Assessment of system's ability to
cope with real-world traffic

■ Validation of audit/assessment
Physical results with real-world behaviour
certification ■ Assessment of system behaviour in
tests fixed set of challenging cases
■ It is possible to reproduce situations

■ Audit of development process and

Audit V&V strategies
■ Assessment of safety concept
(functional safety, safety of use)
and measures taken
Assessment ■ Use of simulation results
Simulation ■ Measuring process maturity

4 Virtual homologation of an ALKS according to UNECE R157 | TÜV SÜD

The line of argumentation for the
positive risk balance in UNECE R157
Annex 4 Appendix 3 is based on the
idea that the incidents and crashes
that we observe today are caused
by limited human ability. In other
words, if the system under test is
shown to perform better than a skilled
and attentive human driver (in all
relevant cases), it can be approved for
the street.
Numerous different scenarios are
required to verify the safety of a
driving function for a positive risk
balance. For a vehicle equipped
with an ALKS, thousands of robust
scenarios contribute to the safety
argument. As a consequence,
virtualisation plays a central role
as this is the key to efficient and
reproducible testing on such a scale.
UNECE R157 proposes a driver
model (see section below – “Human
driver performance model”) where
performance in critical situations can
be compared to the ALKS.
The scenario-based test approach
The scenario-based test approach
essentially breaks down the overall
driving task into scenarios. In a
nutshell, a scenario is a model of the
time sequence of scenes, centred
around a vehicle of interest, with
a start time and an end time. The
different types of scenarios can be
derived from expert knowledge,
crash databases, or real-world
measurements. For practical
purposes, scenarios used as models
for a real driving sequence can be
parameterised.
An essential task in scenario-based
testing is selecting the appropriate
parameter combinations to be tested.
TÜV SÜD | Virtual homologation of an ALKS according to UNECE R157
According to UNECE R157, the key
performance indicators (KPIs) used
to compare the simulation results
from the ALKS under test and the
reference driver model are time to
collision (TTC) and time headway
(THW). Other or additional KPIs might
also be suited to track the outcomes
in terms of severity. This might be
useful in cases where the ALKS under
test is not superior to the driver model
in all scenarios and a compensating
argument therefore needs to be set
up to strike a positive risk balance.
To formalise the approach in a
comparable and reproducible manner,
a scenario-based test methodology
is required to generate reliable
results efficiently in relatively short
periods of time.
To be able to argue the positive
risk balance using simulation data,
trustworthiness arguments for the
simulation are essential. Reliability
arguments for the vehicle and
environment models and for the driver
In a requirement-based approach,
where a verification test directly
corresponds to a requirement, there
are typically one or more preselected
parameter sets that are used to verify
the requirement. In contrast to the
requirement-based approach, you
can vary the scenario parameters in
an explorative manner. This method
helps minimise the risk of unknown
and hazardous situations that are not
covered by the requirement [2].
The scenarios can then be used in
different test environments. This
includes simulation environments
in which the system under test can
be incorporated either as a model
model are described in detail in the
“Reliability of simulation results”
section below. Based on the grid
proposed in UNECE R157 Annex 4
Appendix 3, we have reproduced
the subject vehicle velocity, relative
velocity, lateral velocity, and
longitudinal distance.
In general, the variation of the results
over this parameter grid should be
considered to confirm that no further
sampling points are needed to support
the positive risk balance argument. In
other words, as soon as it is possible
to demonstrate that no further
corner cases occur in a certain area
of the parameter space, no further
simulation cases are needed in this
area. Scenario-based testing is the
key to success in this regard.
(model-in-the-loop) or as a higher
software level, such as a virtual
electronic control unit (V-ECUs),
which is known as software-in-the-
loop (SIL) simulation. In HIL simulation,
the real ECUs are included in the test
environment and are supplied with the
required signals by a real-time PC. In
every in-the-loop test environment,
the vehicle and the environment are
simulated with the required fidelity for
the system under test.
Real test drives on proving grounds or
in the field can also be classified as
scenario-based tests up to a certain
degree. This method produces the
most accurate results and does not
5
require additional reliability measures
since everything is real and the
system under test has the highest
integration level possible. But this test
method is also the most expensive
and most time-consuming. That is
why a typical strategy is to test most
of the cases in SIL, some critical
test cases in HIL, and only the most
critical test cases in real test drives.
The allocation of these methods into
a data-driven development cycle is
illustrated below.

HOMOLOGATION – ARGUMENTATION FOR CERTIFICATION AUTHORITIES

Release criteria – residual risk analysis

End-to-end test and workflow management

(Traceability for automotive certification)

Data collection Machine learning

Compliance / Quality management

and enrichment and prototyping
Process integration

Proto- Data management Integration

typing Raw data > enriched data > simulation scenarios > test data and testing

Series development and

system integration

Operation, execution, maintenance (test execution, …)

Proving system safety

UNECE R157 explicitly allows for
the use of simulation tools to verify
safety concepts. We demonstrate
how to verify system safety for an
ALKS using the dSPACE SIMPHERA
simulation environment with the
industry-proven dSPACE Automotive
Simulation Models (ASM). This
environment can simulate the ALKS
scenarios, including the necessary
vehicle dynamics, in sufficient detail
(see “Reliability of simulation results”
section below). This environment
is also capable of automatically
evaluating the results from thousands
of scenarios and comparing them with
real driving data. We will show that
this environment is well suited for
approval and homologation purposes.

6 Virtual homologation of an ALKS according to UNECE R157 | TÜV SÜD

Identifying hazards
According to UNECE R157 Clause
5.1.1. and Annex 4 Clause 1, the
manufacturer is required to define
risk thresholds to verify freedom from
unreasonable risk.
In general, there are two ways to
consider risks: in absolute values
and relative to a defined reference
system or behaviour. In the first
case, the probabilities of the
presupposed scenario, the probability
of the initiating root cause of the
subsequent harm, and any potential
interdependencies between these
two factors, need to be evaluated.
In the second case, representative
scenarios, including their outcome
for the system under test, are
benchmarked by a reference system
in terms of probability of harm and
severity. In both approaches (absolute
and relative risk evaluation), it is
necessary to consider whether the
target risk threshold is global or
whether there are individual sub-
thresholds for individual scenarios or
groups of scenarios.
This paper demonstrates a relative
risk assessment. We compare the
level of safety achieved by the system
under test with the level of safety
achieved by a skilled and attentive
human driver in a modern vehicle
under good road conditions [2].
Evaluating this comparison by means
of simulation and scenario-based
testing requires two assets: a set of
scenarios and a definition of KPIs
to compare the performance of the
human driver and the ADS.
For an ALKS, UNECE R157 has
limited the scenarios to cut-in, cut-
out, and deceleration. The identified
critical scenarios and hazards form
the basis for considering risk and
verifying safety. UNECE R157
Annex 4 Appendix 3 furthermore
suggests the KPIs time to collision
(TTC) and time headway (THW).
Other KPIs might also consider
severities in case of crash, as well
as compliance with traffic rules
and maintaining safety distances.
For simplicity, we demonstrate
the method by using solely the
suggested scenarios and KPIs from
the UNECE regulation.

Human driver performance model

To prepare the comparison, we will an ALKS must avoid a collision are the “skilled human performance
first create the reference data of determined by simulating the vehicle’s model” shown below to formalise this
the human driver. The conditions behaviour when driven by an attentive behaviour. The driver model is based
under which a vehicle equipped with human driver. UNECE R157 proposes on current research and experiments.

GRAPHICAL REPRESENTATION OF THE "SKILLED HUMAN PERFORMANCE MODEL"

DRIVER BASIC MODEL FOR CUT-IN/CUT-OUT/DECELERATION

Perception Decision Reaction

Decide how to avoid
Foot Time to enable
Release accelerator pedal transfer brake
Accelerator pedal Brake pedal
Pedal angle

Risk Risk Decision Accelerator

perceived evaluation on braking pedal
completely Deceleration
released

Accelerator release time Foot transfer

Perception time Delay in decision time Time
Deceleration occurs

(Source: UN Regulation No. 157: Uniform provisions concerning the approval of vehicles with regards to Automated
Lane Keeping Systems [2021/389]).

TÜV SÜD | Virtual homologation of an ALKS according to UNECE R157 7

The driver behaviour in the the vehicle. For higher speeds, it is reaction times, pedal movements,
given scenarios (cut-in, cut-out, assumed that the braking process is and AEBS engagement as prescribed
deceleration) is split into three parts: supported by an advanced emergency by UNECE R157. The pedal movement
perception, decision, and reaction. braking system (AEBS). and the resulting deceleration for
The risk perception point marks the one simulation run are shown in the
end of the first phase, and the decision This data from UNECE R157 is used figure below.
to brake marks the end of the second to parameterise a driver and AEBS
phase. During the third phase, the model with the dSPACE Automotive These models now let us create
driver releases the acceleration pedal Simulation Models. This results reference data that can be compared
and actuates the brake pedal to stop in models that precisely replicate with the simulation results of the ALKS.

VISUALISATION OF ACCELERATOR AND BRAKE PEDAL ACTUATION,

AND THE RESULTING DECELERATION
Title: Cutin started Title: Accelerator pedal completely released
10 140
18 Time: 27.102000 Time: 28.305000
Title: Cutin perceived
9 Time: 27.291000
16 120
8 Title: Foot on brake pedal
14 Time: 28.426000
Accelerator pedal [%]

100
Deceleration [m/s2]

7
Brake pedal [%]

12
6
80
10
5
8 60
4
6 3 40
4 2
20 Title: Risk perceived Title: Decision on braking
2 1
Time: 27.692000 Time: 27.723000
0 0 0
26.7 26.8 26.9 27.0 27.1 27.2 27.3 27.4 27.5 27.6 27.7 27.8 27.9 28.0 28.1 28.2 28.3 28.4 28.5 28.6 28.7 28.8 28.9 29.0 29.1 29.2

Simulation results to measure the positive risk balance

To demonstrate the risk evaluation, the driver model from the previous ADS leads to a very large number of
a demonstration ALKS system section, we now have all required test cases that are executed to verify
was implemented for this paper. assets for the demonstration. an ALKS. All required simulations were
Together with the cut-in, cut-out, performed, and the demonstration ALKS
and deceleration scenarios; the Executing all parameter combinations system’s behaviour was compared to
relevant UNECE R157 KPIs; and for the human driver model and the the human driver performance model.

CUT-IN SCENARIO AND MAIN VARIABLE PARAMETERS

AS PRESCRIBED BY UNECE R157

dxO
EGO
VeO Vy
dyO
Automated VoO
Drive Challenge vehicle

8 Virtual homologation of an ALKS according to UNECE R157 | TÜV SÜD

 The results from one set of test cases varied over a certain parameter thus the positive risk balance of the
for a cut-in scenario (see figure above) range: for each tuple, a simulation ALKS. The comparison plot (bottom
are shown in the figure below. For is performed and the results of all left) allows the tester to easily identify
each set of test cases, the longitudinal simulation runs are visualised on a parameters where the human driver
velocities of the ego-vehicle (equipped 2-D plot. outperforms the ALKS.
with a human driver or an ALKS)
and the fellow vehicle (participant in The figure below clearly shows the
the scenario) are fixed. The lateral superior performance of the ALKS
velocity and the cut-in distance are compared to the human driver and

VISUALISATION OF THE RESULTS FROM ONE SET OF SIMULATIONS FOR

A CUT-IN SCENARIO (V_EGO = 60KM/H / V_FELLOW = 30KM/H)
Results for the 3.0
3.0
3.0
3.0
3.0
3.0 ALKS 3.0
3.0

Human Driver 2.5 2.5 2.5 2.5

2.5 2.5 2.5 2.5
: no collision : no collision
2.0 2.0 2.0 2.0
:: no collision
collision (front, back) :: no collision
collision (front
2.0 2.0 2.0 2.0 :: collision
collision (front,
(side) back) :: collision
collision (front
(side)
:: collision
interrupt (side)
backward :: collision (side)
interrupt back
1.5 1.5 1.5 1.5 : interrupt backward
VyVy

VyVy

VyVy

VyVy
: interrupt backw
3.0 3.01.5 1.5 1.5 1.5

1.0 1.0 1.0 1.0

2.5 2.51.0 1.0 1.0 1.0

: no collision 0.5 0.5 0.5 : no collision 0.5

2.0 : collision (front, back) 2.00.5 0.5 0.5 : collision (front, back) 0.5
: collision (side) : collision (side)
0 0 0 0
1.5
: interrupt backward 1.5 0 0 10 20 30 40 50 0 600 10 0 0 20 10:30interrupt
2040 backward
3050 4060 50 0 600 10 20 30 40 50 60
Vy

Vy

0 10 20 30
dx0 40 50 600 10 020 10dx0
30 2040 dx0
3050 4060 50 600 10 20 30
dx0 40 50 60
dx0 dx0 dx0 dx0
1.0 1.0
3.0 3.0 3.0 3.0
Human Driver vs. 3.0 3.0 3.0
Crash impact 3.0
0.5 0.5
ALKS (green = ALKS 2.5
2.5
2.5
2.5
2.5
2.5 speed for each 2.5
2.5

20 performs
40 better simulation with
0 0
50 60 0 10 30 50 60 2.00 10 20 30 40 50 2.0
60 2.0 2.0
dx0 2.0 dx0 2.0 2.0 2.0
or as good as the ALKS
1.5 1.5 1.5 1.5
VyVy

VyVy

VyVy

VyVy

3.0
3.0 human driver) 3.01.5
3.0
1.5 1.5 1.5

1.0 1.0 1.0 1.0

2.5 2.51.0 1.0 1.0 1.0
2.5 2.5
0.5 0.5 0.5 0.5
: no collision : no collision
2.0 2.00.5 0.5 0.5 0.5
2.0 : collision (front, back) 2.0 : collision (front, back)
: collision (side) 0 0 0 : collision (side) 0 -32.012 -32.012
1.5 : interrupt backward 1.5 0 0 10 20 30 40 50 0 600 10 0 020 1030 2040 3050 4060 50 0 600
: interrupt
10
-32.012 20 30 40 50 60 -32.012
2040 backward
VyVy

VyVy

1.5 1.5 0 10 20 30
dx0 40 50 600 10 020 1030
dx0 3050 4060
dx0 50 600 10 20 30
dx0 40 50 60
dx0 dx0 dx0 dx0
1.0 1.0
1.0 1.0

0.5 0.5

Possible extensions
0.5 0.5

0 -32.012 0 -32.012
50 600 0 10 20 30 40 50 60 0 0 10 20 30 40 50 60
50 60 0 10 20 30
dx0 40 50 60 0 10 20 30
dx0 40 50 60
dx0 dx0
The outlined example demonstrates which can be calculated during the a collision cannot be avoided.
3.0
how virtual test methods3.0
can be simulation. KPIs can also be refined
2.5 used to build a safety argumentation
2.5 to reflect the severity of potential The parameter combinations for
for an assessment that is aligned impacts. For example, the speed the scenarios in this example were
2.0 2.0
with the UNECE R157 regulation. difference between two vehicles selected by choosing a starting point
1.5
This approach can be extended
1.5
and at the time of impact can be used as for the parameters and iteratively
Vy

Vy

1.0 refined in multiple ways,1.0especially for a measure for the criticality of a increasing the value with each
productive application. crash (as shown in the figure above, simulation. This results in a high
0.5 0.5
bottom right). This can be used to number of executions, especially
50
0
60 0
-32.012
10 UNECE
20 30 R157
40 uses
50 60the two 0KPIs:
0
10 20 30 argue
40 the 60superior
50
-32.012
performance of an for productive applications. Further
dx0 dx0
time to collision and time headway, ALKS compared to the human driver if methods to reduce the test executions

TÜV SÜD | Virtual homologation of an ALKS according to UNECE R157 9

can be used here, for example,
optimisation algorithms or expert
knowledge, to eliminate irrelevant
parameter combinations. This will be
investigated in future research.
Reliability of simulation results
Simulation runs provide results in a
safe, fast, and cost-efficient manner,
as mentioned in the Scenario-Based
Test Approach. Consequently, a major
part of the safety argument should
be based on simulation results (see
“Simulation results to measure the
HIGH LEVEL REQUIREMENTS ON WORKFLOW FOR VIRTUAL VALIDATION
(ACCORDING UNECE R157 ANNEX 4 4.2.)
Scenarios
Validity of models for scenarios
Models
Model validation (correlation
with physical tests)
10
The dSPACE SIMPHERA simulation
environment is used to implement
one test suite for all UNECE R157
test cases. Integration into a
continuous integration and test
positive risk balance” section above).
But how can we trust the results?
UNECE R157 requires that
“Manufacturers shall demonstrate
the scope of the simulation tool, its
validity for the scenario concerned
User
Virtual test platform
Scope of tool chain compliant
with the purpose “virtual
validation of ADS”
Virtual homologation of an ALKS according to UNECE R157 | TÜV SÜD
environment is also possible, to
continuously monitor whether the
acceptance criteria for the risk
evaluation have already been met.
as well as the validation performed for
the simulation tool chain (correlation
of the outcome with physical tests).”
In addition, Schedule 8 of Revision 3 of
the 1958 Agreement must be fulfilled [3].
Safety argumentation
Simulation results
Validation of simulation results
(correlation with physical
tests)
Both scenarios and models are fed
into the simulation tool chain (Figure
on page 10). The scenarios are
defined in such a way that they can
support the safety argument later
(see Scenario-Based Test Approach).
Models describe a selected set of
vehicle characteristics. Models for
sensors, vehicle dynamics, etc. are
used for the virtual validation of
the ADS. The simulation tool chain
contains several features that are
required to fulfil the virtual validation
task, such as data management, test
management, scenario generation,
and simulation execution. The
simulation results available after the
simulation runs support the safety
argument if these results agree with
the real-world behaviour and show
the absence of undesired outcomes.
UNECE R157 addresses several
parts of the simulation workflow
and provides requirements on how
to demonstrate reliability. First, it
is necessary to demonstrate the
scope of the tool chain and whether
it is fit for the purpose of UNECE
R157. The validity of the models for
the considered scenarios must be
demonstrated as well. In addition, the
simulation results must be validated,
which requires correlation with
physical tests. The correlation with
physical tests makes sense at the
vehicle level when generating the
simulation results, but also earlier at
the component level when developing
the model. There are no specific
requirements regarding users in the
regulation.
In summary, all essential parts of
the virtual testing workflow must be
validated and must later be checked
during the homologation process.
An overview of the validation for all
simulation parts is provided in the
following sections.

Scope of the tool chain

The tools used to run the simulation
and support the simulation processes
and methods must also be reviewed
to determine whether they meet the
requirements for virtual validation
in ADS.
There are two established ways to
validate the quality of the tools. One
is to assess the tools according to
ISO 26262, and the other is a “fit for
purpose check” of the tool functions.
Ideally, both methods are performed.
Assessment according to ISO 26262
ensures the integrity of the specific
use cases for the tool, while the “fit
for purpose” check verifies whether
the use cases have been a considered
sufficiently for a homologation task.

SCOPE OF THE VIRTUAL TEST PLATFORM

THE VIRTUAL TEST PLATFORM MUST BE FIT FOR VIRTUAL VALIDATION OF ADS

1 2 1 Software tool qualification

Software tool chain
Correct functioning according to tool specification
Reproductable calculations

2 Fit for purpose check

Configure

Compliance with the scope "virtual validation of ADS"

Operate
Build

Virtual test platform technically capable

Feature adequacy

2 3
Models Virtual test platform
3 Validation
3 Sufficient model accuracy
Input data lies within validated area
I/O
Correlation with physical tests

TÜV SÜD | Virtual homologation of an ALKS according to UNECE R157 11

As a basis for verifying the tool chain,
ISO 26262 refers to the use cases
in the tool specification. Here, the
preparation and execution of the
use cases in the simulation should
be assessed. In general, use cases
can describe user interactions and
the applied subset of features for
the specific simulation task.
Tool chain validity for scenarios
It is necessary to verify not only the
scope of the tool chain (see “Scope
of the tool chain” section above ), but
also its validity for the scenarios used.
As a result, models and simulation
tools can be used only when the
considered scenarios fall within the
How to validate the tool chain (correlation with physical test results)
UNECE R 157 requires simulation tool In addition to the vehicle dynamics, the
chains to be validated. Several other following items must also be validated:
standards are also relevant in terms
of validation requirements, including
■
ISO 11010, ISO 19364, ISO 19365,
■
and ISO/DIS 22140. Most of these
standards are intended for vehicle ■
dynamics models and include specific
scenarios and deterministic evaluation
■
metrics for the validity of the models.
■
The models are validated by correlating
them with physical test results.
VISUALISATION OF THE CUT-OUT SCENARIO WITH CONFIGURATION PARAMETERS
EGO
VeO
Driver
12
Apart from the tool specification, the
main functions for virtual validation
in ADS must be checked during the
fit for purpose check. Namely, the
data management, test management,
scenario generation, and simulation
execution functions must be
reviewed - regardless of whether
one tool covers several functions
validated model and tool chain area.
The general approach to determining
the validated area using interpolation
between several validation points is
described in ISO/DIS 34502.
To verify the validity of a tool chain, a
documented process for determining
Reference driver model
Sensor model
Environment model
ADS function model
Any other models used
dxO dxO_f
Challenge vehicle
VoO
Vy
Virtual homologation of an ALKS according to UNECE R157 | TÜV SÜD
or each function is covered by a
separate tool. The quality of the
functions (performance, consistency,
level of detail, etc.) is sufficient for the
purpose of virtual testing for ADS.
The validation of models, input data,
and output data as an additional
essential part is explained separately
in the next sections.
the validated area is required, in
combination with the verification that
all executed scenarios are inside
the validated area.
The next section explains how
validation is performed.
Further details on model validation
can be found in reference 2) and 3).
Once all simulation models have been
validated, the overall result of the
simulation must be validated as well.
This requires the results of virtual
testing to be correlated with the test
track results. The scenarios used to
validate virtual testing results should
not be known in advance to prevent
the simulation from being adjusted to
match the validation points.
VfO
To demonstrate the general validation
procedures, various scenarios
measured on the test track must
be correlated with their virtual
equivalents from the simulation.
The results from the test track were
recorded by TÜV SÜD as part of the
PEGASUS project.
The figure below shows an example
that was created on the basis of the
dSPACE simulation. Here, a cut-out
scenario was simulated with an ego-
vehicle velocity of 80 km/h (22.2 m/s).

CORRELATION OF TEST TRACK RESULTS WITH SIMULATION RESULTS

FOR THE CUT-OUT SCENARIO WITH VEO=80 KPH

The figure shows that the simulated
car was stopped, whereas the car
on the test track only decelerated
down to 10 km/h (2.8 m/s). There are
also differences in the deceleration
curve. These differences between
the simulation and the real test track
should to be corrected by adjusting
the scenario and vehicle model
parameters to produce reliable
simulation results. This paper focuses
on the general procedure and
therefore does not discuss how to
adjust the parameters.

TÜV SÜD | Virtual homologation of an ALKS according to UNECE R157 13

Conclusion
Under UNECE R157,he the virtual
homologation of an ALKS process
starts by identifying hazards as a
basis for scenario-based testing,
which in turn is used to verify and
validate (V&V) the ALKS. The risk
benefit of a vehicle equipped with
an ALKS is illustrated by comparing
a driver model with the ALKS
performance. Finally, this simulated
comparison builds on a reliable
14
test environment. The theoretical
considerations are supported by
large-scale simulation results and
a reproduction of a proving ground
measurement for the purpose of
model validation.
The authors conclude that the
procedures presented in this paper,
in combination with the test tools used,
are well-suited for demonstrating a
Virtual homologation of an ALKS according to UNECE R157 | TÜV SÜD
positive risk balance in the system
under test. Based on our findings, all
persons involved in this research are
confident that the required processes
can be embedded in established
development environments in an
efficient and straight-forward way
to perform virtual homologation
according to UNECE R157.
GLOSSARY OF ACRONYMS
ADS – Automated driving system PC – Personal Computer
AEBS – Advanced emergency braking system SIL – Software-in-the-loop
ALKS – Automated Lane Keeping System THW – Time headway
ASM – Automotive simulation models TTC – Time to collision
dxO – Longitudinal distance V&V – Verification and validation
dxO_f – longitudinal distance between challenging vehicles V-ECU – Virtual electronic control unit
dyO – Lateral distance VeO – Velocity of EGO
HIL – Hardware-in-the-loop VfO – velocity of upfront vehicle
KPI – Key performance indicator VoO – Velocity of challenging vehicle
ODD – Operational design domain Vy – Lateral velocity

FOOTNOTES
[1] The figure is based on this publication: OICA (2019): OICA views on the certification [3] https://unece.org/fileadmin/DAM/trans/main/wp29/wp29regs/2017/E-ECE-TRANS-
of automated / autonomous vehicle source: https://unece.org/DAM/trans/doc/2019/ 505-Rev.3e.pdf
wp29/WP.29-177-20e.pdf
[2] At this point in time, the incident rates on public roads are dominated by human-driven
vehicles. This means that today, the positive risk balance can be argued solely by
showing that the system under test outperforms a human driver in a vehicle equipped
with AEB under road conditions that allow for a strong deceleration of -0.85 G. In a
future where driving is performed to a significant part by ADAS, it is likely that ADAS
performance will also have to be considered. However, the definition in UNECE R157
Annex 4 Clause 2.16 uses manually driven vehicles by default.

REFERENCES AND RELATED WORK

1) Riedmaier, Stefan, Thomas Ponn, Dieter Ludwig, Bernhard Schick, Frank Diermeyer 4) Gnandt, Christian (2019): Homologation of automated vehicles - The regulatory
(2020): Survey on Scenario-Based Safety Assessment of Automated Vehicles. challenge
In: IEEE Access 5) Koller Benjamin, Matawa Robert (2020): Automated driving requires international
2) Riedmaier, Stefan, Benedikt Danquah, Bernhard Schick, Frank Diermeyer (2020): regulations - A look at the current state of developments
Unified Framework and Survey for Model Verification, Validation and Uncertainty
Quantification.
3) Riedmaier, Stefan, Benedikt Danquah, Bernhard Schick, Frank Diermeyer (2020):
Non-deterministic Model Validation Methodology for Simulation-based Safety
Assessment of Automated Vehicles

COPYRIGHT NOTICE
The information contained in this document represents the current view of TÜV SÜD on the issues discussed as of the date of publication. Because TÜV SÜD must respond to changing
market conditions, it should not be interpreted to be a commitment on the part of TÜV SÜD, and TÜV SÜD cannot guarantee the accuracy of any information presented after the date of
publication. This White Paper is for informational purposes only. TÜV SÜD makes no warranties, express, implied or statutory, as to the information in this document. Complying with all
applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval
system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of TÜV
SÜD. TÜV SÜD may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided
in any written license agreement from TÜV SÜD, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.
ANY REPRODUCTION, ADAPTATION OR TRANSLATION OF THIS DOCUMENT WITHOUT PRIOR WRITTEN PERMISSION IS PROHIBITED, EXCEPT AS ALLOWED UNDER THE COPYRIGHT
LAWS. © TÜV SÜD Group – 2022 – All rights reserved - TÜV SÜD is a registered trademark of TÜV SÜD Group.

DISCLAIMER
All reasonable measures have been taken to ensure the quality, reliability, and accuracy of the information in the content. However, TÜV SÜD is not responsible for the third-party content
contained in this newsletter. TÜV SÜD makes no warranties or representations, expressed or implied, as to the accuracy or completeness of information contained in this newsletter.
This newsletter is intended to provide general information on a particular subject or subjects and is not an exhaustive treatment of such subject(s). Accordingly, the information in
this newsletter is not intended to constitute consulting or professional advice or services. If you are seeking advice on any matters relating to information in this newsletter, you
should – where appropriate – contact us directly with your specific query or seek advice from qualified professional people. TÜV SÜD ensures that the provision of its services meets
independence, impartiality and objective requirements. The information contained in this newsletter may not be copied, quoted, or referred to in any other publication or materials without
the prior written consent of TÜV SÜD. All rights reserved © 2022TÜV SÜD.

TÜV SÜD | Virtual homologation of an ALKS according to UNECE R157 15

Find out more about TÜV SÜD’s
services for autonomous driving
www.tuvsud.com/automated-lane-keeping-system
automotive@tuvsud.com

Add value. Inspire trust.

TÜV SÜD is a trusted partner of choice for safety, security and sustainability solutions. It specialises in testing,
certification, auditing and advisory services. Since 1866, the company has remained committed to its purpose of
enabling progress by protecting people, the environment and assets from technology-related risks. Through more
than 25,000 employees across over 1,000 locations, it adds value to customers and partners by enabling market
access and managing risks. By anticipating technological developments and facilitating.

Developing and Validating Innovations Faster and More Efficient

Simulation is the key to efficient development and reliable validation of complex systems.
dSPACE offers its customers solutions and know-how to drive innovation with software- and hardware-based
simulations. As a market and innovation leader, we consistently support the dynamic transformation of the
automotive industry with solutions for autonomous driving, electromobility, and digitalization. Automobile
manufacturers and their suppliers in particular use our comprehensive, end-to-end solution range to test the
software and hardware components of their new vehicles long before a new model is allowed on the road.
dSPACE is not only a sought-after development partner in vehicle development. Engineers also rely on our
dSPACE knowledge and experience in aerospace and industrial automation. For decades, dSPACE solutions have
accelerated the development of innovative vehicle technology and made its validation more reliable. Our portfolio
ranges from continous tool chains and engineering and consulting services to training and support.
2022 © TÜV SÜD AG | MKG/AM/44.0/en/DE

TÜV SÜD AG dSPACE GmbH

Westendstr. 199 Rathenaustr. 26
80686 Munich, Germany 33102 Paderborn, Germany
+49 89 5791 0 +49 5251 1638 526
www.tuvsud.com www.dspace.com