AVL List GmbH (Headquarters)

Federico Ferrero
Valerio Castelli
Dr. Bernhard Behr

ADAS & Autonomous Driving

Challenges and AVL Solutions for Development and Testing
October 2019

Public
 Market Drivers & Challenges

▪ Accident free driving

▪ Driver relief and comfort functions
▪ Connectivity services
▪ Improved fuel/energy efficiency
▪ Reduced operating cost / new mobility
concepts

Challenges
➢ Robust perception, right decision making
and failure free acting
require new, different to conventional development approaches
& huge testing efforts

Public | | 02 Oktober 2019 | 2

 Areas of ADAS/AD Engineering Services

Tailored Calibration
System Design Testing
Controls & SW
Development Validation
use & test cases, derivative integration,
hazard & risk analysis, AD features
performance
architecture, sensors, development from
optimization,
system safety, feasibility, concept to
testing & validation
benchmark, series,
from lab to proving
target setting market specific
ground and road
modifications

For new levels of vehicle comfort, safety & efficiency

Key Facts AVL List GmbH: 1.75 bn€ TO 2018, 10.400 employees, 45 Affiliates world wide, Founded 1948, Chairman & CEO: Prof. Dr. h.c. Helmut List
Provider of Engineering Services, Instrumentation & Test Systems including Simulation Technologies

Public | | 02 Oktober 2019 | 3

 SOP Application References L2+
European Premium OEMs

Source: Audi.de

ACC , APA, PEA, Speed Limiter
Development of longitudinal
calibration and validation of
various function packages in
all Audi powertrains
LDW, LKA, ISL, BSA, DDM, NextGen
# #
Contribution to development of a AVL is automotive industry
number of L0-L2 ADAS features partner for the development and
for Jaguar Land Rover’s recent + evaluation of objective ADAS
current models including next criteria based on
generation features AVL-DRIVETM ADAS

AVL has agreement from the customers to mention this projects for marketing purpose

AVL is a trusted partner in ADAS SOP projects and has long term relationships
w with OEMs as engineer service provider
Public | | 02 Oktober 2019 | 4
 SOP Reference System Design
European HD Truck OEM

Requirements and Safety

European Truck OEM:
• Lane Centering: ~60 use & 90 test cases
• Traffic Jam Assist: ~80 use & 120 test cases

Work Content
• Use Case Development
• Hazard and Risk Assessment (HARA)
based on ISO 26262
• Validation Procedure and Worst Case Definition
• Validation Test SOP 2019

AVL has best flexibility to design & specify ADAS according OEM preferences
due to multiple supplier openness on system architectures & components

Public | | 02 Oktober 2019 | 5

 System Design Topics
System & requirements engineering, use-& test cases

ADAS/AD Connectivity Functional Design Perception Trajectory & Control &

system design ▪ V2X safety Validation ▪ Sensor input decision actuation
▪ E/E & Ctrl. communication ▪ ISO 26262 Plan ▪ Sensor fusion making ▪ Vehicle dynamics
architectures ▪ Back end systems functional & ▪ Testing and ▪ Perception ▪ Maneuver models
▪ Use cases / test ▪ Data security operational safety validation ▪ Occupancy grid planning ▪ Long./Lat. vehicle
cases requirements strategies for lab ▪ Trajectory motion controls
▪ Hazard & Risk ▪ SOTIF – safety of & road calculation ▪ Actuators
Analysis the intended ▪ Active safety, ▪ Embedded quality interface
▪ Worst Case functionality function & evaluation
scenario performance KPI CONTROLLER
▪ Target Setting

AVL has best flexibility to design & specify ADAS according OEM preferences
due to multiple supplier openness on system architectures & components

Public | | 02 Oktober 2019 | 6

 AVL SW & Functions Development
AVL in-house generic ADAS controls
Environment Perception Planning and Decision Motion Control
cameras

SR/LR Decision ACC/LKA/

radar Perception Making LCA
Lat.
Lidar Sensor Fusion Up to Level 2+ Control Steering

Ultra- Localization Maneuver Emergency HMI

sonic
Planning Status
Occupancy
IMU Decision Trajectory
Grid Long. accelerater &
Making Planning Control braking
V2X Environment
Model Vehicle
Status
Map Level 3,4

3 Types of Projects
▪ Feasibility demonstrations
unmanned driving (L4)
productivity oriented

▪ Tailored features (L3) as

differentiation for an OEM

▪ SW modifications (L0-L2)
because of country specifics

Customer Value: Development according “White Box” approach

provide maximum in-house flexibility and market differentiation potential
Public | | 02 Oktober 2019 | 7
 Predictive Function Examples
smart & intelligent energy management

ECO
ROUTING
PREDICTIVE ADAPTIVE
CRUISE CONTROL

CONNECTED

PREDICTIVE HEV TRAFFIC LIGHT

ENERGYMANAGMENT ASSISTENT

System Tailored Calibration

Design Controls & SW Testing
Development Validation

PREDICTIVE THERMAL
MANAGEMENT PREDICTIVE SHIFTING Fuel COASTING ASSISTENT
Efficiency @High Level FILTER
AD: Platooning
REGENERATION

AVL fully supports development and demonstration of new features improving

energy efficiency, CO2 & driving attributes

Public | | 02 Oktober 2019 | 8

 ADAS Testing Capabilities
at AVL HQ
Partner of
AVL offer: Test Execution

Test Region Styria Test tracks RW Fleet testing

500km public roads multiple proving grounds online OTA assessment

Autonomous Driving & Vehicle Development Center

Location: Graz, Austria

Driver in the loop Vehicle in the loop Virtual Environment

safe & reproducible lab AVL DrivingCube extensive simulation & cloud

Comprehensive test environment for basic and

advanced ADAS + AD features in lab, on XiL and road
Public | | 02 Oktober 2019 | 9
 ADAS Performance Calibration & Optimization
Special Focus: Human Centric Solutions

AVL-DRIVE ADAS
How want drivers to be driven by automated
23
Scale Safety & Comfort
9-10 very convenient
features? 8-9 convenient
7-8
different
acceptable
6-7 almost acceptable
maneuvers
5-6 disturbing

applicable
4-5 very disturbing on

L1 to L3
3-4 almost unacceptable
2-3 unacceptable
1-2 comfort
fail

features

AVL USP: Objective measurement technology for perceived safety & comfort feeling
Customer Benefit: Market success via end consumers agreement on performance

Public | | 02 Oktober 2019 | 10

 Target Setting Function Performance
based on objective ‘AVL-DRIVE Autonomous’ assessment technology

Scale Safety & Comfort

1. AVL USP: In-house developed
9-10 very convenient technology for objective assessment of
8-9 convenient
perceived safety & comfort
7-8 acceptable
6-7 almost acceptable → criteria catalogue (>30)
5-6 disturbing
4-5 very disturbing
3-4 almost unacceptable
2-3 unacceptable
1-2 fail 10
ADAS PERFORMAN CE
AVL Benchmark Database 2012-2018
9

TOTAL RATING
7 base for
targets
setting
6
2. AVL Benchmark Program
competitor comparison 5

→ detailed target diagrams for • ACC

4
calibration ▪ LKA
3
2012 2013 2014 2015 2016 2017 2018 2019

Public | | 02 Oktober 2019 | 11

 ADAS/AD Real World Testing
AVL Fleet Monitoring Technology

Immense amount of testing &

Why estimation of scenario coverage
Demand: time saving approaches & scenario
collection with assessment

AVL Automated logging and evaluation

Approach of geotagged fleet data

x
Client
Com Modul MQTT Client
Live UI

AVL-DRIVE
ZMQ
Client
Com Modul
Kafka Client
Fleet DB
Service Execution of real world testing including
- Live
- Replay
-…
ComModul
Com
Com
Modul
Modul … Client
-
-
-
File
DB
MQTT Client
Offering comprehensive fleet test analysis
- Kafka Client
- ZMQ Client
Queue Manipulator Output Client - UI
Input Client
- …
- Ring FiFo - Filter - Settings
- Settings - Encryption - Monitoring
- Setup
- Monitoring - Manipulation

Fleet HW concept

Public | | 02 Oktober 2019 | 13

 Highlights ADAS/AD Real World Testing
Automated logging and evaluation Country specific identification
of fleet data of significant places
Maneuver Distribution
Statistics

Road segment coverages

Highways
Local roads
Main roads
Minor roads
Geo-tagged view of areas Motorways
with POI markers Pedestrian roads
Regional roads
Walkways
Tunnels
Bridges

Rating below thresholds

Statistics: fleet utilization, road
types, mileage, events distribution…
(target fulfillment)
Identification of geo-areas
of interest (POI)
Over the air transmission
of result parameters
Test routes optimization

Public | | 02 Oktober 2019 | 14

 Comprehensive Proving Ground Testing

AVL function & performance testing

on proving ground in smart city area (HU)

Comprehensive test environment for real world & proving ground testing
enables complete testing + validation packages
Public | | 02 Oktober 2019 | 15
 Safe & Reproducible Lab Testing
with Driver Simulator on Hexapod

Examples of parameter variations

• speed of ego vehicle
• speed of motor biker
• distance to side lane / mid lane
• payload of vehicle
• …

Location: Graz

ADAS Testing and Validation on Driver Simulator

Public | | 02 Oktober 2019 | 16
 How are Automated Driving Functions Tested TODAY

OEMs mainly rely on office simulation for development and in-vehicle testing for validation

Public | | 02 Oktober 2019 | 17

 A Seamless and Open Tool Chain with New
Approaches is Necessary

Simulation Virtual TESTING Proving Ground Real World

Required Elements:
Scenario, Maneuver, Test Case Abstraction incl. KPIs
Environment Abstraction
Vehicle Abstraction
Same Results

The most efficient validation will be done by those who will use the best
combination…
Public | | 02 Oktober 2019 | 18
 Insights: Approach for The Best Combination…
Simulation: VirtualVirtual
TESTING: Proving
Proving Real
Ground and
Simulation Testing
Ground
Real World World

Scale Testing of Variants Integrate, Analyze and

Improve
Finalize and
(Vehicle Configurations and Scenarios)

reproduce Confirm

analyze

Optimize/
Improve
+ specific test cases which + specific test cases which
cannot be done in simulation cannot be done in simulation
and/or virtual testing
Public | | 02 Oktober 2019 | 19
 Scenarios – Core of ADAS
Development & Validation
Examples:

Most scenario cases are uninteresting…

… and some can become very interesting

Public | | 02 Oktober 2019 | 20

 AVL Scenario-Based ADAS/AD V&V Tool Chain

Scenario Preparation Model Preparation Test Systems KPI Preparation

(Sensor) Model (Sensor) Model
Scenarios to test requirements Parametrization Editor • Sensor testbeds
(DVPs)
Generic • Powertrain
testbeds
(Sensor) • Vehicle
Model dynamics
testbeds

Synthetic random scenarios

Specific
(Sensor) Model Test Case Preparation
Integration of Data
reference simulation
Test Sequence Generator environment
Management
Scenarios
Scenarios from road data Online / Offline Evaluation &
Postprocessing

Scenarios
Models
KPIs

Signals, Object lists, Streams (Video, 3D, ..)

Model.CONNECT™ Testbed.CONNECT™

Model.CONNECT™

Data Management

Public Road HIL Platform (Driving

Private Test-Track VIL (Driv.CubeTM) MIL/SIL Cloud MIL/SIL
Testing using Simulator)
reference DAQ

Test Execution

Public | | 02 Oktober 2019 | 21

 Where we are working on …
Dynamic Ground Truth System

Lidar reflections from road objects (e.g. vehicles

Dynamic Ground Truth roof box:

Extracted (Tagged)
Scenarios from
Road drives:

Public | | 02 Oktober 2019 | 22

 Where we are working on …
Test Case Generation Methods
Scenario Preparation
Scenarios to test requirements
AVL Test Case Generator
(DVPs)

Synthetic random scenarios

Scenario Scenario Query scenario
generation information
Database

Test sequence
Scenarios from road data

Test the relevant /

critical scenarios!

Public | | 02 Oktober 2019 | 23

 Where we are working on …
Target: Extract the Relevant Cases

Cut-In → Cut-Out

288.000.000 km to test
>99% nothing happens

Identify relevant cases and

transfer to testbed
Public | | 02 Oktober 2019 | 24
 Where we are working on …
Vehicle in the Loop Testing

Steering Module:

Radar Stimulator:

Public | | 02 Oktober 2019 | 25

 Where we are working on …
Euro-NCAP Evaluation Package

Euro-NCAP Data Processing Package:

Public | | 02 Oktober 2019 | 26

 Where we are working on …
ADAS Parameter Mgt with CRETA

Simulation in LAB

Software Team

Datasets
Neural Network optimisation
Project Coordinator

Make sure that the correct parameters

end up in the correct vehicle and
controller! Parameterisation in the car
Public | | 02 Oktober 2019 | 27
 Some Partnerships

AVL-TNO Cooperation: AVL-NORCOM Cooperation:

Integration of TNO scenario database for Test Integration of NorCom BigData Platform into
Planning and MIL/SIL Validation: AVL Big Data Analytics solution:

AVL Stähle Cooperation: AVL-TÜV-NVIDIA Cooperation:

Development of a steering actuator for ADAS Virtual homologation of piloted highway driving
testing on Chassis dyno testbeds: using NVIDIA DRIVE SIM and NVIDIA DRIVE PX:

Further joint activities:

…
Public | | 02 Oktober 2019 | 28
 Summary ADAS/AD Engineering Offerings

System Design Control & SW Development

• Proof-of-concept new features
• Use & test cases definitions
• Customer specific ADAS & AD features
• HARA – hazard & risk assessment
development from concept to series
with functional safety concept
• Predictive & connected energy
• Controller architecture, sensor layout
management features improving fuel
• Benchmarking & targets setting
consumption

USP, Strengths Calibration, Testing& Validation

• Proven series production experience • Features integration e.g. derivatives
and references • Full capability testing & validation
• In-house SW & controls development real world, proving ground and virtual
• ADAS performance technology for • Calibration of ADAS features with
human perception optimization perceived safety & comfort optimization

Engineering solutions for series (L0-L3) & adv. development (L3+), applicable at PC & CV
PC: passenger cars / CV: commercial vehicles

Public | | 02 Oktober 2019 | 29