Verification, Trustworthiness and

Accountability of Human-Driven
Autonomous Systems
Abstract:
Despite the fact that autonomous systems’ science and control theory have
almost 50 years of history, the community is facing major challenges to
ensure the safety of fully autonomous consumer systems. It mostly concerns
the verification and high fidelity operation of safety-critical systems, may that
be a self-driving car, a homecare robot or a surgical manipulator. The
community still struggles to establish objective criteria for trustworthiness of AI
driven / machine learning based control systems. On one hand, we celebrate
the rise of cognitive capabilities in robotic systems, leading independent
decision making; on the other hand, decisions made in complex
environments, based on multi-sensory data will surly lead to some wrong
conclusions and hazardous outcome, jeopardizing the public trust in entire
application domains. This ambiguity led to the currently ruling safety principle
to offer the possibility for a human-driven override, translating to Level of
Autonomy 3 and 4 with autonomous vehicles.The aim of the development
community is to establish processes and metrics to ensure the reliability of the
takeover process, when the human driver or operator takes back the partial or
full control from the autonomous system. We have been building complex
simulators and data collection systems to benchmark human decision making
against the computer. Situation Awareness (SA) has been identified as a key,
as it defines the level of cognitive understanding and capability of a human
operator in a given environment. Assessing, maintaining and regaining
efficiently SA are core elements of the relevant research projects, reviewed
and compared in this talk. Based on the research at the Antal Bejczy Center
for Intelligent Robotics at Óbuda University, we created an assessment
method for critical handover performance, to quantitatively define the required
level and components of SA with respect to the autonomous functionalities
present. To improve system safety, driver assista...

(Show More)
Published in: 2021 IEEE International Conference on Autonomous Systems
(ICAS)
Date of Conference: 11-13 August 2021
Date Added to IEEE Xplore: 06 October 2021
ISBN Information:
INSPEC Accession Number: 21201489
DOI: 10.1109/ICAS49788.2021.9551163
Publisher: IEEE
Conference Location: Montreal, QC, Canada