Perception simplex: Verifiable collision avoidance in autonomous vehicles amidst obstacle detection faults

Abstract

Advances in deep learning have revolutionized cyber-physical applications, including the development of autonomous vehicles. However, real-world collisions involving autonomous control of vehicles have raised significant safety concerns regarding the use of deep neural networks (DNNs) in safety-critical tasks, particularly perception. The inherent unverifiability of DNNs poses a key challenge in ensuring their safe and reliable operation. In this work, we propose perception simplex ((Formula presented.)), a fault-tolerant application architecture designed for obstacle detection and collision avoidance. We analyse an existing LiDAR-based classical obstacle detection algorithm to establish strict bounds on its capabilities and limitations. Such analysis and verification have not been possible for deep learning-based perception systems yet. By employing verifiable obstacle detection algorithms, (Formula presented.) identifies obstacle existence detection faults in the output of unverifiable DNN-based object detectors. When faults with potential collision risks are detected, appropriate corrective actions are initiated. Through extensive analysis and software-in-the-loop simulations, we demonstrate that (Formula presented.) provides deterministic fault tolerance against obstacle existence detection faults, establishing a robust safety guarantee.