Trustworthy Human-Centered Automation Through Explainable AI and High-Fidelity Simulation

Abstract

As we become more competent developers of artificially intelligent systems, the level of deployment and associated implicit trust in these systems will increase in kind. While this is an attractive concept, with an already-demonstrated capability to positively disrupt industries around the world, it remains a dangerous premise that demands attention and intentional resource allocation to ensure that these systems’ behaviors match our expectations. Until we can develop explainable AI techniques or high-fidelity simulators to enable us to examine the models’ underlying logic for the situations we intend to utilize them in, it will be irresponsible to place our trust in their ability to act on our behalf. In this work we describe and provide guidelines for ongoing efforts in using novel explainable AI techniques and high-fidelity simulation to help establish shared expectations between autonomous systems and the humans who interact with them, discussing collaborative robotics and cybersecurity domains.