A User Interface for Sense-making of the Reasoning Process while Interacting with Robots

Abstract

This paper describes an interface that enables experts to communicate with a virtual robot in a simulated environment via natural language, and to visualize the robot's knowledge representation for them for inspection and correction. The interface visually links the robot's internal reasoning processes and knowledge with the simulated instances in the form of a 3D isometric visualization as well as the robot's first-person view. After 3 weeks of using the system by the roboticists in their daily development, some feedback was collected that provided insights for designing such systems in the future.