Natural, multi-modal interfaces for unmanned systems

Abstract

The prospect of using unmanned systems in dull, dirty, or dangerous jobs to save work or even lives has drawn increasing attention in the DoD. Unmanned ground vehicles are used in theatre to get views into buildings or to destroy suspected IEDs. Unmanned air vehicles are used to get views over the next hill or to deliver munitions on targets thousands of miles away. While the automation and sensing capabilities have increased, interaction with these systems is still fairly rudimentary. Deployed systems typically use tele-operation or waypoint control, in some cases requiring operators to carry heavy operator control units. These approaches place a high burden on the operator in terms of the added weight and the constant attention required to operate the systems. In fact, many of these systems require more than a single operator to control a single platform, which increases the cost and logistics of using them. This paper describes natural, multi-modal interfaces as an alternative to the current state of the practice in controlling unmanned systems, with the goal of leveraging how people already communicate with each other in order to reduce the physical and cognitive burdens of interacting with unmanned systems. We describe approaches and challenges in designing, building, and evaluating natural interfaces. We present our Smart Interaction Device (SID) as an example natural interface for interaction with unmanned systems, and highlight some use cases we have applied it to in the air and ground domains.