Design Approach for Investigating Multimodal Communication in Dismounted Soldier-Robot Interaction

Abstract

For several decades there has been continuous growth in the field of robotics, with recent trends driving towards a vision of humans collaborating in a cohesive unit with automated counterparts. Enabling true mixed-initiative teaming between a human and robot will require communication capabilities and cognition comparable to human teammates. Multimodal communication is a framework in which interfaces can be created supporting the flexible selection of different modalities (e.g. speech, gestures) for these transactions. A major challenge for human factors researchers investigating human robot collaboration with multimodal interfaces is the current limitations of robots. Therefore, simulations and wizard-of-oz type experiments are heavily employed to measure performance, workload, and other factors in future mixed-initiative scenarios. Although these techniques facilitate experimentation, it can be difficult to transition findings to working prototypes of today’s robots. For example, a researcher finds an effective way to convey a robots decision making rationale in a simulation-based study, but has no working robot that can drive the content in reality. Furthermore, the literature regarding multimodal communication with robots applied to the military domain is limited. For example, evaluation of different modalities as part of an interrupting task has been explored in driving scenarios, but not between robot(s) and soldiers. In many cases there is conflict in findings across domains. To address this challenge, this paper describes how a multimodal interface for a real robotic platform developed under the U.S. Army Research Labs Robotics Collaborative Technology Alliance (RCTA) was extended to support standalone simulation of interactions and integration with simulated virtual environments. This functionality enables researchers to assess new interaction techniques using the same software that will interact with a real platform to facilitate transition of their research. Furthermore, experiment design approaches including theory-based tasks in a military relevant mission (Cordon and Search) are discussed.