Feasibility of wearable fitness trackers for adapting multimodal communication

Abstract

In addition to efforts to increase the intelligence and perception capabilities of robots to enable collaboration with human counterparts, there is also a focus towards improving interaction mechanics. Multimodal communication is one such tool under investigation due to its dynamic ability to select explicit and implicit communication modalities with the aim of facilitating robust exchanges of information. Although there is extensive research in the domain of explicit communication using auditory, visual, and tactile interfaces, investigations into systems that incorporate implicit methods, or actually adapt and select appropriate modalities for reporting data from a robot to human is limited. Furthermore, a missing piece is identifying how and when to trigger these changes. A novel strategy to accomplish adaptation is through identification of teammate’s physiological state. From the literature, one can find examples of researchers using high fidelity systems to measure physiological response and predict user workload, but many of these technologies are prohibitively expensive or not suitable for use in many domains of interest for human robot interaction such as dismounted infantry operations. Recent advancements in wearable consumer technologies, specifically fitness trackers supporting integration with third party software, are making it possible for incorporation of low cost systems in a variety of novel applications. A logical extension of these applications being physiological state measurement to drive adaptive automation in the form of multimodal interfaces. This paper describes the results of a study to assess the feasibility of using data from a wearable fitness tracker in an adaptive multimodal interface for squad-level human-robot interaction.