Closing the loop: Towards tightly synchronized robot gesture and speech

Abstract

To engage in natural interactions with humans, social robots should produce speech-accompanying non-verbal behaviors such as hand and arm gestures. Given the special constraints imposed by the physical properties of a humanoid robot, successful multimodal synchronization is difficult to achieve. Introducing the first closed-loop approach to speech and gesture generation for humanoid robots, we propose a multimodal scheduler for improved synchronization based on two novel features, namely an experimentally fitted forward model and a feedback-based adaptation mechanism. Technical results obtained with the implemented scheduler demonstrate the feasibility of our approach; empirical results from an evaluation study highlight the implications of the present work. © Springer International Publishing 2013.