Construction of a Virtual Ballroom Dance Instructor

Abstract

The use of virtual conversational agents is awaited in the tutoring of physical skills such as sports or dances. This paper describes an ongoing project aiming to realize a virtual instructor for ballroom dance. First, a human-human experiment is conducted to collect the interaction corpus between a professional instructor and six learners. The verbal and non-verbal behaviors of the instructor is analyzed and served as the base of a state transition model for ballroom dance tutoring. In order to achieve natural and highly interactive instruction during the multi-modal interaction between the virtual instructor and the learner, it is necessary to divide the learner's motion into small but meaningful segments in real-time. In the case of ballroom dance, the smallest meaningful unit of dance steps is called count which should be synchronized with the beats of accompanied music. A method of automatic extraction from the learner's dance practice motion into count segments is proposed in this paper. This method is based on trajectory similarity comparison between the motion of the learner and the data collected from a professional dance instructor using Angular Matrix for Shape Similarity (AMSS). Another algorithm for identifying the worst performed portion and the corresponding improvement of the learner's motion is also proposed. Finally, a comparison experiment between the prototype system and a non-interactive self-training system was conducted and reported. keywords: virtual agent applications and empirical studies, multimodal interaction, verbal and non-verbal expressiveness, agents in games and virtual environments. © 2013, The Japanese Society for Artificial Intelligence. All rights reserved.