Human–Robot Musical Interaction

Abstract

Music is a form of art, and art is communication, the expression of self: the interaction between artists and their surrounding is a critical part of the artistic process. Artists perform in what we can call the performance space, an environment in which there is usually an audience and often other performers. To give engaging and powerful performances, artists need to not only master artistic techniques, but also effectively interact with the audience and fellow performers at various levels of communication. They need to convey an emotional message to the audience, and technical signals to synchronize with their fellow performers. At the same time, they need to lookout for and acknowledge technical signals from the other performers and sense the audience emotional state as a feedback to adjust their performance. In the scope of musical performance, most of these interactions are in the form of silent ancillary gestures to avoid auditory interference: both technical signaling and emotional gestures must blend in seamlessly and harmonically within the performance. Moreover, due to physical motion constraints related to instrument playing, these gestures are limited to posture changes, gazing, and facial expressions. In this chapter, we discuss the communication challenges that a robot faces during a live performance and some of the technical design choices that enable the robot to overcome such challenges and perform at the same level of human performers.