Towards classification of engagement in human interaction with talking robots

Abstract

In this paper we describe ongoing work to develop an engagement classifier for human-computer interaction systems. We have successfully classified group and individual engagement in a corpus of a conversation among four people called TableTalk, by using a classifier trained with the Support Vector Machine method and audio-visual features. The goal in this paper is to extend that work for the classification of engagement in videos of interaction between an human and a talking robot. For that purpose we are using a corpus of dialogues between participants and a Lego robot named Herme, which was collected during an exhibition. We describe the techniques to improve the engagement detection by taking into account the differences between the characteristics of the videos between the two datasets. Currently we are also conducting an experiment to manually annotate the Herme videos with engagement labels. These annotations will be used for evaluation and further improvements to engagement detection.