The effect of metaphoric gestures on schematic understanding of instruction performed by a pedagogical conversational agent

Abstract

In this paper, we examine the impact of metaphoric gestures performed by Pedagogical Conversational Agent (PCA) on learners’ memorization of technical terms, understanding of relationships between abstract concepts, learning experience, and perception of the PCA. The study employed a one-factor three-level between-participants design where we manipulated gesture factor (speech-gesture match vs. speechgesture mismatch vs. no-gesture). The data of 97 students were acquired in on-line learning environment. As the results, while there was no effect found on memorization of technical terms, we found that students showed accurate schematic understanding of the relationship between abstract concepts when the PCA used metaphoric gestures matched to speech content than when used gestures mismatched, and no gesture. Contrary to the result, we also found that students judged the PCA useful, helpful, and felt the PCA looked like a teacher when performed mismatched gestures to speech content than when performed matched gesture.