ChordAR: An Educational AR Game Design for Children's Music Theory Learning

Abstract

Augmented reality (AR) technology, with its unique immersive interactive experience of virtual reality integration, can show children abstract concepts and theories in an intuitive way. In this paper, we improve the high-precision position and low delay interaction of AR recognition area through edge calculation, so as to enhance the immersion of children's chord knowledge learning. Meanwhile, multisource fusion perception of audio and touch is used, and the AI algorithm model is used to monitor physical events (cloud AI training and edge execution) and effectively realize intelligent services such as chord combination, melody perception, and control. Chord knowledge in music theory is presented to children through multichannels (kinesthetic, visual, auditory, etc.) situationally, to improve children's learning efficiency and experience. We invited 12 children to participate in user experiments to test usability of ChordAR. The test results showed that children can master the method of playing the game through simple learning, ultimately correctly imitating C, G, and E chords, and experience full immersion in the game. Finally, we discuss the positive effects of ChordAR on children's learning and creativity and make suggestions for future AR games.