Conductor tutoring using the microsoft kinect

Abstract

In this paper we present a system that uses the Microsoft Kinect to provide beginner conducting students real time feedback about their performance. Using upper body joint coordinates we detect common mistakes such as swaying, rocking, excessive hinge movement, and mirroring. We compute instant velocities to determine tempo and classify articulation as legato or staccato. Our experiments show that the system performs perfectly when detecting erroneous movements, correctly classifies articulation type most of the time, and can correctly determine tempo by counting the number of beats per minute. The system was well received by conducting students and their instructor, as it allows them to practice by themselves, without an orchestra.