On the use of locality sensitive hashing for Audio Following

Abstract

Audio Following (AF) is the process of mapping a musician’s performance, usually in real-time, to a reference performance that is used as a reference. Such base performance is considered a “correct performance” and thus, the live performace must be aligned to it. The objective of AF is to track the musician’s position throughout the performance. We present a novel approach to AF that uses a locality sensitive hashing (LSH) based index to perform such task. First, we obtain the Audio Fingerprint (AFP) of the base performance. Then, the obtained AFP is indexed using LSH. Such performance’s AFP is used as a reference to align any other performance of the same music. Next, we obtain half-a-second sub-AFP’s of the performance being followed and their corresponding positions in the reference AFP are searched for by querying the index. The system was tested on a set of 22 pianists playing music by Chopin with very good results when comparing the obtained alignment with the ideal alignment.