Enriched Multimodal Representations of Music Performances: Online Access and Visualization

Abstract

The authors provide a first-person outlook on the technical challenges involved in the recording, analysis, archiving, and cloud-based interchange of multimodal string quartet performance data as part of a collaborative research project on ensemble music making. To facilitate the sharing of their own collection of multimodal recordings and extracted descriptors and annotations, they developed a hosting platform through which multimodal data (audio, video, motion capture, and derived signals) can be stored, visualized, annotated, and selectively retrieved via a web interface and a dedicated API. This article offers a twofold contribution: the authors open their collection of enriched multimodal recordings, the Quartet dataset, to the community, and they introduce and enable access to their multimodal data exchange platform and web application, the Repovizz system. This article is part of a special issue on multimedia technologies for enriched music.