Using musical beats to segment videos of Bharatanatyam Adavus

Abstract

We present an algorithm for audio-guided segmentation of the Kinect videos of Adavus in Bharatanatyam dance. Adavus are basic choreographic units of a dance sequence in Bharatanatyam. An Adavu is accompanied by percussion instruments (Tatta Palahai (wooden stick)—Tatta Kozhi (wooden block), Mridangam, Nagaswaram, Flute, Violin, or Veena) and vocal music. It is a combination of events that are either postures or small movements synchronized with rhythmic pattern of beats or Taals.We segment the videos of Adavus according to the percussion beats to determine the events for recognition of Adavus later. We use Blind Source Separation to isolate the instrumental sound from the vocal. Beats are tracked by onset detection to determine the instants in the video where the dancer assumes key postures. We also build a visualizer for test. From over 13000 input frames of 15 Adavus, 74 of the 131 key frames actually present get detected. Every detected key frame is correct. Hence, the system has 100% precision, but only about 56% recall.