Encouraging Improvisation in Piano LearningUsing Adaptive Visualisations and Spatiotemporal Models

Abstract

The process of learning the piano for novices is usually difficult and time-consuming. Several approaches in augmented reality such as piano-roll visualizations have been explored but have not garnered enough success and adoption. These piano roll prototypes have introduced several features and modules that assist novices on aspects such in sight reading, timing and many others. However, improvisation, the act of allowing the piano user to incorporate their personal touch into their performance, and personalised learning have not been much explored in this domain. In this PhD, we are going to explore how we can encourage piano learners to improvise with the use of adaptive piano roll visualisations. Specifically, we are going to investigate how heuristics defined by experts and spatiotemporal models can be used to design visualisations that motivate and encourage learners based on their personalised learning patterns. Using these models and inputs, we will design and build a piano roll training system integrated with adaptive visualisations that serve as intervention helping learners. We will evaluate and compare these visualisations in various user studies where they get to play piano pieces and develop their improvisation skills. We intend to uncover whether these adaptive visualisations will be helpful in the overall training of piano learners. Additionally, we wish to explore whether these adaptive visualisations will allow us to discover affordances that can potentially improve piano learning in general.