Mobile Motion: Multimodal Device Augmentation for Musical Applications

Abstract

Mobile devices have become an integral part of the twenty-first century lifestyle. From social networking and business to day-to-day scheduling and multimedia applications, smart-phones and other portable handsets are now the go-to devices for interaction in the digital world. Currently, mobile devices typically utilise direct user interfaces, such as touch screens or keyboards, where interactions are performed directly by controlling graphical elements or controls on the interface. This project looks to bring device interaction out of the virtual world and into the physical world. Through augmenting existing mobile technologies with custom electronic hardware, it is possible to create a system that can incorporate free gestures within a portable context. With this approach, portable applications can break away from the virtual world and enable the mobile platform to be harnessed as a physical augmented interface. This concept can be exploited for applications within a wide range of contexts including musical performance, games, learning and teaching, and beyond.