Augmenting Virtuality with a Synchronized Dynamic Musical Instrument: A User Evaluation of a Mixed Reality MIDI Keyboard

Abstract

As virtual reality gains popularity, technology which better integrates the user’s physical experience in the virtual environment is needed. Researchers have shown that by including real physical objects to interact with, the experience can be made significantly more convincing and user-friendly. To explore physically connecting the user to the virtual environment, we designed and developed a mixed reality MIDI keyboard. This project explores maintaining the user’s physical connection to the real world in order to align their senses with their true state of augmented virtuality. A user study of the keyboard was performed which verified its ability to improve the VR experience and identified areas for further research. In addition to producing a mixed reality MIDI keyboard, this project serves as a roadmap and foundation for future developments and investigations in integrating physical and virtual environments to improve immersion and presence.