Alternative inputs for games and AR/VR applications: Deep headbanging on the web

Abstract

In multimedia research, scientific progress is often slowed down by high demands on hard- and software. However, hardware continuously improves and today's hardware got powerful enough to meet the performance demands of complex 3D and deep learning applications. With this demo, we demonstrate that utilizing deep learning and 3D modeling is not a major barrier anymore when building prototypes for showcasing research projects. Our web-based game, called "HeadbangZ", showcases a novel gesturebased input methodology realized through deeply learned pose estimation and user interaction in a 3D environment. Since gesturebased inputs increase the immersion in virtual environments, we assume this input methodology to be especially useful for AR/VR applications and games. Furthermore, we demonstrate that rapid prototyping of applications using novel technologies, such as deep learning, is even possible within 48 hours by developing a working demo within this time frame. Finally, we provide insights into what we learned during the development of HeadbangZ to encourage other researchers to make use of novel technologies. In reference to Stephen Harper's quote "Having hit a wall, the next logical step is not to bang our heads against it.", we hope that the presentation of HeadbangZ encourages researchers to bang their heads rhythmically to rock music instead of angrily against a virtual wall created by hard- and software limitations.