Cyberdreams: Visualizing Music in Extended Reality

Abstract

From the visual music films of the twentieth century to the Video Jockey (VJ) performances seen at the latest electronic dance music festivals, there is an extensive body of artistic work that seeks to visualize sound and music. The form that these visualizations take has been shaped significantly by the capabilities of available technologies; thus, we have seen a transition from paint to film; from hand-drawn animations to motion-graphics; and from analog to digital projection systems. In the twenty-first century, visualizations of music are now possible with extended reality (XR) technologies such as virtual reality (VR), augmented/mixed reality (AR/MR), and related forms of multi-projection environment such as fulldome. However, the successful design of visual music and VJ performances using XR technologies requires us to consider the compositional approaches that can be used by artists and designers. To investigate this area, this chapter will begin with an analysis of existing work that visualizes music using XR technologies. This will allow us to consider the spectrum of existing design approaches, and provide a commentary on the possibilities and limitations of the respective technologies. Following this, the chapter will provide an in-depth discussion of Weinel’s practice-led research, which extends from work exhibited at the Carbon Meets Silicon exhibitions held at Wrexham Glyndŵr University (2015, 2017), and includes AR paintings, VJ performances, and a VR application: Cyberdream VR. Through the discussion of these works, the chapter will demonstrate possible compositional principles for visualizing music across media ranging from paint to XR, enabling the realization of work that reinforces the conceptual meanings associated with music.