Metaverse-Empowered Music and Dance: Experience of Emotion and Scene Unification

Abstract

Music and dance are closely related and symbiotic. On the one hand, dance often requires music accompaniment. On the other hand, dance can enrich the melody and style of music. The emergence of the metaverse has taken the experience of music and dance to a new level. This paper studies the three-dimensional situational experience of music and dance in Virtual Reality (VR) empowered by metaverse to feel the beauty of situational integration. After the spherical video is projected onto a two-dimensional plane to form a panoramic video, the two-dimensional panoramic video needs to be converted into a spherical video for users to watch. Therefore, it is more reasonable to take spherical video distortion as the distortion measure of panoramic video coding. In this paper, spherical video distortion is taken as the measurement standard of video quality, and the panoramic video coding technology is optimized. Furthermore, the corresponding weights are introduced to change the distortion ratio of different interpolation regions in the calculation process of rate distortion cost, and a rate distortion optimization technology based on spherical distortion measurement is proposed. The equal weight feature of spherical pixel is realized on two-dimensional plane, which improves the coding efficiency of panoramic video. Experimental results show that compared with the three benchmarks, the proposed algorithm can achieve 1.6157% bit saving on average and achieve a good Quality of Experience (QoE) when other processes are the same.