Audio Super-Resolution via Vision Transformer

Abstract

Audio super-resolution refers to techniques that improve the quality of audio signals, usually by exploiting bandwidth extension methods, whereby audio enhancement is obtained by expanding the phase and the spectrogram of the input audio traces. These techniques are therefore much significant for all those cases where audio traces miss relevant parts of the audible spectrum. In many cases, the given input signal contains the low-band frequencies (the easiest to capture with low-quality recording instruments) whereas the high-band must be generated. In this paper, we illustrate a system for bandwidth extension that works on musical tracks and generates the high-band frequencies starting from the low-band ones. The system, called ViT Super-resolution (ViT-SR), features an architecture based on a Generative Adversarial Network and Vision Transformer model. Some experiments, which are accounted for in the paper, serve the purpose to prove the effectiveness of the presented approach. In particular, our purpose was to demonstrate that it is possible to faithfully reconstruct the high-band signal of an audio file having only its low-band spectrum available as the input, therewith including the usually difficult to synthetically generate harmonics associated with the input track which significantly contribute to the final perceived sound quality.