Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis
This addresses the need for faster and high-quality audio synthesis in applications like speech generation, though it is incremental as it builds on existing vocoder methods.
The paper tackles the problem of high-quality audio synthesis by introducing Vocos, a Fourier-based neural vocoder that matches state-of-the-art audio quality while improving computational efficiency by an order of magnitude compared to time-domain approaches.
Recent advancements in neural vocoding are predominantly driven by Generative Adversarial Networks (GANs) operating in the time-domain. While effective, this approach neglects the inductive bias offered by time-frequency representations, resulting in reduntant and computionally-intensive upsampling operations. Fourier-based time-frequency representation is an appealing alternative, aligning more accurately with human auditory perception, and benefitting from well-established fast algorithms for its computation. Nevertheless, direct reconstruction of complex-valued spectrograms has been historically problematic, primarily due to phase recovery issues. This study seeks to close this gap by presenting Vocos, a new model that directly generates Fourier spectral coefficients. Vocos not only matches the state-of-the-art in audio quality, as demonstrated in our evaluations, but it also substantially improves computational efficiency, achieving an order of magnitude increase in speed compared to prevailing time-domain neural vocoding approaches. The source code and model weights have been open-sourced at https://github.com/gemelo-ai/vocos.