High-Fidelity Music Vocoder using Neural Audio Codecs
This addresses the underexplored challenge of polyphonic music synthesis for audio generation applications, with incremental improvements over existing methods.
The paper tackled the problem of high-fidelity music synthesis by proposing DisCoder, a neural vocoder that reconstructs 44.1 kHz audio from mel spectrograms using a neural audio codec, achieving state-of-the-art performance in objective metrics and a MUSHRA listening study.
While neural vocoders have made significant progress in high-fidelity speech synthesis, their application on polyphonic music has remained underexplored. In this work, we propose DisCoder, a neural vocoder that leverages a generative adversarial encoder-decoder architecture informed by a neural audio codec to reconstruct high-fidelity 44.1 kHz audio from mel spectrograms. Our approach first transforms the mel spectrogram into a lower-dimensional representation aligned with the Descript Audio Codec (DAC) latent space before reconstructing it to an audio signal using a fine-tuned DAC decoder. DisCoder achieves state-of-the-art performance in music synthesis on several objective metrics and in a MUSHRA listening study. Our approach also shows competitive performance in speech synthesis, highlighting its potential as a universal vocoder.