RAF: Relativistic Adversarial Feedback For Universal Speech Synthesis
This addresses the problem of improving speech synthesis quality and efficiency for applications like text-to-speech, though it appears incremental as it builds on existing GAN vocoder frameworks.
The paper tackles the problem of limited generalization in GAN vocoders by proposing Relativistic Adversarial Feedback (RAF), a training objective that improves fidelity and generalization, with RAF-trained BigVGAN-base outperforming LSGAN-trained BigVGAN in perceptual quality using only 12% of the parameters.
We propose Relativistic Adversarial Feedback (RAF), a novel training objective for GAN vocoders that improves in-domain fidelity and generalization to unseen scenarios. Although modern GAN vocoders employ advanced architectures, their training objectives often fail to promote generalizable representations. RAF addresses this problem by leveraging speech self-supervised learning models to assist discriminators in evaluating sample quality, encouraging the generator to learn richer representations. Furthermore, we utilize relativistic pairing for real and fake waveforms to improve the modeling of the training data distribution. Experiments across multiple datasets show consistent gains in both objective and subjective metrics on GAN-based vocoders. Importantly, the RAF-trained BigVGAN-base outperforms the LSGAN-trained BigVGAN in perceptual quality using only 12\% of the parameters. Comparative studies further confirm the effectiveness of RAF as a training framework for GAN vocoders.