PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mapping
This work addresses a specific bottleneck in speech synthesis for audio generation applications, representing an incremental advancement.
The paper tackles the problem of overfitting and periodicity artifacts in GAN-based neural vocoders by introducing PhaseAug, a differentiable augmentation that rotates phase to simulate one-to-many mapping in speech synthesis, resulting in performance improvements over baselines.
Previous generative adversarial network (GAN)-based neural vocoders are trained to reconstruct the exact ground truth waveform from the paired mel-spectrogram and do not consider the one-to-many relationship of speech synthesis. This conventional training causes overfitting for both the discriminators and the generator, leading to the periodicity artifacts in the generated audio signal. In this work, we present PhaseAug, the first differentiable augmentation for speech synthesis that rotates the phase of each frequency bin to simulate one-to-many mapping. With our proposed method, we outperform baselines without any architecture modification. Code and audio samples will be available at https://github.com/mindslab-ai/phaseaug.