AutoCycle-VC: Towards Bottleneck-Independent Zero-Shot Cross-Lingual Voice Conversion
This work addresses voice conversion challenges for applications requiring high-quality, speaker-independent synthesis, representing an incremental improvement over prior methods.
The paper tackles the problem of information loss and poor synthesis quality in zero-shot voice conversion by introducing a cycle-consistency loss and mel-spectrogram pre-processing, resulting in a model that outperforms state-of-the-art methods in subjective and objective evaluations and enables cross-lingual conversions.
This paper proposes a simple and robust zero-shot voice conversion system with a cycle structure and mel-spectrogram pre-processing. Previous works suffer from information loss and poor synthesis quality due to their reliance on a carefully designed bottleneck structure. Moreover, models relying solely on self-reconstruction loss struggled with reproducing different speakers' voices. To address these issues, we suggested a cycle-consistency loss that considers conversion back and forth between target and source speakers. Additionally, stacked random-shuffled mel-spectrograms and a label smoothing method are utilized during speaker encoder training to extract a time-independent global speaker representation from speech, which is the key to a zero-shot conversion. Our model outperforms existing state-of-the-art results in both subjective and objective evaluations. Furthermore, it facilitates cross-lingual voice conversions and enhances the quality of synthesized speech.