How Far Are We from Robust Voice Conversion: A Survey
This survey identifies robustness challenges for voice conversion models in various conditions, particularly for researchers and developers working on real-world VC applications.
This paper surveys the robustness of existing voice conversion (VC) models, finding that sampling rate and audio duration significantly impact performance. It concludes that all VC models struggle with unseen data, though AdaIN-VC is comparatively more robust, and jointly trained speaker embeddings are superior to those from speaker identification.
Voice conversion technologies have been greatly improved in recent years with the help of deep learning, but their capabilities of producing natural sounding utterances in different conditions remain unclear. In this paper, we gave a thorough study of the robustness of known VC models. We also modified these models, such as the replacement of speaker embeddings, to further improve their performances. We found that the sampling rate and audio duration greatly influence voice conversion. All the VC models suffer from unseen data, but AdaIN-VC is relatively more robust. Also, the speaker embedding jointly trained is more suitable for voice conversion than those trained on speaker identification.