Assem-VC: Realistic Voice Conversion by Assembling Modern Speech Synthesis Techniques
This work addresses voice conversion for applications requiring high-quality, speaker-independent speech synthesis, though it is incremental as it builds on and optimizes existing techniques.
The authors tackled the problem of realistic voice conversion by decomposing existing systems into encoders and a decoder, analyzing each component, and reassembling the best ones to create Assem-VC, which achieves state-of-the-art performance in any-to-many non-parallel voice conversion.
Recent works on voice conversion (VC) focus on preserving the rhythm and the intonation as well as the linguistic content. To preserve these features from the source, we decompose current non-parallel VC systems into two encoders and one decoder. We analyze each module with several experiments and reassemble the best components to propose Assem-VC, a new state-of-the-art any-to-many non-parallel VC system. We also examine that PPG and Cotatron features are speaker-dependent, and attempt to remove speaker identity with adversarial training. Code and audio samples are available at https://github.com/mindslab-ai/assem-vc.