DRVC: A Framework of Any-to-Any Voice Conversion with Self-Supervised Learning
This work addresses voice conversion for out-of-training speakers, offering an incremental improvement over existing disentangle-based methods.
The paper tackles the any-to-any voice conversion problem for unseen speakers by addressing the disentanglement overlapping issue in previous models, proposing the DRVC framework with self-supervised learning and cycle losses, resulting in improved speech quality and voice similarity.
Any-to-any voice conversion problem aims to convert voices for source and target speakers, which are out of the training data. Previous works wildly utilize the disentangle-based models. The disentangle-based model assumes the speech consists of content and speaker style information and aims to untangle them to change the style information for conversion. Previous works focus on reducing the dimension of speech to get the content information. But the size is hard to determine to lead to the untangle overlapping problem. We propose the Disentangled Representation Voice Conversion (DRVC) model to address the issue. DRVC model is an end-to-end self-supervised model consisting of the content encoder, timbre encoder, and generator. Instead of the previous work for reducing speech size to get content, we propose a cycle for restricting the disentanglement by the Cycle Reconstruct Loss and Same Loss. The experiments show there is an improvement for converted speech on quality and voice similarity.