Unsupervised Cross-Domain Singing Voice Conversion
This addresses the challenge of creating realistic singing voice conversions for applications like music production, but it is incremental as it builds on existing generative and feature extraction techniques.
The paper tackles the problem of converting singing voices across different identities without supervision, using a wav-to-wav generative model that combines acoustic and melody features, and reports significantly outperforming baseline methods with better audio samples.
We present a wav-to-wav generative model for the task of singing voice conversion from any identity. Our method utilizes both an acoustic model, trained for the task of automatic speech recognition, together with melody extracted features to drive a waveform-based generator. The proposed generative architecture is invariant to the speaker's identity and can be trained to generate target singers from unlabeled training data, using either speech or singing sources. The model is optimized in an end-to-end fashion without any manual supervision, such as lyrics, musical notes or parallel samples. The proposed approach is fully-convolutional and can generate audio in real-time. Experiments show that our method significantly outperforms the baseline methods while generating convincingly better audio samples than alternative attempts.