Self-supervised learning for robust voice cloning
This work addresses voice cloning for TTS systems by improving robustness to noise and acoustic conditions, though it is incremental as it builds on existing self-supervised methods.
The paper tackled robust voice cloning by using self-supervised learning with BYOL and extended audio augmentations to capture speaker identity and noise robustness, achieving multispeaker speech synthesis without additional speaker features and validating results through subjective and objective evaluations.
Voice cloning is a difficult task which requires robust and informative features incorporated in a high quality TTS system in order to effectively copy an unseen speaker's voice. In our work, we utilize features learned in a self-supervised framework via the Bootstrap Your Own Latent (BYOL) method, which is shown to produce high quality speech representations when specific audio augmentations are applied to the vanilla algorithm. We further extend the augmentations in the training procedure to aid the resulting features to capture the speaker identity and to make them robust to noise and acoustic conditions. The learned features are used as pre-trained utterance-level embeddings and as inputs to a Non-Attentive Tacotron based architecture, aiming to achieve multispeaker speech synthesis without utilizing additional speaker features. This method enables us to train our model in an unlabeled multispeaker dataset as well as use unseen speaker embeddings to copy a speaker's voice. Subjective and objective evaluations are used to validate the proposed model, as well as the robustness to the acoustic conditions of the target utterance.