Adversarially Trained Autoencoders for Parallel-Data-Free Voice Conversion
This addresses the challenge of voice conversion for applications like speech synthesis, though it appears incremental as it builds on existing autoencoder and adversarial techniques.
The paper tackles the problem of voice conversion between speakers without requiring parallel data or time-aligned phonemes, achieving generalization to out-of-training speakers through a method using a speaker-independent encoder and speaker-dependent decoders with adversarial training.
We present a method for converting the voices between a set of speakers. Our method is based on training multiple autoencoder paths, where there is a single speaker-independent encoder and multiple speaker-dependent decoders. The autoencoders are trained with an addition of an adversarial loss which is provided by an auxiliary classifier in order to guide the output of the encoder to be speaker independent. The training of the model is unsupervised in the sense that it does not require collecting the same utterances from the speakers nor does it require time aligning over phonemes. Due to the use of a single encoder, our method can generalize to converting the voice of out-of-training speakers to speakers in the training dataset. We present subjective tests corroborating the performance of our method.