AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
This addresses voice style transfer for applications like speech synthesis, offering a simpler alternative to GANs and CVAEs, though it is incremental in improving existing methods.
The authors tackled non-parallel many-to-many and zero-shot voice conversion by proposing AUTOVC, a method using only an autoencoder with a designed bottleneck, which achieved state-of-the-art results and enabled zero-shot conversion for the first time.
Non-parallel many-to-many voice conversion, as well as zero-shot voice conversion, remain under-explored areas. Deep style transfer algorithms, such as generative adversarial networks (GAN) and conditional variational autoencoder (CVAE), are being applied as new solutions in this field. However, GAN training is sophisticated and difficult, and there is no strong evidence that its generated speech is of good perceptual quality. On the other hand, CVAE training is simple but does not come with the distribution-matching property of a GAN. In this paper, we propose a new style transfer scheme that involves only an autoencoder with a carefully designed bottleneck. We formally show that this scheme can achieve distribution-matching style transfer by training only on a self-reconstruction loss. Based on this scheme, we proposed AUTOVC, which achieves state-of-the-art results in many-to-many voice conversion with non-parallel data, and which is the first to perform zero-shot voice conversion.