Many-to-Many Voice Conversion using Cycle-Consistent Variational Autoencoder with Multiple Decoders
This work addresses voice conversion for applications like speech synthesis, but it is incremental as it builds on existing VAE methods.
The paper tackled the problem of low sound quality in many-to-many voice conversion using variational autoencoders (VAEs) with non-parallel data by proposing a cycle consistency loss and multiple decoders, resulting in improved sound quality validated through objective and subjective evaluations.
One of the obstacles in many-to-many voice conversion is the requirement of the parallel training data, which contain pairs of utterances with the same linguistic content spoken by different speakers. Since collecting such parallel data is a highly expensive task, many works attempted to use non-parallel training data for many-to-many voice conversion. One of such approaches is using the variational autoencoder (VAE). Though it can handle many-to-many voice conversion without the parallel training, the VAE based voice conversion methods suffer from low sound qualities of the converted speech. One of the major reasons is because the VAE learns only the self-reconstruction path. The conversion path is not trained at all. In this paper, we propose a cycle consistency loss for VAE to explicitly learn the conversion path. In addition, we propose to use multiple decoders to further improve the sound qualities of the conventional VAE based voice conversion methods. The effectiveness of the proposed method is validated using objective and the subjective evaluations.