TTS Skins: Speaker Conversion via ASR
This addresses speaker conversion for TTS applications, but appears incremental as it builds on existing encoder-decoder and ASR methods.
The paper tackles the problem of converting between speakers' voices without using text, by developing a fully convolutional wav-to-wav network based on an ASR pre-trained encoder and a multi-speaker autoregressive decoder, and demonstrates multi-voice TTS conversion from a TTS robot voice on narrated audiobooks.
We present a fully convolutional wav-to-wav network for converting between speakers' voices, without relying on text. Our network is based on an encoder-decoder architecture, where the encoder is pre-trained for the task of Automatic Speech Recognition, and a multi-speaker waveform decoder is trained to reconstruct the original signal in an autoregressive manner. We train the network on narrated audiobooks, and demonstrate multi-voice TTS in those voices, by converting the voice of a TTS robot.