Towards Fine-Grained Prosody Control for Voice Conversion
This addresses prosody control in voice conversion systems, particularly for Mandarin speech, but appears incremental as it builds on existing feature-based approaches.
The paper tackles the challenge of imperfect prosody description through acoustic features in voice conversion by proposing unsupervised prosody embeddings, which enable fine-grained prosody control and achieve promising results even with singing source speech.
In a typical voice conversion system, prior works utilize various acoustic features (e.g., the pitch, voiced/unvoiced flag, aperiodicity) of the source speech to control the prosody of generated waveform. However, the prosody is related with many factors, such as the intonation, stress and rhythm. It is a challenging task to perfectly describe the prosody through acoustic features. To deal with this problem, we propose prosody embeddings to model prosody. These embeddings are learned from the source speech in an unsupervised manner. We conduct experiments on our Mandarin corpus recoded by professional speakers. Experimental results demonstrate that the proposed method enables fine-grained control of the prosody. In challenging situations (such as the source speech is a singing song), our proposed method can also achieve promising results.