Analytic Study of Text-Free Speech Synthesis for Raw Audio using a Self-Supervised Learning Model
This work addresses the challenge of augmenting datasets for speech synthesis without paired text, which is incremental as it builds on existing self-supervised learning methods.
The study tackled the problem of text-free speech synthesis from raw audio by using self-supervised learning representations, finding that text representations better preserve semantic information while discrete symbol representations excel at retaining acoustic content like prosody and intonation.
We examine the text-free speech representations of raw audio obtained from a self-supervised learning (SSL) model by analyzing the synthesized speech using the SSL representations instead of conventional text representations. Since raw audio does not have paired speech representations as transcribed texts do, obtaining speech representations from unpaired speech is crucial for augmenting available datasets for speech synthesis. Specifically, the proposed speech synthesis is conducted using discrete symbol representations from the SSL model in comparison with text representations, and analytical examinations of the synthesized speech have been carried out. The results empirically show that using text representations is advantageous for preserving semantic information, while using discrete symbol representations is superior for preserving acoustic content, including prosodic and intonational information.