Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
This work addresses the challenge of scalable and robust speech synthesis for applications requiring expressive voice generation, though it is incremental as it builds on existing Tacotron systems.
The paper tackles the problem of modeling and controlling acoustic expressiveness in speech synthesis by proposing global style tokens (GSTs), which are unsupervised embeddings trained within Tacotron, enabling style control and transfer without explicit labels, such as varying speed and speaking style independently of text and replicating styles across long-form text.
In this work, we propose "global style tokens" (GSTs), a bank of embeddings that are jointly trained within Tacotron, a state-of-the-art end-to-end speech synthesis system. The embeddings are trained with no explicit labels, yet learn to model a large range of acoustic expressiveness. GSTs lead to a rich set of significant results. The soft interpretable "labels" they generate can be used to control synthesis in novel ways, such as varying speed and speaking style - independently of the text content. They can also be used for style transfer, replicating the speaking style of a single audio clip across an entire long-form text corpus. When trained on noisy, unlabeled found data, GSTs learn to factorize noise and speaker identity, providing a path towards highly scalable but robust speech synthesis.