Controllable Generation of Artificial Speaker Embeddings through Discovery of Principal Directions
This addresses privacy and data scarcity challenges in speech synthesis for developers and users, though it is incremental as it builds on existing embedding and synthesis methods.
The paper tackles the problem of customizing voice and speaking style in speech synthesis with intuitive controls while avoiding ethical issues, by proposing a method to generate artificial speaker embeddings that offer fine-grained control without requiring labels.
Customizing voice and speaking style in a speech synthesis system with intuitive and fine-grained controls is challenging, given that little data with appropriate labels is available. Furthermore, editing an existing human's voice also comes with ethical concerns. In this paper, we propose a method to generate artificial speaker embeddings that cannot be linked to a real human while offering intuitive and fine-grained control over the voice and speaking style of the embeddings, without requiring any labels for speaker or style. The artificial and controllable embeddings can be fed to a speech synthesis system, conditioned on embeddings of real humans during training, without sacrificing privacy during inference.