Lombard Speech Synthesis for Any Voice with Controllable Style Embeddings
This provides a robust solution for generating more intelligible speech in noisy environments or for hearing-impaired listeners, representing a novel method for a known bottleneck in text-to-speech synthesis.
The paper tackled the problem of synthesizing Lombard speech for any speaker without needing explicit training data, achieving a system that preserves naturalness and speaker identity while enhancing intelligibility under noise with fine-grained prosodic control.
The Lombard effect plays a key role in natural communication, particularly in noisy environments or when addressing hearing-impaired listeners. We present a controllable text-to-speech (TTS) system capable of synthesizing Lombard speech for any speaker without requiring explicit Lombard data during training. Our approach leverages style embeddings learned from a large, prosodically diverse dataset and analyzes their correlation with Lombard attributes using principal component analysis (PCA). By shifting the relevant PCA components, we manipulate the style embeddings and incorporate them into our TTS model to generate speech at desired Lombard levels. Evaluations demonstrate that our method preserves naturalness and speaker identity, enhances intelligibility under noise, and provides fine-grained control over prosody, offering a robust solution for controllable Lombard TTS for any speaker.