FineCombo-TTS: Collaborative and Precise Controllable Speech Synthesis Using Text Descriptions and Reference Speech
For controllable TTS researchers, this provides a unified framework that overcomes the limitations of loosely coupled joint approaches, enabling more precise and flexible attribute control.
FineCombo-TTS enables flexible and precise control over acoustic attributes in speech synthesis by combining reference speech and text descriptions, using a Conditional Flow Matching-based predictor to model fine-grained transformations. Experiments show it achieves expressive and precise controllable TTS.
Controllable text-to-speech (TTS) has become a key research focus. However, methods based on either reference speech or text descriptions lack flexibility and precise control, and recent joint approaches remain loosely coupled, with speech modeling timbre and text controlling global style. We propose FineCombo-TTS, a unified framework for speech synthesis grounded in reference speech and guided by text descriptions, enabling flexible and precise control over acoustic attributes. Instead of explicit attribute disentanglement, we learn a unified acoustic representation and introduce a Conditional Flow Matching (CFM)-based Speech Variance Predictor to model fine-grained reference-to-target transformations guided by text descriptions. To support relative attribute control, we construct FineEdit, a structured paired dataset that explicitly encodes source-to-target attribute variations. Experiments demonstrate that our approach achieves flexible, precise, and expressive controllable TTS.