Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech
For TTS researchers, this provides a more interpretable method for emotion control by revealing that emotional variation is distributed across multiple sparse features, rather than a single global shift.
The paper uses sparse autoencoders to identify interpretable latent features in LLM-based TTS models, enabling bidirectional emotion control (induction/suppression) without modifying backbone parameters. Steering these features achieves comparable or better performance than global steering and existing baselines.
Integrating large language models (LLMs) into text-to-speech (TTS) systems has improved speech expressiveness, yet interpretable emotional control remains challenging. Existing approaches primarily rely on external conditioning or global activation steering, offering limited insight into the internal representations underlying emotional control. In this work, we analyze emotion-related variation in the semantic hidden states of LLM-based TTS models using sparse autoencoders (SAEs) to identify sparse latent features. Our analysis shows that emotional variation is distributed across multiple sparse latent features, while intervening on a small subset enables interpretable emotion control. Building on this observation, we introduce a feature-level intervention framework for bidirectional emotion induction and suppression without modifying backbone parameters. We further show that distinct latent features are associated with specific acoustic attributes (e.g., pitch), suggesting that emotional expression arises from coordinated latent contributions rather than a single global shift. Empirically, steering these sparse latent features achieves comparable or superior emotion induction and suppression performance relative to global steering and existing TTS baselines.