VANI: Very-lightweight Accent-controllable TTS for Native and Non-native speakers with Identity Preservation
This work addresses accent and language control in text-to-speech for native and non-native speakers, but it appears incremental as it builds upon existing disentanglement strategies.
The authors tackled the problem of multilingual accent-controllable speech synthesis by introducing VANI, a very lightweight model that supports explicit control of accent, language, speaker, and fine-grained features, achieving synthesis in 3 different languages while retaining speaker identity and native accents.
We introduce VANI, a very lightweight multi-lingual accent controllable speech synthesis system. Our model builds upon disentanglement strategies proposed in RADMMM and supports explicit control of accent, language, speaker and fine-grained $F_0$ and energy features for speech synthesis. We utilize the Indic languages dataset, released for LIMMITS 2023 as part of ICASSP Signal Processing Grand Challenge, to synthesize speech in 3 different languages. Our model supports transferring the language of a speaker while retaining their voice and the native accent of the target language. We utilize the large-parameter RADMMM model for Track $1$ and lightweight VANI model for Track $2$ and $3$ of the competition.