MahaTTS: A Unified Framework for Multilingual Text-to-Speech Synthesis
This addresses the problem of limited TTS access for non-English speakers, especially in Indic languages, though it appears incremental as it builds on existing methods like Wav2Vec2.0.
The authors tackled the challenge of multilingual text-to-speech synthesis, particularly for Indic languages, by introducing MahaTTS-v2, a system trained on 20K hours of data that shows effectiveness over other frameworks.
Current Text-to-Speech models pose a multilingual challenge, where most of the models traditionally focus on English and European languages, thereby hurting the potential to provide access to information to many more people. To address this gap, we introduce MahaTTS-v2 a Multilingual Multi-speaker Text-To-Speech (TTS) system that has excellent multilingual expressive capabilities in Indic languages. The model has been trained on around 20K hours of data specifically focused on Indian languages. Our approach leverages Wav2Vec2.0 tokens for semantic extraction, and a Language Model (LM) for text-to-semantic modeling. Additionally, we have used a Conditional Flow Model (CFM) for semantics to melspectogram generation. The experimental results indicate the effectiveness of the proposed approach over other frameworks. Our code is available at https://github.com/dubverse-ai/MahaTTSv2