Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applications
This enables real-time, high-quality TTS for low-resource on-device applications, representing an incremental improvement in efficiency.
The paper tackles the problem of computationally complex and memory-consuming end-to-end text-to-speech models by proposing a lightweight model that achieves state-of-the-art performance, with up to 90% smaller parameters and 10x faster real-time factor on the LJSpeech dataset.
Recent works have shown that modelling raw waveform directly from text in an end-to-end (E2E) fashion produces more natural-sounding speech than traditional neural text-to-speech (TTS) systems based on a cascade or two-stage approach. However, current E2E state-of-the-art models are computationally complex and memory-consuming, making them unsuitable for real-time offline on-device applications in low-resource scenarios. To address this issue, we propose a Lightweight E2E-TTS (LE2E) model that generates high-quality speech requiring minimal computational resources. We evaluate the proposed model on the LJSpeech dataset and show that it achieves state-of-the-art performance while being up to $90\%$ smaller in terms of model parameters and $10\times$ faster in real-time-factor. Furthermore, we demonstrate that the proposed E2E training paradigm achieves better quality compared to an equivalent architecture trained in a two-stage approach. Our results suggest that LE2E is a promising approach for developing real-time, high quality, low-resource TTS applications for on-device applications.