EfficientSpeech: An On-Device Text to Speech Model
This work addresses the challenge of enabling standalone speech synthesis on devices with limited resources and no internet access, representing an incremental improvement in efficiency for TTS applications.
The paper tackles the problem of deploying neural text-to-speech models on resource-constrained edge devices by proposing EfficientSpeech, which achieves real-time synthesis on an ARM CPU with 266k parameters and 90 MFLOPS, showing only a slight degradation in audio quality compared to FastSpeech2.
State of the art (SOTA) neural text to speech (TTS) models can generate natural-sounding synthetic voices. These models are characterized by large memory footprints and substantial number of operations due to the long-standing focus on speech quality with cloud inference in mind. Neural TTS models are generally not designed to perform standalone speech syntheses on resource-constrained and no Internet access edge devices. In this work, an efficient neural TTS called EfficientSpeech that synthesizes speech on an ARM CPU in real-time is proposed. EfficientSpeech uses a shallow non-autoregressive pyramid-structure transformer forming a U-Network. EfficientSpeech has 266k parameters and consumes 90 MFLOPS only or about 1% of the size and amount of computation in modern compact models such as Mixer-TTS. EfficientSpeech achieves an average mel generation real-time factor of 104.3 on an RPi4. Human evaluation shows only a slight degradation in audio quality as compared to FastSpeech2.