Conformer-Based Speech Recognition On Extreme Edge-Computing Devices
This work addresses the problem of enabling efficient and accurate speech recognition on smartphones, smart wearables, and smart home devices to protect user privacy, representing a strong specific gain in edge computing.
The paper tackles the challenge of implementing on-device automatic speech recognition on resource-constrained devices by proposing model architecture adaptions, neural network graph transformations, and numerical optimizations, achieving over 5.26 times faster than realtime (0.19 RTF) speech recognition on smart wearables with state-of-the-art accuracy.
With increasingly more powerful compute capabilities and resources in today's devices, traditionally compute-intensive automatic speech recognition (ASR) has been moving from the cloud to devices to better protect user privacy. However, it is still challenging to implement on-device ASR on resource-constrained devices, such as smartphones, smart wearables, and other smart home automation devices. In this paper, we propose a series of model architecture adaptions, neural network graph transformations, and numerical optimizations to fit an advanced Conformer based end-to-end streaming ASR system on resource-constrained devices without accuracy degradation. We achieve over 5.26 times faster than realtime (0.19 RTF) speech recognition on smart wearables while minimizing energy consumption and achieving state-of-the-art accuracy. The proposed methods are widely applicable to other transformer-based server-free AI applications. In addition, we provide a complete theory on optimal pre-normalizers that numerically stabilize layer normalization in any Lp-norm using any floating point precision.