Optimizing Speech Recognition For The Edge
This work addresses the need for small and fast speech recognizers on mobile devices, representing an incremental improvement in optimization techniques for existing architectures.
The paper tackled the problem of creating efficient speech recognition for edge devices by experimenting with more computationally efficient layer types and applying optimization techniques like pruning and quantization, resulting in a system that is an order of magnitude smaller than the baseline without optimizations.
While most deployed speech recognition systems today still run on servers, we are in the midst of a transition towards deployments on edge devices. This leap to the edge is powered by the progression from traditional speech recognition pipelines to end-to-end (E2E) neural architectures, and the parallel development of more efficient neural network topologies and optimization techniques. Thus, we are now able to create highly accurate speech recognizers that are both small and fast enough to execute on typical mobile devices. In this paper, we begin with a baseline RNN-Transducer architecture comprised of Long Short-Term Memory (LSTM) layers. We then experiment with a variety of more computationally efficient layer types, as well as apply optimization techniques like neural connection pruning and parameter quantization to construct a small, high quality, on-device speech recognizer that is an order of magnitude smaller than the baseline system without any optimizations.