Streaming keyword spotting on mobile devices
This work addresses the need for efficient keyword spotting on mobile devices, offering an incremental improvement through automated conversion and model enhancements.
The authors tackled the problem of converting non-streaming keyword spotting models to streaming ones for mobile devices, developing a library that automates this conversion and benchmarking various models to show trade-offs between latency and accuracy, while also introducing novel models with multi-head attention that reduce classification error by 10% on Google speech commands datasets.
In this work we explore the latency and accuracy of keyword spotting (KWS) models in streaming and non-streaming modes on mobile phones. NN model conversion from non-streaming mode (model receives the whole input sequence and then returns the classification result) to streaming mode (model receives portion of the input sequence and classifies it incrementally) may require manual model rewriting. We address this by designing a Tensorflow/Keras based library which allows automatic conversion of non-streaming models to streaming ones with minimum effort. With this library we benchmark multiple KWS models in both streaming and non-streaming modes on mobile phones and demonstrate different tradeoffs between latency and accuracy. We also explore novel KWS models with multi-head attention which reduce the classification error over the state-of-art by 10% on Google speech commands data sets V2. The streaming library with all experiments is open-sourced.