Resource-efficient DNNs for Keyword Spotting using Neural Architecture Search and Quantization
This work provides more resource-efficient deep neural networks for keyword spotting, which is beneficial for deploying AI models on devices with limited computational resources.
This paper explores neural architecture search (NAS) to automatically discover small, efficient deep neural networks for keyword spotting. They achieved 95.4% accuracy on the Google speech commands dataset with 494.8 kB memory and 19.6 million operations, further improving to 96.3% accuracy with 340.1 kB memory and 27.1 million operations by increasing input features and applying quantization.
This paper introduces neural architecture search (NAS) for the automatic discovery of small models for keyword spotting (KWS) in limited resource environments. We employ a differentiable NAS approach to optimize the structure of convolutional neural networks (CNNs) to maximize the classification accuracy while minimizing the number of operations per inference. Using NAS only, we were able to obtain a highly efficient model with 95.4% accuracy on the Google speech commands dataset with 494.8 kB of memory usage and 19.6 million operations. Additionally, weight quantization is used to reduce the memory consumption even further. We show that weight quantization to low bit-widths (e.g. 1 bit) can be used without substantial loss in accuracy. By increasing the number of input features from 10 MFCC to 20 MFCC we were able to increase the accuracy to 96.3% at 340.1 kB of memory usage and 27.1 million operations.