A Channel-Pruned and Weight-Binarized Convolutional Neural Network for Keyword Spotting
This work addresses efficiency challenges for keyword spotting systems, but it is incremental as it builds on existing pruning and binarization techniques.
The authors tackled the problem of reducing model size and computational cost for keyword spotting by combining channel pruning with weight binarization, achieving over 50% channel sparsity with less than 0.25% accuracy loss.
We study channel number reduction in combination with weight binarization (1-bit weight precision) to trim a convolutional neural network for a keyword spotting (classification) task. We adopt a group-wise splitting method based on the group Lasso penalty to achieve over 50% channel sparsity while maintaining the network performance within 0.25% accuracy loss. We show an effective three-stage procedure to balance accuracy and sparsity in network training.