Convolutional Neural Networks Quantization with Attention
This addresses the challenge of deploying efficient neural networks on resource-constrained devices, though it appears incremental as it builds on existing quantization techniques.
The paper tackles the problem of accuracy loss in quantizing deep convolutional neural networks for efficient inference by proposing a double-stage Squeeze-and-Threshold method using attention, achieving state-of-the-art results where a 3-bit model exceeds the accuracy of a full-precision baseline.
It has been proven that, compared to using 32-bit floating-point numbers in the training phase, Deep Convolutional Neural Networks (DCNNs) can operate with low precision during inference, thereby saving memory space and power consumption. However, quantizing networks is always accompanied by an accuracy decrease. Here, we propose a method, double-stage Squeeze-and-Threshold (double-stage ST). It uses the attention mechanism to quantize networks and achieve state-of-art results. Using our method, the 3-bit model can achieve accuracy that exceeds the accuracy of the full-precision baseline model. The proposed double-stage ST activation quantization is easy to apply: inserting it before the convolution.