Standard Deviation-Based Quantization for Deep Neural Networks
This work addresses efficient deployment of deep networks on devices with limited resources, representing an incremental improvement over existing quantization methods.
The paper tackles the problem of reducing inference cost for deep neural networks on resource-restricted devices by proposing a quantization method based on standard deviation and a base-2 logarithmic scheme, achieving better accuracy with 3-bit weights and activations compared to full-precision models on CIFAR10 and ImageNet datasets.
Quantization of deep neural networks is a promising approach that reduces the inference cost, making it feasible to run deep networks on resource-restricted devices. Inspired by existing methods, we propose a new framework to learn the quantization intervals (discrete values) using the knowledge of the network's weight and activation distributions, i.e., standard deviation. Furthermore, we propose a novel base-2 logarithmic quantization scheme to quantize weights to power-of-two discrete values. Our proposed scheme allows us to replace resource-hungry high-precision multipliers with simple shift-add operations. According to our evaluations, our method outperforms existing work on CIFAR10 and ImageNet datasets and even achieves better accuracy performance with 3-bit weights and activations when compared to the full-precision models. Moreover, our scheme simultaneously prunes the network's parameters and allows us to flexibly adjust the pruning ratio during the quantization process.