Quantization Range Estimation for Convolutional Neural Networks
This work addresses the problem of model compression for deployment efficiency in computer vision applications, representing an incremental improvement over existing quantization methods.
The paper tackles the challenge of maintaining accuracy during low-bit post-training quantization of convolutional neural networks by proposing a range estimation method that minimizes quantization errors through layer-wise local minima optimization. Experimental results show the method achieves state-of-the-art top-1 accuracy with almost no loss in 8-bit and 6-bit settings and significant improvements in 4-bit quantization on ResNet series and Inception-v3 models.
Post-training quantization for reducing the storage of deep neural network models has been demonstrated to be an effective way in various tasks. However, low-bit quantization while maintaining model accuracy is a challenging problem. In this paper, we present a range estimation method to improve the quantization performance for post-training quantization. We model the range estimation into an optimization problem of minimizing quantization errors by layer-wise local minima. We prove this problem is locally convex and present an efficient search algorithm to find the optimal solution. We propose the application of the above search algorithm to the transformed weights space to do further improvement in practice. Our experiments demonstrate that our method outperforms state-of-the-art performance generally on top-1 accuracy for image classification tasks on the ResNet series models and Inception-v3 model. The experimental results show that the proposed method has almost no loss of top-1 accuracy in 8-bit and 6-bit settings for image classifications, and the accuracy of 4-bit quantization is also significantly improved. The code is available at https://github.com/codeiscommitting/REQuant.