PD-Quant: Post-Training Quantization based on Prediction Difference Metric
This work addresses the challenge of maintaining model accuracy during compression for deployment in resource-constrained environments, representing an incremental improvement over existing methods.
The paper tackles the problem of accuracy loss in post-training quantization for neural networks, especially at low bit-widths, by proposing PD-Quant, a method that uses prediction differences to determine quantization parameters, resulting in improved accuracy, such as raising ResNet-18 to 53.14% and RegNetX-600MF to 40.67% in 2-bit weight and activation settings.
Post-training quantization (PTQ) is a neural network compression technique that converts a full-precision model into a quantized model using lower-precision data types. Although it can help reduce the size and computational cost of deep neural networks, it can also introduce quantization noise and reduce prediction accuracy, especially in extremely low-bit settings. How to determine the appropriate quantization parameters (e.g., scaling factors and rounding of weights) is the main problem facing now. Existing methods attempt to determine these parameters by minimize the distance between features before and after quantization, but such an approach only considers local information and may not result in the most optimal quantization parameters. We analyze this issue and ropose PD-Quant, a method that addresses this limitation by considering global information. It determines the quantization parameters by using the information of differences between network prediction before and after quantization. In addition, PD-Quant can alleviate the overfitting problem in PTQ caused by the small number of calibration sets by adjusting the distribution of activations. Experiments show that PD-Quant leads to better quantization parameters and improves the prediction accuracy of quantized models, especially in low-bit settings. For example, PD-Quant pushes the accuracy of ResNet-18 up to 53.14% and RegNetX-600MF up to 40.67% in weight 2-bit activation 2-bit. The code is released at https://github.com/hustvl/PD-Quant.