Data-Free Quantization with Accurate Activation Clipping and Adaptive Batch Normalization
This work addresses the problem of compressing neural networks without access to training data, which is incremental but offers specific gains for low bit-width quantization.
The paper tackles data-free quantization for neural networks by introducing accurate activation clipping and adaptive batch normalization to reduce performance degradation, achieving 64.33% top-1 accuracy on ResNet18 with a 3.7% improvement over state-of-the-art methods.
Data-free quantization is a task that compresses the neural network to low bit-width without access to original training data. Most existing data-free quantization methods cause severe performance degradation due to inaccurate activation clipping range and quantization error, especially for low bit-width. In this paper, we present a simple yet effective data-free quantization method with accurate activation clipping and adaptive batch normalization. Accurate activation clipping (AAC) improves the model accuracy by exploiting accurate activation information from the full-precision model. Adaptive batch normalization firstly proposes to address the quantization error from distribution changes by updating the batch normalization layer adaptively. Extensive experiments demonstrate that the proposed data-free quantization method can yield surprisingly performance, achieving 64.33% top-1 accuracy of ResNet18 on ImageNet dataset, with 3.7% absolute improvement outperforming the existing state-of-the-art methods.