EfQAT: An Efficient Framework for Quantization-Aware Training
This addresses the efficiency-accuracy trade-off in model quantization for deployment, but it is incremental as it builds on existing QAT and PTQ schemes.
The paper tackles the computational expense of quantization-aware training (QAT) and accuracy drop of post-training quantization (PTQ) by proposing EfQAT, which optimizes only a subset of parameters in a quantized model, achieving 1.44-1.64x faster backward pass than QAT with little extra compute compared to PTQ.
Quantization-aware training (QAT) schemes have been shown to achieve near-full precision accuracy. They accomplish this by training a quantized model for multiple epochs. This is computationally expensive, mainly because of the full precision backward pass. On the other hand, post-training quantization (PTQ) schemes do not involve training and are therefore computationally cheap, but they usually result in a significant accuracy drop. We address these challenges by proposing EfQAT, which generalizes both schemes by optimizing only a subset of the parameters of a quantized model. EfQAT starts by applying a PTQ scheme to a pre-trained model and only updates the most critical network parameters while freezing the rest, accelerating the backward pass. We demonstrate the effectiveness of EfQAT on various CNNs and Transformer-based models using different GPUs. Specifically, we show that EfQAT is significantly more accurate than PTQ with little extra compute. Furthermore, EfQAT can accelerate the QAT backward pass between 1.44-1.64x while retaining most accuracy.