One Model for All Quantization: A Quantized Network Supporting Hot-Swap Bit-Width Adjustment
This addresses the inconvenience of adapting quantized models for edge devices when precision requirements change, though it is an incremental improvement in quantization methods.
The paper tackles the problem of needing to retrain or fine-tune models for different quantization bit-widths by proposing a single model that supports hot-swap adjustment across bit-widths from 8-bit to 1-bit, achieving accuracy comparable to dedicated models on ImageNet and COCO.
As an effective technique to achieve the implementation of deep neural networks in edge devices, model quantization has been successfully applied in many practical applications. No matter the methods of quantization aware training (QAT) or post-training quantization (PTQ), they all depend on the target bit-widths. When the precision of quantization is adjusted, it is necessary to fine-tune the quantized model or minimize the quantization noise, which brings inconvenience in practical applications. In this work, we propose a method to train a model for all quantization that supports diverse bit-widths (e.g., form 8-bit to 1-bit) to satisfy the online quantization bit-width adjustment. It is hot-swappable that can provide specific quantization strategies for different candidates through multiscale quantization. We use wavelet decomposition and reconstruction to increase the diversity of weights, thus significantly improving the performance of each quantization candidate, especially at ultra-low bit-widths (e.g., 3-bit, 2-bit, and 1-bit). Experimental results on ImageNet and COCO show that our method can achieve accuracy comparable performance to dedicated models trained at the same precision.