CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification
This work addresses a key bottleneck in deploying efficient neural networks for resource-constrained applications, though it is incremental as it builds on existing mixed-precision quantization techniques.
The paper tackles the challenge of determining optimal precision for each layer in mixed-precision quantization of deep neural networks, proposing CSQ to stabilize training and achieve better efficiency-accuracy tradeoffs, with results showing improved performance over previous methods on multiple models and datasets.
Mixed-precision quantization has been widely applied on deep neural networks (DNNs) as it leads to significantly better efficiency-accuracy tradeoffs compared to uniform quantization. Meanwhile, determining the exact precision of each layer remains challenging. Previous attempts on bit-level regularization and pruning-based dynamic precision adjustment during training suffer from noisy gradients and unstable convergence. In this work, we propose Continuous Sparsification Quantization (CSQ), a bit-level training method to search for mixed-precision quantization schemes with improved stability. CSQ stabilizes the bit-level mixed-precision training process with a bi-level gradual continuous sparsification on both the bit values of the quantized weights and the bit selection in determining the quantization precision of each layer. The continuous sparsification scheme enables fully-differentiable training without gradient approximation while achieving an exact quantized model in the end.A budget-aware regularization of total model size enables the dynamic growth and pruning of each layer's precision towards a mixed-precision quantization scheme of the desired size. Extensive experiments show CSQ achieves better efficiency-accuracy tradeoff than previous methods on multiple models and datasets.