BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization
This work addresses the challenge of optimizing neural network compression for deployment, offering a more efficient method than prior approaches, though it is incremental in improving quantization techniques.
The paper tackles the problem of efficiently determining optimal mixed-precision quantization schemes for deep neural networks by proposing bit-level sparsity quantization (BSQ), which induces bit-level sparsity to dynamically reduce precision, achieving higher accuracy and bit reduction on CIFAR-10 and ImageNet datasets compared to previous methods.
Mixed-precision quantization can potentially achieve the optimal tradeoff between performance and compression rate of deep neural networks, and thus, have been widely investigated. However, it lacks a systematic method to determine the exact quantization scheme. Previous methods either examine only a small manually-designed search space or utilize a cumbersome neural architecture search to explore the vast search space. These approaches cannot lead to an optimal quantization scheme efficiently. This work proposes bit-level sparsity quantization (BSQ) to tackle the mixed-precision quantization from a new angle of inducing bit-level sparsity. We consider each bit of quantized weights as an independent trainable variable and introduce a differentiable bit-sparsity regularizer. BSQ can induce all-zero bits across a group of weight elements and realize the dynamic precision reduction, leading to a mixed-precision quantization scheme of the original model. Our method enables the exploration of the full mixed-precision space with a single gradient-based optimization process, with only one hyperparameter to tradeoff the performance and compression. BSQ achieves both higher accuracy and higher bit reduction on various model architectures on the CIFAR-10 and ImageNet datasets comparing to previous methods.