From Quantized DNNs to Quantizable DNNs
This work addresses the need for efficient and adaptable neural networks in resource-constrained environments, offering a novel approach to quantization that improves flexibility and performance.
The paper tackles the problem of enabling deep neural networks to flexibly quantize bit-widths during execution without retraining, resulting in Quantizable DNNs that achieve higher classification accuracy on CIFAR100 and ImageNet compared to standard quantized DNNs.
This paper proposes Quantizable DNNs, a special type of DNNs that can flexibly quantize its bit-width (denoted as `bit modes' thereafter) during execution without further re-training. To simultaneously optimize for all bit modes, a combinational loss of all bit modes is proposed, which enforces consistent predictions ranging from low-bit mode to 32-bit mode. This Consistency-based Loss may also be viewed as certain form of regularization during training. Because outputs of matrix multiplication in different bit modes have different distributions, we introduce Bit-Specific Batch Normalization so as to reduce conflicts among different bit modes. Experiments on CIFAR100 and ImageNet have shown that compared to quantized DNNs, Quantizable DNNs not only have much better flexibility, but also achieve even higher classification accuracy. Ablation studies further verify that the regularization through the consistency-based loss indeed improves the model's generalization performance.