Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
This work addresses the need for flexible and efficient deep network inference across various devices and resource constraints, representing an incremental improvement in adaptive quantization methods.
The paper tackles the problem of training deep networks with adaptive bit-widths for efficient inference by proposing two novel techniques: a collaborative strategy for knowledge transfer from a high-precision teacher and a dynamic block swapping method. The result is improved performance over state-of-the-art methods, demonstrated through extensive experiments on image and video classification benchmarks.
Quantizing deep networks with adaptive bit-widths is a promising technique for efficient inference across many devices and resource constraints. In contrast to static methods that repeat the quantization process and train different models for different constraints, adaptive quantization enables us to flexibly adjust the bit-widths of a single deep network during inference for instant adaptation in different scenarios. While existing research shows encouraging results on common image classification benchmarks, this paper investigates how to train such adaptive networks more effectively. Specifically, we present two novel techniques for quantizing deep neural networks with adaptive bit-widths of weights and activations. First, we propose a collaborative strategy to choose a high-precision teacher for transferring knowledge to the low-precision student while jointly optimizing the model with all bit-widths. Second, to effectively transfer knowledge, we develop a dynamic block swapping method by randomly replacing the blocks in the lower-precision student network with the corresponding blocks in the higher-precision teacher network. Extensive experiments on multiple image classification datasets including video classification benchmarks for the first time, well demonstrate the efficacy of our approach over state-of-the-art methods.