DNQ: Dynamic Network Quantization
This work addresses memory and energy constraints for mobile device deployment, representing an incremental improvement over universal quantization methods.
The paper tackles the problem of deploying neural networks on mobile devices by proposing a Dynamic Network Quantization (DNQ) framework that uses a bit-width controller to learn per-layer bit-widths, achieving a trade-off between accuracy and compression ratio with impressive results on various networks.
Network quantization is an effective method for the deployment of neural networks on memory and energy constrained mobile devices. In this paper, we propose a Dynamic Network Quantization (DNQ) framework which is composed of two modules: a bit-width controller and a quantizer. Unlike most existing quantization methods that use a universal quantization bit-width for the whole network, we utilize policy gradient to train an agent to learn the bit-width of each layer by the bit-width controller. This controller can make a trade-off between accuracy and compression ratio. Given the quantization bit-width sequence, the quantizer adopts the quantization distance as the criterion of the weights importance during quantization. We extensively validate the proposed approach on various main-stream neural networks and obtain impressive results.