DBQ: A Differentiable Branch Quantizer for Lightweight Deep Neural Networks
This addresses the deployment of deep neural networks on resource-constrained devices by improving quantization for lightweight architectures, though it is incremental as it builds on existing quantization techniques.
The paper tackled the problem of quantizing lightweight deep neural networks like MobileNet, which existing methods struggle with, by proposing a differentiable non-uniform quantizer (DBQ) that achieves state-of-the-art accuracy-complexity trade-offs on datasets such as CIFAR-10, ImageNet, and Visual Wake Words.
Deep neural networks have achieved state-of-the art performance on various computer vision tasks. However, their deployment on resource-constrained devices has been hindered due to their high computational and storage complexity. While various complexity reduction techniques, such as lightweight network architecture design and parameter quantization, have been successful in reducing the cost of implementing these networks, these methods have often been considered orthogonal. In reality, existing quantization techniques fail to replicate their success on lightweight architectures such as MobileNet. To this end, we present a novel fully differentiable non-uniform quantizer that can be seamlessly mapped onto efficient ternary-based dot product engines. We conduct comprehensive experiments on CIFAR-10, ImageNet, and Visual Wake Words datasets. The proposed quantizer (DBQ) successfully tackles the daunting task of aggressively quantizing lightweight networks such as MobileNetV1, MobileNetV2, and ShuffleNetV2. DBQ achieves state-of-the art results with minimal training overhead and provides the best (pareto-optimal) accuracy-complexity trade-off.