Adaptive Binary-Ternary Quantization
This work addresses resource constraints for deploying deep networks on devices like wearables and drones, but it is incremental as it builds on existing mixed quantization approaches.
The paper tackles the problem of resource-intensive neural networks by proposing an adaptive binary-ternary quantization method that modifies quantization depth via regularization, requiring only a single training pass. Experimental results demonstrate successful adaptation of quantization depth while maintaining high accuracy on MNIST and CIFAR10 benchmarks.
Neural network models are resource hungry. It is difficult to deploy such deep networks on devices with limited resources, like smart wearables, cellphones, drones, and autonomous vehicles. Low bit quantization such as binary and ternary quantization is a common approach to alleviate this resource requirements. Ternary quantization provides a more flexible model and outperforms binary quantization in terms of accuracy, however doubles the memory footprint and increases the computational cost. Contrary to these approaches, mixed quantized models allow a trade-off between accuracy and memory footprint. In such models, quantization depth is often chosen manually, or is tuned using a separate optimization routine. The latter requires training a quantized network multiple times. Here, we propose an adaptive combination of binary and ternary quantization, namely Smart Quantization (SQ), in which the quantization depth is modified directly via a regularization function, so that the model is trained only once. Our experimental results show that the proposed method adapts quantization depth successfully while keeping the model accuracy high on MNIST and CIFAR10 benchmarks.