Constraint Guided Model Quantization of Neural Networks
This work addresses the critical problem of deploying complex neural networks on resource-constrained edge hardware for practitioners, offering a method to ensure computational budget compliance.
This paper introduces Constraint Guided Model Quantization (CGMQ), a quantization-aware training algorithm that reduces the bit-widths of neural network parameters while adhering to a predefined upper bound on computational resources. CGMQ achieves competitive performance with state-of-the-art methods on MNIST and CIFAR10, uniquely guaranteeing satisfaction of computational cost constraints without hyperparameter tuning.
Deploying neural networks on the edge has become increasingly important as deep learning is being applied in an increasing amount of applications. At the edge computing hardware typically has limited resources disallowing to run neural networks with high complexity. To reduce the complexity of neural networks a wide range of quantization methods have been proposed in recent years. This work proposes Constraint Guided Model Quantization (CGMQ), which is a quantization aware training algorithm that uses an upper bound on the computational resources and reduces the bit-widths of the parameters of the neural network. CGMQ does not require the tuning of a hyperparameter to result in a mixed precision neural network that satisfies the predefined computational cost constraint, while prior work does. It is shown on MNIST and CIFAR10 that the performance of CGMQ is competitive with state-of-the-art quantization aware training algorithms, while guaranteeing the satisfaction of an upper bound on the computational complexity defined by the computational resources of the on edge hardware.