Histogram-Equalized Quantization for logic-gated Residual Neural Networks
This addresses the challenge of efficient hardware deployment for neural networks, though it appears incremental as it builds on existing quantization techniques.
The paper tackles the problem of maintaining accuracy in quantized neural networks by proposing Histogram-Equalized Quantization (HEQ), an adaptive framework that optimizes quantization thresholds. It achieves state-of-the-art results on CIFAR-10 and enables training of logic-gated residual networks on STL-10 with higher accuracy and lower hardware complexity than prior methods.
Adjusting the quantization according to the data or to the model loss seems mandatory to enable a high accuracy in the context of quantized neural networks. This work presents Histogram-Equalized Quantization (HEQ), an adaptive framework for linear symmetric quantization. HEQ automatically adapts the quantization thresholds using a unique step size optimization. We empirically show that HEQ achieves state-of-the-art performances on CIFAR-10. Experiments on the STL-10 dataset even show that HEQ enables a proper training of our proposed logic-gated (OR, MUX) residual networks with a higher accuracy at a lower hardware complexity than previous work.