Minimum Energy Quantized Neural Networks
This work addresses energy efficiency for hardware deployment of neural networks, though it is incremental as it builds on existing quantization methods.
The paper tackles the problem of optimizing energy efficiency in quantized neural networks by analyzing the trade-off between bit precision and network architecture to find the minimum-energy solution, showing that BinaryNets or int4 networks can outperform int8 networks by up to 2-10x at iso-accuracy.
This work targets the automated minimum-energy optimization of Quantized Neural Networks (QNNs) - networks using low precision weights and activations. These networks are trained from scratch at an arbitrary fixed point precision. At iso-accuracy, QNNs using fewer bits require deeper and wider network architectures than networks using higher precision operators, while they require less complex arithmetic and less bits per weights. This fundamental trade-off is analyzed and quantified to find the minimum energy QNN for any benchmark and hence optimize energy-efficiency. To this end, the energy consumption of inference is modeled for a generic hardware platform. This allows drawing several conclusions across different benchmarks. First, energy consumption varies orders of magnitude at iso-accuracy depending on the number of bits used in the QNN. Second, in a typical system, BinaryNets or int4 implementations lead to the minimum energy solution, outperforming int8 networks up to 2-10x at iso-accuracy. All code used for QNN training is available from https://github.com/BertMoons.