Table-Based Neural Units: Fully Quantizing Networks for Multiply-Free Inference
This work addresses the need for efficient, multiply-free neural networks suitable for FPGA and ASIC implementations, offering a novel approach to quantization that is incremental in method but provides strong hardware-specific gains.
The authors tackled the problem of making neural networks more efficient for hardware deployment by fully quantizing all network components and replacing multiplication operations with table-based lookups, achieving results within 1.6% of non-quantized performance on MobileNet with 40 table entries and closing the gap to zero with 320 entries.
In this work, we propose to quantize all parts of standard classification networks and replace the activation-weight--multiply step with a simple table-based lookup. This approach results in networks that are free of floating-point operations and free of multiplications, suitable for direct FPGA and ASIC implementations. It also provides us with two simple measures of per-layer and network-wide compactness as well as insight into the distribution characteristics of activationoutput and weight values. We run controlled studies across different quantization schemes, both fixed and adaptive and, within the set of adaptive approaches, both parametric and model-free. We implement our approach to quantization with minimal, localized changes to the training process, allowing us to benefit from advances in training continuous-valued network architectures. We apply our approach successfully to AlexNet, ResNet, and MobileNet. We show results that are within 1.6% of the reported, non-quantized performance on MobileNet using only 40 entries in our table. This performance gap narrows to zero when we allow tables with 320 entries. Our results give the best accuracies among multiply-free networks.