Deep Neural Network inference with reduced word length
This work addresses the challenge of enabling efficient DNN inference on dedicated hardware for applications limited by computational resources, though it is incremental as it builds on existing low-precision methods.
The paper tackles the problem of high computational complexity and memory requirements in deep neural network inference by proposing a method to evaluate DNNs trained with 32-bit floating-point accuracy using low-precision integer arithmetic, binary shifts, and clipping, achieving results with minimal performance degradation on MNIST using 2-bit or 3-bit integer arithmetic.
Deep neural networks (DNN) are powerful models for many pattern recognition tasks, yet their high computational complexity and memory requirement limit them to applications on high-performance computing platforms. In this paper, we propose a new method to evaluate DNNs trained with 32bit floating point (float32) accuracy using only low precision integer arithmetics in combination with binary shift and clipping operations. Because hardware implementation of these operations is much simpler than high precision floating point calculation, our method can be used for an efficient DNN inference on dedicated hardware. In experiments on MNIST, we demonstrate that DNNs trained with float32 can be evaluated using a combination of 2bit integer arithmetics and a few float32 calculations in each layer or only 3bit integer arithmetics in combination with binary shift and clipping without significant performance degradation.