Deep Positron: A Deep Neural Network Using the Posit Number System
This work addresses the problem of efficient low-precision DNN inference for hardware accelerators, though it is incremental as it builds on existing numerical format research with limited dataset scope.
The paper tackles the computational and memory challenges of deep neural networks by proposing Deep Positron, a DNN architecture using the posit number system at ≤8 bits for inference, achieving better accuracy than 8-bit fixed or floating-point on low-dimensional datasets and comparable accuracy to 32-bit floating-point on an FPGA.
The recent surge of interest in Deep Neural Networks (DNNs) has led to increasingly complex networks that tax computational and memory resources. Many DNNs presently use 16-bit or 32-bit floating point operations. Significant performance and power gains can be obtained when DNN accelerators support low-precision numerical formats. Despite considerable research, there is still a knowledge gap on how low-precision operations can be realized for both DNN training and inference. In this work, we propose a DNN architecture, Deep Positron, with posit numerical format operating successfully at $\leq$8 bits for inference. We propose a precision-adaptable FPGA soft core for exact multiply-and-accumulate for uniform comparison across three numerical formats, fixed, floating-point and posit. Preliminary results demonstrate that 8-bit posit has better accuracy than 8-bit fixed or floating-point for three different low-dimensional datasets. Moreover, the accuracy is comparable to 32-bit floating-point on a Xilinx Virtex-7 FPGA device. The trade-offs between DNN performance and hardware resources, i.e. latency, power, and resource utilization, show that posit outperforms in accuracy and latency at 8-bit and below.