Design Flow of Accelerating Hybrid Extremely Low Bit-width Neural Network in Embedded FPGA
This work addresses the challenge of deploying efficient neural networks on resource-constrained edge devices, though it appears incremental as it builds on existing low-bit quantization and FPGA acceleration techniques.
The authors tackled the problem of creating low-latency, low-energy neural network accelerators for edge computing by proposing a design flow for accelerating extremely low bit-width neural networks with hybrid quantization on embedded FPGAs. Their solution achieved up to 10.3 TOPS and 325.3 images/s/watt while consuming less than 5W, claiming it as the most energy-efficient compared to existing GPU or FPGA implementations.
Neural network accelerators with low latency and low energy consumption are desirable for edge computing. To create such accelerators, we propose a design flow for accelerating the extremely low bit-width neural network (ELB-NN) in embedded FPGAs with hybrid quantization schemes. This flow covers both network training and FPGA-based network deployment, which facilitates the design space exploration and simplifies the tradeoff between network accuracy and computation efficiency. Using this flow helps hardware designers to deliver a network accelerator in edge devices under strict resource and power constraints. We present the proposed flow by supporting hybrid ELB settings within a neural network. Results show that our design can deliver very high performance peaking at 10.3 TOPS and classify up to 325.3 image/s/watt while running large-scale neural networks for less than 5W using embedded FPGA. To the best of our knowledge, it is the most energy efficient solution in comparison to GPU or other FPGA implementations reported so far in the literature.