Efficient Edge AI: Deploying Convolutional Neural Networks on FPGA with the Gemmini Accelerator
This work addresses energy consumption and privacy issues for edge AI deployment, offering an incremental improvement in FPGA-based CNN implementation.
The paper tackles the challenge of deploying Convolutional Neural Networks on edge devices like FPGAs by presenting an end-to-end workflow using the modified Gemmini accelerator, achieving real-time performance with a YOLOv7 model at 36.5 GOP/s/W energy efficiency and superior power efficiency compared to other embedded and FPGA implementations.
The growing concerns regarding energy consumption and privacy have prompted the development of AI solutions deployable on the edge, circumventing the substantial CO2 emissions associated with cloud servers and mitigating risks related to sharing sensitive data. But deploying Convolutional Neural Networks (CNNs) on non-off-the-shelf edge devices remains a complex and labor-intensive task. In this paper, we present and end-to-end workflow for deployment of CNNs on Field Programmable Gate Arrays (FPGAs) using the Gemmini accelerator, which we modified for efficient implementation on FPGAs. We describe how we leverage the use of open source software on each optimization step of the deployment process, the customizations we added to them and its impact on the final system's performance. We were able to achieve real-time performance by deploying a YOLOv7 model on a Xilinx ZCU102 FPGA with an energy efficiency of 36.5 GOP/s/W. Our FPGA-based solution demonstrates superior power efficiency compared with other embedded hardware devices, and even outperforms other FPGA reference implementations. Finally, we present how this kind of solution can be integrated into a wider system, by testing our proposed platform in a traffic monitoring scenario.