An FPGA-Based Accelerator Enabling Efficient Support for CNNs with Arbitrary Kernel Sizes
This addresses efficiency challenges for deploying large-kernel CNNs in vision applications, representing an incremental hardware optimization.
The paper tackles the computational efficiency degradation in deploying CNNs with arbitrary kernel sizes by proposing an FPGA-based accelerator, achieving up to 3.91× better DSP efficiency than prior art and throughputs of 169.68 GOPS for RepLKNet-31 and 244.55 GOPS for PyConvResNet-50.
Convolutional neural networks (CNNs) with large kernels, drawing inspiration from the key operations of vision transformers (ViTs), have demonstrated impressive performance in various vision-based applications. To address the issue of computational efficiency degradation in existing designs for supporting large-kernel convolutions, an FPGA-based inference accelerator is proposed for the efficient deployment of CNNs with arbitrary kernel sizes. Firstly, a Z-flow method is presented to optimize the computing data flow by maximizing data reuse opportunity. Besides, the proposed design, incorporating the kernel-segmentation (Kseg) scheme, enables extended support for large-kernel convolutions, significantly reducing the storage requirements for overlapped data. Moreover, based on the analysis of typical block structures in emerging CNNs, vertical-fused (VF) and horizontal-fused (HF) methods are developed to optimize CNN deployments from both computation and transmission perspectives. The proposed hardware accelerator, evaluated on Intel Arria 10 FPGA, achieves up to 3.91 times better DSP efficiency than prior art on the same network. Particularly, it demonstrates efficient support for large-kernel CNNs, achieving throughputs of 169.68 GOPS and 244.55 GOPS for RepLKNet-31 and PyConvResNet-50, respectively, both of which are implemented on hardware for the first time.