SMOF: Streaming Modern CNNs on FPGAs with Smart Off-Chip Eviction
This addresses a bottleneck for FPGA-based hardware acceleration of modern CNNs in computer vision, enabling deployment on devices with constrained on-chip memory.
The paper tackled the challenge of efficiently mapping modern CNNs with long skip connections (e.g., UNet, YOLO) onto FPGAs with limited on-chip memory by introducing smart off-chip eviction mechanisms for weights and activations, achieving up to 10.65x throughput improvement compared to prior works.
Convolutional Neural Networks (CNNs) have demonstrated their effectiveness in numerous vision tasks. However, their high processing requirements necessitate efficient hardware acceleration to meet the application's performance targets. In the space of FPGAs, streaming-based dataflow architectures are often adopted by users, as significant performance gains can be achieved through layer-wise pipelining and reduced off-chip memory access by retaining data on-chip. However, modern topologies, such as the UNet, YOLO, and X3D models, utilise long skip connections, requiring significant on-chip storage and thus limiting the performance achieved by such system architectures. The paper addresses the above limitation by introducing weight and activation eviction mechanisms to off-chip memory along the computational pipeline, taking into account the available compute and memory resources. The proposed mechanism is incorporated into an existing toolflow, expanding the design space by utilising off-chip memory as a buffer. This enables the mapping of such modern CNNs to devices with limited on-chip memory, under the streaming architecture design approach. SMOF has demonstrated the capacity to deliver competitive and, in some cases, state-of-the-art performance across a spectrum of computer vision tasks, achieving up to 10.65 X throughput improvement compared to previous works.