fpgaHART: A toolflow for throughput-oriented acceleration of 3D CNNs for HAR onto FPGAs
This work addresses resource constraints in systems like surveillance and autonomous vehicles, but it is incremental as it builds on existing FPGA acceleration methods.
The study tackled the high computational and memory requirements of 3D CNNs for Human Action Recognition by proposing a toolflow that optimizes their mapping onto FPGAs, resulting in competitive performance compared to earlier designs.
Surveillance systems, autonomous vehicles, human monitoring systems, and video retrieval are just few of the many applications in which 3D Convolutional Neural Networks are exploited. However, their extensive use is restricted by their high computational and memory requirements, especially when integrated into systems with limited resources. This study proposes a toolflow that optimises the mapping of 3D CNN models for Human Action Recognition onto FPGA devices, taking into account FPGA resources and off-chip memory characteristics. The proposed system employs Synchronous Dataflow (SDF) graphs to model the designs and introduces transformations to expand and explore the design space, resulting in high-throughput designs. A variety of 3D CNN models were evaluated using the proposed toolflow on multiple FPGA devices, demonstrating its potential to deliver competitive performance compared to earlier hand-tuned and model-specific designs.