Integration of a systolic array based hardware accelerator into a DNN operator auto-tuning framework
This work addresses the challenge of deploying neural networks on heterogeneous SoCs with custom accelerators by providing an automated tool for optimizing performance, though it is incremental as it builds on existing frameworks and accelerators.
The authors integrated the TVM auto-tuning framework with the Gemmini systolic array accelerator to automatically generate efficient code for GEMM operations on FPGAs, achieving a peak throughput of 46 GOPs at 100 MHz and outperforming hand-tuned schedules in real-world workloads.
The deployment of neural networks on heterogeneous SoCs coupled with custom accelerators is a challenging task because of the lack of end-to-end software tools provided for these systems. Moreover, the already available low level schedules and mapping strategies provided by the accelerator developers for typical tensor operations are not necessarily the best possible ones for each particular use case. This is why frameworks which automatically test the performance of the generated code on a specific hardware configuration are of special interest. In this work, the integration between the code generation framework TVM and the systolic array-based accelerator Gemmini is presented. A generic schedule to offload the GEneral Matrix Multiply (GEMM) tensor operation onto Gemmini is detailed, and its suitability is tested by executing the AutoTVM tuning process on it. Our generated code achieves a peak throughput of 46 giga-operations per second (GOPs) under a 100 MHz clock on a Xilinx ZCU102 FPGA, outperforming previous work. Furthermore, the code generated by this integration was able to surpass the default hand-tuned schedules provided by the Gemmini developers in real-world workloads.