An ultra-low-power CGRA for accelerating Transformers at the edge
This work addresses the problem of high computational demands for transformers on edge devices, offering a scalable solution for deploying machine learning models in resource-constrained environments, though it is incremental in optimizing existing hardware methods.
The paper tackled the challenge of deploying transformer models on low-power edge devices by introducing an ultra-low-power CGRA architecture, resulting in a design that reduces memory bandwidth demands and enhances data reuse for efficient parallel computation.
Transformers have revolutionized deep learning with applications in natural language processing, computer vision, and beyond. However, their computational demands make it challenging to deploy them on low-power edge devices. This paper introduces an ultra-low-power, Coarse-Grained Reconfigurable Array (CGRA) architecture specifically designed to accelerate General Matrix Multiplication (GEMM) operations in transformer models tailored for the energy and resource constraints of edge applications. The proposed architecture integrates a 4 x 4 array of Processing Elements (PEs) for efficient parallel computation and dedicated 4 x 2 Memory Operation Blocks (MOBs) for optimized LOAD/STORE operations, reducing memory bandwidth demands and enhancing data reuse. A switchless mesh torus interconnect network further minimizes power and latency by enabling direct communication between PEs and MOBs, eliminating the need for centralized switching. Through its heterogeneous array design and efficient dataflow, this CGRA architecture addresses the unique computational needs of transformers, offering a scalable pathway to deploy sophisticated machine learning models on edge devices.