HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration
This work addresses the problem of inefficient and thermally limited hardware for transformer acceleration, which is crucial for AI researchers and engineers, and presents a novel architecture with significant performance gains.
The paper tackles the challenge of designing hardware accelerators for transformer models, which are difficult due to diverse computing kernels and thermal limitations, by proposing HeTraX, a 3D heterogeneous architecture that achieves up to 5.6x speedup and 14.5x improvement in energy-delay product while ensuring thermal feasibility.
Transformers have revolutionized deep learning and generative modeling to enable unprecedented advancements in natural language processing tasks and beyond. However, designing hardware accelerators for executing transformer models is challenging due to the wide variety of computing kernels involved in the transformer architecture. Existing accelerators are either inadequate to accelerate end-to-end transformer models or suffer notable thermal limitations. In this paper, we propose the design of a three-dimensional heterogeneous architecture referred to as HeTraX specifically optimized to accelerate end-to-end transformer models. HeTraX employs hardware resources aligned with the computational kernels of transformers and optimizes both performance and energy. Experimental results show that HeTraX outperforms existing state-of-the-art by up to 5.6x in speedup and improves EDP by 14.5x while ensuring thermally feasibility.