Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures
This work addresses the problem of high compute and memory requirements for transformers in edge applications, offering a domain-specific hardware solution.
The paper tackles the challenge of deploying transformer models on edge devices by proposing Atleus, a 3D heterogeneous architecture optimized for fine-tuning and inference, achieving up to 56x performance and 64.5x energy efficiency improvements over state-of-the-art methods.
Transformer architectures have become the standard neural network model for various machine learning applications including natural language processing and computer vision. However, the compute and memory requirements introduced by transformer models make them challenging to adopt for edge applications. Furthermore, fine-tuning pre-trained transformers (e.g., foundation models) is a common task to enhance the model's predictive performance on specific tasks/applications. Existing transformer accelerators are oblivious to complexities introduced by fine-tuning. In this paper, we propose the design of a three-dimensional (3D) heterogeneous architecture referred to as Atleus that incorporates heterogeneous computing resources specifically optimized to accelerate transformer models for the dual purposes of fine-tuning and inference. Specifically, Atleus utilizes non-volatile memory and systolic array for accelerating transformer computational kernels using an integrated 3D platform. Moreover, we design a suitable NoC to achieve high performance and energy efficiency. Finally, Atleus adopts an effective quantization scheme to support model compression. Experimental results demonstrate that Atleus outperforms existing state-of-the-art by up to 56x and 64.5x in terms of performance and energy efficiency respectively