Integer-only Quantized Transformers for Embedded FPGA-based Time-series Forecasting in AIoT
This work addresses the challenge of efficient on-device inference for time-series forecasting in AIoT systems, representing an incremental improvement in quantization and hardware optimization.
This paper tackles the problem of deploying Transformer models on embedded AIoT devices for time-series forecasting by designing a hardware accelerator with integer-only quantization, achieving a 4-bit quantized model that increases test loss by only 0.63% compared to 8-bit models while operating up to 132.33x faster and consuming 48.19x less energy.
This paper presents the design of a hardware accelerator for Transformers, optimized for on-device time-series forecasting in AIoT systems. It integrates integer-only quantization and Quantization-Aware Training with optimized hardware designs to realize 6-bit and 4-bit quantized Transformer models, which achieved precision comparable to 8-bit quantized models from related research. Utilizing a complete implementation on an embedded FPGA (Xilinx Spartan-7 XC7S15), we examine the feasibility of deploying Transformer models on embedded IoT devices. This includes a thorough analysis of achievable precision, resource utilization, timing, power, and energy consumption for on-device inference. Our results indicate that while sufficient performance can be attained, the optimization process is not trivial. For instance, reducing the quantization bitwidth does not consistently result in decreased latency or energy consumption, underscoring the necessity of systematically exploring various optimization combinations. Compared to an 8-bit quantized Transformer model in related studies, our 4-bit quantized Transformer model increases test loss by only 0.63%, operates up to 132.33x faster, and consumes 48.19x less energy. Relevant source code is provided in the accompanying GitHub repository\footnote{https://github.com/tianheng-ling/TinyTransformer4TS}.