A Study of Quantisation-aware Training on Time Series Transformer Models for Resource-constrained FPGAs
This work addresses model deployment efficiency for time series applications on FPGAs, but it is incremental as it builds on existing quantisation-aware training methods.
The study tackled the problem of quantising time series Transformer models for resource-constrained FPGAs by proposing an adaptive quantisation scheme that dynamically selects between symmetric and asymmetric schemes during training, resulting in reduced computational overhead while maintaining acceptable precision, with most objects quantised to 4 bits.
This study explores the quantisation-aware training (QAT) on time series Transformer models. We propose a novel adaptive quantisation scheme that dynamically selects between symmetric and asymmetric schemes during the QAT phase. Our approach demonstrates that matching the quantisation scheme to the real data distribution can reduce computational overhead while maintaining acceptable precision. Moreover, our approach is robust when applied to real-world data and mixed-precision quantisation, where most objects are quantised to 4 bits. Our findings inform model quantisation and deployment decisions while providing a foundation for advancing quantisation techniques.