APSQ: Additive Partial Sum Quantization with Algorithm-Hardware Co-Design
This addresses power efficiency issues in DNN accelerators for AI hardware developers, though it is incremental as it builds on existing quantization and dataflow techniques.
The paper tackles the high memory and power demands from frequent access to high-precision partial sums in DNN accelerators by introducing Additive Partial Sum Quantization (APSQ), which compresses partial sums to INT8 with nearly lossless performance on NLP and CV tasks, reducing energy costs by 28-87%.
DNN accelerators, significantly advanced by model compression and specialized dataflow techniques, have marked considerable progress. However, the frequent access of high-precision partial sums (PSUMs) leads to excessive memory demands in architectures utilizing input/weight stationary dataflows. Traditional compression strategies have typically overlooked PSUM quantization, which may account for 69% of power consumption. This study introduces a novel Additive Partial Sum Quantization (APSQ) method, seamlessly integrating PSUM accumulation into the quantization framework. A grouping strategy that combines APSQ with PSUM quantization enhanced by a reconfigurable architecture is further proposed. The APSQ performs nearly lossless on NLP and CV tasks across BERT, Segformer, and EfficientViT models while compressing PSUMs to INT8. This leads to a notable reduction in energy costs by 28-87%. Extended experiments on LLaMA2-7B demonstrate the potential of APSQ for large language models. Code is available at https://github.com/Yonghao-Tan/APSQ.