GRAU: Generic Reconfigurable Activation Unit Design for Neural Network Hardware Accelerators
This work addresses hardware efficiency and scalability issues for edge accelerators in AI, though it appears incremental as it builds on existing quantization and activation techniques.
The paper tackles the high hardware cost of multi-threshold activation units in neural network accelerators by proposing GRAU, a reconfigurable design based on piecewise linear fitting with power-of-two slopes, which reduces LUT consumption by over 90% compared to existing methods.
With the continuous growth of neural network scales, low-precision quantization is widely used in edge accelerators. Classic multi-threshold activation hardware requires 2^n thresholds for n-bit outputs, causing a rapid increase in hardware cost as precision increases. We propose a reconfigurable activation hardware, GRAU, based on piecewise linear fitting, where the segment slopes are approximated by powers of two. Our design requires only basic comparators and 1-bit right shifters, supporting mixed-precision quantization and nonlinear functions such as SiLU. Compared with multi-threshold activators, GRAU reduces LUT consumption by over 90%, achieving higher hardware efficiency, flexibility, and scalability.