Technical Report: Activation Residual Hessian Quantization (ARHQ) for Low-Bit LLM Quantization
For LLM practitioners, ARHQ offers a method to reduce error propagation in low-bit activation-weight quantization, though it is tested on a single model and benchmark.
ARHQ introduces a post-training weight splitting method that uses activation quantization residuals to construct a residual Hessian, isolating error-sensitive weight directions into a high-precision low-rank branch. On Qwen3-4B, it improves layer-wise SNR and preserves reasoning performance on ZebraLogic under aggressive quantization.
We present Activation Residual Hessian Quantization (ARHQ), a post-training weight splitting method designed to mitigate error propagation in low-bit activation-weight quantization. By constructing an input-side residual Hessian from activation quantization residuals (G_x), ARHQ analytically identifies and isolates error-sensitive weight directions into a high-precision low-rank branch. This is achieved via a closed-form truncated SVD on the scaled weight matrix W G^{1/2}_x . Experimental results on Qwen3-4B-Thinking-2507 demonstrate that ARHQ significantly improves layer-wise SNR and preserves downstream reasoning performance on ZebraLogic even under aggressive quantization. The code is available at https://github.com/BeautMoonQ/ARHQ.