Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training
For practitioners deploying low-bit quantized LLMs, this work provides a practical diagnosis and mitigation of subtle QAT failures that are invisible to training loss.
The paper identifies two failure modes in HiF8 W8A8 QAT for LLMs—amax saturation and catastrophic forgetting—and proposes fixes (conservative max-algorithm DTS and BF16 warmup) that achieve <0.6% accuracy drop on MMLU, HellaSwag, and ARC-Challenge versus BF16 baseline.
Quantization-aware training (QAT) with low-bit floating-point formats enables efficient LLM deployment, yet introduces subtle failure modes invisible to standard training metrics. We present a systematic study of HiF8 W8A8 QAT for OpenPangu-Embedded-1B through the lens of Delayed Tensor Scaling (DTS). Across eight controlled experiments, we identify and disentangle two orthogonal failure modes: (i)amax saturation, where delayed scale estimates silently corrupt knowledge-sensitive representations via forward-pass clipping, and (ii)catastrophic forgetting, where an aggressive learning rate overwrites pretrained commonsense knowledge independently of quantization. Neither is detectable from training loss alone. We address amax saturation with a conservative max-algorithm DTS strategy over a 64-step history window, and mitigate forgetting via a 500-step BF16 warmup followed by QAT at lr=10^{-5}. Both fixes are necessary and sufficient: our final configuration achieves 0.43% MMLU drop, 0.58% HellaSwag drop, and 0.22% ARC-Challenge drop versus a matched BF16 baseline, with a training loss APE of only 0.11% over 10,000 steps.