SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization
For practitioners deploying large language models, SOAR provides a more accurate 4-bit quantization method that reduces memory footprint without sacrificing performance.
SOAR introduces a post-training quantization framework for NVFP4 format that jointly optimizes global and block-wise scales via closed-form solutions and decouples quantization/dequantization scales through discrete search, consistently outperforming existing NVFP4 baselines across multiple LLMs with no additional hardware overhead.
NVFP4 has recently emerged as an efficient 4-bit microscaling format for large language models (LLMs), offering superior numerical fidelity with native hardware support. However, existing methods often yield suboptimal performance due to inflexible scale selection and the coupled treatment of quantization and dequantization scales. To address these issues, we propose Scale Optimization for Accurate Reconstruction (SOAR), a novel post-training quantization framework that improves the accuracy of NVFP4 quantization. At its core, SOAR features Closed-form Joint Scale Optimization (CJSO), which jointly optimizes global and block-wise scales via analytical solutions derived from reconstruction error minimization. Furthermore, it incorporates Decoupled Scale Search (DSS). DSS decouples the high-precision quantization scale from its constrained dequantization counterpart, and performs discrete search to mitigate precision loss from scale quantization. Extensive experiments across multiple LLMs show that our method consistently outperforms existing NVFP4 quantization baselines, achieving superior accuracy under the same memory footprint with no additional hardware overhead. The code and models will be available at https://github.com/steven-bao1/SOAR.