Scholar Feed

LLM quantization

QuaRot

QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs

Heavily superseded#4 of 80 most-superseded · first seen Mar 30, 2024

Heavily superseded — a standard baseline that newer methods routinely beat

7 papers critique it · 12 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites QuaRot as a baseline.

QuaRot reporting an accuracy loss of approximately 3.5 points
— OSC: Hardware Efficient W4A4 Quantization via Outlier Separation in Channel Dimension
“Interestingly, we can prove analytically and show empirically that rotations improve MXFP4 accuracy, but hurt NVFP4 accuracy when coupled with standard Round-to-Nearest (RTN) quantization.”
— Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization
“Yet, these rotations introduce quadratic complexity, which offsets the potential acceleration.”
— ConvRot: Rotation-Based Plug-and-Play 4-bit Quantization for Diffusion Transformers
“QuaRot, SpinQuant, and ButterflyQuant do not engage with directly [the regime of per-head q_norm/RoPE compatibility failures]”
— Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
“However, these methods operate primarily along the feature dimension and ignore correlations across the sequence dimension.”
— STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization
“QuaRot fails on Qwen models smaller than 14B, suggesting that naive rotation alone is insufficient to suppress quantization error in the presence of severe outliers in small models”
— OffQ: Taming Structured Outliers in LLM Quantization by Offsetting
“However, these predetermined rotations cannot adapt to specific models.”
— ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms

Beaten on benchmarks

Head-to-head results where a newer method reports beating QuaRot. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.