LLM quantization
AQ-SGD
Superseded baseline#27 of 80 most-superseded
Superseded — cited as a baseline and beaten by newer methods
2 papers critique it · 1 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites AQ-SGD as a baseline.
AQ-SGD requires storing previous activations for the whole dataset to compute these changes, resulting in substantial memory overhead. Such an approach poses practical limitations, especially in resource-constrained environments for the large-volumes of training data where storage capacity and system complexity are critical considerations.
“Existing strategies to reduce communication overhead, such as activation quantization wang2023finetuninglanguagemodelsslow, lin2022lglsqlearnedgradientlinear, wu2023estimatormeetsequilibriumperspective, chen2024channel, yang2024gwq, typically use static precision and thus fail to adapt to flexible bit-width throughout training.”
Beaten on benchmarks
Head-to-head results where a newer method reports beating AQ-SGD. Values are copied from the source paper's tables — verify against the cited paper.
AMAQ beats AQ-SGD
1.609 vs 1.919
PPL · [Qwen2.5-14B, Code-Alpaca]
AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
What to use instead
Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.