LLM quantization

DuQuant

DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs

Superseded baseline#39 of 80 most-superseded · first seen Jun 3, 2024

Superseded — cited as a baseline and beaten by newer methods

1 papers critique it · 1 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites DuQuant as a baseline.

lin2024duquant manually design a zigzag permutation pattern in an attempt to distribute outlier weights throughout the network.
Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.