GPTQ

“While GPTQ significantly reduces the local layer-wise MSE, its effect of reducing the global NLL loss is minimal given the same training data and same trainable weights.”

“as these techniques do not involve gradient-based optimization, unless task-specific calibration data is utilized, they can suffer substantial accuracy degradation on more challenging benchmarks, particularly text generation”

“existing PTQ approaches for LLMs minimize the layer-wise reconstruction loss while treating all tokens uniformly lin2023awq,frantar2022gptq,li2025gptqv2, without accounting for token-level informativeness or importance. Such a token-agnostic design inevitably biases the quantized model toward dominant but redundant visual features”

“However, under low-bit quantization, quantization errors from preceding layers accumulate across the network, making local-only basic reconstruction insufficient.”

“However, GPTQ and SmoothQuant~(SQ), which are strong PTQ methods for pure LLMs, do not reliably improve performance in this multimodal setting.”

“We further observe that generic PTQ methods such as GPTQ and AWQ suffer significant performance degradation under W4A16, highlighting the challenge of directly applying standard quantization techniques to VGGT.”

“the same routines collapse on sub-7B models where redundancy is scarce”

“GPTVQ accumulates quantization errors within vector quantization, leading to an inevitable increase in quantization errors as the vector length increases.”

“Despite its empirical success, the GPTQ algorithm was only presented as a sequence of greedily applied algebraic operations: the procedure picks one weight at a time, quantizes it via rounding or clipping, and then optimally updates the not-yet-quantized weights to correct for the remaining per-layer loss; it then continues with the next weight, and so on. This procedure leaves an obvious open question: why does a local greedy rule work so well globally? Current literature does not answer this question, leaving little guidance for principled extensions or failure case analysis.”

“GPTQ~frantar2022gptq poorly handles outliers due to calibration dependence”

“However, the weight updates are computed in a closed form based on second-order gradient information; this is done for each layer separately, which does not consider the dependencies among layers.”

“because GPTQ optimizes only for reconstruction accuracy, it can unintentionally increase group-targeting biases, which we aim to reduce with Fair-GPTQ”