Scholar Feed

LLM quantization

QServe

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

Superseded baseline#47 of 80 most-superseded · first seen May 7, 2024

Cited as a baseline — critiqued by newer work, not yet beaten on a benchmark here

2 papers critique it · 0 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites QServe as a baseline.

QServe~Qserve concludes that W4A4 cannot deliver speedup on Ampere and retreats to W4A8.
— APEX4: Efficient Pure W4A4 LLM Inference via Intra-SM Compute Rebalancing
“the state-of-the-art W4A8 GEMM implementation~lin2024qserve fails to meet expectations: it does not outperform higher-precision methods like W8A8 in memory-bound scenarios and is significantly slower than W8A8 and even FP16 in compute-bound regimes”
— LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.