Scholar Feed

KV-cache compression

TurboQuant

TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate

Superseded baseline#11 of 234 most-superseded · first seen Apr 28, 2025

Superseded — cited as a baseline and beaten by newer methods

7 papers critique it · 4 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites TurboQuant as a baseline.

per-vector methods treat each vector independently, ignoring the structured nature of the KV cache. Within an attention head, a block of n consecutive key or value vectors is not a collection of independent random vectors—it contains a low-rank component reflecting shared structure across tokens. This shared structure means the quantizer's theoretical assumptions (isotropy on the unit sphere) are not fully satisfied, leading to inner product bias.
— eOptShrinkQ: Near-Lossless KV Cache Compression Through Optimal Spectral Denoising and Quantization
“The main drawback is cost. For a head dimension d, a dense orthogonal transform requires O(d²) parameters and arithmetic, which is difficult to justify in latency-sensitive settings such as autoregressive decoding.”
— IsoQuant: Hardware-Aligned SO(4) Isoclinic Rotations for LLM KV Cache Compression
“But it is the wrong geometry. Once a vector has been normalized and Haar-rotated, a block of k consecutive coordinates lies on the unit ball with a specific radial law and a uniform angular component. The coordinates are not an independent product of shifted-Beta marginals. A scalar code sees one coordinate at a time; the source seen by the cache is intrinsically vectorial.”
— FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression
“Based on our experimental results, InnerQ achieves a comparable evaluation score to TurboQuant (Section~sec:accuracy) while having a lower latency (Section~sec:speedup).”
— InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models
“TurboQuant's lower bound is tight---for the problem it solves. That problem is: given an isolated KV vector drawn from the post-rotation distribution, what is the minimum number of bits needed to represent it? The paper's answer is approximately 3 bits per component, and TurboQuant achieves it. But the KV cache is not a collection of isolated vectors.”
— Sequential KV Cache Compression via Probabilistic Language Tries: Beyond the Per-Vector Shannon Limit
“On Mistral-7B, TurboAngle with $n = 64$ (3.0 angle bits) achieves $ = {+}0.0010$, while TurboQuant sym4-g4 at 4.0 bits degrades by ${+}0.0148$: $14.8$ more distortion at a higher bit rate.”
— TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
“All three quantize one coordinate (or one angle) at a time.”
— OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization

Beaten on benchmarks

Head-to-head results where a newer method reports beating TurboQuant. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.