Scholar Feed

KV-cache compression

Palu

Palu: Compressing KV-Cache with Low-Rank Projection

Superseded baseline#13 of 234 most-superseded · first seen Jul 30, 2024

Superseded — cited as a baseline and beaten by newer methods

4 papers critique it · 6 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites Palu as a baseline.

Palu~chang2024palucompressingkvcachelowrank and ReCalKV~yan2025recalkv factorize the model weights into low-rank matrices, cache compressed intermediate states, and reconstruct the full key and value tensors during attention. However, these methods often incur noticeable accuracy degradation due to lossy factorization.
— OjaKV: Context-Aware Online Low-Rank KV Cache Compression with Oja's Rule
“However, as Palu~chang2024palu points out, this will greatly introduce additional computation for recovering the key vectors.”
— SALS: Sparse Attention in Latent Space for KV cache Compression
“A key limitation is that reconstruction error is only an indirect proxy for attention and downstream layer behavior, and accuracy can degrade more sharply at higher compression.”
— Don't be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifold
“However, this approach targets only the projection weights, while prior work yu2023compressing has shown that transformer weights typically have higher rank than the output features (keys/values), suggesting that data-dependent KV-cache compression is more effective.”
— KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs

Beaten on benchmarks

Head-to-head results where a newer method reports beating Palu. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.