Scholar Feed

KV-cache compression

ThinK

ThinK: Thinner Key Cache by Query-Driven Pruning

Superseded baseline#29 of 234 most-superseded · first seen Jul 30, 2024

Superseded — cited as a baseline and beaten by newer methods

2 papers critique it · 4 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites ThinK as a baseline.

previous works on KV cache pruning have been limited to structured pruning, primarily due to the difficulty of efficiently leveraging finer-grained (i.e., unstructured) sparsity during execution.
— Mustafar: Promoting Unstructured Sparsity for KV Cache Pruning in LLM Inference
“direct truncation of the original channels, as exemplified by ThinK, leads to significant performance degradation when pursuing high compression ratios.”
— SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention

Beaten on benchmarks

Head-to-head results where a newer method reports beating ThinK. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.