Long-context / context-window extension
SnapKV
SnapKV: LLM Knows What You are Looking for Before Generation
Superseded — cited as a baseline and beaten by newer methods
2 papers critique it · 3 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites SnapKV as a baseline.
Although these methods generally have low additional overhead, they often lead to noticeable performance degradation.
“While these methods differ in selecting tokens for KV cache retention, they generally apply a uniform budget size across layers, even though the optimal budget size may vary.”
Beaten on benchmarks
Head-to-head results where a newer method reports beating SnapKV. Values are copied from the source paper's tables — verify against the cited paper.
FreqKV beats SnapKV
47.30 vs 0.10
Avg. · [LLaMA-3, 16K evaluation length]
FreqKV: Frequency Domain Key-Value Compression for Efficient Context Window ExtensionZigZagKV beats SnapKV
0.226 vs 0.463
Attention loss · [LLaMA Budget 512]
ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer UncertaintyA2ATS beats SnapKV
86.3 vs 67.6
Accuracy · [MegaBeam-Mistral-7B-512K, Sparsity ~0.062]
A$^2$ATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantization
What to use instead
Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.