Long-context / context-window extension
PyramidKV
PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
Superseded — cited as a baseline and beaten by newer methods
1 papers critique it · 2 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites PyramidKV as a baseline.
While these methods differ in selecting tokens for KV cache retention, they generally apply a uniform budget size across layers, even though the optimal budget size may vary.
Beaten on benchmarks
Head-to-head results where a newer method reports beating PyramidKV. Values are copied from the source paper's tables — verify against the cited paper.
FreqKV beats PyramidKV
47.30 vs 0.10
Avg. · [LLaMA-3, 16K evaluation length]
FreqKV: Frequency Domain Key-Value Compression for Efficient Context Window ExtensionZigZagKV beats PyramidKV
0.226 vs 0.424
Attention loss · [LLaMA Budget 512]
ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty
What to use instead
Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.