TOVA (KV-cache compression): superseded — cited as a baseline and beaten by newer methods. 6 paper(s) critique it, 14 beat it on benchmarks — #7 of 234 most-superseded. Sub-problem: cluster led by SnapKV. Newer alternatives in the same sub-problem include STaR-KV, GRKV, MomentKV, NestedKV, IndexMem.

Is TOVA superseded? Critiques, benchmarks & alternatives

What papers say

Verbatim critique sentences, each from a paper that cites TOVA as a baseline.

these methods require access to the full attention matrix, making them incompatible with Flash Attention~flashattention and thus impractical for modern deployment scenarios

— Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution

“They fix the budget of KV Cache in a finite level, but don't distinguish the differences between layers and between heads.”

— LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation

“These methods, however, often overlook the structure of key information distribution by naively evicting tokens across the entire sequence.”

— TreeKV: Smooth Key-Value Cache Compression with Tree Structures

“While effective, most methods either discard unused tokens too early or require full cache for scoring.”

— PiKV: KV Cache Management System for Mixture of Experts

“However, these methods rely primarily on attention weights and often overlook the contribution of value states in shaping the final model outputs.”

— OBCache: Optimal Brain KV Cache Pruning for Efficient Long-Context LLM Inference

“TOVA~oren2024tova retains attention sinks and a sliding window of recent tokens; a credential at relative depth 0.5 sits 2,000 tokens outside the window.”

— Transactional Attention: Semantic Sponsorship for KV-Cache Retention

Beaten on benchmarks

Head-to-head results where a newer method reports beating TOVA. Values are copied from the source paper's tables — verify against the cited paper.

KVTC beats TOVA
99.3 vs 0.3
LITM · [MN-Minitron 8B]
KV Cache Transform Coding for Compact Storage in LLM Inference
Compactor beats TOVA
38.8 vs 30.9
Total · [Llama 3.1 at 10% KV retention]
Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores
CAKE beats TOVA
29.29 vs 25.31
Avg. · [Llama2-7B-Chat, $B_{\text{total}}=128L$]
CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences
DMS beats TOVA
53.3 vs 46.7
accuracy · [CR4, AIME 24, 7B]
Inference-Time Hyper-Scaling with KV Cache Compression
AhaKV beats TOVA
41.84 vs 37.99
Average · [Qwen2-7B-Inst]
AhaKV: Adaptive Holistic Attention-Driven KV Cache Eviction for Efficient Inference of Large Language Models
KeyDiff beats TOVA
49.19 vs 45.31
Avg. · [Llama3.1-8B at 6K cache budget]
KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments
WeightedKV beats TOVA
4.30 vs 4.69
Perplexity · [Cache Size = 256]
WeightedKV: Attention Scores Weighted Key-Value Cache Merging for Large Language Models
Expected Attention beats TOVA
50.25 vs 48.14
score · [Qwen, Longbench, 25% compression]
Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
TA beats TOVA
30.4 vs 31.2
Perplexity · [K=16]
Transactional Attention: Semantic Sponsorship for KV-Cache Retention
OBC+TOVA beats TOVA
45.95 vs 45.01
Avg. · [10% KV (q-Aware)]
OBCache: Optimal Brain KV Cache Pruning for Efficient Long-Context LLM Inference
TOVA+CAOTE beats TOVA
38.08 vs 37.52
Avg accuracy · [Llama 3.1-8B, 2k budget]
CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Eviction
TA-Rgx beats TOVA
100 vs 0
Accuracy · [Llama-3.2-1B, K=16]
Transactional Attention: Semantic Sponsorship for KV-Cache Retention

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.