Scholar Feed

KV-cache compression

DuoAttention

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads

Superseded baseline#17 of 234 most-superseded · first seen Oct 14, 2024

Superseded — cited as a baseline and beaten by newer methods

4 papers critique it · 5 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites DuoAttention as a baseline.

these methods cannot capture the reasoning behaviors that emerge during dynamically extending CoT generation, as their static heuristics or teacher-forced objectives miss how compression errors accumulate along the autoregressive trajectory
— Which Heads Matter for Reasoning? RL-Guided KV Cache Compression
“Notably, our method replaces DuoAttention's head-score optimization, which originally requires tens of GPU hours, with only a few forward passes completed within a minute”
— KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction
“However, DuoAttention requires an optimization-based offline procedure to classify the heads using synthetic datasets, thus, introducing a computational overhead. In addition, its coarse granularity and reliance on stable head roles limit adaptability across tasks and domains.”
— KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
“their fixed nature overlooks dynamic patterns during inference, leading to significant accuracy losses”
— FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference

Beaten on benchmarks

Head-to-head results where a newer method reports beating DuoAttention. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.