Scholar Feed

Long-context / context-window extension

StreamingLLM

Efficient Streaming Language Models with Attention Sinks

Heavily superseded#5 of 53 most-superseded · first seen Sep 29, 2023

Heavily superseded — a standard baseline that newer methods routinely beat

3 papers critique it · 7 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites StreamingLLM as a baseline.

While these methods differ in selecting tokens for KV cache retention, they generally apply a uniform budget size across layers, even though the optimal budget size may vary.
— ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty
“The sparse attention method StreamingLLM, based on fixed sparse patterns, can guarantee some of the model's capabilities, but due to discarding a large amount of long-context information, it performs poorly on retrieval-related tasks (R.PK, R.Num, R.KV).”
— TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
“StreamingLLM~xiao2023efficient prioritizes continuous generation but compromises accuracy on long-context tasks.”
— LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models

Beaten on benchmarks

Head-to-head results where a newer method reports beating StreamingLLM. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.