Long-context / context-window extension
Mamba
Superseded baseline#20 of 53 most-superseded
Superseded — cited as a baseline and beaten by newer methods
3 papers critique it · 1 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites Mamba as a baseline.
Mamba and RWKV maintain compact fixed-size states but struggle with fine-grained retrieval at long range
“we show, through a series of visualizations, analyses, and empirical measures, that the main barrier is Mamba's implicit bias towards sequence lengths that were seen during training, a phenomenon that we call `limited effective receptive field' (ERF).”
“these recurrent sequence methods are specifically designed as architectural alternatives to Transformers and cannot be directly applied to existing pre-trained LLMs in a plug-and-play manner, requiring models to be trained from scratch and thus limiting their adoption in the current LLM ecosystem”
Beaten on benchmarks
Head-to-head results where a newer method reports beating Mamba. Values are copied from the source paper's tables — verify against the cited paper.
+Deci beats Mamba
5.3 vs 0.3
Accuracy · [Retrieval with varying document count]
DeciMamba: Exploring the Length Extrapolation Potential of MambaDeciMamba beats Mamba
12.61 vs 3.93
LongBench Score · [instruction-tuned Mamba-2.8b zero-shot]
DeciMamba: Exploring the Length Extrapolation Potential of Mamba
What to use instead
Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.