Long-context / context-window extension
CEPE
Superseded baseline#15 of 53 most-superseded
Superseded — cited as a baseline and beaten by newer methods
2 papers critique it · 2 beat it on benchmarks
What papers say
Verbatim critique sentences, each from a paper that cites CEPE as a baseline.
CEPE can process past context chunks in parallel, but these chunks must be passed through all its encoder layers (24-layer RoBERTa in CEPE) and layer-wise linear projections to obtain the final hidden states for cross-attention, leading to even slower inference speed than non-parallel Activation Beacon.
“However, this heterogeneous architecture necessitates meticulous task design for the extra pretraining and warmup stages to stabilize the fine-tuning process.”
Beaten on benchmarks
Head-to-head results where a newer method reports beating CEPE. Values are copied from the source paper's tables — verify against the cited paper.
SharedLLM beats CEPE
2.34 vs 2.43
Perplexity · [ProofPile 8K]
Stacked from One: Multi-Scale Self-Injection for Context Window Extension
What to use instead
Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.