Scholar Feed

Long-context / context-window extension

LongRoPE

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens

Superseded baseline#19 of 53 most-superseded · first seen Feb 21, 2024

Superseded — cited as a baseline and beaten by newer methods

3 papers critique it · 1 beat it on benchmarks

What papers say

Verbatim critique sentences, each from a paper that cites LongRoPE as a baseline.

traditional approaches chen2023extending often suffer from a significant performance drop chen2023clex, ding2024longrope at the target length due to their limited generalization capability.
— DCIS: Efficient Length Extrapolation of LLMs via Divide-and-Conquer Scaling Factor Search
“rescaling factors derived from previous methods often fall short of achieving the effective target context length.”
— LongRoPE2: Near-Lossless LLM Context Window Scaling
“due to the exponential search space complexity, it is challenging for those methods to estimate an optimal frequency; they also need heavy searching cost, for instance, it costs LongRoPE nearly 3 days to search an optimal frequency for a 256k context window using an A100 GPU”
— PSC: Extending Context Window of Large Language Models via Phase Shift Calibration

Beaten on benchmarks

Head-to-head results where a newer method reports beating LongRoPE. Values are copied from the source paper's tables — verify against the cited paper.

What to use instead

Recent methods in the same sub-problem, not yet superseded in the knowledge base — arXiv benchmark leaders, not vetted production recommendations.