LongStream: Long-Sequence Streaming Autoregressive Visual Geometry
This addresses a significant open challenge in 3D reconstruction for applications requiring long-sequence processing, though it appears incremental in improving existing autoregressive approaches.
The paper tackles the problem of long-sequence streaming 3D reconstruction, where existing autoregressive models fail due to attention decay and scale drift, by introducing LongStream, which achieves state-of-the-art performance with stable, metric-scale reconstruction over kilometer-scale sequences at 18 FPS.
Long-sequence streaming 3D reconstruction remains a significant open challenge. Existing autoregressive models often fail when processing long sequences. They typically anchor poses to the first frame, which leads to attention decay, scale drift, and extrapolation errors. We introduce LongStream, a novel gauge-decoupled streaming visual geometry model for metric-scale scene reconstruction across thousands of frames. Our approach is threefold. First, we discard the first-frame anchor and predict keyframe-relative poses. This reformulates long-range extrapolation into a constant-difficulty local task. Second, we introduce orthogonal scale learning. This method fully disentangles geometry from scale estimation to suppress drift. Finally, we solve Transformer cache issues such as attention-sink reliance and long-term KV-cache contamination. We propose cache-consistent training combined with periodic cache refresh. This approach suppresses attention degradation over ultra-long sequences and reduces the gap between training and inference. Experiments show LongStream achieves state-of-the-art performance. It delivers stable, metric-scale reconstruction over kilometer-scale sequences at 18 FPS. Project Page: https://3dagentworld.github.io/longstream/