RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction
This addresses dynamic scene reconstruction for real-time applications like robotics or AR/VR, representing an incremental improvement over existing streaming methods.
The paper tackles the problem of dynamic scene reconstruction from RGB images by proposing RayMap3R, a training-free streaming framework that identifies and suppresses moving objects to reduce artifacts and drift. It achieves state-of-the-art performance among streaming approaches on multiple benchmarks.
Streaming feed-forward 3D reconstruction enables real-time joint estimation of scene geometry and camera poses from RGB images. However, without explicit dynamic reasoning, streaming models can be affected by moving objects, causing artifacts and drift. In this work, we propose RayMap3R, a training-free streaming framework for dynamic scene reconstruction. We observe that RayMap-based predictions exhibit a static-scene bias, providing an internal cue for dynamic identification. Based on this observation, we construct a dual-branch inference scheme that identifies dynamic regions by contrasting RayMap and image predictions, suppressing their interference during memory updates. We further introduce reset metric alignment and state-aware smoothing to preserve metric consistency and stabilize predicted trajectories. Our method achieves state-of-the-art performance among streaming approaches on dynamic scene reconstruction across multiple benchmarks.