Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos
It addresses the challenge of real-time dynamic scene reconstruction for applications like virtual reality or film, though it appears incremental as it builds on existing feed-forward and 3D Gaussian Splatting methods.
The paper tackles the problem of reconstructing dynamic scenes from monocular videos for novel view synthesis, achieving state-of-the-art performance with reconstruction times under 150ms.
Recent advancements in static feed-forward scene reconstruction have demonstrated significant progress in high-quality novel view synthesis. However, these models often struggle with generalizability across diverse environments and fail to effectively handle dynamic content. We present BTimer (short for BulletTimer), the first motion-aware feed-forward model for real-time reconstruction and novel view synthesis of dynamic scenes. Our approach reconstructs the full scene in a 3D Gaussian Splatting representation at a given target ('bullet') timestamp by aggregating information from all the context frames. Such a formulation allows BTimer to gain scalability and generalization by leveraging both static and dynamic scene datasets. Given a casual monocular dynamic video, BTimer reconstructs a bullet-time scene within 150ms while reaching state-of-the-art performance on both static and dynamic scene datasets, even compared with optimization-based approaches.