wildNeRF: Complete view synthesis of in-the-wild dynamic scenes captured using sparse monocular data
This addresses the problem of efficient and high-quality 3D scene reconstruction for applications like robotics and augmented reality, representing a strong incremental improvement over existing methods.
The paper tackles novel-view synthesis of dynamic unstructured scenes from sparse monocular data, achieving state-of-the-art performance on benchmarks like the NVIDIA Dynamic Scenes Dataset with training times of seconds for static scenes and minutes for dynamic ones.
We present a novel neural radiance model that is trainable in a self-supervised manner for novel-view synthesis of dynamic unstructured scenes. Our end-to-end trainable algorithm learns highly complex, real-world static scenes within seconds and dynamic scenes with both rigid and non-rigid motion within minutes. By differentiating between static and motion-centric pixels, we create high-quality representations from a sparse set of images. We perform extensive qualitative and quantitative evaluation on existing benchmarks and set the state-of-the-art on performance measures on the challenging NVIDIA Dynamic Scenes Dataset. Additionally, we evaluate our model performance on challenging real-world datasets such as Cholec80 and SurgicalActions160.