Point-DynRF: Point-based Dynamic Radiance Fields from a Monocular Video
This work addresses a specific challenge in computer vision for 3D scene reconstruction from videos, representing an incremental improvement over existing methods.
The paper tackles the problem of generating novel views from monocular videos using dynamic radiance fields, which previously suffered from geometric inconsistencies at distant viewpoints, by introducing Point-DynRF, a framework that integrates neural point clouds and dynamic radiance fields, achieving improved performance validated on datasets like the NVIDIA Dynamic Scenes Dataset.
Dynamic radiance fields have emerged as a promising approach for generating novel views from a monocular video. However, previous methods enforce the geometric consistency to dynamic radiance fields only between adjacent input frames, making it difficult to represent the global scene geometry and degenerates at the viewpoint that is spatio-temporally distant from the input camera trajectory. To solve this problem, we introduce point-based dynamic radiance fields (\textbf{Point-DynRF}), a novel framework where the global geometric information and the volume rendering process are trained by neural point clouds and dynamic radiance fields, respectively. Specifically, we reconstruct neural point clouds directly from geometric proxies and optimize both radiance fields and the geometric proxies using our proposed losses, allowing them to complement each other. We validate the effectiveness of our method with experiments on the NVIDIA Dynamic Scenes Dataset and several causally captured monocular video clips.