Dynamic View Synthesis from Dynamic Monocular Video
This work addresses dynamic view synthesis for applications in computer vision and graphics, representing an incremental improvement by building on existing neural implicit representations.
The paper tackles the problem of generating novel views from a monocular video of a dynamic scene, achieving results through a method that combines static and dynamic neural radiance fields with regularization to resolve ambiguities.
We present an algorithm for generating novel views at arbitrary viewpoints and any input time step given a monocular video of a dynamic scene. Our work builds upon recent advances in neural implicit representation and uses continuous and differentiable functions for modeling the time-varying structure and the appearance of the scene. We jointly train a time-invariant static NeRF and a time-varying dynamic NeRF, and learn how to blend the results in an unsupervised manner. However, learning this implicit function from a single video is highly ill-posed (with infinitely many solutions that match the input video). To resolve the ambiguity, we introduce regularization losses to encourage a more physically plausible solution. We show extensive quantitative and qualitative results of dynamic view synthesis from casually captured videos.