BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling
This addresses the challenge of under-constrained dynamic scene reconstruction for applications like novel view synthesis, though it builds incrementally on existing NeRF and NRSfM methods.
The paper tackles the problem of modeling dynamic 3D scenes from a single moving camera by bridging non-rigid-structure-from-motion with neural radiance fields, achieving compelling results across complex scenes with lighting and texture changes.
Reasoning the 3D structure of a non-rigid dynamic scene from a single moving camera is an under-constrained problem. Inspired by the remarkable progress of neural radiance fields (NeRFs) in photo-realistic novel view synthesis of static scenes, extensions have been proposed for dynamic settings. These methods heavily rely on neural priors in order to regularize the problem. In this work, we take a step back and reinvestigate how current implementations may entail deleterious effects, including limited expressiveness, entanglement of light and density fields, and sub-optimal motion localization. As a remedy, we advocate for a bridge between classic non-rigid-structure-from-motion (\nrsfm) and NeRF, enabling the well-studied priors of the former to constrain the latter. To this end, we propose a framework that factorizes time and space by formulating a scene as a composition of bandlimited, high-dimensional signals. We demonstrate compelling results across complex dynamic scenes that involve changes in lighting, texture and long-range dynamics.