Neural Radiance Flow for 4D View Synthesis and Video Processing
This work provides a novel method for high-quality 4D view synthesis and video processing for researchers and practitioners working with dynamic scene reconstruction and manipulation.
This paper introduces Neural Radiance Flow (NeRFlow), a 4D spatial-temporal representation learned from RGB images, capable of capturing 3D occupancy, radiance, and dynamics of a scene. It achieves state-of-the-art performance in multi-view rendering for dynamic scenes and can also be used for video processing tasks like super-resolution and de-noising without extra supervision.
We present a method, Neural Radiance Flow (NeRFlow),to learn a 4D spatial-temporal representation of a dynamic scene from a set of RGB images. Key to our approach is the use of a neural implicit representation that learns to capture the 3D occupancy, radiance, and dynamics of the scene. By enforcing consistency across different modalities, our representation enables multi-view rendering in diverse dynamic scenes, including water pouring, robotic interaction, and real images, outperforming state-of-the-art methods for spatial-temporal view synthesis. Our approach works even when inputs images are captured with only one camera. We further demonstrate that the learned representation can serve as an implicit scene prior, enabling video processing tasks such as image super-resolution and de-noising without any additional supervision.