ImmersiveNeRF: Hybrid Radiance Fields for Unbounded Immersive Light Field Reconstruction
This work addresses the challenge of creating immersive AR/VR experiences by enabling larger space 6DoF rendering, though it is incremental as it builds on existing radiance field methods.
The paper tackles the problem of unbounded immersive light field reconstruction by proposing a hybrid radiance field representation that separates foreground and background with adaptive learning, resulting in high-quality rendering and aggressive view extrapolation, as demonstrated through extensive experiments.
This paper proposes a hybrid radiance field representation for unbounded immersive light field reconstruction which supports high-quality rendering and aggressive view extrapolation. The key idea is to first formally separate the foreground and the background and then adaptively balance learning of them during the training process. To fulfill this goal, we represent the foreground and background as two separate radiance fields with two different spatial mapping strategies. We further propose an adaptive sampling strategy and a segmentation regularizer for more clear segmentation and robust convergence. Finally, we contribute a novel immersive light field dataset, named THUImmersive, with the potential to achieve much larger space 6DoF immersive rendering effects compared with existing datasets, by capturing multiple neighboring viewpoints for the same scene, to stimulate the research and AR/VR applications in the immersive light field domain. Extensive experiments demonstrate the strong performance of our method for unbounded immersive light field reconstruction.