NeRF-Enhanced Outpainting for Faithful Field-of-View Extrapolation
This addresses the need for accurate scene representation in domain-specific applications like robotics, though it appears incremental as it builds on existing NeRF and outpainting techniques.
The paper tackles the problem of faithful field-of-view extrapolation for applications like robotic navigation by using pre-captured images as prior knowledge, and presents NeRF-Enhanced Outpainting (NEO) which achieves robust performance on three photorealistic and one real-world dataset.
In various applications, such as robotic navigation and remote visual assistance, expanding the field of view (FOV) of the camera proves beneficial for enhancing environmental perception. Unlike image outpainting techniques aimed solely at generating aesthetically pleasing visuals, these applications demand an extended view that faithfully represents the scene. To achieve this, we formulate a new problem of faithful FOV extrapolation that utilizes a set of pre-captured images as prior knowledge of the scene. To address this problem, we present a simple yet effective solution called NeRF-Enhanced Outpainting (NEO) that uses extended-FOV images generated through NeRF to train a scene-specific image outpainting model. To assess the performance of NEO, we conduct comprehensive evaluations on three photorealistic datasets and one real-world dataset. Extensive experiments on the benchmark datasets showcase the robustness and potential of our method in addressing this challenge. We believe our work lays a strong foundation for future exploration within the research community.