Camera Relocalization in Shadow-free Neural Radiance Fields
This addresses a crucial problem in computer vision and robotics by improving camera pose estimation in dynamic lighting, though it is incremental as it builds on existing NeRF methods.
The paper tackles camera relocalization under varying lighting and shadow conditions by proposing a two-staged pipeline that normalizes images and uses a hash-encoded NeRF with improved gradient techniques, achieving state-of-the-art results on several datasets.
Camera relocalization is a crucial problem in computer vision and robotics. Recent advancements in neural radiance fields (NeRFs) have shown promise in synthesizing photo-realistic images. Several works have utilized NeRFs for refining camera poses, but they do not account for lighting changes that can affect scene appearance and shadow regions, causing a degraded pose optimization process. In this paper, we propose a two-staged pipeline that normalizes images with varying lighting and shadow conditions to improve camera relocalization. We implement our scene representation upon a hash-encoded NeRF which significantly boosts up the pose optimization process. To account for the noisy image gradient computing problem in grid-based NeRFs, we further propose a re-devised truncated dynamic low-pass filter (TDLF) and a numerical gradient averaging technique to smoothen the process. Experimental results on several datasets with varying lighting conditions demonstrate that our method achieves state-of-the-art results in camera relocalization under varying lighting conditions. Code and data will be made publicly available.