FA-BARF: Frequency Adapted Bundle-Adjusting Neural Radiance Fields
This addresses a key limitation in NeRF for photorealistic novel view synthesis, enabling wider applications in dense 3D mapping and reconstruction under real-time requirements, though it appears incremental as it modifies an existing strategy.
The paper tackles the problem of slow joint optimization of scene reconstruction and camera registration in Neural Radiance Fields (NeRF) by replacing a temporal low-pass filter with a frequency-adapted spatial low-pass filter, resulting in accelerated optimization and recovery of real-world scenes with unknown camera poses.
Neural Radiance Fields (NeRF) have exhibited highly effective performance for photorealistic novel view synthesis recently. However, the key limitation it meets is the reliance on a hand-crafted frequency annealing strategy to recover 3D scenes with imperfect camera poses. The strategy exploits a temporal low-pass filter to guarantee convergence while decelerating the joint optimization of implicit scene reconstruction and camera registration. In this work, we introduce the Frequency Adapted Bundle Adjusting Radiance Field (FA-BARF), substituting the temporal low-pass filter for a frequency-adapted spatial low-pass filter to address the decelerating problem. We establish a theoretical framework to interpret the relationship between position encoding of NeRF and camera registration and show that our frequency-adapted filter can mitigate frequency fluctuation caused by the temporal filter. Furthermore, we show that applying a spatial low-pass filter in NeRF can optimize camera poses productively through radial uncertainty overlaps among various views. Extensive experiments show that FA-BARF can accelerate the joint optimization process under little perturbations in object-centric scenes and recover real-world scenes with unknown camera poses. This implies wider possibilities for NeRF applied in dense 3D mapping and reconstruction under real-time requirements. The code will be released upon paper acceptance.