Ternary-Type Opacity and Hybrid Odometry for RGB NeRF-SLAM
This work addresses the problem of deploying NeRF-SLAM in complex environments without depth sensors, which is incremental as it builds on existing NeRF and SLAM methods.
The paper tackles the challenge of using Neural Radiance Fields (NeRFs) for Simultaneous Localization and Mapping (SLAM) with only RGB inputs by introducing a ternary-type opacity model and hybrid odometry scheme, achieving state-of-the-art performance in speed and accuracy on synthetic and real-world datasets.
In this work, we address the challenge of deploying Neural Radiance Field (NeRFs) in Simultaneous Localization and Mapping (SLAM) under the condition of lacking depth information, relying solely on RGB inputs. The key to unlocking the full potential of NeRF in such a challenging context lies in the integration of real-world priors. A crucial prior we explore is the binary opacity prior of 3D space with opaque objects. To effectively incorporate this prior into the NeRF framework, we introduce a ternary-type opacity (TT) model instead, which categorizes points on a ray intersecting a surface into three regions: before, on, and behind the surface. This enables a more accurate rendering of depth, subsequently improving the performance of image warping techniques. Therefore, we further propose a novel hybrid odometry (HO) scheme that merges bundle adjustment and warping-based localization. Our integrated approach of TT and HO achieves state-of-the-art performance on synthetic and real-world datasets, in terms of both speed and accuracy. This breakthrough underscores the potential of NeRF-SLAM in navigating complex environments with high fidelity.